Miscellaneous

Authors: Benjamin Danglot, Martin Monperrus, Walter Rudametkin and Benoit Baudry.

Venue: Empirical Software Engineering 25, 2379–2415 (2020). https://doi.org/10.1007/s10664-019-09794-7

Publication date: March 5, 2020

Abstract: When a developer pushes a change to an application’s codebase, a good practice is to have a test case specifying this behavioral change. Thanks to continuous integration (CI), the test is run on subsequent commits to check that they do no introduce a regression for that behavior. In this paper, we propose an approach that detects behavioral changes in commits. As input, it takes a program, its test suite, and a commit. Its output is a set of test methods that capture the behavioral difference between the pre-commit and post-commit versions of the program. We call our approach DCI (Detecting behavioral changes in CI). It works by generating variations of the existing test cases through (i) assertion amplification and (ii) a search-based exploration of the input space. We evaluate our approach on a curated set of 60 commits from 6 open source Java projects. To our knowledge, this is the first ever curated dataset of real-world behavioral changes. Our evaluation shows that DCI is able to generate test methods that detect behavioral changes. Our approach is fully automated and can be integrated into current development processes. The main limitations are that it targets unit tests and works on a relatively small fraction of commits. More specifically, DCI works on commits that have a unit test that already executes the modified code. In practice, from our benchmark projects, we found 15.29% of commits to meet the conditions required by DCI.

Description: When a developer change the behavior a program, he should provide a new test method (or modify an existing one) that specify the new behavior. However, developers do not always provide such test method because lack of time, expertise or discipline. In the context of the article “An Approach and Benchmark to Detect Behavioral Changes of Commits in Continuous Integration”, we devised a technique that produces automatically a test method that specify the new behavior. On this webpage, you can find: first, the seed test method that we amplified; second, the amplified test method that detect the behavioral change; third, the manual test method provided by the developer with the commit, used as a ground truth.

Tags: Continuous Integration Test amplification Behavioral change detection

Venue: Inria Sophia-Antipolis

Publication date: January 30, 2019

Abstract: Presentation of DSpot. Animation of an hands-on session around the usage of DSpot.

Authors: Benjamin Danglot, Philippe Preux, Benoit Baudry and Martin Monperrus

Venue: Empirical Software Engineering 23, 2086–2119 (2018). https://doi.org/10.1007/s10664-017-9571-8

Publication date: December 21, 2017

Abstract: Can the execution of software be perturbed without breaking the correctness of the output? In this paper, we devise a protocol to answer this question from a novel perspective. In an experimental study, we observe that many perturbations do not break the correctness in ten subject programs. We call this phenomenon “correctness attraction”. The uniqueness of this protocol is that it considers a systematic exploration of the perturbation space as well as perfect oracles to determine the correctness of the output. To this extent, our findings on the stability of software under execution perturbations have a level of validity that has never been reported before in the scarce related work. A qualitative manual analysis enables us to set up the first taxonomy ever of the reasons behind correctness attraction.

Description: Correctness attraction is the phenomenon that a system is able to recover from changes injected into its state, which is called perturbations, and yet produce a correct output. Here, you can apply our protocol to an implementation of a quicksort algorithm. For every integer (literals and expressions), a perturbation consists to add 1 with a given probability. In the other hand, it does the same for boolean literals and expressions by negating them.

Tags: Perturbation analysis Software correctness Empirical study

Venue: Holiday Inn Madrid, 476 Calle Alcalá, Madrid, Organized by Atos Madrid.

Publication date: December 14, 2017

Abstract: Presentation of DSpot: a test suite amplification tool.

Venue: Kungliga Tekniska Högskolan (KTH)

Publication date: December 6, 2017

Abstract: Presentation of Correctness Attraction: A Study of Stability of Software Behavior Under Runtime Perturbation.

Venue: Orange Gardens Innovation Center, Paris - Châtillon

Publication date: June 26, 2017

Abstract: Presentation of Spoon, a library to analyze and transform java code in java. During the event, Spoon received the Community Award of OW2Con'17.