Continuous Integration

Publication date: November 22, 2020

Description: Diff-jjoules is a tool to run in your continuous integration to measure the impact of commits on the energy consumption of the program

Repository: https://github.com/davidson-consulting/diff-jjoules

License: GPL-3.0 License

Tags: java sustainability continuous-integration

Authors: Benjamin Danglot, Jean-Rémy Falleri, Romain Rouvoy

Venue: Registered Reports Track @ 37th International Conference on Software Maintenance and Evolution (ICSME 2021), September 27 - October 1, Luxembourg City

Publication date: September 29, 2021

Abstract: Software Energy Consumption(SEC) is gaining more and more attention. In this paper, we tackle the problem of hinting developers about the SEC of their programs in the context of software developments based on Continuous Integration(CI). In this study, we investigate if the CI can leverage developers' tests to perform a new class of tests: the energy regression testing. Energy regression is similar to performance regression but focused on the energy consumption of the program instead of standard performance indicators, like execution time or memory consumption. We propose to perform an exploratory study of the usage of developers' tests for energy regression testing. We propose to first investigate if developers' tests can be used to obtain stable SEC indicators. Then, to consider if comparing the SEC of developers' tests between two versions can accurately spot energy regressions introduced by automated program mutations. Finally, to assess if it can successfully pinpoint the source code lines guilty of energy regressions. Our study will pave the way for automated SEC regression tools that can be readily deployed inside an existing CI infrastructure to raise awareness of SEC issues among practitioners.

Tags: Continuous Integration Energy Regression Sustainable Software

Authors: Benjamin Danglot, Martin Monperrus, Walter Rudametkin and Benoit Baudry.

Venue: Empirical Software Engineering 25, 2379–2415 (2020). https://doi.org/10.1007/s10664-019-09794-7

Publication date: March 5, 2020

Abstract: When a developer pushes a change to an application’s codebase, a good practice is to have a test case specifying this behavioral change. Thanks to continuous integration (CI), the test is run on subsequent commits to check that they do no introduce a regression for that behavior. In this paper, we propose an approach that detects behavioral changes in commits. As input, it takes a program, its test suite, and a commit. Its output is a set of test methods that capture the behavioral difference between the pre-commit and post-commit versions of the program. We call our approach DCI (Detecting behavioral changes in CI). It works by generating variations of the existing test cases through (i) assertion amplification and (ii) a search-based exploration of the input space. We evaluate our approach on a curated set of 60 commits from 6 open source Java projects. To our knowledge, this is the first ever curated dataset of real-world behavioral changes. Our evaluation shows that DCI is able to generate test methods that detect behavioral changes. Our approach is fully automated and can be integrated into current development processes. The main limitations are that it targets unit tests and works on a relatively small fraction of commits. More specifically, DCI works on commits that have a unit test that already executes the modified code. In practice, from our benchmark projects, we found 15.29% of commits to meet the conditions required by DCI.

Tags: Continuous Integration Test amplification Behavioral change detection

Authors: Benjamin Danglot, Martin Monperrus, Walter Rudametkin and Benoit Baudry.

Venue: Empirical Software Engineering 25, 2379–2415 (2020). https://doi.org/10.1007/s10664-019-09794-7

Publication date: March 5, 2020

Abstract: When a developer pushes a change to an application’s codebase, a good practice is to have a test case specifying this behavioral change. Thanks to continuous integration (CI), the test is run on subsequent commits to check that they do no introduce a regression for that behavior. In this paper, we propose an approach that detects behavioral changes in commits. As input, it takes a program, its test suite, and a commit. Its output is a set of test methods that capture the behavioral difference between the pre-commit and post-commit versions of the program. We call our approach DCI (Detecting behavioral changes in CI). It works by generating variations of the existing test cases through (i) assertion amplification and (ii) a search-based exploration of the input space. We evaluate our approach on a curated set of 60 commits from 6 open source Java projects. To our knowledge, this is the first ever curated dataset of real-world behavioral changes. Our evaluation shows that DCI is able to generate test methods that detect behavioral changes. Our approach is fully automated and can be integrated into current development processes. The main limitations are that it targets unit tests and works on a relatively small fraction of commits. More specifically, DCI works on commits that have a unit test that already executes the modified code. In practice, from our benchmark projects, we found 15.29% of commits to meet the conditions required by DCI.

Description: When a developer change the behavior a program, he should provide a new test method (or modify an existing one) that specify the new behavior. However, developers do not always provide such test method because lack of time, expertise or discipline. In the context of the article “An Approach and Benchmark to Detect Behavioral Changes of Commits in Continuous Integration”, we devised a technique that produces automatically a test method that specify the new behavior. On this webpage, you can find: first, the seed test method that we amplified; second, the amplified test method that detect the behavioral change; third, the manual test method provided by the developer with the commit, used as a ground truth.

Tags: Continuous Integration Test amplification Behavioral change detection