Measure Coverage by Integration Tests with Sonar – Updated
It has now been almost 2 years since JaCoCo was integrated as a plugin in Sonar and more than 18 months since Evgeny published his post to describe how to Measure Code Coverage by Integration Tests with Sonar. A lot happened since then on this JaCoCo integration into Sonar: creation of an ANT task for JaCoCo, support for branch coverage, integration of JaCoCo in Sonar…
This is why I thought it would be a good time to post an updated version of Evgeny’s post.
Let’s first come back to needs and requirements:
- there should be an easy way to assess code coverage by integration / functional / acceptance / UI tests (let’s say integration tests)
- whatever system is used to execute those tests: Maven plugin (maven-surefire-plugin, maven-osgi-test-plugin, maven-failsafe-plugin…), Ant, GreenPepper, Selenium…
- whatever the package format: jar, ear, war…
This can be really tricky and even a nightmare with coverage engines that use source code instrumentation (Clover) or offline bytecode instrumentation (Cobertura or Emma). Indeed an application is usually made of multiple Java libraries and doing static instrumentation of each Java library is most of the time complex to automate. The only suitable approach for this usage is to do on-the-fly byte code instrumentation like JaCoCo does.
To get the code coverage by integration tests on your project, you should simply follow the 5 steps below:
- Attach and configure a JaCoCo agent to the JVM which you will run your integration tests with. To do so, have a look at the detailed JaCoCo documentation to attach the agent: via a command line, using the JaCoCo Maven plugin or using the JaCoCo Ant task.
- Run your automated or manual integration tests within this JVM. At the end of the execution, JaCoCo will dump the code coverage report in the destination you have set in step 1.
- Configure Sonar to reuse this JaCoCo report. In your project properties file (or directly in the command line), add the sonar.jacoco.itReportPath property with the path to the JaCoCo IT coverage report as a value. You can download an example and look at the projects/code-coverage/it directory: for each launcher you will find a sample.
- Run an analysis to feed Sonar with the information provided by the JaCoCo report.
- Browse the results on the Sonar web interface. On project dashboard, you can add the Integration test coverage widget:
As usual, drilldown is available from this widget. At file level, you can access detailed code coverage by integration tests on the Coverage tab:
Note: You can still use Clover, Cobertura or Emma to assess code coverage by unit tests together with JaCoCo for integration tests.
Going further, the next evolution is going to also provide merged coverage by unit tests and integration tests to get a global overview of the code coverage by automated tests. If you are interested in such a feature, just vote for the SONAR-2804 JIRA issue.
Time now to give it a try. Enjoy and give us feedback!