How to configure multi-module Maven + Sonar + JaCoCo to give merged coverage report?

I've searched up and down the internet for this one. There's lots of half-answers out there, to do with Maven properties such as ${sonar.jacoco.reportPath}, or org.jacoco:jacoco-maven-plugin:prepare-agent or setting maven-surefire-plugin argLine with -javaagent.

Some how, none of these answers, either on their own, or in combination, are producing what I'm after: A coverage report which shows a class as covered if it is used in tests higher up the stack, such as entities being used by DAOs, even though it was not fully covered by tests in its own module.

Is there a definitive config somewhere, to achieve this, please?

Solution 1:

I was in the same situation as you, the half answers scattered throughout the Internet were quite annoying, since it seemed that many people had the same issue, but no one could be bothered to fully explain how they solved it.

The Sonar docs refer to a GitHub project with examples that are helpful. What I did to solve this was to apply the integration tests logic to regular unit tests (although proper unit tests should be submodule specific, this isn't always the case).

In the parent pom.xml, add these properties:

<properties>
    <!-- Sonar -->
    <sonar.java.coveragePlugin>jacoco</sonar.java.coveragePlugin>
    <sonar.dynamicAnalysis>reuseReports</sonar.dynamicAnalysis>
    <sonar.jacoco.reportPath>${project.basedir}/../target/jacoco.exec</sonar.jacoco.reportPath>
    <sonar.language>java</sonar.language>
</properties>

This will make Sonar pick up unit testing reports for all submodules in the same place (a target folder in the parent project). It also tells Sonar to reuse reports ran manually instead of rolling its own. We just need to make jacoco-maven-plugin run for all submodules by placing this in the parent pom, inside build/plugins:

<plugin>
    <groupId>org.jacoco</groupId>
    <artifactId>jacoco-maven-plugin</artifactId>
    <version>0.6.0.201210061924</version>
    <configuration>
        <destFile>${sonar.jacoco.reportPath}</destFile>
        <append>true</append>
    </configuration>
    <executions>
        <execution>
            <id>agent</id>
            <goals>
                <goal>prepare-agent</goal>
            </goals>
        </execution>
    </executions>
</plugin>

destFile places the report file in the place where Sonar will look for it and append makes it append to the file rather than overwriting it. This will combine all JaCoCo reports for all submodules in the same file.

Sonar will look at that file for each submodule, since that's what we pointed him at above, giving us combined unit testing results for multi module files in Sonar.

Solution 2:

NEW WAY SINCE VERSION 0.7.7

Since Version 0.7.7 there is a new way to create an aggregated report:

You create a separate 'report' project which collects all the necessary reports (Any goal in the aggregator project is executed before its modules therefore it can't be used).

aggregator pom
  |- parent pom
  |- module a
  |- module b
  |- report module

The root pom looks like this (don't forget to add the new report module under modules):

<build>
<plugins>
  <plugin>
    <groupId>org.jacoco</groupId>
    <artifactId>jacoco-maven-plugin</artifactId>
    <version>0.7.8</version>
    <executions>
      <execution>
        <id>prepare-agent</id>
        <goals>
          <goal>prepare-agent</goal>
        </goals>
      </execution>
    </executions>
  </plugin>
</plugins>

The poms from each sub module doesn't need to be changed at all. The pom from the report module looks like this:

<!-- Add all sub modules as dependencies here -->
<dependencies>
  <dependency>
    <module a>
  </dependency>
  <dependency>
    <module b>
  </dependency>
 ...

  <build>
    <plugins>
      <plugin>
        <groupId>org.jacoco</groupId>
        <artifactId>jacoco-maven-plugin</artifactId>
        <version>0.7.8</version>
        <executions>
          <execution>
            <id>report-aggregate</id>
            <phase>verify</phase>
            <goals>
              <goal>report-aggregate</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>

A full exmple can be found here.

Solution 3:

FAQ

Questions from the top of my head since that time I gone crazy with jacoco.

My application server (jBoss, Glassfish..) located in Iraq, Syria, whatever.. Is it possible to get multi-module coverage when running integration tests on it? Jenkins and Sonar are also on different servers.

Yes. You have to use jacoco agent that runs in mode output=tcpserver, jacoco ant lib. Basically two jars. This will give you 99% success.

How does jacoco agent works?

You append a string

-javaagent:[your_path]/jacocoagent.jar=destfile=/jacoco.exec,output=tcpserver,address=*

to your application server JAVA_OPTS and restart it. In this string only [your_path] have to be replaced with the path to jacocoagent.jar, stored(store it!) on your VM where app server runs. Since that time you start app server, all applications that are deployed will be dynamically monitored and their activity (meaning code usage) will be ready for you to get in jacocos .exec format by tcl request.

Could I reset jacoco agent to start collecting execution data only since the time my test start?

Yes, for that purpose you need jacocoant.jar and ant build script located in your jenkins workspace.

So basically what I need from http://www.eclemma.org/jacoco/ is jacocoant.jar located in my jenkins workspace, and jacocoagent.jar located on my app server VM?

That's right.

I don't want to use ant, I've heard that jacoco maven plugin can do all the things too.

That's not right, jacoco maven plugin can collect unit test data and some integration tests data(see Arquillian Jacoco), but if you have for example rest assured tests as a separated build in jenkins, and want to show multi-module coverage, I can't see how maven plugin can help you.

What exactly does jacoco agent produce?

Only coverage data in .exec format. Sonar then can read it.

Does jacoco need to know where my java classes located are?

No, sonar does, but not jacoco. When you do mvn sonar:sonar path to classes comes into play.

So what about the ant script?

It has to be presented in your jenkins workspace. Mine ant script, I called it jacoco.xml looks like that:

<project name="Jacoco library to collect code coverage remotely" xmlns:jacoco="antlib:org.jacoco.ant">
    <property name="jacoco.port" value="6300"/>
    <property name="jacocoReportFile" location="${workspace}/it-jacoco.exec"/>

    <taskdef uri="antlib:org.jacoco.ant" resource="org/jacoco/ant/antlib.xml">
        <classpath path="${workspace}/tools/jacoco/jacocoant.jar"/>
    </taskdef>

    <target name="jacocoReport">
            <jacoco:dump address="${jacoco.host}" port="${jacoco.port}" dump="true" reset="true" destfile="${jacocoReportFile}" append="false"/>
    </target>

    <target name="jacocoReset">
            <jacoco:dump address="${jacoco.host}" port="${jacoco.port}" reset="true" destfile="${jacocoReportFile}" append="false"/>
        <delete file="${jacocoReportFile}"/>
    </target>
</project>

Two mandatory params you should pass when invoking this script -Dworkspace=$WORKSPACE use it to point to your jenkins workspace and -Djacoco.host=yourappserver.com host without http://

Also notice that I put my jacocoant.jar to ${workspace}/tools/jacoco/jacocoant.jar

What should I do next?

Did you start your app server with jacocoagent.jar?

Did you put ant script and jacocoant.jar in your jenkins workspace?

If yes the last step is to configure a jenkins build. Here is the strategy:

Invoke ant target jacocoReset to reset all previously collected data.
Run your tests
Invoke ant target jacocoReport to get report

If everything is right, you will see it-jacoco.exec in your build workspace.

Look at the screenshot, I also have ant installed in my workspace in $WORKSPACE/tools/ant dir, but you can use one that is installed in your jenkins.

enter image description here

How to push this report in sonar?

Maven sonar:sonar will do the job (don't forget to configure it), point it to main pom.xml so it will run through all modules. Use sonar.jacoco.itReportPath=$WORKSPACE/it-jacoco.exec parameter to tell sonar where your integration test report is located. Every time it will analyse new module classes, it will look for information about coverage in it-jacoco.exec.

I already have jacoco.exec in my `target` dir, `mvn sonar:sonar` ignores/removes it

By default mvn sonar:sonar does clean and deletes your target dir, use sonar.dynamicAnalysis=reuseReports to avoid it.