Thursday, March 13, 2008

Using TestNG with Emma for automating test code coverage report generation

TestNG and Emma can be used together to automate the generation of code coverage report after every test run in the ANT builld script.
Following ant build script snippet shows how both tools are used together for this important metrics collection.

To run it:
$ ant emma test

This will generate the code coverage report for the TestNG tests in ./coverage directory (where, basedir = ".").

<!-- output directory used for EMMA coverage reports: -->

<property name="coverage.dir" value="${basedir}/coverage" />

<!-- directory that contains emma.jar and emma_ant.jar: -->
<property name="emma.dir" value="${lib}/emma" />

<!-- path element used by EMMA taskdef below: -->
<path id="emma.lib">
<pathelement location="${emma.dir}/emma.jar" />
<pathelement location="${emma.dir}/emma_ant.jar" />

<!-- this loads <emma> and <emmajava> custom tasks: -->
<taskdef resource="" classpathref="emma.lib" />

<property environment="env" />
<path id="classpath">
<fileset dir="${lib}">
<include name="**/*.jar" />
<!-- 6. CODE COVERAGE initialization. -->
<target name="emma" description="turns on EMMA's instrumentation/reporting">
<property name="emma.enabled" value="true" />

<!-- this property, if overriden via -Demma.filter=<list of filter specs>
on ANT's command line, will set the coverage filter; by default,
all classes found in 'run.classpath' pathref will be instrumented:
<property name="emma.filter" value="" />

<!-- 7. TEST the application. -->
<taskdef name="testng" classpathref="classpath" classname="org.testng.TestNGAntTask" />

<!-- EMMA ANT tasks are implemented as pseudo-nested tasks: <emma>
container task can contain an arbitrary sequence of <instr>,
<report>, and <merge>. Both the container tag and each of the nested
elements support an optional boolean 'enabled' attribute: setting it
to 'false' will no-op the element. This is convenient for
sandwhiching normal build tasks between EMMA tasks such that coverage
instrumentation and reporting could be enabled on demand. -->

<target name="test" description="execute testng tests" depends="dist">

<emma enabled="${emma.enabled}" verbosity="verbose">
<instr instrpath="${build}" mode="overwrite" metadatafile="${coverage.dir}/metadata.emma">

<!-- note that coverage filters can be set through nested <filter>
elements as well: many of EMMA setting are 'mergeable' in the
sense that they can be specified multiple times and the result
is a union of all such values. Here we are not merging several
filters together but merely demonstrating that it is possible:
<filter value="${emma.filter}" />

<javac srcdir="${test.src.dir}" destdir="${build}" classpathref="classpath" deprecation="${compile.deprecation}" />

<testng classpathref="test.classpath" outputDir="${}" sourcedir="${test.src.dir}" haltOnfailure="true">
<xmlfileset dir="${test.src.dir}" includes="testng.xml" />
<jvmarg value="-Demma.coverage.out.file=${coverage.dir}/coverage.emma" />
<jvmarg value="-Demma.coverage.out.merge=false" />

<!-- if enabled, generate coverage report(s): -->
<emma enabled="${emma.enabled}">
<report sourcepath="${src}" sort="+block,+name,+method,+class" metrics="method:70,block:80,line:80,class:100">
<!-- collect all EMMA data dumps (metadata and runtime)
[this can be done via nested <fileset> fileset elements
or <file> elements pointing to a single file]:
<fileset dir="${coverage.dir}">
<include name="*.emma" />

<!-- for every type of report desired, configure a nested
element; various report parameters
can be inherited from the parent <report>
and individually overridden for each report type:
<txt outfile="${coverage.dir}/coverage.txt" depth="package" columns="class,method,block,line,name" />
<xml outfile="${coverage.dir}/coverage.xml" depth="package" />
<html outfile="${coverage.dir}/coverage.html" depth="method" columns="name,class,method,block,line" />



Anonymous said...

Thanks, this was really helpful

jastazv said...

Thanks, the post was great, very useful.

barisergun said...

Based on various articles including this one I have prepared a simple sample project demonstrating TestNg, Mockito and Emma tools. You can find it from the following link.

15 sorting algorithms visualized in 5 minutes, with awesome arcade sounds

15 sorting algorithms visualized in 5 minutes, with awesome arcade sounds from r/programming