TestNG - java testing framework

Recently i got introduced to TestNG (version 5.7) at work. I was familiar to JUnit from the past and i kind of knew about the existance of TestNG and that it had improvements over JUnit but i never thought that it will gain so much traction that i will be made to use it soon. Here are some of the features:

• JDK 5 Annotations (JDK 1.4 is also supported with JavaDoc annotations).
• Flexible test configuration - using multiple testng XML configuration files one per test suite.
• Support for data-driven testing (with @DataProvider).
• Support for parameters - you can pass parameters to test methods from the testng.xml file.
  
• Allows distribution of tests on slave machines - support for parallel execution of tests and methods.
  
• Powerful execution model (no more TestSuite) - test classes are annotated POJOs and don't have to extend any class or implement interface to have test methods.
• Supported by a variety of tools and plug-ins (Eclipse, IDEA, Ant, Maven, etc...).
• Embeds BeanShell for further flexibility.
• Default JDK functions for runtime and logging (no dependencies).
• Dependent methods for application server testing. - one can specify the dependsOnMethods attribute to the @Test annotation to specify a list of methods that should execute before a certain test method executes. This is a powerful feature and is required for any kind of dependent testing. If a dependent method fails, then all subsequent tests will be skipped, not marked as failed (unlike JUnit).

A good article stating improvements in TestNG over JUnit 4 is found at http://www.ibm.com/developerworks/java/library/j-cq08296/.

I used TestNG today for the first time and found the framework very easy to use and within a day i had it integrated into our build system and made a presentation to the team about its usage in our project. In this post, i am detailing the steps i performed to start using TestNG:

1. Wrote a class using just the 3 basic TestNG annotations to start with:

• @BeforeClass
• @Test (groups = {"xyz.groupname"}) - at class level which gets inherited by all public methods in the class.
• @AfterClass

See http://testng.org/doc/documentation-main.html#annotations for complete list of supported annotations.

2. Wrote a master testng.xml which included all project suite-files and was referenced from the ant build script. Also wrote a testng-regression.xml which was imported in the master testng.xml. In the testng-regression.xml, defined the test runs in the suite. Each such testng-xxx.xml file is for xxx named test suite. Each suite can have one or more test runs. Each test runs identifies the class(es) or package(s) to lookup for annotated test methods. Each test run also identifies filter criteria based on groups to include and exclude in the test run. See http://testng.org/doc/documentation-main.html#testng-xml for more on testng.xml.

One powerful feature i found was the group names could be specified in dot separated (java package name like) notations and follow a hierarchy akin to the Log4j Logger naming hierarchy. So you can use wildcards in the testng.xml to not only include all classes of a group but also include classes from child groups. For example, i could just say xyz.* to include xyz.abc and xyz.def group classes.

3. Lastly, used the ant build file to call the testng ant task and pass the testng.xml location to it so that testng can execute the tests we wanted. We can have multiple targets defined for different types of tests that we may want to automate. See http://testng.org/doc/ant.html for examples.

In the latest releases of JUnit 4, it too uses JDK5 annotations and thus makes up for some of the shortcomings that led Cedric Beust to develop TestNG framework.

If you have not had a chance to explore TestNG so far, then i hope after reading this post you will have the good sense to do so now :).