TestNG is a testing framework inspired by JUnit but introducing a whole set of new functionalities. Let's see how to create a sample project which uses a configuration file to drive a TestNG suite

TestNG Example Test suite

Let's write a proof of concept Hello World example with TestNG. In order to use it, include in your pom.xml the following dependency:

  <dependencies>
    <dependency>
      <groupId>org.testng</groupId>
      <artifactId>testng</artifactId>
      <version>7.0.0</version>
      <scope>test</scope>
    </dependency>
  </dependencies>

On the other hand, if you are using Gradle, you will need the following basic configuration in your build.gradle file:

test { //this is the gradle task to be executed
  useTestNG() { //Tells Gradle to use TestNG
       useDefaultListeners = true // Tells TestNG to execute its default reporting structure
       suites 'src/test/suite.xml' //location of our suite.xml
  }
}

Here is your example App.java file:

package com.example;


public class App 
{

    public int sum(int a, int b) {
        return a+b;
    }

    public int multiply(int a, int b) {
        return a*b;
    }

}

Let's write a Test class for it:

package com.example;

import org.testng.Assert;
import org.testng.annotations.Test;

public class AppTest 
{
    
     
    public void testSum() {
       App a = new App();

       Assert.assertEquals(10, a.sum(5,5));
    }

     
    public void testMultiply() {
        App a = new App();

        Assert.assertEquals(25, a.multiply(5,5));
    }
}

Now include under the resources folder the following testng.xml file:

<suite name="Suite">
 
    <test name="Test">
        <classes>
            <class name="com.example.AppTest"/>
        </classes>
    </test>
</suite>

This configuration file states that the Test suite includes the class com.example.AppTest. We need to provide an hint to the maven-surefire-plugin about the suiteXMLFile name:

<plugin>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.22.1</version>
  <configuration>
    <!-- Suite testng xml file to consider for test execution -->
    <suiteXmlFiles>
      <suiteXmlFile>testng.xml</suiteXmlFile>
    </suiteXmlFiles>
  </configuration>
</plugin>

Here's the Project view:

That's it! You have just executed your first HelloWorld TestNG example

Right-click the XML file to run it:

Suite
Total tests run: 2, Passes: 2, Failures: 0, Skips: 0

Defining Execution groups

Within the XML configuration file it's possible to specify which tests are to be executed. For example, we will define two Test Groups by annotating them in the AppTest class:

package com.example;

import org.testng.Assert;
import org.testng.annotations.Test;

public class AppTest 
{
    
    @Test(groups = { "test-sum" })
    public void testSum() {
       App a = new App();

       Assert.assertEquals(10, a.sum(5,5));
    }

    @Test(groups = { "test-multi" })
    public void testMultiply() {
        App a = new App();

        Assert.assertEquals(25, a.multiply(5,5));
    }
}

Now, within the XML file, add the "test-sum" Group to the include list. Also add the "test-multi" to the exclude list:

<suite name="Suite">
    <groups>
        <run>
            <include name="test-sum" />
            <exclude name="test-multi" />
        </run>
    </groups>
    <test name="Test">
        <classes>
            <class name="com.example.AppTest"/>
        </classes>
    </test>
</suite>

Now if you run the XML TestNG suite, it will just execute one test:

Suite
Total tests run: 1, Passes: 1, Failures: 0, Skips: 0