How to Run TestNG Tests on Jenkins

I'm trying to run TestNG tests (in a contained Java project) from Jenkins but having no luck.

It appears as though the TestNG plugin for Jenkins (https://wiki.jenkins-ci.org/display/JENKINS/testng-plugin) only publishes the results of TestNG tests, but doesn't actually run test classes... or am I wrong?

In any case, how do I actually run TestNG tests in a TestNG project with Jenkins, or is that even possible? Do I have to use a command line statement or batch file (on Windows Server 2008), for example?

Any help much appreciated.

Note I tried entering a post-build command line in Jenkins for the project to run TestNG tests but had a hard time with class paths not being found for TestNG. I posted an earlier question about running TestNG from the command line which I couldn't get working, so I've given up on that route:

How to run TestNG from command line

Solution 1:

There are two steps to accomplished this task:-

Step 1:-

1. Go to localhost:8080/configure (Jenkins configure section)

2. Now go to JDK section and uncheck Install automatically (If you don't uncheck that then it will download latest java every time whenever it is available, and can cause for build failed)

3. put JAVA_HOME in name section and jdk home path in JAVA_HOME section

4. Apply and save

Step 2:-

1. Go to Jenkins and add new Item, also select "Free Style Project" and click on Ok.

2. Click on "Advanced in "Advanced Project Options"

3. Now check option: - "Use custom workspace" and specify your project absolute path in Directory section

4. Apply

5. Now to go "Build" and select "Execute windows batch command"

6. Here in command column give the file name of your batch file
7. Apply and save

Now go to the Jenkins and select your Jenkins project and click on Build :)

Solution 2:

I use gradle to run my testNG tests from Jenkins. Take a look at the gradle docs.

I run the testNG tests using configuration xml files. Take a look at the testNG docs.

There is quite a lot to cover so I suggest reading these sources but I'll provide some relevant pieces from one of my configurations.

The relevant parts from my build.gradle

tasks.withType(Test) {

    useTestNG {
        useDefaultListeners = true
    }

    options {
        outputDirectory = file('test-report')
        listeners << 'org.uncommons.reportng.JUnitXMLReporter'
    }

    testLogging.showStandardStreams = true

    systemProperties System.getProperties()
    systemProperty "org.uncommons.reportng.escape-output", "false"
    systemProperty "org.uncommons.reportng.title", "Test Report"

    ignoreFailures = true
}


task Smoke_Test(type: Test) {
    description "SmokeTest"
    options.suites("resources/testng-smoketest.xml")      
    ignoreFailures = false
}

My testNG xml as referenced above 'testng-smoketest.xml'

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="Smoke Tests" >
    <test name="BootCheck" parallel="false" thread-count="1">
        <classes>
            <class name="com.x.automation.y.tests.smoke.BootCheck" />
        </classes>
    </test>
</suite>    

And from Jenkins, as an 'execute shell' build step run the gradle task, I use gradle wrapper for convenience.

./gradlew clean Smoke_Test

Ensure you're in the correct directory, 'Smoke_Test' is the name specified in the build.gradle.

You can use the testNG Jenkins plugin for saving your results.

I also recommend using reportng for nice formatting of your test reports which can also be shown and saved in Jenkins using the HTML Publisher plugin.

Try getting this to run from a CLI on your local machine first, trying to debug when running from Jenkins will drive you crazy.

Solution 3:

As commented above, please use the following ant script to run TestNG unit tests. Please tweak the below code to meet your requirements.

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<project basedir="." default="build" name="Ant Play">
    <property name="classes.dir" value="bin" />
    <property name="report.dir" value="test-output" />
    <path id="classpath">
        <fileset dir="lib">
            <include name="**/*.jar"/>
         </fileset>
        <pathelement path="${basedir}\${classes.dir}"/>
    </path>
    <target name="init">
        <mkdir dir="${classes.dir}"/>
        <copy includeemptydirs="false" todir="${classes.dir}">
            <fileset dir="src">
                <exclude name="**/*.java"/>
            </fileset>
        </copy>
    </target>
    <target name="clean">
        <delete dir="${classes.dir}"/>
    </target>
    <target depends="clean" name="cleanall"/>
    <target depends="build-project" name="build"/>
    <target depends="init" name="build-project">
        <echo message="${ant.project.name}: ${ant.file}"/>
        <javac debug="true" includeantruntime="false" destdir="${classes.dir}">
            <src path="src"/>
            <classpath refid="classpath"/>
        </javac>
    </target>
    <target  depends="build" name="runTests" description="Running tests" >
        <echo>Running Tests...</echo>
        <taskdef resource="testngtasks" classpathref="classpath"/>
        <testng outputDir="${report.dir}"
            haltonfailure="true"
            useDefaultListeners="false"
            listeners="org.uncommons.reportng.HTMLReporter"
            classpathref="classpath">
            <xmlfileset dir="${basedir}" includes="testng.xml"/>
            <!--<classfileset dir="${classes.dir}" includes="**/*.class" />-->
        </testng>
    </target>
</project>

Let me know if you encounter any issues. BTW, please use the Jenkins ant plugin/task to run this script