Running JavaScript unit tests headlessly in a Continuous Integration build

As I managed to come up with a solution myself, I thought it would be a good idea to share it. The approach might not be flawless, but it's the first one that seemed to work. Feel free to post improvements and suggestions.

What I did in a nutshell:

• Launch an instance of Xvfb, a virtual framebuffer
• Using JsTestDriver:
  • launch an instance of Firefox into the virtual framebuffer (headlessly)
  • capture the Firefox instance and run the test suite
  • generate JUnit-compliant test results .XML
• Use Bamboo to inspect the results file to pass or fail the build

I will next go through the more detailed phases. This is what my my directory structure ended up looking like:

lib/
    JsTestDriver.jar
test/
    qunit/
            equiv.js
            QUnitAdapter.js
    jsTestDriver.conf
    run_js_tests.sh
    tests.js
test-reports/
build.xml

On the build server:

• Install Xvfb (apt-get install Xvfb)
• Install Firefox (apt-get install firefox)

Into your application to be built:

• Install JsTestDriver: http://code.google.com/p/js-test-driver/
  • add the QUnit adapters equiv.js and QUnitAdapter.js
  • configure JsTestDriver (jsTestDriver.conf):

server: http://localhost:4224

load:
# Load QUnit adapters (may be omitted if QUnit is not used)
  - qunit/equiv.js
  - qunit/QUnitAdapter.js   

# Tests themselves (you'll want to add more files)
  - tests.js

Create a script file for running the unit tests and generating test results (example in Bash, run_js_tests.sh):

#!/bin/bash
# directory to write output XML (if this doesn't exist, the results will not be generated!)
OUTPUT_DIR="../test-reports"
mkdir $OUTPUT_DIR

XVFB=`which Xvfb`
if [ "$?" -eq 1 ];
then
    echo "Xvfb not found."
    exit 1
fi

FIREFOX=`which firefox`
if [ "$?" -eq 1 ];
then
    echo "Firefox not found."
    exit 1
fi

$XVFB :99 -ac &    # launch virtual framebuffer into the background
PID_XVFB="$!"      # take the process ID
export DISPLAY=:99 # set display to use that of the xvfb

# run the tests
java -jar ../lib/JsTestDriver.jar --config jsTestDriver.conf --port 4224 --browser $FIREFOX --tests all --testOutput $OUTPUT_DIR

kill $PID_XVFB     # shut down xvfb (firefox will shut down cleanly by JsTestDriver)
echo "Done."

Create an Ant target that calls the script:

<target name="test">        
    <exec executable="cmd" osfamily="windows">
        <!-- This might contain something different in a Windows environment -->
    </exec>

    <exec executable="/bin/bash" dir="test" osfamily="unix">
        <arg value="run_js_tests.sh" />
    </exec>
</target>   

Finally, tell the Bamboo build plan to both invoke the test target and look for JUnit test results. Here the default "**/test-reports/*.xml" will do fine.

For anyone interested in running their Jasmine BDD specs headlessly in maven, you might be interested in the jasmine-maven-plugin I maintain:

http://github.com/searls/jasmine-maven-plugin

As an alternative, you could also try TestSwarm. I've got it up and running using QUnit to run my JS tests.

• http://github.com/jquery/testswarm
• http://swarm.jquery.org/