What unit-testing framework should I use for Qt? [closed]

Solution 1:

You don't have to create separate tests applications. Just use qExec in an independent main() function similar to this one:

int main(int argc, char *argv[])
{
    TestClass1 test1;
    QTest::qExec(&test1, argc, argv);

    TestClass2 test2;
    QTest::qExec(&test2, argc, argv);

    // ...

    return 0;
}

This will execute all test methods in each class in one batch.

Your testclass .h files would look as follows:

class TestClass1 : public QObject
{
Q_OBJECT

private slots:
    void testMethod1();
    // ...
}

Unfortunately this setup isn't really described well in the Qt documentation even though it would seem to be quite useful for a lot of people.

Solution 2:

I started off using QtTest for my app and very, very quickly started running into limitations with it. The two main problems were:

1) My tests run very fast - sufficiently quickly that the overhead of loading an executable, setting up a Q(Core)Application (if needed) etc often dwarfs the running time of the tests themselves! Linking each executable takes up a lot of time, too.

The overhead just kept on increasing as more and more classes were added, and it soon became a problem - one of the goals of unit tests are to have a safety net that runs so fast that it is not a burden at all, and this was rapidly becoming not the case. The solution is to glob multiple test suites into one executable, and while (as shown above) this is mostly do-able, it is not supported and has important limitations.

2) No fixture support - a deal-breaker for me.

So after a while, I switched to Google Test - it is a far more featureful and sophisticated unit testing framework (especially when used with Google Mock) and solves 1) and 2), and moreover, you can still easily use the handy QTestLib features such as QSignalSpy and simulation of GUI events, etc. It was a bit of a pain to switch, but thankfully the project had not advanced too far and many of the changes could be automated.

Personally, I will not be using QtTest over Google Test for future projects - if offers no real advantages that I can see, and has important drawbacks.

Solution 3:

To append to Joe's answer.

Here's a small header I use (testrunner.h), containing an utility class spawning an event loop (which is, for example, needed to test queued signal-slot connections and databases) and "running" QTest-compatible classes:

#ifndef TESTRUNNER_H
#define TESTRUNNER_H

#include <QList>
#include <QTimer>
#include <QCoreApplication>
#include <QtTest>

class TestRunner: public QObject
{
    Q_OBJECT

public:
    TestRunner()
        : m_overallResult(0)
    {}

    void addTest(QObject * test) {
        test->setParent(this);
        m_tests.append(test);
    }

    bool runTests() {
        int argc =0;
        char * argv[] = {0};
        QCoreApplication app(argc, argv);
        QTimer::singleShot(0, this, SLOT(run()) );
        app.exec();

        return m_overallResult == 0;
    }
private slots:
    void run() {
        doRunTests();
        QCoreApplication::instance()->quit();
    }
private:
    void doRunTests() {
        foreach (QObject * test, m_tests) {
            m_overallResult|= QTest::qExec(test);
        }
    }

    QList<QObject *> m_tests;
    int m_overallResult;
};

#endif // TESTRUNNER_H

Use it like this:

#include "testrunner.h"
#include "..." // header for your QTest compatible class here

#include <QDebug>

int main() {
    TestRunner testRunner;
    testRunner.addTest(new ...()); //your QTest compatible class here

    qDebug() << "Overall result: " << (testRunner.runTests()?"PASS":"FAIL");

    return 0;
}

Solution 4:

I don't know that QTestLib is "better" than one framework for another in such general terms. There is one thing that it does well, and that's provide a good way to test Qt based applications.

You could integrate QTest into your new Google Test based setup. I haven't tried it, but based on how QTestLib is architected, it seems like it would not be too complicated.

Tests written with pure QTestLib have an -xml option that you could use, along with some XSLT transformations to convert to the needed format for a continuous integration server. However, a lot of that depends on which CI server you go with. I would imagine the same applies to GTest.

A single test app per test case never caused a lot of friction for me, but that depends on having a build system that would do a decent job of managing the building and execution of the test cases.

I don't know of anything in Qt Creator that would require a seperate project per test case but it could have changed since the last time I looked at Qt Creator.

I would also suggest sticking with QtCore and staying away from the STL. Using QtCore throughout will make dealing with the GUI bits that require the Qt data types easier. You won't have to worry about converting from one data type to another in that case.

Solution 5:

Why not using the unit-testing framework included in Qt? An example : QtTestLib Tutorial.