How do I Dynamically create a Test Suite in JUnit 4?

To create a dynamic test suite, you need to use the @RunWith annotation. There are two common ways to use it:

@RunWith(Suite.class)

This allows you to specify, which classes compose the test suite in question. This is equivalent to the JUnit 3 style:

import junit.framework.TestSuite;
import junit.framework.TestCase;

public final class MasterTester extends TestCase {

  public static TestSuite suite() {
    TestSuite suite = new TestSuite();
    suite.addTestSuite(TestClass1.class);        
    suite.addTestSuite(TestClass2.class);
    // etc...
    return suite;
  }
}

The equivalent JUnit 4 class will be:

import org.junit.runners.Suite;

@RunWith(Suite.class)
@SuiteClasses({TestClass1.class, TestClass2.class})
public final class MasterTester {

}

@RunWith(AllTests.class)

This allows you to dynamically specify the tests, which compose the test suite. If your tests are not known until runtime, you cannot specify them in the annotations. You can use this construction instead. So, if the JUnit 3 code is:

import junit.framework.TestCase;
import junit.framework.TestSuite;
import junit.framework.Test;

public final class MasterTester extends TestCase {

  public static TestSuite suite() {
    TestSuite suite = new TestSuite();
    for (Test test : findAllTestCasesRuntime()) {
      suite.addTest(test);
    }
    return suite;
  }
}

The equivalent JUnit 4 code will be:

import org.junit.runners.AllTests;
import junit.framework.TestSuite;
import junit.framework.Test;

@RunWith(AllTests.class)
public final class MasterTester {

  public static TestSuite suite() {
    TestSuite suite = new TestSuite();
    for (Test test : findAllTestCasesRuntime()) {
      suite.addTest(test);
    }
    return suite;
  }
}

I've tried this using JUnit 4.8 and it works:

@RunWith(AllTests.class)
public class SomeTests
{
    public static TestSuite suite()
    {
        TestSuite suite = new TestSuite();

        suite.addTest(new JUnit4TestAdapter(Test1.class));
        suite.addTest(new JUnit4TestAdapter(Test2.class));

        return suite;
     }
}

I found Classpath suite quite useful when used with a naming convention on my test classes.

https://github.com/takari/takari-cpsuite

Here is an example:

import org.junit.extensions.cpsuite.ClasspathSuite;
import org.junit.runner.RunWith;

@RunWith(ClasspathSuite.class)
@ClassnameFilters({".*UnitTest"})
public class MySuite {
}

I'm not sure what gatherTestClasses() does, but let's say it returns some tests when the OS is Linux and different tests when the OS is Windows. You can replicate that in JUnit 4.4 with assumptions:

@Test
public void onlyOnLinux() {
    assumeThat(getOS(), is(OperatingSystem.LINUX));
    // rest of test
}

@Test
public void onlyOnWindows() {
    assumeThat(getOS(), is(OperatingSystem.WINDOWS));
    // rest of test
}

@Test
public void anyOperatingSystem() {
    // just don't call assumeThat(..)
}

The implementation of getOS() and OperatingSystem being your custom code.