How to run concurrency unit test?

How to use junit to run concurrency test?

Let's say I have a class

public class MessageBoard
{
    public synchronized void postMessage(String message)
    {
        ....
    }

    public void updateMessage(Long id, String message)
    {
        ....
    }
}

I wan to test multiple access to this postMessage concurrently. Any advice on this? I wish to run this kind of concurrency test against all my setter functions (or any method that involves create/update/delete operation).

Solution 1:

Unfortunately I don't believe you can definitively prove that your code is thread-safe by using run-time testing. You can throw as many threads as you like against it, and it may/may not pass depending on the scheduling.

Perhaps you should look at some static analysis tools, such as PMD, that can determine how you're using synchronisation and identify usage problems.

Solution 2:

I would recommend using MultithreadedTC - Written by the concurrency master himself Bill Pugh (and Nat Ayewah). Quote from their overview:

MultithreadedTC is a framework for testing concurrent applications. It features a metronome that is used to provide fine control over the sequence of activities in multiple threads.

This framework allows you to deterministically test every thread interleaving in separate tests

Solution 3:

You can only prove the presence of concurrent bugs, not their absence.

However you can write a specialized test runner that spawns several concurrent thread and then calls your @Test annotated methods.

Solution 4:

In .NET, there are tools like TypeMock Racer or Microsoft CHESS that are designed specifically for unit testing concurrency. These tools not only find multithreading bugs like deadlocks, but also give you the set of thread interleaves that reproduce the errors.

I'd imagine there's something similar for the Java world.

Solution 5:

Running concurrently can lead to unexpected results. For instance, I just discovered, that while my Test suite with 200 tests pass when executed one by one, it fails for concurrent execution, I dug that and it wasn't a thread safety problem, but a test depending on another one, which is a bad thing and I could solve the problem.

Mycila work on JUnit ConcurrentJunitRunner and ConcurrentSuite is very interesting. The article seems a little outdated compared to the latest GA release, in my examples I will show the updated usage.

Annotating a test class like the following will cause to execute test methods concurrently, with a concurrency level of 6:

import com.mycila.junit.concurrent.ConcurrentJunitRunner;
import com.mycila.junit.concurrent.Concurrency;

@RunWith(ConcurrentJunitRunner.class)
@Concurrency(6)
public final class ATest {
...

You can also run all the test classes concurrently:

import com.mycila.junit.concurrent.ConcurrentSuiteRunner;

@RunWith(ConcurrentSuiteRunner.class)
@Suite.SuiteClasses({ATest.class, ATest2.class, ATest3.class})
public class MySuite {
}

The Maven dependency is:

<dependency>
    <groupId>com.mycila</groupId>
    <artifactId>mycila-junit</artifactId>
    <version>1.4.ga</version>
</dependency> 

I am currently investigating how to run methods multiple times and concurrently with this package. It may be possible already, if anyone has an example let me know, below my homebrewed solution.

@Test
public final void runConcurrentMethod() throws InterruptedException {
    ExecutorService exec = Executors.newFixedThreadPool(16);
    for (int i = 0; i < 10000; i++) {
        exec.execute(new Runnable() {
             @Override
             public void run() {
                 concurrentMethod();
             }
        });
    }
    exec.shutdown();
    exec.awaitTermination(50, TimeUnit.SECONDS);
}

private void concurrentMethod() {
    //do and assert something
}

As others noted, is true that you can never be sure whether a concurrency bug would show up or not, but with ten of thousands, or hundred of thousands executions with a concurrency of, say 16, statistics is on your side.