Thread Jiggling

Overview

Thread jiggler is a simple testing framework for exercising code to find threading problems. It works by modifying classes bytecode at runtime to insert Thread.yield() calls between instructions - "jiggling" the threads. This greatly increases the likelihood of discovering threading issues, and does it without you needing to change your production code.

Background

I was recently researching how to test multithreaded code for threading issues, and found out about a tool from IBM called ConTest, but couldn't find any code I could use myself. So naturally, I thought I'd spike my own.

Consider this canonical simple, but thread unsafe class:

    private int count = 0 ;

    public void count() {
        count++;
    }

The count method's byte code is:

DUP
GETFIELD asm/Foo.counter : I
ICONST_1
IADD
PUTFIELD asm/Foo.counter : I

This provides several places where there could be a context switch, which means that count my be increased, but not stored as expected. Let consider a quick unit test:

    Counter counter = new BadCounter();
    int n = 1000;

    @Test
    public void singleThreadedTest() throws Exception {

        for (int i = 0; i < n; i++) {
            counter.count();
        }

        assertEquals(n, counter.getCount());
    }
    ...

This test runs in a single thread, and passes. Lets try and run this on two threads and see if it fails.

    public void threadedTest() throws Exception {
	
        final CompletionService<Void> service = new ExecutorCompletionService<Void>(Executors.newFixedThreadPool(2));

        for (int i = 0; i < n; i++) {
            service.submit(new Callable<Void>() {
                @Override
                public Void call() {
                    counter.count();
                    return null;
                }
            });
        }

        for (int i = 0; i < n; ++i) {
            service.take().get();
        }

        assertEquals(n, counter.getCount());
     }

This also passes. On my computer I can increase n to 100,000 before it starts to fail consistently.

Expected :1000000
Actual   :999661

Just 0.04% of the tests had a problem. What have we learned? We've learned a simple way to run a multithreaded test, but we've learned that, because we can't control when threads do their work, it's a bit trial and error.

Thread Jiggling

So one problem exercising code to find threading defects is that you can't control when threads will yield. However, we can re-write the bytecode to insert Thread.yield() into the bytecode between instructions. In the above example we can get the code to produce more issues by changing the bytecode:

DUP
GETFIELD asm/Foo.counter : I
INVOKESTATIC java/lang/Thread.yield ()V
ICONST_1
IADD
PUTFIELD asm/Foo.counter : I

Using ASM, we can create a rewriter to insert these invocations. The JigglingClassLoader re-writes classes on the fly, adding these calls. From this we can create a JUnit runner to run use the new class loader for the test.

@Jiggle("threadjiggler.test.*")
public class BadCounterTest {
    ...
}

Now running the test:

Expected :1000000
Actual   :836403

The number of test where we see the threading problem jump to 16%. We've done this with out any recompilation of the code, or impacting on other unit tests running in the same JVM.

Exercise for the Reader

SimpleDateFormat is a well know, non-thread safe class in Java. Write a test that jiggles the class. Why is it not thread-safe? How would you rewrite it so that it was thread safe? How can you do so without using a ThreadLocal, locks or synchronization?

Source Code

The code for this can be found on Github.

Further Reading

I've written a post on testing threaded code for correctness. You may also wish to read more generally:

Tags

ant (2)   antlr (1)   applet (1)   apsectj (1)   asm (1)   aspectj (1)   bdd (1)   blog (1)   cd (1)   coldfusion (2)   concurrency (4)   cucumber (1)   dashing (1)   docker (3)   drop-wizard (1)   drupal (1)   dsl (1)   gatling (1)   geb (1)   gist (13)   git (1)   graphite (1)   groovy (2)   hibernate (2)   ioc (1)   java (44)   java-8 (1)   java-fx (1)   jmeter (2)   jpa (2)   junit (5)   links (12)   maven (12)   middleman (2)   mocking (1)   mongodb (1)   oped (9)   os-x (1)   performance (4)   photos (1)   plaf (14)   ruby (3)   scala (1)   selenium (4)   software (10)   spring (5)   spring-mvc (1)   swing (15)   testing (17)   tips (4)   tomcat (3)   tuples (1)   tutorial (1)   unix (6)   vagrant (2)   web (4)   windows (1)