JUnit 4.11 - What's new? Parameterized descriptions

JUnit 4.11 is a major release of the popular testing framework. Amongst the changes were changes to how Parameterized tests are described by the various runners (maven surefire, Eclipse etc).

With JUnit 4.10, when you ran a Parameterized test, all you got in the description was an integer index, which wasn’t very useful. Taking the following test class as an example:

@RunWith(Parameterized.class)
public class ParameterizedTest {
  @Parameters
  public static Iterable<Object[]> data() {
    return Arrays.asList(new Object[][] {
      { 0, 0 },
      { 1, 1 },
      { 2, 1 },
    });
  }

  private int input;
  private int expected;

  public ParameterizedTest(int input, int expected) {
    this.input = input;
    this.expected = expected;
  }

  @Test public void test() {
    assertEquals(expected, Fibonacci.compute(input));
  }

  public static void main(String[] args) {
    RunListener runListener = new RunListener() {
      public void testFailure(Failure failure) throws Exception {
        System.out.println("testFailure " + failure.getDescription().getDisplayName());
      }
    };
    JUnitCore jUnitCore = new JUnitCore();
    jUnitCore.addListener(runListener);
    jUnitCore.run(ParameterizedTest.class);
  }
}

When we run this (as a main method), we get the following output:

testFailure test[0](uk.co.farwell.ParameterizedTest)
testFailure test[1](uk.co.farwell.ParameterizedTest)
testFailure test[2](uk.co.farwell.ParameterizedTest)

The output isn’t very useful when we’re debugging tests, especially if there are lots of them. So, JUnit 4.11 allows the description of the test to be modified by adding a annotation parameter:

@Parameters(name = "{index}: fib({0}) should be {1}")
public static Iterable<Object[]> data() {
  return Arrays.asList(new Object[][] {
    { 0, 0 },
    { 1, 1 },
    { 2, 1 },
  });
}

The {0} is replaced by the first parameter for the test, and so on. Now the description is much more, well, descriptive:

testFailure test[0: fib(0) should be 0](uk.co.farwell.ParameterizedTest)
testFailure test[1: fib(1) should be 1](uk.co.farwell.ParameterizedTest)
testFailure test[2: fib(2) should be 1](uk.co.farwell.ParameterizedTest)

However, when running with the junit runner in Eclipse, you need to be careful. When it creates the descriptions for the test classes, it uses (approximately) ‘uk.co.farwell.ParameterizedTest(test)’. It then uses the name of the test to work out which class and method to jump to when you double click on the test in the junit view. So if, as above, we use parentheses in the description, this doesn’t work. It also screws up the display of the test name in the junit view.

The moral of the story: Don’t use parentheses in your descriptions for parameterized tests.

JUnit 4.11 - What's new? Rules

JUnit 4.11 is a major release of the popular testing framework. Amongst the things I contributed were changes to @Rule and @ClassRule, to allow these annotations to be used on a method as well as a public field. I wanted this because I use Rules heavily in Java, and I wanted to be able to use them in Scala as well. Mainly to make the transition from Java to Scala easier.

Let’s take an example:

public class JavaRuleTest {
  @Rule public ExpectedException expectedException = ExpectedException.none();
 
  @Test
  public void test() {
    expectedException.expect(NumberFormatException.class);
    Integer.valueOf("foo");
  }
}

So this test uses the ExpectedException rule which allows you to say that you expect to receive an exception, in this case a NumberFormatException. Which we do. So the test passes. If we translate this directly to Scala, we get:

class ScalaRuleTest {
  @Rule val expectedException = ExpectedException.none()
 
  @Test def test() {
    expectedException.expect(classOf[NumberFormatException])
    Integer.valueOf("foo")
  }
}

But this produces an error:

The @Rule 'expectedException' must be public.

Although we’re defining a ‘public’ field expectedException, Scala implements this as a private field with an accessor method. So JUnit expects a public field, but finds a private one; the test fails. But now, with JUnit 4.11, we can apply a @Rule to a method:

class ScalaRuleTest {
  val expectedException = ExpectedException.none()
 
  @Rule def expectedExceptionDef = expectedException
 
  @Test def test() {
    expectedException.expect(classOf[NumberFormatException])
    Integer.valueOf("foo")
  }
}

We still need the val expectedException, so that we can refer to it in test(), but we add expectedExceptionDef and move the @Rule annotation to there. And now the test passes.

Similarly, for @ClassRule, you can do the same, but Scala doesn’t have static variables/methods, so you’ll need to add it to the companion object:

object ScalaClassRuleTest {
  @ClassRule def externalResource = new ExternalResource() {
 protected override def before() = println("before")
 protected override def after() = println("after")
  }
}

class ScalaClassRuleTest {
  @Test def test() {
 System.out.println("a test")
  }
}

This change allows me to take my Java code and translate it (almost) directly into Scala, as a stepping stone to rewriting everything completely to use Scalatest or Specs2. Also, other Java programmers may find it useful :-)