Share and Enjoy

David Saff's blog of technological joys.

Tuesday, May 01, 2007


Exploring with JUnit Factory: not null, I assume

In my last post, we used Popper to create a Theory about scoring a bowling game. The end result boiled down to roughly this:

public class BowlingTests extends TheoryContainer {
public static Game STARTING_GAME = new Game();
public static Bowl THREE = new Bowl(3);
public static Bowl FOUR = new Bowl(4);

public void shouldBeTenFramesWithTwoRollsInEach(Game game, Bowl first,
Bowl second) {
assumeThat(game.isAtBeginning(), is(true));
assumeThat(game.getPlayers().size(), is(1));
assumeThat(first.isStrike(), is(false));
assumeThat(second.completesSpareAfter(first), is(false));

for (int frame = 0; frame < 10; frame++) {

assertThat(game.isGameOver(), is(true));

I believe Theories are useful for any Java developer. However, so far, I've been concentrating on test-driven development (TDD). The heartbeat of TDD with Tests and Theories is similar to that with just Tests, with one essential difference.

  1. Use automated or manual exploration to find any data points that invalidate the current Theories. If such a data point exists, add it to the currently accepted data points. Otherwise, write a focused Test or Theory for the next bit of functionality needed. At the end of this step, a Test or Theory should fail.
  2. Change the code so that all current Tests and Theories pass.
  3. Refactor to the best design that passes the current tests and theories.
  4. Repeat.

We've already done step 1--since there were no Theories, I wrote a new one, above, about ten frames with two rolls. I'll do step 2 off-screen...

There. I've written the simplest code I can think of to pass this theory--we can hard-code most of the boolean answers, especially making sure that isGameOver always returns true. Running the test in Eclipse tells me it passes for all of the parameters I've considered (all three of them). However, what about the infinite number of parameters I haven't considered? I can stare at the code and consider other options, or I can just ask JUnit Factory.

JUnit Factory is a free service accessed through a free Eclipse plug-in. Its primary purpose is to generate characterization tests for domain classes. It uses static analysis, dynamic analysis, and tuned heuristics in an attempt to characterize the current behavior of your classes, especially in unanticipated circumstances.

By turning the powerful eye of JUnit Factory on a TheoryContainer, I can see automatically if there are any inputs to my theory that pass the assumptions, but fail the assertions. I've already downloaded the plug-in, and I've made sure I meet these prerequisites:

I focus my editor on BowlingTests push Shift-F9, and in about 30 seconds, I get my first set of characterization tests. Remember that these are tests of the methods of my TheoryContainer, not of the Game or Bowl class themselves. When scanning these tests, I'm looking only for parameters and outcomes, not the assertions themselves, which are unlikely to be interesting--all of my Theory methods, remember, return void. There's usually a few that indicate proper returns from my Theory methods, and a few indicating exceptional returns. Scanning the outline, I see:


So JUnitFactory found at least one way to make the Theory pass, and three ways to make it throw a NullPointerException. Since my tests pass with my current data points, there must be new data points I need to include to find these exceptional behaviors. Let's look at the first NullPointerException test:

public void testShouldBeTenFramesWithTwoRollsInEachThrowsNullPointerException() throws Throwable {
BowlingTests bowlingTests = new BowlingTests();
try {
bowlingTests.shouldBeTenFramesWithTwoRollsInEach(new Game(), BowlingTests.THREE, null);
fail("Expected NullPointerException to be thrown");
} catch (NullPointerException ex) {
assertNull("ex.getMessage()", ex.getMessage());
assertThrownBy(BowlingTests.class, ex);
assertNotNull("bowlingTests.assume", getPrivateField(bowlingTests, "assume"));

This is annoying--JUnitFactory is passing a null Bowl to my theory. Of course, my Theory currently claims that it accepts any value of type Bowl, and null fits that description. The other two NullPointerException tests make use of another null Bowl, and a null Game.

In order to deal with this new information, we add the two new data points to our TheoryContainer:

public static Game STARTING_GAME = new Game();
public static Game NULL_GAME = null;

public static Bowl THREE = new Bowl(3);
public static Bowl FOUR = new Bowl(4);
public static Bowl NULL_BOWL = null;

Running the test, it fails with a NullPointerException, as expected. Now, the theory needs to be updated to assume that the parameters are not null. Using just Popper, we can simply use an attribute of the @Theory annotation, @Theory(nullsAccepted=false). Unfortunately, JUnitFactory does not understand this attribute, so instead, we'll have to explicitly add the assumptions:

public void shouldBeTenFramesWithTwoRollsInEach(Game game, Bowl first,
Bowl second) {
assumeThat(game, isNotNull());
assumeThat(first, isNotNull());
assumeThat(second, isNotNull());

// ...

This is a common pattern when using Popper together with JUnit Factory. In order to make it as painless as possible, you can use a shorthand from Popper 0.5:

public void shouldBeTenFramesWithTwoRollsInEach(Game game, Bowl first,
Bowl second) {
assumeNotNull(game, first, second);

// ...

Now, generating the tests, we see the following methods:


This is what we want to see: sometimes the parameters are invalid, but these are caught by our assumptions. Anything getting past our assumptions is passing the test. Excellent.

This may feel like a lot of work for a simple skeleton. However, this work will be paid off as we move forward--this Theory actually says a lot about bowling games, and as we make Game and Bowl more sophisticated, this Theory will be waiting to catch any weirdness introduced.

And, in the future, I may publish an Eclipse plug-in that will better manage this "mash-up" of Theories and JUnit Factory, for example, by automatically inserting assumeNotNull where desirable.

Comments: Post a Comment

<< Home


February 2005   June 2005   March 2006   August 2006   December 2006   April 2007   May 2007   January 2008  

This page is powered by Blogger. Isn't yours?

Subscribe to Posts [Atom]