Monday, 28 January 2008

Consult Popper for the truth about your code

Can unit tests in the standard form of a collection stated conditions and their outcomes be turned into a statement of truth about your software. Proving the behaviour of some bit of code by listing all the possible passing and failing conditions can get a bit tedious.

If I test a few normal conditions, some edge cases and then some failing conditions, have I done enough? Could someone else understand the intention of the code by looking at all the conditions that I could dream up? Perhaps, but there might be another way.

I've been having a look at the Popper library for JUnit which is aimed at providing developers with a means of transforming assertions into theories about the code tested. I suppose that the name of the library is a reference to Karl Popper.

Popper leverages Hamcrest for adding a declarative flavour to assertions and provides a mechanism for testing your code against all permutations of some input range. A good introduction to Popper is this paper (note the link will download the pdf) from the authors of the library.

So what does a theory look like? The example below has been ripped off from the paper and gives the basic idea.



@Theory
public void multiplyIsInverseOfDivide(int amount, int multiplier){
assertThat(new Dollar(amount).times(multiplier).divideBy(multiplier)
.getAmount(), is(amount));
}


The enclosing test class in this case extends a TheoryContainer which takes care of testing the theory against some input.

That's all very helpful but what is "some input"? This is where DataPoints come into the picture.

DataPoints are providers of inputs for the Dollar amount and int multiplier. Admittedly as we're using JUnit 4 we could get almost the same thing by running this as a Parameterized unit test. However with Popper what you get for free is the fact that the TheoryContainer uses all permutations of the data from the DataPoints to test the theory.

DataPoints can be public member of the theory containing class, they can be a range specified by the @Between annotations, or they can come from a ParameterSupplier.

The theory above should always be true, but of course you might want to filter out some specific inputs. To do this you can state some assumptions which enables the theory to ignore some of the possible permutations.

For example:
@Theory
public void multiplyIsInverseOfDivide(int amount, int multiplier){
assumeNotEquals(0, amount);
assertThat(new Dollar(amount).times(multiplier).divideBy(multiplier)
.getAmount(), is(amount));
}


Obviously you'd want to add some traditional unit tests to exercise exceptional conditions as the assumption above filters out an obvious exceptional case and we'd still want to know the behaviour of the code in such cases.

Have we made any progress?

I like the flavour of Hamcrest assertions and I like the fact that the TheoryContainer handles the permutations of input from the DataPoints. On the other hand given the possibly very large range of input data, would we really want to test a theory against all possible inputs? If so then tests might run for a long time, limited only by the number of bits in the inputs?

In other words since we still need to come up with the DataPoints there remains the question of what represents sensible test data. This is where we'd have to consult the testing literature.

Interestingly Popper's author seems to be playing around with Agitar as a means to find useful DataPoints for testing theories. It looks like Popper is being rolled into JUnit 4 from version 4.4, and it seems like there's some questions about how it plays with the rest of JUnit 4.

I think there's enough there to get interested in and one of my next task is to work it in to some of the examples used in our TDD course which Keith runs. I'll post on my experiences another time.

0 comments :