dave astels

A friend of mine, Misko Hevery, has written a very cool opensource tool for analyzing Java projects and scoring them in terms of how testable they are. His plans are to have it point out what’s wrong, and make suggestions as to what you can do to improve the situation.

Check it out: Testability Explorer

Well worth looking at.

Several people have asked me for the source from my last book “TDD: A Practical Guide”. A lot has happened since that book, and the files from it that were once posted on the Saorsa & Adaption sites were lost. Unfortunately I’ve had to tell people that the source was now longer in existence.

Well, today I was doing some routine housekeeping and… Huzah!!! I found a zip of those very source files. For anyone who has been looking for them… I’m pleased to say that I’m making them available at long last. You can download it here

My coworker, Paul Zabelin, posted here on some ideas that we’ve been experimenting with using RSpec stories and JRuby.

FitNesse uses a single VM to run all pages in a test suite. This causes issues with static variables. They should be reset after each test page.

There are also issues with singletons. They, too, should be reset after each test page has run. So… you have to hunt down and reset each and every singleton in the system. As I was thinking about this, an approach occurred to me that will help with this, as well as making your singletons visible and easily findable.

Marking Singletons

I’ll start by making an interface to mark singletons:

public interface Singleton
{
}

Now I can mark each singeton, for example:

public class PrintQueue implements Singleton
{
    private static PrintQueue instance = null;

    public static PrintQueue getInstance () {
        if (instance == null) {
            instance = new PrintQueue();
        }
        return instance;
    }

    private PrintQueue() {
        //....
    }
}

So that makes the singletons easy to find.. just pull up implementors of Singleton.

Managing Singletons

Next I want to actively manage them. I’ll need to keep track of them:

public class SingletonManager
{
    private static List singletons = new ArrayList();

    public static void registerSingleton (Singleton aSingleton) {
        if (singletons.contains(aSingleton)) return;
        singletons.add(aSingleton);
    }
}

And I need to hook our singletons into the manager:

public class PrintQueue implements Singleton
{
    public static getInstance () {
        if (instance == null) {
            instance = new PrintQueue();
            SingletonManager.registerSingleton(instance);
        }
        return instance;
    }
}

Now I have a lists of all the singletons in the system. This lets me do interesting things.

Resetting Singletons

The first thing (maybe the only thing… and it’s enough… it’s the whole reason for this article) is to reset all the singletons.

public class SingletonManager
{
    public static void reset () {
        Iterator singetonIt = singetons.iterator();
        while (singletonIt.hasNext()) {
            Singleton aSingleton = (Singleton)singletonIt.next();
            aSingleton.reset();
        }
    }
}

Singleton needs a reset method now:

public interface Singleton
{
    void reset();
}

which needs to be implemented:

public class PrintQueue implements Singleton
{
    public void reset() {
        // do anything required to cleanup
        instance = null;
    }
}

Cleaning Up

Now I can use this reset facility from a fixture on the TearDown page on my test suites to reset all my singletons after each test page.

Of course, a better approach would be to not use singletons.

When refactoring some code recently, my pair and I came across yet another reason why state based testing is bad.. and even more.. why testing using access to private state is incredibly bad.

We were doing some serious refactoring…. tackling a class that had become a catch-all… thousands of lines and several dozen methods… and splitting it up into multiple classes, each with a single responsibility.

We did this by creating new classes and moving groups of methods and associated instance variables to the new class, and leaving delegating methods in the original class. These we will clean up on the next pass.

We did this for one responsibility and suddenly tests that had been written long ago started to fail. This perplexed us since the original methods remainded in place in the original class, simply delegating to the real methods in the new class.

A quick exploration of the failing test revealed what was happening. The test in question was using reflection to get access to private fields in the class being tested. This went both ways… private fields were being set with fake values for purposes of the test, and assertions were being made on the values of private fields.

This is evidence of a serious misunderstanding of how you should be writing tests: they should be based on behaviour and function.. not state. An object’s state is simply an implementation detail, and should not be exposed to, or relied upon by, other objects. Guess what.. that means no getters & setters, either. But that’s a discussion for another time.

The problem that we ran into with accessing private fields in a test is that it makes the tests brittle, throwing a monkey wrench into the process of refactoring. When we were refactoring we made a change that did not effect the behaviour. That should NOT cause tests to break. But it did.

Why? Because those tests were accessing a private field of the class via reflection.. and we had moved that field to a new class.

Example:

Let’s start with the example from the first paper in the references section below:

class FieldTest {
  public String publicString = "Foobar";
  private String privateString = "Hello, World!";
}

We’ll pull in the test from that article as well (even if it is contrived):

...

FieldTest f = new FieldTest();
String s = (String)PrivateAccessor.getPrivateField(f, "privateString");
assertEquals("Hello, World!", s);

Now, say we add some behaviour that uses the private field:

public String makeHeader() {
  return "<h1>" + privateString + "</h1>";
}

so now we see some behaviour that doesn’t belong in the class (just go with me on this).. creating the header.. so we can extract that and the variable that only it uses to a new class:

class HeaderCreator {
  private String privateString = "Hello, World!";
  public String makeHeader() {
    return "<h1>" + privateString + "</h1>";
  }
}

class FieldTest {
  public String publicString = "Foobar";
  public String makeHeader() {
    return new HeaderCreator().makeHeader();
  }
}

Now the test that relies on accessing privateField will fail since we’ve, quite legitimately, moved it.

So, it’s a bad idea to write tests based on the state of an object, especially when that state is only available in private fields. That being said, don’t do it..and if you are.. stop doing it… and fix where you did do it.

References

1. Subverting Java Access Protection for Unit Testing by Ross Burton (NOTE: this article presents what NOT to do)
2. State vs Interaction Based Testing by Nat Pryce
3. Mocks Aren’t Stubs by Martin Fowler

This is an article that appeared on my Artima.com blog on February 23, 2004

A while ago there was a bit of fuss on the testdrivendevelopment Yahoo group about the idea of limiting yourself to one assertion per test method, which is a guideline that others and I offer for TDD work.

An address parser was the example of a situation where it was argued that multiple assertions per test made sense. Date was formatted with one address per line, each in one of the following formats:

1. ADDR1$ADDR2$CSP$COUNTRY
2. ADDR1$ADDR2$CSP
3. ADDR1$CSP$COUNTRY
4. ADDR1$CSP

The poster went on to say:

My first inclination is/was to write a test like this:

    a = CreateObject("Address")
    a.GetAddressParts("ADDR1$ADDR2$CITY IL 60563$COUNTRY")
    AssertEquals("ADDR1", a.Addr1)
    AssertEquals("ADDR2", a.Addr2)
    AssertEquals("CITY IL 60563", a.CityStatePostalCd)
    AssertEquals("Country", a.Country)

They didn’t see how to achieve this with one assertion per test, as there are obviously four things to test in this case. I decided that rather than simply reply, I would write some tests and code to illustrate my view on the matter, and offer a solid response.

For this problem, I chose Squeak Smalltalk (see www.squeak.org) and Java. For the sake of conciseness, I’ll omit any required accessors.

So, where to start? Well, when doing TDD it often makes sense to start with something simple to quickly and easily get some code written and working. Then it can be extended and evolved in response to further test driving. Here the simplest case is: tt>ADDR1$CSP. There are two requirements in the parsing of this example: that the ADDR1 was recognized, and that the CSP was recognized. Viewed this way, we need two tests. We start with one for ADDR1:

Squeak:

    testAddr1
        | anAddress |
        anAddress := Address from: 'ADDR1$CITY IL 60563'.
        self assert: anAddress addr1 equals: 'ADDR1'

Java:

    public void testAddr1() throws Exception {
        Address anAddress = new Address("ADDR1$CITY IL 60563");
        assertEquals("ADDR1",  anAddress.getAddr1());
    }

To get this to pass we need an Address class and a from: factory method, which creates an instance and has it parse the address string. For brevity, I’ll skip the “return literal” step.

Squeak:

    Object subclass: #Address
        instanceVariableNames: 'addr1'
        classVariableNames: ''
        poolDictionaries: ''
        category: 'Adaption-One Assertion Test'
(class method)
from: aString
    ^self new parse: aString;
        yourself

parse: aString
    | parts |
    parts := aString findTokens: '$'.
    addr1 := (parts at: 1)

e87520e089973e716b9ec17744f135c7

    public class Address {
    private String addr1;

    public Address(String aString) {
        parse(aString);
    }

    private void parse(String aString) {
        StringTokenizer parts = new StringTokenizer(aString, "$");
        addr1 = parts.nextToken();
    }
}

That’s well & good. The next test is for CSP.

Squeak:

    testCsp
        | anAddress |
        anAddress := Address from: 'ADDR1$CITY IL 60563'.
        self assert: anAddress csp equals: 'CITY IL 60563'

Java:

    public void testCsp() throws Exception {
        Address anAddress = new Address("ADDR1$CITY IL 60563");
        assertEquals("CITY IL 60563",  anAddress.getCsp());
    }

Address>>parse: will need to be extended (and we need to add a csp instance variable and accessors):

Squeak:

    parse: aString
        | parts |
        parts := aString findTokens: '$'.
        addr1 := (parts at: 1).
        csp := (parts at: 2)

Java:

    private void parse(String aString) {
        StringTokenizer parts = new StringTokenizer(aString, "$");
        addr1 = parts.nextToken();
        csp = parts.nextToken();
    }

So. We have two tests for this one situation. Notice the duplication in the tests… the creation of the instance of Address that is being probed. This is the fixture. After refactoring, we have:

Squeak:

    TestCase subclass: #Addr1CspTests
        instanceVariableNames: 'anAddress '
        classVariableNames: ''
        poolDictionaries: ''
        category: 'Adaption-One Assertion Test'!
setUp
    anAddress := Address from: 'ADDR1$CITY IL 60563'

testAddr1
    self assert: anAddress addr1 equals: 'ADDR1'

testCsp
    self assert: anAddress csp equals: 'CITY IL 60563'

ab4725d503a2aa390ab1b11b1a710b99

    public class Addr1CspTests extends TestCase {
    private Address anAddress;

    protected void setUp() throws Exception {
        anAddress = new Address("ADDR1$CITY IL 60563");
    }

    public void testAddr1() throws Exception {
        assertEquals("ADDR1",  anAddress.getAddr1());
    }

    public void testCsp() throws Exception {
        assertEquals("CITY IL 60563",  anAddress.getCsp());
    }
}

So, a fixture that creates the Address instance from the string, and very simple tests that focus on each aspect of that fixture.

The next simplest case is the obvious choice for the next fixture:

Squeak:

    setUp
        anAddress := Address from: 'ADDR1$CITY IL 60563$COUNTRY'

Java:

    protected void setUp() throws Exception {
        anAddress = new Address("ADDR1$CITY IL 60563$COUNTRY");
    }

This set of tests will include ones for addr1 and csp as before (refactoring this to remove that duplication is left to the reader) as well as a new test for country:

Squeak:

    testCountry
        self assert: anAddress country equals: 'COUNTRY'

Java:

    public void testCountry() throws Exception {
        assertEquals("COUNTRY",  anAddress.getCountry());
    }

As before, an instance variable and associated accessors need to be added to the Address class.

This drives Address>>parse: to evolve:

Squeak:

    parse: aString
        | parts |
        parts := aString findTokens: '$'.
        addr1 := (parts at: 1).
        csp := (parts at: 2).
        country := (parts at: 3 ifAbsent: [''])

Java:

    private void parse(String aString) {
        StringTokenizer parts = new StringTokenizer(aString, "$");
        addr1 = parts.nextToken();
        csp = parts.nextToken();
        country = parts.hasMoreTokens() ? parts.nextToken() : "";
    }

From here on, the evolution gets a bit more complex, as we add the ADDR2 option to the mix.

Conclusion

So we took a situation that was thought to require multiple assertions in a test and did it in such as way as to have only one assertion per test.

The key is that instead of using a single TestCase subclass with a complex (i.e. multiple assertion) tests for each situation, we made each of those situations into a separate fixture. Each fixture is implemented by a separate subclass of TestCase. Now each test focuses on a very small, specific aspect of that particular fixture.

I’m convinced writing tests like this is a useful approach. One advantage is that the resulting tests simpler and easier to understand. Just as important, and maybe more so, is that by adding the specification of the behavior one tiny piece at a time, you drive toward evolving the code in small, controllable, understandable steps.

It also fits better into the test fixture centered approach that is the recommended way to organize your tests. We set up the object to test in the setUp method, and tested each aspect of it in individual tests methods.

As I’ve been writing this, something clicked. I see these test methods as specifications of tiny facets of the required behavior. Thus, it makes sense to me to be as gradual as possible about it, driving the evolution of the code in the smallest steps possible. Striving for one assertion per test is a way to do that.

If, however, you view test methods as strictly performing verification, then I can see how it might be seen to make sense to invoke some code and then test all the postconditions. But this view is not TDD, and doesn’t buy you all of the benefits of TDD. I contend that central to TDD is this notion of working in the smallest steps possible, both for the finest-grained long-term verification, and for the most flexible design evolution. Furthermore, this is best done by striving to keep tests as small, focused and simple as ossible.

Aiming for one assertion per test is one way to get there.