dave astels

I’m pleased to say that we are opensourcing our C++ xUnit-style framework. See the official announcement.

Tags: agile C++ google opensource programming tdd testing tools

Several people have asked me for the source from my last book “TDD: A Practical Guide”. A lot has happened since that book, and the files from it that were once posted on the Saorsa & Adaption sites were lost. Unfortunately I’ve had to tell people that the source was now longer in existence.

Well, today I was doing some routine housekeeping and… Huzah!!! I found a zip of those very source files. For anyone who has been looking for them… I’m pleased to say that I’m making them available at long last. You can download it here

Tags: books code java programming tdd

IEEE Software will have a special issue on test-driven development early next summer, edited by Grigori Melnik and Ron Jeffries. I’m a reviewer (as is Brian Marick and likely several others). Check out the Call for Papers and send in submissions. The deadline is December 1.

Tags: tdd

In the fine tradition of Hollywood (who these days is remaking everything from old movies, comic books, and video games in lieu of coming up with original ideas) I am doing a remake of the very first post on my blog (which itself was a repost from my Artima.com blog from Feb 2004). The original post created quite a stir at the time, and lead to many of the ideas that have evolved my understanding of TDD and now BDD.

---

Once upon a time (early 2004) there was a bit of fuss on the testdrivendevelopment Yahoo group about the idea of limiting yourself to one assertion per test method, which is a guideline that others and I offer for TDD work.

An address parser was the example of a situation where it was argued that multiple assertions per test made sense. Date was formatted with one address per line, each in one of the following formats:

1. ADDR1$ADDR2$CSP$COUNTRY
2. ADDR1$ADDR2$CSP
3. ADDR1$CSP$COUNTRY
4. ADDR1$CSP

The poster went on to say:

My first inclination is/was to write a test like this (in Java but you get the idea):

a = CreateObject("Address")
a.GetAddressParts("ADDR1$ADDR2$CITY IL 60563$COUNTRY")
AssertEquals("ADDR1", a.Addr1)
AssertEquals("ADDR2", a.Addr2)
AssertEquals("CITY IL 60563", a.CityStatePostalCd)
AssertEquals("Country", a.Country)

They didn’t see how to achieve this with one assertion per test, as there are obviously four things to test in this case. I decided that rather than simply reply, I would write some tests and code to illustrate my view on the matter, and offer a solid response.

For the original post, I chose Squeak Smalltalk and Java. In this remake I’m going to use just Ruby. In the original I said “For the sake of conciseness, I’ll omit any required accessors.” Well.. this time around I’m not going to use any accessors, since I am more & more of the opinion that they are quite simply an idea whose sole purpose is to undermine the very essence of Object-Oriented software. I’ll also be using rSpec rather than an xUnit framework.

So, where to start? Well, it often makes sense to start with something simple to quickly and easily get some code written and working. Then it can be extended and evolved. Here the simplest case is: ADDR1$CSP. There are two requirements in the parsing of this example: that the ADDR1 was recognized, and that the CSP was recognized. Viewed this way, we need two examples.

If we want to avoid accessors (and for this exercise, I do) we need another way to see inside the object were working with. This is an address.. let’s say (for the sake of argument & this exercise) that one of the purposes of having an address is to generate address labels of some sort. We could do that with an address label maker object that would interrogate the address object for it’s contents and use them. A better way is to have an address label builder that the address object gives information to. It would have methods such as street_number=, csp=, etc. This is something that we can use Mocks Objects for in our examples.

So, let’s start with an example for ADDR1:

context "An address with street & csp" do

  specify "should capture street information" do
    builder = mock("builder", null_object => true)
    builder.should_receive(:addr1=).once.with("ADDR1")
    addr = Address.from_string("ADDR1$CITY IL 60563")
    addr.use(builder)
  end

end

To get this to pass we need an Address class, a from_string factory method, and a use method.

class Address
  def self.from_string(address_string)
    Address.new
  end

  def use(a_builder)
  end
end

This lets our spec run and fail:

An address with street & csp
- should capture street information (FAILED - 1)

1)
Spec::Api::MockExpectationError in 'An address with street & csp should capture street information'
Mock 'builder' expected 'street=' once, but received it 0 times
./address_spec.rb:7:in `'

Finished in 0.003395 seconds

1 specification, 1 failure

Now let’s make that work:

class Address
  def self.from_string(address_string)
    addr1, rest = address_string.split('$')
    Address.new(addr1)
  end

  def initialize(addr1)
    @addr1 = addr1
  end

  def use(a_builder)
    a_builder.addr1 = @addr1
  end
end

This makes the example work:

An address with street & csp
- should capture street information

Finished in 0.003003 seconds

1 specification, 0 failures

That’s well & good. The next test is for CSP.

specify "should capture csp information" do
  builder = mock("builder", :null_object => true)
  builder.should_receive(:csp=).once.with("CITY IL 60563")
  addr = Address.from_string("ADDR1$CITY IL 60563")
  addr.use(builder)
end

Resulting in:

An address with street & csp
- should capture street information
- should capture street information (FAILED - 1)

1)
Spec::Api::MockExpectationError in 'An address with street & csp should capture street information'
Mock 'builder' expected 'csp=' once, but received it 0 times
./address_spec.rb:15:in `'

Finished in 0.005866 seconds

2 specifications, 1 failure

Address#from_string will need to be extended (and we need to add csp support):

class Address
  def self.from_string(address_string)
    addr1, csp, rest = address_string.split('$')
    Address.new(addr1, csp)
  end

  def initialize(addr1, csp)
    @addr1 = addr1
    @csp = csp
  end

  def use(a_builder)
    a_builder.addr1 = @addr1
    a_builder.csp = @csp
  end
end

So. We have two tests for this one situation. Notice the duplication in the tests… the creation of builder & address. This is the context. After refactoring, we have:

require 'address'

context "An address with street & csp" do

  setup do
    @builder = mock("builder", :null_object => true)
    @addr = Address.from_string("ADDR1$CITY IL 60563")
  end

  specify "should capture street information" do
    @builder.should_receive(:addr1=).once.with("ADDR1")
    @addr.use(@builder)
  end

  specify "should capture csp information" do
    @builder.should_receive(:csp=).once.with("CITY IL 60563")
    @addr.use(@builder)
  end
end

So, a context that creates the Address instance from the string as well as the mock, and very simple examples that focus on each aspect of that context.

Note that we don’t have any direct expectations in these examples… What’s up? Well, the mock is autoverifying itself at the end of each example, and that is where our expectations are.. on @address’s interaction with the mock.

The next simplest case is the obvious choice for the next context:

context "An address with street, csp, and country" do

  setup do
    @builder = mock("builder", :null_object => true)
    @addr = Address.from_string("ADDR1$CITY IL 60563$COUNTRY")
  end

end

This set of tests will include ones for addr1 and csp as before (since that behaviour is required in this new context) as well as a new test for country:

specify "should capture country information" do
  @builder.should_receive(:country=).once.with("COUNTRY")
  @addr.use(@builder)
end

As before, an instance variable and such need to be added to the Address class.

This drives Address to evolve:

class Address
  def self.from_string(address_string)
    addr1, csp, country, rest = address_string.split('$')
    Address.new(addr1, csp, country)
  end

  def initialize(addr1, csp, country)
    @addr1 = addr1
    @csp = csp
    @country = country
  end

  def use(a_builder)
    a_builder.addr1 = @addr1
    a_builder.csp = @csp
    a_builder.country = @country
  end
end

Now that we are supporting country, we can go back to the first context and add an example specifying that no country information should be provided to the builder:

specify "should not capture country information" do
  @builder.should_not_receive(:country=)
  @addr.use(@builder)
end

A slight tweak to Address#use will make this work:

def use(a_builder)
  a_builder.addr1 = @addr1
  a_builder.csp = @csp
  a_builder.country = @country unless @country.nil?
end

From here on, the evolution gets a bit more complex, as we add the ADDR2 option to the mix.

The final code and output is:

address_spec.rb

require 'address'

context "An address with street & csp" do

  setup do
    @builder = mock("builder", :null_object => true)
    @addr = Address.from_string("ADDR1$CITY IL 60563")
  end

  specify "should capture street information" do
    @builder.should_receive(:addr1=).once.with("ADDR1")
    @addr.use(@builder)
  end

  specify "should capture csp information" do
    @builder.should_receive(:csp=).once.with("CITY IL 60563")
    @addr.use(@builder)
  end

  specify "should not capture country information" do
    @builder.should_not_receive(:country=)
    @addr.use(@builder)
  end
end

context "An address with street, csp, and country" do

  setup do
    @builder = mock("builder", :null_object => true)
    @addr = Address.from_string("ADDR1$CITY IL 60563$COUNTRY")
  end

  specify "should capture street information" do
    @builder.should_receive(:addr1=).once.with("ADDR1")
    @addr.use(@builder)
  end

  specify "should capture csp information" do
    @builder.should_receive(:csp=).once.with("CITY IL 60563")
    @addr.use(@builder)
  end

  specify "should capture country information" do
    @builder.should_receive(:country=).once.with("COUNTRY")
    @addr.use(@builder)
  end
end

address.rb

class Address
  def self.from_string(address_string)
    addr1, csp, country, rest = address_string.split('$')
    Address.new(addr1, csp, country)
  end

  def initialize(addr1, csp, country)
    @addr1 = addr1
    @csp = csp
    @country = country
  end

  def use(a_builder)
    a_builder.addr1 = @addr1
    a_builder.csp = @csp
    a_builder.country = @country unless @country.nil?
  end
end

spec output

An address with street & csp
- should capture street information
- should capture csp information
- should not capture country information

An address with street, csp, and country
- should capture street information
- should capture csp information
- should capture country information

Finished in 0.014 seconds

6 specifications, 0 failures

Conclusion

So we took a situation that was thought to require multiple assertions/expectations in a test/example and did it in such as way as to have only one per.

The key is that instead of using a single context with a complex (i.e. multiple expectation) example for each situation, we made each of those situations into a separate context. Now each example focuses on a very small, specific aspect of the behaviour dealt with by its context.

I’m convinced writing examples like this is a useful approach. One advantage is that the resulting examples are simpler and easier to understand. Just as important, and maybe more so, is that by adding the specification of the behavior one tiny piece at a time, you drive toward evolving the code in small, controllable, understandable steps.

It also fits better into the context centered approach that is the recommended way to organize your examples. We set up the object being worked on in setup, and wrote examples of it’s behaviour in that particular context in individual specify clauses.

As I was writing this back in early 2004, something clicked. I saw these test methods (as they are in jUnit/test::unit) as specifications of tiny facets of the required behavior. Thus, it made sense to me to be as gradual as possible about it, driving the evolution of the code in the smallest steps possible. Striving for one assertion per test is a way to do that. This epiphany was one of the main drivers in my deepening understanding of what has been called Test Driven Development (and which we are now considering an aspect of Behaviour Driven Development)

If, however, you view test methods as strictly performing verification, then I can see how it might be seen to make sense to invoke some code and then test all the postconditions. But this view is not TDD (and certainly not BDD), and doesn’t buy you all of the benefits that are possible. I contend that central to TDD is this notion of working in the smallest steps possible, both for the finest-grained long-term verification, and for the most flexible design evolution. Furthermore, this is best done by striving to keep tests/examples as small, focused and simple as possible.

Aiming for one expectation each is one way to get there.

Tags: bdd code rspec ruby tdd

Tom White’s Blog: Literate Programming with jMock:

Posted by tomwhite on May 11, 2006 at 11:10 AM

“We’ve been using jMock at our company for some time now. We’ve found it great for test driven development and isolating our unit tests from the rest of the system more effectively. One aspect of jMock that stands out for me is its idea of constraints. In fact, we’ve found this idea so useful that we always use the org.jmock.MockObjectTestCase base class rather than junit.framework.TestCase, even when we aren’t mocking anything out. This seems to have registered with the jMock development team, as they are planning to extract the constraints into a separate project, codenamed Hamcrest (it’s an anagram of “matchers”).”

(Via Steve Freeman.)

Tags: bdd mocks tdd

I did my Intro to jUnit4 talk this morning. It went well with good discussion and a couple dozen people.

If you’re using Java 5 go get jUnit 4.. it’s cool.

Tags: conference tdd

This is an article that appeared on my Artima.com blog on February 23, 2004

---

A while ago there was a bit of fuss on the testdrivendevelopment Yahoo group about the idea of limiting yourself to one assertion per test method, which is a guideline that others and I offer for TDD work.

An address parser was the example of a situation where it was argued that multiple assertions per test made sense. Date was formatted with one address per line, each in one of the following formats:

1. ADDR1$ADDR2$CSP$COUNTRY
2. ADDR1$ADDR2$CSP
3. ADDR1$CSP$COUNTRY
4. ADDR1$CSP

The poster went on to say:

My first inclination is/was to write a test like this:

    a = CreateObject("Address")
    a.GetAddressParts("ADDR1$ADDR2$CITY IL 60563$COUNTRY")
    AssertEquals("ADDR1", a.Addr1)
    AssertEquals("ADDR2", a.Addr2)
    AssertEquals("CITY IL 60563", a.CityStatePostalCd)
    AssertEquals("Country", a.Country)

They didn’t see how to achieve this with one assertion per test, as there are obviously four things to test in this case. I decided that rather than simply reply, I would write some tests and code to illustrate my view on the matter, and offer a solid response.

For this problem, I chose Squeak Smalltalk (see www.squeak.org) and Java. For the sake of conciseness, I’ll omit any required accessors.

So, where to start? Well, when doing TDD it often makes sense to start with something simple to quickly and easily get some code written and working. Then it can be extended and evolved in response to further test driving. Here the simplest case is: tt>ADDR1$CSP. There are two requirements in the parsing of this example: that the ADDR1 was recognized, and that the CSP was recognized. Viewed this way, we need two tests. We start with one for ADDR1:

Squeak:

    testAddr1
        | anAddress |
        anAddress := Address from: 'ADDR1$CITY IL 60563'.
        self assert: anAddress addr1 equals: 'ADDR1'

Java:

    public void testAddr1() throws Exception {
        Address anAddress = new Address("ADDR1$CITY IL 60563");
        assertEquals("ADDR1",  anAddress.getAddr1());
    }

To get this to pass we need an Address class and a from: factory method, which creates an instance and has it parse the address string. For brevity, I’ll skip the “return literal” step.

Squeak:

    Object subclass: #Address
        instanceVariableNames: 'addr1'
        classVariableNames: ''
        poolDictionaries: ''
        category: 'Adaption-One Assertion Test'
(class method)
from: aString
    ^self new parse: aString;
        yourself

parse: aString
    | parts |
    parts := aString findTokens: '$'.
    addr1 := (parts at: 1)

e87520e089973e716b9ec17744f135c7

    public class Address {
    private String addr1;

    public Address(String aString) {
        parse(aString);
    }

    private void parse(String aString) {
        StringTokenizer parts = new StringTokenizer(aString, "$");
        addr1 = parts.nextToken();
    }
}

That’s well & good. The next test is for CSP.

Squeak:

    testCsp
        | anAddress |
        anAddress := Address from: 'ADDR1$CITY IL 60563'.
        self assert: anAddress csp equals: 'CITY IL 60563'

Java:

    public void testCsp() throws Exception {
        Address anAddress = new Address("ADDR1$CITY IL 60563");
        assertEquals("CITY IL 60563",  anAddress.getCsp());
    }

Address>>parse: will need to be extended (and we need to add a csp instance variable and accessors):

Squeak:

    parse: aString
        | parts |
        parts := aString findTokens: '$'.
        addr1 := (parts at: 1).
        csp := (parts at: 2)

Java:

    private void parse(String aString) {
        StringTokenizer parts = new StringTokenizer(aString, "$");
        addr1 = parts.nextToken();
        csp = parts.nextToken();
    }

So. We have two tests for this one situation. Notice the duplication in the tests… the creation of the instance of Address that is being probed. This is the fixture. After refactoring, we have:

Squeak:

    TestCase subclass: #Addr1CspTests
        instanceVariableNames: 'anAddress '
        classVariableNames: ''
        poolDictionaries: ''
        category: 'Adaption-One Assertion Test'!
setUp
    anAddress := Address from: 'ADDR1$CITY IL 60563'

testAddr1
    self assert: anAddress addr1 equals: 'ADDR1'

testCsp
    self assert: anAddress csp equals: 'CITY IL 60563'

ab4725d503a2aa390ab1b11b1a710b99

    public class Addr1CspTests extends TestCase {
    private Address anAddress;

    protected void setUp() throws Exception {
        anAddress = new Address("ADDR1$CITY IL 60563");
    }

    public void testAddr1() throws Exception {
        assertEquals("ADDR1",  anAddress.getAddr1());
    }

    public void testCsp() throws Exception {
        assertEquals("CITY IL 60563",  anAddress.getCsp());
    }
}

So, a fixture that creates the Address instance from the string, and very simple tests that focus on each aspect of that fixture.

The next simplest case is the obvious choice for the next fixture:

Squeak:

    setUp
        anAddress := Address from: 'ADDR1$CITY IL 60563$COUNTRY'

Java:

    protected void setUp() throws Exception {
        anAddress = new Address("ADDR1$CITY IL 60563$COUNTRY");
    }

This set of tests will include ones for addr1 and csp as before (refactoring this to remove that duplication is left to the reader) as well as a new test for country:

Squeak:

    testCountry
        self assert: anAddress country equals: 'COUNTRY'

Java:

    public void testCountry() throws Exception {
        assertEquals("COUNTRY",  anAddress.getCountry());
    }

As before, an instance variable and associated accessors need to be added to the Address class.

This drives Address>>parse: to evolve:

Squeak:

    parse: aString
        | parts |
        parts := aString findTokens: '$'.
        addr1 := (parts at: 1).
        csp := (parts at: 2).
        country := (parts at: 3 ifAbsent: [''])

Java:

    private void parse(String aString) {
        StringTokenizer parts = new StringTokenizer(aString, "$");
        addr1 = parts.nextToken();
        csp = parts.nextToken();
        country = parts.hasMoreTokens() ? parts.nextToken() : "";
    }

From here on, the evolution gets a bit more complex, as we add the ADDR2 option to the mix.

Conclusion

So we took a situation that was thought to require multiple assertions in a test and did it in such as way as to have only one assertion per test.

The key is that instead of using a single TestCase subclass with a complex (i.e. multiple assertion) tests for each situation, we made each of those situations into a separate fixture. Each fixture is implemented by a separate subclass of TestCase. Now each test focuses on a very small, specific aspect of that particular fixture.

I’m convinced writing tests like this is a useful approach. One advantage is that the resulting tests simpler and easier to understand. Just as important, and maybe more so, is that by adding the specification of the behavior one tiny piece at a time, you drive toward evolving the code in small, controllable, understandable steps.

It also fits better into the test fixture centered approach that is the recommended way to organize your tests. We set up the object to test in the setUp method, and tested each aspect of it in individual tests methods.

As I’ve been writing this, something clicked. I see these test methods as specifications of tiny facets of the required behavior. Thus, it makes sense to me to be as gradual as possible about it, driving the evolution of the code in the smallest steps possible. Striving for one assertion per test is a way to do that.

If, however, you view test methods as strictly performing verification, then I can see how it might be seen to make sense to invoke some code and then test all the postconditions. But this view is not TDD, and doesn’t buy you all of the benefits of TDD. I contend that central to TDD is this notion of working in the smallest steps possible, both for the finest-grained long-term verification, and for the most flexible design evolution. Furthermore, this is best done by striving to keep tests as small, focused and simple as ossible.

Aiming for one assertion per test is one way to get there.

Tags: code java quality tdd