One Expectation Per Test(zz)

zz From:

http://jupitermoonbeam.blogspot.com/2007/05/dont-expect-too-much.html

And http://www.daveastels.com/articles/2006/08/26/one-expectation-per-example-a-remake-of-one-assertion-per-test

 

Don't Expect too much

A long while ago (well over two years) there was a lot of fuss made on the testdrivendevelopment Yahoo group about having only one assertion per test. Dave Astels wrote a great little article and a little more fuss was made. It was one of those things you knew made sence but sounded a lot of work. I played with one assertion per test anyway and suddenly felt my code developing more fluidly as I focused on only one thing at a time and my tests looked clearer (more behavoir orientated) and refactoring became a lot simpler too.
Then Dave Astels came along and pushed the envelope further with One Expectation per Example. This really grabbed my attention as I had been enjoying the benefits of keeping my test code clean with one assertion per test but anything with mocks in it just turned into out of control beasts (especially some of the more complex logic such as MVP). Even simple four liners such as below would end up with four expectations:

public AddOrder(OrderDto)
{
Customer customer = session.GetCurrentCustomer();
customer.AddOrder(OrderFactory.CreateOrder(OrderDTO));

workspace.Save(customer);
}

Then everytime I needed to add new functionality I had to add more expectations and anyone who read the work (including myself after 24 hours) would struggle to make head or tale out of the monster. And if tests failed it would take a day to find which expectation went wrong. If you're not careful you end up with the TDD anti-pattern The Mockery.
I had read Dave Astels article several times but couldn't fathom out how it worked especially seeing it was written in Ruby with the behaviour driven RSpec. In the end I had to write it out into .NET myself before I got it.
So here is a break down of how I got Dave's One expectation per example to work for me:
One Expectation Per Example (C#)
One of the first things to note is Dave uses the builder pattern in his example. The idea is that the Address object interacts with a builder to pass it's data to rather than allowing objects to see it's state directly thus breaking encapsulation. I would like to go into this technique in more detail in another article but to deliver the point quickly think that you may create an HTML builder to display the address on the web.
Well let's start with Dave's first test:

[TestFixture]
public class OneExpectationPerExample
{
[Test]
public void ShouldCaptureStreetInformation()
{
    Address addr = Address.Parse("ADDR1$CITY IL 60563");

    Mockery mocks = new Mockery();
    IBuilder builder = mocks.NewMock();

    Expect.Once.On(builder).SetProperty("Address1").To("ADDR1");

    addr.Use(builder);

    mocks.VerifyAllExpectationsHaveBeenMet();
}
}

You may have noticed that I've changed a few things maily to make it look consistant with .NET design practices. Basically I've introduced a Parse method rather than the from_string method Dave uses.
Now we need to get this baby to compile. First we need to create the Address class like so:

public class Address
{
public static Address Parse(string address)
{
  throw new NotImplementedExpection();
}

public void Use(IBuilder builder)
{
  throw new NotImplementedExpection();
}
}

And the IBuilder interface:

public interface IBuilder {}

Now it compiles but when we run it we get the following:

   mock object builder does not have a setter for property Address1

So we need to add the Address1 property to the IBuilder. Then we run and we get:

    TestCase 'OneExpectationPerExample.ShouldCaptureStreetInformation'
failed: NMock2.Internal.ExpectationException : not all expected invocations were performed
Expected:
1 time: builder.Address1 = (equal to "ADDR1") [called 0 times]

Let's implement some working code then:

public class Address
{
private readonly string address1;

private Address(string address1)
{
  this.address1 = address1;
}

public static Address Parse(string address)
{
    string[] splitAddress = address.Split('$');

    return new Address(splitAddress[0]);
}

public void Use(IBuilder builder)
{
    builder.Address1 = address1;
}
}

Run the tests again and they work! So let's move onto the second part of implementing the Csp. Here's the new test:

[Test]
public void ShouldCaptureCspInformation()
{
    Address addr = Address.Parse("ADDR1$CITY IL 60563");

    Mockery mocks = new Mockery();
    IBuilder builder = mocks.NewMock<IBuilder%gt;();

    Expect.Once.On(builder).SetProperty("Csp").To("CITY IL 60563");

    addr.Use(builder);

    mocks.VerifyAllExpectationsHaveBeenMet();
}

Now a little refactoring to get rid off our repeated code we turn it into this:

[TestFixture]
public class OneExpectationPerExample
{
private IBuilder builder;
private Address addr;
private Mockery mocks;

[SetUp]
public void SetUp()
{
    mocks = new Mockery();

    builder = mocks.NewMock();

    addr = Address.Parse("ADDR1$CITY IL 60563");
}

[TearDown]
public void TearDown()
{
mocks.VerifyAllExpectationsHaveBeenMet();
}

[Test]
public void ShouldCaptureStreetInformation()
{
    Expect.Once.On(builder).SetProperty("Address1").To("ADDR1");

    addr.Use(builder);
}

[Test]
public void ShouldCaptureCspInformation()
{
    Expect.Once.On(builder).SetProperty("Csp").To("CITY IL 60563");

    addr.Use(builder);
}
}

Looking good! We run the new test and we get the usual error for having no Csp property on the IBuilder so we add that:

public interface IBuilder
{
string Address1 { set; }
string Csp { set; }
}

Then we run the test again and we get:

   TestCase 'OneExpectationPerExample.ShouldCaptureCspInformation'
failed: NMock2.Internal.ExpectationException : unexpected invocation of builder.Address1 = "ADDR1"
Expected:
1 time: builder.Csp = (equal to "CITY IL 60563") [called 0 times]

Oh no. This is where Dave's article falls apart for .NET.
Basically RSpec has an option to create Quite Mocks which quitely ignore any unexpected calls. Unfortunately I know of no .NET mock libaries that have such behaviour (though I have since been reliably informed by John Donaldson on the tdd yahoo group that it is possible with the NUnit mock library) . Though there is a way out: stub the whole thing out by using Method(Is.Anything):

[Test]
public void ShouldCaptureCspInformation()
{
Expect.Once.On(builder).SetProperty("Csp").To("CITY IL 60563");

// stub it as we're not interested in any other calls.
Stub.On(builder).Method(Is.Anything);

addr.Use(builder);
}

Just be careful to put the Stub AFTER the Expect and not before as NMock will use the Stub rather than the Expect and your test will keep failing.
So now we run the tests and we get:

   TestCase 'OneExpectationPerExample.ShouldCaptureCspInformation'
failed:
TearDown : System.Reflection.TargetInvocationException : Exception has been thrown by the target of an invocation.
----> NMock2.Internal.ExpectationException : not all expected invocations were performed
Expected:
1 time: builder.Csp = (equal to "CITY IL 60563") [called 0 times]

Excellent NMock is now behaving correctly we can finish implementing the code:

public class Address
{
private readonly string address1;
private readonly string csp;

private Address(string address1, string csp)
{
    this.address1 = address1;
    this.csp = csp;
}

public static Address Parse(string address)
{
    string[] splitAddress = address.Split('$');

    return new Address(splitAddress[0], splitAddress[1]);
}

public void Use(IBuilder builder)
{
    builder.Address1 = address1;
    builder.Csp = csp;
}
}

Run the test and it works! Now if we run the whole fixture we get:

   TestCase 'OneExpectationPerExample.ShouldCaptureStreetInformation'
failed: NMock2.Internal.ExpectationException : unexpected invocation of builder.Csp = "CITY IL 60563"

All we need to do do is go back and add the Stub code to the street test. That's a bit of a bummer but we could refactor our tests to do the call in the tear down like so:

[TearDown]
public void UseTheBuilder()
{
Stub.On(builder).Method(Is.Anything);

addr.Use(builder);

mocks.VerifyAllExpectationsHaveBeenMet();
}

[Test]
public void ShouldCaptureStreetInformation()
{
Expect.Once.On(builder).SetProperty("Address1").To("ADDR1");
}

[Test]
public void ShouldCaptureCspInformation()
{
Expect.Once.On(builder).SetProperty("Csp").To("CITY IL 60563");
}

This approach comes across as slightly odd because the expectations are set in the test but the test is run in the tear down. I actually think this is neater in some ways as it ensures you have one test class for each set of behaviours the only off putting thing is the naming convention of the attributes.
I won't bother continuing with the rest of Dave's article as it's just more of the same from here. The only thing I'd add is he does use one class per behaviour set (or context) so when he tests the behaviour of a string with ZIP code he uses a whole new test fixture. This can feel a little extreme in some cases as you get a bit of test class explosion but all in all it does make your life a lot easier.
I hope the translation helps all you .NET B/TDDers out there to free you from The Mockery.
Tip:
In more complex behaviours you may need to pass a value from one mock to another. In those instances you can do:

   Stub.On(x).Method(Is.Anything).Will(Return.Value(y));

One Expectation per Example: A Remake of "One Assertion Per Test"

Posted by dastels on Sunday, August 27, 2006

In the fine tradition of Hollywood (who these days is remaking everything from old movies, comic books, and video games in lieu of coming up with original ideas) I am doing a remake of the very first post on my blog (which itself was a repost from my Artima.com blog from Feb 2004). The original post created quite a stir at the time, and lead to many of the ideas that have evolved my understanding of TDD and now BDD.

---

Once upon a time (early 2004) there was a bit of fuss on the testdrivendevelopment Yahoo group about the idea of limiting yourself to one assertion per test method, which is a guideline that others and I offer for TDD work.

An address parser was the example of a situation where it was argued that multiple assertions per test made sense. Date was formatted with one address per line, each in one of the following formats:

1. ADDR1$ADDR2$CSP$COUNTRY
2. ADDR1$ADDR2$CSP
3. ADDR1$CSP$COUNTRY
4. ADDR1$CSP

The poster went on to say:

My first inclination is/was to write a test like this (in Java but you get the idea):

a = CreateObject("Address")
a.GetAddressParts("ADDR1$ADDR2$CITY IL 60563$COUNTRY")
AssertEquals("ADDR1", a.Addr1)
AssertEquals("ADDR2", a.Addr2)
AssertEquals("CITY IL 60563", a.CityStatePostalCd)
AssertEquals("Country", a.Country)

They didn't see how to achieve this with one assertion per test, as there are obviously four things to test in this case. I decided that rather than simply reply, I would write some tests and code to illustrate my view on the matter, and offer a solid response.

For the original post, I chose Squeak Smalltalk and Java. In this remake I'm going to use just Ruby. In the original I said "For the sake of conciseness, I'll omit any required accessors." Well.. this time around I'm not going to use any accessors, since I am more & more of the opinion that they are quite simply an idea whose sole purpose is to undermine the very essence of Object-Oriented software. I'll also be using rSpec rather than an xUnit framework.

So, where to start? Well, it often makes sense to start with something simple to quickly and easily get some code written and working. Then it can be extended and evolved. Here the simplest case is: ADDR1$CSP. There are two requirements in the parsing of this example: that the ADDR1 was recognized, and that the CSP was recognized. Viewed this way, we need two examples. If we want to avoid accessors (and for this exercise, I do) we need another way to see inside the object were working with. This is an address.. let's say (for the sake of argument & this exercise) that one of the purposes of having an address is to generate address labels of some sort. We could do that with an address label maker object that would interrogate the address object for it's contents and use them. A better way is to have an address label builder that the address object gives information to. It would have methods such as street_number=, csp=, etc. This is something that we can use Mocks Objects for in our examples. So, let's start with an example for ADDR1:

context "An address with street & csp" do
specify "should capture street information" do
builder = mock("builder", null_object => true)
builder.should_receive(:addr1=).once.with("ADDR1")
addr = Address.from_string("ADDR1$CITY IL 60563")
addr.use(builder)
end
end

To get this to pass we need an Address class, a from_string factory method, and a use method.

class Address
def self.from_string(address_string)
Address.new
end
def use(a_builder)
end
end

This lets our spec run and fail:

An address with street & csp
- should capture street information (FAILED - 1)
1)
Spec::Api::MockExpectationError in 'An address with street & csp should capture street information'
Mock 'builder' expected 'street=' once, but received it 0 times
./address_spec.rb:7:in `'
Finished in 0.003395 seconds
1 specification, 1 failure

Now let's make that work:

class Address
def self.from_string(address_string)
addr1, rest = address_string.split('$')
Address.new(addr1)
end
def initialize(addr1)
@addr1 = addr1
end
def use(a_builder)
a_builder.addr1 = @addr1
end
end

This makes the example work:

An address with street & csp
- should capture street information
Finished in 0.003003 seconds
1 specification, 0 failures

That's well & good. The next test is for CSP.

specify "should capture csp information" do
builder = mock("builder", :null_object => true)
builder.should_receive(:csp=).once.with("CITY IL 60563")
addr = Address.from_string("ADDR1$CITY IL 60563")
addr.use(builder)
end

Resulting in:

An address with street & csp
- should capture street information
- should capture street information (FAILED - 1)
1)
Spec::Api::MockExpectationError in 'An address with street & csp should capture street information'
Mock 'builder' expected 'csp=' once, but received it 0 times
./address_spec.rb:15:in `'
Finished in 0.005866 seconds
2 specifications, 1 failure

Address#from_string will need to be extended (and we need to add csp support):

class Address
def self.from_string(address_string)
addr1, csp, rest = address_string.split('$')
Address.new(addr1, csp)
end
def initialize(addr1, csp)
@addr1 = addr1
@csp = csp
end
def use(a_builder)
a_builder.addr1 = @addr1
a_builder.csp = @csp
end
end

So. We have two tests for this one situation. Notice the duplication in the tests... the creation of builder & address. This is the context. After refactoring, we have:

require 'address'
context "An address with street & csp" do
setup do
@builder = mock("builder", :null_object => true)
@addr = Address.from_string("ADDR1$CITY IL 60563")
end
specify "should capture street information" do
@builder.should_receive(:addr1=).once.with("ADDR1")
@addr.use(@builder)
end
specify "should capture csp information" do
@builder.should_receive(:csp=).once.with("CITY IL 60563")
@addr.use(@builder)
end
end

So, a context that creates the Address instance from the string as well as the mock, and very simple examples that focus on each aspect of that context.

Note that we don't have any direct expectations in these examples... What's up? Well, the mock is autoverifying itself at the end of each example, and that is where our expectations are.. on @address's interaction with the mock.

The next simplest case is the obvious choice for the next context:

context "An address with street, csp, and country" do
setup do
@builder = mock("builder", :null_object => true)
@addr = Address.from_string("ADDR1$CITY IL 60563$COUNTRY")
end
end

This set of tests will include ones for addr1 and csp as before (since that behaviour is required in this new context) as well as a new test for country:

specify "should capture country information" do
@builder.should_receive(:country=).once.with("COUNTRY")
@addr.use(@builder)
end

As before, an instance variable and such need to be added to the Address class.

This drives Address to evolve:

class Address
def self.from_string(address_string)
addr1, csp, country, rest = address_string.split('$')
Address.new(addr1, csp, country)
end
def initialize(addr1, csp, country)
@addr1 = addr1
@csp = csp
@country = country
end
def use(a_builder)
a_builder.addr1 = @addr1
a_builder.csp = @csp
a_builder.country = @country
end
end

Now that we are supporting country, we can go back to the first context and add an example specifying that no country information should be provided to the builder:

specify "should not capture country information" do
@builder.should_not_receive(:country=)
@addr.use(@builder)
end

A slight tweak to Address#use will make this work:

def use(a_builder)
a_builder.addr1 = @addr1
a_builder.csp = @csp
a_builder.country = @country unless @country.nil?
end

From here on, the evolution gets a bit more complex, as we add the ADDR2 option to the mix.

The final code and output is:

address_spec.rb

require 'address'
context "An address with street & csp" do
setup do
@builder = mock("builder", :null_object => true)
@addr = Address.from_string("ADDR1$CITY IL 60563")
end
specify "should capture street information" do
@builder.should_receive(:addr1=).once.with("ADDR1")
@addr.use(@builder)
end
specify "should capture csp information" do
@builder.should_receive(:csp=).once.with("CITY IL 60563")
@addr.use(@builder)
end
specify "should not capture country information" do
@builder.should_not_receive(:country=)
@addr.use(@builder)
end
end
context "An address with street, csp, and country" do
setup do
@builder = mock("builder", :null_object => true)
@addr = Address.from_string("ADDR1$CITY IL 60563$COUNTRY")
end
specify "should capture street information" do
@builder.should_receive(:addr1=).once.with("ADDR1")
@addr.use(@builder)
end
specify "should capture csp information" do
@builder.should_receive(:csp=).once.with("CITY IL 60563")
@addr.use(@builder)
end
specify "should capture country information" do
@builder.should_receive(:country=).once.with("COUNTRY")
@addr.use(@builder)
end
end

address.rb

class Address
def self.from_string(address_string)
addr1, csp, country, rest = address_string.split('$')
Address.new(addr1, csp, country)
end
def initialize(addr1, csp, country)
@addr1 = addr1
@csp = csp
@country = country
end
def use(a_builder)
a_builder.addr1 = @addr1
a_builder.csp = @csp
a_builder.country = @country unless @country.nil?
end
end

spec output

An address with street & csp
- should capture street information
- should capture csp information
- should not capture country information
An address with street, csp, and country
- should capture street information
- should capture csp information
- should capture country information
Finished in 0.014 seconds
6 specifications, 0 failures

Conclusion

So we took a situation that was thought to require multiple assertions/expectations in a test/example and did it in such as way as to have only one per.

The key is that instead of using a single context with a complex (i.e. multiple expectation) example for each situation, we made each of those situations into a separate context. Now each example focuses on a very small, specific aspect of the behaviour dealt with by its context.

I'm convinced writing examples like this is a useful approach. One advantage is that the resulting examples are simpler and easier to understand. Just as important, and maybe more so, is that by adding the specification of the behavior one tiny piece at a time, you drive toward evolving the code in small, controllable, understandable steps.

It also fits better into the context centered approach that is the recommended way to organize your examples. We set up the object being worked on in setup, and wrote examples of it's behaviour in that particular context in individual specify clauses.

As I was writing this back in early 2004, something clicked. I saw these test methods (as they are in jUnit/test::unit) as specifications of tiny facets of the required behavior. Thus, it made sense to me to be as gradual as possible about it, driving the evolution of the code in the smallest steps possible. Striving for one assertion per test is a way to do that. This epiphany was one of the main drivers in my deepening understanding of what has been called Test Driven Development (and which we are now considering an aspect of Behaviour Driven Development)

If, however, you view test methods as strictly performing verification, then I can see how it might be seen to make sense to invoke some code and then test all the postconditions. But this view is not TDD (and certainly not BDD), and doesn't buy you all of the benefits that are possible. I contend that central to TDD is this notion of working in the smallest steps possible, both for the finest-grained long-term verification, and for the most flexible design evolution. Furthermore, this is best done by striving to keep tests/examples as small, focused and simple as possible.

Aiming for one expectation each is one way to get there.