The random utterances of David Arno

Using custom constraints to improve NUnit test quality, part 2 of 2

Tue 24th July, 2012 | Category: NUnit,Test Driven Development

Writing good quality unit tests for your code can be hard. Neither multiple asserts in one test, nor multiple single-assert tests are ideal solutions. There are alternatives though, which this article explores. It culminates in a description of a powerful way of writing tests using NUnit.

In part one of this article, I examined two techniques for unit testing: the one test with multiple asserts approach and the many tests with single asserts approach. I discussed why both techniques are flawed and hinted at the fact that there are alternatives. This second part examines those improved solutions.

Improving tests through overriding Equals
One way we can simplify testing of objects that hold many values is by overriding the Equals method in the class under test. Let’s start by adding the following code to our Board class:

Board.cs (expanded)

    public override bool Equals(object obj)
    {
      if (!(obj is Board))
      {
        return false;
      }
 
      var other = (Board)obj;
      return TopLeft == other.TopLeft && 
             Top == other.Top && 
             TopRight == other.TopRight &&
             MiddleLeft == other.MiddleLeft && 
             Middle == other.Middle && 
             MiddleRight == other.MiddleRight &&
             BottomLeft == other.BottomLeft && 
             Bottom == other.Bottom && 
             BottomRight == other.BottomRight;
    }
 
    public override string ToString()
    {
      return "TL=" + TopLeft.ToString() + ", " +
             "T=" + Top.ToString() + ", " +
             "TR=" + TopRight.ToString() + ", " +
             "ML=" + MiddleLeft.ToString() + ", " +
             "M=" + Middle.ToString() + ", " +
             "MR=" + MiddleRight.ToString() + ", " +
             "BL=" + BottomLeft.ToString() + ", " +
             "B=" + Bottom.ToString() + ", " +
             "BR=" + BottomRight.ToString();
    }

By adding this code, we can now greatly simplify our test class:

XMoveOneTestUsingEqualsOverride.cs

using NUnit.Framework;
using TicTacToe;
 
namespace TicTacToeTests
{
  [TestFixture]
  public class XMoveOneTestUsingEqualsOverride
  {
    [Test]
    public void WhenXMovePlayedOnlyThatCellChanges()
    {
      var expectedStateBoard = new Board
      {
        BottomLeft = CellState.Empty,
        Bottom = CellState.X,
        BottomRight = CellState.Empty,
        MiddleLeft = CellState.Empty,
        Middle = CellState.Empty,
        MiddleRight = CellState.Empty,
        TopLeft = CellState.Empty,
        Top = CellState.Empty,
        TopRight = CellState.Empty
      };
 
      var testBoard = new Board();
      testBoard.PlaceX("Bottom");
      Assert.AreEqual(expectedStateBoard, testBoard);
    }
  }
}

Running this test and examining the resultant error, gives us the following information:

Expected: <TL=Empty, T=Empty, TR=Empty, ML=Empty, M=Empty, MR=Empty, 
           BL=Empty, B=X, BR=Empty>
But was:  <TL=Empty, T=Empty, TR=Empty, ML=Empty, M=Empty, MR=Empty, 
           BL=X, B=X, BR=X>

By using this method, we now have just one test and that test has just one assert. Despite this apparent reduction in testing, an examination of the expected and actual values reveals there are two errors in the code. So the test is simplified, yet the error information contents are maintained. Whilst this solution provides a great way to simplify tests, it comes at a price: we have had to add an override of the Equals method to the Board class for the sole reason of aiding testing. In reality, we would also have to define a GetHash override too. We still haven’t achieved a truly desirable testing environment therefore.

At this point, it is time to focus exclusively on NUnit in order to improve the situation still further. As I mentioned in part 1, other frameworks often support extending the framework with new tests, but the following code is specific to NUnit as far as I know. The key to the further improved solution is NUnit’s Constraint class. Constraints are used when asserting via the Assert.That() method.

BoardMatchConstraint.cs

using NUnit.Framework.Constraints;
using TicTacToe;
 
namespace TicTacToeTests
{
  public class BoardMatchConstraint : Constraint
  {
    public BoardMatchConstraint(Board expected)
    {
      _expected = expected;
    }
 
    public override bool Matches(object actual)
    {
      base.actual = actual;
      var valuesMatch = false;
 
      if (actual is Board)
      {
        valuesMatch = TestValuesForEquality(_expected, 
                                            (Board)actual);
      }
 
      return valuesMatch;
    }
 
    public override void WriteDescriptionTo(MessageWriter writer)
    {
      writer.WriteExpectedValue(_expected);
    }
 
    public override void WriteMessageTo(MessageWriter writer)
    {
      writer.DisplayDifferences(_expected, actual);
    }
 
    private static bool TestValuesForEquality(Board expected, 
                                              Board actual)
    {
      return actual.TopLeft == expected.TopLeft &&
             actual.Top == expected.Top &&
             actual.TopRight == expected.TopRight &&
             actual.MiddleLeft == expected.MiddleLeft &&
             actual.Middle == expected.Middle &&
             actual.MiddleRight == expected.MiddleRight &&
             actual.BottomLeft == expected.BottomLeft &&
             actual.Bottom == expected.Bottom &&
             actual.BottomRight == expected.BottomRight;
    }
 
    private readonly Board _expected;
  }
}

By defining the above constraint, we remove the need for the Equals and GetHash method overrides in the Board class. We can then rewrite the test class to be:

XMoveOneTestUsingCustomConstraint.cs

using NUnit.Framework;
using TicTacToe;
 
namespace TicTacToeTests
{
  [TestFixture]
  public class XMoveOneTestUsingCustomConstraint
  {
    [Test]
    public void WhenXMovePlayedOnlyThatCellChanges()
    {
      var expectedStateBoard = new Board
      {
        BottomLeft = CellState.Empty,
        Bottom = CellState.X,
        BottomRight = CellState.Empty,
        MiddleLeft = CellState.Empty,
        Middle = CellState.Empty,
        MiddleRight = CellState.Empty,
        TopLeft = CellState.Empty,
        Top = CellState.Empty,
        TopRight = CellState.Empty
      };
 
      var testBoard = new Board();
      testBoard.PlaceX("Bottom");
      Assert.That(testBoard, 
                  new BoardMatchConstraint(expectedStateBoard));
    }
  }
}

To recap, we now have one test, with one assert. We can readily identify multiple errors in the board state with that one test. Finally, we do not need any extraneous code in the Board class to support this.

Despite all this, we can improve things even further. Assert.That(testBoard, new BoardMatchConstraint(expectedStateBoard)); is functional, but it does not read well. Improving the readability of the test is the subject of a separate blog post.


Share This Post...
1 comment so far, click here to read it or add another

1 Comment so far

  1. The Spaniel July 25th, 2012 14:39

    I think it would be quite useful to see the Assert api expanded to allow for some kind of AssertButContinue behaviour, allowing asserts to build a list which is reported once the test method finally exits.