

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer

Say you have defined some abstract interface and you specify a general contract for that interface to which all implementations must adhere. Are there common techniques that can facilitate testing the conformance of arbitrary implementations to that general contract?

In my environment we have a large collection of such abstract interfaces that define a common interface to several different hardware systems. My current approach is to define a set of conformance tests that work solely with the abstract interfaces. These tests are then packed into a library which again is imported by the actual unit tests of the implementations. These instantiate the tests and also provide a factory which generates concrete testee instances that will be tested by the conformance tests.

In theory this approach sounds fine, but in practice writing the conformance tests is very difficult and the value of their results appears to be very limited, because a lot of additional information about the testee instances is required.

Example (pseudo code):
the abstract interface is

// Triggers some behaviour and returns immediately. While the behaviour is active,
// getState() returns *Running*. If an error occurrs the state changes to *Halted*.
function run();
function getState : State;

This interface and its contract are very simple, yet the test case for this is (relatively) difficult to write and its result says little about whether or not the implementation is doing the right thing or not. Is my approach reasonable or would I be better off just writing separate unit tests for all implementations? Are there other techniques that could help to achieve what I want (i.e.: guarantee the conformance of different implementations to the contract of their general interface).

share|improve this question
Let me see if I understand. Is your problem that the work required to use the common interface is dwarfed by the implementation-specific work required to initialize each implementation? – user246 Oct 2 '12 at 17:49
I understood if differently. I understood that there are some common constrains for different contracts/interfaces that could verified in a similar way. For instance, two collection types in Java (say Set and List) do not accept null values. – dzieciou Nov 1 '12 at 9:50
up vote 5 down vote accepted

Yes. I call them Contract Tests. One easy way to understand them:

Start with tests for a specific implementation of the interface. For example, consider ArrayList implements List. You write tests for ArrayList. One test could be this:

  list = new ArrayList()
  assert list.isEmpty()
  assert_equals 0, list.size()

Notice that LinkedList should also pass this test, but there's only one difference:

list = new LinkedList()

So imagine you have ArrayListTest and LinkedListTest that both have this test, but the only difference is the class name used to instantiate list. Of course, in both cases, list is just a List. You could make these tests identical by extracting a method createEmptyList() : List in each test class. Now you have this:

class ArrayListTest
  List createEmptyList():
    return new ArrayList()

    List list = createEmptyList()
    assert list.isEmpty()
    assert_equals 0, list.size()

class LinkedListTest
  List createEmptyList():
    return new LinkedList()

    List list = createEmptyList()
    assert list.isEmpty()
    assert_equals 0, list.size()

This time I showed the type of list to emphasise that I've made the tests identical.

Now, remove duplication by introducing a new superclass, pulling up the test, and declaring createEmptyList() as abstract.

abstract class ListContract
  abstract List createEmptyList()

    List list = createEmptyList()
    assert list.isEmpty()
    assert_equals 0, list.size()

That is a Contract Test. It should pass for all implementations of List. When you implement List, subclass ListContract and implement createEmptyList().

Now, you can either extract contract tests from existing implementation tests, or you can draft contract tests as you design interfaces. I do both.

I hope this helps.

share|improve this answer

Your Answer


