Before we get back to concrete examples there are a few terms we'll want to define: State Verification, Behavior Verification, Solitary Unit Test, and Sociable Unit Test. These terms have plenty of definitions available on the Internet; unfortunately, many of those definitions differ and some are in direct conflict. This chapter will define those terms for the context of this book.
In the next few sections I'll put some basic definitions around state and behavior verification. If you're looking for additional material on this subject, I would highly recommend reading Mocks Aren't Stubs for a well written, in-depth explanation of mocks, stubs, and their impact on testing.
State Verification describes a style of testing where you exercise one or many methods of an object and then assert the expected state of the object (and/or collaborators). In Mocks Aren't Stubs Martin Fowler identifies developers who generally rely on state verification as Classicists. In my experience, a Classicists' primary argument for their style is: State verification tests specify the least possible implementation detail, thus they will continue to pass even if the internals of the methods being tested are changed.
As long as the external interface remains unchanged,
the following state verification tests should continue to pass despite any
modifications to the internals of Rental and Store.
public class RentalTest {
@Test
public void rentalIsStartedIfInStore() {
Movie movie = a.movie.build();
Rental rental =
a.rental.w(movie).build();
Store store = a.store.w(movie).build();
rental.start(store);
assertTrue(rental.isStarted());
assertEquals(
0, store.getAvailability(movie));
}
@Test
public void
rentalDoesNotStartIfNotAvailable() {
Movie movie = a.movie.build();
Rental rental = a.rental.build();
Store store = a.store.build();
rental.start(store);
assertFalse(rental.isStarted());
assertEquals(
0, store.getAvailability(movie));
}
}In the test above we're verifying the started state of Rental; however, it's
not possible to test a Rental without a Store as well. The Rental would be considered
the SUT in Martin's previously referenced article; however, I prefer the term Class
Under Test (for reasons later described). In the RentalTest the Store instance is
merely a collaborator.
State verification tests are easy to spot due to relying on assertions to verify the
state of our objects; in our example we assert the state of rental.isStarted()
and store.getAvailability(movie).
note: The Classicist/Mockist dichotomy is about more than testing, but the additional details are outside the scope of this book. More information can be found in the previously linked Mocks Aren't Stubs article.
Behavior Verification describes a style of testing where you expect specific interactions to occur between objects as a result of executing other methods. In Mocks Aren't Stubs Martin Fowler identifies developers who generally rely on behavior verification as Mockists. Behavior verification is about specifying how the system should behave rather than specifying the expected result of running the system.
Behavior verification is commonly criticized by state verification advocates who believe internal implementation changes shouldn't cause test failures. Even Mockists will agree that, in it's worst form, behavior verification can be fragile. Unsurprisingly, Mockists have developed a style of development that helps avoid the primary concern of Classicists. Behavior verification tests that minimize collaborators and collaborations can effectively verify interactions without sacrificing maintainability. As a result, a team that relies on Behavior verification will likely produce a codebase with few Law of Demeter violations and a focus on Tell, Don't Ask.
For those unfamiliar with Law of Demeter and Tell, Don't Ask, the Law of Demeter can be defined as Only talk to your immediate friends and Tell, Don't Ask can be defined as Rather than asking an object for data and acting on that data, we should instead tell an object what to do.
The following test verifies the interaction between a Rental and a Store.
public class RentalTest {
@Test
public void rentalIsStartedIfInStore() {
Movie movie = a.movie.build();
Rental rental =
a.rental.w(movie).build();
Store store = mock(Store.class);
when(store.getAvailability(movie))
.thenReturn(1);
rental.start(store);
assertTrue(rental.isStarted());
verify(store).remove(movie);
}
@Test
public void
rentalDoesNotStartIfNotAvailable() {
Rental rental = a.rental.build();
Store store = mock(Store.class);
rental.start(store);
assertFalse(rental.isStarted());
verify(
store, never()).remove(
any(Movie.class));
}
}In the test above we're verifying the Rental using state verification;
however, we've switched to behavior verification for the Store collaborator.
In our examples we're using the Mockito mocking framework to handle verification.
Mockito mocks follow the pattern of create, (optionally) stub, verify. In our tests
we create our mocks using the static method mock. In the first test we stub
the result of store.getAvailability, in the second test no stubbing is necessary.
Finally, both tests use the static method verify to assert the interactions
the Store instance received during test execution.
Behavior verification tests are easy to spot due to their focus on mock verification to ensure a system behaves as expected.
My examples have focused on behavior verification implemented with mocks, as that's how I tend to do behavior verification. While I don't use them in practice, for completeness I'll mention that spies are another popular implementation of behavior verification.
In Mocks Aren't Stubs Martin Fowler asks and answers the following:
So should I be a classicist or a mockist? I find this a difficult question to answer with confidence.
If you've done much unit testing, it's likely that you already have a preference. If you haven't, you may find yourself leaning more toward one approach than the other. Personally, I've never been satisfied with the results from following the advice from either camp. As the book continues we'll explore my favorite approach, which is more of a hybrid. You may or may not prefer my approach, but for now, all we need to agree on are the definitions above.
The definition of Unit Test is quite general:
In computer programming, unit testing is a method by which individual units of source code, sets of one or more computer program modules together with associated control data, usage procedures, and operating procedures are tested to determine if they are fit for use. [...] A unit could be an entire module, but it is more commonly an individual function or procedure. --Wikipedia
The above definition states that a unit can be an individual Java method, but it can also be something much larger that likely includes many collaborating classes. Over the years I've found value in splitting my unit tests into two distinct categories - Solitary Unit Tests and Sociable Unit Tests.
In Java it's common to unit test at the class level. The Foo class will have an associated
FooTests class. Solitary Unit Tests follow two additional constraints:
- Never cross boundaries
- The Class Under Test should be the only concrete class found in a test.
Never cross boundaries is a fairly simple, yet controversial piece of advice. In 2004, Bill Caputo wrote about this advice, and defined a boundary as: "...a database, a queue, another system...". The advice is simple: accessing a database, network, or file system significantly increases the the time it takes to run a test. When the aggregate execution time impacts a developer's decision to run the test suite, the effectiveness of the entire team is at risk. A test suite that isn't run regularly is likely to have negative-ROI.
In the same entry, Bill also defines a boundary as: "... or even an ordinary class if that class is 'outside' the area your [sic] trying to work with or are responsible for". Bill's recommendation is a good one, but I find it too vague. I think Bill's statement is valuable but fails to give concrete advice on where to draw the line. My second constraint is a concrete (and admittedly restrictive) version of Bill's recommendation.
The concept of constraining a unit test such that 'the Class Under Test should be the only concrete class found in a test' sounds extreme, but it's actually not that drastic if you assume a few things.
- You're using a framework such as Mockito that allows you to easily stub most concrete classes
- This constraint does not apply to any primitive or Java class that has a literal (e.g. int, Integer, String, etc)
- You're using some type of automated refactoring tool.
There are pros and cons to this approach, both of which we'll discuss later in the book. For now, it's only important that you understand my definition of Solitary Unit Test, not that you see value in it.
I define Solitary Unit Test as:
Solitary Unit Testing is an activity by which methods of a class or functions of a namespace are tested to determine if they are fit for use. The tests used to determine if a class or namespace is functional should isolate the class or namespace under test by stubbing all collaboration with additional classes and namespaces.
The definition of Sociable Unit Test is simple: Any Unit Test that contains concrete collaborators and/or crosses a boundary. If you have a Unit Test that's not a Solitary Unit Test, it's a Sociable Unit Test.