SEAT: Setup, Exercise, Assert, Teardown
Setup: Setup any objects necessary for tests to run.
Exercise: Make any calls for invoke specific behavior that is necessary for testing a feature or aspect of our program.
Assert: Ensure that specific procedures occurred and that objects under test are in a state that is congruent with what is expected after the “exercise” phase. Here we test that a SUT behavior is as expected, and the side effects of invoking certain behavior on a SUT.
Teardown: Remove any lingering artifacts. This is usually handled by garbage collection. But an example of this may be closing an open socket for a server or closing an open file after we are done using it in a test.
Object under test OR System under test: These are typically the objects or the module that we are testing. An object that we use in our assertions is typically a “object under test”. The SUT may also have collaborator objects necessary for the test to run smoothly. These objects are not considered SUT.
State Verification: We test whether the exercised method worked correctly by examining the state of the SUT and its collaborators after cede method has been exercised.
Behavior verification or interaction testing
When we use mock objects the setup and assertion step are different.
We still have to setup objects to be used under test, but we also have a setup for a series of expectations as well.
What this means is we have two setups, one for data one that lists a setup of expectations we want to pass based on whether our SUT and its collaborators invoke various methods and in what order those invocations occurs.
This second step is a setup which creates a set of expectations for our mock object.
The assertion step is different in this manner. Yes, we still check that state of our SUT. But we also assert that all the expectations in our setup pass as well; we check that all the mocks passed.
This is called behavior verification
We check the that specifics expectations are met from our collaborators and SUT. Those expectations are based on whether the right behaviors were invoked for the SUT and its collaborators.
Mock expectations usually assert: 1. That a specific method was called X number of times 2. That certain arguments were passed to that method 3. And that that method returned a specific value 4. We can also check if a method was called before or after another one, if that method is dependent on another one being called, one that would produce meaningful side effects.
Note that there isn’t any reason to have duplicate logic in your tests with respect to mocks. If one mock in a test suit checks for certain arguments in a method, then you don’t have to check for those arguments again in another test.
Record/Replay mocks also exist. These are slightly different where we have a control object and a mock object.
The control object “records” what method calls are expected in your tests. Once we set what to record. We then “replay” the recording later on which sets all the expectations to run in the assert step.
In the assert step, we verify that the mock expectations are met, but through the control object instead.
Test Double
Generic term for any kind of pretend object that we use for testing purposes.
Dummy
An object that is passed around but not used, usually used to fill parameter lists.
Fake
An object that has a working implementation, but takes a shortcut unlike real thing object; a fake isn’t suitable for production. One example would be an in memory db.
Stub
Provides canned answers for a test. A stub typically doesn’t respond to any message that isn’t needed for the current test.
Spy
A stub that also records some information based on how it was called. An example of this may be a logging object that keeps track of how many message have been logged.
Mock
An object that is pre-programmed with expectations. These expectations assert what messages the object under test and/or it’s collaborators are expected to receive.
Only mocks insist on behavior verification
Mocks behave like the other doubles during the exercise phase, the SUT has to believe it is talking to real collaborators.
But mocks differ in their setup and verification steps.
Stub uses state verification and mock uses behavior verification.
Yet, stubs can use behavior verification as well. But int that case, the stubs are closer to spies.
Classical TDD: Uses real objects. Maybe if it is very inefficient, a double will be used for the collaborators and the real thing for the SUT.
Mockist TDD: Mocks are used for any object of interest in the test, including the SUT.
Offshoot of mockist testing is BDD
BDD adds extra analysis to the TDD paradigm and expands naming styles.
If the collaboration is difficult for what is under test, then it may be best to use mocks for everything.
Otherwise, it is usually ok to use classical (here we can also consider the use of other doubles, depending on the context).
Sometimes we do run into situations that are hard to verify with state verification. In this case, use behavior verification.
TDD drives our implementation. There is a testing style called “outside-in”. We test one SUT, exploring the interactions between the first SUT and its collaborators.
We let this drive our design and the development of our application. We then move onto the next SUT, which allows us to create only what is needed for our application.
1. First start testing, UI using mock layers.
2. Then write tests for the lower layers underneath. Stepping through the system one layer at a time.
We can also use a classical approach with stubs
1. When we need something from a collaborator, we stub it out.
2. Then when we get green tests, we provide the implementation for the SUT.
3. This is a similar approach, an outside-in using classical style and stubs.
There is also another approach, a middle-out approach. This one we build out a feature and get it to work. Once the various functions and objects necessary for this feature work, we layer the UI onto.
Some prefer this approach since it prevent business logic from possibly breaking the UI
Mock style testing can be useful because all we have to do is create the SUT and then everything else can be mocks for the immediate neighbors (collaborators)
A classical TDD style involves a lot of setup and teardown, as well as the reuse of real objects for use in tests.
In that case, it is necessary to create “fixtures” for use across various tests classes.
Fixture An environment that must be set up before a test can run. Usually involves setting up all the doubles and dependencies necessary for a test.
Test often and update your code often. If any new addition starts to cause issues, you can usually find it fairly easily since you’ll know that addition was introduced in the last edit, after the previous green test ran.
Try to separate finely-grained tests for every class, thus reducing potential regression after update and changes from your tests
Whichever style you go with, make sure to combine those unit tests with acceptance tests (feature/integration tests) that are more coarsely-grained, testing the overall system and how well it is integrated in a real world simulation
One issue in using mocks for tests in that they are tightly coupled to the implementations of your players in the application.
If there significant changes to how various methods are called, then that will break your tests.
Classicists only care about the external interface and the state, returns values of the SUT
Mock tests care about he implementations and behaviors, and how our objects interact with one another, which method calls occur to our SUT
Classical testing prefers middle-out style of testing, which mock testing prefers an outside-in style of testing.
Mockist testing tends to encourage a design that is averse to breaking the law of demeter.
One issue with classical testing is that it can encourage application design that pushs layers towards building out classes and methods that must return a specific state.
This can lead to methods that exist solely to satisfy some test, when those methods should be there to fulfill some require of a feature or software layer in your application.
Remember, design your program from “tell, don’t ask”
It may be a good idea to try mockist testing if you run into errors that are hard to pin down, hard to find the module or method from which the error occurs.
Though, this issue can be fixed in a classical approach by making more finely-grained tests.
Another reason to try mockist testing is it encourage the expansion of the behaviors of your classes. This can be great if you are having trouble growing your program.
Whichever style of testings is used, it is always important to remember the trade-off and the difference in workflows that these two styles
encourage. BDD and mockist testing encourages coding out our program through how different actors/objects interact with one another and the
behaviors those different objects invoke. It inspires a "outside-in" approach to software design, focusing on the UI layer first then building
out the lower layers to fill our whichever feature you're working on.
On the other hand, classical testing, which may use stubs, dummies, fakes, and even spies is a bit different. It is state-based and depends
on the end result of your classes and modules. The development of your program will follow a more "middle-out" design, with various backend
modules/classes being built out first, and then layering on the UI to fit the behavior your expect. This design philosophy and testing style
will can be well maintained with finely-grained unit tests in conjunction with coarsely-grained acceptance/integration tests.
Both styles of testing warrant consideration and it really depends on how your team wants to build up your application, project, or program.
Regardless, consider an agile approach, and build up your application layer by layer, with different modules that aren't too tightly coupled.
This will help you create an application via TDD that is easier to grow and maintain as time goes on.