#Abstract
The Ember community has a strong testing culture, but advanced topics like the intersection of integration and isolation testing can be as confusing as they are controversial. To make matters worse we are often tricked into believing that a mock library has the answer to all of our problems, making it difficult to evaluate when to use mocks (or even avoid them).
This talk will give you a fresh perspective on the use and abuse of mock objects, explore why this topic is crucial for test suite reliability, and equip developers with strategies to make these distinctions without the guesswork.
#Details
For years now I've had varying degrees of success with mock objects but until recently I didn't have a narrative in which to tell the story of "good mock/bad mock"
In this talk I'll contrast 2 wildly different component integration testing styles. The first example, which relies heavily on mock objects, is intended to show how verification centric testing can be harmful in 3 unique but related areas.
- Refactoring is painful
- Real regressions are not surfaced
- Tests fail to act as living documentation
Point 1 -> With mock heavy integration testing you begin to encode implementation details about how your components works. These implementation details later slow you down when it comes time to refactor a dependency that component relies on.
Point 2 -> With mock heavy integration testing you will not quickly surface defects that only show up after a component + child components/services are wired together. This generally means less "real" confidence as we make changes in the system. We are also less trusting and will quickly fall back to manual testing/ or throwing it to QA for further verification (both are more costly long term).
Point 3 -> With mock heavy integration testing you often only see one small part of the story (ie: clicking radio button will fire action). With such a focus on isolation you often skip over the impact of that click event (ie: what the user expects should happen). As a result new developers will have less context about "why" this component was written or what makes it useful. Having this tribal knowledge encoded in the test itself acts as living documentation for the next developer.
No talk would be complete without a look at the other side of the coin. For starters, the integration testing style I'm suggesting above^ can certainly be a bigger investment up front. Also the tests themselves have a larger surface area of coverage meaning unrelated changes could ripple outward breaking what might appear like an unrelated test.
Finally I'll give some love to the area in which mock objects shine -> calling out bad design (aka: test friction). I will play devil's advocate a little here by sharing how my experience, and the powerful abstractions in ember specifically, have allowed me to be ignorant of this for the most part.
My hope is that this talk will highlight when and how over mocking can be harmful to ember developers by looking at real examples derived from working application code.