Last month, I read "Unit Testing Principles, Practices, and Patterns" by Vladimir Khorikov. This is a short collection of my thoughts on this work.
Overall, I found the book to be the best intermediate to advanced works on testing that I have read. This is not your 'typical' unit testing book that first tries to convince you that testing is important, than walks you through how to create very rudimentry unit tests. Neither does it expose a particular development practice like TDD. Rather, Khorikov starts with the premise that you already see the value in testing and are already writing tests. The goal of the book is to help you write better, more useful tests by identifying the attributes of a good test. As a developer recogonizes and understands the attributes of a good test suite and how to write one, they begin to write better, more testable code, creating a virtuious cycle in the codebase.
Khorikov's big idea comes in chapter 4, after he has described the state of software testing, different ideas on unit testing and test anatomy using the arrange, act, assert pattern. This leads up to his definition of the four pillars of a good automated test:
- Protection against regressions - how well does the test catch defects
- Resistance to refactoring - Does the test continue to work if the system under test is refactored, or does will the test require substantial refactoring as well?
- Fast Feedback - how fast the test runs
- Maintainability - how hard is the test to set up, and how easy is it to understand the test
The first two are arguably the most important. Protection against regression is the reason to write tests in the first place. However, the test should be written such that it isn't tightly coupled to the implementation of the system under test. This is the second pillar: resistance to refactoring. This pillar is about reducing false positives - failing tests that do not indicate a regression.
Khorikov argues that maximizing a test's value is really about a balance between the first and third principles. The second and fourth are binary - a test either is maintainable or it isn't, and it either is tightly coupled to implementation or not. A good test must have both. But the other two principles are not independent and cannot both be maximized, so a valuable test strikes a compromise. On the one hand, unit tests provide for faster feedback while still catching many regressions. On the other, integration tests are slower, as they usually involve shared, out of process dependencies, But by testing more of the system and its collaborators, they provide better protection.
Khorikov uses these pillars in further discussions on how the use of test doubles (mocks, stubs and spies) can contribute to test fragility. That isn't to say that he is completely opposed to mocking. Rather, mocking should only be used in place of shared, out-of-process dependencies such as external services. The overuse of mocking can hide defects (reducing the first pillar) and couple tests too closely to the implementation (the second pillar). Khorikov argues that code that is both complex and highly collaborative (interacts a lot with other systems, which would get mocked in tests) must be refactored, and he provides tools both for identifying such code, and for refactoring it to make it more testable (notably using the Humble Object pattern).
Overall, I found "Unit Testing Principles, Practices, and Patterns" to be both highly informative and thought provoking. Although Khorikov uses a statically typed and compiled language (C#) for all the examples, I do think that most of the big ideals translate well to Python and other languages. And while I'm not completely convinced on all of his ideas, this book has helped me look more critically at my own tests and how to improve them and, in the process, write better, more testable code.
Well, I just read the part were the author writes on that matter: