+ R1: A good test suite increases confidence that code changes won't cause
+ unexpected side effects without being noticed.
+ R2: Tests reduce the cost of structural changes in later stages of a project
+ because fewer bugs sneak into a system unnoticed.
- R3: Brittle tests can increase the cost of refactoring if they focus on
- superficial details rather than core functionality.
- R4: Each refactoring may involve removing old tests and adding
- new ones, increasing total effort per change.
+ A1: Well tested code tends to be well factored, which makes introducing
+ new features easier.
+ A2: Tests may help developers arrive at better designs and interfaces
+ between systems up front, reducing the amount of rework that is needed later.
- A3: In the early stages of projects, the cost of writing tests reduces the
- speed of iterating through different ideas, so the cost of running into
- a dead end is higher.
- A4: Figuring out which tests to start with (and at what level of abstraction
- to start at) can be challenging in projects that are not well defined.
- A5: Tests force domain modeling decisions early on, making it harder to make
- major decision changes later, and harder to defer design decisions.
- A6: Slow test runs can reduce the benefits of an automated test suite by
- slowing down the feedback loop.
+ L1: A good test suite shows system use cases at various levels of abstraction,
+ allowing developers to gain some insight into how different subsystems
+ are meant to be used.
+ L2: Tests force interfaces to be thought about early on, which tends to
+ increase their ease-of-use and quality.
+ L3: Tests help us resolve inconsistencies between documentation and
+ implementation by verifying specific expectations.
- L4: Learning testing tools and techniques involves a significant
- up-front commitment.
- L5: Test suites are often used as project documentation
- (which can be useful), but are a poor substitute for high-level,
- topical documentation and example programs because they can be
- hard to interpret in isolation.
- L6: Poor quality test suites can be more confusing than
- enlightening if they test via paths that would be awkward
- in real use, use contrived examples, or involve a lot of
- complex setup that is only loosely relevant to the task at hand.
- L7: The seemingly endless variety of testing tools and
- methodologies make for a steep learning curve even for
- experienced developers.
+ C1: Writing tests requires a fairly clear understanding of requirements up
+ front, which may encourage clarifying specifications and resolving their
+ ambiguities early on.
+ C2: Regression tests help ensure that incorrect behavior does not
+ recur once a fix is introduced.
- C3: Tests can represent incorrect assumptions or have defects in
- them, just like implementation code can. This can lead to a false
- sense of security for those who depend on passing test suites as a
- sign of correctness.
+ M1: A good test suite makes changing or updating dependencies safer,
+ because a test suite will help catch dependency-related problems.
+ M2: Well-tested projects make it easier for developers who are
+ new to a codebase to safely make fixes and changes without
+ understanding the nuances of the entire project, allowing
+ maintenance effort to be distributed.
- M3: Every line of code in your tests also needs to be maintained,
- and tests can often have much more code than implementation code does.
- M4: Testing tools introduce new dependencies and potential
- maintenance costs into projects.
+ D1: TDD can act as "training wheels" for software design, reducing the
+ barrier to entry for beginner/intermediate programmers to concepts
+ like SOLID, Law of Demeter, etc. The benefits of these techniques
+ directly improve testability, which makes the effort in applying
+ them feel worthwhile.
+ D2: TDD discourages taking shortcuts in API design, because brittle
+ interfaces are harder to test than well-factored ones.
+ D3: TDD encourages clear, well defined domain models, as well as
+ "discovered objects" (i.e. reusable bits of functionality that
+ weren't directly visible from the project's specification).
- D4: TDD discourages unstructured play early on in a project,
- which may reduce the number of different ideas considered
- before a particular design is settled on.
- D5: Many insights about software design that can be learned via
- TDD can be also be practiced in less formal ways, with similar
- benefits and fewer costs.
- D6: TDD can lead to overengineering in some situations: the tendency
- to focus on domain modeling at all levels of abstraction comes at
- the cost of potentially spending too much energy improving
- unimportant subsystems.
+ Q1: Measurements such as code coverage, test run times, and
+ continuous integration reports give us useful ways of analyzing
+ some important traits of our projects.
- Q2: It is easy to over-emphasize or misinterpret what quantitative metrics
- actually say about our projects.
+ Good software testing practices (in particular TDD), place strong
+ emphasis on software quality and maintainability.
+ Good testing practices lead to increased trust among developers
+ in their own work as well as the work of others, which reduces
+ defensive programming.
- Dogmatism about how to test and what tools to use can be
- distracting and also lead to unproductive fears and insecurity
- about whether one is "doing it right".
- Overemphasizing software testing as a golden hammer can make it
- easy to forget that product quality is not the same as code quality,
- and that the former is more important than the latter. (Of course,
- it is worth keeping in mind that code quality at least implicitly
- affects product quality)
