Skip to content

Instantly share code, notes, and snippets.

@garybernhardt
Created November 26, 2009 20:13
Show Gist options
  • Save garybernhardt/243640 to your computer and use it in GitHub Desktop.
Save garybernhardt/243640 to your computer and use it in GitHub Desktop.
The Limits of TDD
#date 2009-11-09 21:47
#tags python,tdd
<p>
My <a href="/2009/11/how_i_started_tdd.html">last post</a> about TDD
generated some great responses, some of which were skeptical. A few common
complaints about TDD were brought up, and posed with civility, so I'd like
to address them.
</p>
<h4>Complaint: You weren't stupid enough</h4>
<p>
When TDDing Fibonacci, we could get to a point where we have this
function (and I did write exactly this code in my last post):
</p>
<pre><code>def fib(n):
if n == 0:
return 0
else:
return 1</code></pre>
<p>
But why should we write that? Why not this instead?
</p>
<pre><code>def fib(n):
return [0, 1, 1][n]</code></pre>
<p>
This comes down to how we define "simple". In TDD, we make tests pass by
making the simplest possible change. So, which of the above two is
simpler?
</p>
<p>
Defining that word is our job; TDD as a process says nothing about it.
The definition is a <em>huge</em> variable and, in my experience, it's the
primary axis along which our skill as TDDers grows once we reach minimal
competence. Note that we still have to define "simple" even if we're not
doing TDD, but we won't have the test-driving pressure forcing the
definition to be refined.
</p>
<p>
Regardless of how "simple" is defined, we must eventually accept that an
arbitrarily long list is not the simplest thing. At that point, we
refactor. Depending on the definition of simple, it may take seven tests
to get to the final refactor instead of five. So what? Two TDDers need not
generate the same tests, and this isn't a problem at all.
</p>
<h4>Complaint: TDDed tests are prescriptive</h4>
<p>
This is a complaint that TDDed code does exactly what the tests say it
should do, so there might be bugs. If I write the wrong test, the
reasoning goes, then it will drive me to write the wrong code.
</p>
<p>
When would we write the wrong test? Only when we misunderstand the
problem. If we misunderstand the problem, and we go straight to the code,
then we're be encoding our incorrect understanding <em>directly in the
code</em>. That's bad. By writing the tests first, we have some extra
protection against misunderstanding: every assumption about what the
system <em>should</em> do is encoded as a test, and each test has a good
name.
</p>
<p>
Often, this will point out our confusion during the TDD process &ndash;
we'll find that we want to write a test whose name contradicts another
test's name. Even if we translate our misunderstanding into a bug,
however, good test names make it easy to revisit our assumptions later. A
subtle, five-character change to the code may have been driven by a
sixty-character test name, which will be easier to understand.
</p>
<h4>Complaint: Choosing tests is hard</h4>
<p>
When TDDing Fibonacci, I tested fib(0) first. Why did I test fib(1) next
instead of fib(37) or fib(51)?
</p>
<p>
Because it was obvious! The problem domain of a unit test is necessarily
small, so it's usually clear what the next step is. If the next step
isn't clear, it probably means that the unit under test is too large
(making it hard to think about extending it for another case), or that we
don't understand the problem well enough (making it hard to think about
what the code should do at all). In either case, TDD has just helped us:
it's either pointed out a bad design, which we should fix, or it's pointed
a gap in our knowledge about the problem, in which case we should put the
keyboard away and fill that gap.
</p>
<h4>Complaint: The code you TDDed was bad</h4>
<p>
The particular code I came up with in my last blog post was a slow,
recursive Fibonacci solution. Two people mentioned this in the comments.
</p>
<p>
TDD doesn't solve problems like "my run time is superlinear" or "my
database loads aren't eager enough." It's not <em>supposed</em> to solve
those problems! TDD frees us to solve those hard problems well by (1)
pushing us toward a good, decoupled design and (2) providing us with
large, fast test suites.
</p>
<h4>Complaint: TDD requires too much typing</h4>
<p>
This one has the easiest answer of all: <a
href="http://anarchycreek.com/2009/05/26/how-tdd-and-pairing-increase-production/">typing
is not the bottleneck</a>. Just think about it for a minute. Go back
and look at how many lines of code you actually generated yesterday. How
long would it take you to type it all in one long burst? A few minutes?
Seriously, typing is not the bottleneck.
</p>
<h4>TDD is not magic</h4>
<p>
Let's recap:
</p>
<ul>
<li>Complaint: You weren't stupid enough</li>
<li>Response: There's more than one legitimate definition of "stupid".</li>
</ul>
<ul>
<li>Complaint: TDDed tests are prescriptive</li>
<li>Response: This is a feature. Stating our assumptions up front exposes
misunderstandings.</li>
</ul>
<ul>
<li>Complaint: Choosing tests is hard</li>
<li>Response: This is also a feature. It tells us that our design is bad
or that we don't understand the problem.</li>
</ul>
<ul>
<li>Complaint: The code you TDDed was bad!</li>
<li>Response: TDD does not free us from thinking. TDD is not magic.</li>
</ul>
<ul>
<li>Complaint: It's too much typing.</li>
<li>Response: Typing is not the bottleneck.</li>
</ul>
<p>
Many complaints about TDD are complaints that it <em>doesn't</em> solve
some problem. These are not problems with TDD &ndash; it's not supposed
to solve every problem!
</p>
<p>
Dynamic languages don't make coffee, continuous integration doesn't shine
shoes, and TDD doesn't make code scale. It's simply the basis of a solid,
disciplined process for building software &ndash; a beginning, not an end.
</p>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment