test thoughts

gistfile1.txt

So agree with @davetron5000 that defects to users are the most important to prevent. 
I also agree that a test failure doesn't mean a defect occurred as tests themselves 
often just are bad tests that don't test what they should. While you do have to maintain 
and think about a test suite, well written tests should be entirely isolated. 
The app code you do need to reason about the changes of the code effect other pieces of 
the application. So while a buggy tests might be a bit flaky, if it just passes and I am 
not specifically working with that test it isn't increasing my thought load while reasoning 
about the code structure.

I still think people should maintain and keep clean tests, and unless you are doing payments, 
NASA, or other highly important tasks think that 100% test coverage just leads to overly coupled 
tests that end up 'locking' code in so it can't be change rather than testing correct behavior. 
So I still think the LOC ~= Bugs hold true in tests it often just doesn't matter as much and don't 
have as large of a negative impact on a project.

Definitely not trying to make the claim of delete your test suite, less code -> less bugs. Not exactly 
sure of how I weight the importance of test code to production code yet, but I know they don't have exactly 
equal value in my mind.

This was a response to twitter where I couldn't fit my thoughts into 140 chars (https://twitter.com/davetron5000/status/269209497753513984)

My take: If you have a one codebase large enough that the tests can't be maintained along with the code under test then something is awry and is a tell-tale that decomposition needed. I guess what I'm saying is that the code under test is what drives writing tests to begin with. So if you have too many tests it's likely there's too much code (in one library/module/service/whatever).

As far as systems go, those made up of smaller libraries/services, that's where the discussion gets interesting. In that scenario the aggregate of all systems in play is the "codebase" and to some extent unavoidable. Integration-level tests then encounter the same meta-discussion above.