Corey Haines gave his Fast Tests talk at the Golden Gate Ruby Conference 2011 (http://confreaks.net/videos/641-gogaruco2011-fast-rails-tests), here's my feedback on it.
So first up, I think it's very interesting to see another take on how to speed up Rails testing. There's been a focus on it for a couple of years now with various solutions coming for it. From running tests in parallel, to tweaking the filesystem for faster access. Corey Haines take on it is that we should isolate ourselves from the framework that Rails provides and do as much as possible in plain Ruby objects and modules. In his talk he shows how he has been able to massively improve the runtime of his tests by implementing this.
I think it is a logical next step to the current that has been in the community for a while where we have been debating how to improve the design of Rails applications.
During the talk he mentions how he sees the traditional way of testing in Rails as test first rather than test driven. And argues that to be test driven means to change your design when your tests runs too slow. I do not agree with this statement and I will try to explain why.
In test driven development you need to appreciate the pain your tests are giving you. And feeling the pain you should try to alleviate it by changing your design. I agree with the principle in the general sense. As in when I have to do 10 lines of setup, stubbing 5 methods it's a symptom of bad design in that your object under test is probably not well designed.
In the talk Corey Haines focuses on test execution time as the pain. Which implies that practicing TDD our tests should drive us to change our design when running them takes too long. Something about this assertion does not feel right to me and I will try to explain why.
To me a software design is the way the software is structured into separate logical units. So TDD is basically a way to work out the structure of software, with the underlying idea that our design improves when we do TDD. Now using speed of tests as a way to improve the structure of software seems counter intuitive in so far as I can improve the speed and at the same time deteriorate the structure.
In the case of Rails my take on it is that we are using Rails as a DSL. Or stated another way: We are expressing our programs in the language that is Rails. This allows us to write easily comprehensible programs. Isolating Rails from our code is only good design when it improves the structure of our code. Isolating Rails when our code would be easier to understand not doing so is not an improvement of the design even if it leads to faster tests.
Another counter argument to using test speed as a metric for design quality is: If Rails was so fast that the inclusion of Rails in the test suite would have no impact on execution time would the quality of the design be considered to be of higher quality?
This is not to say that there are not valid reasons for wanting a faster test suite. Only that calling it a design issue rings somewhat false to me. I see it more as an orthogonal concern.
I would say that a fast test suite is a requirement to be able to do TDD more effectively. It can even have a profound impact as Corey Haines mentions. But I don't think you can drive a better design by aiming for faster tests.
Given that we want to be able to exclude Rails from a big part of our test suite how do we go about using the parts that we actually want to use even in our isolated classes? I guess this is mostly a matter of gaining access to things in ActiveSupport. Things like Enumerable#sum and Object#present? might be all over our code. Do we need to replace these with array.inject(0, &:+) and !object.nil? or do we include the parts of ActiveSupport that we rely on?
Personally I would favor the option of including ActiveSupport to gain the increased readability of the code. But this again leads to slower tests as then we need to load Rubygems and ActiveSupport.
How do we handle gem version resolution? Can we still use bundler? Even with the goal being to isolate our code as much as possible sometimes we would like to depend on certain gems. Or should we start injecting all dependencies to be able to easily stub them in our tests?
Using ActiveSupport does not mean you have to incur very long loading times. A quick benchmark placed it at 80ms (require 'active_support/core_ext'). See my example of how to add a unit/ test suite to a rails app that uses active support https://github.com/joakimk/fast_unit_tests_example