glv/experience_teaching_ruby_testing.md

## experience_teaching_ruby_testing.md

      
    Raw
  

              experience_teaching_ruby_testing.md
            
          
    Notes on teaching both test/unit and RSpec to new Ruby developers

@tenderlove asked "Is it good to teach RSpec (vs t/u) to people who are totally new to Ruby?" I have experience suggesting that it is a good thing; after a short back and forth, it seemed useful to write it up in detail.
Background

This goes back several years, to when I was the primary Ruby/Rails trainer for Relevance from 2006-2009.  I'm guessing that worked out to probably 6-8 classes a year during those years. Since then, RSpec has changed a fair amount (with the addition of expect) and test/unit has changed radically (it has an entirely new implementation, minitest, that avoids some of the inconsistencies that made test/unit a bit confusing during the time I'm writing about here).
I started out as an RSpec skeptic. I've never been afraid of what a lot of people denigrate as "magic" in Ruby libraries … to me, if you take the trouble to understand it, that stuff's just programming—it's just the way Ruby works.  But I've also known Ruby and test/unit since 2001, and I definitely had a little bit of old man "it was good enough for me, it should be good enough for you" attitude going on.  So I sneered at RSpec quite a bit. And while some at Relevance agreed with me, there were RSpec fans there, ranging from vehement (one thought RSpec was obviously the way everyone would be doing things in the future) to pragmatic (Chad Humphries spent Thanksgiving one year writing micronaut—which went on to become the core of RSpec 2.0—just to get over the "RSpec is too slow" objection so people would let him use RSpec on projects).
As a teacher representing our company, I wanted to emphasize test/unit, but I felt responsible to present both sides, in case a) I was wrong, or b) the students ended up being our clients, having to work with us on a project where we were using RSpec.
Teaching both test/unit and RSpec

So I decided I would teach both. I can't remember the exact details, but IIRC I would teach test/unit on day 2 of a four-day Ruby or Rails class, and then after continuing with some of the main topic, I'd present RSpec as "a popular alternative" on day 3.  And I noticed a consistent pattern: they would really start to get it when I got to RSpec. It wasn't dramatic, but it was noticeable … they wrote better tests, and struggled less, with RSpec.
Naturally, I wondered whether this was just because it was the second time around. They'd already had a learning experience with test/unit; maybe that (plus a good night's sleep) was setting them up for success with RSpec the next day.
So I started doing it the other way around. I would teach RSpec on day 2 and test/unit on day 3. To my surprise, RSpec was still easier for people to grasp. They caught on fairly quickly, but still struggled to write good tests with test/unit. Again, it wasn't a dramatic difference: there were still struggles with RSpec, and successes with test/unit. But the difference was noticeable in the students' work, and also in their reactions. I heard things like "RSpec just makes more sense." (Yes, I really heard that a lot!)
But what conclusions can we really draw from this? Perhaps I just had a knack for teaching RSpec, and my long familiarity with test/unit blinded me to some of the problem areas, so that I didn't teach test/unit as well. Maybe the mix of students had something to do with it … or perhaps the big changes in test/unit since then make all of this irrelevant.
How I Teach

The variable I know the most about is the fact that I was the teacher in all of these classes. So it's worth talking a bit about how I teach.
I have always thought it was a mistake to use abstraction to hide the way things work underneath. Abstraction is useful because it means you don't have to pay attention to all the details all of the time. But you get in trouble if you try to pretend those details don't exist, or think you don't need to understand them at all.
When I'm confronted with a "magic", DSL-ish Ruby API, my first impulse is to open the source and learn a little bit about how the magic works. And I teach things that way, too.  I'll often show how to write the basic code without the "magic" library, and then show how to gradually abstract the details away until you end up with the fancy, magic interface. That provides the proper foundation for using the interface successfully.
For example: I hate it when APIs (or languages, or whatever) are presented as "it's just English!" That doesn't give anyone anything useful to work with; it's just trying to allay fears, and it replaces a mythical danger (the thing people are afraid of simply because it's unknown) with a real danger: you're telling them they don't need to learn anything, when in fact the opposite is true.
So while I acknowledge that there may be some value in the "english-like-ness" of RSpec, that value doesn't mean you don't need to learn the syntax. And I worked hard to build a solid foundation for that, trying to distill the essentials of RSpec down to the simplest possible core. I explained the difference between expectations (should, should_not) and matchers (==, have, be, etc.).  I drilled them on the syntax ("Repeat after me: 'dot should space match …' OK, and now 'dot should underscore not space match …'"). I showed them the fully parenthesized versions of those expressions. I gave a sketchy (but realistic) overview of how RSpec implements its expectations and matchers. And I just can't recall anyone having serious trouble with it, which really surprised me.
And I did a similar thing with test/unit, drilling people on the "expected comma actual" ordering, explaining why that was important, explaining the way the method naming pattern worked, etc.
Caveats / Things to Note


Many of these students were new to unit testing … it was clear to me that RSpec helped students to get over the hump with learning about testing, but it wasn't clear that someone who already got testing did better with RSpec. (But it wasn't clear that they did worse, either.)


The original implementation of test/unit had its own quirks and inconsistencies. One was that positive and negative assertions came in pairs, and it was common for library authors to forget the assert_not version; some of the Rails custom assertions only came in the positive variant. By comparison, RSpec's clean separation of expectations and matchers, with should and should_not as the positive and negative expectation methods, eliminated that inconsistency, which was a big help. Today's minitest, with its paired assert and refute methods, is a big improvement (although I personally detest "refute" as a part of the testing vocabulary).


I know that many people were repulsed by RSpec's insertion of the should method in every object. But it had the advantage of making the expected/actual distinction a bit more intuitive. expected.should == actual doesn't seem quite right; it makes much more sense to say actual.should == expected. That seems like a very subtle point, but in the classes I taught, that ordering was never a problem for students, whereas test/unit's assert_equal expected, actual is arbitrary, and frequently caused confusion (not to mention erroneous, misleading failure messages). RSpec's new expect(actual).to == expected seems slightly worse to me (in that respect), but doesn't seem to cause too many problems; minitest still uses the arbitrary ordering that you simply have to be aware of. To be fair, it quickly becomes second nature, but we're talking here about how easy things are for the newcomer.


RSpec did start out as an experiment, and the developers tried some very bold ideas before settling down a bit and backing off of some things. There were definitely parts of RSpec that went a bit too far, and many parts of RSpec have been deprecated, removed, or extracted into separate gems to keep the basics a little more reasonable.


Many objections to RSpec rest on its supposed complexity: it has APIs and methods that you have to learn for things that Ruby just provides naturally! Why not just use classes and methods?
The problem, as I see it, is that there's a lot of hidden complexity in test/unit that we experienced Ruby developers are just used to, but that often catches new programmers by surprise. The way test/unit uses classes and methods isn't always obvious. What's that method naming convention again? What happens if I lapse back to my Java ways and write a method called "testSomething"? (It's silently ignored.) How does test/unit find those methods? Can I write a module full of test methods and mix that into a test class? (Yes, but it's not a dumb question, because there was a time when that didn't work; the framework must be written so as to make that work, and it's not a foregone conclusion that it was written that way.) And there's more like this. Learning curves can be complex; it's not nearly as clear-cut as "that's simple, this is complex" or "that's easy, this is hard" or even "that's intuitive, this is obscure".