Create a gist now

Instantly share code, notes, and snippets.

What would you like to do?

I highly suspect that the RSpec core team all use black backgrounds in their terminals because sometimes the colors aren’t so nice on my white terminal

I certainly use a black background. I'm not sure about the other RSpec core folks. Regardless, if there are some color changes we can make that would make output look good on a larger variety of backgrounds, we'll certainly consider that (do you have some suggested changes?). In the meantime, the colors are configurable, so you can change the colors to fit your preferences on your machine. First, create a file at ~/.rspec. If this file exists, RSpec will use the command line options configured in this file, allowing you to set personal preferences that apply to any project on your machine. In this file put something like:

--require "~/configure_rspec_colors.rb"

And then in ~/configure_rspec_colors.rb (or whatever file you decide to require from ~/.rspec) put:

RSpec.configure do |c|
  # Each of these can be one of the ANSI color code integers or one
  # of [:black, :white, :red, :green, :yellow, :blue, :magenta, :cyan].
  # The values I've assigned here are the defaults so you'll want to change them.

  # Color config is available in RSpec 2.13+. Since this file will
  # always be loaded by RSpec and you may have projects on earlier
  # versions, it's a good idea to only do this on versions that support it.
  if c.respond_to?(:default_color=)
    c.default_color = :white
    c.detail_color  = :cyan
    c.failure_color = :red
    c.fixed_color   = :blue
    c.pending_color = :yellow
    c.success_color = :green
  end
end

But where is the helper method defined? What’s it’s visibility? Can I put it in a module? Can I use inheritance? Who can call it? Can I call super from the extracted method? If so, where does super go? These are not the types of questions I want to be pondering when I have 3000 test failures and really long spec files to read through. My mind spins off in to thoughts like “I wonder how RSpec works?”, and “I wonder what my cat is doing?”

From what I can gather, calling describe in RSpec essentially defines a class.

This is indeed what it does (plus some other stuff that I'll get into in a bit). I think it'll help you understand by showing an example:

describe "Using an array as a stack" do
  def build_stack
    []
  end

  def stack
    @stack ||= build_stack
  end

  it 'is initially empty' do
    expect(stack).to be_empty
  end

  context "after an item has been pushed" do
    def build_stack
      super.push :item
    end

    it 'allows the pushed item to be popped' do
      expect(stack.pop).to eq(:item)
    end
  end
end

This example is almost exactly (sans some details such as RSpec assigning the classes to different constants) doing this:

class UsingAnArrayAsAStack < RSpec::Core::ExampleGroup
  set_it_up "Using an array as a stack"

  def build_stack
    []
  end

  def stack
    @stack ||= build_stack
  end

  it 'is initially empty' do
    expect(stack).to be_empty
  end

  class AfterAnIteHasBeenPushed < self
    set_it_up "after an item has been pushed"

    def build_stack
      super.push :item
    end

    it 'allows the pushed item to be popped' do
      expect(stack.pop).to eq(:item)
    end
  end

  children << AfterAnIteHasBeenPushed
end

RSpec.world.register UsingAnArrayAsAStack

As you can see, RSpec is in fact creating classes, just like you guessed, and it uses class_exec to evaluate the describe block. Individual examples are evaluated in the context of an instance of the class (like minitest!). Nested contexts are simply subclasses, which means they inherit all helper methods just like you would expect, and you can use super as you would expect. You can also see that RSpec does some extra stuff (set_it_up plus registering the classes via children << and RSpec.world.register for the top-level one) when you call describe or context. (I'd also point out that we do not consider set_it_up, children and register to be part of our public API and this may break in future versions...)

Can I put it in a module?

You can indeed. This works:

module MyHelpers
  def helper
    11
  end
end

describe "something" do
  include MyHelpers

  it "works" do
    expect(10).to eq(helper)
  end

  it "really works" do
    expect(11).to eq(helper)
  end
end

If you wanted the module globally included in all example groups, you can call include on config:

RSpec.configure do |config|
  config.include MyHelpers
end

But if that’s the case, why not just use Ruby classes?

There are a few reasons for this:

  • In its original formulation (as per Dave Astel's and Dan North's articles introducing BDD), one of the things BDD was about was using a new vocabulary to discuss testing, that, in their experience, led you in a better direction when learning TDD. describe and it come from that heritage. I think that they were on to something, but I also think that the alternate vocabulary is most useful for people learning TDD (as they discuss in their articles). For someone who has developed code using TDD for years, the alternate vocabulary probably doesn't matter much, and may even be a hindrance for someone very used to the xUnit style.
  • describe allows you to pass arbitrary English descriptions. Ruby classes have many restrictions on what is allowed in their names. Many people (including myself!) find it very useful to be able to use arbitrary English descriptions. I often include a rationale in my descriptions so that if a test ever fails, the developer is given an explanation for why the behavior expressed in the test was desirable at the time it was written (which can help them decide if they still need to maintain that behavior). For example, here's one test description from a project at work:
it 'ignores subsecond differences in timestamps since MySQL and redis store them at different granularities' do
# ...
end
  • The fact that describe is a method that accepts arguments means we can provide APIs for extensions to be able to get the "class being tested". This is a more explicit alternative to what ActionController::TestCase does: rather than trying to infer the class-under-test from the test class name, the user gives it to us as an explicit argument. Explicit argument means less potential confusion when the user misspells it (they'll get an immediate NameError from ruby!), and the user doesn't have to know anything about how inferrence works.
  • describe also supports RSpec's metadata system, where example groups (test classes) and examples (individual test methods) have attached metadata that can be used to slice and dice your suite in many ways. For example, you can tag an example group as :slow (describe MyClass, :slow do) and exclude groups with that tag (rspec --tag "~slow"). rspec-rails uses :type metadata to know which Rails testing APIs to make available to which example groups. You can tag an example group as :pending to indicate that it should not yet pass, and RSpec will run it, expecting it to fail, and notify you if it now passes. These are all features that would be harder to support if you couldn't pass additional arguments when defining your test class.

You’ll see “hi!” printed 3 times, once for each it, and “hello!” printed once. For the nested context, I would like to print “hello!” before it prints “hi!”. In a normal inheritance situation, I would change the call to super. I am not sure how to do what I just described without learning a new thing

Honestly, I never thought about the fact that the def setup style of hooks allows you to control the order like this! This is indeed something that doesn't have a simple solution in RSpec. I'll think more about if there's anything we can do to improve it.

Overall, when I run into this type of situation -- where I've got a code in a before hook, but an individual test needs to run something else first -- I treat it as a code smell. It suggests that the code doesn't really belong in a before hook since it's not actually something that we always want to run first. Usually I'll pull the code in the hook out into a helper method and then explicitly call it from the places where it needs to happen.

All that said, there is a way to accomplish what you are trying to do, but it's pretty obtuse:

describe "something" do
  def say_hi
    puts "hi!"
  end

  before do |ex|
    say_hi unless ex.metadata[:skip_hi]
  end

  # ...

  context "another thing", :skip_hi do
    before do
      puts "hello!"
      say_hi
    end

    # ...
  end
end

This uses the metadata system (which I mentioned earlier) to skip the logic in the before hook, and then in the nested group we can call it at an appropriate time. As I said, that's far less elegant than using super like you mentioned, and I don't recommend doing this, but it could be a useful temporary step during a larger refactoring.

Another thing that bugs me is that if I do try to refactor my RSpec tests, and I change any lines, then the line that I copy and pasted to run just the one test won’t work anymore. I have to figure out the new line. I can use the name of the test by specifying the -e flag on RSpec, but then I lose the “copy, paste, run” feature that I love so much.

My suggestion is to use RSpec's focus-filtering feature. The spec_helper.rb file generated by rspec --init has a couple lines of config that you'll need for this:

RSpec.configure do |config|
  config.filter_run :focus
  config.run_all_when_everything_filtered = true
end

With that configured, you can (temporarily) add :focus metadata to any example or example group, and RSpec will run just those examples or example groups. If nothing has :focus, then everything will be run (that's what the last line of config is for). With the :focus metadata it doesn't matter if your specs move around to other lines. Given that you'll only want to temporarily change the metadata, and how useful this feature is, we also have some shortcuts: fit is an alias for it with :focus, fdescribe is an alias for describe with :focus and fcontext is an alias for context with :focus -- so just prefix any of these with the letter f to temporarily make RSpec run only them.

Now, do I care which one you use? No. As long as you test your code, I am happy. A professional developer should be able to work in either one of these because they essentially do the same thing: test your code.

<3 <3 I completely agree with this. I'd go so far as to say that as lead RSpec maintainer, I'm extremely grateful that RSpec's not the only testing library around. It's good for the Ruby community that we have multiple well-maintained libraries for testing.

Thanks again, @tenderlove!

Thank you for taking the time to write this up. I really appreciate it, and it definitely helps me understand whats going on from the perspective of a minitest user!

Owner

myronmarston commented Jan 24, 2015

Thank you for taking the time to write this up. I really appreciate it, and it definitely helps me understand whats going on from the perspective of a minitest user!

I'm glad to hear you found it useful!

JEG2 commented Feb 3, 2015

@myronmarston: could RSpec spit out the copy & paste lines using the -e syntax? That seems more robust in the face of changes than a line number.

Owner

myronmarston commented Mar 6, 2015

@myronmarston: could RSpec spit out the copy & paste lines using the -e syntax? That seems more robust in the face of changes than a line number.

Sorry, I just saw your comment now. Github doesn't appear to email me when comments are posted :(.

Anyhow, yes, RSpec could do that. However, I don't think we'll make that change, for a few reasons:

  • Many people use the summary output as a means to locate the examples, not just as a re-run command. Using -e would provide the user with a way to rerun it but would be more difficult to go right to the source line of it.
  • The example doc string is printed after the path/to/spec.rb:15 # bit, so you can always copy that and paste it as a -e option.
  • What is printed out for each run is entirely up to the formatter -- what you see is just what the built-in formatters print. If you wanted a -e rerun line it would be pretty easy to write your own formatter to do that.
  • There are some new features coming in RSpec 3.3 that I think may provide better ways to do what you're wanting. Specifically:
    • As of PR #1884 each example is now assigned a unique ID that is based on its index (e.g. it's the 3rd example of the 2nd nested group within the 1st top-level group of file x) rather than its location and you can use rspec path/to/spec.rb[1:2:3] to run such an example. The ID is robust in the face of changes to line number or changes to the example doc string, but is not robust in the face of changes to the ordering of examples, or changes to the structure (e.g. nesting it in another example group). More importantly, IDs are guaranteed to uniquely identify an example in a way that locations and descriptions are not.
    • One of the features I'm working on that builds on top of the new IDs is a --rerun-failures (or perhaps --only-failures) flag that will make it easy to just re-run the examples that failed. Combined with --fail-fast, you could do something like run rspec --rerun-failures --fail-fast --order defined repeatedly and it will keep targetting the next still failing example until it passes, then the next one, etc. See PR #1888 for more on what we're planning for that.

nathanl commented Mar 6, 2015

@myronmarston Neat! Keep on cranking out the awesome! 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment