The Sad Tale of a Webrat Outcast

gistfile1.rb

	config.before(:suite) do
    DatabaseCleaner.clean_with(:truncation)
  end
  config.before(:each) do
    DatabaseCleaner.strategy = :transaction
  end
  config.before(:each, :js => true) do
    DatabaseCleaner.strategy = :truncation
  end
  config.before(:each) do
    DatabaseCleaner.start
  end
  config.after(:each) do
    DatabaseCleaner.clean
  end

if Rails.env.staging? || Rails.env.production?
  WickedPdf.config = {exe_path: Rails.root.join('bin', 'wkhtmltopdf-amd64').to_s}
else
  WickedPdf.config = { exe_path: File.join(Rails.root, 'bin/wkhtmltopdf')}
end

def title_is_correct(title)
  expect(response.body).to have_xpath("//title", text: "#{BaseTitle} | #{title}")
end
def h1_is_correct(h1_text)
  expect(response.body).to have_xpath("//h1", text: "#{h1_text}")
end

Webrat-refugee.md

#The Long and Sad Story of an Attempted Upgrade to Rspec 3 with lessons for Webrat Outcasts

On a bright sunny day, with a sweeping view of beautiful Port Stephens, I made a terrible mistake. Instead of playing golf, swimming, or sitting in a local restaurant sipping a good wine from a nearby Hunter vineyard, I decided to upgrade to Rspec 3. I know, I know, but what true developer can refuse the opportunity to upgrade their tools when they have downtime?

Of course, I did fall for the siren call of an easy victory. The Relish upgrade instructions do say 'but our hope is to make this the smoothest major-version gem upgrade you've ever done'. I had recently done the Rails 4 upgrade, and that went pretty smoothly, so I thought that an Rspec 3 upgrade would be the perfect thing for a vacation hack. Moderately interesting but not stressful. It will only take me an hour or two. Little did I know...

My target app was The Golf Mentor. This is a few years old, has survived from Rails 2, and has about 2,000 tests that take 6 minutes to run. It was using Rspec 2.14, and in general I like to keep aligned with current releases, so Rspec 3 seemed a good idea.

The upgrade process has been well thought through. It involves first upgrading to rspec 2.99. All your tests should pass at this point, but there will be a long list of deprecations. You can then wave a magic wand called transpec to upgrade almost all your syntax to rspec 3. At this stage all your tests should work, and you might have a few deprecation warnings. You then fix the deprecations, upgrade to rspec 3, and Bobs your Uncle, all tests should pass. I imagine that for most lucky people, this is what happens.

The troubles first started on rspec 2.99, all my tests failed. But I was not worried, generally when all the tests fail at once there is a single failure point. The error message was

     Failure/Error: Unable to find matching line from backtrace
     NoMethodError:
       undefined method `example' for #<RSpec::Core::Example:0x00000102e24138>

I tracked this down to this code in my rails_helper.rb file:

	if example.example.metadata[:js]
      DatabaseCleaner.strategy = :truncation
    else
      DatabaseCleaner.strategy = :transaction
	end

I could not remember why I had inserted this code, so I just went to the database_cleaner github readme and pasted in the example rails code from there. My first big mistake.

I ran rspec spec and it started off well, green dot after green dot, then one red F, then nothing. I waited, still nothing. I hit one ^C, then another one, then another, but the three of them hung there - ^C^C^C, like tear drops waiting to run down my monitor's cheek. I terminated the terminal window, opened a new window and typed rspec rails. Now there was not even any green dots, it just hung from the get go. There must have been some rspec process still running. The only way I could fix it was to reboot my system. I then had to experiment, trying to find which test was causing the problem. Each time I tried something I had to reboot my system. I had not had this sort of delay since my earliest coding experience, using punch cards in a batch entry system on a Cyber mainframe from the Control Data Corporation (yes, I am that old).

I eventually tracked the problem down to my feature specs that used javascript. When a bit more searching on the web, I found the correct way to set up database_cleaner for rspec 3 was

config.before(:suite) do
  DatabaseCleaner.clean_with(:truncation)
end
config.before(:each) do
  DatabaseCleaner.strategy = :transaction
end
config.before(:each, :js => true) do
  DatabaseCleaner.strategy = :truncation
end
config.before(:each) do
  DatabaseCleaner.start
end
config.after(:each) do
  DatabaseCleaner.clean
end

So with a bit more work, I had all my tests passing and was ready for the next step, which was to let transpec do its magic. And indeed it worked well. There were only a few problems, which I fixed relative quickly. Now I had a fully green test suite and no deprecation warnings, so I was ready for the next step, which was to upgrade to rpsec 3. The upgrade guide said that at this stage 'If anything fails, please open a Github issue'. Wow, if they are that confident, then my tests must pass, right?

Wrong, a wave of red Fs amongst placid green dots, a turbulent sea of trouble. There not one cause, there were many. Most of them seem to be connected to selectors, e.g.

expect(response).to have_selector("title", content: "Unsubscribe Newsletter")

I searched the issues on rspec-rails github repository, and did find mention. Someone had raised the issue, but was told that it was a capybara-webrat issue, and to stop using the webrat gem. The issue was then closed. In a nutshell, rspec 3 no longer supported webrat and capybara 2.0 did not support request specs.

In a sense the advice was good, but the solution was not easy. It was not a few copy and pastes. I had used many different selectors with different formats. Worse still, I needed to convert my hundreds of request spec tests to feature tests. The real difficulty is that even if the syntax is the same, capybara is more fussy. For example

fill_in :email, with: 'shelley@example.com'

works with webrat, even though the id of the input is user_email presumably because the text was 'Email'. Capybara would not accept the inaccuracy. From a general viewpoint this is good, but from the viewpoint of upgrading it meant that I would have many, many test failures.

I was already up to 6 hours, with no golf and no wine, I could see days and days of tedious editing and debugging to complete the upgrade. It was too much for me to face, I gave up. Actually I rationalized it as I was now working on a phased upgrade strategy. I resolved to write feature specs instead of request specs for any further integration tests, and when I did any work on old request spec, I would convert it to a feature spec. Above all, I would wait and hope for the day that someone clever writes a trans_webrat_to_capybara gem. The webrat gem has not been updated since 2011, but there have been 1.6 million downloads and it is still being downloaded at the rate of 5,000 a month. There must be a few other outcasts around.

By the way, none of this is a criticism of the rspec team, in the ruby on rails ecosystem, you can only work your part of it, otherwise the scope becomes too wide.

So I have wasted 6 hours, or 7 if you count the time writing this lament. Today is raining here, so I have decided to use rubocop to clean up all my syntax. There is a neat to-do procedure and an auto-correct feature, so it should only take one or two hours.

Postscript (11 months later)

First up, although rubcop did take more than a few hours, it was less than half a day, and I am constantly thankful I spend the time on it. The main pay off is with the SublimeText linter, which picks up many typos as soon as I enter them, rather than only finding them when I run the test or view the web page. It is great time saver and also reduces frustration.

Now, for all the fans of this gist (my wife and uh, actually not even my wife is interested), almost a year later, I was up at Port Stephens and decided to finish the job. It took three days, and it was not pretty. I had kept my resolution to not write any more webrat request specs, but I had not gradually changed over my old request specs to feature specs as I modified them. On the plus side, I had written custom matchers for commonly used tests, e.g. link_is_correct and title_is_correct, which did help. On the downside, it turned out that my controller and helper specs also were using webrat matchers, so the problem was about 3 times bigger than I realised.

It was not pretty because of the suprising number of quite specialised matches e.g have_selector(:input, id: 'something', value: 'something else') from my early days of spec writing. In later days I have either become lazier or manage to avoid the necessity for such detailed tests. Overall, it was a messy fight. I like to think I code with a rapier like precision, but this was aall out muddy wrestling match. Exception after exception, one quarter of the tests broken, no unbroken line of green dots for three days. But still I made it to Rspec 3. I should get fewer unfixable bugs, I became much better at using regex with Sublime Text, and my test suite runs about 30% faster.

So there you are, I have finally migrated to Rspec 3, and left the land of Webrat. There have been almost 400,000 downloads of webrat in the past 11 months, so it still going strong. Given it has not been touched for four years, this is a testament to the creators of the gem. To all webrat lovers, I bid you a fond farewell, but be sure, I am not coming back.