Exploring Minitest's parallelize_me! option, in conjunction with WebMock

README.md

Testing Minitest, parallelize_me! and WebMock

Testing the parallelize_me! option of Minitest, in conjunction with using WebMock to stub some requests.

It would appear that WebMock might not be thread safe as running these tests with parallelisation enabled results in a number of errors:

ParallelisationTest#test_242:
WebMock::NetConnectNotAllowedError: Real HTTP connections are disabled. Unregistered request: GET http://example.com/ with headers {'Accept'=>'*/*', 'Accept-Encoding'=>'gzip;q=1.0,deflate;q=0.6,identity;q=0.3', 'Host'=>'example.com', 'User-Agent'=>'Ruby'}

You can stub this request with the following snippet:

stub_request(:get, "http://example.com/").
  with(:headers => {'Accept'=>'*/*', 'Accept-Encoding'=>'gzip;q=1.0,deflate;q=0.6,identity;q=0.3', 'Host'=>'example.com', 'User-Agent'=>'Ruby'}).
  to_return(:status => 200, :body => "", :headers => {})

============================================================
    /private/tmp/wem-minitest-parallelize/.bundle/gems/gems/webmock-1.21.0/lib/webmock/http_lib_adapters/net_http.rb:114:in `request'
    /Users/chrisroos/.rbenv/versions/2.2.0/lib/ruby/2.2.0/net/http.rb:1285:in `request_get'
    /Users/chrisroos/.rbenv/versions/2.2.0/lib/ruby/2.2.0/net/http.rb:480:in `block in get_response'
    /private/tmp/wem-minitest-parallelize/.bundle/gems/gems/webmock-1.21.0/lib/webmock/http_lib_adapters/net_http.rb:123:in `start_without_connect'
    /private/tmp/wem-minitest-parallelize/.bundle/gems/gems/webmock-1.21.0/lib/webmock/http_lib_adapters/net_http.rb:150:in `start'
    /Users/chrisroos/.rbenv/versions/2.2.0/lib/ruby/2.2.0/net/http.rb:583:in `start'
    /Users/chrisroos/.rbenv/versions/2.2.0/lib/ruby/2.2.0/net/http.rb:478:in `get_response'
    /Users/chrisroos/.rbenv/versions/2.2.0/lib/ruby/2.2.0/net/http.rb:455:in `get'
    parallelisation_with_webmock_test.rb:15:in `block (2 levels) in <class:ParallelisationTest>'

1000 runs, 997 assertions, 0 failures, 3 errors, 0 skips

Gemfile

source 'https://rubygems.org'

gem 'minitest'
gem 'webmock'

Gemfile.lock

GEM
  remote: https://rubygems.org/
  specs:
    addressable (2.3.8)
    crack (0.4.2)
      safe_yaml (~> 1.0.0)
    minitest (5.6.1)
    safe_yaml (1.0.4)
    webmock (1.21.0)
      addressable (>= 2.3.6)
      crack (>= 0.3.2)

PLATFORMS
  ruby

DEPENDENCIES
  minitest
  webmock

parallelisation_with_webmock_test.rb

require 'bundler/setup'
require 'webmock/minitest'
require 'minitest/autorun'

class ParallelisationTest < Minitest::Test
  parallelize_me! if ENV['PARALLELIZE']

  def setup
    stub_request(:get, "http://example.com/").
      to_return(:status => 200, :body => 'stubbed-response', :headers => {})
  end

  1.upto(1_000).each do |count|
    define_method("test_#{count}") do
      response = Net::HTTP.get(URI.parse('http://example.com'))
      assert_equal 'stubbed-response', response
    end
  end
end

I just ran into this. At first, I thought my tests had issues. But I also defined an environment variable that would allow my tests to bypass webmock. I did this prior to enabling parallelization because I needed a way to get data off of the testnet, for quickly verifying new features and regression.

So in testnet mode, I can parallelize, at least. Kind-of reassuring.

My project is here: https://github.com/inertia186/obarc