Custom test doubles vs OpenStruct vs RSpec doubles

results.md

Instantiating: creates I test doubles with N methods each. Calling methods: calls each method on the test double I times.

• 5 methods per test double, 1000 iterations:
  • 4720ms: Instantiating RSpec doubles
  • 163.91ms: Calling methods on RSpec doubles
  • 126.35ms: Instantiating OpenStructs
  • 65.96ms: Instantiating MyDouble class
  • 18.74ms: Calling methods on OpenStructs
  • 17.2ms: Calling methods on MyDoubles
• 15 methods per test double, 1000 iterations:
  • 14600ms: Instantiating RSpec doubles
  • 502.76ms: Instantiating OpenStructs
  • 468.19ms: Calling methods on RSpec doubles
  • 128.82ms: Instantiating MyDoubles
  • 64.73ms: Calling methods on MyDoubles
  • 64.15ms: Calling methods on OpenStructs
• 15 methods per test double, 1 iteration:
  • 4.64ms: Calling methods on RSpec doubles
  • 3.83ms: Instantiating RSpec doubles
  • 0.6ms: Calling methods on OpenStructs
  • 0.57ms: Instantiating OpenStructs
  • 0.37ms: Instantiating MyDoubles
  • 0.34ms: Calling methods on MyDoubles

Raw output:

➜  ~  N=5 I=1000 rspec -p openstruct-benchmark.rb
......

Top 6 slowest examples (5.11 seconds, 100.0% of total time):
  Benchmark instantiates RSpec doubles
    4.72 seconds ./openstruct-benchmark.rb:28
  Benchmark calls methods on RSpec doubles
    0.16391 seconds ./openstruct-benchmark.rb:44
  Benchmark instantiates OpenStructs
    0.12635 seconds ./openstruct-benchmark.rb:24
  Benchmark instantiates MyDoubles
    0.06596 seconds ./openstruct-benchmark.rb:20
  Benchmark calls methods on OpenStructs
    0.01874 seconds ./openstruct-benchmark.rb:38
  Benchmark calls methods on MyDoubles
    0.0172 seconds ./openstruct-benchmark.rb:32

Finished in 5.11 seconds
6 examples, 0 failures
➜  ~  N=15 I=1000 rspec -p openstruct-benchmark.rb
......

Top 6 slowest examples (15.83 seconds, 100.0% of total time):
  Benchmark instantiates RSpec doubles
    14.6 seconds ./openstruct-benchmark.rb:28
  Benchmark instantiates OpenStructs
    0.50276 seconds ./openstruct-benchmark.rb:24
  Benchmark calls methods on RSpec doubles
    0.46819 seconds ./openstruct-benchmark.rb:44
  Benchmark instantiates MyDoubles
    0.12882 seconds ./openstruct-benchmark.rb:20
  Benchmark calls methods on MyDoubles
    0.06473 seconds ./openstruct-benchmark.rb:32
  Benchmark calls methods on OpenStructs
    0.06415 seconds ./openstruct-benchmark.rb:38

Finished in 15.83 seconds
6 examples, 0 failures
➜  ~  N=15 I=1 rspec -p openstruct-benchmark.rb 
......

Top 6 slowest examples (0.01035 seconds, 100.0% of total time):
  Benchmark calls methods on RSpec doubles
    0.00464 seconds ./openstruct-benchmark.rb:44
  Benchmark instantiates RSpec doubles
    0.00383 seconds ./openstruct-benchmark.rb:28
  Benchmark calls methods on OpenStructs
    0.0006 seconds ./openstruct-benchmark.rb:38
  Benchmark instantiates OpenStructs
    0.00057 seconds ./openstruct-benchmark.rb:24
  Benchmark instantiates MyDoubles
    0.00037 seconds ./openstruct-benchmark.rb:20
  Benchmark calls methods on MyDoubles
    0.00034 seconds ./openstruct-benchmark.rb:32

test-double-benchmark.rb

require 'ostruct'

class MyDouble
  def initialize(name, methods={})
    @name = name
    methods.each do |method_name, return_value|
      define_singleton_method(method_name) { return_value }
    end
  end
end

ITERATIONS = (ENV['I'] || 100).to_i
methods = (ENV['N'] || 5).to_i.times.each_with_object({}) {|x, h| h["attr_#{x}"] = 'hello' }

describe 'Benchmark' do
  let(:rspec_double) { double('RSpec double', methods) }
  let(:my_double) { MyDouble.new('double', methods) }
  let(:struct) { OpenStruct.new(methods) }

  it 'instantiates MyDoubles' do
    ITERATIONS.times { MyDouble.new('name', methods) }
  end

  it 'instantiates OpenStructs' do
    ITERATIONS.times { OpenStruct.new(methods) }
  end

  it 'instantiates RSpec doubles' do
    ITERATIONS.times { double('double', methods) }
  end

  it 'calls methods on MyDoubles' do
    ITERATIONS.times do
      methods.keys.each { |method| my_double.send method }
    end
  end

  it 'calls methods on OpenStructs' do
    ITERATIONS.times do
      methods.keys.each { |method| struct.send method }
    end
  end

  it 'calls methods on RSpec doubles' do
    ITERATIONS.times do
      methods.keys.each { |method| rspec_double.send method }
    end
  end
end