Benchmark building a Ruby hash: #each - #each_with_object - #reduce - Hash[map] - #map.zip(map).to_h - #reduce-merge

benchmark_building_a_hash.rb

require 'benchmark/ips'

Benchmark.ips do |x|

  Property = Struct.new(:name, :original_name)

  PROPERTIES = [
    Property.new("Clo",   "Chloe"     ),
    Property.new("Jon",   "Jonathan"  ),
    Property.new("Kris",  "Kristin"   ),
    Property.new("Dave",  "David"     ),
    Property.new("Ana",   "Anastasia" ),
    Property.new("Mike",  "Michael"   ),
    Property.new("Becky", "Rebecca"   ),
    Property.new("Will",  "William"   )
  ]

  x.report('#each') do |times|
    i = 0
    while i < times
      out = {}
      PROPERTIES.each do |property|
        out[property.name] = property.original_name
      end
      out
      i += 1
    end
  end
  
  x.report('#each_with_object') do |times|
    i = 0
    while i < times
      PROPERTIES.each_with_object({}) do |property, memo|
        memo[property.name] = property.original_name
      end
      i += 1
    end
  end

  x.report('#reduce') do |times|
    i = 0
    while i < times
      PROPERTIES.reduce({}) do |memo, property|
        memo[property.name] = property.original_name
        memo
      end
      i += 1
    end
  end

  x.report('Hash[map]') do |times|
    i = 0
    while i < times
      Hash[PROPERTIES.map { |property| [property.name, property.original_name] }]
      i += 1
    end
  end

  x.report('#map.zip(map).to_h') do |times|
    i = 0
    while i < times
      PROPERTIES.map(&:name).zip(PROPERTIES.map(&:original_name)).to_h
      i += 1
    end
  end

  x.report('#reduce-merge') do |times|
    i = 0
    while i < times
      PROPERTIES.reduce({}) { |memo, property| memo.merge(property.name => property.original_name) }
      i += 1
    end
  end

  x.compare!
end

Tests run on a 2012 15" MacBook Pro Retina running the latest OS El Capitan 10.11.1.

Ruby 2.3 seems a little faster than 2.2 in this benchmark 😃 ... would need to verify with a larger sample size 🔍

Interestingly, it’s almost the same for me regardless of whether the while i < times is used or not.

The above:

ruby 2.2.3p173 (2015-08-18 revision 51636) [x86_64-linux]
benchmark_building_a_hash.rb:25: warning: possibly useless use of a variable in void context
Calculating -------------------------------------
               #each    24.433k i/100ms
   #each_with_object    22.095k i/100ms
             #reduce    21.544k i/100ms
           Hash[map]    21.179k i/100ms
  #map.zip(map).to_h    16.915k i/100ms
       #reduce-merge     5.446k i/100ms
-------------------------------------------------
               #each    305.820k (± 1.0%) i/s -      1.539M
   #each_with_object    277.940k (± 1.2%) i/s -      1.392M
             #reduce    265.930k (± 0.9%) i/s -      1.336M
           Hash[map]    254.235k (± 0.8%) i/s -      1.292M
  #map.zip(map).to_h    204.984k (± 0.8%) i/s -      1.032M
       #reduce-merge     58.459k (± 1.2%) i/s -    294.084k

Comparison:
               #each:   305820.2 i/s
   #each_with_object:   277939.5 i/s - 1.10x slower
             #reduce:   265930.2 i/s - 1.15x slower
           Hash[map]:   254234.7 i/s - 1.20x slower
  #map.zip(map).to_h:   204984.0 i/s - 1.49x slower
       #reduce-merge:    58459.1 i/s - 5.23x slower

Above with the while i < times stripped:

ruby 2.2.3p173 (2015-08-18 revision 51636) [x86_64-linux]
Calculating -------------------------------------
               #each    23.752k i/100ms
   #each_with_object    21.417k i/100ms
             #reduce    20.849k i/100ms
           Hash[map]    20.301k i/100ms
  #map.zip(map).to_h    16.542k i/100ms
       #reduce-merge     5.348k i/100ms
-------------------------------------------------
               #each    297.305k (± 1.3%) i/s -      1.496M
   #each_with_object    262.538k (± 1.2%) i/s -      1.328M
             #reduce    255.574k (± 0.8%) i/s -      1.293M
           Hash[map]    244.184k (± 1.6%) i/s -      1.238M
  #map.zip(map).to_h    200.442k (± 1.6%) i/s -      1.009M
       #reduce-merge     57.136k (± 2.0%) i/s -    288.792k

Comparison:
               #each:   297304.5 i/s
   #each_with_object:   262537.7 i/s - 1.13x slower
             #reduce:   255573.9 i/s - 1.16x slower
           Hash[map]:   244184.5 i/s - 1.22x slower
  #map.zip(map).to_h:   200441.7 i/s - 1.48x slower
       #reduce-merge:    57135.9 i/s - 5.20x slower

As for Crystal 0.9.1 – script:

require "benchmark"

struct Property
  property name
  property original_name

  def initialize(@name, @original_name)
  end
end

PROPERTIES = [
  Property.new("Clo",   "Chloe"     ),
  Property.new("Jon",   "Jonathan"  ),
  Property.new("Kris",  "Kristin"   ),
  Property.new("Dave",  "David"     ),
  Property.new("Ana",   "Anastasia" ),
  Property.new("Mike",  "Michael"   ),
  Property.new("Becky", "Rebecca"   ),
  Property.new("Will",  "William"   )
]

Benchmark.ips do |x|
  x.report("#each") do
    out = {} of String => String
    PROPERTIES.each do |property|
      out[property.name] = property.original_name
    end
    out
  end

  x.report("#each_with_object") do
    PROPERTIES.each_with_object({} of String => String) do |property, memo|
      memo[property.name] = property.original_name
    end
  end

  x.report("#reduce") do
    PROPERTIES.inject({} of String => String) do |memo, property|
      memo[property.name] = property.original_name
      memo
    end
  end

  x.report("Hash[map]") do
    PROPERTIES.map { |property| [property.name, property.original_name] }.to_h
  end

  x.report("#map.zip(map).to_h") do
    PROPERTIES.map(&.name).zip(PROPERTIES.map(&.original_name)).to_h
  end

  x.report("#reduce-merge") do
    PROPERTIES.inject({} of String => String) { |memo, property| memo.merge({property.name => property.original_name}) }
  end
end

Results:

             #each   1.04M (±11.45%)       fastest
 #each_with_object 995.49k (± 9.21%)  1.05× slower
           #reduce 972.22k (±12.24%)  1.07× slower
         Hash[map] 617.63k (± 9.35%)  1.69× slower
#map.zip(map).to_h  790.9k (±11.80%)  1.32× slower
     #reduce-merge 159.27k (±10.03%)  6.56× slower

Yes, in this case the while block affects the absolute number of repetitions more than the relative results. As would be expected, while is faster. I don't know at what operation size it makes a difference in relative results.

Your machine is faster, what is it?

Really cool to see the Crystal version! Thanks!

One thing I haven't tried yet is disabling GC as shown in the benchmark-ips README...

Your machine is faster, what is it?

It’s an old-ish (bought in August 2012) Asus Zenbook with i7-3517U CPU @ 1.90GHz and 10 GB of RAM.

Surprised to see that "Hash[map]" is by far the fastest on JRuby 9k.

Calculating -------------------------------------
               #each    20.365k i/100ms
   #each_with_object    19.043k i/100ms
             #reduce    16.627k i/100ms
           Hash[map]    22.174k i/100ms
  #map.zip(map).to_h    14.661k i/100ms
       #reduce-merge     7.936k i/100ms
-------------------------------------------------
               #each    302.582k (± 1.5%) i/s -      1.527M
   #each_with_object    288.721k (± 2.5%) i/s -      1.447M
             #reduce    236.231k (± 3.5%) i/s -      1.181M
           Hash[map]    352.435k (± 2.7%) i/s -      1.774M
  #map.zip(map).to_h    215.814k (± 2.4%) i/s -      1.085M
       #reduce-merge     98.931k (± 5.2%) i/s -    499.968k

Comparison:
           Hash[map]:   352435.2 i/s
               #each:   302582.0 i/s - 1.16x slower
   #each_with_object:   288721.1 i/s - 1.22x slower
             #reduce:   236231.1 i/s - 1.49x slower
  #map.zip(map).to_h:   215814.3 i/s - 1.63x slower
       #reduce-merge:    98931.2 i/s - 3.56x slower

irb(main):153:0> JRUBY_VERSION
=> "9.0.4.0"
irb(main):154:0> ^D
$ java -version
openjdk version "1.8.0_66-internal"
OpenJDK Runtime Environment (build 1.8.0_66-internal-b17)
OpenJDK 64-Bit Server VM (build 25.66-b17, mixed mode)

Thanks for the JRuby version! Interesting! That's cool for Hash[map] because it's my favorite version along with the zip FP-style pipeline. Also interesting that in JRuby reduce seems slower than each_with_object, whereas in MRI they are close and in MRI 2.3.0 they seem to be exactly the same.