Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Benchmark building a Ruby hash: #each - #each_with_object - #reduce - Hash[map] - #map.zip(map).to_h - #reduce-merge
require 'benchmark/ips'
Benchmark.ips do |x|
Property = Struct.new(:name, :original_name)
PROPERTIES = [
Property.new("Clo", "Chloe" ),
Property.new("Jon", "Jonathan" ),
Property.new("Kris", "Kristin" ),
Property.new("Dave", "David" ),
Property.new("Ana", "Anastasia" ),
Property.new("Mike", "Michael" ),
Property.new("Becky", "Rebecca" ),
Property.new("Will", "William" )
]
x.report('#each') do |times|
i = 0
while i < times
out = {}
PROPERTIES.each do |property|
out[property.name] = property.original_name
end
out
i += 1
end
end
x.report('#each_with_object') do |times|
i = 0
while i < times
PROPERTIES.each_with_object({}) do |property, memo|
memo[property.name] = property.original_name
end
i += 1
end
end
x.report('#reduce') do |times|
i = 0
while i < times
PROPERTIES.reduce({}) do |memo, property|
memo[property.name] = property.original_name
memo
end
i += 1
end
end
x.report('Hash[map]') do |times|
i = 0
while i < times
Hash[PROPERTIES.map { |property| [property.name, property.original_name] }]
i += 1
end
end
x.report('#map.zip(map).to_h') do |times|
i = 0
while i < times
PROPERTIES.map(&:name).zip(PROPERTIES.map(&:original_name)).to_h
i += 1
end
end
x.report('#reduce-merge') do |times|
i = 0
while i < times
PROPERTIES.reduce({}) { |memo, property| memo.merge(property.name => property.original_name) }
i += 1
end
end
x.compare!
end
@jonatack

This comment has been minimized.

Copy link
Owner Author

commented Nov 28, 2015

MRI Ruby 2.3.0-preview1:

Comparison:
               #each:   263279.5 i/s
             #reduce:   247168.3 i/s - 1.07x slower
   #each_with_object:   246478.6 i/s - 1.07x slower
          #Hash[map]:   239345.3 i/s - 1.10x slower
  #map.zip(map).to_h:   175519.7 i/s - 1.50x slower
       #reduce-merge:    45550.2 i/s - 5.78x slower
@jonatack

This comment has been minimized.

Copy link
Owner Author

commented Nov 28, 2015

MRI Ruby 2.2.3p173:

Comparison:
               #each:   260677.6 i/s
   #each_with_object:   242110.6 i/s - 1.08x slower
             #reduce:   233343.8 i/s - 1.12x slower
           Hash[map]:   217035.6 i/s - 1.20x slower
  #map.zip(map).to_h:   169006.4 i/s - 1.54x slower
       #reduce-merge:    49080.2 i/s - 5.31x slower
@jonatack

This comment has been minimized.

Copy link
Owner Author

commented Nov 28, 2015

Tests run on a 2012 15" MacBook Pro Retina running the latest OS El Capitan 10.11.1.

Ruby 2.3 seems a little faster than 2.2 in this benchmark 😃 ... would need to verify with a larger sample size 🔍

@chastell

This comment has been minimized.

Copy link

commented Nov 28, 2015

Interestingly, it’s almost the same for me regardless of whether the while i < times is used or not.

The above:

ruby 2.2.3p173 (2015-08-18 revision 51636) [x86_64-linux]
benchmark_building_a_hash.rb:25: warning: possibly useless use of a variable in void context
Calculating -------------------------------------
               #each    24.433k i/100ms
   #each_with_object    22.095k i/100ms
             #reduce    21.544k i/100ms
           Hash[map]    21.179k i/100ms
  #map.zip(map).to_h    16.915k i/100ms
       #reduce-merge     5.446k i/100ms
-------------------------------------------------
               #each    305.820k (± 1.0%) i/s -      1.539M
   #each_with_object    277.940k (± 1.2%) i/s -      1.392M
             #reduce    265.930k (± 0.9%) i/s -      1.336M
           Hash[map]    254.235k (± 0.8%) i/s -      1.292M
  #map.zip(map).to_h    204.984k (± 0.8%) i/s -      1.032M
       #reduce-merge     58.459k (± 1.2%) i/s -    294.084k

Comparison:
               #each:   305820.2 i/s
   #each_with_object:   277939.5 i/s - 1.10x slower
             #reduce:   265930.2 i/s - 1.15x slower
           Hash[map]:   254234.7 i/s - 1.20x slower
  #map.zip(map).to_h:   204984.0 i/s - 1.49x slower
       #reduce-merge:    58459.1 i/s - 5.23x slower

Above with the while i < times stripped:

ruby 2.2.3p173 (2015-08-18 revision 51636) [x86_64-linux]
Calculating -------------------------------------
               #each    23.752k i/100ms
   #each_with_object    21.417k i/100ms
             #reduce    20.849k i/100ms
           Hash[map]    20.301k i/100ms
  #map.zip(map).to_h    16.542k i/100ms
       #reduce-merge     5.348k i/100ms
-------------------------------------------------
               #each    297.305k (± 1.3%) i/s -      1.496M
   #each_with_object    262.538k (± 1.2%) i/s -      1.328M
             #reduce    255.574k (± 0.8%) i/s -      1.293M
           Hash[map]    244.184k (± 1.6%) i/s -      1.238M
  #map.zip(map).to_h    200.442k (± 1.6%) i/s -      1.009M
       #reduce-merge     57.136k (± 2.0%) i/s -    288.792k

Comparison:
               #each:   297304.5 i/s
   #each_with_object:   262537.7 i/s - 1.13x slower
             #reduce:   255573.9 i/s - 1.16x slower
           Hash[map]:   244184.5 i/s - 1.22x slower
  #map.zip(map).to_h:   200441.7 i/s - 1.48x slower
       #reduce-merge:    57135.9 i/s - 5.20x slower
@chastell

This comment has been minimized.

Copy link

commented Nov 28, 2015

As for Crystal 0.9.1 – script:

require "benchmark"

struct Property
  property name
  property original_name

  def initialize(@name, @original_name)
  end
end

PROPERTIES = [
  Property.new("Clo",   "Chloe"     ),
  Property.new("Jon",   "Jonathan"  ),
  Property.new("Kris",  "Kristin"   ),
  Property.new("Dave",  "David"     ),
  Property.new("Ana",   "Anastasia" ),
  Property.new("Mike",  "Michael"   ),
  Property.new("Becky", "Rebecca"   ),
  Property.new("Will",  "William"   )
]

Benchmark.ips do |x|
  x.report("#each") do
    out = {} of String => String
    PROPERTIES.each do |property|
      out[property.name] = property.original_name
    end
    out
  end

  x.report("#each_with_object") do
    PROPERTIES.each_with_object({} of String => String) do |property, memo|
      memo[property.name] = property.original_name
    end
  end

  x.report("#reduce") do
    PROPERTIES.inject({} of String => String) do |memo, property|
      memo[property.name] = property.original_name
      memo
    end
  end

  x.report("Hash[map]") do
    PROPERTIES.map { |property| [property.name, property.original_name] }.to_h
  end

  x.report("#map.zip(map).to_h") do
    PROPERTIES.map(&.name).zip(PROPERTIES.map(&.original_name)).to_h
  end

  x.report("#reduce-merge") do
    PROPERTIES.inject({} of String => String) { |memo, property| memo.merge({property.name => property.original_name}) }
  end
end

Results:

             #each   1.04M (±11.45%)       fastest
 #each_with_object 995.49k (± 9.21%)  1.05× slower
           #reduce 972.22k (±12.24%)  1.07× slower
         Hash[map] 617.63k (± 9.35%)  1.69× slower
#map.zip(map).to_h  790.9k (±11.80%)  1.32× slower
     #reduce-merge 159.27k (±10.03%)  6.56× slower
@jonatack

This comment has been minimized.

Copy link
Owner Author

commented Nov 28, 2015

Yes, in this case the while block affects the absolute number of repetitions more than the relative results. As would be expected, while is faster. I don't know at what operation size it makes a difference in relative results.

Your machine is faster, what is it?

Really cool to see the Crystal version! Thanks!

@jonatack

This comment has been minimized.

Copy link
Owner Author

commented Nov 28, 2015

One thing I haven't tried yet is disabling GC as shown in the benchmark-ips README...

@chastell

This comment has been minimized.

Copy link

commented Nov 28, 2015

Your machine is faster, what is it?

It’s an old-ish (bought in August 2012) Asus Zenbook with i7-3517U CPU @ 1.90GHz and 10 GB of RAM.

@jstepien

This comment has been minimized.

Copy link

commented Nov 28, 2015

Surprised to see that "Hash[map]" is by far the fastest on JRuby 9k.

Calculating -------------------------------------
               #each    20.365k i/100ms
   #each_with_object    19.043k i/100ms
             #reduce    16.627k i/100ms
           Hash[map]    22.174k i/100ms
  #map.zip(map).to_h    14.661k i/100ms
       #reduce-merge     7.936k i/100ms
-------------------------------------------------
               #each    302.582k (± 1.5%) i/s -      1.527M
   #each_with_object    288.721k (± 2.5%) i/s -      1.447M
             #reduce    236.231k (± 3.5%) i/s -      1.181M
           Hash[map]    352.435k (± 2.7%) i/s -      1.774M
  #map.zip(map).to_h    215.814k (± 2.4%) i/s -      1.085M
       #reduce-merge     98.931k (± 5.2%) i/s -    499.968k

Comparison:
           Hash[map]:   352435.2 i/s
               #each:   302582.0 i/s - 1.16x slower
   #each_with_object:   288721.1 i/s - 1.22x slower
             #reduce:   236231.1 i/s - 1.49x slower
  #map.zip(map).to_h:   215814.3 i/s - 1.63x slower
       #reduce-merge:    98931.2 i/s - 3.56x slower

irb(main):153:0> JRUBY_VERSION
=> "9.0.4.0"
irb(main):154:0> ^D
$ java -version
openjdk version "1.8.0_66-internal"
OpenJDK Runtime Environment (build 1.8.0_66-internal-b17)
OpenJDK 64-Bit Server VM (build 25.66-b17, mixed mode)
@jonatack

This comment has been minimized.

Copy link
Owner Author

commented Nov 28, 2015

Thanks for the JRuby version! Interesting! That's cool for Hash[map] because it's my favorite version along with the zip FP-style pipeline. Also interesting that in JRuby reduce seems slower than each_with_object, whereas in MRI they are close and in MRI 2.3.0 they seem to be exactly the same.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.