Skip to content

Instantly share code, notes, and snippets.

@eregon

eregon/original_bench.rb Secret

Last active March 9, 2023 11:42
Show Gist options
  • Save eregon/b65909bd1b9a7c45dc1e3c58df194e85 to your computer and use it in GitHub Desktop.
Save eregon/b65909bd1b9a7c45dc1e3c58df194e85 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
original_bench.rb
puts RUBY_DESCRIPTION
require 'benchmark/ips'
require 'ostruct'
class MyClass
attr_accessor :a, :b, :c
def initialize(a:, b:, c:)
@a = a
@b = b
@c = c
end
end
Benchmark.ips do |x|
result = 0
x.report("OpenStruct") do
obj = OpenStruct.new(a: 4, b: 7, c: 9)
# obj.a = 4
# obj.b = 7
# obj.c = 9
result = obj.a * obj.b * obj.c
end
x.report("MyClass") do # Named "Class" in the table below
obj = MyClass.new(a: 4, b: 7, c: 9)
result = obj.a * obj.b * obj.c
end
x.compare!
end
Raw
results.txt
$ ruby bench.rb
ruby 3.2.0 (2022-12-25 revision a528908271) [x86_64-linux]
Warming up --------------------------------------
OpenStruct 6.913k i/100ms
MyClass 209.528k i/100ms
Calculating -------------------------------------
OpenStruct 68.545k (± 0.9%) i/s - 345.650k in 5.043078s
MyClass 2.108M (± 0.6%) i/s - 10.686M in 5.068759s
Comparison:
MyClass: 2108275.1 i/s
OpenStruct: 68544.5 i/s - 30.76x slower
$ ruby -Ilib bench.rb
ruby 3.2.0 (2022-12-25 revision a528908271) [x86_64-linux]
Warming up --------------------------------------
OpenStruct 43.631k i/100ms
MyClass 201.481k i/100ms
Calculating -------------------------------------
OpenStruct 439.883k (± 0.5%) i/s - 2.225M in 5.058705s
MyClass 1.986M (± 0.7%) i/s - 10.074M in 5.072478s
Comparison:
MyClass: 1986114.8 i/s
OpenStruct: 439883.1 i/s - 4.52x slower
$ ruby --yjit bench.rb
ruby 3.2.0 (2022-12-25 revision a528908271) +YJIT [x86_64-linux]
Warming up --------------------------------------
OpenStruct 925.000 i/100ms
MyClass 243.249k i/100ms
Calculating -------------------------------------
OpenStruct 9.234k (± 0.6%) i/s - 46.250k in 5.008968s
MyClass 2.537M (± 0.9%) i/s - 12.892M in 5.081680s
Comparison:
MyClass: 2537179.8 i/s
OpenStruct: 9233.8 i/s - 274.77x slower
$ ruby --yjit -Ilib bench.rb
ruby 3.2.0 (2022-12-25 revision a528908271) +YJIT [x86_64-linux]
Warming up --------------------------------------
OpenStruct 56.306k i/100ms
MyClass 238.023k i/100ms
Calculating -------------------------------------
OpenStruct 559.264k (± 0.7%) i/s - 2.815M in 5.034156s
MyClass 2.535M (± 0.8%) i/s - 12.853M in 5.071140s
Comparison:
MyClass: 2534753.2 i/s
OpenStruct: 559263.7 i/s - 4.53x slower
$ ruby -Xcompile.invokedynamic=true bench.rb
jruby 9.4.0.0 (3.1.0) 2022-11-23 95c0ec159f OpenJDK 64-Bit Server VM 17.0.6+10 on 17.0.6+10 +indy +jit [x86_64-linux]
Warming up --------------------------------------
OpenStruct 112.000 i/100ms
MyClass 216.053k i/100ms
Calculating -------------------------------------
OpenStruct 168.210k (± 9.2%) i/s - 772.912k in 4.977303s
MyClass 3.377M (±22.2%) i/s - 15.772M in 5.002839s
Comparison:
MyClass: 3377268.5 i/s
OpenStruct: 168209.6 i/s - 20.08x slower
$ ruby -Xcompile.invokedynamic=true -Ilib bench.rb
jruby 9.4.0.0 (3.1.0) 2022-11-23 95c0ec159f OpenJDK 64-Bit Server VM 17.0.6+10 on 17.0.6+10 +indy +jit [x86_64-linux]
Warming up --------------------------------------
OpenStruct 123.572k i/100ms
MyClass 208.632k i/100ms
Calculating -------------------------------------
OpenStruct 1.494M (±13.6%) i/s - 7.414M in 5.070798s
MyClass 2.957M (±18.8%) i/s - 14.187M in 5.032483s
Comparison:
MyClass: 2956595.9 i/s
OpenStruct: 1494112.9 i/s - 1.98x slower
$ ruby bench.rb
truffleruby 23.0.0-dev-82f5ec31, like ruby 3.1.3, GraalVM CE Native [x86_64-linux]
Warming up --------------------------------------
OpenStruct 1.356k i/100ms
MyClass 298.697M i/100ms
Calculating -------------------------------------
OpenStruct 13.695k (± 7.7%) i/s - 69.156k in 5.082275s
MyClass 2.985B (± 0.5%) i/s - 14.935B in 5.003263s
Comparison:
MyClass: 2985117239.3 i/s
OpenStruct: 13695.3 i/s - 217966.88x slower
$ ruby -Ilib bench.rb
truffleruby 23.0.0-dev-82f5ec31, like ruby 3.1.3, GraalVM CE Native [x86_64-linux]
Warming up --------------------------------------
OpenStruct 299.271M i/100ms
MyClass 299.370M i/100ms
Calculating -------------------------------------
OpenStruct 2.990B (± 0.4%) i/s - 14.964B in 5.005294s
MyClass 2.989B (± 0.3%) i/s - 14.969B in 5.007672s
Comparison:
OpenStruct: 2989578258.9 i/s
MyClass: 2989142028.0 i/s - same-ish: difference falls within error
@eregon
Copy link
Author

eregon commented Mar 9, 2023 •
edited
Loading

A variant which forces the allocation:

puts RUBY_DESCRIPTION
require 'benchmark/ips'
require 'ostruct'

class MyClass
  attr_accessor :a, :b, :c
  
  def initialize(a:, b:, c:)
    @a = a
    @b = b
    @c = c
  end
end

Benchmark.ips do |x|
  obj = nil
  result = 0
  
  x.report("OpenStruct") do
    obj = OpenStruct.new(a: 4, b: 7, c: 9)
    # obj.a = 4
    # obj.b = 7
    # obj.c = 9
    result = obj.a * obj.b * obj.c
  end
  
  x.report("MyClass") do
    obj = MyClass.new(a: 4, b: 7, c: 9)
    result = obj.a * obj.b * obj.c
  end

  x.compare!
end

Almost the same results on CRuby (expected), but of course quite different on TruffleRuby which can optimize away unused allocations in the previous benchmark.

$ ruby bench.rb                              
truffleruby 23.0.0-dev-82f5ec31, like ruby 3.1.3, GraalVM CE Native [x86_64-linux]
Warming up --------------------------------------
          OpenStruct     1.445k i/100ms
             MyClass    10.069M i/100ms
Calculating -------------------------------------
          OpenStruct     14.629k (±13.4%) i/s -     72.250k in   5.045561s
             MyClass    106.094M (± 7.5%) i/s -    533.650M in   5.060996s

Comparison:
             MyClass: 106094077.4 i/s
          OpenStruct:    14629.3 i/s - 7252.16x  slower

$ ruby -Ilib bench.rb
truffleruby 23.0.0-dev-82f5ec31, like ruby 3.1.3, GraalVM CE Native [x86_64-linux]
Warming up --------------------------------------
          OpenStruct     2.597M i/100ms
             MyClass    10.720M i/100ms
Calculating -------------------------------------
          OpenStruct     30.141M (± 5.8%) i/s -    150.642M in   5.015945s
             MyClass    104.424M (± 7.3%) i/s -    525.279M in   5.059661s

Comparison:
             MyClass: 104423737.4 i/s
          OpenStruct: 30140917.1 i/s - 3.46x  slower



$ ruby bench.rb && ruby -Ilib bench.rb
ruby 3.2.0 (2022-12-25 revision a528908271) [x86_64-linux]
Warming up --------------------------------------
          OpenStruct     6.871k i/100ms
             MyClass   209.677k i/100ms
Calculating -------------------------------------
          OpenStruct     68.471k (± 0.9%) i/s -    343.550k in   5.017818s
             MyClass      2.079M (± 0.7%) i/s -     10.484M in   5.042548s

Comparison:
             MyClass:  2079167.1 i/s
          OpenStruct:    68471.1 i/s - 30.37x  slower

ruby 3.2.0 (2022-12-25 revision a528908271) [x86_64-linux]
Warming up --------------------------------------
          OpenStruct    44.009k i/100ms
             MyClass   201.714k i/100ms
Calculating -------------------------------------
          OpenStruct    438.532k (± 0.7%) i/s -      2.200M in   5.017991s
             MyClass      2.008M (± 0.6%) i/s -     10.086M in   5.023298s

Comparison:
             MyClass:  2007867.6 i/s
          OpenStruct:   438531.8 i/s - 4.58x  slower




$ ruby --yjit bench.rb && ruby --yjit -Ilib bench.rb
ruby 3.2.0 (2022-12-25 revision a528908271) +YJIT [x86_64-linux]
Warming up --------------------------------------
          OpenStruct   928.000  i/100ms
             MyClass   243.915k i/100ms
Calculating -------------------------------------
          OpenStruct      9.242k (± 0.5%) i/s -     46.400k in   5.020753s
             MyClass      2.582M (± 0.4%) i/s -     12.927M in   5.007140s

Comparison:
             MyClass:  2581846.5 i/s
          OpenStruct:     9241.9 i/s - 279.36x  slower

ruby 3.2.0 (2022-12-25 revision a528908271) +YJIT [x86_64-linux]
Warming up --------------------------------------
          OpenStruct    55.027k i/100ms
             MyClass   235.175k i/100ms
Calculating -------------------------------------
          OpenStruct    563.449k (± 0.9%) i/s -      2.861M in   5.078763s
             MyClass      2.543M (± 0.5%) i/s -     12.935M in   5.086740s

Comparison:
             MyClass:  2542869.1 i/s
          OpenStruct:   563448.7 i/s - 4.51x  slower



$ ruby -Xcompile.invokedynamic=true bench.rb && ruby -Xcompile.invokedynamic=true -Ilib bench.rb
jruby 9.4.0.0 (3.1.0) 2022-11-23 95c0ec159f OpenJDK 64-Bit Server VM 17.0.6+10 on 17.0.6+10 +indy +jit [x86_64-linux]
Warming up --------------------------------------
          OpenStruct   122.000  i/100ms
             MyClass   203.771k i/100ms
Calculating -------------------------------------
          OpenStruct    167.715k (± 9.4%) i/s -    774.700k in   4.979204s
             MyClass      3.018M (±19.3%) i/s -     14.468M in   5.031060s

Comparison:
             MyClass:  3018112.6 i/s
          OpenStruct:   167715.5 i/s - 18.00x  slower

jruby 9.4.0.0 (3.1.0) 2022-11-23 95c0ec159f OpenJDK 64-Bit Server VM 17.0.6+10 on 17.0.6+10 +indy +jit [x86_64-linux]
Warming up --------------------------------------
          OpenStruct   126.238k i/100ms
             MyClass   205.492k i/100ms
Calculating -------------------------------------
          OpenStruct      1.488M (±13.6%) i/s -      7.322M in   5.028592s
             MyClass      2.912M (±19.4%) i/s -     13.973M in   5.033932s

Comparison:
             MyClass:  2911820.8 i/s
          OpenStruct:  1488139.5 i/s - 1.96x  slower

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment