Skip to content

Instantly share code, notes, and snippets.

@PragTob
Last active Oct 23, 2015
Embed
What would you like to do?
define_method meta benchmark
require 'benchmark/ips'
class FakeDimension
def initialize(margin_start)
@margin_start = margin_start
@margin_start_relative = relative? @margin_start
end
def relative?(result)
result.is_a?(Float) && result <= 1
end
def calculate_relative(result)
(result * 100).to_i
end
define_method :full_meta do
instance_variable_name = '@' + :margin_start.to_s
value = instance_variable_get(instance_variable_name)
value = calculate_relative value if relative? value
value
end
IVAR_NAME = "@margin_start"
define_method :hoist_ivar_name do
value = instance_variable_get(IVAR_NAME)
value = calculate_relative value if relative? value
value
end
define_method :direct_ivar do
value = @margin_start
value = calculate_relative value if relative? value
value
end
eval <<-CODE
def full_string
value = @margin_start
value = calculate_relative value if relative? value
value
end
CODE
def full_direct
value = @margin_start
value = calculate_relative value if relative? value
value
end
end
def do_benchmark(description, margin_start)
puts description
Benchmark.ips do |benchmark|
dim = FakeDimension.new margin_start
benchmark.report("full_meta") { dim.full_meta }
benchmark.report("hoist_ivar_name") { dim.hoist_ivar_name }
benchmark.report("direct_ivar") { dim.direct_ivar }
benchmark.report("full_string") { dim.full_string }
benchmark.report("full_direct") { dim.full_direct }
benchmark.compare!
end
end
do_benchmark('Non relative margin start', 10)
do_benchmark('Relative margin start', 0.8)
@PragTob

This comment has been minimized.

Copy link
Owner Author

@PragTob PragTob commented Jul 20, 2015

Results on my machine with 2.2.2:

tobi@airship ~/Desktop $ ruby -v
ruby 2.2.2p95 (2015-04-13 revision 50295) [x86_64-linux]
tobi@airship ~/Desktop $ ruby meta_benchmark.rb 
Non relative margin start
Calculating -------------------------------------
           full_meta    98.331k i/100ms
     hoist_ivar_name   132.884k i/100ms
         direct_ivar   166.998k i/100ms
         full_string   172.453k i/100ms
         full_direct   162.880k i/100ms
-------------------------------------------------
           full_meta      1.840M (± 3.0%) i/s -      9.243M
     hoist_ivar_name      3.147M (± 3.3%) i/s -     15.813M
         direct_ivar      5.288M (± 3.1%) i/s -     26.553M
         full_string      6.034M (± 3.2%) i/s -     30.179M
         full_direct      5.955M (± 3.2%) i/s -     29.807M

Comparison:
         full_string:  6033829.1 i/s
         full_direct:  5954626.6 i/s - 1.01x slower
         direct_ivar:  5288105.5 i/s - 1.14x slower
     hoist_ivar_name:  3146595.7 i/s - 1.92x slower
           full_meta:  1840087.6 i/s - 3.28x slower

Relative margin start
Calculating -------------------------------------
           full_meta    88.643k i/100ms
     hoist_ivar_name   116.964k i/100ms
         direct_ivar   140.267k i/100ms
         full_string   149.543k i/100ms
         full_direct   142.721k i/100ms
-------------------------------------------------
           full_meta      1.547M (± 3.2%) i/s -      7.801M
     hoist_ivar_name      2.357M (± 3.4%) i/s -     11.813M
         direct_ivar      3.488M (± 3.5%) i/s -     17.533M
         full_string      3.836M (± 2.7%) i/s -     19.291M
         full_direct      3.817M (± 3.3%) i/s -     19.125M

Comparison:
         full_string:  3835805.7 i/s
         full_direct:  3817304.9 i/s - 1.00x slower
         direct_ivar:  3488379.0 i/s - 1.10x slower
     hoist_ivar_name:  2357030.1 i/s - 1.63x slower
           full_meta:  1546614.4 i/s - 2.48x slower

And with jruby-1.7.21:

tobi@airship ~/Desktop $ ruby -v
jruby 1.7.21 (1.9.3p551) 2015-07-07 a741a82 on OpenJDK 64-Bit Server VM 1.7.0_79-b14 +jit [linux-amd64]
tobi@airship ~/Desktop $ ruby meta_benchmark.rb 
Non relative margin start
Calculating -------------------------------------
           full_meta    93.360k i/100ms
     hoist_ivar_name   151.934k i/100ms
         direct_ivar   253.792k i/100ms
         full_string   307.861k i/100ms
         full_direct   315.816k i/100ms
-------------------------------------------------
           full_meta      2.158M (± 5.6%) i/s -     10.830M
     hoist_ivar_name      2.514M (± 7.4%) i/s -     12.611M
         direct_ivar      6.856M (± 4.5%) i/s -     34.262M
         full_string     12.508M (± 4.8%) i/s -     62.496M
         full_direct     12.917M (± 4.2%) i/s -     64.426M

Comparison:
         full_direct: 12916681.2 i/s
         full_string: 12508434.5 i/s - 1.03x slower
         direct_ivar:  6856039.9 i/s - 1.88x slower
     hoist_ivar_name:  2514484.8 i/s - 5.14x slower
           full_meta:  2158254.0 i/s - 5.98x slower

Relative margin start
Calculating -------------------------------------
           full_meta   114.869k i/100ms
     hoist_ivar_name   138.199k i/100ms
         direct_ivar   228.379k i/100ms
         full_string   267.545k i/100ms
         full_direct   267.230k i/100ms
-------------------------------------------------
           full_meta      1.992M (± 4.7%) i/s -      9.994M
     hoist_ivar_name      2.332M (± 3.9%) i/s -     11.747M
         direct_ivar      5.518M (± 4.2%) i/s -     27.634M
         full_string      9.068M (± 5.1%) i/s -     45.215M
         full_direct      9.219M (± 5.4%) i/s -     45.964M

Comparison:
         full_direct:  9218901.8 i/s
         full_string:  9067592.8 i/s - 1.02x slower
         direct_ivar:  5517978.3 i/s - 1.67x slower
     hoist_ivar_name:  2332477.7 i/s - 3.95x slower
           full_meta:  1992448.9 i/s - 4.63x slower
@PragTob

This comment has been minimized.

Copy link
Owner Author

@PragTob PragTob commented Oct 23, 2015

For completeness, jruby 9.0.0.0:

tobi@airship ~/Desktop $ ruby meta_benchmark.rb 
Non relative margin start
Calculating -------------------------------------
           full_meta    87.548k i/100ms
     hoist_ivar_name   130.467k i/100ms
         direct_ivar   156.668k i/100ms
         full_string   171.480k i/100ms
         full_direct   171.436k i/100ms
-------------------------------------------------
           full_meta      2.579M (± 6.3%) i/s -     12.870M
     hoist_ivar_name      3.231M (± 8.2%) i/s -     16.047M
         direct_ivar      5.560M (± 6.6%) i/s -     27.730M
         full_string      7.027M (± 5.0%) i/s -     34.982M
         full_direct      7.034M (±11.9%) i/s -     34.459M

Comparison:
         full_direct:  7034295.3 i/s
         full_string:  7027336.0 i/s - 1.00x slower
         direct_ivar:  5559686.4 i/s - 1.27x slower
     hoist_ivar_name:  3231384.6 i/s - 2.18x slower
           full_meta:  2579060.4 i/s - 2.73x slower

Relative margin start
Calculating -------------------------------------
           full_meta   105.522k i/100ms
     hoist_ivar_name   120.094k i/100ms
         direct_ivar   152.049k i/100ms
         full_string   162.297k i/100ms
         full_direct   159.890k i/100ms
-------------------------------------------------
           full_meta      2.316M (± 5.6%) i/s -     11.607M
     hoist_ivar_name      2.942M (± 3.7%) i/s -     14.772M
         direct_ivar      4.549M (± 6.6%) i/s -     22.655M
         full_string      5.664M (± 2.8%) i/s -     28.240M
         full_direct      5.666M (± 3.7%) i/s -     28.301M

Comparison:
         full_direct:  5665954.8 i/s
         full_string:  5663913.6 i/s - 1.00x slower
         direct_ivar:  4549031.3 i/s - 1.25x slower
     hoist_ivar_name:  2942342.7 i/s - 1.93x slower
           full_meta:  2315864.0 i/s - 2.45x slower
@PragTob

This comment has been minimized.

Copy link
Owner Author

@PragTob PragTob commented Oct 23, 2015

And the newly released ruby 9.0.3.0 with some related improvements (7-15% better for some cases and almost the performance as the eval for direct_ivar now):

tobi@airship ~/Desktop $ ruby meta_benchmark.rb 
Non relative margin start
Calculating -------------------------------------
           full_meta    87.493k i/100ms
     hoist_ivar_name   135.651k i/100ms
         direct_ivar   163.963k i/100ms
         full_string   165.898k i/100ms
         full_direct   165.800k i/100ms
-------------------------------------------------
           full_meta      2.754M (± 4.2%) i/s -     13.736M
     hoist_ivar_name      3.682M (± 4.1%) i/s -     18.449M
         direct_ivar      6.953M (± 8.7%) i/s -     34.268M
         full_string      7.145M (± 4.4%) i/s -     35.668M
         full_direct      7.165M (± 4.1%) i/s -     35.813M

Comparison:
         full_direct:  7164776.1 i/s
         full_string:  7144531.3 i/s - 1.00x slower
         direct_ivar:  6952823.3 i/s - 1.03x slower
     hoist_ivar_name:  3682185.7 i/s - 1.95x slower
           full_meta:  2753789.7 i/s - 2.60x slower

Relative margin start
Calculating -------------------------------------
           full_meta   105.745k i/100ms
     hoist_ivar_name   122.527k i/100ms
         direct_ivar   148.861k i/100ms
         full_string   145.214k i/100ms
         full_direct   150.985k i/100ms
-------------------------------------------------
           full_meta      2.367M (± 7.4%) i/s -     11.843M
     hoist_ivar_name      2.882M (± 4.7%) i/s -     14.458M
         direct_ivar      5.317M (± 4.1%) i/s -     26.497M
         full_string      5.335M (±11.2%) i/s -     26.139M
         full_direct      5.490M (± 4.8%) i/s -     27.479M

Comparison:
         full_direct:  5489869.4 i/s
         full_string:  5335464.0 i/s - 1.03x slower
         direct_ivar:  5317361.0 i/s - 1.03x slower
     hoist_ivar_name:  2881805.3 i/s - 1.91x slower
           full_meta:  2367465.1 i/s - 2.32x slower
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment