Skip to content

Instantly share code, notes, and snippets.

@ChrisLundquist

ChrisLundquist/results.txt

Last active December 9, 2015 23:31
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save ChrisLundquist/d06b6cdf1c208d2e7f18 to your computer and use it in GitHub Desktop.
Save ChrisLundquist/d06b6cdf1c208d2e7f18 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
results.txt
$ ruby -v
ruby 2.2.3p173 (2015-08-18 revision 51636) [x86_64-darwin14]
Rehearsal --------------------------------------------------------------
map_compact 0.020000 0.000000 0.020000 ( 0.015315)
map_inplace_compact 0.010000 0.000000 0.010000 ( 0.016069)
reject_map 0.010000 0.000000 0.010000 ( 0.014038)
reject_inplace_map 0.860000 0.010000 0.870000 ( 0.860999)
reject_map_inplace 0.010000 0.000000 0.010000 ( 0.013710)
reject_inplace_map_inplace 0.860000 0.000000 0.860000 ( 0.866945)
----------------------------------------------------- total: 1.780000sec
user system total real
map_compact 0.010000 0.000000 0.010000 ( 0.012554)
map_inplace_compact 0.010000 0.000000 0.010000 ( 0.012114)
reject_map 0.010000 0.000000 0.010000 ( 0.021661)
reject_inplace_map 0.850000 0.000000 0.850000 ( 0.855091)
reject_map_inplace 0.010000 0.000000 0.010000 ( 0.015349)
reject_inplace_map_inplace 0.870000 0.000000 0.870000 ( 0.876531)
Raw
test.rb
#!/usr/bin/env ruby
require 'benchmark'
def data
(0...100_000).to_a
end
def map_compact
data.map do |i|
next if i.odd?
i + i
end.compact
end
def map_inplace_compact
data.map! do |i|
next if i.odd?
i + i
end.compact
end
def reject_map
data.reject { |i| i.odd? }.map { |i| i + i }
end
def reject_inplace_map
data.reject! { |i| i.odd? }.map { |i| i + i }
end
def reject_inplace_map_inplace
data.reject! { |i| i.odd? }.map! { |i| i + i }
end
def reject_map_inplace
data.reject { |i| i.odd? }.map! { |i| i + i }
end
Benchmark.bmbm do |x|
x.report("map_compact") { map_compact }
x.report("map_inplace_compact") { map_inplace_compact }
x.report("reject_map") { reject_map }
x.report("reject_inplace_map") { reject_inplace_map }
x.report("reject_map_inplace") { reject_map_inplace }
x.report("reject_inplace_map_inplace") { reject_inplace_map_inplace }
end
@tenderlove
Copy link

I think there may be too much noise in this test (from GC or other things).

require 'benchmark/ips'

def data
  (0...100_000).to_a
end

def map_compact
  data.map do |i|
    next if i.odd?
    i + i
  end.compact
end

def map_inplace_compact
  data.map! do |i|
    next if i.odd?
    i + i
  end.compact
end

def reject_map
  data.reject { |i| i.odd? }.map { |i| i + i }
end

def reject_inplace_map
  data.reject! { |i| i.odd? }.map { |i| i + i }
end

def reject_inplace_map_inplace
  data.reject! { |i| i.odd? }.map! { |i| i + i }
end

def reject_map_inplace
  data.reject { |i| i.odd? }.map! { |i| i + i }
end

Benchmark.ips do |x|
   x.report("map_compact") { map_compact }
   x.report("map_inplace_compact")  { map_inplace_compact }
   x.report("reject_map") { reject_map }
   x.report("reject_inplace_map") { reject_inplace_map }
   x.report("reject_map_inplace") { reject_map_inplace }
   x.report("reject_inplace_map_inplace") { reject_inplace_map_inplace }
end
Calculating -------------------------------------
         map_compact     2.000  i/100ms
 map_inplace_compact     2.000  i/100ms
          reject_map     1.000  i/100ms
  reject_inplace_map     2.000  i/100ms
  reject_map_inplace     2.000  i/100ms
reject_inplace_map_inplace
                         2.000  i/100ms
-------------------------------------------------
         map_compact     22.854  (± 4.4%) i/s -    114.000 
 map_inplace_compact     23.518  (± 4.3%) i/s -    118.000 
          reject_map     20.189  (± 5.0%) i/s -    101.000 
  reject_inplace_map     20.315  (± 4.9%) i/s -    102.000 
  reject_map_inplace     20.508  (± 4.9%) i/s -    104.000 
reject_inplace_map_inplace
                         20.585  (± 4.9%) i/s -    104.000

They all seem approximately the same in this benchmark.

@ChrisLundquist
Copy link
Author 

Calculating -------------------------------------
         map_compact     7.000  i/100ms
 map_inplace_compact     7.000  i/100ms
          reject_map     6.000  i/100ms
  reject_inplace_map     1.000  i/100ms
  reject_map_inplace     6.000  i/100ms
reject_inplace_map_inplace
                         1.000  i/100ms
-------------------------------------------------
         map_compact     72.187  (± 2.8%) i/s -    364.000 
 map_inplace_compact     76.262  (± 3.9%) i/s -    385.000 
          reject_map     64.845  (± 3.1%) i/s -    324.000 
  reject_inplace_map      1.081  (± 0.0%) i/s -      6.000  in   5.549066s
  reject_map_inplace     65.687  (± 3.0%) i/s -    330.000 
reject_inplace_map_inplace
                          1.091  (± 0.0%) i/s -      6.000  in   5.498228s

Maybe there is something wrong with my install?

@ChrisLundquist
Copy link
Author

Hmm, I reinstalled and see the same strange performance:

$ rbenv install 2.2.3
rbenv: /Users/clundquist/.rbenv/versions/2.2.3 already exists
continue with installation? (y/N) y
Downloading ruby-2.2.3.tar.gz...
-> https://dqw8nmjcqpjn7.cloudfront.net/df795f2f99860745a416092a4004b016ccf77e8b82dec956b120f18bdc71edce
Installing ruby-2.2.3...
Installed ruby-2.2.3 to /Users/clundquist/.rbenv/versions/2.2.3

Running ctags on /Users/clundquist/.rbenv/versions/2.2.3/lib/ruby/2.2.0
Running ctags on /Users/clundquist/.rbenv/versions/2.2.3/lib/ruby/site_ruby/2.2.0
Running ctags on /Users/clundquist/.rbenv/versions/2.2.3/lib/ruby/vendor_ruby/2.2.0
unitys-MacBook-Pro-6:go-cache clundquist$ ruby -v
ruby 2.2.3p173 (2015-08-18 revision 51636) [x86_64-darwin15]

$ gcc -v
Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 7.0.0 (clang-700.1.76)
Target: x86_64-apple-darwin15.0.0
Thread model: posix

Calculating -------------------------------------
         map_compact     7.000  i/100ms
 map_inplace_compact     7.000  i/100ms
          reject_map     6.000  i/100ms
  reject_inplace_map     1.000  i/100ms
  reject_map_inplace     6.000  i/100ms
reject_inplace_map_inplace
                         1.000  i/100ms
-------------------------------------------------
         map_compact     72.082  (± 2.8%) i/s -    364.000 
 map_inplace_compact     76.686  (± 2.6%) i/s -    385.000 
          reject_map     64.012  (± 3.1%) i/s -    324.000 
  reject_inplace_map      1.086  (± 0.0%) i/s -      6.000  in   5.527983s
  reject_map_inplace     65.876  (± 3.0%) i/s -    330.000 
reject_inplace_map_inplace
                          1.090  (± 0.0%) i/s -      6.000  in   5.504853s

@ChrisLundquist
Copy link
Author

Yeah, I see that slowdown across 2.2.0p0, 2.2.2p95, and 1.9.3p484

unitys-MacBook-Pro-6:tmp clundquist$ ruby -v
ruby 2.2.0p0 (2014-12-25 revision 49005) [x86_64-darwin14]
unitys-MacBook-Pro-6:tmp clundquist$ ruby tenderlove.rb 
Calculating -------------------------------------
         map_compact     6.000  i/100ms
 map_inplace_compact     6.000  i/100ms
          reject_map     5.000  i/100ms
  reject_inplace_map     1.000  i/100ms
  reject_map_inplace     5.000  i/100ms
reject_inplace_map_inplace
                         1.000  i/100ms
-------------------------------------------------
         map_compact     64.228  (± 4.7%) i/s -    324.000 
 map_inplace_compact     68.176  (± 2.9%) i/s -    342.000 
          reject_map     55.735  (± 3.6%) i/s -    280.000 
  reject_inplace_map      1.085  (± 0.0%) i/s -      6.000  in   5.528148s
  reject_map_inplace     56.404  (± 3.5%) i/s -    285.000 
reject_inplace_map_inplace
                          1.085  (± 0.0%) i/s -      6.000  in   5.531503s
unitys-MacBook-Pro-6:tmp clundquist$ rbenv shell 2.2.2
unitys-MacBook-Pro-6:tmp clundquist$ ruby -v
ruby 2.2.2p95 (2015-04-13 revision 50295) [x86_64-darwin14]
unitys-MacBook-Pro-6:tmp clundquist$ ruby tenderlove.rb 
Calculating -------------------------------------
         map_compact     7.000  i/100ms
 map_inplace_compact     7.000  i/100ms
          reject_map     6.000  i/100ms
  reject_inplace_map     1.000  i/100ms
  reject_map_inplace     6.000  i/100ms
reject_inplace_map_inplace
                         1.000  i/100ms
-------------------------------------------------
         map_compact     72.320  (± 4.1%) i/s -    364.000 
 map_inplace_compact     77.968  (± 2.6%) i/s -    392.000 
          reject_map     66.188  (± 3.0%) i/s -    336.000 
  reject_inplace_map      1.095  (± 0.0%) i/s -      6.000  in   5.480780s
  reject_map_inplace     67.418  (± 3.0%) i/s -    342.000 
reject_inplace_map_inplace
                          1.083  (± 0.0%) i/s -      6.000  in   5.539186s
unitys-MacBook-Pro-6:tmp clundquist$ rbenv shell 1.9.3-p484
unitys-MacBook-Pro-6:tmp clundquist$ ruby -v
ruby 1.9.3p484 (2013-11-22 revision 43786) [x86_64-darwin14.1.0]
unitys-MacBook-Pro-6:tmp clundquist$ ruby tenderlove.rb 
Calculating -------------------------------------
         map_compact     6.000  i/100ms
 map_inplace_compact     6.000  i/100ms
          reject_map     6.000  i/100ms
  reject_inplace_map     1.000  i/100ms
  reject_map_inplace     6.000  i/100ms
reject_inplace_map_inplace
                         1.000  i/100ms
-------------------------------------------------
         map_compact     72.440  (± 4.1%) i/s -    366.000 
 map_inplace_compact     69.236  (± 2.9%) i/s -    348.000 
          reject_map     63.612  (± 4.7%) i/s -    318.000 
  reject_inplace_map      1.084  (± 0.0%) i/s -      6.000  in   5.535857s
  reject_map_inplace     62.558  (± 3.2%) i/s -    318.000 
reject_inplace_map_inplace
                          1.087  (± 0.0%) i/s -      6.000  in   5.522847s

Maybe it is something else about my laptop?

@ChrisLundquist
Copy link
Author

An EC2 t2.medium instance running Ubuntu 14.04 LTS also shows the slowdown.

$ ruby -v
ruby 1.9.3p484 (2013-11-22 revision 43786) [x86_64-linux]
$ uname -a
Linux production-hubot-1 3.13.0-49-generic #81-Ubuntu SMP Tue Mar 24 19:29:48 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
$ ruby tenderlove.rb 
Calculating -------------------------------------
         map_compact     3.000  i/100ms
 map_inplace_compact     3.000  i/100ms
          reject_map     3.000  i/100ms
  reject_inplace_map     1.000  i/100ms
  reject_map_inplace     3.000  i/100ms
reject_inplace_map_inplace
                         1.000  i/100ms
-------------------------------------------------
         map_compact     36.691  (± 5.5%) i/s -    183.000 
 map_inplace_compact     37.528  (± 5.3%) i/s -    189.000 
          reject_map     34.207  (± 5.8%) i/s -    171.000 
  reject_inplace_map      0.517  (± 0.0%) i/s -      3.000  in   5.799248s
  reject_map_inplace     33.983  (± 5.9%) i/s -    171.000 
reject_inplace_map_inplace
                          0.521  (± 0.0%) i/s -      3.000  in   5.762910s

@ChrisLundquist
Copy link
Author

Using ruby-prof and 1_000_000 elements it took 262 seconds to run.
Using only 1_000_000 elements and no profiler took 131s.
100_000 elements took about 5.5 seconds

$ ruby tenderlove.rb 
Calculating -------------------------------------
reject_inplace_map_inplace
                    x     1.000  i/100ms
-------------------------------------------------
reject_inplace_map_inplace
                          0.008  (± 0.0%) i/s -      1.000  in 131.980742s
Measure Mode: wall_time
Thread ID: 70302340168920
Fiber ID: 70302352779980
Total: 262.674414
Sort by: self_time

 %self      total      self      wait     child     calls  name
 99.58    262.426   261.581     0.000     0.846        2   Array#reject!
  0.32      0.846     0.846     0.000     0.000  2000000   Fixnum#odd?
  0.06      0.167     0.167     0.000     0.000        2   Array#map!
  0.03      0.077     0.077     0.000     0.000        2   Range#each
  0.00      0.003     0.003     0.000     0.000        2   <Module::GC>#start
  0.00    262.674     0.000     0.000   262.674        1   Global#[No method]
  0.00      0.000     0.000     0.000     0.000        9   IO#write
  0.00    262.674     0.000     0.000   262.674        6  *Array#each
  0.00      0.000     0.000     0.000     0.000        5   String#%
  0.00    262.674     0.000     0.000   262.674        1   Benchmark::IPS#ips
  0.00      0.000     0.000     0.000     0.000       11   <Class::Time>#now
  0.00      0.000     0.000     0.000     0.000        5   Rational#/
  0.00      0.000     0.000     0.000     0.000        4   Time#to_f
  0.00      0.000     0.000     0.000     0.000        4   Float#to_i
  0.00      0.000     0.000     0.000     0.000       11   Time#initialize
  0.00      0.000     0.000     0.000     0.000        3   Benchmark::IPS::Helpers#scale
  0.00      0.000     0.000     0.000     0.000        1   Benchmark::IPS::Report::Entry#body
  0.00      0.000     0.000     0.000     0.000       11   <Class::Time>#allocate
  0.00      0.077     0.000     0.000     0.077        2   Enumerable#to_a
  0.00      0.000     0.000     0.000     0.000        1   Time#-
  0.00      0.000     0.000     0.000     0.000        2   Benchmark::IPS::Job::Entry#label_rjust
  0.00    131.983     0.000     0.000   131.983        1   Benchmark::IPS::Job#run
  0.00      0.000     0.000     0.000     0.000        2   Benchmark::IPS::Job#time_us
  0.00      0.000     0.000     0.000     0.000        3   <Module::Math>#log10
  0.00    262.670     0.000     0.000   262.670        2   Object#reject_inplace_map_inplace
  0.00    262.671     0.000     0.000   262.670        2   Benchmark::IPS::Job::Entry#call_times
  0.00      0.000     0.000     0.000     0.000        1   Benchmark::IPS::Job#initialize
  0.00      0.000     0.000     0.000     0.000        5   Numeric#quo
  0.00      0.000     0.000     0.000     0.000        1   Float#**
  0.00      0.000     0.000     0.000     0.000       17   Fixnum#+
  0.00      0.000     0.000     0.000     0.000        4  *Class#new
  0.00      0.000     0.000     0.000     0.000        4   Time#<=>
  0.00      0.003     0.000     0.000     0.003        2   <Module::Benchmark::Timing>#clean_env
  0.00      0.000     0.000     0.000     0.000        3   Float#*
  0.00      0.000     0.000     0.000     0.000        5   Rational#to_f
  0.00      0.000     0.000     0.000     0.000        2   <Module::Benchmark::Timing>#mean
  0.00      0.000     0.000     0.000     0.000        4   Enumerable#inject
  0.00      0.000     0.000     0.000     0.000        1   Benchmark::IPS::Job::Entry#initialize
  0.00      0.000     0.000     0.000     0.000        3   Kernel#hash
  0.00      0.000     0.000     0.000     0.000        4   Comparable#<
  0.00      0.000     0.000     0.000     0.000        2   <Module::Benchmark::IPS>#options
  0.00    262.670     0.000     0.000   262.670        2   Proc#call
  0.00      0.000     0.000     0.000     0.000        7   Fixnum#to_f
  0.00      0.077     0.000     0.000     0.077        2   Object#data
  0.00      0.000     0.000     0.000     0.000        1   Benchmark::IPS::Job#iterations_per_sec
  0.00      0.000     0.000     0.000     0.000        1   Benchmark::IPS::Report::Entry#initialize
  0.00      0.000     0.000     0.000     0.000        4   Kernel#respond_to?
  0.00      0.000     0.000     0.000     0.000        3   IO#puts
  0.00      0.000     0.000     0.000     0.000        1   Hash#[]=
  0.00      0.000     0.000     0.000     0.000        5   Fixnum#fdiv
  0.00      0.000     0.000     0.000     0.000        1   Array#map
  0.00      0.000     0.000     0.000     0.000        2   Time#+
  0.00      0.000     0.000     0.000     0.000        1   IO#printf
  0.00      0.000     0.000     0.000     0.000        4   <Class::BasicObject>#allocate
  0.00      0.000     0.000     0.000     0.000        1   Benchmark::IPS::Job#create_report
  0.00      0.000     0.000     0.000     0.000        4   Float#to_f
  0.00      0.000     0.000     0.000     0.000        1   Benchmark::IPS::Report#add_entry
  0.00      0.000     0.000     0.000     0.000        3   Fixnum#**
  0.00      0.000     0.000     0.000     0.000        2   IO#print
  0.00      0.000     0.000     0.000     0.000        1   Float#<=
  0.00      0.000     0.000     0.000     0.000        1   IO#sync
  0.00      0.000     0.000     0.000     0.000        1   Float#round
  0.00      0.000     0.000     0.000     0.000        1   Benchmark::IPS::Job#cycles_per_100ms
  0.00      0.000     0.000     0.000     0.000        1   Benchmark::IPS::Job#json?
  0.00      0.000     0.000     0.000     0.000        9   Float#/
  0.00      0.000     0.000     0.000     0.000        4   Fixnum#<=>
  0.00      0.000     0.000     0.000     0.000        1   <Module::Benchmark::Timing>#stddev
  0.00      0.000     0.000     0.000     0.000        2   String#to_s
  0.00      0.000     0.000     0.000     0.000        1   Benchmark::IPS::Job#config
  0.00      0.000     0.000     0.000     0.000        2   String#*
  0.00      0.000     0.000     0.000     0.000        1   <Module::Math>#sqrt
  0.00      0.000     0.000     0.000     0.000        1   Float#abs
  0.00      0.000     0.000     0.000     0.000        1   Float#>=
  0.00    130.692     0.000     0.000   130.692        1   Benchmark::IPS::Job#run_warmup
  0.00      0.000     0.000     0.000     0.000        1   Benchmark::IPS::Report::Entry#show_total_time!
  0.00      0.000     0.000     0.000     0.000        1   Benchmark::IPS::Job#item
  0.00      0.000     0.000     0.000     0.000        2   IO#sync=
  0.00      0.000     0.000     0.000     0.000        1   Benchmark::IPS::Job#compare?
  0.00      0.000     0.000     0.000     0.000        1   String#ljust
  0.00      0.000     0.000     0.000     0.000        2   NilClass#nil?
  0.00      0.000     0.000     0.000     0.000        1   Benchmark::IPS::Report::Entry#stddev_percentage
  0.00      0.000     0.000     0.000     0.000        2   Kernel#respond_to_missing?
  0.00      0.000     0.000     0.000     0.000        1   Fixnum#-
  0.00      0.000     0.000     0.000     0.000        1   Proc#arity
  0.00      0.000     0.000     0.000     0.000        1   Benchmark::IPS::Report#initialize
  0.00      0.000     0.000     0.000     0.000        1   Fixnum#*
  0.00      0.000     0.000     0.000     0.000        1   Kernel#kind_of?
  0.00      0.000     0.000     0.000     0.000        1   Benchmark::IPS::Report::Entry#seconds
  0.00      0.000     0.000     0.000     0.000        1   Array#push
  0.00      0.000     0.000     0.000     0.000        1   Fixnum#/
  0.00      0.000     0.000     0.000     0.000        1   Kernel#is_a?
  0.00      0.000     0.000     0.000     0.000        1   Array#last
  0.00      0.000     0.000     0.000     0.000        1   <Module::Benchmark::Timing>#variance

@ChrisLundquist
Copy link
Author

screen shot 2015-12-04 at 7 30 14 pm

require  "time"

def data(size)
  (0...size).to_a
end

def reject_inplace(size)
  filtered = data(size)
  filtered = filtered.reject! { |i| i.odd? }
  filtered
end

def graph
  40.times.reduce({}) do |acc,i|
    size = ( i + 1) * 10000
    start = Time.now
    reject_inplace(size)
    finish = Time.now
    duration = finish - start
    acc[size] = duration
    acc
  end
end

data = graph
data.keys.sort.each do |k| puts("#{ k },#{data[k]}") end

That's a pretty good N^2 curve fit.
Today I learned not to use reject! until ruby > 2.2.3

@ChrisLundquist
Copy link
Author

Fixed in ruby/ruby@5ec029d

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment