Skip to content

Instantly share code, notes, and snippets.

@eregon
Last active October 31, 2020 15:04
Show Gist options
  • Save eregon/8a9437e4ecf23ff300b88f8682ccbe03 to your computer and use it in GitHub Desktop.
Save eregon/8a9437e4ecf23ff300b88f8682ccbe03 to your computer and use it in GitHub Desktop.
Benchmark results on the benchmark from https://chrisseaton.com/truffleruby/ruby-stm/
Ruby Implementation Speedup relative to 2.7.2
Ruby 2.7.2 1.00x
Ruby 2.7.2 --jit 1.16x
Ruby 3 dev 0.81x
Ruby 3 dev --jit 1.08x
TruffleRuby CE 20.2 --jvm 19.73x
TruffleRuby EE 20.2 --jvm 31.54x

The median i/s result out of 3 is used to compute speedups.

These results were run with this commit + changes.diff below to get a better idea of variance.

diff --git a/bench/sequential.rb b/bench/sequential.rb
index e0d7bf8..bdd2a8d 100644
--- a/bench/sequential.rb
+++ b/bench/sequential.rb
@@ -1,5 +1,7 @@
# This benchmark solves the testBoard.
+puts RUBY_DESCRIPTION
+
require 'benchmark/ips'
require_relative '../lib/lee'
@@ -86,8 +88,9 @@ def solve_board(board)
end
Benchmark.ips do |x|
- x.time = 30
- x.warmup = 30
+ x.time = 10
+ x.warmup = 10
+ x.iterations = 3
x.report('testBoard') do
solve_board(board)
diff --git a/lib/lee/board.rb b/lib/lee/board.rb
index 111076c..f933090 100644
--- a/lib/lee/board.rb
+++ b/lib/lee/board.rb
@@ -1,3 +1,5 @@
+require 'set'
+
module Lee
# A point is a location on the board.
$ ruby bench/sequential.rb
ruby 2.7.2p137 (2020-10-01 revision 5445e04352) [x86_64-linux]
Warming up --------------------------------------
testBoard 1.000 i/100ms
testBoard 1.000 i/100ms
testBoard 1.000 i/100ms
Calculating -------------------------------------
testBoard 1.214 (± 0.0%) i/s - 13.000 in 10.714388s
testBoard 1.214 (± 0.0%) i/s - 13.000 in 10.711989s
testBoard 1.215 (± 0.0%) i/s - 13.000 in 10.702990s
$ ruby --jit bench/sequential.rb
ruby 2.7.2p137 (2020-10-01 revision 5445e04352) +JIT [x86_64-linux]
Warming up --------------------------------------
testBoard 1.000 i/100ms
testBoard 1.000 i/100ms
testBoard 1.000 i/100ms
Calculating -------------------------------------
testBoard 1.411 (± 0.0%) i/s - 15.000 in 10.631961s
testBoard 1.410 (± 0.0%) i/s - 15.000 in 10.643642s
testBoard 1.411 (± 0.0%) i/s - 15.000 in 10.636146s
$ ruby bench/sequential.rb
ruby 3.0.0dev (2020-10-30T12:45:23Z master 66bf743b93) [x86_64-linux]
Warming up --------------------------------------
testBoard 1.000 i/100ms
testBoard 1.000 i/100ms
testBoard 1.000 i/100ms
Calculating -------------------------------------
testBoard 0.984 (± 0.0%) i/s - 10.000 in 10.158082s
testBoard 0.989 (± 0.0%) i/s - 10.000 in 10.112835s
testBoard 0.988 (± 0.0%) i/s - 10.000 in 10.119596s
$ ruby --jit bench/sequential.rb
ruby 3.0.0dev (2020-10-30T12:45:23Z master 66bf743b93) +JIT [x86_64-linux]
Warming up --------------------------------------
testBoard 1.000 i/100ms
testBoard 1.000 i/100ms
testBoard 1.000 i/100ms
Calculating -------------------------------------
testBoard 1.304 (± 0.0%) i/s - 14.000 in 10.734479s
testBoard 1.321 (± 0.0%) i/s - 14.000 in 10.597937s
testBoard 1.315 (± 0.0%) i/s - 14.000 in 10.646729s
$ ruby --jvm bench/sequential.rb
truffleruby 20.2.0, like ruby 2.6.6, GraalVM CE JVM [x86_64-linux]
Warming up --------------------------------------
testBoard 1.000 i/100ms
testBoard 2.000 i/100ms
testBoard 2.000 i/100ms
Calculating -------------------------------------
testBoard 23.898 (± 4.2%) i/s - 240.000 in 10.051196s
testBoard 23.948 (± 4.2%) i/s - 240.000 in 10.027479s
testBoard 23.952 (± 0.0%) i/s - 240.000 in 10.021410s
$ ruby --jvm bench/sequential.rb
truffleruby 20.2.0, like ruby 2.6.6, GraalVM EE JVM [x86_64-linux]
Warming up --------------------------------------
testBoard 1.000 i/100ms
testBoard 3.000 i/100ms
testBoard 3.000 i/100ms
Calculating -------------------------------------
testBoard 38.106 (± 2.6%) i/s - 381.000 in 10.010918s
testBoard 38.285 (± 2.6%) i/s - 384.000 in 10.035698s
testBoard 38.436 (± 2.6%) i/s - 387.000 in 10.070958s
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment