Skip to content

Instantly share code, notes, and snippets.

@AaronRustad
Created April 24, 2013 19:53
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save AaronRustad/d6f20229b9f19f6d163e to your computer and use it in GitHub Desktop.
Save AaronRustad/d6f20229b9f19f6d163e to your computer and use it in GitHub Desktop.
require 'rubygems'
gem 'multi_json'
gem 'json'
gem 'oj'
gem 'yajl-ruby'
require 'benchmark'
require 'multi_json'
require 'json'
require 'oj'
require 'yajl'
json = '{"abc":"def"}'
n = 50000
Benchmark.bm(17) do |x|
x.report("multi/json") do
MultiJson.use(:json_pure)
n.times { MultiJson.load(json) }
end
x.report("json") { n.times { JSON::parse(json) } }
x.report("multi/oj") do
MultiJson.use(:oj)
n.times { MultiJson.load(json) }
end
x.report("oj") { n.times { Oj::Doc.parse(json) } }
x.report("multi/yajl") do
MultiJson.use(:yajl)
n.times { MultiJson.load(json) }
end
x.report("yajl") do
n.times { Yajl::Parser.new.parse(json) }
end
end
object = {abc: 'def'}
Benchmark.bm(17) do |x|
x.report("multi/json") do
MultiJson.use(:json_pure)
n.times { MultiJson.dump(object) }
end
x.report("json") { n.times { JSON(json) } }
x.report("multi/oj") do
MultiJson.use(:oj)
n.times { MultiJson.dump(object) }
end
x.report("oj") { n.times { Oj.dump(json) } }
x.report("multi/yajl") do
MultiJson.use(:yajl)
n.times { MultiJson.dump(json) }
end
x.report("yajl") do
n.times { Yajl::Encoder.encode(json) }
end
end
user system total real
multi/json 0.570000 0.010000 0.580000 ( 0.591576)
json 0.170000 0.020000 0.190000 ( 0.178378)
multi/oj 0.440000 0.010000 0.450000 ( 0.456493)
oj 0.210000 0.060000 0.270000 ( 0.268035)
multi/yajl 0.530000 0.030000 0.560000 ( 0.555605)
yajl 0.190000 0.040000 0.230000 ( 0.228279)
user system total real
multi/json 0.550000 0.010000 0.560000 ( 0.562655)
json 0.190000 0.010000 0.200000 ( 0.196212)
multi/oj 0.410000 0.000000 0.410000 ( 0.411601)
oj 0.000000 0.000000 0.000000 ( 0.007718)
multi/yajl 0.520000 0.020000 0.540000 ( 0.530797)
yajl 0.180000 0.020000 0.200000 ( 0.201096)
@darrenboyd
Copy link

This is me, running it on my old MBA...

                        user     system      total        real
multi/json          1.260000   0.030000   1.290000 (  1.300222)
json                0.390000   0.030000   0.420000 (  0.422368)
multi/oj            0.970000   0.030000   1.000000 (  1.012530)
oj                  0.510000   0.150000   0.660000 (  0.741785)
multi/yajl          1.110000   0.080000   1.190000 (  1.187297)
yajl                0.380000   0.060000   0.440000 (  0.457276)
                        user     system      total        real
multi/json          1.210000   0.010000   1.220000 (  1.223367)
json                0.400000   0.020000   0.420000 (  0.431476)
multi/oj            0.890000   0.010000   0.900000 (  0.899011)
oj                  0.010000   0.000000   0.010000 (  0.018243)
multi/yajl          1.120000   0.040000   1.160000 (  1.168490)
yajl                0.320000   0.030000   0.350000 (  0.359170)

@darrenboyd
Copy link

Here it is running on the build server (AWS fairly beefy machine)...

                        user     system      total        real
multi/json          1.190000   0.000000   1.190000 (  1.188965)
json                0.300000   0.000000   0.300000 (  0.293221)
multi/oj            1.030000   0.020000   1.050000 (  1.052230)
oj                  0.310000   0.270000   0.580000 (  0.590445)
multi/yajl          1.210000   0.020000   1.230000 (  1.221959)
yajl                0.270000   0.030000   0.300000 (  0.306176)
                        user     system      total        real
multi/json          1.200000   0.000000   1.200000 (  1.192078)
json                0.290000   0.000000   0.290000 (  0.295167)
multi/oj            0.930000   0.000000   0.930000 (  0.926135)
oj                  0.030000   0.000000   0.030000 (  0.032685)
multi/yajl          1.280000   0.000000   1.280000 (  1.274899)
yajl                0.330000   0.020000   0.350000 (  0.348557)

@AaronRustad
Copy link
Author

Here it is, again, on a rMBP-13, ruby-1.9.3p327

                        user     system      total        real
multi/json          0.540000   0.020000   0.560000 (  0.549747)
json                0.170000   0.000000   0.170000 (  0.176655)
multi/oj            0.410000   0.010000   0.420000 (  0.418114)
oj                  0.240000   0.070000   0.310000 (  0.308255)
multi/yajl          0.480000   0.020000   0.500000 (  0.498978)
yajl                0.140000   0.020000   0.160000 (  0.158191)
                        user     system      total        real
multi/json          0.590000   0.010000   0.600000 (  0.600695)
json                0.180000   0.010000   0.190000 (  0.187899)
multi/oj            4.130000   0.000000   4.130000 (  4.138311)
oj                  3.730000   0.010000   3.740000 (  3.737866)
multi/yajl          0.490000   0.000000   0.490000 (  0.492018)
yajl                0.120000   0.010000   0.130000 (  0.126083)

@krainboltgreene
Copy link

I've improved the script I think:

require "benchmark/ips"
require "json"
require "oj"
require "yajl"
require "yajl/version"

puts "OS Name: #{`uname -a`}"
puts `sw_vers`
puts "Ruby Version: #{`ruby -v`}"
puts "RubyGems Version: #{`gem -v`}"
puts "RVM Version: #{`rvm -v`}"
puts "oj.gem Version: #{Oj::VERSION}"
puts "yajl-ruby.gem Version: #{Yajl::VERSION}"

SMALL_LOAD = %|{"a":"1", "b": 2, "c": { "z": "9", "x": 10 }, "d": ["a", 1, {}, []]}|
SMALL_DUMP = {"a" => "1", "b" => 2, "c" => { "z" => "9", "x" => 10 }, "d" => ["a", 1, {}, []]}

Benchmark.ips do |analysis|
  analysis.time = 5
  analysis.warmup = 2

  analysis.report("stdlib small load") do
    JSON.load(SMALL_LOAD.dup)
  end

  analysis.report("oj small load")do
    Oj.load(SMALL_LOAD.dup)
  end

  analysis.report("yajl small load")do
    Yajl::Parser.new.parse(SMALL_LOAD.dup)
  end

  analysis.compare!
end

Benchmark.ips do |analysis|
  analysis.time = 5
  analysis.warmup = 2

  analysis.report("stdlib small dump") do
    JSON.dump(SMALL_DUMP.dup)
  end

  analysis.report("oj small dump")do
    Oj.dump(SMALL_DUMP.dup)
  end

  analysis.report("yajl small dump")do
    Yajl::Encoder.new.encode(SMALL_DUMP.dup)
  end

  analysis.compare!
end

# ---

BIG_LOAD = File.read("big.json")
BIG_DUMP = Marshal.load(File.read("big.json.rb"))

Benchmark.ips do |analysis|
  analysis.time = 5
  analysis.warmup = 2

  analysis.report("stdlib big load") do
    JSON.load(BIG_LOAD.dup)
  end

  analysis.report("oj big load")do
    Oj.load(BIG_LOAD.dup)
  end

  analysis.report("yajl big load")do
    Yajl::Parser.new.parse(BIG_LOAD.dup)
  end

  analysis.compare!
end

Benchmark.ips do |analysis|
  analysis.time = 5
  analysis.warmup = 2

  analysis.report("stdlib big dump") do
    JSON.dump(BIG_DUMP.dup)
  end

  analysis.report("oj big dump")do
    Oj.dump(BIG_DUMP.dup)
  end

  analysis.report("yajl big dump")do
    Yajl::Encoder.new.encode(BIG_DUMP.dup)
  end

  analysis.compare!
end
▶ ruby json_benchmarks.rb
OS Name: Darwin macbookpro.local 13.3.0 Darwin Kernel Version 13.3.0: Tue Jun  3 21:27:35 PDT 2014; root:xnu-2422.110.17~1/RELEASE_X86_64 x86_64
ProductName:    Mac OS X
ProductVersion: 10.9.4
BuildVersion:   13E28
Ruby Version: ruby 2.1.2p95 (2014-05-08 revision 45877) [x86_64-darwin13.0]
RubyGems Version: 2.2.2
RVM Version: rvm 1.25.28 (stable) by Wayne E. Seguin <wayneeseguin@gmail.com>, Michal Papis <mpapis@gmail.com> [https://rvm.io/]
oj.gem Version: 2.10.2
yajl-ruby.gem Version: 1.2.1

Calculating -------------------------------------
   stdlib small load      6428 i/100ms
       oj small load     13407 i/100ms
     yajl small load      7788 i/100ms
-------------------------------------------------
   stdlib small load    72742.1 (±8.0%) i/s -     366396 in   5.071908s
       oj small load   186584.9 (±10.9%) i/s -     925083 in   5.038102s
     yajl small load   101009.9 (±9.3%) i/s -     506220 in   5.060820s

Comparison:
       oj small load:   186584.9 i/s
     yajl small load:   101009.9 i/s - 1.85x slower
   stdlib small load:    72742.1 i/s - 2.57x slower

--------

Calculating -------------------------------------
   stdlib small dump      7488 i/100ms
       oj small dump     20868 i/100ms
     yajl small dump     10511 i/100ms
-------------------------------------------------
   stdlib small dump    86995.7 (±5.4%) i/s -     434304 in   5.007885s
       oj small dump   284832.1 (±10.7%) i/s -    1419024 in   5.059494s
     yajl small dump   138795.2 (±8.6%) i/s -     693726 in   5.039638s

Comparison:
       oj small dump:   284832.1 i/s
     yajl small dump:   138795.2 i/s - 2.05x slower
   stdlib small dump:    86995.7 i/s - 3.27x slower

--------

Calculating -------------------------------------
   stdlib big load        19 i/100ms
       oj big load        32 i/100ms
     yajl big load        20 i/100ms
-------------------------------------------------
   stdlib big load      185.6 (±8.6%) i/s -        931 in   5.055776s
       oj big load      326.0 (±10.7%) i/s -       1632 in   5.075107s
     yajl big load      212.8 (±9.9%) i/s -       1060 in   5.033603s

Comparison:
       oj big load:      326.0 i/s
     yajl big load:      212.8 i/s - 1.53x slower
   stdlib big load:      185.6 i/s - 1.76x slower

--------

Calculating -------------------------------------
   stdlib big dump        29 i/100ms
       oj big dump       107 i/100ms
     yajl big dump        52 i/100ms
-------------------------------------------------
   stdlib big dump      314.1 (±6.4%) i/s -       1595 in   5.098338s
       oj big dump     1157.4 (±10.3%) i/s -       5778 in   5.053967s
     yajl big dump      539.0 (±7.8%) i/s -       2704 in   5.049070s

Comparison:
       oj big dump:     1157.4 i/s
     yajl big dump:      539.0 i/s - 2.15x slower
   stdlib big dump:      314.1 i/s - 3.68x slower

@ceclinux
Copy link

ceclinux commented Feb 1, 2018

my benchmark on i5-8250U, running Linux ceclinux-pc 4.9.77-1-MANJARO #1 SMP PREEMPT Wed Jan 17 23:04:54 UTC 2018 x86_64 GNU/Linux

Running via Spring preloader in process 23118
                        user     system      total        real
multi/json          0.240000   0.010000   0.250000 (  0.250096)
json                0.110000   0.000000   0.110000 (  0.113757)
multi/oj            0.220000   0.000000   0.220000 (  0.216726)
oj                  0.170000   0.060000   0.230000 (  0.230328)
multi/yajl          2.060000   0.010000   2.070000 (  2.063019)
yajl                0.110000   0.010000   0.120000 (  0.124362)
                        user     system      total        real
multi/json          0.590000   0.000000   0.590000 (  0.596703)
json                0.180000   0.000000   0.180000 (  0.176591)
multi/oj            0.170000   0.000000   0.170000 (  0.168770)
oj                  0.010000   0.010000   0.020000 (  0.008189)
multi/yajl          0.180000   0.000000   0.180000 (  0.186003)
yajl                0.100000   0.000000   0.100000 (  0.095512)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment