Ruby-based Benchmark of MessagePack vs. JSON & Yajl

decode.rb

# encoding: utf-8

require 'bench_press'
require 'json'
require 'yajl'
require 'msgpack'

extend BenchPress
reps 10_000

data = {
  "repository" => {
    "watchers" => 173,
    "has_wiki" => true,
    "url" => "https://github.com/flori/json",
    "open_issues" => 21,
    "homepage" => "http://flori.github.com/json",
    "has_issues" => true,
    "forks" => 35,
    "fork" => false,
    "language" => "Ruby",
    "integrate_branch" => "master",
    "created_at" => "2009/08/24 15:21:39 -0700",
    "master_branch" => "master",
    "size" => 572,
    "private" => false,
    "name" => "json",
    "owner" => "flori",
    "has_downloads" => true,
    "pushed_at" => "2011/07/08 07:34:34 -0700",
    "description" => "JSON implementation for Ruby"
  }
}

json = Yajl::Encoder.encode(data)
msgpack = MessagePack.pack(data)

measure "JSON.parse" do
  JSON.parse(json)
end

measure "Yajl::Parser.parse" do
  Yajl::Parser.parse(json)
end

measure "MessagePack.unpack" do
  MessagePack.unpack(msgpack)
end

decode_result.txt

Decode
======
Date: August 20, 2011

System Information
------------------
    Operating System:    Mac OS X 10.6.8 (10K549)
    Machine:             MacBookPro8,2 (Spring 2011)
    CPU:                 Intel Core i7 2 GHz
    Processor Count:     4
    Memory:              4 GB
    Ruby Gems:           bench_press (0.3.1), json (1.5.3),
                         msgpack (0.4.6), yajl-ruby (0.8.3)


--------------------------------------------------------------------------------
 - ruby 1.9.2p290 (2011-07-09 revision 32553) [x86_64-darwin10.8.0]
--------------------------------------------------------------------------------

"MessagePack.unpack" is up to 61% faster over 10,000 repetitions:

    MessagePack.unpack    0.11150479316711426 secs    Fastest
    Yajl::Parser.parse    0.26775693893432617 secs    58% Slower
    JSON.parse            0.2864522933959961  secs    61% Slower


--------------------------------------------------------------------------------
 - ruby 1.8.7 (2011-06-30 patchlevel 352) [i686-darwin10.8.0]
--------------------------------------------------------------------------------

"MessagePack.unpack" is up to 40% faster over 10,000 repetitions:

    MessagePack.unpack    0.158010005950928 secs    Fastest
    JSON.parse            0.240987062454224 secs    34% Slower
    Yajl::Parser.parse    0.26587986946106  secs    40% Slower


--------------------------------------------------------------------------------
 - ruby 1.8.7 (2011-02-18 patchlevel 334) [i686-darwin10.8.0], MBARI 0x6770,
   Ruby Enterprise Edition 2011.03
--------------------------------------------------------------------------------

"MessagePack.unpack" is up to 62% faster over 10,000 repetitions:

    MessagePack.unpack    0.091127872467041 secs    Fastest
    JSON.parse            0.235518217086792 secs    61% Slower
    Yajl::Parser.parse    0.241642951965332 secs    62% Slower


--------------------------------------------------------------------------------
 - rubinius 1.2.5dev (1.8.7 4cb50041 yyyy-mm-dd JI) [x86_64-apple-darwin10.8.0]
--------------------------------------------------------------------------------

"MessagePack.unpack" is up to 44% faster over 10,000 repetitions:

    MessagePack.unpack    0.8761258125305176 secs    Fastest
    JSON.parse            1.164367914199829  secs    24% Slower
    Yajl::Parser.parse    1.5739879608154297 secs    44% Slower


--------------------------------------------------------------------------------
 - jruby 1.6.3 (ruby-1.8.7-p330) (2011-07-07 965162f) (Java HotSpot(TM) 64-Bit
   Server VM 1.6.0_26) [darwin-x86_64-java]
--------------------------------------------------------------------------------

Tests fail thanks to the bench_press gem using Process.fork to run tests. I
might revisit these tests again later to add JRuby benchmarks too.

encode.rb

# encoding: utf-8

require 'bench_press'
require 'json'
require 'yajl'
require 'msgpack'

extend BenchPress
reps 10_000

data = {
  "repository" => {
    "watchers" => 173,
    "has_wiki" => true,
    "url" => "https://github.com/flori/json",
    "open_issues" => 21,
    "homepage" => "http://flori.github.com/json",
    "has_issues" => true,
    "forks" => 35,
    "fork" => false,
    "language" => "Ruby",
    "integrate_branch" => "master",
    "created_at" => "2009/08/24 15:21:39 -0700",
    "master_branch" => "master",
    "size" => 572,
    "private" => false,
    "name" => "json",
    "owner" => "flori",
    "has_downloads" => true,
    "pushed_at" => "2011/07/08 07:34:34 -0700",
    "description" => "JSON implementation for Ruby"
  }
}

# json

measure "JSON.generate" do
  JSON.generate(data)
end

# yajl

measure "Yajl::Encoder.encode" do
  Yajl::Encoder.encode(data)
end

# messagepack

measure "MessagePack.pack" do
  MessagePack.pack(data)
end

encode_result.txt

Encode
======
Date: August 20, 2011

System Information
------------------
    Operating System:    Mac OS X 10.6.8 (10K549)
    Machine:             MacBookPro8,2 (Spring 2011)
    CPU:                 Intel Core i7 2 GHz
    Processor Count:     4
    Memory:              4 GB
    Ruby Gems:           bench_press (0.3.1), json (1.5.3),
                         msgpack (0.4.6), yajl-ruby (0.8.3)


--------------------------------------------------------------------------------
 - ruby 1.9.2p290 (2011-07-09 revision 32553) [x86_64-darwin10.8.0]
--------------------------------------------------------------------------------

"MessagePack.pack" is up to 87% faster over 10,000 repetitions:

    MessagePack.pack        0.06354403495788574 secs    Fastest
    Yajl::Encoder.encode    0.13566207885742188 secs    53% Slower
    JSON.generate           0.503342866897583   secs    87% Slower


--------------------------------------------------------------------------------
 - ruby 1.8.7 (2011-06-30 patchlevel 352) [i686-darwin10.8.0]
--------------------------------------------------------------------------------

"MessagePack.pack" is up to 71% faster over 10,000 repetitions:

    MessagePack.pack        0.0695998668670654 secs    Fastest
    Yajl::Encoder.encode    0.170566082000732  secs    59% Slower
    JSON.generate           0.246031999588013  secs    71% Slower


--------------------------------------------------------------------------------
 - ruby 1.8.7 (2011-02-18 patchlevel 334) [i686-darwin10.8.0], MBARI 0x6770,
   Ruby Enterprise Edition 2011.03
--------------------------------------------------------------------------------

"MessagePack.pack" is up to 68% faster over 10,000 repetitions:

    MessagePack.pack        0.0694189071655273 secs    Fastest
    Yajl::Encoder.encode    0.165558099746704  secs    58% Slower
    JSON.generate           0.217200040817261  secs    68% Slower


--------------------------------------------------------------------------------
 - rubinius 1.2.5dev (1.8.7 4cb50041 yyyy-mm-dd JI) [x86_64-apple-darwin10.8.0]
--------------------------------------------------------------------------------

"Yajl::Encoder.encode" is up to 14% faster over 10,000 repetitions:

    Yajl::Encoder.encode    0.5371038913726807 secs    Fastest
    MessagePack.pack        0.5457220077514648 secs    1% Slower
    JSON.generate           0.6305360794067383 secs    14% Slower


--------------------------------------------------------------------------------
 - jruby 1.6.3 (ruby-1.8.7-p330) (2011-07-07 965162f) (Java HotSpot(TM) 64-Bit
   Server VM 1.6.0_26) [darwin-x86_64-java]
--------------------------------------------------------------------------------

Tests fail thanks to the bench_press gem using Process.fork to run tests. I
might revisit these tests again later to add JRuby benchmarks too.