Skip to content

Instantly share code, notes, and snippets.

@nicksieger

nicksieger/jruby-memcached-benchmarks.md

Created September 28, 2010 16:37
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save nicksieger/601320 to your computer and use it in GitHub Desktop.
Save nicksieger/601320 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
jruby-memcached-benchmarks.md

JRuby Memcached Client Benchmark Notes

These benchmarks were run using a modified copy of Ruby Memcached's test/profile/benchmark.rb that adds several JRuby-only clients into the mix.

The benchmarking script also runs an Ehcache client for (an unfair) comparison. Ehcache is a configurable Java-based cache, and runs in-memory for this benchmark using the jruby-ehcache-rails3 gem.

The benchmarks were run with the JVM's server (optimizing) compiler. Since the JVM warms up as it goes, later runs are smaller and take into account the warmup.

For cross-Ruby comparison I also ran the same script to exercise the Ruby Memcached/libmemcached client using Ruby 1.9.2-p0.

Clients:

  • Dalli (pure Ruby memcached client)
  • jruby-memcache-client (replacement for memcache-client using the Whalin Java memcache library)
  • Spymemcached (JRuby memcached client using Spymemcached Java library by jamesgolick)
  • Ehcache

Setup

Raw
jruby-memcached-benchmarks.txt
$ jruby -v -I../dalli/lib:../spymemcached-gem/lib test/profile/benchmark.rb
jruby 1.6.0.dev (ruby 1.8.7 patchlevel 249) (2010-09-27 6ec5b2b) (Java HotSpot(TM) 64-Bit Server VM 1.6.0_20) [darwin-x86_64-java]
Darwin vanguard.local 10.4.0 Darwin Kernel Version 10.4.0: Fri Apr 23 18:28:53 PDT 2010; root:xnu-1504.7.4~1/RELEASE_I386 i386
jruby 1.8.7p249
Loops is 20000
Stack depth is 0
Small value size is: 13 bytes
Large value size is: 4158 bytes
No matching processes belonging to you were found
load error: rlibmemcached -- java.lang.UnsatisfiedLinkError: dlopen(/Users/nicksieger/Projects/ruby/memcached/lib/rlibmemcached.bundle, 9): Symbol not found: _environ
Referenced from: /Users/nicksieger/.rvm/rubies/ruby-1.9.2-p0/lib/libruby.1.9.1.dylib
Expected in: flat namespace
in /Users/nicksieger/.rvm/rubies/ruby-1.9.2-p0/lib/libruby.1.9.1.dylib
no such file to load -- remix/stash
2010-09-28 12:04:29.676 INFO net.spy.memcached.MemcachedConnection: Added {QA sa=/127.0.0.1:43042, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2010-09-28 12:04:29.677 INFO net.spy.memcached.MemcachedConnection: Added {QA sa=/127.0.0.1:43043, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2010-09-28 12:04:29.683 INFO net.spy.memcached.MemcachedConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@38f8ff42
2010-09-28 12:04:29.693 INFO net.spy.memcached.MemcachedConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@3b629e96
user system total real
set: dalli:bin 9.449000 0.000000 9.449000 ( 9.448000)
set: ehcache 3.582000 0.000000 3.582000 ( 3.582000)
set: jmclient:ascii 6.838000 0.000000 6.838000 ( 6.838000)
set: spymemcached 9.749000 0.000000 9.749000 ( 9.749000)
get: dalli:bin 7.827000 0.000000 7.827000 ( 7.827000)
get: ehcache 0.942000 0.000000 0.942000 ( 0.942000)
get: jmclient:ascii 5.655000 0.000000 5.655000 ( 5.655000)
get: spymemcached 8.601000 0.000000 8.601000 ( 8.601000)
get-multi: dalli:bin 12.954000 0.000000 12.954000 ( 12.954000)
get-multi: ehcache 1.005000 0.000000 1.005000 ( 1.005000)
get-multi: jmclient:ascii 4.879000 0.000000 4.879000 ( 4.879000)
get-multi: spymemcached 4.487000 0.000000 4.487000 ( 4.487000)
append: dalli:bin 8.642000 0.000000 8.642000 ( 8.642000)
append:ehcache => #<NoMethodError: undefined method `append' for #<ActiveSupport::CacheStoreClient:0xcece5a3>>
append:jmclient:ascii => #<NoMethodError: undefined method `append' for #<MemCache:0x70616389>>
append: spymemcached 13.137000 0.000000 13.137000 ( 13.137000)
prepend: dalli:bin 23.223000 0.000000 23.223000 ( 23.223000)
prepend:ehcache => #<NoMethodError: undefined method `prepend' for #<ActiveSupport::CacheStoreClient:0xcece5a3>>
prepend:jmclient:ascii => #<NoMethodError: undefined method `prepend' for #<MemCache:0x70616389>>
prepend: spymemcached 30.268000 0.000000 30.268000 ( 30.268000)
delete: dalli:bin 6.915000 0.000000 6.915000 ( 6.915000)
delete: ehcache 1.201000 0.000000 1.201000 ( 1.201000)
delete: jmclient:ascii 4.840000 0.000000 4.840000 ( 4.840000)
delete: spymemcached 6.873000 0.000000 6.873000 ( 6.873000)
get-missing: dalli:bin 6.833000 0.000000 6.833000 ( 6.833000)
get-missing: ehcache 0.634000 0.000000 0.634000 ( 0.634000)
get-missing: jmclient:ascii 4.588000 0.000000 4.588000 ( 4.588000)
get-missing: spymemcached 6.321000 0.000000 6.321000 ( 6.321000)
append-missing: dalli:bin 6.922000 0.000000 6.922000 ( 6.922000)
append-missing:ehcache => #<NoMethodError: undefined method `append' for #<ActiveSupport::CacheStoreClient:0xcece5a3>>
append-missing:jmclient:ascii => #<NoMethodError: undefined method `append' for #<MemCache:0x70616389>>
append-missing: spymemcached 6.758000 0.000000 6.758000 ( 6.758000)
prepend-missing: dalli:bin 8.064000 0.000000 8.064000 ( 8.064000)
prepend-missing:ehcache => #<NoMethodError: undefined method `prepend' for #<ActiveSupport::CacheStoreClient:0xcece5a3>>
prepend-missing:jmclient:ascii => #<NoMethodError: undefined method `prepend' for #<MemCache:0x70616389>>
prepend-missing: spymemcached 7.102000 0.000000 7.102000 ( 7.102000)
set-large: dalli:bin 6.734000 0.000000 6.734000 ( 6.734000)
set-large: ehcache 1.516000 0.000000 1.516000 ( 1.516000)
set-large: jmclient:ascii 7.722000 0.000000 7.722000 ( 7.722000)
set-large: spymemcached 9.885000 0.000000 9.885000 ( 9.885000)
get-large: dalli:bin 13.591000 0.000000 13.591000 ( 13.591000)
get-large: ehcache 0.547000 0.000000 0.547000 ( 0.547000)
get-large: jmclient:ascii 9.687000 0.000000 9.687000 ( 9.687000)
get-large: spymemcached 12.305000 0.000000 12.305000 ( 12.305000)
Raw
ruby19-memcached-benchmarks.txt
$ ruby -I../dalli/lib test/profile/benchmark.rb
Darwin vanguard.local 10.4.0 Darwin Kernel Version 10.4.0: Fri Apr 23 18:28:53 PDT 2010; root:xnu-1504.7.4~1/RELEASE_I386 i386
ruby 1.9.2p0
Loops is 20000
Stack depth is 0
Small value size is: 19 bytes
Large value size is: 4189 bytes
No matching processes belonging to you were found
no such file to load -- remix/stash
Could not find RubyGem jruby-memcache-client (>= 0)
no such file to load -- spymemcached
no such file to load -- active_support/cache
user system total real
set: dalli:bin 5.160000 1.300000 6.460000 ( 7.257602)
set: libm:ascii 0.480000 0.780000 1.260000 ( 2.879158)
set: libm:ascii:pipeline 0.140000 0.010000 0.150000 ( 0.170790)
set: libm:ascii:udp 0.410000 0.520000 0.930000 ( 1.908466)
set: libm:bin 0.370000 0.780000 1.150000 ( 2.845852)
set: libm:bin:buffer 0.110000 0.120000 0.230000 ( 0.707210)
set: mclient:ascii 6.300000 1.110000 7.410000 ( 7.798584)
get: dalli:bin 5.630000 1.420000 7.050000 ( 7.485508)
get: libm:ascii 0.560000 0.780000 1.340000 ( 2.988599)
get: libm:ascii:pipeline 0.620000 1.190000 1.810000 ( 3.725585)
get: libm:ascii:udp 0.490000 0.530000 1.020000 ( 1.997827)
get: libm:bin 0.470000 0.780000 1.250000 ( 2.956420)
get: libm:bin:buffer 0.510000 1.110000 1.620000 ( 3.142579)
get: mclient:ascii 7.790000 1.180000 8.970000 ( 9.344592)
get-multi: dalli:bin 6.120000 2.050000 8.170000 ( 8.624984)
get-multi: libm:ascii 0.720000 0.510000 1.230000 ( 2.125837)
get-multi: libm:ascii:pipeline 0.750000 0.530000 1.280000 ( 2.161131)
get-multi: libm:ascii:udp 0.700000 0.430000 1.130000 ( 1.604222)
get-multi: libm:bin 0.660000 0.630000 1.290000 ( 2.699771)
get-multi: libm:bin:buffer 0.680000 0.650000 1.330000 ( 2.698619)
get-multi:mclient:ascii => #<NoMethodError: undefined method `strip' for true:TrueClass>
append: dalli:bin 4.980000 1.330000 6.310000 ( 9.254684)
append: libm:ascii 0.530000 0.800000 1.330000 ( 5.382163)
append: libm:ascii:pipeline 0.180000 0.030000 0.210000 ( 10.685172)
append: libm:ascii:udp 0.480000 0.550000 1.030000 ( 4.759583)
append: libm:bin 0.470000 0.830000 1.300000 ( 12.637884)
append: libm:bin:buffer 0.650000 1.200000 1.850000 ( 19.639100)
append: mclient:ascii 7.110000 1.280000 8.390000 ( 14.282252)
prepend: dalli:bin 5.450000 1.480000 6.930000 ( 11.590226)
prepend: libm:ascii 0.760000 0.920000 1.680000 ( 21.230629)
prepend: libm:ascii:pipeline 0.160000 0.010000 0.170000 ( 0.200603)
prepend: libm:ascii:udp 0.530000 0.590000 1.120000 ( 14.903745)
prepend: libm:bin 0.890000 0.850000 1.740000 ( 3.812533)
prepend: libm:bin:buffer 0.950000 1.100000 2.050000 ( 4.300838)
prepend: mclient:ascii 6.480000 1.140000 7.620000 ( 8.105845)
delete: dalli:bin 5.000000 1.400000 6.400000 ( 6.944967)
delete: libm:ascii 0.900000 0.840000 1.740000 ( 3.614400)
delete: libm:ascii:pipeline 0.230000 0.270000 0.500000 ( 0.709210)
delete: libm:ascii:udp 0.840000 0.590000 1.430000 ( 3.254142)
delete: libm:bin 0.820000 0.890000 1.710000 ( 3.556727)
delete: libm:bin:buffer 0.130000 0.150000 0.280000 ( 0.674715)
delete: mclient:ascii 6.500000 1.130000 7.630000 ( 7.829074)
get-missing: dalli:bin 5.130000 1.360000 6.490000 ( 6.870068)
get-missing: libm:ascii 0.960000 0.860000 1.820000 ( 3.760253)
get-missing: libm:ascii:pipeline 1.000000 1.100000 2.100000 ( 4.325830)
get-missing: libm:ascii:udp 0.880000 0.590000 1.470000 ( 3.262295)
get-missing: libm:bin 0.940000 0.840000 1.780000 ( 3.904349)
get-missing: libm:bin:buffer 0.980000 1.100000 2.080000 ( 4.204083)
get-missing: mclient:ascii 6.300000 1.170000 7.470000 ( 8.267588)
append-missing: dalli:bin 4.960000 1.430000 6.390000 ( 7.709482)
append-missing: libm:ascii 1.010000 0.850000 1.860000 ( 4.380199)
append-missing: libm:ascii:pipeline 0.150000 0.010000 0.160000 ( 0.172640)
append-missing: libm:ascii:udp 0.970000 0.630000 1.600000 ( 2.951318)
append-missing: libm:bin 0.900000 0.830000 1.730000 ( 4.159143)
append-missing: libm:bin:buffer 0.940000 1.100000 2.040000 ( 4.254241)
append-missing: mclient:ascii 6.170000 1.130000 7.300000 ( 8.189961)
prepend-missing: dalli:bin 4.020000 1.250000 5.270000 ( 7.267548)
prepend-missing: libm:ascii 1.010000 0.850000 1.860000 ( 4.059466)
prepend-missing: libm:ascii:pipeline 0.150000 0.010000 0.160000 ( 0.179795)
prepend-missing: libm:ascii:udp 0.940000 0.620000 1.560000 ( 3.550109)
prepend-missing: libm:bin 0.900000 0.860000 1.760000 ( 3.856840)
prepend-missing: libm:bin:buffer 0.930000 0.890000 1.820000 ( 5.293775)
prepend-missing: mclient:ascii 5.530000 1.090000 6.620000 ( 8.162587)
set-large: dalli:bin 5.670000 1.430000 7.100000 ( 8.208441)
set-large: libm:ascii 0.520000 0.820000 1.340000 ( 3.297539)
set-large: libm:ascii:pipeline 0.270000 0.240000 0.510000 ( 0.815074)
set-large: libm:ascii:udp 0.450000 0.590000 1.040000 ( 2.553251)
set-large: libm:bin 0.410000 0.820000 1.230000 ( 2.978832)
set-large: libm:bin:buffer 0.230000 0.400000 0.630000 ( 1.653688)
set-large: mclient:ascii 6.820000 1.190000 8.010000 ( 8.636680)
get-large: dalli:bin 11.400000 1.600000 13.000000 ( 15.237793)
get-large: libm:ascii 0.780000 0.830000 1.610000 ( 5.047453)
get-large: libm:ascii:pipeline 0.840000 1.090000 1.930000 ( 4.747790)
get-large: libm:ascii:udp 0.760000 0.690000 1.450000 ( 2.985629)
get-large: libm:bin 0.680000 0.800000 1.480000 ( 4.980291)
get-large: libm:bin:buffer 0.730000 0.960000 1.690000 ( 5.492779)
get-large: mclient:ascii 9.060000 1.430000 10.490000 ( 11.167996)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment