Skip to content

Instantly share code, notes, and snippets.

@shouyu
Created August 30, 2012 08:46
Embed
What would you like to do?
dalli vs memcached

memcachedクライアントdalliとmemcached gemの調査

memcachedのRubyクライアントである、dallimemcached gemについての調査です

dalli

https://github.com/mperham/dalli

特徴

  • pure Ruby
  • memcache-clientの後継として開発
  • フェイルオーバー機能
  • memcached 1.4+のみサポート
  • バイナリプロトコルのみサポート
  • kgio gemをインストールするとパフォーマンスが上がるらしい
  • シンプル

オプション

github参照

フェイルオーバー

failoverオプションがtrueになっていると

dc = Dalli::Client.new ['localhost:11211', 'localhost:11212']

のようにコンストラクタに複数のアドレスを渡すと自動で分散しタイムアウト等の時に自動でフェイルオーバーします。
全サーバから反応がないとDalli::RingError例外を吐いてしぬ。
https://github.com/mperham/dalli/blob/master/test/test_failover.rb

kgio gem

kgio gemをインストールすると10-20%ほどパフォーマンスが上がるらしい。
ベンチマークにkgio有り無しで載せてます。

memcached gem

https://github.com/evan/memcached

特徴

  • Cで実装されたlibmemcachedのRubyインターフェース
  • Cであるが故高速
  • memcachedのバージョン縛りなし
  • バイナリプロトコルだけでなく従来のテキストプロトコルもサポート
  • Consistent Hashingで分散
  • ノンブロッキングな操作が可能
  • 付属のlibmemcachedが古い(0.3.2が付属されているが最新が1.0.10)

オプション

ドキュメント参照

フェイルオーバー

libmemcached側で実装されている。
dalliとほぼ同じ挙動。

ベンチマーク

dalli(w/o kgio), memcached gem, memcache-client

Linux SVZ1311AJ 3.5.3-1-ARCH #1 SMP PREEMPT Sun Aug 26 09:14:51 CEST 2012 x86_64 GNU/Linux
ruby 1.9.2p290 (2011-07-09 revision 32553) [x86_64-linux]

Ruby 1.9.2p290
Loaded memcached 1.4.3
Loaded remix-stash 1.1.3
Loaded memcache-client 1.8.5
Loaded dalli 2.1.0
PID is 20388
Loops is 50000
Stack depth is 0
Small value size is: 19 bytes
Large value size is: 4189 bytes
                                          user     system      total        real
set: dalli:bin                        9.040000   2.130000  11.170000 ( 11.196209)
set: libm:ascii                       1.930000   2.850000   4.780000 (  7.445127)
set: libm:ascii:pipeline              0.230000   0.010000   0.240000 (  0.232692)
set: libm:ascii:udp                   2.070000   1.620000   3.690000 (  5.485257)
set: libm:bin                         1.300000   2.950000   4.250000 (  6.577973)
set: libm:bin:buffer                  0.230000   0.200000   0.430000 (  0.553087)
set: mclient:ascii                    8.750000   1.990000  10.740000 ( 10.774039)

get: dalli:bin                        9.690000   1.800000  11.490000 ( 11.517524)
get: libm:ascii                       1.980000   3.030000   5.010000 (  6.799963)
get: libm:ascii:pipeline              2.390000   2.620000   5.010000 (  6.933103)
get: libm:ascii:udp                   2.290000   1.650000   3.940000 (  5.106576)
get: libm:bin                         2.120000   5.590000   7.710000 ( 10.394310)
get: libm:bin:buffer                  2.590000   3.940000   6.530000 (  9.314762)
get: mclient:ascii                   10.020000   1.590000  11.610000 ( 11.635575)

get-multi: dalli:bin                  5.270000   2.360000   7.630000 (  7.650404)
get-multi: libm:ascii                 0.980000   0.960000   1.940000 (  2.090035)
get-multi: libm:ascii:pipeline        0.960000   0.770000   1.730000 (  1.812628)
get-multi: libm:ascii:udp             1.580000   0.510000   2.090000 (  2.564980)
get-multi: libm:bin                   0.580000   0.970000   1.550000 (  1.574117)
get-multi: libm:bin:buffer            0.590000   1.010000   1.600000 (  1.608090)

append: dalli:bin                    16.950000   3.940000  20.890000 ( 32.805226)
append: libm:ascii                    3.030000   4.230000   7.260000 ( 29.335895)
append: libm:ascii:pipeline           0.250000   0.000000   0.250000 (  6.048300)
append: libm:ascii:udp                2.180000   1.560000   3.740000 ( 18.613141)
append: libm:bin                      2.490000   4.390000   6.880000 ( 28.505384)
append: libm:bin:buffer               0.760000   0.590000   1.350000 (  7.551046)
append: mclient:ascii                18.800000   5.350000  24.150000 ( 36.332188)

prepend: dalli:bin                   18.200000   4.080000  22.280000 ( 34.818036)
prepend: libm:ascii                   3.000000   4.140000   7.140000 ( 29.462726)
prepend: libm:ascii:pipeline          0.250000   0.000000   0.250000 (  6.011693)
prepend: libm:ascii:udp               2.580000   1.700000   4.280000 ( 21.974421)
prepend: libm:bin                     2.630000   4.250000   6.880000 ( 29.213027)
prepend: libm:bin:buffer              0.760000   0.630000   1.390000 (  7.460614)
prepend: mclient:ascii               18.370000   4.500000  22.870000 ( 35.607866)

delete: dalli:bin                     8.180000   1.560000   9.740000 (  9.753448)
delete: libm:ascii                    3.000000   3.400000   6.400000 (  8.213886)
delete: libm:ascii:pipeline           0.140000   0.000000   0.140000 (  0.142411)
delete: libm:ascii:udp                3.340000   1.820000   5.160000 (  6.314937)
delete: libm:bin                      3.310000   3.140000   6.450000 (  8.266048)
delete: mclient:ascii                 7.960000   1.790000   9.750000 (  9.780266)

get-missing: dalli:bin                8.300000   1.630000   9.930000 (  9.949946)
get-missing: libm:ascii               3.610000   3.240000   6.850000 (  8.536922)
get-missing: libm:ascii:pipeline      4.000000   2.990000   6.990000 (  8.778857)
get-missing: libm:ascii:udp           3.680000   1.780000   5.460000 (  6.532425)
get-missing: libm:bin                 4.030000   5.800000   9.830000 ( 12.541615)
get-missing: libm:bin:buffer          4.380000   4.290000   8.670000 ( 11.282694)
get-missing: mclient:ascii            8.240000   1.600000   9.840000 (  9.857652)

append-missing: dalli:bin             8.550000   1.490000  10.040000 ( 10.070304)
append-missing: libm:ascii            3.910000   3.470000   7.380000 ( 10.118953)
append-missing: libm:ascii:pipeline   0.250000   0.030000   0.280000 (  0.278553)
append-missing: libm:ascii:udp        3.660000   1.630000   5.290000 (  6.827256)
append-missing: libm:bin              3.360000   3.300000   6.660000 (  9.012879)
append-missing: libm:bin:buffer       3.790000   2.930000   6.720000 (  9.175111)
append-missing: mclient:ascii         9.060000   1.630000  10.690000 ( 10.736789)

prepend-missing: dalli:bin            8.740000   1.470000  10.210000 ( 10.235104)
prepend-missing: libm:ascii           3.840000   3.150000   6.990000 (  9.624539)
prepend-missing: libm:ascii:pipeline  0.240000   0.010000   0.250000 (  0.243136)
prepend-missing: libm:ascii:udp       3.970000   1.710000   5.680000 (  7.345762)
prepend-missing: libm:bin             3.340000   3.190000   6.530000 (  8.869057)
prepend-missing: libm:bin:buffer      3.790000   2.930000   6.720000 (  9.169671)
prepend-missing: mclient:ascii        8.470000   1.550000  10.020000 ( 10.061517)

set-large: dalli:bin                  9.180000   1.720000  10.900000 ( 10.935889)
set-large: libm:ascii                 1.940000   3.190000   5.130000 (  8.121204)
set-large: libm:ascii:pipeline        0.370000   0.270000   0.640000 (  0.645946)
set-large: libm:ascii:udp             2.240000   1.700000   3.940000 (  5.969623)
set-large: libm:bin                   1.470000   3.050000   4.520000 (  7.242417)
set-large: libm:bin:buffer            0.300000   0.510000   0.810000 (  0.889591)
set-large: mclient:ascii              8.870000   1.810000  10.680000 ( 10.730371)

get-large: dalli:bin                  9.450000   1.680000  11.130000 ( 11.150904)
get-large: libm:ascii                 1.990000   2.960000   4.950000 (  6.705611)
get-large: libm:ascii:pipeline        2.320000   2.820000   5.140000 (  7.121528)
get-large: libm:ascii:udp             2.290000   1.690000   3.980000 (  5.131094)
get-large: libm:bin                   2.250000   5.610000   7.860000 ( 10.655049)
get-large: libm:bin:buffer            2.730000   3.990000   6.720000 (  9.449588)
get-large: mclient:ascii             10.190000   1.510000  11.700000 ( 11.729039)

hash:default                          0.360000   0.000000   0.360000 (  0.350394)
hash:md5                              0.650000   0.000000   0.650000 (  0.639677)
hash:crc                              0.500000   0.000000   0.500000 (  0.499077)
hash:fnv1_64                          0.330000   0.000000   0.330000 (  0.315859)
hash:fnv1a_64                         0.290000   0.000000   0.290000 (  0.289789)
hash:fnv1_32                          0.310000   0.000000   0.310000 (  0.297607)
hash:fnv1a_32                         0.300000   0.000000   0.300000 (  0.288302)
hash:hsieh                            0.120000   0.000000   0.120000 (  0.123450)
hash:murmur                           0.180000   0.000000   0.180000 (  0.171247)
hash:jenkins                          0.200000   0.000000   0.200000 (  0.195083)
hash:none                             0.120000   0.000000   0.120000 (  0.121242)

dalli(w/ kgio), memcached gem, memcache-client

Linux SVZ1311AJ 3.5.3-1-ARCH #1 SMP PREEMPT Sun Aug 26 09:14:51 CEST 2012 x86_64 GNU/Linux
ruby 1.9.2p290 (2011-07-09 revision 32553) [x86_64-linux]

Ruby 1.9.2p290
Loaded memcached 1.4.3
Loaded remix-stash 1.1.3
Loaded memcache-client 1.8.5
Loaded kgio 2.7.4
Loaded dalli 2.1.0
PID is 1322
Loops is 50000
Stack depth is 0
Small value size is: 19 bytes
Large value size is: 4189 bytes
memcached: no process found
                                          user     system      total        real
set: dalli:bin                        6.560000   2.610000   9.170000 ( 10.072084)
set: libm:ascii                       1.800000   2.920000   4.720000 (  7.402876)
set: libm:ascii:pipeline              0.230000   0.000000   0.230000 (  0.226410)
set: libm:ascii:udp                   1.800000   1.800000   3.600000 (  5.433554)
set: libm:bin                         1.340000   2.800000   4.140000 (  6.488439)
set: libm:bin:buffer                  0.230000   0.200000   0.430000 (  0.553482)
set: mclient:ascii                    8.820000   2.040000  10.860000 ( 10.889803)

get: dalli:bin                        5.240000   1.870000   7.110000 (  7.642416)
get: libm:ascii                       1.830000   2.970000   4.800000 (  6.614017)
get: libm:ascii:pipeline              1.770000   2.770000   4.540000 (  6.375964)
get: libm:ascii:udp                   2.060000   1.730000   3.790000 (  4.975936)
get: libm:bin                         2.050000   5.250000   7.300000 ( 10.050144)
get: libm:bin:buffer                  1.950000   4.180000   6.130000 (  8.951114)
get: mclient:ascii                   10.200000   1.680000  11.880000 ( 11.906878)

get-multi: dalli:bin                  5.360000   2.110000   7.470000 (  7.482966)
get-multi: libm:ascii                 0.960000   0.710000   1.670000 (  1.797262)
get-multi: libm:ascii:pipeline        0.940000   0.810000   1.750000 (  1.893964)
get-multi: libm:ascii:udp             1.530000   0.620000   2.150000 (  2.681157)
get-multi: libm:bin                   0.650000   0.930000   1.580000 (  1.604287)
get-multi: libm:bin:buffer            0.520000   0.990000   1.510000 (  1.569208)

append: dalli:bin                     8.970000   4.830000  13.800000 ( 35.167700)
append: libm:ascii                    2.930000   4.450000   7.380000 ( 29.979601)
append: libm:ascii:pipeline           0.230000   0.010000   0.240000 (  6.131263)
append: libm:ascii:udp                2.080000   1.660000   3.740000 ( 19.110218)
append: libm:bin                      2.320000   4.140000   6.460000 ( 28.034097)
append: libm:bin:buffer               0.800000   0.850000   1.650000 (  7.499054)
append: mclient:ascii                19.260000   4.030000  23.290000 ( 35.076128)

prepend: dalli:bin                    8.680000   4.730000  13.410000 ( 34.824494)
prepend: libm:ascii                   2.680000   3.400000   6.080000 ( 26.188562)
prepend: libm:ascii:pipeline          0.230000   0.010000   0.240000 (  6.023637)
prepend: libm:ascii:udp               2.190000   1.520000   3.710000 ( 20.058748)
prepend: libm:bin                     2.410000   4.440000   6.850000 ( 29.347241)
prepend: libm:bin:buffer              0.740000   0.660000   1.400000 (  7.605870)
prepend: mclient:ascii               19.040000   3.990000  23.030000 ( 35.439866)

delete: dalli:bin                     3.780000   1.700000   5.480000 (  5.837672)
delete: libm:ascii                    3.130000   3.100000   6.230000 (  8.062277)
delete: libm:ascii:pipeline           0.140000   0.000000   0.140000 (  0.141828)
delete: libm:ascii:udp                3.210000   1.730000   4.940000 (  6.102269)
delete: libm:bin                      2.920000   3.160000   6.080000 (  7.819175)
delete: mclient:ascii                 8.350000   1.340000   9.690000 (  9.718666)

get-missing: dalli:bin                3.940000   1.580000   5.520000 (  5.921695)
get-missing: libm:ascii               3.340000   3.230000   6.570000 (  8.265316)
get-missing: libm:ascii:pipeline      3.620000   2.940000   6.560000 (  8.277519)
get-missing: libm:ascii:udp           3.530000   1.780000   5.310000 (  6.417759)
get-missing: libm:bin                 3.770000   5.770000   9.540000 ( 12.253769)
get-missing: libm:bin:buffer          3.940000   4.510000   8.450000 ( 11.187080)
get-missing: mclient:ascii            8.100000   1.450000   9.550000 (  9.559223)

append-missing: dalli:bin             7.470000   3.320000  10.790000 ( 11.899947)
append-missing: libm:ascii            3.920000   3.190000   7.110000 (  9.810621)
append-missing: libm:ascii:pipeline   0.280000   0.010000   0.290000 (  0.295614)
append-missing: libm:ascii:udp        3.720000   1.780000   5.500000 (  7.187822)
append-missing: libm:bin              3.500000   3.180000   6.680000 (  8.955223)
append-missing: libm:bin:buffer       3.260000   3.050000   6.310000 (  8.596548)
append-missing: mclient:ascii         8.880000   1.550000  10.430000 ( 10.472102)

prepend-missing: dalli:bin            7.050000   3.320000  10.370000 ( 11.589129)
prepend-missing: libm:ascii           3.540000   3.270000   6.810000 (  9.304082)
prepend-missing: libm:ascii:pipeline  0.220000   0.010000   0.230000 (  0.234633)
prepend-missing: libm:ascii:udp       3.650000   1.810000   5.460000 (  7.149561)
prepend-missing: libm:bin             3.090000   3.080000   6.170000 (  8.468167)
prepend-missing: libm:bin:buffer      3.380000   3.250000   6.630000 (  9.142553)
prepend-missing: mclient:ascii        8.620000   1.390000  10.010000 ( 10.044911)

set-large: dalli:bin                  9.480000   3.770000  13.250000 ( 14.850681)
set-large: libm:ascii                 2.070000   3.030000   5.100000 (  8.164950)
set-large: libm:ascii:pipeline        0.430000   0.250000   0.680000 (  0.684733)
set-large: libm:ascii:udp             2.100000   1.740000   3.840000 (  5.915178)
set-large: libm:bin                   1.500000   3.030000   4.530000 (  7.280462)
set-large: libm:bin:buffer            0.320000   0.560000   0.880000 (  1.083350)
set-large: mclient:ascii              9.090000   1.740000  10.830000 ( 10.883003)

get-large: dalli:bin                  4.910000   1.710000   6.620000 (  7.057092)
get-large: libm:ascii                 1.970000   2.790000   4.760000 (  6.517797)
get-large: libm:ascii:pipeline        2.020000   2.850000   4.870000 (  6.825777)
get-large: libm:ascii:udp             2.000000   1.640000   3.640000 (  4.796084)
get-large: libm:bin                   1.960000   5.030000   6.990000 (  9.676027)
get-large: libm:bin:buffer            2.030000   4.250000   6.280000 (  9.137414)
get-large: mclient:ascii             10.290000   1.520000  11.810000 ( 11.830625)

hash:default                          0.360000   0.000000   0.360000 (  0.351799)
hash:md5                              0.650000   0.000000   0.650000 (  0.633761)
hash:crc                              0.510000   0.000000   0.510000 (  0.496620)
hash:fnv1_64                          0.330000   0.000000   0.330000 (  0.317144)
hash:fnv1a_64                         0.300000   0.000000   0.300000 (  0.290220)
hash:fnv1_32                          0.300000   0.000000   0.300000 (  0.299574)
hash:fnv1a_32                         0.300000   0.000000   0.300000 (  0.288803)
hash:hsieh                            0.130000   0.000000   0.130000 (  0.124781)
hash:murmur                           0.170000   0.000000   0.170000 (  0.171596)
hash:jenkins                          0.200000   0.000000   0.200000 (  0.196605)
hash:none                             0.130000   0.000000   0.130000 (  0.121153)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment