Skip to content

Instantly share code, notes, and snippets.

@methane
Created April 22, 2015 07:00
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save methane/e15795b9f92471944e6d to your computer and use it in GitHub Desktop.
Save methane/e15795b9f92471944e6d to your computer and use it in GitHub Desktop.

bench

$ numactl -C 16-35 memtier_benchmark -p 2222 -P memcache_binary -n 20000 -t 20 -c 10

go 1.5 / GOMAXPROCS=15

20        Threads
10        Connections per thread
20000     Requests per thread
Type        Ops/sec     Hits/sec   Misses/sec      Latency       KB/sec
------------------------------------------------------------------------
Sets       54294.00          ---          ---      0.33000      4182.58
Gets      542671.38        29.85    542641.54      0.32600     20609.73
Totals    596965.39        29.85    542641.54      0.32600     24792.31

memcached -t15

20        Threads
10        Connections per thread
20000     Requests per thread
Type        Ops/sec     Hits/sec   Misses/sec      Latency       KB/sec
------------------------------------------------------------------------
Sets       69581.25          ---          ---      0.25500      5360.24
Gets      695468.23        74.59    695393.64      0.24500     26412.70
Totals    765049.48        74.59    695393.64      0.24600     31772.94

strace

$ strace -cf numactl -C 0-14 env memcached -p 2222 -t 15
$ numactl -C 16-35 memtier_benchmark -p 2222 -P memcache_binary -n 2000 -t 20 -c 10

20        Threads
10        Connections per thread
2000      Requests per thread
Type        Ops/sec     Hits/sec   Misses/sec      Latency       KB/sec
------------------------------------------------------------------------
Sets        3644.31          ---          ---      5.45700       280.72
Gets       36403.03         0.00     36403.03      4.89300      1382.69
Totals     40047.34         0.00     36403.03      4.94400      1663.41

% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 92.04   14.078867         447     31519           epoll_wait
  3.78    0.577624           1    400000           sendmsg
  3.59    0.549235           1    400463           read
  0.29    0.043748           1     31553           gettimeofday
  0.27    0.040657           1     31607           clock_gettime
  0.03    0.004998          35       143        38 futex
  0.01    0.000859           1       636           mprotect
  0.00    0.000499           2       200           accept
  0.00    0.000351           1       568           fcntl
  0.00    0.000319           1       447           epoll_ctl
  0.00    0.000068          11         6           getsockname
  0.00    0.000041           0       245           write
  0.00    0.000000           0        19         1 open
  0.00    0.000000           0       225           close
  0.00    0.000000           0        15           fstat
  0.00    0.000000           0        72           mmap
  0.00    0.000000           0        18           munmap
  0.00    0.000000           0        13           brk
  0.00    0.000000           0         4           rt_sigaction
  0.00    0.000000           0         1           rt_sigprocmask
  0.00    0.000000           0         3         3 access
  0.00    0.000000           0        15           pipe
  0.00    0.000000           0         1           nanosleep
  0.00    0.000000           0         9           socket
  0.00    0.000000           0         7         1 connect
  0.00    0.000000           0         2           sendto
  0.00    0.000000           0         6           recvmsg
  0.00    0.000000           0         6           bind
  0.00    0.000000           0         2           listen
  0.00    0.000000           0        16           socketpair
  0.00    0.000000           0        68           setsockopt
  0.00    0.000000           0         2           getsockopt
  0.00    0.000000           0        16           clone
  0.00    0.000000           0        26        23 execve
  0.00    0.000000           0         4           getdents
  0.00    0.000000           0         2           getrlimit
  0.00    0.000000           0        49           getuid
  0.00    0.000000           0        48           getgid
  0.00    0.000000           0        49           geteuid
  0.00    0.000000           0        48           getegid
  0.00    0.000000           0         3           arch_prctl
  0.00    0.000000           0         1           setrlimit
  0.00    0.000000           0         1           sched_setaffinity
  0.00    0.000000           0         1           sched_getaffinity
  0.00    0.000000           0        16           epoll_create
  0.00    0.000000           0         1           set_tid_address
  0.00    0.000000           0         2           openat
  0.00    0.000000           0        17           set_robust_list
$ strace -cf numactl -C 0-14 env GOMAXPROCS=15 ./cucache -p 2222
$ numactl -C 16-35 memtier_benchmark -p 2222 -P memcache_binary -n 2000 -t 20 -c 10

20        Threads
10        Connections per thread
2000      Requests per thread
Type        Ops/sec     Hits/sec   Misses/sec      Latency       KB/sec
------------------------------------------------------------------------
Sets        1968.04          ---          ---      9.52300       151.60
Gets       19658.79         0.00     19658.79      9.18600       746.70
Totals     21626.83         0.00     19658.79      9.21700       898.29

% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 64.84   18.655815        8499      2195       157 futex
 29.97    8.622917        4333      1990           select
  2.39    0.687733           1    800267    400050 read
  1.48    0.425496           1    400000           write
  1.23    0.352950           1    406979           clock_gettime
  0.08    0.024254           2     10729           epoll_wait
  0.00    0.000996           5       202           getsockname
  0.00    0.000780           4       207           setsockopt
  0.00    0.000775           4       221           close
  0.00    0.000641           2       402           epoll_ctl
  0.00    0.000578           3       202         2 accept4
  0.00    0.000173           0      1095           sched_yield
  0.00    0.000111           2        56           rt_sigprocmask
  0.00    0.000076           5        15           munmap
  0.00    0.000071           1       120           mmap
  0.00    0.000053           1        41           mprotect
  0.00    0.000000           0        15           open
  0.00    0.000000           0        14           fstat
  0.00    0.000000           0        10           brk
  0.00    0.000000           0        64           rt_sigaction
  0.00    0.000000           0         3         3 access
  0.00    0.000000           0         5           socket
  0.00    0.000000           0         1         1 recvfrom
  0.00    0.000000           0         4           bind
  0.00    0.000000           0         1           listen
  0.00    0.000000           0        18           clone
  0.00    0.000000           0        14        11 execve
  0.00    0.000000           0         4           getdents
  0.00    0.000000           0         1           getrlimit
  0.00    0.000000           0        19           sigaltstack
  0.00    0.000000           0         3           arch_prctl
  0.00    0.000000           0         1           sched_setaffinity
  0.00    0.000000           0         2           sched_getaffinity
  0.00    0.000000           0         1           set_tid_address
  0.00    0.000000           0         3           openat
  0.00    0.000000           0        19           set_robust_list
  0.00    0.000000           0         1           epoll_create1
------ ----------- ----------- --------- --------- ----------------
100.00   28.773419               1624924    400224 total

perf report

  9.65%  cucache  [kernel.kallsyms]  [k] mspin_lock
  9.37%  cucache  [kernel.kallsyms]  [k] pvclock_clocksource_read
  6.17%  cucache  [kernel.kallsyms]  [k] _raw_spin_unlock_irqrestore
  2.26%  cucache  cucache            [.] runtime.cas
  2.18%  cucache  cucache            [.] runtime.xchg
  1.74%  cucache  cucache            [.] runtime.chanrecv
  1.72%  cucache  [kernel.kallsyms]  [k] tcp_sendmsg
  1.31%  cucache  [kernel.kallsyms]  [k] dst_release
  1.22%  cucache  cucache            [.] sync/atomic.CompareAndSwapUint32
  1.09%  cucache  [kernel.kallsyms]  [k] tcp_recvmsg
  1.08%  cucache  cucache            [.] runtime.atomicload
  1.07%  cucache  [kernel.kallsyms]  [k] tcp_ack
  1.01%  cucache  [kernel.kallsyms]  [k] mutex_spin_on_owner
  0.97%  cucache  [kernel.kallsyms]  [k] __fget
  0.94%  cucache  [kernel.kallsyms]  [k] ipv4_dst_check
  0.90%  cucache  cucache            [.] runtime.mallocgc
  0.79%  cucache  [kernel.kallsyms]  [k] __inet_lookup_established
  0.79%  cucache  cucache            [.] runtime.writebarrierptr
  0.72%  cucache  cucache            [.] runtime.chansend
  0.72%  cucache  [kernel.kallsyms]  [k] tcp_transmit_skb
  0.67%  cucache  [kernel.kallsyms]  [k] ip_rcv_finish
  0.67%  cucache  [kernel.kallsyms]  [k] ipv4_mtu
  0.64%  cucache  cucache            [.] sync/atomic.AddUint32
  0.63%  cucache  [kernel.kallsyms]  [k] _raw_spin_lock_bh                                        
  0.60%  cucache  cucache            [.] bufio.(*Reader).fill
  0.58%  cucache  cucache            [.] runtime.goparkunlock
  0.58%  cucache  cucache            [.] sync/atomic.LoadPointer
  0.57%  cucache  cucache            [.] sync/atomic.CompareAndSwapUint64
  0.57%  cucache  [kernel.kallsyms]  [k] skb_release_data
  0.57%  cucache  cucache            [.] runtime.systemstack
  0.56%  cucache  cucache            [.] runtime.atomicstore
  0.56%  cucache  cucache            [.] runtime.deferreturn
  0.56%  cucache  [kernel.kallsyms]  [k] __slab_free
  0.54%  cucache  [kernel.kallsyms]  [k] tcp_wfree
  0.53%  cucache  [kernel.kallsyms]  [k] system_call_after_swapgs
  0.52%  cucache  cucache            [.] runtime.(*waitq).dequeue
  0.52%  cucache  cucache            [.] runtime.schedule
  0.51%  cucache  cucache            [.] runtime.execute
  0.51%  cucache  [kernel.kallsyms]  [k] tcp_write_xmit
  0.48%  cucache  cucache            [.] runtime.xchg64
  0.47%  cucache  cucache            [.] runtime.memmove
  0.47%  cucache  [kernel.kallsyms]  [k] loopback_xmit
  0.47%  cucache  cucache            [.] runtime.scanobject
  0.46%  cucache  [kernel.kallsyms]  [k] _raw_spin_lock
  0.46%  cucache  cucache            [.] github.com/dustin/gomemcached.(*MCRequest).Receive
  0.45%  cucache  cucache            [.] cuckood.(*cmap).kbins
  0.45%  cucache  cucache            [.] net.(*netFD).Read
  0.43%  cucache  [kernel.kallsyms]  [k] sock_def_readable
  0.43%  cucache  cucache            [.] runtime.casgstatus
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment