Skip to content

Instantly share code, notes, and snippets.

@toddlipcon
Created April 14, 2020 23:08
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save toddlipcon/28193dd2d2017bba2aaebdea78bcee6a to your computer and use it in GitHub Desktop.
Save toddlipcon/28193dd2d2017bba2aaebdea78bcee6a to your computer and use it in GitHub Desktop.
Legend
===================
Methods:
fd: client sends a file descriptor for the server to fill
s-write: server writes data the passed fd using write() syscall
s-mmap: server mmaps() the passed fd and writes using memset()
s-reuse: server holds on to the mmapped region across calls
c-reuse: client holds on to the mmapped region across calls
sidecar: server sends back data over unix socket
nt: server uses nontemporal store instructions to write
NOTE: s-reuse without c-reuse isn't very effective since the server
will detect that it cant actually reuse the mapped buffer, since the
client provided a new one.
Columns designate which core each thread is bound to:
client: the client 'user' thread
c-rx: the client reactor thread
s-rx: the server reactor thread
worker: the server RPC worker thread
Core notation:
SaCbTc: socket a core b thread c
Speed-limit (optimal) performance: 18563.2 MB/sec
method client c-rx s-rx worker MB/sec
# all-same-core
fd(s-write) S0C0T0 S0C0T0 S0C0T0 S0C0T0 2545.3
fd(s-mmap) S0C0T0 S0C0T0 S0C0T0 S0C0T0 1934.5
fd(s-mmap,nt) S0C0T0 S0C0T0 S0C0T0 S0C0T0 1739.7
fd(s-mmap,s-reuse) S0C0T0 S0C0T0 S0C0T0 S0C0T0 1973.7
fd(s-mmap,s-reuse,nt) S0C0T0 S0C0T0 S0C0T0 S0C0T0 1731.9
fd(s-write,c-reuse) S0C0T0 S0C0T0 S0C0T0 S0C0T0 5158.5
fd(s-mmap,c-reuse) S0C0T0 S0C0T0 S0C0T0 S0C0T0 6196.4
fd(s-mmap,c-reuse,nt) S0C0T0 S0C0T0 S0C0T0 S0C0T0 4987.3
fd(s-mmap,c-reuse,s-reuse) S0C0T0 S0C0T0 S0C0T0 S0C0T0 13441.2
fd(s-mmap,c-reuse,s-reuse,nt) S0C0T0 S0C0T0 S0C0T0 S0C0T0 9437.3
sidecar S0C0T0 S0C0T0 S0C0T0 S0C0T0 4556.2
sidecar,nt S0C0T0 S0C0T0 S0C0T0 S0C0T0 3736.6
# client-server-cross-core
fd(s-write) S0C0T0 S0C0T0 S0C1T0 S0C1T0 2143.1
fd(s-mmap) S0C0T0 S0C0T0 S0C1T0 S0C1T0 1686.6
fd(s-mmap,nt) S0C0T0 S0C0T0 S0C1T0 S0C1T0 1549.3
fd(s-mmap,s-reuse) S0C0T0 S0C0T0 S0C1T0 S0C1T0 1729.0
fd(s-mmap,s-reuse,nt) S0C0T0 S0C0T0 S0C1T0 S0C1T0 1538.6
fd(s-write,c-reuse) S0C0T0 S0C0T0 S0C1T0 S0C1T0 4228.6
fd(s-mmap,c-reuse) S0C0T0 S0C0T0 S0C1T0 S0C1T0 5296.2
fd(s-mmap,c-reuse,nt) S0C0T0 S0C0T0 S0C1T0 S0C1T0 4188.3
fd(s-mmap,c-reuse,s-reuse) S0C0T0 S0C0T0 S0C1T0 S0C1T0 10142.7
fd(s-mmap,c-reuse,s-reuse,nt) S0C0T0 S0C0T0 S0C1T0 S0C1T0 7642.3
sidecar S0C0T0 S0C0T0 S0C1T0 S0C1T0 3755.6
sidecar,nt S0C0T0 S0C0T0 S0C1T0 S0C1T0 3142.8
# client-server-cross-socket
WARNING: Logging before InitGoogleLogging() is written to STDERR
W0414 16:06:58.495776 26892 rpc_stub-test.cc:1094] Local system doesn't have appropriate CPU topology to execute test scenario client-server-cross-socket
# every-hop-crosses-core
fd(s-write) S0C0T0 S0C1T0 S0C2T0 S0C3T0 1835.6
fd(s-mmap) S0C0T0 S0C1T0 S0C2T0 S0C3T0 1527.8
fd(s-mmap,nt) S0C0T0 S0C1T0 S0C2T0 S0C3T0 1369.6
fd(s-mmap,s-reuse) S0C0T0 S0C1T0 S0C2T0 S0C3T0 1551.2
fd(s-mmap,s-reuse,nt) S0C0T0 S0C1T0 S0C2T0 S0C3T0 1403.2
fd(s-write,c-reuse) S0C0T0 S0C1T0 S0C2T0 S0C3T0 3412.2
fd(s-mmap,c-reuse) S0C0T0 S0C1T0 S0C2T0 S0C3T0 4813.7
fd(s-mmap,c-reuse,nt) S0C0T0 S0C1T0 S0C2T0 S0C3T0 3448.4
fd(s-mmap,c-reuse,s-reuse) S0C0T0 S0C1T0 S0C2T0 S0C3T0 9348.1
fd(s-mmap,c-reuse,s-reuse,nt) S0C0T0 S0C1T0 S0C2T0 S0C3T0 7051.8
sidecar S0C0T0 S0C1T0 S0C2T0 S0C3T0 3289.8
sidecar,nt S0C0T0 S0C1T0 S0C2T0 S0C3T0 2836.2
# every-hop-crosses-socket
W0414 16:07:46.795485 26892 rpc_stub-test.cc:1094] Local system doesn't have appropriate CPU topology to execute test scenario every-hop-crosses-socket
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment