Skip to content

Instantly share code, notes, and snippets.

@toddlipcon
Created April 11, 2020 00:11
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save toddlipcon/e81fcab79d304fa8fd37401e5b28e4b4 to your computer and use it in GitHub Desktop.
Save toddlipcon/e81fcab79d304fa8fd37401e5b28e4b4 to your computer and use it in GitHub Desktop.
Legend
===================
Methods:
fd: client sends a file descriptor for the server to fill
s-write: server writes data the passed fd using write() syscall
s-mmap: server mmaps() the passed fd and writes using memset()
s-reuse: server holds on to the mmapped region across calls
c-reuse: client holds on to the mmapped region across calls
sidecar: server sends back data over unix socket
nt: server uses nontemporal store instructions to write
NOTE: s-reuse without c-reuse isn't very effective since the server
will detect that it cant actually reuse the mapped buffer, since the
client provided a new one.
Columns designate which core each thread is bound to:
client: the client 'user' thread
c-rx: the client reactor thread
s-rx: the server reactor thread
worker: the server RPC worker thread
Core notation:
SaCbTc: socket a core b thread c
Speed-limit (optimal) performance: 21160.5 MB/sec
method client c-rx s-rx worker MB/sec
# all-same-core
fd(s-write) S0C0T0 S0C0T0 S0C0T0 S0C0T0 2450.2
fd(s-mmap) S0C0T0 S0C0T0 S0C0T0 S0C0T0 1878.1
fd(s-mmap,nt) S0C0T0 S0C0T0 S0C0T0 S0C0T0 1686.5
fd(s-mmap,s-reuse) S0C0T0 S0C0T0 S0C0T0 S0C0T0 1853.4
fd(s-mmap,s-reuse,nt) S0C0T0 S0C0T0 S0C0T0 S0C0T0 1658.4
fd(s-write,c-reuse) S0C0T0 S0C0T0 S0C0T0 S0C0T0 4123.1
fd(s-mmap,c-reuse) S0C0T0 S0C0T0 S0C0T0 S0C0T0 5077.5
fd(s-mmap,c-reuse,nt) S0C0T0 S0C0T0 S0C0T0 S0C0T0 4430.9
fd(s-mmap,c-reuse,s-reuse) S0C0T0 S0C0T0 S0C0T0 S0C0T0 11500.8
fd(s-mmap,c-reuse,s-reuse,nt) S0C0T0 S0C0T0 S0C0T0 S0C0T0 8588.5
sidecar S0C0T0 S0C0T0 S0C0T0 S0C0T0 4278.3
sidecar,nt S0C0T0 S0C0T0 S0C0T0 S0C0T0 3478.5
# client-server-cross-core
fd(s-write) S0C0T0 S0C0T0 S0C1T0 S0C1T0 2030.6
fd(s-mmap) S0C0T0 S0C0T0 S0C1T0 S0C1T0 1654.6
fd(s-mmap,nt) S0C0T0 S0C0T0 S0C1T0 S0C1T0 1476.4
fd(s-mmap,s-reuse) S0C0T0 S0C0T0 S0C1T0 S0C1T0 1686.9
fd(s-mmap,s-reuse,nt) S0C0T0 S0C0T0 S0C1T0 S0C1T0 1500.7
fd(s-write,c-reuse) S0C0T0 S0C0T0 S0C1T0 S0C1T0 3675.0
fd(s-mmap,c-reuse) S0C0T0 S0C0T0 S0C1T0 S0C1T0 4439.1
fd(s-mmap,c-reuse,nt) S0C0T0 S0C0T0 S0C1T0 S0C1T0 3751.0
fd(s-mmap,c-reuse,s-reuse) S0C0T0 S0C0T0 S0C1T0 S0C1T0 9415.6
fd(s-mmap,c-reuse,s-reuse,nt) S0C0T0 S0C0T0 S0C1T0 S0C1T0 7274.6
sidecar S0C0T0 S0C0T0 S0C1T0 S0C1T0 3597.9
sidecar,nt S0C0T0 S0C0T0 S0C1T0 S0C1T0 3178.8
# client-server-cross-socket
WARNING: Logging before InitGoogleLogging() is written to STDERR
W0410 17:10:56.228780 25498 rpc_stub-test.cc:1065] Local system doesn't have appropriate CPU topology to execute test scenario client-server-cross-socket
# every-hop-crosses-core
fd(s-write) S0C0T0 S0C1T0 S0C2T0 S0C3T0 1857.5
fd(s-mmap) S0C0T0 S0C1T0 S0C2T0 S0C3T0 1517.3
fd(s-mmap,nt) S0C0T0 S0C1T0 S0C2T0 S0C3T0 1399.2
fd(s-mmap,s-reuse) S0C0T0 S0C1T0 S0C2T0 S0C3T0 1532.1
fd(s-mmap,s-reuse,nt) S0C0T0 S0C1T0 S0C2T0 S0C3T0 1392.6
fd(s-write,c-reuse) S0C0T0 S0C1T0 S0C2T0 S0C3T0 3264.1
fd(s-mmap,c-reuse) S0C0T0 S0C1T0 S0C2T0 S0C3T0 3810.8
fd(s-mmap,c-reuse,nt) S0C0T0 S0C1T0 S0C2T0 S0C3T0 3294.6
fd(s-mmap,c-reuse,s-reuse) S0C0T0 S0C1T0 S0C2T0 S0C3T0 8835.5
fd(s-mmap,c-reuse,s-reuse,nt) S0C0T0 S0C1T0 S0C2T0 S0C3T0 6584.7
sidecar S0C0T0 S0C1T0 S0C2T0 S0C3T0 3334.0
sidecar,nt S0C0T0 S0C1T0 S0C2T0 S0C3T0 2789.9
# every-hop-crosses-socket
W0410 17:11:01.191898 25498 rpc_stub-test.cc:1065] Local system doesn't have appropriate CPU topology to execute test scenario every-hop-crosses-socket
[ OK ] RpcStubTestU
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment