88-core.txt

Legend
===================
Methods:
  fd:      client sends a file descriptor for the server to fill
  s-write: server writes data the passed fd using write() syscall
  s-mmap:  server mmaps() the passed fd and writes using memset()
  s-reuse: server holds on to the mmapped region across calls
  c-reuse: client holds on to the mmapped region across calls
  sidecar: server sends back data over unix socket
  nt:      server uses nontemporal store instructions to write

NOTE: s-reuse without c-reuse isn't very effective since the server
will detect that it cant actually reuse the mapped buffer, since the
client provided a new one.
Columns designate which core each thread is bound to:
  client: the client 'user' thread
  c-rx:   the client reactor thread
  s-rx:   the server reactor thread
  worker: the server RPC worker thread

Core notation:
  SaCbTc: socket a core b thread c

Speed-limit (optimal) performance: 7875.5 MB/sec
                        method client   c-rx   s-rx worker	MB/sec
# all-same-core
                   fd(s-write) S0C0T0 S0C0T0 S0C0T0 S0C0T0	1468.0
                    fd(s-mmap) S0C0T0 S0C0T0 S0C0T0 S0C0T0	1298.2
                 fd(s-mmap,nt) S0C0T0 S0C0T0 S0C0T0 S0C0T0	1063.5
            fd(s-mmap,s-reuse) S0C0T0 S0C0T0 S0C0T0 S0C0T0	1285.9
         fd(s-mmap,s-reuse,nt) S0C0T0 S0C0T0 S0C0T0 S0C0T0	1057.0
           fd(s-write,c-reuse) S0C0T0 S0C0T0 S0C0T0 S0C0T0	3215.6
            fd(s-mmap,c-reuse) S0C0T0 S0C0T0 S0C0T0 S0C0T0	3128.2
         fd(s-mmap,c-reuse,nt) S0C0T0 S0C0T0 S0C0T0 S0C0T0	2190.2
    fd(s-mmap,c-reuse,s-reuse) S0C0T0 S0C0T0 S0C0T0 S0C0T0	6511.3
 fd(s-mmap,c-reuse,s-reuse,nt) S0C0T0 S0C0T0 S0C0T0 S0C0T0	3840.7
                       sidecar S0C0T0 S0C0T0 S0C0T0 S0C0T0	2372.9
                    sidecar,nt S0C0T0 S0C0T0 S0C0T0 S0C0T0	1848.8
# client-server-cross-core
                   fd(s-write) S0C0T0 S0C0T0 S0C1T0 S0C1T0	1421.3
                    fd(s-mmap) S0C0T0 S0C0T0 S0C1T0 S0C1T0	1280.5
                 fd(s-mmap,nt) S0C0T0 S0C0T0 S0C1T0 S0C1T0	1069.8
            fd(s-mmap,s-reuse) S0C0T0 S0C0T0 S0C1T0 S0C1T0	1330.4
         fd(s-mmap,s-reuse,nt) S0C0T0 S0C0T0 S0C1T0 S0C1T0	1107.5
           fd(s-write,c-reuse) S0C0T0 S0C0T0 S0C1T0 S0C1T0	2752.2
            fd(s-mmap,c-reuse) S0C0T0 S0C0T0 S0C1T0 S0C1T0	3155.8
         fd(s-mmap,c-reuse,nt) S0C0T0 S0C0T0 S0C1T0 S0C1T0	2293.9
    fd(s-mmap,c-reuse,s-reuse) S0C0T0 S0C0T0 S0C1T0 S0C1T0	5447.5
 fd(s-mmap,c-reuse,s-reuse,nt) S0C0T0 S0C0T0 S0C1T0 S0C1T0	3923.9
                       sidecar S0C0T0 S0C0T0 S0C1T0 S0C1T0	2091.5
                    sidecar,nt S0C0T0 S0C0T0 S0C1T0 S0C1T0	1556.7
# client-server-cross-socket
                   fd(s-write) S0C0T0 S0C0T0 S1C0T0 S1C0T0	733.9
                    fd(s-mmap) S0C0T0 S0C0T0 S1C0T0 S1C0T0	699.5
                 fd(s-mmap,nt) S0C0T0 S0C0T0 S1C0T0 S1C0T0	687.3
            fd(s-mmap,s-reuse) S0C0T0 S0C0T0 S1C0T0 S1C0T0	696.6
         fd(s-mmap,s-reuse,nt) S0C0T0 S0C0T0 S1C0T0 S1C0T0	667.4
           fd(s-write,c-reuse) S0C0T0 S0C0T0 S1C0T0 S1C0T0	1140.5
            fd(s-mmap,c-reuse) S0C0T0 S0C0T0 S1C0T0 S1C0T0	1482.0
         fd(s-mmap,c-reuse,nt) S0C0T0 S0C0T0 S1C0T0 S1C0T0	1740.1
    fd(s-mmap,c-reuse,s-reuse) S0C0T0 S0C0T0 S1C0T0 S1C0T0	2088.4
 fd(s-mmap,c-reuse,s-reuse,nt) S0C0T0 S0C0T0 S1C0T0 S1C0T0	3534.9
                       sidecar S0C0T0 S0C0T0 S1C0T0 S1C0T0	1015.4
                    sidecar,nt S0C0T0 S0C0T0 S1C0T0 S1C0T0	880.4
# every-hop-crosses-core
                   fd(s-write) S0C0T0 S0C1T0 S0C2T0 S0C3T0	1240.7
                    fd(s-mmap) S0C0T0 S0C1T0 S0C2T0 S0C3T0	1116.0
                 fd(s-mmap,nt) S0C0T0 S0C1T0 S0C2T0 S0C3T0	1108.1
            fd(s-mmap,s-reuse) S0C0T0 S0C1T0 S0C2T0 S0C3T0	1169.5
         fd(s-mmap,s-reuse,nt) S0C0T0 S0C1T0 S0C2T0 S0C3T0	986.3
           fd(s-write,c-reuse) S0C0T0 S0C1T0 S0C2T0 S0C3T0	1967.4
            fd(s-mmap,c-reuse) S0C0T0 S0C1T0 S0C2T0 S0C3T0	2027.6
         fd(s-mmap,c-reuse,nt) S0C0T0 S0C1T0 S0C2T0 S0C3T0	1963.3
    fd(s-mmap,c-reuse,s-reuse) S0C0T0 S0C1T0 S0C2T0 S0C3T0	4756.7
 fd(s-mmap,c-reuse,s-reuse,nt) S0C0T0 S0C1T0 S0C2T0 S0C3T0	3227.4
                       sidecar S0C0T0 S0C1T0 S0C2T0 S0C3T0	1525.4
                    sidecar,nt S0C0T0 S0C1T0 S0C2T0 S0C3T0	1278.6
# every-hop-crosses-socket
                   fd(s-write) S0C0T0 S1C0T0 S0C0T0 S1C0T0	772.7
                    fd(s-mmap) S0C0T0 S1C0T0 S0C0T0 S1C0T0	688.8
                 fd(s-mmap,nt) S0C0T0 S1C0T0 S0C0T0 S1C0T0	678.7
            fd(s-mmap,s-reuse) S0C0T0 S1C0T0 S0C0T0 S1C0T0	677.4
         fd(s-mmap,s-reuse,nt) S0C0T0 S1C0T0 S0C0T0 S1C0T0	654.8
           fd(s-write,c-reuse) S0C0T0 S1C0T0 S0C0T0 S1C0T0	1158.5
            fd(s-mmap,c-reuse) S0C0T0 S1C0T0 S0C0T0 S1C0T0	1536.6
         fd(s-mmap,c-reuse,nt) S0C0T0 S1C0T0 S0C0T0 S1C0T0	1922.3
    fd(s-mmap,c-reuse,s-reuse) S0C0T0 S1C0T0 S0C0T0 S1C0T0	2191.7
 fd(s-mmap,c-reuse,s-reuse,nt) S0C0T0 S1C0T0 S0C0T0 S1C0T0	3924.0
                       sidecar S0C0T0 S1C0T0 S0C0T0 S1C0T0	676.8
                    sidecar,nt S0C0T0 S1C0T0 S0C0T0 S1C0T0	725.5