Hiredis nested bulk benchmarking

benchmarks.md

System info

$ uname -a
Linux towerlinux 4.15.0-96-generic #97-Ubuntu SMP Wed Apr 1 03:25:46 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
$ lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              8
On-line CPU(s) list: 0-7
Thread(s) per core:  2
Core(s) per socket:  4
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
CPU family:          6
Model:               158
Model name:          Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz
[...]

Bulk reply, random leaves

Generation command:

php resp-protogen/proto.php --count 1000000 --type multibulk --bulkleaf rand >| /tmp/mbk.1m.proto

Output:

$ hyperfine --warmup 10 "./raw-bench-master /tmp/mbk.1m.proto" "./raw-bench-unlimited-depth /tmp/mbk.1m.proto"
Benchmark #1: ./raw-bench-master /tmp/mbk.1m.proto
  Time (mean ± σ):     245.5 ms ±   1.7 ms    [User: 232.0 ms, System: 13.0 ms]
  Range (min … max):   243.7 ms … 249.4 ms    12 runs

Benchmark #2: ./raw-bench-unlimited-depth /tmp/mbk.1m.proto
  Time (mean ± σ):     245.1 ms ±   0.8 ms    [User: 231.0 ms, System: 14.0 ms]
  Range (min … max):   244.2 ms … 246.7 ms    12 runs

Summary
  './raw-bench-unlimited-depth /tmp/mbk.1m.proto' ran
    1.00 ± 0.01 times faster than './raw-bench-master /tmp/mbk.1m.proto'

Bulk reply, all integer leaves

Generation command:

php resp-protogen/proto.php --count 1000000 --type multibulk --bulkleaf int >| /tmp/mbk.1m.proto.int_leaves

Output:

$ hyperfine --warmup 10 "./raw-bench-master /tmp/mbk.1m.proto.int_leaves" "./raw-bench-unlimited-depth /tmp/mbk.1m.proto.int_leaves"
Benchmark #1: ./raw-bench-master /tmp/mbk.1m.proto.int_leaves
  Time (mean ± σ):     156.0 ms ±   1.2 ms    [User: 153.7 ms, System: 1.9 ms]
  Range (min … max):   154.8 ms … 158.2 ms    19 runs

Benchmark #2: ./raw-bench-unlimited-depth /tmp/mbk.1m.proto.int_leaves
  Time (mean ± σ):     156.5 ms ±   0.8 ms    [User: 154.2 ms, System: 2.1 ms]
  Range (min … max):   155.8 ms … 158.8 ms    19 runs

Summary
  './raw-bench-master /tmp/mbk.1m.proto.int_leaves' ran
    1.00 ± 0.01 times faster than './raw-bench-unlimited-depth /tmp/mbk.1m.proto.int_leaves'

All integer replies

Generation command:

php resp-protogen/proto.php --count 1000000 --type int >| /tmp/mbk.1m.proto.only_ints

Output:

$ hyperfine --warmup 10 "./raw-bench-master /tmp/mbk.1m.proto.only_ints" "./raw-bench-unlimited-depth /tmp/mbk.1m.proto.only_ints"
Benchmark #1: ./raw-bench-master /tmp/mbk.1m.proto.only_ints
  Time (mean ± σ):      56.9 ms ±   0.5 ms    [User: 55.0 ms, System: 1.9 ms]
  Range (min … max):    56.2 ms …  58.3 ms    52 runs

Benchmark #2: ./raw-bench-unlimited-depth /tmp/mbk.1m.proto.only_ints
  Time (mean ± σ):      57.6 ms ±   0.5 ms    [User: 56.4 ms, System: 1.1 ms]
  Range (min … max):    56.9 ms …  59.0 ms    50 runs

Summary
  './raw-bench-master /tmp/mbk.1m.proto.only_ints' ran
    1.01 ± 0.01 times faster than './raw-bench-unlimited-depth /tmp/mbk.1m.proto.only_ints'