Skip to content

Instantly share code, notes, and snippets.

@mdcallag
Created April 26, 2015 17:15
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save mdcallag/f207bf5373825e6324a3 to your computer and use it in GitHub Desktop.
Save mdcallag/f207bf5373825e6324a3 to your computer and use it in GitHub Desktop.
1M keys, 4kb values 50% compressible, merge operator was "put"
The "Test" column has the test name, number of threads (t1, t2, t4, t8, t16, t32 for 1,2,4,8,16,32 threads) and s0 means "sync=0".
I don't list results
Notes:
* overwrite and mergerandom have similar ops/sec rates. overwrite is slightly faster and I assume that is because compaction is cheaper
from not handling merges.
* overwrite and mergerandom reach peak throughput at 1 thread when sync=0.
* for 4kb values updaterandom is slower than overwrite at low concurrency but then slightly faster at high concurrency. This is odd because update random
does a read with each write. The reads are cheap, from cache, but that is still a lot more work. This looks like RocksDB suffers too much
from mutex contention during writes and there is more contention in the overwrite test because.
** overwrite has more Stall-time than updaterandom at high concurrency
** overwrite has larger commit groups than updaterandom at high concurrency, check out "writes per batch" below
Cumulative writes: 63M writes, 63M keys, 2948K batches, 21.5 writes per batch, ingest: 242.95 GB, 141.48 MB/s --> updaterandom
Cumulative writes: 57M writes, 57M keys, 2579K batches, 22.1 writes per batch, ingest: 219.15 GB, 126.51 MB/s --> overwrite
** While Stall-time was larger for overwrite than updaterandom at high concurrency, stall counts were larger for updaterandom
Stalls(count): 2658 level0_slowdown, 114 level0_numfiles, 0 memtable_compaction, 0 leveln_slowdown_soft, 132 leveln_slowdown_hard --> updaterandom
Stalls(count): 2401 level0_slowdown, 120 level0_numfiles, 0 memtable_compaction, 0 leveln_slowdown_soft, 119 leveln_slowdown_hard --> overwrite
* for 400b values updaterandom was always slower than overwrite, but for that test the database was much larger than RAM so reads were slower,
maybe several hundred usecs/read.
ops/sec mb/sec L0_GB Sum_GB W-Amp W-MB/s usec/op p50 p75 p99 p99.9 p99.99 Uptime Stall-time Stall% Test
27408 107.6 181.8 882.2 4.8 517.3 36.5 11.8 13.7 184 245 1176 1746 00:12:12.367 42.0 overwrite.t1.s0
28230 110.8 186.7 894.5 4.7 525.5 35.4 38.8 48.5 200 246 1158 1743 00:10:31.412 36.2 overwrite.t2.s0
29169 114.5 192.6 895.5 4.6 524.8 34.3 76.4 205.9 382 546 1189 1747 00:10:31.574 36.2 overwrite.t4.s0
30358 119.2 200.9 902.7 4.4 529.1 32.9 149.2 283.2 715 1271 54456 1747 00:10:36.802 36.4 overwrite.t8.s0
31721 124.5 211.0 917.4 4.3 535.5 31.5 325.4 448.0 1169 1366 485893 1754 00:10:58.957 37.6 overwrite.t16.s0
32206 126.4 216.4 936.3 4.3 540.4 31.1 553.0 693.9 1368 1542 979362 1774 00:10:31.967 35.6 overwrite.t32.s0
ops/sec mb/sec L0_GB Sum_GB W-Amp W-MB/s usec/op p50 p75 p99 p99.9 p99.99 Uptime Stall-time Stall% Test
26884 105.5 177.4 843.4 4.7 495.2 37.2 12.7 13.9 190 245 1081 1744 00:11:48.811 40.6 mergerandom.t1.s0
26472 103.9 174.8 847.9 4.8 498.1 37.8 42.7 48.7 200 244 1104 1743 00:10:14.410 35.2 mergerandom.t2.s0
28228 110.8 186.3 862.4 4.6 506.3 35.4 71.6 209.2 388 534 1190 1744 00:10:45.767 37.0 mergerandom.t4.s0
29522 115.9 195.4 873.0 4.4 511.7 33.9 144.4 286.4 757 1252 40750 1747 00:11:6.831 38.2 mergerandom.t8.s0
30689 120.5 203.9 884.5 4.3 516.6 32.6 323.1 457.0 1168 1312 556126 1753 00:11:27.429 39.2 mergerandom.t16.s0
30924 121.4 207.8 897.8 4.3 518.2 32.3 547.9 692.8 1369 1549 1032416 1774 00:11:7.363 37.6 mergerandom.t32.s0
ops/sec mb/sec L0_GB Sum_GB W-Amp W-MB/s usec/op p50 p75 p99 p99.9 p99.99 Uptime Stall-time Stall% Test
20745 162.9 136.8 894.5 6.5 526.1 48.2 50.4 57.6 87 115 163 1741 00:00:7.865 0.5 updaterandom.t1.s0
27629 216.9 182.5 952.2 5.2 559.7 36.2 61.5 74.0 240 272 300 1742 00:02:56.202 10.1 updaterandom.t2.s0
30317 238.0 200.1 986.4 4.9 579.1 33.0 89.6 119.8 284 299 1273 1744 00:06:56.287 23.9 updaterandom.t4.s0
32806 257.5 217.0 1009.1 4.6 591.4 30.5 158.3 277.6 544 1276 66708 1747 00:08:11.389 28.1 updaterandom.t8.s0
34489 270.8 228.3 1015.5 4.4 594.5 29.0 297.1 430.5 1125 1388 337439 1749 00:08:53.080 30.5 updaterandom.t16.s0
36032 282.9 239.9 1048.8 4.3 610.9 27.8 561.9 741.7 1378 1596 781947 1758 00:08:48.396 30.1 updaterandom.t32.s0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment