Skip to content

Instantly share code, notes, and snippets.

@rzarzynski

rzarzynski/with_92a28f033a5272b7dc2c5d726e67b6d09f6166ba_rocksdb_enable_rmrange=true.txt

Last active October 12, 2017 14:33
Show Gist options
  • Save rzarzynski/c4e794f584439bf8d8408bc9c16e578e to your computer and use it in GitHub Desktop.
Save rzarzynski/c4e794f584439bf8d8408bc9c16e578e to your computer and use it in GitHub Desktop.
Download ZIP
Performance tests of commit 92a28f033a5272b7dc2c5d726e67b6d09f6166ba and the rocksdb_enable_rmrange configurable
Raw
with_92a28f033a5272b7dc2c5d726e67b6d09f6166ba_rocksdb_enable_rmrange=true.txt
$ /work/fio/fio /work/fio/examples/rbd.fio
rbd_iodepth32: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=rbd, iodepth=32
fio-3.0-32-g83070
Starting 1 process
Jobs: 1 (f=1): [w(1)][100.0%][r=0KiB/s,w=16.6MiB/s][r=0,w=4248 IOPS][eta 00m:00s]
rbd_iodepth32: (groupid=0, jobs=1): err= 0: pid=14177: Thu Oct 12 14:54:48 2017
write: IOPS=4211, BW=16.5MiB/s (17.3MB/s)(4096MiB/248960msec)
slat (nsec): min=637, max=1264.4k, avg=2790.40, stdev=7299.46
clat (usec): min=768, max=194612, avg=7593.42, stdev=3468.52
lat (usec): min=772, max=194614, avg=7596.21, stdev=3468.28
clat percentiles (msec):
| 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 5], 20.00th=[ 6],
| 30.00th=[ 7], 40.00th=[ 7], 50.00th=[ 8], 60.00th=[ 8],
| 70.00th=[ 9], 80.00th=[ 10], 90.00th=[ 11], 95.00th=[ 12],
| 99.00th=[ 15], 99.50th=[ 21], 99.90th=[ 39], 99.95th=[ 49],
| 99.99th=[ 112]
bw ( KiB/s): min= 5064, max=18818, per=99.99%, avg=16845.69, stdev=1206.02, samples=497
iops : min= 1266, max= 4704, avg=4211.40, stdev=301.50, samples=497
lat (usec) : 1000=0.01%
lat (msec) : 2=0.35%, 4=5.85%, 10=80.81%, 20=12.45%, 50=0.48%
lat (msec) : 100=0.04%, 250=0.01%
cpu : usr=28.69%, sys=67.64%, ctx=603987, majf=0, minf=7
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued rwt: total=0,1048576,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
WRITE: bw=16.5MiB/s (17.3MB/s), 16.5MiB/s-16.5MiB/s (17.3MB/s-17.3MB/s), io=4096MiB (4295MB), run=248960-248960msec
Disk stats (read/write):
nvme0n1: ios=1253/4952944, merge=11/435666, ticks=1856/324080, in_queue=325624, util=31.38%
$ /work/fio/fio /work/fio/examples/rbd.fio
rbd_iodepth32: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=rbd, iodepth=32
fio-3.0-32-g83070
Starting 1 process
Jobs: 1 (f=1): [w(1)][100.0%][r=0KiB/s,w=14.8MiB/s][r=0,w=3780 IOPS][eta 00m:00s]
rbd_iodepth32: (groupid=0, jobs=1): err= 0: pid=14352: Thu Oct 12 14:58:39 2017
write: IOPS=4637, BW=18.1MiB/s (18.0MB/s)(4096MiB/226101msec)
slat (nsec): min=632, max=5363.8k, avg=2736.06, stdev=11545.31
clat (usec): min=748, max=167991, avg=6895.93, stdev=3070.13
lat (usec): min=752, max=167993, avg=6898.67, stdev=3069.94
clat percentiles (msec):
| 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 5], 20.00th=[ 6],
| 30.00th=[ 6], 40.00th=[ 7], 50.00th=[ 7], 60.00th=[ 8],
| 70.00th=[ 8], 80.00th=[ 9], 90.00th=[ 10], 95.00th=[ 11],
| 99.00th=[ 14], 99.50th=[ 16], 99.90th=[ 31], 99.95th=[ 66],
| 99.99th=[ 110]
bw ( KiB/s): min=10634, max=20160, per=99.98%, avg=18546.44, stdev=1540.18, samples=452
iops : min= 2658, max= 5040, avg=4636.58, stdev=385.05, samples=452
lat (usec) : 750=0.01%, 1000=0.01%
lat (msec) : 2=0.46%, 4=8.40%, 10=83.91%, 20=6.99%, 50=0.17%
lat (msec) : 100=0.05%, 250=0.01%
cpu : usr=29.07%, sys=66.33%, ctx=550699, majf=0, minf=5
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued rwt: total=0,1048576,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
WRITE: bw=18.1MiB/s (18.0MB/s), 18.1MiB/s-18.1MiB/s (18.0MB/s-18.0MB/s), io=4096MiB (4295MB), run=226101-226101msec
Disk stats (read/write):
nvme0n1: ios=2142/4693954, merge=7/25814, ticks=3232/344768, in_queue=347672, util=24.73%
$ /work/fio/fio /work/fio/examples/rbd.fio
rbd_iodepth32: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=rbd, iodepth=32
fio-3.0-32-g83070
Starting 1 process
Jobs: 1 (f=1): [w(1)][100.0%][r=0KiB/s,w=19.6MiB/s][r=0,w=5029 IOPS][eta 00m:00s]
rbd_iodepth32: (groupid=0, jobs=1): err= 0: pid=14436: Thu Oct 12 15:02:26 2017
write: IOPS=4731, BW=18.5MiB/s (19.4MB/s)(4096MiB/221600msec)
slat (nsec): min=633, max=3967.0k, avg=2648.73, stdev=8492.13
clat (usec): min=747, max=153416, avg=6758.68, stdev=2772.32
lat (usec): min=764, max=153417, avg=6761.33, stdev=2772.04
clat percentiles (usec):
| 1.00th=[ 2278], 5.00th=[ 3326], 10.00th=[ 4015], 20.00th=[ 4883],
| 30.00th=[ 5538], 40.00th=[ 6063], 50.00th=[ 6652], 60.00th=[ 7177],
| 70.00th=[ 7767], 80.00th=[ 8455], 90.00th=[ 9372], 95.00th=[10290],
| 99.00th=[12387], 99.50th=[14353], 99.90th=[26870], 99.95th=[47449],
| 99.99th=[85459]
bw ( KiB/s): min=12608, max=20200, per=99.98%, avg=18922.51, stdev=1097.60, samples=443
iops : min= 3152, max= 5050, avg=4730.59, stdev=274.41, samples=443
lat (usec) : 750=0.01%, 1000=0.01%
lat (msec) : 2=0.57%, 4=9.42%, 10=83.79%, 20=6.03%, 50=0.14%
lat (msec) : 100=0.04%, 250=0.01%
cpu : usr=28.91%, sys=66.74%, ctx=533894, majf=0, minf=5
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued rwt: total=0,1048576,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
WRITE: bw=18.5MiB/s (19.4MB/s), 18.5MiB/s-18.5MiB/s (19.4MB/s-19.4MB/s), io=4096MiB (4295MB), run=221600-221600msec
Disk stats (read/write):
nvme0n1: ios=1819/4707180, merge=20/3586, ticks=2704/288820, in_queue=292488, util=23.39%
$ /work/fio/fio /work/fio/examples/rbd.fio
rbd_iodepth32: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=rbd, iodepth=32
fio-3.0-32-g83070
Starting 1 process
Jobs: 1 (f=1): [w(1)][100.0%][r=0KiB/s,w=19.7MiB/s][r=0,w=5033 IOPS][eta 00m:00s]
rbd_iodepth32: (groupid=0, jobs=1): err= 0: pid=14510: Thu Oct 12 15:06:14 2017
write: IOPS=4729, BW=18.5MiB/s (19.4MB/s)(4096MiB/221688msec)
slat (nsec): min=628, max=4014.7k, avg=2633.94, stdev=8361.81
clat (usec): min=763, max=147009, avg=6761.37, stdev=2764.84
lat (usec): min=768, max=147010, avg=6764.00, stdev=2764.60
clat percentiles (usec):
| 1.00th=[ 2245], 5.00th=[ 3326], 10.00th=[ 4015], 20.00th=[ 4883],
| 30.00th=[ 5538], 40.00th=[ 6128], 50.00th=[ 6652], 60.00th=[ 7177],
| 70.00th=[ 7767], 80.00th=[ 8455], 90.00th=[ 9372], 95.00th=[10290],
| 99.00th=[12387], 99.50th=[14091], 99.90th=[24773], 99.95th=[40109],
| 99.99th=[96994]
bw ( KiB/s): min=12112, max=20432, per=99.98%, avg=18915.50, stdev=1073.14, samples=443
iops : min= 3028, max= 5108, avg=4728.84, stdev=268.29, samples=443
lat (usec) : 1000=0.01%
lat (msec) : 2=0.57%, 4=9.42%, 10=83.72%, 20=6.13%, 50=0.11%
lat (msec) : 100=0.04%, 250=0.01%
cpu : usr=29.02%, sys=66.62%, ctx=564509, majf=0, minf=5
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued rwt: total=0,1048576,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
WRITE: bw=18.5MiB/s (19.4MB/s), 18.5MiB/s-18.5MiB/s (19.4MB/s-19.4MB/s), io=4096MiB (4295MB), run=221688-221688msec
Disk stats (read/write):
nvme0n1: ios=2039/4709567, merge=25/1603, ticks=2828/275028, in_queue=278300, util=23.46%
$ /work/fio/fio /work/fio/examples/rbd.fio
rbd_iodepth32: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=rbd, iodepth=32
fio-3.0-32-g83070
Starting 1 process
Jobs: 1 (f=1): [w(1)][100.0%][r=0KiB/s,w=19.6MiB/s][r=0,w=5019 IOPS][eta 00m:00sJobs: 1 (f=0): [f(1)][100.0%][r=0KiB/s,w=18.4MiB/s][r=0,w=4702 IOPS][eta 00m:00s]
rbd_iodepth32: (groupid=0, jobs=1): err= 0: pid=14594: Thu Oct 12 15:10:00 2017
write: IOPS=4748, BW=18.5MiB/s (19.4MB/s)(4096MiB/220836msec)
slat (nsec): min=636, max=4077.2k, avg=2735.86, stdev=8629.88
clat (usec): min=761, max=154600, avg=6735.25, stdev=2749.92
lat (usec): min=765, max=154601, avg=6737.99, stdev=2749.65
clat percentiles (usec):
| 1.00th=[ 2376], 5.00th=[ 3458], 10.00th=[ 4113], 20.00th=[ 4948],
| 30.00th=[ 5538], 40.00th=[ 6063], 50.00th=[ 6587], 60.00th=[ 7111],
| 70.00th=[ 7635], 80.00th=[ 8356], 90.00th=[ 9241], 95.00th=[10159],
| 99.00th=[12387], 99.50th=[14484], 99.90th=[27395], 99.95th=[50594],
| 99.99th=[77071]
bw ( KiB/s): min=12848, max=20504, per=99.99%, avg=18989.46, stdev=1109.72, samples=441
iops : min= 3212, max= 5126, avg=4747.34, stdev=277.43, samples=441
lat (usec) : 1000=0.01%
lat (msec) : 2=0.45%, 4=8.62%, 10=85.32%, 20=5.40%, 50=0.15%
lat (msec) : 100=0.05%, 250=0.01%
cpu : usr=29.11%, sys=66.09%, ctx=556840, majf=0, minf=5
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued rwt: total=0,1048576,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
WRITE: bw=18.5MiB/s (19.4MB/s), 18.5MiB/s-18.5MiB/s (19.4MB/s-19.4MB/s), io=4096MiB (4295MB), run=220836-220836msec
Disk stats (read/write):
nvme0n1: ios=2037/4686482, merge=26/1635, ticks=2816/300528, in_queue=302996, util=23.22%
Raw
without_92a28f033a5272b7dc2c5d726e67b6d09f6166ba_rocksdb_enable_rmrange configurable=true.txt
$ /work/fio/fio /work/fio/examples/rbd.fio rbd_iodepth32: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=rbd, iodepth=32
fio-3.0-32-g83070
Starting 1 process
Jobs: 1 (f=0): [f(1)][100.0%][r=0KiB/s,w=16.5MiB/s][r=0,w=4214 IOPS][eta 00m:00s]
rbd_iodepth32: (groupid=0, jobs=1): err= 0: pid=8074: Thu Oct 12 13:52:59 2017
write: IOPS=4196, BW=16.4MiB/s (17.2MB/s)(4096MiB/249866msec)
slat (nsec): min=633, max=1242.0k, avg=2979.85, stdev=7770.18
clat (usec): min=837, max=279704, avg=7620.82, stdev=3833.46
lat (usec): min=841, max=279710, avg=7623.80, stdev=3833.33
clat percentiles (msec):
| 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 6],
| 30.00th=[ 7], 40.00th=[ 7], 50.00th=[ 8], 60.00th=[ 8],
| 70.00th=[ 9], 80.00th=[ 10], 90.00th=[ 11], 95.00th=[ 12],
| 99.00th=[ 15], 99.50th=[ 20], 99.90th=[ 37], 99.95th=[ 50],
| 99.99th=[ 194]
bw ( KiB/s): min= 5136, max=18520, per=99.99%, avg=16784.35, stdev=1350.30, samples=499
iops : min= 1284, max= 4630, avg=4196.06, stdev=337.57, samples=499
lat (usec) : 1000=0.01%
lat (msec) : 2=0.09%, 4=3.33%, 10=85.75%, 20=10.33%, 50=0.44%
lat (msec) : 100=0.03%, 250=0.02%, 500=0.01%
cpu : usr=29.17%, sys=66.40%, ctx=596094, majf=0, minf=5
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued rwt: total=0,1048576,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
WRITE: bw=16.4MiB/s (17.2MB/s), 16.4MiB/s-16.4MiB/s (17.2MB/s-17.2MB/s), io=4096MiB (4295MB), run=249866-249866msec
Disk stats (read/write):
nvme0n1: ios=1252/5001266, merge=12/433798, ticks=1700/407108, in_queue=409692, util=31.54%
$ /work/fio/fio /work/fio/examples/rbd.fio
rbd_iodepth32: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=rbd, iodepth=32
fio-3.0-32-g83070
Starting 1 process
Jobs: 1 (f=1): [w(1)][100.0%][r=0KiB/s,w=17.7MiB/s][r=0,w=4533 IOPS][eta 00m:00s]
rbd_iodepth32: (groupid=0, jobs=1): err= 0: pid=8273: Thu Oct 12 13:57:39 2017
write: IOPS=4640, BW=18.1MiB/s (19.0MB/s)(4096MiB/225961msec)
slat (nsec): min=627, max=3964.3k, avg=2803.00, stdev=8252.53
clat (usec): min=791, max=226626, avg=6891.50, stdev=2968.84
lat (usec): min=794, max=226627, avg=6894.31, stdev=2968.68
clat percentiles (msec):
| 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 5], 20.00th=[ 6],
| 30.00th=[ 6], 40.00th=[ 7], 50.00th=[ 7], 60.00th=[ 8],
| 70.00th=[ 8], 80.00th=[ 9], 90.00th=[ 10], 95.00th=[ 11],
| 99.00th=[ 13], 99.50th=[ 15], 99.90th=[ 26], 99.95th=[ 52],
| 99.99th=[ 105]
bw ( KiB/s): min= 9416, max=19816, per=99.99%, avg=18560.17, stdev=1288.29, samples=451
iops : min= 2354, max= 4954, avg=4640.01, stdev=322.07, samples=451
lat (usec) : 1000=0.01%
lat (msec) : 2=0.05%, 4=5.15%, 10=89.86%, 20=4.75%, 50=0.13%
lat (msec) : 100=0.04%, 250=0.01%
cpu : usr=28.79%, sys=65.94%, ctx=587025, majf=0, minf=6
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued rwt: total=0,1048576,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
WRITE: bw=18.1MiB/s (19.0MB/s), 18.1MiB/s-18.1MiB/s (19.0MB/s-19.0MB/s), io=4096MiB (4295MB), run=225961-225961msec
Disk stats (read/write):
nvme0n1: ios=2055/4754579, merge=9/21695, ticks=3132/391004, in_queue=395964, util=24.64%
$ /work/fio/fio /work/fio/examples/rbd.fio
rbd_iodepth32: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=rbd, iodepth=32
fio-3.0-32-g83070
Starting 1 process
Jobs: 1 (f=1): [w(1)][100.0%][r=0KiB/s,w=18.7MiB/s][r=0,w=4798 IOPS][eta 00m:00s]
rbd_iodepth32: (groupid=0, jobs=1): err= 0: pid=8355: Thu Oct 12 14:01:59 2017
write: IOPS=4682, BW=18.3MiB/s (19.2MB/s)(4096MiB/223918msec)
slat (nsec): min=630, max=2267.7k, avg=2871.14, stdev=8175.09
clat (usec): min=835, max=108418, avg=6829.13, stdev=2432.54
lat (usec): min=838, max=108420, avg=6832.00, stdev=2432.32
clat percentiles (usec):
| 1.00th=[ 2999], 5.00th=[ 3949], 10.00th=[ 4490], 20.00th=[ 5276],
| 30.00th=[ 5800], 40.00th=[ 6259], 50.00th=[ 6652], 60.00th=[ 7111],
| 70.00th=[ 7635], 80.00th=[ 8225], 90.00th=[ 9110], 95.00th=[ 9896],
| 99.00th=[12125], 99.50th=[13960], 99.90th=[23725], 99.95th=[42730],
| 99.99th=[84411]
bw ( KiB/s): min=12552, max=19616, per=99.98%, avg=18728.15, stdev=922.75, samples=447
iops : min= 3138, max= 4904, avg=4682.00, stdev=230.69, samples=447
lat (usec) : 1000=0.01%
lat (msec) : 2=0.05%, 4=5.35%, 10=89.88%, 20=4.56%, 50=0.11%
lat (msec) : 100=0.04%, 250=0.01%
cpu : usr=29.11%, sys=65.57%, ctx=551601, majf=0, minf=6
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued rwt: total=0,1048576,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
WRITE: bw=18.3MiB/s (19.2MB/s), 18.3MiB/s-18.3MiB/s (19.2MB/s-19.2MB/s), io=4096MiB (4295MB), run=223918-223918msec
Disk stats (read/write):
nvme0n1: ios=1792/4748844, merge=28/3429, ticks=2704/249356, in_queue=251772, util=23.70%
$ /work/fio/fio /work/fio/examples/rbd.fio
rbd_iodepth32: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=rbd, iodepth=32
fio-3.0-32-g83070
Starting 1 process
Jobs: 1 (f=1): [w(1)][100.0%][r=0KiB/s,w=18.9MiB/s][r=0,w=4851 IOPS][eta 00m:00sJobs: 1 (f=0): [f(1)][100.0%][r=0KiB/s,w=17.6MiB/s][r=0,w=4494 IOPS][eta 00m:00s]
rbd_iodepth32: (groupid=0, jobs=1): err= 0: pid=8430: Thu Oct 12 14:05:50 2017
write: IOPS=4684, BW=18.3MiB/s (19.2MB/s)(4096MiB/223826msec)
slat (nsec): min=627, max=1329.9k, avg=2863.59, stdev=7884.22
clat (usec): min=810, max=105225, avg=6826.31, stdev=2439.01
lat (usec): min=817, max=105227, avg=6829.18, stdev=2438.77
clat percentiles (usec):
| 1.00th=[ 2966], 5.00th=[ 3916], 10.00th=[ 4490], 20.00th=[ 5211],
| 30.00th=[ 5735], 40.00th=[ 6194], 50.00th=[ 6652], 60.00th=[ 7111],
| 70.00th=[ 7635], 80.00th=[ 8225], 90.00th=[ 9110], 95.00th=[10028],
| 99.00th=[12387], 99.50th=[14353], 99.90th=[28181], 99.95th=[43254],
| 99.99th=[72877]
bw ( KiB/s): min=12216, max=19632, per=99.98%, avg=18735.55, stdev=1000.48, samples=447
iops : min= 3054, max= 4908, avg=4683.84, stdev=250.12, samples=447
lat (usec) : 1000=0.01%
lat (msec) : 2=0.06%, 4=5.60%, 10=89.40%, 20=4.75%, 50=0.15%
lat (msec) : 100=0.04%, 250=0.01%
cpu : usr=28.73%, sys=65.76%, ctx=547906, majf=0, minf=7
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued rwt: total=0,1048576,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
WRITE: bw=18.3MiB/s (19.2MB/s), 18.3MiB/s-18.3MiB/s (19.2MB/s-19.2MB/s), io=4096MiB (4295MB), run=223826-223826msec
Disk stats (read/write):
nvme0n1: ios=2019/4747245, merge=31/1731, ticks=2824/260264, in_queue=262720, util=23.63%
Raw
without_92a28f033a5272b7dc2c5d726e67b6d09f6166ba_rocksdb_enable_rmrange=false.txt
$ /work/fio/fio /work/fio/examples/rbd.fio
rbd_iodepth32: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=rbd, iodepth=32
fio-3.0-32-g83070
Starting 1 process
^Cbs: 1 (f=1): [w(1)][30.6%][r=0KiB/s,w=15.9MiB/s][r=0,w=4079 IOPS][eta 02m:39s]
fio: terminating on signal 2
rbd_iodepth32: (groupid=0, jobs=1): err= 0: pid=5843: Thu Oct 12 13:24:37 2017
write: IOPS=4576, BW=17.9MiB/s (18.7MB/s)(1258MiB/70346msec)
slat (nsec): min=648, max=1255.2k, avg=2906.21, stdev=8856.96
clat (usec): min=769, max=212980, avg=6987.41, stdev=3500.41
lat (usec): min=771, max=212989, avg=6990.31, stdev=3500.25
clat percentiles (msec):
| 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 6],
| 30.00th=[ 6], 40.00th=[ 7], 50.00th=[ 7], 60.00th=[ 8],
| 70.00th=[ 8], 80.00th=[ 9], 90.00th=[ 10], 95.00th=[ 11],
| 99.00th=[ 13], 99.50th=[ 15], 99.90th=[ 27], 99.95th=[ 42],
| 99.99th=[ 201]
bw ( KiB/s): min= 8423, max=19296, per=99.95%, avg=18298.09, stdev=1525.53, samples=140
iops : min= 2105, max= 4824, avg=4574.44, stdev=381.43, samples=140
lat (usec) : 1000=0.01%
lat (msec) : 2=0.05%, 4=4.75%, 10=89.20%, 20=5.82%, 50=0.15%
lat (msec) : 100=0.02%, 250=0.02%
cpu : usr=29.06%, sys=65.36%, ctx=181166, majf=0, minf=7
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued rwt: total=0,321958,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
WRITE: bw=17.9MiB/s (18.7MB/s), 17.9MiB/s-17.9MiB/s (18.7MB/s-18.7MB/s), io=1258MiB (1319MB), run=70346-70346msec
Disk stats (read/write):
nvme0n1: ios=596/1437827, merge=1/6436, ticks=988/132992, in_queue=133880, util=23.88%
$ /work/fio/fio /work/fio/examples/rbd.fio
rbd_iodepth32: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=rbd, iodepth=32
fio-3.0-32-g83070
Starting 1 process
Jobs: 1 (f=1): [w(1)][100.0%][r=0KiB/s,w=18.8MiB/s][r=0,w=4816 IOPS][eta 00m:00s]
rbd_iodepth32: (groupid=0, jobs=1): err= 0: pid=5910: Thu Oct 12 13:28:33 2017
write: IOPS=4646, BW=18.1MiB/s (19.0MB/s)(4096MiB/225689msec)
slat (nsec): min=632, max=2190.5k, avg=2870.34, stdev=8867.78
clat (usec): min=786, max=212922, avg=6883.18, stdev=2896.98
lat (usec): min=788, max=212925, avg=6886.05, stdev=2896.79
clat percentiles (msec):
| 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 6],
| 30.00th=[ 6], 40.00th=[ 7], 50.00th=[ 7], 60.00th=[ 8],
| 70.00th=[ 8], 80.00th=[ 9], 90.00th=[ 10], 95.00th=[ 11],
| 99.00th=[ 13], 99.50th=[ 15], 99.90th=[ 24], 99.95th=[ 39],
| 99.99th=[ 130]
bw ( KiB/s): min=10112, max=19824, per=99.99%, avg=18582.23, stdev=1219.60, samples=451
iops : min= 2528, max= 4956, avg=4645.51, stdev=304.90, samples=451
lat (usec) : 1000=0.01%
lat (msec) : 2=0.04%, 4=4.92%, 10=89.99%, 20=4.89%, 50=0.11%
lat (msec) : 100=0.04%, 250=0.01%
cpu : usr=29.24%, sys=65.16%, ctx=548107, majf=0, minf=6
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued rwt: total=0,1048576,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
WRITE: bw=18.1MiB/s (19.0MB/s), 18.1MiB/s-18.1MiB/s (19.0MB/s-19.0MB/s), io=4096MiB (4295MB), run=225689-225689msec
Disk stats (read/write):
nvme0n1: ios=2079/4687773, merge=10/17094, ticks=3148/359556, in_queue=362956, util=24.16%
$ /work/fio/fio /work/fio/examples/rbd.fio
rbd_iodepth32: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=rbd, iodepth=32
fio-3.0-32-g83070
Starting 1 process
Jobs: 1 (f=1): [w(1)][100.0%][r=0KiB/s,w=17.7MiB/s][r=0,w=4523 IOPS][eta 00m:00s]
rbd_iodepth32: (groupid=0, jobs=1): err= 0: pid=5994: Thu Oct 12 13:32:44 2017
write: IOPS=4668, BW=18.2MiB/s (19.1MB/s)(4096MiB/224613msec)
slat (nsec): min=638, max=1106.5k, avg=2827.03, stdev=7472.67
clat (usec): min=812, max=211014, avg=6850.37, stdev=2808.96
lat (usec): min=818, max=211021, avg=6853.19, stdev=2808.76
clat percentiles (msec):
| 1.00th=[ 4], 5.00th=[ 4], 10.00th=[ 5], 20.00th=[ 6],
| 30.00th=[ 6], 40.00th=[ 7], 50.00th=[ 7], 60.00th=[ 8],
| 70.00th=[ 8], 80.00th=[ 9], 90.00th=[ 10], 95.00th=[ 11],
| 99.00th=[ 13], 99.50th=[ 15], 99.90th=[ 24], 99.95th=[ 45],
| 99.99th=[ 121]
bw ( KiB/s): min= 9680, max=19840, per=99.98%, avg=18668.84, stdev=1260.71, samples=449
iops : min= 2420, max= 4960, avg=4667.15, stdev=315.19, samples=449
lat (usec) : 1000=0.01%
lat (msec) : 2=0.05%, 4=5.15%, 10=89.90%, 20=4.75%, 50=0.11%
lat (msec) : 100=0.03%, 250=0.01%
cpu : usr=29.06%, sys=65.11%, ctx=577795, majf=0, minf=6
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued rwt: total=0,1048576,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
WRITE: bw=18.2MiB/s (19.1MB/s), 18.2MiB/s-18.2MiB/s (19.1MB/s-19.1MB/s), io=4096MiB (4295MB), run=224613-224613msec
Disk stats (read/write):
nvme0n1: ios=2025/4683969, merge=27/6508, ticks=3112/360472, in_queue=363372, util=23.58%
$ /work/fio/fio /work/fio/examples/rbd.fio
rbd_iodepth32: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=rbd, iodepth=32
fio-3.0-32-g83070
Starting 1 process
Jobs: 1 (f=1): [w(1)][100.0%][r=0KiB/s,w=19.2MiB/s][r=0,w=4915 IOPS][eta 00m:00s]
rbd_iodepth32: (groupid=0, jobs=1): err= 0: pid=6071: Thu Oct 12 13:36:36 2017
write: IOPS=4693, BW=18.3MiB/s (19.2MB/s)(4096MiB/223414msec)
slat (nsec): min=636, max=1887.8k, avg=2885.68, stdev=8171.03
clat (usec): min=691, max=203421, avg=6813.74, stdev=2793.19
lat (usec): min=694, max=203428, avg=6816.62, stdev=2792.99
clat percentiles (msec):
| 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 5], 20.00th=[ 6],
| 30.00th=[ 6], 40.00th=[ 7], 50.00th=[ 7], 60.00th=[ 8],
| 70.00th=[ 8], 80.00th=[ 9], 90.00th=[ 10], 95.00th=[ 11],
| 99.00th=[ 13], 99.50th=[ 15], 99.90th=[ 24], 99.95th=[ 35],
| 99.99th=[ 138]
bw ( KiB/s): min=10568, max=19936, per=99.98%, avg=18769.27, stdev=1173.22, samples=446
iops : min= 2642, max= 4984, avg=4692.27, stdev=293.31, samples=446
lat (usec) : 750=0.01%, 1000=0.01%
lat (msec) : 2=0.05%, 4=5.34%, 10=89.75%, 20=4.70%, 50=0.12%
lat (msec) : 100=0.02%, 250=0.01%
cpu : usr=29.36%, sys=64.72%, ctx=564437, majf=0, minf=6
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued rwt: total=0,1048576,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
WRITE: bw=18.3MiB/s (19.2MB/s), 18.3MiB/s-18.3MiB/s (19.2MB/s-19.2MB/s), io=4096MiB (4295MB), run=223414-223414msec
Disk stats (read/write):
nvme0n1: ios=1814/4686062, merge=25/1859, ticks=2408/326256, in_queue=328188, util=23.24%
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment