Skip to content

Instantly share code, notes, and snippets.

@cgwalters

cgwalters/raul.md

Last active December 17, 2019 18:37
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save cgwalters/64752a9544eec662e056f8387a3cefc2 to your computer and use it in GitHub Desktop.
Save cgwalters/64752a9544eec662e056f8387a3cefc2 to your computer and use it in GitHub Desktop.
Download ZIP
fio RHCOS LUKS null cipher vs direct XFS
Raw
raul.md

Same experimental setup as https://gist.github.com/cgwalters/64752a9544eec662e056f8387a3cefc2#file-ssd-test-md

Based on

[global]
bs=2000
ioengine=sync
iodepth=1
runtime=60
directory=/mnt
size=10g
filename=ssd.test.file
fdatasync=1
time_based=1

[seq-write]
rw=write

plain XFS

[root@coreos tmp]# fio raul.fio 
seq-write: (g=0): rw=write, bs=(R) 2000B-2000B, (W) 2000B-2000B, (T) 2000B-2000B, ioengine=sync, iodepth=1
fio-3.14
Starting 1 process
seq-write: Laying out IO file (1 file / 10240MiB)
Jobs: 1 (f=1): [W(1)][100.0%][w=771KiB/s][w=395 IOPS][eta 00m:00s]
seq-write: (groupid=0, jobs=1): err= 0: pid=72: Tue Dec 17 16:19:26 2019
  write: IOPS=377, BW=738KiB/s (756kB/s)(43.2MiB/60004msec); 0 zone resets
    clat (usec): min=4, max=379, avg=20.49, stdev=20.41
     lat (usec): min=4, max=381, avg=21.26, stdev=20.95
    clat percentiles (usec):
     |  1.00th=[    6],  5.00th=[    6], 10.00th=[    6], 20.00th=[    8],
     | 30.00th=[    8], 40.00th=[   10], 50.00th=[   12], 60.00th=[   16],
     | 70.00th=[   24], 80.00th=[   34], 90.00th=[   46], 95.00th=[   60],
     | 99.00th=[   93], 99.50th=[  106], 99.90th=[  165], 99.95th=[  231],
     | 99.99th=[  314]
   bw (  KiB/s): min=  535, max=  855, per=100.00%, avg=737.30, stdev=69.32, samples=120
   iops        : min=  274, max=  438, avg=377.75, stdev=35.52, samples=120
  lat (usec)   : 10=45.35%, 20=20.29%, 50=26.31%, 100=7.36%, 250=0.66%
  lat (usec)   : 500=0.04%
  fsync/fdatasync/sync_file_range:
    sync (usec): min=1020, max=18787, avg=2619.95, stdev=1199.07
    sync percentiles (usec):
     |  1.00th=[ 1287],  5.00th=[ 1385], 10.00th=[ 1434], 20.00th=[ 1516],
     | 30.00th=[ 1631], 40.00th=[ 1893], 50.00th=[ 2606], 60.00th=[ 2966],
     | 70.00th=[ 3130], 80.00th=[ 3425], 90.00th=[ 3982], 95.00th=[ 4621],
     | 99.00th=[ 6259], 99.50th=[ 7373], 99.90th=[11469], 99.95th=[12780],
     | 99.99th=[16712]
  cpu          : usr=0.44%, sys=1.88%, ctx=79095, majf=0, minf=12
  IO depths    : 1=200.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,22668,0,0 short=22668,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
  WRITE: bw=738KiB/s (756kB/s), 738KiB/s-738KiB/s (756kB/s-756kB/s), io=43.2MiB (45.3MB), run=60004-60004msec

Disk stats (read/write):
  vda: ios=0/56303, merge=0/22, ticks=0/60318, in_queue=42120, util=47.73%

LUKS null cipher

[root@coreos tmp]# fio raul.fio 
seq-write: (g=0): rw=write, bs=(R) 2000B-2000B, (W) 2000B-2000B, (T) 2000B-2000B, ioengine=sync, iodepth=1
fio-3.14
Starting 1 process
seq-write: Laying out IO file (1 file / 10240MiB)
Jobs: 1 (f=1): [W(1)][100.0%][w=568KiB/s][w=291 IOPS][eta 00m:00s]
seq-write: (groupid=0, jobs=1): err= 0: pid=142: Tue Dec 17 16:22:25 2019
  write: IOPS=355, BW=695KiB/s (711kB/s)(40.7MiB/60001msec); 0 zone resets
    clat (usec): min=6, max=387, avg=25.41, stdev=24.65
     lat (usec): min=6, max=389, avg=26.10, stdev=25.16
    clat percentiles (usec):
     |  1.00th=[    8],  5.00th=[    8], 10.00th=[    8], 20.00th=[   10],
     | 30.00th=[   10], 40.00th=[   11], 50.00th=[   13], 60.00th=[   19],
     | 70.00th=[   30], 80.00th=[   44], 90.00th=[   60], 95.00th=[   76],
     | 99.00th=[  112], 99.50th=[  127], 99.90th=[  174], 99.95th=[  194],
     | 99.99th=[  289]
   bw (  KiB/s): min=  464, max=  800, per=100.00%, avg=695.79, stdev=76.26, samples=119
   iops        : min=  238, max=  410, avg=356.47, stdev=39.07, samples=119
  lat (usec)   : 10=35.18%, 20=26.45%, 50=23.35%, 100=13.44%, 250=1.56%
  lat (usec)   : 500=0.02%
  fsync/fdatasync/sync_file_range:
    sync (usec): min=1118, max=14395, avg=2780.04, stdev=1251.54
    sync percentiles (usec):
     |  1.00th=[ 1352],  5.00th=[ 1450], 10.00th=[ 1516], 20.00th=[ 1598],
     | 30.00th=[ 1713], 40.00th=[ 2040], 50.00th=[ 2835], 60.00th=[ 3130],
     | 70.00th=[ 3294], 80.00th=[ 3621], 90.00th=[ 4359], 95.00th=[ 5014],
     | 99.00th=[ 6456], 99.50th=[ 7439], 99.90th=[11207], 99.95th=[11731],
     | 99.99th=[12911]
  cpu          : usr=0.44%, sys=2.18%, ctx=67701, majf=0, minf=10
  IO depths    : 1=200.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,21339,0,0 short=21339,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
  WRITE: bw=695KiB/s (711kB/s), 695KiB/s-695KiB/s (711kB/s-711kB/s), io=40.7MiB (42.7MB), run=60001-60001msec
Raw
ssd-test.md

Setup: cosa run -m 16384 on my desktop (i9-9900K, NVMe SSD), otherwise mostly idle during each run. Using a fedora:31 container image with -v /:/host and did yum install fio in there, and cd /host/var/tmp.

Using https://raw.githubusercontent.com/axboe/fio/master/examples/ssd-test.fio but I tweaked the size to just 1g, and changed directory=..

Plain XFS (ssd-test.fio)

[root@coreos tmp]# fio ssd-test.fio 
seq-read: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
rand-read: (g=1): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
seq-write: (g=2): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
rand-write: (g=3): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
fio-3.14
Starting 4 processes
seq-read: Laying out IO file (1 file / 1024MiB)
Jobs: 1 (f=1): [_(3),w(1)][100.0%][w=433MiB/s][w=111k IOPS][eta 00m:00s]                       
seq-read: (groupid=0, jobs=1): err= 0: pid=52: Tue Dec 17 16:03:54 2019
  read: IOPS=131k, BW=510MiB/s (535MB/s)(1024MiB/2008msec)
    slat (nsec): min=1516, max=245651, avg=4051.32, stdev=2705.01
    clat (usec): min=7, max=324, avg=25.61, stdev=11.54
     lat (usec): min=12, max=327, avg=29.87, stdev=11.45
    clat percentiles (usec):
     |  1.00th=[   14],  5.00th=[   15], 10.00th=[   17], 20.00th=[   19],
     | 30.00th=[   20], 40.00th=[   22], 50.00th=[   23], 60.00th=[   25],
     | 70.00th=[   28], 80.00th=[   31], 90.00th=[   37], 95.00th=[   46],
     | 99.00th=[   68], 99.50th=[   74], 99.90th=[  129], 99.95th=[  149],
     | 99.99th=[  265]
   bw (  KiB/s): min=425643, max=524093, per=88.29%, avg=461043.67, stdev=54739.72, samples=3
   iops        : min=106410, max=131023, avg=115260.33, stdev=13685.22, samples=3
  lat (usec)   : 10=0.01%, 20=31.05%, 50=65.58%, 100=3.18%, 250=0.16%
  lat (usec)   : 500=0.02%
  cpu          : usr=11.11%, sys=60.74%, ctx=52894, majf=0, minf=14
  IO depths    : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=262144,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=4
rand-read: (groupid=1, jobs=1): err= 0: pid=53: Tue Dec 17 16:03:54 2019
  read: IOPS=144k, BW=561MiB/s (589MB/s)(1024MiB/1824msec)
    slat (nsec): min=1543, max=233351, avg=4195.80, stdev=2984.45
    clat (usec): min=6, max=588, avg=22.53, stdev=13.76
     lat (usec): min=11, max=610, avg=26.94, stdev=14.80
    clat percentiles (usec):
     |  1.00th=[   13],  5.00th=[   14], 10.00th=[   15], 20.00th=[   17],
     | 30.00th=[   18], 40.00th=[   20], 50.00th=[   21], 60.00th=[   23],
     | 70.00th=[   25], 80.00th=[   27], 90.00th=[   31], 95.00th=[   35],
     | 99.00th=[   48], 99.50th=[   70], 99.90th=[  221], 99.95th=[  277],
     | 99.99th=[  494]
   bw (  KiB/s): min=533784, max=541693, per=93.62%, avg=538200.33, stdev=4034.59, samples=3
   iops        : min=133446, max=135423, avg=134550.00, stdev=1008.54, samples=3
  lat (usec)   : 10=0.02%, 20=45.31%, 50=53.79%, 100=0.57%, 250=0.25%
  lat (usec)   : 500=0.05%, 750=0.01%
  cpu          : usr=12.51%, sys=67.91%, ctx=36371, majf=0, minf=14
  IO depths    : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=262144,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=4
seq-write: (groupid=2, jobs=1): err= 0: pid=54: Tue Dec 17 16:03:54 2019
  write: IOPS=93.6k, BW=366MiB/s (383MB/s)(1024MiB/2801msec); 0 zone resets
    slat (nsec): min=1553, max=266190, avg=4366.68, stdev=3778.06
    clat (usec): min=9, max=1181, avg=37.27, stdev=27.14
     lat (usec): min=15, max=1183, avg=41.86, stdev=28.04
    clat percentiles (usec):
     |  1.00th=[   16],  5.00th=[   19], 10.00th=[   21], 20.00th=[   24],
     | 30.00th=[   27], 40.00th=[   29], 50.00th=[   32], 60.00th=[   35],
     | 70.00th=[   40], 80.00th=[   48], 90.00th=[   61], 95.00th=[   71],
     | 99.00th=[  124], 99.50th=[  139], 99.90th=[  441], 99.95th=[  586],
     | 99.99th=[  660]
   bw (  KiB/s): min=265878, max=355619, per=77.57%, avg=290401.60, stdev=36784.61, samples=5
   iops        : min=66469, max=88904, avg=72599.80, stdev=9196.06, samples=5
  lat (usec)   : 10=0.01%, 20=8.21%, 50=74.57%, 100=15.51%, 250=1.52%
  lat (usec)   : 500=0.10%, 750=0.09%, 1000=0.01%
  lat (msec)   : 2=0.01%
  cpu          : usr=9.46%, sys=43.21%, ctx=115994, majf=0, minf=11
  IO depths    : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,262144,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=4
rand-write: (groupid=3, jobs=1): err= 0: pid=55: Tue Dec 17 16:03:54 2019
  write: IOPS=105k, BW=410MiB/s (430MB/s)(1024MiB/2498msec); 0 zone resets
    slat (nsec): min=1554, max=269702, avg=5606.78, stdev=4189.52
    clat (usec): min=7, max=733, avg=30.89, stdev=15.11
     lat (usec): min=15, max=746, avg=36.77, stdev=16.09
    clat percentiles (usec):
     |  1.00th=[   16],  5.00th=[   19], 10.00th=[   20], 20.00th=[   23],
     | 30.00th=[   25], 40.00th=[   27], 50.00th=[   29], 60.00th=[   31],
     | 70.00th=[   34], 80.00th=[   37], 90.00th=[   43], 95.00th=[   50],
     | 99.00th=[   78], 99.50th=[   92], 99.90th=[  237], 99.95th=[  277],
     | 99.99th=[  375]
   bw (  KiB/s): min=362584, max=448328, per=98.58%, avg=413802.00, stdev=36307.45, samples=4
   iops        : min=90646, max=112082, avg=103450.50, stdev=9076.86, samples=4
  lat (usec)   : 10=0.01%, 20=10.49%, 50=84.63%, 100=4.51%, 250=0.29%
  lat (usec)   : 500=0.08%, 750=0.01%
  cpu          : usr=10.85%, sys=57.75%, ctx=167140, majf=0, minf=9
  IO depths    : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,262144,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=4

Run status group 0 (all jobs):
   READ: bw=510MiB/s (535MB/s), 510MiB/s-510MiB/s (535MB/s-535MB/s), io=1024MiB (1074MB), run=2008-2008msec

Run status group 1 (all jobs):
   READ: bw=561MiB/s (589MB/s), 561MiB/s-561MiB/s (589MB/s-589MB/s), io=1024MiB (1074MB), run=1824-1824msec

Run status group 2 (all jobs):
  WRITE: bw=366MiB/s (383MB/s), 366MiB/s-366MiB/s (383MB/s-383MB/s), io=1024MiB (1074MB), run=2801-2801msec

Run status group 3 (all jobs):
  WRITE: bw=410MiB/s (430MB/s), 410MiB/s-410MiB/s (430MB/s-430MB/s), io=1024MiB (1074MB), run=2498-2498msec

Disk stats (read/write):
  vda: ios=524288/506914, merge=0/0, ticks=10000/12873, in_queue=4, util=90.23%
[root@coreos tmp]#

LUKS null cipher (ssd-test.fio)

[root@coreos tmp]# fio ssd-test.fio 
seq-read: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
rand-read: (g=1): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
seq-write: (g=2): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
rand-write: (g=3): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
fio-3.14
Starting 4 processes
seq-read: Laying out IO file (1 file / 1024MiB)
Jobs: 1 (f=1): [_(3),w(1)][100.0%][w=253MiB/s][w=64.8k IOPS][eta 00m:00s]                        
seq-read: (groupid=0, jobs=1): err= 0: pid=123: Tue Dec 17 16:06:23 2019
  read: IOPS=86.5k, BW=338MiB/s (354MB/s)(1024MiB/3032msec)
    slat (usec): min=2, max=241, avg= 5.96, stdev= 4.45
    clat (usec): min=13, max=1364, avg=39.00, stdev=44.14
     lat (usec): min=19, max=1369, avg=45.23, stdev=45.76
    clat percentiles (usec):
     |  1.00th=[   19],  5.00th=[   21], 10.00th=[   23], 20.00th=[   26],
     | 30.00th=[   29], 40.00th=[   31], 50.00th=[   34], 60.00th=[   37],
     | 70.00th=[   39], 80.00th=[   43], 90.00th=[   55], 95.00th=[   70],
     | 99.00th=[  105], 99.50th=[  178], 99.90th=[  791], 99.95th=[  840],
     | 99.99th=[  938]
   bw (  KiB/s): min=249260, max=270111, per=75.68%, avg=261716.80, stdev=9843.32, samples=5
   iops        : min=62315, max=67527, avg=65428.60, stdev=2460.59, samples=5
  lat (usec)   : 20=3.22%, 50=84.52%, 100=11.17%, 250=0.68%, 500=0.10%
  lat (usec)   : 750=0.17%, 1000=0.14%
  lat (msec)   : 2=0.01%
  cpu          : usr=7.62%, sys=56.05%, ctx=44489, majf=0, minf=14
  IO depths    : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=262144,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=4
rand-read: (groupid=1, jobs=1): err= 0: pid=124: Tue Dec 17 16:06:23 2019
  read: IOPS=94.0k, BW=367MiB/s (385MB/s)(1024MiB/2788msec)
    slat (usec): min=2, max=289, avg= 6.24, stdev= 4.07
    clat (usec): min=11, max=1128, avg=34.80, stdev=26.28
     lat (usec): min=18, max=1184, avg=41.32, stdev=27.49
    clat percentiles (usec):
     |  1.00th=[   18],  5.00th=[   21], 10.00th=[   22], 20.00th=[   25],
     | 30.00th=[   28], 40.00th=[   30], 50.00th=[   33], 60.00th=[   35],
     | 70.00th=[   38], 80.00th=[   41], 90.00th=[   46], 95.00th=[   52],
     | 99.00th=[   83], 99.50th=[   96], 99.90th=[  553], 99.95th=[  668],
     | 99.99th=[  791]
   bw (  KiB/s): min=278217, max=359198, per=81.87%, avg=307921.20, stdev=32457.89, samples=5
   iops        : min=69554, max=89799, avg=76980.00, stdev=8114.37, samples=5
  lat (usec)   : 20=4.11%, 50=89.98%, 100=5.47%, 250=0.26%, 500=0.06%
  lat (usec)   : 750=0.10%, 1000=0.02%
  lat (msec)   : 2=0.01%
  cpu          : usr=9.90%, sys=60.75%, ctx=34689, majf=0, minf=14
  IO depths    : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=262144,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=4
seq-write: (groupid=2, jobs=1): err= 0: pid=125: Tue Dec 17 16:06:23 2019
  write: IOPS=64.6k, BW=252MiB/s (265MB/s)(1024MiB/4058msec); 0 zone resets
    slat (nsec): min=1096, max=231977, avg=2883.96, stdev=2086.66
    clat (usec): min=13, max=3252, avg=58.13, stdev=27.64
     lat (usec): min=24, max=3256, avg=61.22, stdev=27.73
    clat percentiles (usec):
     |  1.00th=[   31],  5.00th=[   37], 10.00th=[   40], 20.00th=[   43],
     | 30.00th=[   46], 40.00th=[   49], 50.00th=[   51], 60.00th=[   55],
     | 70.00th=[   62], 80.00th=[   77], 90.00th=[   85], 95.00th=[   91],
     | 99.00th=[  128], 99.50th=[  163], 99.90th=[  297], 99.95th=[  334],
     | 99.99th=[  619]
   bw (  KiB/s): min=174602, max=267856, per=80.99%, avg=209274.43, stdev=28558.11, samples=7
   iops        : min=43650, max=66964, avg=52318.29, stdev=7139.69, samples=7
  lat (usec)   : 20=0.01%, 50=45.97%, 100=51.50%, 250=2.32%, 500=0.19%
  lat (usec)   : 750=0.01%, 1000=0.01%
  lat (msec)   : 2=0.01%, 4=0.01%
  cpu          : usr=6.85%, sys=24.55%, ctx=72896, majf=0, minf=11
  IO depths    : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,262144,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=4
rand-write: (groupid=3, jobs=1): err= 0: pid=126: Tue Dec 17 16:06:23 2019
  write: IOPS=69.7k, BW=272MiB/s (285MB/s)(1024MiB/3762msec); 0 zone resets
    slat (nsec): min=1123, max=259545, avg=3400.45, stdev=2630.66
    clat (usec): min=17, max=6144, avg=52.83, stdev=36.34
     lat (usec): min=21, max=6155, avg=56.46, stdev=36.63
    clat percentiles (usec):
     |  1.00th=[   31],  5.00th=[   35], 10.00th=[   38], 20.00th=[   41],
     | 30.00th=[   44], 40.00th=[   47], 50.00th=[   50], 60.00th=[   52],
     | 70.00th=[   56], 80.00th=[   61], 90.00th=[   69], 95.00th=[   79],
     | 99.00th=[  121], 99.50th=[  157], 99.90th=[  310], 99.95th=[  429],
     | 99.99th=[  938]
   bw (  KiB/s): min=227696, max=297744, per=99.94%, avg=278547.43, stdev=23578.31, samples=7
   iops        : min=56924, max=74436, avg=69636.86, stdev=5894.58, samples=7
  lat (usec)   : 20=0.01%, 50=52.87%, 100=45.28%, 250=1.64%, 500=0.17%
  lat (usec)   : 750=0.02%, 1000=0.01%
  lat (msec)   : 2=0.01%, 4=0.01%, 10=0.01%
  cpu          : usr=10.34%, sys=29.91%, ctx=71125, majf=0, minf=9
  IO depths    : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,262144,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=4

Run status group 0 (all jobs):
   READ: bw=338MiB/s (354MB/s), 338MiB/s-338MiB/s (354MB/s-354MB/s), io=1024MiB (1074MB), run=3032-3032msec

Run status group 1 (all jobs):
   READ: bw=367MiB/s (385MB/s), 367MiB/s-367MiB/s (385MB/s-385MB/s), io=1024MiB (1074MB), run=2788-2788msec

Run status group 2 (all jobs):
  WRITE: bw=252MiB/s (265MB/s), 252MiB/s-252MiB/s (265MB/s-265MB/s), io=1024MiB (1074MB), run=4058-4058msec

Run status group 3 (all jobs):
  WRITE: bw=272MiB/s (285MB/s), 272MiB/s-272MiB/s (285MB/s-285MB/s), io=1024MiB (1074MB), run=3762-3762msec

Disk stats (read/write):
    dm-0: ios=524288/512243, merge=0/0, ticks=15734/19862, in_queue=35596, util=94.14%, aggrios=524288/406156, aggrmerge=0/118163, aggrticks=12353/12846, aggrin_queue=31, aggrutil=94.08%
  vda: ios=524288/406156, merge=0/118163, ticks=12353/12846, in_queue=31, util=94.08%
48.05%
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment