guest

[nix-shell:~]# fio --filename=./loeoub --direct=1 --rw=randrw --refill_buffers --norandommap --randrepeat=0 --bs=8k --rwmixread=70 --iodepth=16 --numjobs=16 --runtime=60 --group_reporting --name=8k7030test --size=8192000000
8k7030test: (g=0): rw=randrw, bs=(R) 8192B-8192B, (W) 8192B-8192B, (T) 8192B-8192B, ioengine=psync, iodepth=16
...
fio-3.10
Starting 16 processes
8k7030test: Laying out IO file (1 file / 7812MiB)
Jobs: 16 (f=16): [m(16)][100.0%][r=150MiB/s,w=66.2MiB/s][r=19.2k,w=8472 IOPS][eta 00m:00s]
8k7030test: (groupid=0, jobs=16): err= 0: pid=1402: Wed Oct 10 22:06:39 2018
   read: IOPS=16.6k, BW=130MiB/s (136MB/s)(7778MiB/60002msec)
    clat (usec): min=3, max=2232.3k, avg=285.47, stdev=11661.03
     lat (usec): min=3, max=2232.3k, avg=285.70, stdev=11661.03
    clat percentiles (usec):
     |  1.00th=[     3],  5.00th=[     5], 10.00th=[     5], 20.00th=[     6],
     | 30.00th=[     6], 40.00th=[     6], 50.00th=[     6], 60.00th=[     8],
     | 70.00th=[     9], 80.00th=[    46], 90.00th=[   848], 95.00th=[  1090],
     | 99.00th=[  1680], 99.50th=[  1991], 99.90th=[  4359], 99.95th=[  6783],
     | 99.99th=[212861]
   bw (  KiB/s): min=   15, max=20752, per=7.34%, avg=9745.26, stdev=4495.74, samples=1618
   iops        : min=    1, max= 2594, avg=1217.95, stdev=562.09, samples=1618
  write: IOPS=7113, BW=55.6MiB/s (58.3MB/s)(3334MiB/60002msec)
    clat (usec): min=4, max=2247.0k, avg=1571.29, stdev=22492.83
     lat (usec): min=4, max=2247.0k, avg=1571.54, stdev=22492.85
    clat percentiles (usec):
     |  1.00th=[     15],  5.00th=[     29], 10.00th=[     56],
     | 20.00th=[    153], 30.00th=[    318], 40.00th=[    562],
     | 50.00th=[    783], 60.00th=[   1139], 70.00th=[   1500],
     | 80.00th=[   1926], 90.00th=[   2638], 95.00th=[   3392],
     | 99.00th=[   5735], 99.50th=[   7373], 99.90th=[  15401],
     | 99.95th=[  51643], 99.99th=[1199571]
   bw (  KiB/s): min=   15, max= 8688, per=7.34%, avg=4177.09, stdev=1910.61, samples=1618
   iops        : min=    1, max= 1086, avg=521.90, stdev=238.95, samples=1618
  lat (usec)   : 4=1.37%, 10=50.48%, 20=3.99%, 50=3.03%, 100=2.40%
  lat (usec)   : 250=4.05%, 500=5.46%, 750=5.55%, 1000=5.84%
  lat (msec)   : 2=11.86%, 4=4.98%, 10=0.90%, 20=0.05%, 50=0.01%
  lat (msec)   : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
  cpu          : usr=0.64%, sys=15.28%, ctx=945547, majf=0, minf=477
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=995601,426813,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=130MiB/s (136MB/s), 130MiB/s-130MiB/s (136MB/s-136MB/s), io=7778MiB (8156MB), run=60002-60002msec
  WRITE: bw=55.6MiB/s (58.3MB/s), 55.6MiB/s-55.6MiB/s (58.3MB/s-58.3MB/s), io=3334MiB (3496MB), run=60002-60002msec

host

8k7030test: (groupid=0, jobs=16): err= 0: pid=5703: Wed Oct 10 22:09:44 2018
   read: IOPS=3756, BW=29.3MiB/s (30.8MB/s)(1769MiB/60290msec)
    clat (usec): min=2, max=1684.5k, avg=2130.67, stdev=13250.15
     lat (usec): min=2, max=1684.5k, avg=2130.81, stdev=13250.15
    clat percentiles (usec):
     |  1.00th=[     5],  5.00th=[     8], 10.00th=[    27], 20.00th=[    31],
     | 30.00th=[  2409], 40.00th=[  2474], 50.00th=[  2507], 60.00th=[  2507],
     | 70.00th=[  2507], 80.00th=[  2507], 90.00th=[  2540], 95.00th=[  2540],
     | 99.00th=[  3425], 99.50th=[  5800], 99.90th=[ 14746], 99.95th=[ 26870],
     | 99.99th=[591397]
   bw (  KiB/s): min=   32, max= 7520, per=7.07%, avg=2124.33, stdev=989.25, samples=1703
   iops        : min=    4, max=  940, avg=265.52, stdev=123.67, samples=1703
  write: IOPS=1609, BW=12.6MiB/s (13.2MB/s)(758MiB/60290msec)
    clat (usec): min=6, max=1696.9k, avg=4957.50, stdev=49830.69
     lat (usec): min=6, max=1696.9k, avg=4957.67, stdev=49830.69
    clat percentiles (usec):
     |  1.00th=[     15],  5.00th=[     35], 10.00th=[     39],
     | 20.00th=[     43], 30.00th=[   2442], 40.00th=[   2474],
     | 50.00th=[   2507], 60.00th=[   2507], 70.00th=[   2507],
     | 80.00th=[   2540], 90.00th=[   2540], 95.00th=[   2540],
     | 99.00th=[   6521], 99.50th=[  21627], 99.90th=[ 901776],
     | 99.95th=[1052771], 99.99th=[1686111]
   bw (  KiB/s): min=   16, max= 3136, per=7.07%, avg=910.33, stdev=427.37, samples=1702
   iops        : min=    2, max=  392, avg=113.76, stdev=53.43, samples=1702
  lat (usec)   : 4=0.17%, 10=3.68%, 20=1.08%, 50=19.60%, 100=0.99%
  lat (usec)   : 250=0.21%, 500=0.04%, 750=1.13%, 1000=0.50%
  lat (msec)   : 2=0.50%, 4=71.12%, 10=0.63%, 20=0.17%, 50=0.02%
  lat (msec)   : 100=0.01%, 250=0.01%, 500=0.09%, 750=0.03%, 1000=0.03%
  cpu          : usr=0.13%, sys=0.90%, ctx=245780, majf=4, minf=288
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=226472,97024,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=29.3MiB/s (30.8MB/s), 29.3MiB/s-29.3MiB/s (30.8MB/s-30.8MB/s), io=1769MiB (1855MB), run=60290-60290msec
  WRITE: bw=12.6MiB/s (13.2MB/s), 12.6MiB/s-12.6MiB/s (13.2MB/s-13.2MB/s), io=758MiB (795MB), run=60290-60290msec