Skip to content

Instantly share code, notes, and snippets.

@fo40225
Created April 22, 2020 01:37
Show Gist options
  • Save fo40225/d245ede077e591c38b60a1e62b02aa84 to your computer and use it in GitHub Desktop.
Save fo40225/d245ede077e591c38b60a1e62b02aa84 to your computer and use it in GitHub Desktop.
AOC-S3108L-H8iR + 8x Seagate Nytro 3531 1.6TB RAID0
$ fio --loops=5 --size=1g --runtime=5 --time_based=1 --stonewall --direct=1 --group_reporting \
> --name=SeqQ8T1read --bs=1048576 --iodepth=8 --rw=read \
> --name=SeqQ8T1write --bs=1048576 --iodepth=8 --rw=write \
> --name=SeqQ1T1read --bs=1048576 --iodepth=1 --rw=read \
> --name=SeqQ1T1write --bs=1048576 --iodepth=1 --rw=write \
> --name=4kQ32T16read --bs=4096 --iodepth=32 --numjobs=16 --rw=randread \
> --name=4kQ32T16write --bs=4096 --iodepth=32 --numjobs=16 --rw=randwrite \
> --name=4kQ1T1read --bs=4096 --iodepth=1 --rw=randread \
> --name=4kQ1T1write --bs=4096 --iodepth=1 --rw=randwrite
SeqQ8T1read: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=8
SeqQ8T1write: (g=1): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=8
SeqQ1T1read: (g=2): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=1
SeqQ1T1write: (g=3): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=1
4kQ32T16read: (g=4): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=32
...
4kQ32T16write: (g=5): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=32
...
4kQ1T1read: (g=6): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
4kQ1T1write: (g=7): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
fio-3.19
Starting 38 processes
SeqQ8T1read: Laying out IO file (1 file / 1024MiB)
SeqQ8T1write: Laying out IO file (1 file / 1024MiB)
SeqQ1T1read: Laying out IO file (1 file / 1024MiB)
SeqQ1T1write: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ1T1read: Laying out IO file (1 file / 1024MiB)
4kQ1T1write: Laying out IO file (1 file / 1024MiB)
Jobs: 1 (f=1): [_(37),w(1)][44.7%][w=105MiB/s][w=26.9k IOPS][eta 00m:52s]
SeqQ8T1read: (groupid=0, jobs=1): err= 0: pid=70508: Wed Apr 15 23:49:32 2020
read: IOPS=4361, BW=4361MiB/s (4573MB/s)(21.3GiB/5001msec)
clat (usec): min=221, max=2016, avg=228.84, stdev=35.04
lat (usec): min=221, max=2017, avg=228.88, stdev=35.04
clat percentiles (usec):
| 1.00th=[ 223], 5.00th=[ 223], 10.00th=[ 223], 20.00th=[ 223],
| 30.00th=[ 225], 40.00th=[ 225], 50.00th=[ 225], 60.00th=[ 225],
| 70.00th=[ 225], 80.00th=[ 225], 90.00th=[ 227], 95.00th=[ 229],
| 99.00th=[ 392], 99.50th=[ 392], 99.90th=[ 873], 99.95th=[ 873],
| 99.99th=[ 1057]
bw ( MiB/s): min= 3851, max= 4458, per=100.00%, avg=4386.21, stdev=200.41, samples=9
iops : min= 3851, max= 4458, avg=4386.11, stdev=200.71, samples=9
lat (usec) : 250=97.44%, 500=2.45%, 750=0.01%, 1000=0.09%
lat (msec) : 2=0.01%, 4=0.01%
cpu : usr=0.44%, sys=13.26%, ctx=21814, majf=0, minf=266
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=21811,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=8
SeqQ8T1write: (groupid=1, jobs=1): err= 0: pid=70539: Wed Apr 15 23:49:32 2020
write: IOPS=2927, BW=2927MiB/s (3070MB/s)(14.3GiB/5001msec)
clat (usec): min=307, max=8761, avg=323.48, stdev=195.23
lat (usec): min=315, max=8823, avg=340.95, stdev=195.73
clat percentiles (usec):
| 1.00th=[ 310], 5.00th=[ 310], 10.00th=[ 310], 20.00th=[ 310],
| 30.00th=[ 310], 40.00th=[ 310], 50.00th=[ 310], 60.00th=[ 314],
| 70.00th=[ 314], 80.00th=[ 314], 90.00th=[ 318], 95.00th=[ 338],
| 99.00th=[ 408], 99.50th=[ 461], 99.90th=[ 4424], 99.95th=[ 5080],
| 99.99th=[ 5211]
bw ( MiB/s): min= 2720, max= 3034, per=100.00%, avg=2941.24, stdev=110.52, samples=9
iops : min= 2720, max= 3034, avg=2941.22, stdev=110.53, samples=9
lat (usec) : 500=99.75%, 750=0.04%, 1000=0.01%
lat (msec) : 4=0.04%, 10=0.16%
cpu : usr=5.24%, sys=8.84%, ctx=14646, majf=0, minf=11
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,14640,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=8
SeqQ1T1read: (groupid=2, jobs=1): err= 0: pid=70570: Wed Apr 15 23:49:32 2020
read: IOPS=4342, BW=4343MiB/s (4554MB/s)(21.2GiB/5001msec)
clat (usec): min=222, max=2429, avg=229.81, stdev=36.17
lat (usec): min=222, max=2430, avg=229.85, stdev=36.18
clat percentiles (usec):
| 1.00th=[ 223], 5.00th=[ 225], 10.00th=[ 225], 20.00th=[ 225],
| 30.00th=[ 225], 40.00th=[ 225], 50.00th=[ 225], 60.00th=[ 225],
| 70.00th=[ 225], 80.00th=[ 227], 90.00th=[ 227], 95.00th=[ 233],
| 99.00th=[ 392], 99.50th=[ 392], 99.90th=[ 865], 99.95th=[ 865],
| 99.99th=[ 1037]
bw ( MiB/s): min= 3820, max= 4442, per=100.00%, avg=4366.51, stdev=204.78, samples=9
iops : min= 3820, max= 4442, avg=4366.44, stdev=204.98, samples=9
lat (usec) : 250=97.44%, 500=2.44%, 750=0.01%, 1000=0.09%
lat (msec) : 2=0.01%, 4=0.01%
cpu : usr=0.32%, sys=13.94%, ctx=21723, majf=0, minf=516
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=21719,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
SeqQ1T1write: (groupid=3, jobs=1): err= 0: pid=70601: Wed Apr 15 23:49:32 2020
write: IOPS=2960, BW=2960MiB/s (3104MB/s)(14.5GiB/5001msec)
clat (usec): min=306, max=5257, avg=322.04, stdev=164.54
lat (usec): min=313, max=5271, avg=337.09, stdev=165.19
clat percentiles (usec):
| 1.00th=[ 310], 5.00th=[ 310], 10.00th=[ 310], 20.00th=[ 310],
| 30.00th=[ 310], 40.00th=[ 310], 50.00th=[ 310], 60.00th=[ 310],
| 70.00th=[ 310], 80.00th=[ 314], 90.00th=[ 330], 95.00th=[ 359],
| 99.00th=[ 404], 99.50th=[ 461], 99.90th=[ 4359], 99.95th=[ 4752],
| 99.99th=[ 5211]
bw ( MiB/s): min= 2764, max= 3078, per=100.00%, avg=2963.52, stdev=131.41, samples=9
iops : min= 2764, max= 3078, avg=2963.44, stdev=131.46, samples=9
lat (usec) : 500=99.80%, 750=0.05%
lat (msec) : 2=0.01%, 4=0.01%, 10=0.13%
cpu : usr=4.82%, sys=8.64%, ctx=14808, majf=0, minf=13
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,14804,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
4kQ32T16read: (groupid=4, jobs=16): err= 0: pid=70632: Wed Apr 15 23:49:32 2020
read: IOPS=96.4k, BW=376MiB/s (395MB/s)(1882MiB/5001msec)
clat (usec): min=120, max=30370, avg=165.37, stdev=144.63
lat (usec): min=120, max=30371, avg=165.45, stdev=144.64
clat percentiles (usec):
| 1.00th=[ 137], 5.00th=[ 143], 10.00th=[ 147], 20.00th=[ 151],
| 30.00th=[ 155], 40.00th=[ 159], 50.00th=[ 163], 60.00th=[ 167],
| 70.00th=[ 172], 80.00th=[ 176], 90.00th=[ 184], 95.00th=[ 194],
| 99.00th=[ 221], 99.50th=[ 235], 99.90th=[ 265], 99.95th=[ 277],
| 99.99th=[ 367]
bw ( KiB/s): min=354739, max=396760, per=100.00%, avg=385834.11, stdev=762.28, samples=144
iops : min=88679, max=99190, avg=96457.89, stdev=190.67, samples=144
lat (usec) : 250=99.78%, 500=0.21%, 750=0.01%
lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
cpu : usr=1.47%, sys=12.66%, ctx=481941, majf=0, minf=201
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=481894,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
4kQ32T16write: (groupid=5, jobs=16): err= 0: pid=70684: Wed Apr 15 23:49:32 2020
write: IOPS=126k, BW=493MiB/s (517MB/s)(2473MiB/5017msec)
clat (usec): min=40, max=26243, avg=126.16, stdev=144.83
lat (usec): min=40, max=26243, avg=126.23, stdev=144.84
clat percentiles (usec):
| 1.00th=[ 91], 5.00th=[ 104], 10.00th=[ 110], 20.00th=[ 116],
| 30.00th=[ 120], 40.00th=[ 123], 50.00th=[ 126], 60.00th=[ 129],
| 70.00th=[ 131], 80.00th=[ 135], 90.00th=[ 139], 95.00th=[ 145],
| 99.00th=[ 161], 99.50th=[ 182], 99.90th=[ 241], 99.95th=[ 255],
| 99.99th=[ 322]
bw ( KiB/s): min=459536, max=519800, per=100.00%, avg=506736.00, stdev=1148.93, samples=144
iops : min=114878, max=129950, avg=126682.89, stdev=287.36, samples=144
lat (usec) : 50=0.04%, 100=2.98%, 250=96.91%, 500=0.06%, 750=0.01%
lat (usec) : 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
cpu : usr=0.84%, sys=59.48%, ctx=631935, majf=0, minf=567
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,633118,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
4kQ1T1read: (groupid=6, jobs=1): err= 0: pid=70730: Wed Apr 15 23:49:32 2020
read: IOPS=6854, BW=26.8MiB/s (28.1MB/s)(134MiB/5001msec)
clat (usec): min=114, max=424, avg=144.42, stdev=11.23
lat (usec): min=114, max=424, avg=144.61, stdev=11.27
clat percentiles (usec):
| 1.00th=[ 122], 5.00th=[ 126], 10.00th=[ 129], 20.00th=[ 137],
| 30.00th=[ 139], 40.00th=[ 141], 50.00th=[ 143], 60.00th=[ 147],
| 70.00th=[ 151], 80.00th=[ 155], 90.00th=[ 159], 95.00th=[ 163],
| 99.00th=[ 169], 99.50th=[ 174], 99.90th=[ 184], 99.95th=[ 190],
| 99.99th=[ 206]
bw ( KiB/s): min=27208, max=28280, per=100.00%, avg=27448.89, stdev=323.31, samples=9
iops : min= 6802, max= 7070, avg=6862.11, stdev=80.86, samples=9
lat (usec) : 250=100.00%, 500=0.01%
cpu : usr=1.32%, sys=6.28%, ctx=34284, majf=0, minf=11
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=34281,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
4kQ1T1write: (groupid=7, jobs=1): err= 0: pid=70761: Wed Apr 15 23:49:32 2020
write: IOPS=26.3k, BW=103MiB/s (108MB/s)(515MiB/5008msec)
clat (usec): min=34, max=9516, avg=37.41, stdev=26.30
lat (usec): min=34, max=9516, avg=37.50, stdev=26.30
clat percentiles (nsec):
| 1.00th=[35072], 5.00th=[35584], 10.00th=[35584], 20.00th=[36096],
| 30.00th=[36096], 40.00th=[36608], 50.00th=[36608], 60.00th=[37120],
| 70.00th=[37120], 80.00th=[37632], 90.00th=[38656], 95.00th=[42752],
| 99.00th=[45312], 99.50th=[56064], 99.90th=[69120], 99.95th=[70144],
| 99.99th=[85504]
bw ( KiB/s): min=102067, max=107736, per=100.00%, avg=105544.33, stdev=2322.20, samples=9
iops : min=25516, max=26934, avg=26386.00, stdev=580.69, samples=9
lat (usec) : 50=99.29%, 100=0.70%, 250=0.01%, 500=0.01%
lat (msec) : 10=0.01%
cpu : usr=1.68%, sys=21.15%, ctx=131937, majf=0, minf=14
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,131928,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=4361MiB/s (4573MB/s), 4361MiB/s-4361MiB/s (4573MB/s-4573MB/s), io=21.3GiB (22.9GB), run=5001-5001msec
Run status group 1 (all jobs):
WRITE: bw=2927MiB/s (3070MB/s), 2927MiB/s-2927MiB/s (3070MB/s-3070MB/s), io=14.3GiB (15.4GB), run=5001-5001msec
Run status group 2 (all jobs):
READ: bw=4343MiB/s (4554MB/s), 4343MiB/s-4343MiB/s (4554MB/s-4554MB/s), io=21.2GiB (22.8GB), run=5001-5001msec
Run status group 3 (all jobs):
WRITE: bw=2960MiB/s (3104MB/s), 2960MiB/s-2960MiB/s (3104MB/s-3104MB/s), io=14.5GiB (15.5GB), run=5001-5001msec
Run status group 4 (all jobs):
READ: bw=376MiB/s (395MB/s), 376MiB/s-376MiB/s (395MB/s-395MB/s), io=1882MiB (1974MB), run=5001-5001msec
Run status group 5 (all jobs):
WRITE: bw=493MiB/s (517MB/s), 493MiB/s-493MiB/s (517MB/s-517MB/s), io=2473MiB (2593MB), run=5017-5017msec
Run status group 6 (all jobs):
READ: bw=26.8MiB/s (28.1MB/s), 26.8MiB/s-26.8MiB/s (28.1MB/s-28.1MB/s), io=134MiB (140MB), run=5001-5001msec
Run status group 7 (all jobs):
WRITE: bw=103MiB/s (108MB/s), 103MiB/s-103MiB/s (108MB/s-108MB/s), io=515MiB (540MB), run=5008-5008msec
Disk stats (read/write):
sda: ios=690295/878132, merge=0/6417, ticks=103500/96708, in_queue=193344, util=86.78%
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment