Skip to content

Instantly share code, notes, and snippets.

@fo40225
Created April 22, 2020 01:36
Show Gist options
  • Save fo40225/0f4d4d905a0b381a43242e8c96b5f737 to your computer and use it in GitHub Desktop.
Save fo40225/0f4d4d905a0b381a43242e8c96b5f737 to your computer and use it in GitHub Desktop.
Samsung PM983 1.92TB fio test
$ fio --loops=5 --size=1g --runtime=5 --time_based=1 --stonewall --direct=1 --group_reporting \
> --name=SeqQ8T1read --bs=1048576 --iodepth=8 --rw=read \
> --name=SeqQ8T1write --bs=1048576 --iodepth=8 --rw=write \
> --name=SeqQ1T1read --bs=1048576 --iodepth=1 --rw=read \
> --name=SeqQ1T1write --bs=1048576 --iodepth=1 --rw=write \
> --name=4kQ32T16read --bs=4096 --iodepth=32 --numjobs=16 --rw=randread \
> --name=4kQ32T16write --bs=4096 --iodepth=32 --numjobs=16 --rw=randwrite \
> --name=4kQ1T1read --bs=4096 --iodepth=1 --rw=randread \
> --name=4kQ1T1write --bs=4096 --iodepth=1 --rw=randwrite
SeqQ8T1read: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=8
SeqQ8T1write: (g=1): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=8
SeqQ1T1read: (g=2): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=1
SeqQ1T1write: (g=3): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=1
4kQ32T16read: (g=4): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=32
...
4kQ32T16write: (g=5): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=32
...
4kQ1T1read: (g=6): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
4kQ1T1write: (g=7): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
fio-3.19
Starting 38 processes
SeqQ8T1read: Laying out IO file (1 file / 1024MiB)
SeqQ8T1write: Laying out IO file (1 file / 1024MiB)
SeqQ1T1read: Laying out IO file (1 file / 1024MiB)
SeqQ1T1write: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16read: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ32T16write: Laying out IO file (1 file / 1024MiB)
4kQ1T1read: Laying out IO file (1 file / 1024MiB)
4kQ1T1write: Laying out IO file (1 file / 1024MiB)
Jobs: 1 (f=1): [_(37),w(1)][67.7%][w=234MiB/s][w=59.9k IOPS][eta 00m:20s]
SeqQ8T1read: (groupid=0, jobs=1): err= 0: pid=71925: Wed Apr 15 23:53:10 2020
read: IOPS=1888, BW=1889MiB/s (1980MB/s)(9445MiB/5001msec)
clat (usec): min=426, max=1258, avg=528.37, stdev=31.50
lat (usec): min=426, max=1260, avg=528.48, stdev=31.54
clat percentiles (usec):
| 1.00th=[ 453], 5.00th=[ 469], 10.00th=[ 482], 20.00th=[ 498],
| 30.00th=[ 523], 40.00th=[ 529], 50.00th=[ 537], 60.00th=[ 545],
| 70.00th=[ 545], 80.00th=[ 553], 90.00th=[ 562], 95.00th=[ 570],
| 99.00th=[ 578], 99.50th=[ 586], 99.90th=[ 594], 99.95th=[ 603],
| 99.99th=[ 1254]
bw ( MiB/s): min= 1881, max= 1904, per=100.00%, avg=1892.43, stdev= 8.01, samples=9
iops : min= 1881, max= 1904, avg=1892.33, stdev= 8.15, samples=9
lat (usec) : 500=21.81%, 750=78.18%
lat (msec) : 2=0.01%
cpu : usr=0.38%, sys=15.82%, ctx=9447, majf=0, minf=268
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=9445,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=8
SeqQ8T1write: (groupid=1, jobs=1): err= 0: pid=71956: Wed Apr 15 23:53:10 2020
write: IOPS=1933, BW=1934MiB/s (2028MB/s)(9671MiB/5001msec)
clat (usec): min=395, max=1915, avg=491.07, stdev=47.21
lat (usec): min=411, max=1930, avg=515.35, stdev=52.34
clat percentiles (usec):
| 1.00th=[ 416], 5.00th=[ 433], 10.00th=[ 441], 20.00th=[ 453],
| 30.00th=[ 465], 40.00th=[ 482], 50.00th=[ 494], 60.00th=[ 502],
| 70.00th=[ 510], 80.00th=[ 519], 90.00th=[ 529], 95.00th=[ 545],
| 99.00th=[ 619], 99.50th=[ 644], 99.90th=[ 930], 99.95th=[ 1074],
| 99.99th=[ 1909]
bw ( MiB/s): min= 1868, max= 2002, per=99.63%, avg=1926.71, stdev=40.50, samples=9
iops : min= 1868, max= 2002, avg=1926.67, stdev=40.53, samples=9
lat (usec) : 500=56.60%, 750=43.22%, 1000=0.11%
lat (msec) : 2=0.06%
cpu : usr=5.82%, sys=10.54%, ctx=9678, majf=0, minf=12
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,9671,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=8
SeqQ1T1read: (groupid=2, jobs=1): err= 0: pid=71987: Wed Apr 15 23:53:10 2020
read: IOPS=1936, BW=1937MiB/s (2031MB/s)(9685MiB/5001msec)
clat (usec): min=409, max=1705, avg=515.28, stdev=36.12
lat (usec): min=409, max=1706, avg=515.38, stdev=36.17
clat percentiles (usec):
| 1.00th=[ 437], 5.00th=[ 449], 10.00th=[ 457], 20.00th=[ 482],
| 30.00th=[ 494], 40.00th=[ 523], 50.00th=[ 529], 60.00th=[ 537],
| 70.00th=[ 537], 80.00th=[ 545], 90.00th=[ 545], 95.00th=[ 553],
| 99.00th=[ 562], 99.50th=[ 562], 99.90th=[ 594], 99.95th=[ 603],
| 99.99th=[ 1713]
bw ( MiB/s): min= 1910, max= 2022, per=100.00%, avg=1943.36, stdev=41.38, samples=9
iops : min= 1910, max= 2022, avg=1943.33, stdev=41.40, samples=9
lat (usec) : 500=31.32%, 750=68.67%
lat (msec) : 2=0.01%
cpu : usr=0.50%, sys=15.80%, ctx=9688, majf=0, minf=517
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=9685,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
SeqQ1T1write: (groupid=3, jobs=1): err= 0: pid=72024: Wed Apr 15 23:53:10 2020
write: IOPS=1944, BW=1944MiB/s (2039MB/s)(9724MiB/5001msec)
clat (usec): min=379, max=1679, avg=483.87, stdev=47.17
lat (usec): min=393, max=1695, avg=512.12, stdev=52.62
clat percentiles (usec):
| 1.00th=[ 404], 5.00th=[ 416], 10.00th=[ 429], 20.00th=[ 445],
| 30.00th=[ 465], 40.00th=[ 482], 50.00th=[ 490], 60.00th=[ 494],
| 70.00th=[ 502], 80.00th=[ 510], 90.00th=[ 523], 95.00th=[ 545],
| 99.00th=[ 619], 99.50th=[ 644], 99.90th=[ 816], 99.95th=[ 1029],
| 99.99th=[ 1680]
bw ( MiB/s): min= 1872, max= 1976, per=100.00%, avg=1945.17, stdev=34.77, samples=9
iops : min= 1872, max= 1976, avg=1945.11, stdev=34.92, samples=9
lat (usec) : 500=68.83%, 750=31.02%, 1000=0.10%
lat (msec) : 2=0.05%
cpu : usr=6.12%, sys=13.82%, ctx=9728, majf=0, minf=17
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,9724,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
4kQ32T16read: (groupid=4, jobs=16): err= 0: pid=72055: Wed Apr 15 23:53:10 2020
read: IOPS=192k, BW=751MiB/s (788MB/s)(3756MiB/5001msec)
clat (usec): min=47, max=345, avg=82.67, stdev=18.86
lat (usec): min=47, max=345, avg=82.74, stdev=18.86
clat percentiles (usec):
| 1.00th=[ 59], 5.00th=[ 68], 10.00th=[ 69], 20.00th=[ 70],
| 30.00th=[ 72], 40.00th=[ 74], 50.00th=[ 78], 60.00th=[ 84],
| 70.00th=[ 86], 80.00th=[ 90], 90.00th=[ 105], 95.00th=[ 124],
| 99.00th=[ 155], 99.50th=[ 169], 99.90th=[ 200], 99.95th=[ 212],
| 99.99th=[ 245]
bw ( KiB/s): min=762421, max=775037, per=100.00%, avg=769971.44, stdev=233.43, samples=144
iops : min=190600, max=193758, avg=192487.89, stdev=58.41, samples=144
lat (usec) : 50=0.11%, 100=88.22%, 250=11.67%, 500=0.01%
cpu : usr=2.56%, sys=2.44%, ctx=961665, majf=0, minf=213
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=961586,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
4kQ32T16write: (groupid=5, jobs=16): err= 0: pid=72101: Wed Apr 15 23:53:10 2020
write: IOPS=416k, BW=1624MiB/s (1703MB/s)(8122MiB/5002msec)
clat (usec): min=16, max=1855, avg=37.99, stdev= 5.34
lat (usec): min=17, max=1855, avg=38.04, stdev= 5.35
clat percentiles (nsec):
| 1.00th=[31360], 5.00th=[33536], 10.00th=[34560], 20.00th=[35584],
| 30.00th=[36608], 40.00th=[37120], 50.00th=[37632], 60.00th=[38144],
| 70.00th=[38656], 80.00th=[39680], 90.00th=[41216], 95.00th=[42752],
| 99.00th=[52480], 99.50th=[59648], 99.90th=[75264], 99.95th=[81408],
| 99.99th=[95744]
bw ( MiB/s): min= 1578, max= 1668, per=99.76%, avg=1619.89, stdev= 1.74, samples=144
iops : min=404095, max=427112, avg=414686.89, stdev=444.77, samples=144
lat (usec) : 20=0.01%, 50=98.69%, 100=1.30%, 250=0.01%, 500=0.01%
lat (usec) : 1000=0.01%
lat (msec) : 2=0.01%
cpu : usr=2.09%, sys=30.41%, ctx=2079976, majf=0, minf=687
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,2079191,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
4kQ1T1read: (groupid=6, jobs=1): err= 0: pid=72147: Wed Apr 15 23:53:10 2020
read: IOPS=13.5k, BW=52.6MiB/s (55.1MB/s)(263MiB/5001msec)
clat (usec): min=54, max=1656, avg=73.81, stdev=16.03
lat (usec): min=55, max=1656, avg=73.87, stdev=16.03
clat percentiles (usec):
| 1.00th=[ 67], 5.00th=[ 67], 10.00th=[ 68], 20.00th=[ 68],
| 30.00th=[ 69], 40.00th=[ 69], 50.00th=[ 70], 60.00th=[ 71],
| 70.00th=[ 83], 80.00th=[ 84], 90.00th=[ 85], 95.00th=[ 86],
| 99.00th=[ 88], 99.50th=[ 89], 99.90th=[ 106], 99.95th=[ 174],
| 99.99th=[ 996]
bw ( KiB/s): min=52360, max=54240, per=100.00%, avg=53896.56, stdev=615.46, samples=9
iops : min=13090, max=13560, avg=13474.00, stdev=153.88, samples=9
lat (usec) : 100=99.87%, 250=0.10%, 500=0.02%, 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%
cpu : usr=3.00%, sys=1.64%, ctx=67302, majf=0, minf=14
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=67297,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
4kQ1T1write: (groupid=7, jobs=1): err= 0: pid=72178: Wed Apr 15 23:53:10 2020
write: IOPS=58.9k, BW=230MiB/s (241MB/s)(1151MiB/5001msec)
clat (usec): min=10, max=424, avg=16.54, stdev= 2.94
lat (usec): min=10, max=425, avg=16.58, stdev= 2.95
clat percentiles (nsec):
| 1.00th=[11840], 5.00th=[12224], 10.00th=[14784], 20.00th=[15808],
| 30.00th=[16064], 40.00th=[16320], 50.00th=[16512], 60.00th=[16768],
| 70.00th=[16768], 80.00th=[17024], 90.00th=[17792], 95.00th=[20352],
| 99.00th=[24704], 99.50th=[29824], 99.90th=[40704], 99.95th=[42752],
| 99.99th=[54528]
bw ( KiB/s): min=216752, max=259928, per=98.63%, avg=232364.56, stdev=13845.49, samples=9
iops : min=54188, max=64982, avg=58091.11, stdev=3461.41, samples=9
lat (usec) : 20=94.34%, 50=5.65%, 100=0.01%, 250=0.01%, 500=0.01%
cpu : usr=4.34%, sys=38.50%, ctx=294549, majf=0, minf=16
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,294545,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=1889MiB/s (1980MB/s), 1889MiB/s-1889MiB/s (1980MB/s-1980MB/s), io=9445MiB (9904MB), run=5001-5001msec
Run status group 1 (all jobs):
WRITE: bw=1934MiB/s (2028MB/s), 1934MiB/s-1934MiB/s (2028MB/s-2028MB/s), io=9671MiB (10.1GB), run=5001-5001msec
Run status group 2 (all jobs):
READ: bw=1937MiB/s (2031MB/s), 1937MiB/s-1937MiB/s (2031MB/s-2031MB/s), io=9685MiB (10.2GB), run=5001-5001msec
Run status group 3 (all jobs):
WRITE: bw=1944MiB/s (2039MB/s), 1944MiB/s-1944MiB/s (2039MB/s-2039MB/s), io=9724MiB (10.2GB), run=5001-5001msec
Run status group 4 (all jobs):
READ: bw=751MiB/s (788MB/s), 751MiB/s-751MiB/s (788MB/s-788MB/s), io=3756MiB (3939MB), run=5001-5001msec
Run status group 5 (all jobs):
WRITE: bw=1624MiB/s (1703MB/s), 1624MiB/s-1624MiB/s (1703MB/s-1703MB/s), io=8122MiB (8516MB), run=5002-5002msec
Run status group 6 (all jobs):
READ: bw=52.6MiB/s (55.1MB/s), 52.6MiB/s-52.6MiB/s (55.1MB/s-55.1MB/s), io=263MiB (276MB), run=5001-5001msec
Run status group 7 (all jobs):
WRITE: bw=230MiB/s (241MB/s), 230MiB/s-230MiB/s (241MB/s-241MB/s), io=1151MiB (1206MB), run=5001-5001msec
Disk stats (read/write):
nvme0n1: ios=1048013/2390385, merge=0/20275, ticks=89492/66848, in_queue=117500, util=25.67%
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment