Skip to content

Instantly share code, notes, and snippets.

@mazgi
Last active June 22, 2019 12:07
Show Gist options
  • Select an option

  • Save mazgi/21f85139e50fbce8e508adc0c4985f4f to your computer and use it in GitHub Desktop.

Select an option

Save mazgi/21f85139e50fbce8e508adc0c4985f4f to your computer and use it in GitHub Desktop.
Results of fio benchmark on GCP storages
mkdir -p fio.20190605-114903.result
fio --output=fio.20190605-114903.result/job.20190605-114903.filestore-premium.read-4k.out.txt --name=job.20190605-114903.filestore-premium.read-4k --filename=/mnt/filestore-premium/fio.job.20190605-114903.filestore-premium.read-4k.img --direct=1 --readwrite=read --blocksize=4k --size=1G --runtime=30 --numjobs=2 --group_reporting
fio --output=fio.20190605-114903.result/job.20190605-114903.filestore-premium.read-32m.out.txt --name=job.20190605-114903.filestore-premium.read-32m --filename=/mnt/filestore-premium/fio.job.20190605-114903.filestore-premium.read-32m.img --direct=1 --readwrite=read --blocksize=32m --size=1G --runtime=30 --numjobs=2 --group_reporting
fio --output=fio.20190605-114903.result/job.20190605-114903.filestore-premium.write-4k.out.txt --name=job.20190605-114903.filestore-premium.write-4k --filename=/mnt/filestore-premium/fio.job.20190605-114903.filestore-premium.write-4k.img --direct=1 --readwrite=write --blocksize=4k --size=1G --runtime=30 --numjobs=2 --group_reporting
fio --output=fio.20190605-114903.result/job.20190605-114903.filestore-premium.write-32m.out.txt --name=job.20190605-114903.filestore-premium.write-32m --filename=/mnt/filestore-premium/fio.job.20190605-114903.filestore-premium.write-32m.img --direct=1 --readwrite=write --blocksize=32m --size=1G --runtime=30 --numjobs=2 --group_reporting
fio --output=fio.20190605-114903.result/job.20190605-114903.filestore-premium.randread-4k.out.txt --name=job.20190605-114903.filestore-premium.randread-4k --filename=/mnt/filestore-premium/fio.job.20190605-114903.filestore-premium.randread-4k.img --direct=1 --readwrite=randread --blocksize=4k --size=1G --runtime=30 --numjobs=2 --group_reporting
fio --output=fio.20190605-114903.result/job.20190605-114903.filestore-premium.randread-32m.out.txt --name=job.20190605-114903.filestore-premium.randread-32m --filename=/mnt/filestore-premium/fio.job.20190605-114903.filestore-premium.randread-32m.img --direct=1 --readwrite=randread --blocksize=32m --size=1G --runtime=30 --numjobs=2 --group_reporting
fio --output=fio.20190605-114903.result/job.20190605-114903.filestore-premium.randwrite-4k.out.txt --name=job.20190605-114903.filestore-premium.randwrite-4k --filename=/mnt/filestore-premium/fio.job.20190605-114903.filestore-premium.randwrite-4k.img --direct=1 --readwrite=randwrite --blocksize=4k --size=1G --runtime=30 --numjobs=2 --group_reporting
fio --output=fio.20190605-114903.result/job.20190605-114903.filestore-premium.randwrite-32m.out.txt --name=job.20190605-114903.filestore-premium.randwrite-32m --filename=/mnt/filestore-premium/fio.job.20190605-114903.filestore-premium.randwrite-32m.img --direct=1 --readwrite=randwrite --blocksize=32m --size=1G --runtime=30 --numjobs=2 --group_reporting
fio --output=fio.20190605-114903.result/job.20190605-114903.filestore-standard.read-4k.out.txt --name=job.20190605-114903.filestore-standard.read-4k --filename=/mnt/filestore-standard/fio.job.20190605-114903.filestore-standard.read-4k.img --direct=1 --readwrite=read --blocksize=4k --size=1G --runtime=30 --numjobs=2 --group_reporting
fio --output=fio.20190605-114903.result/job.20190605-114903.filestore-standard.read-32m.out.txt --name=job.20190605-114903.filestore-standard.read-32m --filename=/mnt/filestore-standard/fio.job.20190605-114903.filestore-standard.read-32m.img --direct=1 --readwrite=read --blocksize=32m --size=1G --runtime=30 --numjobs=2 --group_reporting
fio --output=fio.20190605-114903.result/job.20190605-114903.filestore-standard.write-4k.out.txt --name=job.20190605-114903.filestore-standard.write-4k --filename=/mnt/filestore-standard/fio.job.20190605-114903.filestore-standard.write-4k.img --direct=1 --readwrite=write --blocksize=4k --size=1G --runtime=30 --numjobs=2 --group_reporting
fio --output=fio.20190605-114903.result/job.20190605-114903.filestore-standard.write-32m.out.txt --name=job.20190605-114903.filestore-standard.write-32m --filename=/mnt/filestore-standard/fio.job.20190605-114903.filestore-standard.write-32m.img --direct=1 --readwrite=write --blocksize=32m --size=1G --runtime=30 --numjobs=2 --group_reporting
fio --output=fio.20190605-114903.result/job.20190605-114903.filestore-standard.randread-4k.out.txt --name=job.20190605-114903.filestore-standard.randread-4k --filename=/mnt/filestore-standard/fio.job.20190605-114903.filestore-standard.randread-4k.img --direct=1 --readwrite=randread --blocksize=4k --size=1G --runtime=30 --numjobs=2 --group_reporting
fio --output=fio.20190605-114903.result/job.20190605-114903.filestore-standard.randread-32m.out.txt --name=job.20190605-114903.filestore-standard.randread-32m --filename=/mnt/filestore-standard/fio.job.20190605-114903.filestore-standard.randread-32m.img --direct=1 --readwrite=randread --blocksize=32m --size=1G --runtime=30 --numjobs=2 --group_reporting
fio --output=fio.20190605-114903.result/job.20190605-114903.filestore-standard.randwrite-4k.out.txt --name=job.20190605-114903.filestore-standard.randwrite-4k --filename=/mnt/filestore-standard/fio.job.20190605-114903.filestore-standard.randwrite-4k.img --direct=1 --readwrite=randwrite --blocksize=4k --size=1G --runtime=30 --numjobs=2 --group_reporting
fio --output=fio.20190605-114903.result/job.20190605-114903.filestore-standard.randwrite-32m.out.txt --name=job.20190605-114903.filestore-standard.randwrite-32m --filename=/mnt/filestore-standard/fio.job.20190605-114903.filestore-standard.randwrite-32m.img --direct=1 --readwrite=randwrite --blocksize=32m --size=1G --runtime=30 --numjobs=2 --group_reporting
fio --output=fio.20190605-114903.result/job.20190605-114903.pd-ssd.read-4k.out.txt --name=job.20190605-114903.pd-ssd.read-4k --filename=/mnt/pd-ssd/fio.job.20190605-114903.pd-ssd.read-4k.img --direct=1 --readwrite=read --blocksize=4k --size=1G --runtime=30 --numjobs=2 --group_reporting
fio --output=fio.20190605-114903.result/job.20190605-114903.pd-ssd.read-32m.out.txt --name=job.20190605-114903.pd-ssd.read-32m --filename=/mnt/pd-ssd/fio.job.20190605-114903.pd-ssd.read-32m.img --direct=1 --readwrite=read --blocksize=32m --size=1G --runtime=30 --numjobs=2 --group_reporting
fio --output=fio.20190605-114903.result/job.20190605-114903.pd-ssd.write-4k.out.txt --name=job.20190605-114903.pd-ssd.write-4k --filename=/mnt/pd-ssd/fio.job.20190605-114903.pd-ssd.write-4k.img --direct=1 --readwrite=write --blocksize=4k --size=1G --runtime=30 --numjobs=2 --group_reporting
fio --output=fio.20190605-114903.result/job.20190605-114903.pd-ssd.write-32m.out.txt --name=job.20190605-114903.pd-ssd.write-32m --filename=/mnt/pd-ssd/fio.job.20190605-114903.pd-ssd.write-32m.img --direct=1 --readwrite=write --blocksize=32m --size=1G --runtime=30 --numjobs=2 --group_reporting
fio --output=fio.20190605-114903.result/job.20190605-114903.pd-ssd.randread-4k.out.txt --name=job.20190605-114903.pd-ssd.randread-4k --filename=/mnt/pd-ssd/fio.job.20190605-114903.pd-ssd.randread-4k.img --direct=1 --readwrite=randread --blocksize=4k --size=1G --runtime=30 --numjobs=2 --group_reporting
fio --output=fio.20190605-114903.result/job.20190605-114903.pd-ssd.randread-32m.out.txt --name=job.20190605-114903.pd-ssd.randread-32m --filename=/mnt/pd-ssd/fio.job.20190605-114903.pd-ssd.randread-32m.img --direct=1 --readwrite=randread --blocksize=32m --size=1G --runtime=30 --numjobs=2 --group_reporting
fio --output=fio.20190605-114903.result/job.20190605-114903.pd-ssd.randwrite-4k.out.txt --name=job.20190605-114903.pd-ssd.randwrite-4k --filename=/mnt/pd-ssd/fio.job.20190605-114903.pd-ssd.randwrite-4k.img --direct=1 --readwrite=randwrite --blocksize=4k --size=1G --runtime=30 --numjobs=2 --group_reporting
fio --output=fio.20190605-114903.result/job.20190605-114903.pd-ssd.randwrite-32m.out.txt --name=job.20190605-114903.pd-ssd.randwrite-32m --filename=/mnt/pd-ssd/fio.job.20190605-114903.pd-ssd.randwrite-32m.img --direct=1 --readwrite=randwrite --blocksize=32m --size=1G --runtime=30 --numjobs=2 --group_reporting
fio --output=fio.20190605-114903.result/job.20190605-114903.pd-standard.read-4k.out.txt --name=job.20190605-114903.pd-standard.read-4k --filename=/mnt/pd-standard/fio.job.20190605-114903.pd-standard.read-4k.img --direct=1 --readwrite=read --blocksize=4k --size=1G --runtime=30 --numjobs=2 --group_reporting
fio --output=fio.20190605-114903.result/job.20190605-114903.pd-standard.read-32m.out.txt --name=job.20190605-114903.pd-standard.read-32m --filename=/mnt/pd-standard/fio.job.20190605-114903.pd-standard.read-32m.img --direct=1 --readwrite=read --blocksize=32m --size=1G --runtime=30 --numjobs=2 --group_reporting
fio --output=fio.20190605-114903.result/job.20190605-114903.pd-standard.write-4k.out.txt --name=job.20190605-114903.pd-standard.write-4k --filename=/mnt/pd-standard/fio.job.20190605-114903.pd-standard.write-4k.img --direct=1 --readwrite=write --blocksize=4k --size=1G --runtime=30 --numjobs=2 --group_reporting
fio --output=fio.20190605-114903.result/job.20190605-114903.pd-standard.write-32m.out.txt --name=job.20190605-114903.pd-standard.write-32m --filename=/mnt/pd-standard/fio.job.20190605-114903.pd-standard.write-32m.img --direct=1 --readwrite=write --blocksize=32m --size=1G --runtime=30 --numjobs=2 --group_reporting
fio --output=fio.20190605-114903.result/job.20190605-114903.pd-standard.randread-4k.out.txt --name=job.20190605-114903.pd-standard.randread-4k --filename=/mnt/pd-standard/fio.job.20190605-114903.pd-standard.randread-4k.img --direct=1 --readwrite=randread --blocksize=4k --size=1G --runtime=30 --numjobs=2 --group_reporting
fio --output=fio.20190605-114903.result/job.20190605-114903.pd-standard.randread-32m.out.txt --name=job.20190605-114903.pd-standard.randread-32m --filename=/mnt/pd-standard/fio.job.20190605-114903.pd-standard.randread-32m.img --direct=1 --readwrite=randread --blocksize=32m --size=1G --runtime=30 --numjobs=2 --group_reporting
fio --output=fio.20190605-114903.result/job.20190605-114903.pd-standard.randwrite-4k.out.txt --name=job.20190605-114903.pd-standard.randwrite-4k --filename=/mnt/pd-standard/fio.job.20190605-114903.pd-standard.randwrite-4k.img --direct=1 --readwrite=randwrite --blocksize=4k --size=1G --runtime=30 --numjobs=2 --group_reporting
fio --output=fio.20190605-114903.result/job.20190605-114903.pd-standard.randwrite-32m.out.txt --name=job.20190605-114903.pd-standard.randwrite-32m --filename=/mnt/pd-standard/fio.job.20190605-114903.pd-standard.randwrite-32m.img --direct=1 --readwrite=randwrite --blocksize=32m --size=1G --runtime=30 --numjobs=2 --group_reporting
job.20190605-114903.filestore-premium.randread-32m: (g=0): rw=randread, bs=(R) 32.0MiB-32.0MiB, (W) 32.0MiB-32.0MiB, (T) 32.0MiB-32.0MiB, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.filestore-premium.randread-32m: Laying out IO file (1 file / 1024MiB)
fio: native_fallocate call failed: Operation not supported
job.20190605-114903.filestore-premium.randread-32m: (groupid=0, jobs=2): err= 0: pid=4152: Wed Jun 5 11:51:22 2019
read: IOPS=49, BW=1575MiB/s (1652MB/s)(2048MiB/1300msec)
clat (usec): min=19907, max=88711, avg=39813.93, stdev=10123.37
lat (usec): min=19908, max=88713, avg=39814.69, stdev=10123.41
clat percentiles (usec):
| 1.00th=[19792], 5.00th=[22414], 10.00th=[29754], 20.00th=[34866],
| 30.00th=[36439], 40.00th=[38011], 50.00th=[39060], 60.00th=[40109],
| 70.00th=[40633], 80.00th=[43779], 90.00th=[50594], 95.00th=[58459],
| 99.00th=[88605], 99.50th=[88605], 99.90th=[88605], 99.95th=[88605],
| 99.99th=[88605]
bw ( KiB/s): min=655360, max=851968, per=48.73%, avg=786039.50, stdev=92685.22, samples=4
iops : min= 20, max= 26, avg=23.75, stdev= 2.87, samples=4
lat (msec) : 20=1.56%, 50=87.50%, 100=10.94%
cpu : usr=0.00%, sys=5.68%, ctx=3101, majf=0, minf=16398
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=64,0,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=1575MiB/s (1652MB/s), 1575MiB/s-1575MiB/s (1652MB/s-1652MB/s), io=2048MiB (2147MB), run=1300-1300msec
job.20190605-114903.filestore-premium.randread-4k: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.filestore-premium.randread-4k: Laying out IO file (1 file / 1024MiB)
fio: native_fallocate call failed: Operation not supported
job.20190605-114903.filestore-premium.randread-4k: (groupid=0, jobs=2): err= 0: pid=4147: Wed Jun 5 11:51:15 2019
read: IOPS=14.1k, BW=54.9MiB/s (57.6MB/s)(1648MiB/30001msec)
clat (usec): min=87, max=9677, avg=140.82, stdev=116.42
lat (usec): min=87, max=9677, avg=140.94, stdev=116.42
clat percentiles (usec):
| 1.00th=[ 94], 5.00th=[ 96], 10.00th=[ 98], 20.00th=[ 101],
| 30.00th=[ 106], 40.00th=[ 115], 50.00th=[ 122], 60.00th=[ 129],
| 70.00th=[ 139], 80.00th=[ 159], 90.00th=[ 196], 95.00th=[ 235],
| 99.00th=[ 396], 99.50th=[ 529], 99.90th=[ 1369], 99.95th=[ 2057],
| 99.99th=[ 4883]
bw ( KiB/s): min=19144, max=31208, per=50.01%, avg=28131.98, stdev=2621.85, samples=120
iops : min= 4786, max= 7802, avg=7032.95, stdev=655.49, samples=120
lat (usec) : 100=16.18%, 250=79.81%, 500=3.45%, 750=0.30%, 1000=0.09%
lat (msec) : 2=0.11%, 4=0.04%, 10=0.01%
cpu : usr=1.62%, sys=4.27%, ctx=441896, majf=0, minf=17
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=421921,0,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=54.9MiB/s (57.6MB/s), 54.9MiB/s-54.9MiB/s (57.6MB/s-57.6MB/s), io=1648MiB (1728MB), run=30001-30001msec
job.20190605-114903.filestore-premium.randwrite-32m: (g=0): rw=randwrite, bs=(R) 32.0MiB-32.0MiB, (W) 32.0MiB-32.0MiB, (T) 32.0MiB-32.0MiB, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.filestore-premium.randwrite-32m: Laying out IO file (1 file / 1024MiB)
fio: native_fallocate call failed: Operation not supported
job.20190605-114903.filestore-premium.randwrite-32m: Laying out IO file (1 file / 1024MiB)
fio: native_fallocate call failed: Operation not supported
job.20190605-114903.filestore-premium.randwrite-32m: (groupid=0, jobs=2): err= 0: pid=4173: Wed Jun 5 11:52:05 2019
write: IOPS=5, BW=167MiB/s (175MB/s)(2048MiB/12247msec)
clat (msec): min=299, max=524, avg=378.95, stdev=35.23
lat (msec): min=301, max=526, avg=382.63, stdev=35.18
clat percentiles (msec):
| 1.00th=[ 300], 5.00th=[ 305], 10.00th=[ 359], 20.00th=[ 372],
| 30.00th=[ 376], 40.00th=[ 380], 50.00th=[ 380], 60.00th=[ 384],
| 70.00th=[ 384], 80.00th=[ 388], 90.00th=[ 397], 95.00th=[ 401],
| 99.00th=[ 527], 99.50th=[ 527], 99.90th=[ 527], 99.95th=[ 527],
| 99.99th=[ 527]
bw ( KiB/s): min=65405, max=132129, per=51.76%, avg=88635.17, stdev=31676.93, samples=46
iops : min= 1, max= 4, avg= 2.65, stdev= 1.02, samples=46
lat (msec) : 500=96.88%, 750=3.12%
cpu : usr=0.88%, sys=0.46%, ctx=1624, majf=0, minf=12
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=0,64,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=167MiB/s (175MB/s), 167MiB/s-167MiB/s (175MB/s-175MB/s), io=2048MiB (2147MB), run=12247-12247msec
job.20190605-114903.filestore-premium.randwrite-4k: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.filestore-premium.randwrite-4k: Laying out IO file (1 file / 1024MiB)
fio: native_fallocate call failed: Operation not supported
job.20190605-114903.filestore-premium.randwrite-4k: Laying out IO file (1 file / 1024MiB)
fio: native_fallocate call failed: Operation not supported
job.20190605-114903.filestore-premium.randwrite-4k: (groupid=0, jobs=2): err= 0: pid=4158: Wed Jun 5 11:51:52 2019
write: IOPS=820, BW=3283KiB/s (3361kB/s)(96.2MiB/30003msec)
clat (usec): min=1452, max=18200, avg=2434.24, stdev=792.88
lat (usec): min=1452, max=18201, avg=2434.76, stdev=792.91
clat percentiles (usec):
| 1.00th=[ 1745], 5.00th=[ 1860], 10.00th=[ 1942], 20.00th=[ 2024],
| 30.00th=[ 2114], 40.00th=[ 2180], 50.00th=[ 2245], 60.00th=[ 2343],
| 70.00th=[ 2442], 80.00th=[ 2606], 90.00th=[ 3032], 95.00th=[ 3556],
| 99.00th=[ 5735], 99.50th=[ 7046], 99.90th=[10552], 99.95th=[11994],
| 99.99th=[17433]
bw ( KiB/s): min= 1152, max= 1904, per=50.02%, avg=1641.69, stdev=189.34, samples=120
iops : min= 288, max= 476, avg=410.33, stdev=47.32, samples=120
lat (msec) : 2=16.53%, 4=80.14%, 10=3.20%, 20=0.13%
cpu : usr=0.22%, sys=0.31%, ctx=48730, majf=0, minf=11
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=0,24622,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=3283KiB/s (3361kB/s), 3283KiB/s-3283KiB/s (3361kB/s-3361kB/s), io=96.2MiB (101MB), run=30003-30003msec
job.20190605-114903.filestore-premium.read-32m: (g=0): rw=read, bs=(R) 32.0MiB-32.0MiB, (W) 32.0MiB-32.0MiB, (T) 32.0MiB-32.0MiB, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.filestore-premium.read-32m: Laying out IO file (1 file / 1024MiB)
fio: native_fallocate call failed: Operation not supported
job.20190605-114903.filestore-premium.read-32m: (groupid=0, jobs=2): err= 0: pid=4121: Wed Jun 5 11:49:58 2019
read: IOPS=57, BW=1843MiB/s (1933MB/s)(2048MiB/1111msec)
clat (usec): min=20167, max=55209, avg=34391.56, stdev=4943.49
lat (usec): min=20169, max=55577, avg=34408.96, stdev=4965.68
clat percentiles (usec):
| 1.00th=[20055], 5.00th=[27395], 10.00th=[29754], 20.00th=[31589],
| 30.00th=[32637], 40.00th=[33424], 50.00th=[33817], 60.00th=[34866],
| 70.00th=[35390], 80.00th=[37487], 90.00th=[40109], 95.00th=[42206],
| 99.00th=[55313], 99.50th=[55313], 99.90th=[55313], 99.95th=[55313],
| 99.99th=[55313]
bw ( KiB/s): min=917504, max=919342, per=48.63%, avg=917963.50, stdev=919.00, samples=4
iops : min= 28, max= 28, avg=28.00, stdev= 0.00, samples=4
lat (msec) : 50=98.44%, 100=1.56%
cpu : usr=0.00%, sys=8.18%, ctx=5854, majf=0, minf=16403
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=64,0,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=1843MiB/s (1933MB/s), 1843MiB/s-1843MiB/s (1933MB/s-1933MB/s), io=2048MiB (2147MB), run=1111-1111msec
job.20190605-114903.filestore-premium.read-4k: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.filestore-premium.read-4k: Laying out IO file (1 file / 1024MiB)
fio: native_fallocate call failed: Operation not supported
job.20190605-114903.filestore-premium.read-4k: (groupid=0, jobs=2): err= 0: pid=4116: Wed Jun 5 11:49:51 2019
read: IOPS=13.6k, BW=53.1MiB/s (55.7MB/s)(1594MiB/30001msec)
clat (usec): min=79, max=9638, avg=145.95, stdev=123.40
lat (usec): min=79, max=9638, avg=146.09, stdev=123.41
clat percentiles (usec):
| 1.00th=[ 91], 5.00th=[ 94], 10.00th=[ 97], 20.00th=[ 100],
| 30.00th=[ 106], 40.00th=[ 115], 50.00th=[ 123], 60.00th=[ 131],
| 70.00th=[ 143], 80.00th=[ 167], 90.00th=[ 208], 95.00th=[ 260],
| 99.00th=[ 465], 99.50th=[ 644], 99.90th=[ 1631], 99.95th=[ 2376],
| 99.99th=[ 4424]
bw ( KiB/s): min=15872, max=31543, per=50.09%, avg=27251.06, stdev=3290.36, samples=120
iops : min= 3968, max= 7885, avg=6812.70, stdev=822.57, samples=120
lat (usec) : 100=19.29%, 250=75.17%, 500=4.70%, 750=0.47%, 1000=0.15%
lat (msec) : 2=0.17%, 4=0.05%, 10=0.01%
cpu : usr=1.66%, sys=3.79%, ctx=425386, majf=0, minf=22
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=408015,0,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=53.1MiB/s (55.7MB/s), 53.1MiB/s-53.1MiB/s (55.7MB/s-55.7MB/s), io=1594MiB (1671MB), run=30001-30001msec
job.20190605-114903.filestore-premium.write-32m: (g=0): rw=write, bs=(R) 32.0MiB-32.0MiB, (W) 32.0MiB-32.0MiB, (T) 32.0MiB-32.0MiB, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.filestore-premium.write-32m: Laying out IO file (1 file / 1024MiB)
fio: native_fallocate call failed: Operation not supported
job.20190605-114903.filestore-premium.write-32m: Laying out IO file (1 file / 1024MiB)
fio: native_fallocate call failed: Operation not supported
job.20190605-114903.filestore-premium.write-32m: (groupid=0, jobs=2): err= 0: pid=4142: Wed Jun 5 11:50:39 2019
write: IOPS=6, BW=199MiB/s (208MB/s)(2048MiB/10307msec)
clat (msec): min=293, max=435, avg=318.59, stdev=23.07
lat (msec): min=296, max=438, avg=322.03, stdev=23.16
clat percentiles (msec):
| 1.00th=[ 296], 5.00th=[ 300], 10.00th=[ 300], 20.00th=[ 309],
| 30.00th=[ 313], 40.00th=[ 313], 50.00th=[ 313], 60.00th=[ 317],
| 70.00th=[ 321], 80.00th=[ 326], 90.00th=[ 334], 95.00th=[ 338],
| 99.00th=[ 435], 99.50th=[ 435], 99.90th=[ 435], 99.95th=[ 435],
| 99.99th=[ 435]
bw ( KiB/s): min=65405, max=131598, per=49.17%, avg=100043.98, stdev=33182.25, samples=40
iops : min= 1, max= 4, avg= 3.02, stdev= 1.05, samples=40
lat (msec) : 500=100.00%
cpu : usr=0.98%, sys=0.56%, ctx=1586, majf=0, minf=19
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=0,64,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=199MiB/s (208MB/s), 199MiB/s-199MiB/s (208MB/s-208MB/s), io=2048MiB (2147MB), run=10307-10307msec
job.20190605-114903.filestore-premium.write-4k: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.filestore-premium.write-4k: Laying out IO file (1 file / 1024MiB)
fio: native_fallocate call failed: Operation not supported
job.20190605-114903.filestore-premium.write-4k: Laying out IO file (1 file / 1024MiB)
fio: native_fallocate call failed: Operation not supported
job.20190605-114903.filestore-premium.write-4k: (groupid=0, jobs=2): err= 0: pid=4126: Wed Jun 5 11:50:28 2019
write: IOPS=841, BW=3366KiB/s (3447kB/s)(98.6MiB/30001msec)
clat (usec): min=930, max=22354, avg=2374.17, stdev=827.37
lat (usec): min=931, max=22355, avg=2374.51, stdev=827.39
clat percentiles (usec):
| 1.00th=[ 1696], 5.00th=[ 1811], 10.00th=[ 1876], 20.00th=[ 1975],
| 30.00th=[ 2040], 40.00th=[ 2114], 50.00th=[ 2212], 60.00th=[ 2278],
| 70.00th=[ 2409], 80.00th=[ 2573], 90.00th=[ 2900], 95.00th=[ 3392],
| 99.00th=[ 5407], 99.50th=[ 7177], 99.90th=[12518], 99.95th=[15401],
| 99.99th=[20579]
bw ( KiB/s): min= 1136, max= 1928, per=50.00%, avg=1683.11, stdev=186.71, samples=120
iops : min= 284, max= 482, avg=420.73, stdev=46.68, samples=120
lat (usec) : 1000=0.01%
lat (msec) : 2=23.85%, 4=73.34%, 10=2.60%, 20=0.18%, 50=0.01%
cpu : usr=0.13%, sys=0.40%, ctx=50073, majf=0, minf=16
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=0,25249,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=3366KiB/s (3447kB/s), 3366KiB/s-3366KiB/s (3447kB/s-3447kB/s), io=98.6MiB (103MB), run=30001-30001msec
job.20190605-114903.filestore-standard.randread-32m: (g=0): rw=randread, bs=(R) 32.0MiB-32.0MiB, (W) 32.0MiB-32.0MiB, (T) 32.0MiB-32.0MiB, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.filestore-standard.randread-32m: Laying out IO file (1 file / 1024MiB)
fio: native_fallocate call failed: Operation not supported
job.20190605-114903.filestore-standard.randread-32m: (groupid=0, jobs=2): err= 0: pid=4227: Wed Jun 5 11:54:52 2019
read: IOPS=42, BW=1362MiB/s (1428MB/s)(2048MiB/1504msec)
clat (usec): min=20975, max=78438, avg=46495.13, stdev=11228.86
lat (usec): min=20975, max=78439, avg=46496.34, stdev=11228.67
clat percentiles (usec):
| 1.00th=[21103], 5.00th=[31589], 10.00th=[33424], 20.00th=[37487],
| 30.00th=[39584], 40.00th=[41681], 50.00th=[44827], 60.00th=[47449],
| 70.00th=[51119], 80.00th=[54789], 90.00th=[60556], 95.00th=[68682],
| 99.00th=[78119], 99.50th=[78119], 99.90th=[78119], 99.95th=[78119],
| 99.99th=[78119]
bw ( KiB/s): min=524288, max=783298, per=46.96%, avg=654733.20, stdev=102635.24, samples=5
iops : min= 16, max= 23, avg=19.80, stdev= 2.86, samples=5
lat (msec) : 50=67.19%, 100=32.81%
cpu : usr=0.10%, sys=5.43%, ctx=4925, majf=0, minf=16400
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=64,0,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=1362MiB/s (1428MB/s), 1362MiB/s-1362MiB/s (1428MB/s-1428MB/s), io=2048MiB (2147MB), run=1504-1504msec
job.20190605-114903.filestore-standard.randread-4k: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.filestore-standard.randread-4k: Laying out IO file (1 file / 1024MiB)
fio: native_fallocate call failed: Operation not supported
job.20190605-114903.filestore-standard.randread-4k: (groupid=0, jobs=2): err= 0: pid=4221: Wed Jun 5 11:54:40 2019
read: IOPS=15.9k, BW=62.1MiB/s (65.1MB/s)(1863MiB/30001msec)
clat (usec): min=84, max=12109, avg=124.48, stdev=80.33
lat (usec): min=84, max=12109, avg=124.58, stdev=80.34
clat percentiles (usec):
| 1.00th=[ 92], 5.00th=[ 95], 10.00th=[ 99], 20.00th=[ 105],
| 30.00th=[ 113], 40.00th=[ 116], 50.00th=[ 119], 60.00th=[ 122],
| 70.00th=[ 126], 80.00th=[ 133], 90.00th=[ 145], 95.00th=[ 165],
| 99.00th=[ 235], 99.50th=[ 285], 99.90th=[ 824], 99.95th=[ 1500],
| 99.99th=[ 3097]
bw ( KiB/s): min=29232, max=33819, per=49.98%, avg=31772.36, stdev=999.79, samples=118
iops : min= 7308, max= 8454, avg=7943.03, stdev=249.97, samples=118
lat (usec) : 100=12.60%, 250=86.57%, 500=0.65%, 750=0.06%, 1000=0.03%
lat (msec) : 2=0.06%, 4=0.02%, 10=0.01%, 20=0.01%
cpu : usr=2.04%, sys=4.45%, ctx=490096, majf=0, minf=17
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=476820,0,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=62.1MiB/s (65.1MB/s), 62.1MiB/s-62.1MiB/s (65.1MB/s-65.1MB/s), io=1863MiB (1953MB), run=30001-30001msec
job.20190605-114903.filestore-standard.randwrite-32m: (g=0): rw=randwrite, bs=(R) 32.0MiB-32.0MiB, (W) 32.0MiB-32.0MiB, (T) 32.0MiB-32.0MiB, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.filestore-standard.randwrite-32m: Laying out IO file (1 file / 1024MiB)
fio: native_fallocate call failed: Operation not supported
job.20190605-114903.filestore-standard.randwrite-32m: Laying out IO file (1 file / 1024MiB)
fio: native_fallocate call failed: Operation not supported
job.20190605-114903.filestore-standard.randwrite-32m: (groupid=0, jobs=2): err= 0: pid=4247: Wed Jun 5 11:55:43 2019
write: IOPS=3, BW=107MiB/s (112MB/s)(2048MiB/19200msec)
clat (msec): min=356, max=777, avg=595.94, stdev=76.69
lat (msec): min=360, max=780, avg=599.85, stdev=76.96
clat percentiles (msec):
| 1.00th=[ 355], 5.00th=[ 384], 10.00th=[ 592], 20.00th=[ 600],
| 30.00th=[ 609], 40.00th=[ 609], 50.00th=[ 609], 60.00th=[ 617],
| 70.00th=[ 617], 80.00th=[ 617], 90.00th=[ 634], 95.00th=[ 651],
| 99.00th=[ 776], 99.50th=[ 776], 99.90th=[ 776], 99.95th=[ 776],
| 99.99th=[ 776]
bw ( KiB/s): min=65405, max=66064, per=60.23%, avg=65791.45, stdev=263.81, samples=62
iops : min= 1, max= 2, avg= 1.98, stdev= 0.13, samples=62
lat (msec) : 500=9.38%, 750=87.50%, 1000=3.12%
cpu : usr=0.51%, sys=0.37%, ctx=1635, majf=0, minf=13
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=0,64,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=107MiB/s (112MB/s), 107MiB/s-107MiB/s (112MB/s-112MB/s), io=2048MiB (2147MB), run=19200-19200msec
job.20190605-114903.filestore-standard.randwrite-4k: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.filestore-standard.randwrite-4k: Laying out IO file (1 file / 1024MiB)
fio: native_fallocate call failed: Operation not supported
job.20190605-114903.filestore-standard.randwrite-4k: Laying out IO file (1 file / 1024MiB)
fio: native_fallocate call failed: Operation not supported
job.20190605-114903.filestore-standard.randwrite-4k: (groupid=0, jobs=2): err= 0: pid=4232: Wed Jun 5 11:55:23 2019
write: IOPS=918, BW=3673KiB/s (3761kB/s)(108MiB/30002msec)
clat (usec): min=1053, max=40625, avg=2175.44, stdev=502.38
lat (usec): min=1054, max=40625, avg=2175.91, stdev=502.39
clat percentiles (usec):
| 1.00th=[ 1762], 5.00th=[ 1860], 10.00th=[ 1926], 20.00th=[ 1991],
| 30.00th=[ 2040], 40.00th=[ 2089], 50.00th=[ 2147], 60.00th=[ 2180],
| 70.00th=[ 2245], 80.00th=[ 2311], 90.00th=[ 2409], 95.00th=[ 2507],
| 99.00th=[ 3195], 99.50th=[ 4228], 99.90th=[ 7242], 99.95th=[ 9503],
| 99.99th=[15139]
bw ( KiB/s): min= 1672, max= 2040, per=50.02%, avg=1836.60, stdev=76.18, samples=120
iops : min= 418, max= 510, avg=459.07, stdev=19.08, samples=120
lat (msec) : 2=22.03%, 4=77.41%, 10=0.52%, 20=0.04%, 50=0.01%
cpu : usr=0.18%, sys=0.39%, ctx=53480, majf=0, minf=15
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=0,27548,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=3673KiB/s (3761kB/s), 3673KiB/s-3673KiB/s (3761kB/s-3761kB/s), io=108MiB (113MB), run=30002-30002msec
job.20190605-114903.filestore-standard.read-32m: (g=0): rw=read, bs=(R) 32.0MiB-32.0MiB, (W) 32.0MiB-32.0MiB, (T) 32.0MiB-32.0MiB, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.filestore-standard.read-32m: Laying out IO file (1 file / 1024MiB)
fio: native_fallocate call failed: Operation not supported
job.20190605-114903.filestore-standard.read-32m: (groupid=0, jobs=2): err= 0: pid=4194: Wed Jun 5 11:53:11 2019
read: IOPS=42, BW=1352MiB/s (1417MB/s)(2048MiB/1515msec)
clat (usec): min=23210, max=83602, avg=46822.63, stdev=13344.93
lat (usec): min=23211, max=83603, avg=46828.58, stdev=13343.24
clat percentiles (usec):
| 1.00th=[23200], 5.00th=[30016], 10.00th=[33424], 20.00th=[36963],
| 30.00th=[38011], 40.00th=[40633], 50.00th=[42206], 60.00th=[46400],
| 70.00th=[52167], 80.00th=[58459], 90.00th=[65274], 95.00th=[72877],
| 99.00th=[83362], 99.50th=[83362], 99.90th=[83362], 99.95th=[83362],
| 99.99th=[83362]
bw ( KiB/s): min=458752, max=917504, per=47.26%, avg=654263.20, stdev=179221.58, samples=5
iops : min= 14, max= 28, avg=19.60, stdev= 5.41, samples=5
lat (msec) : 50=67.19%, 100=32.81%
cpu : usr=0.00%, sys=5.49%, ctx=4811, majf=0, minf=16405
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=64,0,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=1352MiB/s (1417MB/s), 1352MiB/s-1352MiB/s (1417MB/s-1417MB/s), io=2048MiB (2147MB), run=1515-1515msec
job.20190605-114903.filestore-standard.read-4k: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.filestore-standard.read-4k: Laying out IO file (1 file / 1024MiB)
fio: native_fallocate call failed: Operation not supported
job.20190605-114903.filestore-standard.read-4k: (groupid=0, jobs=2): err= 0: pid=4178: Wed Jun 5 11:52:53 2019
read: IOPS=16.2k, BW=63.4MiB/s (66.5MB/s)(1901MiB/30001msec)
clat (usec): min=83, max=11058, avg=122.35, stdev=87.72
lat (usec): min=83, max=11058, avg=122.46, stdev=87.72
clat percentiles (usec):
| 1.00th=[ 90], 5.00th=[ 94], 10.00th=[ 97], 20.00th=[ 104],
| 30.00th=[ 111], 40.00th=[ 114], 50.00th=[ 117], 60.00th=[ 120],
| 70.00th=[ 124], 80.00th=[ 130], 90.00th=[ 143], 95.00th=[ 161],
| 99.00th=[ 231], 99.50th=[ 281], 99.90th=[ 807], 99.95th=[ 1401],
| 99.99th=[ 4752]
bw ( KiB/s): min=29104, max=34744, per=50.04%, avg=32473.35, stdev=1262.43, samples=118
iops : min= 7276, max= 8686, avg=8118.28, stdev=315.61, samples=118
lat (usec) : 100=15.28%, 250=83.93%, 500=0.62%, 750=0.06%, 1000=0.03%
lat (msec) : 2=0.05%, 4=0.02%, 10=0.01%, 20=0.01%
cpu : usr=1.79%, sys=4.38%, ctx=497657, majf=0, minf=22
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=486731,0,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=63.4MiB/s (66.5MB/s), 63.4MiB/s-63.4MiB/s (66.5MB/s-66.5MB/s), io=1901MiB (1994MB), run=30001-30001msec
job.20190605-114903.filestore-standard.write-32m: (g=0): rw=write, bs=(R) 32.0MiB-32.0MiB, (W) 32.0MiB-32.0MiB, (T) 32.0MiB-32.0MiB, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.filestore-standard.write-32m: Laying out IO file (1 file / 1024MiB)
fio: native_fallocate call failed: Operation not supported
job.20190605-114903.filestore-standard.write-32m: Laying out IO file (1 file / 1024MiB)
fio: native_fallocate call failed: Operation not supported
job.20190605-114903.filestore-standard.write-32m: (groupid=0, jobs=2): err= 0: pid=4204: Wed Jun 5 11:53:54 2019
write: IOPS=5, BW=170MiB/s (178MB/s)(2048MiB/12073msec)
clat (msec): min=353, max=527, avg=373.76, stdev=29.09
lat (msec): min=356, max=534, avg=377.19, stdev=29.36
clat percentiles (msec):
| 1.00th=[ 355], 5.00th=[ 359], 10.00th=[ 359], 20.00th=[ 363],
| 30.00th=[ 368], 40.00th=[ 368], 50.00th=[ 368], 60.00th=[ 368],
| 70.00th=[ 372], 80.00th=[ 376], 90.00th=[ 384], 95.00th=[ 401],
| 99.00th=[ 527], 99.50th=[ 527], 99.90th=[ 527], 99.95th=[ 527],
| 99.99th=[ 527]
bw ( KiB/s): min=65405, max=131598, per=50.95%, avg=88502.65, stdev=31626.56, samples=46
iops : min= 1, max= 4, avg= 2.67, stdev= 0.99, samples=46
lat (msec) : 500=96.88%, 750=3.12%
cpu : usr=0.73%, sys=0.58%, ctx=1537, majf=0, minf=19
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=0,64,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=170MiB/s (178MB/s), 170MiB/s-170MiB/s (178MB/s-178MB/s), io=2048MiB (2147MB), run=12073-12073msec
job.20190605-114903.filestore-standard.write-4k: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.filestore-standard.write-4k: Laying out IO file (1 file / 1024MiB)
fio: native_fallocate call failed: Operation not supported
job.20190605-114903.filestore-standard.write-4k: Laying out IO file (1 file / 1024MiB)
fio: native_fallocate call failed: Operation not supported
job.20190605-114903.filestore-standard.write-4k: (groupid=0, jobs=2): err= 0: pid=4199: Wed Jun 5 11:53:41 2019
write: IOPS=931, BW=3725KiB/s (3814kB/s)(109MiB/30003msec)
clat (usec): min=979, max=13650, avg=2145.74, stdev=436.97
lat (usec): min=979, max=13651, avg=2146.23, stdev=436.97
clat percentiles (usec):
| 1.00th=[ 1631], 5.00th=[ 1729], 10.00th=[ 1778], 20.00th=[ 1876],
| 30.00th=[ 1958], 40.00th=[ 2024], 50.00th=[ 2114], 60.00th=[ 2180],
| 70.00th=[ 2245], 80.00th=[ 2311], 90.00th=[ 2474], 95.00th=[ 2671],
| 99.00th=[ 3785], 99.50th=[ 4424], 99.90th=[ 7242], 99.95th=[ 8717],
| 99.99th=[12387]
bw ( KiB/s): min= 1448, max= 2192, per=50.01%, avg=1862.56, stdev=172.63, samples=120
iops : min= 362, max= 548, avg=465.58, stdev=43.18, samples=120
lat (usec) : 1000=0.01%
lat (msec) : 2=36.14%, 4=63.10%, 10=0.73%, 20=0.03%
cpu : usr=0.18%, sys=0.35%, ctx=44134, majf=0, minf=17
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=0,27937,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=3725KiB/s (3814kB/s), 3725KiB/s-3725KiB/s (3814kB/s-3814kB/s), io=109MiB (114MB), run=30003-30003msec
job.20190605-114903.pd-ssd.randread-32m: (g=0): rw=randread, bs=(R) 32.0MiB-32.0MiB, (W) 32.0MiB-32.0MiB, (T) 32.0MiB-32.0MiB, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.pd-ssd.randread-32m: Laying out IO file (1 file / 1024MiB)
job.20190605-114903.pd-ssd.randread-32m: (groupid=0, jobs=2): err= 0: pid=4311: Wed Jun 5 11:59:00 2019
read: IOPS=7, BW=242MiB/s (254MB/s)(2048MiB/8447msec)
clat (msec): min=142, max=267, avg=261.88, stdev=21.68
lat (msec): min=142, max=267, avg=261.88, stdev=21.68
clat percentiles (msec):
| 1.00th=[ 144], 5.00th=[ 266], 10.00th=[ 266], 20.00th=[ 266],
| 30.00th=[ 268], 40.00th=[ 268], 50.00th=[ 268], 60.00th=[ 268],
| 70.00th=[ 268], 80.00th=[ 268], 90.00th=[ 268], 95.00th=[ 268],
| 99.00th=[ 268], 99.50th=[ 268], 99.90th=[ 268], 99.95th=[ 268],
| 99.99th=[ 268]
bw ( KiB/s): min=65536, max=132129, per=49.65%, avg=123271.66, stdev=22086.76, samples=32
iops : min= 2, max= 4, avg= 3.72, stdev= 0.68, samples=32
lat (msec) : 250=4.69%, 500=95.31%
cpu : usr=0.00%, sys=0.54%, ctx=87, majf=0, minf=16396
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=64,0,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=242MiB/s (254MB/s), 242MiB/s-242MiB/s (254MB/s-254MB/s), io=2048MiB (2147MB), run=8447-8447msec
Disk stats (read/write):
sdb: ios=8128/2, merge=0/1, ticks=1570792/0, in_queue=1569812, util=98.48%
job.20190605-114903.pd-ssd.randread-4k: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.pd-ssd.randread-4k: Laying out IO file (1 file / 1024MiB)
job.20190605-114903.pd-ssd.randread-4k: (groupid=0, jobs=2): err= 0: pid=4295: Wed Jun 5 11:58:37 2019
read: IOPS=3521, BW=13.8MiB/s (14.4MB/s)(413MiB/30001msec)
clat (usec): min=191, max=15500, avg=566.38, stdev=212.33
lat (usec): min=191, max=15500, avg=566.52, stdev=212.33
clat percentiles (usec):
| 1.00th=[ 277], 5.00th=[ 334], 10.00th=[ 379], 20.00th=[ 461],
| 30.00th=[ 502], 40.00th=[ 529], 50.00th=[ 553], 60.00th=[ 578],
| 70.00th=[ 611], 80.00th=[ 644], 90.00th=[ 709], 95.00th=[ 766],
| 99.00th=[ 1172], 99.50th=[ 1729], 99.90th=[ 2835], 99.95th=[ 3032],
| 99.99th=[ 4686]
bw ( KiB/s): min= 6398, max= 8475, per=50.48%, avg=7110.18, stdev=392.27, samples=119
iops : min= 1599, max= 2118, avg=1777.24, stdev=98.03, samples=119
lat (usec) : 250=0.36%, 500=29.65%, 750=63.97%, 1000=4.54%
lat (msec) : 2=1.09%, 4=0.37%, 10=0.02%, 20=0.01%
cpu : usr=0.57%, sys=2.69%, ctx=105803, majf=0, minf=19
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=105655,0,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=13.8MiB/s (14.4MB/s), 13.8MiB/s-13.8MiB/s (14.4MB/s-14.4MB/s), io=413MiB (433MB), run=30001-30001msec
Disk stats (read/write):
sdb: ios=105615/2, merge=0/1, ticks=58192/0, in_queue=29812, util=99.13%
job.20190605-114903.pd-ssd.randwrite-32m: (g=0): rw=randwrite, bs=(R) 32.0MiB-32.0MiB, (W) 32.0MiB-32.0MiB, (T) 32.0MiB-32.0MiB, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.pd-ssd.randwrite-32m: Laying out IO file (1 file / 1024MiB)
job.20190605-114903.pd-ssd.randwrite-32m: (groupid=0, jobs=2): err= 0: pid=4321: Wed Jun 5 12:00:00 2019
write: IOPS=2, BW=71.2MiB/s (74.7MB/s)(2048MiB/28758msec)
clat (msec): min=441, max=1785, avg=881.29, stdev=428.65
lat (msec): min=445, max=1790, avg=884.79, stdev=428.92
clat percentiles (msec):
| 1.00th=[ 443], 5.00th=[ 443], 10.00th=[ 443], 20.00th=[ 447],
| 30.00th=[ 447], 40.00th=[ 447], 50.00th=[ 894], 60.00th=[ 894],
| 70.00th=[ 1334], 80.00th=[ 1334], 90.00th=[ 1334], 95.00th=[ 1368],
| 99.00th=[ 1787], 99.50th=[ 1787], 99.90th=[ 1787], 99.95th=[ 1787],
| 99.99th=[ 1787]
bw ( KiB/s): min=65536, max=132933, per=96.71%, avg=70527.34, stdev=16884.18, samples=58
iops : min= 2, max= 4, avg= 2.14, stdev= 0.51, samples=58
lat (msec) : 500=42.19%, 750=1.56%, 1000=18.75%, 2000=37.50%
cpu : usr=0.34%, sys=0.14%, ctx=220, majf=0, minf=15
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=0,64,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=71.2MiB/s (74.7MB/s), 71.2MiB/s-71.2MiB/s (74.7MB/s-74.7MB/s), io=2048MiB (2147MB), run=28758-28758msec
Disk stats (read/write):
sdb: ios=0/8176, merge=0/796, ticks=0/1822092, in_queue=1794892, util=97.71%
job.20190605-114903.pd-ssd.randwrite-4k: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.pd-ssd.randwrite-4k: Laying out IO file (1 file / 1024MiB)
job.20190605-114903.pd-ssd.randwrite-4k: (groupid=0, jobs=2): err= 0: pid=4316: Wed Jun 5 11:59:31 2019
write: IOPS=2666, BW=10.4MiB/s (10.9MB/s)(313MiB/30001msec)
clat (usec): min=254, max=40390, avg=748.05, stdev=3084.40
lat (usec): min=255, max=40391, avg=748.38, stdev=3084.41
clat percentiles (usec):
| 1.00th=[ 285], 5.00th=[ 306], 10.00th=[ 314], 20.00th=[ 330],
| 30.00th=[ 343], 40.00th=[ 355], 50.00th=[ 367], 60.00th=[ 379],
| 70.00th=[ 392], 80.00th=[ 412], 90.00th=[ 441], 95.00th=[ 474],
| 99.00th=[25035], 99.50th=[26870], 99.90th=[29492], 99.95th=[30802],
| 99.99th=[32375]
bw ( KiB/s): min= 4728, max= 5928, per=50.04%, avg=5337.80, stdev=260.09, samples=119
iops : min= 1182, max= 1482, avg=1334.35, stdev=65.00, samples=119
lat (usec) : 500=96.53%, 750=1.70%, 1000=0.18%
lat (msec) : 2=0.08%, 4=0.03%, 10=0.01%, 20=0.06%, 50=1.41%
cpu : usr=0.53%, sys=3.92%, ctx=161458, majf=0, minf=18
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=0,80008,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=10.4MiB/s (10.9MB/s), 10.4MiB/s-10.4MiB/s (10.9MB/s-10.9MB/s), io=313MiB (328MB), run=30001-30001msec
Disk stats (read/write):
sdb: ios=0/79781, merge=0/2399, ticks=0/27660, in_queue=28, util=0.03%
job.20190605-114903.pd-ssd.read-32m: (g=0): rw=read, bs=(R) 32.0MiB-32.0MiB, (W) 32.0MiB-32.0MiB, (T) 32.0MiB-32.0MiB, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.pd-ssd.read-32m: Laying out IO file (1 file / 1024MiB)
job.20190605-114903.pd-ssd.read-32m: (groupid=0, jobs=2): err= 0: pid=4269: Wed Jun 5 11:56:52 2019
read: IOPS=7, BW=242MiB/s (254MB/s)(2048MiB/8446msec)
clat (msec): min=158, max=268, avg=261.88, stdev=21.55
lat (msec): min=158, max=268, avg=261.88, stdev=21.55
clat percentiles (msec):
| 1.00th=[ 159], 5.00th=[ 266], 10.00th=[ 266], 20.00th=[ 268],
| 30.00th=[ 268], 40.00th=[ 268], 50.00th=[ 268], 60.00th=[ 268],
| 70.00th=[ 268], 80.00th=[ 268], 90.00th=[ 268], 95.00th=[ 268],
| 99.00th=[ 268], 99.50th=[ 268], 99.90th=[ 268], 99.95th=[ 268],
| 99.99th=[ 268]
bw ( KiB/s): min=65536, max=132129, per=49.69%, avg=123375.44, stdev=22115.58, samples=32
iops : min= 2, max= 4, avg= 3.75, stdev= 0.67, samples=32
lat (msec) : 250=4.69%, 500=95.31%
cpu : usr=0.00%, sys=0.53%, ctx=83, majf=0, minf=16403
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=64,0,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=242MiB/s (254MB/s), 242MiB/s-242MiB/s (254MB/s-254MB/s), io=2048MiB (2147MB), run=8446-8446msec
Disk stats (read/write):
sdb: ios=8105/2, merge=0/1, ticks=1567352/0, in_queue=1569152, util=98.56%
job.20190605-114903.pd-ssd.read-4k: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.pd-ssd.read-4k: Laying out IO file (1 file / 1024MiB)
job.20190605-114903.pd-ssd.read-4k: (groupid=0, jobs=2): err= 0: pid=4254: Wed Jun 5 11:56:28 2019
read: IOPS=4579, BW=17.9MiB/s (18.8MB/s)(537MiB/30001msec)
clat (usec): min=163, max=14280, avg=435.53, stdev=140.05
lat (usec): min=163, max=14280, avg=435.67, stdev=140.05
clat percentiles (usec):
| 1.00th=[ 297], 5.00th=[ 330], 10.00th=[ 347], 20.00th=[ 367],
| 30.00th=[ 383], 40.00th=[ 400], 50.00th=[ 420], 60.00th=[ 437],
| 70.00th=[ 457], 80.00th=[ 486], 90.00th=[ 529], 95.00th=[ 578],
| 99.00th=[ 742], 99.50th=[ 955], 99.90th=[ 2376], 99.95th=[ 2769],
| 99.99th=[ 4752]
bw ( KiB/s): min= 7816, max=10324, per=50.02%, avg=9162.53, stdev=698.08, samples=120
iops : min= 1954, max= 2581, avg=2290.57, stdev=174.53, samples=120
lat (usec) : 250=0.18%, 500=83.88%, 750=15.00%, 1000=0.49%
lat (msec) : 2=0.33%, 4=0.11%, 10=0.01%, 20=0.01%
cpu : usr=0.77%, sys=3.00%, ctx=137595, majf=0, minf=22
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=137394,0,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=17.9MiB/s (18.8MB/s), 17.9MiB/s-17.9MiB/s (18.8MB/s-18.8MB/s), io=537MiB (563MB), run=30001-30001msec
Disk stats (read/write):
sdb: ios=136574/2, merge=0/1, ticks=56968/0, in_queue=28856, util=96.44%
job.20190605-114903.pd-ssd.write-32m: (g=0): rw=write, bs=(R) 32.0MiB-32.0MiB, (W) 32.0MiB-32.0MiB, (T) 32.0MiB-32.0MiB, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.pd-ssd.write-32m: Laying out IO file (1 file / 1024MiB)
job.20190605-114903.pd-ssd.write-32m: (groupid=0, jobs=2): err= 0: pid=4280: Wed Jun 5 11:57:52 2019
write: IOPS=2, BW=71.5MiB/s (74.9MB/s)(2048MiB/28652msec)
clat (msec): min=442, max=1786, avg=885.10, stdev=480.46
lat (msec): min=445, max=1790, avg=888.37, stdev=480.69
clat percentiles (msec):
| 1.00th=[ 443], 5.00th=[ 443], 10.00th=[ 443], 20.00th=[ 443],
| 30.00th=[ 447], 40.00th=[ 447], 50.00th=[ 447], 60.00th=[ 1334],
| 70.00th=[ 1334], 80.00th=[ 1334], 90.00th=[ 1334], 95.00th=[ 1787],
| 99.00th=[ 1787], 99.50th=[ 1787], 99.90th=[ 1787], 99.95th=[ 1787],
| 99.99th=[ 1787]
bw ( KiB/s): min=65536, max=131598, per=95.91%, avg=70196.26, stdev=16821.85, samples=58
iops : min= 2, max= 4, avg= 2.14, stdev= 0.51, samples=58
lat (msec) : 500=51.56%, 1000=4.69%, 2000=43.75%
cpu : usr=0.33%, sys=0.12%, ctx=225, majf=0, minf=24
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=0,64,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=71.5MiB/s (74.9MB/s), 71.5MiB/s-71.5MiB/s (74.9MB/s-74.9MB/s), io=2048MiB (2147MB), run=28652-28652msec
Disk stats (read/write):
sdb: ios=0/8190, merge=0/34, ticks=0/1822128, in_queue=1787300, util=97.57%
job.20190605-114903.pd-ssd.write-4k: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.pd-ssd.write-4k: Laying out IO file (1 file / 1024MiB)
job.20190605-114903.pd-ssd.write-4k: (groupid=0, jobs=2): err= 0: pid=4275: Wed Jun 5 11:57:22 2019
write: IOPS=2808, BW=10.0MiB/s (11.5MB/s)(329MiB/30001msec)
clat (usec): min=235, max=48191, avg=710.40, stdev=3383.70
lat (usec): min=235, max=48192, avg=710.78, stdev=3383.72
clat percentiles (usec):
| 1.00th=[ 269], 5.00th=[ 285], 10.00th=[ 297], 20.00th=[ 314],
| 30.00th=[ 326], 40.00th=[ 334], 50.00th=[ 347], 60.00th=[ 359],
| 70.00th=[ 371], 80.00th=[ 392], 90.00th=[ 420], 95.00th=[ 449],
| 99.00th=[28181], 99.50th=[32375], 99.90th=[38011], 99.95th=[40633],
| 99.99th=[44303]
bw ( KiB/s): min= 4928, max= 6296, per=50.02%, avg=5619.11, stdev=294.62, samples=120
iops : min= 1232, max= 1574, avg=1404.69, stdev=73.64, samples=120
lat (usec) : 250=0.07%, 500=97.65%, 750=0.94%, 1000=0.12%
lat (msec) : 2=0.05%, 4=0.04%, 10=0.01%, 20=0.02%, 50=1.10%
cpu : usr=0.55%, sys=3.13%, ctx=169646, majf=0, minf=21
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=0,84262,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=10.0MiB/s (11.5MB/s), 10.0MiB/s-10.0MiB/s (11.5MB/s-11.5MB/s), io=329MiB (345MB), run=30001-30001msec
Disk stats (read/write):
sdb: ios=0/83973, merge=0/44, ticks=0/28100, in_queue=4, util=0.01%
job.20190605-114903.pd-standard.randread-32m: (g=0): rw=randread, bs=(R) 32.0MiB-32.0MiB, (W) 32.0MiB-32.0MiB, (T) 32.0MiB-32.0MiB, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.pd-standard.randread-32m: Laying out IO file (1 file / 1024MiB)
job.20190605-114903.pd-standard.randread-32m: (groupid=0, jobs=2): err= 0: pid=4385: Wed Jun 5 12:03:36 2019
read: IOPS=3, BW=116MiB/s (121MB/s)(2048MiB/17719msec)
clat (msec): min=295, max=568, avg=549.24, stdev=46.39
lat (msec): min=295, max=568, avg=549.24, stdev=46.39
clat percentiles (msec):
| 1.00th=[ 296], 5.00th=[ 550], 10.00th=[ 558], 20.00th=[ 558],
| 30.00th=[ 558], 40.00th=[ 558], 50.00th=[ 558], 60.00th=[ 558],
| 70.00th=[ 558], 80.00th=[ 558], 90.00th=[ 567], 95.00th=[ 567],
| 99.00th=[ 567], 99.50th=[ 567], 99.90th=[ 567], 99.95th=[ 567],
| 99.99th=[ 567]
bw ( KiB/s): min=65536, max=66331, per=55.66%, avg=65877.21, stdev=337.44, samples=62
iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=62
lat (msec) : 500=4.69%, 750=95.31%
cpu : usr=0.01%, sys=0.24%, ctx=76, majf=0, minf=16398
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=64,0,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=116MiB/s (121MB/s), 116MiB/s-116MiB/s (121MB/s-121MB/s), io=2048MiB (2147MB), run=17719-17719msec
Disk stats (read/write):
sdc: ios=8163/2, merge=0/1, ticks=3326292/1032, in_queue=3308096, util=99.03%
job.20190605-114903.pd-standard.randread-4k: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.pd-standard.randread-4k: Laying out IO file (1 file / 1024MiB)
job.20190605-114903.pd-standard.randread-4k: (groupid=0, jobs=2): err= 0: pid=4380: Wed Jun 5 12:03:03 2019
read: IOPS=205, BW=822KiB/s (842kB/s)(24.1MiB/30002msec)
clat (usec): min=263, max=105565, avg=9725.27, stdev=10509.33
lat (usec): min=263, max=105566, avg=9725.61, stdev=10509.36
clat percentiles (usec):
| 1.00th=[ 322], 5.00th=[ 355], 10.00th=[ 379], 20.00th=[ 429],
| 30.00th=[ 506], 40.00th=[ 1565], 50.00th=[ 8455], 60.00th=[ 11207],
| 70.00th=[ 13829], 80.00th=[ 17171], 90.00th=[ 22938], 95.00th=[ 28705],
| 99.00th=[ 43779], 99.50th=[ 49546], 99.90th=[ 71828], 99.95th=[ 83362],
| 99.99th=[105382]
bw ( KiB/s): min= 184, max= 1288, per=50.00%, avg=410.99, stdev=249.84, samples=120
iops : min= 46, max= 322, avg=102.72, stdev=62.44, samples=120
lat (usec) : 500=29.72%, 750=4.33%, 1000=1.93%
lat (msec) : 2=5.06%, 4=1.49%, 10=12.84%, 20=30.50%, 50=13.65%
lat (msec) : 100=0.45%, 250=0.02%
cpu : usr=0.08%, sys=0.32%, ctx=6193, majf=0, minf=15
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=6167,0,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=822KiB/s (842kB/s), 822KiB/s-822KiB/s (842kB/s-842kB/s), io=24.1MiB (25.3MB), run=30002-30002msec
Disk stats (read/write):
sdc: ios=6159/2, merge=0/1, ticks=59676/0, in_queue=29296, util=97.51%
job.20190605-114903.pd-standard.randwrite-32m: (g=0): rw=randwrite, bs=(R) 32.0MiB-32.0MiB, (W) 32.0MiB-32.0MiB, (T) 32.0MiB-32.0MiB, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.pd-standard.randwrite-32m: Laying out IO file (1 file / 1024MiB)
job.20190605-114903.pd-standard.randwrite-32m: (groupid=0, jobs=2): err= 0: pid=4406: Wed Jun 5 12:04:36 2019
write: IOPS=2, BW=71.4MiB/s (74.9MB/s)(2048MiB/28684msec)
clat (msec): min=442, max=1361, avg=878.59, stdev=412.70
lat (msec): min=445, max=1364, avg=882.47, stdev=412.87
clat percentiles (msec):
| 1.00th=[ 443], 5.00th=[ 443], 10.00th=[ 443], 20.00th=[ 443],
| 30.00th=[ 447], 40.00th=[ 456], 50.00th=[ 894], 60.00th=[ 1334],
| 70.00th=[ 1334], 80.00th=[ 1334], 90.00th=[ 1334], 95.00th=[ 1334],
| 99.00th=[ 1368], 99.50th=[ 1368], 99.90th=[ 1368], 99.95th=[ 1368],
| 99.99th=[ 1368]
bw ( KiB/s): min=65536, max=131334, per=93.38%, avg=68275.28, stdev=11796.11, samples=60
iops : min= 2, max= 4, avg= 2.07, stdev= 0.36, samples=60
lat (msec) : 500=43.75%, 1000=15.62%, 2000=40.62%
cpu : usr=0.42%, sys=0.10%, ctx=225, majf=0, minf=17
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=0,64,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=71.4MiB/s (74.9MB/s), 71.4MiB/s-71.4MiB/s (74.9MB/s-74.9MB/s), io=2048MiB (2147MB), run=28684-28684msec
Disk stats (read/write):
sdc: ios=0/8185, merge=0/560, ticks=0/1823364, in_queue=1793628, util=97.56%
job.20190605-114903.pd-standard.randwrite-4k: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.pd-standard.randwrite-4k: Laying out IO file (1 file / 1024MiB)
job.20190605-114903.pd-standard.randwrite-4k: (groupid=0, jobs=2): err= 0: pid=4390: Wed Jun 5 12:04:07 2019
write: IOPS=1537, BW=6152KiB/s (6299kB/s)(180MiB/30002msec)
clat (usec): min=303, max=80537, avg=1298.37, stdev=5173.05
lat (usec): min=303, max=80537, avg=1298.67, stdev=5173.05
clat percentiles (usec):
| 1.00th=[ 334], 5.00th=[ 355], 10.00th=[ 367], 20.00th=[ 383],
| 30.00th=[ 396], 40.00th=[ 408], 50.00th=[ 424], 60.00th=[ 441],
| 70.00th=[ 465], 80.00th=[ 1467], 90.00th=[ 1647], 95.00th=[ 1729],
| 99.00th=[40109], 99.50th=[42730], 99.90th=[50070], 99.95th=[52691],
| 99.99th=[55313]
bw ( KiB/s): min= 2760, max= 4312, per=50.02%, avg=3076.94, stdev=176.27, samples=120
iops : min= 690, max= 1078, avg=769.17, stdev=44.07, samples=120
lat (usec) : 500=76.65%, 750=2.99%, 1000=0.12%
lat (msec) : 2=18.52%, 4=0.10%, 10=0.01%, 20=0.01%, 50=1.49%
lat (msec) : 100=0.11%
cpu : usr=0.28%, sys=2.49%, ctx=93201, majf=0, minf=20
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=0,46141,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=6152KiB/s (6299kB/s), 6152KiB/s-6152KiB/s (6299kB/s-6299kB/s), io=180MiB (189MB), run=30002-30002msec
Disk stats (read/write):
sdc: ios=0/46013, merge=0/1577, ticks=0/28524, in_queue=92, util=0.19%
job.20190605-114903.pd-standard.read-32m: (g=0): rw=read, bs=(R) 32.0MiB-32.0MiB, (W) 32.0MiB-32.0MiB, (T) 32.0MiB-32.0MiB, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.pd-standard.read-32m: Laying out IO file (1 file / 1024MiB)
job.20190605-114903.pd-standard.read-32m: (groupid=0, jobs=2): err= 0: pid=4343: Wed Jun 5 12:01:18 2019
read: IOPS=3, BW=116MiB/s (121MB/s)(2048MiB/17729msec)
clat (msec): min=313, max=576, avg=549.61, stdev=45.94
lat (msec): min=313, max=576, avg=549.61, stdev=45.94
clat percentiles (msec):
| 1.00th=[ 313], 5.00th=[ 550], 10.00th=[ 550], 20.00th=[ 558],
| 30.00th=[ 558], 40.00th=[ 558], 50.00th=[ 558], 60.00th=[ 558],
| 70.00th=[ 558], 80.00th=[ 567], 90.00th=[ 567], 95.00th=[ 567],
| 99.00th=[ 575], 99.50th=[ 575], 99.90th=[ 575], 99.95th=[ 575],
| 99.99th=[ 575]
bw ( KiB/s): min=65536, max=66064, per=55.58%, avg=65750.63, stdev=216.42, samples=62
iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=62
lat (msec) : 500=4.69%, 750=95.31%
cpu : usr=0.00%, sys=0.26%, ctx=80, majf=0, minf=16406
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=64,0,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=116MiB/s (121MB/s), 116MiB/s-116MiB/s (121MB/s-121MB/s), io=2048MiB (2147MB), run=17729-17729msec
Disk stats (read/write):
sdc: ios=8144/2, merge=0/1, ticks=3308268/864, in_queue=3294296, util=98.93%
job.20190605-114903.pd-standard.read-4k: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.pd-standard.read-4k: Laying out IO file (1 file / 1024MiB)
job.20190605-114903.pd-standard.read-4k: (groupid=0, jobs=2): err= 0: pid=4337: Wed Jun 5 12:00:45 2019
read: IOPS=4907, BW=19.2MiB/s (20.1MB/s)(575MiB/30001msec)
clat (usec): min=199, max=75257, avg=406.38, stdev=1315.98
lat (usec): min=199, max=75257, avg=406.51, stdev=1315.98
clat percentiles (usec):
| 1.00th=[ 237], 5.00th=[ 258], 10.00th=[ 269], 20.00th=[ 285],
| 30.00th=[ 293], 40.00th=[ 306], 50.00th=[ 318], 60.00th=[ 326],
| 70.00th=[ 343], 80.00th=[ 359], 90.00th=[ 388], 95.00th=[ 424],
| 99.00th=[ 832], 99.50th=[ 4359], 99.90th=[21890], 99.95th=[28443],
| 99.99th=[42206]
bw ( KiB/s): min= 6136, max=11312, per=50.15%, avg=9845.47, stdev=809.33, samples=118
iops : min= 1534, max= 2828, avg=2461.32, stdev=202.35, samples=118
lat (usec) : 250=3.23%, 500=94.40%, 750=1.24%, 1000=0.31%
lat (msec) : 2=0.25%, 4=0.07%, 10=0.17%, 20=0.20%, 50=0.12%
lat (msec) : 100=0.01%
cpu : usr=0.68%, sys=3.45%, ctx=147507, majf=0, minf=24
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=147244,0,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=19.2MiB/s (20.1MB/s), 19.2MiB/s-19.2MiB/s (20.1MB/s-20.1MB/s), io=575MiB (603MB), run=30001-30001msec
Disk stats (read/write):
sdc: ios=146940/2, merge=0/1, ticks=57392/4, in_queue=29384, util=98.07%
job.20190605-114903.pd-standard.write-32m: (g=0): rw=write, bs=(R) 32.0MiB-32.0MiB, (W) 32.0MiB-32.0MiB, (T) 32.0MiB-32.0MiB, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.pd-standard.write-32m: Laying out IO file (1 file / 1024MiB)
job.20190605-114903.pd-standard.write-32m: (groupid=0, jobs=2): err= 0: pid=4363: Wed Jun 5 12:02:18 2019
write: IOPS=2, BW=71.5MiB/s (74.0MB/s)(2048MiB/28649msec)
clat (msec): min=442, max=1786, avg=884.44, stdev=368.53
lat (msec): min=445, max=1790, avg=888.30, stdev=368.89
clat percentiles (msec):
| 1.00th=[ 443], 5.00th=[ 443], 10.00th=[ 443], 20.00th=[ 447],
| 30.00th=[ 447], 40.00th=[ 885], 50.00th=[ 894], 60.00th=[ 894],
| 70.00th=[ 911], 80.00th=[ 1334], 90.00th=[ 1334], 95.00th=[ 1334],
| 99.00th=[ 1787], 99.50th=[ 1787], 99.90th=[ 1787], 99.95th=[ 1787],
| 99.99th=[ 1787]
bw ( KiB/s): min=65536, max=131334, per=94.13%, avg=68906.08, stdev=14532.65, samples=59
iops : min= 2, max= 4, avg= 2.10, stdev= 0.44, samples=59
lat (msec) : 500=32.81%, 1000=37.50%, 2000=29.69%
cpu : usr=0.44%, sys=0.06%, ctx=218, majf=0, minf=20
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=0,64,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=71.5MiB/s (74.0MB/s), 71.5MiB/s-71.5MiB/s (74.0MB/s-74.0MB/s), io=2048MiB (2147MB), run=28649-28649msec
Disk stats (read/write):
sdc: ios=0/8183, merge=0/31, ticks=0/1817604, in_queue=1785076, util=97.49%
job.20190605-114903.pd-standard.write-4k: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
...
fio-3.1
Starting 2 processes
job.20190605-114903.pd-standard.write-4k: Laying out IO file (1 file / 1024MiB)
job.20190605-114903.pd-standard.write-4k: (groupid=0, jobs=2): err= 0: pid=4358: Wed Jun 5 12:01:49 2019
write: IOPS=2503, BW=9.78MiB/s (10.3MB/s)(293MiB/30001msec)
clat (usec): min=284, max=62875, avg=797.30, stdev=3860.65
lat (usec): min=285, max=62876, avg=797.66, stdev=3860.66
clat percentiles (usec):
| 1.00th=[ 322], 5.00th=[ 338], 10.00th=[ 347], 20.00th=[ 363],
| 30.00th=[ 371], 40.00th=[ 379], 50.00th=[ 392], 60.00th=[ 400],
| 70.00th=[ 412], 80.00th=[ 429], 90.00th=[ 453], 95.00th=[ 478],
| 99.00th=[32637], 99.50th=[38011], 99.90th=[41681], 99.95th=[43254],
| 99.99th=[45876]
bw ( KiB/s): min= 4344, max= 5448, per=50.03%, avg=5009.29, stdev=212.89, samples=119
iops : min= 1086, max= 1362, avg=1252.27, stdev=53.21, samples=119
lat (usec) : 500=96.92%, 750=1.80%, 1000=0.09%
lat (msec) : 2=0.04%, 4=0.06%, 10=0.01%, 20=0.01%, 50=1.07%
lat (msec) : 100=0.01%
cpu : usr=0.51%, sys=2.63%, ctx=151171, majf=0, minf=23
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=0,75106,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=9.78MiB/s (10.3MB/s), 9.78MiB/s-9.78MiB/s (10.3MB/s-10.3MB/s), io=293MiB (308MB), run=30001-30001msec
Disk stats (read/write):
sdc: ios=0/74833, merge=0/33, ticks=0/28380, in_queue=16, util=0.05%
/dev/sdb1 /mnt/pd-ssd ext4 rw,relatime,data=ordered 0 0
/dev/sdc1 /mnt/pd-standard ext4 rw,relatime,data=ordered 0 0
10.121.215.122:/default /mnt/filestore-standard nfs rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.121.215.122,mountvers=3,mountport=2050,mountproto=tcp,local_lock=none,addr=10.121.215.122 0 0
10.249.22.218:/default /mnt/filestore-premium nfs rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.249.22.218,mountvers=3,mountport=2050,mountproto=tcp,local_lock=none,addr=10.249.22.218 0 0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment