Created
May 8, 2019 20:53
-
-
Save jtaleric/99037179ad0dd6b2662d0b343c081c0e to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<example-benchmark-fio-server-951-benchmark-9cd4p> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:54 2019 | |
write: IOPS=36, BW=2354KiB/s (2411kB/s)(276MiB/120060msec) | |
slat (nsec): min=0, max=504592k, avg=441726.08, stdev=10395455.15 | |
clat (nsec): min=0, max=995152k, avg=108771492.32, stdev=167527907.41 | |
lat (nsec): min=0, max=995235k, avg=109214317.91, stdev=167771537.56 | |
clat percentiles (usec): | |
| 1.00th=[ 988], 5.00th=[ 1172], 10.00th=[ 1336], 20.00th=[ 1778], | |
| 30.00th=[ 2671], 40.00th=[ 6194], 50.00th=[ 34341], 60.00th=[ 77071], | |
| 70.00th=[107480], 80.00th=[158335], 90.00th=[358613], 95.00th=[501220], | |
| 99.00th=[759170], 99.50th=[868221], 99.90th=[935330], 99.95th=[935330], | |
| 99.99th=[994051] | |
bw ( KiB/s): min= 0, max= 3002, per=52.90%, avg=2352.50, stdev=918.53, samples=2 | |
iops : min= 0, max= 46, avg=36.00, stdev=14.14, samples=2 | |
lat (usec) : 4=0.02%, 500=0.02%, 750=0.07%, 1000=1.13% | |
lat (msec) : 2=22.09%, 4=12.19%, 10=7.36%, 20=4.44%, 50=4.51% | |
lat (msec) : 100=15.57%, 250=17.99%, 500=9.61%, 750=4.03%, 1000=1.02% | |
cpu : usr=0.04%, sys=0.16%, ctx=8445, majf=0, minf=31 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,4413,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:54 2019 | |
read: IOPS=348, BW=21.8MiB/s (22.8MB/s)(2614MiB/120030msec) | |
slat (nsec): min=0, max=498780k, avg=50078.56, stdev=3579794.07 | |
clat (nsec): min=0, max=937417k, avg=11464988.97, stdev=59818482.14 | |
lat (nsec): min=0, max=937452k, avg=11515884.91, stdev=59935424.53 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 732], 40.00th=[ 740], | |
| 50.00th=[ 764], 60.00th=[ 852], 70.00th=[ 924], | |
| 80.00th=[ 1288], 90.00th=[ 1941504], 95.00th=[ 51118080], | |
| 99.00th=[362807296], 99.50th=[471859200], 99.90th=[692060160], | |
| 99.95th=[734003200], 99.99th=[843055104] | |
bw ( KiB/s): min= 0, max=28792, per=91.76%, avg=22304.00, stdev=9175.42, samples=2 | |
iops : min= 0, max= 450, avg=348.50, stdev=143.54, samples=2 | |
lat (nsec) : 750=47.82%, 1000=25.51% | |
lat (usec) : 2=9.42%, 4=0.02%, 10=0.63%, 20=0.59%, 50=0.28% | |
lat (usec) : 100=0.10%, 250=0.04%, 500=0.29%, 750=1.01%, 1000=1.91% | |
lat (msec) : 2=2.44%, 4=1.35%, 10=0.63%, 20=0.48%, 50=2.41% | |
lat (msec) : 100=2.11%, 250=1.42%, 500=1.13%, 750=0.38%, 1000=0.03% | |
cpu : usr=0.11%, sys=0.37%, ctx=12986, majf=0, minf=31 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=41881,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:54 2019 | |
read: IOPS=31, BW=2014KiB/s (2062kB/s)(236MiB/120065msec) | |
slat (nsec): min=0, max=464438k, avg=251788.17, stdev=10003721.86 | |
clat (nsec): min=0, max=761107k, avg=7488674.89, stdev=41769913.15 | |
lat (nsec): min=0, max=761136k, avg=7741358.10, stdev=42918867.18 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 756], 30.00th=[ 844], 40.00th=[ 924], | |
| 50.00th=[ 1080], 60.00th=[ 1208], 70.00th=[ 1336], | |
| 80.00th=[ 1624], 90.00th=[ 1204224], 95.00th=[ 38535168], | |
| 99.00th=[214958080], 99.50th=[325058560], 99.90th=[534773760], | |
| 99.95th=[675282944], 99.99th=[759169024] | |
bw ( KiB/s): min= 0, max= 2421, per=8.29%, avg=2014.50, stdev=574.88, samples=2 | |
iops : min= 0, max= 37, avg=31.00, stdev= 8.49, samples=2 | |
write: IOPS=32, BW=2094KiB/s (2144kB/s)(246MiB/120065msec) | |
slat (nsec): min=0, max=552114k, avg=319489.42, stdev=8814876.07 | |
clat (nsec): min=0, max=1106.9M, avg=115024253.69, stdev=172725727.36 | |
lat (nsec): min=0, max=1107.2M, avg=115344697.54, stdev=172831915.20 | |
clat percentiles (usec): | |
| 1.00th=[ 1020], 5.00th=[ 1205], 10.00th=[ 1401], | |
| 20.00th=[ 1876], 30.00th=[ 2900], 40.00th=[ 8029], | |
| 50.00th=[ 46924], 60.00th=[ 82314], 70.00th=[ 111674], | |
| 80.00th=[ 175113], 90.00th=[ 371196], 95.00th=[ 509608], | |
| 99.00th=[ 759170], 99.50th=[ 876610], 99.90th=[ 960496], | |
| 99.95th=[ 994051], 99.99th=[1098908] | |
bw ( KiB/s): min= 0, max= 2517, per=47.05%, avg=2092.50, stdev=600.33, samples=2 | |
iops : min= 0, max= 39, avg=32.50, stdev= 9.19, samples=2 | |
lat (nsec) : 750=9.36%, 1000=12.80% | |
lat (usec) : 2=17.65%, 4=0.09%, 10=0.25%, 20=0.70%, 50=0.53% | |
lat (usec) : 100=0.62%, 250=0.32%, 500=0.42%, 750=0.45%, 1000=1.00% | |
lat (msec) : 2=11.62%, 4=6.70%, 10=3.98%, 20=2.15%, 50=4.18% | |
lat (msec) : 100=9.19%, 250=9.53%, 500=5.70%, 750=2.26%, 1000=0.52% | |
cpu : usr=0.04%, sys=0.19%, ctx=8692, majf=0, minf=34 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=3779,3925,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=23.7MiB/s (24.9MB/s), 2014KiB/s-21.8MiB/s (2062kB/s-22.8MB/s), io=2850MiB (2989MB), run=120030-120065msec | |
WRITE: bw=4448KiB/s (4554kB/s), 2094KiB/s-2354KiB/s (2144kB/s-2411kB/s), io=522MiB (547MB), run=120060-120065msec | |
Disk stats (read/write): | |
rbd300: ios=7336/8516, merge=0/1, ticks=525672/830529, in_queue=1348119, util=12.06% | |
Jobs: 117 (f=116): [_(2),m(1)][99.9%][r=1334M,w=800M][r=20.8k,w=12.5k IOPS][eta 00m:56s] <example-benchmark-fio-server-255-benchmark-v6glb> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:54 2019 | |
write: IOPS=33, BW=2171KiB/s (2223kB/s)(254MiB/120036msec) | |
slat (nsec): min=0, max=536048k, avg=417781.11, stdev=10887713.42 | |
clat (nsec): min=0, max=1261.6M, avg=118027748.44, stdev=178274062.80 | |
lat (nsec): min=0, max=1261.6M, avg=118446560.37, stdev=178442905.58 | |
clat percentiles (usec): | |
| 1.00th=[ 947], 5.00th=[ 1205], 10.00th=[ 1467], | |
| 20.00th=[ 2114], 30.00th=[ 3654], 40.00th=[ 10028], | |
| 50.00th=[ 58459], 60.00th=[ 86508], 70.00th=[ 111674], | |
| 80.00th=[ 177210], 90.00th=[ 371196], 95.00th=[ 530580], | |
| 99.00th=[ 775947], 99.50th=[ 901776], 99.90th=[1098908], | |
| 99.95th=[1098908], 99.99th=[1266680] | |
bw ( KiB/s): min= 0, max= 3001, per=53.65%, avg=2168.50, stdev=1177.33, samples=2 | |
iops : min= 0, max= 46, avg=33.00, stdev=18.38, samples=2 | |
lat (usec) : 750=0.05%, 1000=1.38% | |
lat (msec) : 2=17.77%, 4=11.92%, 10=8.85%, 20=3.91%, 50=4.47% | |
lat (msec) : 100=17.16%, 250=19.59%, 500=9.05%, 750=4.62%, 1000=0.96% | |
cpu : usr=0.04%, sys=0.15%, ctx=7612, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,4068,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:54 2019 | |
read: IOPS=273, BW=17.1MiB/s (17.9MB/s)(2052MiB/120023msec) | |
slat (nsec): min=0, max=393291k, avg=46338.23, stdev=2796058.39 | |
clat (nsec): min=0, max=1038.1M, avg=14624047.78, stdev=65201130.73 | |
lat (nsec): min=0, max=1038.6M, avg=14671240.74, stdev=65271622.43 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 732], 40.00th=[ 756], | |
| 50.00th=[ 844], 60.00th=[ 908], 70.00th=[ 1144], | |
| 80.00th=[ 724992], 90.00th=[ 25559040], 95.00th=[ 67633152], | |
| 99.00th=[379584512], 99.50th=[488636416], 99.90th=[700448768], | |
| 99.95th=[750780416], 99.99th=[943718400] | |
bw ( KiB/s): min= 0, max=22634, per=90.78%, avg=17511.00, stdev=7245.02, samples=2 | |
iops : min= 0, max= 354, avg=273.50, stdev=113.84, samples=2 | |
lat (nsec) : 750=39.24%, 1000=25.97% | |
lat (usec) : 2=12.59%, 4=0.05%, 10=0.41%, 20=0.41%, 50=0.17% | |
lat (usec) : 100=0.09%, 250=0.03%, 500=0.24%, 750=1.12%, 1000=2.31% | |
lat (msec) : 2=2.99%, 4=1.91%, 10=0.98%, 20=0.94%, 50=3.84% | |
lat (msec) : 100=2.95%, 250=1.98%, 500=1.34%, 750=0.40%, 1000=0.05% | |
cpu : usr=0.10%, sys=0.33%, ctx=13746, majf=0, minf=29 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=32856,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:54 2019 | |
read: IOPS=27, BW=1786KiB/s (1829kB/s)(209MiB/120059msec) | |
slat (nsec): min=0, max=535574k, avg=187960.89, stdev=9251743.42 | |
clat (nsec): min=0, max=794000k, avg=10453672.56, stdev=48478017.97 | |
lat (nsec): min=0, max=794034k, avg=10642538.71, stdev=49327238.20 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 764], 30.00th=[ 876], 40.00th=[ 988], | |
| 50.00th=[ 1128], 60.00th=[ 1256], 70.00th=[ 1448], | |
| 80.00th=[ 100864], 90.00th=[ 3194880], 95.00th=[ 53739520], | |
| 99.00th=[270532608], 99.50th=[362807296], 99.90th=[522190848], | |
| 99.95th=[608174080], 99.99th=[792723456] | |
bw ( KiB/s): min= 0, max= 2412, per=9.26%, avg=1786.50, stdev=884.59, samples=2 | |
iops : min= 0, max= 37, avg=27.50, stdev=13.44, samples=2 | |
write: IOPS=29, BW=1873KiB/s (1918kB/s)(220MiB/120059msec) | |
slat (nsec): min=0, max=400348k, avg=279434.69, stdev=6757705.47 | |
clat (nsec): min=0, max=1261.4M, avg=126879957.42, stdev=186908714.01 | |
lat (nsec): min=0, max=1261.5M, avg=127160436.98, stdev=186957434.42 | |
clat percentiles (usec): | |
| 1.00th=[ 1004], 5.00th=[ 1287], 10.00th=[ 1549], | |
| 20.00th=[ 2278], 30.00th=[ 4228], 40.00th=[ 15795], | |
| 50.00th=[ 63701], 60.00th=[ 89654], 70.00th=[ 117965], | |
| 80.00th=[ 196084], 90.00th=[ 396362], 95.00th=[ 549454], | |
| 99.00th=[ 826278], 99.50th=[ 935330], 99.90th=[1098908], | |
| 99.95th=[1098908], 99.99th=[1266680] | |
bw ( KiB/s): min= 0, max= 2512, per=46.29%, avg=1871.00, stdev=906.51, samples=2 | |
iops : min= 0, max= 39, avg=29.00, stdev=14.14, samples=2 | |
lat (nsec) : 750=8.58%, 1000=11.38% | |
lat (usec) : 2=16.15%, 4=0.07%, 10=0.28%, 20=0.76%, 50=0.80% | |
lat (usec) : 100=1.02%, 250=0.67%, 500=0.63%, 750=0.63%, 1000=1.46% | |
lat (msec) : 2=9.58%, 4=7.00%, 10=4.52%, 20=2.42%, 50=3.92% | |
lat (msec) : 100=10.00%, 250=11.09%, 500=5.71%, 750=2.48%, 1000=0.71% | |
cpu : usr=0.04%, sys=0.17%, ctx=8048, majf=0, minf=33 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=3351,3510,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=18.8MiB/s (19.8MB/s), 1786KiB/s-17.1MiB/s (1829kB/s-17.9MB/s), io=2262MiB (2372MB), run=120023-120059msec | |
WRITE: bw=4043KiB/s (4140kB/s), 1873KiB/s-2171KiB/s (1918kB/s-2223kB/s), io=474MiB (497MB), run=120036-120059msec | |
Disk stats (read/write): | |
rbd75: ios=8536/9131, merge=0/1, ticks=554901/875234, in_queue=1421062, util=12.57% | |
<example-benchmark-fio-server-424-benchmark-tlngh> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:54 2019 | |
write: IOPS=41, BW=2666KiB/s (2730kB/s)(313MiB/120074msec) | |
slat (nsec): min=0, max=400546k, avg=326325.35, stdev=6957801.48 | |
clat (nsec): min=0, max=1102.3M, avg=96092666.22, stdev=166659881.38 | |
lat (nsec): min=0, max=1102.3M, avg=96420079.84, stdev=167148622.48 | |
clat percentiles (usec): | |
| 1.00th=[ 1012], 5.00th=[ 1254], 10.00th=[ 1467], | |
| 20.00th=[ 1991], 30.00th=[ 2835], 40.00th=[ 4686], | |
| 50.00th=[ 11338], 60.00th=[ 45876], 70.00th=[ 86508], | |
| 80.00th=[ 133694], 90.00th=[ 346031], 95.00th=[ 480248], | |
| 99.00th=[ 801113], 99.50th=[ 843056], 99.90th=[ 943719], | |
| 99.95th=[ 985662], 99.99th=[1098908] | |
bw ( KiB/s): min= 0, max= 2965, per=55.11%, avg=2665.00, stdev=424.26, samples=2 | |
iops : min= 0, max= 46, avg=41.00, stdev= 7.07, samples=2 | |
lat (usec) : 500=0.02%, 750=0.04%, 1000=0.78% | |
lat (msec) : 2=19.24%, 4=16.86%, 10=12.00%, 20=6.18%, 50=5.52% | |
lat (msec) : 100=13.06%, 250=12.44%, 500=9.76%, 750=2.90%, 1000=1.22% | |
cpu : usr=0.05%, sys=0.18%, ctx=9369, majf=0, minf=31 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,4999,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:54 2019 | |
read: IOPS=318, BW=19.9MiB/s (20.9MB/s)(2389MiB/120031msec) | |
slat (nsec): min=0, max=320204k, avg=38057.84, stdev=2181510.66 | |
clat (nsec): min=0, max=1057.4M, avg=12541947.82, stdev=61296772.33 | |
lat (nsec): min=0, max=1057.4M, avg=12580866.82, stdev=61342276.53 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 740], 40.00th=[ 788], | |
| 50.00th=[ 852], 60.00th=[ 900], 70.00th=[ 1096], | |
| 80.00th=[ 43776], 90.00th=[ 6324224], 95.00th=[ 58982400], | |
| 99.00th=[362807296], 99.50th=[480247808], 99.90th=[700448768], | |
| 99.95th=[767557632], 99.99th=[893386752] | |
bw ( KiB/s): min= 0, max=25069, per=90.81%, avg=20383.00, stdev=6627.00, samples=2 | |
iops : min= 0, max= 392, avg=318.50, stdev=103.94, samples=2 | |
lat (nsec) : 750=33.09%, 1000=34.07% | |
lat (usec) : 2=11.70%, 4=0.05%, 10=0.57%, 20=0.41%, 50=0.14% | |
lat (usec) : 100=0.10%, 250=0.03%, 500=0.14%, 750=1.09%, 1000=2.45% | |
lat (msec) : 2=3.49%, 4=1.98%, 10=1.14%, 20=0.49%, 50=3.07% | |
lat (msec) : 100=2.94%, 250=1.55%, 500=1.06%, 750=0.39%, 1000=0.06% | |
cpu : usr=0.10%, sys=0.37%, ctx=15077, majf=0, minf=31 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=38240,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:54 2019 | |
read: IOPS=32, BW=2065KiB/s (2115kB/s)(242MiB/120022msec) | |
slat (nsec): min=0, max=399843k, avg=206891.70, stdev=7876972.64 | |
clat (nsec): min=0, max=737486k, avg=8865298.83, stdev=46977819.17 | |
lat (nsec): min=0, max=976153k, avg=9073210.18, stdev=48841859.71 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 732], | |
| 20.00th=[ 772], 30.00th=[ 876], 40.00th=[ 972], | |
| 50.00th=[ 1112], 60.00th=[ 1256], 70.00th=[ 1480], | |
| 80.00th=[ 109056], 90.00th=[ 2506752], 95.00th=[ 45875200], | |
| 99.00th=[244318208], 99.50th=[400556032], 99.90th=[583008256], | |
| 99.95th=[717225984], 99.99th=[734003200] | |
bw ( KiB/s): min= 0, max= 2312, per=9.20%, avg=2065.00, stdev=349.31, samples=2 | |
iops : min= 0, max= 36, avg=32.00, stdev= 5.66, samples=2 | |
write: IOPS=33, BW=2171KiB/s (2223kB/s)(255MiB/120022msec) | |
slat (nsec): min=0, max=10291k, avg=180936.82, stdev=303505.98 | |
clat (nsec): min=0, max=1040.1M, avg=109529981.00, stdev=175834302.36 | |
lat (nsec): min=0, max=1040.2M, avg=109712031.36, stdev=175783939.65 | |
clat percentiles (usec): | |
| 1.00th=[ 1057], 5.00th=[ 1287], 10.00th=[ 1565], | |
| 20.00th=[ 2278], 30.00th=[ 3621], 40.00th=[ 6587], | |
| 50.00th=[ 19530], 60.00th=[ 66847], 70.00th=[ 102237], | |
| 80.00th=[ 160433], 90.00th=[ 371196], 95.00th=[ 497026], | |
| 99.00th=[ 809501], 99.50th=[ 893387], 99.90th=[ 952108], | |
| 99.95th=[1002439], 99.99th=[1035994] | |
bw ( KiB/s): min= 0, max= 2405, per=44.85%, avg=2169.00, stdev=333.75, samples=2 | |
iops : min= 0, max= 37, avg=33.50, stdev= 4.95, samples=2 | |
lat (nsec) : 750=8.20%, 1000=12.22% | |
lat (usec) : 2=14.93%, 4=0.05%, 10=0.39%, 20=1.15%, 50=1.02% | |
lat (usec) : 100=0.96%, 250=0.79%, 500=0.94%, 750=0.52%, 1000=1.17% | |
lat (msec) : 2=9.61%, 4=8.78%, 10=6.61%, 20=3.36%, 50=4.33% | |
lat (msec) : 100=8.36%, 250=7.93%, 500=6.17%, 750=1.85%, 1000=0.68% | |
cpu : usr=0.05%, sys=0.19%, ctx=9240, majf=0, minf=35 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=3874,4069,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=21.9MiB/s (22.0MB/s), 2065KiB/s-19.9MiB/s (2115kB/s-20.9MB/s), io=2631MiB (2759MB), run=120022-120031msec | |
WRITE: bw=4836KiB/s (4953kB/s), 2171KiB/s-2666KiB/s (2223kB/s-2730kB/s), io=567MiB (595MB), run=120022-120074msec | |
Disk stats (read/write): | |
rbd121: ios=8410/9542, merge=0/1, ticks=542895/868442, in_queue=1402193, util=12.62% | |
<example-benchmark-fio-server-325-benchmark-fbmfd> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:54 2019 | |
write: IOPS=46, BW=2969KiB/s (3040kB/s)(348MiB/120063msec) | |
slat (nsec): min=0, max=428628k, avg=322718.17, stdev=6988005.94 | |
clat (nsec): min=0, max=1469.7M, avg=86229567.91, stdev=147784132.72 | |
lat (nsec): min=0, max=1469.8M, avg=86553448.19, stdev=148051015.99 | |
clat percentiles (usec): | |
| 1.00th=[ 988], 5.00th=[ 1188], 10.00th=[ 1401], | |
| 20.00th=[ 1827], 30.00th=[ 2573], 40.00th=[ 4359], | |
| 50.00th=[ 10814], 60.00th=[ 61604], 70.00th=[ 91751], | |
| 80.00th=[ 120062], 90.00th=[ 261096], 95.00th=[ 417334], | |
| 99.00th=[ 683672], 99.50th=[ 809501], 99.90th=[ 977273], | |
| 99.95th=[1283458], 99.99th=[1468007] | |
bw ( KiB/s): min= 0, max= 3621, per=54.56%, avg=2967.50, stdev=924.19, samples=2 | |
iops : min= 0, max= 56, avg=46.00, stdev=14.14, samples=2 | |
lat (usec) : 50=0.02%, 250=0.02%, 750=0.07%, 1000=1.02% | |
lat (msec) : 2=21.88%, 4=15.52%, 10=11.03%, 20=4.01%, 50=3.92% | |
lat (msec) : 100=15.99%, 250=16.39%, 500=6.95%, 750=2.48%, 1000=0.68% | |
cpu : usr=0.07%, sys=0.19%, ctx=10557, majf=0, minf=31 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,5566,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:54 2019 | |
read: IOPS=295, BW=18.5MiB/s (19.4MB/s)(2218MiB/120022msec) | |
slat (nsec): min=0, max=413228k, avg=42515.07, stdev=2589450.95 | |
clat (nsec): min=0, max=1400.0M, avg=13517573.15, stdev=63183262.34 | |
lat (nsec): min=0, max=1400.3M, avg=13560936.96, stdev=63244141.80 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 724], | |
| 20.00th=[ 724], 30.00th=[ 740], 40.00th=[ 772], | |
| 50.00th=[ 844], 60.00th=[ 900], 70.00th=[ 1080], | |
| 80.00th=[ 264192], 90.00th=[ 22937600], 95.00th=[ 61079552], | |
| 99.00th=[375390208], 99.50th=[480247808], 99.90th=[700448768], | |
| 99.95th=[767557632], 99.99th=[926941184] | |
bw ( KiB/s): min= 0, max=24299, per=88.89%, avg=18927.00, stdev=7597.16, samples=2 | |
iops : min= 0, max= 380, avg=295.50, stdev=119.50, samples=2 | |
lat (nsec) : 750=36.46%, 1000=30.96% | |
lat (usec) : 2=11.74%, 4=0.06%, 10=0.34%, 20=0.18%, 50=0.17% | |
lat (usec) : 100=0.06%, 250=0.05%, 500=0.17%, 750=1.13%, 1000=2.16% | |
lat (msec) : 2=2.93%, 4=1.72%, 10=1.00%, 20=0.66%, 50=3.78% | |
lat (msec) : 100=3.29%, 250=1.50%, 500=1.19%, 750=0.41%, 1000=0.06% | |
cpu : usr=0.10%, sys=0.35%, ctx=14216, majf=0, minf=31 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=35512,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:54 2019 | |
read: IOPS=37, BW=2372KiB/s (2429kB/s)(278MiB/120051msec) | |
slat (nsec): min=0, max=405259k, avg=173230.37, stdev=7000712.24 | |
clat (nsec): min=0, max=676905k, avg=10063171.52, stdev=47810644.50 | |
lat (nsec): min=0, max=676931k, avg=10237323.34, stdev=48310601.69 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 812], 30.00th=[ 876], 40.00th=[ 980], | |
| 50.00th=[ 1112], 60.00th=[ 1256], 70.00th=[ 1416], | |
| 80.00th=[ 73216], 90.00th=[ 3588096], 95.00th=[ 55836672], | |
| 99.00th=[265289728], 99.50th=[362807296], 99.90th=[583008256], | |
| 99.95th=[599785472], 99.99th=[675282944] | |
bw ( KiB/s): min= 0, max= 2913, per=11.14%, avg=2373.00, stdev=763.68, samples=2 | |
iops : min= 0, max= 45, avg=36.50, stdev=12.02, samples=2 | |
write: IOPS=38, BW=2471KiB/s (2531kB/s)(290MiB/120051msec) | |
slat (nsec): min=0, max=4214.0k, avg=178472.88, stdev=241439.55 | |
clat (nsec): min=0, max=1469.7M, avg=93902576.19, stdev=154811764.20 | |
lat (nsec): min=0, max=1469.8M, avg=94082275.96, stdev=154763959.10 | |
clat percentiles (usec): | |
| 1.00th=[ 1012], 5.00th=[ 1237], 10.00th=[ 1450], | |
| 20.00th=[ 1991], 30.00th=[ 3097], 40.00th=[ 5604], | |
| 50.00th=[ 22152], 60.00th=[ 70779], 70.00th=[ 95945], | |
| 80.00th=[ 129500], 90.00th=[ 291505], 95.00th=[ 442500], | |
| 99.00th=[ 750781], 99.50th=[ 876610], 99.90th=[1098908], | |
| 99.95th=[1132463], 99.99th=[1468007] | |
bw ( KiB/s): min= 0, max= 3003, per=45.41%, avg=2470.00, stdev=753.78, samples=2 | |
iops : min= 0, max= 46, avg=38.00, stdev=11.31, samples=2 | |
lat (nsec) : 750=6.72%, 1000=13.63% | |
lat (usec) : 2=17.22%, 4=0.02%, 10=0.23%, 20=0.47%, 50=0.44% | |
lat (usec) : 100=0.66%, 250=0.53%, 500=0.37%, 750=0.68%, 1000=1.45% | |
lat (msec) : 2=11.21%, 4=8.28%, 10=5.91%, 20=2.37%, 50=3.56% | |
lat (msec) : 100=10.54%, 250=9.54%, 500=4.41%, 750=1.28%, 1000=0.43% | |
cpu : usr=0.07%, sys=0.22%, ctx=10658, majf=0, minf=31 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=4451,4633,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=20.8MiB/s (21.8MB/s), 2372KiB/s-18.5MiB/s (2429kB/s-19.4MB/s), io=2496MiB (2618MB), run=120022-120051msec | |
WRITE: bw=5440KiB/s (5570kB/s), 2471KiB/s-2969KiB/s (2531kB/s-3040kB/s), io=638MiB (669MB), run=120051-120063msec | |
Disk stats (read/write): | |
rbd103: ios=8316/10505, merge=0/1, ticks=549459/828531, in_queue=1368379, util=14.25% | |
<example-benchmark-fio-server-336-benchmark-tzn4v> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:54 2019 | |
write: IOPS=37, BW=2378KiB/s (2435kB/s)(279MiB/120104msec) | |
slat (nsec): min=0, max=340097k, avg=326004.44, stdev=6645041.68 | |
clat (nsec): min=0, max=1280.6M, avg=107678890.86, stdev=175225797.88 | |
lat (nsec): min=0, max=1280.7M, avg=108005935.42, stdev=175238266.51 | |
clat percentiles (usec): | |
| 1.00th=[ 955], 5.00th=[ 1172], 10.00th=[ 1369], | |
| 20.00th=[ 1844], 30.00th=[ 2868], 40.00th=[ 6128], | |
| 50.00th=[ 28967], 60.00th=[ 76022], 70.00th=[ 104334], | |
| 80.00th=[ 154141], 90.00th=[ 337642], 95.00th=[ 488637], | |
| 99.00th=[ 817890], 99.50th=[ 935330], 99.90th=[1132463], | |
| 99.95th=[1283458], 99.99th=[1283458] | |
bw ( KiB/s): min= 0, max= 1784, per=40.93%, avg=1784.00, stdev= 0.00, samples=1 | |
iops : min= 0, max= 27, avg=27.00, stdev= 0.00, samples=1 | |
lat (usec) : 4=0.02%, 100=0.02%, 250=0.02%, 500=0.02%, 750=0.09% | |
lat (usec) : 1000=1.68% | |
lat (msec) : 2=20.36%, 4=13.09%, 10=8.88%, 20=3.61%, 50=5.94% | |
lat (msec) : 100=14.64%, 250=17.60%, 500=9.35%, 750=3.27%, 1000=1.03% | |
cpu : usr=0.04%, sys=0.17%, ctx=8470, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,4460,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:54 2019 | |
read: IOPS=456, BW=28.5MiB/s (29.9MB/s)(3422MiB/120027msec) | |
slat (nsec): min=0, max=338266k, avg=27425.06, stdev=1786682.20 | |
clat (nsec): min=0, max=999544k, avg=8748171.03, stdev=51624016.82 | |
lat (nsec): min=0, max=999606k, avg=8776407.06, stdev=51663169.95 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 708], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 724], 40.00th=[ 732], | |
| 50.00th=[ 748], 60.00th=[ 828], 70.00th=[ 884], | |
| 80.00th=[ 1112], 90.00th=[ 1044480], 95.00th=[ 36962304], | |
| 99.00th=[270532608], 99.50th=[417333248], 99.90th=[650117120], | |
| 99.95th=[742391808], 99.99th=[910163968] | |
bw ( KiB/s): min= 0, max=44025, per=93.94%, avg=29198.00, stdev=20968.54, samples=2 | |
iops : min= 0, max= 688, avg=456.00, stdev=328.10, samples=2 | |
lat (nsec) : 750=50.41%, 1000=27.05% | |
lat (usec) : 2=8.04%, 4=0.05%, 10=1.12%, 20=0.45%, 50=0.17% | |
lat (usec) : 100=0.10%, 250=0.06%, 500=0.31%, 750=0.70%, 1000=1.39% | |
lat (msec) : 2=1.85%, 4=1.13%, 10=0.65%, 20=0.46%, 50=2.03% | |
lat (msec) : 100=1.78%, 250=1.14%, 500=0.82%, 750=0.24%, 1000=0.05% | |
cpu : usr=0.12%, sys=0.47%, ctx=13690, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=54795,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:54 2019 | |
read: IOPS=29, BW=1895KiB/s (1941kB/s)(222MiB/120051msec) | |
slat (nsec): min=0, max=263179k, avg=101624.02, stdev=4414051.05 | |
clat (nsec): min=0, max=655890k, avg=9618104.13, stdev=46649931.58 | |
lat (nsec): min=0, max=655927k, avg=9720798.57, stdev=46853921.91 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 804], 30.00th=[ 876], 40.00th=[ 956], | |
| 50.00th=[ 1096], 60.00th=[ 1240], 70.00th=[ 1416], | |
| 80.00th=[ 35584], 90.00th=[ 2670592], 95.00th=[ 50069504], | |
| 99.00th=[256901120], 99.50th=[367001600], 99.90th=[522190848], | |
| 99.95th=[599785472], 99.99th=[658505728] | |
bw ( KiB/s): min= 0, max= 2421, per=6.10%, avg=1895.50, stdev=743.17, samples=2 | |
iops : min= 0, max= 37, avg=29.00, stdev=11.31, samples=2 | |
write: IOPS=30, BW=1983KiB/s (2030kB/s)(232MiB/120051msec) | |
slat (nsec): min=0, max=279271k, avg=268855.21, stdev=4731857.77 | |
clat (nsec): min=0, max=1280.7M, avg=120132593.80, stdev=184042868.06 | |
lat (nsec): min=0, max=1280.8M, avg=120402554.96, stdev=184195357.79 | |
clat percentiles (usec): | |
| 1.00th=[ 988], 5.00th=[ 1221], 10.00th=[ 1450], | |
| 20.00th=[ 2057], 30.00th=[ 3654], 40.00th=[ 10159], | |
| 50.00th=[ 51643], 60.00th=[ 86508], 70.00th=[ 114820], | |
| 80.00th=[ 187696], 90.00th=[ 367002], 95.00th=[ 505414], | |
| 99.00th=[ 876610], 99.50th=[ 977273], 99.90th=[1132463], | |
| 99.95th=[1132463], 99.99th=[1283458] | |
bw ( KiB/s): min= 0, max= 2540, per=45.43%, avg=1980.50, stdev=791.25, samples=2 | |
iops : min= 0, max= 39, avg=30.50, stdev=12.02, samples=2 | |
lat (nsec) : 750=6.97%, 1000=14.22% | |
lat (usec) : 2=16.17%, 4=0.08%, 10=0.36%, 20=0.89%, 50=0.76% | |
lat (usec) : 100=0.98%, 250=0.47%, 500=0.58%, 750=0.58%, 1000=1.20% | |
lat (msec) : 2=10.26%, 4=6.75%, 10=4.73%, 20=1.90%, 50=4.94% | |
lat (msec) : 100=8.73%, 250=10.95%, 500=5.79%, 750=1.76%, 1000=0.80% | |
cpu : usr=0.05%, sys=0.18%, ctx=8332, majf=0, minf=34 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=3556,3716,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=30.4MiB/s (31.8MB/s), 1895KiB/s-28.5MiB/s (1941kB/s-29.9MB/s), io=3644MiB (3821MB), run=120027-120051msec | |
WRITE: bw=4360KiB/s (4465kB/s), 1983KiB/s-2378KiB/s (2030kB/s-2435kB/s), io=511MiB (536MB), run=120051-120104msec | |
Disk stats (read/write): | |
rbd111: ios=7834/8600, merge=0/1, ticks=524255/818436, in_queue=1334384, util=12.21% | |
fio: inflate error -5 | |
fio: failed decompressing log | |
fio: failed converting IO log | |
<example-benchmark-fio-server-239-benchmark-4zbns> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:54 2019 | |
write: IOPS=45, BW=2936KiB/s (3006kB/s)(344MiB/120067msec) | |
slat (nsec): min=0, max=534902k, avg=347032.55, stdev=8533504.59 | |
clat (nsec): min=0, max=889389k, avg=86828735.19, stdev=144120444.95 | |
lat (nsec): min=0, max=1368.6M, avg=87177012.08, stdev=145119573.59 | |
clat percentiles (usec): | |
| 1.00th=[ 947], 5.00th=[ 1156], 10.00th=[ 1336], 20.00th=[ 1795], | |
| 30.00th=[ 2606], 40.00th=[ 4948], 50.00th=[ 17433], 60.00th=[ 55313], | |
| 70.00th=[ 94897], 80.00th=[131597], 90.00th=[238027], 95.00th=[421528], | |
| 99.00th=[700449], 99.50th=[742392], 99.90th=[826278], 99.95th=[859833], | |
| 99.99th=[893387] | |
bw ( KiB/s): min= 0, max= 3771, per=55.94%, avg=2935.00, stdev=1182.28, samples=2 | |
iops : min= 0, max= 58, avg=45.00, stdev=18.38, samples=2 | |
lat (usec) : 250=0.02%, 500=0.04%, 750=0.02%, 1000=1.76% | |
lat (msec) : 2=21.13%, 4=14.35%, 10=9.14%, 20=4.60%, 50=7.45% | |
lat (msec) : 100=13.01%, 250=18.91%, 500=6.21%, 750=2.98%, 1000=0.45% | |
cpu : usr=0.05%, sys=0.20%, ctx=10277, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,5505,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:54 2019 | |
read: IOPS=255, BW=15.9MiB/s (16.7MB/s)(1912MiB/120021msec) | |
slat (nsec): min=0, max=533552k, avg=48056.18, stdev=3101874.04 | |
clat (nsec): min=0, max=925627k, avg=15680255.74, stdev=65921046.09 | |
lat (nsec): min=0, max=925689k, avg=15729185.26, stdev=66004552.52 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 732], 40.00th=[ 764], | |
| 50.00th=[ 868], 60.00th=[ 956], 70.00th=[ 1256], | |
| 80.00th=[ 970752], 90.00th=[ 31850496], 95.00th=[ 73924608], | |
| 99.00th=[375390208], 99.50th=[497025024], 99.90th=[692060160], | |
| 99.95th=[725614592], 99.99th=[834666496] | |
bw ( KiB/s): min= 0, max=22069, per=100.00%, avg=22069.00, stdev= 0.00, samples=1 | |
iops : min= 0, max= 345, avg=345.00, stdev= 0.00, samples=1 | |
lat (nsec) : 750=37.72%, 1000=24.33% | |
lat (usec) : 2=12.99%, 4=0.07%, 10=0.30%, 20=0.28%, 50=0.21% | |
lat (usec) : 100=0.03%, 250=0.04%, 500=0.32%, 750=1.28%, 1000=2.65% | |
lat (msec) : 2=3.61%, 4=2.23%, 10=1.10%, 20=1.05%, 50=4.52% | |
lat (msec) : 100=3.20%, 250=2.16%, 500=1.44%, 750=0.45%, 1000=0.03% | |
cpu : usr=0.08%, sys=0.33%, ctx=14593, majf=0, minf=100 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=30613,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:54 2019 | |
read: IOPS=34, BW=2211KiB/s (2264kB/s)(259MiB/120027msec) | |
slat (nsec): min=0, max=534280k, avg=237292.72, stdev=9787518.53 | |
clat (nsec): min=0, max=711810k, avg=12200922.95, stdev=52573178.22 | |
lat (nsec): min=0, max=711830k, avg=12439097.46, stdev=53435173.79 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 780], 30.00th=[ 884], 40.00th=[ 988], | |
| 50.00th=[ 1128], 60.00th=[ 1256], 70.00th=[ 1528], | |
| 80.00th=[ 238592], 90.00th=[ 20840448], 95.00th=[ 67633152], | |
| 99.00th=[283115520], 99.50th=[358612992], 99.90th=[675282944], | |
| 99.95th=[708837376], 99.99th=[708837376] | |
bw ( KiB/s): min= 0, max= 2634, per=11.93%, avg=2210.00, stdev=599.63, samples=2 | |
iops : min= 0, max= 41, avg=34.00, stdev= 9.90, samples=2 | |
write: IOPS=36, BW=2313KiB/s (2368kB/s)(271MiB/120027msec) | |
slat (nsec): min=0, max=6942.8k, avg=174147.77, stdev=263171.86 | |
clat (nsec): min=0, max=859436k, avg=98737609.79, stdev=152398197.87 | |
lat (nsec): min=0, max=859576k, avg=98912930.74, stdev=152347644.57 | |
clat percentiles (usec): | |
| 1.00th=[ 1020], 5.00th=[ 1303], 10.00th=[ 1549], 20.00th=[ 2180], | |
| 30.00th=[ 3687], 40.00th=[ 10159], 50.00th=[ 39584], 60.00th=[ 73925], | |
| 70.00th=[105382], 80.00th=[141558], 90.00th=[295699], 95.00th=[463471], | |
| 99.00th=[692061], 99.50th=[750781], 99.90th=[801113], 99.95th=[843056], | |
| 99.99th=[859833] | |
bw ( KiB/s): min= 0, max= 2733, per=44.05%, avg=2311.50, stdev=596.09, samples=2 | |
iops : min= 0, max= 42, avg=35.50, stdev= 9.19, samples=2 | |
lat (nsec) : 750=7.59%, 1000=12.44% | |
lat (usec) : 2=15.02%, 4=0.05%, 10=0.29%, 20=1.04%, 50=0.85% | |
lat (usec) : 100=0.93%, 250=0.97%, 500=0.55%, 750=0.57%, 1000=1.29% | |
lat (msec) : 2=9.95%, 4=7.29%, 10=5.11%, 20=3.11%, 50=5.72% | |
lat (msec) : 100=9.42%, 250=11.52%, 500=3.96%, 750=2.13%, 1000=0.22% | |
cpu : usr=0.05%, sys=0.20%, ctx=9894, majf=0, minf=36 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=4146,4334,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=18.1MiB/s (18.0MB/s), 2211KiB/s-15.9MiB/s (2264kB/s-16.7MB/s), io=2171MiB (2277MB), run=120021-120027msec | |
WRITE: bw=5248KiB/s (5374kB/s), 2313KiB/s-2936KiB/s (2368kB/s-3006kB/s), io=615MiB (645MB), run=120027-120067msec | |
Disk stats (read/write): | |
rbd95: ios=9683/11004, merge=0/1, ticks=566653/848675, in_queue=1404758, util=14.29% | |
<example-benchmark-fio-server-747-benchmark-jh49p> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:54 2019 | |
write: IOPS=41, BW=2650KiB/s (2714kB/s)(311MiB/120014msec) | |
slat (nsec): min=0, max=444994k, avg=406213.93, stdev=8650946.30 | |
clat (nsec): min=0, max=1306.6M, avg=96754907.14, stdev=155893560.13 | |
lat (nsec): min=0, max=1306.7M, avg=97162272.04, stdev=156021316.52 | |
clat percentiles (usec): | |
| 1.00th=[ 963], 5.00th=[ 1156], 10.00th=[ 1352], | |
| 20.00th=[ 1811], 30.00th=[ 2704], 40.00th=[ 5473], | |
| 50.00th=[ 35914], 60.00th=[ 78119], 70.00th=[ 100140], | |
| 80.00th=[ 130548], 90.00th=[ 278922], 95.00th=[ 471860], | |
| 99.00th=[ 717226], 99.50th=[ 817890], 99.90th=[1044382], | |
| 99.95th=[1098908], 99.99th=[1300235] | |
bw ( KiB/s): min= 0, max= 3854, per=54.00%, avg=2648.00, stdev=1705.54, samples=2 | |
iops : min= 0, max= 60, avg=41.00, stdev=26.87, samples=2 | |
lat (usec) : 500=0.04%, 750=0.02%, 1000=1.43% | |
lat (msec) : 2=21.20%, 4=14.15%, 10=7.79%, 20=3.60%, 50=3.85% | |
lat (msec) : 100=17.74%, 250=19.33%, 500=6.83%, 750=3.32%, 1000=0.62% | |
cpu : usr=0.04%, sys=0.18%, ctx=9502, majf=0, minf=29 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,4967,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:54 2019 | |
read: IOPS=565, BW=35.3MiB/s (36.0MB/s)(4233MiB/120022msec) | |
slat (nsec): min=0, max=426774k, avg=30081.64, stdev=2287324.85 | |
clat (nsec): min=0, max=1057.1M, avg=7035019.24, stdev=47212773.76 | |
lat (nsec): min=0, max=1057.1M, avg=7065919.65, stdev=47275485.63 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 708], 10.00th=[ 716], | |
| 20.00th=[ 716], 30.00th=[ 724], 40.00th=[ 740], | |
| 50.00th=[ 796], 60.00th=[ 852], 70.00th=[ 900], | |
| 80.00th=[ 1112], 90.00th=[ 25216], 95.00th=[ 13959168], | |
| 99.00th=[227540992], 99.50th=[396361728], 99.90th=[633339904], | |
| 99.95th=[725614592], 99.99th=[918552576] | |
bw ( KiB/s): min= 0, max=54834, per=94.38%, avg=36118.50, stdev=26467.71, samples=2 | |
iops : min= 0, max= 860, avg=565.50, stdev=416.49, samples=2 | |
lat (nsec) : 750=45.37%, 1000=32.21% | |
lat (usec) : 2=6.94%, 4=0.05%, 10=2.83%, 20=2.39%, 50=0.47% | |
lat (usec) : 100=0.13%, 250=0.04%, 500=0.09%, 750=0.50%, 1000=1.06% | |
lat (msec) : 2=1.45%, 4=0.88%, 10=0.46%, 20=0.29%, 50=1.68% | |
lat (msec) : 100=1.44%, 250=0.80%, 500=0.68%, 750=0.19%, 1000=0.04% | |
cpu : usr=0.16%, sys=0.52%, ctx=13040, majf=0, minf=29 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=67920,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:54 2019 | |
read: IOPS=33, BW=2161KiB/s (2213kB/s)(253MiB/120042msec) | |
slat (nsec): min=0, max=4021.6k, avg=26307.06, stdev=106378.46 | |
clat (nsec): min=0, max=626922k, avg=7607568.80, stdev=38180756.57 | |
lat (nsec): min=0, max=626958k, avg=7634781.88, stdev=38187137.32 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 756], 30.00th=[ 852], 40.00th=[ 948], | |
| 50.00th=[ 1080], 60.00th=[ 1224], 70.00th=[ 1384], | |
| 80.00th=[ 17024], 90.00th=[ 1581056], 95.00th=[ 47448064], | |
| 99.00th=[191889408], 99.50th=[304087040], 99.90th=[480247808], | |
| 99.95th=[484442112], 99.99th=[624951296] | |
bw ( KiB/s): min= 0, max= 2935, per=5.65%, avg=2161.50, stdev=1093.89, samples=2 | |
iops : min= 0, max= 45, avg=33.00, stdev=16.97, samples=2 | |
write: IOPS=35, BW=2255KiB/s (2309kB/s)(264MiB/120042msec) | |
slat (nsec): min=0, max=444199k, avg=469547.59, stdev=10265636.14 | |
clat (nsec): min=0, max=1282.2M, avg=105745971.30, stdev=159456781.24 | |
lat (nsec): min=0, max=1282.3M, avg=106216461.29, stdev=159801960.04 | |
clat percentiles (usec): | |
| 1.00th=[ 996], 5.00th=[ 1205], 10.00th=[ 1401], | |
| 20.00th=[ 1909], 30.00th=[ 3228], 40.00th=[ 9503], | |
| 50.00th=[ 60031], 60.00th=[ 86508], 70.00th=[ 107480], | |
| 80.00th=[ 139461], 90.00th=[ 320865], 95.00th=[ 484443], | |
| 99.00th=[ 734004], 99.50th=[ 801113], 99.90th=[ 943719], | |
| 99.95th=[1027605], 99.99th=[1283458] | |
bw ( KiB/s): min= 0, max= 3069, per=45.93%, avg=2252.50, stdev=1154.71, samples=2 | |
iops : min= 0, max= 47, avg=34.50, stdev=17.68, samples=2 | |
lat (nsec) : 750=8.79%, 1000=13.03% | |
lat (usec) : 2=16.53%, 4=0.04%, 10=0.41%, 20=0.77%, 50=0.72% | |
lat (usec) : 100=0.92%, 250=0.56%, 500=0.41%, 750=0.45%, 1000=1.28% | |
lat (msec) : 2=11.07%, 4=6.54%, 10=4.13%, 20=1.80%, 50=3.83% | |
lat (msec) : 100=10.84%, 250=11.47%, 500=4.07%, 750=1.91%, 1000=0.43% | |
cpu : usr=0.05%, sys=0.21%, ctx=9390, majf=0, minf=34 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=4056,4226,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=37.4MiB/s (39.2MB/s), 2161KiB/s-35.3MiB/s (2213kB/s-36.0MB/s), io=4486MiB (4704MB), run=120022-120042msec | |
WRITE: bw=4904KiB/s (5022kB/s), 2255KiB/s-2650KiB/s (2309kB/s-2714kB/s), io=575MiB (603MB), run=120014-120042msec | |
Disk stats (read/write): | |
rbd244: ios=7107/9294, merge=0/3, ticks=515812/787469, in_queue=1295011, util=12.37% | |
<example-benchmark-fio-server-238-benchmark-wczfn> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:54 2019 | |
write: IOPS=39, BW=2517KiB/s (2577kB/s)(295MiB/120014msec) | |
slat (nsec): min=0, max=341058k, avg=292693.61, stdev=5575637.86 | |
clat (nsec): min=0, max=1342.0M, avg=102021795.41, stdev=158984969.89 | |
lat (nsec): min=0, max=1342.1M, avg=102315643.79, stdev=158993907.50 | |
clat percentiles (usec): | |
| 1.00th=[ 963], 5.00th=[ 1221], 10.00th=[ 1483], | |
| 20.00th=[ 2073], 30.00th=[ 3392], 40.00th=[ 9372], | |
| 50.00th=[ 45351], 60.00th=[ 78119], 70.00th=[ 106431], | |
| 80.00th=[ 143655], 90.00th=[ 295699], 95.00th=[ 459277], | |
| 99.00th=[ 725615], 99.50th=[ 868221], 99.90th=[1149240], | |
| 99.95th=[1300235], 99.99th=[1333789] | |
bw ( KiB/s): min= 0, max= 3384, per=53.24%, avg=2514.50, stdev=1229.66, samples=2 | |
iops : min= 0, max= 52, avg=38.50, stdev=19.09, samples=2 | |
lat (usec) : 50=0.02%, 250=0.02%, 500=0.06%, 750=0.02%, 1000=1.34% | |
lat (msec) : 2=17.45%, 4=13.19%, 10=8.25%, 20=3.90%, 50=7.19% | |
lat (msec) : 100=16.22%, 250=20.54%, 500=7.70%, 750=3.41%, 1000=0.49% | |
cpu : usr=0.05%, sys=0.17%, ctx=8969, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,4717,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:54 2019 | |
read: IOPS=379, BW=23.7MiB/s (24.9MB/s)(2846MiB/120038msec) | |
slat (nsec): min=0, max=313498k, avg=28520.52, stdev=1565230.47 | |
clat (nsec): min=0, max=1558.7M, avg=10539488.90, stdev=56101509.52 | |
lat (nsec): min=0, max=1558.8M, avg=10568843.17, stdev=56135662.02 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 724], 40.00th=[ 732], | |
| 50.00th=[ 764], 60.00th=[ 860], 70.00th=[ 924], | |
| 80.00th=[ 1288], 90.00th=[ 2441216], 95.00th=[ 49020928], | |
| 99.00th=[316669952], 99.50th=[450887680], 99.90th=[666894336], | |
| 99.95th=[734003200], 99.99th=[977272832] | |
bw ( KiB/s): min= 0, max=34871, per=92.05%, avg=24286.50, stdev=14968.74, samples=2 | |
iops : min= 0, max= 545, avg=379.50, stdev=234.05, samples=2 | |
lat (nsec) : 750=47.29%, 1000=26.10% | |
lat (usec) : 2=9.46%, 4=0.05%, 10=1.23%, 20=0.46%, 50=0.15% | |
lat (usec) : 100=0.07%, 250=0.06%, 500=0.26%, 750=0.85%, 1000=1.47% | |
lat (msec) : 2=2.07%, 4=1.36%, 10=0.71%, 20=0.54%, 50=2.97% | |
lat (msec) : 100=2.30%, 250=1.30%, 500=0.98%, 750=0.30%, 1000=0.03% | |
cpu : usr=0.12%, sys=0.40%, ctx=13768, majf=0, minf=42 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=45580,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:54 2019 | |
read: IOPS=32, BW=2105KiB/s (2155kB/s)(247MiB/120008msec) | |
slat (nsec): min=0, max=267135k, avg=128561.10, stdev=4765559.54 | |
clat (nsec): min=0, max=638238k, avg=9563016.49, stdev=43149253.15 | |
lat (nsec): min=0, max=638271k, avg=9692556.97, stdev=43396123.16 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 788], 30.00th=[ 876], 40.00th=[ 996], | |
| 50.00th=[ 1128], 60.00th=[ 1256], 70.00th=[ 1432], | |
| 80.00th=[ 40192], 90.00th=[ 3751936], 95.00th=[ 54788096], | |
| 99.00th=[254803968], 99.50th=[333447168], 99.90th=[497025024], | |
| 99.95th=[509607936], 99.99th=[641728512] | |
bw ( KiB/s): min= 0, max= 2695, per=7.97%, avg=2104.00, stdev=835.80, samples=2 | |
iops : min= 0, max= 42, avg=32.50, stdev=13.44, samples=2 | |
write: IOPS=34, BW=2206KiB/s (2259kB/s)(259MiB/120008msec) | |
slat (nsec): min=0, max=10176k, avg=175306.89, stdev=326702.12 | |
clat (nsec): min=0, max=1839.1M, avg=107317664.14, stdev=164884666.88 | |
lat (nsec): min=0, max=1839.2M, avg=107494142.40, stdev=164838936.04 | |
clat percentiles (usec): | |
| 1.00th=[ 963], 5.00th=[ 1221], 10.00th=[ 1467], | |
| 20.00th=[ 2212], 30.00th=[ 3949], 40.00th=[ 12256], | |
| 50.00th=[ 52167], 60.00th=[ 81265], 70.00th=[ 108528], | |
| 80.00th=[ 154141], 90.00th=[ 312476], 95.00th=[ 484443], | |
| 99.00th=[ 734004], 99.50th=[ 876610], 99.90th=[1149240], | |
| 99.95th=[1300235], 99.99th=[1837106] | |
bw ( KiB/s): min= 0, max= 2809, per=46.68%, avg=2204.50, stdev=854.89, samples=2 | |
iops : min= 0, max= 43, avg=34.00, stdev=12.73, samples=2 | |
lat (nsec) : 750=7.67%, 1000=12.10% | |
lat (usec) : 2=17.50%, 4=0.09%, 10=0.22%, 20=0.95%, 50=0.78% | |
lat (usec) : 100=1.11%, 250=0.52%, 500=0.40%, 750=0.49%, 1000=1.18% | |
lat (msec) : 2=9.58%, 4=6.85%, 10=4.79%, 20=2.12%, 50=5.21% | |
lat (msec) : 100=10.05%, 250=11.35%, 500=4.74%, 750=1.89%, 1000=0.30% | |
cpu : usr=0.06%, sys=0.20%, ctx=9317, majf=0, minf=37 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=3948,4134,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=25.8MiB/s (27.0MB/s), 2105KiB/s-23.7MiB/s (2155kB/s-24.9MB/s), io=3093MiB (3243MB), run=120008-120038msec | |
WRITE: bw=4723KiB/s (4837kB/s), 2206KiB/s-2517KiB/s (2259kB/s-2577kB/s), io=554MiB (580MB), run=120008-120014msec | |
Disk stats (read/write): | |
rbd67: ios=8366/9287, merge=0/1, ticks=537421/816981, in_queue=1345358, util=13.52% | |
<example-benchmark-fio-server-965-benchmark-cvg7r> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:54 2019 | |
write: IOPS=46, BW=2948KiB/s (3018kB/s)(346MiB/120088msec) | |
slat (nsec): min=0, max=479406k, avg=350130.57, stdev=8587720.43 | |
clat (nsec): min=0, max=1103.4M, avg=86740940.20, stdev=143041701.87 | |
lat (nsec): min=0, max=1103.5M, avg=87092248.17, stdev=143156153.60 | |
clat percentiles (usec): | |
| 1.00th=[ 955], 5.00th=[ 1156], 10.00th=[ 1352], | |
| 20.00th=[ 1795], 30.00th=[ 2737], 40.00th=[ 5735], | |
| 50.00th=[ 20317], 60.00th=[ 67634], 70.00th=[ 96994], | |
| 80.00th=[ 129500], 90.00th=[ 229639], 95.00th=[ 387974], | |
| 99.00th=[ 700449], 99.50th=[ 801113], 99.90th=[1044382], | |
| 99.95th=[1098908], 99.99th=[1098908] | |
bw ( KiB/s): min= 0, max= 3585, per=54.32%, avg=2947.50, stdev=901.56, samples=2 | |
iops : min= 0, max= 56, avg=46.00, stdev=14.14, samples=2 | |
lat (usec) : 4=0.02%, 500=0.02%, 750=0.02%, 1000=1.74% | |
lat (msec) : 2=21.58%, 4=12.23%, 10=9.90%, 20=4.40%, 50=5.70% | |
lat (msec) : 100=15.77%, 250=19.52%, 500=6.10%, 750=2.35%, 1000=0.52% | |
cpu : usr=0.07%, sys=0.19%, ctx=10526, majf=0, minf=26 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,5528,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:54 2019 | |
read: IOPS=291, BW=18.2MiB/s (19.1MB/s)(2187MiB/120030msec) | |
slat (nsec): min=0, max=467384k, avg=48463.95, stdev=3199541.13 | |
clat (nsec): min=0, max=987567k, avg=13728849.56, stdev=63522045.77 | |
lat (nsec): min=0, max=987595k, avg=13778150.67, stdev=63621954.04 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 732], 40.00th=[ 748], | |
| 50.00th=[ 820], 60.00th=[ 884], 70.00th=[ 1096], | |
| 80.00th=[ 522240], 90.00th=[ 20840448], 95.00th=[ 63700992], | |
| 99.00th=[371195904], 99.50th=[492830720], 99.90th=[683671552], | |
| 99.95th=[750780416], 99.99th=[851443712] | |
bw ( KiB/s): min= 0, max=24862, per=88.70%, avg=18656.00, stdev=8776.61, samples=2 | |
iops : min= 0, max= 388, avg=291.00, stdev=137.18, samples=2 | |
lat (nsec) : 750=42.00%, 1000=25.39% | |
lat (usec) : 2=11.31%, 4=0.08%, 10=0.45%, 20=0.12%, 50=0.11% | |
lat (usec) : 100=0.05%, 250=0.06%, 500=0.44%, 750=1.18%, 1000=2.35% | |
lat (msec) : 2=3.08%, 4=1.93%, 10=0.88%, 20=0.54%, 50=3.67% | |
lat (msec) : 100=3.09%, 250=1.55%, 500=1.29%, 750=0.44%, 1000=0.05% | |
cpu : usr=0.10%, sys=0.34%, ctx=14233, majf=0, minf=74 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=35008,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:54 2019 | |
read: IOPS=37, BW=2379KiB/s (2436kB/s)(279MiB/120022msec) | |
slat (nsec): min=0, max=2365.6k, avg=27578.90, stdev=85435.33 | |
clat (nsec): min=0, max=660093k, avg=10380038.22, stdev=49666588.17 | |
lat (nsec): min=0, max=660371k, avg=10408597.54, stdev=49675165.24 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 724], | |
| 20.00th=[ 756], 30.00th=[ 868], 40.00th=[ 964], | |
| 50.00th=[ 1112], 60.00th=[ 1240], 70.00th=[ 1448], | |
| 80.00th=[ 94720], 90.00th=[ 3555328], 95.00th=[ 53739520], | |
| 99.00th=[278921216], 99.50th=[408944640], 99.90th=[541065216], | |
| 99.95th=[608174080], 99.99th=[658505728] | |
bw ( KiB/s): min= 0, max= 2746, per=11.31%, avg=2379.00, stdev=519.02, samples=2 | |
iops : min= 0, max= 42, avg=36.50, stdev= 7.78, samples=2 | |
write: IOPS=38, BW=2480KiB/s (2540kB/s)(291MiB/120022msec) | |
slat (nsec): min=0, max=421952k, avg=357499.62, stdev=8700755.43 | |
clat (nsec): min=0, max=1108.1M, avg=93392464.33, stdev=148165561.77 | |
lat (nsec): min=0, max=1108.1M, avg=93751123.75, stdev=148302960.36 | |
clat percentiles (usec): | |
| 1.00th=[ 1037], 5.00th=[ 1237], 10.00th=[ 1500], | |
| 20.00th=[ 2180], 30.00th=[ 3884], 40.00th=[ 9765], | |
| 50.00th=[ 42206], 60.00th=[ 78119], 70.00th=[ 103285], | |
| 80.00th=[ 133694], 90.00th=[ 235930], 95.00th=[ 396362], | |
| 99.00th=[ 734004], 99.50th=[ 868221], 99.90th=[1061159], | |
| 99.95th=[1098908], 99.99th=[1115685] | |
bw ( KiB/s): min= 0, max= 2824, per=45.67%, avg=2478.00, stdev=489.32, samples=2 | |
iops : min= 0, max= 44, avg=38.50, stdev= 7.78, samples=2 | |
lat (nsec) : 750=9.30%, 1000=11.41% | |
lat (usec) : 2=15.74%, 4=0.14%, 10=0.26%, 20=0.79%, 50=0.70% | |
lat (usec) : 100=0.91%, 250=0.64%, 500=0.68%, 750=0.66%, 1000=1.39% | |
lat (msec) : 2=9.94%, 4=7.20%, 10=5.30%, 20=2.69%, 50=4.98% | |
lat (msec) : 100=9.83%, 250=12.06%, 500=3.59%, 750=1.35%, 1000=0.34% | |
cpu : usr=0.06%, sys=0.22%, ctx=10593, majf=0, minf=35 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=4464,4648,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=20.5MiB/s (21.5MB/s), 2379KiB/s-18.2MiB/s (2436kB/s-19.1MB/s), io=2465MiB (2585MB), run=120022-120030msec | |
WRITE: bw=5426KiB/s (5557kB/s), 2480KiB/s-2948KiB/s (2540kB/s-3018kB/s), io=636MiB (667MB), run=120022-120088msec | |
Disk stats (read/write): | |
rbd329: ios=9355/10893, merge=0/1, ticks=557544/828062, in_queue=1375322, util=14.62% | |
<example-benchmark-fio-server-714-benchmark-g6ljr> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:55 2019 | |
write: IOPS=49, BW=3166KiB/s (3242kB/s)(371MiB/120038msec) | |
slat (nsec): min=0, max=483547k, avg=436729.22, stdev=9798160.86 | |
clat (nsec): min=0, max=1052.2M, avg=80745876.19, stdev=137146591.70 | |
lat (nsec): min=0, max=1052.3M, avg=81183814.06, stdev=137524748.71 | |
clat percentiles (usec): | |
| 1.00th=[ 996], 5.00th=[ 1205], 10.00th=[ 1401], | |
| 20.00th=[ 1909], 30.00th=[ 2802], 40.00th=[ 5145], | |
| 50.00th=[ 19268], 60.00th=[ 59507], 70.00th=[ 89654], | |
| 80.00th=[ 116917], 90.00th=[ 208667], 95.00th=[ 404751], | |
| 99.00th=[ 675283], 99.50th=[ 742392], 99.90th=[ 926942], | |
| 99.95th=[1027605], 99.99th=[1052771] | |
bw ( KiB/s): min= 0, max= 4302, per=54.75%, avg=3164.50, stdev=1608.67, samples=2 | |
iops : min= 0, max= 67, avg=49.00, stdev=25.46, samples=2 | |
lat (usec) : 4=0.03%, 100=0.02%, 750=0.07%, 1000=0.91% | |
lat (msec) : 2=20.52%, 4=14.91%, 10=9.35%, 20=4.46%, 50=7.40% | |
lat (msec) : 100=16.83%, 250=17.62%, 500=4.94%, 750=2.53%, 1000=0.40% | |
cpu : usr=0.06%, sys=0.22%, ctx=11193, majf=0, minf=29 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,5936,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:55 2019 | |
read: IOPS=300, BW=18.8MiB/s (19.7MB/s)(2254MiB/120012msec) | |
slat (nsec): min=0, max=484975k, avg=65282.63, stdev=4143390.88 | |
clat (nsec): min=0, max=1203.3M, avg=13303433.72, stdev=63562701.91 | |
lat (nsec): min=0, max=1203.6M, avg=13369542.88, stdev=63721578.58 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 716], 10.00th=[ 724], | |
| 20.00th=[ 732], 30.00th=[ 740], 40.00th=[ 756], | |
| 50.00th=[ 844], 60.00th=[ 900], 70.00th=[ 1128], | |
| 80.00th=[ 23424], 90.00th=[ 13434880], 95.00th=[ 60555264], | |
| 99.00th=[ 367001600], 99.50th=[ 476053504], 99.90th=[ 700448768], | |
| 99.95th=[ 792723456], 99.99th=[1044381696] | |
bw ( KiB/s): min= 0, max=25968, per=88.44%, avg=19235.50, stdev=9521.19, samples=2 | |
iops : min= 0, max= 406, avg=300.50, stdev=149.20, samples=2 | |
lat (nsec) : 750=38.01%, 1000=28.33% | |
lat (usec) : 2=10.58%, 4=0.05%, 10=1.86%, 20=1.10%, 50=1.26% | |
lat (usec) : 100=0.10%, 250=0.07%, 500=0.09%, 750=0.86%, 1000=2.03% | |
lat (msec) : 2=2.74%, 4=1.86%, 10=0.90%, 20=0.53%, 50=3.54% | |
lat (msec) : 100=2.80%, 250=1.62%, 500=1.25%, 750=0.37%, 1000=0.05% | |
cpu : usr=0.09%, sys=0.35%, ctx=13433, majf=0, minf=29 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=36110,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:55 2019 | |
read: IOPS=39, BW=2518KiB/s (2578kB/s)(295MiB/120029msec) | |
slat (nsec): min=0, max=349857k, avg=101363.58, stdev=5089683.13 | |
clat (nsec): min=0, max=1044.2M, avg=9561601.99, stdev=48388043.08 | |
lat (nsec): min=0, max=1044.2M, avg=9663946.77, stdev=48647228.37 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 780], 30.00th=[ 876], 40.00th=[ 996], | |
| 50.00th=[ 1128], 60.00th=[ 1256], 70.00th=[ 1432], | |
| 80.00th=[ 43776], 90.00th=[ 2899968], 95.00th=[ 51118080], | |
| 99.00th=[ 248512512], 99.50th=[ 379584512], 99.90th=[ 566231040], | |
| 99.95th=[ 666894336], 99.99th=[1044381696] | |
bw ( KiB/s): min= 0, max= 3013, per=11.57%, avg=2517.50, stdev=700.74, samples=2 | |
iops : min= 0, max= 47, avg=39.00, stdev=11.31, samples=2 | |
write: IOPS=40, BW=2614KiB/s (2677kB/s)(306MiB/120029msec) | |
slat (nsec): min=0, max=764714k, avg=504417.57, stdev=13821816.46 | |
clat (nsec): min=0, max=1313.0M, avg=88688665.47, stdev=146690691.98 | |
lat (nsec): min=0, max=1313.1M, avg=89194015.61, stdev=147209124.43 | |
clat percentiles (usec): | |
| 1.00th=[ 1037], 5.00th=[ 1254], 10.00th=[ 1516], | |
| 20.00th=[ 2180], 30.00th=[ 3654], 40.00th=[ 8586], | |
| 50.00th=[ 33817], 60.00th=[ 67634], 70.00th=[ 93848], | |
| 80.00th=[ 123208], 90.00th=[ 227541], 95.00th=[ 438305], | |
| 99.00th=[ 683672], 99.50th=[ 775947], 99.90th=[1044382], | |
| 99.95th=[1061159], 99.99th=[1317012] | |
bw ( KiB/s): min= 0, max= 3117, per=45.20%, avg=2612.50, stdev=713.47, samples=2 | |
iops : min= 0, max= 48, avg=40.00, stdev=11.31, samples=2 | |
lat (nsec) : 750=8.05%, 1000=11.86% | |
lat (usec) : 2=17.28%, 4=0.05%, 10=0.37%, 20=1.02%, 50=0.76% | |
lat (usec) : 100=0.80%, 250=0.45%, 500=0.43%, 750=0.39%, 1000=1.44% | |
lat (msec) : 2=9.94%, 4=7.65%, 10=5.33%, 20=2.76%, 50=5.62% | |
lat (msec) : 100=10.70%, 250=10.22%, 500=2.99%, 750=1.54%, 1000=0.20% | |
cpu : usr=0.06%, sys=0.25%, ctx=11131, majf=0, minf=36 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=4725,4900,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=21.2MiB/s (22.3MB/s), 2518KiB/s-18.8MiB/s (2578kB/s-19.7MB/s), io=2550MiB (2673MB), run=120012-120029msec | |
WRITE: bw=5781KiB/s (5919kB/s), 2614KiB/s-3166KiB/s (2677kB/s-3242kB/s), io=678MiB (711MB), run=120029-120038msec | |
Disk stats (read/write): | |
rbd292: ios=7536/10929, merge=0/2, ticks=534002/809056, in_queue=1333704, util=13.66% | |
<example-benchmark-fio-server-386-benchmark-nzznc> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:55 2019 | |
write: IOPS=40, BW=2596KiB/s (2658kB/s)(304MiB/120024msec) | |
slat (nsec): min=0, max=579931k, avg=462348.78, stdev=11856183.51 | |
clat (nsec): min=0, max=1090.2M, avg=98730771.42, stdev=156908561.14 | |
lat (nsec): min=0, max=1289.8M, avg=99194171.74, stdev=157794900.50 | |
clat percentiles (usec): | |
| 1.00th=[ 1012], 5.00th=[ 1221], 10.00th=[ 1418], | |
| 20.00th=[ 1991], 30.00th=[ 3228], 40.00th=[ 7242], | |
| 50.00th=[ 30802], 60.00th=[ 72877], 70.00th=[ 100140], | |
| 80.00th=[ 131597], 90.00th=[ 316670], 95.00th=[ 484443], | |
| 99.00th=[ 708838], 99.50th=[ 742392], 99.90th=[ 826278], | |
| 99.95th=[ 910164], 99.99th=[1082131] | |
bw ( KiB/s): min= 0, max= 3369, per=55.35%, avg=2593.50, stdev=1096.72, samples=2 | |
iops : min= 0, max= 52, avg=40.00, stdev=16.97, samples=2 | |
lat (usec) : 250=0.02%, 750=0.04%, 1000=0.84% | |
lat (msec) : 2=19.24%, 4=13.86%, 10=8.49%, 20=4.61%, 50=6.41% | |
lat (msec) : 100=16.55%, 250=17.58%, 500=7.77%, 750=4.17%, 1000=0.43% | |
cpu : usr=0.05%, sys=0.17%, ctx=9233, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,4864,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:55 2019 | |
read: IOPS=290, BW=18.1MiB/s (18.0MB/s)(2174MiB/120020msec) | |
slat (nsec): min=0, max=689525k, avg=62199.26, stdev=4622504.13 | |
clat (nsec): min=0, max=1083.4M, avg=13794212.01, stdev=64703199.17 | |
lat (nsec): min=0, max=1083.5M, avg=13857268.54, stdev=64873106.38 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 740], 40.00th=[ 788], | |
| 50.00th=[ 844], 60.00th=[ 908], 70.00th=[ 1128], | |
| 80.00th=[ 8896], 90.00th=[ 15007744], 95.00th=[ 62128128], | |
| 99.00th=[ 387973120], 99.50th=[ 492830720], 99.90th=[ 692060160], | |
| 99.95th=[ 734003200], 99.99th=[1019215872] | |
bw ( KiB/s): min= 0, max=22016, per=90.23%, avg=18550.50, stdev=4900.96, samples=2 | |
iops : min= 0, max= 344, avg=289.50, stdev=77.07, samples=2 | |
lat (nsec) : 750=33.40%, 1000=32.84% | |
lat (usec) : 2=11.88%, 4=0.05%, 10=2.36%, 20=0.94%, 50=0.36% | |
lat (usec) : 100=0.11%, 250=0.05%, 500=0.18%, 750=0.86%, 1000=1.88% | |
lat (msec) : 2=2.49%, 4=1.47%, 10=0.79%, 20=0.64%, 50=3.51% | |
lat (msec) : 100=2.72%, 250=1.69%, 500=1.29%, 750=0.44%, 1000=0.03% | |
cpu : usr=0.09%, sys=0.32%, ctx=12530, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=34807,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:55 2019 | |
read: IOPS=31, BW=2009KiB/s (2057kB/s)(236MiB/120021msec) | |
slat (nsec): min=0, max=427153k, avg=136375.18, stdev=6958530.80 | |
clat (nsec): min=0, max=702092k, avg=9087273.02, stdev=46342012.02 | |
lat (nsec): min=0, max=702124k, avg=9224584.28, stdev=46846870.09 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 780], 30.00th=[ 868], 40.00th=[ 948], | |
| 50.00th=[ 1080], 60.00th=[ 1240], 70.00th=[ 1384], | |
| 80.00th=[ 31872], 90.00th=[ 1777664], 95.00th=[ 46399488], | |
| 99.00th=[261095424], 99.50th=[367001600], 99.90th=[549453824], | |
| 99.95th=[583008256], 99.99th=[700448768] | |
bw ( KiB/s): min= 0, max= 2533, per=9.77%, avg=2008.50, stdev=741.76, samples=2 | |
iops : min= 0, max= 39, avg=31.00, stdev=11.31, samples=2 | |
write: IOPS=32, BW=2091KiB/s (2141kB/s)(245MiB/120021msec) | |
slat (nsec): min=0, max=701141k, avg=429937.85, stdev=12610360.16 | |
clat (nsec): min=0, max=1018.3M, avg=113826493.31, stdev=166482932.38 | |
lat (nsec): min=0, max=1018.4M, avg=114257403.10, stdev=166748154.43 | |
clat percentiles (usec): | |
| 1.00th=[ 1057], 5.00th=[ 1336], 10.00th=[ 1631], | |
| 20.00th=[ 2442], 30.00th=[ 4555], 40.00th=[ 17171], | |
| 50.00th=[ 56886], 60.00th=[ 86508], 70.00th=[ 109577], | |
| 80.00th=[ 156238], 90.00th=[ 358613], 95.00th=[ 513803], | |
| 99.00th=[ 725615], 99.50th=[ 759170], 99.90th=[ 809501], | |
| 99.95th=[1019216], 99.99th=[1019216] | |
bw ( KiB/s): min= 0, max= 2647, per=44.58%, avg=2089.00, stdev=789.13, samples=2 | |
iops : min= 0, max= 41, avg=32.00, stdev=12.73, samples=2 | |
lat (nsec) : 750=7.31%, 1000=13.84% | |
lat (usec) : 2=16.54%, 4=0.08%, 10=0.25%, 20=0.98%, 50=0.70% | |
lat (usec) : 100=1.03%, 250=0.57%, 500=0.57%, 750=0.57%, 1000=1.14% | |
lat (msec) : 2=8.69%, 4=6.87%, 10=4.38%, 20=2.67%, 50=4.96% | |
lat (msec) : 100=10.14%, 250=10.73%, 500=5.18%, 750=2.52%, 1000=0.29% | |
cpu : usr=0.04%, sys=0.19%, ctx=8800, majf=0, minf=34 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=3768,3918,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=20.1MiB/s (21.1MB/s), 2009KiB/s-18.1MiB/s (2057kB/s-18.0MB/s), io=2410MiB (2527MB), run=120020-120021msec | |
WRITE: bw=4687KiB/s (4799kB/s), 2091KiB/s-2596KiB/s (2141kB/s-2658kB/s), io=549MiB (576MB), run=120021-120024msec | |
Disk stats (read/write): | |
rbd113: ios=7017/9006, merge=0/1, ticks=547807/860241, in_queue=1399971, util=11.72% | |
<example-benchmark-fio-server-616-benchmark-vm6nt> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:55 2019 | |
write: IOPS=42, BW=2740KiB/s (2806kB/s)(321MiB/120064msec) | |
slat (nsec): min=0, max=522799k, avg=448436.58, stdev=10743519.39 | |
clat (nsec): min=0, max=1524.9M, avg=93344224.78, stdev=161360206.70 | |
lat (nsec): min=0, max=1524.9M, avg=93793721.83, stdev=161971351.87 | |
clat percentiles (usec): | |
| 1.00th=[ 938], 5.00th=[ 1156], 10.00th=[ 1369], | |
| 20.00th=[ 1876], 30.00th=[ 2868], 40.00th=[ 5145], | |
| 50.00th=[ 19268], 60.00th=[ 60031], 70.00th=[ 87557], | |
| 80.00th=[ 125305], 90.00th=[ 304088], 95.00th=[ 459277], | |
| 99.00th=[ 801113], 99.50th=[ 935330], 99.90th=[1149240], | |
| 99.95th=[1233126], 99.99th=[1518339] | |
bw ( KiB/s): min= 0, max= 1793, per=34.93%, avg=1793.00, stdev= 0.00, samples=1 | |
iops : min= 0, max= 28, avg=28.00, stdev= 0.00, samples=1 | |
lat (usec) : 500=0.02%, 750=0.04%, 1000=1.87% | |
lat (msec) : 2=19.66%, 4=14.99%, 10=9.93%, 20=3.89%, 50=6.64% | |
lat (msec) : 100=17.05%, 250=14.05%, 500=8.00%, 750=2.76%, 1000=0.91% | |
cpu : usr=0.04%, sys=0.20%, ctx=9807, majf=0, minf=32 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,5138,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:55 2019 | |
read: IOPS=383, BW=23.9MiB/s (25.1MB/s)(2872MiB/120013msec) | |
slat (nsec): min=0, max=471122k, avg=46847.59, stdev=3381194.72 | |
clat (nsec): min=0, max=1407.2M, avg=10409463.13, stdev=55502628.48 | |
lat (nsec): min=0, max=1407.2M, avg=10457142.71, stdev=55610776.23 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 708], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 732], 40.00th=[ 740], | |
| 50.00th=[ 812], 60.00th=[ 876], 70.00th=[ 972], | |
| 80.00th=[ 1432], 90.00th=[ 3424256], 95.00th=[ 50593792], | |
| 99.00th=[ 291504128], 99.50th=[ 450887680], 99.90th=[ 675282944], | |
| 99.95th=[ 767557632], 99.99th=[1098907648] | |
bw ( KiB/s): min= 0, max=34035, per=100.00%, avg=34035.00, stdev= 0.00, samples=1 | |
iops : min= 0, max= 532, avg=532.00, stdev= 0.00, samples=1 | |
lat (nsec) : 750=43.64%, 1000=27.38% | |
lat (usec) : 2=10.12%, 4=0.03%, 10=0.80%, 20=0.30%, 50=0.14% | |
lat (usec) : 100=0.06%, 250=0.06%, 500=0.22%, 750=0.88%, 1000=2.09% | |
lat (msec) : 2=2.76%, 4=1.76%, 10=0.97%, 20=0.57%, 50=3.17% | |
lat (msec) : 100=2.66%, 250=1.23%, 500=0.83%, 750=0.29%, 1000=0.04% | |
cpu : usr=0.13%, sys=0.41%, ctx=16042, majf=0, minf=31 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=45981,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:55 2019 | |
read: IOPS=35, BW=2297KiB/s (2352kB/s)(269MiB/120051msec) | |
slat (nsec): min=0, max=3687.3k, avg=25139.08, stdev=91477.78 | |
clat (nsec): min=0, max=1038.4M, avg=7695728.63, stdev=45093642.98 | |
lat (nsec): min=0, max=1038.6M, avg=7721793.49, stdev=45105435.26 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 732], | |
| 20.00th=[ 756], 30.00th=[ 852], 40.00th=[ 948], | |
| 50.00th=[ 1112], 60.00th=[ 1240], 70.00th=[ 1384], | |
| 80.00th=[ 15808], 90.00th=[ 1777664], 95.00th=[ 41156608], | |
| 99.00th=[ 183500800], 99.50th=[ 304087040], 99.90th=[ 666894336], | |
| 99.95th=[ 725614592], 99.99th=[1035993088] | |
bw ( KiB/s): min= 0, max= 2781, per=8.57%, avg=2297.00, stdev=684.48, samples=2 | |
iops : min= 0, max= 43, avg=35.50, stdev=10.61, samples=2 | |
write: IOPS=37, BW=2393KiB/s (2451kB/s)(281MiB/120051msec) | |
slat (nsec): min=0, max=521733k, avg=510207.12, stdev=12528343.03 | |
clat (nsec): min=0, max=1456.0M, avg=99230619.88, stdev=167653400.45 | |
lat (nsec): min=0, max=1456.1M, avg=99741774.10, stdev=168293190.74 | |
clat percentiles (usec): | |
| 1.00th=[ 971], 5.00th=[ 1254], 10.00th=[ 1500], | |
| 20.00th=[ 2089], 30.00th=[ 3326], 40.00th=[ 6456], | |
| 50.00th=[ 23987], 60.00th=[ 61080], 70.00th=[ 88605], | |
| 80.00th=[ 131597], 90.00th=[ 341836], 95.00th=[ 476054], | |
| 99.00th=[ 801113], 99.50th=[ 935330], 99.90th=[1216349], | |
| 99.95th=[1367344], 99.99th=[1451230] | |
bw ( KiB/s): min= 0, max= 2873, per=46.60%, avg=2392.00, stdev=680.24, samples=2 | |
iops : min= 0, max= 44, avg=36.50, stdev=10.61, samples=2 | |
lat (nsec) : 750=9.00%, 1000=12.22% | |
lat (usec) : 2=17.29%, 4=0.08%, 10=0.19%, 20=0.72%, 50=0.70% | |
lat (usec) : 100=0.61%, 250=0.48%, 500=0.49%, 750=0.44%, 1000=1.53% | |
lat (msec) : 2=10.24%, 4=7.73%, 10=6.29%, 20=2.58%, 50=5.10% | |
lat (msec) : 100=9.71%, 250=7.68%, 500=4.64%, 750=1.66%, 1000=0.50% | |
cpu : usr=0.05%, sys=0.21%, ctx=10071, majf=0, minf=38 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=4311,4486,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=26.2MiB/s (27.4MB/s), 2297KiB/s-23.9MiB/s (2352kB/s-25.1MB/s), io=3141MiB (3294MB), run=120013-120051msec | |
WRITE: bw=5133KiB/s (5256kB/s), 2393KiB/s-2740KiB/s (2451kB/s-2806kB/s), io=602MiB (631MB), run=120051-120064msec | |
Disk stats (read/write): | |
rbd226: ios=8971/10050, merge=0/2, ticks=526494/837841, in_queue=1354664, util=14.05% | |
<example-benchmark-fio-server-810-benchmark-s6m2k> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:55 2019 | |
write: IOPS=45, BW=2945KiB/s (3016kB/s)(345MiB/120041msec) | |
slat (nsec): min=0, max=573788k, avg=367710.09, stdev=9878806.33 | |
clat (nsec): min=0, max=1214.2M, avg=86620290.25, stdev=149814104.02 | |
lat (nsec): min=0, max=1214.3M, avg=86989216.49, stdev=149987111.38 | |
clat percentiles (usec): | |
| 1.00th=[ 996], 5.00th=[ 1221], 10.00th=[ 1418], | |
| 20.00th=[ 1909], 30.00th=[ 2900], 40.00th=[ 5211], | |
| 50.00th=[ 13435], 60.00th=[ 47973], 70.00th=[ 85459], | |
| 80.00th=[ 120062], 90.00th=[ 283116], 95.00th=[ 438305], | |
| 99.00th=[ 708838], 99.50th=[ 817890], 99.90th=[ 952108], | |
| 99.95th=[1044382], 99.99th=[1216349] | |
bw ( KiB/s): min= 0, max= 3560, per=55.22%, avg=2944.00, stdev=871.16, samples=2 | |
iops : min= 0, max= 55, avg=45.50, stdev=13.44, samples=2 | |
lat (usec) : 4=0.02%, 750=0.04%, 1000=1.00% | |
lat (msec) : 2=20.12%, 4=14.45%, 10=12.19%, 20=5.11%, 50=7.63% | |
lat (msec) : 100=14.69%, 250=13.80%, 500=7.66%, 750=2.54%, 1000=0.76% | |
cpu : usr=0.06%, sys=0.19%, ctx=10456, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,5521,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:55 2019 | |
read: IOPS=294, BW=18.4MiB/s (19.3MB/s)(2204MiB/120030msec) | |
slat (nsec): min=0, max=310412k, avg=40472.37, stdev=2303449.06 | |
clat (nsec): min=0, max=1217.2M, avg=13618848.09, stdev=62825076.11 | |
lat (nsec): min=0, max=1217.3M, avg=13660203.59, stdev=62875748.57 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 708], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 732], 40.00th=[ 764], | |
| 50.00th=[ 836], 60.00th=[ 892], 70.00th=[ 1096], | |
| 80.00th=[ 561152], 90.00th=[ 23461888], 95.00th=[ 63176704], | |
| 99.00th=[ 354418688], 99.50th=[ 476053504], 99.90th=[ 700448768], | |
| 99.95th=[ 759169024], 99.99th=[1061158912] | |
bw ( KiB/s): min= 0, max=24758, per=89.20%, avg=18807.00, stdev=8415.98, samples=2 | |
iops : min= 0, max= 387, avg=293.50, stdev=132.23, samples=2 | |
lat (nsec) : 750=35.79%, 1000=31.21% | |
lat (usec) : 2=11.36%, 4=0.04%, 10=0.62%, 20=0.46%, 50=0.17% | |
lat (usec) : 100=0.10%, 250=0.05%, 500=0.15%, 750=1.01%, 1000=2.07% | |
lat (msec) : 2=3.01%, 4=1.86%, 10=0.96%, 20=0.78%, 50=3.94% | |
lat (msec) : 100=3.04%, 250=1.73%, 500=1.21%, 750=0.38%, 1000=0.05% | |
cpu : usr=0.09%, sys=0.34%, ctx=14244, majf=0, minf=29 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=35289,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:55 2019 | |
read: IOPS=35, BW=2281KiB/s (2336kB/s)(267MiB/120036msec) | |
slat (nsec): min=0, max=378161k, avg=188024.60, stdev=7512404.09 | |
clat (nsec): min=0, max=739179k, avg=10245570.83, stdev=48682354.00 | |
lat (nsec): min=0, max=739211k, avg=10434521.92, stdev=49233760.49 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 732], | |
| 20.00th=[ 772], 30.00th=[ 868], 40.00th=[ 980], | |
| 50.00th=[ 1112], 60.00th=[ 1240], 70.00th=[ 1416], | |
| 80.00th=[ 56576], 90.00th=[ 4014080], 95.00th=[ 52690944], | |
| 99.00th=[283115520], 99.50th=[354418688], 99.90th=[557842432], | |
| 99.95th=[616562688], 99.99th=[742391808] | |
bw ( KiB/s): min= 0, max= 2588, per=10.82%, avg=2281.50, stdev=433.46, samples=2 | |
iops : min= 0, max= 40, avg=35.00, stdev= 7.07, samples=2 | |
write: IOPS=37, BW=2386KiB/s (2444kB/s)(280MiB/120036msec) | |
slat (nsec): min=0, max=6032.8k, avg=173025.17, stdev=261709.00 | |
clat (nsec): min=0, max=1270.0M, avg=97171527.70, stdev=157884329.78 | |
lat (nsec): min=0, max=1270.3M, avg=97346023.25, stdev=157836529.25 | |
clat percentiles (usec): | |
| 1.00th=[ 1029], 5.00th=[ 1270], 10.00th=[ 1532], | |
| 20.00th=[ 2180], 30.00th=[ 3654], 40.00th=[ 7701], | |
| 50.00th=[ 24773], 60.00th=[ 64750], 70.00th=[ 94897], | |
| 80.00th=[ 135267], 90.00th=[ 312476], 95.00th=[ 476054], | |
| 99.00th=[ 742392], 99.50th=[ 843056], 99.90th=[ 935330], | |
| 99.95th=[ 952108], 99.99th=[1266680] | |
bw ( KiB/s): min= 0, max= 2692, per=44.73%, avg=2384.50, stdev=434.87, samples=2 | |
iops : min= 0, max= 42, avg=37.00, stdev= 7.07, samples=2 | |
lat (nsec) : 750=8.44%, 1000=11.81% | |
lat (usec) : 2=17.02%, 4=0.05%, 10=0.14%, 20=0.80%, 50=0.69% | |
lat (usec) : 100=0.74%, 250=0.49%, 500=0.41%, 750=0.56%, 1000=1.36% | |
lat (msec) : 2=9.93%, 4=7.71%, 10=5.94%, 20=3.15%, 50=5.67% | |
lat (msec) : 100=9.30%, 250=8.91%, 500=4.67%, 750=1.75%, 1000=0.47% | |
cpu : usr=0.06%, sys=0.21%, ctx=10250, majf=0, minf=34 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=4280,4473,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=20.6MiB/s (21.6MB/s), 2281KiB/s-18.4MiB/s (2336kB/s-19.3MB/s), io=2472MiB (2592MB), run=120030-120036msec | |
WRITE: bw=5332KiB/s (5459kB/s), 2386KiB/s-2945KiB/s (2444kB/s-3016kB/s), io=625MiB (655MB), run=120036-120041msec | |
Disk stats (read/write): | |
rbd188: ios=7908/10343, merge=0/1, ticks=544904/835140, in_queue=1370788, util=13.49% | |
<example-benchmark-fio-server-363-benchmark-v4496> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:55 2019 | |
write: IOPS=41, BW=2646KiB/s (2709kB/s)(310MiB/120098msec) | |
slat (nsec): min=0, max=483563k, avg=335624.44, stdev=7666597.67 | |
clat (nsec): min=0, max=1102.5M, avg=96744121.92, stdev=159436843.91 | |
lat (nsec): min=0, max=1102.5M, avg=97080769.02, stdev=159472833.05 | |
clat percentiles (usec): | |
| 1.00th=[ 938], 5.00th=[ 1106], 10.00th=[ 1303], | |
| 20.00th=[ 1663], 30.00th=[ 2343], 40.00th=[ 3818], | |
| 50.00th=[ 23200], 60.00th=[ 72877], 70.00th=[ 103285], | |
| 80.00th=[ 129500], 90.00th=[ 308282], 95.00th=[ 488637], | |
| 99.00th=[ 742392], 99.50th=[ 801113], 99.90th=[ 901776], | |
| 99.95th=[ 935330], 99.99th=[1098908] | |
bw ( KiB/s): min= 0, max= 4098, per=55.36%, avg=2645.50, stdev=2054.15, samples=2 | |
iops : min= 0, max= 64, avg=41.00, stdev=32.53, samples=2 | |
lat (usec) : 2=0.02%, 4=0.04%, 500=0.02%, 750=0.08%, 1000=1.67% | |
lat (msec) : 2=24.00%, 4=14.89%, 10=6.15%, 20=2.56%, 50=4.90% | |
lat (msec) : 100=14.53%, 250=19.87%, 500=6.59%, 750=3.89%, 1000=0.81% | |
cpu : usr=0.05%, sys=0.17%, ctx=9368, majf=0, minf=27 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,4962,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:55 2019 | |
read: IOPS=323, BW=20.2MiB/s (21.2MB/s)(2424MiB/120013msec) | |
slat (nsec): min=0, max=477001k, avg=36573.14, stdev=2637904.02 | |
clat (nsec): min=0, max=940724k, avg=12367744.96, stdev=60911558.88 | |
lat (nsec): min=0, max=941214k, avg=12405146.29, stdev=60976937.14 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 708], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 732], 40.00th=[ 740], | |
| 50.00th=[ 796], 60.00th=[ 860], 70.00th=[ 940], | |
| 80.00th=[ 1304], 90.00th=[ 6127616], 95.00th=[ 55836672], | |
| 99.00th=[367001600], 99.50th=[471859200], 99.90th=[700448768], | |
| 99.95th=[767557632], 99.99th=[843055104] | |
bw ( KiB/s): min= 0, max=25084, per=91.05%, avg=20685.00, stdev=6221.13, samples=2 | |
iops : min= 0, max= 392, avg=323.00, stdev=97.58, samples=2 | |
lat (nsec) : 750=44.89%, 1000=27.73% | |
lat (usec) : 2=9.91%, 4=0.08%, 10=0.35%, 20=0.21%, 50=0.19% | |
lat (usec) : 100=0.06%, 250=0.04%, 500=0.16%, 750=0.79%, 1000=1.59% | |
lat (msec) : 2=2.21%, 4=1.37%, 10=0.80%, 20=0.56%, 50=3.41% | |
lat (msec) : 100=2.67%, 250=1.38%, 500=1.21%, 750=0.34%, 1000=0.06% | |
cpu : usr=0.10%, sys=0.35%, ctx=12840, majf=0, minf=29 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=38815,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:55 2019 | |
read: IOPS=31, BW=2033KiB/s (2082kB/s)(238MiB/120015msec) | |
slat (nsec): min=0, max=436760k, avg=141784.50, stdev=7073229.93 | |
clat (nsec): min=0, max=698805k, avg=9375440.61, stdev=44524086.18 | |
lat (nsec): min=0, max=698827k, avg=9518114.96, stdev=45063482.36 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 732], 10.00th=[ 748], | |
| 20.00th=[ 820], 30.00th=[ 884], 40.00th=[ 988], | |
| 50.00th=[ 1112], 60.00th=[ 1224], 70.00th=[ 1416], | |
| 80.00th=[ 67072], 90.00th=[ 5537792], 95.00th=[ 54788096], | |
| 99.00th=[221249536], 99.50th=[325058560], 99.90th=[599785472], | |
| 99.95th=[683671552], 99.99th=[700448768] | |
bw ( KiB/s): min= 0, max= 3070, per=8.94%, avg=2032.00, stdev=1467.95, samples=2 | |
iops : min= 0, max= 47, avg=31.00, stdev=22.63, samples=2 | |
write: IOPS=33, BW=2135KiB/s (2186kB/s)(250MiB/120015msec) | |
slat (nsec): min=0, max=192477k, avg=220163.16, stdev=3048609.65 | |
clat (nsec): min=0, max=1102.4M, avg=110902683.24, stdev=169976312.05 | |
lat (nsec): min=0, max=1102.6M, avg=111123871.68, stdev=169947686.95 | |
clat percentiles (usec): | |
| 1.00th=[ 963], 5.00th=[ 1156], 10.00th=[ 1369], | |
| 20.00th=[ 1926], 30.00th=[ 3032], 40.00th=[ 8586], | |
| 50.00th=[ 54264], 60.00th=[ 86508], 70.00th=[ 111674], | |
| 80.00th=[ 141558], 90.00th=[ 350225], 95.00th=[ 522191], | |
| 99.00th=[ 750781], 99.50th=[ 843056], 99.90th=[ 926942], | |
| 99.95th=[ 935330], 99.99th=[1098908] | |
bw ( KiB/s): min= 0, max= 3198, per=44.64%, avg=2133.50, stdev=1505.43, samples=2 | |
iops : min= 0, max= 49, avg=32.50, stdev=23.33, samples=2 | |
lat (nsec) : 750=5.14%, 1000=14.97% | |
lat (usec) : 2=16.56%, 4=0.08%, 10=0.14%, 20=0.82%, 50=0.78% | |
lat (usec) : 100=1.10%, 250=0.52%, 500=0.59%, 750=0.64%, 1000=1.61% | |
lat (msec) : 2=11.03%, 4=7.50%, 10=3.43%, 20=1.87%, 50=4.54% | |
lat (msec) : 100=9.75%, 250=11.70%, 500=4.33%, 750=2.42%, 1000=0.50% | |
cpu : usr=0.05%, sys=0.19%, ctx=9130, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=3814,4000,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=22.2MiB/s (23.3MB/s), 2033KiB/s-20.2MiB/s (2082kB/s-21.2MB/s), io=2663MiB (2792MB), run=120013-120015msec | |
WRITE: bw=4779KiB/s (4894kB/s), 2135KiB/s-2646KiB/s (2186kB/s-2709kB/s), io=561MiB (588MB), run=120015-120098msec | |
Disk stats (read/write): | |
rbd141: ios=7942/9637, merge=0/3, ticks=548741/850068, in_queue=1389903, util=12.74% | |
Jobs: 63 (f=59): [_(2),m(1)][99.8%][r=1231M,w=976M][r=19.2k,w=15.2k IOPS][eta 00m:55s] <example-benchmark-fio-server-910-benchmark-74gxv> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:55 2019 | |
write: IOPS=47, BW=3049KiB/s (3122kB/s)(358MiB/120101msec) | |
slat (nsec): min=0, max=587401k, avg=499377.44, stdev=12120076.80 | |
clat (nsec): min=0, max=1699.1M, avg=83679046.09, stdev=152339670.45 | |
lat (nsec): min=0, max=1699.1M, avg=84179546.98, stdev=152850121.23 | |
clat percentiles (usec): | |
| 1.00th=[ 979], 5.00th=[ 1156], 10.00th=[ 1336], | |
| 20.00th=[ 1778], 30.00th=[ 2638], 40.00th=[ 4686], | |
| 50.00th=[ 14222], 60.00th=[ 51643], 70.00th=[ 87557], | |
| 80.00th=[ 117965], 90.00th=[ 225444], 95.00th=[ 413139], | |
| 99.00th=[ 725615], 99.50th=[ 884999], 99.90th=[1149240], | |
| 99.95th=[1300235], 99.99th=[1702888] | |
bw ( KiB/s): min= 0, max= 3874, per=54.17%, avg=3049.50, stdev=1166.02, samples=2 | |
iops : min= 0, max= 60, avg=47.00, stdev=18.38, samples=2 | |
lat (usec) : 750=0.07%, 1000=1.42% | |
lat (msec) : 2=21.72%, 4=14.44%, 10=10.21%, 20=4.77%, 50=6.92% | |
lat (msec) : 100=14.43%, 250=16.89%, 500=5.58%, 750=2.64%, 1000=0.65% | |
cpu : usr=0.06%, sys=0.20%, ctx=10898, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,5719,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:55 2019 | |
read: IOPS=369, BW=23.0MiB/s (24.2MB/s)(2766MiB/120003msec) | |
slat (nsec): min=0, max=586968k, avg=49643.35, stdev=3526447.54 | |
clat (nsec): min=0, max=1687.4M, avg=10816930.67, stdev=56864630.85 | |
lat (nsec): min=0, max=1687.5M, avg=10867413.28, stdev=56980443.13 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 732], 40.00th=[ 740], | |
| 50.00th=[ 812], 60.00th=[ 868], 70.00th=[ 948], | |
| 80.00th=[ 1384], 90.00th=[ 3555328], 95.00th=[ 50593792], | |
| 99.00th=[ 308281344], 99.50th=[ 438304768], 99.90th=[ 658505728], | |
| 99.95th=[ 734003200], 99.99th=[1233125376] | |
bw ( KiB/s): min= 0, max=34327, per=90.47%, avg=23600.50, stdev=15169.56, samples=2 | |
iops : min= 0, max= 537, avg=369.00, stdev=237.59, samples=2 | |
lat (nsec) : 750=43.42%, 1000=28.81% | |
lat (usec) : 2=9.35%, 4=0.04%, 10=0.87%, 20=1.08%, 50=0.38% | |
lat (usec) : 100=0.12%, 250=0.07%, 500=0.10%, 750=0.73%, 1000=1.59% | |
lat (msec) : 2=2.31%, 4=1.29%, 10=0.76%, 20=0.71%, 50=3.29% | |
lat (msec) : 100=2.40%, 250=1.39%, 500=0.95%, 750=0.28%, 1000=0.03% | |
cpu : usr=0.10%, sys=0.41%, ctx=14046, majf=0, minf=97 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=44330,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:55 2019 | |
read: IOPS=38, BW=2487KiB/s (2547kB/s)(292MiB/120019msec) | |
slat (nsec): min=0, max=368100k, avg=106961.37, stdev=5389940.62 | |
clat (nsec): min=0, max=759558k, avg=9002016.33, stdev=40516136.77 | |
lat (nsec): min=0, max=759588k, avg=9109980.22, stdev=40868878.36 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 804], 30.00th=[ 884], 40.00th=[ 996], | |
| 50.00th=[ 1144], 60.00th=[ 1256], 70.00th=[ 1432], | |
| 80.00th=[ 71168], 90.00th=[ 5275648], 95.00th=[ 54263808], | |
| 99.00th=[212860928], 99.50th=[308281344], 99.90th=[429916160], | |
| 99.95th=[476053504], 99.99th=[759169024] | |
bw ( KiB/s): min= 0, max= 2916, per=9.53%, avg=2486.50, stdev=607.40, samples=2 | |
iops : min= 0, max= 45, avg=38.50, stdev= 9.19, samples=2 | |
write: IOPS=40, BW=2583KiB/s (2645kB/s)(303MiB/120019msec) | |
slat (nsec): min=0, max=626891k, avg=490839.66, stdev=12588951.98 | |
clat (nsec): min=0, max=1796.9M, avg=90155001.53, stdev=160192506.95 | |
lat (nsec): min=0, max=1796.0M, avg=90646821.59, stdev=160496253.46 | |
clat percentiles (usec): | |
| 1.00th=[ 1012], 5.00th=[ 1237], 10.00th=[ 1434], | |
| 20.00th=[ 2040], 30.00th=[ 3294], 40.00th=[ 6587], | |
| 50.00th=[ 21627], 60.00th=[ 59507], 70.00th=[ 91751], | |
| 80.00th=[ 121111], 90.00th=[ 252707], 95.00th=[ 442500], | |
| 99.00th=[ 759170], 99.50th=[ 884999], 99.90th=[1300235], | |
| 99.95th=[1686111], 99.99th=[1803551] | |
bw ( KiB/s): min= 0, max= 3015, per=45.85%, avg=2581.50, stdev=613.06, samples=2 | |
iops : min= 0, max= 47, avg=40.00, stdev= 9.90, samples=2 | |
lat (nsec) : 750=7.48%, 1000=12.19% | |
lat (usec) : 2=17.28%, 4=0.13%, 10=0.34%, 20=0.79%, 50=0.62% | |
lat (usec) : 100=0.64%, 250=0.64%, 500=0.63%, 750=0.41%, 1000=1.38% | |
lat (msec) : 2=10.86%, 4=7.39%, 10=6.11%, 20=3.06%, 50=5.52% | |
lat (msec) : 100=9.36%, 250=9.71%, 500=3.46%, 750=1.48%, 1000=0.39% | |
cpu : usr=0.05%, sys=0.24%, ctx=11003, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=4666,4841,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=25.5MiB/s (26.7MB/s), 2487KiB/s-23.0MiB/s (2547kB/s-24.2MB/s), io=3058MiB (3206MB), run=120003-120019msec | |
WRITE: bw=5630KiB/s (5766kB/s), 2583KiB/s-3049KiB/s (2645kB/s-3122kB/s), io=660MiB (692MB), run=120019-120101msec | |
Disk stats (read/write): | |
rbd320: ios=8138/10744, merge=0/1, ticks=543117/803685, in_queue=1337178, util=13.96% | |
<example-benchmark-fio-server-697-benchmark-v5m2j> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:55 2019 | |
write: IOPS=52, BW=3334KiB/s (3414kB/s)(391MiB/120024msec) | |
slat (nsec): min=0, max=950382k, avg=452969.44, stdev=14552434.79 | |
clat (nsec): min=0, max=1440.0M, avg=76854293.56, stdev=153490325.63 | |
lat (nsec): min=0, max=1441.2M, avg=77308421.84, stdev=154089812.05 | |
clat percentiles (usec): | |
| 1.00th=[ 955], 5.00th=[ 1205], 10.00th=[ 1401], | |
| 20.00th=[ 1762], 30.00th=[ 2376], 40.00th=[ 3589], | |
| 50.00th=[ 6849], 60.00th=[ 23725], 70.00th=[ 76022], | |
| 80.00th=[ 107480], 90.00th=[ 208667], 95.00th=[ 392168], | |
| 99.00th=[ 784335], 99.50th=[ 910164], 99.90th=[1199571], | |
| 99.95th=[1300235], 99.99th=[1434452] | |
bw ( KiB/s): min= 0, max= 5185, per=58.75%, avg=3333.00, stdev=2619.12, samples=2 | |
iops : min= 0, max= 81, avg=52.00, stdev=41.01, samples=2 | |
lat (usec) : 100=0.02%, 250=0.03%, 750=0.03%, 1000=1.36% | |
lat (msec) : 2=23.31%, 4=17.55%, 10=11.94%, 20=4.78%, 50=4.61% | |
lat (msec) : 100=14.32%, 250=13.89%, 500=4.88%, 750=2.08%, 1000=0.99% | |
cpu : usr=0.06%, sys=0.22%, ctx=11738, majf=0, minf=27 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,6250,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:55 2019 | |
read: IOPS=341, BW=21.3MiB/s (22.4MB/s)(2561MiB/120020msec) | |
slat (nsec): min=0, max=871267k, avg=58333.28, stdev=5300696.13 | |
clat (nsec): min=0, max=1267.3M, avg=11691015.06, stdev=59871844.84 | |
lat (nsec): min=0, max=1267.5M, avg=11750191.55, stdev=60112618.15 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 732], 40.00th=[ 772], | |
| 50.00th=[ 836], 60.00th=[ 892], 70.00th=[ 1032], | |
| 80.00th=[ 1576], 90.00th=[ 4882432], 95.00th=[ 53739520], | |
| 99.00th=[ 337641472], 99.50th=[ 467664896], 99.90th=[ 692060160], | |
| 99.95th=[ 784334848], 99.99th=[1082130432] | |
bw ( KiB/s): min= 0, max=28268, per=90.74%, avg=21854.00, stdev=9070.77, samples=2 | |
iops : min= 0, max= 442, avg=341.50, stdev=142.13, samples=2 | |
lat (nsec) : 750=36.43%, 1000=32.41% | |
lat (usec) : 2=11.69%, 4=0.06%, 10=0.56%, 20=0.22%, 50=0.16% | |
lat (usec) : 100=0.09%, 250=0.04%, 500=0.25%, 750=1.03%, 1000=2.28% | |
lat (msec) : 2=2.99%, 4=1.55%, 10=0.76%, 20=0.64%, 50=3.41% | |
lat (msec) : 100=2.65%, 250=1.38%, 500=1.02%, 750=0.31%, 1000=0.04% | |
cpu : usr=0.11%, sys=0.39%, ctx=14938, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=41006,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:55 2019 | |
read: IOPS=34, BW=2232KiB/s (2286kB/s)(262MiB/120028msec) | |
slat (nsec): min=0, max=3261.9k, avg=30353.32, stdev=99290.01 | |
clat (nsec): min=0, max=1235.7M, avg=9591222.87, stdev=49457877.63 | |
lat (nsec): min=0, max=1235.9M, avg=9622550.11, stdev=49466608.08 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 804], 30.00th=[ 884], 40.00th=[ 1020], | |
| 50.00th=[ 1160], 60.00th=[ 1288], 70.00th=[ 1496], | |
| 80.00th=[ 193536], 90.00th=[ 6127616], 95.00th=[ 52690944], | |
| 99.00th=[ 235929600], 99.50th=[ 354418688], 99.90th=[ 517996544], | |
| 99.95th=[ 641728512], 99.99th=[1233125376] | |
bw ( KiB/s): min= 0, max= 3177, per=9.27%, avg=2232.00, stdev=1336.43, samples=2 | |
iops : min= 0, max= 49, avg=34.50, stdev=20.51, samples=2 | |
write: IOPS=36, BW=2340KiB/s (2396kB/s)(274MiB/120028msec) | |
slat (nsec): min=0, max=839359k, avg=446443.00, stdev=13829654.09 | |
clat (nsec): min=0, max=1464.5M, avg=100143464.64, stdev=174579671.22 | |
lat (nsec): min=0, max=1464.6M, avg=100590859.57, stdev=175258011.37 | |
clat percentiles (usec): | |
| 1.00th=[ 1057], 5.00th=[ 1287], 10.00th=[ 1532], | |
| 20.00th=[ 2212], 30.00th=[ 3589], 40.00th=[ 7373], | |
| 50.00th=[ 29754], 60.00th=[ 71828], 70.00th=[ 95945], | |
| 80.00th=[ 125305], 90.00th=[ 278922], 95.00th=[ 484443], | |
| 99.00th=[ 884999], 99.50th=[ 943719], 99.90th=[1350566], | |
| 99.95th=[1468007], 99.99th=[1468007] | |
bw ( KiB/s): min= 0, max= 3323, per=41.20%, avg=2337.50, stdev=1393.71, samples=2 | |
iops : min= 0, max= 51, avg=36.00, stdev=21.21, samples=2 | |
lat (nsec) : 750=7.20%, 1000=11.69% | |
lat (usec) : 2=17.30%, 4=0.08%, 10=0.29%, 20=0.59%, 50=0.77% | |
lat (usec) : 100=0.80%, 250=0.59%, 500=0.42%, 750=0.65%, 1000=1.42% | |
lat (msec) : 2=10.28%, 4=8.01%, 10=6.04%, 20=2.99%, 50=4.37% | |
lat (msec) : 100=11.07%, 250=9.46%, 500=3.51%, 750=1.54%, 1000=0.75% | |
cpu : usr=0.06%, sys=0.21%, ctx=10176, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=4187,4385,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=23.5MiB/s (24.7MB/s), 2232KiB/s-21.3MiB/s (2286kB/s-22.4MB/s), io=2823MiB (2960MB), run=120020-120028msec | |
WRITE: bw=5674KiB/s (5810kB/s), 2340KiB/s-3334KiB/s (2396kB/s-3414kB/s), io=665MiB (697MB), run=120024-120028msec | |
Disk stats (read/write): | |
rbd234: ios=8956/11204, merge=0/1, ticks=542224/827259, in_queue=1359118, util=14.50% | |
<example-benchmark-fio-server-112-benchmark-wpgw6> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:55 2019 | |
write: IOPS=47, BW=3020KiB/s (3092kB/s)(354MiB/120030msec) | |
slat (nsec): min=0, max=434516k, avg=325543.33, stdev=7343542.39 | |
clat (nsec): min=0, max=953098k, avg=84734749.94, stdev=150303929.00 | |
lat (nsec): min=0, max=953156k, avg=85061546.03, stdev=150791631.58 | |
clat percentiles (usec): | |
| 1.00th=[ 996], 5.00th=[ 1205], 10.00th=[ 1401], 20.00th=[ 1844], | |
| 30.00th=[ 2573], 40.00th=[ 4113], 50.00th=[ 8848], 60.00th=[ 36963], | |
| 70.00th=[ 79168], 80.00th=[119014], 90.00th=[295699], 95.00th=[459277], | |
| 99.00th=[683672], 99.50th=[725615], 99.90th=[792724], 99.95th=[876610], | |
| 99.99th=[952108] | |
bw ( KiB/s): min= 0, max= 4542, per=57.71%, avg=3017.50, stdev=2155.97, samples=2 | |
iops : min= 0, max= 70, avg=46.50, stdev=33.23, samples=2 | |
lat (usec) : 4=0.02%, 50=0.02%, 250=0.02%, 500=0.04%, 750=0.02% | |
lat (usec) : 1000=0.97% | |
lat (msec) : 2=21.73%, 4=16.86%, 10=11.47%, 20=4.84%, 50=6.17% | |
lat (msec) : 100=13.85%, 250=13.06%, 500=7.26%, 750=3.43%, 1000=0.32% | |
cpu : usr=0.05%, sys=0.19%, ctx=10421, majf=0, minf=25 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,5660,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:55 2019 | |
read: IOPS=336, BW=21.0MiB/s (22.0MB/s)(2523MiB/120021msec) | |
slat (nsec): min=0, max=406062k, avg=39732.49, stdev=2589915.75 | |
clat (nsec): min=0, max=1114.5M, avg=11880232.54, stdev=59995718.21 | |
lat (nsec): min=0, max=1114.6M, avg=11920800.50, stdev=60062257.19 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 708], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 732], 40.00th=[ 740], | |
| 50.00th=[ 796], 60.00th=[ 868], 70.00th=[ 972], | |
| 80.00th=[ 1432], 90.00th=[ 3620864], 95.00th=[ 53739520], | |
| 99.00th=[362807296], 99.50th=[471859200], 99.90th=[675282944], | |
| 99.95th=[742391808], 99.99th=[901775360] | |
bw ( KiB/s): min= 0, max=27225, per=91.10%, avg=21528.00, stdev=8056.77, samples=2 | |
iops : min= 0, max= 425, avg=336.00, stdev=125.87, samples=2 | |
lat (nsec) : 750=44.70%, 1000=26.30% | |
lat (usec) : 2=10.01%, 4=0.04%, 10=0.50%, 20=0.32%, 50=0.16% | |
lat (usec) : 100=0.06%, 250=0.06%, 500=0.29%, 750=1.02%, 1000=2.18% | |
lat (msec) : 2=2.88%, 4=1.61%, 10=0.76%, 20=0.62%, 50=3.14% | |
lat (msec) : 100=2.47%, 250=1.40%, 500=1.09%, 750=0.37%, 1000=0.04% | |
cpu : usr=0.11%, sys=0.37%, ctx=14426, majf=0, minf=29 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=40393,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:55 2019 | |
read: IOPS=32, BW=2106KiB/s (2156kB/s)(247MiB/120003msec) | |
slat (nsec): min=0, max=3284.3k, avg=29352.61, stdev=98817.29 | |
clat (nsec): min=0, max=785351k, avg=10373089.41, stdev=48731284.43 | |
lat (nsec): min=0, max=785379k, avg=10403500.78, stdev=48741066.99 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 732], | |
| 20.00th=[ 772], 30.00th=[ 884], 40.00th=[ 1004], | |
| 50.00th=[ 1144], 60.00th=[ 1288], 70.00th=[ 1480], | |
| 80.00th=[ 154624], 90.00th=[ 11337728], 95.00th=[ 54788096], | |
| 99.00th=[265289728], 99.50th=[387973120], 99.90th=[591396864], | |
| 99.95th=[616562688], 99.99th=[784334848] | |
bw ( KiB/s): min= 0, max= 3043, per=8.91%, avg=2104.50, stdev=1327.24, samples=2 | |
iops : min= 0, max= 47, avg=32.50, stdev=20.51, samples=2 | |
write: IOPS=34, BW=2211KiB/s (2264kB/s)(259MiB/120003msec) | |
slat (nsec): min=0, max=392423k, avg=356288.72, stdev=8053940.39 | |
clat (nsec): min=0, max=1000.3M, avg=105913790.38, stdev=167085986.59 | |
lat (nsec): min=0, max=1000.4M, avg=106271139.65, stdev=167201360.65 | |
clat percentiles (usec): | |
| 1.00th=[ 1037], 5.00th=[ 1237], 10.00th=[ 1467], | |
| 20.00th=[ 2114], 30.00th=[ 3392], 40.00th=[ 6980], | |
| 50.00th=[ 27395], 60.00th=[ 68682], 70.00th=[ 99091], | |
| 80.00th=[ 147850], 90.00th=[ 371196], 95.00th=[ 497026], | |
| 99.00th=[ 717226], 99.50th=[ 750781], 99.90th=[ 876610], | |
| 99.95th=[ 876610], 99.99th=[1002439] | |
bw ( KiB/s): min= 0, max= 3193, per=42.24%, avg=2208.50, stdev=1392.29, samples=2 | |
iops : min= 0, max= 49, avg=34.00, stdev=21.21, samples=2 | |
lat (nsec) : 750=8.26%, 1000=11.10% | |
lat (usec) : 2=16.60%, 4=0.10%, 10=0.26%, 20=0.94%, 50=0.78% | |
lat (usec) : 100=0.75%, 250=0.61%, 500=0.54%, 750=0.48%, 1000=1.27% | |
lat (msec) : 2=10.62%, 4=7.74%, 10=6.02%, 20=2.82%, 50=5.01% | |
lat (msec) : 100=9.83%, 250=8.37%, 500=5.39%, 750=2.29%, 1000=0.27% | |
cpu : usr=0.05%, sys=0.19%, ctx=9532, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=3949,4142,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=23.1MiB/s (24.2MB/s), 2106KiB/s-21.0MiB/s (2156kB/s-22.0MB/s), io=2770MiB (2904MB), run=120003-120021msec | |
WRITE: bw=5230KiB/s (5355kB/s), 2211KiB/s-3020KiB/s (2264kB/s-3092kB/s), io=613MiB (643MB), run=120003-120030msec | |
Disk stats (read/write): | |
rbd37: ios=8286/10343, merge=0/3, ticks=542121/837826, in_queue=1370444, util=13.14% | |
<example-benchmark-fio-server-5-benchmark-hchc9> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:55 2019 | |
write: IOPS=43, BW=2784KiB/s (2851kB/s)(326MiB/120043msec) | |
slat (nsec): min=0, max=1469.3M, avg=542823.26, stdev=21352335.09 | |
clat (nsec): min=0, max=1719.5M, avg=91545953.17, stdev=150529149.13 | |
lat (nsec): min=0, max=2232.0M, avg=92089805.76, stdev=153224563.06 | |
clat percentiles (usec): | |
| 1.00th=[ 988], 5.00th=[ 1237], 10.00th=[ 1418], | |
| 20.00th=[ 1958], 30.00th=[ 3228], 40.00th=[ 6980], | |
| 50.00th=[ 42730], 60.00th=[ 78119], 70.00th=[ 102237], | |
| 80.00th=[ 127402], 90.00th=[ 219153], 95.00th=[ 408945], | |
| 99.00th=[ 725615], 99.50th=[ 834667], 99.90th=[1350566], | |
| 99.95th=[1468007], 99.99th=[1719665] | |
bw ( KiB/s): min= 0, max= 3203, per=54.41%, avg=2782.50, stdev=594.68, samples=2 | |
iops : min= 0, max= 50, avg=43.00, stdev= 9.90, samples=2 | |
lat (usec) : 4=0.02%, 750=0.11%, 1000=1.03% | |
lat (msec) : 2=19.37%, 4=12.93%, 10=9.37%, 20=3.37%, 50=5.31% | |
lat (msec) : 100=17.24%, 250=22.05%, 500=6.05%, 750=2.28%, 1000=0.73% | |
cpu : usr=0.05%, sys=0.18%, ctx=9915, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,5219,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:55 2019 | |
read: IOPS=348, BW=21.8MiB/s (22.9MB/s)(2615MiB/120001msec) | |
slat (nsec): min=0, max=1466.4M, avg=67849.19, stdev=7423963.40 | |
clat (nsec): min=0, max=1973.0M, avg=11410932.15, stdev=57537264.39 | |
lat (nsec): min=0, max=1974.0M, avg=11479646.23, stdev=58075993.63 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 708], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 740], 40.00th=[ 780], | |
| 50.00th=[ 852], 60.00th=[ 932], 70.00th=[ 1192], | |
| 80.00th=[ 839680], 90.00th=[ 14352384], 95.00th=[ 54788096], | |
| 99.00th=[ 304087040], 99.50th=[ 438304768], 99.90th=[ 683671552], | |
| 99.95th=[ 792723456], 99.99th=[1216348160] | |
bw ( KiB/s): min= 0, max=31764, per=90.94%, avg=22314.00, stdev=13364.32, samples=2 | |
iops : min= 0, max= 496, avg=348.50, stdev=208.60, samples=2 | |
lat (nsec) : 750=35.07%, 1000=28.45% | |
lat (usec) : 2=12.71%, 4=0.05%, 10=0.53%, 20=0.29%, 50=0.14% | |
lat (usec) : 100=0.06%, 250=0.07%, 500=0.41%, 750=1.13%, 1000=2.65% | |
lat (msec) : 2=3.94%, 4=2.52%, 10=1.49%, 20=1.02%, 50=3.88% | |
lat (msec) : 100=3.09%, 250=1.23%, 500=0.96%, 750=0.25%, 1000=0.05% | |
cpu : usr=0.12%, sys=0.42%, ctx=18819, majf=0, minf=27 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=41856,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:55 2019 | |
read: IOPS=34, BW=2226KiB/s (2280kB/s)(261MiB/120024msec) | |
slat (nsec): min=0, max=1469.5M, avg=485436.54, stdev=23708009.49 | |
clat (nsec): min=0, max=1068.9M, avg=10575940.19, stdev=49916812.10 | |
lat (nsec): min=0, max=1470.0M, avg=11062295.90, stdev=55220809.67 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 732], | |
| 20.00th=[ 780], 30.00th=[ 884], 40.00th=[ 1012], | |
| 50.00th=[ 1160], 60.00th=[ 1272], 70.00th=[ 1480], | |
| 80.00th=[ 193536], 90.00th=[ 21102592], 95.00th=[ 57933824], | |
| 99.00th=[ 235929600], 99.50th=[ 362807296], 99.90th=[ 616562688], | |
| 99.95th=[ 734003200], 99.99th=[1069547520] | |
bw ( KiB/s): min= 0, max= 2374, per=9.07%, avg=2226.00, stdev=209.30, samples=2 | |
iops : min= 0, max= 37, avg=34.50, stdev= 3.54, samples=2 | |
write: IOPS=36, BW=2331KiB/s (2387kB/s)(273MiB/120024msec) | |
slat (nsec): min=0, max=6481.8k, avg=159585.38, stdev=274880.69 | |
clat (nsec): min=0, max=1784.7M, avg=99395294.56, stdev=159205507.24 | |
lat (nsec): min=0, max=1784.7M, avg=99556016.47, stdev=159164612.17 | |
clat percentiles (usec): | |
| 1.00th=[ 1037], 5.00th=[ 1287], 10.00th=[ 1549], | |
| 20.00th=[ 2343], 30.00th=[ 4228], 40.00th=[ 10683], | |
| 50.00th=[ 54264], 60.00th=[ 81265], 70.00th=[ 106431], | |
| 80.00th=[ 131597], 90.00th=[ 270533], 95.00th=[ 442500], | |
| 99.00th=[ 784335], 99.50th=[ 918553], 99.90th=[1384121], | |
| 99.95th=[1468007], 99.99th=[1786774] | |
bw ( KiB/s): min= 0, max= 2476, per=45.53%, avg=2328.50, stdev=208.60, samples=2 | |
iops : min= 0, max= 38, avg=36.00, stdev= 2.83, samples=2 | |
lat (nsec) : 750=7.96%, 1000=11.17% | |
lat (usec) : 2=16.96%, 4=0.06%, 10=0.27%, 20=0.83%, 50=0.71% | |
lat (usec) : 100=0.70%, 250=0.57%, 500=0.54%, 750=0.55%, 1000=1.05% | |
lat (msec) : 2=9.34%, 4=7.48%, 10=5.65%, 20=1.94%, 50=5.16% | |
lat (msec) : 100=11.00%, 250=12.30%, 500=3.83%, 750=1.32%, 1000=0.49% | |
cpu : usr=0.04%, sys=0.22%, ctx=10113, majf=0, minf=34 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=4175,4368,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=23.0MiB/s (25.1MB/s), 2226KiB/s-21.8MiB/s (2280kB/s-22.9MB/s), io=2876MiB (3016MB), run=120001-120024msec | |
WRITE: bw=5114KiB/s (5237kB/s), 2331KiB/s-2784KiB/s (2387kB/s-2851kB/s), io=600MiB (629MB), run=120024-120043msec | |
Disk stats (read/write): | |
rbd2: ios=11019/10797, merge=0/1, ticks=541896/812590, in_queue=1343285, util=15.83% | |
<example-benchmark-fio-server-752-benchmark-kfgvn> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:55 2019 | |
write: IOPS=38, BW=2456KiB/s (2515kB/s)(288MiB/120075msec) | |
slat (nsec): min=0, max=486355k, avg=347151.01, stdev=8370452.54 | |
clat (nsec): min=0, max=1379.9M, avg=104506667.37, stdev=171039013.89 | |
lat (nsec): min=0, max=1379.0M, avg=104854912.10, stdev=171182329.65 | |
clat percentiles (usec): | |
| 1.00th=[ 988], 5.00th=[ 1188], 10.00th=[ 1385], | |
| 20.00th=[ 1860], 30.00th=[ 2835], 40.00th=[ 5735], | |
| 50.00th=[ 19006], 60.00th=[ 63701], 70.00th=[ 99091], | |
| 80.00th=[ 154141], 90.00th=[ 346031], 95.00th=[ 501220], | |
| 99.00th=[ 742392], 99.50th=[ 801113], 99.90th=[1132463], | |
| 99.95th=[1300235], 99.99th=[1384121] | |
bw ( KiB/s): min= 0, max= 3208, per=55.27%, avg=2455.00, stdev=1064.90, samples=2 | |
iops : min= 0, max= 50, avg=38.00, stdev=16.97, samples=2 | |
lat (usec) : 500=0.07%, 750=0.02%, 1000=1.13% | |
lat (msec) : 2=21.35%, 4=13.07%, 10=9.29%, 20=5.52%, 50=5.91% | |
lat (msec) : 100=13.77%, 250=15.44%, 500=9.38%, 750=4.13%, 1000=0.83% | |
cpu : usr=0.05%, sys=0.16%, ctx=8673, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,4605,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:55 2019 | |
read: IOPS=336, BW=21.0MiB/s (22.0MB/s)(2521MiB/120024msec) | |
slat (nsec): min=0, max=476699k, avg=39713.39, stdev=2703973.90 | |
clat (nsec): min=0, max=1301.7M, avg=11885616.09, stdev=61578485.33 | |
lat (nsec): min=0, max=1301.7M, avg=11926172.49, stdev=61648490.33 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 708], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 732], 40.00th=[ 764], | |
| 50.00th=[ 828], 60.00th=[ 876], 70.00th=[ 996], | |
| 80.00th=[ 1432], 90.00th=[ 2670592], 95.00th=[ 53215232], | |
| 99.00th=[ 371195904], 99.50th=[ 476053504], 99.90th=[ 675282944], | |
| 99.95th=[ 742391808], 99.99th=[1019215872] | |
bw ( KiB/s): min= 0, max=25368, per=91.91%, avg=21512.50, stdev=5452.50, samples=2 | |
iops : min= 0, max= 396, avg=335.50, stdev=85.56, samples=2 | |
lat (nsec) : 750=36.50%, 1000=33.72% | |
lat (usec) : 2=11.17%, 4=0.10%, 10=0.82%, 20=0.63%, 50=0.14% | |
lat (usec) : 100=0.07%, 250=0.05%, 500=0.21%, 750=0.93%, 1000=1.97% | |
lat (msec) : 2=2.80%, 4=1.64%, 10=0.82%, 20=0.49%, 50=2.68% | |
lat (msec) : 100=2.35%, 250=1.38%, 500=1.12%, 750=0.36%, 1000=0.04% | |
cpu : usr=0.11%, sys=0.37%, ctx=13493, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=40363,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=14: Wed May 8 20:34:55 2019 | |
read: IOPS=29, BW=1897KiB/s (1943kB/s)(222MiB/120034msec) | |
slat (nsec): min=0, max=394240k, avg=221723.67, stdev=8238630.85 | |
clat (nsec): min=0, max=533183k, avg=8440051.27, stdev=43256219.94 | |
lat (nsec): min=0, max=533276k, avg=8662693.17, stdev=44003117.30 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 772], 30.00th=[ 860], 40.00th=[ 956], | |
| 50.00th=[ 1096], 60.00th=[ 1240], 70.00th=[ 1432], | |
| 80.00th=[ 53504], 90.00th=[ 1630208], 95.00th=[ 42205184], | |
| 99.00th=[250609664], 99.50th=[346030080], 99.90th=[488636416], | |
| 99.95th=[530579456], 99.99th=[534773760] | |
bw ( KiB/s): min= 0, max= 2520, per=8.10%, avg=1896.50, stdev=881.76, samples=2 | |
iops : min= 0, max= 39, avg=29.00, stdev=14.14, samples=2 | |
write: IOPS=31, BW=1987KiB/s (2035kB/s)(233MiB/120034msec) | |
slat (nsec): min=0, max=4499.2k, avg=170586.55, stdev=218293.04 | |
clat (nsec): min=0, max=1384.8M, avg=121032677.53, stdev=182906307.19 | |
lat (nsec): min=0, max=1384.9M, avg=121204248.52, stdev=182846462.11 | |
clat percentiles (usec): | |
| 1.00th=[ 1012], 5.00th=[ 1254], 10.00th=[ 1483], | |
| 20.00th=[ 2089], 30.00th=[ 3621], 40.00th=[ 9765], | |
| 50.00th=[ 45351], 60.00th=[ 81265], 70.00th=[ 112722], | |
| 80.00th=[ 206570], 90.00th=[ 392168], 95.00th=[ 541066], | |
| 99.00th=[ 767558], 99.50th=[ 901776], 99.90th=[1052771], | |
| 99.95th=[1300235], 99.99th=[1384121] | |
bw ( KiB/s): min= 0, max= 2640, per=44.71%, avg=1986.00, stdev=924.90, samples=2 | |
iops : min= 0, max= 41, avg=30.50, stdev=14.85, samples=2 | |
lat (nsec) : 750=8.23%, 1000=12.94% | |
lat (usec) : 2=15.82%, 4=0.07%, 10=0.12%, 20=0.73%, 50=1.07% | |
lat (usec) : 100=1.07%, 250=0.65%, 500=0.91%, 750=0.58%, 1000=1.33% | |
lat (msec) : 2=10.42%, 4=6.85%, 10=4.79%, 20=2.93%, 50=4.44% | |
lat (msec) : 100=8.27%, 250=9.74%, 500=5.82%, 750=2.66%, 1000=0.52% | |
cpu : usr=0.06%, sys=0.16%, ctx=8298, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=3559,3723,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=22.9MiB/s (23.0MB/s), 1897KiB/s-21.0MiB/s (1943kB/s-22.0MB/s), io=2744MiB (2877MB), run=120024-120034msec | |
WRITE: bw=4443KiB/s (4549kB/s), 1987KiB/s-2456KiB/s (2035kB/s-2515kB/s), io=521MiB (546MB), run=120034-120075msec | |
Disk stats (read/write): | |
rbd272: ios=7654/8582, merge=0/1, ticks=537074/870601, in_queue=1399336, util=11.51% | |
<example-benchmark-fio-server-427-benchmark-hzqrn> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:55 2019 | |
write: IOPS=55, BW=3533KiB/s (3617kB/s)(414MiB/120097msec) | |
slat (nsec): min=0, max=456194k, avg=355259.52, stdev=8398375.70 | |
clat (nsec): min=0, max=991198k, avg=72238614.86, stdev=141200455.04 | |
lat (nsec): min=0, max=991288k, avg=72595361.15, stdev=141328353.19 | |
clat percentiles (usec): | |
| 1.00th=[ 1004], 5.00th=[ 1221], 10.00th=[ 1401], 20.00th=[ 1795], | |
| 30.00th=[ 2409], 40.00th=[ 3654], 50.00th=[ 6521], 60.00th=[ 16450], | |
| 70.00th=[ 69731], 80.00th=[104334], 90.00th=[204473], 95.00th=[392168], | |
| 99.00th=[708838], 99.50th=[792724], 99.90th=[926942], 99.95th=[952108], | |
| 99.99th=[994051] | |
bw ( KiB/s): min= 0, max= 4756, per=56.38%, avg=3533.00, stdev=1729.58, samples=2 | |
iops : min= 0, max= 74, avg=55.00, stdev=26.87, samples=2 | |
lat (usec) : 500=0.02%, 750=0.03%, 1000=0.92% | |
lat (msec) : 2=23.29%, 4=17.60%, 10=13.67%, 20=5.77%, 50=4.87% | |
lat (msec) : 100=12.65%, 250=12.93%, 500=4.95%, 750=2.69%, 1000=0.66% | |
cpu : usr=0.08%, sys=0.23%, ctx=12600, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,6626,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:55 2019 | |
read: IOPS=336, BW=21.0MiB/s (22.1MB/s)(2526MiB/120015msec) | |
slat (nsec): min=0, max=362838k, avg=45577.70, stdev=2841029.73 | |
clat (nsec): min=0, max=1029.8M, avg=11837195.23, stdev=59499909.85 | |
lat (nsec): min=0, max=1029.9M, avg=11883621.69, stdev=59574212.09 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 708], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 732], 40.00th=[ 748], | |
| 50.00th=[ 836], 60.00th=[ 892], 70.00th=[ 1080], | |
| 80.00th=[ 60672], 90.00th=[ 9240576], 95.00th=[ 54788096], | |
| 99.00th=[346030080], 99.50th=[476053504], 99.90th=[692060160], | |
| 99.95th=[750780416], 99.99th=[943718400] | |
bw ( KiB/s): min= 0, max=28561, per=89.07%, avg=21554.00, stdev=9909.39, samples=2 | |
iops : min= 0, max= 446, avg=336.50, stdev=154.86, samples=2 | |
lat (nsec) : 750=39.64%, 1000=28.22% | |
lat (usec) : 2=10.44%, 4=0.04%, 10=0.88%, 20=0.50%, 50=0.24% | |
lat (usec) : 100=0.10%, 250=0.06%, 500=0.15%, 750=0.98%, 1000=2.14% | |
lat (msec) : 2=3.42%, 4=2.12%, 10=1.14%, 20=0.74%, 50=3.59% | |
lat (msec) : 100=2.99%, 250=1.23%, 500=0.95%, 750=0.37%, 1000=0.05% | |
cpu : usr=0.12%, sys=0.38%, ctx=16094, majf=0, minf=102 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=40443,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:55 2019 | |
read: IOPS=41, BW=2654KiB/s (2718kB/s)(311MiB/120055msec) | |
slat (nsec): min=0, max=238280k, avg=92609.11, stdev=3549619.81 | |
clat (nsec): min=0, max=774600k, avg=8859266.13, stdev=41684463.38 | |
lat (nsec): min=0, max=774650k, avg=8952847.07, stdev=41829723.60 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 732], 10.00th=[ 764], | |
| 20.00th=[ 844], 30.00th=[ 924], 40.00th=[ 1048], | |
| 50.00th=[ 1192], 60.00th=[ 1320], 70.00th=[ 1608], | |
| 80.00th=[ 370688], 90.00th=[ 9895936], 95.00th=[ 52690944], | |
| 99.00th=[191889408], 99.50th=[312475648], 99.90th=[476053504], | |
| 99.95th=[658505728], 99.99th=[775946240] | |
bw ( KiB/s): min= 0, max= 3376, per=10.97%, avg=2655.00, stdev=1019.65, samples=2 | |
iops : min= 0, max= 52, avg=41.00, stdev=15.56, samples=2 | |
write: IOPS=42, BW=2735KiB/s (2801kB/s)(321MiB/120055msec) | |
slat (nsec): min=0, max=634946k, avg=345574.69, stdev=9626503.84 | |
clat (nsec): min=0, max=1219.3M, avg=84920091.81, stdev=152543276.10 | |
lat (nsec): min=0, max=1219.3M, avg=85266747.75, stdev=152708991.55 | |
clat percentiles (usec): | |
| 1.00th=[ 1012], 5.00th=[ 1237], 10.00th=[ 1434], | |
| 20.00th=[ 1942], 30.00th=[ 2933], 40.00th=[ 4817], | |
| 50.00th=[ 10290], 60.00th=[ 45876], 70.00th=[ 84411], | |
| 80.00th=[ 115868], 90.00th=[ 254804], 95.00th=[ 467665], | |
| 99.00th=[ 725615], 99.50th=[ 801113], 99.90th=[ 952108], | |
| 99.95th=[ 985662], 99.99th=[1216349] | |
bw ( KiB/s): min= 0, max= 3446, per=43.63%, avg=2734.00, stdev=1006.92, samples=2 | |
iops : min= 0, max= 53, avg=42.00, stdev=15.56, samples=2 | |
lat (nsec) : 750=4.32%, 1000=13.77% | |
lat (usec) : 2=17.31%, 4=0.07%, 10=0.30%, 20=0.92%, 50=0.85% | |
lat (usec) : 100=0.87%, 250=0.72%, 500=0.62%, 750=0.53%, 1000=1.40% | |
lat (msec) : 2=11.97%, 4=8.73%, 10=7.17%, 20=3.20%, 50=4.70% | |
lat (msec) : 100=9.37%, 250=7.66%, 500=3.41%, 750=1.75%, 1000=0.36% | |
cpu : usr=0.07%, sys=0.25%, ctx=11962, majf=0, minf=31 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=4980,5128,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=23.6MiB/s (24.8MB/s), 2654KiB/s-21.0MiB/s (2718kB/s-22.1MB/s), io=2837MiB (2975MB), run=120015-120055msec | |
WRITE: bw=6267KiB/s (6417kB/s), 2735KiB/s-3533KiB/s (2801kB/s-3617kB/s), io=735MiB (771MB), run=120055-120097msec | |
Disk stats (read/write): | |
rbd131: ios=9638/11955, merge=0/1, ticks=544140/818314, in_queue=1351514, util=15.67% | |
<example-benchmark-fio-server-731-benchmark-2p24w> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:55 2019 | |
write: IOPS=46, BW=2954KiB/s (3025kB/s)(346MiB/120061msec) | |
slat (nsec): min=0, max=750892k, avg=445086.93, stdev=11721042.12 | |
clat (nsec): min=0, max=1628.2M, avg=86222905.58, stdev=150130186.58 | |
lat (nsec): min=0, max=1628.3M, avg=86669252.57, stdev=150470281.13 | |
clat percentiles (usec): | |
| 1.00th=[ 963], 5.00th=[ 1221], 10.00th=[ 1500], | |
| 20.00th=[ 2147], 30.00th=[ 3228], 40.00th=[ 5604], | |
| 50.00th=[ 13173], 60.00th=[ 55837], 70.00th=[ 93848], | |
| 80.00th=[ 123208], 90.00th=[ 240124], 95.00th=[ 421528], | |
| 99.00th=[ 708838], 99.50th=[ 801113], 99.90th=[1098908], | |
| 99.95th=[1115685], 99.99th=[1635779] | |
bw ( KiB/s): min= 0, max= 4555, per=52.95%, avg=2953.00, stdev=2265.57, samples=2 | |
iops : min= 0, max= 71, avg=46.00, stdev=35.36, samples=2 | |
lat (usec) : 100=0.02%, 250=0.04%, 500=0.04%, 750=0.07%, 1000=1.39% | |
lat (msec) : 2=16.67%, 4=16.38%, 10=13.04%, 20=4.75%, 50=6.41% | |
lat (msec) : 100=13.20%, 250=18.62%, 500=5.99%, 750=2.67%, 1000=0.58% | |
cpu : usr=0.05%, sys=0.20%, ctx=10208, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,5538,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:55 2019 | |
read: IOPS=529, BW=33.0MiB/s (34.7MB/s)(3966MiB/120019msec) | |
slat (nsec): min=0, max=482645k, avg=34306.21, stdev=2614937.41 | |
clat (nsec): min=0, max=1336.0M, avg=7539479.66, stdev=48759461.40 | |
lat (nsec): min=0, max=1336.0M, avg=7574591.66, stdev=48839095.66 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 732], 40.00th=[ 740], | |
| 50.00th=[ 804], 60.00th=[ 868], 70.00th=[ 964], | |
| 80.00th=[ 8640], 90.00th=[ 1089536], 95.00th=[ 32636928], | |
| 99.00th=[ 217055232], 99.50th=[ 404750336], 99.90th=[ 633339904], | |
| 99.95th=[ 742391808], 99.99th=[1149239296] | |
bw ( KiB/s): min= 0, max=46913, per=93.04%, avg=33844.00, stdev=18482.36, samples=2 | |
iops : min= 0, max= 735, avg=529.50, stdev=290.62, samples=2 | |
lat (nsec) : 750=44.39%, 1000=26.82% | |
lat (usec) : 2=7.58%, 4=0.06%, 10=4.95%, 20=2.19%, 50=0.69% | |
lat (usec) : 100=0.15%, 250=0.06%, 500=0.17%, 750=0.85%, 1000=1.70% | |
lat (msec) : 2=2.34%, 4=1.32%, 10=0.59%, 20=0.39%, 50=2.28% | |
lat (msec) : 100=1.84%, 250=0.73%, 500=0.67%, 750=0.20%, 1000=0.03% | |
cpu : usr=0.16%, sys=0.53%, ctx=16677, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=63607,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:55 2019 | |
read: IOPS=39, BW=2534KiB/s (2595kB/s)(297MiB/120003msec) | |
slat (nsec): min=0, max=2602.2k, avg=26859.27, stdev=68750.48 | |
clat (nsec): min=0, max=750910k, avg=6636280.46, stdev=36858791.12 | |
lat (nsec): min=0, max=750930k, avg=6664125.19, stdev=36867160.13 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 732], 10.00th=[ 748], | |
| 20.00th=[ 804], 30.00th=[ 884], 40.00th=[ 980], | |
| 50.00th=[ 1128], 60.00th=[ 1272], 70.00th=[ 1592], | |
| 80.00th=[ 84480], 90.00th=[ 1613824], 95.00th=[ 38535168], | |
| 99.00th=[160432128], 99.50th=[295698432], 99.90th=[480247808], | |
| 99.95th=[488636416], 99.99th=[750780416] | |
bw ( KiB/s): min= 0, max= 3603, per=6.97%, avg=2534.00, stdev=1511.79, samples=2 | |
iops : min= 0, max= 56, avg=39.00, stdev=24.04, samples=2 | |
write: IOPS=40, BW=2625KiB/s (2688kB/s)(308MiB/120003msec) | |
slat (nsec): min=0, max=750900k, avg=473560.27, stdev=12692496.43 | |
clat (nsec): min=0, max=1632.7M, avg=90831011.95, stdev=157168249.28 | |
lat (nsec): min=0, max=1632.8M, avg=91305477.41, stdev=157556825.13 | |
clat percentiles (usec): | |
| 1.00th=[ 1004], 5.00th=[ 1237], 10.00th=[ 1467], | |
| 20.00th=[ 2147], 30.00th=[ 3326], 40.00th=[ 5866], | |
| 50.00th=[ 15795], 60.00th=[ 62129], 70.00th=[ 96994], | |
| 80.00th=[ 127402], 90.00th=[ 256902], 95.00th=[ 442500], | |
| 99.00th=[ 742392], 99.50th=[ 868221], 99.90th=[1233126], | |
| 99.95th=[1283458], 99.99th=[1635779] | |
bw ( KiB/s): min= 0, max= 3730, per=47.02%, avg=2622.50, stdev=1566.24, samples=2 | |
iops : min= 0, max= 58, avg=40.50, stdev=24.75, samples=2 | |
lat (nsec) : 750=5.76%, 1000=14.44% | |
lat (usec) : 2=15.05%, 4=0.09%, 10=0.52%, 20=1.30%, 50=1.23% | |
lat (usec) : 100=1.25%, 250=1.03%, 500=1.04%, 750=0.69%, 1000=1.35% | |
lat (msec) : 2=10.10%, 4=8.67%, 10=6.74%, 20=2.49%, 50=4.65% | |
lat (msec) : 100=8.20%, 250=9.94%, 500=3.54%, 750=1.44%, 1000=0.35% | |
cpu : usr=0.06%, sys=0.23%, ctx=11015, majf=0, minf=31 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=4755,4919,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=35.5MiB/s (37.2MB/s), 2534KiB/s-33.0MiB/s (2595kB/s-34.7MB/s), io=4263MiB (4470MB), run=120003-120019msec | |
WRITE: bw=5577KiB/s (5711kB/s), 2625KiB/s-2954KiB/s (2688kB/s-3025kB/s), io=654MiB (686MB), run=120003-120061msec | |
Disk stats (read/write): | |
rbd178: ios=9145/10566, merge=0/3, ticks=516068/791944, in_queue=1297922, util=13.11% | |
Jobs: 51 (f=51): [_(2),m(1)][99.8%][r=1375M,w=1387M][r=21.5k,w=21.7k IOPS][eta 00m:54s] Jobs: 51 (f=51): [_(2),m(1)][99.8%][r=1410M,w=1427M][r=22.0k,w=22.3k IOPS][eta 00m:53s] <example-benchmark-fio-server-794-benchmark-xkzrq> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:57 2019 | |
write: IOPS=39, BW=2511KiB/s (2572kB/s)(294MiB/120011msec) | |
slat (nsec): min=0, max=830857k, avg=575567.16, stdev=16158635.08 | |
clat (nsec): min=0, max=1469.7M, avg=101665439.36, stdev=163496335.02 | |
lat (nsec): min=0, max=1469.8M, avg=102242584.08, stdev=164095874.59 | |
clat percentiles (usec): | |
| 1.00th=[ 1020], 5.00th=[ 1270], 10.00th=[ 1598], | |
| 20.00th=[ 2245], 30.00th=[ 3720], 40.00th=[ 9503], | |
| 50.00th=[ 42730], 60.00th=[ 72877], 70.00th=[ 102237], | |
| 80.00th=[ 143655], 90.00th=[ 291505], 95.00th=[ 438305], | |
| 99.00th=[ 801113], 99.50th=[ 918553], 99.90th=[1434452], | |
| 99.95th=[1468007], 99.99th=[1468007] | |
bw ( KiB/s): min= 0, max= 3129, per=48.79%, avg=2509.00, stdev=876.81, samples=2 | |
iops : min= 0, max= 48, avg=38.50, stdev=13.44, samples=2 | |
lat (usec) : 2=0.02%, 4=0.02%, 500=0.02%, 750=0.04%, 1000=0.70% | |
lat (msec) : 2=15.68%, 4=14.53%, 10=9.31%, 20=4.12%, 50=7.39% | |
lat (msec) : 100=17.68%, 250=18.25%, 500=8.35%, 750=2.57%, 1000=1.13% | |
cpu : usr=0.06%, sys=0.17%, ctx=8853, majf=0, minf=26 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,4706,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:57 2019 | |
read: IOPS=321, BW=20.1MiB/s (21.1MB/s)(2412MiB/120011msec) | |
slat (nsec): min=0, max=593474k, avg=62460.39, stdev=4735684.38 | |
clat (nsec): min=0, max=1023.2M, avg=12403573.49, stdev=60386500.08 | |
lat (nsec): min=0, max=1023.3M, avg=12466969.14, stdev=60576303.42 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 732], 40.00th=[ 740], | |
| 50.00th=[ 812], 60.00th=[ 876], 70.00th=[ 996], | |
| 80.00th=[ 1528], 90.00th=[ 11075584], 95.00th=[ 57409536], | |
| 99.00th=[346030080], 99.50th=[467664896], 99.90th=[700448768], | |
| 99.95th=[775946240], 99.99th=[952107008] | |
bw ( KiB/s): min= 0, max=27268, per=89.00%, avg=20583.50, stdev=9453.31, samples=2 | |
iops : min= 0, max= 426, avg=321.50, stdev=147.79, samples=2 | |
lat (nsec) : 750=42.48%, 1000=27.64% | |
lat (usec) : 2=10.36%, 4=0.04%, 10=0.65%, 20=0.38%, 50=0.32% | |
lat (usec) : 100=0.13%, 250=0.04%, 500=0.20%, 750=0.85%, 1000=1.83% | |
lat (msec) : 2=2.50%, 4=1.57%, 10=0.91%, 20=0.91%, 50=3.52% | |
lat (msec) : 100=2.58%, 250=1.53%, 500=1.16%, 750=0.33%, 1000=0.06% | |
cpu : usr=0.11%, sys=0.36%, ctx=13842, majf=0, minf=109 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=38631,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:57 2019 | |
read: IOPS=39, BW=2544KiB/s (2605kB/s)(298MiB/120003msec) | |
slat (nsec): min=0, max=578056k, avg=151252.96, stdev=8368754.74 | |
clat (nsec): min=0, max=1230.4M, avg=8238059.24, stdev=45356635.31 | |
lat (nsec): min=0, max=1230.4M, avg=8390234.99, stdev=46106318.45 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 732], 10.00th=[ 740], | |
| 20.00th=[ 780], 30.00th=[ 876], 40.00th=[ 988], | |
| 50.00th=[ 1144], 60.00th=[ 1272], 70.00th=[ 1560], | |
| 80.00th=[ 232448], 90.00th=[ 3686400], 95.00th=[ 41156608], | |
| 99.00th=[ 221249536], 99.50th=[ 354418688], 99.90th=[ 480247808], | |
| 99.95th=[ 801112064], 99.99th=[1233125376] | |
bw ( KiB/s): min= 0, max= 3363, per=11.00%, avg=2543.00, stdev=1159.66, samples=2 | |
iops : min= 0, max= 52, avg=39.00, stdev=18.38, samples=2 | |
write: IOPS=41, BW=2632KiB/s (2695kB/s)(308MiB/120003msec) | |
slat (nsec): min=0, max=451778k, avg=272568.46, stdev=6436634.86 | |
clat (nsec): min=0, max=1469.7M, avg=88949844.53, stdev=152195506.47 | |
lat (nsec): min=0, max=1469.8M, avg=89223694.54, stdev=152274142.94 | |
clat percentiles (usec): | |
| 1.00th=[ 1045], 5.00th=[ 1401], 10.00th=[ 1729], | |
| 20.00th=[ 2540], 30.00th=[ 4015], 40.00th=[ 7308], | |
| 50.00th=[ 19792], 60.00th=[ 54789], 70.00th=[ 87557], | |
| 80.00th=[ 124257], 90.00th=[ 265290], 95.00th=[ 421528], | |
| 99.00th=[ 742392], 99.50th=[ 876610], 99.90th=[ 952108], | |
| 99.95th=[1434452], 99.99th=[1468007] | |
bw ( KiB/s): min= 0, max= 3460, per=51.16%, avg=2630.50, stdev=1173.09, samples=2 | |
iops : min= 0, max= 54, avg=41.00, stdev=18.38, samples=2 | |
lat (nsec) : 750=6.93%, 1000=13.11% | |
lat (usec) : 2=15.45%, 4=0.07%, 10=0.41%, 20=0.79%, 50=0.92% | |
lat (usec) : 100=0.97%, 250=0.75%, 500=0.82%, 750=0.54%, 1000=1.22% | |
lat (msec) : 2=8.42%, 4=9.14%, 10=7.65%, 20=3.96%, 50=5.69% | |
lat (msec) : 100=9.08%, 250=8.22%, 500=4.08%, 750=1.28%, 1000=0.48% | |
cpu : usr=0.07%, sys=0.23%, ctx=11104, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=4771,4932,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=22.6MiB/s (23.7MB/s), 2544KiB/s-20.1MiB/s (2605kB/s-21.1MB/s), io=2711MiB (2842MB), run=120003-120011msec | |
WRITE: bw=5143KiB/s (5266kB/s), 2511KiB/s-2632KiB/s (2572kB/s-2695kB/s), io=603MiB (632MB), run=120003-120011msec | |
Disk stats (read/write): | |
rbd206: ios=9196/10145, merge=0/2, ticks=549346/854017, in_queue=1393497, util=13.88% | |
<example-benchmark-fio-server-694-benchmark-wvfzs> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:57 2019 | |
write: IOPS=40, BW=2571KiB/s (2633kB/s)(307MiB/122324msec) | |
slat (nsec): min=0, max=25427k, avg=195885.07, stdev=562273.47 | |
clat (nsec): min=0, max=8959.5M, avg=99109794.52, stdev=322776851.58 | |
lat (nsec): min=0, max=8959.6M, avg=99307113.25, stdev=322783856.59 | |
clat percentiles (usec): | |
| 1.00th=[ 848], 5.00th=[ 1827], 10.00th=[ 2802], | |
| 20.00th=[ 4293], 30.00th=[ 6718], 40.00th=[ 10421], | |
| 50.00th=[ 15533], 60.00th=[ 27395], 70.00th=[ 54264], | |
| 80.00th=[ 130548], 90.00th=[ 287310], 95.00th=[ 429917], | |
| 99.00th=[ 717226], 99.50th=[1317012], 99.90th=[6878659], | |
| 99.95th=[8556381], 99.99th=[8925479] | |
bw ( KiB/s): min= 0, max= 3233, per=47.42%, avg=2618.50, stdev=869.03, samples=2 | |
iops : min= 0, max= 50, avg=40.50, stdev=13.44, samples=2 | |
lat (usec) : 2=0.02%, 4=0.02%, 100=0.02%, 250=0.02%, 500=0.06% | |
lat (usec) : 750=0.37%, 1000=1.06% | |
lat (msec) : 2=4.28%, 4=11.89%, 10=21.32%, 20=15.64%, 50=14.05% | |
lat (msec) : 100=8.90%, 250=10.59%, 500=8.33%, 750=2.61%, 1000=0.20% | |
cpu : usr=0.03%, sys=0.15%, ctx=7781, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,4911,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:57 2019 | |
read: IOPS=68, BW=4371KiB/s (4476kB/s)(513MiB/120237msec) | |
slat (nsec): min=0, max=9176.8k, avg=52739.78, stdev=188486.51 | |
clat (nsec): min=0, max=1825.5M, avg=58615474.01, stdev=197991591.34 | |
lat (nsec): min=0, max=1825.5M, avg=58669431.87, stdev=198008807.92 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 732], 10.00th=[ 772], | |
| 20.00th=[ 820], 30.00th=[ 884], 40.00th=[ 1032], | |
| 50.00th=[ 1256], 60.00th=[ 1640], 70.00th=[ 1417216], | |
| 80.00th=[ 3424256], 90.00th=[ 49020928], 95.00th=[ 599785472], | |
| 99.00th=[ 952107008], 99.50th=[1069547520], 99.90th=[1350565888], | |
| 99.95th=[1518338048], 99.99th=[1820327936] | |
bw ( KiB/s): min= 0, max= 4448, per=60.15%, avg=4376.50, stdev=101.12, samples=2 | |
iops : min= 0, max= 69, avg=68.00, stdev= 1.41, samples=2 | |
lat (nsec) : 750=6.54%, 1000=32.33% | |
lat (usec) : 2=21.82%, 4=0.22%, 10=0.11%, 20=0.52%, 50=0.17% | |
lat (usec) : 100=0.19%, 250=0.10%, 500=0.06%, 750=0.82%, 1000=3.16% | |
lat (msec) : 2=7.88%, 4=7.27%, 10=4.92%, 20=2.01%, 50=1.99% | |
lat (msec) : 100=0.97%, 250=0.83%, 500=2.22%, 750=3.12%, 1000=2.05% | |
cpu : usr=0.04%, sys=0.11%, ctx=6246, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=8209,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=14: Wed May 8 20:34:57 2019 | |
read: IOPS=45, BW=2911KiB/s (2981kB/s)(341MiB/120007msec) | |
slat (nsec): min=0, max=3786.0k, avg=42845.43, stdev=115101.89 | |
clat (nsec): min=0, max=809090k, avg=19029910.56, stdev=72180861.64 | |
lat (nsec): min=0, max=809121k, avg=19073885.97, stdev=72190890.86 | |
clat percentiles (nsec): | |
| 1.00th=[ 724], 5.00th=[ 740], 10.00th=[ 748], | |
| 20.00th=[ 868], 30.00th=[ 1004], 40.00th=[ 1144], | |
| 50.00th=[ 1304], 60.00th=[ 52480], 70.00th=[ 1105920], | |
| 80.00th=[ 3227648], 90.00th=[ 43778048], 95.00th=[ 87556096], | |
| 99.00th=[446693376], 99.50th=[509607936], 99.90th=[734003200], | |
| 99.95th=[784334848], 99.99th=[809500672] | |
bw ( KiB/s): min= 0, max= 4526, per=39.99%, avg=2909.50, stdev=2286.08, samples=2 | |
iops : min= 0, max= 70, avg=45.00, stdev=35.36, samples=2 | |
write: IOPS=46, BW=3008KiB/s (3081kB/s)(353MiB/120007msec) | |
slat (nsec): min=0, max=5596.6k, avg=184642.48, stdev=236894.22 | |
clat (nsec): min=0, max=852576k, avg=66695651.86, stdev=130745032.75 | |
lat (nsec): min=0, max=852605k, avg=66881887.03, stdev=130698526.14 | |
clat percentiles (usec): | |
| 1.00th=[ 1020], 5.00th=[ 1221], 10.00th=[ 1369], 20.00th=[ 1696], | |
| 30.00th=[ 2212], 40.00th=[ 3163], 50.00th=[ 5145], 60.00th=[ 10945], | |
| 70.00th=[ 49546], 80.00th=[102237], 90.00th=[200279], 95.00th=[383779], | |
| 99.00th=[616563], 99.50th=[683672], 99.90th=[750781], 99.95th=[826278], | |
| 99.99th=[851444] | |
bw ( KiB/s): min= 0, max= 4664, per=54.45%, avg=3007.00, stdev=2343.35, samples=2 | |
iops : min= 0, max= 72, avg=46.50, stdev=36.06, samples=2 | |
lat (nsec) : 750=4.73%, 1000=9.89% | |
lat (usec) : 2=14.22%, 4=0.05%, 10=0.07%, 20=0.32%, 50=0.23% | |
lat (usec) : 100=0.32%, 250=0.35%, 500=0.22%, 750=1.08%, 1000=2.54% | |
lat (msec) : 2=17.03%, 4=12.00%, 10=8.42%, 20=3.30%, 50=5.66% | |
lat (msec) : 100=6.99%, 250=7.36%, 500=3.60%, 750=1.53%, 1000=0.12% | |
cpu : usr=0.09%, sys=0.25%, ctx=14803, majf=0, minf=37 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=5457,5640,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=7277KiB/s (7451kB/s), 2911KiB/s-4371KiB/s (2981kB/s-4476kB/s), io=854MiB (896MB), run=120007-120237msec | |
WRITE: bw=5522KiB/s (5655kB/s), 2571KiB/s-3008KiB/s (2633kB/s-3081kB/s), io=660MiB (692MB), run=120007-122324msec | |
Disk stats (read/write): | |
rbd204: ios=5634/12337, merge=0/3, ticks=642225/878157, in_queue=1511367, util=4.27% | |
<example-benchmark-fio-server-612-benchmark-2dg6w> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:58 2019 | |
write: IOPS=27, BW=1758KiB/s (1800kB/s)(207MiB/120534msec) | |
slat (nsec): min=0, max=4257.1M, avg=2010805.23, stdev=76515967.14 | |
clat (nsec): min=0, max=6824.8M, avg=143809839.17, stdev=306619330.65 | |
lat (nsec): min=0, max=6833.9M, avg=145820705.46, stdev=319679458.15 | |
clat percentiles (usec): | |
| 1.00th=[ 1221], 5.00th=[ 1663], 10.00th=[ 2114], | |
| 20.00th=[ 3032], 30.00th=[ 4178], 40.00th=[ 6063], | |
| 50.00th=[ 9634], 60.00th=[ 16909], 70.00th=[ 88605], | |
| 80.00th=[ 333448], 90.00th=[ 492831], 95.00th=[ 616563], | |
| 99.00th=[ 809501], 99.50th=[1035994], 99.90th=[3506439], | |
| 99.95th=[6811550], 99.99th=[6811550] | |
bw ( KiB/s): min= 0, max= 2243, per=33.59%, avg=1763.00, stdev=678.82, samples=2 | |
iops : min= 0, max= 35, avg=27.50, stdev=10.61, samples=2 | |
lat (usec) : 500=0.03%, 750=0.03%, 1000=0.18% | |
lat (msec) : 2=8.41%, 4=19.78%, 10=22.32%, 20=11.82%, 50=6.23% | |
lat (msec) : 100=1.45%, 250=5.47%, 500=14.45%, 750=8.50%, 1000=0.88% | |
cpu : usr=0.02%, sys=0.12%, ctx=5571, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,3307,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:58 2019 | |
read: IOPS=136, BW=8764KiB/s (8975kB/s)(1032MiB/120562msec) | |
slat (nsec): min=0, max=12461k, avg=36945.42, stdev=180100.44 | |
clat (nsec): min=0, max=1972.8M, avg=29204481.49, stdev=143415318.41 | |
lat (nsec): min=0, max=1972.8M, avg=29242530.82, stdev=143430361.26 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 732], 10.00th=[ 756], | |
| 20.00th=[ 788], 30.00th=[ 820], 40.00th=[ 868], | |
| 50.00th=[ 900], 60.00th=[ 1032], 70.00th=[ 1320], | |
| 80.00th=[ 995328], 90.00th=[ 4685824], 95.00th=[ 51642368], | |
| 99.00th=[ 826277888], 99.50th=[ 952107008], 99.90th=[1333788672], | |
| 99.95th=[1535115264], 99.99th=[1837105152] | |
bw ( KiB/s): min= 0, max= 9481, per=72.35%, avg=8802.50, stdev=959.54, samples=2 | |
iops : min= 0, max= 148, avg=137.00, stdev=15.56, samples=2 | |
lat (nsec) : 750=9.74%, 1000=49.14% | |
lat (usec) : 2=17.42%, 4=0.25%, 10=0.19%, 20=0.59%, 50=0.16% | |
lat (usec) : 100=0.12%, 250=0.12%, 500=0.13%, 750=0.43%, 1000=1.79% | |
lat (msec) : 2=4.74%, 4=4.43%, 10=3.01%, 20=1.31%, 50=1.40% | |
lat (msec) : 100=0.57%, 250=0.58%, 500=1.05%, 750=1.38%, 1000=1.13% | |
cpu : usr=0.06%, sys=0.18%, ctx=7446, majf=0, minf=29 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=16507,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:58 2019 | |
read: IOPS=53, BW=3419KiB/s (3501kB/s)(401MiB/120002msec) | |
slat (nsec): min=0, max=1954.4k, avg=31149.84, stdev=77107.69 | |
clat (nsec): min=0, max=734117k, avg=11037698.80, stdev=54963807.10 | |
lat (nsec): min=0, max=734145k, avg=11069791.59, stdev=54971279.93 | |
clat percentiles (nsec): | |
| 1.00th=[ 724], 5.00th=[ 732], 10.00th=[ 740], | |
| 20.00th=[ 812], 30.00th=[ 908], 40.00th=[ 1032], | |
| 50.00th=[ 1176], 60.00th=[ 1304], 70.00th=[ 1816], | |
| 80.00th=[ 856064], 90.00th=[ 5275648], 95.00th=[ 51118080], | |
| 99.00th=[320864256], 99.50th=[455081984], 99.90th=[616562688], | |
| 99.95th=[650117120], 99.99th=[734003200] | |
bw ( KiB/s): min= 0, max= 4692, per=28.10%, avg=3418.50, stdev=1801.00, samples=2 | |
iops : min= 0, max= 73, avg=53.00, stdev=28.28, samples=2 | |
write: IOPS=54, BW=3508KiB/s (3592kB/s)(411MiB/120002msec) | |
slat (nsec): min=0, max=10682k, avg=192837.44, stdev=258303.43 | |
clat (nsec): min=0, max=1267.3M, avg=62261381.68, stdev=127104471.59 | |
lat (nsec): min=0, max=1267.3M, avg=62455492.49, stdev=127055529.35 | |
clat percentiles (usec): | |
| 1.00th=[ 1012], 5.00th=[ 1221], 10.00th=[ 1401], | |
| 20.00th=[ 1795], 30.00th=[ 2376], 40.00th=[ 3458], | |
| 50.00th=[ 5997], 60.00th=[ 14484], 70.00th=[ 32900], | |
| 80.00th=[ 99091], 90.00th=[ 187696], 95.00th=[ 325059], | |
| 99.00th=[ 675283], 99.50th=[ 750781], 99.90th=[ 901776], | |
| 99.95th=[ 977273], 99.99th=[1266680] | |
bw ( KiB/s): min= 0, max= 4778, per=66.77%, avg=3505.00, stdev=1800.29, samples=2 | |
iops : min= 0, max= 74, avg=54.00, stdev=28.28, samples=2 | |
lat (nsec) : 750=5.97%, 1000=12.84% | |
lat (usec) : 2=15.87%, 4=0.08%, 10=0.23%, 20=0.78%, 50=0.68% | |
lat (usec) : 100=0.62%, 250=0.69%, 500=0.44%, 750=0.65%, 1000=1.93% | |
lat (msec) : 2=14.01%, 4=11.19%, 10=7.22%, 20=4.84%, 50=5.47% | |
lat (msec) : 100=5.08%, 250=7.06%, 500=3.13%, 750=0.98%, 1000=0.25% | |
cpu : usr=0.09%, sys=0.31%, ctx=15021, majf=0, minf=31 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=6410,6574,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=11.9MiB/s (12.5MB/s), 3419KiB/s-8764KiB/s (3501kB/s-8975kB/s), io=1433MiB (1502MB), run=120002-120562msec | |
WRITE: bw=5250KiB/s (5376kB/s), 1758KiB/s-3508KiB/s (1800kB/s-3592kB/s), io=618MiB (648MB), run=120002-120534msec | |
Disk stats (read/write): | |
rbd279: ios=5460/10356, merge=0/3, ticks=596117/878665, in_queue=1466829, util=4.57% | |
Jobs: 48 (f=45): [_(2),m(1)][99.8%][r=1436M,w=1434M][r=22.4k,w=22.4k IOPS][eta 00m:52s] <example-benchmark-fio-server-898-benchmark-2q9b7> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:58 2019 | |
write: IOPS=13, BW=835KiB/s (855kB/s)(98.1MiB/120351msec) | |
slat (nsec): min=0, max=2022.3M, avg=2728134.72, stdev=58249237.85 | |
clat (nsec): min=0, max=3275.6M, avg=304687648.20, stdev=456863340.63 | |
lat (nsec): min=0, max=3275.7M, avg=307412545.76, stdev=460022219.28 | |
clat percentiles (usec): | |
| 1.00th=[ 1303], 5.00th=[ 2024], 10.00th=[ 2704], | |
| 20.00th=[ 4490], 30.00th=[ 6915], 40.00th=[ 11338], | |
| 50.00th=[ 18744], 60.00th=[ 50070], 70.00th=[ 392168], | |
| 80.00th=[ 633340], 90.00th=[1010828], 95.00th=[1317012], | |
| 99.00th=[1635779], 99.50th=[1920992], 99.90th=[2231370], | |
| 99.95th=[3271558], 99.99th=[3271558] | |
bw ( KiB/s): min= 0, max= 1076, per=21.35%, avg=833.00, stdev=343.65, samples=2 | |
iops : min= 0, max= 16, avg=12.50, stdev= 4.95, samples=2 | |
lat (usec) : 1000=0.19% | |
lat (msec) : 2=4.53%, 4=12.95%, 10=19.85%, 20=13.85%, 50=8.74% | |
lat (msec) : 100=1.91%, 250=0.89%, 500=11.81%, 750=8.42%, 1000=6.76% | |
cpu : usr=0.02%, sys=0.07%, ctx=2848, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,1567,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:58 2019 | |
read: IOPS=307, BW=19.2MiB/s (20.1MB/s)(2315MiB/120566msec) | |
slat (nsec): min=0, max=1464.6M, avg=73420.58, stdev=8011473.67 | |
clat (nsec): min=0, max=2370.1M, avg=12938782.83, stdev=99222958.48 | |
lat (nsec): min=0, max=2370.1M, avg=13013350.53, stdev=99551960.23 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 772], 30.00th=[ 788], 40.00th=[ 804], | |
| 50.00th=[ 836], 60.00th=[ 868], 70.00th=[ 900], | |
| 80.00th=[ 1020], 90.00th=[ 11840], 95.00th=[ 2703360], | |
| 99.00th=[ 624951296], 99.50th=[ 792723456], 99.90th=[1199570944], | |
| 99.95th=[1384120320], 99.99th=[1937768448] | |
bw ( KiB/s): min= 0, max=26548, per=87.30%, avg=19751.50, stdev=9611.70, samples=2 | |
iops : min= 0, max= 415, avg=308.50, stdev=150.61, samples=2 | |
lat (nsec) : 750=12.45%, 1000=67.02% | |
lat (usec) : 2=9.06%, 4=0.23%, 10=0.46%, 20=2.23%, 50=0.58% | |
lat (usec) : 100=0.18%, 250=0.09%, 500=0.13%, 750=0.15%, 1000=0.44% | |
lat (msec) : 2=1.38%, 4=1.28%, 10=1.15%, 20=0.51%, 50=0.46% | |
lat (msec) : 100=0.24%, 250=0.24%, 500=0.46%, 750=0.65%, 1000=0.42% | |
cpu : usr=0.08%, sys=0.34%, ctx=5906, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=37091,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:58 2019 | |
read: IOPS=46, BW=2977KiB/s (3048kB/s)(349MiB/120021msec) | |
slat (nsec): min=0, max=5457.3k, avg=23255.68, stdev=98205.93 | |
clat (nsec): min=0, max=786199k, avg=7736512.34, stdev=47943367.55 | |
lat (nsec): min=0, max=786220k, avg=7760701.96, stdev=47953450.62 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 724], 10.00th=[ 724], | |
| 20.00th=[ 748], 30.00th=[ 828], 40.00th=[ 916], | |
| 50.00th=[ 1048], 60.00th=[ 1192], 70.00th=[ 1368], | |
| 80.00th=[ 19328], 90.00th=[ 978944], 95.00th=[ 24772608], | |
| 99.00th=[242221056], 99.50th=[404750336], 99.90th=[608174080], | |
| 99.95th=[700448768], 99.99th=[784334848] | |
bw ( KiB/s): min= 0, max= 4710, per=13.16%, avg=2976.50, stdev=2451.54, samples=2 | |
iops : min= 0, max= 73, avg=46.00, stdev=38.18, samples=2 | |
write: IOPS=48, BW=3076KiB/s (3150kB/s)(361MiB/120021msec) | |
slat (nsec): min=0, max=6098.2k, avg=209590.76, stdev=291288.10 | |
clat (nsec): min=0, max=1088.8M, avg=75761080.52, stdev=143915707.64 | |
lat (nsec): min=0, max=1088.8M, avg=75971772.86, stdev=143854208.57 | |
clat percentiles (usec): | |
| 1.00th=[ 963], 5.00th=[ 1123], 10.00th=[ 1287], | |
| 20.00th=[ 1663], 30.00th=[ 2278], 40.00th=[ 3556], | |
| 50.00th=[ 6980], 60.00th=[ 21890], 70.00th=[ 68682], | |
| 80.00th=[ 112722], 90.00th=[ 231736], 95.00th=[ 400557], | |
| 99.00th=[ 708838], 99.50th=[ 817890], 99.90th=[ 968885], | |
| 99.95th=[ 994051], 99.99th=[1082131] | |
bw ( KiB/s): min= 0, max= 4848, per=78.78%, avg=3074.00, stdev=2508.81, samples=2 | |
iops : min= 0, max= 75, avg=47.50, stdev=38.89, samples=2 | |
lat (nsec) : 750=10.49%, 1000=12.60% | |
lat (usec) : 2=14.41%, 4=0.04%, 10=0.41%, 20=1.55%, 50=1.02% | |
lat (usec) : 100=0.98%, 250=1.01%, 500=0.77%, 750=0.50%, 1000=1.36% | |
lat (msec) : 2=13.40%, 4=8.68%, 10=6.46%, 20=3.22%, 50=4.30% | |
lat (msec) : 100=6.09%, 250=7.50%, 500=3.56%, 750=1.25%, 1000=0.42% | |
cpu : usr=0.06%, sys=0.27%, ctx=11681, majf=0, minf=34 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=5582,5765,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=22.1MiB/s (23.2MB/s), 2977KiB/s-19.2MiB/s (3048kB/s-20.1MB/s), io=2664MiB (2793MB), run=120021-120566msec | |
WRITE: bw=3902KiB/s (3996kB/s), 835KiB/s-3076KiB/s (855kB/s-3150kB/s), io=459MiB (481MB), run=120021-120351msec | |
Disk stats (read/write): | |
rbd321: ios=3932/7745, merge=0/1, ticks=559319/910120, in_queue=1463629, util=5.16% | |
<example-benchmark-fio-server-417-benchmark-gs9tf> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:58 2019 | |
write: IOPS=22, BW=1436KiB/s (1471kB/s)(171MiB/121813msec) | |
slat (nsec): min=0, max=318460k, avg=653825.73, stdev=10094001.52 | |
clat (nsec): min=0, max=4853.7M, avg=175745024.98, stdev=330156121.69 | |
lat (nsec): min=0, max=4853.0M, avg=176399966.50, stdev=330429601.02 | |
clat percentiles (usec): | |
| 1.00th=[ 1074], 5.00th=[ 1762], 10.00th=[ 2606], | |
| 20.00th=[ 4424], 30.00th=[ 7046], 40.00th=[ 10945], | |
| 50.00th=[ 17433], 60.00th=[ 46400], 70.00th=[ 223347], | |
| 80.00th=[ 367002], 90.00th=[ 557843], 95.00th=[ 666895], | |
| 99.00th=[ 943719], 99.50th=[1887437], 99.90th=[4798284], | |
| 99.95th=[4865393], 99.99th=[4865393] | |
bw ( KiB/s): min= 0, max= 1607, per=31.79%, avg=1455.50, stdev=214.25, samples=2 | |
iops : min= 0, max= 25, avg=22.50, stdev= 3.54, samples=2 | |
lat (usec) : 4=0.07%, 750=0.07%, 1000=0.66% | |
lat (msec) : 2=5.42%, 4=12.23%, 10=19.00%, 20=14.54%, 50=8.53% | |
lat (msec) : 100=4.43%, 250=7.14%, 500=15.49%, 750=9.85%, 1000=1.90% | |
cpu : usr=0.03%, sys=0.10%, ctx=4795, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,2731,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:58 2019 | |
read: IOPS=123, BW=7924KiB/s (8115kB/s)(934MiB/120724msec) | |
slat (nsec): min=0, max=10609k, avg=31616.62, stdev=166024.40 | |
clat (nsec): min=0, max=1785.3M, avg=32298334.44, stdev=152225712.97 | |
lat (nsec): min=0, max=1785.3M, avg=32330977.46, stdev=152246360.23 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 740], | |
| 20.00th=[ 780], 30.00th=[ 796], 40.00th=[ 844], | |
| 50.00th=[ 876], 60.00th=[ 908], 70.00th=[ 1112], | |
| 80.00th=[ 1496], 90.00th=[ 3096576], 95.00th=[ 61603840], | |
| 99.00th=[ 851443712], 99.50th=[ 977272832], 99.90th=[1266679808], | |
| 99.95th=[1468006400], 99.99th=[1769996288] | |
bw ( KiB/s): min= 0, max= 9050, per=72.45%, avg=7959.00, stdev=1542.91, samples=2 | |
iops : min= 0, max= 141, avg=124.00, stdev=24.04, samples=2 | |
lat (nsec) : 750=10.84%, 1000=55.88% | |
lat (usec) : 2=14.46%, 4=0.16%, 10=0.30%, 20=0.97%, 50=0.21% | |
lat (usec) : 100=0.13%, 250=0.14%, 500=0.11%, 750=0.27%, 1000=1.05% | |
lat (msec) : 2=3.48%, 4=3.04%, 10=2.30%, 20=0.89%, 50=0.66% | |
lat (msec) : 100=0.31%, 250=0.31%, 500=1.02%, 750=1.93%, 1000=1.09% | |
cpu : usr=0.04%, sys=0.16%, ctx=5094, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=14945,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:58 2019 | |
read: IOPS=48, BW=3080KiB/s (3154kB/s)(361MiB/120004msec) | |
slat (nsec): min=0, max=6708.7k, avg=29217.79, stdev=128662.97 | |
clat (nsec): min=0, max=1112.9M, avg=9348407.86, stdev=55207623.48 | |
lat (nsec): min=0, max=1112.0M, avg=9378582.70, stdev=55212255.77 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 764], 30.00th=[ 868], 40.00th=[ 964], | |
| 50.00th=[ 1112], 60.00th=[ 1240], 70.00th=[ 1528], | |
| 80.00th=[ 148480], 90.00th=[ 2146304], 95.00th=[ 33423360], | |
| 99.00th=[ 312475648], 99.50th=[ 467664896], 99.90th=[ 624951296], | |
| 99.95th=[ 742391808], 99.99th=[1115684864] | |
bw ( KiB/s): min= 0, max= 4738, per=28.03%, avg=3079.50, stdev=2345.47, samples=2 | |
iops : min= 0, max= 74, avg=48.00, stdev=36.77, samples=2 | |
write: IOPS=49, BW=3190KiB/s (3266kB/s)(374MiB/120004msec) | |
slat (nsec): min=0, max=7569.2k, avg=191882.63, stdev=320415.62 | |
clat (nsec): min=0, max=1050.9M, avg=71244375.78, stdev=134209200.19 | |
lat (nsec): min=0, max=1050.9M, avg=71437577.53, stdev=134155113.65 | |
clat percentiles (usec): | |
| 1.00th=[ 1037], 5.00th=[ 1287], 10.00th=[ 1516], | |
| 20.00th=[ 2073], 30.00th=[ 2900], 40.00th=[ 4359], | |
| 50.00th=[ 7308], 60.00th=[ 18744], 70.00th=[ 60031], | |
| 80.00th=[ 109577], 90.00th=[ 217056], 95.00th=[ 379585], | |
| 99.00th=[ 641729], 99.50th=[ 692061], 99.90th=[ 834667], | |
| 99.95th=[ 901776], 99.99th=[1052771] | |
bw ( KiB/s): min= 0, max= 4899, per=69.63%, avg=3187.50, stdev=2420.43, samples=2 | |
iops : min= 0, max= 76, avg=49.50, stdev=37.48, samples=2 | |
lat (nsec) : 750=9.10%, 1000=11.98% | |
lat (usec) : 2=14.25%, 4=0.09%, 10=0.37%, 20=1.14%, 50=0.95% | |
lat (usec) : 100=0.88%, 250=1.16%, 500=0.80%, 750=0.48%, 1000=1.41% | |
lat (msec) : 2=10.99%, 4=10.91%, 10=8.98%, 20=3.69%, 50=4.57% | |
lat (msec) : 100=5.85%, 250=7.58%, 500=3.06%, 750=1.60%, 1000=0.14% | |
cpu : usr=0.07%, sys=0.28%, ctx=12640, majf=0, minf=31 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=5775,5978,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=10.7MiB/s (11.2MB/s), 3080KiB/s-7924KiB/s (3154kB/s-8115kB/s), io=1295MiB (1358MB), run=120004-120724msec | |
WRITE: bw=4579KiB/s (4689kB/s), 1436KiB/s-3190KiB/s (1471kB/s-3266kB/s), io=545MiB (571MB), run=120004-121813msec | |
Disk stats (read/write): | |
rbd128: ios=3825/8987, merge=0/2, ticks=579612/909061, in_queue=1482232, util=3.77% | |
<example-benchmark-fio-server-702-benchmark-72kl9> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:58 2019 | |
write: IOPS=22, BW=1418KiB/s (1452kB/s)(168MiB/121297msec) | |
slat (nsec): min=0, max=427537k, avg=661838.85, stdev=12390156.81 | |
clat (nsec): min=0, max=2646.7M, avg=182451904.64, stdev=362745931.13 | |
lat (nsec): min=0, max=2646.8M, avg=183115311.85, stdev=362747429.03 | |
clat percentiles (usec): | |
| 1.00th=[ 1123], 5.00th=[ 1549], 10.00th=[ 1893], | |
| 20.00th=[ 2573], 30.00th=[ 3326], 40.00th=[ 4424], | |
| 50.00th=[ 6063], 60.00th=[ 9634], 70.00th=[ 23200], | |
| 80.00th=[ 396362], 90.00th=[ 725615], 95.00th=[ 968885], | |
| 99.00th=[1535116], 99.50th=[1551893], 99.90th=[2566915], | |
| 99.95th=[2634023], 99.99th=[2634023] | |
bw ( KiB/s): min= 0, max= 1689, per=33.81%, avg=1430.50, stdev=365.57, samples=2 | |
iops : min= 0, max= 26, avg=22.00, stdev= 5.66, samples=2 | |
lat (usec) : 750=0.26%, 1000=0.48% | |
lat (msec) : 2=10.95%, 4=25.33%, 10=23.39%, 20=8.68%, 50=3.87% | |
lat (msec) : 100=2.38%, 250=2.38%, 500=5.85%, 750=7.56%, 1000=4.17% | |
cpu : usr=0.03%, sys=0.10%, ctx=4842, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,2685,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:58 2019 | |
read: IOPS=202, BW=12.7MiB/s (13.3MB/s)(1525MiB/120525msec) | |
slat (nsec): min=0, max=67962k, avg=31963.04, stdev=493698.59 | |
clat (nsec): min=0, max=1958.1M, avg=19763723.06, stdev=119137048.25 | |
lat (nsec): min=0, max=1958.2M, avg=19796745.38, stdev=119154585.48 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 740], | |
| 20.00th=[ 780], 30.00th=[ 796], 40.00th=[ 828], | |
| 50.00th=[ 868], 60.00th=[ 900], 70.00th=[ 1004], | |
| 80.00th=[ 1400], 90.00th=[ 1777664], 95.00th=[ 8028160], | |
| 99.00th=[ 742391808], 99.50th=[ 893386752], 99.90th=[1249902592], | |
| 99.95th=[1300234240], 99.99th=[1686110208] | |
bw ( KiB/s): min= 0, max=15112, per=82.69%, avg=12982.00, stdev=3012.27, samples=2 | |
iops : min= 0, max= 236, avg=202.50, stdev=47.38, samples=2 | |
lat (nsec) : 750=11.73%, 1000=58.10% | |
lat (usec) : 2=12.66%, 4=0.15%, 10=0.42%, 20=1.68%, 50=0.48% | |
lat (usec) : 100=0.20%, 250=0.13%, 500=0.11%, 750=0.26%, 1000=1.27% | |
lat (msec) : 2=3.29%, 4=2.92%, 10=1.97%, 20=0.82%, 50=0.57% | |
lat (msec) : 100=0.27%, 250=0.28%, 500=0.62%, 750=1.13%, 1000=0.68% | |
cpu : usr=0.07%, sys=0.24%, ctx=7087, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=24401,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:58 2019 | |
read: IOPS=43, BW=2754KiB/s (2820kB/s)(323MiB/120018msec) | |
slat (nsec): min=0, max=2284.4k, avg=27384.79, stdev=79125.39 | |
clat (nsec): min=0, max=935607k, avg=7895742.05, stdev=47845118.61 | |
lat (nsec): min=0, max=935651k, avg=7924026.04, stdev=47851357.49 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 764], 30.00th=[ 852], 40.00th=[ 948], | |
| 50.00th=[ 1096], 60.00th=[ 1240], 70.00th=[ 1512], | |
| 80.00th=[ 126464], 90.00th=[ 2113536], 95.00th=[ 32112640], | |
| 99.00th=[200278016], 99.50th=[392167424], 99.90th=[641728512], | |
| 99.95th=[717225984], 99.99th=[935329792] | |
bw ( KiB/s): min= 0, max= 4245, per=17.54%, avg=2753.50, stdev=2109.30, samples=2 | |
iops : min= 0, max= 66, avg=42.50, stdev=33.23, samples=2 | |
write: IOPS=44, BW=2843KiB/s (2911kB/s)(333MiB/120018msec) | |
slat (nsec): min=0, max=10903k, avg=179886.99, stdev=309272.59 | |
clat (nsec): min=0, max=1047.2M, avg=82461684.37, stdev=148141789.84 | |
lat (nsec): min=0, max=1047.5M, avg=82642759.28, stdev=148090585.05 | |
clat percentiles (usec): | |
| 1.00th=[ 971], 5.00th=[ 1205], 10.00th=[ 1385], | |
| 20.00th=[ 1893], 30.00th=[ 2573], 40.00th=[ 3982], | |
| 50.00th=[ 9765], 60.00th=[ 26608], 70.00th=[ 79168], | |
| 80.00th=[ 123208], 90.00th=[ 265290], 95.00th=[ 446694], | |
| 99.00th=[ 692061], 99.50th=[ 742392], 99.90th=[ 876610], | |
| 99.95th=[ 910164], 99.99th=[1044382] | |
bw ( KiB/s): min= 0, max= 4369, per=67.16%, avg=2841.50, stdev=2160.21, samples=2 | |
iops : min= 0, max= 68, avg=44.00, stdev=33.94, samples=2 | |
lat (nsec) : 750=8.04%, 1000=13.42% | |
lat (usec) : 2=13.92%, 4=0.02%, 10=0.30%, 20=1.21%, 50=0.87% | |
lat (usec) : 100=1.30%, 250=0.97%, 500=0.61%, 750=0.59%, 1000=1.70% | |
lat (msec) : 2=12.33%, 4=10.48%, 10=5.81%, 20=3.69%, 50=5.07% | |
lat (msec) : 100=5.65%, 250=8.36%, 500=3.69%, 750=1.73%, 1000=0.26% | |
cpu : usr=0.05%, sys=0.25%, ctx=11299, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=5164,5328,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=15.3MiB/s (16.1MB/s), 2754KiB/s-12.7MiB/s (2820kB/s-13.3MB/s), io=1848MiB (1938MB), run=120018-120525msec | |
WRITE: bw=4231KiB/s (4333kB/s), 1418KiB/s-2843KiB/s (1452kB/s-2911kB/s), io=501MiB (526MB), run=120018-121297msec | |
Disk stats (read/write): | |
rbd182: ios=4729/8233, merge=0/2, ticks=563379/937246, in_queue=1494170, util=4.62% | |
<example-benchmark-fio-server-620-benchmark-rlhzh> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:58 2019 | |
write: IOPS=36, BW=2346KiB/s (2402kB/s)(276MiB/120414msec) | |
slat (nsec): min=0, max=1809.7M, avg=1451143.21, stdev=41416492.97 | |
clat (nsec): min=0, max=2608.3M, avg=107987024.10, stdev=273072237.75 | |
lat (nsec): min=0, max=2673.9M, avg=109505679.21, stdev=278381371.46 | |
clat percentiles (usec): | |
| 1.00th=[ 1139], 5.00th=[ 1450], 10.00th=[ 1663], | |
| 20.00th=[ 2147], 30.00th=[ 2769], 40.00th=[ 3654], | |
| 50.00th=[ 5145], 60.00th=[ 8586], 70.00th=[ 16909], | |
| 80.00th=[ 50594], 90.00th=[ 488637], 95.00th=[ 759170], | |
| 99.00th=[1283458], 99.50th=[1384121], 99.90th=[1870660], | |
| 99.95th=[1887437], 99.99th=[2600469] | |
bw ( KiB/s): min= 0, max= 3078, per=46.17%, avg=2351.50, stdev=1027.43, samples=2 | |
iops : min= 0, max= 48, avg=36.50, stdev=16.26, samples=2 | |
lat (usec) : 500=0.02%, 750=0.09%, 1000=0.25% | |
lat (msec) : 2=16.78%, 4=26.14%, 10=19.43%, 20=9.00%, 50=8.34% | |
lat (msec) : 100=5.01%, 250=2.68%, 500=2.40%, 750=4.87%, 1000=2.47% | |
cpu : usr=0.04%, sys=0.17%, ctx=8054, majf=0, minf=29 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,4411,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:58 2019 | |
read: IOPS=832, BW=51.8MiB/s (54.4MB/s)(6223MiB/120042msec) | |
slat (nsec): min=0, max=1627.4M, avg=61070.51, stdev=7625575.00 | |
clat (nsec): min=0, max=2303.2M, avg=4747375.68, stdev=57025931.76 | |
lat (nsec): min=0, max=2303.2M, avg=4809469.51, stdev=57538826.11 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 732], | |
| 20.00th=[ 764], 30.00th=[ 780], 40.00th=[ 796], | |
| 50.00th=[ 836], 60.00th=[ 868], 70.00th=[ 900], | |
| 80.00th=[ 1400], 90.00th=[ 12864], 95.00th=[ 782336], | |
| 99.00th=[ 43778048], 99.50th=[ 438304768], 99.90th=[ 901775360], | |
| 99.95th=[1027604480], 99.99th=[1417674752] | |
bw ( KiB/s): min= 0, max=88652, per=95.26%, avg=53098.00, stdev=50280.95, samples=2 | |
iops : min= 0, max= 1391, avg=832.50, stdev=789.84, samples=2 | |
lat (nsec) : 750=14.75%, 1000=60.85% | |
lat (usec) : 2=5.43%, 4=0.15%, 10=1.52%, 20=9.01%, 50=2.48% | |
lat (usec) : 100=0.27%, 250=0.18%, 500=0.12%, 750=0.17%, 1000=0.56% | |
lat (msec) : 2=1.15%, 4=0.85%, 10=0.60%, 20=0.40%, 50=0.61% | |
lat (msec) : 100=0.20%, 250=0.10%, 500=0.16%, 750=0.25%, 1000=0.14% | |
cpu : usr=0.22%, sys=0.80%, ctx=11085, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=99919,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:58 2019 | |
read: IOPS=41, BW=2655KiB/s (2719kB/s)(311MiB/120015msec) | |
slat (nsec): min=0, max=4456.3k, avg=32168.09, stdev=97756.59 | |
clat (nsec): min=0, max=989312k, avg=12693057.98, stdev=60321579.79 | |
lat (nsec): min=0, max=989349k, avg=12726223.12, stdev=60329190.75 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 740], | |
| 20.00th=[ 820], 30.00th=[ 916], 40.00th=[ 1080], | |
| 50.00th=[ 1224], 60.00th=[ 1384], 70.00th=[ 44288], | |
| 80.00th=[ 1122304], 90.00th=[ 15138816], 95.00th=[ 62128128], | |
| 99.00th=[320864256], 99.50th=[425721856], 99.90th=[767557632], | |
| 99.95th=[935329792], 99.99th=[985661440] | |
bw ( KiB/s): min= 0, max= 3947, per=4.76%, avg=2655.00, stdev=1827.16, samples=2 | |
iops : min= 0, max= 61, avg=41.00, stdev=28.28, samples=2 | |
write: IOPS=43, BW=2757KiB/s (2823kB/s)(323MiB/120015msec) | |
slat (nsec): min=0, max=8114.2k, avg=169031.29, stdev=274467.99 | |
clat (nsec): min=0, max=1041.7M, avg=80800586.43, stdev=146966844.61 | |
lat (nsec): min=0, max=1041.8M, avg=80970787.08, stdev=146927156.52 | |
clat percentiles (usec): | |
| 1.00th=[ 1057], 5.00th=[ 1270], 10.00th=[ 1483], | |
| 20.00th=[ 1926], 30.00th=[ 2671], 40.00th=[ 4047], | |
| 50.00th=[ 8586], 60.00th=[ 24773], 70.00th=[ 65799], | |
| 80.00th=[ 111674], 90.00th=[ 295699], 95.00th=[ 455082], | |
| 99.00th=[ 641729], 99.50th=[ 700449], 99.90th=[ 851444], | |
| 99.95th=[ 868221], 99.99th=[1044382] | |
bw ( KiB/s): min= 0, max= 4086, per=54.10%, avg=2755.50, stdev=1881.61, samples=2 | |
iops : min= 0, max= 63, avg=42.50, stdev=28.99, samples=2 | |
lat (nsec) : 750=6.03%, 1000=11.10% | |
lat (usec) : 2=15.61%, 4=0.11%, 10=0.19%, 20=0.82%, 50=0.69% | |
lat (usec) : 100=0.59%, 250=0.89%, 500=0.49%, 750=0.73%, 1000=1.93% | |
lat (msec) : 2=12.85%, 4=11.24%, 10=6.76%, 20=3.80%, 50=6.10% | |
lat (msec) : 100=6.97%, 250=6.51%, 500=4.52%, 750=1.90%, 1000=0.18% | |
cpu : usr=0.07%, sys=0.23%, ctx=12113, majf=0, minf=33 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=4978,5168,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=54.4MiB/s (57.1MB/s), 2655KiB/s-51.8MiB/s (2719kB/s-54.4MB/s), io=6534MiB (6852MB), run=120015-120042msec | |
WRITE: bw=5094KiB/s (5216kB/s), 2346KiB/s-2757KiB/s (2402kB/s-2823kB/s), io=599MiB (628MB), run=120015-120414msec | |
Disk stats (read/write): | |
rbd291: ios=6612/9793, merge=0/1, ticks=541800/872212, in_queue=1405759, util=7.93% | |
<example-benchmark-fio-server-139-benchmark-7btww> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:58 2019 | |
write: IOPS=21, BW=1355KiB/s (1388kB/s)(160MiB/120980msec) | |
slat (nsec): min=0, max=859342k, avg=1241964.81, stdev=25538249.96 | |
clat (nsec): min=0, max=2242.1M, avg=188335014.13, stdev=370328253.41 | |
lat (nsec): min=0, max=2242.8M, avg=189577480.82, stdev=371271934.52 | |
clat percentiles (usec): | |
| 1.00th=[ 1090], 5.00th=[ 1434], 10.00th=[ 1778], | |
| 20.00th=[ 2474], 30.00th=[ 3326], 40.00th=[ 4752], | |
| 50.00th=[ 7242], 60.00th=[ 13566], 70.00th=[ 27657], | |
| 80.00th=[ 392168], 90.00th=[ 817890], 95.00th=[1035994], | |
| 99.00th=[1484784], 99.50th=[1635779], 99.90th=[2139096], | |
| 99.95th=[2139096], 99.99th=[2231370] | |
bw ( KiB/s): min= 0, max= 1415, per=29.91%, avg=1358.50, stdev=79.90, samples=2 | |
iops : min= 0, max= 22, avg=21.00, stdev= 1.41, samples=2 | |
lat (usec) : 10=0.04%, 500=0.04%, 750=0.08%, 1000=0.31% | |
lat (msec) : 2=12.90%, 4=21.81%, 10=20.75%, 20=9.73%, 50=8.75% | |
lat (msec) : 100=1.99%, 250=1.80%, 500=4.85%, 750=5.31%, 1000=5.90% | |
cpu : usr=0.02%, sys=0.11%, ctx=4636, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,2559,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:58 2019 | |
read: IOPS=376, BW=23.5MiB/s (24.7MB/s)(2837MiB/120564msec) | |
slat (nsec): min=0, max=691432k, avg=43024.80, stdev=3613085.44 | |
clat (nsec): min=0, max=2143.1M, avg=10554204.56, stdev=88854248.08 | |
lat (nsec): min=0, max=2143.2M, avg=10598187.22, stdev=88937193.66 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 748], | |
| 20.00th=[ 780], 30.00th=[ 796], 40.00th=[ 828], | |
| 50.00th=[ 860], 60.00th=[ 892], 70.00th=[ 932], | |
| 80.00th=[ 1144], 90.00th=[ 13632], 95.00th=[ 2342912], | |
| 99.00th=[ 541065216], 99.50th=[ 759169024], 99.90th=[1052770304], | |
| 99.95th=[1266679808], 99.99th=[1602224128] | |
bw ( KiB/s): min= 0, max=34499, per=89.06%, avg=24203.50, stdev=14560.04, samples=2 | |
iops : min= 0, max= 539, avg=378.00, stdev=227.69, samples=2 | |
lat (nsec) : 750=10.71%, 1000=65.67% | |
lat (usec) : 2=10.08%, 4=0.17%, 10=0.54%, 20=3.27%, 50=1.05% | |
lat (usec) : 100=0.19%, 250=0.11%, 500=0.19%, 750=0.19%, 1000=0.72% | |
lat (msec) : 2=1.76%, 4=1.56%, 10=1.17%, 20=0.52%, 50=0.42% | |
lat (msec) : 100=0.13%, 250=0.16%, 500=0.33%, 750=0.55%, 1000=0.37% | |
cpu : usr=0.10%, sys=0.42%, ctx=7607, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=45440,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:58 2019 | |
read: IOPS=48, BW=3100KiB/s (3175kB/s)(363MiB/120004msec) | |
slat (nsec): min=0, max=3502.5k, avg=30685.11, stdev=91394.21 | |
clat (nsec): min=0, max=886323k, avg=8285032.64, stdev=51086846.62 | |
lat (nsec): min=0, max=886388k, avg=8316623.23, stdev=51091851.38 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 764], 30.00th=[ 876], 40.00th=[ 996], | |
| 50.00th=[ 1144], 60.00th=[ 1288], 70.00th=[ 1672], | |
| 80.00th=[ 179200], 90.00th=[ 1974272], 95.00th=[ 14221312], | |
| 99.00th=[320864256], 99.50th=[438304768], 99.90th=[583008256], | |
| 99.95th=[658505728], 99.99th=[884998144] | |
bw ( KiB/s): min= 0, max= 4320, per=11.41%, avg=3100.00, stdev=1725.34, samples=2 | |
iops : min= 0, max= 67, avg=48.00, stdev=26.87, samples=2 | |
write: IOPS=50, BW=3213KiB/s (3290kB/s)(377MiB/120004msec) | |
slat (nsec): min=0, max=13105k, avg=189168.32, stdev=269636.09 | |
clat (nsec): min=0, max=1091.8M, avg=71891687.37, stdev=135619142.37 | |
lat (nsec): min=0, max=1091.9M, avg=72082105.36, stdev=135568803.97 | |
clat percentiles (usec): | |
| 1.00th=[ 1020], 5.00th=[ 1221], 10.00th=[ 1418], | |
| 20.00th=[ 1827], 30.00th=[ 2442], 40.00th=[ 3818], | |
| 50.00th=[ 7832], 60.00th=[ 21103], 70.00th=[ 68682], | |
| 80.00th=[ 107480], 90.00th=[ 200279], 95.00th=[ 392168], | |
| 99.00th=[ 658506], 99.50th=[ 717226], 99.90th=[ 859833], | |
| 99.95th=[ 926942], 99.99th=[1098908] | |
bw ( KiB/s): min= 0, max= 4449, per=70.71%, avg=3211.50, stdev=1750.09, samples=2 | |
iops : min= 0, max= 69, avg=49.50, stdev=27.58, samples=2 | |
lat (nsec) : 750=8.45%, 1000=11.48% | |
lat (usec) : 2=14.69%, 4=0.07%, 10=0.48%, 20=1.30%, 50=0.89% | |
lat (usec) : 100=1.14%, 250=1.15%, 500=0.68%, 750=0.79%, 1000=1.64% | |
lat (msec) : 2=13.27%, 4=10.32%, 10=6.92%, 20=3.79%, 50=4.01% | |
lat (msec) : 100=6.76%, 250=7.48%, 500=3.02%, 750=1.48%, 1000=0.19% | |
cpu : usr=0.08%, sys=0.28%, ctx=12857, majf=0, minf=34 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=5813,6022,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=26.5MiB/s (27.8MB/s), 3100KiB/s-23.5MiB/s (3175kB/s-24.7MB/s), io=3200MiB (3355MB), run=120004-120564msec | |
WRITE: bw=4543KiB/s (4652kB/s), 1355KiB/s-3213KiB/s (1388kB/s-3290kB/s), io=537MiB (563MB), run=120004-120980msec | |
Disk stats (read/write): | |
rbd58: ios=5113/8988, merge=0/1, ticks=567154/914911, in_queue=1474958, util=6.12% | |
<example-benchmark-fio-server-289-benchmark-jhcgn> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:58 2019 | |
write: IOPS=44, BW=2824KiB/s (2892kB/s)(331MiB/120072msec) | |
slat (nsec): min=0, max=714127k, avg=491923.63, stdev=13745623.96 | |
clat (nsec): min=0, max=1605.9M, avg=90397146.42, stdev=161216053.44 | |
lat (nsec): min=0, max=1766.4M, avg=90890178.16, stdev=162453244.77 | |
clat percentiles (usec): | |
| 1.00th=[ 938], 5.00th=[ 1123], 10.00th=[ 1270], | |
| 20.00th=[ 1614], 30.00th=[ 2114], 40.00th=[ 3064], | |
| 50.00th=[ 7046], 60.00th=[ 62129], 70.00th=[ 94897], | |
| 80.00th=[ 135267], 90.00th=[ 254804], 95.00th=[ 438305], | |
| 99.00th=[ 759170], 99.50th=[ 943719], 99.90th=[1132463], | |
| 99.95th=[1149240], 99.99th=[1602225] | |
bw ( KiB/s): min= 0, max= 3586, per=45.15%, avg=2823.00, stdev=1079.04, samples=2 | |
iops : min= 0, max= 56, avg=44.00, stdev=16.97, samples=2 | |
lat (usec) : 250=0.02%, 500=0.02%, 750=0.09%, 1000=2.00% | |
lat (msec) : 2=26.38%, 4=15.98%, 10=7.84%, 20=2.72%, 50=3.36% | |
lat (msec) : 100=13.54%, 250=17.79%, 500=6.35%, 750=2.85%, 1000=0.64% | |
cpu : usr=0.07%, sys=0.18%, ctx=9958, majf=0, minf=26 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,5295,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:58 2019 | |
read: IOPS=311, BW=19.5MiB/s (20.4MB/s)(2338MiB/120002msec) | |
slat (nsec): min=0, max=710630k, avg=63218.86, stdev=4979859.96 | |
clat (nsec): min=0, max=1434.2M, avg=12791436.07, stdev=62473523.58 | |
lat (nsec): min=0, max=1434.4M, avg=12855512.68, stdev=62676595.96 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 732], 40.00th=[ 748], | |
| 50.00th=[ 820], 60.00th=[ 876], 70.00th=[ 1032], | |
| 80.00th=[ 8768], 90.00th=[ 7045120], 95.00th=[ 60030976], | |
| 99.00th=[ 362807296], 99.50th=[ 484442112], 99.90th=[ 683671552], | |
| 99.95th=[ 767557632], 99.99th=[1019215872] | |
bw ( KiB/s): min= 0, max=25118, per=85.72%, avg=19949.00, stdev=7310.07, samples=2 | |
iops : min= 0, max= 392, avg=311.00, stdev=114.55, samples=2 | |
lat (nsec) : 750=41.73%, 1000=27.46% | |
lat (usec) : 2=10.55%, 4=0.03%, 10=0.58%, 20=0.31%, 50=0.27% | |
lat (usec) : 100=0.09%, 250=0.07%, 500=0.27%, 750=0.98%, 1000=2.19% | |
lat (msec) : 2=3.06%, 4=1.82%, 10=0.83%, 20=0.52%, 50=3.20% | |
lat (msec) : 100=2.93%, 250=1.53%, 500=1.11%, 750=0.40%, 1000=0.05% | |
cpu : usr=0.11%, sys=0.35%, ctx=14140, majf=0, minf=97 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=37428,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:58 2019 | |
read: IOPS=51, BW=3322KiB/s (3402kB/s)(389MiB/120007msec) | |
slat (nsec): min=0, max=663988k, avg=171583.65, stdev=8737229.14 | |
clat (nsec): min=0, max=632295k, avg=6775755.36, stdev=37927924.18 | |
lat (nsec): min=0, max=664065k, avg=6948327.42, stdev=38902864.79 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 740], | |
| 20.00th=[ 796], 30.00th=[ 892], 40.00th=[ 1004], | |
| 50.00th=[ 1160], 60.00th=[ 1304], 70.00th=[ 17792], | |
| 80.00th=[ 864256], 90.00th=[ 3031040], 95.00th=[ 33816576], | |
| 99.00th=[179306496], 99.50th=[308281344], 99.90th=[492830720], | |
| 99.95th=[574619648], 99.99th=[633339904] | |
bw ( KiB/s): min= 0, max= 4763, per=14.27%, avg=3322.00, stdev=2037.88, samples=2 | |
iops : min= 0, max= 74, avg=51.50, stdev=31.82, samples=2 | |
write: IOPS=53, BW=3430KiB/s (3513kB/s)(402MiB/120007msec) | |
slat (nsec): min=0, max=11878k, avg=197961.98, stdev=307898.64 | |
clat (nsec): min=0, max=1469.7M, avg=68202610.29, stdev=138950257.20 | |
lat (nsec): min=0, max=1469.8M, avg=68401706.46, stdev=138903708.14 | |
clat percentiles (usec): | |
| 1.00th=[ 996], 5.00th=[ 1221], 10.00th=[ 1401], | |
| 20.00th=[ 1762], 30.00th=[ 2212], 40.00th=[ 2999], | |
| 50.00th=[ 4621], 60.00th=[ 12387], 70.00th=[ 57410], | |
| 80.00th=[ 103285], 90.00th=[ 200279], 95.00th=[ 354419], | |
| 99.00th=[ 683672], 99.50th=[ 801113], 99.90th=[1098908], | |
| 99.95th=[1132463], 99.99th=[1468007] | |
bw ( KiB/s): min= 0, max= 4867, per=54.84%, avg=3428.50, stdev=2034.35, samples=2 | |
iops : min= 0, max= 76, avg=53.50, stdev=31.82, samples=2 | |
lat (nsec) : 750=7.08%, 1000=12.30% | |
lat (usec) : 2=14.32%, 4=0.03%, 10=0.24%, 20=0.81%, 50=0.70% | |
lat (usec) : 100=0.59%, 250=0.86%, 500=0.89%, 750=0.79%, 1000=2.20% | |
lat (msec) : 2=15.49%, 4=12.24%, 10=7.09%, 20=2.65%, 50=4.11% | |
lat (msec) : 100=6.31%, 250=7.16%, 500=2.66%, 750=1.15%, 1000=0.28% | |
cpu : usr=0.07%, sys=0.31%, ctx=14996, majf=0, minf=33 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=6231,6429,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=22.7MiB/s (23.8MB/s), 3322KiB/s-19.5MiB/s (3402kB/s-20.4MB/s), io=2728MiB (2860MB), run=120002-120007msec | |
WRITE: bw=6252KiB/s (6402kB/s), 2824KiB/s-3430KiB/s (2892kB/s-3513kB/s), io=733MiB (769MB), run=120007-120072msec | |
Disk stats (read/write): | |
rbd123: ios=9289/12253, merge=0/2, ticks=557195/852287, in_queue=1398563, util=14.81% | |
<example-benchmark-fio-server-115-benchmark-krrp7> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:59 2019 | |
write: IOPS=26, BW=1687KiB/s (1727kB/s)(198MiB/120401msec) | |
slat (nsec): min=0, max=464962k, avg=535888.93, stdev=11242241.37 | |
clat (nsec): min=0, max=2030.7M, avg=151464747.66, stdev=258316057.55 | |
lat (nsec): min=0, max=2030.8M, avg=152001879.93, stdev=258431932.77 | |
clat percentiles (usec): | |
| 1.00th=[ 1270], 5.00th=[ 1942], 10.00th=[ 2573], | |
| 20.00th=[ 3851], 30.00th=[ 5473], 40.00th=[ 7832], | |
| 50.00th=[ 11207], 60.00th=[ 20055], 70.00th=[ 122160], | |
| 80.00th=[ 329253], 90.00th=[ 549454], 95.00th=[ 700449], | |
| 99.00th=[ 943719], 99.50th=[1182794], 99.90th=[2021655], | |
| 99.95th=[2021655], 99.99th=[2038432] | |
bw ( KiB/s): min= 0, max= 1993, per=31.33%, avg=1688.50, stdev=430.63, samples=2 | |
iops : min= 0, max= 31, avg=26.00, stdev= 7.07, samples=2 | |
lat (usec) : 4=0.13%, 250=0.03%, 500=0.03%, 750=0.06%, 1000=0.06% | |
lat (msec) : 2=5.21%, 4=15.36%, 10=26.12%, 20=13.19%, 50=6.50% | |
lat (msec) : 100=2.62%, 250=6.18%, 500=13.15%, 750=7.48%, 1000=3.19% | |
cpu : usr=0.03%, sys=0.12%, ctx=5496, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,3170,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:59 2019 | |
read: IOPS=123, BW=7920KiB/s (8110kB/s)(933MiB/120639msec) | |
slat (nsec): min=0, max=7129.5k, avg=37269.24, stdev=139259.60 | |
clat (nsec): min=0, max=1959.1M, avg=32268522.52, stdev=150613641.83 | |
lat (nsec): min=0, max=1959.1M, avg=32306809.86, stdev=150630782.48 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 748], | |
| 20.00th=[ 788], 30.00th=[ 820], 40.00th=[ 868], | |
| 50.00th=[ 908], 60.00th=[ 1080], 70.00th=[ 1368], | |
| 80.00th=[ 1138688], 90.00th=[ 4882432], 95.00th=[ 56885248], | |
| 99.00th=[ 843055104], 99.50th=[ 960495616], 99.90th=[1216348160], | |
| 99.95th=[1400897536], 99.99th=[1803550720] | |
bw ( KiB/s): min= 0, max= 8521, per=68.97%, avg=7955.50, stdev=799.74, samples=2 | |
iops : min= 0, max= 133, avg=124.00, stdev=12.73, samples=2 | |
lat (nsec) : 750=10.00%, 1000=47.62% | |
lat (usec) : 2=17.35%, 4=0.25%, 10=0.15%, 20=0.74%, 50=0.16% | |
lat (usec) : 100=0.14%, 250=0.13%, 500=0.11%, 750=0.44%, 1000=1.89% | |
lat (msec) : 2=5.43%, 4=4.56%, 10=3.44%, 20=1.44%, 50=1.05% | |
lat (msec) : 100=0.33%, 250=0.31%, 500=1.02%, 750=1.94%, 1000=1.11% | |
cpu : usr=0.04%, sys=0.18%, ctx=7043, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=14926,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:59 2019 | |
read: IOPS=56, BW=3635KiB/s (3722kB/s)(426MiB/120006msec) | |
slat (nsec): min=0, max=1680.6k, avg=30012.19, stdev=72067.62 | |
clat (nsec): min=0, max=747366k, avg=10032263.04, stdev=54279230.59 | |
lat (nsec): min=0, max=747386k, avg=10063205.13, stdev=54284115.55 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 796], 30.00th=[ 892], 40.00th=[ 1012], | |
| 50.00th=[ 1144], 60.00th=[ 1304], 70.00th=[ 15808], | |
| 80.00th=[ 921600], 90.00th=[ 3522560], 95.00th=[ 45350912], | |
| 99.00th=[329252864], 99.50th=[442499072], 99.90th=[675282944], | |
| 99.95th=[700448768], 99.99th=[750780416] | |
bw ( KiB/s): min= 0, max= 5302, per=31.51%, avg=3635.00, stdev=2357.49, samples=2 | |
iops : min= 0, max= 82, avg=56.00, stdev=36.77, samples=2 | |
write: IOPS=58, BW=3715KiB/s (3804kB/s)(435MiB/120006msec) | |
slat (nsec): min=0, max=29166k, avg=196391.82, stdev=440398.02 | |
clat (nsec): min=0, max=905267k, avg=58898123.51, stdev=124023895.80 | |
lat (nsec): min=0, max=906784k, avg=59095996.36, stdev=123976481.96 | |
clat percentiles (usec): | |
| 1.00th=[ 979], 5.00th=[ 1172], 10.00th=[ 1352], 20.00th=[ 1680], | |
| 30.00th=[ 2114], 40.00th=[ 2868], 50.00th=[ 4686], 60.00th=[ 10421], | |
| 70.00th=[ 32113], 80.00th=[ 93848], 90.00th=[160433], 95.00th=[320865], | |
| 99.00th=[675283], 99.50th=[767558], 99.90th=[868221], 99.95th=[893387], | |
| 99.99th=[901776] | |
bw ( KiB/s): min= 0, max= 5360, per=68.90%, avg=3713.00, stdev=2329.21, samples=2 | |
iops : min= 0, max= 83, avg=57.50, stdev=36.06, samples=2 | |
lat (nsec) : 750=7.63%, 1000=11.81% | |
lat (usec) : 2=14.57%, 4=0.04%, 10=0.25%, 20=0.73%, 50=0.65% | |
lat (usec) : 100=0.67%, 250=0.86%, 500=0.48%, 750=0.68%, 1000=2.32% | |
lat (msec) : 2=16.48%, 4=11.47%, 10=6.95%, 20=4.01%, 50=4.48% | |
lat (msec) : 100=5.18%, 250=6.82%, 500=2.74%, 750=0.91%, 1000=0.28% | |
cpu : usr=0.08%, sys=0.33%, ctx=15897, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=6816,6963,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=11.3MiB/s (11.8MB/s), 3635KiB/s-7920KiB/s (3722kB/s-8110kB/s), io=1359MiB (1425MB), run=120006-120639msec | |
WRITE: bw=5389KiB/s (5519kB/s), 1687KiB/s-3715KiB/s (1727kB/s-3804kB/s), io=634MiB (664MB), run=120006-120401msec | |
Disk stats (read/write): | |
rbd33: ios=5721/11388, merge=0/3, ticks=605434/883833, in_queue=1480636, util=5.12% | |
<example-benchmark-fio-server-594-benchmark-dz8fd> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:59 2019 | |
write: IOPS=39, BW=2556KiB/s (2617kB/s)(300MiB/120181msec) | |
slat (nsec): min=0, max=746936k, avg=484272.82, stdev=14806221.57 | |
clat (nsec): min=0, max=1378.3M, avg=100241276.02, stdev=175257826.40 | |
lat (nsec): min=0, max=1378.4M, avg=100726680.74, stdev=175669845.78 | |
clat percentiles (usec): | |
| 1.00th=[ 996], 5.00th=[ 1237], 10.00th=[ 1450], | |
| 20.00th=[ 1926], 30.00th=[ 2868], 40.00th=[ 4883], | |
| 50.00th=[ 11076], 60.00th=[ 55837], 70.00th=[ 99091], | |
| 80.00th=[ 139461], 90.00th=[ 337642], 95.00th=[ 484443], | |
| 99.00th=[ 809501], 99.50th=[ 952108], 99.90th=[1283458], | |
| 99.95th=[1317012], 99.99th=[1384121] | |
bw ( KiB/s): min= 0, max= 2893, per=46.14%, avg=2557.00, stdev=475.18, samples=2 | |
iops : min= 0, max= 45, avg=39.50, stdev= 7.78, samples=2 | |
lat (usec) : 100=0.02%, 500=0.02%, 750=0.04%, 1000=0.92% | |
lat (msec) : 2=20.20%, 4=15.22%, 10=12.42%, 20=5.88%, 50=4.48% | |
lat (msec) : 100=11.03%, 250=16.76%, 500=8.46%, 750=3.25%, 1000=0.96% | |
cpu : usr=0.05%, sys=0.17%, ctx=8988, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,4797,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:59 2019 | |
read: IOPS=336, BW=21.0MiB/s (22.0MB/s)(2525MiB/120182msec) | |
slat (nsec): min=0, max=688109k, avg=55153.02, stdev=4613265.28 | |
clat (nsec): min=0, max=1316.2M, avg=11849012.02, stdev=62320874.61 | |
lat (nsec): min=0, max=1316.2M, avg=11905146.43, stdev=62497607.01 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 732], 40.00th=[ 756], | |
| 50.00th=[ 844], 60.00th=[ 908], 70.00th=[ 1112], | |
| 80.00th=[ 354304], 90.00th=[ 2998272], 95.00th=[ 55312384], | |
| 99.00th=[ 358612992], 99.50th=[ 471859200], 99.90th=[ 725614592], | |
| 99.95th=[ 859832320], 99.99th=[1199570944] | |
bw ( KiB/s): min= 0, max=31963, per=88.33%, avg=21542.00, stdev=14737.52, samples=2 | |
iops : min= 0, max= 499, avg=336.00, stdev=230.52, samples=2 | |
lat (nsec) : 750=38.94%, 1000=27.33% | |
lat (usec) : 2=11.77%, 4=0.04%, 10=0.52%, 20=0.59%, 50=0.19% | |
lat (usec) : 100=0.11%, 250=0.08%, 500=0.87%, 750=1.40%, 1000=2.68% | |
lat (msec) : 2=3.81%, 4=2.41%, 10=1.09%, 20=0.57%, 50=2.13% | |
lat (msec) : 100=2.62%, 250=1.33%, 500=1.11%, 750=0.33%, 1000=0.07% | |
cpu : usr=0.11%, sys=0.40%, ctx=16448, majf=0, minf=108 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=40399,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:59 2019 | |
read: IOPS=45, BW=2881KiB/s (2950kB/s)(338MiB/120007msec) | |
slat (nsec): min=0, max=3273.4k, avg=38406.71, stdev=105301.68 | |
clat (nsec): min=0, max=585036k, avg=9336840.69, stdev=43652742.88 | |
lat (nsec): min=0, max=585053k, avg=9376204.46, stdev=43661535.41 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 732], 10.00th=[ 740], | |
| 20.00th=[ 820], 30.00th=[ 924], 40.00th=[ 1064], | |
| 50.00th=[ 1208], 60.00th=[ 1400], 70.00th=[ 209920], | |
| 80.00th=[ 1155072], 90.00th=[ 6782976], 95.00th=[ 46399488], | |
| 99.00th=[244318208], 99.50th=[350224384], 99.90th=[509607936], | |
| 99.95th=[541065216], 99.99th=[583008256] | |
bw ( KiB/s): min= 0, max= 4260, per=11.81%, avg=2879.50, stdev=1952.32, samples=2 | |
iops : min= 0, max= 66, avg=44.50, stdev=30.41, samples=2 | |
write: IOPS=46, BW=2991KiB/s (3063kB/s)(351MiB/120007msec) | |
slat (nsec): min=0, max=57340k, avg=188996.30, stdev=811463.73 | |
clat (nsec): min=0, max=1109.1M, avg=76748413.93, stdev=143499222.92 | |
lat (nsec): min=0, max=1109.1M, avg=76938515.68, stdev=143449607.79 | |
clat percentiles (usec): | |
| 1.00th=[ 1074], 5.00th=[ 1303], 10.00th=[ 1532], | |
| 20.00th=[ 2024], 30.00th=[ 2900], 40.00th=[ 4817], | |
| 50.00th=[ 10814], 60.00th=[ 26346], 70.00th=[ 67634], | |
| 80.00th=[ 107480], 90.00th=[ 235930], 95.00th=[ 425722], | |
| 99.00th=[ 692061], 99.50th=[ 759170], 99.90th=[ 943719], | |
| 99.95th=[ 985662], 99.99th=[1115685] | |
bw ( KiB/s): min= 0, max= 4384, per=53.94%, avg=2989.50, stdev=1972.12, samples=2 | |
iops : min= 0, max= 68, avg=46.00, stdev=31.11, samples=2 | |
lat (nsec) : 750=6.66%, 1000=10.83% | |
lat (usec) : 2=14.09%, 4=0.10%, 10=0.29%, 20=0.60%, 50=0.65% | |
lat (usec) : 100=0.66%, 250=0.60%, 500=1.10%, 750=1.01%, 1000=2.00% | |
lat (msec) : 2=12.96%, 4=10.63%, 10=7.37%, 20=4.72%, 50=6.18% | |
lat (msec) : 100=7.20%, 250=7.00%, 500=3.70%, 750=1.34%, 1000=0.31% | |
cpu : usr=0.06%, sys=0.28%, ctx=13512, majf=0, minf=34 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=5402,5605,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=23.8MiB/s (24.0MB/s), 2881KiB/s-21.0MiB/s (2950kB/s-22.0MB/s), io=2862MiB (3001MB), run=120007-120182msec | |
WRITE: bw=5543KiB/s (5676kB/s), 2556KiB/s-2991KiB/s (2617kB/s-3063kB/s), io=651MiB (682MB), run=120007-120181msec | |
Disk stats (read/write): | |
rbd248: ios=11875/11980, merge=0/2, ticks=556245/857749, in_queue=1401822, util=15.70% | |
<example-benchmark-fio-server-130-benchmark-ckmtx> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:59 2019 | |
write: IOPS=51, BW=3328KiB/s (3408kB/s)(390MiB/120061msec) | |
slat (nsec): min=0, max=657788k, avg=347116.30, stdev=8981861.13 | |
clat (nsec): min=0, max=1140.8M, avg=76624588.97, stdev=145436001.13 | |
lat (nsec): min=0, max=1140.8M, avg=76973399.87, stdev=145639513.75 | |
clat percentiles (usec): | |
| 1.00th=[ 955], 5.00th=[ 1156], 10.00th=[ 1319], | |
| 20.00th=[ 1696], 30.00th=[ 2376], 40.00th=[ 3589], | |
| 50.00th=[ 7177], 60.00th=[ 24773], 70.00th=[ 71828], | |
| 80.00th=[ 108528], 90.00th=[ 233833], 95.00th=[ 400557], | |
| 99.00th=[ 725615], 99.50th=[ 843056], 99.90th=[ 952108], | |
| 99.95th=[1002439], 99.99th=[1132463] | |
bw ( KiB/s): min= 0, max= 3477, per=50.05%, avg=3327.50, stdev=211.42, samples=2 | |
iops : min= 0, max= 54, avg=51.50, stdev= 3.54, samples=2 | |
lat (usec) : 2=0.02%, 4=0.02%, 500=0.03%, 750=0.05%, 1000=1.47% | |
lat (msec) : 2=23.52%, 4=17.03%, 10=11.04%, 20=5.26%, 50=6.68% | |
lat (msec) : 100=12.39%, 250=13.27%, 500=6.18%, 750=2.20%, 1000=0.87% | |
cpu : usr=0.07%, sys=0.22%, ctx=11790, majf=0, minf=26 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,6241,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:59 2019 | |
read: IOPS=338, BW=21.1MiB/s (22.1MB/s)(2533MiB/120041msec) | |
slat (nsec): min=0, max=520529k, avg=41330.26, stdev=2721988.27 | |
clat (nsec): min=0, max=1107.1M, avg=11867312.79, stdev=59671897.16 | |
lat (nsec): min=0, max=1107.1M, avg=11909494.08, stdev=59741193.49 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 708], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 732], 40.00th=[ 748], | |
| 50.00th=[ 836], 60.00th=[ 892], 70.00th=[ 1096], | |
| 80.00th=[ 24960], 90.00th=[ 5210112], 95.00th=[ 56360960], | |
| 99.00th=[ 337641472], 99.50th=[ 471859200], 99.90th=[ 700448768], | |
| 99.95th=[ 759169024], 99.99th=[1061158912] | |
bw ( KiB/s): min= 0, max=31047, per=87.08%, avg=21615.00, stdev=13338.86, samples=2 | |
iops : min= 0, max= 485, avg=337.50, stdev=208.60, samples=2 | |
lat (nsec) : 750=41.36%, 1000=26.06% | |
lat (usec) : 2=10.59%, 4=0.04%, 10=0.87%, 20=1.02%, 50=0.41% | |
lat (usec) : 100=0.08%, 250=0.04%, 500=0.19%, 750=1.10%, 1000=2.41% | |
lat (msec) : 2=3.42%, 4=2.02%, 10=0.98%, 20=0.62%, 50=3.06% | |
lat (msec) : 100=2.91%, 250=1.43%, 500=0.96%, 750=0.36%, 1000=0.05% | |
cpu : usr=0.11%, sys=0.39%, ctx=15868, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=40576,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:59 2019 | |
read: IOPS=50, BW=3214KiB/s (3291kB/s)(377MiB/120006msec) | |
slat (nsec): min=0, max=657744k, avg=145579.23, stdev=8474050.87 | |
clat (nsec): min=0, max=1066.6M, avg=8382053.59, stdev=40166155.39 | |
lat (nsec): min=0, max=1066.6M, avg=8528658.07, stdev=41033365.30 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 732], 10.00th=[ 740], | |
| 20.00th=[ 804], 30.00th=[ 908], 40.00th=[ 1048], | |
| 50.00th=[ 1176], 60.00th=[ 1320], 70.00th=[ 18560], | |
| 80.00th=[ 1003520], 90.00th=[ 7897088], 95.00th=[ 48496640], | |
| 99.00th=[ 183500800], 99.50th=[ 274726912], 99.90th=[ 484442112], | |
| 99.95th=[ 517996544], 99.99th=[1069547520] | |
bw ( KiB/s): min= 0, max= 4074, per=12.95%, avg=3214.00, stdev=1216.22, samples=2 | |
iops : min= 0, max= 63, avg=49.50, stdev=19.09, samples=2 | |
write: IOPS=51, BW=3323KiB/s (3403kB/s)(389MiB/120006msec) | |
slat (nsec): min=0, max=63599k, avg=206804.07, stdev=945686.55 | |
clat (nsec): min=0, max=1233.8M, avg=68800246.38, stdev=134987032.75 | |
lat (nsec): min=0, max=1233.9M, avg=69008286.97, stdev=134933795.73 | |
clat percentiles (usec): | |
| 1.00th=[ 996], 5.00th=[ 1221], 10.00th=[ 1434], | |
| 20.00th=[ 1942], 30.00th=[ 2769], 40.00th=[ 4490], | |
| 50.00th=[ 8717], 60.00th=[ 21890], 70.00th=[ 54264], | |
| 80.00th=[ 96994], 90.00th=[ 204473], 95.00th=[ 375391], | |
| 99.00th=[ 650118], 99.50th=[ 775947], 99.90th=[ 943719], | |
| 99.95th=[ 952108], 99.99th=[1233126] | |
bw ( KiB/s): min= 0, max= 4184, per=49.95%, avg=3321.00, stdev=1220.47, samples=2 | |
iops : min= 0, max= 65, avg=51.50, stdev=19.09, samples=2 | |
lat (nsec) : 750=6.62%, 1000=11.64% | |
lat (usec) : 2=15.40%, 4=0.07%, 10=0.26%, 20=0.58%, 50=0.62% | |
lat (usec) : 100=0.55%, 250=0.57%, 500=0.71%, 750=0.72%, 1000=2.13% | |
lat (msec) : 2=12.83%, 4=10.11%, 10=7.98%, 20=4.37%, 50=6.58% | |
lat (msec) : 100=7.44%, 250=6.36%, 500=3.20%, 750=0.96%, 1000=0.32% | |
cpu : usr=0.09%, sys=0.29%, ctx=14868, majf=0, minf=34 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=6028,6228,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=24.2MiB/s (25.4MB/s), 3214KiB/s-21.1MiB/s (3291kB/s-22.1MB/s), io=2910MiB (3051MB), run=120006-120041msec | |
WRITE: bw=6650KiB/s (6810kB/s), 3323KiB/s-3328KiB/s (3403kB/s-3408kB/s), io=780MiB (818MB), run=120006-120061msec | |
Disk stats (read/write): | |
rbd49: ios=9506/12982, merge=0/1, ticks=545666/834922, in_queue=1369109, util=15.51% | |
<example-benchmark-fio-server-600-benchmark-fvxhx> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:59 2019 | |
write: IOPS=19, BW=1236KiB/s (1265kB/s)(145MiB/120516msec) | |
slat (nsec): min=0, max=694928k, avg=1293864.89, stdev=25365740.43 | |
clat (nsec): min=0, max=2243.2M, avg=206617078.50, stdev=402630615.34 | |
lat (nsec): min=0, max=2243.3M, avg=207910976.65, stdev=403420903.13 | |
clat percentiles (usec): | |
| 1.00th=[ 1106], 5.00th=[ 1418], 10.00th=[ 1647], | |
| 20.00th=[ 2180], 30.00th=[ 2802], 40.00th=[ 3556], | |
| 50.00th=[ 4817], 60.00th=[ 7832], 70.00th=[ 21103], | |
| 80.00th=[ 467665], 90.00th=[ 884999], 95.00th=[1199571], | |
| 99.00th=[1501561], 99.50th=[1635779], 99.90th=[1920992], | |
| 99.95th=[2139096], 99.99th=[2231370] | |
bw ( KiB/s): min= 0, max= 1654, per=26.83%, avg=1238.50, stdev=587.61, samples=2 | |
iops : min= 0, max= 25, avg=18.50, stdev= 9.19, samples=2 | |
lat (usec) : 750=0.09%, 1000=0.17% | |
lat (msec) : 2=16.44%, 4=27.54%, 10=18.76%, 20=6.45%, 50=5.16% | |
lat (msec) : 100=1.42%, 250=1.98%, 500=2.41%, 750=6.88%, 1000=5.59% | |
cpu : usr=0.02%, sys=0.09%, ctx=4282, majf=0, minf=29 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,2324,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:59 2019 | |
read: IOPS=685, BW=42.7MiB/s (44.8MB/s)(5151MiB/120619msec) | |
slat (nsec): min=0, max=645074k, avg=32638.70, stdev=3275070.39 | |
clat (nsec): min=0, max=2092.4M, avg=5821962.72, stdev=68390851.36 | |
lat (nsec): min=0, max=2092.8M, avg=5855536.09, stdev=68477975.18 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 740], | |
| 20.00th=[ 772], 30.00th=[ 788], 40.00th=[ 804], | |
| 50.00th=[ 836], 60.00th=[ 868], 70.00th=[ 900], | |
| 80.00th=[ 1112], 90.00th=[ 12480], 95.00th=[ 33024], | |
| 99.00th=[ 27394048], 99.50th=[ 633339904], 99.90th=[1019215872], | |
| 99.95th=[1199570944], 99.99th=[1518338048] | |
bw ( KiB/s): min= 0, max=69544, per=93.51%, avg=43953.50, stdev=36190.43, samples=2 | |
iops : min= 0, max= 1091, avg=689.00, stdev=568.51, samples=2 | |
lat (nsec) : 750=12.78%, 1000=65.59% | |
lat (usec) : 2=4.88%, 4=0.16%, 10=2.00%, 20=7.08%, 50=3.16% | |
lat (usec) : 100=0.23%, 250=0.11%, 500=0.14%, 750=0.15%, 1000=0.43% | |
lat (msec) : 2=0.94%, 4=0.68%, 10=0.41%, 20=0.18%, 50=0.21% | |
lat (msec) : 100=0.07%, 250=0.08%, 500=0.10%, 750=0.29%, 1000=0.23% | |
cpu : usr=0.16%, sys=0.65%, ctx=6935, majf=0, minf=31 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=82711,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:59 2019 | |
read: IOPS=51, BW=3292KiB/s (3371kB/s)(386MiB/120003msec) | |
slat (nsec): min=0, max=65904k, avg=36780.12, stdev=841937.14 | |
clat (nsec): min=0, max=749872k, avg=6530086.82, stdev=44937011.76 | |
lat (nsec): min=0, max=749907k, avg=6567813.82, stdev=44947365.91 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 756], 30.00th=[ 852], 40.00th=[ 956], | |
| 50.00th=[ 1096], 60.00th=[ 1272], 70.00th=[ 1672], | |
| 80.00th=[ 95744], 90.00th=[ 1368064], 95.00th=[ 14614528], | |
| 99.00th=[145752064], 99.50th=[425721856], 99.90th=[624951296], | |
| 99.95th=[666894336], 99.99th=[750780416] | |
bw ( KiB/s): min= 0, max= 5060, per=7.00%, avg=3292.00, stdev=2500.33, samples=2 | |
iops : min= 0, max= 79, avg=51.00, stdev=39.60, samples=2 | |
write: IOPS=53, BW=3395KiB/s (3477kB/s)(398MiB/120003msec) | |
slat (nsec): min=0, max=19783k, avg=197295.51, stdev=375928.11 | |
clat (nsec): min=0, max=852237k, avg=68838535.89, stdev=129062909.48 | |
lat (nsec): min=0, max=852306k, avg=69037001.18, stdev=129005986.35 | |
clat percentiles (usec): | |
| 1.00th=[ 996], 5.00th=[ 1237], 10.00th=[ 1418], 20.00th=[ 1844], | |
| 30.00th=[ 2442], 40.00th=[ 3458], 50.00th=[ 6718], 60.00th=[ 20841], | |
| 70.00th=[ 58983], 80.00th=[105382], 90.00th=[212861], 95.00th=[371196], | |
| 99.00th=[624952], 99.50th=[675283], 99.90th=[759170], 99.95th=[759170], | |
| 99.99th=[851444] | |
bw ( KiB/s): min= 0, max= 5222, per=73.49%, avg=3392.50, stdev=2587.30, samples=2 | |
iops : min= 0, max= 81, avg=52.50, stdev=40.31, samples=2 | |
lat (nsec) : 750=9.04%, 1000=12.40% | |
lat (usec) : 2=13.32%, 4=0.10%, 10=0.85%, 20=1.64%, 50=1.03% | |
lat (usec) : 100=1.11%, 250=1.43%, 500=0.82%, 750=0.69%, 1000=1.48% | |
lat (msec) : 2=13.19%, 4=10.76%, 10=5.76%, 20=3.66%, 50=5.25% | |
lat (msec) : 100=6.13%, 250=6.62%, 500=3.37%, 750=1.33%, 1000=0.06% | |
cpu : usr=0.08%, sys=0.29%, ctx=13113, majf=0, minf=29 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=6173,6363,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=45.9MiB/s (48.1MB/s), 3292KiB/s-42.7MiB/s (3371kB/s-44.8MB/s), io=5537MiB (5806MB), run=120003-120619msec | |
WRITE: bw=4616KiB/s (4727kB/s), 1236KiB/s-3395KiB/s (1265kB/s-3477kB/s), io=543MiB (570MB), run=120003-120516msec | |
Disk stats (read/write): | |
rbd199: ios=4257/8892, merge=0/1, ticks=531020/899787, in_queue=1424167, util=6.02% | |
<example-benchmark-fio-server-771-benchmark-l7n6n> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:59 2019 | |
write: IOPS=21, BW=1351KiB/s (1384kB/s)(159MiB/120291msec) | |
slat (nsec): min=0, max=412566k, avg=603490.14, stdev=11034782.35 | |
clat (nsec): min=0, max=4347.8M, avg=188943138.26, stdev=309675547.61 | |
lat (nsec): min=0, max=4348.0M, avg=189547874.67, stdev=309681324.16 | |
clat percentiles (usec): | |
| 1.00th=[ 1139], 5.00th=[ 1778], 10.00th=[ 2311], | |
| 20.00th=[ 3687], 30.00th=[ 5669], 40.00th=[ 8979], | |
| 50.00th=[ 15270], 60.00th=[ 120062], 70.00th=[ 258999], | |
| 80.00th=[ 367002], 90.00th=[ 549454], 95.00th=[ 775947], | |
| 99.00th=[1115685], 99.50th=[1300235], 99.90th=[4328522], | |
| 99.95th=[4328522], 99.99th=[4328522] | |
bw ( KiB/s): min= 0, max= 1528, per=28.54%, avg=1351.50, stdev=249.61, samples=2 | |
iops : min= 0, max= 23, avg=20.50, stdev= 3.54, samples=2 | |
lat (usec) : 250=0.08%, 500=0.04%, 750=0.04%, 1000=0.16% | |
lat (msec) : 2=6.86%, 4=15.06%, 10=20.34%, 20=10.21%, 50=5.24% | |
lat (msec) : 100=1.81%, 250=9.18%, 500=19.71%, 750=6.23%, 1000=3.59% | |
cpu : usr=0.04%, sys=0.09%, ctx=4463, majf=0, minf=29 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,2537,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:59 2019 | |
read: IOPS=163, BW=10.2MiB/s (10.7MB/s)(1237MiB/120719msec) | |
slat (nsec): min=0, max=10631k, avg=33850.06, stdev=183395.68 | |
clat (nsec): min=0, max=1826.2M, avg=24367822.86, stdev=129614967.16 | |
lat (nsec): min=0, max=1826.2M, avg=24402732.11, stdev=129638893.04 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 756], | |
| 20.00th=[ 780], 30.00th=[ 804], 40.00th=[ 860], | |
| 50.00th=[ 892], 60.00th=[ 948], 70.00th=[ 1192], | |
| 80.00th=[ 1704], 90.00th=[ 3293184], 95.00th=[ 33423360], | |
| 99.00th=[ 784334848], 99.50th=[ 935329792], 99.90th=[1216348160], | |
| 99.95th=[1317011456], 99.99th=[1686110208] | |
bw ( KiB/s): min= 0, max=11221, per=76.60%, avg=10553.50, stdev=943.99, samples=2 | |
iops : min= 0, max= 175, avg=164.50, stdev=14.85, samples=2 | |
lat (nsec) : 750=9.87%, 1000=54.06% | |
lat (usec) : 2=16.47%, 4=0.22%, 10=0.24%, 20=0.74%, 50=0.15% | |
lat (usec) : 100=0.12%, 250=0.07%, 500=0.09%, 750=0.30%, 1000=1.27% | |
lat (msec) : 2=4.08%, 4=2.97%, 10=2.30%, 20=1.21%, 50=1.45% | |
lat (msec) : 100=0.61%, 250=0.52%, 500=0.91%, 750=1.16%, 1000=0.86% | |
cpu : usr=0.07%, sys=0.21%, ctx=7263, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=19789,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:59 2019 | |
read: IOPS=51, BW=3305KiB/s (3385kB/s)(387MiB/120008msec) | |
slat (nsec): min=0, max=3588.7k, avg=25761.76, stdev=73931.98 | |
clat (nsec): min=0, max=959944k, avg=10211143.33, stdev=51913237.46 | |
lat (nsec): min=0, max=959982k, avg=10237801.22, stdev=51919638.05 | |
clat percentiles (nsec): | |
| 1.00th=[ 724], 5.00th=[ 732], 10.00th=[ 740], | |
| 20.00th=[ 788], 30.00th=[ 876], 40.00th=[ 972], | |
| 50.00th=[ 1112], 60.00th=[ 1256], 70.00th=[ 1512], | |
| 80.00th=[ 216064], 90.00th=[ 3850240], 95.00th=[ 48496640], | |
| 99.00th=[325058560], 99.50th=[387973120], 99.90th=[541065216], | |
| 99.95th=[616562688], 99.99th=[960495616] | |
bw ( KiB/s): min= 0, max= 4802, per=23.99%, avg=3305.50, stdev=2116.37, samples=2 | |
iops : min= 0, max= 75, avg=51.50, stdev=33.23, samples=2 | |
write: IOPS=52, BW=3393KiB/s (3475kB/s)(398MiB/120008msec) | |
slat (nsec): min=0, max=3952.3k, avg=190717.24, stdev=251762.76 | |
clat (nsec): min=0, max=994975k, avg=65641195.08, stdev=133569305.73 | |
lat (nsec): min=0, max=995050k, avg=65833017.35, stdev=133524713.73 | |
clat percentiles (usec): | |
| 1.00th=[ 963], 5.00th=[ 1172], 10.00th=[ 1336], 20.00th=[ 1663], | |
| 30.00th=[ 2073], 40.00th=[ 2835], 50.00th=[ 5211], 60.00th=[ 21365], | |
| 70.00th=[ 64226], 80.00th=[100140], 90.00th=[154141], 95.00th=[329253], | |
| 99.00th=[734004], 99.50th=[834667], 99.90th=[935330], 99.95th=[952108], | |
| 99.99th=[994051] | |
bw ( KiB/s): min= 0, max= 4852, per=71.60%, avg=3391.00, stdev=2066.17, samples=2 | |
iops : min= 0, max= 75, avg=52.50, stdev=31.82, samples=2 | |
lat (nsec) : 750=7.23%, 1000=13.74% | |
lat (usec) : 2=14.70%, 4=0.04%, 10=0.29%, 20=1.01%, 50=1.01% | |
lat (usec) : 100=0.84%, 250=0.68%, 500=0.45%, 750=0.44%, 1000=1.74% | |
lat (msec) : 2=15.66%, 4=10.29%, 10=4.86%, 20=2.95%, 50=4.81% | |
lat (msec) : 100=7.88%, 250=7.40%, 500=2.55%, 750=0.94%, 1000=0.50% | |
cpu : usr=0.07%, sys=0.30%, ctx=13776, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=6198,6360,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=13.5MiB/s (14.1MB/s), 3305KiB/s-10.2MiB/s (3385kB/s-10.7MB/s), io=1624MiB (1703MB), run=120008-120719msec | |
WRITE: bw=4737KiB/s (4850kB/s), 1351KiB/s-3393KiB/s (1384kB/s-3475kB/s), io=556MiB (583MB), run=120008-120291msec | |
Disk stats (read/write): | |
rbd230: ios=5135/9900, merge=0/2, ticks=595516/887454, in_queue=1475455, util=4.69% | |
Jobs: 37 (f=37): [_(2),m(1)][99.7%][r=1206M,w=1205M][r=18.8k,w=18.8k IOPS][eta 00m:51s] <example-benchmark-fio-server-372-benchmark-lc6p4> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:59 2019 | |
write: IOPS=25, BW=1650KiB/s (1689kB/s)(194MiB/120291msec) | |
slat (nsec): min=0, max=773509k, avg=852811.07, stdev=20150575.43 | |
clat (nsec): min=0, max=2310.7M, avg=154359745.14, stdev=327539286.90 | |
lat (nsec): min=0, max=2310.8M, avg=155213400.86, stdev=327842778.04 | |
clat percentiles (usec): | |
| 1.00th=[ 1037], 5.00th=[ 1352], 10.00th=[ 1598], | |
| 20.00th=[ 2114], 30.00th=[ 2900], 40.00th=[ 3851], | |
| 50.00th=[ 5276], 60.00th=[ 8160], 70.00th=[ 17171], | |
| 80.00th=[ 105382], 90.00th=[ 675283], 95.00th=[ 893387], | |
| 99.00th=[1333789], 99.50th=[1484784], 99.90th=[2021655], | |
| 99.95th=[2264925], 99.99th=[2298479] | |
bw ( KiB/s): min= 0, max= 2336, per=36.05%, avg=1650.50, stdev=969.44, samples=2 | |
iops : min= 0, max= 36, avg=25.50, stdev=14.85, samples=2 | |
lat (usec) : 500=0.10%, 750=0.13%, 1000=0.55% | |
lat (msec) : 2=17.43%, 4=22.82%, 10=23.08%, 20=7.00%, 50=3.52% | |
lat (msec) : 100=5.07%, 250=1.94%, 500=2.61%, 750=8.20%, 1000=4.07% | |
cpu : usr=0.03%, sys=0.12%, ctx=5780, majf=0, minf=27 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,3098,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:59 2019 | |
read: IOPS=610, BW=38.0MiB/s (39.9MB/s)(4588MiB/120573msec) | |
slat (nsec): min=0, max=771813k, avg=42177.78, stdev=4068032.92 | |
clat (nsec): min=0, max=1795.7M, avg=6503281.36, stdev=66799159.40 | |
lat (nsec): min=0, max=1795.8M, avg=6546418.17, stdev=66930745.83 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 764], 30.00th=[ 780], 40.00th=[ 796], | |
| 50.00th=[ 828], 60.00th=[ 860], 70.00th=[ 884], | |
| 80.00th=[ 972], 90.00th=[ 11840], 95.00th=[ 995328], | |
| 99.00th=[ 91750400], 99.50th=[ 599785472], 99.90th=[ 952107008], | |
| 99.95th=[1115684864], 99.99th=[1400897536] | |
bw ( KiB/s): min= 0, max=63106, per=93.37%, avg=39021.00, stdev=34061.33, samples=2 | |
iops : min= 0, max= 990, avg=611.50, stdev=535.28, samples=2 | |
lat (nsec) : 750=15.86%, 1000=64.91% | |
lat (usec) : 2=5.86%, 4=0.14%, 10=1.15%, 20=5.13%, 50=0.69% | |
lat (usec) : 100=0.19%, 250=0.10%, 500=0.19%, 750=0.28%, 1000=0.49% | |
lat (msec) : 2=1.20%, 4=1.00%, 10=0.73%, 20=0.33%, 50=0.46% | |
lat (msec) : 100=0.31%, 250=0.10%, 500=0.23%, 750=0.38%, 1000=0.18% | |
cpu : usr=0.15%, sys=0.61%, ctx=8969, majf=0, minf=31 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=73655,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:59 2019 | |
read: IOPS=44, BW=2840KiB/s (2908kB/s)(333MiB/120025msec) | |
slat (nsec): min=0, max=2616.8k, avg=30791.64, stdev=88068.41 | |
clat (nsec): min=0, max=642034k, avg=7264005.10, stdev=39713153.45 | |
lat (nsec): min=0, max=642063k, avg=7295930.78, stdev=39719952.99 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 788], 30.00th=[ 876], 40.00th=[ 988], | |
| 50.00th=[ 1144], 60.00th=[ 1304], 70.00th=[ 9920], | |
| 80.00th=[ 684032], 90.00th=[ 2899968], 95.00th=[ 41680896], | |
| 99.00th=[181403648], 99.50th=[362807296], 99.90th=[501219328], | |
| 99.95th=[530579456], 99.99th=[641728512] | |
bw ( KiB/s): min= 0, max= 4536, per=6.80%, avg=2840.00, stdev=2398.51, samples=2 | |
iops : min= 0, max= 70, avg=43.50, stdev=37.48, samples=2 | |
write: IOPS=45, BW=2935KiB/s (3005kB/s)(344MiB/120025msec) | |
slat (nsec): min=0, max=5247.4k, avg=183124.89, stdev=260290.31 | |
clat (nsec): min=0, max=922637k, avg=80167211.55, stdev=145551293.96 | |
lat (nsec): min=0, max=922680k, avg=80351609.68, stdev=145505979.46 | |
clat percentiles (usec): | |
| 1.00th=[ 1029], 5.00th=[ 1237], 10.00th=[ 1434], 20.00th=[ 1860], | |
| 30.00th=[ 2573], 40.00th=[ 3916], 50.00th=[ 8979], 60.00th=[ 28705], | |
| 70.00th=[ 68682], 80.00th=[113771], 90.00th=[278922], 95.00th=[450888], | |
| 99.00th=[650118], 99.50th=[725615], 99.90th=[859833], 99.95th=[918553], | |
| 99.99th=[918553] | |
bw ( KiB/s): min= 0, max= 4651, per=64.06%, avg=2932.50, stdev=2430.33, samples=2 | |
iops : min= 0, max= 72, avg=45.00, stdev=38.18, samples=2 | |
lat (nsec) : 750=7.63%, 1000=12.33% | |
lat (usec) : 2=13.99%, 4=0.14%, 10=0.38%, 20=0.85%, 50=0.85% | |
lat (usec) : 100=0.89%, 250=1.16%, 500=0.70%, 750=0.79%, 1000=1.73% | |
lat (msec) : 2=13.41%, 4=10.59%, 10=5.92%, 20=3.28%, 50=5.86% | |
lat (msec) : 100=7.03%, 250=6.65%, 500=3.85%, 750=1.83%, 1000=0.17% | |
cpu : usr=0.06%, sys=0.26%, ctx=12141, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=5326,5501,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=40.8MiB/s (42.8MB/s), 2840KiB/s-38.0MiB/s (2908kB/s-39.9MB/s), io=4921MiB (5160MB), run=120025-120573msec | |
WRITE: bw=4578KiB/s (4688kB/s), 1650KiB/s-2935KiB/s (1689kB/s-3005kB/s), io=538MiB (564MB), run=120025-120291msec | |
Disk stats (read/write): | |
rbd130: ios=6082/9151, merge=0/1, ticks=546406/923365, in_queue=1462130, util=7.65% | |
<example-benchmark-fio-server-452-benchmark-7lt6f> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:59 2019 | |
write: IOPS=45, BW=2916KiB/s (2986kB/s)(346MiB/121551msec) | |
slat (nsec): min=0, max=8601.9M, avg=1768759.47, stdev=115624208.02 | |
clat (nsec): min=0, max=11024M, avg=85317961.64, stdev=306099695.43 | |
lat (nsec): min=0, max=11024M, avg=87087195.73, stdev=327190446.01 | |
clat percentiles (usec): | |
| 1.00th=[ 914], 5.00th=[ 1745], 10.00th=[ 2507], | |
| 20.00th=[ 3720], 30.00th=[ 5145], 40.00th=[ 7504], | |
| 50.00th=[ 10421], 60.00th=[ 15533], 70.00th=[ 29492], | |
| 80.00th=[ 102237], 90.00th=[ 219153], 95.00th=[ 484443], | |
| 99.00th=[ 759170], 99.50th=[ 935330], 99.90th=[ 3305112], | |
| 99.95th=[ 8657044], 99.99th=[11072963] | |
bw ( KiB/s): min= 0, max= 3061, per=47.12%, avg=2950.00, stdev=156.98, samples=2 | |
iops : min= 0, max= 47, avg=45.50, stdev= 2.12, samples=2 | |
lat (usec) : 4=0.04%, 50=0.02%, 250=0.07%, 500=0.04%, 750=0.33% | |
lat (usec) : 1000=0.89% | |
lat (msec) : 2=5.53%, 4=15.37%, 10=26.41%, 20=16.22%, 50=9.83% | |
lat (msec) : 100=5.17%, 250=11.51%, 500=3.97%, 750=3.63%, 1000=0.63% | |
cpu : usr=0.05%, sys=0.19%, ctx=9026, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,5535,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:59 2019 | |
read: IOPS=79, BW=5087KiB/s (5210kB/s)(599MiB/120566msec) | |
slat (nsec): min=0, max=11811k, avg=55106.52, stdev=241135.38 | |
clat (nsec): min=0, max=1838.1M, avg=50275141.81, stdev=187121374.36 | |
lat (nsec): min=0, max=1838.1M, avg=50331333.59, stdev=187141422.67 | |
clat percentiles (nsec): | |
| 1.00th=[ 724], 5.00th=[ 764], 10.00th=[ 788], | |
| 20.00th=[ 852], 30.00th=[ 908], 40.00th=[ 1012], | |
| 50.00th=[ 1256], 60.00th=[ 1512], 70.00th=[ 1138688], | |
| 80.00th=[ 2932736], 90.00th=[ 15400960], 95.00th=[ 549453824], | |
| 99.00th=[ 935329792], 99.50th=[1052770304], 99.90th=[1350565888], | |
| 99.95th=[1585446912], 99.99th=[1837105152] | |
bw ( KiB/s): min= 0, max= 5128, per=61.11%, avg=5102.50, stdev=36.06, samples=2 | |
iops : min= 0, max= 80, avg=79.50, stdev= 0.71, samples=2 | |
lat (nsec) : 750=4.34%, 1000=35.13% | |
lat (usec) : 2=23.43%, 4=0.21%, 10=0.10%, 20=0.85%, 50=0.27% | |
lat (usec) : 100=0.21%, 250=0.15%, 500=0.19%, 750=0.66%, 1000=3.03% | |
lat (msec) : 2=7.55%, 4=6.70%, 10=5.32%, 20=2.76%, 50=1.40% | |
lat (msec) : 100=0.45%, 250=0.47%, 500=1.35%, 750=2.96%, 1000=1.87% | |
cpu : usr=0.04%, sys=0.14%, ctx=6779, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=9581,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:59 2019 | |
read: IOPS=51, BW=3278KiB/s (3357kB/s)(384MiB/120011msec) | |
slat (nsec): min=0, max=4415.6k, avg=40057.69, stdev=108722.41 | |
clat (nsec): min=0, max=810846k, avg=15542310.91, stdev=68280778.67 | |
lat (nsec): min=0, max=810878k, avg=15583357.86, stdev=68290237.45 | |
clat percentiles (nsec): | |
| 1.00th=[ 724], 5.00th=[ 740], 10.00th=[ 788], | |
| 20.00th=[ 884], 30.00th=[ 996], 40.00th=[ 1144], | |
| 50.00th=[ 1288], 60.00th=[ 14016], 70.00th=[ 954368], | |
| 80.00th=[ 2179072], 90.00th=[ 31850496], 95.00th=[ 64749568], | |
| 99.00th=[400556032], 99.50th=[574619648], 99.90th=[717225984], | |
| 99.95th=[725614592], 99.99th=[809500672] | |
bw ( KiB/s): min= 0, max= 5222, per=39.26%, avg=3278.00, stdev=2749.23, samples=2 | |
iops : min= 0, max= 81, avg=50.50, stdev=43.13, samples=2 | |
write: IOPS=52, BW=3387KiB/s (3469kB/s)(397MiB/120011msec) | |
slat (nsec): min=0, max=11202k, avg=195281.65, stdev=275030.68 | |
clat (nsec): min=0, max=980601k, avg=60513733.50, stdev=123539267.98 | |
lat (nsec): min=0, max=980657k, avg=60710238.12, stdev=123494453.74 | |
clat percentiles (usec): | |
| 1.00th=[ 1012], 5.00th=[ 1221], 10.00th=[ 1385], 20.00th=[ 1745], | |
| 30.00th=[ 2278], 40.00th=[ 3195], 50.00th=[ 5342], 60.00th=[ 14615], | |
| 70.00th=[ 45351], 80.00th=[ 87557], 90.00th=[154141], 95.00th=[367002], | |
| 99.00th=[574620], 99.50th=[658506], 99.90th=[884999], 99.95th=[943719], | |
| 99.99th=[977273] | |
bw ( KiB/s): min= 0, max= 5397, per=54.07%, avg=3385.00, stdev=2845.40, samples=2 | |
iops : min= 0, max= 84, avg=52.50, stdev=44.55, samples=2 | |
lat (nsec) : 750=3.39%, 1000=11.52% | |
lat (usec) : 2=14.24%, 4=0.07%, 10=0.18%, 20=0.42%, 50=0.50% | |
lat (usec) : 100=0.38%, 250=0.55%, 500=0.29%, 750=0.99%, 1000=2.86% | |
lat (msec) : 2=16.39%, 4=12.80%, 10=7.15%, 20=3.89%, 50=6.17% | |
lat (msec) : 100=8.40%, 250=5.23%, 500=3.12%, 750=1.36%, 1000=0.14% | |
cpu : usr=0.08%, sys=0.31%, ctx=16088, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=6146,6350,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=8350KiB/s (8551kB/s), 3278KiB/s-5087KiB/s (3357kB/s-5210kB/s), io=983MiB (1031MB), run=120011-120566msec | |
WRITE: bw=6260KiB/s (6411kB/s), 2916KiB/s-3387KiB/s (2986kB/s-3469kB/s), io=743MiB (779MB), run=120011-121551msec | |
Disk stats (read/write): | |
rbd166: ios=6007/12363, merge=0/3, ticks=619271/875615, in_queue=1485609, util=4.64% | |
<example-benchmark-fio-server-170-benchmark-7ll8b> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:59 2019 | |
write: IOPS=25, BW=1658KiB/s (1698kB/s)(195MiB/120385msec) | |
slat (nsec): min=0, max=887718k, avg=1213744.90, stdev=27334605.55 | |
clat (nsec): min=0, max=2050.6M, avg=153222238.41, stdev=315111553.22 | |
lat (nsec): min=0, max=2050.6M, avg=154436698.50, stdev=316707758.24 | |
clat percentiles (usec): | |
| 1.00th=[ 1156], 5.00th=[ 1483], 10.00th=[ 1827], | |
| 20.00th=[ 2474], 30.00th=[ 3392], 40.00th=[ 5080], | |
| 50.00th=[ 7504], 60.00th=[ 13173], 70.00th=[ 31851], | |
| 80.00th=[ 121111], 90.00th=[ 675283], 95.00th=[ 884999], | |
| 99.00th=[1283458], 99.50th=[1384121], 99.90th=[1887437], | |
| 99.95th=[2021655], 99.99th=[2055209] | |
bw ( KiB/s): min= 0, max= 1787, per=32.78%, avg=1660.50, stdev=178.90, samples=2 | |
iops : min= 0, max= 27, avg=25.00, stdev= 2.83, samples=2 | |
lat (usec) : 2=0.03%, 750=0.16%, 1000=0.19% | |
lat (msec) : 2=12.45%, 4=21.57%, 10=21.57%, 20=9.34%, 50=7.64% | |
lat (msec) : 100=5.49%, 250=3.59%, 500=2.86%, 750=7.54%, 1000=4.40% | |
cpu : usr=0.04%, sys=0.12%, ctx=5748, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,3116,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:59 2019 | |
read: IOPS=671, BW=41.8MiB/s (43.8MB/s)(5034MiB/120435msec) | |
slat (nsec): min=0, max=880995k, avg=42469.66, stdev=4585790.36 | |
clat (nsec): min=0, max=1644.8M, avg=5920176.44, stdev=63524581.82 | |
lat (nsec): min=0, max=1644.8M, avg=5963596.00, stdev=63696049.76 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 724], | |
| 20.00th=[ 764], 30.00th=[ 780], 40.00th=[ 804], | |
| 50.00th=[ 844], 60.00th=[ 876], 70.00th=[ 908], | |
| 80.00th=[ 1480], 90.00th=[ 12864], 95.00th=[ 757760], | |
| 99.00th=[ 63176704], 99.50th=[ 574619648], 99.90th=[ 952107008], | |
| 99.95th=[1044381696], 99.99th=[1300234240] | |
bw ( KiB/s): min= 0, max=69975, per=93.01%, avg=42879.00, stdev=38319.53, samples=2 | |
iops : min= 0, max= 1098, avg=672.00, stdev=602.45, samples=2 | |
lat (nsec) : 750=16.86%, 1000=58.14% | |
lat (usec) : 2=5.53%, 4=0.13%, 10=1.32%, 20=9.56%, 50=2.74% | |
lat (usec) : 100=0.26%, 250=0.13%, 500=0.15%, 750=0.17%, 1000=0.53% | |
lat (msec) : 2=1.20%, 4=0.87%, 10=0.59%, 20=0.30%, 50=0.40% | |
lat (msec) : 100=0.25%, 250=0.07%, 500=0.22%, 750=0.34%, 1000=0.18% | |
cpu : usr=0.16%, sys=0.67%, ctx=8878, majf=0, minf=34 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=80815,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:59 2019 | |
read: IOPS=51, BW=3314KiB/s (3394kB/s)(389MiB/120035msec) | |
slat (nsec): min=0, max=1675.7k, avg=27582.43, stdev=64411.66 | |
clat (nsec): min=0, max=694594k, avg=8549233.54, stdev=46985004.43 | |
lat (nsec): min=0, max=694628k, avg=8577802.47, stdev=46991864.10 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 756], 30.00th=[ 860], 40.00th=[ 964], | |
| 50.00th=[ 1112], 60.00th=[ 1256], 70.00th=[ 1656], | |
| 80.00th=[ 337920], 90.00th=[ 2801664], 95.00th=[ 39583744], | |
| 99.00th=[256901120], 99.50th=[387973120], 99.90th=[526385152], | |
| 99.95th=[624951296], 99.99th=[692060160] | |
bw ( KiB/s): min= 0, max= 5128, per=7.19%, avg=3314.50, stdev=2564.68, samples=2 | |
iops : min= 0, max= 80, avg=51.50, stdev=40.31, samples=2 | |
write: IOPS=53, BW=3418KiB/s (3500kB/s)(401MiB/120035msec) | |
slat (nsec): min=0, max=42017k, avg=186096.19, stdev=597925.65 | |
clat (nsec): min=0, max=1006.8M, avg=66420593.68, stdev=125702557.93 | |
lat (nsec): min=0, max=1006.9M, avg=66607957.18, stdev=125661614.59 | |
clat percentiles (usec): | |
| 1.00th=[ 1029], 5.00th=[ 1237], 10.00th=[ 1450], | |
| 20.00th=[ 1827], 30.00th=[ 2311], 40.00th=[ 3294], | |
| 50.00th=[ 6521], 60.00th=[ 22938], 70.00th=[ 61604], | |
| 80.00th=[ 105382], 90.00th=[ 173016], 95.00th=[ 358613], | |
| 99.00th=[ 633340], 99.50th=[ 692061], 99.90th=[ 801113], | |
| 99.95th=[ 884999], 99.99th=[1010828] | |
bw ( KiB/s): min= 0, max= 5280, per=67.45%, avg=3416.50, stdev=2635.39, samples=2 | |
iops : min= 0, max= 82, avg=53.00, stdev=41.01, samples=2 | |
lat (nsec) : 750=9.02%, 1000=11.91% | |
lat (usec) : 2=13.84%, 4=0.06%, 10=0.25%, 20=1.00%, 50=0.92% | |
lat (usec) : 100=1.05%, 250=1.01%, 500=0.73%, 750=0.59%, 1000=1.78% | |
lat (msec) : 2=13.49%, 4=11.63%, 10=5.38%, 20=3.46%, 50=5.36% | |
lat (msec) : 100=6.70%, 250=7.63%, 500=2.93%, 750=1.18%, 1000=0.10% | |
cpu : usr=0.07%, sys=0.30%, ctx=13879, majf=0, minf=33 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=6215,6408,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=45.0MiB/s (47.2MB/s), 3314KiB/s-41.8MiB/s (3394kB/s-43.8MB/s), io=5422MiB (5686MB), run=120035-120435msec | |
WRITE: bw=5066KiB/s (5187kB/s), 1658KiB/s-3418KiB/s (1698kB/s-3500kB/s), io=596MiB (624MB), run=120035-120385msec | |
Disk stats (read/write): | |
rbd54: ios=5720/9776, merge=0/1, ticks=553989/877423, in_queue=1423562, util=7.76% | |
<example-benchmark-fio-server-305-benchmark-bvd8p> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:59 2019 | |
write: IOPS=46, BW=2967KiB/s (3038kB/s)(348MiB/120084msec) | |
slat (nsec): min=0, max=591926k, avg=385076.58, stdev=10580791.50 | |
clat (nsec): min=0, max=1249.9M, avg=86328692.16, stdev=149286241.82 | |
lat (nsec): min=0, max=1249.0M, avg=86715682.26, stdev=149627952.29 | |
clat percentiles (usec): | |
| 1.00th=[ 1004], 5.00th=[ 1237], 10.00th=[ 1467], | |
| 20.00th=[ 1958], 30.00th=[ 2868], 40.00th=[ 4686], | |
| 50.00th=[ 11076], 60.00th=[ 53216], 70.00th=[ 90702], | |
| 80.00th=[ 123208], 90.00th=[ 258999], 95.00th=[ 408945], | |
| 99.00th=[ 708838], 99.50th=[ 809501], 99.90th=[1052771], | |
| 99.95th=[1233126], 99.99th=[1249903] | |
bw ( KiB/s): min= 0, max= 3388, per=45.08%, avg=2966.50, stdev=596.09, samples=2 | |
iops : min= 0, max= 52, avg=45.50, stdev= 9.19, samples=2 | |
lat (usec) : 4=0.02%, 250=0.02%, 750=0.02%, 1000=0.93% | |
lat (msec) : 2=19.57%, 4=16.80%, 10=11.79%, 20=4.85%, 50=5.59% | |
lat (msec) : 100=13.28%, 250=16.71%, 500=7.08%, 750=2.50%, 1000=0.74% | |
cpu : usr=0.05%, sys=0.21%, ctx=10453, majf=0, minf=29 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,5564,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:59 2019 | |
read: IOPS=316, BW=19.7MiB/s (20.7MB/s)(2370MiB/120032msec) | |
slat (nsec): min=0, max=590286k, avg=50581.37, stdev=3970019.69 | |
clat (nsec): min=0, max=1148.2M, avg=12614914.63, stdev=60910243.05 | |
lat (nsec): min=0, max=1148.2M, avg=12666319.66, stdev=61045819.97 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 708], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 732], 40.00th=[ 748], | |
| 50.00th=[ 836], 60.00th=[ 892], 70.00th=[ 1112], | |
| 80.00th=[ 18560], 90.00th=[ 9371648], 95.00th=[ 59506688], | |
| 99.00th=[ 350224384], 99.50th=[ 476053504], 99.90th=[ 692060160], | |
| 99.95th=[ 750780416], 99.99th=[1010827264] | |
bw ( KiB/s): min= 0, max=28456, per=85.14%, avg=20222.00, stdev=11644.63, samples=2 | |
iops : min= 0, max= 444, avg=315.50, stdev=181.73, samples=2 | |
lat (nsec) : 750=40.78%, 1000=26.21% | |
lat (usec) : 2=10.59%, 4=0.03%, 10=1.41%, 20=1.02%, 50=0.30% | |
lat (usec) : 100=0.13%, 250=0.06%, 500=0.15%, 750=0.86%, 1000=2.15% | |
lat (msec) : 2=3.24%, 4=1.98%, 10=1.16%, 20=0.65%, 50=3.31% | |
lat (msec) : 100=2.89%, 250=1.61%, 500=1.07%, 750=0.37%, 1000=0.03% | |
cpu : usr=0.09%, sys=0.37%, ctx=14721, majf=0, minf=29 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=37938,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:59 2019 | |
read: IOPS=55, BW=3535KiB/s (3620kB/s)(414MiB/120039msec) | |
slat (nsec): min=0, max=484899k, avg=107021.58, stdev=5955708.76 | |
clat (nsec): min=0, max=759194k, avg=6801948.12, stdev=36574853.34 | |
lat (nsec): min=0, max=759246k, avg=6909910.12, stdev=37050983.59 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 796], 30.00th=[ 892], 40.00th=[ 1004], | |
| 50.00th=[ 1160], 60.00th=[ 1320], 70.00th=[ 15040], | |
| 80.00th=[ 888832], 90.00th=[ 4145152], 95.00th=[ 37486592], | |
| 99.00th=[170917888], 99.50th=[258998272], 99.90th=[476053504], | |
| 99.95th=[608174080], 99.99th=[759169024] | |
bw ( KiB/s): min= 0, max= 4618, per=14.88%, avg=3535.00, stdev=1531.59, samples=2 | |
iops : min= 0, max= 72, avg=55.00, stdev=24.04, samples=2 | |
write: IOPS=56, BW=3615KiB/s (3702kB/s)(424MiB/120039msec) | |
slat (nsec): min=0, max=589992k, avg=284036.47, stdev=7194313.68 | |
clat (nsec): min=0, max=977765k, avg=64022066.99, stdev=122762405.28 | |
lat (nsec): min=0, max=1018.0M, avg=64307305.36, stdev=123184191.18 | |
clat percentiles (usec): | |
| 1.00th=[ 1057], 5.00th=[ 1287], 10.00th=[ 1483], 20.00th=[ 1926], | |
| 30.00th=[ 2573], 40.00th=[ 3785], 50.00th=[ 6849], 60.00th=[ 17957], | |
| 70.00th=[ 53216], 80.00th=[ 98042], 90.00th=[187696], 95.00th=[350225], | |
| 99.00th=[591397], 99.50th=[675283], 99.90th=[859833], 99.95th=[884999], | |
| 99.99th=[977273] | |
bw ( KiB/s): min= 0, max= 4677, per=54.92%, avg=3614.00, stdev=1503.31, samples=2 | |
iops : min= 0, max= 73, avg=56.00, stdev=24.04, samples=2 | |
lat (nsec) : 750=7.62%, 1000=12.02% | |
lat (usec) : 2=14.45%, 4=0.07%, 10=0.25%, 20=0.67%, 50=0.63% | |
lat (usec) : 100=0.78%, 250=0.84%, 500=0.75%, 750=0.63%, 1000=1.73% | |
lat (msec) : 2=13.27%, 4=11.49%, 10=8.05%, 20=3.89%, 50=5.53% | |
lat (msec) : 100=6.67%, 250=6.58%, 500=2.96%, 750=1.01%, 1000=0.12% | |
cpu : usr=0.06%, sys=0.33%, ctx=15762, majf=0, minf=36 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=6631,6777,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=23.2MiB/s (24.3MB/s), 3535KiB/s-19.7MiB/s (3620kB/s-20.7MB/s), io=2784MiB (2920MB), run=120032-120039msec | |
WRITE: bw=6580KiB/s (6738kB/s), 2967KiB/s-3615KiB/s (3038kB/s-3702kB/s), io=772MiB (809MB), run=120039-120084msec | |
Disk stats (read/write): | |
rbd105: ios=9005/12689, merge=0/1, ticks=537857/830714, in_queue=1357530, util=14.73% | |
<example-benchmark-fio-server-214-benchmark-g26rt> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:59 2019 | |
write: IOPS=39, BW=2501KiB/s (2561kB/s)(293MiB/120100msec) | |
slat (nsec): min=0, max=878508k, avg=512354.66, stdev=14978981.02 | |
clat (nsec): min=0, max=1750.0M, avg=102508712.84, stdev=189336994.89 | |
lat (nsec): min=0, max=1750.1M, avg=103022267.99, stdev=189729647.89 | |
clat percentiles (usec): | |
| 1.00th=[ 1045], 5.00th=[ 1237], 10.00th=[ 1434], | |
| 20.00th=[ 1942], 30.00th=[ 2835], 40.00th=[ 4752], | |
| 50.00th=[ 10552], 60.00th=[ 51643], 70.00th=[ 103285], | |
| 80.00th=[ 139461], 90.00th=[ 304088], 95.00th=[ 549454], | |
| 99.00th=[ 843056], 99.50th=[1010828], 99.90th=[1602225], | |
| 99.95th=[1702888], 99.99th=[1753220] | |
bw ( KiB/s): min= 0, max= 1899, per=33.04%, avg=1899.00, stdev= 0.00, samples=1 | |
iops : min= 0, max= 29, avg=29.00, stdev= 0.00, samples=1 | |
lat (usec) : 500=0.02%, 750=0.02%, 1000=0.58% | |
lat (msec) : 2=20.34%, 4=16.12%, 10=12.30%, 20=5.71%, 50=4.80% | |
lat (msec) : 100=9.08%, 250=19.02%, 500=6.48%, 750=3.67%, 1000=1.36% | |
cpu : usr=0.05%, sys=0.17%, ctx=8743, majf=0, minf=29 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,4690,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:59 2019 | |
read: IOPS=371, BW=23.2MiB/s (24.3MB/s)(2787MiB/120081msec) | |
slat (nsec): min=0, max=864080k, avg=51361.43, stdev=4763726.52 | |
clat (nsec): min=0, max=1359.3M, avg=10722193.91, stdev=64689666.11 | |
lat (nsec): min=0, max=1359.4M, avg=10774402.02, stdev=64870471.03 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 716], 10.00th=[ 724], | |
| 20.00th=[ 732], 30.00th=[ 764], 40.00th=[ 804], | |
| 50.00th=[ 844], 60.00th=[ 884], 70.00th=[ 940], | |
| 80.00th=[ 1160], 90.00th=[ 1105920], 95.00th=[ 41680896], | |
| 99.00th=[ 350224384], 99.50th=[ 497025024], 99.90th=[ 817889280], | |
| 99.95th=[ 918552576], 99.99th=[1166016512] | |
bw ( KiB/s): min= 0, max=36891, per=88.40%, avg=23777.50, stdev=18545.29, samples=2 | |
iops : min= 0, max= 577, avg=371.50, stdev=290.62, samples=2 | |
lat (nsec) : 750=27.86%, 1000=47.00% | |
lat (usec) : 2=10.92%, 4=0.06%, 10=0.77%, 20=0.56%, 50=0.15% | |
lat (usec) : 100=0.09%, 250=0.05%, 500=0.21%, 750=0.58%, 1000=1.35% | |
lat (msec) : 2=2.12%, 4=1.40%, 10=0.69%, 20=0.34%, 50=1.35% | |
lat (msec) : 100=1.96%, 250=1.11%, 500=0.95%, 750=0.32%, 1000=0.14% | |
cpu : usr=0.12%, sys=0.38%, ctx=11228, majf=0, minf=108 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=44650,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:59 2019 | |
read: IOPS=49, BW=3136KiB/s (3212kB/s)(368MiB/120005msec) | |
slat (nsec): min=0, max=1914.8k, avg=31155.07, stdev=74360.36 | |
clat (nsec): min=0, max=744812k, avg=9110926.27, stdev=44539320.61 | |
lat (nsec): min=0, max=744837k, avg=9143035.67, stdev=44544914.81 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 732], 10.00th=[ 740], | |
| 20.00th=[ 828], 30.00th=[ 908], 40.00th=[ 1032], | |
| 50.00th=[ 1160], 60.00th=[ 1304], 70.00th=[ 19584], | |
| 80.00th=[ 987136], 90.00th=[ 7176192], 95.00th=[ 50069504], | |
| 99.00th=[233832448], 99.50th=[341835776], 99.90th=[574619648], | |
| 99.95th=[616562688], 99.99th=[742391808] | |
bw ( KiB/s): min= 0, max= 4258, per=11.66%, avg=3136.00, stdev=1586.75, samples=2 | |
iops : min= 0, max= 66, avg=48.50, stdev=24.75, samples=2 | |
write: IOPS=50, BW=3250KiB/s (3328kB/s)(381MiB/120005msec) | |
slat (nsec): min=0, max=878437k, avg=346407.87, stdev=11510788.20 | |
clat (nsec): min=0, max=1737.6M, avg=69625248.52, stdev=133022418.53 | |
lat (nsec): min=0, max=1737.7M, avg=69972917.80, stdev=134278546.34 | |
clat percentiles (usec): | |
| 1.00th=[ 1045], 5.00th=[ 1287], 10.00th=[ 1516], | |
| 20.00th=[ 2057], 30.00th=[ 3097], 40.00th=[ 5145], | |
| 50.00th=[ 10683], 60.00th=[ 24773], 70.00th=[ 70779], | |
| 80.00th=[ 105382], 90.00th=[ 175113], 95.00th=[ 337642], | |
| 99.00th=[ 700449], 99.50th=[ 742392], 99.90th=[ 884999], | |
| 99.95th=[ 935330], 99.99th=[1736442] | |
bw ( KiB/s): min= 0, max= 4382, per=56.51%, avg=3248.00, stdev=1603.72, samples=2 | |
iops : min= 0, max= 68, avg=50.50, stdev=24.75, samples=2 | |
lat (nsec) : 750=5.64%, 1000=12.77% | |
lat (usec) : 2=15.11%, 4=0.10%, 10=0.12%, 20=0.71%, 50=0.65% | |
lat (usec) : 100=0.73%, 250=0.74%, 500=0.56%, 750=0.63%, 1000=1.97% | |
lat (msec) : 2=12.09%, 4=9.67%, 10=8.15%, 20=5.10%, 50=5.26% | |
lat (msec) : 100=7.80%, 250=8.05%, 500=2.76%, 750=1.18%, 1000=0.21% | |
cpu : usr=0.07%, sys=0.29%, ctx=14274, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=5882,6091,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=26.3MiB/s (27.5MB/s), 3136KiB/s-23.2MiB/s (3212kB/s-24.3MB/s), io=3154MiB (3307MB), run=120005-120081msec | |
WRITE: bw=5748KiB/s (5886kB/s), 2501KiB/s-3250KiB/s (2561kB/s-3328kB/s), io=674MiB (707MB), run=120005-120100msec | |
Disk stats (read/write): | |
rbd60: ios=7813/11452, merge=0/1, ticks=560837/843308, in_queue=1394412, util=12.65% | |
<example-benchmark-fio-server-573-benchmark-29b45> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:59 2019 | |
write: IOPS=44, BW=2846KiB/s (2915kB/s)(334MiB/120177msec) | |
slat (nsec): min=0, max=820038k, avg=481068.24, stdev=13479085.23 | |
clat (nsec): min=0, max=1048.1M, avg=89624595.19, stdev=149912487.94 | |
lat (nsec): min=0, max=1048.2M, avg=90106805.68, stdev=150298868.66 | |
clat percentiles (usec): | |
| 1.00th=[ 988], 5.00th=[ 1205], 10.00th=[ 1385], | |
| 20.00th=[ 1893], 30.00th=[ 2868], 40.00th=[ 5735], | |
| 50.00th=[ 16581], 60.00th=[ 74974], 70.00th=[ 102237], | |
| 80.00th=[ 125305], 90.00th=[ 235930], 95.00th=[ 425722], | |
| 99.00th=[ 725615], 99.50th=[ 843056], 99.90th=[ 935330], | |
| 99.95th=[ 943719], 99.99th=[1044382] | |
bw ( KiB/s): min= 0, max= 3072, per=44.56%, avg=2848.50, stdev=316.08, samples=2 | |
iops : min= 0, max= 48, avg=44.50, stdev= 4.95, samples=2 | |
lat (usec) : 4=0.04%, 100=0.02%, 250=0.02%, 500=0.02%, 750=0.06% | |
lat (usec) : 1000=1.03% | |
lat (msec) : 2=20.57%, 4=13.91%, 10=9.88%, 20=5.93%, 50=4.98% | |
lat (msec) : 100=12.56%, 250=21.47%, 500=5.88%, 750=2.85%, 1000=0.80% | |
cpu : usr=0.06%, sys=0.19%, ctx=10059, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,5342,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:59 2019 | |
read: IOPS=326, BW=20.4MiB/s (21.4MB/s)(2449MiB/120025msec) | |
slat (nsec): min=0, max=690834k, avg=54024.69, stdev=4310215.39 | |
clat (nsec): min=0, max=1444.3M, avg=12206634.97, stdev=63174388.43 | |
lat (nsec): min=0, max=1444.4M, avg=12261505.28, stdev=63326547.19 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 732], 40.00th=[ 748], | |
| 50.00th=[ 828], 60.00th=[ 876], 70.00th=[ 948], | |
| 80.00th=[ 1304], 90.00th=[ 2670592], 95.00th=[ 56885248], | |
| 99.00th=[ 362807296], 99.50th=[ 484442112], 99.90th=[ 734003200], | |
| 99.95th=[ 809500672], 99.99th=[1010827264] | |
bw ( KiB/s): min= 0, max=30908, per=85.83%, avg=20894.50, stdev=14161.23, samples=2 | |
iops : min= 0, max= 484, avg=327.00, stdev=222.03, samples=2 | |
lat (nsec) : 750=40.44%, 1000=32.16% | |
lat (usec) : 2=9.98%, 4=0.05%, 10=0.85%, 20=0.80%, 50=0.18% | |
lat (usec) : 100=0.07%, 250=0.05%, 500=0.11%, 750=0.68%, 1000=1.57% | |
lat (msec) : 2=2.35%, 4=1.47%, 10=0.79%, 20=0.54%, 50=2.30% | |
lat (msec) : 100=2.60%, 250=1.44%, 500=1.10%, 750=0.38%, 1000=0.07% | |
cpu : usr=0.10%, sys=0.35%, ctx=12084, majf=0, minf=44 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=39241,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=14: Wed May 8 20:34:59 2019 | |
read: IOPS=53, BW=3453KiB/s (3536kB/s)(405MiB/120005msec) | |
slat (nsec): min=0, max=377941k, avg=145377.66, stdev=6367946.16 | |
clat (nsec): min=0, max=824320k, avg=8948377.22, stdev=45281721.38 | |
lat (nsec): min=0, max=824512k, avg=9094746.36, stdev=45750151.63 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 732], 10.00th=[ 740], | |
| 20.00th=[ 820], 30.00th=[ 924], 40.00th=[ 1048], | |
| 50.00th=[ 1192], 60.00th=[ 1384], 70.00th=[ 67072], | |
| 80.00th=[ 1138688], 90.00th=[ 7700480], 95.00th=[ 47448064], | |
| 99.00th=[233832448], 99.50th=[379584512], 99.90th=[583008256], | |
| 99.95th=[599785472], 99.99th=[826277888] | |
bw ( KiB/s): min= 0, max= 5171, per=14.18%, avg=3452.50, stdev=2430.33, samples=2 | |
iops : min= 0, max= 80, avg=53.50, stdev=37.48, samples=2 | |
write: IOPS=55, BW=3551KiB/s (3637kB/s)(416MiB/120005msec) | |
slat (nsec): min=0, max=114884k, avg=206767.42, stdev=1440085.66 | |
clat (nsec): min=0, max=978894k, avg=63265817.90, stdev=119018938.09 | |
lat (nsec): min=0, max=979480k, avg=63473688.03, stdev=118978188.84 | |
clat percentiles (usec): | |
| 1.00th=[ 1037], 5.00th=[ 1303], 10.00th=[ 1516], 20.00th=[ 1958], | |
| 30.00th=[ 2704], 40.00th=[ 4228], 50.00th=[ 7767], 60.00th=[ 19530], | |
| 70.00th=[ 69731], 80.00th=[103285], 90.00th=[156238], 95.00th=[304088], | |
| 99.00th=[624952], 99.50th=[708838], 99.90th=[843056], 99.95th=[918553], | |
| 99.99th=[977273] | |
bw ( KiB/s): min= 0, max= 5238, per=55.52%, avg=3549.00, stdev=2388.61, samples=2 | |
iops : min= 0, max= 81, avg=55.00, stdev=36.77, samples=2 | |
lat (nsec) : 750=5.93%, 1000=12.18% | |
lat (usec) : 2=14.23%, 4=0.05%, 10=0.21%, 20=0.74%, 50=0.85% | |
lat (usec) : 100=0.91%, 250=0.71%, 500=0.76%, 750=0.63%, 1000=1.97% | |
lat (msec) : 2=12.87%, 4=11.19%, 10=8.16%, 20=4.69%, 50=4.58% | |
lat (msec) : 100=7.81%, 250=7.90%, 500=2.55%, 750=0.96%, 1000=0.14% | |
cpu : usr=0.07%, sys=0.32%, ctx=15689, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=6475,6656,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=23.8MiB/s (24.9MB/s), 3453KiB/s-20.4MiB/s (3536kB/s-21.4MB/s), io=2853MiB (2992MB), run=120005-120025msec | |
WRITE: bw=6393KiB/s (6546kB/s), 2846KiB/s-3551KiB/s (2915kB/s-3637kB/s), io=750MiB (787MB), run=120005-120177msec | |
Disk stats (read/write): | |
rbd256: ios=8297/12558, merge=0/2, ticks=571393/819813, in_queue=1380594, util=14.16% | |
<example-benchmark-fio-server-294-benchmark-tmh99> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:34:59 2019 | |
write: IOPS=20, BW=1329KiB/s (1361kB/s)(157MiB/120857msec) | |
slat (nsec): min=0, max=200899k, avg=380636.09, stdev=5677598.13 | |
clat (nsec): min=0, max=6808.5M, avg=192323685.98, stdev=404033626.27 | |
lat (nsec): min=0, max=6808.8M, avg=192705644.37, stdev=404002602.14 | |
clat percentiles (usec): | |
| 1.00th=[ 1450], 5.00th=[ 2147], 10.00th=[ 2671], | |
| 20.00th=[ 3720], 30.00th=[ 4948], 40.00th=[ 6915], | |
| 50.00th=[ 9765], 60.00th=[ 14091], 70.00th=[ 34866], | |
| 80.00th=[ 383779], 90.00th=[ 666895], 95.00th=[ 901776], | |
| 99.00th=[1451230], 99.50th=[1635779], 99.90th=[4395631], | |
| 99.95th=[6811550], 99.99th=[6811550] | |
bw ( KiB/s): min= 0, max= 1782, per=26.56%, avg=1334.00, stdev=633.57, samples=2 | |
iops : min= 0, max= 27, avg=20.00, stdev= 9.90, samples=2 | |
lat (usec) : 4=0.04%, 250=0.04%, 750=0.04%, 1000=0.12% | |
lat (msec) : 2=3.79%, 4=18.48%, 10=28.49%, 20=15.16%, 50=4.51% | |
lat (msec) : 100=0.52%, 250=3.87%, 500=8.34%, 750=8.86%, 1000=3.59% | |
cpu : usr=0.02%, sys=0.09%, ctx=4422, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,2506,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:34:59 2019 | |
read: IOPS=201, BW=12.6MiB/s (13.2MB/s)(1520MiB/120584msec) | |
slat (nsec): min=0, max=30825k, avg=33031.54, stdev=251563.33 | |
clat (nsec): min=0, max=1843.3M, avg=19822397.80, stdev=118197969.25 | |
lat (nsec): min=0, max=1843.3M, avg=19856547.54, stdev=118217208.93 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 732], 10.00th=[ 756], | |
| 20.00th=[ 788], 30.00th=[ 828], 40.00th=[ 868], | |
| 50.00th=[ 900], 60.00th=[ 964], 70.00th=[ 1240], | |
| 80.00th=[ 11712], 90.00th=[ 2703360], 95.00th=[ 12648448], | |
| 99.00th=[ 742391808], 99.50th=[ 901775360], 99.90th=[1166016512], | |
| 99.95th=[1249902592], 99.99th=[1719664640] | |
bw ( KiB/s): min= 0, max=13908, per=78.34%, avg=12965.50, stdev=1332.90, samples=2 | |
iops : min= 0, max= 217, avg=202.00, stdev=21.21, samples=2 | |
lat (nsec) : 750=9.20%, 1000=53.40% | |
lat (usec) : 2=16.93%, 4=0.19%, 10=0.23%, 20=0.72%, 50=0.21% | |
lat (usec) : 100=0.15%, 250=0.12%, 500=0.08%, 750=0.22%, 1000=1.69% | |
lat (msec) : 2=5.01%, 4=3.90%, 10=2.58%, 20=1.08%, 50=0.98% | |
lat (msec) : 100=0.39%, 250=0.30%, 500=0.68%, 750=0.99%, 1000=0.72% | |
cpu : usr=0.07%, sys=0.27%, ctx=9250, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=24313,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:34:59 2019 | |
read: IOPS=57, BW=3662KiB/s (3750kB/s)(429MiB/120007msec) | |
slat (nsec): min=0, max=7169.2k, avg=30253.59, stdev=126833.96 | |
clat (nsec): min=0, max=745444k, avg=7485190.03, stdev=44225344.30 | |
lat (nsec): min=0, max=745480k, avg=7516432.89, stdev=44236569.51 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 764], 30.00th=[ 876], 40.00th=[ 980], | |
| 50.00th=[ 1144], 60.00th=[ 1288], 70.00th=[ 14656], | |
| 80.00th=[ 749568], 90.00th=[ 2867200], 95.00th=[ 33816576], | |
| 99.00th=[208666624], 99.50th=[408944640], 99.90th=[599785472], | |
| 99.95th=[624951296], 99.99th=[742391808] | |
bw ( KiB/s): min= 0, max= 5649, per=22.13%, avg=3662.00, stdev=2810.04, samples=2 | |
iops : min= 0, max= 88, avg=57.00, stdev=43.84, samples=2 | |
write: IOPS=58, BW=3720KiB/s (3809kB/s)(436MiB/120007msec) | |
slat (nsec): min=0, max=4886.1k, avg=176323.86, stdev=232612.07 | |
clat (nsec): min=0, max=1093.2M, avg=61434190.62, stdev=127604648.95 | |
lat (nsec): min=0, max=1093.5M, avg=61611749.80, stdev=127566168.49 | |
clat percentiles (usec): | |
| 1.00th=[ 1029], 5.00th=[ 1303], 10.00th=[ 1483], | |
| 20.00th=[ 1827], 30.00th=[ 2311], 40.00th=[ 3228], | |
| 50.00th=[ 5080], 60.00th=[ 10814], 70.00th=[ 35390], | |
| 80.00th=[ 91751], 90.00th=[ 175113], 95.00th=[ 350225], | |
| 99.00th=[ 608175], 99.50th=[ 692061], 99.90th=[ 910164], | |
| 99.95th=[1027605], 99.99th=[1098908] | |
bw ( KiB/s): min= 0, max= 5666, per=74.02%, avg=3717.50, stdev=2755.60, samples=2 | |
iops : min= 0, max= 88, avg=57.50, stdev=43.13, samples=2 | |
lat (nsec) : 750=8.47%, 1000=12.07% | |
lat (usec) : 2=13.36%, 4=0.04%, 10=0.32%, 20=1.28%, 50=0.92% | |
lat (usec) : 100=0.85%, 250=1.39%, 500=0.57%, 750=0.47%, 1000=1.69% | |
lat (msec) : 2=14.34%, 4=12.44%, 10=7.88%, 20=4.06%, 50=3.80% | |
lat (msec) : 100=6.09%, 250=5.90%, 500=2.64%, 750=1.28%, 1000=0.12% | |
cpu : usr=0.08%, sys=0.32%, ctx=15155, majf=0, minf=31 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=6867,6972,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=16.2MiB/s (16.9MB/s), 3662KiB/s-12.6MiB/s (3750kB/s-13.2MB/s), io=1949MiB (2044MB), run=120007-120584msec | |
WRITE: bw=5022KiB/s (5143kB/s), 1329KiB/s-3720KiB/s (1361kB/s-3809kB/s), io=593MiB (622MB), run=120007-120857msec | |
Disk stats (read/write): | |
rbd98: ios=6656/10952, merge=0/2, ticks=583320/912510, in_queue=1486887, util=5.81% | |
<example-benchmark-fio-server-977-benchmark-wgh5n> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:35:00 2019 | |
write: IOPS=33, BW=2177KiB/s (2229kB/s)(256MiB/120220msec) | |
slat (nsec): min=0, max=1172.5M, avg=774123.61, stdev=23284904.99 | |
clat (nsec): min=0, max=1437.9M, avg=117417416.74, stdev=205641999.08 | |
lat (nsec): min=0, max=1437.9M, avg=118192524.39, stdev=206589593.23 | |
clat percentiles (usec): | |
| 1.00th=[ 1004], 5.00th=[ 1237], 10.00th=[ 1434], | |
| 20.00th=[ 1942], 30.00th=[ 2868], 40.00th=[ 5342], | |
| 50.00th=[ 13173], 60.00th=[ 60031], 70.00th=[ 103285], | |
| 80.00th=[ 179307], 90.00th=[ 375391], 95.00th=[ 574620], | |
| 99.00th=[ 935330], 99.50th=[1052771], 99.90th=[1182794], | |
| 99.95th=[1333789], 99.99th=[1434452] | |
bw ( KiB/s): min= 0, max= 2643, per=42.38%, avg=2178.50, stdev=656.90, samples=2 | |
iops : min= 0, max= 41, avg=33.50, stdev=10.61, samples=2 | |
lat (usec) : 750=0.05%, 1000=0.86% | |
lat (msec) : 2=20.00%, 4=15.10%, 10=11.63%, 20=6.61%, 50=5.02% | |
lat (msec) : 100=9.64%, 250=15.47%, 500=9.08%, 750=3.65%, 1000=2.28% | |
cpu : usr=0.05%, sys=0.15%, ctx=7711, majf=0, minf=31 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,4086,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:35:00 2019 | |
read: IOPS=434, BW=27.1MiB/s (28.4MB/s)(3256MiB/120173msec) | |
slat (nsec): min=0, max=1168.0M, avg=69320.30, stdev=6851183.35 | |
clat (nsec): min=0, max=1338.9M, avg=9166220.50, stdev=60043152.24 | |
lat (nsec): min=0, max=1338.9M, avg=9236403.72, stdev=60436829.32 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 708], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 732], 40.00th=[ 772], | |
| 50.00th=[ 812], 60.00th=[ 860], 70.00th=[ 900], | |
| 80.00th=[ 1160], 90.00th=[ 1028096], 95.00th=[ 20054016], | |
| 99.00th=[ 308281344], 99.50th=[ 476053504], 99.90th=[ 775946240], | |
| 99.95th=[ 876609536], 99.99th=[1098907648] | |
bw ( KiB/s): min= 0, max=43740, per=90.78%, avg=27779.00, stdev=22572.26, samples=2 | |
iops : min= 0, max= 684, avg=434.00, stdev=353.55, samples=2 | |
lat (nsec) : 750=34.58%, 1000=41.49% | |
lat (usec) : 2=9.00%, 4=0.09%, 10=0.55%, 20=1.01%, 50=0.38% | |
lat (usec) : 100=0.11%, 250=0.08%, 500=0.40%, 750=0.68%, 1000=1.52% | |
lat (msec) : 2=2.37%, 4=1.52%, 10=0.76%, 20=0.48%, 50=1.41% | |
lat (msec) : 100=1.42%, 250=0.92%, 500=0.81%, 750=0.30%, 1000=0.11% | |
cpu : usr=0.13%, sys=0.46%, ctx=13251, majf=0, minf=38 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=52164,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:35:00 2019 | |
read: IOPS=44, BW=2864KiB/s (2933kB/s)(336MiB/120003msec) | |
slat (nsec): min=0, max=816568k, avg=185923.53, stdev=11141851.34 | |
clat (nsec): min=0, max=724201k, avg=8174567.07, stdev=43029992.72 | |
lat (nsec): min=0, max=817041k, avg=8361466.82, stdev=44428754.62 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 740], | |
| 20.00th=[ 820], 30.00th=[ 900], 40.00th=[ 1032], | |
| 50.00th=[ 1176], 60.00th=[ 1352], 70.00th=[ 32640], | |
| 80.00th=[ 856064], 90.00th=[ 4620288], 95.00th=[ 39059456], | |
| 99.00th=[235929600], 99.50th=[320864256], 99.90th=[517996544], | |
| 99.95th=[633339904], 99.99th=[725614592] | |
bw ( KiB/s): min= 0, max= 4423, per=9.36%, avg=2863.50, stdev=2205.47, samples=2 | |
iops : min= 0, max= 69, avg=44.50, stdev=34.65, samples=2 | |
write: IOPS=46, BW=2970KiB/s (3041kB/s)(348MiB/120003msec) | |
slat (nsec): min=0, max=54255k, avg=183966.16, stdev=752724.15 | |
clat (nsec): min=0, max=1040.4M, avg=78172935.18, stdev=141799380.38 | |
lat (nsec): min=0, max=1040.4M, avg=78358082.12, stdev=141760902.55 | |
clat percentiles (usec): | |
| 1.00th=[ 1004], 5.00th=[ 1237], 10.00th=[ 1483], | |
| 20.00th=[ 2089], 30.00th=[ 3064], 40.00th=[ 5735], | |
| 50.00th=[ 13829], 60.00th=[ 30278], 70.00th=[ 74974], | |
| 80.00th=[ 110625], 90.00th=[ 229639], 95.00th=[ 387974], | |
| 99.00th=[ 725615], 99.50th=[ 801113], 99.90th=[ 918553], | |
| 99.95th=[ 952108], 99.99th=[1044382] | |
bw ( KiB/s): min= 0, max= 4562, per=57.73%, avg=2967.50, stdev=2254.96, samples=2 | |
iops : min= 0, max= 71, avg=46.00, stdev=35.36, samples=2 | |
lat (nsec) : 750=5.63%, 1000=13.06% | |
lat (usec) : 2=14.03%, 4=0.06%, 10=0.30%, 20=0.90%, 50=0.85% | |
lat (usec) : 100=0.80%, 250=0.84%, 500=0.92%, 750=1.02%, 1000=2.10% | |
lat (msec) : 2=11.43%, 4=9.95%, 10=6.71%, 20=5.17%, 50=6.26% | |
lat (msec) : 100=7.18%, 250=7.72%, 500=3.57%, 750=1.09%, 1000=0.42% | |
cpu : usr=0.07%, sys=0.26%, ctx=12751, majf=0, minf=33 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=5371,5565,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=29.9MiB/s (31.3MB/s), 2864KiB/s-27.1MiB/s (2933kB/s-28.4MB/s), io=3591MiB (3766MB), run=120003-120173msec | |
WRITE: bw=5141KiB/s (5264kB/s), 2177KiB/s-2970KiB/s (2229kB/s-3041kB/s), io=604MiB (633MB), run=120003-120220msec | |
Disk stats (read/write): | |
rbd324: ios=8914/10591, merge=0/1, ticks=549035/867403, in_queue=1406550, util=12.49% | |
<example-benchmark-fio-server-138-benchmark-9r9wf> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:35:00 2019 | |
write: IOPS=46, BW=2976KiB/s (3047kB/s)(349MiB/120102msec) | |
slat (nsec): min=0, max=624921k, avg=477947.36, stdev=12874979.64 | |
clat (nsec): min=0, max=1497.6M, avg=85883724.77, stdev=166458648.87 | |
lat (nsec): min=0, max=1497.6M, avg=86362829.03, stdev=167067375.65 | |
clat percentiles (usec): | |
| 1.00th=[ 1004], 5.00th=[ 1172], 10.00th=[ 1336], | |
| 20.00th=[ 1745], 30.00th=[ 2442], 40.00th=[ 3752], | |
| 50.00th=[ 7046], 60.00th=[ 27657], 70.00th=[ 88605], | |
| 80.00th=[ 113771], 90.00th=[ 242222], 95.00th=[ 476054], | |
| 99.00th=[ 759170], 99.50th=[ 943719], 99.90th=[1300235], | |
| 99.95th=[1468007], 99.99th=[1501561] | |
bw ( KiB/s): min= 0, max= 3030, per=42.42%, avg=2975.50, stdev=77.07, samples=2 | |
iops : min= 0, max= 47, avg=46.00, stdev= 1.41, samples=2 | |
lat (usec) : 4=0.02%, 500=0.02%, 750=0.02%, 1000=0.88% | |
lat (msec) : 2=23.29%, 4=17.02%, 10=12.20%, 20=5.36%, 50=3.10% | |
lat (msec) : 100=12.97%, 250=15.39%, 500=5.36%, 750=3.39%, 1000=0.75% | |
cpu : usr=0.05%, sys=0.21%, ctx=10541, majf=0, minf=29 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,5581,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:35:00 2019 | |
read: IOPS=377, BW=23.6MiB/s (24.7MB/s)(2830MiB/120105msec) | |
slat (nsec): min=0, max=473114k, avg=47081.33, stdev=3532967.60 | |
clat (nsec): min=0, max=1438.1M, avg=10554888.20, stdev=60565769.03 | |
lat (nsec): min=0, max=1438.2M, avg=10602804.25, stdev=60674181.23 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 732], 40.00th=[ 748], | |
| 50.00th=[ 820], 60.00th=[ 876], 70.00th=[ 932], | |
| 80.00th=[ 1352], 90.00th=[ 1843200], 95.00th=[ 50069504], | |
| 99.00th=[ 329252864], 99.50th=[ 476053504], 99.90th=[ 734003200], | |
| 99.95th=[ 868220928], 99.99th=[1199570944] | |
bw ( KiB/s): min= 0, max=35189, per=85.90%, avg=24145.50, stdev=15617.87, samples=2 | |
iops : min= 0, max= 551, avg=377.50, stdev=245.37, samples=2 | |
lat (nsec) : 750=40.49%, 1000=32.27% | |
lat (usec) : 2=8.76%, 4=0.06%, 10=1.83%, 20=0.98%, 50=0.34% | |
lat (usec) : 100=0.11%, 250=0.07%, 500=0.22%, 750=0.70%, 1000=1.89% | |
lat (msec) : 2=2.58%, 4=1.61%, 10=0.93%, 20=0.33%, 50=1.87% | |
lat (msec) : 100=2.63%, 250=1.04%, 500=0.88%, 750=0.35%, 1000=0.06% | |
cpu : usr=0.10%, sys=0.41%, ctx=13738, majf=0, minf=51 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=45385,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:35:00 2019 | |
read: IOPS=62, BW=3986KiB/s (4082kB/s)(467MiB/120016msec) | |
slat (nsec): min=0, max=516982k, avg=103312.94, stdev=5980122.37 | |
clat (nsec): min=0, max=999725k, avg=7809295.40, stdev=43750017.09 | |
lat (nsec): min=0, max=999792k, avg=7913667.08, stdev=44149149.06 | |
clat percentiles (nsec): | |
| 1.00th=[ 724], 5.00th=[ 732], 10.00th=[ 740], | |
| 20.00th=[ 820], 30.00th=[ 916], 40.00th=[ 1048], | |
| 50.00th=[ 1192], 60.00th=[ 1368], 70.00th=[ 78336], | |
| 80.00th=[ 1089536], 90.00th=[ 5144576], 95.00th=[ 39583744], | |
| 99.00th=[ 189792256], 99.50th=[ 329252864], 99.90th=[ 513802240], | |
| 99.95th=[ 817889280], 99.99th=[1002438656] | |
bw ( KiB/s): min= 0, max= 5655, per=14.18%, avg=3985.50, stdev=2361.03, samples=2 | |
iops : min= 0, max= 88, avg=62.00, stdev=36.77, samples=2 | |
write: IOPS=63, BW=4042KiB/s (4139kB/s)(474MiB/120016msec) | |
slat (nsec): min=0, max=633040k, avg=353080.90, stdev=9365055.55 | |
clat (nsec): min=0, max=1236.7M, avg=55364701.88, stdev=121922412.63 | |
lat (nsec): min=0, max=1236.7M, avg=55718902.59, stdev=122175546.16 | |
clat percentiles (usec): | |
| 1.00th=[ 996], 5.00th=[ 1221], 10.00th=[ 1385], | |
| 20.00th=[ 1762], 30.00th=[ 2311], 40.00th=[ 3130], | |
| 50.00th=[ 5014], 60.00th=[ 11076], 70.00th=[ 28181], | |
| 80.00th=[ 85459], 90.00th=[ 133694], 95.00th=[ 299893], | |
| 99.00th=[ 650118], 99.50th=[ 725615], 99.90th=[ 918553], | |
| 99.95th=[ 977273], 99.99th=[1233126] | |
bw ( KiB/s): min= 0, max= 5676, per=57.60%, avg=4040.00, stdev=2313.65, samples=2 | |
iops : min= 0, max= 88, avg=62.50, stdev=36.06, samples=2 | |
lat (nsec) : 750=6.30%, 1000=11.97% | |
lat (usec) : 2=14.31%, 4=0.12%, 10=0.32%, 20=0.72%, 50=0.60% | |
lat (usec) : 100=0.66%, 250=0.88%, 500=0.72%, 750=0.75%, 1000=2.44% | |
lat (msec) : 2=15.07%, 4=12.30%, 10=7.92%, 20=4.76%, 50=4.62% | |
lat (msec) : 100=6.83%, 250=5.36%, 500=2.17%, 750=1.00%, 1000=0.17% | |
cpu : usr=0.10%, sys=0.35%, ctx=18104, majf=0, minf=36 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=7476,7576,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=27.5MiB/s (28.8MB/s), 3986KiB/s-23.6MiB/s (4082kB/s-24.7MB/s), io=3297MiB (3457MB), run=120016-120105msec | |
WRITE: bw=7014KiB/s (7183kB/s), 2976KiB/s-4042KiB/s (3047kB/s-4139kB/s), io=823MiB (863MB), run=120016-120102msec | |
Disk stats (read/write): | |
rbd40: ios=9658/13474, merge=0/1, ticks=566891/831540, in_queue=1386680, util=15.13% | |
Jobs: 28 (f=28): [_(2),m(1)][99.6%][r=1044M,w=1019M][r=16.3k,w=15.9k IOPS][eta 00m:50s] <example-benchmark-fio-server-446-benchmark-2rtd4> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:35:00 2019 | |
write: IOPS=14, BW=956KiB/s (979kB/s)(112MiB/120085msec) | |
slat (nsec): min=0, max=899001k, avg=734283.87, stdev=21245487.71 | |
clat (nsec): min=0, max=3218.8M, avg=267401220.62, stdev=389066586.33 | |
lat (nsec): min=0, max=3218.0M, avg=268136022.10, stdev=389320204.11 | |
clat percentiles (usec): | |
| 1.00th=[ 1663], 5.00th=[ 2737], 10.00th=[ 3720], | |
| 20.00th=[ 5407], 30.00th=[ 8029], 40.00th=[ 12256], | |
| 50.00th=[ 20055], 60.00th=[ 84411], 70.00th=[ 383779], | |
| 80.00th=[ 650118], 90.00th=[ 826278], 95.00th=[ 985662], | |
| 99.00th=[1484784], 99.50th=[1635779], 99.90th=[3170894], | |
| 99.95th=[3204449], 99.99th=[3204449] | |
bw ( KiB/s): min= 0, max= 1412, per=25.82%, avg=1412.00, stdev= 0.00, samples=1 | |
iops : min= 0, max= 22, avg=22.00, stdev= 0.00, samples=1 | |
lat (usec) : 250=0.06%, 1000=0.06% | |
lat (msec) : 2=2.01%, 4=9.49%, 10=23.90%, 20=14.57%, 50=8.21% | |
lat (msec) : 100=2.51%, 250=5.36%, 500=7.82%, 750=13.01%, 1000=8.43% | |
cpu : usr=0.02%, sys=0.07%, ctx=3024, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,1791,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:35:00 2019 | |
read: IOPS=168, BW=10.5MiB/s (11.0MB/s)(1266MiB/120432msec) | |
slat (nsec): min=0, max=23034k, avg=26427.91, stdev=210110.57 | |
clat (nsec): min=0, max=1849.9M, avg=23799251.42, stdev=130255782.34 | |
lat (nsec): min=0, max=1849.0M, avg=23826708.01, stdev=130277592.76 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 740], | |
| 20.00th=[ 780], 30.00th=[ 796], 40.00th=[ 836], | |
| 50.00th=[ 868], 60.00th=[ 892], 70.00th=[ 956], | |
| 80.00th=[ 1256], 90.00th=[ 1482752], 95.00th=[ 20578304], | |
| 99.00th=[ 784334848], 99.50th=[ 926941184], 99.90th=[1182793728], | |
| 99.95th=[1333788672], 99.99th=[1619001344] | |
bw ( KiB/s): min= 0, max=11333, per=70.98%, avg=10797.00, stdev=758.02, samples=2 | |
iops : min= 0, max= 177, avg=168.50, stdev=12.02, samples=2 | |
lat (nsec) : 750=12.58%, 1000=60.81% | |
lat (usec) : 2=12.84%, 4=0.23%, 10=0.21%, 20=1.00%, 50=0.10% | |
lat (usec) : 100=0.08%, 250=0.07%, 500=0.04%, 750=0.26%, 1000=0.62% | |
lat (msec) : 2=2.02%, 4=1.88%, 10=1.45%, 20=0.80%, 50=1.01% | |
lat (msec) : 100=0.40%, 250=0.42%, 500=0.79%, 750=1.25%, 1000=0.82% | |
cpu : usr=0.06%, sys=0.19%, ctx=5020, majf=0, minf=29 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=20248,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:35:00 2019 | |
read: IOPS=69, BW=4467KiB/s (4574kB/s)(524MiB/120003msec) | |
slat (nsec): min=0, max=8029.9k, avg=25453.88, stdev=121721.00 | |
clat (nsec): min=0, max=668541k, avg=6380971.94, stdev=41066870.55 | |
lat (nsec): min=0, max=668579k, avg=6407382.95, stdev=41072836.25 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 756], 30.00th=[ 836], 40.00th=[ 924], | |
| 50.00th=[ 1064], 60.00th=[ 1224], 70.00th=[ 1512], | |
| 80.00th=[ 97792], 90.00th=[ 1433600], 95.00th=[ 11862016], | |
| 99.00th=[206569472], 99.50th=[341835776], 99.90th=[541065216], | |
| 99.95th=[599785472], 99.99th=[666894336] | |
bw ( KiB/s): min= 0, max= 7498, per=29.36%, avg=4466.50, stdev=4287.19, samples=2 | |
iops : min= 0, max= 117, avg=69.50, stdev=67.18, samples=2 | |
write: IOPS=70, BW=4515KiB/s (4623kB/s)(529MiB/120003msec) | |
slat (nsec): min=0, max=11441k, avg=189852.16, stdev=277675.89 | |
clat (nsec): min=0, max=970672k, avg=50276865.16, stdev=111353429.62 | |
lat (nsec): min=0, max=970761k, avg=50467942.12, stdev=111311973.94 | |
clat percentiles (usec): | |
| 1.00th=[ 971], 5.00th=[ 1188], 10.00th=[ 1385], 20.00th=[ 1762], | |
| 30.00th=[ 2311], 40.00th=[ 3228], 50.00th=[ 4817], 60.00th=[ 9241], | |
| 70.00th=[ 20579], 80.00th=[ 68682], 90.00th=[139461], 95.00th=[287310], | |
| 99.00th=[574620], 99.50th=[675283], 99.90th=[834667], 99.95th=[918553], | |
| 99.99th=[968885] | |
bw ( KiB/s): min= 0, max= 7545, per=82.53%, avg=4512.50, stdev=4288.60, samples=2 | |
iops : min= 0, max= 117, avg=70.00, stdev=66.47, samples=2 | |
lat (nsec) : 750=9.66%, 1000=13.03% | |
lat (usec) : 2=13.06%, 4=0.08%, 10=0.62%, 20=1.50%, 50=0.99% | |
lat (usec) : 100=0.87%, 250=1.65%, 500=0.76%, 750=0.59%, 1000=1.67% | |
lat (msec) : 2=13.33%, 4=11.49%, 10=8.41%, 20=4.92%, 50=4.16% | |
lat (msec) : 100=4.61%, 250=5.36%, 500=2.45%, 750=0.65%, 1000=0.14% | |
cpu : usr=0.09%, sys=0.39%, ctx=17391, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=8376,8463,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=14.9MiB/s (15.6MB/s), 4467KiB/s-10.5MiB/s (4574kB/s-11.0MB/s), io=1789MiB (1876MB), run=120003-120432msec | |
WRITE: bw=5468KiB/s (5599kB/s), 956KiB/s-4515KiB/s (979kB/s-4623kB/s), io=641MiB (672MB), run=120003-120085msec | |
Disk stats (read/write): | |
rbd126: ios=3914/10574, merge=0/2, ticks=578140/913384, in_queue=1484315, util=4.51% | |
<example-benchmark-fio-server-185-benchmark-lp9ct> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:35:00 2019 | |
write: IOPS=39, BW=2553KiB/s (2614kB/s)(301MiB/120774msec) | |
slat (nsec): min=0, max=35777k, avg=174425.73, stdev=601217.79 | |
clat (nsec): min=0, max=8251.2M, avg=100036644.27, stdev=335178831.09 | |
lat (nsec): min=0, max=8251.3M, avg=100212323.58, stdev=335174890.69 | |
clat percentiles (usec): | |
| 1.00th=[ 750], 5.00th=[ 1237], 10.00th=[ 1778], | |
| 20.00th=[ 3261], 30.00th=[ 5080], 40.00th=[ 7701], | |
| 50.00th=[ 11731], 60.00th=[ 19268], 70.00th=[ 40109], | |
| 80.00th=[ 139461], 90.00th=[ 278922], 95.00th=[ 459277], | |
| 99.00th=[ 759170], 99.50th=[1300235], 99.90th=[4395631], | |
| 99.95th=[8220836], 99.99th=[8220836] | |
bw ( KiB/s): min= 0, max= 3221, per=42.04%, avg=2567.00, stdev=924.90, samples=2 | |
iops : min= 0, max= 50, avg=39.50, stdev=14.85, samples=2 | |
lat (usec) : 4=0.02%, 50=0.02%, 500=0.06%, 750=0.91%, 1000=2.04% | |
lat (msec) : 2=8.64%, 4=13.08%, 10=22.28%, 20=13.46%, 50=11.71% | |
lat (msec) : 100=4.82%, 250=10.55%, 500=8.41%, 750=2.99%, 1000=0.27% | |
cpu : usr=0.04%, sys=0.14%, ctx=7681, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,4815,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:35:00 2019 | |
read: IOPS=68, BW=4361KiB/s (4465kB/s)(513MiB/120497msec) | |
slat (nsec): min=0, max=21635k, avg=53416.96, stdev=327483.41 | |
clat (nsec): min=0, max=1829.5M, avg=58787431.59, stdev=199791627.08 | |
lat (nsec): min=0, max=1829.5M, avg=58841997.19, stdev=199805850.51 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 732], 10.00th=[ 772], | |
| 20.00th=[ 820], 30.00th=[ 884], 40.00th=[ 1004], | |
| 50.00th=[ 1240], 60.00th=[ 1528], 70.00th=[ 1318912], | |
| 80.00th=[ 3620864], 90.00th=[ 40108032], 95.00th=[ 608174080], | |
| 99.00th=[ 968884224], 99.50th=[1082130432], 99.90th=[1350565888], | |
| 99.95th=[1468006400], 99.99th=[1837105152] | |
bw ( KiB/s): min= 0, max= 4444, per=55.77%, avg=4375.50, stdev=96.87, samples=2 | |
iops : min= 0, max= 69, avg=68.00, stdev= 1.41, samples=2 | |
lat (nsec) : 750=7.03%, 1000=32.83% | |
lat (usec) : 2=22.27%, 4=0.26%, 10=0.12%, 20=0.39%, 50=0.15% | |
lat (usec) : 100=0.15%, 250=0.06%, 500=0.07%, 750=0.77%, 1000=2.62% | |
lat (msec) : 2=7.37%, 4=6.77%, 10=5.08%, 20=2.33%, 50=2.17% | |
lat (msec) : 100=0.73%, 250=0.88%, 500=1.93%, 750=3.19%, 1000=1.99% | |
cpu : usr=0.04%, sys=0.10%, ctx=6071, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=8207,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:35:00 2019 | |
read: IOPS=54, BW=3500KiB/s (3584kB/s)(410MiB/120005msec) | |
slat (nsec): min=0, max=9183.3k, avg=39914.76, stdev=142373.27 | |
clat (nsec): min=0, max=749534k, avg=18005844.75, stdev=76443612.62 | |
lat (nsec): min=0, max=749561k, avg=18046837.72, stdev=76451706.73 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 732], 10.00th=[ 740], | |
| 20.00th=[ 852], 30.00th=[ 996], 40.00th=[ 1160], | |
| 50.00th=[ 1304], 60.00th=[ 35072], 70.00th=[ 995328], | |
| 80.00th=[ 2441216], 90.00th=[ 29491200], 95.00th=[ 71827456], | |
| 99.00th=[463470592], 99.50th=[608174080], 99.90th=[683671552], | |
| 99.95th=[725614592], 99.99th=[750780416] | |
bw ( KiB/s): min= 0, max= 5429, per=44.60%, avg=3499.00, stdev=2729.43, samples=2 | |
iops : min= 0, max= 84, avg=54.00, stdev=42.43, samples=2 | |
write: IOPS=55, BW=3576KiB/s (3662kB/s)(419MiB/120005msec) | |
slat (nsec): min=0, max=5692.4k, avg=182624.16, stdev=237777.68 | |
clat (nsec): min=0, max=1091.5M, avg=53980894.32, stdev=112363439.06 | |
lat (nsec): min=0, max=1091.7M, avg=54164773.18, stdev=112319425.21 | |
clat percentiles (usec): | |
| 1.00th=[ 1037], 5.00th=[ 1237], 10.00th=[ 1418], | |
| 20.00th=[ 1762], 30.00th=[ 2212], 40.00th=[ 3130], | |
| 50.00th=[ 4752], 60.00th=[ 10159], 70.00th=[ 27132], | |
| 80.00th=[ 78119], 90.00th=[ 162530], 95.00th=[ 304088], | |
| 99.00th=[ 541066], 99.50th=[ 599786], 99.90th=[ 792724], | |
| 99.95th=[ 817890], 99.99th=[1098908] | |
bw ( KiB/s): min= 0, max= 5484, per=58.55%, avg=3575.00, stdev=2699.73, samples=2 | |
iops : min= 0, max= 85, avg=55.50, stdev=41.72, samples=2 | |
lat (nsec) : 750=5.61%, 1000=9.29% | |
lat (usec) : 2=13.91%, 4=0.05%, 10=0.22%, 20=0.48%, 50=0.32% | |
lat (usec) : 100=0.38%, 250=0.49%, 500=0.34%, 750=1.14%, 1000=2.80% | |
lat (msec) : 2=16.52%, 4=13.11%, 10=9.01%, 20=4.52%, 50=5.64% | |
lat (msec) : 100=6.54%, 250=4.89%, 500=3.69%, 750=1.02%, 1000=0.05% | |
cpu : usr=0.08%, sys=0.31%, ctx=17110, majf=0, minf=31 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=6561,6704,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=7846KiB/s (8035kB/s), 3500KiB/s-4361KiB/s (3584kB/s-4465kB/s), io=923MiB (968MB), run=120005-120497msec | |
WRITE: bw=6106KiB/s (6253kB/s), 2553KiB/s-3576KiB/s (2614kB/s-3662kB/s), io=720MiB (755MB), run=120005-120774msec | |
Disk stats (read/write): | |
rbd73: ios=5816/13073, merge=0/3, ticks=653844/876526, in_queue=1520869, util=4.37% | |
<example-benchmark-fio-server-637-benchmark-4fhxd> | |
<example-benchmark-fio-server-567-benchmark-tkk6d> | |
<example-benchmark-fio-server-155-benchmark-scltw> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:35:00 2019 | |
write: IOPS=47, BW=3031KiB/s (3104kB/s)(355MiB/120027msec) | |
slat (nsec): min=0, max=713321k, avg=471599.79, stdev=12371015.09 | |
clat (nsec): min=0, max=1162.4M, avg=84194593.79, stdev=147118350.80 | |
lat (nsec): min=0, max=1297.7M, avg=84667236.21, stdev=147923628.18 | |
clat percentiles (usec): | |
| 1.00th=[ 963], 5.00th=[ 1123], 10.00th=[ 1287], | |
| 20.00th=[ 1680], 30.00th=[ 2409], 40.00th=[ 4424], | |
| 50.00th=[ 14222], 60.00th=[ 53216], 70.00th=[ 91751], | |
| 80.00th=[ 119014], 90.00th=[ 235930], 95.00th=[ 425722], | |
| 99.00th=[ 742392], 99.50th=[ 834667], 99.90th=[ 943719], | |
| 99.95th=[ 985662], 99.99th=[1166017] | |
bw ( KiB/s): min= 0, max= 1897, per=27.66%, avg=1897.00, stdev= 0.00, samples=1 | |
iops : min= 0, max= 29, avg=29.00, stdev= 0.00, samples=1 | |
lat (usec) : 500=0.04%, 750=0.09%, 1000=1.67% | |
lat (msec) : 2=23.76%, 4=13.08%, 10=8.50%, 20=4.84%, 50=7.22% | |
lat (msec) : 100=14.15%, 250=17.35%, 500=6.04%, 750=2.38%, 1000=0.92% | |
cpu : usr=0.06%, sys=0.20%, ctx=10812, majf=0, minf=28 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,5682,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:35:00 2019 | |
read: IOPS=331, BW=20.7MiB/s (21.7MB/s)(2485MiB/120014msec) | |
slat (nsec): min=0, max=714213k, avg=59830.16, stdev=4581236.52 | |
clat (nsec): min=0, max=1125.4M, avg=12024678.97, stdev=58728270.03 | |
lat (nsec): min=0, max=1125.4M, avg=12085365.90, stdev=58924449.76 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 724], | |
| 20.00th=[ 732], 30.00th=[ 748], 40.00th=[ 820], | |
| 50.00th=[ 860], 60.00th=[ 932], 70.00th=[ 1128], | |
| 80.00th=[ 309248], 90.00th=[ 9633792], 95.00th=[ 55836672], | |
| 99.00th=[341835776], 99.50th=[467664896], 99.90th=[675282944], | |
| 99.95th=[742391808], 99.99th=[968884224] | |
bw ( KiB/s): min= 0, max=28049, per=84.99%, avg=21206.50, stdev=9676.76, samples=2 | |
iops : min= 0, max= 438, avg=331.00, stdev=151.32, samples=2 | |
lat (nsec) : 750=30.11%, 1000=35.20% | |
lat (usec) : 2=13.03%, 4=0.06%, 10=0.82%, 20=0.40%, 50=0.14% | |
lat (usec) : 100=0.07%, 250=0.07%, 500=0.35%, 750=1.07%, 1000=2.45% | |
lat (msec) : 2=3.18%, 4=1.99%, 10=1.08%, 20=0.75%, 50=3.52% | |
lat (msec) : 100=2.76%, 250=1.48%, 500=1.06%, 750=0.34%, 1000=0.04% | |
cpu : usr=0.12%, sys=0.38%, ctx=15922, majf=0, minf=37 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=39785,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:35:00 2019 | |
read: IOPS=58, BW=3746KiB/s (3836kB/s)(439MiB/120005msec) | |
slat (nsec): min=0, max=9850.8k, avg=37565.76, stdev=156835.35 | |
clat (nsec): min=0, max=680660k, avg=6520405.48, stdev=33535024.37 | |
lat (nsec): min=0, max=680698k, avg=6558922.65, stdev=33543170.85 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 732], 10.00th=[ 740], | |
| 20.00th=[ 820], 30.00th=[ 924], 40.00th=[ 1064], | |
| 50.00th=[ 1208], 60.00th=[ 1416], 70.00th=[ 142336], | |
| 80.00th=[ 1073152], 90.00th=[ 5013504], 95.00th=[ 34865152], | |
| 99.00th=[168820736], 99.50th=[258998272], 99.90th=[455081984], | |
| 99.95th=[505413632], 99.99th=[683671552] | |
bw ( KiB/s): min= 0, max= 5781, per=15.01%, avg=3745.50, stdev=2878.63, samples=2 | |
iops : min= 0, max= 90, avg=58.00, stdev=45.25, samples=2 | |
write: IOPS=59, BW=3828KiB/s (3920kB/s)(449MiB/120005msec) | |
slat (nsec): min=0, max=340981k, avg=298619.27, stdev=5238520.33 | |
clat (nsec): min=0, max=1050.3M, avg=60301551.27, stdev=126652467.05 | |
lat (nsec): min=0, max=1050.4M, avg=60601269.70, stdev=126682799.90 | |
clat percentiles (usec): | |
| 1.00th=[ 1004], 5.00th=[ 1221], 10.00th=[ 1401], | |
| 20.00th=[ 1778], 30.00th=[ 2343], 40.00th=[ 3425], | |
| 50.00th=[ 5669], 60.00th=[ 11731], 70.00th=[ 39060], | |
| 80.00th=[ 92799], 90.00th=[ 158335], 95.00th=[ 341836], | |
| 99.00th=[ 633340], 99.50th=[ 775947], 99.90th=[ 943719], | |
| 99.95th=[ 943719], 99.99th=[1052771] | |
bw ( KiB/s): min= 0, max= 5822, per=55.78%, avg=3825.50, stdev=2823.48, samples=2 | |
iops : min= 0, max= 90, avg=59.00, stdev=43.84, samples=2 | |
lat (nsec) : 750=5.86%, 1000=11.94% | |
lat (usec) : 2=13.77%, 4=0.07%, 10=0.26%, 20=0.95%, 50=0.75% | |
lat (usec) : 100=0.71%, 250=0.82%, 500=1.11%, 750=1.04%, 1000=2.37% | |
lat (msec) : 2=15.03%, 4=11.20%, 10=9.35%, 20=3.85%, 50=5.03% | |
lat (msec) : 100=5.85%, 250=6.41%, 500=2.51%, 750=0.85%, 1000=0.30% | |
cpu : usr=0.09%, sys=0.32%, ctx=17167, majf=0, minf=35 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=7025,7175,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=24.4MiB/s (25.6MB/s), 3746KiB/s-20.7MiB/s (3836kB/s-21.7MB/s), io=2924MiB (3066MB), run=120005-120014msec | |
WRITE: bw=6859KiB/s (7023kB/s), 3031KiB/s-3828KiB/s (3104kB/s-3920kB/s), io=804MiB (843MB), run=120005-120027msec | |
Disk stats (read/write): | |
rbd236: ios=11231/13877, merge=0/1, ticks=555717/840618, in_queue=1383488, util=16.97% | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:35:00 2019 | |
write: IOPS=15, BW=1000KiB/s (1024kB/s)(118MiB/120408msec) | |
slat (nsec): min=0, max=980064k, avg=1663581.52, stdev=35294572.35 | |
clat (nsec): min=0, max=2257.3M, avg=255484057.39, stdev=465350314.36 | |
lat (nsec): min=0, max=2257.8M, avg=257147165.96, stdev=465938509.66 | |
clat percentiles (usec): | |
| 1.00th=[ 1237], 5.00th=[ 1500], 10.00th=[ 1844], | |
| 20.00th=[ 2737], 30.00th=[ 3884], 40.00th=[ 6128], | |
| 50.00th=[ 10421], 60.00th=[ 21627], 70.00th=[ 49546], | |
| 80.00th=[ 675283], 90.00th=[1010828], 95.00th=[1367344], | |
| 99.00th=[1669333], 99.50th=[1887437], 99.90th=[2164261], | |
| 99.95th=[2264925], 99.99th=[2264925] | |
bw ( KiB/s): min= 0, max= 1497, per=21.63%, avg=1000.50, stdev=702.16, samples=2 | |
iops : min= 0, max= 23, avg=15.00, stdev=11.31, samples=2 | |
lat (usec) : 50=0.05%, 250=0.05% | |
lat (msec) : 2=12.30%, 4=18.48%, 10=18.74%, 20=9.11%, 50=11.77% | |
lat (msec) : 100=5.27%, 250=0.53%, 500=0.75%, 750=4.79%, 1000=7.72% | |
cpu : usr=0.03%, sys=0.07%, ctx=3491, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,1878,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:35:00 2019 | |
read: IOPS=627, BW=39.0MiB/s (40.9MB/s)(4684MiB/120072msec) | |
slat (nsec): min=0, max=979374k, avg=48936.40, stdev=5532284.90 | |
clat (nsec): min=0, max=2121.5M, avg=6347450.41, stdev=70775025.09 | |
lat (nsec): min=0, max=2121.5M, avg=6397358.95, stdev=71004998.05 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 724], | |
| 20.00th=[ 764], 30.00th=[ 780], 40.00th=[ 788], | |
| 50.00th=[ 828], 60.00th=[ 860], 70.00th=[ 884], | |
| 80.00th=[ 924], 90.00th=[ 8768], 95.00th=[ 12736], | |
| 99.00th=[ 76021760], 99.50th=[ 624951296], 99.90th=[1082130432], | |
| 99.95th=[1266679808], 99.99th=[1635778560] | |
bw ( KiB/s): min= 0, max=59778, per=91.83%, avg=39953.00, stdev=28036.78, samples=2 | |
iops : min= 0, max= 939, avg=627.00, stdev=441.23, samples=2 | |
lat (nsec) : 750=17.24%, 1000=67.21% | |
lat (usec) : 2=4.63%, 4=0.13%, 10=1.42%, 20=5.52%, 50=0.52% | |
lat (usec) : 100=0.15%, 250=0.10%, 500=0.08%, 750=0.08%, 1000=0.17% | |
lat (msec) : 2=0.38%, 4=0.34%, 10=0.30%, 20=0.26%, 50=0.36% | |
lat (msec) : 100=0.18%, 250=0.20%, 500=0.15%, 750=0.30%, 1000=0.18% | |
cpu : usr=0.15%, sys=0.59%, ctx=5019, majf=0, minf=40 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=75286,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:35:00 2019 | |
read: IOPS=55, BW=3563KiB/s (3649kB/s)(418MiB/120002msec) | |
slat (nsec): min=0, max=633777k, avg=116582.49, stdev=7753908.85 | |
clat (nsec): min=0, max=613214k, avg=6368642.28, stdev=41422627.12 | |
lat (nsec): min=0, max=633913k, avg=6486171.40, stdev=42134167.47 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 748], 30.00th=[ 828], 40.00th=[ 916], | |
| 50.00th=[ 1064], 60.00th=[ 1224], 70.00th=[ 1592], | |
| 80.00th=[ 73216], 90.00th=[ 1335296], 95.00th=[ 10289152], | |
| 99.00th=[206569472], 99.50th=[350224384], 99.90th=[526385152], | |
| 99.95th=[583008256], 99.99th=[616562688] | |
bw ( KiB/s): min= 0, max= 5756, per=8.19%, avg=3562.50, stdev=3102.08, samples=2 | |
iops : min= 0, max= 89, avg=55.00, stdev=48.08, samples=2 | |
write: IOPS=56, BW=3638KiB/s (3726kB/s)(426MiB/120002msec) | |
slat (nsec): min=0, max=16871k, avg=187455.60, stdev=310103.99 | |
clat (nsec): min=0, max=1166.3M, avg=64088245.30, stdev=127371794.56 | |
lat (nsec): min=0, max=1166.4M, avg=64276809.91, stdev=127323659.95 | |
clat percentiles (usec): | |
| 1.00th=[ 1045], 5.00th=[ 1319], 10.00th=[ 1532], | |
| 20.00th=[ 2008], 30.00th=[ 2704], 40.00th=[ 3752], | |
| 50.00th=[ 6390], 60.00th=[ 16188], 70.00th=[ 59507], | |
| 80.00th=[ 101188], 90.00th=[ 173016], 95.00th=[ 316670], | |
| 99.00th=[ 666895], 99.50th=[ 792724], 99.90th=[ 918553], | |
| 99.95th=[ 952108], 99.99th=[1166017] | |
bw ( KiB/s): min= 0, max= 5860, per=78.63%, avg=3636.50, stdev=3144.50, samples=2 | |
iops : min= 0, max= 91, avg=56.50, stdev=48.79, samples=2 | |
lat (nsec) : 750=10.52%, 1000=12.57% | |
lat (usec) : 2=11.90%, 4=0.07%, 10=0.64%, 20=1.83%, 50=1.19% | |
lat (usec) : 100=1.39%, 250=1.77%, 500=1.04%, 750=0.42%, 1000=1.01% | |
lat (msec) : 2=10.89%, 4=11.91%, 10=7.83%, 20=3.60%, 50=4.21% | |
lat (msec) : 100=6.04%, 250=7.55%, 500=2.39%, 750=0.92%, 1000=0.30% | |
cpu : usr=0.07%, sys=0.30%, ctx=13525, majf=0, minf=31 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=6681,6819,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=42.5MiB/s (44.6MB/s), 3563KiB/s-39.0MiB/s (3649kB/s-40.9MB/s), io=5102MiB (5349MB), run=120002-120072msec | |
WRITE: bw=4626KiB/s (4737kB/s), 1000KiB/s-3638KiB/s (1024kB/s-3726kB/s), io=544MiB (570MB), run=120002-120408msec | |
Disk stats (read/write): | |
rbd245: ios=3467/8915, merge=0/1, ticks=551293/914532, in_queue=1459579, util=5.67% | |
<example-benchmark-fio-server-737-benchmark-pfzcw> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:35:00 2019 | |
write: IOPS=34, BW=2211KiB/s (2264kB/s)(262MiB/121542msec) | |
slat (nsec): min=0, max=279070k, avg=313105.26, stdev=4584630.48 | |
clat (nsec): min=0, max=9623.8M, avg=120787623.04, stdev=322490503.56 | |
lat (nsec): min=0, max=9623.8M, avg=121102071.47, stdev=322508076.74 | |
clat percentiles (usec): | |
| 1.00th=[ 1074], 5.00th=[ 1696], 10.00th=[ 2343], | |
| 20.00th=[ 3458], 30.00th=[ 4883], 40.00th=[ 6718], | |
| 50.00th=[ 10028], 60.00th=[ 16712], 70.00th=[ 57410], | |
| 80.00th=[ 229639], 90.00th=[ 400557], 95.00th=[ 566232], | |
| 99.00th=[ 893387], 99.50th=[ 935330], 99.90th=[2231370], | |
| 99.95th=[8556381], 99.99th=[9596568] | |
bw ( KiB/s): min= 0, max= 2521, per=39.07%, avg=2236.50, stdev=402.34, samples=2 | |
iops : min= 0, max= 39, avg=34.50, stdev= 6.36, samples=2 | |
lat (usec) : 250=0.02%, 500=0.02%, 750=0.33%, 1000=0.43% | |
lat (msec) : 2=6.48%, 4=16.85%, 10=25.81%, 20=12.82%, 50=6.34% | |
lat (msec) : 100=3.57%, 250=9.32%, 500=11.42%, 750=4.43%, 1000=1.74% | |
cpu : usr=0.03%, sys=0.15%, ctx=7087, majf=0, minf=31 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,4196,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:35:00 2019 | |
read: IOPS=98, BW=6322KiB/s (6473kB/s)(744MiB/120507msec) | |
slat (nsec): min=0, max=13961k, avg=41369.64, stdev=206220.35 | |
clat (nsec): min=0, max=1884.5M, avg=40508976.77, stdev=167911107.34 | |
lat (nsec): min=0, max=1884.5M, avg=40551550.07, stdev=167924859.19 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 764], | |
| 20.00th=[ 780], 30.00th=[ 828], 40.00th=[ 868], | |
| 50.00th=[ 924], 60.00th=[ 1192], 70.00th=[ 1496], | |
| 80.00th=[ 1564672], 90.00th=[ 10551296], 95.00th=[ 354418688], | |
| 99.00th=[ 901775360], 99.50th=[1002438656], 99.90th=[1333788672], | |
| 99.95th=[1535115264], 99.99th=[1837105152] | |
bw ( KiB/s): min= 0, max= 6847, per=64.89%, avg=6344.50, stdev=710.64, samples=2 | |
iops : min= 0, max= 106, avg=98.50, stdev=10.61, samples=2 | |
lat (nsec) : 750=8.55%, 1000=44.78% | |
lat (usec) : 2=18.65%, 4=0.16%, 10=0.05%, 20=0.62%, 50=0.21% | |
lat (usec) : 100=0.18%, 250=0.10%, 500=0.14%, 750=0.44%, 1000=2.25% | |
lat (msec) : 2=5.77%, 4=4.82%, 10=3.20%, 20=1.59%, 50=1.57% | |
lat (msec) : 100=0.82%, 250=0.71%, 500=1.29%, 750=2.08%, 1000=1.56% | |
cpu : usr=0.04%, sys=0.15%, ctx=6384, majf=0, minf=31 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=11900,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=14: Wed May 8 20:35:00 2019 | |
read: IOPS=54, BW=3471KiB/s (3555kB/s)(407MiB/120007msec) | |
slat (nsec): min=0, max=2714.2k, avg=33168.77, stdev=89049.83 | |
clat (nsec): min=0, max=1276.5M, avg=9409156.58, stdev=48225664.37 | |
lat (nsec): min=0, max=1276.6M, avg=9443278.32, stdev=48232813.61 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 772], 30.00th=[ 900], 40.00th=[ 1032], | |
| 50.00th=[ 1176], 60.00th=[ 1368], 70.00th=[ 88576], | |
| 80.00th=[ 1122304], 90.00th=[ 5931008], 95.00th=[ 46923776], | |
| 99.00th=[ 242221056], 99.50th=[ 383778816], 99.90th=[ 583008256], | |
| 99.95th=[ 608174080], 99.99th=[1283457024] | |
bw ( KiB/s): min= 0, max= 5833, per=35.49%, avg=3470.50, stdev=3341.08, samples=2 | |
iops : min= 0, max= 91, avg=54.00, stdev=52.33, samples=2 | |
write: IOPS=55, BW=3559KiB/s (3644kB/s)(417MiB/120007msec) | |
slat (nsec): min=0, max=8103.8k, avg=181515.33, stdev=260812.54 | |
clat (nsec): min=0, max=1065.1M, avg=62753756.72, stdev=126482037.44 | |
lat (nsec): min=0, max=1065.2M, avg=62936479.59, stdev=126442634.22 | |
clat percentiles (usec): | |
| 1.00th=[ 996], 5.00th=[ 1221], 10.00th=[ 1401], | |
| 20.00th=[ 1811], 30.00th=[ 2409], 40.00th=[ 3458], | |
| 50.00th=[ 5473], 60.00th=[ 13435], 70.00th=[ 43254], | |
| 80.00th=[ 90702], 90.00th=[ 164627], 95.00th=[ 387974], | |
| 99.00th=[ 599786], 99.50th=[ 675283], 99.90th=[ 843056], | |
| 99.95th=[ 910164], 99.99th=[1061159] | |
bw ( KiB/s): min= 0, max= 5939, per=62.15%, avg=3557.50, stdev=3367.95, samples=2 | |
iops : min= 0, max= 92, avg=55.00, stdev=52.33, samples=2 | |
lat (nsec) : 750=8.15%, 1000=10.53% | |
lat (usec) : 2=13.47%, 4=0.02%, 10=0.25%, 20=0.86%, 50=0.61% | |
lat (usec) : 100=0.84%, 250=0.90%, 500=0.42%, 750=0.88%, 1000=2.41% | |
lat (msec) : 2=14.72%, 4=11.69%, 10=8.41%, 20=3.60%, 50=5.27% | |
lat (msec) : 100=6.93%, 250=5.47%, 500=3.38%, 750=1.08%, 1000=0.10% | |
cpu : usr=0.08%, sys=0.30%, ctx=15450, majf=0, minf=33 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=6509,6670,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=9778KiB/s (10.0MB/s), 3471KiB/s-6322KiB/s (3555kB/s-6473kB/s), io=1151MiB (1207MB), run=120007-120507msec | |
WRITE: bw=5725KiB/s (5862kB/s), 2211KiB/s-3559KiB/s (2264kB/s-3644kB/s), io=680MiB (713MB), run=120007-121542msec | |
Disk stats (read/write): | |
rbd53: ios=5395/11625, merge=0/3, ticks=602058/947991, in_queue=1541538, util=4.56% | |
<example-benchmark-fio-server-653-benchmark-6cpgt> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:35:00 2019 | |
write: IOPS=34, BW=2218KiB/s (2271kB/s)(260MiB/120080msec) | |
slat (nsec): min=0, max=1186.4M, avg=1481401.31, stdev=33991422.05 | |
clat (nsec): min=0, max=2099.0M, avg=114488164.29, stdev=238384745.73 | |
lat (nsec): min=0, max=2099.4M, avg=115970057.69, stdev=240216513.94 | |
clat percentiles (usec): | |
| 1.00th=[ 1037], 5.00th=[ 1336], 10.00th=[ 1614], | |
| 20.00th=[ 2245], 30.00th=[ 3130], 40.00th=[ 4424], | |
| 50.00th=[ 7504], 60.00th=[ 18744], 70.00th=[ 54789], | |
| 80.00th=[ 125305], 90.00th=[ 463471], 95.00th=[ 675283], | |
| 99.00th=[1035994], 99.50th=[1149240], 99.90th=[1786774], | |
| 99.95th=[1870660], 99.99th=[2105541] | |
bw ( KiB/s): min= 0, max= 2634, per=39.17%, avg=2216.50, stdev=590.43, samples=2 | |
iops : min= 0, max= 41, avg=34.50, stdev= 9.19, samples=2 | |
lat (usec) : 750=0.10%, 1000=0.53% | |
lat (msec) : 2=15.54%, 4=20.92%, 10=16.50%, 20=7.36%, 50=8.49% | |
lat (msec) : 100=7.24%, 250=9.16%, 500=5.39%, 750=4.88%, 1000=2.77% | |
cpu : usr=0.04%, sys=0.15%, ctx=7482, majf=0, minf=31 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,4158,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:35:00 2019 | |
read: IOPS=519, BW=32.2MiB/s (33.8MB/s)(3869MiB/120140msec) | |
slat (nsec): min=0, max=1032.2M, avg=66123.48, stdev=6143379.66 | |
clat (nsec): min=0, max=2075.3M, avg=7681293.84, stdev=65084937.34 | |
lat (nsec): min=0, max=2075.4M, avg=7748304.75, stdev=65378311.95 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 724], | |
| 20.00th=[ 740], 30.00th=[ 780], 40.00th=[ 796], | |
| 50.00th=[ 844], 60.00th=[ 876], 70.00th=[ 916], | |
| 80.00th=[ 1240], 90.00th=[ 16768], 95.00th=[ 1417216], | |
| 99.00th=[ 304087040], 99.50th=[ 522190848], 99.90th=[ 893386752], | |
| 99.95th=[1027604480], 99.99th=[1317011456] | |
bw ( KiB/s): min= 0, max=52272, per=90.92%, avg=33012.50, stdev=27237.05, samples=2 | |
iops : min= 0, max= 824, avg=519.00, stdev=431.34, samples=2 | |
lat (nsec) : 750=23.18%, 1000=52.62% | |
lat (usec) : 2=6.72%, 4=0.13%, 10=1.52%, 20=7.48%, 50=1.36% | |
lat (usec) : 100=0.16%, 250=0.08%, 500=0.06%, 750=0.22%, 1000=0.74% | |
lat (msec) : 2=1.30%, 4=0.82%, 10=0.41%, 20=0.25%, 50=0.73% | |
lat (msec) : 100=0.75%, 250=0.36%, 500=0.55%, 750=0.34%, 1000=0.16% | |
cpu : usr=0.11%, sys=0.53%, ctx=8615, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=62370,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:35:00 2019 | |
read: IOPS=52, BW=3337KiB/s (3417kB/s)(391MiB/120013msec) | |
slat (nsec): min=0, max=599202k, avg=127308.18, stdev=7575305.82 | |
clat (nsec): min=0, max=668739k, avg=7431754.00, stdev=41464138.35 | |
lat (nsec): min=0, max=668798k, avg=7559998.10, stdev=42139523.63 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 780], 30.00th=[ 884], 40.00th=[ 1004], | |
| 50.00th=[ 1160], 60.00th=[ 1320], 70.00th=[ 22912], | |
| 80.00th=[ 888832], 90.00th=[ 3194880], 95.00th=[ 35913728], | |
| 99.00th=[204472320], 99.50th=[329252864], 99.90th=[530579456], | |
| 99.95th=[583008256], 99.99th=[666894336] | |
bw ( KiB/s): min= 0, max= 5328, per=9.19%, avg=3337.00, stdev=2815.70, samples=2 | |
iops : min= 0, max= 83, avg=52.00, stdev=43.84, samples=2 | |
write: IOPS=53, BW=3443KiB/s (3526kB/s)(404MiB/120013msec) | |
slat (nsec): min=0, max=6146.0k, avg=181618.18, stdev=222082.45 | |
clat (nsec): min=0, max=1423.9M, avg=67042871.01, stdev=133485106.00 | |
lat (nsec): min=0, max=1423.9M, avg=67225673.93, stdev=133436561.24 | |
clat percentiles (usec): | |
| 1.00th=[ 1029], 5.00th=[ 1237], 10.00th=[ 1385], | |
| 20.00th=[ 1745], 30.00th=[ 2245], 40.00th=[ 3097], | |
| 50.00th=[ 4948], 60.00th=[ 12518], 70.00th=[ 41681], | |
| 80.00th=[ 98042], 90.00th=[ 214959], 95.00th=[ 379585], | |
| 99.00th=[ 641729], 99.50th=[ 717226], 99.90th=[ 809501], | |
| 99.95th=[ 817890], 99.99th=[1417675] | |
bw ( KiB/s): min= 0, max= 5473, per=60.81%, avg=3441.50, stdev=2872.97, samples=2 | |
iops : min= 0, max= 85, avg=53.50, stdev=44.55, samples=2 | |
lat (nsec) : 750=7.70%, 1000=11.79% | |
lat (usec) : 2=13.52%, 4=0.05%, 10=0.30%, 20=0.98%, 50=0.92% | |
lat (usec) : 100=0.77%, 250=1.10%, 500=0.69%, 750=0.68%, 1000=2.14% | |
lat (msec) : 2=15.40%, 4=12.12%, 10=6.92%, 20=3.92%, 50=4.73% | |
lat (msec) : 100=5.37%, 250=6.36%, 500=3.18%, 750=1.25%, 1000=0.13% | |
cpu : usr=0.07%, sys=0.30%, ctx=14601, majf=0, minf=34 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=6257,6454,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=35.5MiB/s (37.2MB/s), 3337KiB/s-32.2MiB/s (3417kB/s-33.8MB/s), io=4260MiB (4467MB), run=120013-120140msec | |
WRITE: bw=5659KiB/s (5795kB/s), 2218KiB/s-3443KiB/s (2271kB/s-3526kB/s), io=664MiB (696MB), run=120013-120080msec | |
Disk stats (read/write): | |
rbd213: ios=5674/10923, merge=0/3, ticks=532365/860881, in_queue=1384865, util=8.80% | |
<example-benchmark-fio-server-666-benchmark-vhp22> | |
<example-benchmark-fio-server-674-benchmark-5pb2m> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:35:00 2019 | |
write: IOPS=41, BW=2641KiB/s (2704kB/s)(310MiB/120145msec) | |
slat (nsec): min=0, max=575779k, avg=600805.43, stdev=14798680.70 | |
clat (nsec): min=0, max=1310.3M, avg=96705119.71, stdev=167414675.40 | |
lat (nsec): min=0, max=1338.3M, avg=97306982.18, stdev=168344306.33 | |
clat percentiles (usec): | |
| 1.00th=[ 971], 5.00th=[ 1188], 10.00th=[ 1401], | |
| 20.00th=[ 1926], 30.00th=[ 2835], 40.00th=[ 5342], | |
| 50.00th=[ 19268], 60.00th=[ 78119], 70.00th=[ 102237], | |
| 80.00th=[ 127402], 90.00th=[ 267387], 95.00th=[ 463471], | |
| 99.00th=[ 834667], 99.50th=[ 943719], 99.90th=[1149240], | |
| 99.95th=[1300235], 99.99th=[1317012] | |
bw ( KiB/s): min= 0, max= 2877, per=42.41%, avg=2641.50, stdev=333.05, samples=2 | |
iops : min= 0, max= 44, avg=40.50, stdev= 4.95, samples=2 | |
lat (usec) : 250=0.02%, 1000=1.31% | |
lat (msec) : 2=19.74%, 4=14.77%, 10=9.97%, 20=4.56%, 50=4.38% | |
lat (msec) : 100=14.57%, 250=20.14%, 500=6.16%, 750=2.83%, 1000=1.25% | |
cpu : usr=0.06%, sys=0.17%, ctx=9207, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,4955,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:35:00 2019 | |
read: IOPS=486, BW=30.3MiB/s (31.8MB/s)(3637MiB/120064msec) | |
slat (nsec): min=0, max=878989k, avg=51948.57, stdev=4719575.63 | |
clat (nsec): min=0, max=1394.2M, avg=8192226.33, stdev=53775826.59 | |
lat (nsec): min=0, max=1394.2M, avg=8245018.59, stdev=53987800.95 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 748], 40.00th=[ 788], | |
| 50.00th=[ 828], 60.00th=[ 868], 70.00th=[ 924], | |
| 80.00th=[ 1192], 90.00th=[ 757760], 95.00th=[ 23199744], | |
| 99.00th=[267386880], 99.50th=[442499072], 99.90th=[717225984], | |
| 99.95th=[784334848], 99.99th=[952107008] | |
bw ( KiB/s): min= 0, max=50771, per=89.87%, avg=31032.50, stdev=27914.45, samples=2 | |
iops : min= 0, max= 796, avg=486.00, stdev=438.41, samples=2 | |
lat (nsec) : 750=30.36%, 1000=45.16% | |
lat (usec) : 2=8.76%, 4=0.08%, 10=1.60%, 20=3.14%, 50=0.19% | |
lat (usec) : 100=0.13%, 250=0.06%, 500=0.09%, 750=0.39%, 1000=1.05% | |
lat (msec) : 2=1.69%, 4=1.16%, 10=0.62%, 20=0.42%, 50=1.52% | |
lat (msec) : 100=1.69%, 250=0.83%, 500=0.72%, 750=0.28%, 1000=0.06% | |
cpu : usr=0.13%, sys=0.51%, ctx=12296, majf=0, minf=105 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=58366,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:35:00 2019 | |
read: IOPS=54, BW=3515KiB/s (3600kB/s)(412MiB/120011msec) | |
slat (nsec): min=0, max=3688.4k, avg=35129.82, stdev=104218.83 | |
clat (nsec): min=0, max=681349k, avg=8268770.25, stdev=41687470.94 | |
lat (nsec): min=0, max=681373k, avg=8304838.00, stdev=41694110.97 | |
clat percentiles (nsec): | |
| 1.00th=[ 724], 5.00th=[ 732], 10.00th=[ 748], | |
| 20.00th=[ 828], 30.00th=[ 924], 40.00th=[ 1064], | |
| 50.00th=[ 1208], 60.00th=[ 1384], 70.00th=[ 64256], | |
| 80.00th=[ 1089536], 90.00th=[ 5734400], 95.00th=[ 43778048], | |
| 99.00th=[214958080], 99.50th=[333447168], 99.90th=[509607936], | |
| 99.95th=[608174080], 99.99th=[683671552] | |
bw ( KiB/s): min= 0, max= 5158, per=10.18%, avg=3515.50, stdev=2322.85, samples=2 | |
iops : min= 0, max= 80, avg=54.50, stdev=36.06, samples=2 | |
write: IOPS=56, BW=3593KiB/s (3679kB/s)(421MiB/120011msec) | |
slat (nsec): min=0, max=566283k, avg=315230.86, stdev=8140260.33 | |
clat (nsec): min=0, max=1079.2M, avg=62967107.48, stdev=118454148.51 | |
lat (nsec): min=0, max=1079.3M, avg=63283412.71, stdev=118888184.71 | |
clat percentiles (usec): | |
| 1.00th=[ 1004], 5.00th=[ 1254], 10.00th=[ 1467], | |
| 20.00th=[ 1909], 30.00th=[ 2704], 40.00th=[ 4293], | |
| 50.00th=[ 8455], 60.00th=[ 22414], 70.00th=[ 65274], | |
| 80.00th=[ 102237], 90.00th=[ 158335], 95.00th=[ 304088], | |
| 99.00th=[ 591397], 99.50th=[ 750781], 99.90th=[ 935330], | |
| 99.95th=[ 952108], 99.99th=[1082131] | |
bw ( KiB/s): min= 0, max= 5207, per=57.64%, avg=3590.50, stdev=2286.08, samples=2 | |
iops : min= 0, max= 81, avg=55.50, stdev=36.06, samples=2 | |
lat (nsec) : 750=4.97%, 1000=12.73% | |
lat (usec) : 2=14.70%, 4=0.03%, 10=0.33%, 20=0.82%, 50=0.73% | |
lat (usec) : 100=0.86%, 250=1.03%, 500=0.58%, 750=0.76%, 1000=2.14% | |
lat (msec) : 2=13.36%, 4=10.56%, 10=7.76%, 20=4.22%, 50=5.60% | |
lat (msec) : 100=7.45%, 250=7.89%, 500=2.56%, 750=0.71%, 1000=0.24% | |
cpu : usr=0.07%, sys=0.33%, ctx=15735, majf=0, minf=33 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=6592,6734,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=33.7MiB/s (35.4MB/s), 3515KiB/s-30.3MiB/s (3600kB/s-31.8MB/s), io=4049MiB (4246MB), run=120011-120064msec | |
WRITE: bw=6230KiB/s (6379kB/s), 2641KiB/s-3593KiB/s (2704kB/s-3679kB/s), io=731MiB (766MB), run=120011-120145msec | |
Disk stats (read/write): | |
rbd173: ios=8283/12086, merge=0/1, ticks=553856/811994, in_queue=1355561, util=13.18% | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:35:00 2019 | |
write: IOPS=29, BW=1877KiB/s (1922kB/s)(222MiB/120954msec) | |
slat (nsec): min=0, max=1178.4M, avg=1070070.99, stdev=26922015.16 | |
clat (nsec): min=0, max=1811.8M, avg=135023864.34, stdev=288341801.46 | |
lat (nsec): min=0, max=1811.9M, avg=136095701.53, stdev=289170559.85 | |
clat percentiles (usec): | |
| 1.00th=[ 1029], 5.00th=[ 1287], 10.00th=[ 1532], | |
| 20.00th=[ 2024], 30.00th=[ 2704], 40.00th=[ 3851], | |
| 50.00th=[ 5997], 60.00th=[ 13435], 70.00th=[ 38536], | |
| 80.00th=[ 127402], 90.00th=[ 591397], 95.00th=[ 876610], | |
| 99.00th=[1249903], 99.50th=[1317012], 99.90th=[1635779], | |
| 99.95th=[1669333], 99.99th=[1803551] | |
bw ( KiB/s): min= 0, max= 2954, per=33.73%, avg=1889.50, stdev=1505.43, samples=2 | |
iops : min= 0, max= 46, avg=29.00, stdev=24.04, samples=2 | |
lat (usec) : 750=0.06%, 1000=0.73% | |
lat (msec) : 2=18.96%, 4=21.24%, 10=16.19%, 20=7.14%, 50=7.02% | |
lat (msec) : 100=4.51%, 250=9.37%, 500=3.02%, 750=4.91%, 1000=3.92% | |
cpu : usr=0.04%, sys=0.13%, ctx=6342, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,3545,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:35:00 2019 | |
read: IOPS=517, BW=32.2MiB/s (33.8MB/s)(3876MiB/120265msec) | |
slat (nsec): min=0, max=1009.8M, avg=58501.48, stdev=5796659.72 | |
clat (nsec): min=0, max=2075.2M, avg=7667833.39, stdev=70093238.96 | |
lat (nsec): min=0, max=2075.3M, avg=7727229.62, stdev=70340481.38 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 716], 10.00th=[ 724], | |
| 20.00th=[ 740], 30.00th=[ 772], 40.00th=[ 788], | |
| 50.00th=[ 804], 60.00th=[ 852], 70.00th=[ 884], | |
| 80.00th=[ 932], 90.00th=[ 8768], 95.00th=[ 790528], | |
| 99.00th=[ 375390208], 99.50th=[ 624951296], 99.90th=[ 943718400], | |
| 99.95th=[1044381696], 99.99th=[1300234240] | |
bw ( KiB/s): min= 0, max=51751, per=89.94%, avg=32982.50, stdev=26542.67, samples=2 | |
iops : min= 0, max= 812, avg=517.00, stdev=417.19, samples=2 | |
lat (nsec) : 750=24.97%, 1000=58.54% | |
lat (usec) : 2=6.06%, 4=0.17%, 10=0.98%, 20=3.16%, 50=0.53% | |
lat (usec) : 100=0.11%, 250=0.08%, 500=0.08%, 750=0.25%, 1000=0.54% | |
lat (msec) : 2=0.92%, 4=0.67%, 10=0.47%, 20=0.24%, 50=0.56% | |
lat (msec) : 100=0.48%, 250=0.13%, 500=0.38%, 750=0.41%, 1000=0.23% | |
cpu : usr=0.13%, sys=0.49%, ctx=7065, majf=0, minf=36 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=62257,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:35:00 2019 | |
read: IOPS=57, BW=3678KiB/s (3767kB/s)(431MiB/120039msec) | |
slat (nsec): min=0, max=753868k, avg=277793.85, stdev=12125320.64 | |
clat (nsec): min=0, max=806006k, avg=7819411.97, stdev=47157480.87 | |
lat (nsec): min=0, max=806066k, avg=8098127.39, stdev=48659920.24 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 764], 30.00th=[ 876], 40.00th=[ 996], | |
| 50.00th=[ 1144], 60.00th=[ 1320], 70.00th=[ 18560], | |
| 80.00th=[ 749568], 90.00th=[ 2637824], 95.00th=[ 26869760], | |
| 99.00th=[265289728], 99.50th=[387973120], 99.90th=[624951296], | |
| 99.95th=[675282944], 99.99th=[809500672] | |
bw ( KiB/s): min= 0, max= 5620, per=10.03%, avg=3679.00, stdev=2744.99, samples=2 | |
iops : min= 0, max= 87, avg=57.00, stdev=42.43, samples=2 | |
write: IOPS=58, BW=3753KiB/s (3844kB/s)(440MiB/120039msec) | |
slat (nsec): min=0, max=4929.1k, avg=192875.32, stdev=240267.68 | |
clat (nsec): min=0, max=1279.8M, avg=60267779.32, stdev=129578981.16 | |
lat (nsec): min=0, max=1279.9M, avg=60461861.12, stdev=129540203.48 | |
clat percentiles (usec): | |
| 1.00th=[ 1037], 5.00th=[ 1287], 10.00th=[ 1483], | |
| 20.00th=[ 1893], 30.00th=[ 2442], 40.00th=[ 3392], | |
| 50.00th=[ 5407], 60.00th=[ 11207], 70.00th=[ 30016], | |
| 80.00th=[ 85459], 90.00th=[ 170918], 95.00th=[ 346031], | |
| 99.00th=[ 666895], 99.50th=[ 784335], 99.90th=[ 935330], | |
| 99.95th=[ 968885], 99.99th=[1283458] | |
bw ( KiB/s): min= 0, max= 5680, per=66.99%, avg=3752.50, stdev=2725.90, samples=2 | |
iops : min= 0, max= 88, avg=58.00, stdev=42.43, samples=2 | |
lat (nsec) : 750=8.39%, 1000=11.65% | |
lat (usec) : 2=13.20%, 4=0.04%, 10=0.39%, 20=1.22%, 50=0.88% | |
lat (usec) : 100=0.91%, 250=1.23%, 500=0.78%, 750=0.98%, 1000=1.87% | |
lat (msec) : 2=13.50%, 4=12.72%, 10=8.19%, 20=4.71%, 50=4.54% | |
lat (msec) : 100=5.34%, 250=5.01%, 500=3.30%, 750=0.83%, 1000=0.32% | |
cpu : usr=0.08%, sys=0.33%, ctx=15516, majf=0, minf=31 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=6899,7037,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=35.8MiB/s (37.6MB/s), 3678KiB/s-32.2MiB/s (3767kB/s-33.8MB/s), io=4307MiB (4516MB), run=120039-120265msec | |
WRITE: bw=5602KiB/s (5737kB/s), 1877KiB/s-3753KiB/s (1922kB/s-3844kB/s), io=662MiB (694MB), run=120039-120954msec | |
Disk stats (read/write): | |
rbd205: ios=5496/10983, merge=0/1, ticks=553117/872605, in_queue=1417429, util=8.18% | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:35:00 2019 | |
write: IOPS=22, BW=1415KiB/s (1449kB/s)(167MiB/120791msec) | |
slat (nsec): min=0, max=560200k, avg=579809.47, stdev=13166429.06 | |
clat (nsec): min=0, max=3879.5M, avg=180201402.17, stdev=290186331.72 | |
lat (nsec): min=0, max=3879.7M, avg=180784362.43, stdev=290262817.96 | |
clat percentiles (usec): | |
| 1.00th=[ 1565], 5.00th=[ 2573], 10.00th=[ 3425], | |
| 20.00th=[ 5407], 30.00th=[ 8586], 40.00th=[ 13829], | |
| 50.00th=[ 22152], 60.00th=[ 80217], 70.00th=[ 214959], | |
| 80.00th=[ 371196], 90.00th=[ 583009], 95.00th=[ 708838], | |
| 99.00th=[ 985662], 99.50th=[1266680], 99.90th=[3875537], | |
| 99.95th=[3875537], 99.99th=[3875537] | |
bw ( KiB/s): min= 0, max= 2022, per=28.42%, avg=1421.00, stdev=849.94, samples=2 | |
iops : min= 0, max= 31, avg=21.50, stdev=13.44, samples=2 | |
lat (usec) : 4=0.07%, 250=0.04%, 500=0.04%, 1000=0.04% | |
lat (msec) : 2=2.77%, 4=10.09%, 10=20.17%, 20=14.62%, 50=9.75% | |
lat (msec) : 100=4.50%, 250=10.16%, 500=14.59%, 750=8.92%, 1000=3.45% | |
cpu : usr=0.03%, sys=0.10%, ctx=4530, majf=0, minf=31 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,2667,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:35:00 2019 | |
read: IOPS=124, BW=7969KiB/s (8160kB/s)(938MiB/120482msec) | |
slat (nsec): min=0, max=11516k, avg=36049.89, stdev=185515.83 | |
clat (nsec): min=0, max=1889.0M, avg=32130791.70, stdev=149359827.92 | |
lat (nsec): min=0, max=1890.1M, avg=32167996.44, stdev=149380240.45 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 740], | |
| 20.00th=[ 780], 30.00th=[ 804], 40.00th=[ 852], | |
| 50.00th=[ 892], 60.00th=[ 956], 70.00th=[ 1240], | |
| 80.00th=[ 30592], 90.00th=[ 5603328], 95.00th=[ 81264640], | |
| 99.00th=[ 834666496], 99.50th=[ 960495616], 99.90th=[1216348160], | |
| 99.95th=[1367343104], 99.99th=[1853882368] | |
bw ( KiB/s): min= 0, max= 8257, per=69.65%, avg=7998.50, stdev=365.57, samples=2 | |
iops : min= 0, max= 129, avg=124.50, stdev= 6.36, samples=2 | |
lat (nsec) : 750=11.93%, 1000=50.52% | |
lat (usec) : 2=16.29%, 4=0.27%, 10=0.13%, 20=0.79%, 50=0.21% | |
lat (usec) : 100=0.11%, 250=0.09%, 500=0.09%, 750=0.31%, 1000=1.35% | |
lat (msec) : 2=3.72%, 4=3.19%, 10=2.74%, 20=1.33%, 50=1.45% | |
lat (msec) : 100=0.61%, 250=0.61%, 500=0.96%, 750=1.81%, 1000=1.11% | |
cpu : usr=0.04%, sys=0.17%, ctx=5951, majf=0, minf=31 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=14999,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:35:00 2019 | |
read: IOPS=55, BW=3529KiB/s (3614kB/s)(414MiB/120011msec) | |
slat (nsec): min=0, max=3964.5k, avg=26293.80, stdev=81795.50 | |
clat (nsec): min=0, max=739318k, avg=10488878.55, stdev=55028154.30 | |
lat (nsec): min=0, max=739351k, avg=10516103.18, stdev=55036378.95 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 772], 30.00th=[ 876], 40.00th=[ 988], | |
| 50.00th=[ 1144], 60.00th=[ 1288], 70.00th=[ 1768], | |
| 80.00th=[ 692224], 90.00th=[ 4816896], 95.00th=[ 44302336], | |
| 99.00th=[329252864], 99.50th=[450887680], 99.90th=[624951296], | |
| 99.95th=[675282944], 99.99th=[742391808] | |
bw ( KiB/s): min= 0, max= 5220, per=30.73%, avg=3529.00, stdev=2391.44, samples=2 | |
iops : min= 0, max= 81, avg=54.50, stdev=37.48, samples=2 | |
write: IOPS=56, BW=3609KiB/s (3695kB/s)(423MiB/120011msec) | |
slat (nsec): min=0, max=9699.0k, avg=182579.58, stdev=251670.03 | |
clat (nsec): min=0, max=948820k, avg=60636613.95, stdev=123178432.93 | |
lat (nsec): min=0, max=948884k, avg=60820348.26, stdev=123136199.28 | |
clat percentiles (usec): | |
| 1.00th=[ 1012], 5.00th=[ 1221], 10.00th=[ 1401], 20.00th=[ 1778], | |
| 30.00th=[ 2343], 40.00th=[ 3523], 50.00th=[ 6652], 60.00th=[ 15401], | |
| 70.00th=[ 37487], 80.00th=[ 90702], 90.00th=[166724], 95.00th=[333448], | |
| 99.00th=[641729], 99.50th=[717226], 99.90th=[893387], 99.95th=[918553], | |
| 99.99th=[952108] | |
bw ( KiB/s): min= 0, max= 5319, per=72.13%, avg=3606.50, stdev=2421.84, samples=2 | |
iops : min= 0, max= 83, avg=56.00, stdev=38.18, samples=2 | |
lat (nsec) : 750=8.29%, 1000=11.81% | |
lat (usec) : 2=14.65%, 4=0.04%, 10=0.39%, 20=1.12%, 50=0.75% | |
lat (usec) : 100=0.76%, 250=1.17%, 500=0.37%, 750=0.46%, 1000=1.48% | |
lat (msec) : 2=13.97%, 4=10.30%, 10=7.65%, 20=4.71%, 50=5.79% | |
lat (msec) : 100=5.75%, 250=6.33%, 500=3.01%, 750=1.09%, 1000=0.13% | |
cpu : usr=0.07%, sys=0.31%, ctx=14900, majf=0, minf=31 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=6618,6764,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=11.2MiB/s (11.8MB/s), 3529KiB/s-7969KiB/s (3614kB/s-8160kB/s), io=1351MiB (1417MB), run=120011-120482msec | |
WRITE: bw=5000KiB/s (5120kB/s), 1415KiB/s-3609KiB/s (1449kB/s-3695kB/s), io=590MiB (618MB), run=120011-120791msec | |
Disk stats (read/write): | |
rbd184: ios=4567/10108, merge=0/2, ticks=592804/909666, in_queue=1495157, util=4.30% | |
<example-benchmark-fio-server-400-benchmark-fj44s> | |
<example-benchmark-fio-server-644-benchmark-z4l9v> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:35:00 2019 | |
write: IOPS=19, BW=1259KiB/s (1289kB/s)(148MiB/120652msec) | |
slat (nsec): min=0, max=661885k, avg=818240.79, stdev=16567186.30 | |
clat (nsec): min=0, max=1579.3M, avg=202990774.12, stdev=318293903.98 | |
lat (nsec): min=0, max=1579.9M, avg=203809884.73, stdev=318373055.97 | |
clat percentiles (usec): | |
| 1.00th=[ 1254], 5.00th=[ 1827], 10.00th=[ 2343], | |
| 20.00th=[ 3425], 30.00th=[ 5211], 40.00th=[ 7963], | |
| 50.00th=[ 12911], 60.00th=[ 25297], 70.00th=[ 287310], | |
| 80.00th=[ 434111], 90.00th=[ 692061], 95.00th=[ 893387], | |
| 99.00th=[1283458], 99.50th=[1434452], 99.90th=[1484784], | |
| 99.95th=[1484784], 99.99th=[1585447] | |
bw ( KiB/s): min= 0, max= 1388, per=22.62%, avg=1259.00, stdev=182.43, samples=2 | |
iops : min= 0, max= 21, avg=19.00, stdev= 2.83, samples=2 | |
lat (usec) : 750=0.08%, 1000=0.17% | |
lat (msec) : 2=6.12%, 4=17.30%, 10=20.55%, 20=12.62%, 50=8.31% | |
lat (msec) : 100=1.60%, 250=1.18%, 500=14.56%, 750=9.70%, 1000=4.68% | |
cpu : usr=0.03%, sys=0.09%, ctx=4052, majf=0, minf=31 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,2370,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:35:00 2019 | |
read: IOPS=204, BW=12.8MiB/s (13.4MB/s)(1540MiB/120482msec) | |
slat (nsec): min=0, max=123504k, avg=30115.36, stdev=798302.18 | |
clat (nsec): min=0, max=1866.4M, avg=19516391.16, stdev=117399582.99 | |
lat (nsec): min=0, max=1866.4M, avg=19547550.88, stdev=117418991.93 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 772], 30.00th=[ 788], 40.00th=[ 812], | |
| 50.00th=[ 852], 60.00th=[ 876], 70.00th=[ 924], | |
| 80.00th=[ 1240], 90.00th=[ 1368064], 95.00th=[ 9895936], | |
| 99.00th=[ 734003200], 99.50th=[ 884998144], 99.90th=[1166016512], | |
| 99.95th=[1283457024], 99.99th=[1837105152] | |
bw ( KiB/s): min= 0, max=16005, per=75.84%, avg=13136.50, stdev=4056.67, samples=2 | |
iops : min= 0, max= 250, avg=205.00, stdev=63.64, samples=2 | |
lat (nsec) : 750=12.70%, 1000=61.83% | |
lat (usec) : 2=11.12%, 4=0.20%, 10=0.34%, 20=1.36%, 50=0.21% | |
lat (usec) : 100=0.10%, 250=0.12%, 500=0.05%, 750=0.19%, 1000=0.78% | |
lat (msec) : 2=2.21%, 4=2.08%, 10=1.74%, 20=0.78%, 50=0.82% | |
lat (msec) : 100=0.38%, 250=0.33%, 500=0.72%, 750=1.06%, 1000=0.62% | |
cpu : usr=0.06%, sys=0.24%, ctx=6050, majf=0, minf=31 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=24637,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=14: Wed May 8 20:35:00 2019 | |
read: IOPS=66, BW=4250KiB/s (4352kB/s)(498MiB/120010msec) | |
slat (nsec): min=0, max=2540.8k, avg=24466.37, stdev=63425.35 | |
clat (nsec): min=0, max=1096.5M, avg=8256931.65, stdev=50119470.91 | |
lat (nsec): min=0, max=1096.6M, avg=8282370.65, stdev=50126584.41 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 756], 30.00th=[ 852], 40.00th=[ 940], | |
| 50.00th=[ 1080], 60.00th=[ 1224], 70.00th=[ 1560], | |
| 80.00th=[ 168960], 90.00th=[ 2473984], 95.00th=[ 25821184], | |
| 99.00th=[ 274726912], 99.50th=[ 425721856], 99.90th=[ 616562688], | |
| 99.95th=[ 742391808], 99.99th=[1098907648] | |
bw ( KiB/s): min= 0, max= 6593, per=24.54%, avg=4250.50, stdev=3312.80, samples=2 | |
iops : min= 0, max= 103, avg=66.00, stdev=52.33, samples=2 | |
write: IOPS=67, BW=4330KiB/s (4434kB/s)(507MiB/120010msec) | |
slat (nsec): min=0, max=7458.3k, avg=189853.89, stdev=259526.74 | |
clat (nsec): min=0, max=894153k, avg=50960770.68, stdev=106485643.26 | |
lat (nsec): min=0, max=894188k, avg=51151899.21, stdev=106440642.32 | |
clat percentiles (usec): | |
| 1.00th=[ 979], 5.00th=[ 1221], 10.00th=[ 1385], 20.00th=[ 1745], | |
| 30.00th=[ 2245], 40.00th=[ 3064], 50.00th=[ 4752], 60.00th=[ 10290], | |
| 70.00th=[ 25822], 80.00th=[ 81265], 90.00th=[141558], 95.00th=[267387], | |
| 99.00th=[557843], 99.50th=[650118], 99.90th=[742392], 99.95th=[767558], | |
| 99.99th=[893387] | |
bw ( KiB/s): min= 0, max= 6640, per=77.77%, avg=4328.00, stdev=3269.66, samples=2 | |
iops : min= 0, max= 103, avg=67.00, stdev=50.91, samples=2 | |
lat (nsec) : 750=8.95%, 1000=12.87% | |
lat (usec) : 2=13.44%, 4=0.02%, 10=0.40%, 20=1.39%, 50=0.77% | |
lat (usec) : 100=1.04%, 250=1.31%, 500=0.63%, 750=0.53%, 1000=1.59% | |
lat (msec) : 2=14.34%, 4=11.53%, 10=7.63%, 20=4.38%, 50=4.83% | |
lat (msec) : 100=5.20%, 250=5.91%, 500=2.44%, 750=0.76%, 1000=0.06% | |
cpu : usr=0.07%, sys=0.38%, ctx=17220, majf=0, minf=31 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=7970,8116,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=16.9MiB/s (17.7MB/s), 4250KiB/s-12.8MiB/s (4352kB/s-13.4MB/s), io=2038MiB (2137MB), run=120010-120482msec | |
WRITE: bw=5565KiB/s (5699kB/s), 1259KiB/s-4330KiB/s (1289kB/s-4434kB/s), io=656MiB (688MB), run=120010-120652msec | |
Disk stats (read/write): | |
rbd162: ios=4669/11042, merge=0/1, ticks=588851/900875, in_queue=1481867, util=5.90% | |
<example-benchmark-fio-server-216-benchmark-88f76> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:35:00 2019 | |
write: IOPS=44, BW=2862KiB/s (2931kB/s)(336MiB/120068msec) | |
slat (nsec): min=0, max=556126k, avg=395599.01, stdev=9573679.68 | |
clat (nsec): min=0, max=1375.0M, avg=89644510.34, stdev=147243605.16 | |
lat (nsec): min=0, max=1375.2M, avg=90041328.80, stdev=147426996.10 | |
clat percentiles (usec): | |
| 1.00th=[ 979], 5.00th=[ 1172], 10.00th=[ 1352], | |
| 20.00th=[ 1762], 30.00th=[ 2507], 40.00th=[ 5014], | |
| 50.00th=[ 21103], 60.00th=[ 68682], 70.00th=[ 96994], | |
| 80.00th=[ 127402], 90.00th=[ 258999], 95.00th=[ 421528], | |
| 99.00th=[ 708838], 99.50th=[ 750781], 99.90th=[ 918553], | |
| 99.95th=[1132463], 99.99th=[1367344] | |
bw ( KiB/s): min= 0, max= 3492, per=45.92%, avg=2861.00, stdev=892.37, samples=2 | |
iops : min= 0, max= 54, avg=44.00, stdev=14.14, samples=2 | |
lat (usec) : 4=0.02%, 500=0.04%, 750=0.07%, 1000=1.12% | |
lat (msec) : 2=22.93%, 4=13.12%, 10=7.85%, 20=4.34%, 50=6.04% | |
lat (msec) : 100=15.64%, 250=18.55%, 500=6.93%, 750=2.80%, 1000=0.54% | |
cpu : usr=0.07%, sys=0.18%, ctx=10180, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,5365,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:35:00 2019 | |
read: IOPS=305, BW=19.1MiB/s (20.0MB/s)(2292MiB/120015msec) | |
slat (nsec): min=0, max=510621k, avg=52091.46, stdev=3300689.27 | |
clat (nsec): min=0, max=1192.9M, avg=13093072.10, stdev=61828654.90 | |
lat (nsec): min=0, max=1192.9M, avg=13145990.94, stdev=61925908.23 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 740], 40.00th=[ 780], | |
| 50.00th=[ 844], 60.00th=[ 908], 70.00th=[ 1128], | |
| 80.00th=[ 60672], 90.00th=[ 15400960], 95.00th=[ 60555264], | |
| 99.00th=[362807296], 99.50th=[484442112], 99.90th=[683671552], | |
| 99.95th=[734003200], 99.99th=[826277888] | |
bw ( KiB/s): min= 0, max=24684, per=85.72%, avg=19554.50, stdev=7254.21, samples=2 | |
iops : min= 0, max= 386, avg=305.50, stdev=113.84, samples=2 | |
lat (nsec) : 750=35.59%, 1000=30.25% | |
lat (usec) : 2=12.30%, 4=0.03%, 10=0.67%, 20=0.68%, 50=0.45% | |
lat (usec) : 100=0.10%, 250=0.06%, 500=0.31%, 750=1.12%, 1000=2.37% | |
lat (msec) : 2=3.13%, 4=1.82%, 10=0.85%, 20=0.52%, 50=3.62% | |
lat (msec) : 100=3.02%, 250=1.50%, 500=1.20%, 750=0.40%, 1000=0.04% | |
cpu : usr=0.10%, sys=0.36%, ctx=14507, majf=0, minf=31 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=36693,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:35:00 2019 | |
read: IOPS=50, BW=3258KiB/s (3337kB/s)(382MiB/120007msec) | |
slat (nsec): min=0, max=64156k, avg=44426.52, stdev=826886.07 | |
clat (nsec): min=0, max=612522k, avg=7346455.67, stdev=38544224.56 | |
lat (nsec): min=0, max=612553k, avg=7391819.68, stdev=38557927.00 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 732], 10.00th=[ 740], | |
| 20.00th=[ 812], 30.00th=[ 900], 40.00th=[ 1020], | |
| 50.00th=[ 1176], 60.00th=[ 1336], 70.00th=[ 33024], | |
| 80.00th=[ 946176], 90.00th=[ 4489216], 95.00th=[ 39059456], | |
| 99.00th=[183500800], 99.50th=[295698432], 99.90th=[501219328], | |
| 99.95th=[526385152], 99.99th=[616562688] | |
bw ( KiB/s): min= 0, max= 4812, per=14.28%, avg=3258.00, stdev=2197.69, samples=2 | |
iops : min= 0, max= 75, avg=50.50, stdev=34.65, samples=2 | |
write: IOPS=52, BW=3370KiB/s (3451kB/s)(395MiB/120007msec) | |
slat (nsec): min=0, max=240431k, avg=228343.91, stdev=3035908.98 | |
clat (nsec): min=0, max=1074.2M, avg=68752131.71, stdev=128360758.51 | |
lat (nsec): min=0, max=1074.3M, avg=68981714.33, stdev=128329619.31 | |
clat percentiles (usec): | |
| 1.00th=[ 1004], 5.00th=[ 1237], 10.00th=[ 1467], | |
| 20.00th=[ 2008], 30.00th=[ 2868], 40.00th=[ 4555], | |
| 50.00th=[ 10159], 60.00th=[ 22152], 70.00th=[ 66847], | |
| 80.00th=[ 105382], 90.00th=[ 187696], 95.00th=[ 350225], | |
| 99.00th=[ 650118], 99.50th=[ 734004], 99.90th=[ 834667], | |
| 99.95th=[ 893387], 99.99th=[1082131] | |
bw ( KiB/s): min= 0, max= 4920, per=54.05%, avg=3367.50, stdev=2195.57, samples=2 | |
iops : min= 0, max= 76, avg=52.00, stdev=33.94, samples=2 | |
lat (nsec) : 750=5.93%, 1000=13.00% | |
lat (usec) : 2=14.02%, 4=0.05%, 10=0.27%, 20=0.83%, 50=0.79% | |
lat (usec) : 100=0.76%, 250=0.79%, 500=0.92%, 750=0.79%, 1000=1.89% | |
lat (msec) : 2=12.49%, 4=10.62%, 10=7.48%, 20=5.36%, 50=5.31% | |
lat (msec) : 100=6.99%, 250=7.43%, 500=3.07%, 750=1.05%, 1000=0.18% | |
cpu : usr=0.06%, sys=0.30%, ctx=14609, majf=0, minf=31 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=6111,6316,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=22.3MiB/s (23.4MB/s), 3258KiB/s-19.1MiB/s (3337kB/s-20.0MB/s), io=2674MiB (2804MB), run=120007-120015msec | |
WRITE: bw=6230KiB/s (6380kB/s), 2862KiB/s-3370KiB/s (2931kB/s-3451kB/s), io=731MiB (766MB), run=120007-120068msec | |
Disk stats (read/write): | |
rbd246: ios=9675/12516, merge=0/1, ticks=561703/838638, in_queue=1389101, util=15.18% | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:35:00 2019 | |
write: IOPS=27, BW=1792KiB/s (1835kB/s)(211MiB/120300msec) | |
slat (nsec): min=0, max=710004k, avg=768267.47, stdev=17305830.82 | |
clat (nsec): min=0, max=2194.6M, avg=142987318.59, stdev=256277381.03 | |
lat (nsec): min=0, max=2194.7M, avg=143756261.10, stdev=257049780.78 | |
clat percentiles (usec): | |
| 1.00th=[ 1123], 5.00th=[ 1418], 10.00th=[ 1696], | |
| 20.00th=[ 2409], 30.00th=[ 3326], 40.00th=[ 5014], | |
| 50.00th=[ 9110], 60.00th=[ 38011], 70.00th=[ 105382], | |
| 80.00th=[ 221250], 90.00th=[ 526386], 95.00th=[ 725615], | |
| 99.00th=[1115685], 99.50th=[1249903], 99.90th=[1417675], | |
| 99.95th=[1870660], 99.99th=[2197816] | |
bw ( KiB/s): min= 0, max= 2231, per=31.72%, avg=1794.00, stdev=618.01, samples=2 | |
iops : min= 0, max= 34, avg=27.50, stdev= 9.19, samples=2 | |
lat (usec) : 1000=0.27% | |
lat (msec) : 2=14.38%, 4=20.68%, 10=15.75%, 20=5.61%, 50=4.58% | |
lat (msec) : 100=7.34%, 250=13.13%, 500=7.49%, 750=6.24%, 1000=3.00% | |
cpu : usr=0.03%, sys=0.12%, ctx=6249, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,3366,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:35:00 2019 | |
read: IOPS=486, BW=30.3MiB/s (31.8MB/s)(3644MiB/120302msec) | |
slat (nsec): min=0, max=481883k, avg=40883.35, stdev=3292713.26 | |
clat (nsec): min=0, max=2142.9M, avg=8229375.61, stdev=64916449.31 | |
lat (nsec): min=0, max=2143.1M, avg=8271109.54, stdev=65013247.85 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 724], | |
| 20.00th=[ 740], 30.00th=[ 772], 40.00th=[ 796], | |
| 50.00th=[ 844], 60.00th=[ 876], 70.00th=[ 916], | |
| 80.00th=[ 1096], 90.00th=[ 12736], 95.00th=[ 2039808], | |
| 99.00th=[ 354418688], 99.50th=[ 505413632], 99.90th=[ 859832320], | |
| 99.95th=[ 960495616], 99.99th=[1384120320] | |
bw ( KiB/s): min= 0, max=47390, per=89.35%, avg=31095.00, stdev=23044.61, samples=2 | |
iops : min= 0, max= 743, avg=487.00, stdev=362.04, samples=2 | |
lat (nsec) : 750=26.66%, 1000=51.00% | |
lat (usec) : 2=7.21%, 4=0.10%, 10=1.34%, 20=5.40%, 50=0.82% | |
lat (usec) : 100=0.15%, 250=0.08%, 500=0.08%, 750=0.24%, 1000=0.64% | |
lat (msec) : 2=1.24%, 4=0.96%, 10=0.53%, 20=0.22%, 50=0.79% | |
lat (msec) : 100=0.88%, 250=0.44%, 500=0.71%, 750=0.34%, 1000=0.13% | |
cpu : usr=0.13%, sys=0.48%, ctx=8589, majf=0, minf=99 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=58472,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:35:00 2019 | |
read: IOPS=59, BW=3790KiB/s (3881kB/s)(444MiB/120012msec) | |
slat (nsec): min=0, max=2853.8k, avg=26733.30, stdev=79399.49 | |
clat (nsec): min=0, max=768103k, avg=6051219.37, stdev=36611268.74 | |
lat (nsec): min=0, max=768124k, avg=6078828.42, stdev=36617018.80 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 764], 30.00th=[ 852], 40.00th=[ 948], | |
| 50.00th=[ 1096], 60.00th=[ 1240], 70.00th=[ 9536], | |
| 80.00th=[ 370688], 90.00th=[ 2605056], 95.00th=[ 27394048], | |
| 99.00th=[139460608], 99.50th=[252706816], 99.90th=[526385152], | |
| 99.95th=[641728512], 99.99th=[767557632] | |
bw ( KiB/s): min= 0, max= 6400, per=10.89%, avg=3790.00, stdev=3691.10, samples=2 | |
iops : min= 0, max= 100, avg=59.00, stdev=57.98, samples=2 | |
write: IOPS=60, BW=3873KiB/s (3966kB/s)(454MiB/120012msec) | |
slat (nsec): min=0, max=368178k, avg=260495.36, stdev=4957786.34 | |
clat (nsec): min=0, max=1192.9M, avg=60005346.24, stdev=128875303.67 | |
lat (nsec): min=0, max=1192.9M, avg=60267041.02, stdev=129001065.91 | |
clat percentiles (usec): | |
| 1.00th=[ 1012], 5.00th=[ 1254], 10.00th=[ 1467], | |
| 20.00th=[ 1942], 30.00th=[ 2573], 40.00th=[ 3589], | |
| 50.00th=[ 5473], 60.00th=[ 11600], 70.00th=[ 29230], | |
| 80.00th=[ 90702], 90.00th=[ 160433], 95.00th=[ 333448], | |
| 99.00th=[ 650118], 99.50th=[ 725615], 99.90th=[ 876610], | |
| 99.95th=[1052771], 99.99th=[1199571] | |
bw ( KiB/s): min= 0, max= 6501, per=68.45%, avg=3871.00, stdev=3719.38, samples=2 | |
iops : min= 0, max= 101, avg=60.00, stdev=57.98, samples=2 | |
lat (nsec) : 750=8.53%, 1000=13.09% | |
lat (usec) : 2=12.74%, 4=0.03%, 10=0.36%, 20=1.38%, 50=0.77% | |
lat (usec) : 100=1.00%, 250=1.13%, 500=0.86%, 750=0.65%, 1000=1.64% | |
lat (msec) : 2=12.52%, 4=12.26%, 10=8.76%, 20=4.87%, 50=4.41% | |
lat (msec) : 100=5.01%, 250=6.32%, 500=2.24%, 750=1.23%, 1000=0.17% | |
cpu : usr=0.07%, sys=0.33%, ctx=15734, majf=0, minf=33 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=7108,7259,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=33.0MiB/s (35.6MB/s), 3790KiB/s-30.3MiB/s (3881kB/s-31.8MB/s), io=4088MiB (4287MB), run=120012-120302msec | |
WRITE: bw=5656KiB/s (5791kB/s), 1792KiB/s-3873KiB/s (1835kB/s-3966kB/s), io=664MiB (697MB), run=120012-120300msec | |
Disk stats (read/write): | |
rbd62: ios=6007/10900, merge=0/1, ticks=553071/887900, in_queue=1432491, util=9.76% | |
<example-benchmark-fio-server-7-benchmark-n8krm> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:35:00 2019 | |
write: IOPS=50, BW=3256KiB/s (3335kB/s)(382MiB/120041msec) | |
slat (nsec): min=0, max=754365k, avg=524418.01, stdev=13825228.46 | |
clat (nsec): min=0, max=1437.4M, avg=78366317.32, stdev=145975603.65 | |
lat (nsec): min=0, max=1438.1M, avg=78891857.37, stdev=146417463.46 | |
clat percentiles (usec): | |
| 1.00th=[ 963], 5.00th=[ 1188], 10.00th=[ 1369], | |
| 20.00th=[ 1762], 30.00th=[ 2442], 40.00th=[ 3621], | |
| 50.00th=[ 6259], 60.00th=[ 18220], 70.00th=[ 81265], | |
| 80.00th=[ 112722], 90.00th=[ 250610], 95.00th=[ 400557], | |
| 99.00th=[ 700449], 99.50th=[ 759170], 99.90th=[ 901776], | |
| 99.95th=[ 926942], 99.99th=[1434452] | |
bw ( KiB/s): min= 0, max= 3328, per=42.00%, avg=3256.00, stdev=101.82, samples=2 | |
iops : min= 0, max= 52, avg=50.50, stdev= 2.12, samples=2 | |
lat (usec) : 500=0.02%, 750=0.07%, 1000=1.54% | |
lat (msec) : 2=22.49%, 4=18.21%, 10=13.19%, 20=5.03%, 50=3.37% | |
lat (msec) : 100=12.14%, 250=13.94%, 500=6.75%, 750=2.74%, 1000=0.54% | |
cpu : usr=0.06%, sys=0.21%, ctx=11444, majf=0, minf=26 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,6105,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:35:00 2019 | |
read: IOPS=344, BW=21.5MiB/s (22.6MB/s)(2581MiB/120018msec) | |
slat (nsec): min=0, max=507784k, avg=64993.68, stdev=4454457.57 | |
clat (nsec): min=0, max=1336.8M, avg=11573441.71, stdev=60446260.20 | |
lat (nsec): min=0, max=1336.8M, avg=11639287.24, stdev=60613041.39 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 724], | |
| 20.00th=[ 724], 30.00th=[ 732], 40.00th=[ 748], | |
| 50.00th=[ 820], 60.00th=[ 884], 70.00th=[ 1048], | |
| 80.00th=[ 11968], 90.00th=[ 3391488], 95.00th=[ 57409536], | |
| 99.00th=[337641472], 99.50th=[480247808], 99.90th=[750780416], | |
| 99.95th=[809500672], 99.99th=[977272832] | |
bw ( KiB/s): min= 0, max=31315, per=83.20%, avg=22024.00, stdev=13139.46, samples=2 | |
iops : min= 0, max= 490, avg=344.00, stdev=206.48, samples=2 | |
lat (nsec) : 750=41.49%, 1000=27.40% | |
lat (usec) : 2=9.78%, 4=0.07%, 10=1.14%, 20=0.61%, 50=0.39% | |
lat (usec) : 100=0.12%, 250=0.05%, 500=0.34%, 750=1.01%, 1000=2.49% | |
lat (msec) : 2=3.47%, 4=1.99%, 10=1.02%, 20=0.42%, 50=2.40% | |
lat (msec) : 100=3.23%, 250=1.24%, 500=0.93%, 750=0.34%, 1000=0.09% | |
cpu : usr=0.11%, sys=0.39%, ctx=15643, majf=0, minf=64 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=41370,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:35:00 2019 | |
read: IOPS=69, BW=4450KiB/s (4557kB/s)(522MiB/120002msec) | |
slat (nsec): min=0, max=750545k, avg=167895.13, stdev=9048304.87 | |
clat (nsec): min=0, max=997524k, avg=6591996.29, stdev=37271220.87 | |
lat (nsec): min=0, max=997578k, avg=6760880.42, stdev=38337231.16 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 740], | |
| 20.00th=[ 804], 30.00th=[ 916], 40.00th=[ 1064], | |
| 50.00th=[ 1208], 60.00th=[ 1496], 70.00th=[ 610304], | |
| 80.00th=[ 1286144], 90.00th=[ 4145152], 95.00th=[ 34865152], | |
| 99.00th=[141557760], 99.50th=[265289728], 99.90th=[574619648], | |
| 99.95th=[591396864], 99.99th=[994050048] | |
bw ( KiB/s): min= 0, max= 6443, per=16.81%, avg=4449.00, stdev=2819.94, samples=2 | |
iops : min= 0, max= 100, avg=69.00, stdev=43.84, samples=2 | |
write: IOPS=70, BW=4498KiB/s (4605kB/s)(527MiB/120002msec) | |
slat (nsec): min=0, max=516019k, avg=258656.32, stdev=5651967.71 | |
clat (nsec): min=0, max=1458.4M, avg=50155127.58, stdev=113685456.08 | |
lat (nsec): min=0, max=1458.4M, avg=50415075.39, stdev=113756135.61 | |
clat percentiles (usec): | |
| 1.00th=[ 1020], 5.00th=[ 1254], 10.00th=[ 1434], | |
| 20.00th=[ 1778], 30.00th=[ 2245], 40.00th=[ 2900], | |
| 50.00th=[ 3916], 60.00th=[ 6325], 70.00th=[ 19268], | |
| 80.00th=[ 74974], 90.00th=[ 129500], 95.00th=[ 295699], | |
| 99.00th=[ 591397], 99.50th=[ 683672], 99.90th=[ 784335], | |
| 99.95th=[ 901776], 99.99th=[1451230] | |
bw ( KiB/s): min= 0, max= 6458, per=57.99%, avg=4495.50, stdev=2775.39, samples=2 | |
iops : min= 0, max= 100, avg=69.50, stdev=43.13, samples=2 | |
lat (nsec) : 750=7.04%, 1000=11.01% | |
lat (usec) : 2=12.70%, 4=0.05%, 10=0.27%, 20=0.67%, 50=0.56% | |
lat (usec) : 100=0.59%, 250=0.83%, 500=0.88%, 750=1.10%, 1000=2.94% | |
lat (msec) : 2=16.24%, 4=15.20%, 10=8.56%, 20=3.31%, 50=4.02% | |
lat (msec) : 100=5.96%, 250=4.79%, 500=2.43%, 750=0.76%, 1000=0.09% | |
cpu : usr=0.08%, sys=0.39%, ctx=20699, majf=0, minf=31 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=8344,8429,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=25.9MiB/s (27.1MB/s), 4450KiB/s-21.5MiB/s (4557kB/s-22.6MB/s), io=3103MiB (3253MB), run=120002-120018msec | |
WRITE: bw=7753KiB/s (7939kB/s), 3256KiB/s-4498KiB/s (3335kB/s-4605kB/s), io=909MiB (953MB), run=120002-120041msec | |
Disk stats (read/write): | |
rbd6: ios=11216/15012, merge=0/3, ticks=566580/831401, in_queue=1384528, util=16.71% | |
<example-benchmark-fio-server-3-benchmark-tx76m> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:35:00 2019 | |
write: IOPS=18, BW=1156KiB/s (1184kB/s)(136MiB/120093msec) | |
slat (nsec): min=0, max=1871.0M, avg=2769651.20, stdev=56431218.59 | |
clat (nsec): min=0, max=2617.7M, avg=220504187.67, stdev=349885888.10 | |
lat (nsec): min=0, max=2617.8M, avg=223272360.64, stdev=353092581.18 | |
clat percentiles (usec): | |
| 1.00th=[ 1074], 5.00th=[ 1303], 10.00th=[ 1565], | |
| 20.00th=[ 2180], 30.00th=[ 3228], 40.00th=[ 5473], | |
| 50.00th=[ 11731], 60.00th=[ 80217], 70.00th=[ 204473], | |
| 80.00th=[ 501220], 90.00th=[ 734004], 95.00th=[ 918553], | |
| 99.00th=[1400898], 99.50th=[1870660], 99.90th=[2071987], | |
| 99.95th=[2197816], 99.99th=[2634023] | |
bw ( KiB/s): min= 0, max= 1699, per=21.12%, avg=1155.00, stdev=769.33, samples=2 | |
iops : min= 0, max= 26, avg=17.50, stdev=12.02, samples=2 | |
lat (usec) : 4=0.05%, 1000=0.51% | |
lat (msec) : 2=16.75%, 4=17.17%, 10=13.71%, 20=6.46%, 50=2.58% | |
lat (msec) : 100=4.80%, 250=8.86%, 500=9.23%, 750=10.57%, 1000=5.86% | |
cpu : usr=0.02%, sys=0.09%, ctx=4098, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,2167,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:35:00 2019 | |
read: IOPS=545, BW=33.9MiB/s (35.6MB/s)(4074MiB/120118msec) | |
slat (nsec): min=0, max=1870.9M, avg=90467.46, stdev=10011085.33 | |
clat (nsec): min=0, max=1903.2M, avg=7293770.07, stdev=69808358.17 | |
lat (nsec): min=0, max=1903.2M, avg=7385182.62, stdev=70527189.25 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 724], | |
| 20.00th=[ 740], 30.00th=[ 788], 40.00th=[ 812], | |
| 50.00th=[ 852], 60.00th=[ 876], 70.00th=[ 916], | |
| 80.00th=[ 1004], 90.00th=[ 12736], 95.00th=[ 24960], | |
| 99.00th=[ 354418688], 99.50th=[ 557842432], 99.90th=[ 952107008], | |
| 99.95th=[1166016512], 99.99th=[1853882368] | |
bw ( KiB/s): min= 0, max=54319, per=89.19%, avg=34761.00, stdev=27659.19, samples=2 | |
iops : min= 0, max= 855, avg=546.00, stdev=436.99, samples=2 | |
lat (nsec) : 750=21.71%, 1000=58.02% | |
lat (usec) : 2=5.73%, 4=0.13%, 10=0.90%, 20=5.66%, 50=4.13% | |
lat (usec) : 100=0.14%, 250=0.06%, 500=0.06%, 750=0.15%, 1000=0.34% | |
lat (msec) : 2=0.50%, 4=0.38%, 10=0.18%, 20=0.10%, 50=0.26% | |
lat (msec) : 100=0.30%, 250=0.14%, 500=0.52%, 750=0.34%, 1000=0.17% | |
cpu : usr=0.12%, sys=0.51%, ctx=4867, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=65572,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:35:00 2019 | |
read: IOPS=66, BW=4247KiB/s (4349kB/s)(498MiB/120004msec) | |
slat (nsec): min=0, max=657901k, avg=106802.92, stdev=7372430.83 | |
clat (nsec): min=0, max=744115k, avg=4777220.47, stdev=35459077.18 | |
lat (nsec): min=0, max=744165k, avg=4884922.55, stdev=36213602.95 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 756], 30.00th=[ 852], 40.00th=[ 932], | |
| 50.00th=[ 1080], 60.00th=[ 1224], 70.00th=[ 1576], | |
| 80.00th=[ 84480], 90.00th=[ 1122304], 95.00th=[ 5799936], | |
| 99.00th=[141557760], 99.50th=[256901120], 99.90th=[557842432], | |
| 99.95th=[633339904], 99.99th=[742391808] | |
bw ( KiB/s): min= 0, max= 7251, per=10.89%, avg=4246.00, stdev=4249.71, samples=2 | |
iops : min= 0, max= 113, avg=66.00, stdev=66.47, samples=2 | |
write: IOPS=67, BW=4317KiB/s (4421kB/s)(506MiB/120004msec) | |
slat (nsec): min=0, max=445053k, avg=240599.24, stdev=4950521.64 | |
clat (nsec): min=0, max=1125.9M, avg=54461828.03, stdev=123421580.22 | |
lat (nsec): min=0, max=1125.0M, avg=54703547.45, stdev=123454883.67 | |
clat percentiles (usec): | |
| 1.00th=[ 1012], 5.00th=[ 1221], 10.00th=[ 1369], | |
| 20.00th=[ 1713], 30.00th=[ 2147], 40.00th=[ 2835], | |
| 50.00th=[ 4178], 60.00th=[ 7898], 70.00th=[ 19792], | |
| 80.00th=[ 80217], 90.00th=[ 149947], 95.00th=[ 304088], | |
| 99.00th=[ 633340], 99.50th=[ 775947], 99.90th=[ 935330], | |
| 99.95th=[ 943719], 99.99th=[1132463] | |
bw ( KiB/s): min= 0, max= 7336, per=78.88%, avg=4315.00, stdev=4272.34, samples=2 | |
iops : min= 0, max= 114, avg=67.00, stdev=66.47, samples=2 | |
lat (nsec) : 750=8.84%, 1000=13.77% | |
lat (usec) : 2=12.64%, 4=0.04%, 10=0.65%, 20=1.71%, 50=0.92% | |
lat (usec) : 100=1.36%, 250=1.94%, 500=0.83%, 750=0.54%, 1000=1.45% | |
lat (msec) : 2=14.44%, 4=12.42%, 10=7.87%, 20=3.83%, 50=3.23% | |
lat (msec) : 100=4.75%, 250=5.61%, 500=2.03%, 750=0.84%, 1000=0.29% | |
cpu : usr=0.09%, sys=0.37%, ctx=16379, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=7964,8092,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=38.1MiB/s (39.9MB/s), 4247KiB/s-33.9MiB/s (4349kB/s-35.6MB/s), io=4572MiB (4794MB), run=120004-120118msec | |
WRITE: bw=5470KiB/s (5602kB/s), 1156KiB/s-4317KiB/s (1184kB/s-4421kB/s), io=642MiB (673MB), run=120004-120093msec | |
Disk stats (read/write): | |
rbd1: ios=3345/10492, merge=0/1, ticks=511851/876978, in_queue=1381859, util=7.20% | |
<example-benchmark-fio-server-225-benchmark-d2hts> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:35:00 2019 | |
write: IOPS=16, BW=1079KiB/s (1105kB/s)(127MiB/120445msec) | |
slat (nsec): min=0, max=860283k, avg=1696157.53, stdev=32922611.14 | |
clat (nsec): min=0, max=2310.5M, avg=236942025.07, stdev=429727850.91 | |
lat (nsec): min=0, max=2311.1M, avg=238639284.04, stdev=430204025.53 | |
clat percentiles (usec): | |
| 1.00th=[ 1221], 5.00th=[ 1827], 10.00th=[ 2278], | |
| 20.00th=[ 3228], 30.00th=[ 4424], 40.00th=[ 6128], | |
| 50.00th=[ 10028], 60.00th=[ 21627], 70.00th=[ 78119], | |
| 80.00th=[ 513803], 90.00th=[ 943719], 95.00th=[1233126], | |
| 99.00th=[1568670], 99.50th=[1853883], 99.90th=[2105541], | |
| 99.95th=[2122318], 99.99th=[2298479] | |
bw ( KiB/s): min= 0, max= 1129, per=20.01%, avg=1079.00, stdev=70.71, samples=2 | |
iops : min= 0, max= 17, avg=16.50, stdev= 0.71, samples=2 | |
lat (usec) : 4=0.05%, 500=0.05%, 750=0.05%, 1000=0.35% | |
lat (msec) : 2=6.42%, 4=19.69%, 10=23.54%, 20=8.88%, 50=8.05% | |
lat (msec) : 100=4.79%, 250=3.80%, 500=4.15%, 750=4.84%, 1000=6.56% | |
cpu : usr=0.02%, sys=0.08%, ctx=3498, majf=0, minf=31 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,2026,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:35:00 2019 | |
read: IOPS=369, BW=23.0MiB/s (24.1MB/s)(2774MiB/120466msec) | |
slat (nsec): min=0, max=813265k, avg=56599.31, stdev=4929365.07 | |
clat (nsec): min=0, max=1847.6M, avg=10781330.84, stdev=91567779.88 | |
lat (nsec): min=0, max=1847.9M, avg=10838874.42, stdev=91712392.55 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 772], 30.00th=[ 780], 40.00th=[ 804], | |
| 50.00th=[ 844], 60.00th=[ 868], 70.00th=[ 900], | |
| 80.00th=[ 972], 90.00th=[ 11584], 95.00th=[ 1187840], | |
| 99.00th=[ 574619648], 99.50th=[ 775946240], 99.90th=[1166016512], | |
| 99.95th=[1350565888], 99.99th=[1719664640] | |
bw ( KiB/s): min= 0, max=31363, per=85.02%, avg=23663.00, stdev=10889.44, samples=2 | |
iops : min= 0, max= 491, avg=370.00, stdev=171.12, samples=2 | |
lat (nsec) : 750=14.31%, 1000=66.73% | |
lat (usec) : 2=7.44%, 4=0.16%, 10=0.95%, 20=3.40%, 50=0.79% | |
lat (usec) : 100=0.18%, 250=0.11%, 500=0.11%, 750=0.11%, 1000=0.43% | |
lat (msec) : 2=1.18%, 4=0.97%, 10=0.74%, 20=0.29%, 50=0.38% | |
lat (msec) : 100=0.24%, 250=0.14%, 500=0.28%, 750=0.51%, 1000=0.44% | |
cpu : usr=0.09%, sys=0.39%, ctx=5480, majf=0, minf=31 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=44518,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:35:00 2019 | |
read: IOPS=66, BW=4267KiB/s (4370kB/s)(500MiB/120003msec) | |
slat (nsec): min=0, max=4304.4k, avg=23613.57, stdev=74508.36 | |
clat (nsec): min=0, max=759154k, avg=6090334.59, stdev=46592721.81 | |
lat (nsec): min=0, max=759266k, avg=6114859.38, stdev=46599886.35 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 756], 30.00th=[ 836], 40.00th=[ 908], | |
| 50.00th=[ 1048], 60.00th=[ 1208], 70.00th=[ 1496], | |
| 80.00th=[ 67072], 90.00th=[ 1204224], 95.00th=[ 5996544], | |
| 99.00th=[168820736], 99.50th=[434110464], 99.90th=[616562688], | |
| 99.95th=[675282944], 99.99th=[759169024] | |
bw ( KiB/s): min= 0, max= 5781, per=15.33%, avg=4267.00, stdev=2141.12, samples=2 | |
iops : min= 0, max= 90, avg=66.50, stdev=33.23, samples=2 | |
write: IOPS=67, BW=4330KiB/s (4434kB/s)(507MiB/120003msec) | |
slat (nsec): min=0, max=6756.8k, avg=175613.08, stdev=232944.45 | |
clat (nsec): min=0, max=1093.4M, avg=53149774.18, stdev=110172503.70 | |
lat (nsec): min=0, max=1093.5M, avg=53326567.81, stdev=110133857.81 | |
clat percentiles (usec): | |
| 1.00th=[ 1074], 5.00th=[ 1319], 10.00th=[ 1483], | |
| 20.00th=[ 1909], 30.00th=[ 2507], 40.00th=[ 3687], | |
| 50.00th=[ 6390], 60.00th=[ 12649], 70.00th=[ 26346], | |
| 80.00th=[ 80217], 90.00th=[ 145753], 95.00th=[ 316670], | |
| 99.00th=[ 522191], 99.50th=[ 608175], 99.90th=[ 843056], | |
| 99.95th=[ 910164], 99.99th=[1098908] | |
bw ( KiB/s): min= 0, max= 5804, per=80.26%, avg=4327.50, stdev=2088.09, samples=2 | |
iops : min= 0, max= 90, avg=67.00, stdev=32.53, samples=2 | |
lat (nsec) : 750=9.35%, 1000=13.87% | |
lat (usec) : 2=12.52%, 4=0.04%, 10=0.68%, 20=1.67%, 50=1.02% | |
lat (usec) : 100=1.19%, 250=1.66%, 500=0.94%, 750=0.46%, 1000=1.06% | |
lat (msec) : 2=12.31%, 4=11.09%, 10=8.16%, 20=5.56%, 50=4.91% | |
lat (msec) : 100=4.41%, 250=5.52%, 500=2.78%, 750=0.69%, 1000=0.13% | |
cpu : usr=0.09%, sys=0.36%, ctx=16242, majf=0, minf=31 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=8001,8116,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=27.2MiB/s (28.5MB/s), 4267KiB/s-23.0MiB/s (4370kB/s-24.1MB/s), io=3274MiB (3433MB), run=120003-120466msec | |
WRITE: bw=5393KiB/s (5522kB/s), 1079KiB/s-4330KiB/s (1105kB/s-4434kB/s), io=634MiB (665MB), run=120003-120445msec | |
Disk stats (read/write): | |
rbd76: ios=3853/10268, merge=0/1, ticks=568825/896607, in_queue=1458316, util=5.45% | |
<example-benchmark-fio-server-499-benchmark-hgwjk> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:35:00 2019 | |
write: IOPS=20, BW=1336KiB/s (1369kB/s)(158MiB/121394msec) | |
slat (nsec): min=0, max=521835k, avg=602332.19, stdev=11489635.80 | |
clat (nsec): min=0, max=2418.6M, avg=191197413.08, stdev=299955700.19 | |
lat (nsec): min=0, max=2418.6M, avg=191801070.37, stdev=299979092.81 | |
clat percentiles (usec): | |
| 1.00th=[ 1057], 5.00th=[ 1418], 10.00th=[ 1762], | |
| 20.00th=[ 2507], 30.00th=[ 3523], 40.00th=[ 5080], | |
| 50.00th=[ 10159], 60.00th=[ 63701], 70.00th=[ 244319], | |
| 80.00th=[ 408945], 90.00th=[ 633340], 95.00th=[ 759170], | |
| 99.00th=[1182794], 99.50th=[1400898], 99.90th=[2432697], | |
| 99.95th=[2432697], 99.99th=[2432697] | |
bw ( KiB/s): min= 0, max= 1506, per=24.22%, avg=1348.00, stdev=223.45, samples=2 | |
iops : min= 0, max= 23, avg=20.50, stdev= 3.54, samples=2 | |
lat (usec) : 4=0.04%, 100=0.04%, 500=0.04%, 750=0.04%, 1000=0.51% | |
lat (msec) : 2=12.99%, 4=19.87%, 10=16.43%, 20=6.04%, 50=2.61% | |
lat (msec) : 100=4.19%, 250=7.58%, 500=14.81%, 750=9.87%, 1000=2.45% | |
cpu : usr=0.03%, sys=0.09%, ctx=4216, majf=0, minf=31 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,2532,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:35:00 2019 | |
read: IOPS=139, BW=8931KiB/s (9145kB/s)(1048MiB/120186msec) | |
slat (nsec): min=0, max=9380.9k, avg=29479.14, stdev=141498.82 | |
clat (nsec): min=0, max=1852.7M, avg=28689495.96, stdev=142920150.94 | |
lat (nsec): min=0, max=1852.8M, avg=28720090.77, stdev=142943141.98 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 772], 30.00th=[ 788], 40.00th=[ 836], | |
| 50.00th=[ 868], 60.00th=[ 900], 70.00th=[ 1048], | |
| 80.00th=[ 1416], 90.00th=[ 2703360], 95.00th=[ 43253760], | |
| 99.00th=[ 817889280], 99.50th=[ 943718400], 99.90th=[1283457024], | |
| 99.95th=[1451229184], 99.99th=[1837105152] | |
bw ( KiB/s): min= 0, max= 9136, per=67.97%, avg=8932.50, stdev=287.79, samples=2 | |
iops : min= 0, max= 142, avg=139.00, stdev= 4.24, samples=2 | |
lat (nsec) : 750=12.69%, 1000=55.86% | |
lat (usec) : 2=14.55%, 4=0.18%, 10=0.29%, 20=0.39%, 50=0.17% | |
lat (usec) : 100=0.13%, 250=0.04%, 500=0.08%, 750=0.26%, 1000=1.25% | |
lat (msec) : 2=3.16%, 4=2.30%, 10=1.74%, 20=0.89%, 50=1.22% | |
lat (msec) : 100=0.52%, 250=0.43%, 500=0.91%, 750=1.57%, 1000=1.01% | |
cpu : usr=0.05%, sys=0.18%, ctx=5400, majf=0, minf=31 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=16768,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:35:00 2019 | |
read: IOPS=65, BW=4218KiB/s (4319kB/s)(494MiB/120008msec) | |
slat (nsec): min=0, max=3874.9k, avg=26448.84, stdev=87557.02 | |
clat (nsec): min=0, max=904621k, avg=9496297.79, stdev=57007692.17 | |
lat (nsec): min=0, max=905014k, avg=9523688.60, stdev=57015747.35 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 748], 30.00th=[ 852], 40.00th=[ 948], | |
| 50.00th=[ 1096], 60.00th=[ 1256], 70.00th=[ 1544], | |
| 80.00th=[ 164864], 90.00th=[ 2473984], 95.00th=[ 30801920], | |
| 99.00th=[346030080], 99.50th=[488636416], 99.90th=[666894336], | |
| 99.95th=[717225984], 99.99th=[901775360] | |
bw ( KiB/s): min= 0, max= 6002, per=32.10%, avg=4218.00, stdev=2522.96, samples=2 | |
iops : min= 0, max= 93, avg=65.50, stdev=38.89, samples=2 | |
write: IOPS=66, BW=4278KiB/s (4381kB/s)(501MiB/120008msec) | |
slat (nsec): min=0, max=8021.8k, avg=195191.63, stdev=241106.21 | |
clat (nsec): min=0, max=930859k, avg=50434385.70, stdev=108648012.81 | |
lat (nsec): min=0, max=930927k, avg=50630816.52, stdev=108598568.70 | |
clat percentiles (usec): | |
| 1.00th=[ 988], 5.00th=[ 1188], 10.00th=[ 1369], 20.00th=[ 1729], | |
| 30.00th=[ 2212], 40.00th=[ 2966], 50.00th=[ 4555], 60.00th=[ 9896], | |
| 70.00th=[ 26346], 80.00th=[ 79168], 90.00th=[131597], 95.00th=[261096], | |
| 99.00th=[549454], 99.50th=[683672], 99.90th=[884999], 99.95th=[918553], | |
| 99.99th=[926942] | |
bw ( KiB/s): min= 0, max= 6036, per=76.84%, avg=4276.00, stdev=2489.02, samples=2 | |
iops : min= 0, max= 94, avg=66.50, stdev=38.89, samples=2 | |
lat (nsec) : 750=9.89%, 1000=11.90% | |
lat (usec) : 2=13.76%, 4=0.02%, 10=0.42%, 20=1.27%, 50=0.87% | |
lat (usec) : 100=1.00%, 250=1.09%, 500=0.65%, 750=0.62%, 1000=1.59% | |
lat (msec) : 2=14.31%, 4=11.99%, 10=7.35%, 20=3.99%, 50=4.79% | |
lat (msec) : 100=5.32%, 250=5.96%, 500=2.30%, 750=0.78%, 1000=0.16% | |
cpu : usr=0.09%, sys=0.36%, ctx=16941, majf=0, minf=34 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=7909,8019,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=12.8MiB/s (13.5MB/s), 4218KiB/s-8931KiB/s (4319kB/s-9145kB/s), io=1543MiB (1617MB), run=120008-120186msec | |
WRITE: bw=5566KiB/s (5699kB/s), 1336KiB/s-4278KiB/s (1369kB/s-4381kB/s), io=660MiB (692MB), run=120008-121394msec | |
Disk stats (read/write): | |
rbd210: ios=4224/10915, merge=0/2, ticks=600452/894257, in_queue=1487171, util=4.82% | |
<example-benchmark-fio-server-352-benchmark-swk2k> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:35:01 2019 | |
write: IOPS=19, BW=1250KiB/s (1280kB/s)(149MiB/122256msec) | |
slat (nsec): min=0, max=3241.0M, avg=1882431.65, stdev=67552986.62 | |
clat (nsec): min=0, max=3320.5M, avg=201732832.03, stdev=317272332.29 | |
lat (nsec): min=0, max=3373.9M, avg=203614730.47, stdev=324396055.25 | |
clat percentiles (usec): | |
| 1.00th=[ 1139], 5.00th=[ 1713], 10.00th=[ 2442], | |
| 20.00th=[ 3818], 30.00th=[ 6456], 40.00th=[ 10421], | |
| 50.00th=[ 18482], 60.00th=[ 54264], 70.00th=[ 233833], | |
| 80.00th=[ 450888], 90.00th=[ 692061], 95.00th=[ 826278], | |
| 99.00th=[1132463], 99.50th=[1367344], 99.90th=[3238003], | |
| 99.95th=[3305112], 99.99th=[3305112] | |
bw ( KiB/s): min= 0, max= 1384, per=23.96%, avg=1270.50, stdev=160.51, samples=2 | |
iops : min= 0, max= 21, avg=19.50, stdev= 2.12, samples=2 | |
lat (usec) : 500=0.04%, 1000=0.42% | |
lat (msec) : 2=6.71%, 4=13.67%, 10=18.54%, 20=12.33%, 50=7.76% | |
lat (msec) : 100=4.49%, 250=7.09%, 500=11.91%, 750=9.77%, 1000=4.82% | |
cpu : usr=0.03%, sys=0.08%, ctx=4156, majf=0, minf=29 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,2384,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:35:01 2019 | |
read: IOPS=150, BW=9662KiB/s (9894kB/s)(1137MiB/120533msec) | |
slat (nsec): min=0, max=11726k, avg=30343.88, stdev=150033.44 | |
clat (nsec): min=0, max=1774.4M, avg=26494562.90, stdev=136028998.42 | |
lat (nsec): min=0, max=1774.5M, avg=26526057.61, stdev=136047028.65 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 780], 30.00th=[ 796], 40.00th=[ 836], | |
| 50.00th=[ 876], 60.00th=[ 916], 70.00th=[ 1112], | |
| 80.00th=[ 1512], 90.00th=[ 2965504], 95.00th=[ 36962304], | |
| 99.00th=[ 792723456], 99.50th=[ 935329792], 99.90th=[1233125376], | |
| 99.95th=[1350565888], 99.99th=[1669332992] | |
bw ( KiB/s): min= 0, max= 9968, per=70.80%, avg=9702.50, stdev=375.47, samples=2 | |
iops : min= 0, max= 155, avg=151.00, stdev= 5.66, samples=2 | |
lat (nsec) : 750=13.35%, 1000=53.24% | |
lat (usec) : 2=14.88%, 4=0.20%, 10=0.15%, 20=0.68%, 50=0.20% | |
lat (usec) : 100=0.08%, 250=0.06%, 500=0.10%, 750=0.29%, 1000=1.26% | |
lat (msec) : 2=3.73%, 4=2.91%, 10=2.18%, 20=0.86%, 50=1.28% | |
lat (msec) : 100=0.60%, 250=0.36%, 500=0.92%, 750=1.40%, 1000=0.96% | |
cpu : usr=0.05%, sys=0.19%, ctx=6330, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=18194,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=14: Wed May 8 20:35:01 2019 | |
read: IOPS=63, BW=4060KiB/s (4157kB/s)(476MiB/120009msec) | |
slat (nsec): min=0, max=2641.9k, avg=25605.77, stdev=72611.64 | |
clat (nsec): min=0, max=995069k, avg=6187294.75, stdev=39693365.11 | |
lat (nsec): min=0, max=995130k, avg=6213825.74, stdev=39700139.79 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 724], | |
| 20.00th=[ 748], 30.00th=[ 844], 40.00th=[ 924], | |
| 50.00th=[ 1064], 60.00th=[ 1224], 70.00th=[ 1528], | |
| 80.00th=[ 142336], 90.00th=[ 1761280], 95.00th=[ 11993088], | |
| 99.00th=[200278016], 99.50th=[320864256], 99.90th=[517996544], | |
| 99.95th=[641728512], 99.99th=[994050048] | |
bw ( KiB/s): min= 0, max= 6873, per=29.62%, avg=4059.50, stdev=3978.89, samples=2 | |
iops : min= 0, max= 107, avg=63.00, stdev=62.23, samples=2 | |
write: IOPS=64, BW=4129KiB/s (4228kB/s)(484MiB/120009msec) | |
slat (nsec): min=0, max=6809.1k, avg=193732.28, stdev=258346.27 | |
clat (nsec): min=0, max=871312k, avg=55941812.81, stdev=118854266.47 | |
lat (nsec): min=0, max=871345k, avg=56136695.07, stdev=118806735.79 | |
clat percentiles (usec): | |
| 1.00th=[ 979], 5.00th=[ 1188], 10.00th=[ 1385], 20.00th=[ 1696], | |
| 30.00th=[ 2147], 40.00th=[ 2868], 50.00th=[ 4178], 60.00th=[ 7504], | |
| 70.00th=[ 29230], 80.00th=[ 87557], 90.00th=[160433], 95.00th=[325059], | |
| 99.00th=[608175], 99.50th=[666895], 99.90th=[775947], 99.95th=[843056], | |
| 99.99th=[868221] | |
bw ( KiB/s): min= 0, max= 6961, per=77.84%, avg=4127.00, stdev=4007.88, samples=2 | |
iops : min= 0, max= 108, avg=64.00, stdev=62.23, samples=2 | |
lat (nsec) : 750=9.91%, 1000=12.67% | |
lat (usec) : 2=12.85%, 4=0.07%, 10=0.40%, 20=1.26%, 50=0.87% | |
lat (usec) : 100=1.07%, 250=1.32%, 500=0.80%, 750=0.54%, 1000=1.93% | |
lat (msec) : 2=15.05%, 4=12.04%, 10=8.38%, 20=2.77%, 50=3.50% | |
lat (msec) : 100=4.94%, 250=6.21%, 500=2.23%, 750=1.11%, 1000=0.09% | |
cpu : usr=0.07%, sys=0.36%, ctx=16160, majf=0, minf=34 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=7613,7740,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=13.4MiB/s (14.0MB/s), 4060KiB/s-9662KiB/s (4157kB/s-9894kB/s), io=1613MiB (1691MB), run=120009-120533msec | |
WRITE: bw=5303KiB/s (5430kB/s), 1250KiB/s-4129KiB/s (1280kB/s-4228kB/s), io=633MiB (664MB), run=120009-122256msec | |
Disk stats (read/write): | |
rbd118: ios=4729/10780, merge=0/2, ticks=572879/917855, in_queue=1482954, util=4.44% | |
<example-benchmark-fio-server-706-benchmark-xxdl7> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:35:01 2019 | |
write: IOPS=26, BW=1719KiB/s (1760kB/s)(206MiB/122670msec) | |
slat (nsec): min=0, max=673225k, avg=684647.51, stdev=15798809.05 | |
clat (nsec): min=0, max=1908.0M, avg=146098595.77, stdev=316285717.18 | |
lat (nsec): min=0, max=1909.0M, avg=146784260.50, stdev=316853377.15 | |
clat percentiles (usec): | |
| 1.00th=[ 1205], 5.00th=[ 1647], 10.00th=[ 1958], | |
| 20.00th=[ 2737], 30.00th=[ 3687], 40.00th=[ 4817], | |
| 50.00th=[ 6718], 60.00th=[ 9896], 70.00th=[ 16450], | |
| 80.00th=[ 240124], 90.00th=[ 566232], 95.00th=[ 901776], | |
| 99.00th=[1384121], 99.50th=[1501561], 99.90th=[1887437], | |
| 99.95th=[1904215], 99.99th=[1904215] | |
bw ( KiB/s): min= 0, max= 2234, per=29.13%, avg=1748.00, stdev=687.31, samples=2 | |
iops : min= 0, max= 34, avg=26.50, stdev=10.61, samples=2 | |
lat (usec) : 100=0.03%, 750=0.15%, 1000=0.24% | |
lat (msec) : 2=10.27%, 4=22.48%, 10=27.40%, 20=12.00%, 50=5.80% | |
lat (msec) : 100=0.94%, 250=0.79%, 500=8.35%, 750=4.86%, 1000=2.89% | |
cpu : usr=0.04%, sys=0.12%, ctx=5526, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,3292,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:35:01 2019 | |
read: IOPS=233, BW=14.6MiB/s (15.3MB/s)(1759MiB/120651msec) | |
slat (nsec): min=0, max=525196k, avg=46795.53, stdev=3133292.21 | |
clat (nsec): min=0, max=1842.0M, avg=17115408.03, stdev=109396301.96 | |
lat (nsec): min=0, max=1842.1M, avg=17163208.16, stdev=109539992.27 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 732], 10.00th=[ 748], | |
| 20.00th=[ 780], 30.00th=[ 796], 40.00th=[ 836], | |
| 50.00th=[ 868], 60.00th=[ 900], 70.00th=[ 988], | |
| 80.00th=[ 1384], 90.00th=[ 2039808], 95.00th=[ 13565952], | |
| 99.00th=[ 708837376], 99.50th=[ 868220928], 99.90th=[1166016512], | |
| 99.95th=[1266679808], 99.99th=[1702887424] | |
bw ( KiB/s): min= 0, max=17562, per=78.09%, avg=15005.00, stdev=3616.14, samples=2 | |
iops : min= 0, max= 274, avg=234.00, stdev=56.57, samples=2 | |
lat (nsec) : 750=10.45%, 1000=59.97% | |
lat (usec) : 2=13.45%, 4=0.21%, 10=0.24%, 20=0.76%, 50=0.25% | |
lat (usec) : 100=0.13%, 250=0.11%, 500=0.06%, 750=0.26%, 1000=1.06% | |
lat (msec) : 2=2.98%, 4=2.55%, 10=1.99%, 20=1.12%, 50=1.38% | |
lat (msec) : 100=0.50%, 250=0.35%, 500=0.55%, 750=0.81%, 1000=0.61% | |
cpu : usr=0.08%, sys=0.29%, ctx=8279, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=28138,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:35:01 2019 | |
read: IOPS=67, BW=4309KiB/s (4413kB/s)(505MiB/120043msec) | |
slat (nsec): min=0, max=3424.6k, avg=31999.20, stdev=100206.55 | |
clat (nsec): min=0, max=740074k, avg=9433999.90, stdev=51294559.77 | |
lat (nsec): min=0, max=740112k, avg=9466955.19, stdev=51304338.52 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 740], | |
| 20.00th=[ 796], 30.00th=[ 892], 40.00th=[ 1020], | |
| 50.00th=[ 1176], 60.00th=[ 1336], 70.00th=[ 18560], | |
| 80.00th=[ 839680], 90.00th=[ 4358144], 95.00th=[ 42205184], | |
| 99.00th=[287309824], 99.50th=[417333248], 99.90th=[633339904], | |
| 99.95th=[700448768], 99.99th=[742391808] | |
bw ( KiB/s): min= 0, max= 6309, per=22.43%, avg=4310.00, stdev=2827.01, samples=2 | |
iops : min= 0, max= 98, avg=67.00, stdev=43.84, samples=2 | |
write: IOPS=68, BW=4377KiB/s (4482kB/s)(513MiB/120043msec) | |
slat (nsec): min=0, max=5049.0k, avg=193015.97, stdev=236476.27 | |
clat (nsec): min=0, max=1324.3M, avg=49248474.97, stdev=103898012.15 | |
lat (nsec): min=0, max=1324.3M, avg=49442877.34, stdev=103853368.88 | |
clat percentiles (usec): | |
| 1.00th=[ 1012], 5.00th=[ 1221], 10.00th=[ 1369], | |
| 20.00th=[ 1680], 30.00th=[ 2212], 40.00th=[ 3064], | |
| 50.00th=[ 4948], 60.00th=[ 10552], 70.00th=[ 26870], | |
| 80.00th=[ 74974], 90.00th=[ 137364], 95.00th=[ 256902], | |
| 99.00th=[ 513803], 99.50th=[ 599786], 99.90th=[ 859833], | |
| 99.95th=[ 868221], 99.99th=[1317012] | |
bw ( KiB/s): min= 0, max= 6345, per=72.92%, avg=4376.00, stdev=2784.59, samples=2 | |
iops : min= 0, max= 99, avg=68.00, stdev=43.84, samples=2 | |
lat (nsec) : 750=7.50%, 1000=11.70% | |
lat (usec) : 2=14.09%, 4=0.07%, 10=0.43%, 20=1.12%, 50=0.98% | |
lat (usec) : 100=0.71%, 250=1.18%, 500=0.66%, 750=0.73%, 1000=2.00% | |
lat (msec) : 2=15.26%, 4=11.44%, 10=7.63%, 20=4.57%, 50=5.46% | |
lat (msec) : 100=5.42%, 250=5.95%, 500=2.44%, 750=0.60%, 1000=0.10% | |
cpu : usr=0.11%, sys=0.38%, ctx=18225, majf=0, minf=38 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=8083,8206,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=18.8MiB/s (19.7MB/s), 4309KiB/s-14.6MiB/s (4413kB/s-15.3MB/s), io=2264MiB (2374MB), run=120043-120651msec | |
WRITE: bw=6002KiB/s (6146kB/s), 1719KiB/s-4377KiB/s (1760kB/s-4482kB/s), io=719MiB (754MB), run=120043-122670msec | |
Disk stats (read/write): | |
rbd257: ios=6205/11856, merge=0/2, ticks=596108/895995, in_queue=1483029, util=6.30% | |
<example-benchmark-fio-server-473-benchmark-s88vn> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:35:01 2019 | |
write: IOPS=26, BW=1692KiB/s (1733kB/s)(200MiB/121075msec) | |
slat (nsec): min=0, max=225666k, avg=347336.80, stdev=4842478.92 | |
clat (nsec): min=0, max=3886.0M, avg=150346798.75, stdev=285940322.20 | |
lat (nsec): min=0, max=3886.2M, avg=150695339.12, stdev=285937535.10 | |
clat percentiles (usec): | |
| 1.00th=[ 1156], 5.00th=[ 1778], 10.00th=[ 2442], | |
| 20.00th=[ 3752], 30.00th=[ 5276], 40.00th=[ 7439], | |
| 50.00th=[ 10814], 60.00th=[ 17433], 70.00th=[ 54789], | |
| 80.00th=[ 304088], 90.00th=[ 566232], 95.00th=[ 750781], | |
| 99.00th=[1019216], 99.50th=[1182794], 99.90th=[3506439], | |
| 99.95th=[3841983], 99.99th=[3875537] | |
bw ( KiB/s): min= 0, max= 2070, per=32.19%, avg=1702.50, stdev=519.72, samples=2 | |
iops : min= 0, max= 32, avg=26.00, stdev= 8.49, samples=2 | |
lat (usec) : 500=0.03%, 1000=0.28% | |
lat (msec) : 2=6.35%, 4=15.57%, 10=25.95%, 20=13.91%, 50=7.60% | |
lat (msec) : 100=2.03%, 250=4.97%, 500=12.16%, 750=6.22%, 1000=3.91% | |
cpu : usr=0.04%, sys=0.11%, ctx=5496, majf=0, minf=31 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,3198,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:35:01 2019 | |
read: IOPS=209, BW=13.1MiB/s (13.7MB/s)(1579MiB/120646msec) | |
slat (nsec): min=0, max=10947k, avg=35755.18, stdev=179088.85 | |
clat (nsec): min=0, max=1843.5M, avg=19068443.95, stdev=112056013.01 | |
lat (nsec): min=0, max=1843.5M, avg=19105175.28, stdev=112070145.59 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 740], | |
| 20.00th=[ 772], 30.00th=[ 796], 40.00th=[ 844], | |
| 50.00th=[ 884], 60.00th=[ 964], 70.00th=[ 1272], | |
| 80.00th=[ 839680], 90.00th=[ 3719168], 95.00th=[ 23461888], | |
| 99.00th=[ 734003200], 99.50th=[ 868220928], 99.90th=[1132462080], | |
| 99.95th=[1266679808], 99.99th=[1619001344] | |
bw ( KiB/s): min= 0, max=14679, per=79.45%, avg=13458.00, stdev=1726.75, samples=2 | |
iops : min= 0, max= 229, avg=210.00, stdev=26.87, samples=2 | |
lat (nsec) : 750=11.34%, 1000=49.95% | |
lat (usec) : 2=15.71%, 4=0.21%, 10=0.19%, 20=1.02%, 50=0.23% | |
lat (usec) : 100=0.12%, 250=0.11%, 500=0.14%, 750=0.47%, 1000=1.90% | |
lat (msec) : 2=4.98%, 4=3.97%, 10=2.82%, 20=1.45%, 50=1.75% | |
lat (msec) : 100=0.66%, 250=0.52%, 500=0.72%, 750=0.84%, 1000=0.71% | |
cpu : usr=0.07%, sys=0.28%, ctx=10686, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=25262,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=14: Wed May 8 20:35:01 2019 | |
read: IOPS=55, BW=3556KiB/s (3641kB/s)(417MiB/120042msec) | |
slat (nsec): min=0, max=1662.4k, avg=27905.04, stdev=68083.88 | |
clat (nsec): min=0, max=930076k, avg=7187406.33, stdev=38705839.51 | |
lat (nsec): min=0, max=930119k, avg=7216297.76, stdev=38712737.80 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 732], | |
| 20.00th=[ 780], 30.00th=[ 868], 40.00th=[ 988], | |
| 50.00th=[ 1144], 60.00th=[ 1304], 70.00th=[ 15424], | |
| 80.00th=[ 765952], 90.00th=[ 4079616], 95.00th=[ 39059456], | |
| 99.00th=[183500800], 99.50th=[283115520], 99.90th=[591396864], | |
| 99.95th=[675282944], 99.99th=[926941184] | |
bw ( KiB/s): min= 0, max= 5559, per=20.99%, avg=3556.50, stdev=2831.96, samples=2 | |
iops : min= 0, max= 86, avg=55.00, stdev=43.84, samples=2 | |
write: IOPS=56, BW=3628KiB/s (3715kB/s)(425MiB/120042msec) | |
slat (nsec): min=0, max=6981.7k, avg=189076.12, stdev=241129.58 | |
clat (nsec): min=0, max=1469.3M, avg=63533638.70, stdev=133759133.98 | |
lat (nsec): min=0, max=1469.4M, avg=63723896.57, stdev=133715671.18 | |
clat percentiles (usec): | |
| 1.00th=[ 988], 5.00th=[ 1172], 10.00th=[ 1336], | |
| 20.00th=[ 1647], 30.00th=[ 2147], 40.00th=[ 3130], | |
| 50.00th=[ 5932], 60.00th=[ 16712], 70.00th=[ 44303], | |
| 80.00th=[ 96994], 90.00th=[ 162530], 95.00th=[ 329253], | |
| 99.00th=[ 700449], 99.50th=[ 801113], 99.90th=[ 952108], | |
| 99.95th=[1098908], 99.99th=[1468007] | |
bw ( KiB/s): min= 0, max= 5653, per=68.58%, avg=3627.00, stdev=2865.20, samples=2 | |
iops : min= 0, max= 88, avg=56.50, stdev=44.55, samples=2 | |
lat (nsec) : 750=8.09%, 1000=12.14% | |
lat (usec) : 2=13.44%, 4=0.07%, 10=0.36%, 20=1.26%, 50=0.78% | |
lat (usec) : 100=1.02%, 250=1.04%, 500=0.65%, 750=0.75%, 1000=1.96% | |
lat (msec) : 2=15.50%, 4=10.12%, 10=5.89%, 20=4.30%, 50=6.07% | |
lat (msec) : 100=5.89%, 250=7.10%, 500=2.34%, 750=0.87%, 1000=0.36% | |
cpu : usr=0.07%, sys=0.32%, ctx=14974, majf=0, minf=34 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=6669,6802,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=16.5MiB/s (17.3MB/s), 3556KiB/s-13.1MiB/s (3641kB/s-13.7MB/s), io=1996MiB (2093MB), run=120042-120646msec | |
WRITE: bw=5289KiB/s (5416kB/s), 1692KiB/s-3628KiB/s (1733kB/s-3715kB/s), io=625MiB (656MB), run=120042-121075msec | |
Disk stats (read/write): | |
rbd146: ios=7112/10366, merge=0/2, ticks=570356/925505, in_queue=1487123, util=5.21% | |
<example-benchmark-fio-server-663-benchmark-7t28d> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:35:01 2019 | |
write: IOPS=36, BW=2338KiB/s (2394kB/s)(274MiB/120024msec) | |
slat (nsec): min=0, max=1108.8M, avg=1563338.65, stdev=34895725.20 | |
clat (nsec): min=0, max=2157.2M, avg=108691853.62, stdev=172953769.21 | |
lat (nsec): min=0, max=2157.2M, avg=110458971.82, stdev=181332612.36 | |
clat percentiles (usec): | |
| 1.00th=[ 1012], 5.00th=[ 1254], 10.00th=[ 1483], | |
| 20.00th=[ 2040], 30.00th=[ 3425], 40.00th=[ 9634], | |
| 50.00th=[ 51119], 60.00th=[ 84411], 70.00th=[ 108528], | |
| 80.00th=[ 143655], 90.00th=[ 346031], 95.00th=[ 488637], | |
| 99.00th=[ 834667], 99.50th=[ 910164], 99.90th=[1115685], | |
| 99.95th=[1182794], 99.99th=[2164261] | |
bw ( KiB/s): min= 0, max= 3665, per=35.50%, avg=2337.00, stdev=1878.08, samples=2 | |
iops : min= 0, max= 57, avg=36.00, stdev=29.70, samples=2 | |
lat (usec) : 750=0.07%, 1000=0.80% | |
lat (msec) : 2=18.44%, 4=12.71%, 10=8.33%, 20=4.41%, 50=5.02% | |
lat (msec) : 100=16.80%, 250=20.82%, 500=7.90%, 750=3.33%, 1000=1.21% | |
cpu : usr=0.05%, sys=0.15%, ctx=8296, majf=0, minf=26 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,4381,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:35:01 2019 | |
read: IOPS=505, BW=31.4MiB/s (32.9MB/s)(3763MiB/120006msec) | |
slat (nsec): min=0, max=1108.8M, avg=95001.81, stdev=8079368.15 | |
clat (nsec): min=0, max=2157.1M, avg=7861988.18, stdev=51822844.43 | |
lat (nsec): min=0, max=2157.2M, avg=7957796.46, stdev=52813178.22 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 732], 40.00th=[ 756], | |
| 50.00th=[ 836], 60.00th=[ 892], 70.00th=[ 1112], | |
| 80.00th=[ 8768], 90.00th=[ 45312], 95.00th=[ 26869760], | |
| 99.00th=[ 261095424], 99.50th=[ 421527552], 99.90th=[ 658505728], | |
| 99.95th=[ 767557632], 99.99th=[1098907648] | |
bw ( KiB/s): min= 0, max=46009, per=88.48%, avg=32107.50, stdev=19659.69, samples=2 | |
iops : min= 0, max= 727, avg=505.50, stdev=313.25, samples=2 | |
lat (nsec) : 750=38.28%, 1000=29.30% | |
lat (usec) : 2=7.41%, 4=0.05%, 10=6.17%, 20=4.86%, 50=4.02% | |
lat (usec) : 100=0.24%, 250=0.07%, 500=0.08%, 750=0.42%, 1000=1.00% | |
lat (msec) : 2=1.37%, 4=0.76%, 10=0.40%, 20=0.29%, 50=1.86% | |
lat (msec) : 100=1.61%, 250=0.81%, 500=0.77%, 750=0.20%, 1000=0.05% | |
cpu : usr=0.14%, sys=0.48%, ctx=11673, majf=0, minf=105 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=60714,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:35:01 2019 | |
read: IOPS=65, BW=4187KiB/s (4288kB/s)(491MiB/120038msec) | |
slat (nsec): min=0, max=358717k, avg=71754.97, stdev=4047648.69 | |
clat (nsec): min=0, max=959407k, avg=3762438.41, stdev=26003167.77 | |
lat (nsec): min=0, max=959440k, avg=3835087.41, stdev=26316511.26 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 724], | |
| 20.00th=[ 748], 30.00th=[ 844], 40.00th=[ 932], | |
| 50.00th=[ 1080], 60.00th=[ 1256], 70.00th=[ 13504], | |
| 80.00th=[ 444416], 90.00th=[ 1957888], 95.00th=[ 8978432], | |
| 99.00th=[ 79167488], 99.50th=[152043520], 99.90th=[442499072], | |
| 99.95th=[497025024], 99.99th=[960495616] | |
bw ( KiB/s): min= 0, max= 7293, per=11.54%, avg=4188.00, stdev=4391.13, samples=2 | |
iops : min= 0, max= 113, avg=64.50, stdev=68.59, samples=2 | |
write: IOPS=66, BW=4246KiB/s (4347kB/s)(498MiB/120038msec) | |
slat (nsec): min=0, max=616615k, avg=277859.02, stdev=7123826.65 | |
clat (nsec): min=0, max=1058.7M, avg=56496877.05, stdev=127492223.13 | |
lat (nsec): min=0, max=1058.8M, avg=56775886.69, stdev=127619809.89 | |
clat percentiles (usec): | |
| 1.00th=[ 996], 5.00th=[ 1188], 10.00th=[ 1336], | |
| 20.00th=[ 1614], 30.00th=[ 1958], 40.00th=[ 2540], | |
| 50.00th=[ 3884], 60.00th=[ 7111], 70.00th=[ 21627], | |
| 80.00th=[ 85459], 90.00th=[ 143655], 95.00th=[ 350225], | |
| 99.00th=[ 641729], 99.50th=[ 775947], 99.90th=[ 943719], | |
| 99.95th=[ 943719], 99.99th=[1061159] | |
bw ( KiB/s): min= 0, max= 7357, per=64.48%, avg=4244.50, stdev=4401.74, samples=2 | |
iops : min= 0, max= 114, avg=65.50, stdev=68.59, samples=2 | |
lat (nsec) : 750=10.03%, 1000=12.38% | |
lat (usec) : 2=11.41%, 4=0.06%, 10=0.43%, 20=1.47%, 50=0.82% | |
lat (usec) : 100=0.96%, 250=1.24%, 500=1.10%, 750=0.87%, 1000=2.21% | |
lat (msec) : 2=17.35%, 4=11.41%, 10=7.51%, 20=3.66%, 50=3.38% | |
lat (msec) : 100=4.86%, 250=5.46%, 500=2.27%, 750=0.89%, 1000=0.26% | |
cpu : usr=0.08%, sys=0.35%, ctx=17449, majf=0, minf=33 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=7855,7959,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=35.4MiB/s (37.2MB/s), 4187KiB/s-31.4MiB/s (4288kB/s-32.9MB/s), io=4254MiB (4460MB), run=120006-120038msec | |
WRITE: bw=6584KiB/s (6742kB/s), 2338KiB/s-4246KiB/s (2394kB/s-4347kB/s), io=772MiB (809MB), run=120024-120038msec | |
Disk stats (read/write): | |
rbd284: ios=7426/12487, merge=0/2, ticks=505851/806981, in_queue=1302709, util=12.68% | |
<example-benchmark-fio-server-37-benchmark-k5hwt> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:35:01 2019 | |
write: IOPS=39, BW=2540KiB/s (2601kB/s)(298MiB/120173msec) | |
slat (nsec): min=0, max=696899k, avg=411800.47, stdev=10858781.84 | |
clat (nsec): min=0, max=1250.4M, avg=100680898.92, stdev=165640343.18 | |
lat (nsec): min=0, max=1250.6M, avg=101093747.11, stdev=165819419.43 | |
clat percentiles (usec): | |
| 1.00th=[ 971], 5.00th=[ 1205], 10.00th=[ 1418], | |
| 20.00th=[ 1909], 30.00th=[ 2868], 40.00th=[ 5735], | |
| 50.00th=[ 25035], 60.00th=[ 71828], 70.00th=[ 103285], | |
| 80.00th=[ 139461], 90.00th=[ 308282], 95.00th=[ 480248], | |
| 99.00th=[ 792724], 99.50th=[ 901776], 99.90th=[1115685], | |
| 99.95th=[1115685], 99.99th=[1249903] | |
bw ( KiB/s): min= 0, max= 2981, per=39.78%, avg=2541.00, stdev=622.25, samples=2 | |
iops : min= 0, max= 46, avg=39.00, stdev= 9.90, samples=2 | |
lat (usec) : 250=0.02%, 500=0.04%, 750=0.04%, 1000=1.26% | |
lat (msec) : 2=19.87%, 4=14.23%, 10=8.64%, 20=4.51%, 50=6.53% | |
lat (msec) : 100=13.91%, 250=18.74%, 500=7.74%, 750=3.34%, 1000=0.97% | |
cpu : usr=0.06%, sys=0.17%, ctx=9069, majf=0, minf=27 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,4766,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:35:01 2019 | |
read: IOPS=381, BW=23.8MiB/s (24.9MB/s)(2856MiB/120067msec) | |
slat (nsec): min=0, max=408609k, avg=33102.72, stdev=2270301.13 | |
clat (nsec): min=0, max=1243.1M, avg=10502054.70, stdev=58942956.54 | |
lat (nsec): min=0, max=1243.2M, avg=10535980.10, stdev=58995317.19 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 708], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 732], 40.00th=[ 756], | |
| 50.00th=[ 820], 60.00th=[ 876], 70.00th=[ 988], | |
| 80.00th=[ 8640], 90.00th=[ 1155072], 95.00th=[ 49020928], | |
| 99.00th=[325058560], 99.50th=[467664896], 99.90th=[734003200], | |
| 99.95th=[843055104], 99.99th=[960495616] | |
bw ( KiB/s): min= 0, max=37141, per=86.65%, avg=24365.50, stdev=18067.29, samples=2 | |
iops : min= 0, max= 581, avg=381.00, stdev=282.84, samples=2 | |
lat (nsec) : 750=38.95%, 1000=31.59% | |
lat (usec) : 2=9.13%, 4=0.05%, 10=3.93%, 20=2.24%, 50=1.34% | |
lat (usec) : 100=0.13%, 250=0.06%, 500=0.12%, 750=0.56%, 1000=1.34% | |
lat (msec) : 2=1.89%, 4=1.03%, 10=0.54%, 20=0.37%, 50=1.82% | |
lat (msec) : 100=2.31%, 250=1.27%, 500=0.96%, 750=0.29%, 1000=0.08% | |
cpu : usr=0.11%, sys=0.38%, ctx=11528, majf=0, minf=31 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=45763,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:35:01 2019 | |
read: IOPS=58, BW=3769KiB/s (3859kB/s)(442MiB/120004msec) | |
slat (nsec): min=0, max=156853k, avg=69439.21, stdev=2318254.95 | |
clat (nsec): min=0, max=912880k, avg=6950330.88, stdev=42236370.38 | |
lat (nsec): min=0, max=913024k, avg=7020687.52, stdev=42300601.87 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 740], | |
| 20.00th=[ 804], 30.00th=[ 876], 40.00th=[ 988], | |
| 50.00th=[ 1144], 60.00th=[ 1288], 70.00th=[ 14144], | |
| 80.00th=[ 716800], 90.00th=[ 2867200], 95.00th=[ 23986176], | |
| 99.00th=[217055232], 99.50th=[325058560], 99.90th=[549453824], | |
| 99.95th=[641728512], 99.99th=[910163968] | |
bw ( KiB/s): min= 0, max= 5579, per=13.40%, avg=3768.00, stdev=2561.14, samples=2 | |
iops : min= 0, max= 87, avg=58.50, stdev=40.31, samples=2 | |
write: IOPS=60, BW=3854KiB/s (3946kB/s)(452MiB/120004msec) | |
slat (nsec): min=0, max=17335k, avg=183883.83, stdev=342978.00 | |
clat (nsec): min=0, max=1171.4M, avg=59432543.96, stdev=120575360.93 | |
lat (nsec): min=0, max=1171.5M, avg=59617587.11, stdev=120535896.17 | |
clat percentiles (usec): | |
| 1.00th=[ 1020], 5.00th=[ 1221], 10.00th=[ 1418], | |
| 20.00th=[ 1762], 30.00th=[ 2343], 40.00th=[ 3490], | |
| 50.00th=[ 6718], 60.00th=[ 16057], 70.00th=[ 42730], | |
| 80.00th=[ 90702], 90.00th=[ 154141], 95.00th=[ 320865], | |
| 99.00th=[ 608175], 99.50th=[ 700449], 99.90th=[ 910164], | |
| 99.95th=[ 943719], 99.99th=[1166017] | |
bw ( KiB/s): min= 0, max= 5643, per=60.29%, avg=3851.50, stdev=2533.56, samples=2 | |
iops : min= 0, max= 88, avg=60.00, stdev=39.60, samples=2 | |
lat (nsec) : 750=6.79%, 1000=13.47% | |
lat (usec) : 2=13.65%, 4=0.09%, 10=0.23%, 20=1.02%, 50=0.93% | |
lat (usec) : 100=0.99%, 250=1.08%, 500=0.86%, 750=0.66%, 1000=2.05% | |
lat (msec) : 2=14.38%, 4=10.35%, 10=7.07%, 20=4.86%, 50=5.50% | |
lat (msec) : 100=6.33%, 250=5.87%, 500=2.76%, 750=0.91%, 1000=0.14% | |
cpu : usr=0.08%, sys=0.34%, ctx=16157, majf=0, minf=31 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=7067,7224,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=27.5MiB/s (28.8MB/s), 3769KiB/s-23.8MiB/s (3859kB/s-24.9MB/s), io=3297MiB (3457MB), run=120004-120067msec | |
WRITE: bw=6388KiB/s (6541kB/s), 2540KiB/s-3854KiB/s (2601kB/s-3946kB/s), io=750MiB (786MB), run=120004-120173msec | |
Disk stats (read/write): | |
rbd12: ios=7283/12276, merge=0/1, ticks=539258/835111, in_queue=1364388, util=12.96% | |
Jobs: 7 (f=7): [_(2),m(1)][98.8%][r=455M,w=443M][r=7105,w=6923 IOPS][eta 00m:49s] Jobs: 7 (f=7): [_(2),m(1)][98.7%][r=583M,w=562M][r=9104,w=8780 IOPS][eta 00m:48s] <example-benchmark-fio-server-435-benchmark-5cvqg> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:35:02 2019 | |
write: IOPS=22, BW=1412KiB/s (1446kB/s)(166MiB/120304msec) | |
slat (nsec): min=0, max=777619k, avg=1045993.83, stdev=24696843.15 | |
clat (nsec): min=0, max=3272.3M, avg=181650449.15, stdev=336408176.45 | |
lat (nsec): min=0, max=3273.1M, avg=182696909.40, stdev=336849572.89 | |
clat percentiles (usec): | |
| 1.00th=[ 1139], 5.00th=[ 1467], 10.00th=[ 1811], | |
| 20.00th=[ 2540], 30.00th=[ 3720], 40.00th=[ 5997], | |
| 50.00th=[ 14746], 60.00th=[ 39584], 70.00th=[ 92799], | |
| 80.00th=[ 425722], 90.00th=[ 675283], 95.00th=[ 884999], | |
| 99.00th=[1367344], 99.50th=[1669333], 99.90th=[2332034], | |
| 99.95th=[2902459], 99.99th=[3271558] | |
bw ( KiB/s): min= 0, max= 1468, per=23.39%, avg=1412.00, stdev=79.20, samples=2 | |
iops : min= 0, max= 22, avg=21.50, stdev= 0.71, samples=2 | |
lat (usec) : 4=0.04%, 50=0.04%, 100=0.04%, 500=0.04%, 1000=0.23% | |
lat (msec) : 2=12.79%, 4=18.75%, 10=15.01%, 20=6.79%, 50=8.45% | |
lat (msec) : 100=9.02%, 250=6.56%, 500=5.62%, 750=8.26%, 1000=5.62% | |
cpu : usr=0.02%, sys=0.11%, ctx=4893, majf=0, minf=27 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,2651,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:35:02 2019 | |
read: IOPS=540, BW=33.6MiB/s (35.3MB/s)(4039MiB/120089msec) | |
slat (nsec): min=0, max=682499k, avg=37730.14, stdev=3821435.40 | |
clat (nsec): min=0, max=3266.9M, avg=7397397.95, stdev=73619786.81 | |
lat (nsec): min=0, max=3266.9M, avg=7436102.24, stdev=73726006.00 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 724], | |
| 20.00th=[ 748], 30.00th=[ 780], 40.00th=[ 804], | |
| 50.00th=[ 844], 60.00th=[ 868], 70.00th=[ 900], | |
| 80.00th=[ 1144], 90.00th=[ 12608], 95.00th=[ 24448], | |
| 99.00th=[ 350224384], 99.50th=[ 591396864], 99.90th=[ 977272832], | |
| 99.95th=[1249902592], 99.99th=[1837105152] | |
bw ( KiB/s): min= 0, max=54965, per=88.02%, avg=34356.50, stdev=29144.82, samples=2 | |
iops : min= 0, max= 863, avg=538.50, stdev=458.91, samples=2 | |
lat (nsec) : 750=20.50%, 1000=57.38% | |
lat (usec) : 2=5.04%, 4=0.15%, 10=0.99%, 20=9.96%, 50=1.54% | |
lat (usec) : 100=0.15%, 250=0.11%, 500=0.09%, 750=0.15%, 1000=0.35% | |
lat (msec) : 2=0.72%, 4=0.49%, 10=0.29%, 20=0.17%, 50=0.41% | |
lat (msec) : 100=0.38%, 250=0.09%, 500=0.39%, 750=0.37%, 1000=0.19% | |
cpu : usr=0.13%, sys=0.52%, ctx=5705, majf=0, minf=34 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=64882,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:35:02 2019 | |
read: IOPS=71, BW=4595KiB/s (4705kB/s)(538MiB/120003msec) | |
slat (nsec): min=0, max=3033.9k, avg=26259.86, stdev=80062.73 | |
clat (nsec): min=0, max=687222k, avg=4836633.32, stdev=32108832.60 | |
lat (nsec): min=0, max=687261k, avg=4863797.12, stdev=32118036.94 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 740], | |
| 20.00th=[ 764], 30.00th=[ 860], 40.00th=[ 972], | |
| 50.00th=[ 1160], 60.00th=[ 1352], 70.00th=[ 19072], | |
| 80.00th=[ 460800], 90.00th=[ 1597440], 95.00th=[ 7962624], | |
| 99.00th=[130547712], 99.50th=[238026752], 99.90th=[434110464], | |
| 99.95th=[526385152], 99.99th=[683671552] | |
bw ( KiB/s): min= 0, max= 8049, per=11.77%, avg=4593.50, stdev=4886.81, samples=2 | |
iops : min= 0, max= 125, avg=71.00, stdev=76.37, samples=2 | |
write: IOPS=72, BW=4638KiB/s (4750kB/s)(544MiB/120003msec) | |
slat (nsec): min=0, max=9475.9k, avg=183809.96, stdev=263225.44 | |
clat (nsec): min=0, max=998063k, avg=50489057.96, stdev=117048370.19 | |
lat (nsec): min=0, max=998145k, avg=50674028.51, stdev=117014939.57 | |
clat percentiles (usec): | |
| 1.00th=[ 963], 5.00th=[ 1139], 10.00th=[ 1287], 20.00th=[ 1532], | |
| 30.00th=[ 1876], 40.00th=[ 2507], 50.00th=[ 3720], 60.00th=[ 7439], | |
| 70.00th=[ 21627], 80.00th=[ 71828], 90.00th=[121111], 95.00th=[312476], | |
| 99.00th=[574620], 99.50th=[683672], 99.90th=[926942], 99.95th=[935330], | |
| 99.99th=[994051] | |
bw ( KiB/s): min= 0, max= 8099, per=76.81%, avg=4637.50, stdev=4895.30, samples=2 | |
iops : min= 0, max= 126, avg=72.00, stdev=76.37, samples=2 | |
lat (nsec) : 750=7.98%, 1000=12.81% | |
lat (usec) : 2=12.09%, 4=0.05%, 10=0.40%, 20=1.62%, 50=1.02% | |
lat (usec) : 100=1.18%, 250=1.75%, 500=1.09%, 750=0.99%, 1000=2.65% | |
lat (msec) : 2=18.00%, 4=10.84%, 10=6.70%, 20=3.30%, 50=4.31% | |
lat (msec) : 100=5.60%, 250=4.41%, 500=2.16%, 750=0.86%, 1000=0.18% | |
cpu : usr=0.09%, sys=0.40%, ctx=18391, majf=0, minf=37 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=8615,8694,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=38.1MiB/s (39.0MB/s), 4595KiB/s-33.6MiB/s (4705kB/s-35.3MB/s), io=4578MiB (4800MB), run=120003-120089msec | |
WRITE: bw=6039KiB/s (6183kB/s), 1412KiB/s-4638KiB/s (1446kB/s-4750kB/s), io=709MiB (744MB), run=120003-120304msec | |
Disk stats (read/write): | |
rbd110: ios=4843/11517, merge=0/1, ticks=536620/888979, in_queue=1417352, util=7.88% | |
Jobs: 6 (f=6): [_(2),m(1)][98.6%][r=487M,w=494M][r=7605,w=7724 IOPS][eta 00m:47s] Jobs: 6 (f=6): [_(2),m(1)][98.6%][r=439M,w=449M][r=6851,w=7007 IOPS][eta 00m:45s] Jobs: 6 (f=6): [_(2),m(1)][98.6%][r=483M,w=472M][r=7539,w=7372 IOPS][eta 00m:45s] Jobs: 6 (f=6): [_(2),m(1)][98.7%][r=423M,w=420M][r=6605,w=6560 IOPS][eta 00m:44s] Jobs: 6 (f=6): [_(2),m(1)][98.7%][r=454M,w=452M][r=7088,w=7060 IOPS][eta 00m:42s] Jobs: 6 (f=6): [_(2),m(1)][98.8%][r=175M,w=169M][r=2727,w=2633 IOPS][eta 00m:41s] Jobs: 6 (f=6): [_(2),m(1)][98.8%][r=452M,w=453M][r=7063,w=7085 IOPS][eta 00m:41s] Jobs: 6 (f=6): [_(2),m(1)][98.8%][r=443M,w=438M][r=6916,w=6842 IOPS][eta 00m:40s] <example-benchmark-fio-server-503-benchmark-6fx6x> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:35:10 2019 | |
write: IOPS=57, BW=3665KiB/s (3753kB/s)(430MiB/120026msec) | |
slat (nsec): min=0, max=622630k, avg=345543.41, stdev=9113718.13 | |
clat (nsec): min=0, max=1384.4M, avg=69695686.91, stdev=136504038.48 | |
lat (nsec): min=0, max=1384.4M, avg=70042389.89, stdev=136680584.74 | |
clat percentiles (usec): | |
| 1.00th=[ 938], 5.00th=[ 1123], 10.00th=[ 1287], | |
| 20.00th=[ 1565], 30.00th=[ 1958], 40.00th=[ 2638], | |
| 50.00th=[ 4293], 60.00th=[ 16319], 70.00th=[ 63701], | |
| 80.00th=[ 101188], 90.00th=[ 208667], 95.00th=[ 371196], | |
| 99.00th=[ 675283], 99.50th=[ 759170], 99.90th=[ 918553], | |
| 99.95th=[ 935330], 99.99th=[1384121] | |
bw ( KiB/s): min= 0, max= 5002, per=29.12%, avg=3663.00, stdev=1893.63, samples=2 | |
iops : min= 0, max= 78, avg=57.00, stdev=29.70, samples=2 | |
lat (usec) : 4=0.01%, 500=0.01%, 750=0.09%, 1000=1.89% | |
lat (msec) : 2=28.85%, 4=18.14%, 10=8.84%, 20=2.87%, 50=5.82% | |
lat (msec) : 100=13.11%, 250=11.95%, 500=5.62%, 750=2.21%, 1000=0.61% | |
cpu : usr=0.07%, sys=0.24%, ctx=12882, majf=0, minf=27 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,6870,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:35:10 2019 | |
read: IOPS=363, BW=22.7MiB/s (23.8MB/s)(2725MiB/120010msec) | |
slat (nsec): min=0, max=597722k, avg=43845.37, stdev=3451713.75 | |
clat (nsec): min=0, max=952019k, avg=10981275.55, stdev=56162989.38 | |
lat (nsec): min=0, max=952050k, avg=11025965.24, stdev=56275129.48 | |
clat percentiles (nsec): | |
| 1.00th=[ 700], 5.00th=[ 708], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 732], 40.00th=[ 772], | |
| 50.00th=[ 836], 60.00th=[ 884], 70.00th=[ 1048], | |
| 80.00th=[ 8768], 90.00th=[ 3489792], 95.00th=[ 52166656], | |
| 99.00th=[312475648], 99.50th=[446693376], 99.90th=[683671552], | |
| 99.95th=[759169024], 99.99th=[843055104] | |
bw ( KiB/s): min= 0, max=33364, per=72.38%, avg=23248.00, stdev=14306.18, samples=2 | |
iops : min= 0, max= 522, avg=363.50, stdev=224.15, samples=2 | |
lat (nsec) : 750=35.04%, 1000=33.44% | |
lat (usec) : 2=11.20%, 4=0.06%, 10=0.69%, 20=0.35%, 50=0.13% | |
lat (usec) : 100=0.09%, 250=0.03%, 500=0.41%, 750=1.28%, 1000=2.38% | |
lat (msec) : 2=3.27%, 4=1.88%, 10=0.96%, 20=0.58%, 50=2.99% | |
lat (msec) : 100=2.50%, 250=1.42%, 500=0.98%, 750=0.28%, 1000=0.05% | |
cpu : usr=0.12%, sys=0.40%, ctx=16531, majf=0, minf=37 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=43643,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:35:10 2019 | |
read: IOPS=138, BW=8872KiB/s (9085kB/s)(1040MiB/120001msec) | |
slat (nsec): min=0, max=423034k, avg=64612.42, stdev=3280735.86 | |
clat (nsec): min=0, max=1022.7M, avg=3889780.33, stdev=26493181.11 | |
lat (nsec): min=0, max=1022.8M, avg=3955401.07, stdev=26697984.84 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 740], 10.00th=[ 772], | |
| 20.00th=[ 900], 30.00th=[ 1128], 40.00th=[ 1544], | |
| 50.00th=[ 432128], 60.00th=[ 774144], 70.00th=[ 921600], | |
| 80.00th=[ 1269760], 90.00th=[ 2637824], 95.00th=[ 5603328], | |
| 99.00th=[ 79167488], 99.50th=[ 183500800], 99.90th=[ 379584512], | |
| 99.95th=[ 413138944], 99.99th=[1019215872] | |
bw ( KiB/s): min= 0, max= 2046, per=6.37%, avg=2046.00, stdev= 0.00, samples=1 | |
iops : min= 0, max= 32, avg=32.00, stdev= 0.00, samples=1 | |
write: IOPS=139, BW=8917KiB/s (9131kB/s)(1045MiB/120001msec) | |
slat (nsec): min=0, max=360725k, avg=196120.52, stdev=2797797.23 | |
clat (nsec): min=0, max=951017k, avg=24667440.35, stdev=78295914.29 | |
lat (nsec): min=0, max=951106k, avg=24864692.52, stdev=78319107.19 | |
clat percentiles (usec): | |
| 1.00th=[ 914], 5.00th=[ 1057], 10.00th=[ 1139], 20.00th=[ 1270], | |
| 30.00th=[ 1401], 40.00th=[ 1565], 50.00th=[ 1860], 60.00th=[ 2507], | |
| 70.00th=[ 3916], 80.00th=[ 8455], 90.00th=[ 63701], 95.00th=[128451], | |
| 99.00th=[459277], 99.50th=[534774], 99.90th=[717226], 99.95th=[784335], | |
| 99.99th=[943719] | |
bw ( KiB/s): min= 0, max= 2136, per=16.98%, avg=2136.00, stdev= 0.00, samples=1 | |
iops : min= 0, max= 33, avg=33.00, stdev= 0.00, samples=1 | |
lat (nsec) : 750=3.87%, 1000=8.51% | |
lat (usec) : 2=7.98%, 4=0.03%, 10=0.07%, 20=0.29%, 50=0.43% | |
lat (usec) : 100=0.41%, 250=0.92%, 500=2.95%, 750=3.83%, 1000=8.77% | |
lat (msec) : 2=31.61%, 4=12.24%, 10=6.91%, 20=2.28%, 50=2.35% | |
lat (msec) : 100=2.80%, 250=2.19%, 500=1.23%, 750=0.30%, 1000=0.04% | |
cpu : usr=0.15%, sys=0.66%, ctx=47021, majf=0, minf=32 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=16637,16716,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=31.4MiB/s (32.9MB/s), 8872KiB/s-22.7MiB/s (9085kB/s-23.8MB/s), io=3764MiB (3947MB), run=120001-120010msec | |
WRITE: bw=12.3MiB/s (12.9MB/s), 3665KiB/s-8917KiB/s (3753kB/s-9131kB/s), io=1475MiB (1546MB), run=120001-120026msec | |
Disk stats (read/write): | |
rbd208: ios=19607/24303, merge=0/1, ticks=580676/840563, in_queue=1399084, util=21.89% | |
Jobs: 5 (f=5): [_(2),m(1)][98.6%][r=401M,w=400M][r=6270,w=6243 IOPS][eta 00m:39s] Jobs: 5 (f=5): [_(2),m(1)][98.7%][r=402M,w=388M][r=6277,w=6068 IOPS][eta 00m:38s] Jobs: 5 (f=5): [_(2),m(1)][98.7%][r=341M,w=330M][r=5323,w=5149 IOPS][eta 00m:37s] Jobs: 5 (f=5): [_(2),m(1)][98.7%][r=372M,w=365M][r=5806,w=5706 IOPS][eta 00m:36s] Jobs: 5 (f=5): [_(2),m(1)][98.8%][r=390M,w=381M][r=6092,w=5949 IOPS][eta 00m:34s] <example-benchmark-fio-server-668-benchmark-bhqmd> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:35:15 2019 | |
write: IOPS=15, BW=1003KiB/s (1027kB/s)(119MiB/121774msec) | |
slat (nsec): min=0, max=800208k, avg=1315104.52, stdev=25820013.84 | |
clat (nsec): min=0, max=1877.4M, avg=252516056.14, stdev=421021517.60 | |
lat (nsec): min=0, max=1877.8M, avg=253830909.29, stdev=421422000.84 | |
clat percentiles (usec): | |
| 1.00th=[ 1020], 5.00th=[ 1467], 10.00th=[ 1860], | |
| 20.00th=[ 2573], 30.00th=[ 3654], 40.00th=[ 5276], | |
| 50.00th=[ 9110], 60.00th=[ 26084], 70.00th=[ 242222], | |
| 80.00th=[ 574620], 90.00th=[ 977273], 95.00th=[1233126], | |
| 99.00th=[1484784], 99.50th=[1635779], 99.90th=[1870660], | |
| 99.95th=[1870660], 99.99th=[1870660] | |
bw ( KiB/s): min= 0, max= 1290, per=7.19%, avg=1011.50, stdev=393.86, samples=2 | |
iops : min= 0, max= 20, avg=15.50, stdev= 6.36, samples=2 | |
lat (usec) : 4=0.05%, 500=0.05%, 1000=0.73% | |
lat (msec) : 2=10.97%, 4=20.89%, 10=18.43%, 20=6.72%, 50=6.98% | |
lat (msec) : 100=3.78%, 250=1.63%, 500=8.61%, 750=6.09%, 1000=5.83% | |
cpu : usr=0.01%, sys=0.08%, ctx=3340, majf=0, minf=29 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,1905,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:35:15 2019 | |
read: IOPS=267, BW=16.7MiB/s (17.5MB/s)(2005MiB/120115msec) | |
slat (nsec): min=0, max=680756k, avg=56808.85, stdev=4611819.23 | |
clat (nsec): min=0, max=1845.9M, avg=14932473.14, stdev=106837887.10 | |
lat (nsec): min=0, max=1845.0M, avg=14990289.36, stdev=106948423.50 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 716], 10.00th=[ 724], | |
| 20.00th=[ 764], 30.00th=[ 780], 40.00th=[ 796], | |
| 50.00th=[ 836], 60.00th=[ 868], 70.00th=[ 900], | |
| 80.00th=[ 1004], 90.00th=[ 12480], 95.00th=[ 2768896], | |
| 99.00th=[ 692060160], 99.50th=[ 851443712], 99.90th=[1182793728], | |
| 99.95th=[1283457024], 99.99th=[1619001344] | |
bw ( KiB/s): min= 0, max=20554, per=67.59%, avg=20554.00, stdev= 0.00, samples=1 | |
iops : min= 0, max= 321, avg=321.00, stdev= 0.00, samples=1 | |
lat (nsec) : 750=18.62%, 1000=61.37% | |
lat (usec) : 2=9.01%, 4=0.20%, 10=0.18%, 20=1.19%, 50=0.23% | |
lat (usec) : 100=0.11%, 250=0.09%, 500=0.09%, 750=0.24%, 1000=0.82% | |
lat (msec) : 2=2.05%, 4=1.55%, 10=1.02%, 20=0.52%, 50=0.48% | |
lat (msec) : 100=0.19%, 250=0.12%, 500=0.23%, 750=0.97%, 1000=0.50% | |
cpu : usr=0.07%, sys=0.29%, ctx=5937, majf=0, minf=30 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=32116,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=14: Wed May 8 20:35:15 2019 | |
read: IOPS=208, BW=13.0MiB/s (13.7MB/s)(1562MiB/120002msec) | |
slat (nsec): min=0, max=3572.6k, avg=32152.11, stdev=62686.37 | |
clat (nsec): min=0, max=610833k, avg=2659805.67, stdev=24685079.65 | |
lat (nsec): min=0, max=610861k, avg=2693027.06, stdev=24687759.61 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 740], | |
| 20.00th=[ 844], 30.00th=[ 1032], 40.00th=[ 1368], | |
| 50.00th=[ 268288], 60.00th=[ 700416], 70.00th=[ 831488], | |
| 80.00th=[ 1056768], 90.00th=[ 1941504], 95.00th=[ 3325952], | |
| 99.00th=[ 41680896], 99.50th=[101187584], 99.90th=[467664896], | |
| 99.95th=[509607936], 99.99th=[599785472] | |
bw ( KiB/s): min= 0, max=23191, per=43.83%, avg=13330.00, stdev=13945.56, samples=2 | |
iops : min= 0, max= 362, avg=208.00, stdev=217.79, samples=2 | |
write: IOPS=207, BW=12.9MiB/s (13.6MB/s)(1554MiB/120002msec) | |
slat (nsec): min=0, max=12405k, avg=162543.64, stdev=196155.68 | |
clat (nsec): min=0, max=677169k, avg=16443104.90, stdev=52031115.06 | |
lat (nsec): min=0, max=677230k, avg=16606816.60, stdev=52011422.69 | |
clat percentiles (usec): | |
| 1.00th=[ 881], 5.00th=[ 1029], 10.00th=[ 1106], 20.00th=[ 1221], | |
| 30.00th=[ 1336], 40.00th=[ 1483], 50.00th=[ 1696], 60.00th=[ 2147], | |
| 70.00th=[ 3294], 80.00th=[ 5997], 90.00th=[ 34866], 95.00th=[103285], | |
| 99.00th=[263193], 99.50th=[383779], 99.90th=[549454], 99.95th=[591397], | |
| 99.99th=[650118] | |
bw ( KiB/s): min= 0, max=22936, per=94.24%, avg=13255.50, stdev=13690.29, samples=2 | |
iops : min= 0, max= 358, avg=206.50, stdev=214.25, samples=2 | |
lat (nsec) : 750=5.86%, 1000=8.55% | |
lat (usec) : 2=6.79%, 4=0.03%, 10=0.24%, 20=0.76%, 50=0.65% | |
lat (usec) : 100=0.61%, 250=1.32%, 500=3.16%, 750=3.98%, 1000=9.39% | |
lat (msec) : 2=32.63%, 4=11.09%, 10=6.29%, 20=2.00%, 50=1.86% | |
lat (msec) : 100=1.92%, 250=2.17%, 500=0.61%, 750=0.11% | |
cpu : usr=0.24%, sys=0.94%, ctx=66524, majf=0, minf=34 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=24996,24854,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=29.7MiB/s (31.1MB/s), 13.0MiB/s-16.7MiB/s (13.7MB/s-17.5MB/s), io=3567MiB (3740MB), run=120002-120115msec | |
WRITE: bw=13.7MiB/s (14.4MB/s), 1003KiB/s-12.9MiB/s (1027kB/s-13.6MB/s), io=1673MiB (1754MB), run=120002-121774msec | |
Disk stats (read/write): | |
rbd227: ios=15912/26988, merge=0/1, ticks=587346/884907, in_queue=1450898, util=10.17% | |
<example-benchmark-fio-server-700-benchmark-lbhjl> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:35:15 2019 | |
write: IOPS=46, BW=2954KiB/s (3025kB/s)(346MiB/120020msec) | |
slat (nsec): min=0, max=941199k, avg=450397.43, stdev=14470997.85 | |
clat (nsec): min=0, max=1098.0M, avg=86745193.42, stdev=158139041.80 | |
lat (nsec): min=0, max=1098.1M, avg=87196573.48, stdev=158617381.78 | |
clat percentiles (usec): | |
| 1.00th=[ 1004], 5.00th=[ 1205], 10.00th=[ 1418], | |
| 20.00th=[ 1844], 30.00th=[ 2376], 40.00th=[ 3720], | |
| 50.00th=[ 8848], 60.00th=[ 42206], 70.00th=[ 88605], | |
| 80.00th=[ 125305], 90.00th=[ 274727], 95.00th=[ 408945], | |
| 99.00th=[ 801113], 99.50th=[ 926942], 99.90th=[1044382], | |
| 99.95th=[1098908], 99.99th=[1098908] | |
bw ( KiB/s): min= 0, max= 2189, per=14.78%, avg=2189.00, stdev= 0.00, samples=1 | |
iops : min= 0, max= 34, avg=34.00, stdev= 0.00, samples=1 | |
lat (usec) : 500=0.02%, 1000=0.94% | |
lat (msec) : 2=22.85%, 4=17.54%, 10=9.64%, 20=4.55%, 50=5.82% | |
lat (msec) : 100=12.14%, 250=15.66%, 500=7.57%, 750=1.99%, 1000=1.07% | |
cpu : usr=0.06%, sys=0.19%, ctx=10237, majf=0, minf=30 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,5537,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:35:15 2019 | |
read: IOPS=435, BW=27.2MiB/s (28.5MB/s)(3266MiB/120022msec) | |
slat (nsec): min=0, max=519236k, avg=36005.92, stdev=2710731.95 | |
clat (nsec): min=0, max=1273.1M, avg=9152643.07, stdev=51653585.32 | |
lat (nsec): min=0, max=1273.1M, avg=9189505.59, stdev=51731663.29 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 708], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 732], 40.00th=[ 740], | |
| 50.00th=[ 804], 60.00th=[ 876], 70.00th=[ 980], | |
| 80.00th=[ 1480], 90.00th=[ 2441216], 95.00th=[ 44826624], | |
| 99.00th=[ 254803968], 99.50th=[ 413138944], 99.90th=[ 683671552], | |
| 99.95th=[ 750780416], 99.99th=[1019215872] | |
bw ( KiB/s): min= 0, max=42058, per=70.17%, avg=27869.00, stdev=20066.28, samples=2 | |
iops : min= 0, max= 658, avg=435.50, stdev=314.66, samples=2 | |
lat (nsec) : 750=43.47%, 1000=27.31% | |
lat (usec) : 2=9.90%, 4=0.05%, 10=1.01%, 20=0.67%, 50=0.25% | |
lat (usec) : 100=0.11%, 250=0.08%, 500=0.90%, 750=1.13%, 1000=1.88% | |
lat (msec) : 2=2.63%, 4=1.72%, 10=0.99%, 20=0.62%, 50=2.79% | |
lat (msec) : 100=2.38%, 250=1.09%, 500=0.75%, 750=0.22%, 1000=0.04% | |
cpu : usr=0.13%, sys=0.48%, ctx=17884, majf=0, minf=32 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=52311,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:35:15 2019 | |
read: IOPS=185, BW=11.6MiB/s (12.1MB/s)(1389MiB/120002msec) | |
slat (nsec): min=0, max=2063.9k, avg=37114.12, stdev=56251.65 | |
clat (nsec): min=0, max=969207k, avg=2652385.66, stdev=19531361.29 | |
lat (nsec): min=0, max=969226k, avg=2690596.59, stdev=19533369.42 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 740], 10.00th=[ 772], | |
| 20.00th=[ 940], 30.00th=[ 1272], 40.00th=[ 288768], | |
| 50.00th=[ 626688], 60.00th=[ 782336], 70.00th=[ 913408], | |
| 80.00th=[ 1204224], 90.00th=[ 2342912], 95.00th=[ 4227072], | |
| 99.00th=[ 53739520], 99.50th=[ 85458944], 99.90th=[333447168], | |
| 99.95th=[425721856], 99.99th=[517996544] | |
bw ( KiB/s): min= 0, max=22243, per=29.84%, avg=11850.00, stdev=14697.92, samples=2 | |
iops : min= 0, max= 347, avg=184.50, stdev=229.81, samples=2 | |
write: IOPS=185, BW=11.6MiB/s (12.1MB/s)(1390MiB/120002msec) | |
slat (nsec): min=0, max=11228k, avg=166988.35, stdev=176151.67 | |
clat (nsec): min=0, max=1018.4M, avg=18784576.99, stdev=68641704.52 | |
lat (nsec): min=0, max=1018.5M, avg=18952667.28, stdev=68621574.76 | |
clat percentiles (usec): | |
| 1.00th=[ 889], 5.00th=[ 1020], 10.00th=[ 1106], 20.00th=[ 1205], | |
| 30.00th=[ 1303], 40.00th=[ 1434], 50.00th=[ 1598], 60.00th=[ 1909], | |
| 70.00th=[ 2704], 80.00th=[ 4752], 90.00th=[ 22414], 95.00th=[106431], | |
| 99.00th=[383779], 99.50th=[501220], 99.90th=[700449], 99.95th=[725615], | |
| 99.99th=[926942] | |
bw ( KiB/s): min= 0, max=22164, per=80.07%, avg=11862.00, stdev=14569.23, samples=2 | |
iops : min= 0, max= 346, avg=185.00, stdev=227.69, samples=2 | |
lat (nsec) : 750=3.61%, 1000=7.51% | |
lat (usec) : 2=5.58%, 4=0.02%, 10=0.09%, 20=0.35%, 50=0.45% | |
lat (usec) : 100=0.39%, 250=1.19%, 500=4.59%, 750=4.92%, 1000=10.38% | |
lat (msec) : 2=35.97%, 4=11.08%, 10=5.80%, 20=1.87%, 50=1.52% | |
lat (msec) : 100=1.77%, 250=1.74%, 500=0.91%, 750=0.23%, 1000=0.02% | |
cpu : usr=0.21%, sys=0.82%, ctx=64428, majf=0, minf=103 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=22220,22241,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=38.8MiB/s (40.7MB/s), 11.6MiB/s-27.2MiB/s (12.1MB/s-28.5MB/s), io=4655MiB (4881MB), run=120002-120022msec | |
WRITE: bw=14.5MiB/s (15.2MB/s), 2954KiB/s-11.6MiB/s (3025kB/s-12.1MB/s), io=1737MiB (1821MB), run=120002-120020msec | |
Disk stats (read/write): | |
rbd274: ios=26881/29490, merge=0/1, ticks=576168/834987, in_queue=1382881, util=24.06% | |
<example-benchmark-fio-server-292-benchmark-bqpc4> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:35:16 2019 | |
write: IOPS=31, BW=2033KiB/s (2082kB/s)(239MiB/120229msec) | |
slat (nsec): min=0, max=538330k, avg=792992.72, stdev=17014015.86 | |
clat (nsec): min=0, max=1096.8M, avg=125120214.34, stdev=193494723.33 | |
lat (nsec): min=0, max=1096.8M, avg=125914117.71, stdev=194040761.47 | |
clat percentiles (usec): | |
| 1.00th=[ 971], 5.00th=[ 1188], 10.00th=[ 1434], | |
| 20.00th=[ 2073], 30.00th=[ 3556], 40.00th=[ 7242], | |
| 50.00th=[ 22676], 60.00th=[ 85459], 70.00th=[ 125305], | |
| 80.00th=[ 210764], 90.00th=[ 392168], 95.00th=[ 549454], | |
| 99.00th=[ 868221], 99.50th=[ 918553], 99.90th=[1098908], | |
| 99.95th=[1098908], 99.99th=[1098908] | |
bw ( KiB/s): min= 0, max= 2264, per=14.20%, avg=2034.50, stdev=324.56, samples=2 | |
iops : min= 0, max= 35, avg=31.50, stdev= 4.95, samples=2 | |
lat (usec) : 500=0.05%, 750=0.03%, 1000=1.49% | |
lat (msec) : 2=17.42%, 4=13.13%, 10=11.08%, 20=5.87%, 50=5.32% | |
lat (msec) : 100=9.90%, 250=18.08%, 500=11.53%, 750=3.90%, 1000=2.04% | |
cpu : usr=0.04%, sys=0.14%, ctx=7226, majf=0, minf=29 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,3817,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:35:16 2019 | |
read: IOPS=470, BW=29.4MiB/s (30.8MB/s)(3525MiB/120058msec) | |
slat (nsec): min=0, max=528245k, avg=50365.71, stdev=3931550.84 | |
clat (nsec): min=0, max=976416k, avg=8467552.73, stdev=53420881.21 | |
lat (nsec): min=0, max=976446k, avg=8518714.93, stdev=53571321.10 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 708], 10.00th=[ 716], | |
| 20.00th=[ 724], 30.00th=[ 724], 40.00th=[ 740], | |
| 50.00th=[ 812], 60.00th=[ 852], 70.00th=[ 900], | |
| 80.00th=[ 1032], 90.00th=[ 23168], 95.00th=[ 27656192], | |
| 99.00th=[274726912], 99.50th=[442499072], 99.90th=[700448768], | |
| 99.95th=[784334848], 99.99th=[910163968] | |
bw ( KiB/s): min= 0, max=45037, per=70.93%, avg=30076.50, stdev=21157.34, samples=2 | |
iops : min= 0, max= 706, avg=471.00, stdev=332.34, samples=2 | |
lat (nsec) : 750=41.94%, 1000=37.25% | |
lat (usec) : 2=8.02%, 4=0.04%, 10=1.72%, 20=0.97%, 50=0.35% | |
lat (usec) : 100=0.10%, 250=0.03%, 500=0.09%, 750=0.46%, 1000=0.92% | |
lat (msec) : 2=1.38%, 4=0.84%, 10=0.46%, 20=0.27%, 50=1.30% | |
lat (msec) : 100=1.72%, 250=1.03%, 500=0.78%, 750=0.27%, 1000=0.06% | |
cpu : usr=0.12%, sys=0.46%, ctx=10834, majf=0, minf=29 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=56535,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:35:16 2019 | |
read: IOPS=192, BW=12.1MiB/s (12.6MB/s)(1446MiB/120008msec) | |
slat (nsec): min=0, max=537040k, avg=77887.67, stdev=4761859.59 | |
clat (nsec): min=0, max=1008.6M, avg=2244051.44, stdev=19678423.59 | |
lat (nsec): min=0, max=1018.6M, avg=2322966.53, stdev=20791907.39 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 732], 10.00th=[ 748], | |
| 20.00th=[ 844], 30.00th=[ 1032], 40.00th=[ 1528], | |
| 50.00th=[ 342016], 60.00th=[ 733184], 70.00th=[ 864256], | |
| 80.00th=[ 1155072], 90.00th=[ 2179072], 95.00th=[ 3751936], | |
| 99.00th=[ 39583744], 99.50th=[ 72876032], 99.90th=[329252864], | |
| 99.95th=[463470592], 99.99th=[608174080] | |
bw ( KiB/s): min= 0, max=22722, per=29.11%, avg=12342.00, stdev=14679.54, samples=2 | |
iops : min= 0, max= 355, avg=192.50, stdev=229.81, samples=2 | |
write: IOPS=192, BW=12.0MiB/s (12.6MB/s)(1444MiB/120008msec) | |
slat (nsec): min=0, max=410477k, avg=203439.38, stdev=3377915.14 | |
clat (nsec): min=0, max=1034.9M, avg=18357830.34, stdev=67889277.51 | |
lat (nsec): min=0, max=1034.9M, avg=18562367.41, stdev=67958632.05 | |
clat percentiles (usec): | |
| 1.00th=[ 881], 5.00th=[ 1012], 10.00th=[ 1090], 20.00th=[ 1205], | |
| 30.00th=[ 1319], 40.00th=[ 1434], 50.00th=[ 1631], 60.00th=[ 1958], | |
| 70.00th=[ 2769], 80.00th=[ 4948], 90.00th=[ 20579], 95.00th=[ 98042], | |
| 99.00th=[404751], 99.50th=[492831], 99.90th=[608175], 99.95th=[708838], | |
| 99.99th=[918553] | |
bw ( KiB/s): min= 0, max=22592, per=85.97%, avg=12319.00, stdev=14528.22, samples=2 | |
iops : min= 0, max= 353, avg=192.00, stdev=227.69, samples=2 | |
lat (nsec) : 750=5.08%, 1000=9.32% | |
lat (usec) : 2=5.93%, 4=0.01%, 10=0.22%, 20=0.58%, 50=0.52% | |
lat (usec) : 100=0.53%, 250=1.30%, 500=3.34%, 750=3.99%, 1000=9.36% | |
lat (msec) : 2=34.81%, 4=11.10%, 10=6.41%, 20=1.81%, 50=1.59% | |
lat (msec) : 100=1.50%, 250=1.44%, 500=0.93%, 750=0.24%, 1000=0.01% | |
cpu : usr=0.23%, sys=0.83%, ctx=63019, majf=0, minf=37 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=23145,23098,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=41.4MiB/s (43.4MB/s), 12.1MiB/s-29.4MiB/s (12.6MB/s-30.8MB/s), io=4971MiB (5213MB), run=120008-120058msec | |
WRITE: bw=13.0MiB/s (14.7MB/s), 2033KiB/s-12.0MiB/s (2082kB/s-12.6MB/s), io=1683MiB (1764MB), run=120008-120229msec | |
Disk stats (read/write): | |
rbd153: ios=17769/27091, merge=0/1, ticks=542581/849541, in_queue=1369843, util=18.48% | |
Jobs: 2 (f=2): [_(2),m(1)][97.1%][r=158M,w=151M][r=2463,w=2356 IOPS][eta 00m:34s] Jobs: 2 (f=2): [_(2),m(1)][97.2%][r=178M,w=165M][r=2784,w=2585 IOPS][eta 00m:33s] Jobs: 2 (f=2): [_(2),m(1)][97.3%][r=172M,w=186M][r=2691,w=2905 IOPS][eta 00m:32s] Jobs: 2 (f=2): [_(2),m(1)][97.4%][r=174M,w=180M][r=2711,w=2820 IOPS][eta 00m:31s] Jobs: 2 (f=2): [_(2),m(1)][97.5%][r=151M,w=157M][r=2364,w=2454 IOPS][eta 00m:30s] Jobs: 2 (f=2): [_(2),m(1)][97.5%][r=34.6M,w=36.0M][r=539,w=561 IOPS][eta 00m:29s] Jobs: 2 (f=2): [_(2),m(1)][97.6%][r=194M,w=176M][r=3028,w=2746 IOPS][eta 00m:28s] Jobs: 2 (f=2): [_(2),m(1)][97.7%][r=182M,w=173M][r=2837,w=2706 IOPS][eta 00m:27s] <example-benchmark-fio-server-561-benchmark-p2zwl> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:35:23 2019 | |
write: IOPS=25, BW=1646KiB/s (1686kB/s)(194MiB/120815msec) | |
slat (nsec): min=0, max=904337k, avg=1214381.02, stdev=28710695.03 | |
clat (nsec): min=0, max=2205.6M, avg=154348561.57, stdev=313344406.92 | |
lat (nsec): min=0, max=2205.7M, avg=155563464.57, stdev=315846325.37 | |
clat percentiles (usec): | |
| 1.00th=[ 1012], 5.00th=[ 1254], 10.00th=[ 1483], | |
| 20.00th=[ 1926], 30.00th=[ 2671], 40.00th=[ 3752], | |
| 50.00th=[ 5735], 60.00th=[ 11469], 70.00th=[ 44303], | |
| 80.00th=[ 183501], 90.00th=[ 641729], 95.00th=[ 868221], | |
| 99.00th=[1317012], 99.50th=[1417675], 99.90th=[1988101], | |
| 99.95th=[2164261], 99.99th=[2197816] | |
bw ( KiB/s): min= 0, max= 2024, per=8.70%, avg=1655.00, stdev=521.84, samples=2 | |
iops : min= 0, max= 31, avg=25.50, stdev= 7.78, samples=2 | |
lat (usec) : 500=0.03%, 750=0.06%, 1000=0.81% | |
lat (msec) : 2=20.35%, 4=20.32%, 10=16.46%, 20=7.02%, 50=5.96% | |
lat (msec) : 100=6.02%, 250=4.35%, 500=3.74%, 750=7.83%, 1000=3.96% | |
cpu : usr=0.03%, sys=0.11%, ctx=5679, majf=0, minf=26 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,3105,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:35:23 2019 | |
read: IOPS=488, BW=30.4MiB/s (31.9MB/s)(3658MiB/120210msec) | |
slat (nsec): min=0, max=893326k, avg=59818.55, stdev=5752672.97 | |
clat (nsec): min=0, max=1867.1M, avg=8176169.39, stdev=74702180.11 | |
lat (nsec): min=0, max=1867.1M, avg=8236986.67, stdev=74929978.25 | |
clat percentiles (nsec): | |
| 1.00th=[ 716], 5.00th=[ 724], 10.00th=[ 724], | |
| 20.00th=[ 740], 30.00th=[ 780], 40.00th=[ 796], | |
| 50.00th=[ 828], 60.00th=[ 860], 70.00th=[ 892], | |
| 80.00th=[ 940], 90.00th=[ 11584], 95.00th=[ 790528], | |
| 99.00th=[ 404750336], 99.50th=[ 624951296], 99.90th=[1061158912], | |
| 99.95th=[1233125376], 99.99th=[1417674752] | |
bw ( KiB/s): min= 0, max=45246, per=64.08%, avg=31208.00, stdev=19852.73, samples=2 | |
iops : min= 0, max= 710, avg=489.00, stdev=312.54, samples=2 | |
lat (nsec) : 750=21.79%, 1000=60.61% | |
lat (usec) : 2=5.91%, 4=0.14%, 10=0.58%, 20=4.82%, 50=0.60% | |
lat (usec) : 100=0.20%, 250=0.09%, 500=0.09%, 750=0.12%, 1000=0.44% | |
lat (msec) : 2=1.01%, 4=0.75%, 10=0.49%, 20=0.23%, 50=0.43% | |
lat (msec) : 100=0.45%, 250=0.11%, 500=0.45%, 750=0.43%, 1000=0.16% | |
cpu : usr=0.11%, sys=0.48%, ctx=6521, majf=0, minf=32 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=58746,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:35:23 2019 | |
read: IOPS=274, BW=17.2MiB/s (17.0MB/s)(2060MiB/120002msec) | |
slat (nsec): min=0, max=11991k, avg=33428.44, stdev=87379.39 | |
clat (nsec): min=0, max=1079.6M, avg=1861929.90, stdev=18135558.66 | |
lat (nsec): min=0, max=1079.6M, avg=1896414.99, stdev=18137606.47 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 732], 10.00th=[ 756], | |
| 20.00th=[ 908], 30.00th=[ 1352], 40.00th=[ 284672], | |
| 50.00th=[ 505856], 60.00th=[ 741376], 70.00th=[ 839680], | |
| 80.00th=[ 1036288], 90.00th=[ 1843200], 95.00th=[ 3063808], | |
| 99.00th=[ 22413312], 99.50th=[ 58458112], 99.90th=[263192576], | |
| 99.95th=[375390208], 99.99th=[926941184] | |
bw ( KiB/s): min= 0, max=34082, per=36.09%, avg=17577.00, stdev=23341.59, samples=2 | |
iops : min= 0, max= 532, avg=274.00, stdev=364.87, samples=2 | |
write: IOPS=273, BW=17.1MiB/s (17.9MB/s)(2051MiB/120002msec) | |
slat (nsec): min=0, max=386496k, avg=167469.00, stdev=2137773.06 | |
clat (nsec): min=0, max=1040.4M, avg=12588891.95, stdev=54899280.21 | |
lat (nsec): min=0, max=1040.4M, avg=12757420.24, stdev=54996504.54 | |
clat percentiles (usec): | |
| 1.00th=[ 857], 5.00th=[ 988], 10.00th=[ 1057], 20.00th=[ 1156], | |
| 30.00th=[ 1237], 40.00th=[ 1336], 50.00th=[ 1450], 60.00th=[ 1647], | |
| 70.00th=[ 2114], 80.00th=[ 3392], 90.00th=[ 9110], 95.00th=[ 42730], | |
| 99.00th=[337642], 99.50th=[438305], 99.90th=[566232], 99.95th=[608175], | |
| 99.99th=[692061] | |
bw ( KiB/s): min= 0, max=33863, per=91.97%, avg=17504.00, stdev=23135.12, samples=2 | |
iops : min= 0, max= 529, avg=273.00, stdev=362.04, samples=2 | |
lat (nsec) : 750=4.52%, 1000=7.01% | |
lat (usec) : 2=4.12%, 4=0.02%, 10=0.13%, 20=0.47%, 50=0.50% | |
lat (usec) : 100=0.44%, 250=1.78%, 500=6.01%, 750=5.68%, 1000=11.66% | |
lat (msec) : 2=37.37%, 4=10.03%, 10=4.85%, 20=1.67%, 50=1.06% | |
lat (msec) : 100=0.95%, 250=0.97%, 500=0.62%, 750=0.15%, 1000=0.01% | |
cpu : usr=0.31%, sys=1.09%, ctx=95248, majf=0, minf=34 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=32958,32820,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=47.6MiB/s (49.9MB/s), 17.2MiB/s-30.4MiB/s (17.0MB/s-31.9MB/s), io=5717MiB (5995MB), run=120002-120210msec | |
WRITE: bw=18.6MiB/s (19.5MB/s), 1646KiB/s-17.1MiB/s (1686kB/s-17.9MB/s), io=2246MiB (2355MB), run=120002-120815msec | |
Disk stats (read/write): | |
rbd281: ios=24610/36374, merge=0/1, ticks=576701/893934, in_queue=1440916, util=15.37% | |
Jobs: 1 (f=1): [_(2),m(1)][94.7%][r=92.5M,w=89.8M][r=1446,w=1403 IOPS][eta 00m:26s] Jobs: 1 (f=1): [_(2),m(1)][94.9%][r=82.6M,w=89.0M][r=1291,w=1391 IOPS][eta 00m:25s] Jobs: 1 (f=1): [_(2),m(1)][95.1%][r=93.7M,w=90.5M][r=1463,w=1413 IOPS][eta 00m:24s] Jobs: 1 (f=1): [_(2),m(1)][95.3%][r=89.0M,w=94.5M][r=1406,w=1476 IOPS][eta 00m:23s] Jobs: 1 (f=1): [_(2),m(1)][95.5%][r=91.4M,w=89.8M][r=1428,w=1403 IOPS][eta 00m:22s] Jobs: 1 (f=1): [_(2),m(1)][95.7%][r=100M,w=97.9M][r=1564,w=1530 IOPS][eta 00m:21s] Jobs: 1 (f=1): [_(2),m(1)][95.9%][r=36.5M,w=39.4M][r=571,w=615 IOPS][eta 00m:20s] Jobs: 1 (f=1): [_(2),m(1)][96.1%][r=36.7M,w=40.8M][r=574,w=638 IOPS][eta 00m:19s] Jobs: 1 (f=1): [_(2),m(1)][96.3%][r=95.3M,w=94.1M][r=1489,w=1470 IOPS][eta 00m:18s] Jobs: 1 (f=1): [_(2),m(1)][96.5%][r=116M,w=114M][r=1811,w=1787 IOPS][eta 00m:17s] Jobs: 1 (f=1): [_(2),m(1)][96.9%][r=113M,w=115M][r=1765,w=1797 IOPS][eta 00m:15s] Jobs: 1 (f=1): [_(2),m(1)][96.9%][r=118M,w=119M][r=1851,w=1857 IOPS][eta 00m:15s] Jobs: 1 (f=1): [_(2),m(1)][97.1%][r=113M,w=117M][r=1769,w=1825 IOPS][eta 00m:14s] Jobs: 1 (f=1): [_(2),m(1)][97.6%][r=117M,w=117M][r=1835,w=1833 IOPS][eta 00m:12s] Jobs: 1 (f=1): [_(2),m(1)][97.6%][r=112M,w=114M][r=1747,w=1778 IOPS][eta 00m:12s] Jobs: 1 (f=1): [_(2),m(1)][97.8%][r=120M,w=118M][r=1871,w=1840 IOPS][eta 00m:11s] Jobs: 1 (f=1): [_(2),m(1)][98.0%][r=114M,w=120M][r=1775,w=1872 IOPS][eta 00m:10s] Jobs: 1 (f=1): [_(2),m(1)][98.4%][r=110M,w=102M][r=1720,w=1599 IOPS][eta 00m:08s] Jobs: 1 (f=1): [_(2),m(1)][98.4%][r=20.8M,w=25.1M][r=324,w=391 IOPS][eta 00m:08s] Jobs: 1 (f=1): [_(2),m(1)][98.6%][r=106M,w=102M][r=1663,w=1601 IOPS][eta 00m:07s] Jobs: 1 (f=1): [_(2),m(1)][98.8%][r=107M,w=107M][r=1677,w=1665 IOPS][eta 00m:06s] Jobs: 1 (f=1): [_(2),m(1)][99.0%][r=107M,w=105M][r=1667,w=1635 IOPS][eta 00m:05s] Jobs: 1 (f=1): [_(2),m(1)][99.2%][r=105M,w=104M][r=1645,w=1622 IOPS][eta 00m:04s] Jobs: 1 (f=1): [_(2),m(1)][99.4%][r=106M,w=111M][r=1663,w=1736 IOPS][eta 00m:03s] Jobs: 1 (f=1): [_(2),m(1)][99.6%][r=107M,w=101M][r=1677,w=1585 IOPS][eta 00m:02s] Jobs: 1 (f=1): [_(2),m(1)][99.8%][r=106M,w=107M][r=1654,w=1665 IOPS][eta 00m:01s] Jobs: 1 (f=1): [_(2),m(1)][100.0%][r=108M,w=107M][r=1686,w=1669 IOPS][eta 00m:00s] <example-benchmark-fio-server-657-benchmark-nfsfh> | |
randwrite: (groupid=0, jobs=1): err= 0: pid=11: Wed May 8 20:35:50 2019 | |
write: IOPS=40, BW=2566KiB/s (2628kB/s)(301MiB/120045msec) | |
slat (nsec): min=0, max=482145k, avg=438482.98, stdev=10952692.52 | |
clat (nsec): min=0, max=1658.2M, avg=99703536.50, stdev=166032787.67 | |
lat (nsec): min=0, max=1658.3M, avg=100143246.46, stdev=166305537.98 | |
clat percentiles (usec): | |
| 1.00th=[ 947], 5.00th=[ 1188], 10.00th=[ 1418], | |
| 20.00th=[ 1909], 30.00th=[ 3163], 40.00th=[ 7635], | |
| 50.00th=[ 40109], 60.00th=[ 70779], 70.00th=[ 101188], | |
| 80.00th=[ 130548], 90.00th=[ 316670], 95.00th=[ 446694], | |
| 99.00th=[ 750781], 99.50th=[ 935330], 99.90th=[1451230], | |
| 99.95th=[1602225], 99.99th=[1652556] | |
bw ( KiB/s): min= 0, max= 1573, per=3.75%, avg=1573.00, stdev= 0.00, samples=1 | |
iops : min= 0, max= 24, avg=24.00, stdev= 0.00, samples=1 | |
lat (usec) : 500=0.02%, 750=0.06%, 1000=1.64% | |
lat (msec) : 2=19.38%, 4=12.47%, 10=8.40%, 20=3.24%, 50=7.46% | |
lat (msec) : 100=16.80%, 250=18.79%, 500=8.40%, 750=2.35%, 1000=0.64% | |
cpu : usr=0.05%, sys=0.16%, ctx=8915, majf=0, minf=26 | |
IO depths : 1=0.1%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=0,4810,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
read: (groupid=0, jobs=1): err= 0: pid=12: Wed May 8 20:35:50 2019 | |
read: IOPS=612, BW=38.2MiB/s (40.0MB/s)(4582MiB/120023msec) | |
slat (nsec): min=0, max=463662k, avg=27019.50, stdev=2410645.20 | |
clat (nsec): min=0, max=1544.2M, avg=6507416.01, stdev=46288893.88 | |
lat (nsec): min=0, max=1544.3M, avg=6535236.00, stdev=46359211.29 | |
clat percentiles (nsec): | |
| 1.00th=[ 708], 5.00th=[ 708], 10.00th=[ 716], | |
| 20.00th=[ 716], 30.00th=[ 724], 40.00th=[ 732], | |
| 50.00th=[ 740], 60.00th=[ 812], 70.00th=[ 868], | |
| 80.00th=[ 948], 90.00th=[ 10432], 95.00th=[ 8454144], | |
| 99.00th=[ 191889408], 99.50th=[ 358612992], 99.90th=[ 616562688], | |
| 99.95th=[ 750780416], 99.99th=[1149239296] | |
bw ( KiB/s): min= 0, max=57456, per=49.78%, avg=39101.00, stdev=25957.89, samples=2 | |
iops : min= 0, max= 901, avg=612.50, stdev=408.00, samples=2 | |
lat (nsec) : 750=52.09%, 1000=30.09% | |
lat (usec) : 2=6.66%, 4=0.04%, 10=1.10%, 20=0.87%, 50=0.30% | |
lat (usec) : 100=0.07%, 250=0.04%, 500=0.30%, 750=0.53%, 1000=0.91% | |
lat (msec) : 2=1.09%, 4=0.63%, 10=0.33%, 20=0.24%, 50=1.71% | |
lat (msec) : 100=1.41%, 250=0.79%, 500=0.62%, 750=0.14%, 1000=0.04% | |
cpu : usr=0.16%, sys=0.56%, ctx=12882, majf=0, minf=108 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=73566,0,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
readwrite: (groupid=0, jobs=1): err= 0: pid=13: Wed May 8 20:35:50 2019 | |
read: IOPS=616, BW=38.5MiB/s (40.4MB/s)(4624MiB/120002msec) | |
slat (nsec): min=0, max=4051.2k, avg=34070.14, stdev=38397.97 | |
clat (nsec): min=0, max=983244k, avg=1157929.31, stdev=11857203.78 | |
lat (nsec): min=0, max=983271k, avg=1193149.93, stdev=11857821.07 | |
clat percentiles (nsec): | |
| 1.00th=[ 724], 5.00th=[ 780], 10.00th=[ 996], | |
| 20.00th=[ 248832], 30.00th=[ 313344], 40.00th=[ 399360], | |
| 50.00th=[ 643072], 60.00th=[ 733184], 70.00th=[ 806912], | |
| 80.00th=[ 921600], 90.00th=[ 1302528], 95.00th=[ 2211840], | |
| 99.00th=[ 5668864], 99.50th=[ 11862016], 99.90th=[ 98041856], | |
| 99.95th=[212860928], 99.99th=[692060160] | |
bw ( KiB/s): min= 0, max=76727, per=50.24%, avg=39461.00, stdev=52702.08, samples=2 | |
iops : min= 0, max= 1198, avg=616.00, stdev=823.07, samples=2 | |
write: IOPS=615, BW=38.5MiB/s (40.4MB/s)(4619MiB/120002msec) | |
slat (nsec): min=0, max=32795k, avg=144062.14, stdev=169244.18 | |
clat (nsec): min=0, max=985893k, avg=5161799.35, stdev=27012639.31 | |
lat (nsec): min=0, max=985966k, avg=5306860.77, stdev=27005522.14 | |
clat percentiles (usec): | |
| 1.00th=[ 848], 5.00th=[ 979], 10.00th=[ 1057], 20.00th=[ 1139], | |
| 30.00th=[ 1221], 40.00th=[ 1287], 50.00th=[ 1352], 60.00th=[ 1434], | |
| 70.00th=[ 1565], 80.00th=[ 1860], 90.00th=[ 3392], 95.00th=[ 6783], | |
| 99.00th=[111674], 99.50th=[189793], 99.90th=[375391], 99.95th=[413139], | |
| 99.99th=[574620] | |
bw ( KiB/s): min= 0, max=76539, per=93.92%, avg=39416.50, stdev=52499.14, samples=2 | |
iops : min= 0, max= 1195, avg=615.00, stdev=820.24, samples=2 | |
lat (nsec) : 750=1.86%, 1000=3.17% | |
lat (usec) : 2=1.65%, 4=0.01%, 10=0.05%, 20=0.16%, 50=0.19% | |
lat (usec) : 100=0.18%, 250=2.78%, 500=12.46%, 750=8.78%, 1000=13.88% | |
lat (msec) : 2=43.04%, 4=6.77%, 10=2.70%, 20=0.68%, 50=0.48% | |
lat (msec) : 100=0.50%, 250=0.48%, 500=0.17%, 750=0.01%, 1000=0.01% | |
cpu : usr=0.67%, sys=2.18%, ctx=235909, majf=0, minf=101 | |
IO depths : 1=0.0%, 2=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% | |
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% | |
issued rwts: total=73990,73908,0,0 short=0,0,0,0 dropped=0,0,0,0 | |
latency : target=0, window=0, percentile=100.00%, depth=4 | |
Run status group 0 (all jobs): | |
READ: bw=76.7MiB/s (80.4MB/s), 38.2MiB/s-38.5MiB/s (40.0MB/s-40.4MB/s), io=9207MiB (9654MB), run=120002-120023msec | |
WRITE: bw=40.0MiB/s (42.0MB/s), 2566KiB/s-38.5MiB/s (2628kB/s-40.4MB/s), io=4920MiB (5159MB), run=120002-120045msec | |
Disk stats (read/write): | |
rbd266: ios=69727/79136, merge=0/1, ticks=587936/790828, in_queue=1307271, util=33.49% |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment