Skip to content

Instantly share code, notes, and snippets.

@b333z

b333z/issue-1803.txt

Last active December 28, 2015 07:19
Show Gist options
  • Save b333z/7463871 to your computer and use it in GitHub Desktop.
Save b333z/7463871 to your computer and use it in GitHub Desktop.
Download ZIP
zfs issue-1803 test results
Raw
issue-1803.txt
Test started: 2013-11-14_13-21-20
pre_pkg_setup() {
EGIT_REPO_URI="file:///zfs/zfs"
EGIT_BRANCH="issue-1803"
}
pre_pkg_setup() {
EGIT_REPO_URI="https://github.com/zfsonlinux/spl.git"
EGIT_BRANCH="master"
}
Parallel dd's
128000+0 records in
128000+0 records out
524288000 bytes (524 MB) copied, 10.537 s, 49.8 MB/s
128000+0 records in
128000+0 records out
524288000 bytes (524 MB) copied, 8.70059 s, 60.3 MB/s
128000+0 records in
128000+0 records out
524288000 bytes (524 MB) copied, 12.3756 s, 42.4 MB/s
128000+0 records in
128000+0 records out
524288000 bytes (524 MB) copied, 13.231 s, 39.6 MB/s
fio tests:
read-rand-4k: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=1
...
read-rand-4k: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=1
read-rand-128k: (g=1): rw=randread, bs=128K-128K/128K-128K/128K-128K, ioengine=sync, iodepth=1
...
read-rand-128k: (g=1): rw=randread, bs=128K-128K/128K-128K/128K-128K, ioengine=sync, iodepth=1
read-seq-4k: (g=2): rw=read, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=1
...
read-seq-4k: (g=2): rw=read, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=1
read-seq-128k: (g=3): rw=read, bs=128K-128K/128K-128K/128K-128K, ioengine=sync, iodepth=1
...
read-seq-128k: (g=3): rw=read, bs=128K-128K/128K-128K/128K-128K, ioengine=sync, iodepth=1
fio-2.0.14
Starting 32 processes
read-rand-4k: (groupid=0, jobs=1): err= 0: pid=3701: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=3254.4KB/s, iops=813 , runt=157343msec
clat (usec): min=14 , max=272878 , avg=1224.21, stdev=7643.65
lat (usec): min=14 , max=272879 , avg=1224.62, stdev=7643.73
clat percentiles (usec):
| 1.00th=[ 15], 5.00th=[ 16], 10.00th=[ 17], 20.00th=[ 18],
| 30.00th=[ 18], 40.00th=[ 18], 50.00th=[ 18], 60.00th=[ 18],
| 70.00th=[ 21], 80.00th=[ 34], 90.00th=[ 41], 95.00th=[ 95],
| 99.00th=[36608], 99.50th=[57600], 99.90th=[97792], 99.95th=[115200],
| 99.99th=[171008]
bw (KB/s) : min= 62, max=179696, per=11.04%, avg=2873.53, stdev=18636.33
lat (usec) : 20=68.67%, 50=23.29%, 100=3.07%, 250=0.22%, 500=0.01%
lat (usec) : 1000=0.01%
lat (msec) : 2=0.01%, 4=0.02%, 10=1.31%, 20=1.46%, 50=1.30%
lat (msec) : 100=0.54%, 250=0.10%, 500=0.01%
cpu : usr=0.46%, sys=2.47%, ctx=6503, majf=0, minf=20
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-rand-4k: (groupid=0, jobs=1): err= 0: pid=3702: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=3363.7KB/s, iops=840 , runt=152216msec
clat (usec): min=14 , max=212388 , avg=1183.92, stdev=6833.44
lat (usec): min=14 , max=212389 , avg=1184.33, stdev=6833.51
clat percentiles (usec):
| 1.00th=[ 16], 5.00th=[ 17], 10.00th=[ 17], 20.00th=[ 18],
| 30.00th=[ 19], 40.00th=[ 19], 50.00th=[ 20], 60.00th=[ 21],
| 70.00th=[ 33], 80.00th=[ 36], 90.00th=[ 44], 95.00th=[ 4320],
| 99.00th=[34560], 99.50th=[49920], 99.90th=[86528], 99.95th=[100864],
| 99.99th=[129536]
bw (KB/s) : min= 80, max=165824, per=11.83%, avg=3078.48, stdev=18185.55
lat (usec) : 20=41.36%, 50=49.98%, 100=3.35%, 250=0.17%, 500=0.01%
lat (usec) : 750=0.01%, 1000=0.01%
lat (msec) : 2=0.02%, 4=0.03%, 10=1.51%, 20=1.56%, 50=1.49%
lat (msec) : 100=0.45%, 250=0.05%
cpu : usr=0.51%, sys=2.69%, ctx=7087, majf=0, minf=20
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-rand-4k: (groupid=0, jobs=1): err= 0: pid=3703: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=3761.3KB/s, iops=940 , runt=136124msec
clat (usec): min=14 , max=177798 , avg=1058.10, stdev=6514.90
lat (usec): min=14 , max=177799 , avg=1058.54, stdev=6514.96
clat percentiles (usec):
| 1.00th=[ 15], 5.00th=[ 16], 10.00th=[ 17], 20.00th=[ 17],
| 30.00th=[ 18], 40.00th=[ 18], 50.00th=[ 18], 60.00th=[ 22],
| 70.00th=[ 33], 80.00th=[ 34], 90.00th=[ 40], 95.00th=[ 71],
| 99.00th=[33024], 99.50th=[47872], 99.90th=[82432], 99.95th=[101888],
| 99.99th=[126464]
bw (KB/s) : min= 72, max=176952, per=12.87%, avg=3350.45, stdev=18294.16
lat (usec) : 20=56.56%, 50=35.23%, 100=3.60%, 250=0.20%, 500=0.02%
lat (usec) : 750=0.01%
lat (msec) : 2=0.01%, 4=0.02%, 10=1.24%, 20=1.35%, 50=1.31%
lat (msec) : 100=0.41%, 250=0.06%
cpu : usr=0.60%, sys=2.92%, ctx=6133, majf=0, minf=20
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-rand-4k: (groupid=0, jobs=1): err= 0: pid=3704: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=4169.2KB/s, iops=1042 , runt=122782msec
clat (usec): min=14 , max=203546 , avg=953.85, stdev=6226.84
lat (usec): min=14 , max=203547 , avg=954.29, stdev=6226.90
clat percentiles (usec):
| 1.00th=[ 15], 5.00th=[ 16], 10.00th=[ 17], 20.00th=[ 17],
| 30.00th=[ 18], 40.00th=[ 18], 50.00th=[ 18], 60.00th=[ 22],
| 70.00th=[ 34], 80.00th=[ 34], 90.00th=[ 40], 95.00th=[ 62],
| 99.00th=[30848], 99.50th=[46336], 99.90th=[80384], 99.95th=[91648],
| 99.99th=[130560]
bw (KB/s) : min= 79, max=160056, per=14.73%, avg=3835.01, stdev=19638.23
lat (usec) : 20=57.01%, 50=35.21%, 100=3.65%, 250=0.22%, 500=0.02%
lat (usec) : 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.02%, 10=1.08%, 20=1.16%, 50=1.18%
lat (msec) : 100=0.40%, 250=0.04%
cpu : usr=0.66%, sys=3.18%, ctx=5588, majf=0, minf=20
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-rand-4k: (groupid=0, jobs=1): err= 0: pid=3705: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=3324.7KB/s, iops=831 , runt=154000msec
clat (usec): min=13 , max=236860 , avg=1197.81, stdev=6922.40
lat (usec): min=14 , max=236861 , avg=1198.23, stdev=6922.46
clat percentiles (usec):
| 1.00th=[ 15], 5.00th=[ 16], 10.00th=[ 17], 20.00th=[ 17],
| 30.00th=[ 18], 40.00th=[ 18], 50.00th=[ 19], 60.00th=[ 20],
| 70.00th=[ 33], 80.00th=[ 35], 90.00th=[ 47], 95.00th=[ 4512],
| 99.00th=[35072], 99.50th=[51456], 99.90th=[86528], 99.95th=[101888],
| 99.99th=[138240]
bw (KB/s) : min= 86, max=169672, per=13.18%, avg=3431.29, stdev=19778.54
lat (usec) : 20=52.50%, 50=38.44%, 100=3.68%, 250=0.24%, 500=0.02%
lat (usec) : 750=0.01%, 1000=0.01%
lat (msec) : 2=0.02%, 4=0.03%, 10=1.43%, 20=1.66%, 50=1.43%
lat (msec) : 100=0.48%, 250=0.05%
cpu : usr=0.52%, sys=2.62%, ctx=7089, majf=0, minf=20
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-rand-4k: (groupid=0, jobs=1): err= 0: pid=3706: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=4675.6KB/s, iops=1168 , runt=109507msec
clat (usec): min=13 , max=186721 , avg=850.31, stdev=5866.37
lat (usec): min=14 , max=186722 , avg=850.75, stdev=5866.43
clat percentiles (usec):
| 1.00th=[ 15], 5.00th=[ 16], 10.00th=[ 17], 20.00th=[ 17],
| 30.00th=[ 18], 40.00th=[ 18], 50.00th=[ 18], 60.00th=[ 20],
| 70.00th=[ 33], 80.00th=[ 34], 90.00th=[ 37], 95.00th=[ 64],
| 99.00th=[27008], 99.50th=[44288], 99.90th=[78336], 99.95th=[94720],
| 99.99th=[125440]
bw (KB/s) : min= 103, max=168944, per=16.51%, avg=4296.96, stdev=21339.45
lat (usec) : 20=58.97%, 50=33.34%, 100=3.77%, 250=0.38%, 500=0.03%
lat (usec) : 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%, 10=0.99%, 20=1.11%, 50=0.97%
lat (msec) : 100=0.38%, 250=0.04%
cpu : usr=0.67%, sys=3.51%, ctx=5108, majf=0, minf=20
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-rand-4k: (groupid=0, jobs=1): err= 0: pid=3707: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=3404.6KB/s, iops=851 , runt=150409msec
clat (usec): min=14 , max=163583 , avg=1169.66, stdev=6704.21
lat (usec): min=14 , max=163584 , avg=1170.09, stdev=6704.28
clat percentiles (usec):
| 1.00th=[ 16], 5.00th=[ 17], 10.00th=[ 17], 20.00th=[ 18],
| 30.00th=[ 18], 40.00th=[ 19], 50.00th=[ 20], 60.00th=[ 21],
| 70.00th=[ 33], 80.00th=[ 35], 90.00th=[ 46], 95.00th=[ 3952],
| 99.00th=[34560], 99.50th=[49920], 99.90th=[85504], 99.95th=[96768],
| 99.99th=[125440]
bw (KB/s) : min= 84, max=171528, per=12.92%, avg=3363.17, stdev=19204.23
lat (usec) : 20=48.78%, 50=42.31%, 100=3.63%, 250=0.23%, 500=0.02%
lat (usec) : 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.02%, 10=1.46%, 20=1.54%, 50=1.50%
lat (msec) : 100=0.46%, 250=0.04%
cpu : usr=0.53%, sys=2.73%, ctx=6998, majf=0, minf=20
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-rand-4k: (groupid=0, jobs=1): err= 0: pid=3708: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=3619.7KB/s, iops=904 , runt=141452msec
clat (usec): min=13 , max=208839 , avg=1099.55, stdev=6611.14
lat (usec): min=13 , max=208840 , avg=1099.99, stdev=6611.21
clat percentiles (usec):
| 1.00th=[ 15], 5.00th=[ 16], 10.00th=[ 17], 20.00th=[ 17],
| 30.00th=[ 18], 40.00th=[ 18], 50.00th=[ 18], 60.00th=[ 32],
| 70.00th=[ 33], 80.00th=[ 34], 90.00th=[ 44], 95.00th=[ 80],
| 99.00th=[34560], 99.50th=[49408], 99.90th=[84480], 99.95th=[95744],
| 99.99th=[128512]
bw (KB/s) : min= 103, max=170856, per=13.91%, avg=3620.63, stdev=19472.73
lat (usec) : 20=54.23%, 50=37.33%, 100=3.60%, 250=0.24%, 500=0.03%
lat (usec) : 750=0.01%
lat (msec) : 2=0.01%, 4=0.02%, 10=1.28%, 20=1.38%, 50=1.38%
lat (msec) : 100=0.45%, 250=0.04%
cpu : usr=0.55%, sys=2.91%, ctx=6393, majf=0, minf=20
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-rand-128k: (groupid=1, jobs=1): err= 0: pid=3709: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=5283.9KB/s, iops=41 , runt= 96899msec
clat (usec): min=166 , max=169448 , avg=24193.88, stdev=22006.96
lat (usec): min=167 , max=169450 , avg=24196.13, stdev=22006.91
clat percentiles (usec):
| 1.00th=[ 406], 5.00th=[ 4704], 10.00th=[ 6240], 20.00th=[ 8384],
| 30.00th=[10432], 40.00th=[13248], 50.00th=[16320], 60.00th=[20608],
| 70.00th=[27008], 80.00th=[36608], 90.00th=[54016], 95.00th=[70144],
| 99.00th=[102912], 99.50th=[119296], 99.90th=[148480], 99.95th=[154624],
| 99.99th=[168960]
bw (KB/s) : min= 1665, max=12298, per=12.63%, avg=5289.34, stdev=1279.81
lat (usec) : 250=0.78%, 500=0.33%, 750=0.05%
lat (msec) : 2=1.20%, 4=1.43%, 10=23.83%, 20=30.62%, 50=29.20%
lat (msec) : 100=11.38%, 250=1.20%
cpu : usr=0.18%, sys=1.64%, ctx=4007, majf=0, minf=51
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
read-rand-128k: (groupid=1, jobs=1): err= 0: pid=3710: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=5349.8KB/s, iops=41 , runt= 95705msec
clat (usec): min=199 , max=175956 , avg=23895.31, stdev=21622.19
lat (usec): min=201 , max=175958 , avg=23897.55, stdev=21622.21
clat percentiles (usec):
| 1.00th=[ 310], 5.00th=[ 4768], 10.00th=[ 6304], 20.00th=[ 8640],
| 30.00th=[10560], 40.00th=[13376], 50.00th=[16192], 60.00th=[20352],
| 70.00th=[26752], 80.00th=[36096], 90.00th=[53504], 95.00th=[69120],
| 99.00th=[101888], 99.50th=[116224], 99.90th=[144384], 99.95th=[158720],
| 99.99th=[175104]
bw (KB/s) : min= 3121, max= 8836, per=12.78%, avg=5354.30, stdev=1052.42
lat (usec) : 250=0.90%, 500=0.48%, 750=0.12%
lat (msec) : 2=1.10%, 4=1.05%, 10=23.48%, 20=31.85%, 50=29.78%
lat (msec) : 100=10.10%, 250=1.15%
cpu : usr=0.20%, sys=1.63%, ctx=4000, majf=0, minf=54
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
read-rand-128k: (groupid=1, jobs=1): err= 0: pid=3711: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=5452.7KB/s, iops=42 , runt= 93899msec
clat (usec): min=165 , max=178202 , avg=23443.93, stdev=21328.96
lat (usec): min=166 , max=178204 , avg=23446.11, stdev=21328.96
clat percentiles (usec):
| 1.00th=[ 241], 5.00th=[ 4704], 10.00th=[ 6240], 20.00th=[ 8384],
| 30.00th=[10304], 40.00th=[12992], 50.00th=[16064], 60.00th=[20352],
| 70.00th=[26240], 80.00th=[35584], 90.00th=[51968], 95.00th=[68096],
| 99.00th=[105984], 99.50th=[113152], 99.90th=[162816], 99.95th=[168960],
| 99.99th=[179200]
bw (KB/s) : min= 2577, max= 8015, per=13.05%, avg=5465.69, stdev=1037.36
lat (usec) : 250=1.03%, 500=0.70%, 750=0.10%, 1000=0.10%
lat (msec) : 2=0.75%, 4=1.23%, 10=24.80%, 20=30.68%, 50=29.73%
lat (msec) : 100=9.68%, 250=1.23%
cpu : usr=0.17%, sys=1.70%, ctx=3984, majf=0, minf=52
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
read-rand-128k: (groupid=1, jobs=1): err= 0: pid=3712: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=5235.1KB/s, iops=40 , runt= 97786msec
clat (usec): min=164 , max=262570 , avg=24415.71, stdev=23083.09
lat (usec): min=165 , max=262573 , avg=24417.84, stdev=23083.10
clat percentiles (usec):
| 1.00th=[ 237], 5.00th=[ 4320], 10.00th=[ 5920], 20.00th=[ 8160],
| 30.00th=[10304], 40.00th=[12864], 50.00th=[16320], 60.00th=[20864],
| 70.00th=[27264], 80.00th=[36608], 90.00th=[55040], 95.00th=[71168],
| 99.00th=[103936], 99.50th=[119296], 99.90th=[164864], 99.95th=[177152],
| 99.99th=[264192]
bw (KB/s) : min= 2475, max=21375, per=12.41%, avg=5196.57, stdev=1693.13
lat (usec) : 250=1.05%, 500=0.35%, 750=0.10%, 1000=0.05%
lat (msec) : 2=1.80%, 4=1.38%, 10=24.03%, 20=29.18%, 50=29.65%
lat (msec) : 100=10.93%, 250=1.48%, 500=0.03%
cpu : usr=0.15%, sys=1.65%, ctx=3998, majf=0, minf=54
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
read-rand-128k: (groupid=1, jobs=1): err= 0: pid=3713: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=5361.2KB/s, iops=41 , runt= 95502msec
clat (usec): min=166 , max=242096 , avg=23844.09, stdev=22191.49
lat (usec): min=167 , max=242097 , avg=23846.25, stdev=22191.45
clat percentiles (usec):
| 1.00th=[ 231], 5.00th=[ 4896], 10.00th=[ 6432], 20.00th=[ 8512],
| 30.00th=[10432], 40.00th=[12992], 50.00th=[16064], 60.00th=[20352],
| 70.00th=[26752], 80.00th=[35584], 90.00th=[52480], 95.00th=[68096],
| 99.00th=[110080], 99.50th=[122368], 99.90th=[181248], 99.95th=[183296],
| 99.99th=[242688]
bw (KB/s) : min= 2555, max= 8170, per=12.83%, avg=5374.48, stdev=1060.82
lat (usec) : 250=1.23%, 500=0.58%, 750=0.08%, 1000=0.03%
lat (msec) : 2=0.80%, 4=1.20%, 10=24.18%, 20=30.98%, 50=29.58%
lat (msec) : 100=9.98%, 250=1.40%
cpu : usr=0.20%, sys=1.64%, ctx=3982, majf=0, minf=54
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
read-rand-128k: (groupid=1, jobs=1): err= 0: pid=3714: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=5286.4KB/s, iops=41 , runt= 96853msec
clat (usec): min=165 , max=199601 , avg=24182.18, stdev=21619.34
lat (usec): min=165 , max=199603 , avg=24184.36, stdev=21619.36
clat percentiles (usec):
| 1.00th=[ 426], 5.00th=[ 5024], 10.00th=[ 6624], 20.00th=[ 8640],
| 30.00th=[10688], 40.00th=[13376], 50.00th=[16512], 60.00th=[20864],
| 70.00th=[27520], 80.00th=[37120], 90.00th=[52992], 95.00th=[68096],
| 99.00th=[103936], 99.50th=[115200], 99.90th=[160768], 99.95th=[168960],
| 99.99th=[199680]
bw (KB/s) : min= 2509, max=12047, per=12.63%, avg=5290.01, stdev=1258.37
lat (usec) : 250=0.75%, 500=0.35%, 750=0.05%
lat (msec) : 2=1.10%, 4=1.45%, 10=23.30%, 20=30.55%, 50=31.23%
lat (msec) : 100=10.03%, 250=1.20%
cpu : usr=0.18%, sys=1.64%, ctx=4007, majf=0, minf=54
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
read-rand-128k: (groupid=1, jobs=1): err= 0: pid=3715: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=5320.5KB/s, iops=41 , runt= 96232msec
clat (usec): min=155 , max=170950 , avg=24027.55, stdev=21887.60
lat (usec): min=155 , max=170953 , avg=24029.76, stdev=21887.60
clat percentiles (usec):
| 1.00th=[ 386], 5.00th=[ 4768], 10.00th=[ 6240], 20.00th=[ 8384],
| 30.00th=[10176], 40.00th=[13120], 50.00th=[16512], 60.00th=[20864],
| 70.00th=[26752], 80.00th=[36096], 90.00th=[53504], 95.00th=[69120],
| 99.00th=[102912], 99.50th=[118272], 99.90th=[168960], 99.95th=[168960],
| 99.99th=[171008]
bw (KB/s) : min= 2043, max= 8942, per=12.71%, avg=5325.35, stdev=1195.24
lat (usec) : 250=0.78%, 500=0.38%, 750=0.08%
lat (msec) : 2=1.03%, 4=1.35%, 10=25.33%, 20=29.25%, 50=29.53%
lat (msec) : 100=11.12%, 250=1.18%
cpu : usr=0.16%, sys=1.67%, ctx=4013, majf=0, minf=55
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
read-rand-128k: (groupid=1, jobs=1): err= 0: pid=3716: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=5235.4KB/s, iops=40 , runt= 97796msec
clat (usec): min=188 , max=205812 , avg=24417.95, stdev=23886.87
lat (usec): min=190 , max=205815 , avg=24420.11, stdev=23886.86
clat percentiles (usec):
| 1.00th=[ 243], 5.00th=[ 2320], 10.00th=[ 5408], 20.00th=[ 7712],
| 30.00th=[ 9664], 40.00th=[12352], 50.00th=[15680], 60.00th=[20608],
| 70.00th=[27776], 80.00th=[37632], 90.00th=[55040], 95.00th=[73216],
| 99.00th=[112128], 99.50th=[128512], 99.90th=[185344], 99.95th=[193536],
| 99.99th=[205824]
bw (KB/s) : min= 2808, max=22528, per=12.41%, avg=5196.43, stdev=1936.96
lat (usec) : 250=1.03%, 500=0.50%, 750=0.12%, 1000=0.03%
lat (msec) : 2=2.53%, 4=1.58%, 10=25.88%, 20=27.33%, 50=27.83%
lat (msec) : 100=11.60%, 250=1.60%
cpu : usr=0.18%, sys=1.63%, ctx=3987, majf=0, minf=55
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
read-seq-4k: (groupid=2, jobs=1): err= 0: pid=3717: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=20430KB/s, iops=5107 , runt= 25061msec
clat (usec): min=12 , max=1703.7K, avg=192.62, stdev=13031.26
lat (usec): min=12 , max=1703.7K, avg=192.96, stdev=13031.26
clat percentiles (usec):
| 1.00th=[ 13], 5.00th=[ 14], 10.00th=[ 14], 20.00th=[ 15],
| 30.00th=[ 15], 40.00th=[ 16], 50.00th=[ 16], 60.00th=[ 17],
| 70.00th=[ 26], 80.00th=[ 27], 90.00th=[ 29], 95.00th=[ 31],
| 99.00th=[ 69], 99.50th=[ 342], 99.90th=[ 1640], 99.95th=[ 5088],
| 99.99th=[733184]
bw (KB/s) : min= 170, max=51192, per=15.99%, avg=22613.52, stdev=13145.77
lat (usec) : 20=67.05%, 50=31.22%, 100=1.01%, 250=0.12%, 500=0.29%
lat (usec) : 750=0.09%, 1000=0.04%
lat (msec) : 2=0.10%, 4=0.02%, 10=0.01%, 20=0.01%, 50=0.01%
lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2000=0.01%
cpu : usr=2.08%, sys=10.29%, ctx=1095, majf=0, minf=25
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-seq-4k: (groupid=2, jobs=1): err= 0: pid=3718: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=18415KB/s, iops=4603 , runt= 27804msec
clat (usec): min=12 , max=2730.1K, avg=214.43, stdev=15584.25
lat (usec): min=12 , max=2730.1K, avg=214.74, stdev=15584.25
clat percentiles (usec):
| 1.00th=[ 13], 5.00th=[ 14], 10.00th=[ 14], 20.00th=[ 14],
| 30.00th=[ 15], 40.00th=[ 15], 50.00th=[ 16], 60.00th=[ 16],
| 70.00th=[ 17], 80.00th=[ 23], 90.00th=[ 28], 95.00th=[ 30],
| 99.00th=[ 48], 99.50th=[ 67], 99.90th=[ 1208], 99.95th=[11968],
| 99.99th=[659456]
bw (KB/s) : min= 398, max=166640, per=17.96%, avg=25399.60, stdev=36460.35
lat (usec) : 20=78.76%, 50=20.31%, 100=0.61%, 250=0.07%, 500=0.07%
lat (usec) : 750=0.03%, 1000=0.02%
lat (msec) : 2=0.05%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2000=0.01%, >=2000=0.01%
cpu : usr=1.54%, sys=8.58%, ctx=623, majf=0, minf=25
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-seq-4k: (groupid=2, jobs=1): err= 0: pid=3719: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=25078KB/s, iops=6269 , runt= 20416msec
clat (usec): min=12 , max=1419.1K, avg=156.46, stdev=10386.71
lat (usec): min=12 , max=1419.1K, avg=156.79, stdev=10386.71
clat percentiles (usec):
| 1.00th=[ 13], 5.00th=[ 13], 10.00th=[ 14], 20.00th=[ 14],
| 30.00th=[ 14], 40.00th=[ 15], 50.00th=[ 16], 60.00th=[ 17],
| 70.00th=[ 19], 80.00th=[ 27], 90.00th=[ 28], 95.00th=[ 31],
| 99.00th=[ 68], 99.50th=[ 358], 99.90th=[ 1768], 99.95th=[ 7072],
| 99.99th=[522240]
bw (KB/s) : min= 2689, max=101128, per=20.60%, avg=29135.09, stdev=22197.91
lat (usec) : 20=70.99%, 50=27.35%, 100=0.91%, 250=0.11%, 500=0.30%
lat (usec) : 750=0.11%, 1000=0.03%
lat (msec) : 2=0.10%, 4=0.02%, 10=0.03%, 20=0.01%, 50=0.01%
lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2000=0.01%
cpu : usr=2.37%, sys=12.21%, ctx=1134, majf=0, minf=25
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-seq-4k: (groupid=2, jobs=1): err= 0: pid=3720: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=24443KB/s, iops=6110 , runt= 20947msec
clat (usec): min=12 , max=2405.7K, avg=160.56, stdev=13668.52
lat (usec): min=12 , max=2405.7K, avg=160.91, stdev=13668.52
clat percentiles (usec):
| 1.00th=[ 13], 5.00th=[ 13], 10.00th=[ 14], 20.00th=[ 14],
| 30.00th=[ 14], 40.00th=[ 15], 50.00th=[ 16], 60.00th=[ 16],
| 70.00th=[ 18], 80.00th=[ 27], 90.00th=[ 28], 95.00th=[ 31],
| 99.00th=[ 66], 99.50th=[ 338], 99.90th=[ 1704], 99.95th=[ 3920],
| 99.99th=[444416]
bw (KB/s) : min= 2556, max=73448, per=21.83%, avg=30873.28, stdev=18724.99
lat (usec) : 20=72.15%, 50=26.26%, 100=0.92%, 250=0.11%, 500=0.25%
lat (usec) : 750=0.12%, 1000=0.03%
lat (msec) : 2=0.09%, 4=0.03%, 10=0.02%, 20=0.01%, 50=0.01%
lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2000=0.01%, >=2000=0.01%
cpu : usr=2.60%, sys=11.46%, ctx=1041, majf=0, minf=25
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-seq-4k: (groupid=2, jobs=1): err= 0: pid=3721: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=20378KB/s, iops=5094 , runt= 25125msec
clat (usec): min=12 , max=2848.7K, avg=193.25, stdev=13592.33
lat (usec): min=12 , max=2848.7K, avg=193.62, stdev=13592.34
clat percentiles (usec):
| 1.00th=[ 13], 5.00th=[ 14], 10.00th=[ 14], 20.00th=[ 14],
| 30.00th=[ 15], 40.00th=[ 15], 50.00th=[ 16], 60.00th=[ 17],
| 70.00th=[ 18], 80.00th=[ 26], 90.00th=[ 28], 95.00th=[ 30],
| 99.00th=[ 63], 99.50th=[ 322], 99.90th=[ 2192], 99.95th=[16192],
| 99.99th=[460800]
bw (KB/s) : min= 131, max=86656, per=19.74%, avg=27921.50, stdev=24907.04
lat (usec) : 20=73.56%, 50=25.01%, 100=0.79%, 250=0.09%, 500=0.21%
lat (usec) : 750=0.09%, 1000=0.05%
lat (msec) : 2=0.10%, 4=0.02%, 10=0.02%, 20=0.01%, 50=0.01%
lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2000=0.01%, >=2000=0.01%
cpu : usr=1.87%, sys=9.78%, ctx=1072, majf=0, minf=24
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-seq-4k: (groupid=2, jobs=1): err= 0: pid=3722: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=20799KB/s, iops=5199 , runt= 24616msec
clat (usec): min=12 , max=2197.9K, avg=189.43, stdev=13924.05
lat (usec): min=12 , max=2197.9K, avg=189.73, stdev=13924.05
clat percentiles (usec):
| 1.00th=[ 13], 5.00th=[ 14], 10.00th=[ 14], 20.00th=[ 14],
| 30.00th=[ 15], 40.00th=[ 15], 50.00th=[ 16], 60.00th=[ 16],
| 70.00th=[ 17], 80.00th=[ 20], 90.00th=[ 28], 95.00th=[ 30],
| 99.00th=[ 51], 99.50th=[ 71], 99.90th=[ 1096], 99.95th=[ 7136],
| 99.99th=[509952]
bw (KB/s) : min= 258, max=82936, per=18.43%, avg=26066.04, stdev=25810.95
lat (usec) : 20=79.65%, 50=19.29%, 100=0.69%, 250=0.08%, 500=0.12%
lat (usec) : 750=0.03%, 1000=0.03%
lat (msec) : 2=0.05%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2000=0.01%, >=2000=0.01%
cpu : usr=1.92%, sys=9.41%, ctx=688, majf=0, minf=24
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-seq-4k: (groupid=2, jobs=1): err= 0: pid=3723: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=20815KB/s, iops=5203 , runt= 24598msec
clat (usec): min=12 , max=1977.1K, avg=189.03, stdev=12091.94
lat (usec): min=12 , max=1977.1K, avg=189.45, stdev=12091.99
clat percentiles (usec):
| 1.00th=[ 13], 5.00th=[ 14], 10.00th=[ 14], 20.00th=[ 14],
| 30.00th=[ 14], 40.00th=[ 15], 50.00th=[ 15], 60.00th=[ 16],
| 70.00th=[ 19], 80.00th=[ 27], 90.00th=[ 29], 95.00th=[ 32],
| 99.00th=[ 63], 99.50th=[ 89], 99.90th=[ 1432], 99.95th=[11328],
| 99.99th=[700416]
bw (KB/s) : min= 219, max=51120, per=17.16%, avg=24264.83, stdev=16904.95
lat (usec) : 20=70.25%, 50=28.16%, 100=1.14%, 250=0.11%, 500=0.12%
lat (usec) : 750=0.05%, 1000=0.02%
lat (msec) : 2=0.07%, 4=0.01%, 10=0.02%, 20=0.01%, 50=0.01%
lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2000=0.01%
cpu : usr=1.94%, sys=10.25%, ctx=751, majf=0, minf=24
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-seq-4k: (groupid=2, jobs=1): err= 0: pid=3724: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=17677KB/s, iops=4419 , runt= 28965msec
clat (usec): min=12 , max=2673.6K, avg=223.45, stdev=18475.17
lat (usec): min=12 , max=2673.6K, avg=223.76, stdev=18475.17
clat percentiles (usec):
| 1.00th=[ 13], 5.00th=[ 14], 10.00th=[ 14], 20.00th=[ 14],
| 30.00th=[ 14], 40.00th=[ 15], 50.00th=[ 15], 60.00th=[ 16],
| 70.00th=[ 17], 80.00th=[ 26], 90.00th=[ 28], 95.00th=[ 30],
| 99.00th=[ 54], 99.50th=[ 90], 99.90th=[ 1672], 99.95th=[ 7456],
| 99.99th=[509952]
bw (KB/s) : min= 172, max=144552, per=23.96%, avg=33879.77, stdev=39715.31
lat (usec) : 20=77.06%, 50=21.79%, 100=0.68%, 250=0.09%, 500=0.18%
lat (usec) : 750=0.06%, 1000=0.02%
lat (msec) : 2=0.06%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2000=0.01%, >=2000=0.01%
cpu : usr=1.54%, sys=8.25%, ctx=817, majf=0, minf=24
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-seq-128k: (groupid=3, jobs=1): err= 0: pid=3725: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=23094KB/s, iops=180 , runt= 22170msec
clat (usec): min=82 , max=2883.1K, avg=5530.60, stdev=93498.40
lat (usec): min=82 , max=2883.1K, avg=5531.76, stdev=93498.41
clat percentiles (usec):
| 1.00th=[ 87], 5.00th=[ 94], 10.00th=[ 96], 20.00th=[ 115],
| 30.00th=[ 159], 40.00th=[ 165], 50.00th=[ 181], 60.00th=[ 258],
| 70.00th=[ 940], 80.00th=[ 1032], 90.00th=[ 1272], 95.00th=[ 2160],
| 99.00th=[15552], 99.50th=[100864], 99.90th=[2277376], 99.95th=[2605056],
| 99.99th=[2899968]
bw (KB/s) : min= 2957, max=68649, per=21.07%, avg=30624.11, stdev=21546.85
lat (usec) : 100=14.33%, 250=44.98%, 500=4.05%, 750=0.73%, 1000=11.48%
lat (msec) : 2=18.45%, 4=4.45%, 10=0.40%, 20=0.18%, 50=0.35%
lat (msec) : 100=0.10%, 250=0.15%, 500=0.12%, 750=0.05%, 1000=0.05%
lat (msec) : 2000=0.05%, >=2000=0.10%
cpu : usr=0.32%, sys=4.12%, ctx=1556, majf=0, minf=58
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
read-seq-128k: (groupid=3, jobs=1): err= 0: pid=3726: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=22796KB/s, iops=178 , runt= 22460msec
clat (usec): min=83 , max=2945.4K, avg=5606.04, stdev=93890.34
lat (usec): min=83 , max=2945.4K, avg=5606.95, stdev=93890.34
clat percentiles (usec):
| 1.00th=[ 86], 5.00th=[ 92], 10.00th=[ 94], 20.00th=[ 145],
| 30.00th=[ 157], 40.00th=[ 159], 50.00th=[ 165], 60.00th=[ 175],
| 70.00th=[ 217], 80.00th=[ 836], 90.00th=[ 1160], 95.00th=[ 1896],
| 99.00th=[29568], 99.50th=[86528], 99.90th=[2007040], 99.95th=[2473984],
| 99.99th=[2932736]
bw (KB/s) : min= 225, max=100967, per=20.85%, avg=30310.35, stdev=32329.31
lat (usec) : 100=15.70%, 250=58.25%, 500=4.17%, 750=0.90%, 1000=6.15%
lat (msec) : 2=10.12%, 4=2.93%, 10=0.35%, 20=0.23%, 50=0.48%
lat (msec) : 100=0.28%, 250=0.05%, 500=0.12%, 750=0.05%, 1000=0.08%
lat (msec) : 2000=0.05%, >=2000=0.10%
cpu : usr=0.21%, sys=3.72%, ctx=987, majf=0, minf=55
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
read-seq-128k: (groupid=3, jobs=1): err= 0: pid=3727: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=23016KB/s, iops=179 , runt= 22245msec
clat (usec): min=91 , max=2662.6K, avg=5550.39, stdev=89067.09
lat (usec): min=92 , max=2662.6K, avg=5551.50, stdev=89067.11
clat percentiles (usec):
| 1.00th=[ 93], 5.00th=[ 96], 10.00th=[ 107], 20.00th=[ 157],
| 30.00th=[ 159], 40.00th=[ 165], 50.00th=[ 175], 60.00th=[ 213],
| 70.00th=[ 486], 80.00th=[ 1004], 90.00th=[ 1336], 95.00th=[ 2160],
| 99.00th=[24448], 99.50th=[107008], 99.90th=[2007040], 99.95th=[2113536],
| 99.99th=[2670592]
bw (KB/s) : min= 115, max=130299, per=28.95%, avg=42084.74, stdev=47107.81
lat (usec) : 100=7.30%, 250=58.10%, 500=4.75%, 750=1.07%, 1000=8.12%
lat (msec) : 2=14.98%, 4=3.85%, 10=0.55%, 20=0.18%, 50=0.38%
lat (msec) : 100=0.15%, 250=0.25%, 500=0.08%, 1000=0.05%, 2000=0.10%
lat (msec) : >=2000=0.10%
cpu : usr=0.22%, sys=4.17%, ctx=1284, majf=0, minf=58
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
read-seq-128k: (groupid=3, jobs=1): err= 0: pid=3728: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=22202KB/s, iops=173 , runt= 23061msec
clat (usec): min=81 , max=3267.5K, avg=5757.70, stdev=96466.91
lat (usec): min=82 , max=3267.5K, avg=5758.54, stdev=96466.92
clat percentiles (usec):
| 1.00th=[ 88], 5.00th=[ 94], 10.00th=[ 96], 20.00th=[ 101],
| 30.00th=[ 133], 40.00th=[ 159], 50.00th=[ 163], 60.00th=[ 169],
| 70.00th=[ 183], 80.00th=[ 223], 90.00th=[ 972], 95.00th=[ 1480],
| 99.00th=[18816], 99.50th=[195584], 99.90th=[2179072], 99.95th=[2342912],
| 99.99th=[3260416]
bw (KB/s) : min= 823, max=59904, per=16.50%, avg=23982.18, stdev=18782.38
lat (usec) : 100=17.75%, 250=64.73%, 500=4.80%, 750=0.35%, 1000=3.18%
lat (msec) : 2=6.08%, 4=1.85%, 10=0.12%, 20=0.15%, 50=0.20%
lat (msec) : 100=0.10%, 250=0.28%, 500=0.15%, 750=0.08%, 1000=0.03%
lat (msec) : 2000=0.05%, >=2000=0.12%
cpu : usr=0.25%, sys=3.24%, ctx=596, majf=0, minf=59
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
read-seq-128k: (groupid=3, jobs=1): err= 0: pid=3729: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=19081KB/s, iops=149 , runt= 26833msec
clat (usec): min=89 , max=2687.6K, avg=6698.37, stdev=90128.81
lat (usec): min=89 , max=2687.6K, avg=6699.33, stdev=90128.83
clat percentiles (usec):
| 1.00th=[ 92], 5.00th=[ 95], 10.00th=[ 106], 20.00th=[ 157],
| 30.00th=[ 159], 40.00th=[ 163], 50.00th=[ 169], 60.00th=[ 201],
| 70.00th=[ 233], 80.00th=[ 916], 90.00th=[ 1512], 95.00th=[ 2128],
| 99.00th=[29312], 99.50th=[276480], 99.90th=[1875968], 99.95th=[1908736],
| 99.99th=[2703360]
bw (KB/s) : min= 132, max=61593, per=13.45%, avg=19547.86, stdev=17205.95
lat (usec) : 100=8.45%, 250=63.00%, 500=5.98%, 750=0.98%, 1000=3.38%
lat (msec) : 2=12.83%, 4=2.93%, 10=0.98%, 20=0.33%, 50=0.33%
lat (msec) : 100=0.10%, 250=0.20%, 500=0.08%, 750=0.18%, 1000=0.10%
lat (msec) : 2000=0.15%, >=2000=0.05%
cpu : usr=0.23%, sys=3.26%, ctx=1013, majf=0, minf=58
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
read-seq-128k: (groupid=3, jobs=1): err= 0: pid=3730: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=18717KB/s, iops=146 , runt= 27355msec
clat (usec): min=80 , max=2424.6K, avg=6830.61, stdev=90708.60
lat (usec): min=81 , max=2424.6K, avg=6831.43, stdev=90708.62
clat percentiles (usec):
| 1.00th=[ 85], 5.00th=[ 92], 10.00th=[ 94], 20.00th=[ 153],
| 30.00th=[ 157], 40.00th=[ 159], 50.00th=[ 165], 60.00th=[ 171],
| 70.00th=[ 201], 80.00th=[ 239], 90.00th=[ 1032], 95.00th=[ 1656],
| 99.00th=[34048], 99.50th=[358400], 99.90th=[1613824], 99.95th=[1646592],
| 99.99th=[2408448]
bw (KB/s) : min= 226, max=120320, per=15.16%, avg=22040.52, stdev=28987.75
lat (usec) : 100=13.78%, 250=67.38%, 500=6.85%, 750=0.48%, 1000=1.28%
lat (msec) : 2=6.53%, 4=1.33%, 10=0.53%, 20=0.30%, 50=0.80%
lat (msec) : 100=0.12%, 250=0.08%, 500=0.08%, 750=0.15%, 1000=0.10%
lat (msec) : 2000=0.20%, >=2000=0.05%
cpu : usr=0.14%, sys=3.06%, ctx=565, majf=0, minf=57
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
read-seq-128k: (groupid=3, jobs=1): err= 0: pid=3731: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=18568KB/s, iops=145 , runt= 27575msec
clat (usec): min=89 , max=2629.5K, avg=6884.32, stdev=93455.30
lat (usec): min=89 , max=2629.5K, avg=6885.33, stdev=93455.31
clat percentiles (usec):
| 1.00th=[ 93], 5.00th=[ 153], 10.00th=[ 157], 20.00th=[ 159],
| 30.00th=[ 161], 40.00th=[ 165], 50.00th=[ 169], 60.00th=[ 187],
| 70.00th=[ 217], 80.00th=[ 278], 90.00th=[ 1192], 95.00th=[ 1912],
| 99.00th=[43264], 99.50th=[374784], 99.90th=[1744896], 99.95th=[2572288],
| 99.99th=[2637824]
bw (KB/s) : min= 166, max=64000, per=16.84%, avg=24473.74, stdev=20765.36
lat (usec) : 100=3.15%, 250=73.23%, 500=6.98%, 750=0.48%, 1000=2.70%
lat (msec) : 2=8.68%, 4=1.88%, 10=0.45%, 20=0.25%, 50=1.30%
lat (msec) : 100=0.18%, 250=0.15%, 500=0.23%, 750=0.10%, 1000=0.12%
lat (msec) : 2000=0.08%, >=2000=0.08%
cpu : usr=0.15%, sys=3.29%, ctx=753, majf=0, minf=57
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
read-seq-128k: (groupid=3, jobs=1): err= 0: pid=3732: Thu Nov 14 13:27:43 2013
read : io=512000KB, bw=18170KB/s, iops=141 , runt= 28179msec
clat (usec): min=83 , max=2233.1K, avg=7036.95, stdev=87799.51
lat (usec): min=83 , max=2233.1K, avg=7037.77, stdev=87799.52
clat percentiles (usec):
| 1.00th=[ 84], 5.00th=[ 91], 10.00th=[ 95], 20.00th=[ 137],
| 30.00th=[ 155], 40.00th=[ 157], 50.00th=[ 163], 60.00th=[ 169],
| 70.00th=[ 199], 80.00th=[ 237], 90.00th=[ 964], 95.00th=[ 1576],
| 99.00th=[54016], 99.50th=[464896], 99.90th=[1581056], 99.95th=[1646592],
| 99.99th=[2244608]
bw (KB/s) : min= 161, max=121223, per=14.23%, avg=20684.81, stdev=28478.96
lat (usec) : 100=13.75%, 250=67.40%, 500=5.98%, 750=0.50%, 1000=3.18%
lat (msec) : 2=5.75%, 4=1.60%, 10=0.23%, 20=0.18%, 50=0.38%
lat (msec) : 100=0.20%, 250=0.30%, 500=0.10%, 750=0.15%, 1000=0.05%
lat (msec) : 2000=0.23%, >=2000=0.05%
cpu : usr=0.15%, sys=2.91%, ctx=693, majf=0, minf=58
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4000.0MB, aggrb=26032KB/s, minb=3254KB/s, maxb=4675KB/s, mint=109507msec, maxt=157343msec
Run status group 1 (all jobs):
READ: io=4000.0MB, aggrb=41883KB/s, minb=5235KB/s, maxb=5452KB/s, mint=93899msec, maxt=97796msec
Run status group 2 (all jobs):
READ: io=4000.0MB, aggrb=141412KB/s, minb=17676KB/s, maxb=25078KB/s, mint=20416msec, maxt=28965msec
Run status group 3 (all jobs):
READ: io=4000.0MB, aggrb=145356KB/s, minb=18169KB/s, maxb=23094KB/s, mint=22170msec, maxt=28179msec
Raw
master.txt
Test started: 2013-11-14_19-17-01
pre_pkg_setup() {
EGIT_REPO_URI="file:///zfs/zfs"
EGIT_BRANCH="master"
}
pre_pkg_setup() {
EGIT_REPO_URI="https://github.com/zfsonlinux/spl.git"
EGIT_BRANCH="master"
}
Parallel dd's
128000+0 records in
128000+0 records out
524288000 bytes (524 MB) copied, 10.7909 s, 48.6 MB/s
128000+0 records in
128000+0 records out
524288000 bytes (524 MB) copied, 10.8855 s, 48.2 MB/s
128000+0 records in
128000+0 records out
524288000 bytes (524 MB) copied, 12.503 s, 41.9 MB/s
128000+0 records in
128000+0 records out
524288000 bytes (524 MB) copied, 13.5288 s, 38.8 MB/s
fio tests:
read-rand-4k: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=1
...
read-rand-4k: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=1
read-rand-128k: (g=1): rw=randread, bs=128K-128K/128K-128K/128K-128K, ioengine=sync, iodepth=1
...
read-rand-128k: (g=1): rw=randread, bs=128K-128K/128K-128K/128K-128K, ioengine=sync, iodepth=1
read-seq-4k: (g=2): rw=read, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=1
...
read-seq-4k: (g=2): rw=read, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=1
read-seq-128k: (g=3): rw=read, bs=128K-128K/128K-128K/128K-128K, ioengine=sync, iodepth=1
...
read-seq-128k: (g=3): rw=read, bs=128K-128K/128K-128K/128K-128K, ioengine=sync, iodepth=1
fio-2.0.14
Starting 32 processes
read-rand-4k: (groupid=0, jobs=1): err= 0: pid=3620: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=3193.4KB/s, iops=798 , runt=160334msec
clat (usec): min=14 , max=259302 , avg=1247.60, stdev=7399.88
lat (usec): min=14 , max=259304 , avg=1247.96, stdev=7399.96
clat percentiles (usec):
| 1.00th=[ 16], 5.00th=[ 16], 10.00th=[ 17], 20.00th=[ 18],
| 30.00th=[ 18], 40.00th=[ 18], 50.00th=[ 19], 60.00th=[ 19],
| 70.00th=[ 21], 80.00th=[ 36], 90.00th=[ 43], 95.00th=[ 1912],
| 99.00th=[36096], 99.50th=[54016], 99.90th=[94720], 99.95th=[112128],
| 99.99th=[148480]
bw (KB/s) : min= 85, max=175048, per=12.90%, avg=3295.08, stdev=20365.35
lat (usec) : 20=62.75%, 50=29.08%, 100=3.05%, 250=0.11%, 500=0.01%
lat (usec) : 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.02%, 10=1.36%, 20=1.59%, 50=1.44%
lat (msec) : 100=0.51%, 250=0.08%, 500=0.01%
cpu : usr=0.48%, sys=2.49%, ctx=6824, majf=0, minf=20
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-rand-4k: (groupid=0, jobs=1): err= 0: pid=3621: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=3279.1KB/s, iops=819 , runt=156100msec
clat (usec): min=14 , max=230872 , avg=1214.14, stdev=7026.30
lat (usec): min=14 , max=230873 , avg=1214.52, stdev=7026.37
clat percentiles (usec):
| 1.00th=[ 16], 5.00th=[ 17], 10.00th=[ 17], 20.00th=[ 18],
| 30.00th=[ 18], 40.00th=[ 19], 50.00th=[ 20], 60.00th=[ 22],
| 70.00th=[ 34], 80.00th=[ 35], 90.00th=[ 46], 95.00th=[ 4448],
| 99.00th=[35584], 99.50th=[51968], 99.90th=[86528], 99.95th=[102912],
| 99.99th=[140288]
bw (KB/s) : min= 63, max=167040, per=13.23%, avg=3379.35, stdev=19294.29
lat (usec) : 20=47.99%, 50=43.15%, 100=3.50%, 250=0.23%, 500=0.02%
lat (usec) : 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.02%, 10=1.44%, 20=1.61%, 50=1.47%
lat (msec) : 100=0.49%, 250=0.06%
cpu : usr=0.52%, sys=2.73%, ctx=7065, majf=0, minf=20
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-rand-4k: (groupid=0, jobs=1): err= 0: pid=3622: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=3500.2KB/s, iops=875 , runt=146285msec
clat (usec): min=14 , max=169372 , avg=1137.32, stdev=6781.69
lat (usec): min=14 , max=169372 , avg=1137.71, stdev=6781.76
clat percentiles (usec):
| 1.00th=[ 16], 5.00th=[ 17], 10.00th=[ 17], 20.00th=[ 18],
| 30.00th=[ 18], 40.00th=[ 18], 50.00th=[ 19], 60.00th=[ 33],
| 70.00th=[ 34], 80.00th=[ 35], 90.00th=[ 41], 95.00th=[ 87],
| 99.00th=[35072], 99.50th=[50432], 99.90th=[86528], 99.95th=[101888],
| 99.99th=[138240]
bw (KB/s) : min= 58, max=167976, per=13.06%, avg=3337.55, stdev=18084.57
lat (usec) : 20=53.58%, 50=38.09%, 100=3.41%, 250=0.18%, 500=0.03%
lat (usec) : 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.02%, 10=1.31%, 20=1.45%, 50=1.41%
lat (msec) : 100=0.46%, 250=0.05%
cpu : usr=0.58%, sys=2.89%, ctx=6593, majf=0, minf=20
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-rand-4k: (groupid=0, jobs=1): err= 0: pid=3623: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=4420.8KB/s, iops=1105 , runt=115817msec
clat (usec): min=14 , max=186689 , avg=899.54, stdev=6106.61
lat (usec): min=14 , max=186690 , avg=899.91, stdev=6106.67
clat percentiles (usec):
| 1.00th=[ 16], 5.00th=[ 17], 10.00th=[ 17], 20.00th=[ 18],
| 30.00th=[ 18], 40.00th=[ 18], 50.00th=[ 19], 60.00th=[ 20],
| 70.00th=[ 34], 80.00th=[ 35], 90.00th=[ 38], 95.00th=[ 60],
| 99.00th=[29056], 99.50th=[44800], 99.90th=[82432], 99.95th=[95744],
| 99.99th=[128512]
bw (KB/s) : min= 86, max=160208, per=17.58%, avg=4490.47, stdev=21756.70
lat (usec) : 20=58.02%, 50=34.61%, 100=3.36%, 250=0.29%, 500=0.04%
lat (usec) : 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.02%, 10=1.07%, 20=1.07%, 50=1.08%
lat (msec) : 100=0.37%, 250=0.04%
cpu : usr=0.68%, sys=3.40%, ctx=5325, majf=0, minf=20
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-rand-4k: (groupid=0, jobs=1): err= 0: pid=3624: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=3231.8KB/s, iops=807 , runt=158431msec
clat (usec): min=14 , max=202757 , avg=1232.28, stdev=7056.31
lat (usec): min=14 , max=202758 , avg=1232.66, stdev=7056.38
clat percentiles (usec):
| 1.00th=[ 16], 5.00th=[ 17], 10.00th=[ 18], 20.00th=[ 18],
| 30.00th=[ 18], 40.00th=[ 19], 50.00th=[ 20], 60.00th=[ 22],
| 70.00th=[ 35], 80.00th=[ 37], 90.00th=[ 48], 95.00th=[ 5088],
| 99.00th=[35584], 99.50th=[51456], 99.90th=[89600], 99.95th=[103936],
| 99.99th=[144384]
bw (KB/s) : min= 69, max=157248, per=12.19%, avg=3114.95, stdev=17849.28
lat (usec) : 20=43.54%, 50=47.14%, 100=3.82%, 250=0.27%, 500=0.02%
lat (usec) : 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.03%, 10=1.42%, 20=1.70%, 50=1.51%
lat (msec) : 100=0.47%, 250=0.06%
cpu : usr=0.54%, sys=2.73%, ctx=7170, majf=0, minf=20
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-rand-4k: (groupid=0, jobs=1): err= 0: pid=3625: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=3595.6KB/s, iops=898 , runt=142397msec
clat (usec): min=14 , max=213477 , avg=1106.85, stdev=6680.27
lat (usec): min=14 , max=213478 , avg=1107.26, stdev=6680.33
clat percentiles (usec):
| 1.00th=[ 16], 5.00th=[ 17], 10.00th=[ 18], 20.00th=[ 18],
| 30.00th=[ 18], 40.00th=[ 19], 50.00th=[ 20], 60.00th=[ 34],
| 70.00th=[ 35], 80.00th=[ 36], 90.00th=[ 45], 95.00th=[ 87],
| 99.00th=[35072], 99.50th=[51456], 99.90th=[84480], 99.95th=[98816],
| 99.99th=[126464]
bw (KB/s) : min= 92, max=159048, per=14.16%, avg=3617.85, stdev=18680.26
lat (usec) : 20=47.54%, 50=43.65%, 100=3.96%, 250=0.28%, 500=0.04%
lat (usec) : 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.02%, 10=1.24%, 20=1.39%, 50=1.34%
lat (msec) : 100=0.49%, 250=0.05%
cpu : usr=0.58%, sys=3.04%, ctx=6402, majf=0, minf=20
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-rand-4k: (groupid=0, jobs=1): err= 0: pid=3626: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=3378.4KB/s, iops=844 , runt=151552msec
clat (usec): min=14 , max=174338 , avg=1178.66, stdev=6842.37
lat (usec): min=14 , max=174339 , avg=1179.04, stdev=6842.43
clat percentiles (usec):
| 1.00th=[ 16], 5.00th=[ 17], 10.00th=[ 17], 20.00th=[ 18],
| 30.00th=[ 18], 40.00th=[ 18], 50.00th=[ 19], 60.00th=[ 20],
| 70.00th=[ 34], 80.00th=[ 35], 90.00th=[ 46], 95.00th=[ 197],
| 99.00th=[35072], 99.50th=[50944], 99.90th=[86528], 99.95th=[101888],
| 99.99th=[127488]
bw (KB/s) : min= 118, max=173896, per=12.60%, avg=3219.06, stdev=18790.51
lat (usec) : 20=58.37%, 50=32.88%, 100=3.57%, 250=0.22%, 500=0.02%
lat (usec) : 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%, 10=1.36%, 20=1.57%, 50=1.46%
lat (msec) : 100=0.46%, 250=0.05%
cpu : usr=0.53%, sys=2.74%, ctx=6850, majf=0, minf=20
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-rand-4k: (groupid=0, jobs=1): err= 0: pid=3627: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=3704.2KB/s, iops=926 , runt=138222msec
clat (usec): min=14 , max=253639 , avg=1074.52, stdev=6640.79
lat (usec): min=14 , max=253640 , avg=1074.90, stdev=6640.85
clat percentiles (usec):
| 1.00th=[ 16], 5.00th=[ 17], 10.00th=[ 17], 20.00th=[ 18],
| 30.00th=[ 18], 40.00th=[ 18], 50.00th=[ 19], 60.00th=[ 21],
| 70.00th=[ 34], 80.00th=[ 35], 90.00th=[ 40], 95.00th=[ 69],
| 99.00th=[33536], 99.50th=[49408], 99.90th=[85504], 99.95th=[99840],
| 99.99th=[134144]
bw (KB/s) : min= 111, max=162768, per=13.25%, avg=3383.69, stdev=18588.28
lat (usec) : 20=54.20%, 50=37.93%, 100=3.29%, 250=0.18%, 500=0.02%
lat (usec) : 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%, 10=1.16%, 20=1.37%, 50=1.33%
lat (msec) : 100=0.44%, 250=0.05%, 500=0.01%
cpu : usr=0.57%, sys=2.97%, ctx=6192, majf=0, minf=20
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-rand-128k: (groupid=1, jobs=1): err= 0: pid=3641: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=5210.2KB/s, iops=40 , runt= 98269msec
clat (usec): min=162 , max=169602 , avg=24535.30, stdev=22102.38
lat (usec): min=163 , max=169605 , avg=24537.49, stdev=22102.35
clat percentiles (usec):
| 1.00th=[ 362], 5.00th=[ 4448], 10.00th=[ 6112], 20.00th=[ 8384],
| 30.00th=[10560], 40.00th=[13376], 50.00th=[16768], 60.00th=[21120],
| 70.00th=[28288], 80.00th=[38144], 90.00th=[54016], 95.00th=[69120],
| 99.00th=[103936], 99.50th=[115200], 99.90th=[154624], 99.95th=[154624],
| 99.99th=[168960]
bw (KB/s) : min= 2976, max=12800, per=12.58%, avg=5163.51, stdev=1191.49
lat (usec) : 250=0.93%, 500=0.30%, 750=0.15%, 1000=0.03%
lat (msec) : 2=1.33%, 4=1.75%, 10=23.00%, 20=30.00%, 50=30.25%
lat (msec) : 100=10.95%, 250=1.33%
cpu : usr=0.15%, sys=1.71%, ctx=4001, majf=0, minf=51
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
read-rand-128k: (groupid=1, jobs=1): err= 0: pid=3642: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=5224.7KB/s, iops=40 , runt= 98008msec
clat (usec): min=160 , max=179805 , avg=24470.60, stdev=22561.19
lat (usec): min=161 , max=179807 , avg=24472.85, stdev=22561.15
clat percentiles (usec):
| 1.00th=[ 213], 5.00th=[ 4512], 10.00th=[ 6048], 20.00th=[ 8512],
| 30.00th=[10560], 40.00th=[12864], 50.00th=[16192], 60.00th=[20864],
| 70.00th=[28032], 80.00th=[37120], 90.00th=[54016], 95.00th=[70144],
| 99.00th=[107008], 99.50th=[124416], 99.90th=[154624], 99.95th=[162816],
| 99.99th=[179200]
bw (KB/s) : min= 2681, max=11613, per=12.73%, avg=5224.53, stdev=1145.93
lat (usec) : 250=1.25%, 500=0.45%, 750=0.18%
lat (msec) : 2=0.98%, 4=1.38%, 10=23.60%, 20=30.05%, 50=29.65%
lat (msec) : 100=11.10%, 250=1.38%
cpu : usr=0.17%, sys=1.69%, ctx=3976, majf=0, minf=54
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
read-rand-128k: (groupid=1, jobs=1): err= 0: pid=3643: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=5448.4KB/s, iops=42 , runt= 93973msec
clat (usec): min=165 , max=206917 , avg=23461.80, stdev=21237.45
lat (usec): min=166 , max=206919 , avg=23463.96, stdev=21237.47
clat percentiles (usec):
| 1.00th=[ 247], 5.00th=[ 4768], 10.00th=[ 6432], 20.00th=[ 8512],
| 30.00th=[10432], 40.00th=[12992], 50.00th=[15936], 60.00th=[20352],
| 70.00th=[26752], 80.00th=[35584], 90.00th=[51456], 95.00th=[67072],
| 99.00th=[101888], 99.50th=[118272], 99.90th=[152576], 99.95th=[154624],
| 99.99th=[207872]
bw (KB/s) : min= 2500, max= 8347, per=13.29%, avg=5456.29, stdev=1113.60
lat (usec) : 250=1.07%, 500=0.35%, 750=0.15%, 1000=0.05%
lat (msec) : 2=0.88%, 4=1.18%, 10=24.65%, 20=30.70%, 50=30.38%
lat (msec) : 100=9.45%, 250=1.15%
cpu : usr=0.17%, sys=1.77%, ctx=3990, majf=0, minf=52
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
read-rand-128k: (groupid=1, jobs=1): err= 0: pid=3644: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=5242.2KB/s, iops=40 , runt= 97670msec
clat (usec): min=159 , max=170039 , avg=24386.08, stdev=22654.02
lat (usec): min=160 , max=170041 , avg=24388.24, stdev=22654.01
clat percentiles (usec):
| 1.00th=[ 207], 5.00th=[ 4384], 10.00th=[ 6112], 20.00th=[ 8384],
| 30.00th=[10432], 40.00th=[12992], 50.00th=[16064], 60.00th=[20608],
| 70.00th=[27520], 80.00th=[37632], 90.00th=[54528], 95.00th=[70144],
| 99.00th=[105984], 99.50th=[120320], 99.90th=[160768], 99.95th=[166912],
| 99.99th=[171008]
bw (KB/s) : min= 2475, max= 7968, per=12.79%, avg=5249.17, stdev=1091.36
lat (usec) : 250=1.62%, 500=0.55%, 750=0.30%
lat (msec) : 2=1.03%, 4=1.15%, 10=23.18%, 20=30.68%, 50=29.05%
lat (msec) : 100=11.10%, 250=1.35%
cpu : usr=0.17%, sys=1.70%, ctx=3958, majf=0, minf=54
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
read-rand-128k: (groupid=1, jobs=1): err= 0: pid=3645: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=5497.9KB/s, iops=42 , runt= 93127msec
clat (usec): min=165 , max=161392 , avg=23249.66, stdev=20591.12
lat (usec): min=166 , max=161394 , avg=23251.85, stdev=20591.14
clat percentiles (usec):
| 1.00th=[ 213], 5.00th=[ 4704], 10.00th=[ 6240], 20.00th=[ 8384],
| 30.00th=[10432], 40.00th=[12992], 50.00th=[15680], 60.00th=[20352],
| 70.00th=[26496], 80.00th=[36096], 90.00th=[51456], 95.00th=[67072],
| 99.00th=[95744], 99.50th=[104960], 99.90th=[140288], 99.95th=[140288],
| 99.99th=[160768]
bw (KB/s) : min= 3310, max= 8652, per=13.44%, avg=5516.09, stdev=1012.77
lat (usec) : 250=1.35%, 500=0.55%, 750=0.23%, 1000=0.08%
lat (msec) : 2=0.85%, 4=1.25%, 10=23.60%, 20=31.25%, 50=29.95%
lat (msec) : 100=10.15%, 250=0.75%
cpu : usr=0.18%, sys=1.78%, ctx=3970, majf=0, minf=54
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
read-rand-128k: (groupid=1, jobs=1): err= 0: pid=3646: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=5310.7KB/s, iops=41 , runt= 96411msec
clat (usec): min=187 , max=177768 , avg=24070.69, stdev=21255.82
lat (usec): min=189 , max=177770 , avg=24072.85, stdev=21255.79
clat percentiles (usec):
| 1.00th=[ 422], 5.00th=[ 5408], 10.00th=[ 6752], 20.00th=[ 8768],
| 30.00th=[10816], 40.00th=[13376], 50.00th=[16320], 60.00th=[20608],
| 70.00th=[27264], 80.00th=[36096], 90.00th=[52992], 95.00th=[69120],
| 99.00th=[103936], 99.50th=[117248], 99.90th=[134144], 99.95th=[134144],
| 99.99th=[177152]
bw (KB/s) : min= 3000, max= 8273, per=12.95%, avg=5315.16, stdev=1031.57
lat (usec) : 250=0.78%, 500=0.28%, 750=0.18%, 1000=0.05%
lat (msec) : 2=0.60%, 4=1.25%, 10=22.90%, 20=32.20%, 50=30.12%
lat (msec) : 100=10.40%, 250=1.25%
cpu : usr=0.20%, sys=1.70%, ctx=4004, majf=0, minf=54
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
read-rand-128k: (groupid=1, jobs=1): err= 0: pid=3647: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=5248.3KB/s, iops=41 , runt= 97556msec
clat (usec): min=160 , max=179770 , avg=24356.91, stdev=22596.71
lat (usec): min=161 , max=179772 , avg=24359.16, stdev=22596.66
clat percentiles (usec):
| 1.00th=[ 207], 5.00th=[ 3984], 10.00th=[ 5920], 20.00th=[ 8384],
| 30.00th=[10560], 40.00th=[13120], 50.00th=[16320], 60.00th=[20864],
| 70.00th=[27008], 80.00th=[37120], 90.00th=[54016], 95.00th=[70144],
| 99.00th=[111104], 99.50th=[118272], 99.90th=[152576], 99.95th=[164864],
| 99.99th=[179200]
bw (KB/s) : min= 2504, max= 9826, per=12.83%, avg=5265.56, stdev=1161.70
lat (usec) : 250=1.65%, 500=0.60%, 750=0.25%, 1000=0.03%
lat (msec) : 2=1.18%, 4=1.33%, 10=22.70%, 20=30.30%, 50=29.68%
lat (msec) : 100=10.88%, 250=1.43%
cpu : usr=0.17%, sys=1.70%, ctx=3966, majf=0, minf=55
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
read-rand-128k: (groupid=1, jobs=1): err= 0: pid=3648: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=5130.2KB/s, iops=40 , runt= 99786msec
clat (usec): min=192 , max=185689 , avg=24915.39, stdev=24056.97
lat (usec): min=194 , max=185691 , avg=24917.61, stdev=24056.96
clat percentiles (usec):
| 1.00th=[ 1272], 5.00th=[ 2192], 10.00th=[ 5344], 20.00th=[ 7712],
| 30.00th=[ 9920], 40.00th=[12736], 50.00th=[16320], 60.00th=[20608],
| 70.00th=[28032], 80.00th=[38656], 90.00th=[58624], 95.00th=[76288],
| 99.00th=[110080], 99.50th=[125440], 99.90th=[162816], 99.95th=[164864],
| 99.99th=[185344]
bw (KB/s) : min= 2299, max=24174, per=12.34%, avg=5064.04, stdev=2315.04
lat (usec) : 250=0.62%, 500=0.33%, 750=0.03%
lat (msec) : 2=3.43%, 4=1.68%, 10=24.28%, 20=28.53%, 50=27.35%
lat (msec) : 100=12.20%, 250=1.58%
cpu : usr=0.18%, sys=1.66%, ctx=4013, majf=0, minf=55
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
read-seq-4k: (groupid=2, jobs=1): err= 0: pid=3653: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=18193KB/s, iops=4548 , runt= 28143msec
clat (usec): min=12 , max=2258.3K, avg=217.05, stdev=14874.44
lat (usec): min=12 , max=2258.3K, avg=217.35, stdev=14874.44
clat percentiles (usec):
| 1.00th=[ 13], 5.00th=[ 14], 10.00th=[ 14], 20.00th=[ 14],
| 30.00th=[ 14], 40.00th=[ 15], 50.00th=[ 16], 60.00th=[ 16],
| 70.00th=[ 17], 80.00th=[ 21], 90.00th=[ 28], 95.00th=[ 30],
| 99.00th=[ 60], 99.50th=[ 195], 99.90th=[ 1640], 99.95th=[ 8768],
| 99.99th=[528384]
bw (KB/s) : min= 141, max=91448, per=17.02%, avg=24771.74, stdev=27882.26
lat (usec) : 20=78.62%, 50=20.06%, 100=0.77%, 250=0.09%, 500=0.22%
lat (usec) : 750=0.06%, 1000=0.03%
lat (msec) : 2=0.07%, 4=0.02%, 10=0.02%, 20=0.01%, 50=0.01%
lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2000=0.01%, >=2000=0.01%
cpu : usr=1.52%, sys=8.69%, ctx=902, majf=0, minf=25
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-seq-4k: (groupid=2, jobs=1): err= 0: pid=3654: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=25698KB/s, iops=6424 , runt= 19924msec
clat (usec): min=12 , max=1782.8K, avg=152.74, stdev=10699.88
lat (usec): min=12 , max=1782.8K, avg=153.09, stdev=10699.89
clat percentiles (usec):
| 1.00th=[ 13], 5.00th=[ 14], 10.00th=[ 14], 20.00th=[ 14],
| 30.00th=[ 15], 40.00th=[ 16], 50.00th=[ 16], 60.00th=[ 17],
| 70.00th=[ 17], 80.00th=[ 26], 90.00th=[ 28], 95.00th=[ 30],
| 99.00th=[ 52], 99.50th=[ 68], 99.90th=[ 1304], 99.95th=[ 7136],
| 99.99th=[501760]
bw (KB/s) : min= 356, max=80968, per=18.72%, avg=27240.00, stdev=19026.07
lat (usec) : 20=77.57%, 50=21.31%, 100=0.81%, 250=0.08%, 500=0.07%
lat (usec) : 750=0.02%, 1000=0.01%
lat (msec) : 2=0.04%, 4=0.01%, 10=0.03%, 20=0.01%, 50=0.01%
lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2000=0.01%
cpu : usr=2.49%, sys=12.05%, ctx=640, majf=0, minf=25
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-seq-4k: (groupid=2, jobs=1): err= 0: pid=3655: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=26422KB/s, iops=6605 , runt= 19378msec
clat (usec): min=12 , max=1609.2K, avg=148.38, stdev=9947.37
lat (usec): min=12 , max=1609.2K, avg=148.75, stdev=9947.38
clat percentiles (usec):
| 1.00th=[ 13], 5.00th=[ 14], 10.00th=[ 14], 20.00th=[ 14],
| 30.00th=[ 15], 40.00th=[ 16], 50.00th=[ 16], 60.00th=[ 17],
| 70.00th=[ 18], 80.00th=[ 27], 90.00th=[ 28], 95.00th=[ 31],
| 99.00th=[ 60], 99.50th=[ 80], 99.90th=[ 2128], 99.95th=[ 9280],
| 99.99th=[382976]
bw (KB/s) : min= 608, max=155368, per=21.71%, avg=31600.50, stdev=34032.42
lat (usec) : 20=75.06%, 50=23.63%, 100=0.90%, 250=0.08%, 500=0.09%
lat (usec) : 750=0.07%, 1000=0.02%
lat (msec) : 2=0.05%, 4=0.02%, 10=0.04%, 20=0.01%, 50=0.01%
lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2000=0.01%
cpu : usr=2.54%, sys=12.67%, ctx=715, majf=0, minf=25
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-seq-4k: (groupid=2, jobs=1): err= 0: pid=3656: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=28949KB/s, iops=7237 , runt= 17686msec
clat (usec): min=12 , max=2276.5K, avg=135.30, stdev=10537.99
lat (usec): min=12 , max=2276.5K, avg=135.62, stdev=10537.99
clat percentiles (usec):
| 1.00th=[ 13], 5.00th=[ 14], 10.00th=[ 14], 20.00th=[ 14],
| 30.00th=[ 14], 40.00th=[ 15], 50.00th=[ 15], 60.00th=[ 16],
| 70.00th=[ 18], 80.00th=[ 27], 90.00th=[ 28], 95.00th=[ 31],
| 99.00th=[ 64], 99.50th=[ 229], 99.90th=[ 1832], 99.95th=[10816],
| 99.99th=[374784]
bw (KB/s) : min= 1111, max=97680, per=21.67%, avg=31544.63, stdev=25128.97
lat (usec) : 20=74.59%, 50=23.91%, 100=0.92%, 250=0.10%, 500=0.23%
lat (usec) : 750=0.09%, 1000=0.02%
lat (msec) : 2=0.06%, 4=0.02%, 10=0.02%, 20=0.01%, 50=0.01%
lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2000=0.01%, >=2000=0.01%
cpu : usr=2.62%, sys=13.85%, ctx=984, majf=0, minf=25
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-seq-4k: (groupid=2, jobs=1): err= 0: pid=3657: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=20281KB/s, iops=5070 , runt= 25245msec
clat (usec): min=12 , max=1986.5K, avg=194.20, stdev=11022.54
lat (usec): min=12 , max=1986.5K, avg=194.52, stdev=11022.54
clat percentiles (usec):
| 1.00th=[ 13], 5.00th=[ 14], 10.00th=[ 14], 20.00th=[ 14],
| 30.00th=[ 15], 40.00th=[ 16], 50.00th=[ 16], 60.00th=[ 17],
| 70.00th=[ 18], 80.00th=[ 27], 90.00th=[ 29], 95.00th=[ 31],
| 99.00th=[ 62], 99.50th=[ 91], 99.90th=[ 3216], 99.95th=[21376],
| 99.99th=[585728]
bw (KB/s) : min= 189, max=58961, per=16.03%, avg=23328.84, stdev=18982.11
lat (usec) : 20=72.01%, 50=26.57%, 100=0.96%, 250=0.08%, 500=0.13%
lat (usec) : 750=0.07%, 1000=0.03%
lat (msec) : 2=0.06%, 4=0.01%, 10=0.03%, 20=0.01%, 50=0.01%
lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2000=0.01%
cpu : usr=1.89%, sys=10.09%, ctx=847, majf=0, minf=24
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-seq-4k: (groupid=2, jobs=1): err= 0: pid=3658: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=18823KB/s, iops=4705 , runt= 27201msec
clat (usec): min=12 , max=2316.3K, avg=209.60, stdev=14141.94
lat (usec): min=12 , max=2316.3K, avg=209.91, stdev=14141.94
clat percentiles (usec):
| 1.00th=[ 13], 5.00th=[ 14], 10.00th=[ 14], 20.00th=[ 14],
| 30.00th=[ 15], 40.00th=[ 15], 50.00th=[ 16], 60.00th=[ 17],
| 70.00th=[ 17], 80.00th=[ 22], 90.00th=[ 28], 95.00th=[ 30],
| 99.00th=[ 51], 99.50th=[ 80], 99.90th=[ 1624], 99.95th=[12096],
| 99.99th=[577536]
bw (KB/s) : min= 110, max=117738, per=17.50%, avg=25466.73, stdev=28506.22
lat (usec) : 20=78.74%, 50=20.18%, 100=0.63%, 250=0.09%, 500=0.14%
lat (usec) : 750=0.06%, 1000=0.03%
lat (msec) : 2=0.05%, 4=0.01%, 10=0.02%, 20=0.01%, 50=0.01%
lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2000=0.01%, >=2000=0.01%
cpu : usr=1.81%, sys=8.76%, ctx=774, majf=0, minf=24
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-seq-4k: (groupid=2, jobs=1): err= 0: pid=3659: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=21198KB/s, iops=5299 , runt= 24153msec
clat (usec): min=12 , max=1930.5K, avg=185.73, stdev=11395.15
lat (usec): min=12 , max=1930.5K, avg=186.09, stdev=11395.16
clat percentiles (usec):
| 1.00th=[ 13], 5.00th=[ 14], 10.00th=[ 14], 20.00th=[ 14],
| 30.00th=[ 15], 40.00th=[ 15], 50.00th=[ 16], 60.00th=[ 17],
| 70.00th=[ 18], 80.00th=[ 27], 90.00th=[ 29], 95.00th=[ 31],
| 99.00th=[ 65], 99.50th=[ 330], 99.90th=[ 3280], 99.95th=[17024],
| 99.99th=[618496]
bw (KB/s) : min= 194, max=80160, per=17.48%, avg=25434.70, stdev=19747.05
lat (usec) : 20=75.77%, 50=22.66%, 100=0.90%, 250=0.12%, 500=0.19%
lat (usec) : 750=0.14%, 1000=0.04%
lat (msec) : 2=0.07%, 4=0.02%, 10=0.03%, 20=0.01%, 50=0.01%
lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2000=0.01%
cpu : usr=1.90%, sys=10.24%, ctx=1032, majf=0, minf=24
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-seq-4k: (groupid=2, jobs=1): err= 0: pid=3660: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=20232KB/s, iops=5058 , runt= 25306msec
clat (usec): min=12 , max=2445.1K, avg=194.71, stdev=14693.36
lat (usec): min=12 , max=2445.1K, avg=195.01, stdev=14693.36
clat percentiles (usec):
| 1.00th=[ 13], 5.00th=[ 14], 10.00th=[ 14], 20.00th=[ 15],
| 30.00th=[ 15], 40.00th=[ 16], 50.00th=[ 16], 60.00th=[ 17],
| 70.00th=[ 17], 80.00th=[ 24], 90.00th=[ 28], 95.00th=[ 30],
| 99.00th=[ 54], 99.50th=[ 93], 99.90th=[ 1704], 99.95th=[ 7136],
| 99.99th=[593920]
bw (KB/s) : min= 157, max=86592, per=19.72%, avg=28701.58, stdev=22434.41
lat (usec) : 20=78.10%, 50=20.74%, 100=0.67%, 250=0.10%, 500=0.14%
lat (usec) : 750=0.07%, 1000=0.02%
lat (msec) : 2=0.07%, 4=0.01%, 10=0.04%, 20=0.01%, 50=0.01%
lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 2000=0.01%
lat (msec) : >=2000=0.01%
cpu : usr=1.81%, sys=9.58%, ctx=787, majf=0, minf=24
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=128000/w=0/d=0, short=r=0/w=0/d=0
read-seq-128k: (groupid=3, jobs=1): err= 0: pid=3661: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=20245KB/s, iops=158 , runt= 25290msec
clat (usec): min=84 , max=2185.9K, avg=6311.52, stdev=77900.42
lat (usec): min=84 , max=2185.9K, avg=6312.68, stdev=77900.42
clat percentiles (usec):
| 1.00th=[ 87], 5.00th=[ 94], 10.00th=[ 98], 20.00th=[ 155],
| 30.00th=[ 161], 40.00th=[ 167], 50.00th=[ 179], 60.00th=[ 219],
| 70.00th=[ 708], 80.00th=[ 988], 90.00th=[ 1160], 95.00th=[ 2096],
| 99.00th=[44288], 99.50th=[350208], 99.90th=[1482752], 99.95th=[1581056],
| 99.99th=[2179072]
bw (KB/s) : min= 418, max=65015, per=14.04%, avg=22354.95, stdev=16237.22
lat (usec) : 100=11.03%, 250=52.93%, 500=5.10%, 750=1.15%, 1000=11.25%
lat (msec) : 2=13.05%, 4=3.78%, 10=0.20%, 20=0.35%, 50=0.18%
lat (msec) : 100=0.15%, 250=0.20%, 500=0.28%, 750=0.12%, 1000=0.05%
lat (msec) : 2000=0.18%, >=2000=0.03%
cpu : usr=0.22%, sys=3.66%, ctx=1324, majf=0, minf=55
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
read-seq-128k: (groupid=3, jobs=1): err= 0: pid=3662: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=25735KB/s, iops=201 , runt= 19895msec
clat (usec): min=83 , max=1320.5K, avg=4964.51, stdev=59939.48
lat (usec): min=83 , max=1320.5K, avg=4965.38, stdev=59939.46
clat percentiles (usec):
| 1.00th=[ 86], 5.00th=[ 88], 10.00th=[ 93], 20.00th=[ 101],
| 30.00th=[ 155], 40.00th=[ 159], 50.00th=[ 165], 60.00th=[ 175],
| 70.00th=[ 211], 80.00th=[ 290], 90.00th=[ 1048], 95.00th=[ 1592],
| 99.00th=[38656], 99.50th=[240640], 99.90th=[1138688], 99.95th=[1236992],
| 99.99th=[1318912]
bw (KB/s) : min= 2740, max=106582, per=17.93%, avg=28542.33, stdev=26187.07
lat (usec) : 100=18.88%, 250=56.78%, 500=6.65%, 750=0.58%, 1000=5.40%
lat (msec) : 2=7.62%, 4=2.00%, 10=0.43%, 20=0.40%, 50=0.40%
lat (msec) : 100=0.18%, 250=0.20%, 500=0.12%, 750=0.12%, 1000=0.12%
lat (msec) : 2000=0.12%
cpu : usr=0.22%, sys=4.25%, ctx=803, majf=0, minf=58
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
read-seq-128k: (groupid=3, jobs=1): err= 0: pid=3663: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=23578KB/s, iops=184 , runt= 21715msec
clat (usec): min=84 , max=1962.4K, avg=5418.08, stdev=69696.94
lat (usec): min=84 , max=1962.4K, avg=5419.18, stdev=69696.93
clat percentiles (usec):
| 1.00th=[ 86], 5.00th=[ 93], 10.00th=[ 96], 20.00th=[ 126],
| 30.00th=[ 157], 40.00th=[ 163], 50.00th=[ 167], 60.00th=[ 185],
| 70.00th=[ 235], 80.00th=[ 908], 90.00th=[ 1128], 95.00th=[ 1768],
| 99.00th=[42752], 99.50th=[246784], 99.90th=[1384448], 99.95th=[1417216],
| 99.99th=[1957888]
bw (KB/s) : min= 1960, max=86016, per=17.24%, avg=27452.05, stdev=22849.34
lat (usec) : 100=13.12%, 250=58.05%, 500=4.53%, 750=0.68%, 1000=9.50%
lat (msec) : 2=9.47%, 4=2.70%, 10=0.20%, 20=0.38%, 50=0.43%
lat (msec) : 100=0.20%, 250=0.25%, 500=0.18%, 750=0.08%, 1000=0.10%
lat (msec) : 2000=0.15%
cpu : usr=0.33%, sys=3.98%, ctx=1051, majf=0, minf=58
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
read-seq-128k: (groupid=3, jobs=1): err= 0: pid=3664: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=21064KB/s, iops=164 , runt= 24307msec
clat (usec): min=87 , max=1834.5K, avg=6067.55, stdev=71812.60
lat (usec): min=87 , max=1834.5K, avg=6068.39, stdev=71812.59
clat percentiles (usec):
| 1.00th=[ 92], 5.00th=[ 94], 10.00th=[ 99], 20.00th=[ 153],
| 30.00th=[ 159], 40.00th=[ 163], 50.00th=[ 167], 60.00th=[ 177],
| 70.00th=[ 209], 80.00th=[ 255], 90.00th=[ 1064], 95.00th=[ 1336],
| 99.00th=[88576], 99.50th=[313344], 99.90th=[1220608], 99.95th=[1253376],
| 99.99th=[1826816]
bw (KB/s) : min= 184, max=56910, per=13.33%, avg=21221.46, stdev=14124.45
lat (usec) : 100=10.73%, 250=68.70%, 500=6.60%, 750=0.30%, 1000=2.73%
lat (msec) : 2=7.10%, 4=1.83%, 10=0.12%, 20=0.28%, 50=0.40%
lat (msec) : 100=0.28%, 250=0.30%, 500=0.28%, 750=0.12%, 1000=0.05%
lat (msec) : 2000=0.20%
cpu : usr=0.19%, sys=3.58%, ctx=656, majf=0, minf=59
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
read-seq-128k: (groupid=3, jobs=1): err= 0: pid=3665: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=21055KB/s, iops=164 , runt= 24317msec
clat (usec): min=84 , max=2808.8K, avg=6071.59, stdev=92750.68
lat (usec): min=84 , max=2808.8K, avg=6072.34, stdev=92750.68
clat percentiles (usec):
| 1.00th=[ 87], 5.00th=[ 93], 10.00th=[ 95], 20.00th=[ 107],
| 30.00th=[ 155], 40.00th=[ 157], 50.00th=[ 163], 60.00th=[ 167],
| 70.00th=[ 175], 80.00th=[ 211], 90.00th=[ 258], 95.00th=[ 988],
| 99.00th=[29312], 99.50th=[216064], 99.90th=[2007040], 99.95th=[2572288],
| 99.99th=[2801664]
bw (KB/s) : min= 1961, max=109312, per=18.27%, avg=29089.10, stdev=29865.43
lat (usec) : 100=15.25%, 250=73.83%, 500=3.88%, 750=0.50%, 1000=1.68%
lat (msec) : 2=2.23%, 4=0.62%, 10=0.12%, 20=0.58%, 50=0.53%
lat (msec) : 100=0.03%, 250=0.33%, 500=0.12%, 750=0.08%, 1000=0.10%
lat (msec) : 2000=0.05%, >=2000=0.10%
cpu : usr=0.15%, sys=3.26%, ctx=399, majf=0, minf=58
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
read-seq-128k: (groupid=3, jobs=1): err= 0: pid=3666: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=20210KB/s, iops=157 , runt= 25334msec
clat (usec): min=83 , max=2545.8K, avg=6325.46, stdev=96083.46
lat (usec): min=84 , max=2545.8K, avg=6326.24, stdev=96083.47
clat percentiles (usec):
| 1.00th=[ 88], 5.00th=[ 93], 10.00th=[ 96], 20.00th=[ 106],
| 30.00th=[ 139], 40.00th=[ 157], 50.00th=[ 161], 60.00th=[ 167],
| 70.00th=[ 177], 80.00th=[ 211], 90.00th=[ 266], 95.00th=[ 924],
| 99.00th=[49920], 99.50th=[185344], 99.90th=[2072576], 99.95th=[2244608],
| 99.99th=[2539520]
bw (KB/s) : min= 264, max=83273, per=19.83%, avg=31562.33, stdev=29898.43
lat (usec) : 100=14.40%, 250=73.23%, 500=6.35%, 750=0.33%, 1000=1.33%
lat (msec) : 2=1.73%, 4=0.68%, 10=0.15%, 20=0.28%, 50=0.53%
lat (msec) : 100=0.33%, 250=0.28%, 500=0.15%, 1000=0.05%, 2000=0.12%
lat (msec) : >=2000=0.10%
cpu : usr=0.17%, sys=3.12%, ctx=354, majf=0, minf=57
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
read-seq-128k: (groupid=3, jobs=1): err= 0: pid=3667: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=20115KB/s, iops=157 , runt= 25454msec
clat (usec): min=83 , max=2668.2K, avg=6354.74, stdev=96216.62
lat (usec): min=84 , max=2668.2K, avg=6355.57, stdev=96216.62
clat percentiles (usec):
| 1.00th=[ 89], 5.00th=[ 95], 10.00th=[ 98], 20.00th=[ 133],
| 30.00th=[ 159], 40.00th=[ 163], 50.00th=[ 169], 60.00th=[ 177],
| 70.00th=[ 187], 80.00th=[ 219], 90.00th=[ 286], 95.00th=[ 1020],
| 99.00th=[50432], 99.50th=[222208], 99.90th=[1777664], 99.95th=[2572288],
| 99.99th=[2670592]
bw (KB/s) : min= 552, max=92547, per=18.02%, avg=28691.06, stdev=27743.78
lat (usec) : 100=11.53%, 250=75.17%, 500=5.93%, 750=0.55%, 1000=1.58%
lat (msec) : 2=2.55%, 4=0.60%, 10=0.10%, 20=0.43%, 50=0.55%
lat (msec) : 100=0.33%, 250=0.23%, 500=0.20%, 750=0.05%, 2000=0.15%
lat (msec) : >=2000=0.08%
cpu : usr=0.18%, sys=3.24%, ctx=445, majf=0, minf=57
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
read-seq-128k: (groupid=3, jobs=1): err= 0: pid=3668: Thu Nov 14 19:23:26 2013
read : io=512000KB, bw=19900KB/s, iops=155 , runt= 25729msec
clat (usec): min=85 , max=2236.2K, avg=6424.51, stdev=81093.11
lat (usec): min=86 , max=2236.2K, avg=6425.34, stdev=81093.11
clat percentiles (usec):
| 1.00th=[ 87], 5.00th=[ 94], 10.00th=[ 98], 20.00th=[ 111],
| 30.00th=[ 159], 40.00th=[ 161], 50.00th=[ 167], 60.00th=[ 173],
| 70.00th=[ 185], 80.00th=[ 219], 90.00th=[ 298], 95.00th=[ 1128],
| 99.00th=[101888], 99.50th=[403456], 99.90th=[1662976], 99.95th=[1712128],
| 99.99th=[2244608]
bw (KB/s) : min= 190, max=89730, per=15.00%, avg=23879.69, stdev=21986.41
lat (usec) : 100=11.53%, 250=73.62%, 500=7.05%, 750=0.35%, 1000=1.80%
lat (msec) : 2=2.95%, 4=0.68%, 10=0.15%, 20=0.25%, 50=0.40%
lat (msec) : 100=0.20%, 250=0.43%, 500=0.20%, 750=0.23%, 2000=0.12%
lat (msec) : >=2000=0.05%
cpu : usr=0.18%, sys=3.19%, ctx=408, majf=0, minf=58
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=4000/w=0/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4000.0MB, aggrb=25546KB/s, minb=3193KB/s, maxb=4420KB/s, mint=115817msec, maxt=160334msec
Run status group 1 (all jobs):
READ: io=4000.0MB, aggrb=41047KB/s, minb=5130KB/s, maxb=5497KB/s, mint=93127msec, maxt=99786msec
Run status group 2 (all jobs):
READ: io=4000.0MB, aggrb=145542KB/s, minb=18192KB/s, maxb=28949KB/s, mint=17686msec, maxt=28143msec
Run status group 3 (all jobs):
READ: io=4000.0MB, aggrb=159197KB/s, minb=19899KB/s, maxb=25735KB/s, mint=19895msec, maxt=25729msec
Raw
test
#!/bin/bash
start=$(date +"%Y-%m-%d_%H-%M-%S")
log="test_$start.log"
echo "Test started: $start" >> $log
cat env-zfs | grep -v -E "^#" &>> $log
cat env-spl | grep -v -E "^#" &>> $log
echo "Parallel dd's" >> $log
for i in `seq 1 4`; do
dd bs=4k of=/dev/null if=/tank3/asyncreads.$i.0 &>> $log.$i &
done
wait
for i in `seq 1 4`; do
cat $log.$i >> $log
done
echo " " >> $log
echo "fio tests:" >> $log
fio test.fio &>> $log
ended=$(date +"%Y-%m-%d_%H-%M-%S")
echo "Test ended: $ended" >> $log
Raw
test.fio
[global]
size=500m
numjobs=8
directory=/tank3
[read-rand-4k]
rw=randread
bs=4k
[read-rand-128k]
stonewall
rw=randread
bs=128k
[read-seq-4k]
stonewall
rw=read
bs=4k
[read-seq-128k]
stonewall
rw=read
bs=128k
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment