I've added a file /vendor/random.data with 4GB of random data from /dev/urandom. We are comparing block sizes 2K, 4K, 8K and 16K for ext4 and erofs.
For erofs, we are also using BOARD_EROFS_PCLUSTER_SIZE := 262144 according to the Google documentation. We will compare lz4hc, lz4 and disabling compression.
I did mount /vendor -o remount,cache_strategy=disabled for erofs to make sure nothing gets cached. The 5.10 kernel on cupid does not seem to support dax=always or the legacy dax flags, neither does it support deduplicating blocks. That may result in more accurate and/or better results, but the numbers difference is huge as is.
Also, ext4 has quite stable read numbers, while erofs with compression fluctuates by ~5-10% on subsequent runs.
Won't work, fio: io_u error on file /vendor/random.data: Invalid argument: read offset=112640, buflen=2048 spam
Sequential reads:
$ fio --readonly --ioengine=libaio --direct=1 --gtod_reduce=1 --name=ext4-4k-sequential-read --bs=4K --iodepth=64 --size=4G --filename=/vendor/random.data --readwrite=read
ext4-4k-sequential-read: (groupid=0, jobs=1): err= 0: pid=5635: Thu Jun 1 21:05:26 2023
read: IOPS=189k, BW=739MiB/s (775MB/s)(4096MiB/5541msec)
bw ( KiB/s): min=652296, max=776408, per=100.00%, avg=757747.64, stdev=35268.43, samples=11
iops : min=163076, max=194102, avg=189437.09, stdev=8816.51, samples=11
cpu : usr=5.43%, sys=48.16%, ctx=210793, majf=0, minf=70
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=1048576,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=739MiB/s (775MB/s), 739MiB/s-739MiB/s (775MB/s-775MB/s), io=4096MiB (4295MB), run=5541-5541msec
Disk stats (read/write):
dm-4: ios=1019760/0, merge=0/0, ticks=268560/0, in_queue=268560, util=98.40%, aggrios=408673/0, aggrmerge=639901/0, aggrticks=92013/0, aggrin_queue=92012, aggrutil=97.43%
sda: ios=408673/0, merge=639901/0, ticks=92013/0, in_queue=92012, util=97.43%
Random reads:
$ fio --readonly --ioengine=libaio --direct=1 --gtod_reduce=1 --name=ext4-4k-random-read --bs=4K --iodepth=64 --size=4G --filename=/vendor/random.data --readwrite=randread
ext4-4k-random-read: (groupid=0, jobs=1): err= 0: pid=5662: Thu Jun 1 21:05:58 2023
read: IOPS=134k, BW=522MiB/s (547MB/s)(4096MiB/7845msec)
bw ( KiB/s): min=398664, max=649904, per=99.08%, avg=529706.67, stdev=63400.28, samples=15
iops : min=99666, max=162476, avg=132426.80, stdev=15850.32, samples=15
cpu : usr=4.91%, sys=47.77%, ctx=255225, majf=0, minf=70
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=1048576,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=522MiB/s (547MB/s), 522MiB/s-522MiB/s (547MB/s-547MB/s), io=4096MiB (4295MB), run=7845-7845msec
Disk stats (read/write):
dm-4: ios=1014816/0, merge=0/0, ticks=303796/0, in_queue=303796, util=98.80%, aggrios=1048419/0, aggrmerge=155/0, aggrticks=304250/0, aggrin_queue=304250, aggrutil=98.48%
sda: ios=1048419/0, merge=155/0, ticks=304250/0, in_queue=304250, util=98.48%
Sequential reads:
$ fio --readonly --ioengine=libaio --direct=1 --gtod_reduce=1 --name=ext4-8k-sequential-read --bs=8K --iodepth=64 --size=4G --filename=/vendor/random.data --readwrite=read
ext4-8k-sequential-read: (groupid=0, jobs=1): err= 0: pid=5848: Thu Jun 1 21:09:32 2023
read: IOPS=148k, BW=1154MiB/s (1211MB/s)(4096MiB/3548msec)
bw ( MiB/s): min= 978, max= 1279, per=100.00%, avg=1156.82, stdev=97.15, samples=7
iops : min=125234, max=163728, avg=148073.71, stdev=12434.89, samples=7
cpu : usr=5.61%, sys=42.94%, ctx=118255, majf=0, minf=133
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=524288,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=1154MiB/s (1211MB/s), 1154MiB/s-1154MiB/s (1211MB/s-1211MB/s), io=4096MiB (4295MB), run=3548-3548msec
Disk stats (read/write):
dm-4: ios=502232/0, merge=0/0, ticks=183672/0, in_queue=183672, util=97.08%, aggrios=196856/0, aggrmerge=327455/0, aggrticks=59802/0, aggrin_queue=59802, aggrutil=96.10%
sda: ios=196856/0, merge=327455/0, ticks=59802/0, in_queue=59802, util=96.10%
Random reads:
$ fio --readonly --ioengine=libaio --direct=1 --gtod_reduce=1 --name=ext4-8k-random-read --bs=8K --iodepth=64 --size=4G --filename=/vendor/random.data --readwrite=randread
ext4-8k-random-read: (groupid=0, jobs=1): err= 0: pid=6030: Thu Jun 1 21:11:47 2023
read: IOPS=81.8k, BW=639MiB/s (670MB/s)(4096MiB/6406msec)
bw ( KiB/s): min=588672, max=685488, per=99.54%, avg=651748.00, stdev=25704.65, samples=12
iops : min=73584, max=85686, avg=81468.67, stdev=3213.06, samples=12
cpu : usr=4.61%, sys=43.06%, ctx=146625, majf=0, minf=134
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=524288,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=639MiB/s (670MB/s), 639MiB/s-639MiB/s (670MB/s-670MB/s), io=4096MiB (4295MB), run=6406-6406msec
Disk stats (read/write):
dm-4: ios=522469/0, merge=0/0, ticks=352152/0, in_queue=352152, util=98.54%, aggrios=522457/0, aggrmerge=1854/0, aggrticks=347617/0, aggrin_queue=367460, aggrutil=98.13%
sda: ios=522457/0, merge=1854/0, ticks=347617/0, in_queue=367460, util=98.13%
Sequential reads:
$ fio --readonly --ioengine=libaio --direct=1 --gtod_reduce=1 --name=ext4-16k-sequential-read --bs=16K --iodepth=64 --size=4G --filename=/vendor/random.data --readwrite=read
ext4-16k-sequential-read: (groupid=0, jobs=1): err= 0: pid=6079: Thu Jun 1 21:13:08 2023
read: IOPS=97.8k, BW=1528MiB/s (1603MB/s)(4096MiB/2680msec)
bw ( MiB/s): min= 1216, max= 1638, per=99.90%, avg=1526.76, stdev=176.69, samples=5
iops : min=77824, max=104878, avg=97712.80, stdev=11307.87, samples=5
cpu : usr=5.11%, sys=37.33%, ctx=58531, majf=0, minf=263
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=262144,0,0,0 short=1,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=1528MiB/s (1603MB/s), 1528MiB/s-1528MiB/s (1603MB/s-1603MB/s), io=4096MiB (4295MB), run=2680-2680msec
Disk stats (read/write):
dm-4: ios=257973/0, merge=0/0, ticks=149844/0, in_queue=149844, util=96.32%, aggrios=88316/0, aggrmerge=173858/0, aggrticks=44939/0, aggrin_queue=44938, aggrutil=95.33%
sda: ios=88316/0, merge=173858/0, ticks=44939/0, in_queue=44938, util=95.33%
Random reads:
$ fio --readonly --ioengine=libaio --direct=1 --gtod_reduce=1 --name=ext4-16k-random-read --bs=16K --iodepth=64 --size=4G --filename=/vendor/random.data --readwrite=randread
ext4-16k-random-read: (groupid=0, jobs=1): err= 0: pid=6112: Thu Jun 1 21:13:58 2023
read: IOPS=57.7k, BW=901MiB/s (945MB/s)(4096MiB/4545msec)
bw ( KiB/s): min=747872, max=955104, per=99.88%, avg=921752.89, stdev=65928.11, samples=9
iops : min=46744, max=59694, avg=57609.78, stdev=4119.85, samples=9
cpu : usr=4.60%, sys=39.72%, ctx=97173, majf=0, minf=262
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=262145,0,0,0 short=1,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=901MiB/s (945MB/s), 901MiB/s-901MiB/s (945MB/s-945MB/s), io=4096MiB (4295MB), run=4545-4545msec
Disk stats (read/write):
dm-4: ios=252129/0, merge=0/0, ticks=264000/0, in_queue=264000, util=98.02%, aggrios=260253/0, aggrmerge=1921/0, aggrticks=268904/0, aggrin_queue=268904, aggrutil=97.28%
sda: ios=260253/0, merge=1921/0, ticks=268904/0, in_queue=268904, util=97.28%
Sequential reads:
$ fio --readonly --ioengine=libaio --gtod_reduce=1 --name=erofs-2k-sequential-read --bs=2K --iodepth=64 --size=4G --filename=/vendor/random.data --readwrite=read
erofs-2k-sequential-read: (groupid=0, jobs=1): err= 0: pid=5741: Thu Jun 1 23:32:24 2023
read: IOPS=584k, BW=1140MiB/s (1196MB/s)(4096MiB/3592msec)
bw ( MiB/s): min= 651, max= 1256, per=99.92%, avg=1139.38, stdev=216.45, samples=7
iops : min=333632, max=643090, avg=583362.57, stdev=110823.54, samples=7
cpu : usr=18.91%, sys=78.50%, ctx=45, majf=0, minf=38
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=2097152,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=1140MiB/s (1196MB/s), 1140MiB/s-1140MiB/s (1196MB/s-1196MB/s), io=4096MiB (4295MB), run=3592-3592msec
Disk stats (read/write):
dm-4: ios=3844/0, merge=0/0, ticks=2092/0, in_queue=2092, util=92.47%, aggrios=8196/2, aggrmerge=0/0, aggrticks=3624/0, aggrin_queue=3625, aggrutil=92.11%
sda: ios=8196/2, merge=0/0, ticks=3624/0, in_queue=3625, util=92.11%
Random reads:
$ fio --readonly --ioengine=libaio --gtod_reduce=1 --name=erofs-2k-random-read --bs=2K --iodepth=64 --size=4G --filename=/vendor/random.data --readwrite=randread
erofs-2k-random-read: (groupid=0, jobs=1): err= 0: pid=6375: Thu Jun 1 23:47:01 2023
read: IOPS=4535, BW=9072KiB/s (9289kB/s)(4096MiB/462354msec)
bw ( KiB/s): min= 5640, max=11544, per=100.00%, avg=9079.37, stdev=777.93, samples=922
iops : min= 2820, max= 5772, avg=4539.54, stdev=388.93, samples=922
cpu : usr=3.62%, sys=28.18%, ctx=1669536, majf=2, minf=40
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=2097152,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=9072KiB/s (9289kB/s), 9072KiB/s-9072KiB/s (9289kB/s-9289kB/s), io=4096MiB (4295MB), run=462354-462354msec
Disk stats (read/write):
dm-4: ios=1659945/0, merge=0/0, ticks=302676/0, in_queue=302676, util=100.00%, aggrios=1661001/574, aggrmerge=788/0, aggrticks=337357/316, aggrin_queue=337748, aggrutil=100.00%
sda: ios=1661001/574, merge=788/0, ticks=337357/316, in_queue=337748, util=100.00%
Sequential reads:
$ fio --readonly --ioengine=libaio --gtod_reduce=1 --name=erofs-4k-sequential-read --bs=4K --iodepth=64 --size=4G --filename=/vendor/random.data --readwrite=read
erofs-4k-sequential-read: (groupid=0, jobs=1): err= 0: pid=6703: Thu Jun 1 23:49:54 2023
read: IOPS=326k, BW=1272MiB/s (1334MB/s)(4096MiB/3220msec)
bw ( MiB/s): min= 203, max= 1511, per=98.80%, avg=1256.75, stdev=516.74, samples=6
iops : min=52074, max=386988, avg=321728.67, stdev=132285.48, samples=6
cpu : usr=10.25%, sys=78.60%, ctx=2420, majf=0, minf=71
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=1048576,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=1272MiB/s (1334MB/s), 1272MiB/s-1272MiB/s (1334MB/s-1334MB/s), io=4096MiB (4295MB), run=3220-3220msec
Disk stats (read/write):
dm-4: ios=29870/0, merge=0/0, ticks=14932/0, in_queue=14932, util=85.71%, aggrios=31623/0, aggrmerge=1/0, aggrticks=15709/0, aggrin_queue=15709, aggrutil=85.14%
sda: ios=31623/0, merge=1/0, ticks=15709/0, in_queue=15709, util=85.14%
Random reads:
$ fio --readonly --ioengine=libaio --gtod_reduce=1 --name=erofs-4k-random-read --bs=4K --iodepth=64 --size=4G --filename=/vendor/random.data --readwrite=randread
erofs-4k-random-read: (groupid=0, jobs=1): err= 0: pid=6720: Thu Jun 1 23:54:22 2023
read: IOPS=4146, BW=16.2MiB/s (17.0MB/s)(4096MiB/252904msec)
bw ( KiB/s): min= 7488, max=21301, per=100.00%, avg=16601.43, stdev=1158.71, samples=504
iops : min= 1872, max= 5325, avg=4150.16, stdev=289.67, samples=504
cpu : usr=3.11%, sys=27.99%, ctx=1030103, majf=0, minf=70
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=1048576,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=16.2MiB/s (17.0MB/s), 16.2MiB/s-16.2MiB/s (17.0MB/s-17.0MB/s), io=4096MiB (4295MB), run=252904-252904msec
Disk stats (read/write):
dm-4: ios=1025492/0, merge=0/0, ticks=169076/0, in_queue=169076, util=100.00%, aggrios=1025680/35, aggrmerge=716/0, aggrticks=186026/20, aggrin_queue=186064, aggrutil=100.00%
sda: ios=1025680/35, merge=716/0, ticks=186026/20, in_queue=186064, util=100.00%
Sequential reads:
$ fio --readonly --ioengine=libaio --gtod_reduce=1 --name=erofs-8k-sequential-read --bs=8K --iodepth=64 --size=4G --filename=/vendor/random.data --readwrite=read
erofs-8k-sequential-read: (groupid=0, jobs=1): err= 0: pid=6857: Thu Jun 1 23:54:50 2023
read: IOPS=179k, BW=1400MiB/s (1468MB/s)(4096MiB/2925msec)
bw ( MiB/s): min= 484, max= 1626, per=97.91%, avg=1371.10, stdev=496.33, samples=5
iops : min=62058, max=208192, avg=175500.40, stdev=63530.38, samples=5
cpu : usr=5.13%, sys=78.63%, ctx=3271, majf=0, minf=135
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=524288,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=1400MiB/s (1468MB/s), 1400MiB/s-1400MiB/s (1468MB/s-1468MB/s), io=4096MiB (4295MB), run=2925-2925msec
Disk stats (read/write):
dm-4: ios=24578/0, merge=0/0, ticks=11616/0, in_queue=11616, util=85.22%, aggrios=26699/0, aggrmerge=0/0, aggrticks=12515/0, aggrin_queue=12515, aggrutil=84.47%
sda: ios=26699/0, merge=0/0, ticks=12515/0, in_queue=12515, util=84.47%
Random reads:
$ fio --readonly --ioengine=libaio --gtod_reduce=1 --name=erofs-8k-random-read --bs=8K --iodepth=64 --size=4G --filename=/vendor/random.data --readwrite=randread
erofs-8k-random-read: (groupid=0, jobs=1): err= 0: pid=6882: Thu Jun 1 23:57:37 2023
read: IOPS=3843, BW=30.0MiB/s (31.5MB/s)(4096MiB/136400msec)
bw ( KiB/s): min=10960, max=36950, per=100.00%, avg=30783.24, stdev=2159.16, samples=272
iops : min= 1370, max= 4618, avg=3847.67, stdev=269.91, samples=272
cpu : usr=3.26%, sys=28.92%, ctx=526278, majf=0, minf=134
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=524288,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=30.0MiB/s (31.5MB/s), 30.0MiB/s-30.0MiB/s (31.5MB/s-31.5MB/s), io=4096MiB (4295MB), run=136400-136400msec
Disk stats (read/write):
dm-4: ios=523795/0, merge=0/0, ticks=88620/0, in_queue=88620, util=99.92%, aggrios=524112/21, aggrmerge=213/0, aggrticks=98321/23, aggrin_queue=98346, aggrutil=99.89%
sda: ios=524112/21, merge=213/0, ticks=98321/23, in_queue=98346, util=99.89%
Sequential reads:
$ fio --readonly --ioengine=libaio --gtod_reduce=1 --name=erofs-16k-sequential-read --bs=16K --iodepth=64 --size=4G --filename=/vendor/random.data --readwrite=read
erofs-16k-sequential-read: (groupid=0, jobs=1): err= 0: pid=6985: Thu Jun 1 23:58:38 2023
read: IOPS=90.8k, BW=1419MiB/s (1488MB/s)(4096MiB/2886msec)
bw ( MiB/s): min= 397, max= 1665, per=97.78%, avg=1387.72, stdev=553.82, samples=5
iops : min=25466, max=106588, avg=88814.00, stdev=35444.35, samples=5
cpu : usr=3.12%, sys=74.84%, ctx=3701, majf=0, minf=261
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=262144,0,0,0 short=1,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=1419MiB/s (1488MB/s), 1419MiB/s-1419MiB/s (1488MB/s-1488MB/s), io=4096MiB (4295MB), run=2886-2886msec
Disk stats (read/write):
dm-4: ios=21084/0, merge=0/0, ticks=10680/0, in_queue=10680, util=84.20%, aggrios=23836/0, aggrmerge=0/0, aggrticks=12585/0, aggrin_queue=12586, aggrutil=83.20%
sda: ios=23836/0, merge=0/0, ticks=12585/0, in_queue=12586, util=83.20%
Random reads:
$ fio --readonly --ioengine=libaio --gtod_reduce=1 --name=erofs-16k-random-read --bs=16K --iodepth=64 --size=4G --filename=/vendor/random.data --readwrite=randread
erofs-16k-random-read: (groupid=0, jobs=1): err= 0: pid=7016: Fri Jun 2 00:00:48 2023
read: IOPS=3141, BW=49.1MiB/s (51.5MB/s)(4096MiB/83433msec)
bw ( KiB/s): min=18464, max=71025, per=99.96%, avg=50249.73, stdev=7559.55, samples=166
iops : min= 1154, max= 4439, avg=3140.51, stdev=472.50, samples=166
cpu : usr=3.02%, sys=40.57%, ctx=265568, majf=0, minf=262
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=262145,0,0,0 short=1,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=49.1MiB/s (51.5MB/s), 49.1MiB/s-49.1MiB/s (51.5MB/s-51.5MB/s), io=4096MiB (4295MB), run=83433-83433msec
Disk stats (read/write):
dm-4: ios=263693/0, merge=0/0, ticks=47332/0, in_queue=47332, util=99.78%, aggrios=274323/54, aggrmerge=356/0, aggrticks=56002/20, aggrin_queue=56034, aggrutil=99.73%
sda: ios=274323/54, merge=356/0, ticks=56002/20, in_queue=56034, util=99.73%
Sequential reads:
$ fio --readonly --ioengine=libaio --gtod_reduce=1 --name=erofs-2k-sequential-read --bs=2K --iodepth=64 --size=4G --filename=/vendor/random.data --readwrite=read
erofs-2k-sequential-read: (groupid=0, jobs=1): err= 0: pid=5470: Fri Jun 2 00:24:37 2023
read: IOPS=653k, BW=1275MiB/s (1337MB/s)(4096MiB/3213msec)
bw ( MiB/s): min= 1128, max= 1312, per=99.90%, avg=1273.50, stdev=71.25, samples=6
iops : min=577834, max=671904, avg=652030.67, stdev=36481.19, samples=6
cpu : usr=18.31%, sys=78.21%, ctx=544, majf=0, minf=38
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=2097152,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=1275MiB/s (1337MB/s), 1275MiB/s-1275MiB/s (1337MB/s-1337MB/s), io=4096MiB (4295MB), run=3213-3213msec
Disk stats (read/write):
dm-4: ios=4492/0, merge=0/0, ticks=2208/0, in_queue=2208, util=96.91%, aggrios=8708/0, aggrmerge=0/0, aggrticks=3466/0, aggrin_queue=3466, aggrutil=96.23%
sda: ios=8708/0, merge=0/0, ticks=3466/0, in_queue=3466, util=96.23%
Random reads:
$ fio --readonly --ioengine=libaio --gtod_reduce=1 --name=erofs-2k-random-read --bs=2K --iodepth=64 --size=4G --filename=/vendor/random.data --readwrite=randread
erofs-2k-random-read: (groupid=0, jobs=1): err= 0: pid=5487: Fri Jun 2 00:31:01 2023
read: IOPS=5659, BW=11.1MiB/s (11.6MB/s)(4096MiB/370585msec)
bw ( KiB/s): min= 6024, max=14836, per=100.00%, avg=11331.03, stdev=1193.36, samples=740
iops : min= 3012, max= 7418, avg=5665.52, stdev=596.68, samples=740
cpu : usr=3.48%, sys=25.91%, ctx=1555029, majf=1, minf=40
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=2097152,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=11.1MiB/s (11.6MB/s), 11.1MiB/s-11.1MiB/s (11.6MB/s-11.6MB/s), io=4096MiB (4295MB), run=370585-370585msec
Disk stats (read/write):
dm-4: ios=1548374/0, merge=0/0, ticks=248740/0, in_queue=248740, util=100.00%, aggrios=1549448/641, aggrmerge=347/0, aggrticks=275898/125, aggrin_queue=276085, aggrutil=100.00%
sda: ios=1549448/641, merge=347/0, ticks=275898/125, in_queue=276085, util=100.00%
Sequential reads:
$ fio --readonly --ioengine=libaio --gtod_reduce=1 --name=erofs-4k-sequential-read --bs=4K --iodepth=64 --size=4G --filename=/vendor/random.data --readwrite=read
erofs-4k-sequential-read: (groupid=0, jobs=1): err= 0: pid=5910: Fri Jun 2 00:31:40 2023
read: IOPS=368k, BW=1436MiB/s (1506MB/s)(4096MiB/2852msec)
bw ( MiB/s): min= 55, max= 1811, per=97.16%, avg=1395.42, stdev=750.56, samples=5
iops : min=14186, max=463742, avg=357226.80, stdev=192144.12, samples=5
cpu : usr=8.77%, sys=86.60%, ctx=353, majf=0, minf=71
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=1048576,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=1436MiB/s (1506MB/s), 1436MiB/s-1436MiB/s (1506MB/s-1506MB/s), io=4096MiB (4295MB), run=2852-2852msec
Disk stats (read/write):
dm-4: ios=3709/0, merge=0/0, ticks=2056/0, in_queue=2056, util=80.88%, aggrios=8207/13, aggrmerge=0/0, aggrticks=3666/3, aggrin_queue=3668, aggrutil=81.31%
sda: ios=8207/13, merge=0/0, ticks=3666/3, in_queue=3668, util=81.31%
Random reads:
$ fio --readonly --ioengine=libaio --gtod_reduce=1 --name=erofs-4k-random-read --bs=4K --iodepth=64 --size=4G --filename=/vendor/random.data --readwrite=randread
erofs-4k-random-read: (groupid=0, jobs=1): err= 0: pid=5935: Fri Jun 2 00:36:07 2023
read: IOPS=4280, BW=16.7MiB/s (17.5MB/s)(4096MiB/244940msec)
bw ( KiB/s): min= 8584, max=22624, per=100.00%, avg=17144.76, stdev=1050.34, samples=489
iops : min= 2146, max= 5656, avg=4286.13, stdev=262.58, samples=489
cpu : usr=3.02%, sys=25.29%, ctx=1052497, majf=0, minf=68
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=1048576,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=16.7MiB/s (17.5MB/s), 16.7MiB/s-16.7MiB/s (17.5MB/s-17.5MB/s), io=4096MiB (4295MB), run=244940-244940msec
Disk stats (read/write):
dm-4: ios=1048481/0, merge=0/0, ticks=167580/0, in_queue=167580, util=100.00%, aggrios=1048729/124, aggrmerge=341/0, aggrticks=185198/38, aggrin_queue=185267, aggrutil=100.00%
sda: ios=1048729/124, merge=341/0, ticks=185198/38, in_queue=185267, util=100.00%
Sequential reads:
$ fio --readonly --ioengine=libaio --gtod_reduce=1 --name=erofs-8k-sequential-read --bs=8K --iodepth=64 --size=4G --filename=/vendor/random.data --readwrite=read
erofs-8k-sequential-read: (groupid=0, jobs=1): err= 0: pid=6114: Fri Jun 2 00:36:52 2023
read: IOPS=207k, BW=1619MiB/s (1698MB/s)(4096MiB/2530msec)
bw ( MiB/s): min= 753, max= 1861, per=99.90%, avg=1617.42, stdev=483.78, samples=5
iops : min=96416, max=238223, avg=207029.80, stdev=61923.42, samples=5
cpu : usr=5.77%, sys=80.31%, ctx=1969, majf=0, minf=135
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=524288,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=1619MiB/s (1698MB/s), 1619MiB/s-1619MiB/s (1698MB/s-1698MB/s), io=4096MiB (4295MB), run=2530-2530msec
Disk stats (read/write):
dm-4: ios=3814/0, merge=0/0, ticks=2340/0, in_queue=2340, util=86.84%, aggrios=8195/0, aggrmerge=0/0, aggrticks=3772/0, aggrin_queue=3772, aggrutil=86.03%
sda: ios=8195/0, merge=0/0, ticks=3772/0, in_queue=3772, util=86.03%
Random reads:
$ fio --readonly --ioengine=libaio --gtod_reduce=1 --name=erofs-8k-random-read --bs=8K --iodepth=64 --size=4G --filename=/vendor/random.data --readwrite=randread
erofs-8k-random-read: (groupid=0, jobs=1): err= 0: pid=6132: Fri Jun 2 00:39:17 2023
read: IOPS=3997, BW=31.2MiB/s (32.7MB/s)(4096MiB/131154msec)
bw ( KiB/s): min=14096, max=39808, per=100.00%, avg=32007.98, stdev=2237.52, samples=261
iops : min= 1762, max= 4976, avg=4000.95, stdev=279.69, samples=261
cpu : usr=2.93%, sys=26.76%, ctx=526537, majf=0, minf=134
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=524288,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=31.2MiB/s (32.7MB/s), 31.2MiB/s-31.2MiB/s (32.7MB/s-32.7MB/s), io=4096MiB (4295MB), run=131154-131154msec
Disk stats (read/write):
dm-4: ios=523283/0, merge=0/0, ticks=88284/0, in_queue=88284, util=99.92%, aggrios=524572/279, aggrmerge=577/2, aggrticks=97027/83, aggrin_queue=97126, aggrutil=99.88%
sda: ios=524572/279, merge=577/2, ticks=97027/83, in_queue=97126, util=99.88%
Sequential reads:
$ fio --readonly --ioengine=libaio --gtod_reduce=1 --name=erofs-16k-sequential-read --bs=16K --iodepth=64 --size=4G --filename=/vendor/random.data --readwrite=read
erofs-16k-sequential-read: (groupid=0, jobs=1): err= 0: pid=6310: Fri Jun 2 00:40:09 2023
read: IOPS=110k, BW=1723MiB/s (1807MB/s)(4096MiB/2377msec)
bw ( MiB/s): min= 963, max= 1970, per=97.46%, avg=1679.48, stdev=478.47, samples=4
iops : min=61690, max=126080, avg=107486.50, stdev=30622.27, samples=4
cpu : usr=2.10%, sys=81.10%, ctx=2875, majf=0, minf=263
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=262144,0,0,0 short=1,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=1723MiB/s (1807MB/s), 1723MiB/s-1723MiB/s (1807MB/s-1807MB/s), io=4096MiB (4295MB), run=2377-2377msec
Disk stats (read/write):
dm-4: ios=4098/0, merge=0/0, ticks=2420/0, in_queue=2420, util=86.39%, aggrios=8195/0, aggrmerge=0/0, aggrticks=3784/0, aggrin_queue=3784, aggrutil=85.06%
sda: ios=8195/0, merge=0/0, ticks=3784/0, in_queue=3784, util=85.06%
Random reads:
$ fio --readonly --ioengine=libaio --gtod_reduce=1 --name=erofs-16k-random-read --bs=16K --iodepth=64 --size=4G --filename=/vendor/random.data --readwrite=randread
erofs-16k-random-read: (groupid=0, jobs=1): err= 0: pid=6327: Fri Jun 2 00:41:56 2023
read: IOPS=2830, BW=44.2MiB/s (46.4MB/s)(4096MiB/92601msec)
bw ( KiB/s): min=22464, max=49888, per=100.00%, avg=45316.67, stdev=3234.44, samples=184
iops : min= 1404, max= 3118, avg=2832.18, stdev=202.11, samples=184
cpu : usr=3.01%, sys=38.86%, ctx=264434, majf=0, minf=263
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=262145,0,0,0 short=1,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=44.2MiB/s (46.4MB/s), 44.2MiB/s-44.2MiB/s (46.4MB/s-46.4MB/s), io=4096MiB (4295MB), run=92601-92601msec
Disk stats (read/write):
dm-4: ios=261759/0, merge=0/0, ticks=52896/0, in_queue=52896, util=99.85%, aggrios=262252/268, aggrmerge=302/1, aggrticks=61197/169, aggrin_queue=61366, aggrutil=99.81%
sda: ios=262252/268, merge=302/1, ticks=61197/169, in_queue=61366, util=99.81%
Sequential reads:
$ fio --readonly --ioengine=libaio --gtod_reduce=1 --name=erofs-2k-sequential-read --bs=2K --iodepth=64 --size=4G --filename=/vendor/random.data --readwrite=read
erofs-2k-sequential-read: (groupid=0, jobs=1): err= 0: pid=5443: Fri Jun 2 01:39:16 2023
read: IOPS=739k, BW=1443MiB/s (1513MB/s)(4096MiB/2838msec)
bw ( MiB/s): min= 1197, max= 1516, per=99.63%, avg=1437.96, stdev=134.96, samples=5
iops : min=613202, max=776430, avg=736237.60, stdev=69100.23, samples=5
cpu : usr=18.93%, sys=78.75%, ctx=155, majf=0, minf=41
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=2097152,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=1443MiB/s (1513MB/s), 1443MiB/s-1443MiB/s (1513MB/s-1513MB/s), io=4096MiB (4295MB), run=2838-2838msec
Disk stats (read/write):
dm-4: ios=3864/0, merge=0/0, ticks=2080/0, in_queue=2080, util=96.52%, aggrios=8313/1, aggrmerge=0/0, aggrticks=3494/1, aggrin_queue=3494, aggrutil=95.53%
sda: ios=8313/1, merge=0/0, ticks=3494/1, in_queue=3494, util=95.53%
Random reads:
$ fio --readonly --ioengine=libaio --gtod_reduce=1 --name=erofs-2k-random-read --bs=2K --iodepth=64 --size=4G --filename=/vendor/random.data --readwrite=randread
erofs-2k-random-read: (groupid=0, jobs=1): err= 0: pid=5532: Fri Jun 2 01:44:47 2023
read: IOPS=6784, BW=13.3MiB/s (13.9MB/s)(4096MiB/309099msec)
bw ( KiB/s): min= 6528, max=16994, per=100.00%, avg=13583.32, stdev=1527.38, samples=617
iops : min= 3264, max= 8497, avg=6791.65, stdev=763.69, samples=617
cpu : usr=4.49%, sys=39.78%, ctx=1529127, majf=1, minf=40
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=2097152,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=13.3MiB/s (13.9MB/s), 13.3MiB/s-13.3MiB/s (13.9MB/s-13.9MB/s), io=4096MiB (4295MB), run=309099-309099msec
Disk stats (read/write):
dm-4: ios=1527094/0, merge=0/0, ticks=203880/0, in_queue=203880, util=100.00%, aggrios=1527269/758, aggrmerge=71/0, aggrticks=231108/183, aggrin_queue=231413, aggrutil=100.00%
sda: ios=1527269/758, merge=71/0, ticks=231108/183, in_queue=231413, util=100.00%
Sequential reads:
$ fio --readonly --ioengine=libaio --gtod_reduce=1 --name=erofs-4k-sequential-read --bs=4K --iodepth=64 --size=4G --filename=/vendor/random.data --readwrite=read
erofs-4k-sequential-read: (groupid=0, jobs=1): err= 0: pid=5938: Fri Jun 2 01:45:07 2023
read: IOPS=428k, BW=1673MiB/s (1754MB/s)(4096MiB/2449msec)
bw ( MiB/s): min= 975, max= 1897, per=97.78%, avg=1635.43, stdev=441.10, samples=4
iops : min=249674, max=485722, avg=418671.75, stdev=112919.96, samples=4
cpu : usr=9.72%, sys=77.98%, ctx=1847, majf=0, minf=70
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=1048576,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=1673MiB/s (1754MB/s), 1673MiB/s-1673MiB/s (1754MB/s-1754MB/s), io=4096MiB (4295MB), run=2449-2449msec
Disk stats (read/write):
dm-4: ios=3967/0, merge=0/0, ticks=2300/0, in_queue=2300, util=86.88%, aggrios=8196/0, aggrmerge=0/0, aggrticks=3736/0, aggrin_queue=3737, aggrutil=86.07%
sda: ios=8196/0, merge=0/0, ticks=3736/0, in_queue=3737, util=86.07%
Random reads:
$ fio --readonly --ioengine=libaio --gtod_reduce=1 --name=erofs-4k-random-read --bs=4K --iodepth=64 --size=4G --filename=/vendor/random.data --readwrite=randread
erofs-4k-random-read: (groupid=0, jobs=1): err= 0: pid=5958: Fri Jun 2 01:48:43 2023
read: IOPS=5096, BW=19.9MiB/s (20.9MB/s)(4096MiB/205764msec)
bw ( KiB/s): min= 9640, max=25104, per=100.00%, avg=20407.33, stdev=1177.24, samples=411
iops : min= 2410, max= 6276, avg=5101.77, stdev=294.31, samples=411
cpu : usr=3.89%, sys=37.39%, ctx=1049270, majf=0, minf=68
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=1048576,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=19.9MiB/s (20.9MB/s), 19.9MiB/s-19.9MiB/s (20.9MB/s-20.9MB/s), io=4096MiB (4295MB), run=205764-205764msec
Disk stats (read/write):
dm-4: ios=1048035/0, merge=0/0, ticks=135868/0, in_queue=135868, util=100.00%, aggrios=1048761/316, aggrmerge=158/0, aggrticks=156870/62, aggrin_queue=156957, aggrutil=99.97%
sda: ios=1048761/316, merge=158/0, ticks=156870/62, in_queue=156957, util=99.97%
Sequential reads:
$ fio --readonly --ioengine=libaio --gtod_reduce=1 --name=erofs-8k-sequential-read --bs=8K --iodepth=64 --size=4G --filename=/vendor/random.data --readwrite=read
erofs-8k-sequential-read: (groupid=0, jobs=1): err= 0: pid=6130: Fri Jun 2 01:49:07 2023
read: IOPS=221k, BW=1723MiB/s (1807MB/s)(4096MiB/2377msec)
bw ( MiB/s): min= 1035, max= 1978, per=97.98%, avg=1688.32, stdev=439.50, samples=4
iops : min=132604, max=253294, avg=216105.25, stdev=56255.50, samples=4
cpu : usr=7.70%, sys=70.16%, ctx=3026, majf=0, minf=133
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=524288,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=1723MiB/s (1807MB/s), 1723MiB/s-1723MiB/s (1807MB/s-1807MB/s), io=4096MiB (4295MB), run=2377-2377msec
Disk stats (read/write):
dm-4: ios=3625/0, merge=0/0, ticks=2372/0, in_queue=2372, util=85.46%, aggrios=8195/0, aggrmerge=0/0, aggrticks=4104/0, aggrin_queue=4105, aggrutil=86.29%
sda: ios=8195/0, merge=0/0, ticks=4104/0, in_queue=4105, util=86.29%
Random reads:
$ fio --readonly --ioengine=libaio --gtod_reduce=1 --name=erofs-8k-random-read --bs=8K --iodepth=64 --size=4G --filename=/vendor/random.data --readwrite=randread
erofs-8k-random-read: (groupid=0, jobs=1): err= 0: pid=6147: Fri Jun 2 01:51:07 2023
read: IOPS=4792, BW=37.4MiB/s (39.3MB/s)(4096MiB/109397msec)
bw ( KiB/s): min=18944, max=46896, per=100.00%, avg=38395.13, stdev=2211.88, samples=218
iops : min= 2368, max= 5862, avg=4799.25, stdev=276.53, samples=218
cpu : usr=3.69%, sys=39.15%, ctx=525278, majf=0, minf=141
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=524288,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=37.4MiB/s (39.3MB/s), 37.4MiB/s-37.4MiB/s (39.3MB/s-39.3MB/s), io=4096MiB (4295MB), run=109397-109397msec
Disk stats (read/write):
dm-4: ios=523636/0, merge=0/0, ticks=71572/0, in_queue=71572, util=99.86%, aggrios=524342/44, aggrmerge=242/0, aggrticks=81866/12, aggrin_queue=81879, aggrutil=99.82%
sda: ios=524342/44, merge=242/0, ticks=81866/12, in_queue=81879, util=99.82%
Sequential reads:
$ fio --readonly --ioengine=libaio --gtod_reduce=1 --name=erofs-16k-sequential-read --bs=16K --iodepth=64 --size=4G --filename=/vendor/random.data --readwrite=read
erofs-16k-sequential-read: (groupid=0, jobs=1): err= 0: pid=6275: Fri Jun 2 01:51:46 2023
read: IOPS=111k, BW=1735MiB/s (1819MB/s)(4096MiB/2361msec)
bw ( MiB/s): min= 961, max= 1974, per=98.38%, avg=1706.78, stdev=496.87, samples=4
iops : min=61562, max=126368, avg=109234.00, stdev=31799.82, samples=4
cpu : usr=6.57%, sys=72.63%, ctx=2953, majf=0, minf=261
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=262144,0,0,0 short=1,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=1735MiB/s (1819MB/s), 1735MiB/s-1735MiB/s (1819MB/s-1819MB/s), io=4096MiB (4295MB), run=2361-2361msec
Disk stats (read/write):
dm-4: ios=3663/0, merge=0/0, ticks=2344/0, in_queue=2344, util=84.82%, aggrios=8338/0, aggrmerge=10/0, aggrticks=4085/0, aggrin_queue=4085, aggrutil=86.47%
sda: ios=8338/0, merge=10/0, ticks=4085/0, in_queue=4085, util=86.47%
Random reads:
$ fio --readonly --ioengine=libaio --gtod_reduce=1 --name=erofs-16k-random-read --bs=16K --iodepth=64 --size=4G --filename=/vendor/random.data --readwrite=randread
erofs-16k-random-read: (groupid=0, jobs=1): err= 0: pid=6293: Fri Jun 2 01:52:57 2023
read: IOPS=4396, BW=68.7MiB/s (72.0MB/s)(4096MiB/59625msec)
bw ( KiB/s): min=26656, max=80000, per=100.00%, avg=70401.97, stdev=5087.59, samples=119
iops : min= 1666, max= 5000, avg=4400.03, stdev=318.01, samples=119
cpu : usr=3.42%, sys=40.68%, ctx=262655, majf=0, minf=262
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=262145,0,0,0 short=1,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=68.7MiB/s (72.0MB/s), 68.7MiB/s-68.7MiB/s (72.0MB/s-72.0MB/s), io=4096MiB (4295MB), run=59625-59625msec
Disk stats (read/write):
dm-4: ios=261037/0, merge=0/0, ticks=38772/0, in_queue=38772, util=99.59%, aggrios=262170/3, aggrmerge=0/0, aggrticks=43372/0, aggrin_queue=43373, aggrutil=99.51%
sda: ios=262170/3, merge=0/0, ticks=43372/0, in_queue=43373, util=99.51%