The tests
fio full write:
fio --name=writefile --size=100G --filesize=100G --filename=/dev/sdX --bs=1M --nrfiles=1 --direct=1 --sync=0 --randrepeat=0 --rw=write --refill_buffers --end_fsync=1 --iodepth=200 --ioengine=libaio
fio random read:
fio --time_based --name=benchmark --size=100G --runtime=30 --filename=/dev/sdX --ioengine=libaio --randrepeat=0 --iodepth=128 --direct=1 --invalidate=1 --verify=0 --verify_fatal=0 --numjobs=4 --rw=randread --blocksize=4k --group_reporting
fio random write:
fio --time_based --name=benchmark --size=100G --runtime=30 --filename=/dev/sdX --ioengine=libaio --randrepeat=0 --iodepth=128 --direct=1 --invalidate=1 --verify=0 --verify_fatal=0 --numjobs=4 --rw=randwrite --blocksize=4k --group_reporting
The results
| Description | fio full write | fio random read | fio random write |
|---|---|---|---|
| 1x Intel P3700 400GB raw | io=102400MB | io=46874MB | io=12323MB |
| bw=356306KB/s | bw=1562.4MB/s | bw=420533KB/s | |
| iops=347 | iops=399962 | iops=105133 | |
| runt=294291msec | runt=30002msec | runt=30006msec | |
| 2x WD 1TB HW RAID-0 | io=102400MB | io=66832KB | io=72444KB |
| w/ Writeback caching | bw=262623KB/s | bw=2185.6KB/s | bw=2395.9KB/s |
| iops=256 | iops=546 | iops=598 | |
| runt=399271msec | runt=30586msec | runt=30247msec | |
| Openstack VM | io=163840MB | io=258504KB | io=181976KB |
| Backed by HW RAID-10 SSD | bw=786872KB/s | bw=8609.4KB/s | bw=6059.9KB/s |
| local storage w/ Writeback | iops=768 | iops=2152 | iops=1514 |
| caching | runt=213214msec | runt=30026msec | runt=30030msec |
| krbd with librbd | io=102400MB | io=1049.1MB | io=259704KB |
| caching | bw=341576KB/s | bw=35478KB/s | bw=5592.7KB/s |
| iops=333 | iops=8869 | iops=1398 | |
| runt=306982msec | runt=30304msec | runt=46437msec | |
| krbd + LIO iscsi | io=102400MB | io=95092KB | io=58096KB |
| bw=274826KB/s | bw=3112.9KB/s | bw=1890.9KB/s | |
| iops=268 | iops=778 | iops=472 | |
| runt=381542msec | runt=30548msec | runt=30725msec |