Skip to content

Instantly share code, notes, and snippets.

@bassu

bassu/zfsonlinux-benchmark

Forked from anonymous/zfsonlinux-benchmark
Created October 20, 2013 20:15
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save bassu/7074736 to your computer and use it in GitHub Desktop.
Save bassu/7074736 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
zfsonlinux-benchmark
ZFSOnLinux Benchmark On Stock Hardware
======================================
Hardware
--------
- Supermicro X8DTL
- 92GB RAM
- LSI 9211-4i HBA
- WD3000F9YZ 6Gbps 3TB x 4 HDDs
Software
--------
- CentOS 6.4
- Md software raid for / & /boot
- Xen 4.3.2 Hypervisor Dom0
- 4 pinned cpu cores
- 16GB dedicated RAM
- ZFSOnLinux 0.6.2-1
- pool: dual mirror
- zil & l2arc: no
- compression: lz4
- ashift: default (automatic to 12)
- arc_max: 12GB
Tests from DomUs are listed first and then tests from the Dom0.
------------------------------------------------------------------------------
DomU (Windows) I/O on ZVOL (via CrystalDiskMark3)
------------------------------------------------------------------------------
Sequential Read : 1080.370 MB/s
Sequential Write : 236.033 MB/s
Random Read 512KB : 853.167 MB/s
Random Write 512KB : 256.090 MB/s
Random Read 4KB (QD=1) : 35.507 MB/s [ 8668.8 IOPS]
Random Write 4KB (QD=1) : 27.543 MB/s [ 6724.3 IOPS]
Random Read 4KB (QD=32) : 205.050 MB/s [ 50060.9 IOPS]
Random Write 4KB (QD=32) : 113.603 MB/s [ 27735.2 IOPS]
Test : 1000 MB
Date : 2013/10/20 5:15:42
OS : Windows Server 2008 R2
---------------------------------------------------------------------------------
DomU (Linux) I/0 on ZVOL (via iozone)
---------------------------------------------------------------------------------
Machine = Linux localhost.localdomain 2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22
Auto Mode
Using minimum file size of 1024 kilobytes.
Using maximum file size of 1048576 kilobytes.
Record Size 4 KB
Record Size 512 KB
Include fsync in write timing
Command line used: iozone -M -a -n1M -g1G -r4k -r512k -e -f /root/iozone.dat -i0 -i1 -i2
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
random random
KB reclen write rewrite read reread read write
1024 4 264667 334869 1002839 2439205 1349004 299409
1024 512 336285 437422 1244266 7644933 7257394 434809
2048 4 131867 386780 1104558 1655474 1026055 286428
2048 512 169480 558193 2029850 8498106 8357547 557288
4096 4 245007 339914 1281620 1665715 1299948 428224
4096 512 513033 652561 438588 8360824 8426437 271675
8192 4 283400 505455 1961641 2540122 1660628 452823
8192 512 406509 675909 3574371 6912307 6809564 670318
16384 4 197714 385315 2105646 2303100 1577516 453738
16384 512 555862 816176 4277907 5555420 5714334 712962
32768 4 256758 493070 2178845 2265621 1502840 489601
32768 512 637771 819572 4763451 5384103 5251212 797918
65536 4 264872 509916 2221705 2277443 1485306 499322
65536 512 691929 837703 5074843 5386403 5280040 855249
131072 4 288877 556620 2248997 2281259 1466293 479225
131072 512 624199 512963 4003295 5271525 5246221 467255
262144 4 321606 558223 2242597 2266133 1437477 543872
262144 512 172833 851352 5346498 5404471 5332676 933179
524288 4 205219 492180 2035526 2235932 1394561 255852
524288 512 114866 633309 4621390 5240938 5144361 409208
1048576 4 126998 212212 445797 720175 139256 145422
1048576 512 97883 325383 477395 803282 1482268 360152
---------------------------------------------------------------------------------
Dom0 I/0 on root mdraid (via iozone)
---------------------------------------------------------------------------------
Machine = Linux newton 3.4.61-9.el6.centos.alt.x86_64 #1 SMP Wed Sep 11 15:34
Using minimum file size of 1024 kilobytes.
Using maximum file size of 1048576 kilobytes.
Record Size 4 KB
Record Size 512 KB
Include fsync in write timing
Command line used: iozone -M -a -n1M -g1G -r4k -r512k -e -f /root/iozone.dat -i0 -i1 -i2
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
random random
KB reclen write rewrite read reread read write
1024 4 20442 32147 39256 99941 680357 28055
1024 512 18076 24587 69512 144408 2159571 22564
2048 4 40883 55315 62385 145341 510582 53069
2048 512 54375 66660 81616 188166 114266 51798
4096 4 81756 88476 207194 219849 169768 86298
4096 512 81960 77613 150096 3583764 4153935 97380
8192 4 104176 117404 338273 425403 505276 121101
8192 512 98306 110438 342004 494927 374408 123569
16384 4 122804 137004 466914 493387 530776 143554
16384 512 151217 135605 431804 859919 1159845 81058
32768 4 144757 184742 670637 1133210 752666 140972
32768 512 157335 186509 971507 1949438 1403701 191059
65536 4 152841 186660 942625 991394 831600 165052
65536 512 227649 190177 1257891 1899918 2104492 235915
131072 4 163074 198676 1102418 1277691 955947 175986
131072 512 235330 255822 1551935 1685381 2500517 220444
262144 4 168213 219648 1235206 1669770 1125409 214867
262144 512 183786 248056 2797306 3812509 3749466 248667
524288 4 179997 231367 1241655 1695160 1000137 212983
524288 512 221925 271777 2694541 3820562 3633573 266922
1048576 4 214323 253000 1641259 1615256 811242 200568
1048576 512 267144 298299 3232912 4046648 3945190 292888
---------------------------------------------------------------------------------
Dom0 I/0 on a mount ZFS dataset (via iozone)
---------------------------------------------------------------------------------
Machine = Linux newton 3.4.61-9.el6.centos.alt.x86_64 #1 SMP Wed Sep 11 15:34
Using minimum file size of 1024 kilobytes.
Using maximum file size of 1048576 kilobytes.
Record Size 4 KB
Record Size 512 KB
Include fsync in write timing
Command line used: iozone -M -a -n1M -g1G -r4k -r512k -e -f /vms/tgrounds/iozone.dat -i0 -i1 -i2
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
random random
KB reclen write rewrite read reread read write
1024 4 14694 50398 440427 427198 376493 49054
1024 512 85740 53587 4654248 5690162 5507740 71567
2048 4 64597 77876 658343 451488 374259 76332
2048 512 176265 160176 5184636 5836323 5674397 174909
4096 4 80613 89325 440952 447161 385901 94799
4096 512 260479 319602 5696610 5736555 5512011 220108
8192 4 80568 124576 430795 564071 572902 111443
8192 512 334669 350624 4943985 5025700 5329730 558079
16384 4 114513 127273 471033 628897 551203 131598
16384 512 409108 551371 4013799 4724349 5100984 569918
32768 4 126116 142484 517881 632796 545079 138714
32768 512 463008 652022 4162066 4460593 4418008 604005
65536 4 135938 137858 626359 615597 526208 134542
65536 512 1104784 1121846 4355687 4435603 4402497 1084045
131072 4 128002 145508 624411 631660 492544 150026
131072 512 1021486 1205269 4344769 4368903 4304086 1176557
262144 4 135035 152051 515474 582577 426706 138092
262144 512 1267082 1319155 4378992 4384632 4290830 1363238
524288 4 159906 148497 554127 578796 430724 190973
524288 512 1359947 1575726 4364563 4372747 4286128 1454666
1048576 4 172347 166865 558197 617659 437586 227524
1048576 512 1573535 1495414 4310974 4374042 4304944 1505186
---------------------------------------------------------------------------------
Dom0 I/0 on a ZVOL, formated and mounted as ext4 filesystem (via iozone)
---------------------------------------------------------------------------------
Machine = Linux newton 3.4.61-9.el6.centos.alt.x86_64 #1 SMP Wed Sep 11 15:34
Using minimum file size of 1024 kilobytes.
Using maximum file size of 1048576 kilobytes.
Record Size 4 KB
Record Size 512 KB
Include fsync in write timing
Command line used: iozone -M -a -n1M -g1G -r4k -r512k -e -f /mnt/iozone.dat -i0 -i1 -i2
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
random random
KB reclen write rewrite read reread read write
1024 4 41917 44381 53685 1442326 984676 39736
1024 512 50902 52889 40434 3880507 4282950 22956
2048 4 63541 86162 119131 1519319 995610 71451
2048 512 85880 84996 120698 4875642 4889518 89852
4096 4 71988 86966 239207 1535177 1010829 64299
4096 512 106070 94438 242438 5038371 4971306 97924
8192 4 86509 89139 340438 1483785 998777 106464
8192 512 112820 111387 490686 4207758 4483963 126197
16384 4 110616 127857 956112 1364407 971591 125495
16384 512 114803 129422 983029 3328656 3534078 134785
32768 4 127171 138037 968290 1492234 1265810 134293
32768 512 137772 138352 1296926 3934718 3357672 134195
65536 4 123433 140662 1114102 1636306 1248379 143645
65536 512 151805 155298 1958059 3474681 3942212 150967
131072 4 129927 155842 510522 1355872 1101788 133933
131072 512 105606 134264 1416798 3760381 3994279 170225
262144 4 94948 139797 1650512 1838264 1194386 145374
262144 512 144398 86350 1950405 3262734 3625977 74683
524288 4 151150 122921 805663 1188354 1105941 125015
524288 512 115066 113725 2866933 2851915 798176 143994
1048576 4 142273 149417 973214 1749483 1145239 153193
1048576 512 138190 161120 2736467 2559505 2423366 159748
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment