Created
November 18, 2022 23:44
-
-
Save mdcallag/bcfcc8213357ab36b9077934c35977fa to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Server: | |
* Explained at http://smalldatum.blogspot.com/2018/12/new-small-servers-for-performance.html | |
* Intel NUC with 4 cores, hyperthread disabled, 16G RAM | |
* Samsung 970 EVO | |
* Ubuntu 22.04, 5.15.0-52-generic kernel | |
* fio used 8 files, 20G per file for dir and buf with XFS filesystem | |
Example command lines, repeated for numjobs in 1 2 4 8 16 32 48 64 | |
raw = no filesystem | |
fio --filename=/dev/nvme0n1 --direct=1 --rw=randread --bs=4096 --ioengine=psync --iodepth=1 --runtime=600 --numjobs=1 --time_based --group_reporting --name=iops-test-job --eta-newline=1 --eta-interval=1 --randseed=1668657773 --readonly --eta=always | |
dir = O_DIRECT | |
fio --filename=/data/m/fio.in.1:/data/m/fio.in.2:/data/m/fio.in.3:/data/m/fio.in.4:/data/m/fio.in.5:/data/m/fio.in.6:/data/m/fio.in.7:/data/m/fio.in.8 --filesize=20480m --rw=randread --bs=4096 --ioengine=psync --iodepth=1 --time_based --group_reporting --name=iops-test-job --eta-newline=1 --eta-interval=1 --randrepeat=0 --norandommap --invalidate=0 --eta=always --direct=1 --numjobs=1 --runtime=600 | |
buf = buffered IO | |
fio --filename=/data/m/fio.in.1:/data/m/fio.in.2:/data/m/fio.in.3:/data/m/fio.in.4:/data/m/fio.in.5:/data/m/fio.in.6:/data/m/fio.in.7:/data/m/fio.in.8 --filesize=20480m --rw=randread --bs=4096 --ioengine=psync --iodepth=1 --time_based --group_reporting --name=iops-test-job --eta-newline=1 --eta-interval=1 --randrepeat=0 --norandommap --invalidate=0 --eta=always --direct=0 --fadvise_hint=random --numjobs=1 --runtime=600 | |
IOPs per fio | |
1 2 4 8 16 32 48 64 <- njobs | |
24470 49785 97255 163767 298737 439166 499544 498873 fio.4kb.raw.psync | |
9715 19123 38309 86112 143291 207611 237557 252481 fio.4kb.dir.psync | |
9242 19478 42212 92064 148756 223649 255541 272594 fio.4kb.buf.psync | |
- | |
22873 44656 85435 139814 229258 316238 351337 366243 fio.8kb.raw.psync | |
9116 17612 33060 66826 97391 121228 129733 133908 fio.8kb.dir.psync | |
8675 17029 36450 71135 105196 131658 141951 146795 fio.8kb.buf.psync | |
response time per fio - usecs | |
1 2 4 8 16 32 48 64 <- njobs | |
40.2 39.5 40.4 47.9 52.7 71.8 95.0 127.0 fio.4kb.raw.psync | |
100.8 102.2 102.6 92.2 111.0 153.5 201.4 252.9 fio.4kb.dir.psync | |
105.7 100.8 93.2 86.3 106.8 142.4 187.0 234.1 fio.4kb.buf.psync | |
- | |
43.2 43.3 46.2 56.4 68.8 100.2 135.7 173.7 fio.8kb.raw.psync | |
107.9 111.5 119.2 119.0 163.5 263.3 369.4 477.5 fio.8kb.dir.psync | |
113.4 115.4 108.4 111.7 151.4 242.4 337.6 435.4 fio.8kb.buf.psync | |
vmstat us+sy | |
1 2 4 8 16 32 48 64 <- njobs | |
3.2 6.0 12.2 28.8 59.6 93.1 98.9 99.9 fio.4kb.raw.psync | |
5.8 11.3 21.0 14.8 28.8 46.8 54.9 58.2 fio.4kb.dir.psync | |
7.4 15.1 23.6 22.5 44.4 67.4 80.0 84.2 fio.4kb.buf.psync | |
- | |
3.2 5.9 11.5 25.1 48.8 73.4 83.4 87.7 fio.8kb.raw.psync | |
5.6 11.2 20.0 12.3 21.0 27.6 29.4 31.7 fio.8kb.dir.psync | |
8.3 17.6 24.0 22.9 37.0 50.9 54.3 57.1 fio.8kb.buf.psync | |
cpu_usecs per operation | |
1 2 4 8 16 32 48 64 <- njobs | |
6.0 5.9 5.9 8.7 7.8 8.2 7.9 8.0 fio.4kb.raw.psync | |
28.1 28.4 25.1 8.0 8.9 9.4 9.3 9.2 fio.4kb.dir.psync | |
38.9 36.1 25.3 10.9 13.6 12.1 12.5 12.3 fio.4kb.buf.psync | |
- | |
6.4 6.2 6.3 8.8 8.7 8.9 9.1 9.2 fio.8kb.raw.psync | |
29.2 30.1 28.2 8.6 9.6 9.9 9.9 10.3 fio.8kb.dir.psync | |
43.1 45.3 28.5 13.7 14.6 16.4 15.9 15.9 fio.8kb.buf.psync | |
iostat r_await in usecs | |
1 2 4 8 16 32 48 64 <- njobs | |
40 39 40 40 45 50 60 60 fio.4kb.raw.psync | |
80 80 85 90 100 148 190 241 fio.4kb.dir.psync | |
81 81 81 90 100 140 180 230 fio.4kb.buf.psync | |
- | |
40 40 40 50 60 90 120 160 fio.8kb.raw.psync | |
90 90 100 110 160 260 360 470 fio.8kb.dir.psync | |
90 90 100 110 150 250 350 452 fio.8kb.buf.psync | |
iostat reads per query | |
1 2 4 8 16 32 48 64 <- njobs | |
~1 ~1 ~1 ~1 ~1 ~1 ~1 ~1 fio.4kb.raw.psync | |
~1 ~1 ~1 ~1 ~1 ~1 ~1 ~1 fio.4kb.dir.psync | |
~1 ~1 ~1 ~1 ~1 ~1 ~1 ~1 fio.4kb.buf.psync | |
- | |
~1 ~1 ~1 ~1 ~1 ~1 ~1 ~1 fio.8kb.raw.psync | |
~1 ~1 ~1 ~1 ~1 ~1 ~1 ~1 fio.8kb.dir.psync | |
~1 ~1 ~1 ~1 ~1 ~1 ~1 ~1 fio.8kb.buf.psync | |
iostat rareq-sz in KB | |
1 2 4 8 16 32 48 64 <- njobs | |
4.0 4.0 4.0 4.0 4.0 4.0 4.0 4.0 fio.4kb.raw.psync | |
4.0 4.0 4.0 4.0 4.0 4.0 4.0 4.0 fio.4kb.dir.psync | |
4.0 4.0 4.0 4.0 4.0 4.0 4.0 4.0 fio.4kb.buf.psync | |
- | |
8.0 8.0 8.0 8.0 8.0 8.0 8.0 8.0 fio.8kb.raw.psync | |
8.0 8.0 8.0 8.0 8.0 8.0 8.0 8.0 fio.8kb.dir.psync | |
7.8 8.0 8.0 8.0 8.0 8.0 8.0 8.0 fio.8kb.buf.psync | |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment