gistfile1.txt

Summary:
* kernels have similar performance
* ext4 vs xfs are close with a few exceptions
  ** ext4 is ~12% faster on read-only with 4 threads
  ** ext4 is ~6% faster on read-write with 4 threads and nobarrier
  ** xfs is ~6% faster on write-only with 4 threads !nobarrier
* dioread_nolock has a small impact
* nobarrier has a big impact

Results from the i3 NUC
* details on servers at http://smalldatum.blogspot.com/2017/05/small-servers-for-database-performance.html
* NUC5i3ryh with 8gb RAM, Samsung 850 EVO SSD that uses SATA interface
* OS is Ubuntu 16.04 -- 4.4.0-38 and 4.4.0-109. Meltdown fix is in 4.4.0-109 but not in 4.4.0-38. Tests
    for 4.4.0-109 are run with fix enabled (pti=on) and disabled (pti=off)
* Server CPU supports PCID. I think the OS uses it.

Tests are run:
* for 1 and 4 client threads
* for ext4 with and without nobarrier, with and without dioread_nolock
* for xfs with and without nobarrier
* the 4.4 kernel does not have /sys/block/$dev/queue/write_cache 

Filesystem mount options:
UUID=...	/data	xfs	noatime,nodiratime,discard,noauto	0 1
UUID=...	/data	xfs	noatime,nodiratime,discard,noauto,nobarrier	0 1
UUID=... 	/data	ext4	data=ordered,discard,noauto	0 1
UUID=... 	/data	ext4	data=ordered,discard,noauto,dioread_nolock	0 1
UUID=... 	/data	ext4	nobarrier,data=ordered,discard,noauto	0 1
UUID=... 	/data	ext4	nobarrier,data=ordered,discard,noauto,dioread_nolock	0 1

Legend:
* numbers are MB/s throughput
* 1t, 4t - 1 and 4 client threads
* ext4, xfs - file system
* dioread_nolock, nobarrier - filesystem mount options

- rndrd - read-only
4.4     4.4     4.4
pti=on  pti=off no-pti
117     120     116     1t,xfs,nobarrier
116     120     115     1t,xfs
118     119     117     1t,ext4
118     118     118     1t,ext4,dioread_nolock
119     118     121     1t,ext4,nobarrier
116     123     116     1t,ext4,nobarrier,dioread_nolock
-
227     230     240     4t,xfs,nobarrier
240     230     253     4t,xfs
276     276     276     4t,ext4
283     284     283     4t,ext4,dioread_nolock
277     277     280     4t,ext4,nobarrier
281     280     283     4t,ext4,nobarrier,dioread_nolock

- rndrw - read-write
4.4     4.4     4.4
pti=on  pti=off no-pti
 90      91      85     1t,xfs,nobarrier
 83      85      84     1t,xfs
 88      85      86     1t,ext4
 86      84      87     1t,ext4,dioread_nolock
 94      90      88     1t,ext4,nobarrier
 89      89      90     1t,ext4,nobarrier,dioread_nolock
-
169     180     163     4t,xfs,nobarrier
138     137     138     4t,xfs
131     134     130     4t,ext4
140     135     135     4t,ext4,dioread_nolock
167     176     173     4t,ext4,nobarrier
185     173     171     4t,ext4,nobarrier,dioread_nolock

- rndwr - write-only
4.4     4.4     4.4
pti=on  pti=off no-pti
197     199     196     1t,xfs,nobarrier
115     113     114     1t,xfs
114     112     116     1t,ext4
112     113     111     1t,ext4,dioread_nolock
196     195     191     1t,ext4,nobarrier
190     194     190     1t,ext4,nobarrier,dioread_nolock
-
200     209     214     4t,xfs,nobarrier
128     131     128     4t,xfs
111     112     111     4t,ext4
121     118     121     4t,ext4,dioread_nolock
187     192     192     4t,ext4,nobarrier
206     208     209     4t,ext4,nobarrer,dioread_nolock