mdcallag

-- 250m rows, 1 collections, 16 threads, mongo, in-memory

Command line:
bash iq.sh wt "" $bd/bin/mongo $bd/data md2 1 16 yes no yes no no 250000000 no

- load mongo
ips.av  ips.75  ips.95  ips.99  r/i     rkb/i   wkb/i   Mcpu/i  size    rss     r/s     rmb/s   wmb/s   cpu     max-i   engine
 11716    673     645     642   0       0       20.94   371     125     121.3   0       0       245.3    4.4    1.047   mm.m365
 12399    707     677     671   0       0       20.74   349     124     121.3   0       0       257.2    4.3    0.952   mm.m400
 45005   2890     511     440   0       0        3.76   770      95      61.0   0       0       169.0   34.7    7.651   wt.m365.v1-180

mdcallag

-- 250m rows, 16 collections, 16 threads, mongo, in-memory

Command line:
bash iq.sh wt "" $bd/bin/mongo $bd/data md2 1 16 yes no no no no 250000000 no

Configuration summary:
* must tune WiredTiger beyond cacheSizeGB for great performance
* best average insert performance comes with eviction_dirty_trigger/target set to >= 80. But this comes at a cost.
    First, mongod RSS will be much larger than cacheSizeGB. Second, write stalls (p99 write time) are worse.
* mongod RSS can almost be predicted by cacheSizeGB * ((100 + eviction_dirty_trig)/100)

mdcallag

tests used: 
* server has 56 core with HT, 256gb RAM, Linux 4.6.7-73, 3 fast SSD w/ 5tb
* insert benchmark - in-memory and io-bound, http://smalldatum.blogspot.com/2017/06/the-insert-benchmark.html

legend for load:
* ips.av - average inserts/second, global 
* ips.75, ips.95, ips.99 - inserts/second at 75th, 95th and 99th percentile, per insert client
* r/i, rkb/i, wkb/i - reads/insert, read KB/insert, write KB/insert, from iostat
* Mcpu/i - normalized CPU overhead/insert
* size - database size in GB at test end 

mdcallag

tests used:
* server has 56-core, 256gb RAM, Linux 4.6.7-73, 3 SSD w/ 5tb
* insert benchmark - in-memory and io-bound, http://smalldatum.blogspot.com/2017/06/the-insert-benchmark.html

legend:
* ips.av - average inserts/second
* ips.75, ips.95, ips.99 - inserts/second at 75th, 95th and 99th percentile
* r/i, rkb/i, wkb/i - reads/insert, read KB/insert, write KB/insert, from iostat
* Mcpu/i - normalized CPU overhead/insert
* size - database size in GB at test end

mdcallag

--- linkbench id1=1b, 28 clients

- load, ldop=4
ips     r/i     rkb/i   wkb/i   Mcpu/i  size    rss     r/s     rMB/s   wMB/s   cpu     engine
171840  0       0       1.44    86      693      3.5      5      0.5    247.4   14.9    myrocks.16jun17.none
172902  0       0       1.43    85      693      5.4      3      0.2    247.5   14.7    myrocks.8may18.none
175002  0       0.31    1.36    83      693     12.1    431     53.8    238.2   14.5    myrocks.8may18.none.dio
172326  0       0       1.81    94      333      3.4      8      0.8    312.8   16.3    myrocks.16jun17.zstd
172287  0       0       1.81    92      331      3.5      8      0.8    312.0   15.8    myrocks.8may18.zstd
172520  0       0.48    1.71    89      332      3.5    663     83.0    294.9   15.4    myrocks.8may18.zstd.dio

mdcallag

scan
pre     post    pre     post    pre     post
secs    secs    Mrps    Mrps    Mcpu/o  Mcpu/o  engine
29      33      2.758   2.424   1.050   1.325   myrocks.8may18.none
28      35      2.857   2.285   1.150   1.400   myrocks.8may18.none.fix
17      16      4.705   5.000   1.375   1.275   inno.o5717.ahi
17      17      4.705   4.705   1.550   1.250   inno.o5717.noahi

update-inlist
1       2       4       8       16      24      32      40      48      64

mdcallag

scan
pre     post    pre     post    pre     post
secs    secs    Mrps    Mrps    Mcpu/o  Mcpu/o  engine
29      33      2.758   2.424   1.050   1.325   myrocks.8may18.none
28      35      2.857   2.285   1.150   1.400   myrocks.8may18.none.fix
17      16      4.705   5.000   1.375   1.275   inno.o5717.ahi
17      17      4.705   4.705   1.550   1.250   inno.o5717.noahi

update-inlist
1       2       4       8       16      24      32      40      48      64

mdcallag

sysbench
* 8 tables, 100M rows/table
* more details at http://smalldatum.blogspot.com/2017/02/using-modern-sysbench-to-compare.html
* server changed to have ~56gb of RAM, 56 cores with HT enabled, fast SSD
* InnoDB has 35gb buffer pool, MyRocks has 30gb block cache

Summary for impact of direct IO
* full scan with O_DIRECT is more than 4X slower for MyRocks
* read rate from storage (rMB/s) is 2X to 3X faster without O_DIRECT
* many tests get 10% to 20% less QPS with O_DIRECT

mdcallag

Insert benchmark with 2b rows, 16 tables, 16 clients
* server changed to have ~56gb of RAM, 56 cores with HT enabled, fast SSD
* InnoDB has 35gb buffer pool, MyRocks has 30gb block cache

Insert benchmark has 4 steps - see http://smalldatum.blogspot.com/2017/06/the-insert-benchmark.html
1) concurrently load 16 tables with 1 client/table
2) concurrently scan 16 tables with 1 client/table - scan is done for PK, then each secondary index, then PK index again
3) OLTP using 16 write clients and 16 read clients (1 read/write pair per table), each writer does 1000 inserts/second
3) OLTP using 16 write clients and 16 read clients (1 read/write pair per table), each writer does 100 inserts/second

mdcallag

CREATE TABLE `sbtest1` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `k` int(11) NOT NULL DEFAULT '0',
  `c` char(120) NOT NULL DEFAULT '',
  `pad` char(60) NOT NULL DEFAULT '',
  KEY `xid` (`id`),
  KEY `k_1` (`k`)
) ENGINE=InnoDB AUTO_INCREMENT=2000001 DEFAULT CHARSET=latin1;