Skip to content

Instantly share code, notes, and snippets.

@mdcallag
Last active July 14, 2018 21:27
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save mdcallag/17b2998fb31b0929874d15a05e2c22a0 to your computer and use it in GitHub Desktop.
Save mdcallag/17b2998fb31b0929874d15a05e2c22a0 to your computer and use it in GitHub Desktop.
tests used:
* server has 56-core, 256gb RAM, Linux 4.6.7-73, 3 SSD w/ 5tb
* insert benchmark - in-memory and io-bound, http://smalldatum.blogspot.com/2017/06/the-insert-benchmark.html
legend:
* ips.av - average inserts/second
* ips.75, ips.95, ips.99 - inserts/second at 75th, 95th and 99th percentile
* r/i, rkb/i, wkb/i - reads/insert, read KB/insert, write KB/insert, from iostat
* Mcpu/i - normalized CPU overhead/insert
* size - database size in GB at test end
* rss - mongod RSS in GB at test end
* r/s, rmb/s, wmb/s - r/s, read MB/s and write MB/s, from iostat
* cpu - average CPU utilization, us+sy from vmstat
* max-i - max time per insert call in seconds
engines:
* mm.365, mm.m400 - mmapv1 with MongoDB 3.6.5, 4.0.0
* wt.m365.vX-180 - WiredTiger, MongoDB 3.6.5, vX is the configuration, cache_size = 180gb
* wt.m400.vX-180 - WiredTiger, MongoDB 4.0.0, vX is the configuration, cache_size = 180gb
The base configuration is:
processManagement:
fork: true
systemLog:
destination: file
path: /blah/blah/blah
logAppend: true
storage:
syncPeriodSecs: 60
dbPath: /blah/blah/blah
journal:
enabled: true
operationProfiling.slowOpThresholdMs: 2000
replication.oplogSizeMB: 4000
storage.wiredTiger.collectionConfig.blockCompressor: none
storage.wiredTiger.engineConfig.journalCompressor: none
storage.wiredTiger.engineConfig.cacheSizeGB: 180
configurations - "cS" means storage.wiredTiger.engineConfig.configString, "sps" is syncPeriodSecs, "cache" is cacheSizeGB
* v1 - base config
* v2 - base & sps=300
* v3 - base & cS: eviction=(threads_min=8,threads_max=8)
* v4 - base & cS: eviction_checkpoint_target=0
* v5 - base & cS: eviction_checkpoint_target=0,eviction_dirty_trigger=90,eviction_dirty_target=85,eviction_trigger=95,eviction_target=90
* v6 - base & cS: eviction_checkpoint_target=1,eviction_dirty_trigger=90,eviction_dirty_target=85,eviction_trigger=95,eviction_target=90
* v7 - base & cS: eviction_checkpoint_target=10,eviction_dirty_trigger=90,eviction_dirty_target=85,eviction_trigger=95,eviction_target=90
* v8 - base & cS: eviction_checkpoint_target=85,eviction_dirty_trigger=90,eviction_dirty_target=85,eviction_trigger=95,eviction_target=90
* v9 - base & cs: sps=300, eviction_checkpoint_target=0,eviction_dirty_trigger=90,eviction_dirty_target=85,eviction_trigger=95,eviction_target=90
* v10 - base & cs: sps=300, eviction_checkpoint_target=1,eviction_dirty_trigger=90,eviction_dirty_target=85,eviction_trigger=95,eviction_target=90
* v11 - base & cs: sps=300, eviction_checkpoint_target=10,eviction_dirty_trigger=90,eviction_dirty_target=85,eviction_trigger=95,eviction_target=90
* v12 - base & cs: sps=300, eviction_checkpoint_target=85,eviction_dirty_trigger=90,eviction_dirty_target=85,eviction_trigger=95,eviction_target=90
* v13 - base & cs: cache=150, sps=300, eviction_checkpoint_target=0,eviction_dirty_trigger=20,eviction_dirty_target=15,eviction_trigger=95,eviction_target=90
* v14 - base & cs: cache=150, sps=300, eviction_checkpoint_target=1,eviction_dirty_trigger=20,eviction_dirty_target=15,eviction_trigger=95,eviction_target=90
* v15 - base & cs: cache=150, sps=300, eviction_checkpoint_target=10,eviction_dirty_trigger=20,eviction_dirty_target=15,eviction_trigger=95,eviction_target=90
* v16 - base & cs: cache=120, sps=300, eviction_checkpoint_target=0,eviction_dirty_trigger=40,eviction_dirty_target=35,eviction_trigger=95,eviction_target=90
* v17 - base & cs: cache=120, sps=300, eviction_checkpoint_target=1,eviction_dirty_trigger=40,eviction_dirty_target=35,eviction_trigger=95,eviction_target=90
* v18 - base & cs: cache=120, sps=300, eviction_checkpoint_target=10,eviction_dirty_trigger=40,eviction_dirty_target=35,eviction_trigger=95,eviction_target=90
* v19 - base & cs: cache=100, sps=300, eviction_checkpoint_target=0,eviction_dirty_trigger=60,eviction_dirty_target=55,eviction_trigger=95,eviction_target=90
* v20 - base & cs: cache=100, sps=300, eviction_checkpoint_target=1,eviction_dirty_trigger=60,eviction_dirty_target=55,eviction_trigger=95,eviction_target=90
* v21 - base & cs: cache=100, sps=300, eviction_checkpoint_target=10,eviction_dirty_trigger=60,eviction_dirty_target=55,eviction_trigger=95,eviction_target=90
* v22 - base & cS: cache=100, sps=300, eviction_checkpoint_target=0,eviction_dirty_trigger=80,eviction_dirty_target=75,eviction_trigger=95,eviction_target=90
* v23 - base & cS: cache=100, sps=300, eviction_checkpoint_target=1,eviction_dirty_trigger=80,eviction_dirty_target=75,eviction_trigger=95,eviction_target=90
* v24 - base & cS: cache=100, sps=300, eviction_checkpoint_target=10,eviction_dirty_trigger=80,eviction_dirty_target=75,eviction_trigger=95,eviction_target=90
* v14a - v14 & eviction=(threads_min=8,threads_max=8)
* v17a - v17 & eviction=(threads_min=8,threads_max=8)
* v20a - v20 & eviction=(threads_min=8,threads_max=8)
* v23a - v23 & eviction=(threads_min=8,threads_max=8)
-- 100m rows, 1 collection, 1 thread, mongo, in-memory, 100 docs/insert
- load mongo
ips.av ips.75 ips.95 ips.99 r/i rkb/i wkb/i Mcpu/i size rss r/s rmb/s wmb/s cpu max-i engine
12979 11972 10964 10046 0 0 15.87 564 52 49.0 0 0 206.0 7.3 2.623 mm.m365
13014 11952 10891 9864 0 0 15.87 303 56 49.3 0 0 206.6 3.9 2.416 mm.m400
-
20446 17323 15752 10150 0 0 5.95 545 36 39.0 0 0 121.6 11.1 0.367 wt.m365.v1-180
20610 17072 16114 13944 0 0 5.23 514 37 37.1 0 0 107.8 10.6 0.431 wt.m365.v2-180
20530 16750 15490 14975 0 0 6.01 639 35 38.9 0 0 123.4 13.1 0.414 wt.m365.v3-180
20916 17372 16596 16013 0 0 5.74 529 35 40.0 0 0 120.0 11.1 0.391 wt.m365.v4-180
28153 27349 26275 23323 0 0 1.49 280 28 60.9 0 0 41.8 7.9 0.412 wt.m365.v5-180
28121 27220 26322 23202 0 0 1.49 278 28 60.9 0 0 41.9 7.8 0.288 wt.m365.v6-180
28337 27434 26418 23517 0 0 1.50 278 29 60.8 0 0 42.5 7.9 0.301 wt.m365.v7-180
28313 27490 26478 23339 0 0 1.49 280 29 61.0 0 0 42.3 7.9 0.450 wt.m365.v8-180
28531 27695 26744 23104 0 0 1.00 260 29 60.8 0 0 28.5 7.4 0.390 wt.m365.v9-180
28249 27422 26379 23290 0 0 1.00 262 29 60.8 0 0 28.2 7.4 0.343 wt.m365.v10-180
28555 27703 26733 23771 0 0 0.99 261 30 60.9 0 0 28.1 7.4 0.389 wt.m365.v11-180
28450 27581 26649 22808 0 0 1.00 260 28 60.7 0 0 28.5 7.4 0.413 wt.m365.v12-180
27167 25999 21729 20912 0 0 1.46 284 32 52.9 0 0 39.6 7.7 0.570 wt.m365.v13-180
27203 26174 21629 20839 0 0 1.52 285 33 53.4 0 0 41.3 7.7 0.439 wt.m365.v14-180
27071 26182 21522 20930 0 0 1.38 286 33 53.6 0 0 37.4 7.7 0.389 wt.m365.v15-180
28321 27470 26495 22879 0 0 1.01 261 29 60.9 0 0 28.6 7.4 0.389 wt.m365.v16-180
28337 27460 26608 23536 0 0 0.98 260 28 60.8 0 0 27.8 7.4 0.390 wt.m365.v17-180
28273 27439 26558 23205 0 0 1.00 259 28 60.8 0 0 28.2 7.3 0.390 wt.m365.v18-180
28313 27517 26483 23288 0 0 1.00 262 29 61.0 0 0 28.2 7.4 0.388 wt.m365.v19-180
27996 27190 26096 22607 0 0 1.06 262 28 60.8 0 0 29.7 7.3 0.341 wt.m365.v20-180
28329 27473 26540 23886 0 0 0.98 263 29 60.8 0 0 27.9 7.5 0.390 wt.m365.v21-180
-
20371 16917 15966 10659 0 0 5.94 379 x 39.1 0 0 121.0 7.7 0.398 wt.m400.v1-180
20454 16688 16023 15126 0 0 5.28 354 x 37.1 0 0 107.9 7.2 0.390 wt.m400.v2-180
20602 16623 15352 14999 0 0 6.08 475 x 38.8 0 0 125.3 9.8 0.369 wt.m400.v3-180
20881 17183 16383 16144 0 0 5.84 365 x 39.7 0 0 122.0 7.6 0.402 wt.m400.v4-180
28547 27690 26922 23707 0 0 1.48 164 x 60.9 0 0 42.2 4.7 0.482 wt.m400.v5-180
28401 27546 26566 23157 0 0 1.47 167 x 61.0 0 0 41.9 4.8 0.222 wt.m400.v6-180
28629 27783 27034 23362 0 0 1.46 165 x 61.0 0 0 41.7 4.7 0.367 wt.m400.v7-180
28563 27718 26783 23186 0 0 1.45 168 x 60.9 0 0 41.5 4.8 0.247 wt.m400.v8-180
28129 27279 26600 23345 0 0 0.99 149 x 60.9 0 0 27.9 4.2 0.406 wt.m400.v9-180
28588 27676 26893 23448 0 0 0.99 150 x 60.9 0 0 28.2 4.3 0.501 wt.m400.v10-180
28474 27606 26880 23775 0 0 0.98 151 x 61.0 0 0 27.8 4.3 0.340 wt.m400.v11-180
28604 27771 27102 23482 0 0 1.00 147 x 60.9 0 0 28.7 4.2 0.437 wt.m400.v12-180
27337 25787 21829 21209 0 0 1.49 166 35 52.8 0 0 40.7 4.5 0.564 wt.m400.v13-180
27241 25581 22050 21196 0 0 1.45 170 34 52.8 0 0 39.4 4.6 0.511 wt.m400.v14-180
26925 25443 21638 20929 0 0 1.51 172 34 53.1 0 0 40.7 4.6 0.524 wt.m400.v15-180
28596 27756 26951 23419 0 0 1.00 150 29 60.9 0 0 28.5 4.3 0.401 wt.m400.v16-180
28273 27412 26515 23517 0 0 1.00 149 28 60.9 0 0 28.2 4.2 0.415 wt.m400.v17-180
28563 27708 26945 23377 0 0 0.98 148 29 60.9 0 0 27.9 4.2 0.366 wt.m400.v18-180
28450 27609 26601 23162 0 0 1.02 150 30 60.9 0 0 29.0 4.3 0.397 wt.m400.v19-180
28662 27779 26983 23667 0 0 0.98 147 28 60.9 0 0 27.9 4.2 0.366 wt.m400.v20-180
28563 27690 26904 23567 0 0 1.02 149 29 61.0 0 0 29.0 4.3 0.412 wt.m400.v21-180
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment