Skip to content

Instantly share code, notes, and snippets.

@mdcallag
Last active July 16, 2018 15:37
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save mdcallag/400c7615a74063ce43523c1005d88c4a to your computer and use it in GitHub Desktop.
Save mdcallag/400c7615a74063ce43523c1005d88c4a to your computer and use it in GitHub Desktop.
tests used:
* server has 56 core with HT, 256gb RAM, Linux 4.6.7-73, 3 fast SSD w/ 5tb
* insert benchmark - in-memory and io-bound, http://smalldatum.blogspot.com/2017/06/the-insert-benchmark.html
legend for load:
* ips.av - average inserts/second, global
* ips.75, ips.95, ips.99 - inserts/second at 75th, 95th and 99th percentile, per insert client
* r/i, rkb/i, wkb/i - reads/insert, read KB/insert, write KB/insert, from iostat
* Mcpu/i - normalized CPU overhead/insert
* size - database size in GB at test end
* rss - mongod RSS in GB at test end
* r/s, rmb/s, wmb/s - r/s, read MB/s and write MB/s, from iostat
* cpu - average CPU utilization, us+sy from vmstat
* max-i - max time per insert call in seconds
legend for wps=...
* ips.av, ips.99 -- average and 99th percentile insert rates. ips.av is global, ips.p99 is per insert client, insert clients are rate limited
* qps.av, qps.99 -- similar to ips.av & ips.p99, but for queries, query clients are not rate limited
* r/i, rkb/i, wkb/i - reads/insert, read KB/insert, write KB/insert from iostat
* r/q, rkb/q, wkb/q - reads/query, read KB/query, write KB/query from iostat
* Mcpu/i, Mcpu/q - normalized CPU overhead /insert and /query, from us+sy columns in vmstat
* size - database size in GB at test end
* rss - mongod RSS in GB at test end
* r/s, rmb/s, wmb/s - reads/second, read MB/second, write MB/second from iostat
* CPU utilization - from us+sy columns in vmstat
* max-i,q - max time in seconds for an insert and query
engines:
* mm.365, mm.m400 - mmapv1 with MongoDB 3.6.5, 4.0.0
* wt.m365.vX-180 - WiredTiger, MongoDB 3.6.5, vX is the configuration
* wt.m400.vX-180 - WiredTiger, MongoDB 4.0.0, vX is the configuration
The base configuration is:
processManagement:
fork: true
systemLog:
destination: file
path: /blah/blah/blah
logAppend: true
storage:
syncPeriodSecs: 60
dbPath: /blah/blah/blah
journal:
enabled: true
operationProfiling.slowOpThresholdMs: 2000
replication.oplogSizeMB: 4000
storage.wiredTiger.collectionConfig.blockCompressor: none
storage.wiredTiger.engineConfig.journalCompressor: none
storage.wiredTiger.engineConfig.cacheSizeGB: 180
configurations - "cS" means storage.wiredTiger.engineConfig.configString, "sps" is syncPeriodSecs, "cache" is cacheSizeGB
* v1 - base config
* v2 - base & sps=300
* v3 - base & cS: eviction=(threads_min=8,threads_max=8)
* v4 - base & cS: eviction_checkpoint_target=0
* v5 - base & cS: eviction_checkpoint_target=0,eviction_dirty_trigger=90,eviction_dirty_target=85,eviction_trigger=95,eviction_target=90
* v6 - base & cS: eviction_checkpoint_target=1,eviction_dirty_trigger=90,eviction_dirty_target=85,eviction_trigger=95,eviction_target=90
* v7 - base & cS: eviction_checkpoint_target=10,eviction_dirty_trigger=90,eviction_dirty_target=85,eviction_trigger=95,eviction_target=90
* v8 - base & cS: eviction_checkpoint_target=85,eviction_dirty_trigger=90,eviction_dirty_target=85,eviction_trigger=95,eviction_target=90
* v9 - base & cs: sps=300, eviction_checkpoint_target=0,eviction_dirty_trigger=90,eviction_dirty_target=85,eviction_trigger=95,eviction_target=90
* v10 - base & cs: sps=300, eviction_checkpoint_target=1,eviction_dirty_trigger=90,eviction_dirty_target=85,eviction_trigger=95,eviction_target=90
* v11 - base & cs: sps=300, eviction_checkpoint_target=10,eviction_dirty_trigger=90,eviction_dirty_target=85,eviction_trigger=95,eviction_target=90
* v12 - base & cs: sps=300, eviction_checkpoint_target=85,eviction_dirty_trigger=90,eviction_dirty_target=85,eviction_trigger=95,eviction_target=90
* v13 - base & cs: cache=150, sps=300, eviction_checkpoint_target=0,eviction_dirty_trigger=20,eviction_dirty_target=15,eviction_trigger=95,eviction_target=90
* v14 - base & cs: cache=150, sps=300, eviction_checkpoint_target=1,eviction_dirty_trigger=20,eviction_dirty_target=15,eviction_trigger=95,eviction_target=90
* v15 - base & cs: cache=150, sps=300, eviction_checkpoint_target=10,eviction_dirty_trigger=20,eviction_dirty_target=15,eviction_trigger=95,eviction_target=90
* v16 - base & cs: cache=120, sps=300, eviction_checkpoint_target=0,eviction_dirty_trigger=40,eviction_dirty_target=35,eviction_trigger=95,eviction_target=90
* v17 - base & cs: cache=120, sps=300, eviction_checkpoint_target=1,eviction_dirty_trigger=40,eviction_dirty_target=35,eviction_trigger=95,eviction_target=90
* v18 - base & cs: cache=120, sps=300, eviction_checkpoint_target=10,eviction_dirty_trigger=40,eviction_dirty_target=35,eviction_trigger=95,eviction_target=90
* v19 - base & cs: cache=100, sps=300, eviction_checkpoint_target=0,eviction_dirty_trigger=60,eviction_dirty_target=55,eviction_trigger=95,eviction_target=90
* v20 - base & cs: cache=100, sps=300, eviction_checkpoint_target=1,eviction_dirty_trigger=60,eviction_dirty_target=55,eviction_trigger=95,eviction_target=90
* v21 - base & cs: cache=100, sps=300, eviction_checkpoint_target=10,eviction_dirty_trigger=60,eviction_dirty_target=55,eviction_trigger=95,eviction_target=90
* v22 - base & cS: cache=100, sps=300, eviction_checkpoint_target=0,eviction_dirty_trigger=80,eviction_dirty_target=75,eviction_trigger=95,eviction_target=90
* v23 - base & cS: cache=100, sps=300, eviction_checkpoint_target=1,eviction_dirty_trigger=80,eviction_dirty_target=75,eviction_trigger=95,eviction_target=90
* v24 - base & cS: cache=100, sps=300, eviction_checkpoint_target=10,eviction_dirty_trigger=80,eviction_dirty_target=75,eviction_trigger=95,eviction_target=90
* v14a - v14 & eviction=(threads_min=8,threads_max=8)
* v17a - v17 & eviction=(threads_min=8,threads_max=8)
* v20a - v20 & eviction=(threads_min=8,threads_max=8)
* v23a - v23 & eviction=(threads_min=8,threads_max=8)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment