LMDB Put Performance Results (random keys)

lmdb_put_perf.md

Batch size = 400

Keys are random.

The results are below.

10s_lmdb_put_large_8MB.txt

$ make write-large-hurry
go test -bench PutLarge -benchtime 10s
testing: warning: no tests to run
PASS
BenchmarkPutLarge_LMDB-4	write_test.go:51: bench put LARGE (8MB), take 1 (n=1)
write_test.go:97: last put took: 18.058786ms
write_test.go:98: last key: 52fdfc072182654f163f5f0f9a621d72
write_test.go:148: depth: 1, branch pg: 0, leaf pg: 1, entries: 1
write_test.go:154: gc: 182.756µs
write_test.go:157: mem: heap 8MB/8MB, alloc total 24MB
write_test.go:48: take 1 (n=1) done in 19.912282ms

write_test.go:51: bench put LARGE (8MB), take 2 (n=100)
write_test.go:97: last put took: 1.440733884s
write_test.go:98: last key: 136385cdc838f0bdd4c812f042577410
write_test.go:148: depth: 2, branch pg: 1, leaf pg: 2, entries: 100
write_test.go:154: gc: 246.659µs
write_test.go:157: mem: heap 8MB/8MB, alloc total 24MB
write_test.go:48: take 2 (n=100) done in 1.967043561s

write_test.go:51: bench put LARGE (8MB), take 3 (n=1000)
write_test.go:97: last put took: 3.051093977s
write_test.go:98: last key: f4164168d965a2c0fb9cc8c73d9e776e
write_test.go:148: depth: 2, branch pg: 1, leaf pg: 19, entries: 1000
write_test.go:154: gc: 621.095µs
write_test.go:157: mem: heap 8MB/8MB, alloc total 24MB
write_test.go:48: take 3 (n=1000) done in 21.921219344s

    1000	  21920596 ns/op	 382.68 MB/s	     168 B/op	       5 allocs/op
ok  	github.com/zenhotels/lmdb-go/lmdb/bench	23.955s

10s_lmdb_put_small_8KB.txt

$ make write-hurry
go test -bench PutSmall -benchtime 10s
testing: warning: no tests to run
PASS
BenchmarkPutSmall_LMDB-4	write_test.go:38: bench put SMALL (8KB), take 1 (n=1)
write_test.go:97: last put took: 292.959µs
write_test.go:98: last key: 52fdfc072182654f163f5f0f9a621d72
write_test.go:148: depth: 1, branch pg: 0, leaf pg: 1, entries: 1
write_test.go:154: gc: 133.664µs
write_test.go:157: mem: heap 8MB/8MB, alloc total 24MB
write_test.go:35: take 1 (n=1) done in 1.489914ms

write_test.go:38: bench put SMALL (8KB), take 2 (n=100)
write_test.go:97: last put took: 2.291753ms
write_test.go:98: last key: 136385cdc838f0bdd4c812f042577410
write_test.go:148: depth: 2, branch pg: 1, leaf pg: 2, entries: 100
write_test.go:154: gc: 190.763µs
write_test.go:157: mem: heap 8MB/8MB, alloc total 24MB
write_test.go:35: take 2 (n=100) done in 5.3043ms

write_test.go:38: bench put SMALL (8KB), take 3 (n=10000)
write_test.go:97: last put took: 10.188102ms
write_test.go:98: last key: 99e97542e2075290412a71cf6d8d885f
write_test.go:148: depth: 3, branch pg: 3, leaf pg: 178, entries: 10000
write_test.go:154: gc: 381.422µs
write_test.go:157: mem: heap 8MB/8MB, alloc total 25MB
write_test.go:35: take 3 (n=10000) done in 288.66556ms

write_test.go:38: bench put SMALL (8KB), take 4 (n=500000)
write_test.go:97: last put took: 16.002261ms
write_test.go:98: last key: 9aca490c637bdf2ed979deea828eb97f
write_test.go:148: depth: 4, branch pg: 134, leaf pg: 8961, entries: 500000
write_test.go:154: gc: 2.172246ms
write_test.go:157: mem: heap 8MB/8MB, alloc total 102MB
write_test.go:35: take 4 (n=500000) done in 37.059813713s

  500000	     74118 ns/op	 110.53 MB/s	     160 B/op	       5 allocs/op
ok  	github.com/zenhotels/lmdb-go/lmdb/bench	37.396s

20s_lmdb_put_large_8MB.txt

$ make write-large
go test -bench PutLarge -benchtime 20s
testing: warning: no tests to run
PASS
BenchmarkPutLarge_LMDB-4	write_test.go:51: bench put LARGE (8MB), take 1 (n=1)
write_test.go:97: last put took: 24.287748ms
write_test.go:98: last key: 52fdfc072182654f163f5f0f9a621d72
write_test.go:148: depth: 1, branch pg: 0, leaf pg: 1, entries: 1
write_test.go:154: gc: 248.744µs
write_test.go:157: mem: heap 8MB/8MB, alloc total 24MB
write_test.go:48: take 1 (n=1) done in 26.588564ms

write_test.go:51: bench put LARGE (8MB), take 2 (n=100)
write_test.go:97: last put took: 1.427091489s
write_test.go:98: last key: 136385cdc838f0bdd4c812f042577410
write_test.go:148: depth: 2, branch pg: 1, leaf pg: 2, entries: 100
write_test.go:154: gc: 174.737µs
write_test.go:157: mem: heap 8MB/8MB, alloc total 24MB
write_test.go:48: take 2 (n=100) done in 1.984328468s

write_test.go:51: bench put LARGE (8MB), take 3 (n=2000)
write_test.go:97: last put took: 7.737076432s
write_test.go:98: last key: deb8bee22c4dc0b9b4d4cc70a500d2c8
write_test.go:148: depth: 2, branch pg: 1, leaf pg: 36, entries: 2000
write_test.go:154: gc: 1.582502ms
write_test.go:157: mem: heap 8MB/8MB, alloc total 24MB
write_test.go:48: take 3 (n=2000) done in 48.721131943s

    2000	  24360281 ns/op	 344.36 MB/s	     164 B/op	       5 allocs/op
ok  	github.com/zenhotels/lmdb-go/lmdb/bench	50.781s

20s_lmdb_put_small_8KB.txt

$ make write
go test -bench PutSmall -benchtime 20s
testing: warning: no tests to run
PASS
BenchmarkPutSmall_LMDB-4	write_test.go:38: bench put SMALL (8KB), take 1 (n=1)
write_test.go:97: last put took: 320.715µs
write_test.go:98: last key: 52fdfc072182654f163f5f0f9a621d72
write_test.go:148: depth: 1, branch pg: 0, leaf pg: 1, entries: 1
write_test.go:154: gc: 128.707µs
write_test.go:157: mem: heap 8MB/8MB, alloc total 24MB
write_test.go:35: take 1 (n=1) done in 1.461125ms

write_test.go:38: bench put SMALL (8KB), take 2 (n=100)
write_test.go:97: last put took: 2.356655ms
write_test.go:98: last key: 136385cdc838f0bdd4c812f042577410
write_test.go:148: depth: 2, branch pg: 1, leaf pg: 2, entries: 100
write_test.go:154: gc: 170.844µs
write_test.go:157: mem: heap 8MB/8MB, alloc total 24MB
write_test.go:35: take 2 (n=100) done in 5.315844ms

write_test.go:38: bench put SMALL (8KB), take 3 (n=10000)
write_test.go:97: last put took: 10.39134ms
write_test.go:98: last key: 99e97542e2075290412a71cf6d8d885f
write_test.go:148: depth: 3, branch pg: 3, leaf pg: 178, entries: 10000
write_test.go:154: gc: 389.974µs
write_test.go:157: mem: heap 8MB/8MB, alloc total 25MB
write_test.go:35: take 3 (n=10000) done in 302.670297ms

write_test.go:38: bench put SMALL (8KB), take 4 (n=1000000)
write_test.go:97: last put took: 175.775517ms
write_test.go:98: last key: 2608ee7aef084fc8ad75bd970c5373ff
write_test.go:148: depth: 4, branch pg: 264, leaf pg: 17883, entries: 1000000
write_test.go:154: gc: 2.013232ms
write_test.go:157: mem: heap 8MB/8MB, alloc total 178MB
write_test.go:35: take 4 (n=1000000) done in 1m56.620681968s

 1000000	    116620 ns/op	  70.25 MB/s	     160 B/op	       5 allocs/op
ok  	github.com/zenhotels/lmdb-go/lmdb/bench	116.971s

5s_lmdb_put_large_8MB.txt

$ make write-large-quick
go test -bench PutLarge -benchtime 5s
testing: warning: no tests to run
PASS
BenchmarkPutLarge_LMDB-4	write_test.go:51: bench put LARGE (8MB), take 1 (n=1)
write_test.go:97: last put took: 17.603395ms
write_test.go:98: last key: 52fdfc072182654f163f5f0f9a621d72
write_test.go:148: depth: 1, branch pg: 0, leaf pg: 1, entries: 1
write_test.go:154: gc: 179.097µs
write_test.go:157: mem: heap 8MB/8MB, alloc total 24MB
write_test.go:48: take 1 (n=1) done in 19.898695ms

write_test.go:51: bench put LARGE (8MB), take 2 (n=100)
write_test.go:97: last put took: 1.419926586s
write_test.go:98: last key: 136385cdc838f0bdd4c812f042577410
write_test.go:148: depth: 2, branch pg: 1, leaf pg: 2, entries: 100
write_test.go:154: gc: 195.296µs
write_test.go:157: mem: heap 8MB/8MB, alloc total 24MB
write_test.go:48: take 2 (n=100) done in 1.967846494s

write_test.go:51: bench put LARGE (8MB), take 3 (n=500)
write_test.go:97: last put took: 1.354998198s
write_test.go:98: last key: dccc016d5895b2121ba4066e44d694f6
write_test.go:148: depth: 2, branch pg: 1, leaf pg: 9, entries: 500
write_test.go:154: gc: 223.18µs
write_test.go:157: mem: heap 8MB/8MB, alloc total 24MB
write_test.go:48: take 3 (n=500) done in 10.540197531s

     500	  21079194 ns/op	 397.96 MB/s	     177 B/op	       5 allocs/op
ok  	github.com/zenhotels/lmdb-go/lmdb/bench	12.573s

5s_lmdb_put_small_8KB.txt

$ make write-quick
go test -bench PutSmall -benchtime 5s
testing: warning: no tests to run
PASS
BenchmarkPutSmall_LMDB-4	write_test.go:38: bench put SMALL (8KB), take 1 (n=1)
write_test.go:97: last put took: 328.211µs
write_test.go:98: last key: 52fdfc072182654f163f5f0f9a621d72
write_test.go:148: depth: 1, branch pg: 0, leaf pg: 1, entries: 1
write_test.go:154: gc: 126.904µs
write_test.go:157: mem: heap 8MB/8MB, alloc total 24MB
write_test.go:35: take 1 (n=1) done in 1.481737ms

write_test.go:38: bench put SMALL (8KB), take 2 (n=100)
write_test.go:97: last put took: 2.291922ms
write_test.go:98: last key: 136385cdc838f0bdd4c812f042577410
write_test.go:148: depth: 2, branch pg: 1, leaf pg: 2, entries: 100
write_test.go:154: gc: 173.78µs
write_test.go:157: mem: heap 8MB/8MB, alloc total 24MB
write_test.go:35: take 2 (n=100) done in 5.356321ms

write_test.go:38: bench put SMALL (8KB), take 3 (n=10000)
write_test.go:97: last put took: 10.60237ms
write_test.go:98: last key: 99e97542e2075290412a71cf6d8d885f
write_test.go:148: depth: 3, branch pg: 3, leaf pg: 178, entries: 10000
write_test.go:154: gc: 368.565µs
write_test.go:157: mem: heap 8MB/8MB, alloc total 25MB
write_test.go:35: take 3 (n=10000) done in 291.986119ms

write_test.go:38: bench put SMALL (8KB), take 4 (n=300000)
write_test.go:97: last put took: 16.530883ms
write_test.go:98: last key: 312644a6d87884693ce7655ca8135b0f
write_test.go:148: depth: 3, branch pg: 67, leaf pg: 5412, entries: 300000
write_test.go:154: gc: 2.270591ms
write_test.go:157: mem: heap 8MB/8MB, alloc total 71MB
write_test.go:35: take 4 (n=300000) done in 13.916012036s

  300000	     46384 ns/op	 176.61 MB/s	     160 B/op	       5 allocs/op
ok  	github.com/zenhotels/lmdb-go/lmdb/bench	14.241s