This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Just using MDB_WRITEMAP|MDB_NOSYNC, the put speed went from .1M/s to .3M/s | |
perf testing: LMDB aka Lightning MDB | |
running tests on: via command: 'cat /proc/cpuinfo | egrep 'model name' | head -n 1' | |
running tests on: `model name : Intel(R) Core(TM)2 Extreme CPU Q9300 @ 2.53GHz` | |
-OP MMAP REMAP SHRK PART TOTAL ------PERCENT OPERATIONS PER PROCESS PER SECOND -OPS | |
--- -k/s --k/s --/s --/s M-OPS 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 -M/s | |
PUT 0.0 0.0 0 0 0.0100 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.0 | |
PUT 0.0 0.0 0 0 1.1 99 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1.1 - | |
PUT 0.0 0.0 0 0 2.0 1 2 96 0 0 0 0 0 0 0 0 0 0 0 0 0 0.8 - |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Was trying to duplicate this test: https://github.com/facebook/rocksdb/wiki/RocksDB-In-Memory-Workload-Performance-Benchmarks | |
using git rev 0365eaf12e9e896ea5902fb3bf3db5e6da275d2 | |
but all I get is this: | |
./run ./db_bench.rocks --db=/tmp/test1 --num_levels=6 --key_size=20 --prefix_size=20 --keys_per_prefix=0 --value_size=100 --block_size=4096 --cache_size=17179869184 --cache_numshardbits=6 --compression_type=none --compression_ratio=1 --min_level_to_compress=-1 --disable_seek_compaction=1 --hard_rate_limit=2 --write_buffer_size=134217728 --max_write_buffer_number=2 --level0_file_num_compaction_trigger=8 --target_file_size_base=134217728 --max_bytes_for_level_base=1073741824 --disable_wal=0 --wal_dir=/mnt/hyc/WAL --sync=0 --disable_data_sync=1 --verify_checksum=1 --delete_obsolete_files_period_micros=314572800 --max_grandparent_overlap_factor=10 --max_background_compactions=4 --max_background_flushes=0 --level0_slowdown_writes_trigger=16 --level0_stop_writes_trigger=24 --statistics=0 --stats_per_interval=0 --s |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
SSD | |
These tests are incomplete because I still can't get the HyperLevelDB driver to build, and various other oddities in the build tree. | |
violino:/home/software/influxdb> ./benchmark-storage -path=/home/test/db -points=3000000 | |
################ Benchmarking: lmdb | |
Writing 3000000 points in batches of 1000 points took 8.579291321s (2.859764 microsecond per point) | |
Querying 3000000 points took 1.743559302s (0.581186 microseconds per point) | |
Size: 11M | |
Took 1.925530865s to delete 1500000 points |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
16GB data, 8GB RAM, Samsung 830 512GB SSD, 160000000 records, 16 byte keys, 100 byte values | |
Sophia: version 1.1 | |
fillrandsync : 8.519 micros/op 117387 ops/sec; 13.0 MB/s (160000 ops) | |
20784 /home/test/dbbench_sph-1 | |
20788 /home/test | |
fillrandom : 13.487 micros/op 74143 ops/sec; 8.2 MB/s | |
26220260 /home/test/dbbench_sph-2 | |
26220264 /home/test | |
fillrandbatch : 7.334 micros/op 136352 ops/sec; 15.1 MB/s |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
violino:~/OD/mdb/libraries/liblmdb> rm -rf /tmp/leveldbtest-1000/* | |
violino:~/OD/mdb/libraries/liblmdb> ./db_bench_mdb.no_profile --num=10000000 | |
LMDB: version LMDB 0.9.14: (June 20, 2014) | |
Date: Sat Jul 5 10:47:17 2014 | |
CPU: 4 * Intel(R) Core(TM)2 Extreme CPU Q9300 @ 2.53GHz | |
CPUCache: 6144 KB | |
Keys: 16 bytes each | |
Values: 100 bytes each (50 bytes after compression) | |
Entries: 10000000 | |
RawSize: 1106.3 MB (estimated) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
https://github.com/hyc/leveldb/commit/d05251bb51138f8f77a08e0e01f22c5048d3ccb5 | |
violino:/home/software/leveldb> ./db_bench_ntdb | |
NTDB: version 1.0 | |
Date: Mon Sep 15 16:51:55 2014 | |
CPU: 4 * Intel(R) Core(TM)2 Extreme CPU Q9300 @ 2.53GHz | |
CPUCache: 6144 KB | |
Keys: 16 bytes each | |
Values: 100 bytes each (50 bytes after compression) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Using https://github.com/hyc/leveldb/commit/0cbfeaa4caa6f6615c0a0caf611e4cdff909465d | |
Had to revert the TDB_MUTEX_LOCKING code since that doesn't support transactions, and all of the other tests are transactional. | |
./db_bench_tdb --stats_interval=100000 --benchmarks=fillseqbatch --new_hash=1 | |
TDB: version 1.3.0 | |
Date: Tue Sep 16 02:56:01 2014 | |
CPU: 4 * Intel(R) Core(TM)2 Extreme CPU Q9300 @ 2.53GHz | |
CPUCache: 6144 KB | |
Keys: 16 bytes each |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
violino:/home/software/leveldb> ./db_bench_mdb.vl32 | |
LMDB: version LMDB 0.9.14: (September 15, 2014) | |
Date: Thu Sep 18 20:52:49 2014 | |
CPU: 4 * Intel(R) Core(TM)2 Extreme CPU Q9300 @ 2.53GHz | |
CPUCache: 6144 KB | |
Keys: 16 bytes each | |
Values: 100 bytes each (50 bytes after compression) | |
Entries: 1000000 | |
RawSize: 110.6 MB (estimated) | |
FileSize: 62.9 MB (estimated) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Look at all the bugs in BerkeleyDB's history that are directly attributable to needing | |
to manage multiple files in a transactional context. The SleepyCat/BerkeleyDB folks were | |
smart guys but they still got tripped up by this, over and over again. One of the key | |
principles to LMDB is "don't do anything that's hard" - this is why it's so reliable, | |
and why it's so efficient. All of the bugs listed here are a type of bug that is | |
impossible for LMDB to encounter, because LMDB never performs any such operations. | |
File-related bugs fixed in BDB 4.0 | |
https://gitorious.org/berkeleydb/berkeleydb/source/347d239a1e44ed4f773ae9274c2a32cf2b8999c0:docs/upgrading/changelog_4_0_14.html | |
#4307 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Doing mdc2 for 3s on 16 size blocks: 1424278 mdc2's in 2.93s | |
Doing mdc2 for 3s on 64 size blocks: 385955 mdc2's in 3.00s | |
Doing mdc2 for 3s on 256 size blocks: 98679 mdc2's in 2.99s | |
Doing mdc2 for 3s on 1024 size blocks: 24859 mdc2's in 2.94s | |
Doing mdc2 for 3s on 8192 size blocks: 3038 mdc2's in 2.93s | |
Doing md4 for 3s on 16 size blocks: 8688711 md4's in 2.94s | |
Doing md4 for 3s on 64 size blocks: 6804170 md4's in 2.93s | |
Doing md4 for 3s on 256 size blocks: 4225590 md4's in 2.96s | |
Doing md4 for 3s on 1024 size blocks: 1637754 md4's in 2.92s | |
Doing md4 for 3s on 8192 size blocks: 247283 md4's in 2.91s |
OlderNewer