Last active
February 2, 2017 03:12
-
-
Save hyc/cddcc1c2f4c5981a186f to your computer and use it in GitHub Desktop.
In-Memory DB tests, part 1
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Inspired by this RocksDB post, I started playing with this "readwhilewriting" test as well. | |
| https://github.com/facebook/rocksdb/wiki/RocksDB-In-Memory-Workload-Performance-Benchmarks | |
| Of course the data here is only a tiny fraction of the size of the RocksDB report. (I have | |
| a larger test running that's of a more comparable size, but it will probably be a couple | |
| more days before all of the tests finish.) | |
| So - tested LevelDB 1.17, Basho LevelDB 1.9, BerkeleyDB 5.3.21, HyperLevelDB 1.14, | |
| LMDB 0.9.12rc, RocksDB 3.2, TokuDB 4.6.119, and WiredTiger 2.2.0 (both LSM and Btree). | |
| That's a lot of tests to run, trust me. | |
| The test system is an Asus NP56D laptop with quadcore AMD A10-4600M APU and 16GB DRAM. The | |
| test first loads a database with 10M small records (16 byte keys, 100 byte values) as fast | |
| as possible, then fires off 4 threads to read the DB and 1 thread to write it, all using | |
| random keys. The writer is constrained to no more than 10240 writes per second. The | |
| purpose of the test is to measure read throughput. | |
| Summary... | |
| Engine Load Time Load Size Writes/Sec Reads/Sec Run Time Final Size | |
| Wall, User, Sys KB Wall, User, Sys KB | |
| LevelDB 00:17s 00:22s 00:03s 1123024 10231 158919 04:12s 15:00s 01:15s 1186640 | |
| Basho 00:19s 00:35s 00:07s 2536940 10231 61490 10:51s 40:36s 02:14s 1450180 | |
| BerkeleyDB 01:07s 01:01s 00:06s 1918364 9584 77603 08:36s 23:30s 09:51s 1940912 | |
| HyperLvlDB 00:19s 00:24s 00:03s 1123224 10196 146328 04:34s 16:37s 01:01s 1146308 | |
| LMDB 00:05s 00:04s 00:01s 1258128 10227 1501891 00:27s 01:44s 00:01s 1289908 | |
| RocksDB 00:11s 00:17s 00:03s 1130692 10232 82226 08:08s 29:33s 02:28s 1514476 | |
| TokuDB 01:03s 00:55s 00:33s 1612296 9801 114568 05:54s 17:59s 03:23s 2326852 | |
| WiredLSM 00:34s 01:14s 00:09s 2044484 10214 280009 02:23s 09:04s 00:06s 1407764 | |
| WiredBtree 00:09s 00:08s 00:01s 1190944 10207 128953 05:11s 14:55s 05:01s 2382104 | |
| The same test, using 20M records instead of 10M | |
| Engine Load Time Load Size Writes/Sec Reads/Sec Run Time Final Size | |
| Wall, User, Sys KB Wall, User, Sys KB | |
| LevelDB 00:35s 00:45s 00:07s 2246004 10232 24626 54:09s 96:33s 58:33s 3422060 | |
| Basho 00:41s 01:23s 00:17s 6281124 10233 47742 27:56s 103:29s 06:47s 2614212 | |
| BerkeleyDB 02:08s 01:57s 00:11s 3262628 9475 74154 17:59s 49:08s 20:47s 3258468 | |
| HyperLvlDB 00:38s 00:49s 00:06s 2247448 10192 135028 09:53s 36:05s 02:10s 2291036 | |
| LMDB 00:11s 00:08s 00:02s 2516192 10241 1444797 00:56s 03:35s 00:02s 2547976 | |
| RocksDB 00:22s 00:35s 00:06s 2256032 10232 64513 20:40s 74:14s 07:22s 3348364 | |
| TokuDB 01:45s 01:41s 00:47s 2718360 9884 109104 12:16s 37:57s 07:10s 3898032 | |
| WiredLSM 01:09s 02:37s 00:19s 2405856 10219 255603 05:14s 20:12s 00:11s 2762792 | |
| WiredBtree 00:17s 00:15s 00:02s 2381876 10218 105235 12:41s 36:47s 12:45s 4763968 | |
| The binaries and script to run this test are at http://highlandsun.com/hyc/bench-20140609.tgz | |
| Raw data for the 10M record test is at http://highlandsun.com/hyc/raw10M.txt |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Software revisions used: | |
| violino:/home/software/leveldb> g++ --version | |
| g++ (Ubuntu/Linaro 4.7.3-1ubuntu1) 4.7.3 | |
| Copyright (C) 2012 Free Software Foundation, Inc. | |
| This is free software; see the source for copying conditions. There is NO | |
| warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. | |
| violino:/home/software/leveldb> git log -1 --pretty=format:"%H %ci" master | |
| e353fbc7ea81f12a5694991b708f8f45343594b1 2014-05-01 13:44:03 -0700 | |
| violino:/home/software/basho_leveldb> git log -1 --pretty=format:"%H %ci" develop | |
| 16b22c8198975b62a938dff9910f4432772d253a 2014-06-06 12:25:40 -0400 | |
| violino:/home/software/db-5.3.21> ls -l README | |
| -rw-r--r-- 1 hyc hyc 234 May 11 2012 README | |
| violino:/home/software/HyperLevelDB> git log -1 --pretty=format:"%H %ci" releases/1.0 | |
| a7a707e303ec1953d08cbc586312ac7b2988eebb 2014-02-10 09:43:03 -0500 | |
| violino:~/OD/mdb> git log -1 --pretty=format:"%H %ci" | |
| a93810cc3d1a062bf5edbe9c14795d0360cda8a4 2014-05-30 23:39:44 -0700 | |
| violino:/home/software/rocksdb> git log -1 --pretty=format:"%H %ci" | |
| 0365eaf12e9e896ea5902fb3bf3db5e6da275d2e 2014-06-06 18:27:44 -0700 | |
| violino:/home/software/sophia> git log -1 --pretty=format:"%H %ci" | |
| 3f16d0ed3f512dfa5d4d2036bba7deb4fe6ae4ec 2014-02-25 20:56:53 +0400 | |
| violino:/home/software/sqlite3.git> git log -1 --pretty=format:"%H %ci" master | |
| c18a797d6853081493732fa9a3a77e1334a9ef12 2013-09-12 19:28:34 -0700 | |
| violino:/home/software/ft-index> git log -1 --pretty=format:"%H %ci" master | |
| f51c7180db1eafdd9e6efb915c396d894c2d0ab1 2014-05-30 12:58:28 -0400 | |
| violino:/home/software/kyotocabinet-1.2.76> ls -l ChangeLog | |
| -rw-r--r-- 1 hyc hyc 29158 May 24 2012 ChangeLog | |
| violino:/home/software/wiredtiger> git log -1 --pretty=format:"%H %ci" | |
| 91da74e5946c409b8e05c53927a7b447129a6933 2014-05-21 17:05:08 +1000 |
Hi Michael, I just followed whatever build instructions for each of those other libraries, nothing special. However in every case I disabled use of any 3rd party libraries (e.g. tcmalloc or jemalloc) and disabled use of any compression libraries.
I actually don't have a use case for this myself, was just poking at the RocksDB performance report linked above. I can't think of any sane reason to run a production DB on a tmpfs. It's a horrible waste of RAM, and a horrible waste of time to work with all that data on a non-persistent medium.
By the way, the full report is now online http://symas.com/mdb/inmem/
As an afterthought, the readwhilewriting test is probably close to what an LDAP workload looks like - very read-heavy.
As another afterthought, I would consider using NVDIMMs and a reset-survivable RAMdisk, like we used to do back in the 1980s... https://groups.google.com/d/msg/comp.sys.atari.st/4Vg8rGUuGC4/TJrFrnt5WaMJ
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi Howard, do you have a build script for the dependencies?
Also, what's the goal with this work -- what's driving the choice of data volume, concurrency, test machine, etc.? Do they represent some real-world workload?