Skip to content

Instantly share code, notes, and snippets.

/results.md Secret

Created June 7, 2016 21:12
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save anonymous/638202a41699f24c80fd4b5457940b74 to your computer and use it in GitHub Desktop.
Save anonymous/638202a41699f24c80fd4b5457940b74 to your computer and use it in GitHub Desktop.
Results of testing LMDB, LevelDB, and SQLite3 backends in Minetest.

Results

All results sorted by wall time.

Source Destination Time
LMDB Dummy 0:02.72
LMDB Dummy 0:02.80
LMDB Dummy 0:02.25
LevelDB Dummy 0:09.72
LevelDB Dummy 0:09.67
LevelDB Dummy 0:09.91
LMDB Dummy 0:13.63
LevelDB Dummy 0:16.14
LMDB LevelDB 0:16.88
LMDB LevelDB 0:17.22
LMDB LevelDB 0:17.49
LMDB SQLite3 0:19.98
LevelDB LMDB 0:19.34
LevelDB LMDB 0:19.05
LevelDB LMDB 0:19.11
SQLite3 Dummy 0:20.82
SQLite3 Dummy 0:20.85
LMDB SQLite3 0:20.39
SQLite3 Dummy 0:21.21
LMDB SQLite3 0:21.17
LMDB LevelDB 0:27.67
LevelDB LMDB 0:27.58
SQLite3 LMDB 0:30.79
LMDB SQLite3 0:31.67
SQLite3 LMDB 0:31.56
SQLite3 LMDB 0:31.90
SQLite3 Dummy 0:32.31
SQLite3 LevelDB 0:38.69
SQLite3 LevelDB 0:39.08
SQLite3 LevelDB 0:39.96
LevelDB SQLite3 0:40.76
LevelDB SQLite3 0:41.22
LevelDB SQLite3 0:42.07
LevelDB SQLite3 0:46.66
SQLite3 LMDB 0:50.61
SQLite3 LevelDB 1:01.14

As you can see, LMDB, performs quite well. The only time LevelDB beats it on read is when you compare LMDB after clearing caches to LevelDB with primed caches. SQLite3's performance is an embarrassment, but it's also the only time we can directly compare LevelDB and LMDB on write. As you can see, SQLite3->LMDB is faster than SQLite3->LevelDB.

Read Test

For this test, I created all the databases by using emergeblocks on a world with an LMDB backend, then migrated it to the other backends. Before each database was tested, a sync was performed with the caches dropped right after. Since editing world.mt was to happen so often, vi was fired up with that file after the caches were dropped. No sync or crash drop was performed in the middle of the sequence, so the first value is cacheless, while the three after are migrations with the cache primed by the previous run. The command used was a server-mode migration from the database listed to dummy in order to estimate the raw read of the databases.

LMDB

2016-06-06 23:50:41: ACTION[Main]: Successfully migrated 1483063 blocks 2.42user 0.41system 0:13.63elapsed 20%CPU (0avgtext+0avgdata 499612maxresident)k 432952inputs+32outputs (762major+79373minor)pagefaults 0swaps

2016-06-06 23:50:53: ACTION[Main]: Successfully migrated 1483063 blocks 2.43user 0.28system 0:02.72elapsed 99%CPU (0avgtext+0avgdata 499980maxresident)k 0inputs+8outputs (0major+78099minor)pagefaults 0swaps

2016-06-06 23:51:02: ACTION[Main]: Successfully migrated 1483063 blocks 2.50user 0.29system 0:02.80elapsed 99%CPU (0avgtext+0avgdata 500016maxresident)k 0inputs+32outputs (0major+78098minor)pagefaults 0swaps

2016-06-06 23:51:10: ACTION[Main]: Successfully migrated 1483063 blocks 2.03user 0.21system 0:02.25elapsed 99%CPU (0avgtext+0avgdata 500024maxresident)k 0inputs+8outputs (0major+78099minor)pagefaults 0swaps

LevelDb

2016-06-06 23:51:55: ACTION[Main]: Successfully migrated 1483063 blocks 10.69user 0.34system 0:16.14elapsed 68%CPU (0avgtext+0avgdata 352388maxresident)k 117896inputs+72outputs (229major+78620minor)pagefaults 0swaps

2016-06-06 23:52:14: ACTION[Main]: Successfully migrated 1483063 blocks 9.36user 0.26system 0:09.72elapsed 98%CPU (0avgtext+0avgdata 352960maxresident)k 0inputs+64outputs (0major+78316minor)pagefaults 0swaps

2016-06-06 23:52:30: ACTION[Main]: Successfully migrated 1483063 blocks 9.21user 0.36system 0:09.67elapsed 98%CPU (0avgtext+0avgdata 352960maxresident)k 0inputs+48outputs (0major+78317minor)pagefaults 0swaps

2016-06-06 23:52:48: ACTION[Main]: Successfully migrated 1483063 blocks 9.50user 0.32system 0:09.91elapsed 99%CPU (0avgtext+0avgdata 352860maxresident)k 0inputs+40outputs (0major+78313minor)pagefaults 0swaps

SQLite3

2016-06-06 23:53:45: ACTION[Main]: Successfully migrated 1483063 blocks 13.87user 9.66system 0:32.31elapsed 72%CPU (0avgtext+0avgdata 319616maxresident)k 405192inputs+40outputs (190major+79005minor)pagefaults 0swaps

2016-06-06 23:54:13: ACTION[Main]: Successfully migrated 1483063 blocks 12.16user 8.63system 0:20.82elapsed 99%CPU (0avgtext+0avgdata 320252maxresident)k 0inputs+24outputs (0major+79055minor)pagefaults 0swaps

2016-06-06 23:54:41: ACTION[Main]: Successfully migrated 1483063 blocks 12.06user 8.77system 0:20.85elapsed 99%CPU (0avgtext+0avgdata 320216maxresident)k 0inputs+32outputs (0major+79056minor)pagefaults 0swaps

2016-06-06 23:55:10: ACTION[Main]: Successfully migrated 1483063 blocks 12.69user 8.50system 0:21.21elapsed 99%CPU (0avgtext+0avgdata 320208maxresident)k 0inputs+32outputs (0major+79055minor)pagefaults 0swaps

Write Test

For the write portion, I migrated from each backend to each other. Once again, caches are dropped before the first, but the remaining three runs occur with the cache primed by the previous run.

LMDB

LevelDB

2016-06-07 10:12:57: ACTION[Main]: Successfully migrated 1483063 blocks 12.93user 5.26system 0:27.67elapsed 65%CPU (0avgtext+0avgdata 245768maxresident)k 425872inputs+576888outputs (740major+14182minor)pagefaults 0swaps

2016-06-07 10:14:59: ACTION[Main]: Successfully migrated 1483063 blocks 12.71user 5.05system 0:17.22elapsed 103%CPU (0avgtext+0avgdata 247788maxresident)k 208inputs+583288outputs (0major+13104minor)pagefaults 0swaps

2016-06-07 10:16:09: ACTION[Main]: Successfully migrated 1483063 blocks 13.16user 5.25system 0:17.49elapsed 105%CPU (0avgtext+0avgdata 246012maxresident)k 8inputs+578512outputs (0major+12998minor)pagefaults 0swaps

2016-06-07 10:16:58: ACTION[Main]: Successfully migrated 1483063 blocks 13.12user 5.10system 0:16.88elapsed 107%CPU (0avgtext+0avgdata 246028maxresident)k 0inputs+577240outputs (0major+13013minor)pagefaults 0swaps

SQLite3

2016-06-07 10:19:13: ACTION[Main]: Successfully migrated 1483063 blocks 9.93user 0.68system 0:31.67elapsed 33%CPU (0avgtext+0avgdata 227012maxresident)k 426160inputs+368864outputs (740major+11052minor)pagefaults 0swaps

2016-06-07 10:20:21: ACTION[Main]: Successfully migrated 1483063 blocks 9.44user 0.62system 0:21.17elapsed 47%CPU (0avgtext+0avgdata 227428maxresident)k 288inputs+368928outputs (0major+9779minor)pagefaults 0swaps

2016-06-07 10:21:16: ACTION[Main]: Successfully migrated 1483063 blocks 9.51user 0.62system 0:20.39elapsed 49%CPU (0avgtext+0avgdata 227432maxresident)k 104inputs+368912outputs (0major+9780minor)pagefaults 0swaps

2016-06-07 10:22:20: ACTION[Main]: Successfully migrated 1483063 blocks 9.13user 0.49system 0:19.98elapsed 48%CPU (0avgtext+0avgdata 227392maxresident)k 128inputs+368568outputs (0major+9780minor)pagefaults 0swaps

LevelDB

LMDB

2016-06-07 10:25:40: ACTION[Main]: Successfully migrated 1483063 blocks 14.16user 0.74system 0:27.58elapsed 54%CPU (0avgtext+0avgdata 111584maxresident)k 116944inputs+513800outputs (205major+16562minor)pagefaults 0swaps

2016-06-07 10:26:39: ACTION[Main]: Successfully migrated 1483063 blocks 10.51user 0.40system 0:19.34elapsed 56%CPU (0avgtext+0avgdata 110776maxresident)k 104inputs+394208outputs (1major+16224minor)pagefaults 0swaps

2016-06-07 10:27:39: ACTION[Main]: Successfully migrated 1483063 blocks 10.63user 0.41system 0:19.05elapsed 57%CPU (0avgtext+0avgdata 109036maxresident)k 8inputs+394112outputs (1major+15853minor)pagefaults 0swaps

2016-06-07 10:29:21: ACTION[Main]: Successfully migrated 1483063 blocks 10.83user 0.38system 0:19.11elapsed 58%CPU (0avgtext+0avgdata 104480maxresident)k 0inputs+394136outputs (1major+14707minor)pagefaults 0swaps

SQLite3

2016-06-07 10:32:53: ACTION[Main]: Successfully migrated 1483063 blocks 23.62user 0.80system 0:46.66elapsed 52%CPU (0avgtext+0avgdata 79812maxresident)k 110592inputs+377144outputs (198major+10158minor)pagefaults 0swaps

2016-06-07 10:34:02: ACTION[Main]: Successfully migrated 1483063 blocks 23.54user 0.64system 0:42.07elapsed 57%CPU (0avgtext+0avgdata 80288maxresident)k 136inputs+376792outputs (0major+9962minor)pagefaults 0swaps

2016-06-07 10:35:27: ACTION[Main]: Successfully migrated 1483063 blocks 22.96user 0.61system 0:40.76elapsed 57%CPU (0avgtext+0avgdata 80292maxresident)k 24inputs+376008outputs (0major+9969minor)pagefaults 0swaps

2016-06-07 10:36:35: ACTION[Main]: Successfully migrated 1483063 blocks 22.86user 0.62system 0:41.22elapsed 56%CPU (0avgtext+0avgdata 80276maxresident)k 32inputs+376480outputs (0major+9966minor)pagefaults 0swaps

SQLite3

LMDB

2016-06-07 10:41:23: ACTION[Main]: Successfully migrated 1483063 blocks 16.38user 11.76system 0:50.61elapsed 55%CPU (0avgtext+0avgdata 40752maxresident)k 396168inputs+365800outputs (165major+8336minor)pagefaults 0swaps

2016-06-07 10:47:30: ACTION[Main]: Successfully migrated 1483063 blocks 13.60user 9.45system 0:31.56elapsed 73%CPU (0avgtext+0avgdata 42860maxresident)k 8inputs+365416outputs (1major+8972minor)pagefaults 0swaps

2016-06-07 10:51:09: ACTION[Main]: Successfully migrated 1483063 blocks 13.08user 9.64system 0:30.79elapsed 73%CPU (0avgtext+0avgdata 42532maxresident)k 0inputs+365376outputs (1major+8974minor)pagefaults 0swaps

2016-06-07 10:52:23: ACTION[Main]: Successfully migrated 1483063 blocks 13.70user 9.78system 0:31.90elapsed 73%CPU (0avgtext+0avgdata 43712maxresident)k 0inputs+365416outputs (1major+9145minor)pagefaults 0swaps

LevelDB

2016-06-07 10:55:11: ACTION[Main]: Successfully migrated 1483063 blocks 29.69user 17.90system 1:01.14elapsed 77%CPU (0avgtext+0avgdata 61004maxresident)k 396376inputs+599200outputs (164major+9970minor)pagefaults 0swaps

2016-06-07 10:56:34: ACTION[Main]: Successfully migrated 1483063 blocks 26.75user 16.13system 0:39.08elapsed 109%CPU (0avgtext+0avgdata 62760maxresident)k 184inputs+599208outputs (0major+10379minor)pagefaults 0swaps

2016-06-07 10:57:35: ACTION[Main]: Successfully migrated 1483063 blocks 27.24user 16.47system 0:39.96elapsed 109%CPU (0avgtext+0avgdata 63236maxresident)k 16inputs+599200outputs (0major+10477minor)pagefaults 0swaps

2016-06-07 10:58:33: ACTION[Main]: Successfully migrated 1483063 blocks 26.26user 16.20system 0:38.69elapsed 109%CPU (0avgtext+0avgdata 61440maxresident)k 16inputs+599184outputs (0major+10067minor)pagefaults 0swaps

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment