InfluxDB testing

AA

SSD

These tests are incomplete because I still can't get the HyperLevelDB driver to build, and various other oddities in the build tree.

violino:/home/software/influxdb> ./benchmark-storage -path=/home/test/db -points=3000000
################ Benchmarking: lmdb
Writing 3000000 points in batches of 1000 points took 8.579291321s (2.859764 microsecond per point)
Querying 3000000 points took 1.743559302s (0.581186 microseconds per point)
Size: 11M
Took 1.925530865s to delete 1500000 points
Took 1.118us to compact
Querying 1500002 points took 844.85632ms (0.563237 microseconds per point)
Size: 11M
Writing 1500000 points in batches of 1000 points took 4.162090524s (2.774727 microsecond per point)
Size: 11M
################ Benchmarking: leveldb
Writing 3000000 points in batches of 1000 points took 11.573577863s (3.857859 microsecond per point)
Querying 3000000 points took 2.925480119s (0.975160 microseconds per point)
Size: 66M
Took 4.259787827s to delete 1500000 points
Took 5.733182801s to compact
Querying 1500000 points took 1.386606671s (0.924404 microseconds per point)
Size: 33M
Writing 1500000 points in batches of 1000 points took 5.796011589s (3.864008 microsecond per point)
Size: 68M
################ Benchmarking: rocksdb
Writing 3000000 points in batches of 1000 points took 10.028515133s (3.342838 microsecond per point)
Querying 3000000 points took 2.951286316s (0.983762 microseconds per point)
Size: 92M
Took 2.374496266s to delete 1500000 points
Took 6.537264862s to compact
Querying 1500000 points took 1.248761975s (0.832508 microseconds per point)
Size: 46M
Writing 1500000 points in batches of 1000 points took 4.897056185s (3.264704 microsecond per point)
Size: 94M
violino:/home/software/influxdb> ./benchmark-storage -path=/home/test/db -points=30000000
################ Benchmarking: lmdb
Writing 30000000 points in batches of 1000 points took 1m26.404038187s (2.880135 microsecond per point)
Querying 30000000 points took 18.093649965s (0.603122 microseconds per point)
Size: 1.3G
Took 20.194356605s to delete 15000000 points
Took 1.536us to compact
Querying 15000000 points took 8.898250048s (0.593217 microseconds per point)
Size: 2.0G
Writing 15000000 points in batches of 1000 points took 42.97290309s (2.864860 microsecond per point)
Size: 2.0G
################ Benchmarking: leveldb
Writing 30000000 points in batches of 1000 points took 2m5.937896748s (4.197930 microsecond per point)
Querying 30000000 points took 30.234725191s (1.007824 microseconds per point)
Size: 647M
Took 41.348762307s to delete 15000000 points
Took 1m13.872348346s to compact
Querying 15000000 points took 13.97266771s (0.931511 microseconds per point)
Size: 324M
Writing 15000000 points in batches of 1000 points took 1m0.777067561s (4.051805 microsecond per point)
Size: 648M
################ Benchmarking: rocksdb
Writing 30000000 points in batches of 1000 points took 1m48.979719649s (3.632657 microsecond per point)
Querying 30000000 points took 29.560162299s (0.985339 microseconds per point)
Size: 927M
Took 25.000730523s to delete 15000000 points
Took 59.666849377s to compact
Querying 15000000 points took 13.888019542s (0.925868 microseconds per point)
Size: 494M
Writing 15000000 points in batches of 1000 points took 56.759511542s (3.783967 microsecond per point)
Size: 953M
violino:/home/software/influxdb>  ./benchmark-storage -path=/home/test/db -points=100000000
################ Benchmarking: lmdb
Writing 100000000 points in batches of 1000 points took 4m53.04954666s (2.930495 microsecond per point)
Querying 100000000 points took 1m5.43905056s (0.654391 microseconds per point)
Size: 4.3G
Took 1m16.695704766s to delete 50000000 points
Took 388.183us to compact
Querying 50000000 points took 32.59158313s (0.651832 microseconds per point)
Size: 6.4G
Writing 50000000 points in batches of 1000 points took 2m37.797819839s (3.155956 microsecond per point)
Size: 6.4G
################ Benchmarking: leveldb
Writing 100000000 points in batches of 1000 points took 7m7.246269261s (4.272463 microsecond per point)
Querying 100000000 points took 1m41.400523494s (1.014005 microseconds per point)
Size: 2.2G
Took 2m30.613254933s to delete 50000000 points
Took 5m12.345083712s to compact
Querying 50000000 points took 46.14220981s (0.922844 microseconds per point)
Size: 1.1G
Writing 50000000 points in batches of 1000 points took 3m29.78487308s (4.195697 microsecond per point)
Size: 2.2G
################ Benchmarking: rocksdb
Writing 100000000 points in batches of 1000 points took 6m11.153530942s (3.711535 microsecond per point)
Querying 100000000 points took 1m46.161943384s (1.061619 microseconds per point)
Size: 3.1G
Took 1m39.841894083s to delete 50000000 points
Took 4m28.481438731s to compact
Querying 50000000 points took 41.348830623s (0.826977 microseconds per point)
Size: 1.6G
Writing 50000000 points in batches of 1000 points took 3m3.317511175s (3.666350 microsecond per point)
Size: 3.1G

After

Your test uncovered some page allocation stupidity in our code when handling deletes. With that fixed, the DB no longer grows during deletes:

violino:/home/software/influxdb> ./benchmark-storage -path=/home/test/db -points=100000000
################ Benchmarking: lmdb
Writing 100000000 points in batches of 1000 points took 4m49.551874408s (2.895519 microsecond per point)
Querying 100000000 points took 1m2.715090003s (0.627151 microseconds per point)
Size: 4.3G
Took 1m2.297495295s to delete 50000000 points
Took 1.677us to compact
Querying 50000000 points took 30.656723697s (0.613134 microseconds per point)
Size: 4.3G
Writing 50000000 points in batches of 1000 points took 2m20.864793642s (2.817296 microsecond per point)
Size: 4.3G