CouchDB Benchmark

Results from http://gist.github.com/166802 script.

This test is to see the performance hit on CouchDB as more rows are loaded into the database.

Specifically how long it takes to insert 1000 rows in a batch insert as the database continues to grow.

This is what I'm seeing on an EC2 small instance.
Measurements are in the number of seconds to insert that batch of 1000 into CouchDB.
(Batch 1 - 1,000 rows in DB)
1       1.02298092842102
2       1.2157130241394
3       1.27432703971863
... (Batch 1000 - 1,000,000 rows in DB)
1000    2.84813213348389
1001    2.77256989479065
1002    2.7364330291748
... (Batch 2000 - 2,000,000 rows in DB)
2000    3.1963050365448
2001    3.39697504043579
2002    3.12854719161987
... (Batch 3000 - 3,000,000 rows in DB)
3000    3.18745613098145
3001    3.53188800811768
3002    3.15942788124084
... (Batch 4000 - 4,000,000 rows in DB)
4000    5.17613315582275
4001    4.85200691223145
4002    4.81283807754517
... (Batch 5000 - 5,000,000 rows in DB)
5000    5.90975499153137
5001    6.04520988464355
5002    6.80835199356079
... (Batch 6000 - 6,000,000 rows in DB)
6000    7.45973920822144
6001    7.0675630569458
6002    7.61369109153748
... (Batch 7000 - 7,000,000 rows in DB)
7000    9.27594900131226
7001    8.96303296089172
7002    8.10933589935303
... (Batch 8000 - 8,000,000 rows in DB)
8000    7.96305799484253
8001    8.68896293640137
8002    11.5279979705811
... (Batch 9000 - 9,000,000 rows in DB)
9000    10.8936779499054
9001    10.2960419654846
9002    10.0917880535126
... (Batch 10000 - 10,000,000 rows in DB)
9998    11.4294641017914
9999    11.2450361251831
10000   11.7544960975647

# time ruby couchload.rb

real    1068m23.757s
user    39m1.858s
sys     4m0.451s


This was the problem I wanted to demonstrate.  As the DB continues to grow the time to insert 1000 rows in a
single batch insert grows linearly.  I plan to benchmark this on other btree based DBs to see if this is a
problem with other key value stores or just with CouchDB.