Instantly share code, notes, and snippets.

Embed
What would you like to do?
PouchDB 2.2.0 performance report, April 2014

PouchDB 2.2.0 performance report

TL;DR

PouchDB 2.2.0, which will be released in early May, will boast an 80x-100x speed improvement in query() calls in IndexedDB (Chrome, Firefox, IE) and a roughly 40x speed improvement in WebSQL (Safari, Cordova). This requires that you first save the view before querying (so it can be indexed), but we will provide sugar to make that easy.

The old-style query() calls with temporary views, however, will be up to 100x slower due to no longer being performed in-memory. So developers will need to upgrade carefully.

Edit: Take these numbers with a grain of salt; it depends on your database size. Our tests are on a database with 1,000 documents.

Performance for allDocs() (a.k.a. bulk reads) is roughly 2x-8x improved. Calling allDocs() using the skip/limit pattern is confirmed to be a disaster for IndexedDB, but the startkey pattern performs fine, which is good, because otherwise I'd be a liar.

Performance test results

These are results from browser tests I ran on April 19, 2014, comparing PouchDB 2.2.0-alpha to PouchDB 2.1.2. The 2.2.0-alpha version is an experimental build that hasn't been fully code-reviewed and merged yet, but I expect it'll be merged without significant changes in time for the May release.

Chrome 34, Safari 6.1, Firefox 28, and IE 11 were tested. Node.js (LevelDB) was not tested due to technical difficulties with the test harness, but hopefully that'll be fixed soon.

All numbers are reported in total milliseconds to run the tests, so smaller numbers are better.

The tests were run on a 2013 MacBook Air. IE 11 was run in Virtual Box, using the 8.1 version of Windows, so it's probably not a fair fight if you try to compare the absolute numbers to the other browsers'.

Basic Tests

Let's start with allDocs(). In these tests, I'm explicitly trying to call out skip/limit as being terrible for performance, so it shouldn't surprise us that the startkey pattern does better. 1,000 docs are loaded into the database, then we page through all of them in 100-page increments (i.e. 10 queries). This is performed 50 times.

Change in speed:

Chrome (IDB) Firefox IE Chrome (SQL) Safari
2.00x 0.96x 1.15x 1.75x 2.19x

Change in speed:

Chrome (IDB) Firefox IE Chrome (SQL) Safari
8.04x 1.60x 3.30x 1.30x 3.07x

I also ran a basic-inserts test, which just inserts 1,000 docs using db.post(). Nothing much to see here.

Map/reduce query tests

Of course, persistent map/reduce is the star of the show for 2.2.0, so this is where it gets really interesting. In order to keep the numbers pure, a design doc with a view is inserted, then an initial query is performed to do the first build of the database.

So in a sense it's cheating a bit, because it discounts the cost of building the index. I think that's fair, though, since otherwise the performance would get "better" with every additional iteration, which wouldn't give us the information we want to isolate.

If you're wondering how this used to work pre-2.2.0, there's no trick: it just did everything in memory, regardless of whether you saved a design doc with a view or not.

Change in speed:

Chrome (IDB) Firefox IE Chrome (SQL) Safari
61.32x 72.80x 58.79x 10.67x 11.84x

Now the same test with {stale: 'ok'}, which tells PouchDB not to bother checking for updates to the database:

Change in speed:

Chrome (IDB) Firefox IE Chrome (SQL) Safari
83.25x 94.36x 94.52x 34.91x 46.95x

These two are pretty much the same, but {stale: 'ok'} is a bit faster due to being able to skip the update check. (In these tests, there's nothing to update anyway.)

Now, the best way to measure the cost of building the views is in the temp-views test. Here we compare the old method (building and querying the whole index in memory) with the new, more CouchDB-style method (build a temporary view, query it, then delete it).

Change in speed:

Chrome (IDB) Firefox IE Chrome (SQL) Safari
0.02x 0.03x N/A 0.01x 0.01x

The values are kinda hard to see (check the link at the bottom for the numbers), but what's obvious it that the performance has nosedived. IE never even finished.

This is why we're going to recommend that, before upgrading to 2.2.0, developers should analyze their use of query() and decide whether to switch to an allDocs() call (and continue doing things in-memory) or to start using persistent views (probably the superior choice in most cases, especially on mobile).

If these numbers scare you, though, keep in mind that they represent queries on a database containing 1,000 documents, and the tests are performed 10 times. So the fact that we can index 10,000 documents in ~10 minutes (and on a MacBook Air) is actually not so bad. That's 45 docs/minute in the fastest browser (Safari) and 12 docs/minute in the slowest (Firefox). If your database is small, you'll probably be able to upgrade to 2.2.0 with your code unchanged, and without seeing any visible dip in performance.

Furthermore, this kind of behavior is in line with what you'd typically expect from CouchDB – temporary views are slow, so unless you're just testing during development, you better save your views first.

Conclusion

PouchDB 2.2.0 is gonna be screaming fast. In addition to what you see here, there were also many performance improvements to LevelDB, which we'll try to report soon.

And of course, this is just the beginning. We're always working to make PouchDB leaner, meaner, and funner, now and beyond the 2.2.0 release.

References

To reproduce the tests yourself, use this PouchDB version and this PouchDB Map/Reduce version. Instructions are in CONTRIBUTING.md.

Google Doc containing the data is here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment