Skip to content

Instantly share code, notes, and snippets.

@max-mapper
Created January 14, 2012 02:27
Show Gist options
  • Save max-mapper/1609973 to your computer and use it in GitHub Desktop.
Save max-mapper/1609973 to your computer and use it in GitHub Desktop.
streaming functional transformer for couchdb using node
var request = require('request').defaults({json: true}),
transfuse = require('transfuse'),
JSONStream = require('JSONStream');
function transform(couchdb, funcString, headers) {
var down = request({url: couchdb + '/_all_docs?include_docs=true'}),
up = request({url: couchdb + '/_bulk_docs', method: "POST", headers: headers}),
tr = transfuse(['rows', /./, 'doc'], funcString, JSONStream.stringify("{\"docs\":[\n", "\n,\n", "\n]}\n"));
down.pipe(tr)
tr.pipe(up)
}
// transform('http://localhost:5984/cats', 'function(doc, map) { doc.type = "cat"; map(doc) }')
@dscape
Copy link

dscape commented Jan 14, 2012

yeah :) but you might be streaming something large into couch :)

also selectors like mikeal suggested really help performance. I only implemented simple dot notation but in the best case scenario you could go from taking 2 seconds to parse npmjs to 0.05.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment