Skip to content

Instantly share code, notes, and snippets.

@derickson
Created July 16, 2015 15:18
Show Gist options
  • Save derickson/0ea776df13786093ebec to your computer and use it in GitHub Desktop.
Save derickson/0ea776df13786093ebec to your computer and use it in GitHub Desktop.
Example of NodeJS Loop of Mongo to Elasticsearch
// npm install elasticsearch
// setup nodejs client for elasticsearch
// documentation: https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current/index.html
var elasticsearch = require('elasticsearch');
var EsClient = new elasticsearch.Client({
host: 'localhost:9200',
log: 'info'
});
// npm install mongodb
// setup nodjs driver for MongoDB
var MongoClient = require('mongodb').MongoClient;
var ObjectID = require('mongodb').ObjectID;
var mongoDBName = 'heroes';
var mongoCollectionName = 'characters';
var connectionString = 'mongodb://127.0.0.1:27017/'; // put username and password for mongo here
var esIndexName = 'herocharacters';
// connect to Mongodb
MongoClient.connect(connectionString+mongoDBName, function(err, db) {
if(err) throw err;
// for each object in a collection
var collection = db.collection(mongoCollectionName);
var counter = 0;
collection.find().each(function(err, item) {
if(item != null) {
if(counter % 100 == 0) console.log( "Syncing object id: "+ item['_id'] + " #: " + counter);
EsClient.index(
{index: esIndexName, type:mongoCollectionName, id: item._id.toString() , body: item },
function(error, response) {
if(err) throw err;
}
);
}
counter += 1;
});
});
@derickson
Copy link
Author

Apologies for this not being the most awesome NodeJS code ever written

@psyanite
Copy link

Nice job!

@saiharshanani
Copy link

Field [_id] is a metadata field and cannot be added inside a document. Use the index API request parameters. Having this issue. Tried renaming id field to other name but no luck. It processed some of the documents.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment