Skip to content

Instantly share code, notes, and snippets.

@korczis
Created February 9, 2014 20:00
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save korczis/8905062 to your computer and use it in GitHub Desktop.
Save korczis/8905062 to your computer and use it in GitHub Desktop.
Simple NGA Geonames to Mongo Importer
var mongo = new Mongo(config);
var bulk = [];
mongo.initialize().then(function() {
return mongo.getCollection("geonames");
}).then(function(collection) {
var d = deferred();
var etl = new Etl();
etl.load("c:\\Downloads\\geonames_20140203.txt", {delimiter: "\t", columns: true, relax: true}).on('record', function(row, index) {
//console.log(JSON.stringify(row, null, 4));
console.log(index);
bulk.push(row);
if(bulk.length === 1000) {
collection.insert(bulk, function(err, docs) {
});
bulk = [];
}
}).on('error', function(error){
console.log(error.message);
}).on('end', function() {
collection.insert(bulk, function(err, docs) {
});
bulk = [];
d.resolve(true);
});
return d.promise();
});
return;
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment