Skip to content

Instantly share code, notes, and snippets.

@csabapalfi
Last active March 25, 2016 09:51
Show Gist options
  • Save csabapalfi/01cc60b6b918b8628909 to your computer and use it in GitHub Desktop.
Save csabapalfi/01cc60b6b918b8628909 to your computer and use it in GitHub Desktop.
Dedupe records in Mongo with mongoose
db.downloads.aggregate(
{ '$group': { _id: '$userId', total: { '$sum': 1 } } },
{ '$match': { total: { '$gte': 2 } } }
);
_.forEach(dupeIds, function(dupeId) {
db.find({ userId: dupeId }, function(err, result) {
if(err || !result || result && result.length < 2) {
console.log('skipping ', dupe);
} else {
//manually dedupe into first record, this is custom to your schema
var resources1 = _.indexBy(result[0].resources, 'resourceId');
var resources2 = _.indexBy(result[1].resources, 'resourceId');
var mergedResources = _.values(_.merge(resources1, resources2));
result[0].resources = mergedResources;
//save first, remove duplicate
result[0].save(function(err) {
if(err) console.error(err);
result[1].remove(function(err) {
if(err) console.error(err);
});
});
}
});
});
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment