Skip to content

Instantly share code, notes, and snippets.

@intolerance
Last active November 6, 2019 21:12
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save intolerance/683b406b652ebe17bc0a38dfc6cc0132 to your computer and use it in GitHub Desktop.
Save intolerance/683b406b652ebe17bc0a38dfc6cc0132 to your computer and use it in GitHub Desktop.
Finds and remove duplicates from MongoDB without aggregation
const devs = {};
Device.find({}, (err, devices) => {
devices.forEach((device) => {
if (!devs || !devs[device.uid]) {
devs[device.uid] = [];
}
devs[device.uid].push(device);
});
const dupesOnly = {};
Object.keys(devs).forEach((key) => {
if (devs[key].length > 1) {
dupesOnly[key] = devs[key];
console.log(`Keeping ${devs[key][0]._id} and Removing ${devs[key][1]._id}!`);
Device.findByIdAndRemove({ _id: devs[key][1]._id }, (err3) => {
if (err3) {
console.error(err3);
}
});
}
});
// const dupesOnly = Object.keys(devs).filter(dev => dev.length > 1);
fs.writeFileSync('./devsDups.json', JSON.stringify(dupesOnly, '\t', 2), 'utf8');
@intolerance
Copy link
Author

Finds and remove duplicates from MongoDB without aggregation (it wasn't available to me)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment