Skip to content

Instantly share code, notes, and snippets.

@nesbert
Last active April 15, 2023 15:24
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save nesbert/b790fc6096d8c8c4ffe0d1b57f5f7711 to your computer and use it in GitHub Desktop.
Save nesbert/b790fc6096d8c8c4ffe0d1b57f5f7711 to your computer and use it in GitHub Desktop.
Need a quick way to remove old (unwanted) documents from MongoDB without needing schema context.

MongoDB Delete By ObjectId Creation Time

MongoDB's ObjectId contains a timestamp, which can help you filter documents based on their creation date. To reduce the size of your collection, you can use a strategy that involves deleting or archiving old documents based on a specific date or time range.

Here's a high-level overview of the process:

  • Determine the date threshold: Decide on a date or time range beyond which you would like to remove or archive the documents. For example, you might want to remove documents older than 6 months or 1 year.
  • Query documents based on the ObjectId's timestamp: You can use MongoDB's aggregation framework or the find() method to filter documents based on the creation date embedded in the ObjectId.

Step 1: Find Documents Older Than...

Here's an example of how to create a filter based on the ObjectId's timestamp using the MongoDB shell:

var thresholdDate = new Date();
thresholdDate.setMonth(thresholdDate.getMonth() - 6); // This sets the threshold to 6 months ago.
var objectIdThreshold = Math.floor(thresholdDate.getTime() / 1000).toString(16) + "0000000000000000";

db.yourCollection.find({ "_id": { "$lt": ObjectId(objectIdThreshold) } });

Replace yourCollection with the name of your collection and adjust the thresholdDate calculation based on your desired date range.

Step 2a: Delete Documents

Depending on your requirements, you can either delete the documents permanently or archive them to another collection or storage system.

db.yourCollection.deleteMany({ "_id": { "$lt": ObjectId(objectIdThreshold) } });

Step 2b: Archive Documents

To archive the documents, you can use the aggregate() function with $out or $merge stages to create a new collection with the filtered documents, and then remove them from the original collection:

db.yourCollection.aggregate([
  { "$match": { "_id": { "$lt": ObjectId(objectIdThreshold) } } },
  { "$out": "yourArchiveCollection" } // Use "$merge" instead of "$out" if you want to append to an existing collection.
]);

db.yourCollection.deleteMany({ "_id": { "$lt": ObjectId(objectIdThreshold) } });

Replace yourArchiveCollection with the name of your archive collection.

By following these steps, you can effectively reduce the size of your collection by removing or archiving old documents based on their creation date. Note that these operations can be resource-intensive, so it's a good idea to perform them during periods of low database usage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment