Skip to content

Instantly share code, notes, and snippets.

@comerford
Last active October 6, 2022 15:13
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save comerford/9250698 to your computer and use it in GitHub Desktop.
Save comerford/9250698 to your computer and use it in GitHub Desktop.
Function to Pre-Heat data using ObjectID timestamp component in MongoDB
// function will take a number of days, a collection name, an index Field name, and a boolean as args
// it assumes the index is ObjectID based and creates an ObjectID with a timestamp X days in the past
// Finally, it queries that index/data (controlled by boolean), loading it into memory
//
// Example - 2 days data, foo collection, _id index, pre-heat index only
// preHeatData(2, "foo", "_id", true)
// Example - 7 days data, foo collection, _id index, pre-heat data also
// preHeatData(7, "foo", "_id", false)
// Example - 2 days data, bar collection, blah index, pre-heat index only
// preHeatData(2, "bar", "blah", false)
preHeatData = function(numDays, collName, indexField, indexOnly) {
// rounding is needed here to chop off the fractional seconds
// ObjectID is to the second
lastXDays = Math.round(((new Date().getTime() - (numDays * 24 * 60 * 60))/1000));
// convert to hexadecimal
hexDays = lastXDays.toString(16);
// create and pad a string and pass it to the ObjectId constructor
historicId = ObjectId(hexDays+"0000000000000000");
// Now query for it:
if (indexOnly && indexField != "_id") {
// if indexOnly is true, and not the _id index include a projection to make this a covered query
db.collName.find({indexField : {"$gt" : historicId}}, {"_id" : 0, indexField : 1}).hint({indexField : 1}).explain();
print("Loaded index Only, pre-heated the " + indexField + " index from the " + collName + " collection");
}
else if(indexOnly) {
// if it is the _id field, simpler projection
db.collName.find({indexField : {"$gt" : historicId}}, {indexField : 1}).hint({indexField : 1}).explain();
print("Loaded index Only, pre-heated the " + indexField + " index from the " + collName + " collection");
}
else {
// finally, the general case, where we return the data also
db.collName.find({indexField : {"$gt" : historicId}}).hint({indexField : 1}).explain();
print("Pre-heated the " + indexField + " index and data from the " + collName + " collection");
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment