Skip to content

Instantly share code, notes, and snippets.

@Xetera
Created February 15, 2019 10:40
Show Gist options
  • Save Xetera/a9c9d7649fdcb3aeff0c776416b57a91 to your computer and use it in GitHub Desktop.
Save Xetera/a9c9d7649fdcb3aeff0c776416b57a91 to your computer and use it in GitHub Desktop.
Processing an array of promises* in batches of X requests at a time
/**
* (*) Due to the eager nature of promises, `items` must be an
* array of promise returning functions and not an array of promises
*/
const batchProcess = (count, items) => {
const chunked = R.splitEvery(count, items);
const process = async ([head, ...tail]) => {
const promises = head.map(func => func());
const results = await Promise.all(promises);
if (!tail.length) {
return results;
}
return [...results, ...await process(tail)];
}
return process(chunked);
}
const results = await batchProcess(50, aVeryVeryLargeAmountOfRequests);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment