Skip to content

Instantly share code, notes, and snippets.

@Deepak13245
Created June 2, 2020 20:17
Show Gist options
  • Save Deepak13245/6d5aac83a4ec5a6976be06f8626552d5 to your computer and use it in GitHub Desktop.
Save Deepak13245/6d5aac83a4ec5a6976be06f8626552d5 to your computer and use it in GitHub Desktop.
Run async functions in parallel with limited amout of concurrency.
async function inParallelWithLimit(list, concurrency, fn) {
// chunk is imported from here :- https://gist.github.com/Deepak13245/129ecdbb72c9f0f81827eb421260a8da
const batches = chunk(list, concurrency);
// Process batches in series
// inSeries :- https://gist.github.com/Deepak13245/c39c67c45abbf5d2a357fba5beeabbde
// inParallel :- https://gist.github.com/Deepak13245/e49c1853ebceacbd3ab6f725c42b4ba6
const data = await inSeries(batches, async (items, batch) => {
// Process items in a batch in parallel
return inParallel(items, fn);
});
// Flatten the array
return data.reduce((acc, items) => [...acc,...items], []);
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment