Skip to content

Instantly share code, notes, and snippets.

View carpntr's full-sized avatar
⌨️

Andrew Carpenter carpntr

⌨️
View GitHub Profile
@thehesiod
thehesiod / async_worker_pool.py
Last active June 30, 2023 11:01
Asynchronous Worker Pool, allows for limiting number of concurrent tasks
import asyncio
from datetime import datetime, timezone
import os
def utc_now():
# utcnow returns a naive datetime, so we have to set the timezone manually <sigh>
return datetime.utcnow().replace(tzinfo=timezone.utc)
class Terminator:
pass
@alexpsi
alexpsi / asyncChunkedMap.js
Created February 25, 2017 20:31
Like Promise.all but executed in chunks, so you can have some async concurrency control.
const chunks = (arr, chunkSize) => {
let results = [];
while (arr.length) results.push(arr.splice(0, chunkSize));
return results;
};
module.exports = (xs, f, concurrency) => {
if (xs.length == 0) return Promise.resolve();
return Promise.all(chunks(xs, concurrency).reduce(
(acc, chunk) => acc.then(() => Promise.all(chunk.map(f))),