Skip to content

Instantly share code, notes, and snippets.

@fasiha
Last active June 13, 2024 05:02
Show Gist options
  • Save fasiha/c04744393e0aafd07b0603593daacaa1 to your computer and use it in GitHub Desktop.
Save fasiha/c04744393e0aafd07b0603593daacaa1 to your computer and use it in GitHub Desktop.
Node.js, very simple work queue
export type Task = () => Promise<void>;
export function addTask(doTask: Task) {
TASKS.push(doTask);
kickOffWorkers(MAX_WORKERS);
}
const TASKS: Task[] = [];
const workerPool: number[] = []; // "number" but really doesn't matter, this is just to keep track of how many parallel workers we already have
const MAX_WORKERS = 3;
function kickOffWorkers(numWorkers: number = MAX_WORKERS) {
const numCurrentWorkers = workerPool.length;
if (numCurrentWorkers < numWorkers) {
const takeTaskAndRecurse = async () => {
const task = TASKS.pop();
if (!task) {
// delete
workerPool.splice(numCurrentWorkers, 1);
} else {
queueMicrotask(async () => {
await task();
queueMicrotask(takeTaskAndRecurse);
});
}
};
queueMicrotask(takeTaskAndRecurse);
workerPool.push(Date.now()); // like I said, contents really don't matter
// try to fill the worker pool
kickOffWorkers(numWorkers);
}
}
// DEMO
(async function () {
const sleep = (milliseconds: number) => new Promise((resolve) => setTimeout(resolve, milliseconds));
const sleepAndLog = (milliseconds: number, log: string) =>
sleep(milliseconds).then(() => console.log(log, new Date()));
console.log(`${MAX_WORKERS} threads`);
console.log("_", new Date());
addTask(async () => await sleepAndLog(1000, "1"));
addTask(async () => await sleepAndLog(1000, "2"));
addTask(async () => await sleepAndLog(1000, "3"));
await sleep(4000);
console.log("_", new Date());
addTask(async () => await sleepAndLog(1000, "4"));
addTask(async () => await sleepAndLog(1000, "5"));
addTask(async () => await sleepAndLog(1000, "6"));
})();
@fasiha
Copy link
Author

fasiha commented Jun 13, 2024

Output with MAX_THREADS going from 1 to 3:

$ npx tsx index.ts
1 threads
_ 2024-06-13T04:47:46.777Z
3 2024-06-13T04:47:47.780Z
2 2024-06-13T04:47:48.781Z
1 2024-06-13T04:47:49.783Z
_ 2024-06-13T04:47:50.778Z
6 2024-06-13T04:47:51.780Z
5 2024-06-13T04:47:52.781Z
4 2024-06-13T04:47:53.783Z

$ npx tsx index.ts
2 threads
_ 2024-06-13T04:47:55.682Z
3 2024-06-13T04:47:56.686Z
2 2024-06-13T04:47:56.687Z
1 2024-06-13T04:47:57.687Z
_ 2024-06-13T04:47:59.685Z
6 2024-06-13T04:48:00.686Z
5 2024-06-13T04:48:00.686Z
4 2024-06-13T04:48:01.688Z

$ npx tsx index.ts
3 threads
_ 2024-06-13T04:48:04.119Z
3 2024-06-13T04:48:05.123Z
2 2024-06-13T04:48:05.124Z
1 2024-06-13T04:48:05.124Z
_ 2024-06-13T04:48:08.122Z
6 2024-06-13T04:48:09.124Z
5 2024-06-13T04:48:09.126Z
4 2024-06-13T04:48:10.127Z

@fasiha
Copy link
Author

fasiha commented Jun 13, 2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment