Skip to content

Instantly share code, notes, and snippets.

@darius-sas
Last active March 25, 2021 14:33
Show Gist options
  • Save darius-sas/df4aac73416fcae92892be787e6eac20 to your computer and use it in GitHub Desktop.
Save darius-sas/df4aac73416fcae92892be787e6eac20 to your computer and use it in GitHub Desktop.
Run jobs in parallel in bash
#!/bin/bash
# Copied from https://milhouse.dev/2015/11/20/writing-a-process-pool-in-bash/
# Define the pool size (aka number of processes to run concurrently)
POOL_SIZE=4
parallel() {
local proc procs
declare -a procs=() # this declares procs as an array
morework=true
while $morework; do
if [[ "${#procs[@]}" -lt "$POOL_SIZE" ]]; then
read proc || { morework=false; continue ;}
eval "$proc" &
procs["${#procs[@]}"]="$!"
fi
for n in "${!procs[@]}"; do
kill -0 "${procs[n]}" 2>/dev/null && continue
unset procs[n]
done
done
wait
}
# invoke the process pool (one task per line)
parallel <<EOF
sleep 2 && echo "hello 1"
sleep 1 && echo "hello 2"
echo "hello 3"
EOF
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment