Skip to content

Instantly share code, notes, and snippets.

@coderofsalvation
Created December 20, 2023 08:42
Show Gist options
  • Save coderofsalvation/6987b37222ee37ba2adace7844ef5757 to your computer and use it in GitHub Desktop.
Save coderofsalvation/6987b37222ee37ba2adace7844ef5757 to your computer and use it in GitHub Desktop.
parallel workflow orchestration using wait-cmd in shellscript #bash #parallel #shellscript #speedupCI #windmill #airflow
#!/bin/bash
#
# Q: how can I track & run tasks in parallel (speed up CI pipeline e.g.)
# confidently without introducing platforms like Apache
# Airflow or Windmilll?
#
# A: the 'wait' command in shellscript
#
set -eE && trap "echo '❌ halting flow'" ERR
run(){ time "$@" 2>&1 | sed "s/^/[$1] /g" | tee .log; }
run ./mycmd &
run ./othercmd &
wait
grep -q ❌ .log && exit 1 || exit 0
# OUTPUT:
#
# [./othercmd] dolor sit amet
# real 0m2.010s
# user 0m0.012s
# sys 0m0.018s
# [./mycmd] lorem ipsum
# real 0m3.010s
# user 0m0.012s
# sys 0m0.018s
# shell returned 0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment