Created
December 20, 2023 08:42
-
-
Save coderofsalvation/6987b37222ee37ba2adace7844ef5757 to your computer and use it in GitHub Desktop.
parallel workflow orchestration using wait-cmd in shellscript #bash #parallel #shellscript #speedupCI #windmill #airflow
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
# | |
# Q: how can I track & run tasks in parallel (speed up CI pipeline e.g.) | |
# confidently without introducing platforms like Apache | |
# Airflow or Windmilll? | |
# | |
# A: the 'wait' command in shellscript | |
# | |
set -eE && trap "echo '❌ halting flow'" ERR | |
run(){ time "$@" 2>&1 | sed "s/^/[$1] /g" | tee .log; } | |
run ./mycmd & | |
run ./othercmd & | |
wait | |
grep -q ❌ .log && exit 1 || exit 0 | |
# OUTPUT: | |
# | |
# [./othercmd] dolor sit amet | |
# real 0m2.010s | |
# user 0m0.012s | |
# sys 0m0.018s | |
# [./mycmd] lorem ipsum | |
# real 0m3.010s | |
# user 0m0.012s | |
# sys 0m0.018s | |
# shell returned 0 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment