Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save HakimCassimallyBBC/e6d4d88dd20b03eaf2514124505d78a4 to your computer and use it in GitHub Desktop.
Save HakimCassimallyBBC/e6d4d88dd20b03eaf2514124505d78a4 to your computer and use it in GitHub Desktop.
multicurl attempt
#!/bin/bash
FIFO=$(mktemp /tmp/multicurl.XXXXX)
echo FIFO=$FIFO
for arg in "$@"
do
curl $arg > $FIFO &
done
# this doesn't work as:
# - both are noisy
# - both streams overlap (want line buffered)
# - ctrl-c leaves the curls running in background
@fishface60
Copy link

To avoid interleaving you might need something like:

for arg in "$@"; do
    curl | while read -r line; do printf "%s\n" "$line" >"$FIFO"; done &
done

while true; do
    cat "$FIFO"
done

Terminating when all the curl processes have finished is not solved.

@fishface60
Copy link

# Assuming bash
pids=()
for arg in "$@"; do
    curl | while read -r line; do printf "%s\n" "$line" >"$FIFO"; done &
    pids+="$!"
done

trap 'for pid in "${PIDS}"; do kill $pid; done' EXIT

while true; do cat "$FIFO"; done

@HakimCassimallyBBC
Copy link
Author

# easier in Perl probably?
use strict;
use warnings;

my @FH = map {
    open (my $FH, '-|', 'curl', '-s', '-k', $_);
    $FH;
} @ARGV;

while (1) {
    my @lines = grep defined, map scalar <$_>, @FH or last;
    print @lines;
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment