Skip to content

Instantly share code, notes, and snippets.

@KhanMaytok
Forked from CMCDragonkai/parallel_curl.sh
Created December 20, 2017 21:29
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save KhanMaytok/900603987913310e00b9e589f2a82b23 to your computer and use it in GitHub Desktop.
Save KhanMaytok/900603987913310e00b9e589f2a82b23 to your computer and use it in GitHub Desktop.
Bash: GNU Parallel with Curl
# do it once
seq 1 | parallel -n0 "curl -H 'Content-Type: application/json' http://httpbin.org/post -X POST -d '{\"url\":\"http://google.com/\"}'"
# do it twice
seq 2 | parallel -n0 "curl -H 'Content-Type: application/json' http://httpbin.org/post -X POST -d '{\"url\":\"http://google.com/\"}'"
# do it 4 times, but at 2 a time
seq 4 | parallel -n0 -j2 "curl -H 'Content-Type: application/json' http://httpbin.org/post -X POST -d '{\"url\":\"http://google.com/\"}'"
# you can also put all your commands into a file
echo "ls\nls" > somefile
cat somefile | parallel
# finally, just straight command line parameters
parallel echo ::: a b c
# by default, it runs as many jobs in parallel as there are cores in your computer
# if you try and set more jobs than there are cores, they will be concurrent, but
# they will only ever be parallel up to your core count
# for httpie use the --ignore-stdin flag, or else it gets mixed up
seq 1 | parallel -n0 "http --ignore-stdin POST http://httpbin.org/post url=http://google.com/"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment