Skip to content

Instantly share code, notes, and snippets.

@TonyStark
Created December 28, 2019 08:42
Show Gist options
  • Save TonyStark/f0b4fc34b31d326434d2ea08b894065d to your computer and use it in GitHub Desktop.
Save TonyStark/f0b4fc34b31d326434d2ea08b894065d to your computer and use it in GitHub Desktop.
parallel 50 files download
#!/bin/bash
for a in {1487054155500..1487055000000}
do
if [ $((a%50)) = 0 ]
then
curl -s -I http://link.com/updateOTA_$((a)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+1)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+2)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+3)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+4)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+5)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+6)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+7)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+8)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+9)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+10)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+11)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+12)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+13)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+14)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+15)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+16)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+17)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+18)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+19)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+20)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+21)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+22)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+23)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+24)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+25)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+26)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+27)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+28)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+29)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+30)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+31)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+32)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+33)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+34)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+35)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+36)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+37)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+38)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+39)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+40)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+41)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+42)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+43)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+44)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+45)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+46)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+47)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+48)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+49)).zip | head -n1 &
curl -s -I http://link.com/updateOTA_$((a+50)).zip | head -n1
wait
echo "$((a))"
fi
done
@TonyStark
Copy link
Author

aria2c -i url -s16 -x16 --max-concurrent-downloads=1000

@TonyStark
Copy link
Author

This loop doesn't require pre-generating a list of a million integers

for ((a=1487054155500; a<=1487055000000; a+=50)); do
for(k=0; k<50; k++)); do
printf 'url = %s\n' "http://link.com/updateOTA_$((a+k)).zip"
done | curl -I -K - -w 'result: %{http_code} %{url_effective}' | grep -F 'result:' > batch-$a.txt
done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment