Skip to content

Instantly share code, notes, and snippets.

/download.sh

Created June 10, 2016 19:09
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save anonymous/ae10b42c8242e95a3c03ba6c404aec81 to your computer and use it in GitHub Desktop.
Save anonymous/ae10b42c8242e95a3c03ba6c404aec81 to your computer and use it in GitHub Desktop.
Download ZIP
Curl custom recursive download example
Raw
download.sh
# Was searching for a certain lyric, couldn't find a site that allowed you to search all of a specific artists lyrics
# thanks to http://apple.stackexchange.com/a/100576
curl -O -H "User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.84 Safari/537.36" http://www.azlyrics.com/19/50cent.html
for file in $(cat 50cent.html | grep href | sed 's/.*href="//' | sed 's/".*//' | grep '50cent' | grep '\.\.' | sed 's/\.\./www.azlyrics.com/'); do curl -O -H "User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.84 Safari/537.36" $file ; done
grep -nir --color=always watch | less -r
# seems ip got banned after 50 or so requests so maybe need to add some delays
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment