Skip to content

Instantly share code, notes, and snippets.

@dasginganinja

dasginganinja/check-status.sh

Created October 29, 2015 16:27
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save dasginganinja/d0615bb45537a2322790 to your computer and use it in GitHub Desktop.
Save dasginganinja/d0615bb45537a2322790 to your computer and use it in GitHub Desktop.
Download ZIP
HTTP Status checking
Raw
check-status.sh
# Check for number of urls with 200 statuses
grep 200 out.txt | wc -l
Raw
lookup-urls.sh
# One at a time
cat urls.txt | xargs -I ^ curl -o /dev/null -sw "%{url_effective}\\t %{http_code}\\t %{redirect_url}\\n" ^ > out.txt
# parallelization using xargs
cat urls.txt | xargs -P 20 -I ^ curl -o /dev/null -sw "%{url_effective}\\t %{http_code}\\t %{redirect_url}\\n" ^ > out.txt
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment