Skip to content

Instantly share code, notes, and snippets.

@jnv
Created November 8, 2016 21:08
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save jnv/806d5fc001afb5161ce09813df84ca2f to your computer and use it in GitHub Desktop.
Save jnv/806d5fc001afb5161ce09813df84ca2f to your computer and use it in GitHub Desktop.
Bulk check list of URLs with curl; outputs semicolon separated csv
#!/bin/bash
# Usage: check-urls.sh <urls list>
# <urls list> is a text file with 1 URL per line
echo "url;http_code;url_effective;error"
while read -r LINE; do
echo -n "$LINE;"
curl -o /dev/null -sS --max-time 5 --write-out '%{http_code};%{redirect_url};' "$LINE" 2> >(sed -e 's/$/;/' | tr -d '\n' | tr -d '\r')
sync
echo ""
done < "$1"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment