Skip to content

Instantly share code, notes, and snippets.

@stephen-puiszis

stephen-puiszis/gist:522ebdd15849bc8e0000

Created September 23, 2014 00:09
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save stephen-puiszis/522ebdd15849bc8e0000 to your computer and use it in GitHub Desktop.
Save stephen-puiszis/522ebdd15849bc8e0000 to your computer and use it in GitHub Desktop.
Download ZIP
Read a Sitemap.xml file and save any links to a file in that same directory. Then CURL every link from that file to read headers. Best used to quickly verify your site map is returning 200s.
Raw
gistfile1.txt
grep <sitemap file> --only-matching -E -e 'http(s?):\/\/[^ \"\(\)\<\>]*' >> links.txt
for i in `cat links.txt` ; do curl -I $i ; done (edited)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment