Skip to content

Instantly share code, notes, and snippets.

@bmcguirk

bmcguirk/find_404s_using_wget.sh

Last active February 4, 2024 13:48
Show Gist options
  • Star 3 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save bmcguirk/f75caf04236096d843fc0206f01ec9f7 to your computer and use it in GitHub Desktop.
Save bmcguirk/f75caf04236096d843fc0206f01ec9f7 to your computer and use it in GitHub Desktop.
Download ZIP
Find broken links (404s) via wget and grep.
Raw
find_404s_using_wget.sh
# Full credit to Pete here: http://www.createdbypete.com/articles/simple-way-to-find-broken-links-with-wget/
wget --spider -nd -o ~/wget.log -e robots=off -w 1 -r http://www.example.com
grep -B 2 '404' example.com.log >> example.com.404.log
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment