Skip to content

Instantly share code, notes, and snippets.

@Ereza

Ereza/download_full_site.sh

Last active July 20, 2017 17:50
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Ereza/6e7c78d03c9efa3f4e3fd732ea1610b5 to your computer and use it in GitHub Desktop.
Save Ereza/6e7c78d03c9efa3f4e3fd732ea1610b5 to your computer and use it in GitHub Desktop.
Download ZIP
Download full site with wget
Raw
download_full_site.sh
#Adjust --level as needed (max recursive depth)
#For rate-limited sites (random wait between 10 and 20 s)
wget -w 10 --random-wait -e robots=off --recursive --level 30 --no-clobber --page-requisites --html-extension --convert-links --restrict-file-names=windows --domains mydomain.com --no-parent http://mydomain.com/path/
#For own sites with no limit:
wget -e robots=off --recursive --level 30 --page-requisites --html-extension --convert-links --restrict-file-names=windows --domains mydomain.com --no-parent http://mydomain.com/path/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment