Skip to content

Instantly share code, notes, and snippets.

@dcosson dcosson/gist:2918201
Created Jun 12, 2012

Embed
What would you like to do?
Archive a website (in this case, tourbie.com)
# Just a note to myself on how to archive a website
mkdir tourbie_archive
cd tourbie_archive
wget --mirror -p -nH -e robots=off --convert-links http://tourbie.com
# --mirror mirrors the site (recurses all links)
# -p downloads all the links necessary to view the site
# --convert-links converts all links starting with http://tourbie.com to be relative
# -e robots=off optional, ignore robots.txt (on tourbie.com, I had disallowed the static files directory in robots.txt)
# -nH take out domain name (won't put everything in a "tourbie.com" folder)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.