Skip to content

Instantly share code, notes, and snippets.

@nfarina
Created August 4, 2021 13:30
Show Gist options
  • Save nfarina/365029c6a73b96bfad4c4585061a3417 to your computer and use it in GitHub Desktop.
Save nfarina/365029c6a73b96bfad4c4585061a3417 to your computer and use it in GitHub Desktop.
#!/bin/bash
#
# Backs up my entire website, in case Tumblr or CloudApp goes down someday.
# Last time I ran this, it took 18 minutes.
#
wget \
--mirror `# turns on recursion and timestamping, basically says we want to "mirror" the whole site` \
--convert-links `# after download, convert all links to point to localhost` \
--page-requisites `# download images and stylesheets and stuff` \
--adjust-extension `# add .html extensions for local viewing` \
-H `# allow crawling to external hosts besides nfarina.com (needed for cl.ly image links)` \
-Dcl.ly,nfarina.com `# allowed hosts` \
-e robots=off `# cl.ly's robots.txt requests no crawling at all. Makes sense for Google, not for us` \
-w 2 `# wait two seconds between HTTP requests` \
http://nfarina.com
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment