Skip to content

Instantly share code, notes, and snippets.

@beatngu13
Last active April 9, 2021 20:48
Show Gist options
  • Save beatngu13/7ab3166291c5d854697ec12384319303 to your computer and use it in GitHub Desktop.
Save beatngu13/7ab3166291c5d854697ec12384319303 to your computer and use it in GitHub Desktop.
Download a list of web pages (including assets) for offline usage with Wget
# The user agent to use (see https://en.wikipedia.org/wiki/User_agent).
USER_AGENT='Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:68.0) Gecko/20100101 Firefox/68.0'
# The web pages to download.
WEB_PAGES=(
'google.com'
'youtube.com'
'facebook.com'
)
function download_web_page() {
web_page=$1
wget --header='Accept: text/html' \
--header='Accept-Language: en-US' \
--adjust-extension \
--backup-converted \
--convert-links \
--page-requisites \
--span-hosts \
--user-agent="${USER_AGENT}" \
--restrict-file-names=windows \
"${web_page}"
}
for web_page in "${WEB_PAGES[@]}"; do
download_web_page ${web_page}
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment