Skip to content

Instantly share code, notes, and snippets.

View vhscom's full-sized avatar

Josh Habdas vhscom

View GitHub Profile
@vhscom
vhscom / wget.txt
Created December 11, 2023 15:12 — forked from suzannealdrich/wget.txt
wget spider cache warmer
wget --spider -o wget.log -e robots=off -r -l 5 -p -S --header="X-Bypass-Cache: 1" --limit-rate=124k www.example.com
# Options explained
# --spider: Crawl the site
# -o wget.log: Keep the log
# -e robots=off: Ignore robots.txt
# -r: specify recursive download
# -l 5: Depth to search. I.e 1 means 'crawl the homepages'.  2 means 'crawl the homepage and all pages it links to'...
# -p: get all images, etc. needed to display HTML page
# -S: print server response
# How to use a GPG key for SSH authentication
# Tested on Ubuntu 20.04 LTS
# Enable SSH support in gpg-agent:
echo enable-ssh-support > ~/.gnupg/gpg-agent.conf
# Cache GPG & SSH passwords for 24 hours (the reboot will clear the password cache)
echo max-cache-ttl 86440 >> ~/.gnupg/gpg-agent.conf
echo max-cache-ttl-ssh 86440 86440 >> ~/.gnupg/gpg-agent.conf
echo default-cache-ttl 86440 >> ~/.gnupg/gpg-agent.conf