Skip to content

Instantly share code, notes, and snippets.

@janiosarmento

janiosarmento/cachewarmer.sh Secret

Created July 8, 2018 19:10
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save janiosarmento/08f0b173ff7c0fcb419aa7cff001c7c3 to your computer and use it in GitHub Desktop.
Save janiosarmento/08f0b173ff7c0fcb419aa7cff001c7c3 to your computer and use it in GitHub Desktop.
Download ZIP
Cache Warmer
Raw
cachewarmer.sh
#!/bin/bash
UAString='Mozilla/5.0 (Windows NT 6.3; Trident/7.0; rv:11.0) like Gecko'
IgnoreExtensions="\.(jpeg|jpg|png|gif|svg|pdf)"
fqdn=$1
time wget --user-agent "$UAString" --quiet https://$fqdn/sitemap_index.xml --no-cache --output-document - | egrep -o "https://$fqdn[^<]+" | while read line; do
wget --user-agent "$UAString" --quiet $line --no-cache --output-document - | egrep -o "https://$fqdn[^<]+" | egrep -v -i $IgnoreExtensions | sort | while read line2; do
curl -A "$UAString" -s -L $line2 > /dev/null 2>&1
echo $line2
done
echo $line
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment