Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Quick, easy and dirty cache warmer based on sitemap.xml
#!/bin/bash
URL='www.domain.nd'
SITEMAP='sitemap.xml'
wget --quiet http://$URL/$SITEMAP --no-cache --output-document - | egrep -o "http://$URL[^<]+" | while read line; do
time curl -A 'Cache Warmer' -s -L $line > /dev/null 2>&1
echo $line
done
@osopolar

This comment has been minimized.

Copy link

@osopolar osopolar commented Aug 31, 2016

wouldn't it be more efficient (less bandwidth use) to use curl -I, like:
time curl -A 'Cache Warmer' -s -I $line > /dev/null 2>&1

curl -I ... just ask for the header and close the connection.
see: http://serverfault.com/a/140977/184446

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment