Skip to content

Instantly share code, notes, and snippets.

@zacscott

zacscott/precache.sh

Created Jun 15, 2020
Embed
What would you like to do?
Precache sitemaps automatically via a smart Bash script.
!/bin/bash
# Automatically hits all sitemaps on a site to pre-cache then at the edge.
# CONFIG
SITEMAP_URL=https://www.oursite.com.au/sitemap_index.xml
# Pull the index sitemap and get a list of all of the individual sitemaps.
echo "PULLING $SITEMAP_URL"
sitemaps=`curl $SITEMAP_URL | awk -F '[<>]' '/loc/{print $3}'`
# Loop over each of the sitemap URLs and Curl it to warm up the cache.
for sitemap in $sitemaps; do
count=0
# Curl the sitemap URL until we confirm that it is cached by Cloudflare.
while :; do
let "count=count+1"
echo "HITTING $sitemap"
result=`curl -sI $sitemap | tr -d '\r' | tr '[:upper:]' '[:lower:]' | sed -En 's/^cf-cache-status: (.*)/\1/p'`
# If URL is cached, move onto the next sitemap.
if [ "hit" == "$result" ]; then
echo "CACHED $sitemap"
break
fi
# If we reached the limit, move on, don't get stuck in an infinite loop.
if [ $count -ge "10" ]; then
echo "REACHED LIMIT"
break
fi
done
echo "NEXT"
done
echo "DONE"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.