Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
A cache warmer in bash using curl
#!/bin/bash
if [ "$#" -ne 1 ] || [ "x$1" == "x" ] ; then
echo "Usage: $0 <sitemap.xml>"
exit 0;
fi
if [ ! -f "$1" ]; then
echo "Sitemap file $1 not found! Exit!"
exit 1
fi
cat "$1" | perl -ne 'while (/>(http.+?)</g) { print "$1\n"; }' | while read line; do
echo " Crawling $line "
curl -so /dev/null -w "%{time_connect} - %{time_starttransfer} - %{time_total} " $line
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.