Skip to content

Instantly share code, notes, and snippets.

@digitalbricklayer
Last active November 23, 2020 14:15
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save digitalbricklayer/a245948b7d6960330a7dda9a11692a32 to your computer and use it in GitHub Desktop.
#!/usr/bin/env bash
# Validate the contents of the sitemap
# See https://blog.atj.me/2018/05/crawl-sitemap-xml-with-curl/ for inspiration
SITEMAP_URI="/sitemap.xml"
SITEMAP_URL="http://localhost:1313"$SITEMAP_URI
# Start the test server
docker-compose up -d
# Wait for the test server to start up
until $(curl --output /dev/null --silent --head --fail $SITEMAP_URL); do
printf '.'
sleep 1
done
curl -s $SITEMAP_URL | \
grep -e loc | \
sed 's|<loc>\(.*\)<\/loc>$|\1|g' | \
xargs -I {} curl -s -o /dev/null -w "%{http_code} %{url_effective}\n" {} | \
grep -v ^200
docker-compose down
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment