Skip to content

Instantly share code, notes, and snippets.

@medigeek
Created July 26, 2021 13:52
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save medigeek/df0a351481ece09a83749bbffc04e9de to your computer and use it in GitHub Desktop.
Save medigeek/df0a351481ece09a83749bbffc04e9de to your computer and use it in GitHub Desktop.
asynchronous crawler / warmer based on sitemap.xml
#!/bin/bash
#run multiple concurrent connections to speed up cache warming
#install eliashaeussler/cache-warmup
mkdir cache-warmer-elias
cd cache-warmer-elias
composer require eliashaeussler/cache-warmup
#run it
vendor/bin/cache-warmup -v 'https://www.example.com/sitemap.xml'
#Default concurrency is set to 5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment