Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
# anything from amazonaws is not human
(.*)\.compute(.*)\.amazonaws\.com\.
# your-server.de is home to many bots: https://www.webmasterworld.com/search_engine_spiders/3963600.htm
(.*)\.clients\.your-server\.de\.
# anything with 'crawl' in its reverse DNS is probably a bot
(.*)crawl(.*)\.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.