Skip to content

Instantly share code, notes, and snippets.

@joostvanveen

joostvanveen/public.blacklist

Created October 19, 2017 10:30
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save joostvanveen/e608c987ac0ca4d4ad371be998b9becd to your computer and use it in GitHub Desktop.
Save joostvanveen/e608c987ac0ca4d4ad371be998b9becd to your computer and use it in GitHub Desktop.
Download ZIP
Nginx configuration file for Hypernode to block unwanted bots
Raw
public.blacklist
if ($http_user_agent ~ "PHPCrawl|bGenius|SemrushBot|Yandex|YandexBot|MegaIndex|spbot|BLEXBot|ltx71|Daum|SpotBot|DotBot|seoscanners|domaincrawler|CCBot|SeznamBot|SMTBot") {
return 403;
break;
}
@mikebranderhorst
Copy link

Better use status 410 and you don't need break after return. More bots are listed here: https://gist.github.com/peterjaap/ab5801152e1d0f874476b47e0c9a4682

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment