Skip to content

Instantly share code, notes, and snippets.

@HuanMeng0

HuanMeng0/spider.conf

Created July 30, 2016 12:22
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save HuanMeng0/321af6b8bf5cc1ba88b85efd4d512eec to your computer and use it in GitHub Desktop.
Save HuanMeng0/321af6b8bf5cc1ba88b85efd4d512eec to your computer and use it in GitHub Desktop.
Download ZIP
屏蔽搜索引擎爬虫(Nginx)
Raw
spider.conf
location / {
if ($http_user_agent ~* "qihoobot|Baiduspider|Googlebot|Googlebot-Mobile|Googlebot-Image|Mediapartners-Google|Adsbot-Google|Feedfetcher-Google|Yahoo! Slurp|Yahoo! Slurp China|YoudaoBot|Sosospider|Sogou spider|Sogou web spider|MSNBot|ia_archiver|Tomato Bot")
{
return 403;
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment