Skip to content

Instantly share code, notes, and snippets.

@jemekite
Created July 25, 2015 13:28
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save jemekite/10a5f8e264281476e50c to your computer and use it in GitHub Desktop.
Save jemekite/10a5f8e264281476e50c to your computer and use it in GitHub Desktop.
NginX badbot blocker conf
if ($http_user_agent ~ "WebCrawlerJohn|Sphere|OptimizationCrawler|Apache-HttpClient|LinkpadBot|MegaIndex.ru|WeCrawlForThePeace|Xenu|SemanticScholarBot|Domain Re-Animator Bot|proximic|Sosospider|Sogou|Jorgee|ZumBot|MJ12bot|spbot|rojerbot|Exabot|dotbot|gigabot|AhrefsBot|accelobot|searchmetrics|awcheckBot|CompSpyBot|EasouSpider|purebot|Ezooms|SurveyBot|sitebot|dotnetdotcom|dotbot|SolomonoBot|ZmEu|lipperhey|WBSearchBot|Snoopy|AhrefsBot|DinoPing|dataprovider.com|discoverybot|www.integromedb.org|360Spider|80legs|YamanaLab-Robot|ip-web-crawler.com|Aboundex|Sleuth|NCBot|JikeSpider|Curious|visaduhoc.info|SemrushBot|Dow Jones Searchbot|SISTRIX|brandwatch.net|magpie-crawler|YoudaoBot|TurnitinBot|aiHitBot|PeoplePal|EzineArticlesLinkScanner|ProCogSEOBot|ScreenerBot|PagesInventory|SeznamBot|libwww-perl|Lynx|Nutch|Java|SEOstats|Python-urllib|python-requests|HTTP_Request|HTTP_Request2|Zend_Http_Client|Jakarta Commons-HttpClient|The Incutio XML-RPC PHP Library|YisouSpider|SEOENGWorldBot|Netseer|Riddler|CLIPish|proximic|Mail.RU_Bot|WebCapture|Indy Library|Add Catalog|Butterfly|SocialSearcher|xpymep.exe|linkdex.com|lindex.com|YodaoBot|POGS|WebInDetail.com|WEBSITEtheWEB.COM|CatchBot|NextGenSearchBot|BacklinkCrawler|rarely used|DomainCrawler|ltbot|Wotbot|QuerySeekerSpider|netEstate NE Crawler|Synapse|news bot|DomainSigmaCrawler|omgilibot|Owlin|Mozilla--no-parent|SimplePie|OrangeBot|Feed Parser|FeedlyBot|Fetchbot|BLEXBot|PHPCrawl|libwww|User-Agent|lwp-trivial|PhantomJS|SV1|R6_FeedFetcher|x00_-gawa.sa.pilipinas.2015|Proxy Gear Pro|PHP/5\.|XML-RPC.NET|chroot|DataCha0s|tools.ua.random|ZmEu|immoral|mobmail android|Scanner") {
return 444;
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment