Skip to content

Instantly share code, notes, and snippets.

@ronaldbaltus
Last active August 25, 2018 12:48
Show Gist options
  • Save ronaldbaltus/7c9b9e4a7998a55c8b0154d393a3917b to your computer and use it in GitHub Desktop.
Save ronaldbaltus/7c9b9e4a7998a55c8b0154d393a3917b to your computer and use it in GitHub Desktop.
Rotbot .htaccess - Block annoying bots with a simple htaccess
# Not so evil bots, but they are bots. If you want to block them, just remove the # in the front.
#SetEnvIfNoCase User-Agent "googlebot" rotbot
#SetEnvIfNoCase User-Agent "bingbot" rotbot
#SetEnvIfNoCase User-Agent "AppEngine-Google" rotbot
#SetEnvIfNoCase User-Agent "Google-Image" rotbot
#SetEnvIfNoCase User-Agent "msnbot-media" rotbot
SetEnvIfNoCase User-Agent "AdsBot-Google" rotbot
# Block evil bots
SetEnvIfNoCase User-Agent "AhrefsBot" rotbot
SetEnvIfNoCase User-Agent "archive.org_bot" rotbot
SetEnvIfNoCase User-Agent "Baiduspider" rotbot
SetEnvIfNoCase User-Agent "BLEXBot" rotbot
SetEnvIfNoCase User-Agent "DotBot" rotbot
SetEnvIfNoCase User-Agent "Exabot" rotbot
SetEnvIfNoCase User-Agent "heritrix" rotbot
SetEnvIfNoCase User-Agent "ltx71" rotbot
SetEnvIfNoCase User-Agent "MJ12bot" rotbot
SetEnvIfNoCase User-Agent "oBot" rotbot
SetEnvIfNoCase User-Agent "SemrushBot" rotbot
SetEnvIfNoCase User-Agent "SEOkicks-Robot" rotbot
SetEnvIfNoCase User-Agent "spbot/4.4.2" rotbot
SetEnvIfNoCase User-Agent "TurnitinBot" rotbot
SetEnvIfNoCase User-Agent "Vagabondo" rotbot
SetEnvIfNoCase User-Agent "worldwebheritage.org" rotbot
SetEnvIfNoCase User-Agent "YandexBot" rotbot
SetEnvIfNoCase User-Agent "YandexImages" rotbot
SetEnvIfNoCase User-Agent "MegaIndex.ru" rotbot
SetEnvIfNoCase User-Agent "Kraken" rotbot
SetEnvIfNoCase User-Agent "psbot" rotbot
SetEnvIfNoCase User-Agent "CCBot" rotbot
SetEnvIfNoCase User-Agent "GroupHigh" rotbot
SetEnvIfNoCase User-Agent "Sogou" rotbot
SetEnvIfNoCase User-Agent "NetcraftSurveyAgent" rotbot
SetEnvIfNoCase User-Agent "WBSearchBot" rotbot
SetEnvIfNoCase User-Agent "seoscanners.net" rotbot
SetEnvIfNoCase User-Agent "XoviBot" rotbot
SetEnvIfNoCase User-Agent "Twitterbot" rotbot
SetEnvIfNoCase User-Agent "Applebot" rotbot
SetEnvIfNoCase User-Agent "Mediatoolkitbot" rotbot
SetEnvIfNoCase User-Agent "Findxbot" rotbot
SetEnvIfNoCase User-Agent "LivelapBot" rotbot
SetEnvIfNoCase User-Agent "TweetmemeBot" rotbot
SetEnvIfNoCase User-Agent "epicbot" rotbot
SetEnvIfNoCase User-Agent "IndeedBot" rotbot
SetEnvIfNoCase User-Agent "PaperLiBot" rotbot
SetEnvIfNoCase User-Agent "Uptimebot" rotbot
SetEnvIfNoCase User-Agent "coccocbot-web" rotbot
SetEnvIfNoCase User-Agent "Leikibot" rotbot
SetEnvIfNoCase User-Agent "dlvr.it" rotbot
SetEnvIfNoCase User-Agent "Laserlikebot" rotbot
SetEnvIfNoCase User-Agent "omgili" rotbot
SetEnvIfNoCase User-Agent "Pinterestbot" rotbot
SetEnvIfNoCase User-Agent "trendictionbot" rotbot
SetEnvIfNoCase User-Agent "YaK" rotbot
SetEnvIfNoCase User-Agent "Findxbot" rotbot
SetEnvIfNoCase User-Agent "SeznamBot" rotbot
<Limit POST GET HEAD PUT>
Order Allow,Deny
Allow from all
Deny from env=rotbot
</Limit>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment