Skip to content

Instantly share code, notes, and snippets.

@werdan
Created November 16, 2015 10:49
Show Gist options
  • Save werdan/e65dfa4e1608ce502eaf to your computer and use it in GitHub Desktop.
Save werdan/e65dfa4e1608ce502eaf to your computer and use it in GitHub Desktop.
Boodmo Bad Bot List
omniexplorer_bot
yandeximages
Spinn3r
sogou
sosospider+
jikespider
ia_archiver
PaperLiBot
ahrefsbot
SiteBot
DNS-Digger
boardreader
radian6
R6_FeedFetcher
R6_CommentReader
ScoutJet
ezooms
CC-rget
magpie-crawler
discobot
MJ12bot
SemrushBot
MLBot
butterfly
SeznamBot
HuaweiSymantecSpider
Exabot
netseer
psbot
moreoverbot
SocialSpider-Finder
^artron
^Baidu
^mixpod
^Mp3Bear
^mp3raid
^music
^pcmusic
^qzone
^Sina
^wuzam
^xnimg
^Yandex
^Aport
^mail.ru
^Dalvik
BLEXBot
msnbot
^Jakarta
IDBot
id-search
User-Agent
ConveraCrawler
^Mozilla$
lwp-trivial
Snoopy
MFC_Tear_Sample
PHPCrawl
panscient.com
IBM EVV
Bork-edition
Fetch API Request
PleaseCrawl
layeredtech.com
WEP Search
Wells Search II
Missigua Locator
ISC Systems iRc Search
Microsoft URL Control
Indy Library
8484 Boston Project
Atomic_Email_Hunter
atSpider
autoemailspider
China Local Browse
ContactBot
ContentSmartz
DataCha0s
DBrowse
Demo Bot
DSurf15a
EBrowse
Educate Search VxB
EmailSiphon
EmailWolf
ESurf15a
ExtractorPro
Franklin Locator
FSurf15a
Full Web Bot
Guestbook Auto Submitter
Industry Program
ISC Systems iRc Search
IUPUI Research Bot
LARBIN-EXPERIMENTAL
LetsCrawl.com
Lincoln State Web Browser
LMQueueBot
LWP::Simple
Mac Finder
MFC Foundation Class Library
Microsoft URL Control
Missauga Locate
Missigua Locator
Missouri College Browse
Mizzu Labs
Mo College
MVAClient
NameOfAgent \(CMS Spider\)
NASA Search
Nsauditor
PBrowse
PEval
Poirot
Port Huron Labs
Production Bot
Program Shareware
PSurf15a
psycheclone
searchbot admin@google.com
ShablastBot
snap.com beta crawler
Snapbot
sohu agent
SSurf15a
TSurf15a
Under the Rainbow
VadixBot
WebVulnCrawl.blogspot.com
Wells Search
WEP Search
Xenu Link Sleuth
www.integromedb.org/Crawler
www.proximic.com/info/spider.php
ShopWiki/1.0
EasouSpider
www.80legs.com/webcrawler.html
GrapeshotCrawler
360Spider
MaxPointCrawler
URLAppendBot
GarlikCrawler
Wotbox
www.exb.de/crawler
www.seoengine.com/seoengbot.htm
help.naver.com/robots
SISTRIX Crawler
wootbot
YisouSpider
MSIECrawler
YRSpider
ZumBot
SnapPreviewBot
TurnitinBot
UptimeRobot
filterdb.iss.net/crawler/
ContextAd Bot
Diffbot
FatBot
AppCodesCrawler
uMBot
QuerySeekerSpider
adbeat_bot
TweetmemeBot
Scope PreviewBot
linkdexbot
CompSpyBot
Goodzer
Zing-BottaBot
SiteSucker
MailChimp.com
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment