-
-
Save slavivanov/86438b0f145830225cca7976d2282bb4 to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Mozilla/5.0 (compatible; woriobot support [at] worio [dot] com +http://worio.com) | |
EventMachine HttpClient | |
MxAgent | |
Mozilla | |
Comodo-Certificates-Spider | |
JS-Kit URL Resolver, http://js-kit.com/ | |
curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5 | |
Googlebot/2.1 (http://www.googlebot.com/bot.html) | |
libwww-perl/5.834 | |
Jakarta Commons-HttpClient/3.1 | |
ia_archiver-web.archive.org | |
Java/1.6.0_24 | |
NING/1.0 | |
Mozilla/5.0 (compatible; woriobot +http://worio.com) | |
Mozilla/5.0 (compatible; AntBot/1.0; +http://www.ant.com/) | |
msnbot-NewsBlogs/2.0b (+http://search.msn.com/msnbot.htm) | |
WordPress/3.0.1; http://aiesecchennai.com | |
Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07) | |
PycURL/7.18.2 | |
Mozilla/5.0 (compatible; heritrix/3.0.0 +http://dataclip.com/crawler.html) | |
MetaURI API/2.0 +metauri.com | |
Voyager/1.0 | |
Python-urllib/2.6 | |
Python-urllib/2.4 | |
Mozilla/4.0 (compatible; MSIE 6.0;) | |
example/1.0 | |
Who.is Bot | |
LinkedInBot/1.0 (compatible; Mozilla/5.0; Jakarta Commons-HttpClient/3.1 +http://www.linkedin.com) | |
bitlybot | |
Twitterbot/0.1 | |
ia_archiver (+http://www.alexa.com/site/help/webmasters; crawler@alexa.com) | |
Mozilla/3.0 | |
OrangeAskBot/2.0; +http://www.orangeask.com | |
curl/7.15.5 | |
PostRank/2.0 (postrank.com) | |
Mozilla/5.0 (compatible; AboutUsBot Johnny5/2.0; +http://www.AboutUs.org/) | |
Googlebot/2.1 (+http://www.google.com/bot.html) | |
Microsoft Data Access Internet Publishing Provider Protocol Discovery | |
UnwindFetchor/1.0 (+http://www.gnip.com/) | |
Apple-PubSub/65.11 | |
Mozilla/4.0 (compatible; MSIE 5.0; Mac_PowerPC) | |
Mozilla/5.0 (compatible; Birubot/1.0) Gecko/2009032608 Firefox/3.0.8 | |
Mozilla/5.0 (compatible; Butterfly/1.0; +http://labs.topsy.com/butterfly/) Gecko/2009032608 Firefox/3.0.8 | |
Mozilla/5.0 (compatible; DotBot/1.1; http://www.dotnetdotcom.org/, crawler@dotnetdotcom.org) | |
Facebook share follower | |
Microsoft-WebDAV-MiniRedir/5.1.2600 | |
Shockwave Flash | |
MLBot (www.metadatalabs.com/mlbot) | |
Microsoft Office Existence Discovery | |
Mozilla/5.0 (compatible; Search17Bot/1.1; http://www.search17.com/bot.php) | |
Microsoft Office Protocol Discovery | |
Mozilla/5.0 (compatible; SiteBot/0.1; +http://www.sitebot.org/robot/) | |
axtar.az/axtar-1 (axtar crawler; http://axtar.az; admin@axtar.az) | |
Apple-PubSub/65.21 | |
Mozilla/5.0 (compatible; Google Desktop/5.9.1005.12335; http://desktop.google.com/) | |
Mozilla/5.0 (compatible; AhrefsBot/1.0; +http://ahrefs.com/robot/) | |
Java/1.6.0_20 | |
Adobe Flash Player 10 | |
Microsoft-WebDAV-MiniRedir/6.1.7601 | |
Mozilla/4.0 (compatible; Netcraft Web Server Survey) | |
WordPress/3.0; http://127.0.0.1/wordpress | |
Mozilla/5.0 (compatible; NetcraftSurveyAgent/1.0; +info@netcraft.com) | |
Mail.Ru/1.0 | |
Mozilla/5.0 (compatible; 008/0.83; http://www.80legs.com/webcrawler.html;) Gecko/2008032620 | |
CatchBot/3.0; +http://www.catchbot.com | |
msnbot/2.0b (+http://search.msn.com/msnbot.htm)._ | |
Mozilla/5.0 (compatible; ScoutJet; +http://www.scoutjet.com/) | |
Huaweisymantecspider (compatible; MSIE 8.0; DSE-support@huaweisymantec.com) | |
Java/1.6.0_23 | |
Mozilla/4.0 (compatible;) | |
DomainCrawler/2.0 (info@domaincrawler.com; http://www.domaincrawler.com/aiesecchennai.com | |
Wget/1.9+cvs-stable (Red Hat modified) | |
Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp) | |
Apple-PubSub/65 | |
Mozilla/5.0 (compatible; MJ12bot/v1.3.3; http://www.majestic12.co.uk/bot.php?+) | |
facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php) | |
ZmEu | |
Apple-PubSub/65.28 | |
Mozilla/5.0 (compatible; YandexImages/3.0; +http://yandex.com/bots) | |
Mozilla/5.0 (compatible; Ezooms/1.0; ezooms.bot@gmail.com) | |
Mozilla/5.0 (compatible; Exabot/3.0; +http://www.exabot.com/go/robot) | |
Mozilla/5.0 (compatible; Purebot/1.1; +http://www.puritysearch.net/) | |
Baiduspider-image+(+http://www.baidu.com/search/spider.htm) | |
Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots) | |
msnbot-media/1.1 (+http://search.msn.com/msnbot.htm) | |
facebookexternalhit/1.0 (+http://www.facebook.com/externalhit_uatext.php) | |
Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) | |
Apple-PubSub/65.20 | |
- | |
Googlebot-Image/1.0 | |
WordPress/3.0.1; http://www.aiesecchennai.com | |
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) | |
Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html) | |
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) | |
Baiduspider+(+http://www.baidu.com/search/spider.htm) | |
Mozilla/5.0 (compatible; 008/0.83; http://www.80legs.com/webcrawler.html) Gecko/2008032620 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment