-
-
Save djprmf/f95260efd4c6ab4a52af to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
User-agent: Mediapartners-Google | |
Disallow: | |
User-agent: Mediapartners-Google* | |
Disallow: | |
User-agent: * | |
Disallow: /abuse | |
Disallow: /admgt/ | |
Disallow: /contact | |
Disallow: /donate | |
Disallow: /go/ | |
Disallow: /login | |
Disallow: /modcp | |
Disallow: /post | |
Disallow: /privmsg | |
Disallow: /spa/ | |
Disallow: /sta/ | |
User-agent: 008 | |
User-agent: Accoona | |
User-agent: aipbot | |
User-agent: aipbot* | |
User-agent: aipbot/1.0 | |
User-agent: Alexa | |
User-agent: Alexa Bitlybot | |
User-agent: Alexibot | |
User-agent: AltaVista Intranet V2.0 AVS EVAL search@freeit.com | |
User-agent: AltaVista Intranet V2.0 Compaq Altavista Eval sveand@altavista.net | |
User-agent: AltaVista Intranet V2.0 evreka.com crawler@evreka.com | |
User-agent: AltaVista V2.0B crawler@evreka.com | |
User-agent: Anonymous | |
User-agent: ApocalXExplorerBot | |
User-agent: appie | |
User-agent: Aqua_Products | |
User-agent: Argus/1.1 | |
User-agent: Artabus | |
User-agent: Ask Jeeves | |
User-agent: asterias | |
User-agent: atSpider | |
User-agent: attentio | |
User-agent: AV Fetch 1.0 | |
User-agent: AVSearch-3.0(AltaVista/AVC) | |
User-agent: AWS Cloud Based | |
User-agent: b2w | |
User-agent: b2w/0.1 | |
User-agent: BackDoorBot | |
User-agent: BackDoorBot/1.0 | |
User-agent: Baiduspider | |
User-agent: becomebot | |
User-agent: BecomeBot | |
User-agent: BigBrother | |
User-agent: BIGLOTRON (BETA 2;GNU/Linux) | |
User-agent: BizInformation | |
User-agent: Black Hole | |
User-agent: Black.Hole | |
User-agent: BlackWidow | |
User-agent: BlowFish | |
User-agent: BlowFish/1.0 | |
User-agent: BoardPulse | |
User-agent: boitho.com-dc | |
User-agent: Bookmark search tool | |
User-agent: bot/1.0 | |
User-agent: BotALot | |
User-agent: Bot mailto:craftbot@yahoo.com | |
User-agent: BotRightHere | |
User-agent: BrandProtect | |
User-agent: BuiltBotTough | |
User-agent: Bullseye | |
User-agent: Bullseye/1.0 | |
User-agent: BunnySlippers | |
User-agent: CazoodleBot | |
User-agent: Cegbfeieh | |
User-agent: cfetch | |
User-agent: cfetch/1.0 | |
User-agent: CheeseBot | |
User-agent: CherryPicker | |
User-agent: CherryPicker /1.0 | |
User-agent: CherryPickerElite/1.0 | |
User-agent: CherryPickerSE/1.0 | |
User-agent: ChinaClaw | |
User-agent: Collage | |
User-agent: cometrics-bot | |
User-agent: complex_network_group | |
User-agent: convera | |
User-agent: ConveraCrawler | |
User-agent: ConveraCrawler/0.2 | |
User-agent: ConveraCrawler/0.9d | |
User-agent: Convera Internet Spider V6.x | |
User-agent: ConveraMultiMediaCrawler/0.1 | |
User-agent: Copernic | |
User-agent: CopyRightCheck | |
User-agent: cosmos | |
User-agent: Crescent | |
User-agent: Crescent Internet ToolPak HTTP OLE Control v.1.0 | |
User-agent: Crescent Internet ToolPak HTTPOLE Control v.1.0 | |
User-agent: Curl | |
User-agent: Custo | |
User-agent: CydralSpider | |
User-agent: Deepnet Explorer | |
User-agent: default.ida | |
User-agent: DigExt | |
User-agent: DISCo | |
User-agent: discobot | |
User-agent: DISCoFinder | |
User-agent: DISCo Pump | |
User-agent: DISCo Pump 3.0 | |
User-agent: DISCo Pump 3.1 | |
User-agent: DISCo Pump 3.2 | |
User-agent: DittoSpyder | |
User-agent: DOC | |
User-agent: dotbot | |
User-agent: DotBot | |
User-agent: DotBot/1.1 | |
User-agent: Download Demon | |
User-agent: Download Demon/3.2.0.8 | |
User-agent: Download Demon/3.5.0.11 | |
User-agent: Download Ninja | |
User-agent: Download Wonder | |
User-agent: DSurf | |
User-agent: Dulance bot | |
User-agent: dumbot | |
User-agent: eCatch | |
User-agent: eCatch/3.0 | |
User-agent: echo! | |
User-agent: EchO!/2.0 | |
User-agent: EirGrabber | |
User-agent: EliteSys Entry | |
User-agent: EmailCollector | |
User-agent: Email Extractor | |
User-agent: EmailSiphon | |
User-agent: EmailSmartz | |
User-agent: EmailWolf | |
User-agent: Enterprise_Search | |
User-agent: Enterprise_Search/1.0 | |
User-agent: EroCrawler | |
User-agent: es | |
User-agent: ESIRover | |
User-agent: e-SocietyRobot | |
User-agent: Exabot | |
User-agent: Exabot/2.0 | |
User-agent: Exabot-Images | |
User-agent: Express WebPictures | |
User-agent: Express WebPictures (www.express-soft.com) | |
User-agent: ExtractorPro | |
User-agent: EyeNetIE | |
User-agent: FairAd Client | |
User-agent: Fairshare | |
User-agent: Fasterfox | |
User-agent: Fetch | |
User-agent: Flaming AttackBot | |
User-agent: Flamingo_SearchEngine | |
User-agent: FlashGet | |
User-agent: FlashGet WebWasher 3.2 | |
User-agent: Foobot | |
User-agent: FreeFind | |
User-agent: FreeWebMonitoring SiteChecker/0.1 | |
User-agent: FrontPage | |
User-agent: FrontPage [NC,OR] | |
User-agent: FurlBot | |
User-agent: Gaisbot | |
User-agent: Gaisbot/3.0 | |
User-agent: GetBot | |
User-agent: GetRight | |
User-agent: GetRight/2.11 | |
User-agent: GetRight/3.1 | |
User-agent: GetRight/3.2 | |
User-agent: GetRight/3.3 | |
User-agent: GetRight/3.3.3 | |
User-agent: GetRight/3.3.4 | |
User-agent: GetRight/4.0.0 | |
User-agent: GetRight/4.1.0 | |
User-agent: GetRight/4.1.1 | |
User-agent: GetRight/4.1.2 | |
User-agent: GetRight/4.2 | |
User-agent: GetRight/4.2b (Portuguxeas) | |
User-agent: GetRight/4.2c | |
User-agent: GetRight/4.3 | |
User-agent: GetRight/4.5 | |
User-agent: GetRight/4.5a | |
User-agent: GetRight/4.5b | |
User-agent: GetRight/4.5b1 | |
User-agent: GetRight/4.5b2 | |
User-agent: GetRight/4.5b3 | |
User-agent: GetRight/4.5b6 | |
User-agent: GetRight/4.5b7 | |
User-agent: GetRight/4.5c | |
User-agent: GetRight/4.5d | |
User-agent: GetRight/4.5e | |
User-agent: GetRight/5.0beta1 | |
User-agent: GetRight/5.0beta2 | |
User-agent: GetUrl | |
User-agent: GetWeb! | |
User-agent: Gigabot | |
User-agent: Go-Ahead-Got-It | |
User-agent: Go!Zilla | |
User-agent: Go!Zilla 3.3 (www.gozilla.com) | |
User-agent: Go!Zilla 3.5 (www.gozilla.com) | |
User-agent: Go!Zilla (www.gozilla.com) | |
User-agent: GrabNet | |
User-agent: Grafula | |
User-agent: grub | |
User-agent: grub-client | |
User-agent: Hackertarget.com | |
User-agent: Harvest | |
User-agent: Harvest/1.5 | |
User-agent: Hatena Antenna | |
User-agent: HavIndex | |
User-agent: heritrix | |
User-agent: hloader | |
User-agent: HMView | |
User-agent: httplib | |
User-agent: httrack | |
User-agent: HTTrack | |
User-agent: HTTrack 3.0 | |
User-agent: HTTrack 3.0x | |
User-agent: HTTrack [NC,OR] | |
User-agent: humanlinks | |
User-agent: ichiro | |
User-agent: IconSurf | |
User-agent: Igentia | |
User-agent: Image Collector | |
User-agent: Image Stripper | |
User-agent: Image Sucker | |
User-agent: Indy Library | |
User-agent: Indy Library [NC,OR] | |
User-agent: InfoNaviRobot | |
User-agent: InfoSpiders | |
User-agent: InterGET | |
User-agent: Internet Explore | |
User-agent: Internet Ninja | |
User-agent: Internet Ninja 4.0 | |
User-agent: Internet Ninja 5.0 | |
User-agent: Internet Ninja 6.0 | |
User-agent: InternetSupervision | |
User-agent: IRLbot | |
User-agent: Iron | |
User-agent: Iron33/1.0.2 | |
User-agent: Jeeves | |
User-agent: JennyBot | |
User-agent: Jetbot | |
User-agent: Jetbot/1.0 | |
User-agent: JetCar | |
User-agent: Jobo | |
User-agent: JOC Web Spider | |
User-agent: kalooga | |
User-agent: KDD Exploror | |
User-agent: Kenjin Spider | |
User-agent: Kenjin.Spider | |
User-agent: Keyword Density | |
User-agent: Keyword.Density | |
User-agent: Keyword Density/0.9 | |
User-agent: larbin | |
User-agent: Larbin | |
User-agent: larbin_2.6.2 (kabura@sushi.com) | |
User-agent: larbin_2.6.2 kabura@sushi.com | |
User-agent: larbin_2.6.2 (larbin2.6.2@unspecified.mail) | |
User-agent: larbin_2.6.2 larbin2.6.2@unspecified.mail | |
User-agent: larbin_2.6.2 larbin@correa.org | |
User-agent: larbin_2.6.2 listonATccDOTgatechDOTedu | |
User-agent: larbin_2.6.2 (listonATccDOTgatechDOTedu) | |
User-agent: larbin_2.6.2 (vitalbox1@hotmail.com) | |
User-agent: larbin_2.6.2 vitalbox1@hotmail.com | |
User-agent: larbin (samualt9@bigfoot.com) | |
User-agent: larbin samualt9@bigfoot.com | |
User-agent: LBot | |
User-agent: LeechFTP | |
User-agent: LexiBot | |
User-agent: libWeb/clsHTTP | |
User-agent: libWeb/clsHTTPDisallow: / | |
User-agent: libwww | |
User-agent: LightningDownload | |
User-agent: Linguee | |
User-agent: LinkedIn | |
User-agent: LinkextractorPro | |
User-agent: Linknzbot | |
User-agent: Linknzbot* | |
User-agent: Linknzbot 2004 | |
User-agent: LinkScan | |
User-agent: LinkScan/8.1a Unix | |
User-agent: LinkScan/8.1a.Unix | |
User-agent: LinkScan/8.1a Unix Disallow: / | |
User-agent: linksmanager | |
User-agent: LinksManager | |
User-agent: LinksManager.com_bot | |
User-agent: LinkWalker | |
User-agent: LjSEEK | |
User-agent: LNSpiderguy | |
User-agent: looksmart | |
User-agent: LWP | |
User-agent: LWP* | |
User-agent: lwp-trivial | |
User-agent: lwp-trivial/1.34 | |
User-agent: magpie-crawler | |
User-agent: Mail Sweeper | |
User-agent: Marketwirebot | |
User-agent: Mass Downloader | |
User-agent: Mass Downloader/2.2 | |
User-agent: Mata Hari | |
User-agent: Mata.Hari | |
User-agent: MetagerBot | |
User-agent: MetaURI | |
User-agent: Microsoft.URL | |
User-agent: Microsoft URL Control | |
User-agent: Microsoft URL Control* | |
User-agent: Microsoft.URL.Control | |
User-agent: Microsoft URL Control - 5.01.4511 | |
User-agent: Microsoft URL Control - 6.00.8169 | |
User-agent: Microsoft URL Control - 6.01.9782 | |
User-agent: MIDown tool | |
User-agent: MIIxpc | |
User-agent: MIIxpc/4.2 | |
User-agent: Missigua Locator | |
User-agent: Mister PiX | |
User-agent: Mister.PiX | |
User-agent: Mister Pix II 2.01 | |
User-agent: Mister Pix II 2.02a | |
User-agent: Mister PiX version.dll | |
User-agent: MJ12bot | |
User-agent: MLBot | |
User-agent: moget | |
User-agent: moget/2.1 | |
User-agent: mozilla | |
User-agent: Mozilla | |
User-agent: Mozilla/2.0 (compatible; Ask Jeeves) | |
User-agent: mozilla/3 | |
User-agent: mozilla/4 | |
User-agent: Mozilla/4.0 (compatible; BullsEye; Windows 95) | |
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 2000) | |
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 95) | |
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 98) | |
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows ME) | |
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows NT) | |
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows XP) | |
User-agent: mozilla/5 | |
User-agent: MRSPUTNIK | |
User-agent: MSIECrawler | |
User-agent: MSRBOT | |
User-agent: MS Search 4.0 Robot | |
User-agent: MS Search 5.0 Robot | |
User-agent: munky | |
User-agent: naver | |
User-agent: Naverbot | |
User-agent: NaverBot | |
User-agent: NaverBot-1.0 | |
User-agent: Navroad | |
User-agent: NearSite | |
User-agent: NetAnts | |
User-agent: NetAnts/1.10 | |
User-agent: NetAnts/1.23 | |
User-agent: NetAnts/1.24 | |
User-agent: NetAnts/1.25 | |
User-agent: NetAttache | |
User-agent: NetAttache Light 1.1 | |
User-agent: Netcraft Web Server Survey | |
User-agent: NetMechanic | |
User-agent: NetSpider | |
User-agent: Net Vampire | |
User-agent: Net Vampire/3.0 | |
User-agent: NetZIP | |
User-agent: NetZip-Downloader | |
User-agent: NetZip-Downloader/1.0.62 (Win32; Dec 7 1998) | |
User-agent: NetZip Downloader 1.0 Win32(Nov 12 1998) | |
User-agent: NetZippy+(http://www.innerprise.net/usp-spider.asp) | |
User-agent: NetZippy+(http:/www.innerprise.net/usp-spider.asp) | |
User-agent: NICErsPRO | |
User-agent: NimbleCrawler | |
User-agent: NPbot | |
User-agent: NPBot | |
User-agent: NPBot/3 | |
User-agent: Nutch | |
User-agent: Nutch* | |
User-agent: NutchCVS/0.06-dev | |
User-agent: NutchCVS/0.7.1 | |
User-agent: NutchOrg | |
User-agent: oBot | |
User-agent: Ocelli | |
User-agent: Octopus | |
User-agent: Offline Explorer | |
User-agent: Offline.Explorer | |
User-agent: Offline Explorer/1.2 | |
User-agent: Offline Explorer/1.4 | |
User-agent: Offline Explorer/1.6 | |
User-agent: Offline Explorer/1.7 | |
User-agent: Offline Explorer/1.9 | |
User-agent: Offline Explorer/2.0 | |
User-agent: Offline Explorer/2.1 | |
User-agent: Offline Explorer/2.3 | |
User-agent: Offline Explorer/2.4 | |
User-agent: Offline Explorer/2.5 | |
User-agent: Offline Navigator | |
User-agent: OmniExplorer_Bot | |
User-agent: oneriot | |
User-agent: Openbot | |
User-agent: Openfind | |
User-agent: Openfind data gathere | |
User-agent: Openfind data gatherer | |
User-agent: Oracle Ultra Search | |
User-agent: OutfoxBot/0.5 | |
User-agent: PageGrabber | |
User-agent: Papa Foto | |
User-agent: pavuk | |
User-agent: PBWF | |
User-agent: pcBrowser | |
User-agent: penthesilea | |
User-agent: PerMan | |
User-agent: PGBot | |
User-agent: PhpDig | |
User-agent: Pingdom GIGRIB (http://www.pingdom.com) | |
User-agent: postrank | |
User-agent: ProPowerBot | |
User-agent: ProPowerBot/2.14 | |
User-agent: ProWebWalker | |
User-agent: psbot | |
User-agent: psycheclone | |
User-agent: Psycheclone | |
User-agent: Python-urllib | |
User-agent: QuepasaCreep | |
User-agent: QueryN Metasearch | |
User-agent: QueryN.Metasearch | |
User-agent: radian6 comment reader | |
User-agent: radian6 Feedfetcher | |
User-agent: Radiation Retriever | |
User-agent: Radiation Retriever 1.1 | |
User-agent: RB2B-bot | |
User-agent: RealDownload | |
User-agent: RealDownload/4.0.0.40 | |
User-agent: RealDownload/4.0.0.41 | |
User-agent: RealDownload/4.0.0.42 | |
User-agent: ReGet | |
User-agent: RepoMonkey | |
User-agent: RepoMonkey Bait & Tackle/v1.01 | |
User-agent: RepoMonkey Bait & Tackle | |
User-agent: RepoMonkey Bait & Tackle/v1.01 | |
User-agent: research-spider | |
User-agent: RMA | |
User-agent: Robozilla | |
User-agent: Roverbot | |
User-agent: RufusBot | |
User-agent: sbider | |
User-agent: Scooter/1.0 | |
User-agent: Scooter/1.0 scooter@pa.dec.com | |
User-agent: Scooter/1.1 (custom) | |
User-agent: Scooter/2.0 G.R.A.B. V1.1.0 | |
User-agent: Scooter/2.0 G.R.A.B. X2.0 | |
User-agent: Scooter2_Mercator_x-x.0 | |
User-agent: Scooter-3.0.EU | |
User-agent: Scooter-3.0.FS | |
User-agent: Scooter-3.0.HD | |
User-agent: Scooter-3.0QI | |
User-agent: Scooter-3.0.VNS | |
User-agent: Scooter-3.2 | |
User-agent: Scooter-3.2.BT | |
User-agent: Scooter-3.2.DIL | |
User-agent: Scooter-3.2.EX | |
User-agent: Scooter-3.2.JT | |
User-agent: Scooter-3.2.NIV | |
User-agent: Scooter-3.2.SF0 | |
User-agent: Scooter-3.2.snippet | |
User-agent: Scooter/3.3 | |
User-agent: Scooter-3.3dev | |
User-agent: Scooter/3.3.QA.pczukor | |
User-agent: Scooter/3.3_SF | |
User-agent: Scooter/3.3.vscooter | |
User-agent: Scooter-ARS-1.1 | |
User-agent: Scooter-ARS-1.1-ih | |
User-agent: Scooter_bh0-3.0.3 | |
User-agent: Scooter_trk3-3.0.3 | |
User-agent: scooter-venus-3.0.vns | |
User-agent: Scooter-W3-1.0 | |
User-agent: Scooter-W3.1.2 | |
User-agent: Scrubby | |
User-agent: SearchDaimon.com-dc | |
User-agent: searchpreview | |
User-agent: seekbot | |
User-agent: Seekbot | |
User-agent: Seekbot/1.0 | |
User-agent: SEOprofiler | |
User-agent: Shai'Hulud | |
User-agent: Shim-Crawler | |
User-agent: ShopWiki | |
User-agent: ShopWiki/1.0 | |
User-agent: SightupBot | |
User-agent: SiteBot | |
User-agent: SiteSnagger | |
User-agent: Slurp China | |
User-agent: SlySearch | |
User-agent: SmartDownload | |
User-agent: SmartDownload/1.2.76 (Win32; Apr 1 1999) | |
User-agent: SmartDownload/1.2.77 (Win32; Aug 17 1999) | |
User-agent: SmartDownload/1.2.77 (Win32; Feb 1 2000) | |
User-agent: SmartDownload/1.2.77 (Win32; Jun 19 2001) | |
User-agent: Snapbot | |
User-agent: Snappy | |
User-agent: Softlayer Server | |
User-agent: Sogou web spider | |
User-agent: sootle | |
User-agent: sosospider | |
User-agent: SpankBot | |
User-agent: spanner | |
User-agent: Speedy | |
User-agent: SpiderBot | |
User-agent: Sqworm | |
User-agent: Sqworm/2.9.85-BETA (beta_release; 20011115-775; i686-pc-linux | |
User-agent: ssearcher100 | |
User-agent: Stanford | |
User-agent: Stanford Comp Sci | |
User-agent: suggybot | |
User-agent: SuperBot | |
User-agent: SuperBot/2.6 | |
User-agent: SuperBot/3.0 (Win32) | |
User-agent: SuperBot/3.1 (Win32) | |
User-agent: SuperHTTP | |
User-agent: SuperHTTP/1.0 | |
User-agent: Surfbot | |
User-agent: SurveyBot | |
User-agent: suzuran | |
User-agent: Szukacz | |
User-agent: Szukacz/1.4 | |
User-agent: tAkeOut | |
User-agent: Teleport | |
User-agent: TeleportPro | |
User-agent: Teleport Pro | |
User-agent: Teleport Pro/1.29 | |
User-agent: Teleport Pro/1.29.1590 | |
User-agent: Teleport Pro/1.29.1634 | |
User-agent: Teleport Pro/1.29.1718 | |
User-agent: Teleport Pro/1.29.1820 | |
User-agent: Teleport Pro/1.29.1847 | |
User-agent: Telesoft | |
User-agent: Templeton | |
User-agent: Teoma | |
User-agent: The Intraformant | |
User-agent: The.Intraformant | |
User-agent: TheNomad | |
User-agent: TightTwatBot | |
User-agent: Titan | |
User-agent: toCrawl | |
User-agent: toCrawl/UrlDispatcher | |
User-agent: True_Robot | |
User-agent: True_Robot/1.0 | |
User-agent: turingos | |
User-agent: TurnitinBot | |
User-agent: TurnitinBot/1.5 | |
User-agent: Tweetmeme | |
User-agent: TwengaBot | |
User-agent: Twiceler | |
User-agent: URL Control | |
User-agent: UrlDispatcher | |
User-agent: ://URLFAN | |
User-agent: URL_Spider_Pro | |
User-agent: URLy Warning | |
User-agent: URLy.Warning | |
User-agent: VCI | |
User-agent: VCI WebViewer VCI WebViewer Win32 | |
User-agent: vobsub | |
User-agent: VoidEYE | |
User-agent: vscooter | |
User-agent: w3mir | |
User-agent: WatchDog/3.0 | |
User-agent: WebAuto | |
User-agent: WebAuto/3.40 (Win98; I) | |
User-agent: WebBandit | |
User-agent: WebBandit/3.50 | |
User-agent: WebCapture | |
User-agent: WebCapture 2.0 | |
User-agent: WebCatcher | |
User-agent: webcopier | |
User-agent: WebCopier | |
User-agent: WebCopier v.2.2 | |
User-agent: WebCopier v2.5 | |
User-agent: WebCopier v2.6 | |
User-agent: WebCopier v2.7a | |
User-agent: WebCopier v2.8 | |
User-agent: WebCopier v3.0 | |
User-agent: WebCopier v3.0.1 | |
User-agent: WebCopier v3.2 | |
User-agent: WebCopier v3.2a | |
User-agent: webcopy | |
User-agent: WebCopy | |
User-agent: webcrawl.net | |
User-agent: WebEmailExtrac | |
User-agent: WebEMailExtrac.* | |
User-agent: WebEnhancer | |
User-agent: WebFetch | |
User-agent: webfetch/2.1.0 | |
User-agent: WebFetcher | |
User-agent: WebGo IS | |
User-agent: Web Image Collector | |
User-agent: Web.Image.Collector | |
User-agent: WebLeacher | |
User-agent: WebmasterWorld Extractor | |
User-agent: WebmasterWorldForumBot | |
User-agent: webmirror | |
User-agent: WebMirror | |
User-agent: WebReaper | |
User-agent: Web Reaper | |
User-agent: WebReaper [info@webreaper.net] | |
User-agent: WebReaper v9.1 - www.otway.com/webreaper | |
User-agent: WebReaper v9.7 - www.webreaper.net | |
User-agent: WebReaper v9.8 - www.webreaper.net | |
User-agent: WebReaper vWebReaper v7.3 - www,otway.com/webreaper | |
User-agent: WebReaper [webreaper@otway.com] | |
User-agent: WebSauger | |
User-agent: WebSauger 1.20b | |
User-agent: WebSauger 1.20j | |
User-agent: WebSauger 1.20k | |
User-agent: website extractor | |
User-agent: Website eXtractor | |
User-agent: Website eXtractor (http:/www.asona.org) | |
User-agent: Website Quester | |
User-agent: Website.Quester | |
User-agent: Website Quester - www.asona.org | |
User-agent: Website Quester - www.esalesbiz.com/extra/ | |
User-agent: Webster Pro | |
User-agent: Webster.Pro | |
User-agent: WebStripper | |
User-agent: WebStripper/2.02 | |
User-agent: WebStripper/2.03 | |
User-agent: WebStripper/2.10 | |
User-agent: WebStripper/2.12 | |
User-agent: WebStripper/2.13 | |
User-agent: WebStripper/2.15 | |
User-agent: WebStripper/2.16 | |
User-agent: WebStripper/2.19 | |
User-agent: Web Sucker | |
User-agent: webvac | |
User-agent: WebVac | |
User-agent: WebVulnCrawl | |
User-agent: WebVulnScan | |
User-agent: WebWalk | |
User-agent: WebWasher | |
User-agent: WebWhacker | |
User-agent: WebZip | |
User-agent: WebZIP | |
User-agent: WebZIP/2.75 (http://www.spidersoft.com) | |
User-agent: WebZIP/2.75 (http:/www.spidersoft.com) | |
User-agent: WebZIP/3.65 (http://www.spidersoft.com) | |
User-agent: WebZIP/3.65 (http:/www.spidersoft.com) | |
User-agent: WebZIP/3.80 (http://www.spidersoft.com) | |
User-agent: WebZIP/3.80 (http:/www.spidersoft.com) | |
User-agent: WebZip/4.0 | |
User-agent: WebZIP/4.0 (http://www.spidersoft.com) | |
User-agent: WebZIP/4.0 (http:/www.spidersoft.com) | |
User-agent: WebZIP/4.1 (http://www.spidersoft.com) | |
User-agent: WebZIP/4.1 (http:/www.spidersoft.com) | |
User-agent: WebZIP/4.21 | |
User-agent: WebZIP/4.21 (http://www.spidersoft.com) | |
User-agent: WebZIP/4.21 (http:/www.spidersoft.com) | |
User-agent: WebZIP/5.0 | |
User-agent: WebZIP/5.0 (http://www.spidersoft.com) | |
User-agent: WebZIP/5.0 (http:/www.spidersoft.com) | |
User-agent: WebZIP/5.0 PR1 (http://www.spidersoft.com) | |
User-agent: WebZIP/5.0 PR1 (http:/www.spidersoft.com) | |
User-agent: wget | |
User-agent: wGet | |
User-agent: Wget | |
User-agent: Wget/1.10.2 | |
User-agent: Wget/1.5.2 | |
User-agent: Wget/1.5.3 | |
User-agent: Wget/1.6 | |
User-agent: Wget/1.7 | |
User-agent: Wget/1.8 | |
User-agent: Wget/1.8.1 | |
User-agent: Wget/1.8.1+cvs | |
User-agent: Wget/1.8.2 | |
User-agent: Wget/1.9-beta | |
User-agent: whitevector crawler | |
User-agent: Whitevector+Crawler | |
User-agent: Widow | |
User-agent: WikioFeedBot | |
User-agent: wikiwix-bot-3.0 | |
User-agent: Willow | |
User-agent: WinHTTrack | |
User-agent: Wise-Guys | |
User-agent: woozweb-monitoring | |
User-agent: woriobot | |
User-agent: WWW-Collector | |
User-agent: WWW-Collector-E | |
User-agent: WWWOFFLE | |
User-agent: Xaldon WebSpider | |
User-agent: Xaldon WebSpider 2.5.b3 | |
User-agent: Xenu | |
User-agent: Xenu Link Sleuth | |
User-agent: Xenu's | |
User-agent: Xenu's Link Sleuth 1.1c | |
User-agent: xGet | |
User-agent: Yahoo-MMCrawler | |
User-agent: YahooSeeker/CafeKelsa | |
User-agent: Yeti | |
User-agent: YodaoBot | |
User-agent: YRSPider | |
User-agent: Zao | |
User-agent: Zealbot | |
User-agent: Zeus | |
User-agent: Zeus 11389 Webster Pro V2.9 Win32 | |
User-agent: Zeus 11652 Webster Pro V2.9 Win32 | |
User-agent: Zeus 18018 Webster Pro V2.9 Win32 | |
User-agent: Zeus 26378 Webster Pro V2.9 Win32 | |
User-agent: Zeus 30747 Webster Pro V2.9 Win32 | |
User-agent: Zeus 32297 Webster Pro V2.9 Win32 | |
User-agent: Zeus 39206 Webster Pro V2.9 Win32 | |
User-agent: Zeus 41641 Webster Pro V2.9 Win32 | |
User-agent: Zeus 44238 Webster Pro V2.9 Win32 | |
User-agent: Zeus 51070 Webster Pro V2.9 Win32 | |
User-agent: Zeus 51674 Webster Pro V2.9 Win32 | |
User-agent: Zeus 51837 Webster Pro V2.9 Win32 | |
User-agent: Zeus 63567 Webster Pro V2.9 Win32 | |
User-agent: Zeus 6694 Webster Pro V2.9 Win32 | |
User-agent: Zeus 71129 Webster Pro V2.9 Win32 | |
User-agent: Zeus 82016 Webster Pro V2.9 Win32 | |
User-agent: Zeus 82900 Webster Pro V2.9 Win32 | |
User-agent: Zeus 84842 Webster Pro V2.9 Win32 | |
User-agent: Zeus 90872 Webster Pro V2.9 Win32 | |
User-agent: Zeus 94934 Webster Pro V2.9 Win32 | |
User-agent: Zeus 95245 Webster Pro V2.9 Win32 | |
User-agent: Zeus 95351 Webster Pro V2.9 Win32 | |
User-agent: Zeus 97371 Webster Pro V2.9 Win32 | |
User-agent: Zeus Link Scout | |
User-agent: ZyBorg | |
Disallow: / | |
Sitemap: http://tugatech.com.pt/sitemap.xml |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment