| User-agent: Mediapartners-Google | |
| Disallow: | |
| User-agent: Mediapartners-Google* | |
| Disallow: | |
| User-agent: * | |
| Disallow: /abuse | |
| Disallow: /admgt/ | |
| Disallow: /contact | |
| Disallow: /donate | |
| Disallow: /go/ | |
| Disallow: /login | |
| Disallow: /modcp | |
| Disallow: /post | |
| Disallow: /privmsg | |
| Disallow: /spa/ | |
| Disallow: /sta/ | |
| User-agent: 008 | |
| User-agent: Accoona | |
| User-agent: aipbot | |
| User-agent: aipbot* | |
| User-agent: aipbot/1.0 | |
| User-agent: Alexa | |
| User-agent: Alexa Bitlybot | |
| User-agent: Alexibot | |
| User-agent: AltaVista Intranet V2.0 AVS EVAL search@freeit.com | |
| User-agent: AltaVista Intranet V2.0 Compaq Altavista Eval sveand@altavista.net | |
| User-agent: AltaVista Intranet V2.0 evreka.com crawler@evreka.com | |
| User-agent: AltaVista V2.0B crawler@evreka.com | |
| User-agent: Anonymous | |
| User-agent: ApocalXExplorerBot | |
| User-agent: appie | |
| User-agent: Aqua_Products | |
| User-agent: Argus/1.1 | |
| User-agent: Artabus | |
| User-agent: Ask Jeeves | |
| User-agent: asterias | |
| User-agent: atSpider | |
| User-agent: attentio | |
| User-agent: AV Fetch 1.0 | |
| User-agent: AVSearch-3.0(AltaVista/AVC) | |
| User-agent: AWS Cloud Based | |
| User-agent: b2w | |
| User-agent: b2w/0.1 | |
| User-agent: BackDoorBot | |
| User-agent: BackDoorBot/1.0 | |
| User-agent: Baiduspider | |
| User-agent: becomebot | |
| User-agent: BecomeBot | |
| User-agent: BigBrother | |
| User-agent: BIGLOTRON (BETA 2;GNU/Linux) | |
| User-agent: BizInformation | |
| User-agent: Black Hole | |
| User-agent: Black.Hole | |
| User-agent: BlackWidow | |
| User-agent: BlowFish | |
| User-agent: BlowFish/1.0 | |
| User-agent: BoardPulse | |
| User-agent: boitho.com-dc | |
| User-agent: Bookmark search tool | |
| User-agent: bot/1.0 | |
| User-agent: BotALot | |
| User-agent: Bot mailto:craftbot@yahoo.com | |
| User-agent: BotRightHere | |
| User-agent: BrandProtect | |
| User-agent: BuiltBotTough | |
| User-agent: Bullseye | |
| User-agent: Bullseye/1.0 | |
| User-agent: BunnySlippers | |
| User-agent: CazoodleBot | |
| User-agent: Cegbfeieh | |
| User-agent: cfetch | |
| User-agent: cfetch/1.0 | |
| User-agent: CheeseBot | |
| User-agent: CherryPicker | |
| User-agent: CherryPicker /1.0 | |
| User-agent: CherryPickerElite/1.0 | |
| User-agent: CherryPickerSE/1.0 | |
| User-agent: ChinaClaw | |
| User-agent: Collage | |
| User-agent: cometrics-bot | |
| User-agent: complex_network_group | |
| User-agent: convera | |
| User-agent: ConveraCrawler | |
| User-agent: ConveraCrawler/0.2 | |
| User-agent: ConveraCrawler/0.9d | |
| User-agent: Convera Internet Spider V6.x | |
| User-agent: ConveraMultiMediaCrawler/0.1 | |
| User-agent: Copernic | |
| User-agent: CopyRightCheck | |
| User-agent: cosmos | |
| User-agent: Crescent | |
| User-agent: Crescent Internet ToolPak HTTP OLE Control v.1.0 | |
| User-agent: Crescent Internet ToolPak HTTPOLE Control v.1.0 | |
| User-agent: Curl | |
| User-agent: Custo | |
| User-agent: CydralSpider | |
| User-agent: Deepnet Explorer | |
| User-agent: default.ida | |
| User-agent: DigExt | |
| User-agent: DISCo | |
| User-agent: discobot | |
| User-agent: DISCoFinder | |
| User-agent: DISCo Pump | |
| User-agent: DISCo Pump 3.0 | |
| User-agent: DISCo Pump 3.1 | |
| User-agent: DISCo Pump 3.2 | |
| User-agent: DittoSpyder | |
| User-agent: DOC | |
| User-agent: dotbot | |
| User-agent: DotBot | |
| User-agent: DotBot/1.1 | |
| User-agent: Download Demon | |
| User-agent: Download Demon/3.2.0.8 | |
| User-agent: Download Demon/3.5.0.11 | |
| User-agent: Download Ninja | |
| User-agent: Download Wonder | |
| User-agent: DSurf | |
| User-agent: Dulance bot | |
| User-agent: dumbot | |
| User-agent: eCatch | |
| User-agent: eCatch/3.0 | |
| User-agent: echo! | |
| User-agent: EchO!/2.0 | |
| User-agent: EirGrabber | |
| User-agent: EliteSys Entry | |
| User-agent: EmailCollector | |
| User-agent: Email Extractor | |
| User-agent: EmailSiphon | |
| User-agent: EmailSmartz | |
| User-agent: EmailWolf | |
| User-agent: Enterprise_Search | |
| User-agent: Enterprise_Search/1.0 | |
| User-agent: EroCrawler | |
| User-agent: es | |
| User-agent: ESIRover | |
| User-agent: e-SocietyRobot | |
| User-agent: Exabot | |
| User-agent: Exabot/2.0 | |
| User-agent: Exabot-Images | |
| User-agent: Express WebPictures | |
| User-agent: Express WebPictures (www.express-soft.com) | |
| User-agent: ExtractorPro | |
| User-agent: EyeNetIE | |
| User-agent: FairAd Client | |
| User-agent: Fairshare | |
| User-agent: Fasterfox | |
| User-agent: Fetch | |
| User-agent: Flaming AttackBot | |
| User-agent: Flamingo_SearchEngine | |
| User-agent: FlashGet | |
| User-agent: FlashGet WebWasher 3.2 | |
| User-agent: Foobot | |
| User-agent: FreeFind | |
| User-agent: FreeWebMonitoring SiteChecker/0.1 | |
| User-agent: FrontPage | |
| User-agent: FrontPage [NC,OR] | |
| User-agent: FurlBot | |
| User-agent: Gaisbot | |
| User-agent: Gaisbot/3.0 | |
| User-agent: GetBot | |
| User-agent: GetRight | |
| User-agent: GetRight/2.11 | |
| User-agent: GetRight/3.1 | |
| User-agent: GetRight/3.2 | |
| User-agent: GetRight/3.3 | |
| User-agent: GetRight/3.3.3 | |
| User-agent: GetRight/3.3.4 | |
| User-agent: GetRight/4.0.0 | |
| User-agent: GetRight/4.1.0 | |
| User-agent: GetRight/4.1.1 | |
| User-agent: GetRight/4.1.2 | |
| User-agent: GetRight/4.2 | |
| User-agent: GetRight/4.2b (Portuguxeas) | |
| User-agent: GetRight/4.2c | |
| User-agent: GetRight/4.3 | |
| User-agent: GetRight/4.5 | |
| User-agent: GetRight/4.5a | |
| User-agent: GetRight/4.5b | |
| User-agent: GetRight/4.5b1 | |
| User-agent: GetRight/4.5b2 | |
| User-agent: GetRight/4.5b3 | |
| User-agent: GetRight/4.5b6 | |
| User-agent: GetRight/4.5b7 | |
| User-agent: GetRight/4.5c | |
| User-agent: GetRight/4.5d | |
| User-agent: GetRight/4.5e | |
| User-agent: GetRight/5.0beta1 | |
| User-agent: GetRight/5.0beta2 | |
| User-agent: GetUrl | |
| User-agent: GetWeb! | |
| User-agent: Gigabot | |
| User-agent: Go-Ahead-Got-It | |
| User-agent: Go!Zilla | |
| User-agent: Go!Zilla 3.3 (www.gozilla.com) | |
| User-agent: Go!Zilla 3.5 (www.gozilla.com) | |
| User-agent: Go!Zilla (www.gozilla.com) | |
| User-agent: GrabNet | |
| User-agent: Grafula | |
| User-agent: grub | |
| User-agent: grub-client | |
| User-agent: Hackertarget.com | |
| User-agent: Harvest | |
| User-agent: Harvest/1.5 | |
| User-agent: Hatena Antenna | |
| User-agent: HavIndex | |
| User-agent: heritrix | |
| User-agent: hloader | |
| User-agent: HMView | |
| User-agent: httplib | |
| User-agent: httrack | |
| User-agent: HTTrack | |
| User-agent: HTTrack 3.0 | |
| User-agent: HTTrack 3.0x | |
| User-agent: HTTrack [NC,OR] | |
| User-agent: humanlinks | |
| User-agent: ichiro | |
| User-agent: IconSurf | |
| User-agent: Igentia | |
| User-agent: Image Collector | |
| User-agent: Image Stripper | |
| User-agent: Image Sucker | |
| User-agent: Indy Library | |
| User-agent: Indy Library [NC,OR] | |
| User-agent: InfoNaviRobot | |
| User-agent: InfoSpiders | |
| User-agent: InterGET | |
| User-agent: Internet Explore | |
| User-agent: Internet Ninja | |
| User-agent: Internet Ninja 4.0 | |
| User-agent: Internet Ninja 5.0 | |
| User-agent: Internet Ninja 6.0 | |
| User-agent: InternetSupervision | |
| User-agent: IRLbot | |
| User-agent: Iron | |
| User-agent: Iron33/1.0.2 | |
| User-agent: Jeeves | |
| User-agent: JennyBot | |
| User-agent: Jetbot | |
| User-agent: Jetbot/1.0 | |
| User-agent: JetCar | |
| User-agent: Jobo | |
| User-agent: JOC Web Spider | |
| User-agent: kalooga | |
| User-agent: KDD Exploror | |
| User-agent: Kenjin Spider | |
| User-agent: Kenjin.Spider | |
| User-agent: Keyword Density | |
| User-agent: Keyword.Density | |
| User-agent: Keyword Density/0.9 | |
| User-agent: larbin | |
| User-agent: Larbin | |
| User-agent: larbin_2.6.2 (kabura@sushi.com) | |
| User-agent: larbin_2.6.2 kabura@sushi.com | |
| User-agent: larbin_2.6.2 (larbin2.6.2@unspecified.mail) | |
| User-agent: larbin_2.6.2 larbin2.6.2@unspecified.mail | |
| User-agent: larbin_2.6.2 larbin@correa.org | |
| User-agent: larbin_2.6.2 listonATccDOTgatechDOTedu | |
| User-agent: larbin_2.6.2 (listonATccDOTgatechDOTedu) | |
| User-agent: larbin_2.6.2 (vitalbox1@hotmail.com) | |
| User-agent: larbin_2.6.2 vitalbox1@hotmail.com | |
| User-agent: larbin (samualt9@bigfoot.com) | |
| User-agent: larbin samualt9@bigfoot.com | |
| User-agent: LBot | |
| User-agent: LeechFTP | |
| User-agent: LexiBot | |
| User-agent: libWeb/clsHTTP | |
| User-agent: libWeb/clsHTTPDisallow: / | |
| User-agent: libwww | |
| User-agent: LightningDownload | |
| User-agent: Linguee | |
| User-agent: LinkedIn | |
| User-agent: LinkextractorPro | |
| User-agent: Linknzbot | |
| User-agent: Linknzbot* | |
| User-agent: Linknzbot 2004 | |
| User-agent: LinkScan | |
| User-agent: LinkScan/8.1a Unix | |
| User-agent: LinkScan/8.1a.Unix | |
| User-agent: LinkScan/8.1a Unix Disallow: / | |
| User-agent: linksmanager | |
| User-agent: LinksManager | |
| User-agent: LinksManager.com_bot | |
| User-agent: LinkWalker | |
| User-agent: LjSEEK | |
| User-agent: LNSpiderguy | |
| User-agent: looksmart | |
| User-agent: LWP | |
| User-agent: LWP* | |
| User-agent: lwp-trivial | |
| User-agent: lwp-trivial/1.34 | |
| User-agent: magpie-crawler | |
| User-agent: Mail Sweeper | |
| User-agent: Marketwirebot | |
| User-agent: Mass Downloader | |
| User-agent: Mass Downloader/2.2 | |
| User-agent: Mata Hari | |
| User-agent: Mata.Hari | |
| User-agent: MetagerBot | |
| User-agent: MetaURI | |
| User-agent: Microsoft.URL | |
| User-agent: Microsoft URL Control | |
| User-agent: Microsoft URL Control* | |
| User-agent: Microsoft.URL.Control | |
| User-agent: Microsoft URL Control - 5.01.4511 | |
| User-agent: Microsoft URL Control - 6.00.8169 | |
| User-agent: Microsoft URL Control - 6.01.9782 | |
| User-agent: MIDown tool | |
| User-agent: MIIxpc | |
| User-agent: MIIxpc/4.2 | |
| User-agent: Missigua Locator | |
| User-agent: Mister PiX | |
| User-agent: Mister.PiX | |
| User-agent: Mister Pix II 2.01 | |
| User-agent: Mister Pix II 2.02a | |
| User-agent: Mister PiX version.dll | |
| User-agent: MJ12bot | |
| User-agent: MLBot | |
| User-agent: moget | |
| User-agent: moget/2.1 | |
| User-agent: mozilla | |
| User-agent: Mozilla | |
| User-agent: Mozilla/2.0 (compatible; Ask Jeeves) | |
| User-agent: mozilla/3 | |
| User-agent: mozilla/4 | |
| User-agent: Mozilla/4.0 (compatible; BullsEye; Windows 95) | |
| User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 2000) | |
| User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 95) | |
| User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 98) | |
| User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows ME) | |
| User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows NT) | |
| User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows XP) | |
| User-agent: mozilla/5 | |
| User-agent: MRSPUTNIK | |
| User-agent: MSIECrawler | |
| User-agent: MSRBOT | |
| User-agent: MS Search 4.0 Robot | |
| User-agent: MS Search 5.0 Robot | |
| User-agent: munky | |
| User-agent: naver | |
| User-agent: Naverbot | |
| User-agent: NaverBot | |
| User-agent: NaverBot-1.0 | |
| User-agent: Navroad | |
| User-agent: NearSite | |
| User-agent: NetAnts | |
| User-agent: NetAnts/1.10 | |
| User-agent: NetAnts/1.23 | |
| User-agent: NetAnts/1.24 | |
| User-agent: NetAnts/1.25 | |
| User-agent: NetAttache | |
| User-agent: NetAttache Light 1.1 | |
| User-agent: Netcraft Web Server Survey | |
| User-agent: NetMechanic | |
| User-agent: NetSpider | |
| User-agent: Net Vampire | |
| User-agent: Net Vampire/3.0 | |
| User-agent: NetZIP | |
| User-agent: NetZip-Downloader | |
| User-agent: NetZip-Downloader/1.0.62 (Win32; Dec 7 1998) | |
| User-agent: NetZip Downloader 1.0 Win32(Nov 12 1998) | |
| User-agent: NetZippy+(http://www.innerprise.net/usp-spider.asp) | |
| User-agent: NetZippy+(http:/www.innerprise.net/usp-spider.asp) | |
| User-agent: NICErsPRO | |
| User-agent: NimbleCrawler | |
| User-agent: NPbot | |
| User-agent: NPBot | |
| User-agent: NPBot/3 | |
| User-agent: Nutch | |
| User-agent: Nutch* | |
| User-agent: NutchCVS/0.06-dev | |
| User-agent: NutchCVS/0.7.1 | |
| User-agent: NutchOrg | |
| User-agent: oBot | |
| User-agent: Ocelli | |
| User-agent: Octopus | |
| User-agent: Offline Explorer | |
| User-agent: Offline.Explorer | |
| User-agent: Offline Explorer/1.2 | |
| User-agent: Offline Explorer/1.4 | |
| User-agent: Offline Explorer/1.6 | |
| User-agent: Offline Explorer/1.7 | |
| User-agent: Offline Explorer/1.9 | |
| User-agent: Offline Explorer/2.0 | |
| User-agent: Offline Explorer/2.1 | |
| User-agent: Offline Explorer/2.3 | |
| User-agent: Offline Explorer/2.4 | |
| User-agent: Offline Explorer/2.5 | |
| User-agent: Offline Navigator | |
| User-agent: OmniExplorer_Bot | |
| User-agent: oneriot | |
| User-agent: Openbot | |
| User-agent: Openfind | |
| User-agent: Openfind data gathere | |
| User-agent: Openfind data gatherer | |
| User-agent: Oracle Ultra Search | |
| User-agent: OutfoxBot/0.5 | |
| User-agent: PageGrabber | |
| User-agent: Papa Foto | |
| User-agent: pavuk | |
| User-agent: PBWF | |
| User-agent: pcBrowser | |
| User-agent: penthesilea | |
| User-agent: PerMan | |
| User-agent: PGBot | |
| User-agent: PhpDig | |
| User-agent: Pingdom GIGRIB (http://www.pingdom.com) | |
| User-agent: postrank | |
| User-agent: ProPowerBot | |
| User-agent: ProPowerBot/2.14 | |
| User-agent: ProWebWalker | |
| User-agent: psbot | |
| User-agent: psycheclone | |
| User-agent: Psycheclone | |
| User-agent: Python-urllib | |
| User-agent: QuepasaCreep | |
| User-agent: QueryN Metasearch | |
| User-agent: QueryN.Metasearch | |
| User-agent: radian6 comment reader | |
| User-agent: radian6 Feedfetcher | |
| User-agent: Radiation Retriever | |
| User-agent: Radiation Retriever 1.1 | |
| User-agent: RB2B-bot | |
| User-agent: RealDownload | |
| User-agent: RealDownload/4.0.0.40 | |
| User-agent: RealDownload/4.0.0.41 | |
| User-agent: RealDownload/4.0.0.42 | |
| User-agent: ReGet | |
| User-agent: RepoMonkey | |
| User-agent: RepoMonkey Bait & Tackle/v1.01 | |
| User-agent: RepoMonkey Bait & Tackle | |
| User-agent: RepoMonkey Bait & Tackle/v1.01 | |
| User-agent: research-spider | |
| User-agent: RMA | |
| User-agent: Robozilla | |
| User-agent: Roverbot | |
| User-agent: RufusBot | |
| User-agent: sbider | |
| User-agent: Scooter/1.0 | |
| User-agent: Scooter/1.0 scooter@pa.dec.com | |
| User-agent: Scooter/1.1 (custom) | |
| User-agent: Scooter/2.0 G.R.A.B. V1.1.0 | |
| User-agent: Scooter/2.0 G.R.A.B. X2.0 | |
| User-agent: Scooter2_Mercator_x-x.0 | |
| User-agent: Scooter-3.0.EU | |
| User-agent: Scooter-3.0.FS | |
| User-agent: Scooter-3.0.HD | |
| User-agent: Scooter-3.0QI | |
| User-agent: Scooter-3.0.VNS | |
| User-agent: Scooter-3.2 | |
| User-agent: Scooter-3.2.BT | |
| User-agent: Scooter-3.2.DIL | |
| User-agent: Scooter-3.2.EX | |
| User-agent: Scooter-3.2.JT | |
| User-agent: Scooter-3.2.NIV | |
| User-agent: Scooter-3.2.SF0 | |
| User-agent: Scooter-3.2.snippet | |
| User-agent: Scooter/3.3 | |
| User-agent: Scooter-3.3dev | |
| User-agent: Scooter/3.3.QA.pczukor | |
| User-agent: Scooter/3.3_SF | |
| User-agent: Scooter/3.3.vscooter | |
| User-agent: Scooter-ARS-1.1 | |
| User-agent: Scooter-ARS-1.1-ih | |
| User-agent: Scooter_bh0-3.0.3 | |
| User-agent: Scooter_trk3-3.0.3 | |
| User-agent: scooter-venus-3.0.vns | |
| User-agent: Scooter-W3-1.0 | |
| User-agent: Scooter-W3.1.2 | |
| User-agent: Scrubby | |
| User-agent: SearchDaimon.com-dc | |
| User-agent: searchpreview | |
| User-agent: seekbot | |
| User-agent: Seekbot | |
| User-agent: Seekbot/1.0 | |
| User-agent: SEOprofiler | |
| User-agent: Shai'Hulud | |
| User-agent: Shim-Crawler | |
| User-agent: ShopWiki | |
| User-agent: ShopWiki/1.0 | |
| User-agent: SightupBot | |
| User-agent: SiteBot | |
| User-agent: SiteSnagger | |
| User-agent: Slurp China | |
| User-agent: SlySearch | |
| User-agent: SmartDownload | |
| User-agent: SmartDownload/1.2.76 (Win32; Apr 1 1999) | |
| User-agent: SmartDownload/1.2.77 (Win32; Aug 17 1999) | |
| User-agent: SmartDownload/1.2.77 (Win32; Feb 1 2000) | |
| User-agent: SmartDownload/1.2.77 (Win32; Jun 19 2001) | |
| User-agent: Snapbot | |
| User-agent: Snappy | |
| User-agent: Softlayer Server | |
| User-agent: Sogou web spider | |
| User-agent: sootle | |
| User-agent: sosospider | |
| User-agent: SpankBot | |
| User-agent: spanner | |
| User-agent: Speedy | |
| User-agent: SpiderBot | |
| User-agent: Sqworm | |
| User-agent: Sqworm/2.9.85-BETA (beta_release; 20011115-775; i686-pc-linux | |
| User-agent: ssearcher100 | |
| User-agent: Stanford | |
| User-agent: Stanford Comp Sci | |
| User-agent: suggybot | |
| User-agent: SuperBot | |
| User-agent: SuperBot/2.6 | |
| User-agent: SuperBot/3.0 (Win32) | |
| User-agent: SuperBot/3.1 (Win32) | |
| User-agent: SuperHTTP | |
| User-agent: SuperHTTP/1.0 | |
| User-agent: Surfbot | |
| User-agent: SurveyBot | |
| User-agent: suzuran | |
| User-agent: Szukacz | |
| User-agent: Szukacz/1.4 | |
| User-agent: tAkeOut | |
| User-agent: Teleport | |
| User-agent: TeleportPro | |
| User-agent: Teleport Pro | |
| User-agent: Teleport Pro/1.29 | |
| User-agent: Teleport Pro/1.29.1590 | |
| User-agent: Teleport Pro/1.29.1634 | |
| User-agent: Teleport Pro/1.29.1718 | |
| User-agent: Teleport Pro/1.29.1820 | |
| User-agent: Teleport Pro/1.29.1847 | |
| User-agent: Telesoft | |
| User-agent: Templeton | |
| User-agent: Teoma | |
| User-agent: The Intraformant | |
| User-agent: The.Intraformant | |
| User-agent: TheNomad | |
| User-agent: TightTwatBot | |
| User-agent: Titan | |
| User-agent: toCrawl | |
| User-agent: toCrawl/UrlDispatcher | |
| User-agent: True_Robot | |
| User-agent: True_Robot/1.0 | |
| User-agent: turingos | |
| User-agent: TurnitinBot | |
| User-agent: TurnitinBot/1.5 | |
| User-agent: Tweetmeme | |
| User-agent: TwengaBot | |
| User-agent: Twiceler | |
| User-agent: URL Control | |
| User-agent: UrlDispatcher | |
| User-agent: ://URLFAN | |
| User-agent: URL_Spider_Pro | |
| User-agent: URLy Warning | |
| User-agent: URLy.Warning | |
| User-agent: VCI | |
| User-agent: VCI WebViewer VCI WebViewer Win32 | |
| User-agent: vobsub | |
| User-agent: VoidEYE | |
| User-agent: vscooter | |
| User-agent: w3mir | |
| User-agent: WatchDog/3.0 | |
| User-agent: WebAuto | |
| User-agent: WebAuto/3.40 (Win98; I) | |
| User-agent: WebBandit | |
| User-agent: WebBandit/3.50 | |
| User-agent: WebCapture | |
| User-agent: WebCapture 2.0 | |
| User-agent: WebCatcher | |
| User-agent: webcopier | |
| User-agent: WebCopier | |
| User-agent: WebCopier v.2.2 | |
| User-agent: WebCopier v2.5 | |
| User-agent: WebCopier v2.6 | |
| User-agent: WebCopier v2.7a | |
| User-agent: WebCopier v2.8 | |
| User-agent: WebCopier v3.0 | |
| User-agent: WebCopier v3.0.1 | |
| User-agent: WebCopier v3.2 | |
| User-agent: WebCopier v3.2a | |
| User-agent: webcopy | |
| User-agent: WebCopy | |
| User-agent: webcrawl.net | |
| User-agent: WebEmailExtrac | |
| User-agent: WebEMailExtrac.* | |
| User-agent: WebEnhancer | |
| User-agent: WebFetch | |
| User-agent: webfetch/2.1.0 | |
| User-agent: WebFetcher | |
| User-agent: WebGo IS | |
| User-agent: Web Image Collector | |
| User-agent: Web.Image.Collector | |
| User-agent: WebLeacher | |
| User-agent: WebmasterWorld Extractor | |
| User-agent: WebmasterWorldForumBot | |
| User-agent: webmirror | |
| User-agent: WebMirror | |
| User-agent: WebReaper | |
| User-agent: Web Reaper | |
| User-agent: WebReaper [info@webreaper.net] | |
| User-agent: WebReaper v9.1 - www.otway.com/webreaper | |
| User-agent: WebReaper v9.7 - www.webreaper.net | |
| User-agent: WebReaper v9.8 - www.webreaper.net | |
| User-agent: WebReaper vWebReaper v7.3 - www,otway.com/webreaper | |
| User-agent: WebReaper [webreaper@otway.com] | |
| User-agent: WebSauger | |
| User-agent: WebSauger 1.20b | |
| User-agent: WebSauger 1.20j | |
| User-agent: WebSauger 1.20k | |
| User-agent: website extractor | |
| User-agent: Website eXtractor | |
| User-agent: Website eXtractor (http:/www.asona.org) | |
| User-agent: Website Quester | |
| User-agent: Website.Quester | |
| User-agent: Website Quester - www.asona.org | |
| User-agent: Website Quester - www.esalesbiz.com/extra/ | |
| User-agent: Webster Pro | |
| User-agent: Webster.Pro | |
| User-agent: WebStripper | |
| User-agent: WebStripper/2.02 | |
| User-agent: WebStripper/2.03 | |
| User-agent: WebStripper/2.10 | |
| User-agent: WebStripper/2.12 | |
| User-agent: WebStripper/2.13 | |
| User-agent: WebStripper/2.15 | |
| User-agent: WebStripper/2.16 | |
| User-agent: WebStripper/2.19 | |
| User-agent: Web Sucker | |
| User-agent: webvac | |
| User-agent: WebVac | |
| User-agent: WebVulnCrawl | |
| User-agent: WebVulnScan | |
| User-agent: WebWalk | |
| User-agent: WebWasher | |
| User-agent: WebWhacker | |
| User-agent: WebZip | |
| User-agent: WebZIP | |
| User-agent: WebZIP/2.75 (http://www.spidersoft.com) | |
| User-agent: WebZIP/2.75 (http:/www.spidersoft.com) | |
| User-agent: WebZIP/3.65 (http://www.spidersoft.com) | |
| User-agent: WebZIP/3.65 (http:/www.spidersoft.com) | |
| User-agent: WebZIP/3.80 (http://www.spidersoft.com) | |
| User-agent: WebZIP/3.80 (http:/www.spidersoft.com) | |
| User-agent: WebZip/4.0 | |
| User-agent: WebZIP/4.0 (http://www.spidersoft.com) | |
| User-agent: WebZIP/4.0 (http:/www.spidersoft.com) | |
| User-agent: WebZIP/4.1 (http://www.spidersoft.com) | |
| User-agent: WebZIP/4.1 (http:/www.spidersoft.com) | |
| User-agent: WebZIP/4.21 | |
| User-agent: WebZIP/4.21 (http://www.spidersoft.com) | |
| User-agent: WebZIP/4.21 (http:/www.spidersoft.com) | |
| User-agent: WebZIP/5.0 | |
| User-agent: WebZIP/5.0 (http://www.spidersoft.com) | |
| User-agent: WebZIP/5.0 (http:/www.spidersoft.com) | |
| User-agent: WebZIP/5.0 PR1 (http://www.spidersoft.com) | |
| User-agent: WebZIP/5.0 PR1 (http:/www.spidersoft.com) | |
| User-agent: wget | |
| User-agent: wGet | |
| User-agent: Wget | |
| User-agent: Wget/1.10.2 | |
| User-agent: Wget/1.5.2 | |
| User-agent: Wget/1.5.3 | |
| User-agent: Wget/1.6 | |
| User-agent: Wget/1.7 | |
| User-agent: Wget/1.8 | |
| User-agent: Wget/1.8.1 | |
| User-agent: Wget/1.8.1+cvs | |
| User-agent: Wget/1.8.2 | |
| User-agent: Wget/1.9-beta | |
| User-agent: whitevector crawler | |
| User-agent: Whitevector+Crawler | |
| User-agent: Widow | |
| User-agent: WikioFeedBot | |
| User-agent: wikiwix-bot-3.0 | |
| User-agent: Willow | |
| User-agent: WinHTTrack | |
| User-agent: Wise-Guys | |
| User-agent: woozweb-monitoring | |
| User-agent: woriobot | |
| User-agent: WWW-Collector | |
| User-agent: WWW-Collector-E | |
| User-agent: WWWOFFLE | |
| User-agent: Xaldon WebSpider | |
| User-agent: Xaldon WebSpider 2.5.b3 | |
| User-agent: Xenu | |
| User-agent: Xenu Link Sleuth | |
| User-agent: Xenu's | |
| User-agent: Xenu's Link Sleuth 1.1c | |
| User-agent: xGet | |
| User-agent: Yahoo-MMCrawler | |
| User-agent: YahooSeeker/CafeKelsa | |
| User-agent: Yeti | |
| User-agent: YodaoBot | |
| User-agent: YRSPider | |
| User-agent: Zao | |
| User-agent: Zealbot | |
| User-agent: Zeus | |
| User-agent: Zeus 11389 Webster Pro V2.9 Win32 | |
| User-agent: Zeus 11652 Webster Pro V2.9 Win32 | |
| User-agent: Zeus 18018 Webster Pro V2.9 Win32 | |
| User-agent: Zeus 26378 Webster Pro V2.9 Win32 | |
| User-agent: Zeus 30747 Webster Pro V2.9 Win32 | |
| User-agent: Zeus 32297 Webster Pro V2.9 Win32 | |
| User-agent: Zeus 39206 Webster Pro V2.9 Win32 | |
| User-agent: Zeus 41641 Webster Pro V2.9 Win32 | |
| User-agent: Zeus 44238 Webster Pro V2.9 Win32 | |
| User-agent: Zeus 51070 Webster Pro V2.9 Win32 | |
| User-agent: Zeus 51674 Webster Pro V2.9 Win32 | |
| User-agent: Zeus 51837 Webster Pro V2.9 Win32 | |
| User-agent: Zeus 63567 Webster Pro V2.9 Win32 | |
| User-agent: Zeus 6694 Webster Pro V2.9 Win32 | |
| User-agent: Zeus 71129 Webster Pro V2.9 Win32 | |
| User-agent: Zeus 82016 Webster Pro V2.9 Win32 | |
| User-agent: Zeus 82900 Webster Pro V2.9 Win32 | |
| User-agent: Zeus 84842 Webster Pro V2.9 Win32 | |
| User-agent: Zeus 90872 Webster Pro V2.9 Win32 | |
| User-agent: Zeus 94934 Webster Pro V2.9 Win32 | |
| User-agent: Zeus 95245 Webster Pro V2.9 Win32 | |
| User-agent: Zeus 95351 Webster Pro V2.9 Win32 | |
| User-agent: Zeus 97371 Webster Pro V2.9 Win32 | |
| User-agent: Zeus Link Scout | |
| User-agent: ZyBorg | |
| Disallow: / | |
| Sitemap: http://tugatech.com.pt/sitemap.xml |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment