User-agent: http://www.almaden.ibm.com/cs/crawler User-agent: http://www.almaden.ibm.com/cs/crawler [c01] User-agent: Anawave User-agent: baiduspider User-agent: Baiduspider+ User-agent: Baiduspider+(+http://www.baidu.com/search/spider.htm) User-agent: BDFetch User-agent: DoCoMo User-agent: EmailCollector User-agent: EmailSiphon User-agent: ExtractorPro User-agent: FlashSite User-agent: Gigabot User-agent: Go-Get-It User-agent: Grab-a-Site User-agent: HotCargo User-agent: HttpLoader User-agent: ia_archiver User-agent: ichiro/1.0 User-agent: http://www.istarthere.com User-agent: http://www.kinja.com User-agent: larbin User-agent: LARBIN-EXPERIMENTAL efp@gmx.net User-agent: larbin_2.6.3 (larbin2.6.3@unspecified.mail) User-agent: LinkWalker User-agent: linksmanager User-agent: LinksManager.com (http://linksmanager.com/linkchecker.html) User-agent: lmspider User-agent: lmspider (lmspider@scansoft.com) User-agent: Mail Sweeper User-agent: MemoWeb User-agent: MetaGer-LinkChecker User-agent: MicrosoftPrototypeCrawler User-agent: MSNBOT User-agent: msnbot User-agent: MSNBOT/0.1 User-agent: MSNBOT/0.2 User-agent: NaverBot-1.0 (NHN Corp. / +82-2-3011-1954 / nhnbot@naver.com) User-agent: NaverBot-1.0 User-agent: NearSite User-agent: NetAttache User-agent: NPBot User-agent: NPBot-1/2.0 User-agent: obot User-agent: Polybot User-agent: psbot User-agent: TurnitinBot/1.5 User-agent: Radview User-agent: Radview/HttpLoader User-agent: Second Site User-agent: SecondSite User-agent: seekbot User-agent: Sitecheck.internetseer.com User-agent: SiteSnagger User-agent: SpidyBot User-agent: Steeler User-agent: Teleport User-agent: Teleport Pro User-agent: TurnitinBot User-agent: Visual Web User-agent: VisualWeb User-agent: WBI_Client User-agent: WebCompass User-agent: WebCopy User-agent: WebDownloader User-agent: WebGather 3.0 User-agent: WebRetriever User-agent: WebSnake User-agent: WebVCR User-agent: WebWhacker User-agent: WebZIP User-agent: Wget User-agent: Wget/1.6 User-agent: Zeus User-agent: Zao Disallow: / User-agent: Googlebot Disallow: /*.txt$ User-agent: * Disallow: /test/ Disallow: /growlery-images/ Disallow: /private/ Disallow: /logs/ Disallow: /photos/ Disallow: /contact/ Disallow: /stats/ Disallow: /phpMyAdmin/ Disallow: /some-bad-bot/