User-agent: Baiduspider Disallow: / User-agent: BecomeBot/3.0 Disallow: / User-agent: boitho.com-dc Disallow: / User-agent: BruinBot Disallow: / User-agent: CatchBot Disallow: / User-agent: CatchBot/1.0 Disallow: / User-agent: CazoodleBot Crawler Disallow: / User-agent: ConveraCrawler Disallow: / User-agent: CyberPatrol Disallow: / User-Agent: DealGates Bot Disallow: / User-Agent: DealGates Bot/1.1 Disallow: / User-Agent: DealGates Bot/1.2 Disallow: / User-Agent: discobot Disallow: / User-Agent: discobot/1.0 Disallow: / User-agent: Exabot-Images/1.0 Disallow: / User-agent: Exabot-Images Disallow: / User-agent: Exabot/3.0 Disallow: / User-agent: Exabot Disallow: / User-agent: Fast Disallow: / User-agent: FindSeekWebRobot Disallow: / User-agent: Gigablast Robot Disallow: / User-agent: Gigabot Disallow: / User-agent: GingerCrawler Disallow: / User-agent: GingerCrawler/1.0 Disallow: / User-agent: GurujiBot Disallow: / User-agent: GurujiBot/1.0 Disallow: / User-agent: heritrix/1.12.1 Disallow: / User-agent: Holmes Crawler Disallow: / User-agent: ia_archiver Crawl-delay: 300 User-agent: ichiro/3.0 Disallow: / User-agent: Jyxobot/1 Disallow: / User-agent: KaloogaBot Disallow: / User-agent: librabot/1.0 Disallow: / User-agent: librabot Disallow: / User-agent: LinksManager.com_bot Disallow: / User-agent: Mail.Ru/1.0 Disallow: / User-agent: MJ12bot Disallow: / User-agent: MSR-ISRCCrawler Disallow: / User-agent: my-heritrix-crawler Disallow: / User-agent: NaverBot/1.0 Disallow: / User-agent: NimbleCrawler Disallow: / User-agent: Nutch Disallow: / User-agent: Nutch Spider/Nutch-1.0-dev Disallow: / User-agent: Nutch-1.0-dev Disallow: / User-agent: OmniExplorer_Bot/4.32 Disallow: / User-agent: panscient.com Disallow: / User-agent: Slurp Crawl-delay: 300 User-agent: Snapbot/1.0 Disallow: / User-agent: Sosospider+ Disallow: / User-agent: Speedy Spider Disallow: / User-agent: StackRambler/2.0 Disallow: / User-agent: SurveyBot/2.3 Disallow: / User-agent: Twiceler www.cuill.com/robots.html Disallow: / User-agent: TurnitinBot Disallow: / User-agent: Visbot Disallow: / User-agent: Visbot/1.1 Disallow: / User-agent: VoilaBot BETA 1.2 Disallow: / User-agent: voyager-hc/1.0 Disallow: / User-agent: WebAlta Disallow: / User-agent: WebAlta Crawler/2.0 Disallow: / User-agent: WebDataCentreBot Disallow: / User-agent: WebDataCentreBot/1.0 Disallow: / User-agent: WinHTTP Robot/1.0 Disallow: / User-agent: Yandex Disallow: / User-agent: Yanga WorldSearch Bot v1.1/beta Disallow: / User-agent: YoudaoBot Disallow: / User-agent: YoudaoBot/1.0 Disallow: / User-agent: zibber-v0.1(www.zibb.com/crawler/) Disallow: / User-agent: * Disallow: /cgi-bin/ Disallow: /logs/ Disallow: /404page.html Disallow: /500page.html Disallow: /accorderx.html Disallow: /cat1* Disallow: /cat2* Disallow: /cat3* Disallow: /cat4* Disallow: /cat5* Disallow: /cat6* Disallow: /cat7* Disallow: /cat8* Disallow: /cat9* Disallow: /*?action= Disallow: /*?image_id= Disallow: /config.php Disallow: /config.new.php Disallow: /headers/ Disallow: /00-styles.css Disallow: /admin/ Disallow: /includes/ Disallow: /lang/ Disallow: /lightbox.html Disallow: /lightbox.htm Disallow: /lightbox.php Disallow: /search.php Disallow: /search.htm Disallow: /licence.html Disallow: /contactus.html Disallow: /transfers/ Disallow: /new.php Disallow: /uploads/ Disallow: /uploads0/ Disallow: /uploads1/ Disallow: /uploads2/ Disallow: /uploads3/ Disallow: /uploads4/ Disallow: /uploads5/ Disallow: /uploads6/ Disallow: /uploads7/ Disallow: /uploads8/ Disallow: /uploads9/ # updated November 18 2009 (version 84)