User-agent: 008/ Disallow: / User-agent: 008/0.83 Disallow: / User-agent: atraxbot Disallow: / User-agent: atraxbot/0.2 Disallow: / User-agent: atraxbot/0.3 Disallow: / User-agent: Baiduspider Disallow: / User-agent: BecomeBot/3.0 Disallow: / User-agent: boitho.com-dc Disallow: / User-agent: BruinBot Disallow: / User-agent: Canookle/1.1 Disallow: / User-agent: CatchBot Disallow: / User-agent: CatchBot/1.0 Disallow: / User-agent: CazoodleBot Crawler Disallow: / User-agent: CCBot/1.0 Disallow: / User-agent: Charlotte/0.9t Disallow: / User-agent: complex_network_group Disallow: / User-agent: ConveraCrawler Disallow: / User-agent: crawly_2.6.3 Disallow: / User-agent: CyberPatrol Disallow: / User-agent: DBLBot/1.0 Disallow: / User-Agent: DealGates Bot Disallow: / User-Agent: DealGates Bot/1.1 Disallow: / User-Agent: DealGates Bot/1.2 Disallow: / User-Agent: DepSpid Disallow: / User-Agent: discobot Disallow: / User-Agent: discobot/1.0 Disallow: / User-Agent: DotBot/1.0.1 Disallow: / User-agent: e-SocietyRobot Disallow: / User-agent: EmeraldShield.com Web Spider Disallow: / User-agent: EnaBot/1.2 Disallow: / User-agent: Exabot-Images/1.0 Disallow: / User-agent: Exabot-Images Disallow: / User-agent: Exabot/3.0 Disallow: / User-agent: Exabot Disallow: / User-agent: exooba/exooba crawler Disallow: / User-agent: Fast Disallow: / User-agent: FindSeekWebRobot Disallow: / User-agent: flatlandbot/baypup Disallow: / User-agent: FubraSpider Disallow: / User-agent: Gaisbot/3.0+ Disallow: / User-agent: genieBot Disallow: / User-agent: GeonaBot/1.2 Disallow: / User-agent: Gigablast Robot Disallow: / User-agent: Gigabot Disallow: / User-agent: GingerCrawler Disallow: / User-agent: GingerCrawler/1.0 Disallow: / User-agent: Girafabot Disallow: / User-agent: Grub Crawler Disallow: / User-agent: GurujiBot Disallow: / User-agent: GurujiBot/1.0 Disallow: / User-agent: heritrix/1.12.1 Disallow: / User-agent: Holmes Crawler Disallow: / User-agent: ia_archiver Crawl-delay: 300 User-agent: ichiro/3.0 Disallow: / User-agent: ilial Disallow: / User-agent: IRLbot Disallow: / User-agent: Isidorus/2.0 Disallow: / User-agent: itsapic.com_crawler Disallow: / User-agent: JadynAveBot Disallow: / User-agent: Jyxobot/1 Disallow: / User-agent: KaloogaBot Disallow: / User-agent: KS Crawler Disallow: / User-agent: LexxeBot Disallow: / User-agent: LexxeBot/1.0 Disallow: / User-agent: librabot/1.0 Disallow: / User-agent: librabot Disallow: / User-agent: LinksManager.com_bot Disallow: / User-agent: Mail.Ru/1.0 Disallow: / User-agent: MJ12bot Disallow: / User-agent: MLBot Disallow: / User-agent: MSR-ISRCCrawler Disallow: / User-agent: multicrawler Disallow: / User-agent: my-heritrix-crawler Disallow: / User-agent: NaverBot/1.0 Disallow: / User-agent: NextGenSearchBot Disallow: / User-agent: Nikita The Spider Disallow: / User-agent: NimbleCrawler Disallow: / User-agent: Nutch Disallow: / User-agent: Nutch Spider/Nutch-1.0-dev Disallow: / User-agent: Nutch-1.0-dev Disallow: / User-agent: obot Disallow: / User-agent: OmniExplorer_Bot/4.32 Disallow: / User-agent: OOZBOT Disallow: / User-agent: OOZBOT/0.20 Disallow: / User-agent: PAGEBULLBOT Disallow: / User-agent: panscient.com Disallow: / User-agent: psbot Disallow: / User-agent: REBI-Shoveler v0.1 Disallow: / User-agent: RufusBot Disallow: / User-agent: Runnk RSS aggregator Disallow: / User-agent: ScoutJet Disallow: / User-agent: ShablastBot/1.0 Disallow: / User-agent: Sirketcebot/v.01 Disallow: / User-agent: Slurp Crawl-delay: 300 User-agent: Snapbot/1.0 Disallow: / User-agent: Sosospider+ Disallow: / User-agent: Speedy Spider Disallow: / User-agent: Spock Crawler Disallow: / User-agent: SpokeSpider/1.0 Disallow: / User-agent: Sproose Disallow: / User-agent: StackRambler/2.0 Disallow: / User-agent: SurveyBot/2.3 Disallow: / User-agent: Tagoobot Disallow: / User-agent: Tagoobot/3.0 Disallow: / User-agent: Teoma Disallow: / User-agent: Twiceler www.cuill.com/robots.html Disallow: / User-agent: TurnitinBot Disallow: / User-agent: Visbot Disallow: / User-agent: Visbot/1.1 Disallow: / User-agent: VoilaBot BETA 1.2 Disallow: / User-agent: voyager-hc/1.0 Disallow: / User-agent: WebAlta Disallow: / User-agent: WebAlta Crawler/2.0 Disallow: / User-agent: WebDataCentreBot Disallow: / User-agent: WebDataCentreBot/1.0 Disallow: / User-agent: Wget Disallow: / User-agent: WinHTTP Robot/1.0 Disallow: / User-agent: WiseNut Robot Disallow: / User-agent: woriobot Disallow: / User-agent: Yandex Disallow: / User-agent: Yanga WorldSearch Bot v1.1/beta Disallow: / User-agent: YebolBot Disallow: / User-agent: YesupBot/1.0 Disallow: / User-agent: YodaoBot/1.0 Disallow: / User-agent: YoudaoBot Disallow: / User-agent: YoudaoBot/1.0 Disallow: / User-agent: Zeusbot Disallow: / User-agent: zibber-v0.1 Disallow: / User-agent: zibber Disallow: / User-agent: * Disallow: /cgi-bin/ Disallow: /logs/ Disallow: /404page.html Disallow: /500page.html Disallow: /accorderx.html Disallow: /config.php Disallow: /config.new.php Disallow: /headers/ Disallow: /00-styles.css Disallow: /admin/ Disallow: /includes/ Disallow: /lang/ Disallow: /lightbox.html Disallow: /lightbox.php Disallow: /search.php Disallow: /search.htm Disallow: /licence.html Disallow: /contactus.html Disallow: /transfers/ Disallow: /new.php Disallow: /uploads/ Disallow: /uploads0/ Disallow: /uploads1/ Disallow: /uploads2/ Disallow: /uploads3/ Disallow: /uploads4/ Disallow: /uploads5/ Disallow: /uploads6/ Disallow: /uploads7/ Disallow: /uploads8/ Disallow: /uploads9/ # updated November 6 2009 (version 79)