User-agent: VideoParser Disallow: / User-agent: 008 Disallow: / User-Agent: BLEXBot Disallow: / User-Agent: AhrefsBot Disallow: / User-Agent: Vagabondo Disallow: / User-Agent: SEOkicks-Robot Disallow: / User-Agent: ia_archiver Disallow: / User-Agent: archive.org_bot Disallow: / User-Agent: special_archiver Disallow: / User-Agent: MJ12bot Disallow: / User-Agent: special_archiver Disallow: / User-Agent: heritrix Disallow: / User-Agent: netEstate NE Crawler Disallow: / User-Agent: sistrix Disallow: / User-Agent: WBSearchBot Disallow: / User-Agent: QuerySeekerSpider Disallow: / User-agent: proximic Disallow: / User-agent: SiteExplorer Disallow: / User-agent: SemrushBot Disallow: / User-agent: SemrushBot-SA Disallow: / User-agent: DotBot Disallow: / User-agent: Mail.Ru Disallow: / User-agent: Baiduspider Disallow: / User-agent: MSNBot Disallow: / User-agent: Slurp Disallow: / User-agent: SeznamBot Disallow: / User-agent: spbot Disallow: / User-agent: DeuSu Disallow: /