User-agent: Googlebot Disallow: /prereg/ Disallow: /scrape/ User-agent: MSNBot Disallow: /prereg/ Disallow: /scrape/ User-agent: Slurp Disallow: / #Crawl-delay: 5 #Disallow: /prereg/ #Disallow: /scrape/ User-agent: Teoma Disallow: /prereg/ Disallow: /scrape/ User-agent: * Disallow: /