# For disallowing googlebot to crawl old pages User-agent: Googlebot Disallow: /customerlist.htm Disallow: /legal.htm Disallow: /services.htm # For disallowing other search spiders User-agent: * Disallow: /customerlist.htm Disallow: /legal.htm Disallow: /services.htm