bots, scram User-agent: gsa-crawler Disallow: User-agent: htdig Disallow: User-agent: * Disallow: /cgi-bin Disallow: /analog Disallow: /stats