# Tells Scanning Robots Where They Are And Are Not Welcome # # User-agent: can also specify by name; "*" is for everyone # Disallow: disallow if this matches first part of requested path # # For now disallow all we can modify this as needed to allow certain crawlers. # User-agent: * Disallow: /air Disallow: /aspnet_client Disallow: /awstats Disallow: /cgi_bin Disallow: /cust Disallow: /custdocs Disallow: /docs Disallow: /flash Disallow: /images Disallow: /jac Disallow: /keys Disallow: /log Disallow: /mail Disallow: /oldSite Disallow: /share Disallow: /stats Disallow: /webdav Disallow: /wx Disallow: tulz.shtml