User-agent: * Disallow: /cgi-bin/ # updated 2005-09-07 (disallow folders)