User-agent: * Disallow: /scripts/ Disallow: /secure/ Disallow: /sphider/ Disallow: /pdf/