# General anyMeta robots.txt file User-agent: msnbot Crawl-delay: 120 User-agent: * Crawl-delay: 10 # templates are not handy to be indexed User-agent: * Disallow: /templates/ Disallow: /logon Disallow: /logoff Disallow: /register