User-agent: * Disallow: / # but allow only important bots User-agent: Googlebot User-agent: Googlebot-Image User-agent: Mediapartners-Google User-agent: msnbot User-agent: msnbot-media User-agent: Slurp User-agent: Yahoo-Blogs User-agent: Yahoo-MMCrawler # Directories Disallow: /App_Browsers/ Disallow: /App_Code/ Disallow: /App_Data/ Disallow: /App_GlobalResources/ Disallow: /App_WebReferences/ Disallow: /aspnet_client/ Disallow: /AssetManagement/ Disallow: /bin/ Disallow: /configs/ Disallow: /controls/ Disallow: /ErrorPages/ Disallow: /logs/ Disallow: /MasterPages/ Disallow: /members/Trainers/te350/ Disallow: /My Project/ Disallow: /nomination/ Disallow: /nxtbook/ Disallow: /old/ Disallow: /PrivateAssets/ Disallow: /Reports/ Disallow: /regalert/ Disallow: /Service References/ Disallow: /Stream/ Disallow: /tabs/ Disallow: /temp/ Disallow: /test/ Disallow: /web_reports/ Disallow: /webservices/ Disallow: /widgets/ Disallow: /wifi/ Disallow: /workarea/ Disallow: /ws/