User-agent: * Disallow: /passwordreset Disallow: /activate Disallow: /mobile/passwordreset Disallow: /mobile/activate User-agent: bingbot Crawl-delay: 10 User-agent: * Disallow: /get_weatherdata.asp Disallow: /get_routeviewsdata.asp User-agent: SMTBot (similartech.com/smtbot) Disallow: / User-agent: Mozilla/5.0 (compatible; EasouSpider; +http://www.easou.com/search/spider.html) Disallow: / User-agent: Mozilla/5.0 (compatible; XoviBot/2.0; +http://www.xovibot.net/) Disallow: /