# www.robotstxt.org/ - how to # http://code.google.com/web/controlcrawlindex/ - how to # https://www.google.com/webmasters/tools/robots-testing-tool? - test your edits User-agent: * Disallow: /assets/ Disallow: /test_* Disallow: /test-* Disallow: /my/mavlink.php Disallow: /_includes/ Disallow: /news/events/2016/ Disallow: /news/events/2017/ Disallow: /news/events/2018/ Disallow: /college-of-arts-and-sciences/news/events/2016 Disallow: /college-of-arts-and-sciences/news/events/2017 Disallow: /college-of-arts-and-sciences/news/events/2018 Disallow: college-of-business-administration/news/events/2016 Disallow: college-of-business-administration/news/events/2017 Disallow: college-of-business-administration/news/events/2018 Disallow: college-of-communication-fine-arts-and-media/news/events/2016 Disallow: college-of-communication-fine-arts-and-media/news/events/2017 Disallow: college-of-communication-fine-arts-and-media/news/events/2018 Disallow: college-of-education/news/events/2016 Disallow: college-of-education/news/events/2017 Disallow: college-of-education/news/events/2018 Disallow: college-of-information-science-and-technology/news/events/2016 Disallow: college-of-information-science-and-technology/news/events/2017 Disallow: college-of-information-science-and-technology/news/events/2018 Disallow: college-of-public-affairs-and-community-service/news/events/2016 Disallow: college-of-public-affairs-and-community-service/news/events/2017 Disallow: college-of-public-affairs-and-community-service/news/events/2018