app/soc/content/robots.txt
author Lennard de Rijk <ljvderijk@gmail.com>
Thu, 30 Jul 2009 09:53:22 +0200
changeset 2688 dfe0439a0711
parent 73 211a3eeacf27
permissions -rw-r--r--
Added list of Surveys to the Project's manage page. This list will show wether or not a Survey has been taken and will also feature links to (re)take a Survey and in the near future to see the actual SurveyRecord. Note that the take Survey link is working however Org Admins will receive an access denied message when following this link.

# Directions for web crawlers.
# See http://www.robotstxt.org/wc/norobots.html.

User-agent: HTTrack
User-agent: puf
User-agent: MSIECrawler
User-agent: Nutch
Disallow: /