app/soc/content/robots.txt
author Lennard de Rijk <ljvderijk@gmail.com>
Mon, 10 Aug 2009 15:28:42 -0700
changeset 2744 6f5e303f64b5
parent 73 211a3eeacf27
permissions -rw-r--r--
Removed the grade_choices keyword argument from GradeSurveyTakeForm. This is because this class already assumes very specific things about the grade_choices keyword argument. Therefore it is now a constant inside this class.

# Directions for web crawlers.
# See http://www.robotstxt.org/wc/norobots.html.

User-agent: HTTrack
User-agent: puf
User-agent: MSIECrawler
User-agent: Nutch
Disallow: /