app/soc/content/robots.txt
author Lennard de Rijk <ljvderijk@gmail.com>
Mon, 13 Jul 2009 21:56:01 +0200
changeset 2634 6e71545533b6
parent 73 211a3eeacf27
permissions -rw-r--r--
Added failed_evaluations property to StudentProject model. This is used to store GradingRecords which state a fail for this project. It's a list because manual changes in GradingRecords might force multiple GradingRecords to state a fail. Also it needs to be clear which GradingRecords have been processed and with which grade_decision.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
73
211a3eeacf27 Created robots.txtx and simple Melange favicon.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
     1
# Directions for web crawlers.
211a3eeacf27 Created robots.txtx and simple Melange favicon.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
     2
# See http://www.robotstxt.org/wc/norobots.html.
211a3eeacf27 Created robots.txtx and simple Melange favicon.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
     3
211a3eeacf27 Created robots.txtx and simple Melange favicon.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
     4
User-agent: HTTrack
211a3eeacf27 Created robots.txtx and simple Melange favicon.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
     5
User-agent: puf
211a3eeacf27 Created robots.txtx and simple Melange favicon.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
     6
User-agent: MSIECrawler
211a3eeacf27 Created robots.txtx and simple Melange favicon.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
     7
User-agent: Nutch
211a3eeacf27 Created robots.txtx and simple Melange favicon.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
     8
Disallow: /