app/soc/content/robots.txt
author Lennard de Rijk <ljvderijk@gmail.com>
Wed, 08 Jul 2009 15:44:54 +0200
changeset 2573 f09f317769c4
parent 73 211a3eeacf27
permissions -rw-r--r--
Changed the StudentProject model to handle a dynamic amount of GradeRecords. This allows for a dynamic amount of moments where a StudentProject can be evaluated. Also included in this patch are changes to the status property that will help to enable PA's to withdraw StudentProjects from the program without the need for a developer to step in.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
73
211a3eeacf27 Created robots.txtx and simple Melange favicon.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
     1
# Directions for web crawlers.
211a3eeacf27 Created robots.txtx and simple Melange favicon.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
     2
# See http://www.robotstxt.org/wc/norobots.html.
211a3eeacf27 Created robots.txtx and simple Melange favicon.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
     3
211a3eeacf27 Created robots.txtx and simple Melange favicon.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
     4
User-agent: HTTrack
211a3eeacf27 Created robots.txtx and simple Melange favicon.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
     5
User-agent: puf
211a3eeacf27 Created robots.txtx and simple Melange favicon.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
     6
User-agent: MSIECrawler
211a3eeacf27 Created robots.txtx and simple Melange favicon.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
     7
User-agent: Nutch
211a3eeacf27 Created robots.txtx and simple Melange favicon.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
     8
Disallow: /