app/soc/content/robots.txt
author Pawel Solyga <Pawel.Solyga@gmail.com>
Fri, 17 Oct 2008 13:57:54 +0000
changeset 369 2955eff2bf94
parent 73 211a3eeacf27
permissions -rw-r--r--
Replace GROUP_TYPE_PLURAL and GROUP_TYPE_SHORT constants in Group models with more generic TYPE_NAME, TYPE_NAME_SHORT, TYPE_NAME_PLURAL names and apply changes in affected files. Use ugettext_lazy for TYPE_NAME and TYPE_NAME_PLURAL constants. This approach is similar to the upcoming refactor of views. Patch by: Pawel Solyga Review by: to-be-reviewed

# Directions for web crawlers.
# See http://www.robotstxt.org/wc/norobots.html.

User-agent: HTTrack
User-agent: puf
User-agent: MSIECrawler
User-agent: Nutch
Disallow: /