Mercurial Mercurial > py_tasks_melange / file revision
summary | shortlog | changelog | graph | tags | bookmarks | branches | files | changeset | file | latest | revisions | annotate | diff | comparison | raw | help
Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
app/soc/content/robots.txt
author Sverre Rabbelier <srabbelier@gmail.com>
Sat, 12 Sep 2009 00:59:58 +0200
changeset 2899 691735924efb
parent 73 211a3eeacf27
permissions -rw-r--r--
Fix broken maintenance mode Maintenance mode has been broken since r7fe19ebfe0d3 (Thu Jun 4) "Display caught errors on the page itself", fix it by passing the site entity, rather than the site module to isActivePeriod.

# Directions for web crawlers.
# See http://www.robotstxt.org/wc/norobots.html.

User-agent: HTTrack
User-agent: puf
User-agent: MSIECrawler
User-agent: Nutch
Disallow: /
py_tasks_melange