1 # Directions for web crawlers.

     2 # See http://www.robotstxt.org/wc/norobots.html.

     3 

     4 User-agent: HTTrack

     5 User-agent: puf

     6 User-agent: MSIECrawler

     7 User-agent: Nutch

     8 Disallow: /