Changes in tags are saved to the data store.
Also, when a task is created, its arbit tags are stored. Issue 696 fixed.
# Directions for web crawlers.# See http://www.robotstxt.org/wc/norobots.html.User-agent: HTTrackUser-agent: pufUser-agent: MSIECrawlerUser-agent: NutchDisallow: /