Automatic Buildout Updates
==========================

When a buildout is run, one of the first steps performed is to check
for updates to either zc.buildout or setuptools.  To demonstrate this,
we've created some "new releases" of buildout and setuptools in a
new_releases folder:

    >>> ls(new_releases)
    d  setuptools
    -  setuptools-99.99-py2.4.egg
    d  zc.buildout
    -  zc.buildout-100.0b1-pyN.N.egg
    -  zc.buildout-99.99-py2.4.egg

Let's update the sample buildout.cfg to look in this area:

    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... find-links = %(new_releases)s
    ... index = %(new_releases)s
    ... parts = show-versions
    ... develop = showversions
    ...
    ... [show-versions]
    ... recipe = showversions
    ... """ % dict(new_releases=new_releases))

We'll also include a recipe that echos the versions of setuptools and
zc.buildout used:

    >>> mkdir(sample_buildout, 'showversions')

    >>> write(sample_buildout, 'showversions', 'showversions.py',
    ... """
    ... import pkg_resources
    ...
    ... class Recipe:
    ...
    ...     def __init__(self, buildout, name, options):
    ...         pass
    ...
    ...     def install(self):
    ...         for project in 'zc.buildout', 'setuptools':
    ...             req = pkg_resources.Requirement.parse(project)
    ...             print project, pkg_resources.working_set.find(req).version
    ...         return ()
    ...     update = install
    ... """)


    >>> write(sample_buildout, 'showversions', 'setup.py',
    ... """
    ... from setuptools import setup
    ...
    ... setup(
    ...     name = "showversions",
    ...     entry_points = {'zc.buildout': ['default = showversions:Recipe']},
    ...     )
    ... """)


Now if we run the buildout, the buildout will upgrade itself to the
new versions found in new releases:

    >>> print system(buildout),
    Getting distribution for 'zc.buildout'.
    Got zc.buildout 99.99.
    Getting distribution for 'setuptools'.
    Got setuptools 99.99.
    Upgraded:
      zc.buildout version 99.99,
      setuptools version 99.99;
    restarting.
    Generated script '/sample-buildout/bin/buildout'.
    Develop: '/sample-buildout/showversions'
    Installing show-versions.
    zc.buildout 99.99
    setuptools 99.99

Notice that, even though we have a newer beta version of zc.buildout
available, the final "99.99" was selected.  If you want to get non-final
versions, specify a specific version in your buildout's versions
section, you typically want to use the --accept-buildout-test-releases
option to the bootstrap script, which internally uses the
``accept-buildout-test-releases = true`` discussed below.

Our buildout script's site.py has been updated to use the new eggs:

    >>> cat(sample_buildout, 'parts', 'buildout', 'site.py')
    ... # doctest: +NORMALIZE_WHITESPACE +ELLIPSIS
    "...
    def addsitepackages(known_paths):
        """Add site packages, as determined by zc.buildout.
    <BLANKLINE>
        See original_addsitepackages, below, for the original version."""
        setuptools_path = '/sample-buildout/eggs/setuptools-99.99-pyN.N.egg'
        sys.path.append(setuptools_path)
        known_paths.add(os.path.normcase(setuptools_path))
        import pkg_resources
        buildout_paths = [
            '/sample-buildout/eggs/zc.buildout-99.99-pyN.N.egg',
            '/sample-buildout/eggs/setuptools-99.99-pyN.N.egg'
            ]
        for path in buildout_paths:
            sitedir, sitedircase = makepath(path)
            if not sitedircase in known_paths and os.path.exists(sitedir):
                sys.path.append(sitedir)
                known_paths.add(sitedircase)
                pkg_resources.working_set.add_entry(sitedir)
        sys.__egginsert = len(buildout_paths) # Support setuptools.
        original_paths = [
            ...
            ]
        for path in original_paths:
            if path == setuptools_path or path not in known_paths:
                addsitedir(path, known_paths)
        return known_paths
    ...

Now, let's recreate the sample buildout. If we specify constraints on
the versions of zc.buildout and setuptools (or distribute) to use,
running the buildout will install earlier versions of these packages:

    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... find-links = %(new_releases)s
    ... index = %(new_releases)s
    ... parts = show-versions
    ... develop = showversions
    ... zc.buildout-version = < 99
    ... setuptools-version = < 99
    ... distribute-version = < 99
    ...
    ... [show-versions]
    ... recipe = showversions
    ... """ % dict(new_releases=new_releases))

Now we can see that we actually "upgrade" to an earlier version.

    >>> print system(buildout),
    Upgraded:
      zc.buildout version 1.0.0,
      setuptools version 0.6;
    restarting.
    Develop: '/sample-buildout/showversions'
    Updating show-versions.
    zc.buildout 1.0.0
    setuptools 0.6

There are a number of cases, described below, in which the updates
don't happen.

We won't upgrade in offline mode:

    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... find-links = %(new_releases)s
    ... index = %(new_releases)s
    ... parts = show-versions
    ... develop = showversions
    ...
    ... [show-versions]
    ... recipe = showversions
    ... """ % dict(new_releases=new_releases))

    >>> print system(buildout+' -o'),
    Develop: '/sample-buildout/showversions'
    Updating show-versions.
    zc.buildout 1.0.0
    setuptools 0.6

Or in non-newest mode:

    >>> print system(buildout+' -N'),
    Develop: '/sample-buildout/showversions'
    Updating show-versions.
    zc.buildout 1.0.0
    setuptools 0.6

We also won't upgrade if the buildout script being run isn't in the
buildout's bin directory.  To see this we'll create a new buildout
directory:

    >>> sample_buildout2 = tmpdir('sample_buildout2')
    >>> write(sample_buildout2, 'buildout.cfg',
    ... """
    ... [buildout]
    ... find-links = %(new_releases)s
    ... index = %(new_releases)s
    ... parts =
    ... """ % dict(new_releases=new_releases))

    >>> cd(sample_buildout2)
    >>> print system(buildout),
    Creating directory '/sample_buildout2/bin'.
    Creating directory '/sample_buildout2/parts'.
    Creating directory '/sample_buildout2/eggs'.
    Creating directory '/sample_buildout2/develop-eggs'.
    Getting distribution for 'zc.buildout'.
    Got zc.buildout 99.99.
    Getting distribution for 'setuptools'.
    Got setuptools 99.99.
    Not upgrading because not running a local buildout command.

    >>> ls('bin')

As mentioned above, the ``accept-buildout-test-releases = true`` means that
newer non-final versions of these dependencies are preferred.  Typically
users are not expected to actually manipulate this value.  Instead, the
bootstrap script creates a buildout buildout script that passes in the
value as a command line override. This then results in the buildout
script being rewritten to remember the decision.

We'll mimic this by passing the argument actually in the command line.

    >>> cd(sample_buildout)
    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... find-links = %(new_releases)s
    ... index = %(new_releases)s
    ... parts = show-versions
    ... develop = showversions
    ...
    ... [show-versions]
    ... recipe = showversions
    ... """ % dict(new_releases=new_releases))

    >>> print system(buildout +
    ...              ' buildout:accept-buildout-test-releases=true'),
    ... # doctest: +NORMALIZE_WHITESPACE
    Getting distribution for 'zc.buildout'.
    Got zc.buildout 100.0b1.
    Upgraded:
      zc.buildout version 100.0b1,
      setuptools version 99.99;
    restarting.
    Generated script '/sample-buildout/bin/buildout'.
    NOTE: Accepting early releases of build system packages.  Rerun bootstrap
          without --accept-buildout-test-releases (-t) to return to default
          behavior.
    Develop: '/sample-buildout/showversions'
    Updating show-versions.
    zc.buildout 100.0b1
    setuptools 99.99

The buildout script shows the change.

    >>> buildout_script = join(sample_buildout, 'bin', 'buildout')
    >>> import sys
    >>> if sys.platform.startswith('win'):
    ...     buildout_script += '-script.py'
    >>> print open(buildout_script).read() # doctest: +ELLIPSIS
    #...
    sys.argv.insert(1, 'buildout:accept-buildout-test-releases=true')
    print ('NOTE: Accepting early releases of build system packages.  Rerun '
           'bootstrap without --accept-buildout-test-releases (-t) to return to '
           'default behavior.')
    ...