Metadata-Version: 1.0

Name: zc.buildout

Version: 1.5.2

Summary: System for managing development buildouts

Home-page: http://pypi.python.org/pypi/zc.buildout

Author: Jim Fulton

Author-email: jim@zope.com

License: ZPL 2.1

Description: ********

        Buildout

        ********

        .. contents::

        The Buildout project provides support for creating applications,

        especially Python applications.  It provides tools for assembling

        applications from multiple parts, Python or otherwise.  An application

        may actually contain multiple programs, processes, and configuration

        settings.

        The word "buildout" refers to a description of a set of parts and the

        software to create and assemble them.  It is often used informally to

        refer to an installed system based on a buildout definition.  For

        example, if we are creating an application named "Foo", then "the Foo

        buildout" is the collection of configuration and application-specific

        software that allows an instance of the application to be created.  We

        may refer to such an instance of the application informally as "a Foo

        buildout".

        To get a feel for some of the things you might use buildouts for, see

        the `Buildout examples`_.

        To lean more about using buildouts, see `Detailed Documentation`_.

        To see screencasts, talks, useful links and more documentation, visit

        the `Buildout website <http://www.buildout.org>`_.

        Recipes

        *******

        Existing recipes include:

        `zc.recipe.egg <http://pypi.python.org/pypi/zc.recipe.egg>`_

           The egg recipe installes one or more eggs, with their

           dependencies.  It installs their console-script entry points with

           the needed eggs included in their paths.  It is suitable for use with

           a "clean" Python: one without packages installed in site-packages.

        `z3c.recipe.scripts <http://pypi.python.org/pypi/z3c.recipe.scripts>`_

          Like zc.recipe.egg, this recipe builds interpreter scripts and entry

          point scripts based on eggs.  It can be used with a Python that has

          packages installed in site-packages, such as a system Python.  The

          interpreter also has more features than the one offered by

          zc.recipe.egg.

        `zc.recipe.testrunner <http://pypi.python.org/pypi/zc.recipe.testrunner>`_

           The testrunner egg creates a test runner script for one or

           more eggs.

        `zc.recipe.zope3checkout <http://pypi.python.org/pypi/zc.recipe.zope3checkout>`_

           The zope3checkout recipe installs a Zope 3 checkout into a

           buildout.

        `zc.recipe.zope3instance <http://pypi.python.org/pypi/zc.recipe.zope3instance>`_

           The zope3instance recipe sets up a Zope 3 instance.

        `zc.recipe.filestorage <http://pypi.python.org/pypi/zc.recipe.filestorage>`_

           The filestorage recipe sets up a ZODB file storage for use in a

           Zope 3 instance created by the zope3instance recipe.

        Buildout examples

        *****************

        Here are a few examples of what you can do with buildouts.  We'll

        present these as a set of use cases.

        Try out an egg

        ==============

        Sometimes you want to try an egg (or eggs) that someone has released.

        You'd like to get a Python interpreter that lets you try things

        interactively or run sample scripts without having to do path

        manipulations.  If you can and don't mind modifying your Python

        installation, you could use easy_install, otherwise, you could create

        a directory somewhere and create a buildout.cfg file in that directory

        containing::

          [buildout]

          parts = mypython

          [mypython]

          recipe = zc.recipe.egg

          interpreter = mypython

          eggs = theegg

        where theegg is the name of the egg you want to try out.

        Run buildout in this directory.  It will create a bin subdirectory

        that includes a mypython script.  If you run mypython without any

        arguments you'll get an interactive interpreter with the egg in the

        path. If you run it with a script and script arguments, the script

        will run with the egg in its path.  Of course, you can specify as many

        eggs as you want in the eggs option.

        If the egg provides any scripts (console_scripts entry points), those

        will be installed in your bin directory too.

        Work on a package

        =================

        I often work on packages that are managed separately.  They don't have

        scripts to be installed, but I want to be able to run their tests

        using the `zope.testing test runner

        <http://www.python.org/pypi/zope.testing>`_.  In this kind of

        application, the program to be installed is the test runner.  A good

        example of this is `zc.ngi <http://svn.zope.org/zc.ngi/trunk/>`_.

        Here I have a subversion project for the zc.ngi package.  The software

        is in the src directory.  The configuration file is very simple::

          [buildout]

          develop = .

          parts = test

          [test]

          recipe = zc.recipe.testrunner

          eggs = zc.ngi

        I use the develop option to create a develop egg based on the current

        directory.  I request a test script named "test" using the

        zc.recipe.testrunner recipe.  In the section for the test script, I

        specify that I want to run the tests in the zc.ngi package.

        When I check out this project into a new sandbox, I run bootstrap.py

        to get setuptools and zc.buildout and to create bin/buildout.  I run

        bin/buildout, which installs the test script, bin/test, which I can

        then use to run the tests.

        This is probably the most common type of buildout.

        If I need to run a previous version of zc.buildout, I use the

        `--version` option of the bootstrap.py script::

            $ python bootstrap.py --version 1.1.3

        The `zc.buildout project <http://svn.zope.org/zc.buildout/trunk>`_

        is a slightly more complex example of this type of buildout.

        Install egg-based scripts

        =========================

        A variation of the `Try out an egg`_ use case is to install scripts

        into your ~/bin directory (on Unix, of course).  My ~/bin directory is

        a buildout with a configuration file that looks like::

          [buildout]

          parts = foo bar

          bin-directory = .

          [foo]

          ...

        where foo and bar are packages with scripts that I want available.  As

        I need new scripts, I can add additional sections.  The bin-directory

        option specified that scripts should be installed into the current

        directory.

        Multi-program multi-machine systems

        ===================================

        Using an older prototype version of the buildout, we've build a number

        of systems involving multiple programs, databases, and machines.  One

        typical example consists of:

        - Multiple Zope instances

        - Multiple ZEO servers

        - An LDAP server

        - Cache-invalidation and Mail delivery servers

        - Dozens of add-on packages

        - Multiple test runners

        - Multiple deployment modes, including dev, stage, and prod,

          with prod deployment over multiple servers

        Parts installed include:

        - Application software installs, including Zope, ZEO and LDAP

          software

        - Add-on packages

        - Bundles of configuration that define Zope, ZEO and LDAP instances

        - Utility scripts such as test runners, server-control

          scripts, cron jobs.

        Questions and Bug Reporting

        ***************************

        Please send questions and comments to the

        `distutils SIG mailing list <mailto://distutils-sig@python.org>`_.

        Report bugs using the `zc.buildout Launchpad Bug Tracker

        <https://launchpad.net/zc.buildout/+bugs>`_.

        System Python and zc.buildout 1.5

        *********************************

        The 1.5 line of zc.buildout introduced a number of changes.

        Problems

        ========

        As usual, please send questions and comments to the `distutils SIG

        mailing list <mailto://distutils-sig@python.org>`_. Report bugs using

        the `zc.buildout Launchpad Bug Tracker

        <https://launchpad.net/zc.buildout/+bugs>`_.

        If problems are keeping you from your work, here's an easy way to

        revert to the old code temporarily: switch to a custom "emergency"

        bootstrap script, available from

        http://svn.zope.org/repos/main/zc.buildout/branches/1.4/bootstrap/bootstrap.py .

        This customized script will select zc.buildout 1.4.4 by default.

        zc.buildout 1.4.4 will not upgrade itself unless you explicitly specify

        a new version.  It will also prefer older versions of zc.recipe.egg and

        some other common recipes.  If you have trouble with other recipes,

        consider using a standard buildout "versions" section to specify older

        versions of these, as described in the Buildout documentation

        (http://pypi.python.org/pypi/zc.buildout#repeatable-buildouts-controlling-eggs-used).

        Working with a System Python

        ============================

        While there are a number of new features available in zc.buildout 1.5,

        the biggest is that Buildout itself supports usage with a system Python.

        This can work if you follow a couple of simple rules.

        1. Use the new bootstrap.py (available from

           svn://svn.zope.org/repos/main/zc.buildout/trunk/bootstrap/bootstrap.py).

        2. Use buildout recipes that have been upgraded to work with zc.buildout 1.5

           and higher.  Specifically, they should use

           ``zc.buildout.easy_install.sitepackage_safe_scripts`` to generate

           their scripts, if any, rather than ``zc.buildout.easy_install.scripts``.

           See the `Recipes That Support a System Python`_ section below for more

           details on recipes that are available as of this writing, and

           `Updating Recipes to Support a System Python`_ for instructions on

           how to update a recipe.  Note that you should generally only need to

           update recipes that generate scripts.

        You can then use ``include-site-packages = false`` and

        ``exec-sitecustomize = false`` buildout options to eliminate access to

        your Python's site packages and not execute its sitecustomize file, if

        it exists, respectively.

        Alternately, you can use the ``allowed-eggs-from-site-packages`` buildout

        option as a glob-aware whitelist of eggs that may come from site-packages.

        This value defaults to "*", accepting all eggs.

        It's important to note that recipes not upgraded for zc.buildout 1.5.0

        should continue to work--just without internal support for a system Python.

        Using a system Python is inherently fragile.  Using a clean,

        freshly-installed Python without customization in site-packages is more

        robust and repeatable.  See some of the regression tests added to the

        1.5.0 line for the kinds of issues that you can encounter with a system

        Python, and see

        http://pypi.python.org/pypi/z3c.recipe.scripts#including-site-packages-and-sitecustomize

        for more discussion.

        However, using a system Python can be very convenient, and the

        zc.buildout code for this feature has been tested by many users already.

        Moreover, it has automated tests to exercise the problems that have been

        encountered and fixed.  Many people rely on it.

        Recipes That Support a System Python

        ====================================

        zc.recipe.egg continues to generate old-style scripts that are not safe

        for use with a system Python.  This was done for backwards

        compatibility, because it is integral to so many buildouts and used as a

        dependency of so many other recipes.

        If you want to generate new-style scripts that do support system Python

        usage, use z3c.recipe.scripts instead

        (http://pypi.python.org/pypi/z3c.recipe.scripts). z3c.recipe.scripts has

        the same script and interpreter generation options as zc.recipe.egg,

        plus a few more for the new features mentioned above.  In the simplest

        case, you should be able to simply change ``recipe = zc.recipe.egg`` to

        ``recipe = z3c.recipe.scripts`` in the pertinent sections of your

        buildout configuration and your generated scripts will work with a system

        Python.

        Other updated recipes include zc.recipe.testrunner 1.4.0 and

        z3c.recipe.tag 0.4.0.  Others should be updated soon: see their change

        documents for details, or see `Updating Recipes to Support a System

        Python`_ for instructions on how to update recipes yourself.

        Templates for creating Python scripts with the z3c.recipe.filetemplate

        recipe can be easily changed to support a system Python.

        - If you don't care about supporting relative paths, simply using a

          generated interpreter with the eggs you want should be sufficient, as

          it was before. For instance, if the interpreter is named "py", use

          ``#!${buildout:bin-directory/py}`` or ``#!/usr/bin/env

          ${buildout:bin-directory/py}``).

        - If you do care about relative paths,  (``relative-paths = true`` in

          your buildout configuration), then z3c.recipe.scripts does require a

          bit more changes, as is usual for the relative path support in that

          package.  First, use z3c.recipe.scripts to generate a script or

          interpreter with the dependencies you want.  This will create a

          directory in ``parts`` that has a site.py and sitecustomize.py.  Then,

          begin your script as in the snippet below.  The example assumes that

          the z3c.recipe.scripts generated were from a Buildout configuration

          section labeled "scripts": adjust accordingly.

          ::

            #!${buildout:executable} -S

            ${python-relative-path-setup}

            import sys

            sys.path.insert(0, ${scripts:parts-directory|path-repr})

            import site

        Updating Recipes to Support a System Python

        ===========================================

        You should generally only need to update recipes that generate scripts.

        These recipes need to change from using ``zc.buildout.easy_install.scripts``

        to be using ``zc.buildout.easy_install.sitepackage_safe_scripts``.

        The signatures of the two functions are different.  Please compare::

            def scripts(

                reqs, working_set, executable, dest,

                scripts=None,

                extra_paths=(),

                arguments='',

                interpreter=None,

                initialization='',

                relative_paths=False,

                ):

            def sitepackage_safe_scripts(

                dest, working_set, executable, site_py_dest,

                reqs=(),

                scripts=None,

                interpreter=None,

                extra_paths=(),

                initialization='',

                include_site_packages=False,

                exec_sitecustomize=False,

                relative_paths=False,

                script_arguments='',

                script_initialization='',

                ):

        In most cases, the arguments are merely reordered.  The ``reqs``

        argument is no longer required in order to make it easier to generate an

        interpreter alone.  The ``arguments`` argument was renamed to

        ``script_arguments`` to clarify that it did not affect interpreter

        generation.

        The only new required argument is ``site_py_dest``.  It must be the path

        to a directory in which the customized site.py and sitecustomize.py

        files will be written.  A typical generation in a recipe will look like

        this.

        (In the recipe's __init__ method...)

        ::

                self.options = options

                b_options = buildout['buildout']

                options['parts-directory'] = os.path.join(

                    b_options['parts-directory'], self.name)

        (In the recipe's install method...)

        ::

                options = self.options

                generated = []

                if not os.path.exists(options['parts-directory']):

                    os.mkdir(options['parts-directory'])

                    generated.append(options['parts-directory'])

        Then ``options['parts-directory']`` can be used for the ``site_py_dest``

        value.

        If you want to support the other arguments (``include_site_packages``,

        ``exec_sitecustomize``, ``script_initialization``, as well as the

        ``allowed-eggs-from-site-packages`` option),  you might want to look at

        some of the code in

        svn://svn.zope.org/repos/main/zc.buildout/trunk/z3c.recipe.scripts\_/src/z3c/recipe/scripts/scripts.py .

        You might even be able to adopt some of it by subclassing or delegating.

        The Scripts class in that file is the closest to what you might be used

        to from zc.recipe.egg.

        Important note for recipe authors: As of buildout 1.5.2, the code in

        recipes is *always run with the access to the site-packages as

        configured in the buildout section*.

        virtualenv

        ==========

        Using virtualenv (http://pypi.python.org/pypi/virtualenv) with the

        --no-site-packages option already provided a simple way of using a

        system Python.  This is intended to continue to work, and some automated

        tests exist to demonstrate this.

        However, it is only supported to the degree that people have found it to

        work in the past.  The existing Buildout tests for virtualenv are only

        for problems encountered previously.  They are very far from

        comprehensive.

        Using Buildout with a system python has at least three advantages over

        using Buildout in conjunction with virtualenv.  They may or may not be

        pertinent to your desired usage.

        - Unlike ``virtualenv --no-site-packages``, Buildout's support allows you

          to choose to let packages from your system Python be available to your

          software (see ``include-site-packages`` in

          http://pypi.python.org/pypi/z3c.recipe.scripts).

          You can even specify which eggs installed in your system Python can be

          allowed to fulfill some of your packages' dependencies (see

          ``allowed-eggs-from-site-packages`` in

          http://pypi.python.org/pypi/z3c.recipe.scripts).

          At the expense of some repeatability and platform dependency, this

          flexibility means that, for instance, you can rely on

          difficult-to-build eggs like lxml coming from your system Python.

        - Buildout's implementation has a full set of automated tests.

        - An integral Buildout implementation means fewer steps and fewer dependencies

          to work with a system Python.

        Detailed Documentation

        **********************

        Buildouts

        =========

        The word "buildout" refers to a description of a set of parts and the

        software to create and assemble them.  It is often used informally to

        refer to an installed system based on a buildout definition.  For

        example, if we are creating an application named "Foo", then "the Foo

        buildout" is the collection of configuration and application-specific

        software that allows an instance of the application to be created.  We

        may refer to such an instance of the application informally as "a Foo

        buildout".

        This document describes how to define buildouts using buildout

        configuration files and recipes.  There are three ways to set up the

        buildout software and create a buildout instance:

        1. Install the zc.buildout egg with easy_install and use the buildout

           script installed in a Python scripts area.

        2. Use the buildout bootstrap script to create a buildout that

           includes both the setuptools and zc.buildout eggs.  This allows you

           to use the buildout software without modifying a Python install.

           The buildout script is installed into your buildout local scripts

           area.

        3. Use a buildout command from an already installed buildout to

           bootstrap a new buildout.  (See the section on bootstraping later

           in this document.)

        Often, a software project will be managed in a software repository,

        such as a subversion repository, that includes some software source

        directories, buildout configuration files, and a copy of the buildout

        bootstrap script.  To work on the project, one would check out the

        project from the repository and run the bootstrap script which

        installs setuptools and zc.buildout into the checkout as well as any

        parts defined.

        We have a sample buildout that we created using the bootstrap command

        of an existing buildout (method 3 above).  It has the absolute minimum

        information.  We have bin, develop-eggs, eggs and parts directories,

        and a configuration file:

            >>> ls(sample_buildout)

            d  bin

            -  buildout.cfg

            d  develop-eggs

            d  eggs

            d  parts

        The bin directory contains scripts.

            >>> ls(sample_buildout, 'bin')

            -  buildout

            >>> ls(sample_buildout, 'eggs')

            -  setuptools-0.6-py2.4.egg

            -  zc.buildout-1.0-py2.4.egg

        The develop-eggs directory is initially empty:

            >>> ls(sample_buildout, 'develop-eggs')

        The develop-eggs directory holds egg links for software being

        developed in the buildout.  We separate develop-eggs and other eggs to

        allow eggs directories to be shared across multiple buildouts.  For

        example, a common developer technique is to define a common eggs

        directory in their home that all non-develop eggs are stored in.  This

        allows larger buildouts to be set up much more quickly and saves disk

        space.

        The parts directory just contains some helpers for the buildout script

        itself.

            >>> ls(sample_buildout, 'parts')

            d  buildout

        The parts directory provides an area where recipes can install

        part data.  For example, if we built a custom Python, we would

        install it in the part directory.  Part data is stored in a

        sub-directory of the parts directory with the same name as the part.