eggs/zc.buildout-1.5.2-py2.6.egg/zc/buildout/buildout.txt
changeset 69 c6bca38c1cbf
equal deleted inserted replaced
68:5ff1fc726848 69:c6bca38c1cbf
       
     1 Buildouts
       
     2 =========
       
     3 
       
     4 The word "buildout" refers to a description of a set of parts and the
       
     5 software to create and assemble them.  It is often used informally to
       
     6 refer to an installed system based on a buildout definition.  For
       
     7 example, if we are creating an application named "Foo", then "the Foo
       
     8 buildout" is the collection of configuration and application-specific
       
     9 software that allows an instance of the application to be created.  We
       
    10 may refer to such an instance of the application informally as "a Foo
       
    11 buildout".
       
    12 
       
    13 This document describes how to define buildouts using buildout
       
    14 configuration files and recipes.  There are three ways to set up the
       
    15 buildout software and create a buildout instance:
       
    16 
       
    17 1. Install the zc.buildout egg with easy_install and use the buildout
       
    18    script installed in a Python scripts area.
       
    19 
       
    20 2. Use the buildout bootstrap script to create a buildout that
       
    21    includes both the setuptools and zc.buildout eggs.  This allows you
       
    22    to use the buildout software without modifying a Python install.
       
    23    The buildout script is installed into your buildout local scripts
       
    24    area.
       
    25 
       
    26 3. Use a buildout command from an already installed buildout to
       
    27    bootstrap a new buildout.  (See the section on bootstraping later
       
    28    in this document.)
       
    29 
       
    30 Often, a software project will be managed in a software repository,
       
    31 such as a subversion repository, that includes some software source
       
    32 directories, buildout configuration files, and a copy of the buildout
       
    33 bootstrap script.  To work on the project, one would check out the
       
    34 project from the repository and run the bootstrap script which
       
    35 installs setuptools and zc.buildout into the checkout as well as any
       
    36 parts defined.
       
    37 
       
    38 We have a sample buildout that we created using the bootstrap command
       
    39 of an existing buildout (method 3 above).  It has the absolute minimum
       
    40 information.  We have bin, develop-eggs, eggs and parts directories,
       
    41 and a configuration file:
       
    42 
       
    43     >>> ls(sample_buildout)
       
    44     d  bin
       
    45     -  buildout.cfg
       
    46     d  develop-eggs
       
    47     d  eggs
       
    48     d  parts
       
    49 
       
    50 The bin directory contains scripts.
       
    51 
       
    52     >>> ls(sample_buildout, 'bin')
       
    53     -  buildout
       
    54 
       
    55     >>> ls(sample_buildout, 'eggs')
       
    56     -  setuptools-0.6-py2.4.egg
       
    57     -  zc.buildout-1.0-py2.4.egg
       
    58 
       
    59 The develop-eggs directory is initially empty:
       
    60 
       
    61     >>> ls(sample_buildout, 'develop-eggs')
       
    62 
       
    63 The develop-eggs directory holds egg links for software being
       
    64 developed in the buildout.  We separate develop-eggs and other eggs to
       
    65 allow eggs directories to be shared across multiple buildouts.  For
       
    66 example, a common developer technique is to define a common eggs
       
    67 directory in their home that all non-develop eggs are stored in.  This
       
    68 allows larger buildouts to be set up much more quickly and saves disk
       
    69 space.
       
    70 
       
    71 The parts directory just contains some helpers for the buildout script
       
    72 itself.
       
    73 
       
    74     >>> ls(sample_buildout, 'parts')
       
    75     d  buildout
       
    76 
       
    77 The parts directory provides an area where recipes can install
       
    78 part data.  For example, if we built a custom Python, we would
       
    79 install it in the part directory.  Part data is stored in a
       
    80 sub-directory of the parts directory with the same name as the part.
       
    81 
       
    82 Buildouts are defined using configuration files.  These are in the
       
    83 format defined by the Python ConfigParser module, with extensions
       
    84 that we'll describe later.  By default, when a buildout is run, it
       
    85 looks for the file buildout.cfg in the directory where the buildout is
       
    86 run.
       
    87 
       
    88 The minimal configuration file has a buildout section that defines no
       
    89 parts:
       
    90 
       
    91     >>> cat(sample_buildout, 'buildout.cfg')
       
    92     [buildout]
       
    93     parts =
       
    94 
       
    95 A part is simply something to be created by a buildout.  It can be
       
    96 almost anything, such as a Python package, a program, a directory, or
       
    97 even a configuration file.
       
    98 
       
    99 Recipes
       
   100 -------
       
   101 
       
   102 A part is created by a recipe.  Recipes are always installed as Python
       
   103 eggs. They can be downloaded from a package server, such as the
       
   104 Python Package Index, or they can be developed as part of a project
       
   105 using a "develop" egg.
       
   106 
       
   107 A develop egg is a special kind of egg that gets installed as an "egg
       
   108 link" that contains the name of a source directory.  Develop eggs
       
   109 don't have to be packaged for distribution to be used and can be
       
   110 modified in place, which is especially useful while they are being
       
   111 developed.
       
   112 
       
   113 Let's create a recipe as part of the sample project.  We'll create a
       
   114 recipe for creating directories.  First, we'll create a recipes source
       
   115 directory for our local recipes:
       
   116 
       
   117     >>> mkdir(sample_buildout, 'recipes')
       
   118 
       
   119 and then we'll create a source file for our mkdir recipe:
       
   120 
       
   121     >>> write(sample_buildout, 'recipes', 'mkdir.py',
       
   122     ... """
       
   123     ... import logging, os, zc.buildout
       
   124     ...
       
   125     ... class Mkdir:
       
   126     ...
       
   127     ...     def __init__(self, buildout, name, options):
       
   128     ...         self.name, self.options = name, options
       
   129     ...         options['path'] = os.path.join(
       
   130     ...                               buildout['buildout']['directory'],
       
   131     ...                               options['path'],
       
   132     ...                               )
       
   133     ...         if not os.path.isdir(os.path.dirname(options['path'])):
       
   134     ...             logging.getLogger(self.name).error(
       
   135     ...                 'Cannot create %s. %s is not a directory.',
       
   136     ...                 options['path'], os.path.dirname(options['path']))
       
   137     ...             raise zc.buildout.UserError('Invalid Path')
       
   138     ...
       
   139     ...
       
   140     ...     def install(self):
       
   141     ...         path = self.options['path']
       
   142     ...         logging.getLogger(self.name).info(
       
   143     ...             'Creating directory %s', os.path.basename(path))
       
   144     ...         os.mkdir(path)
       
   145     ...         return path
       
   146     ...
       
   147     ...     def update(self):
       
   148     ...         pass
       
   149     ... """)
       
   150 
       
   151 Currently, recipes must define 3 methods [#future_recipe_methods]_:
       
   152 
       
   153 - a constructor,
       
   154 
       
   155 - an install method, and
       
   156 
       
   157 - an update method.
       
   158 
       
   159 The constructor is responsible for updating a parts options to reflect
       
   160 data read from other sections.  The buildout system keeps track of
       
   161 whether a part specification has changed.  A part specification has
       
   162 changed if it's options, after adjusting for data read from other
       
   163 sections, has changed, or if the recipe has changed.  Only the options
       
   164 for the part are considered.  If data are read from other sections,
       
   165 then that information has to be reflected in the parts options.  In
       
   166 the Mkdir example, the given path is interpreted relative to the
       
   167 buildout directory, and data from the buildout directory is read.  The
       
   168 path option is updated to reflect this.  If the directory option was
       
   169 changed in the buildout sections, we would know to update parts
       
   170 created using the mkdir recipe using relative path names.
       
   171 
       
   172 When buildout is run, it saves configuration data for installed parts
       
   173 in a file named ".installed.cfg".  In subsequent runs, it compares
       
   174 part-configuration data stored in the .installed.cfg file and the
       
   175 part-configuration data loaded from the configuration files as
       
   176 modified by recipe constructors to decide if the configuration of a
       
   177 part has changed. If the configuration has changed, or if the recipe
       
   178 has changed, then the part is uninstalled and reinstalled.  The
       
   179 buildout only looks at the part's options, so any data used to
       
   180 configure the part needs to be reflected in the part's options.  It is
       
   181 the job of a recipe constructor to make sure that the options include
       
   182 all relevant data.
       
   183 
       
   184 Of course, parts are also uninstalled if they are no-longer used.
       
   185 
       
   186 The recipe defines a constructor that takes a buildout object, a part
       
   187 name, and an options dictionary. It saves them in instance attributes.
       
   188 If the path is relative, we'll interpret it as relative to the
       
   189 buildout directory.  The buildout object passed in is a mapping from
       
   190 section name to a mapping of options for that section. The buildout
       
   191 directory is available as the directory option of the buildout
       
   192 section.  We normalize the path and save it back into the options
       
   193 directory.
       
   194 
       
   195 The install method is responsible for creating the part.  In this
       
   196 case, we need the path of the directory to create.  We'll use a path
       
   197 option from our options dictionary.  The install method logs what it's
       
   198 doing using the Python logging call.  We return the path that we
       
   199 installed.  If the part is uninstalled or reinstalled, then the path
       
   200 returned will be removed by the buildout machinery.  A recipe install
       
   201 method is expected to return a string, or an iterable of strings
       
   202 containing paths to be removed if a part is uninstalled.  For most
       
   203 recipes, this is all of the uninstall support needed. For more complex
       
   204 uninstallation scenarios use `Uninstall recipes`_.
       
   205 
       
   206 The update method is responsible for updating an already installed
       
   207 part.  An empty method is often provided, as in this example, if parts
       
   208 can't be updated.  An update method can return None, a string, or an
       
   209 iterable of strings.  If a string or iterable of strings is returned,
       
   210 then the saved list of paths to be uninstalled is updated with the new
       
   211 information by adding any new files returned by the update method.
       
   212 
       
   213 We need to provide packaging information so that our recipe can be
       
   214 installed as a develop egg. The minimum information we need to specify
       
   215 [#packaging_info]_ is a name.  For recipes, we also need to define the
       
   216 names of the recipe classes as entry points.  Packaging information is
       
   217 provided via a setup.py script:
       
   218 
       
   219     >>> write(sample_buildout, 'recipes', 'setup.py',
       
   220     ... """
       
   221     ... from setuptools import setup
       
   222     ...
       
   223     ... setup(
       
   224     ...     name = "recipes",
       
   225     ...     entry_points = {'zc.buildout': ['mkdir = mkdir:Mkdir']},
       
   226     ...     )
       
   227     ... """)
       
   228 
       
   229 Our setup script defines an entry point. Entry points provide
       
   230 a way for an egg to define the services it provides.  Here we've said
       
   231 that we define a zc.buildout entry point named mkdir.  Recipe
       
   232 classes must be exposed as entry points in the zc.buildout group.  we
       
   233 give entry points names within the group.
       
   234 
       
   235 We also need a README.txt for our recipes to avoid an annoying warning
       
   236 from distutils, on which setuptools and zc.buildout are based:
       
   237 
       
   238     >>> write(sample_buildout, 'recipes', 'README.txt', " ")
       
   239 
       
   240 Now let's update our buildout.cfg:
       
   241 
       
   242     >>> write(sample_buildout, 'buildout.cfg',
       
   243     ... """
       
   244     ... [buildout]
       
   245     ... develop = recipes
       
   246     ... parts = data-dir
       
   247     ...
       
   248     ... [data-dir]
       
   249     ... recipe = recipes:mkdir
       
   250     ... path = mystuff
       
   251     ... """)
       
   252 
       
   253 Let's go through the changes one by one::
       
   254 
       
   255     develop = recipes
       
   256 
       
   257 This tells the buildout to install a development egg for our recipes.
       
   258 Any number of paths can be listed.  The paths can be relative or
       
   259 absolute.  If relative, they are treated as relative to the buildout
       
   260 directory.  They can be directory or file paths.  If a file path is
       
   261 given, it should point to a Python setup script.  If a directory path
       
   262 is given, it should point to a directory containing a setup.py file.
       
   263 Development eggs are installed before building any parts, as they may
       
   264 provide locally-defined recipes needed by the parts.
       
   265 
       
   266 ::
       
   267 
       
   268     parts = data-dir
       
   269 
       
   270 Here we've named a part to be "built".  We can use any name we want
       
   271 except that different part names must be unique and recipes will often
       
   272 use the part name to decide what to do.
       
   273 
       
   274 ::
       
   275 
       
   276     [data-dir]
       
   277     recipe = recipes:mkdir
       
   278     path = mystuff
       
   279 
       
   280 
       
   281 When we name a part, we also create a section of the same
       
   282 name that contains part data.  In this section, we'll define
       
   283 the recipe to be used to install the part.  In this case, we also
       
   284 specify the path to be created.
       
   285 
       
   286 Let's run the buildout.  We do so by running the build script in the
       
   287 buildout:
       
   288 
       
   289     >>> import os
       
   290     >>> os.chdir(sample_buildout)
       
   291     >>> buildout = os.path.join(sample_buildout, 'bin', 'buildout')
       
   292     >>> print system(buildout),
       
   293     Develop: '/sample-buildout/recipes'
       
   294     Installing data-dir.
       
   295     data-dir: Creating directory mystuff
       
   296 
       
   297 We see that the recipe created the directory, as expected:
       
   298 
       
   299     >>> ls(sample_buildout)
       
   300     -  .installed.cfg
       
   301     d  bin
       
   302     -  buildout.cfg
       
   303     d  develop-eggs
       
   304     d  eggs
       
   305     d  mystuff
       
   306     d  parts
       
   307     d  recipes
       
   308 
       
   309 In addition, .installed.cfg has been created containing information
       
   310 about the part we installed:
       
   311 
       
   312     >>> cat(sample_buildout, '.installed.cfg')
       
   313     [buildout]
       
   314     installed_develop_eggs = /sample-buildout/develop-eggs/recipes.egg-link
       
   315     parts = data-dir
       
   316     <BLANKLINE>
       
   317     [data-dir]
       
   318     __buildout_installed__ = /sample-buildout/mystuff
       
   319     __buildout_signature__ = recipes-c7vHV6ekIDUPy/7fjAaYjg==
       
   320     path = /sample-buildout/mystuff
       
   321     recipe = recipes:mkdir
       
   322 
       
   323 Note that the directory we installed is included in .installed.cfg.
       
   324 In addition, the path option includes the actual destination
       
   325 directory.
       
   326 
       
   327 If we change the name of the directory in the configuration file,
       
   328 we'll see that the directory gets removed and recreated:
       
   329 
       
   330     >>> write(sample_buildout, 'buildout.cfg',
       
   331     ... """
       
   332     ... [buildout]
       
   333     ... develop = recipes
       
   334     ... parts = data-dir
       
   335     ...
       
   336     ... [data-dir]
       
   337     ... recipe = recipes:mkdir
       
   338     ... path = mydata
       
   339     ... """)
       
   340 
       
   341     >>> print system(buildout),
       
   342     Develop: '/sample-buildout/recipes'
       
   343     Uninstalling data-dir.
       
   344     Installing data-dir.
       
   345     data-dir: Creating directory mydata
       
   346 
       
   347     >>> ls(sample_buildout)
       
   348     -  .installed.cfg
       
   349     d  bin
       
   350     -  buildout.cfg
       
   351     d  develop-eggs
       
   352     d  eggs
       
   353     d  mydata
       
   354     d  parts
       
   355     d  recipes
       
   356 
       
   357 If any of the files or directories created by a recipe are removed,
       
   358 the part will be reinstalled:
       
   359 
       
   360     >>> rmdir(sample_buildout, 'mydata')
       
   361     >>> print system(buildout),
       
   362     Develop: '/sample-buildout/recipes'
       
   363     Uninstalling data-dir.
       
   364     Installing data-dir.
       
   365     data-dir: Creating directory mydata
       
   366 
       
   367 Error reporting
       
   368 ---------------
       
   369 
       
   370 If a user makes an error, an error needs to be printed and work needs
       
   371 to stop.  This is accomplished by logging a detailed error message and
       
   372 then raising a (or an instance of a subclass of a)
       
   373 zc.buildout.UserError exception.  Raising an error other than a
       
   374 UserError still displays the error, but labels it as a bug in the
       
   375 buildout software or recipe. In the sample above, of someone gives a
       
   376 non-existent directory to create the directory in:
       
   377 
       
   378 
       
   379     >>> write(sample_buildout, 'buildout.cfg',
       
   380     ... """
       
   381     ... [buildout]
       
   382     ... develop = recipes
       
   383     ... parts = data-dir
       
   384     ...
       
   385     ... [data-dir]
       
   386     ... recipe = recipes:mkdir
       
   387     ... path = /xxx/mydata
       
   388     ... """)
       
   389 
       
   390 We'll get a user error, not a traceback.
       
   391 
       
   392     >>> print system(buildout),
       
   393     Develop: '/sample-buildout/recipes'
       
   394     data-dir: Cannot create /xxx/mydata. /xxx is not a directory.
       
   395     While:
       
   396       Installing.
       
   397       Getting section data-dir.
       
   398       Initializing part data-dir.
       
   399     Error: Invalid Path
       
   400 
       
   401 
       
   402 Recipe Error Handling
       
   403 ---------------------
       
   404 
       
   405 If an error occurs during installation, it is up to the recipe to
       
   406 clean up any system side effects, such as files created.  Let's update
       
   407 the mkdir recipe to support multiple paths:
       
   408 
       
   409     >>> write(sample_buildout, 'recipes', 'mkdir.py',
       
   410     ... """
       
   411     ... import logging, os, zc.buildout
       
   412     ...
       
   413     ... class Mkdir:
       
   414     ...
       
   415     ...     def __init__(self, buildout, name, options):
       
   416     ...         self.name, self.options = name, options
       
   417     ...
       
   418     ...         # Normalize paths and check that their parent
       
   419     ...         # directories exist:
       
   420     ...         paths = []
       
   421     ...         for path in options['path'].split():
       
   422     ...             path = os.path.join(buildout['buildout']['directory'], path)
       
   423     ...             if not os.path.isdir(os.path.dirname(path)):
       
   424     ...                 logging.getLogger(self.name).error(
       
   425     ...                     'Cannot create %s. %s is not a directory.',
       
   426     ...                     options['path'], os.path.dirname(options['path']))
       
   427     ...                 raise zc.buildout.UserError('Invalid Path')
       
   428     ...             paths.append(path)
       
   429     ...         options['path'] = ' '.join(paths)
       
   430     ...
       
   431     ...     def install(self):
       
   432     ...         paths = self.options['path'].split()
       
   433     ...         for path in paths:
       
   434     ...             logging.getLogger(self.name).info(
       
   435     ...                 'Creating directory %s', os.path.basename(path))
       
   436     ...             os.mkdir(path)
       
   437     ...         return paths
       
   438     ...
       
   439     ...     def update(self):
       
   440     ...         pass
       
   441     ... """)
       
   442 
       
   443 If there is an error creating a path, the install method will exit and
       
   444 leave previously created paths in place:
       
   445 
       
   446     >>> write(sample_buildout, 'buildout.cfg',
       
   447     ... """
       
   448     ... [buildout]
       
   449     ... develop = recipes
       
   450     ... parts = data-dir
       
   451     ...
       
   452     ... [data-dir]
       
   453     ... recipe = recipes:mkdir
       
   454     ... path = foo bin
       
   455     ... """)
       
   456 
       
   457     >>> print system(buildout), # doctest: +ELLIPSIS
       
   458     Develop: '/sample-buildout/recipes'
       
   459     Uninstalling data-dir.
       
   460     Installing data-dir.
       
   461     data-dir: Creating directory foo
       
   462     data-dir: Creating directory bin
       
   463     While:
       
   464       Installing data-dir.
       
   465     <BLANKLINE>
       
   466     An internal error occurred due to a bug in either zc.buildout or in a
       
   467     recipe being used:
       
   468     Traceback (most recent call last):
       
   469       ...
       
   470     OSError: [Errno 17] File exists: '/sample-buildout/bin'
       
   471 
       
   472 We meant to create a directory bins, but typed bin.  Now foo was
       
   473 left behind.
       
   474 
       
   475     >>> os.path.exists('foo')
       
   476     True
       
   477 
       
   478 If we fix the typo:
       
   479 
       
   480     >>> write(sample_buildout, 'buildout.cfg',
       
   481     ... """
       
   482     ... [buildout]
       
   483     ... develop = recipes
       
   484     ... parts = data-dir
       
   485     ...
       
   486     ... [data-dir]
       
   487     ... recipe = recipes:mkdir
       
   488     ... path = foo bins
       
   489     ... """)
       
   490 
       
   491     >>> print system(buildout), # doctest: +ELLIPSIS
       
   492     Develop: '/sample-buildout/recipes'
       
   493     Installing data-dir.
       
   494     data-dir: Creating directory foo
       
   495     While:
       
   496       Installing data-dir.
       
   497     <BLANKLINE>
       
   498     An internal error occurred due to a bug in either zc.buildout or in a
       
   499     recipe being used:
       
   500     Traceback (most recent call last):
       
   501     ...
       
   502     OSError: [Errno 17] File exists: '/sample-buildout/foo'
       
   503 
       
   504 Now they fail because foo exists, because it was left behind.
       
   505 
       
   506     >>> remove('foo')
       
   507 
       
   508 Let's fix the recipe:
       
   509 
       
   510     >>> write(sample_buildout, 'recipes', 'mkdir.py',
       
   511     ... """
       
   512     ... import logging, os, zc.buildout
       
   513     ...
       
   514     ... class Mkdir:
       
   515     ...
       
   516     ...     def __init__(self, buildout, name, options):
       
   517     ...         self.name, self.options = name, options
       
   518     ...
       
   519     ...         # Normalize paths and check that their parent
       
   520     ...         # directories exist:
       
   521     ...         paths = []
       
   522     ...         for path in options['path'].split():
       
   523     ...             path = os.path.join(buildout['buildout']['directory'], path)
       
   524     ...             if not os.path.isdir(os.path.dirname(path)):
       
   525     ...                 logging.getLogger(self.name).error(
       
   526     ...                     'Cannot create %s. %s is not a directory.',
       
   527     ...                     options['path'], os.path.dirname(options['path']))
       
   528     ...                 raise zc.buildout.UserError('Invalid Path')
       
   529     ...             paths.append(path)
       
   530     ...         options['path'] = ' '.join(paths)
       
   531     ...
       
   532     ...     def install(self):
       
   533     ...         paths = self.options['path'].split()
       
   534     ...         created = []
       
   535     ...         try:
       
   536     ...             for path in paths:
       
   537     ...                 logging.getLogger(self.name).info(
       
   538     ...                     'Creating directory %s', os.path.basename(path))
       
   539     ...                 os.mkdir(path)
       
   540     ...                 created.append(path)
       
   541     ...         except:
       
   542     ...             for d in created:
       
   543     ...                 os.rmdir(d)
       
   544     ...             raise
       
   545     ...
       
   546     ...         return paths
       
   547     ...
       
   548     ...     def update(self):
       
   549     ...         pass
       
   550     ... """)
       
   551 
       
   552 And put back the typo:
       
   553 
       
   554     >>> write(sample_buildout, 'buildout.cfg',
       
   555     ... """
       
   556     ... [buildout]
       
   557     ... develop = recipes
       
   558     ... parts = data-dir
       
   559     ...
       
   560     ... [data-dir]
       
   561     ... recipe = recipes:mkdir
       
   562     ... path = foo bin
       
   563     ... """)
       
   564 
       
   565 When we rerun the buildout:
       
   566 
       
   567     >>> print system(buildout), # doctest: +ELLIPSIS
       
   568     Develop: '/sample-buildout/recipes'
       
   569     Installing data-dir.
       
   570     data-dir: Creating directory foo
       
   571     data-dir: Creating directory bin
       
   572     While:
       
   573       Installing data-dir.
       
   574     <BLANKLINE>
       
   575     An internal error occurred due to a bug in either zc.buildout or in a
       
   576     recipe being used:
       
   577     Traceback (most recent call last):
       
   578     ...
       
   579     OSError: [Errno 17] File exists: '/sample-buildout/bin'
       
   580 
       
   581 .. Wait for the file to really disappear. My linux is weird.
       
   582 
       
   583     >>> wait_until("foo goes away", lambda : not os.path.exists('foo'),
       
   584     ...            timeout=200)
       
   585 
       
   586 we get the same error, but we don't get the directory left behind:
       
   587 
       
   588     >>> os.path.exists('foo')
       
   589     False
       
   590 
       
   591 It's critical that recipes clean up partial effects when errors
       
   592 occur.  Because recipes most commonly create files and directories,
       
   593 buildout provides a helper API for removing created files when an
       
   594 error occurs.  Option objects have a created method that can be called
       
   595 to record files as they are created.  If the install or update method
       
   596 returns with an error, then any registered paths are removed
       
   597 automatically.  The method returns the files registered and can be
       
   598 used to return the files created.  Let's use this API to simplify the
       
   599 recipe:
       
   600 
       
   601     >>> write(sample_buildout, 'recipes', 'mkdir.py',
       
   602     ... """
       
   603     ... import logging, os, zc.buildout
       
   604     ...
       
   605     ... class Mkdir:
       
   606     ...
       
   607     ...     def __init__(self, buildout, name, options):
       
   608     ...         self.name, self.options = name, options
       
   609     ...
       
   610     ...         # Normalize paths and check that their parent
       
   611     ...         # directories exist:
       
   612     ...         paths = []
       
   613     ...         for path in options['path'].split():
       
   614     ...             path = os.path.join(buildout['buildout']['directory'], path)
       
   615     ...             if not os.path.isdir(os.path.dirname(path)):
       
   616     ...                 logging.getLogger(self.name).error(
       
   617     ...                     'Cannot create %s. %s is not a directory.',
       
   618     ...                     options['path'], os.path.dirname(options['path']))
       
   619     ...                 raise zc.buildout.UserError('Invalid Path')
       
   620     ...             paths.append(path)
       
   621     ...         options['path'] = ' '.join(paths)
       
   622     ...
       
   623     ...     def install(self):
       
   624     ...         paths = self.options['path'].split()
       
   625     ...         for path in paths:
       
   626     ...             logging.getLogger(self.name).info(
       
   627     ...                 'Creating directory %s', os.path.basename(path))
       
   628     ...             os.mkdir(path)
       
   629     ...             self.options.created(path)
       
   630     ...
       
   631     ...         return self.options.created()
       
   632     ...
       
   633     ...     def update(self):
       
   634     ...         pass
       
   635     ... """)
       
   636 
       
   637 ..
       
   638 
       
   639     >>> remove(sample_buildout, 'recipes', 'mkdir.pyc')
       
   640 
       
   641 We returned by calling created, taking advantage of the fact that it
       
   642 returns the registered paths.  We did this for illustrative purposes.
       
   643 It would be simpler just to return the paths as before.
       
   644 
       
   645 If we rerun the buildout, again, we'll get the error and no
       
   646 directories will be created:
       
   647 
       
   648     >>> print system(buildout), # doctest: +ELLIPSIS
       
   649     Develop: '/sample-buildout/recipes'
       
   650     Installing data-dir.
       
   651     data-dir: Creating directory foo
       
   652     data-dir: Creating directory bin
       
   653     While:
       
   654       Installing data-dir.
       
   655     <BLANKLINE>
       
   656     An internal error occurred due to a bug in either zc.buildout or in a
       
   657     recipe being used:
       
   658     Traceback (most recent call last):
       
   659     ...
       
   660     OSError: [Errno 17] File exists: '/sample-buildout/bin'
       
   661 
       
   662     >>> os.path.exists('foo')
       
   663     False
       
   664 
       
   665 Now, we'll fix the typo again and we'll get the directories we expect:
       
   666 
       
   667     >>> write(sample_buildout, 'buildout.cfg',
       
   668     ... """
       
   669     ... [buildout]
       
   670     ... develop = recipes
       
   671     ... parts = data-dir
       
   672     ...
       
   673     ... [data-dir]
       
   674     ... recipe = recipes:mkdir
       
   675     ... path = foo bins
       
   676     ... """)
       
   677 
       
   678     >>> print system(buildout),
       
   679     Develop: '/sample-buildout/recipes'
       
   680     Installing data-dir.
       
   681     data-dir: Creating directory foo
       
   682     data-dir: Creating directory bins
       
   683 
       
   684     >>> os.path.exists('foo')
       
   685     True
       
   686     >>> os.path.exists('bins')
       
   687     True
       
   688 
       
   689 Configuration file syntax
       
   690 -------------------------
       
   691 
       
   692 As mentioned earlier, buildout configuration files use the format
       
   693 defined by the Python ConfigParser module with extensions.  The
       
   694 extensions are:
       
   695 
       
   696 - option names are case sensitive
       
   697 
       
   698 - option values can use a substitution syntax, described below, to
       
   699   refer to option values in specific sections.
       
   700 
       
   701 - option values can be appended or removed using the - and +
       
   702   operators.
       
   703 
       
   704 The ConfigParser syntax is very flexible.  Section names can contain
       
   705 any characters other than newlines and right square braces ("]").
       
   706 Option names can contain any characters other than newlines, colons,
       
   707 and equal signs, can not start with a space, and don't include
       
   708 trailing spaces.
       
   709 
       
   710 It is likely that, in the future, some characters will be given
       
   711 special buildout-defined meanings.  This is already true of the
       
   712 characters ":", "$", "%", "(", and ")".  For now, it is a good idea to
       
   713 keep section and option names simple, sticking to alphanumeric
       
   714 characters, hyphens, and periods.
       
   715 
       
   716 Annotated sections
       
   717 ------------------
       
   718 
       
   719 When used with the `annotate` command, buildout displays annotated sections.
       
   720 All sections are displayed, sorted alphabetically. For each section,
       
   721 all key-value pairs are displayed, sorted alphabetically, along with
       
   722 the origin of the value (file name or COMPUTED_VALUE, DEFAULT_VALUE,
       
   723 COMMAND_LINE_VALUE).
       
   724 
       
   725     >>> print system(buildout+ ' annotate'),
       
   726     ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
       
   727     <BLANKLINE>
       
   728     Annotated sections
       
   729     ==================
       
   730     <BLANKLINE>
       
   731     [buildout]
       
   732     accept-buildout-test-releases= false
       
   733         DEFAULT_VALUE
       
   734     allow-hosts= *
       
   735         DEFAULT_VALUE
       
   736     allow-picked-versions= true
       
   737         DEFAULT_VALUE
       
   738     allowed-eggs-from-site-packages= *
       
   739         DEFAULT_VALUE
       
   740     bin-directory= bin
       
   741         DEFAULT_VALUE
       
   742     develop= recipes
       
   743         /sample-buildout/buildout.cfg
       
   744     develop-eggs-directory= develop-eggs
       
   745         DEFAULT_VALUE
       
   746     directory= /sample-buildout
       
   747         COMPUTED_VALUE
       
   748     eggs-directory= eggs
       
   749         DEFAULT_VALUE
       
   750     exec-sitecustomize= true
       
   751         DEFAULT_VALUE
       
   752     executable= ...
       
   753         DEFAULT_VALUE
       
   754     find-links=
       
   755         DEFAULT_VALUE
       
   756     include-site-packages= true
       
   757         DEFAULT_VALUE
       
   758     install-from-cache= false
       
   759         DEFAULT_VALUE
       
   760     installed= .installed.cfg
       
   761         DEFAULT_VALUE
       
   762     log-format=
       
   763         DEFAULT_VALUE
       
   764     log-level= INFO
       
   765         DEFAULT_VALUE
       
   766     newest= true
       
   767         DEFAULT_VALUE
       
   768     offline= false
       
   769         DEFAULT_VALUE
       
   770     parts= data-dir
       
   771         /sample-buildout/buildout.cfg
       
   772     parts-directory= parts
       
   773         DEFAULT_VALUE
       
   774     prefer-final= false
       
   775         DEFAULT_VALUE
       
   776     python= buildout
       
   777         DEFAULT_VALUE
       
   778     relative-paths= false
       
   779         DEFAULT_VALUE
       
   780     socket-timeout=
       
   781         DEFAULT_VALUE
       
   782     unzip= false
       
   783         DEFAULT_VALUE
       
   784     use-dependency-links= true
       
   785         DEFAULT_VALUE
       
   786     <BLANKLINE>
       
   787     [data-dir]
       
   788     path= foo bins
       
   789         /sample-buildout/buildout.cfg
       
   790     recipe= recipes:mkdir
       
   791         /sample-buildout/buildout.cfg
       
   792     <BLANKLINE>
       
   793 
       
   794 Variable substitutions
       
   795 ----------------------
       
   796 
       
   797 Buildout configuration files support variable substitution.
       
   798 To illustrate this, we'll create an debug recipe to
       
   799 allow us to see interactions with the buildout:
       
   800 
       
   801     >>> write(sample_buildout, 'recipes', 'debug.py',
       
   802     ... """
       
   803     ... class Debug:
       
   804     ...
       
   805     ...     def __init__(self, buildout, name, options):
       
   806     ...         self.buildout = buildout
       
   807     ...         self.name = name
       
   808     ...         self.options = options
       
   809     ...
       
   810     ...     def install(self):
       
   811     ...         items = self.options.items()
       
   812     ...         items.sort()
       
   813     ...         for option, value in items:
       
   814     ...             print option, value
       
   815     ...         return ()
       
   816     ...
       
   817     ...     update = install
       
   818     ... """)
       
   819 
       
   820 This recipe doesn't actually create anything. The install method
       
   821 doesn't return anything, because it didn't create any files or
       
   822 directories.
       
   823 
       
   824 We also have to update our setup script:
       
   825 
       
   826     >>> write(sample_buildout, 'recipes', 'setup.py',
       
   827     ... """
       
   828     ... from setuptools import setup
       
   829     ... entry_points = (
       
   830     ... '''
       
   831     ... [zc.buildout]
       
   832     ... mkdir = mkdir:Mkdir
       
   833     ... debug = debug:Debug
       
   834     ... ''')
       
   835     ... setup(name="recipes", entry_points=entry_points)
       
   836     ... """)
       
   837 
       
   838 We've rearranged the script a bit to make the entry points easier to
       
   839 edit.  In particular, entry points are now defined as a configuration
       
   840 string, rather than a dictionary.
       
   841 
       
   842 Let's update our configuration to provide variable substitution
       
   843 examples:
       
   844 
       
   845     >>> write(sample_buildout, 'buildout.cfg',
       
   846     ... """
       
   847     ... [buildout]
       
   848     ... develop = recipes
       
   849     ... parts = data-dir debug
       
   850     ... log-level = INFO
       
   851     ...
       
   852     ... [debug]
       
   853     ... recipe = recipes:debug
       
   854     ... File 1 = ${data-dir:path}/file
       
   855     ... File 2 = ${debug:File 1}/log
       
   856     ...
       
   857     ... [data-dir]
       
   858     ... recipe = recipes:mkdir
       
   859     ... path = mydata
       
   860     ... """)
       
   861 
       
   862 We used a string-template substitution for File 1 and File 2.  This
       
   863 type of substitution uses the string.Template syntax.  Names
       
   864 substituted are qualified option names, consisting of a section name
       
   865 and option name joined by a colon.
       
   866 
       
   867 Now, if we run the buildout, we'll see the options with the values
       
   868 substituted.
       
   869 
       
   870     >>> print system(buildout),
       
   871     Develop: '/sample-buildout/recipes'
       
   872     Uninstalling data-dir.
       
   873     Installing data-dir.
       
   874     data-dir: Creating directory mydata
       
   875     Installing debug.
       
   876     File 1 /sample-buildout/mydata/file
       
   877     File 2 /sample-buildout/mydata/file/log
       
   878     recipe recipes:debug
       
   879 
       
   880 Note that the substitution of the data-dir path option reflects the
       
   881 update to the option performed by the mkdir recipe.
       
   882 
       
   883 It might seem surprising that mydata was created again.  This is
       
   884 because we changed our recipes package by adding the debug module.
       
   885 The buildout system didn't know if this module could effect the mkdir
       
   886 recipe, so it assumed it could and reinstalled mydata.  If we rerun
       
   887 the buildout:
       
   888 
       
   889     >>> print system(buildout),
       
   890     Develop: '/sample-buildout/recipes'
       
   891     Updating data-dir.
       
   892     Updating debug.
       
   893     File 1 /sample-buildout/mydata/file
       
   894     File 2 /sample-buildout/mydata/file/log
       
   895     recipe recipes:debug
       
   896 
       
   897 We can see that mydata was not recreated.
       
   898 
       
   899 Note that, in this case, we didn't specify a log level, so
       
   900 we didn't get output about what the buildout was doing.
       
   901 
       
   902 Section and option names in variable substitutions are only allowed to
       
   903 contain alphanumeric characters, hyphens, periods and spaces. This
       
   904 restriction might be relaxed in future releases.
       
   905 
       
   906 We can ommit the section name in a variable substitution to refer to
       
   907 the current section.  We can also use the special option,
       
   908 _buildout_section_name_ to get the current section name.
       
   909 
       
   910     >>> write(sample_buildout, 'buildout.cfg',
       
   911     ... """
       
   912     ... [buildout]
       
   913     ... develop = recipes
       
   914     ... parts = data-dir debug
       
   915     ... log-level = INFO
       
   916     ...
       
   917     ... [debug]
       
   918     ... recipe = recipes:debug
       
   919     ... File 1 = ${data-dir:path}/file
       
   920     ... File 2 = ${:File 1}/log
       
   921     ... my_name = ${:_buildout_section_name_}
       
   922     ...
       
   923     ... [data-dir]
       
   924     ... recipe = recipes:mkdir
       
   925     ... path = mydata
       
   926     ... """)
       
   927 
       
   928     >>> print system(buildout),
       
   929     Develop: '/sample-buildout/recipes'
       
   930     Uninstalling debug.
       
   931     Updating data-dir.
       
   932     Installing debug.
       
   933     File 1 /sample-buildout/mydata/file
       
   934     File 2 /sample-buildout/mydata/file/log
       
   935     my_name debug
       
   936     recipe recipes:debug
       
   937 
       
   938 Automatic part selection and ordering
       
   939 -------------------------------------
       
   940 
       
   941 When a section with a recipe is referred to, either through variable
       
   942 substitution or by an initializing recipe, the section is treated as a
       
   943 part and added to the part list before the referencing part.  For
       
   944 example, we can leave data-dir out of the parts list:
       
   945 
       
   946     >>> write(sample_buildout, 'buildout.cfg',
       
   947     ... """
       
   948     ... [buildout]
       
   949     ... develop = recipes
       
   950     ... parts = debug
       
   951     ... log-level = INFO
       
   952     ...
       
   953     ... [debug]
       
   954     ... recipe = recipes:debug
       
   955     ... File 1 = ${data-dir:path}/file
       
   956     ... File 2 = ${debug:File 1}/log
       
   957     ...
       
   958     ... [data-dir]
       
   959     ... recipe = recipes:mkdir
       
   960     ... path = mydata
       
   961     ... """)
       
   962 
       
   963 
       
   964 It will still be treated as a part:
       
   965 
       
   966     >>> print system(buildout),
       
   967     Develop: '/sample-buildout/recipes'
       
   968     Uninstalling debug.
       
   969     Updating data-dir.
       
   970     Installing debug.
       
   971     File 1 /sample-buildout/mydata/file
       
   972     File 2 /sample-buildout/mydata/file/log
       
   973     recipe recipes:debug
       
   974 
       
   975     >>> cat('.installed.cfg') # doctest: +ELLIPSIS
       
   976     [buildout]
       
   977     installed_develop_eggs = /sample-buildout/develop-eggs/recipes.egg-link
       
   978     parts = data-dir debug
       
   979     ...
       
   980 
       
   981 Note that the data-dir part is included *before* the debug part,
       
   982 because the debug part refers to the data-dir part.  Even if we list
       
   983 the data-dir part after the debug part, it will be included before:
       
   984 
       
   985     >>> write(sample_buildout, 'buildout.cfg',
       
   986     ... """
       
   987     ... [buildout]
       
   988     ... develop = recipes
       
   989     ... parts = debug data-dir
       
   990     ... log-level = INFO
       
   991     ...
       
   992     ... [debug]
       
   993     ... recipe = recipes:debug
       
   994     ... File 1 = ${data-dir:path}/file
       
   995     ... File 2 = ${debug:File 1}/log
       
   996     ...
       
   997     ... [data-dir]
       
   998     ... recipe = recipes:mkdir
       
   999     ... path = mydata
       
  1000     ... """)
       
  1001 
       
  1002 
       
  1003 It will still be treated as a part:
       
  1004 
       
  1005     >>> print system(buildout),
       
  1006     Develop: '/sample-buildout/recipes'
       
  1007     Updating data-dir.
       
  1008     Updating debug.
       
  1009     File 1 /sample-buildout/mydata/file
       
  1010     File 2 /sample-buildout/mydata/file/log
       
  1011     recipe recipes:debug
       
  1012 
       
  1013     >>> cat('.installed.cfg') # doctest: +ELLIPSIS
       
  1014     [buildout]
       
  1015     installed_develop_eggs = /sample-buildout/develop-eggs/recipes.egg-link
       
  1016     parts = data-dir debug
       
  1017     ...
       
  1018 
       
  1019 Extending sections (macros)
       
  1020 ---------------------------
       
  1021 
       
  1022 A section (other than the buildout section) can extend one or more
       
  1023 other sections using the ``<=`` option.  Options from the referenced
       
  1024 sections are copied to the refering section *before* variable
       
  1025 substitution.  This, together with the ability to refer to variables
       
  1026 of the current section allows sections to be used as macros.
       
  1027 
       
  1028     >>> write(sample_buildout, 'buildout.cfg',
       
  1029     ... """
       
  1030     ... [buildout]
       
  1031     ... develop = recipes
       
  1032     ... parts = myfiles
       
  1033     ... log-level = INFO
       
  1034     ...
       
  1035     ... [debug]
       
  1036     ... recipe = recipes:debug
       
  1037     ...
       
  1038     ... [with_file1]
       
  1039     ... <= debug
       
  1040     ... file1 = ${:path}/file1
       
  1041     ... color = red
       
  1042     ...
       
  1043     ... [with_file2]
       
  1044     ... <= debug
       
  1045     ... file2 = ${:path}/file2
       
  1046     ... color = blue
       
  1047     ...
       
  1048     ... [myfiles]
       
  1049     ... <= with_file1
       
  1050     ...    with_file2
       
  1051     ... path = mydata
       
  1052     ... """)
       
  1053 
       
  1054     >>> print system(buildout),
       
  1055     Develop: '/sample-buildout/recipes'
       
  1056     Uninstalling debug.
       
  1057     Uninstalling data-dir.
       
  1058     Installing myfiles.
       
  1059     color blue
       
  1060     file1 mydata/file1
       
  1061     file2 mydata/file2
       
  1062     path mydata
       
  1063     recipe recipes:debug
       
  1064 
       
  1065 In this example, the debug, with_file1 and with_file2 sections act as
       
  1066 macros. In particular, the variable substitutions are performed
       
  1067 relative to the myfiles section.
       
  1068 
       
  1069 Adding and removing options
       
  1070 ---------------------------
       
  1071 
       
  1072 We can append and remove values to an option by using the + and -
       
  1073 operators.
       
  1074 
       
  1075 This is illustrated below; first we define a base configuration.
       
  1076 
       
  1077     >>> write(sample_buildout, 'base.cfg',
       
  1078     ... """
       
  1079     ... [buildout]
       
  1080     ... parts = part1 part2 part3
       
  1081     ...
       
  1082     ... [part1]
       
  1083     ... recipe =
       
  1084     ... option = a1 a2
       
  1085     ...
       
  1086     ... [part2]
       
  1087     ... recipe =
       
  1088     ... option = b1 b2 b3 b4
       
  1089     ...
       
  1090     ... [part3]
       
  1091     ... recipe =
       
  1092     ... option = c1 c2
       
  1093     ...
       
  1094     ... """)
       
  1095 
       
  1096 Extending this configuration, we can "adjust" the values set in the
       
  1097 base configuration file.
       
  1098 
       
  1099     >>> write(sample_buildout, 'extension1.cfg',
       
  1100     ... """
       
  1101     ... [buildout]
       
  1102     ... extends = base.cfg
       
  1103     ...
       
  1104     ... # appending values
       
  1105     ... [part1]
       
  1106     ... option += a3 a4
       
  1107     ...
       
  1108     ... # removing values
       
  1109     ... [part2]
       
  1110     ... option -= b1 b2
       
  1111     ...
       
  1112     ... # alt. spelling
       
  1113     ... [part3]
       
  1114     ... option+=c3 c4 c5
       
  1115     ...
       
  1116     ... # normal assignment
       
  1117     ... [part4]
       
  1118     ... option = h1 h2
       
  1119     ...
       
  1120     ... """)
       
  1121 
       
  1122 An additional extension.
       
  1123 
       
  1124     >>> write(sample_buildout, 'extension2.cfg',
       
  1125     ... """
       
  1126     ... [buildout]
       
  1127     ... extends = extension1.cfg
       
  1128     ...
       
  1129     ... # appending values
       
  1130     ... [part1]
       
  1131     ... option += a5
       
  1132     ...
       
  1133     ... # removing values
       
  1134     ... [part2]
       
  1135     ... option -= b1 b2 b3
       
  1136     ...
       
  1137     ... """)
       
  1138 
       
  1139 To verify that the options are adjusted correctly, we'll set up an
       
  1140 extension that prints out the options.
       
  1141 
       
  1142     >>> mkdir(sample_buildout, 'demo')
       
  1143     >>> write(sample_buildout, 'demo', 'demo.py',
       
  1144     ... """
       
  1145     ... def ext(buildout):
       
  1146     ...     print [part['option'] for name, part in buildout.items() \
       
  1147     ...           if name.startswith('part')]
       
  1148     ... """)
       
  1149 
       
  1150     >>> write(sample_buildout, 'demo', 'setup.py',
       
  1151     ... """
       
  1152     ... from setuptools import setup
       
  1153     ...
       
  1154     ... setup(
       
  1155     ...     name="demo",
       
  1156     ...     entry_points={'zc.buildout.extension': ['ext = demo:ext']},
       
  1157     ...     )
       
  1158     ... """)
       
  1159 
       
  1160 Set up a buildout configuration for this extension.
       
  1161 
       
  1162     >>> write(sample_buildout, 'buildout.cfg',
       
  1163     ... """
       
  1164     ... [buildout]
       
  1165     ... develop = demo
       
  1166     ... parts =
       
  1167     ... """)
       
  1168 
       
  1169     >>> os.chdir(sample_buildout)
       
  1170     >>> print system(os.path.join(sample_buildout, 'bin', 'buildout')),
       
  1171     Develop: '/sample-buildout/demo'
       
  1172     Uninstalling myfiles.
       
  1173     Getting distribution for 'recipes'.
       
  1174     zip_safe flag not set; analyzing archive contents...
       
  1175     Got recipes 0.0.0.
       
  1176     warning: install_lib: 'build/lib' does not exist -- no Python modules to install
       
  1177 
       
  1178 Verify option values.
       
  1179 
       
  1180     >>> write(sample_buildout, 'buildout.cfg',
       
  1181     ... """
       
  1182     ... [buildout]
       
  1183     ... develop = demo
       
  1184     ... extensions = demo
       
  1185     ... extends = extension2.cfg
       
  1186     ... """)
       
  1187 
       
  1188     >>> print system(os.path.join('bin', 'buildout')),
       
  1189     ['a1 a2/na3 a4/na5', 'b1 b2 b3 b4', 'c1 c2/nc3 c4 c5', 'h1 h2']
       
  1190     Develop: '/sample-buildout/demo'
       
  1191 
       
  1192 Annotated sections output shows which files are responsible for which
       
  1193 operations.
       
  1194 
       
  1195     >>> print system(os.path.join('bin', 'buildout') + ' annotate'),
       
  1196     ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
       
  1197     <BLANKLINE>
       
  1198     Annotated sections
       
  1199     ==================
       
  1200     ...
       
  1201     <BLANKLINE>
       
  1202     [part1]
       
  1203     option= a1 a2
       
  1204     a3 a4
       
  1205     a5
       
  1206         /sample-buildout/base.cfg
       
  1207     +=  /sample-buildout/extension1.cfg
       
  1208     +=  /sample-buildout/extension2.cfg
       
  1209     recipe=
       
  1210         /sample-buildout/base.cfg
       
  1211     <BLANKLINE>
       
  1212     [part2]
       
  1213     option= b1 b2 b3 b4
       
  1214         /sample-buildout/base.cfg
       
  1215     -=  /sample-buildout/extension1.cfg
       
  1216     -=  /sample-buildout/extension2.cfg
       
  1217     recipe=
       
  1218         /sample-buildout/base.cfg
       
  1219     <BLANKLINE>
       
  1220     [part3]
       
  1221     option= c1 c2
       
  1222     c3 c4 c5
       
  1223         /sample-buildout/base.cfg
       
  1224     +=  /sample-buildout/extension1.cfg
       
  1225     recipe=
       
  1226         /sample-buildout/base.cfg
       
  1227     <BLANKLINE>
       
  1228     [part4]
       
  1229     option= h1 h2
       
  1230         /sample-buildout/extension1.cfg
       
  1231 
       
  1232 Cleanup.
       
  1233 
       
  1234     >>> os.remove(os.path.join(sample_buildout, 'base.cfg'))
       
  1235     >>> os.remove(os.path.join(sample_buildout, 'extension1.cfg'))
       
  1236     >>> os.remove(os.path.join(sample_buildout, 'extension2.cfg'))
       
  1237 
       
  1238 Multiple configuration files
       
  1239 ----------------------------
       
  1240 
       
  1241 A configuration file can "extend" another configuration file.
       
  1242 Options are read from the other configuration file if they aren't
       
  1243 already defined by your configuration file.
       
  1244 
       
  1245 The configuration files your file extends can extend
       
  1246 other configuration files.  The same file may be
       
  1247 used more than once although, of course, cycles aren't allowed.
       
  1248 
       
  1249 To see how this works, we use an example:
       
  1250 
       
  1251     >>> write(sample_buildout, 'buildout.cfg',
       
  1252     ... """
       
  1253     ... [buildout]
       
  1254     ... extends = base.cfg
       
  1255     ...
       
  1256     ... [debug]
       
  1257     ... op = buildout
       
  1258     ... """)
       
  1259 
       
  1260     >>> write(sample_buildout, 'base.cfg',
       
  1261     ... """
       
  1262     ... [buildout]
       
  1263     ... develop = recipes
       
  1264     ... parts = debug
       
  1265     ...
       
  1266     ... [debug]
       
  1267     ... recipe = recipes:debug
       
  1268     ... op = base
       
  1269     ... """)
       
  1270 
       
  1271     >>> print system(buildout),
       
  1272     Develop: '/sample-buildout/recipes'
       
  1273     Installing debug.
       
  1274     op buildout
       
  1275     recipe recipes:debug
       
  1276 
       
  1277 The example is pretty trivial, but the pattern it illustrates is
       
  1278 pretty common.  In a more practical example, the base buildout might
       
  1279 represent a product and the extending buildout might be a
       
  1280 customization.
       
  1281 
       
  1282 Here is a more elaborate example.
       
  1283 
       
  1284     >>> other = tmpdir('other')
       
  1285 
       
  1286     >>> write(sample_buildout, 'buildout.cfg',
       
  1287     ... """
       
  1288     ... [buildout]
       
  1289     ... extends = b1.cfg b2.cfg %(b3)s
       
  1290     ...
       
  1291     ... [debug]
       
  1292     ... op = buildout
       
  1293     ... """ % dict(b3=os.path.join(other, 'b3.cfg')))
       
  1294 
       
  1295     >>> write(sample_buildout, 'b1.cfg',
       
  1296     ... """
       
  1297     ... [buildout]
       
  1298     ... extends = base.cfg
       
  1299     ...
       
  1300     ... [debug]
       
  1301     ... op1 = b1 1
       
  1302     ... op2 = b1 2
       
  1303     ... """)
       
  1304 
       
  1305     >>> write(sample_buildout, 'b2.cfg',
       
  1306     ... """
       
  1307     ... [buildout]
       
  1308     ... extends = base.cfg
       
  1309     ...
       
  1310     ... [debug]
       
  1311     ... op2 = b2 2
       
  1312     ... op3 = b2 3
       
  1313     ... """)
       
  1314 
       
  1315     >>> write(other, 'b3.cfg',
       
  1316     ... """
       
  1317     ... [buildout]
       
  1318     ... extends = b3base.cfg
       
  1319     ...
       
  1320     ... [debug]
       
  1321     ... op4 = b3 4
       
  1322     ... """)
       
  1323 
       
  1324     >>> write(other, 'b3base.cfg',
       
  1325     ... """
       
  1326     ... [debug]
       
  1327     ... op5 = b3base 5
       
  1328     ... """)
       
  1329 
       
  1330     >>> write(sample_buildout, 'base.cfg',
       
  1331     ... """
       
  1332     ... [buildout]
       
  1333     ... develop = recipes
       
  1334     ... parts = debug
       
  1335     ...
       
  1336     ... [debug]
       
  1337     ... recipe = recipes:debug
       
  1338     ... name = base
       
  1339     ... """)
       
  1340 
       
  1341     >>> print system(buildout),
       
  1342     Develop: '/sample-buildout/recipes'
       
  1343     Uninstalling debug.
       
  1344     Installing debug.
       
  1345     name base
       
  1346     op buildout
       
  1347     op1 b1 1
       
  1348     op2 b2 2
       
  1349     op3 b2 3
       
  1350     op4 b3 4
       
  1351     op5 b3base 5
       
  1352     recipe recipes:debug
       
  1353 
       
  1354 There are several things to note about this example:
       
  1355 
       
  1356 - We can name multiple files in an extends option.
       
  1357 
       
  1358 - We can reference files recursively.
       
  1359 
       
  1360 - Relative file names in extended options are interpreted relative to
       
  1361   the directory containing the referencing configuration file.
       
  1362 
       
  1363 Loading Configuration from URLs
       
  1364 -------------------------------
       
  1365 
       
  1366 Configuration files can be loaded from URLs.  To see how this works,
       
  1367 we'll set up a web server with some configuration files.
       
  1368 
       
  1369     >>> server_data = tmpdir('server_data')
       
  1370 
       
  1371     >>> write(server_data, "r1.cfg",
       
  1372     ... """
       
  1373     ... [debug]
       
  1374     ... op1 = r1 1
       
  1375     ... op2 = r1 2
       
  1376     ... """)
       
  1377 
       
  1378     >>> write(server_data, "r2.cfg",
       
  1379     ... """
       
  1380     ... [buildout]
       
  1381     ... extends = r1.cfg
       
  1382     ...
       
  1383     ... [debug]
       
  1384     ... op2 = r2 2
       
  1385     ... op3 = r2 3
       
  1386     ... """)
       
  1387 
       
  1388     >>> server_url = start_server(server_data)
       
  1389 
       
  1390     >>> write('client.cfg',
       
  1391     ... """
       
  1392     ... [buildout]
       
  1393     ... develop = recipes
       
  1394     ... parts = debug
       
  1395     ... extends = %(url)s/r2.cfg
       
  1396     ...
       
  1397     ... [debug]
       
  1398     ... recipe = recipes:debug
       
  1399     ... name = base
       
  1400     ... """ % dict(url=server_url))
       
  1401 
       
  1402 
       
  1403     >>> print system(buildout+ ' -c client.cfg'),
       
  1404     Develop: '/sample-buildout/recipes'
       
  1405     Uninstalling debug.
       
  1406     Installing debug.
       
  1407     name base
       
  1408     op1 r1 1
       
  1409     op2 r2 2
       
  1410     op3 r2 3
       
  1411     recipe recipes:debug
       
  1412 
       
  1413 Here we specified a URL for the file we extended.  The file we
       
  1414 downloaded, itself referred to a file on the server using a relative
       
  1415 URL reference.  Relative references are interpreted relative to the
       
  1416 base URL when they appear in configuration files loaded via URL.
       
  1417 
       
  1418 We can also specify a URL as the configuration file to be used by a
       
  1419 buildout.
       
  1420 
       
  1421     >>> os.remove('client.cfg')
       
  1422     >>> write(server_data, 'remote.cfg',
       
  1423     ... """
       
  1424     ... [buildout]
       
  1425     ... develop = recipes
       
  1426     ... parts = debug
       
  1427     ... extends = r2.cfg
       
  1428     ...
       
  1429     ... [debug]
       
  1430     ... recipe = recipes:debug
       
  1431     ... name = remote
       
  1432     ... """)
       
  1433 
       
  1434     >>> print system(buildout + ' -c ' + server_url + '/remote.cfg'),
       
  1435     While:
       
  1436       Initializing.
       
  1437     Error: Missing option: buildout:directory
       
  1438 
       
  1439 Normally, the buildout directory defaults to directory
       
  1440 containing a configuration file.  This won't work for configuration
       
  1441 files loaded from URLs.  In this case, the buildout directory would
       
  1442 normally be defined on the command line:
       
  1443 
       
  1444     >>> print system(buildout
       
  1445     ...              + ' -c ' + server_url + '/remote.cfg'
       
  1446     ...              + ' buildout:directory=' + sample_buildout
       
  1447     ...              ),
       
  1448     Develop: '/sample-buildout/recipes'
       
  1449     Uninstalling debug.
       
  1450     Installing debug.
       
  1451     name remote
       
  1452     op1 r1 1
       
  1453     op2 r2 2
       
  1454     op3 r2 3
       
  1455     recipe recipes:debug
       
  1456 
       
  1457 User defaults
       
  1458 -------------
       
  1459 
       
  1460 If the file $HOME/.buildout/default.cfg, exists, it is read before
       
  1461 reading the configuration file.  ($HOME is the value of the HOME
       
  1462 environment variable. The '/' is replaced by the operating system file
       
  1463 delimiter.)
       
  1464 
       
  1465     >>> old_home = os.environ['HOME']
       
  1466     >>> home = tmpdir('home')
       
  1467     >>> mkdir(home, '.buildout')
       
  1468     >>> write(home, '.buildout', 'default.cfg',
       
  1469     ... """
       
  1470     ... [debug]
       
  1471     ... op1 = 1
       
  1472     ... op7 = 7
       
  1473     ... """)
       
  1474 
       
  1475     >>> os.environ['HOME'] = home
       
  1476     >>> print system(buildout),
       
  1477     Develop: '/sample-buildout/recipes'
       
  1478     Uninstalling debug.
       
  1479     Installing debug.
       
  1480     name base
       
  1481     op buildout
       
  1482     op1 b1 1
       
  1483     op2 b2 2
       
  1484     op3 b2 3
       
  1485     op4 b3 4
       
  1486     op5 b3base 5
       
  1487     op7 7
       
  1488     recipe recipes:debug
       
  1489 
       
  1490 A buildout command-line argument, -U, can be used to suppress reading
       
  1491 user defaults:
       
  1492 
       
  1493     >>> print system(buildout + ' -U'),
       
  1494     Develop: '/sample-buildout/recipes'
       
  1495     Uninstalling debug.
       
  1496     Installing debug.
       
  1497     name base
       
  1498     op buildout
       
  1499     op1 b1 1
       
  1500     op2 b2 2
       
  1501     op3 b2 3
       
  1502     op4 b3 4
       
  1503     op5 b3base 5
       
  1504     recipe recipes:debug
       
  1505 
       
  1506     >>> os.environ['HOME'] = old_home
       
  1507 
       
  1508 Log level
       
  1509 ---------
       
  1510 
       
  1511 We can control the level of logging by specifying a log level in out
       
  1512 configuration file.  For example, so suppress info messages, we can
       
  1513 set the logging level to WARNING
       
  1514 
       
  1515     >>> write(sample_buildout, 'buildout.cfg',
       
  1516     ... """
       
  1517     ... [buildout]
       
  1518     ... log-level = WARNING
       
  1519     ... extends = b1.cfg b2.cfg
       
  1520     ... """)
       
  1521 
       
  1522     >>> print system(buildout),
       
  1523     name base
       
  1524     op1 b1 1
       
  1525     op2 b2 2
       
  1526     op3 b2 3
       
  1527     recipe recipes:debug
       
  1528 
       
  1529 Uninstall recipes
       
  1530 -----------------
       
  1531 
       
  1532 As we've seen, when parts are installed, buildout keeps track of files
       
  1533 and directories that they create. When the parts are uninstalled these
       
  1534 files and directories are deleted.
       
  1535 
       
  1536 Sometimes more clean up is needed. For example, a recipe might add a
       
  1537 system service by calling chkconfig --add during installation. Later
       
  1538 during uninstallation, chkconfig --del will need to be called to
       
  1539 remove the system service.
       
  1540 
       
  1541 In order to deal with these uninstallation issues, you can register
       
  1542 uninstall recipes. Uninstall recipes are registered using the
       
  1543 'zc.buildout.uninstall' entry point. Parts specify uninstall recipes
       
  1544 using the 'uninstall' option.
       
  1545 
       
  1546 In comparison to regular recipes, uninstall recipes are much
       
  1547 simpler. They are simply callable objects that accept the name of the
       
  1548 part to be uninstalled and the part's options dictionary. Uninstall
       
  1549 recipes don't have access to the part itself since it maybe not be
       
  1550 able to be instantiated at uninstallation time.
       
  1551 
       
  1552 Here's a recipe that simulates installation of a system service, along
       
  1553 with an uninstall recipe that simulates removing the service.
       
  1554 
       
  1555     >>> write(sample_buildout, 'recipes', 'service.py',
       
  1556     ... """
       
  1557     ... class Service:
       
  1558     ...
       
  1559     ...     def __init__(self, buildout, name, options):
       
  1560     ...         self.buildout = buildout
       
  1561     ...         self.name = name
       
  1562     ...         self.options = options
       
  1563     ...
       
  1564     ...     def install(self):
       
  1565     ...         print "chkconfig --add %s" % self.options['script']
       
  1566     ...         return ()
       
  1567     ...
       
  1568     ...     def update(self):
       
  1569     ...         pass
       
  1570     ...
       
  1571     ...
       
  1572     ... def uninstall_service(name, options):
       
  1573     ...     print "chkconfig --del %s" % options['script']
       
  1574     ... """)
       
  1575 
       
  1576 To use these recipes we must register them using entry points. Make
       
  1577 sure to use the same name for the recipe and uninstall recipe. This is
       
  1578 required to let buildout know which uninstall recipe goes with which
       
  1579 recipe.
       
  1580 
       
  1581     >>> write(sample_buildout, 'recipes', 'setup.py',
       
  1582     ... """
       
  1583     ... from setuptools import setup
       
  1584     ... entry_points = (
       
  1585     ... '''
       
  1586     ... [zc.buildout]
       
  1587     ... mkdir = mkdir:Mkdir
       
  1588     ... debug = debug:Debug
       
  1589     ... service = service:Service
       
  1590     ...
       
  1591     ... [zc.buildout.uninstall]
       
  1592     ... service = service:uninstall_service
       
  1593     ... ''')
       
  1594     ... setup(name="recipes", entry_points=entry_points)
       
  1595     ... """)
       
  1596 
       
  1597 Here's how these recipes could be used in a buildout:
       
  1598 
       
  1599     >>> write(sample_buildout, 'buildout.cfg',
       
  1600     ... """
       
  1601     ... [buildout]
       
  1602     ... develop = recipes
       
  1603     ... parts = service
       
  1604     ...
       
  1605     ... [service]
       
  1606     ... recipe = recipes:service
       
  1607     ... script = /path/to/script
       
  1608     ... """)
       
  1609 
       
  1610 When the buildout is run the service will be installed
       
  1611 
       
  1612     >>> print system(buildout)
       
  1613     Develop: '/sample-buildout/recipes'
       
  1614     Uninstalling debug.
       
  1615     Installing service.
       
  1616     chkconfig --add /path/to/script
       
  1617     <BLANKLINE>
       
  1618 
       
  1619 The service has been installed. If the buildout is run again with no
       
  1620 changes, the service shouldn't be changed.
       
  1621 
       
  1622     >>> print system(buildout)
       
  1623     Develop: '/sample-buildout/recipes'
       
  1624     Updating service.
       
  1625     <BLANKLINE>
       
  1626 
       
  1627 Now we change the service part to trigger uninstallation and
       
  1628 re-installation.
       
  1629 
       
  1630     >>> write(sample_buildout, 'buildout.cfg',
       
  1631     ... """
       
  1632     ... [buildout]
       
  1633     ... develop = recipes
       
  1634     ... parts = service
       
  1635     ...
       
  1636     ... [service]
       
  1637     ... recipe = recipes:service
       
  1638     ... script = /path/to/a/different/script
       
  1639     ... """)
       
  1640 
       
  1641     >>> print system(buildout)
       
  1642     Develop: '/sample-buildout/recipes'
       
  1643     Uninstalling service.
       
  1644     Running uninstall recipe.
       
  1645     chkconfig --del /path/to/script
       
  1646     Installing service.
       
  1647     chkconfig --add /path/to/a/different/script
       
  1648     <BLANKLINE>
       
  1649 
       
  1650 Now we remove the service part, and add another part.
       
  1651 
       
  1652     >>> write(sample_buildout, 'buildout.cfg',
       
  1653     ... """
       
  1654     ... [buildout]
       
  1655     ... develop = recipes
       
  1656     ... parts = debug
       
  1657     ...
       
  1658     ... [debug]
       
  1659     ... recipe = recipes:debug
       
  1660     ... """)
       
  1661 
       
  1662     >>> print system(buildout)
       
  1663     Develop: '/sample-buildout/recipes'
       
  1664     Uninstalling service.
       
  1665     Running uninstall recipe.
       
  1666     chkconfig --del /path/to/a/different/script
       
  1667     Installing debug.
       
  1668     recipe recipes:debug
       
  1669     <BLANKLINE>
       
  1670 
       
  1671 Uninstall recipes don't have to take care of removing all the files
       
  1672 and directories created by the part. This is still done automatically,
       
  1673 following the execution of the uninstall recipe. An upshot is that an
       
  1674 uninstallation recipe can access files and directories created by a
       
  1675 recipe before they are deleted.
       
  1676 
       
  1677 For example, here's an uninstallation recipe that simulates backing up
       
  1678 a directory before it is deleted. It is designed to work with the
       
  1679 mkdir recipe introduced earlier.
       
  1680 
       
  1681     >>> write(sample_buildout, 'recipes', 'backup.py',
       
  1682     ... """
       
  1683     ... import os
       
  1684     ... def backup_directory(name, options):
       
  1685     ...     path = options['path']
       
  1686     ...     size = len(os.listdir(path))
       
  1687     ...     print "backing up directory %s of size %s" % (path, size)
       
  1688     ... """)
       
  1689 
       
  1690 It must be registered with the zc.buildout.uninstall entry
       
  1691 point. Notice how it is given the name 'mkdir' to associate it with
       
  1692 the mkdir recipe.
       
  1693 
       
  1694     >>> write(sample_buildout, 'recipes', 'setup.py',
       
  1695     ... """
       
  1696     ... from setuptools import setup
       
  1697     ... entry_points = (
       
  1698     ... '''
       
  1699     ... [zc.buildout]
       
  1700     ... mkdir = mkdir:Mkdir
       
  1701     ... debug = debug:Debug
       
  1702     ... service = service:Service
       
  1703     ...
       
  1704     ... [zc.buildout.uninstall]
       
  1705     ... uninstall_service = service:uninstall_service
       
  1706     ... mkdir = backup:backup_directory
       
  1707     ... ''')
       
  1708     ... setup(name="recipes", entry_points=entry_points)
       
  1709     ... """)
       
  1710 
       
  1711 Now we can use it with a mkdir part.
       
  1712 
       
  1713     >>> write(sample_buildout, 'buildout.cfg',
       
  1714     ... """
       
  1715     ... [buildout]
       
  1716     ... develop = recipes
       
  1717     ... parts = dir debug
       
  1718     ...
       
  1719     ... [dir]
       
  1720     ... recipe = recipes:mkdir
       
  1721     ... path = my_directory
       
  1722     ...
       
  1723     ... [debug]
       
  1724     ... recipe = recipes:debug
       
  1725     ... """)
       
  1726 
       
  1727 Run the buildout to install the part.
       
  1728 
       
  1729     >>> print system(buildout)
       
  1730     Develop: '/sample-buildout/recipes'
       
  1731     Uninstalling debug.
       
  1732     Installing dir.
       
  1733     dir: Creating directory my_directory
       
  1734     Installing debug.
       
  1735     recipe recipes:debug
       
  1736     <BLANKLINE>
       
  1737 
       
  1738 Now we remove the part from the configuration file.
       
  1739 
       
  1740     >>> write(sample_buildout, 'buildout.cfg',
       
  1741     ... """
       
  1742     ... [buildout]
       
  1743     ... develop = recipes
       
  1744     ... parts = debug
       
  1745     ...
       
  1746     ... [debug]
       
  1747     ... recipe = recipes:debug
       
  1748     ... """)
       
  1749 
       
  1750 When the buildout is run the part is removed, and the uninstall recipe
       
  1751 is run before the directory is deleted.
       
  1752 
       
  1753     >>> print system(buildout)
       
  1754     Develop: '/sample-buildout/recipes'
       
  1755     Uninstalling dir.
       
  1756     Running uninstall recipe.
       
  1757     backing up directory /sample-buildout/my_directory of size 0
       
  1758     Updating debug.
       
  1759     recipe recipes:debug
       
  1760     <BLANKLINE>
       
  1761 
       
  1762 Now we will return the registration to normal for the benefit of the
       
  1763 rest of the examples.
       
  1764 
       
  1765     >>> write(sample_buildout, 'recipes', 'setup.py',
       
  1766     ... """
       
  1767     ... from setuptools import setup
       
  1768     ... entry_points = (
       
  1769     ... '''
       
  1770     ... [zc.buildout]
       
  1771     ... mkdir = mkdir:Mkdir
       
  1772     ... debug = debug:Debug
       
  1773     ... ''')
       
  1774     ... setup(name="recipes", entry_points=entry_points)
       
  1775     ... """)
       
  1776 
       
  1777 
       
  1778 Command-line usage
       
  1779 ------------------
       
  1780 
       
  1781 A number of arguments can be given on the buildout command line.  The
       
  1782 command usage is::
       
  1783 
       
  1784   buildout [options and assignments] [command [command arguments]]
       
  1785 
       
  1786 The following options are supported:
       
  1787 
       
  1788 -h (or --help)
       
  1789     Print basic usage information.  If this option is used, then all
       
  1790     other options are ignored.
       
  1791 
       
  1792 -c filename
       
  1793     The -c option can be used to specify a configuration file, rather than
       
  1794     buildout.cfg in the current directory.
       
  1795 
       
  1796 
       
  1797 -t socket_timeout
       
  1798 
       
  1799    Specify the socket timeout in seconds.
       
  1800 
       
  1801 -v
       
  1802     Increment the verbosity by 10.  The verbosity is used to adjust
       
  1803     the logging level.  The verbosity is subtracted from the numeric
       
  1804     value of the log-level option specified in the configuration file.
       
  1805 
       
  1806 -q
       
  1807     Decrement the verbosity by 10.
       
  1808 
       
  1809 -U
       
  1810     Don't read user-default configuration.
       
  1811 
       
  1812 -o
       
  1813     Run in off-line mode.  This is equivalent to the assignment
       
  1814     buildout:offline=true.
       
  1815 
       
  1816 -O
       
  1817     Run in non-off-line mode.  This is equivalent to the assignment
       
  1818     buildout:offline=false.  This is the default buildout mode.  The
       
  1819     -O option would normally be used to override a true offline
       
  1820     setting in a configuration file.
       
  1821 
       
  1822 -n
       
  1823     Run in newest mode.  This is equivalent to the assignment
       
  1824     buildout:newest=true.  With this setting, which is the default,
       
  1825     buildout will try to find the newest versions of distributions
       
  1826     available that satisfy its requirements.
       
  1827 
       
  1828 -N
       
  1829     Run in non-newest mode.  This is equivalent to the assignment
       
  1830     buildout:newest=false.  With this setting, buildout will not seek
       
  1831     new distributions if installed distributions satisfy it's
       
  1832     requirements.
       
  1833 
       
  1834 Assignments are of the form::
       
  1835 
       
  1836   section_name:option_name=value
       
  1837 
       
  1838 Options and assignments can be given in any order.
       
  1839 
       
  1840 Here's an example:
       
  1841 
       
  1842     >>> write(sample_buildout, 'other.cfg',
       
  1843     ... """
       
  1844     ... [buildout]
       
  1845     ... develop = recipes
       
  1846     ... parts = debug
       
  1847     ... installed = .other.cfg
       
  1848     ... log-level = WARNING
       
  1849     ...
       
  1850     ... [debug]
       
  1851     ... name = other
       
  1852     ... recipe = recipes:debug
       
  1853     ... """)
       
  1854 
       
  1855 Note that we used the installed buildout option to specify an
       
  1856 alternate file to store information about installed parts.
       
  1857 
       
  1858     >>> print system(buildout+' -c other.cfg debug:op1=foo -v'),
       
  1859     Develop: '/sample-buildout/recipes'
       
  1860     Installing debug.
       
  1861     name other
       
  1862     op1 foo
       
  1863     recipe recipes:debug
       
  1864 
       
  1865 Here we used the -c option to specify an alternate configuration file,
       
  1866 and the -v option to increase the level of logging from the default,
       
  1867 WARNING.
       
  1868 
       
  1869 Options can also be combined in the usual Unix way, as in:
       
  1870 
       
  1871     >>> print system(buildout+' -vcother.cfg debug:op1=foo'),
       
  1872     Develop: '/sample-buildout/recipes'
       
  1873     Updating debug.
       
  1874     name other
       
  1875     op1 foo
       
  1876     recipe recipes:debug
       
  1877 
       
  1878 Here we combined the -v and -c options with the configuration file
       
  1879 name.  Note that the -c option has to be last, because it takes an
       
  1880 argument.
       
  1881 
       
  1882     >>> os.remove(os.path.join(sample_buildout, 'other.cfg'))
       
  1883     >>> os.remove(os.path.join(sample_buildout, '.other.cfg'))
       
  1884 
       
  1885 The most commonly used command is 'install' and it takes a list of
       
  1886 parts to install. if any parts are specified, only those parts are
       
  1887 installed.  To illustrate this, we'll update our configuration and run
       
  1888 the buildout in the usual way:
       
  1889 
       
  1890     >>> write(sample_buildout, 'buildout.cfg',
       
  1891     ... """
       
  1892     ... [buildout]
       
  1893     ... develop = recipes
       
  1894     ... parts = debug d1 d2 d3
       
  1895     ...
       
  1896     ... [d1]
       
  1897     ... recipe = recipes:mkdir
       
  1898     ... path = d1
       
  1899     ...
       
  1900     ... [d2]
       
  1901     ... recipe = recipes:mkdir
       
  1902     ... path = d2
       
  1903     ...
       
  1904     ... [d3]
       
  1905     ... recipe = recipes:mkdir
       
  1906     ... path = d3
       
  1907     ...
       
  1908     ... [debug]
       
  1909     ... recipe = recipes:debug
       
  1910     ... """)
       
  1911 
       
  1912     >>> print system(buildout),
       
  1913     Develop: '/sample-buildout/recipes'
       
  1914     Uninstalling debug.
       
  1915     Installing debug.
       
  1916     recipe recipes:debug
       
  1917     Installing d1.
       
  1918     d1: Creating directory d1
       
  1919     Installing d2.
       
  1920     d2: Creating directory d2
       
  1921     Installing d3.
       
  1922     d3: Creating directory d3
       
  1923 
       
  1924     >>> ls(sample_buildout)
       
  1925     -  .installed.cfg
       
  1926     -  b1.cfg
       
  1927     -  b2.cfg
       
  1928     -  base.cfg
       
  1929     d  bin
       
  1930     -  buildout.cfg
       
  1931     d  d1
       
  1932     d  d2
       
  1933     d  d3
       
  1934     d  demo
       
  1935     d  develop-eggs
       
  1936     d  eggs
       
  1937     d  parts
       
  1938     d  recipes
       
  1939 
       
  1940     >>> cat(sample_buildout, '.installed.cfg')
       
  1941     ... # doctest: +NORMALIZE_WHITESPACE
       
  1942     [buildout]
       
  1943     installed_develop_eggs = /sample-buildout/develop-eggs/recipes.egg-link
       
  1944     parts = debug d1 d2 d3
       
  1945     <BLANKLINE>
       
  1946     [debug]
       
  1947     __buildout_installed__ =
       
  1948     __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg==
       
  1949     recipe = recipes:debug
       
  1950     <BLANKLINE>
       
  1951     [d1]
       
  1952     __buildout_installed__ = /sample-buildout/d1
       
  1953     __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg==
       
  1954     path = /sample-buildout/d1
       
  1955     recipe = recipes:mkdir
       
  1956     <BLANKLINE>
       
  1957     [d2]
       
  1958     __buildout_installed__ = /sample-buildout/d2
       
  1959     __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg==
       
  1960     path = /sample-buildout/d2
       
  1961     recipe = recipes:mkdir
       
  1962     <BLANKLINE>
       
  1963     [d3]
       
  1964     __buildout_installed__ = /sample-buildout/d3
       
  1965     __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg==
       
  1966     path = /sample-buildout/d3
       
  1967     recipe = recipes:mkdir
       
  1968 
       
  1969 Now we'll update our configuration file:
       
  1970 
       
  1971     >>> write(sample_buildout, 'buildout.cfg',
       
  1972     ... """
       
  1973     ... [buildout]
       
  1974     ... develop = recipes
       
  1975     ... parts = debug d2 d3 d4
       
  1976     ...
       
  1977     ... [d2]
       
  1978     ... recipe = recipes:mkdir
       
  1979     ... path = data2
       
  1980     ...
       
  1981     ... [d3]
       
  1982     ... recipe = recipes:mkdir
       
  1983     ... path = data3
       
  1984     ...
       
  1985     ... [d4]
       
  1986     ... recipe = recipes:mkdir
       
  1987     ... path = ${d2:path}-extra
       
  1988     ...
       
  1989     ... [debug]
       
  1990     ... recipe = recipes:debug
       
  1991     ... x = 1
       
  1992     ... """)
       
  1993 
       
  1994 and run the buildout specifying just d3 and d4:
       
  1995 
       
  1996     >>> print system(buildout+' install d3 d4'),
       
  1997     Develop: '/sample-buildout/recipes'
       
  1998     Uninstalling d3.
       
  1999     Installing d3.
       
  2000     d3: Creating directory data3
       
  2001     Installing d4.
       
  2002     d4: Creating directory data2-extra
       
  2003 
       
  2004     >>> ls(sample_buildout)
       
  2005     -  .installed.cfg
       
  2006     -  b1.cfg
       
  2007     -  b2.cfg
       
  2008     -  base.cfg
       
  2009     d  bin
       
  2010     -  buildout.cfg
       
  2011     d  d1
       
  2012     d  d2
       
  2013     d  data2-extra
       
  2014     d  data3
       
  2015     d  demo
       
  2016     d  develop-eggs
       
  2017     d  eggs
       
  2018     d  parts
       
  2019     d  recipes
       
  2020 
       
  2021 Only the d3 and d4 recipes ran.  d3 was removed and data3 and data2-extra
       
  2022 were created.
       
  2023 
       
  2024 The .installed.cfg is only updated for the recipes that ran:
       
  2025 
       
  2026     >>> cat(sample_buildout, '.installed.cfg')
       
  2027     ... # doctest: +NORMALIZE_WHITESPACE
       
  2028     [buildout]
       
  2029     installed_develop_eggs = /sample-buildout/develop-eggs/recipes.egg-link
       
  2030     parts = debug d1 d2 d3 d4
       
  2031     <BLANKLINE>
       
  2032     [debug]
       
  2033     __buildout_installed__ =
       
  2034     __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg==
       
  2035     recipe = recipes:debug
       
  2036     <BLANKLINE>
       
  2037     [d1]
       
  2038     __buildout_installed__ = /sample-buildout/d1
       
  2039     __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg==
       
  2040     path = /sample-buildout/d1
       
  2041     recipe = recipes:mkdir
       
  2042     <BLANKLINE>
       
  2043     [d2]
       
  2044     __buildout_installed__ = /sample-buildout/d2
       
  2045     __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg==
       
  2046     path = /sample-buildout/d2
       
  2047     recipe = recipes:mkdir
       
  2048     <BLANKLINE>
       
  2049     [d3]
       
  2050     __buildout_installed__ = /sample-buildout/data3
       
  2051     __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg==
       
  2052     path = /sample-buildout/data3
       
  2053     recipe = recipes:mkdir
       
  2054     <BLANKLINE>
       
  2055     [d4]
       
  2056     __buildout_installed__ = /sample-buildout/data2-extra
       
  2057     __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg==
       
  2058     path = /sample-buildout/data2-extra
       
  2059     recipe = recipes:mkdir
       
  2060 
       
  2061 Note that the installed data for debug, d1, and d2 haven't changed,
       
  2062 because we didn't install those parts and that the d1 and d2
       
  2063 directories are still there.
       
  2064 
       
  2065 Now, if we run the buildout without the install command:
       
  2066 
       
  2067     >>> print system(buildout),
       
  2068     Develop: '/sample-buildout/recipes'
       
  2069     Uninstalling d2.
       
  2070     Uninstalling d1.
       
  2071     Uninstalling debug.
       
  2072     Installing debug.
       
  2073     recipe recipes:debug
       
  2074     x 1
       
  2075     Installing d2.
       
  2076     d2: Creating directory data2
       
  2077     Updating d3.
       
  2078     Updating d4.
       
  2079 
       
  2080 We see the output of the debug recipe and that data2 was created.  We
       
  2081 also see that d1 and d2 have gone away:
       
  2082 
       
  2083     >>> ls(sample_buildout)
       
  2084     -  .installed.cfg
       
  2085     -  b1.cfg
       
  2086     -  b2.cfg
       
  2087     -  base.cfg
       
  2088     d  bin
       
  2089     -  buildout.cfg
       
  2090     d  data2
       
  2091     d  data2-extra
       
  2092     d  data3
       
  2093     d  demo
       
  2094     d  develop-eggs
       
  2095     d  eggs
       
  2096     d  parts
       
  2097     d  recipes
       
  2098 
       
  2099 Alternate directory and file locations
       
  2100 --------------------------------------
       
  2101 
       
  2102 The buildout normally puts the bin, eggs, and parts directories in the
       
  2103 directory in the directory containing the configuration file. You can
       
  2104 provide alternate locations, and even names for these directories.
       
  2105 
       
  2106     >>> alt = tmpdir('sample-alt')
       
  2107 
       
  2108     >>> write(sample_buildout, 'buildout.cfg',
       
  2109     ... """
       
  2110     ... [buildout]
       
  2111     ... develop = recipes
       
  2112     ... parts =
       
  2113     ... develop-eggs-directory = %(developbasket)s
       
  2114     ... eggs-directory = %(basket)s
       
  2115     ... bin-directory = %(scripts)s
       
  2116     ... parts-directory = %(work)s
       
  2117     ... """ % dict(
       
  2118     ...    developbasket = os.path.join(alt, 'developbasket'),
       
  2119     ...    basket = os.path.join(alt, 'basket'),
       
  2120     ...    scripts = os.path.join(alt, 'scripts'),
       
  2121     ...    work = os.path.join(alt, 'work'),
       
  2122     ... ))
       
  2123 
       
  2124     >>> print system(buildout),
       
  2125     Creating directory '/sample-alt/scripts'.
       
  2126     Creating directory '/sample-alt/work'.
       
  2127     Creating directory '/sample-alt/basket'.
       
  2128     Creating directory '/sample-alt/developbasket'.
       
  2129     Develop: '/sample-buildout/recipes'
       
  2130     Uninstalling d4.
       
  2131     Uninstalling d3.
       
  2132     Uninstalling d2.
       
  2133     Uninstalling debug.
       
  2134 
       
  2135     >>> ls(alt)
       
  2136     d  basket
       
  2137     d  developbasket
       
  2138     d  scripts
       
  2139     d  work
       
  2140 
       
  2141     >>> ls(alt, 'developbasket')
       
  2142     -  recipes.egg-link
       
  2143 
       
  2144 You can also specify an alternate buildout directory:
       
  2145 
       
  2146     >>> rmdir(alt)
       
  2147     >>> alt = tmpdir('sample-alt')
       
  2148 
       
  2149     >>> write(sample_buildout, 'buildout.cfg',
       
  2150     ... """
       
  2151     ... [buildout]
       
  2152     ... directory = %(alt)s
       
  2153     ... develop = %(recipes)s
       
  2154     ... parts =
       
  2155     ... """ % dict(
       
  2156     ...    alt=alt,
       
  2157     ...    recipes=os.path.join(sample_buildout, 'recipes'),
       
  2158     ...    ))
       
  2159 
       
  2160     >>> print system(buildout),
       
  2161     Creating directory '/sample-alt/bin'.
       
  2162     Creating directory '/sample-alt/parts'.
       
  2163     Creating directory '/sample-alt/eggs'.
       
  2164     Creating directory '/sample-alt/develop-eggs'.
       
  2165     Develop: '/sample-buildout/recipes'
       
  2166 
       
  2167     >>> ls(alt)
       
  2168     -  .installed.cfg
       
  2169     d  bin
       
  2170     d  develop-eggs
       
  2171     d  eggs
       
  2172     d  parts
       
  2173 
       
  2174     >>> ls(alt, 'develop-eggs')
       
  2175     -  recipes.egg-link
       
  2176 
       
  2177 Logging control
       
  2178 ---------------
       
  2179 
       
  2180 Three buildout options are used to control logging:
       
  2181 
       
  2182 log-level
       
  2183    specifies the log level
       
  2184 
       
  2185 verbosity
       
  2186    adjusts the log level
       
  2187 
       
  2188 log-format
       
  2189    allows an alternate logging for mat to be specified
       
  2190 
       
  2191 We've already seen the log level and verbosity.  Let's look at an example
       
  2192 of changing the format:
       
  2193 
       
  2194     >>> write(sample_buildout, 'buildout.cfg',
       
  2195     ... """
       
  2196     ... [buildout]
       
  2197     ... develop = recipes
       
  2198     ... parts =
       
  2199     ... log-level = 25
       
  2200     ... verbosity = 5
       
  2201     ... log-format = %(levelname)s %(message)s
       
  2202     ... """)
       
  2203 
       
  2204 Here, we've changed the format to include the log-level name, rather
       
  2205 than the logger name.
       
  2206 
       
  2207 We've also illustrated, with a contrived example, that the log level
       
  2208 can be a numeric value and that the verbosity can be specified in the
       
  2209 configuration file.  Because the verbosity is subtracted from the log
       
  2210 level, we get a final log level of 20, which is the INFO level.
       
  2211 
       
  2212     >>> print system(buildout),
       
  2213     INFO Develop: '/sample-buildout/recipes'
       
  2214 
       
  2215 Predefined buildout options
       
  2216 ---------------------------
       
  2217 
       
  2218 Buildouts have a number of predefined options that recipes can use
       
  2219 and that users can override in their configuration files.  To see
       
  2220 these, we'll run a minimal buildout configuration with a debug logging
       
  2221 level.  One of the features of debug logging is that the configuration
       
  2222 database is shown.
       
  2223 
       
  2224     >>> write(sample_buildout, 'buildout.cfg',
       
  2225     ... """
       
  2226     ... [buildout]
       
  2227     ... parts =
       
  2228     ... """)
       
  2229 
       
  2230     >>> print system(buildout+' -vv'), # doctest: +NORMALIZE_WHITESPACE
       
  2231     Installing 'zc.buildout', 'setuptools'.
       
  2232     We have a develop egg: zc.buildout X.X.
       
  2233     We have the best distribution that satisfies 'setuptools'.
       
  2234     Picked: setuptools = V.V
       
  2235     <BLANKLINE>
       
  2236     Configuration data:
       
  2237     [buildout]
       
  2238     accept-buildout-test-releases = false
       
  2239     allow-hosts = *
       
  2240     allow-picked-versions = true
       
  2241     allowed-eggs-from-site-packages = *
       
  2242     bin-directory = /sample-buildout/bin
       
  2243     develop-eggs-directory = /sample-buildout/develop-eggs
       
  2244     directory = /sample-buildout
       
  2245     eggs-directory = /sample-buildout/eggs
       
  2246     exec-sitecustomize = true
       
  2247     executable = python
       
  2248     find-links =
       
  2249     include-site-packages = true
       
  2250     install-from-cache = false
       
  2251     installed = /sample-buildout/.installed.cfg
       
  2252     log-format =
       
  2253     log-level = INFO
       
  2254     newest = true
       
  2255     offline = false
       
  2256     parts =
       
  2257     parts-directory = /sample-buildout/parts
       
  2258     prefer-final = false
       
  2259     python = buildout
       
  2260     relative-paths = false
       
  2261     socket-timeout =
       
  2262     unzip = false
       
  2263     use-dependency-links = true
       
  2264     verbosity = 20
       
  2265     <BLANKLINE>
       
  2266 
       
  2267 All of these options can be overridden by configuration files or by
       
  2268 command-line assignments.  We've discussed most of these options
       
  2269 already, but let's review them and touch on some we haven't discussed:
       
  2270 
       
  2271 allowed-eggs-from-site-packages
       
  2272     Sometimes you need or want to control what eggs from site-packages are
       
  2273     used. The allowed-eggs-from-site-packages option allows you to specify a
       
  2274     whitelist of project names that may be included from site-packages.  You
       
  2275     can use globs to specify the value.  It defaults to a single value of '*',
       
  2276     indicating that any package may come from site-packages.
       
  2277 
       
  2278     Here's a usage example::
       
  2279 
       
  2280         [buildout]
       
  2281         ...
       
  2282 
       
  2283         allowed-eggs-from-site-packages =
       
  2284             demo
       
  2285             bigdemo
       
  2286             zope.*
       
  2287 
       
  2288     This option interacts with the ``include-site-packages`` option in the
       
  2289     following ways.
       
  2290 
       
  2291     If ``include-site-packages`` is true, then
       
  2292     ``allowed-eggs-from-site-packages`` filters what eggs from site-packages
       
  2293     may be chosen.  Therefore, if ``allowed-eggs-from-site-packages`` is an
       
  2294     empty list, then no eggs from site-packages are chosen, but site-packages
       
  2295     will still be included at the end of path lists.
       
  2296 
       
  2297     If ``include-site-packages`` is false, the value of
       
  2298     ``allowed-eggs-from-site-packages`` is irrelevant.
       
  2299 
       
  2300     See the ``include-site-packages`` description for more information.
       
  2301 
       
  2302 bin-directory
       
  2303    The directory path where scripts are written.  This can be a
       
  2304    relative path, which is interpreted relative to the directory
       
  2305    option.
       
  2306 
       
  2307 develop-eggs-directory
       
  2308    The directory path where development egg links are created for software
       
  2309    being created in the local project.  This can be a relative path,
       
  2310    which is interpreted relative to the directory option.
       
  2311 
       
  2312 directory
       
  2313    The buildout directory.  This is the base for other buildout file
       
  2314    and directory locations, when relative locations are used.
       
  2315 
       
  2316 eggs-directory
       
  2317    The directory path where downloaded eggs are put.  It is common to share
       
  2318    this directory across buildouts. Eggs in this directory should
       
  2319    *never* be modified.  This can be a relative path, which is
       
  2320    interpreted relative to the directory option.
       
  2321 
       
  2322 exec-sitecustomize
       
  2323     Normally the Python's real sitecustomize module is processed.
       
  2324     If you do not want it to be processed in order to increase the
       
  2325     repeatability of your buildout, set this value to 'false'.  This will
       
  2326     be honored irrespective of the setting for include-site-packages.
       
  2327     This option will be honored by some recipes and not others.
       
  2328     z3c.recipe.scripts honors this and zc.recipe.egg does not, for
       
  2329     instance.
       
  2330 
       
  2331 executable
       
  2332    The Python executable used to run the buildout.  See the python
       
  2333    option below.
       
  2334 
       
  2335 include-site-packages
       
  2336     You can choose not to have the site-packages of the underlying Python
       
  2337     available to your script or interpreter, in addition to the packages
       
  2338     from your eggs.  This can increase repeatability for your buildout.
       
  2339     This option will be better used by some recipes than others.
       
  2340     z3c.recipe.scripts honors this fully and zc.recipe.egg only
       
  2341     partially, for instance.
       
  2342 
       
  2343 installed
       
  2344    The file path where information about the results of the previous
       
  2345    buildout run is written.  This can be a relative path, which is
       
  2346    interpreted relative to the directory option.  This file provides
       
  2347    an inventory of installed parts with information needed to decide
       
  2348    which if any parts need to be uninstalled.
       
  2349 
       
  2350 log-format
       
  2351    The format used for logging messages.
       
  2352 
       
  2353 log-level
       
  2354    The log level before verbosity adjustment
       
  2355 
       
  2356 parts
       
  2357    A white space separated list of parts to be installed.
       
  2358 
       
  2359 parts-directory
       
  2360    A working directory that parts can used to store data.
       
  2361 
       
  2362 python
       
  2363    The name of a section containing information about the default
       
  2364    Python interpreter.  Recipes that need a installation
       
  2365    typically have options to tell them which Python installation to
       
  2366    use.  By convention, if a section-specific option isn't used, the
       
  2367    option is looked for in the buildout section.  The option must
       
  2368    point to a section with an executable option giving the path to a
       
  2369    Python executable.  By default, the buildout section defines the
       
  2370    default Python as the Python used to run the buildout.
       
  2371 
       
  2372 relative-paths
       
  2373     The paths generated by zc.buildout are absolute by default, and this
       
  2374     option is ``false``.  However, if you set this value to be ``true``,
       
  2375     bin/buildout will be generated with code that makes the paths relative.
       
  2376     Some recipes, such as zc.recipe.egg and z3c.recipe.scripts, honor this
       
  2377     value as well.
       
  2378 
       
  2379 unzip
       
  2380     By default, zc.buildout doesn't unzip zip-safe eggs ("unzip = false").
       
  2381     This follows the policy followed by setuptools itself.  Experience shows
       
  2382     this policy to to be inconvenient.  Zipped eggs make debugging more
       
  2383     difficult and often import more slowly.  You can include an unzip option in
       
  2384     the buildout section to change the default unzipping policy ("unzip =
       
  2385     true").
       
  2386 
       
  2387 use-dependency-links
       
  2388     By default buildout will obey the setuptools dependency_links metadata
       
  2389     when it looks for dependencies. This behavior can be controlled with
       
  2390     the use-dependency-links buildout option::
       
  2391 
       
  2392       [buildout]
       
  2393       ...
       
  2394       use-dependency-links = false
       
  2395 
       
  2396     The option defaults to true. If you set it to false, then dependency
       
  2397     links are only looked for in the locations specified by find-links.
       
  2398 
       
  2399 verbosity
       
  2400    A log-level adjustment.  Typically, this is set via the -q and -v
       
  2401    command-line options.
       
  2402 
       
  2403 
       
  2404 Creating new buildouts and bootstrapping
       
  2405 ----------------------------------------
       
  2406 
       
  2407 If zc.buildout is installed, you can use it to create a new buildout
       
  2408 with it's own local copies of zc.buildout and setuptools and with
       
  2409 local buildout scripts.
       
  2410 
       
  2411     >>> sample_bootstrapped = tmpdir('sample-bootstrapped')
       
  2412 
       
  2413     >>> print system(buildout
       
  2414     ...              +' -c'+os.path.join(sample_bootstrapped, 'setup.cfg')
       
  2415     ...              +' init'),
       
  2416     Creating '/sample-bootstrapped/setup.cfg'.
       
  2417     Creating directory '/sample-bootstrapped/bin'.
       
  2418     Creating directory '/sample-bootstrapped/parts'.
       
  2419     Creating directory '/sample-bootstrapped/eggs'.
       
  2420     Creating directory '/sample-bootstrapped/develop-eggs'.
       
  2421     Generated script '/sample-bootstrapped/bin/buildout'.
       
  2422 
       
  2423 Note that a basic setup.cfg was created for us.
       
  2424 
       
  2425     >>> ls(sample_bootstrapped)
       
  2426     d  bin
       
  2427     d  develop-eggs
       
  2428     d  eggs
       
  2429     d  parts
       
  2430     -  setup.cfg
       
  2431 
       
  2432     >>> ls(sample_bootstrapped, 'bin')
       
  2433     -  buildout
       
  2434 
       
  2435     >>> _ = (ls(sample_bootstrapped, 'eggs'),
       
  2436     ...      ls(sample_bootstrapped, 'develop-eggs'))
       
  2437     -  setuptools-0.6-py2.3.egg
       
  2438     -  zc.buildout-1.0-py2.3.egg
       
  2439 
       
  2440 (We list both the eggs and develop-eggs directories because the
       
  2441 buildout or setuptools egg could be installed in the develop-eggs
       
  2442 directory if the original buildout had develop eggs for either
       
  2443 buildout or setuptools.)
       
  2444 
       
  2445 If relative-paths is ``true``, the buildout script uses relative paths.
       
  2446 
       
  2447     >>> write(sample_bootstrapped, 'setup.cfg',
       
  2448     ... '''
       
  2449     ... [buildout]
       
  2450     ... relative-paths = true
       
  2451     ... parts =
       
  2452     ... ''')
       
  2453 
       
  2454     >>> print system(buildout
       
  2455     ...              +' -c'+os.path.join(sample_bootstrapped, 'setup.cfg')
       
  2456     ...              +' bootstrap'),
       
  2457     Generated script '/sample-bootstrapped/bin/buildout'.
       
  2458 
       
  2459     >>> buildout_script = join(sample_bootstrapped, 'bin', 'buildout')
       
  2460     >>> import sys
       
  2461     >>> if sys.platform.startswith('win'):
       
  2462     ...     buildout_script += '-script.py'
       
  2463     >>> print open(buildout_script).read() # doctest: +ELLIPSIS
       
  2464     #!... -S
       
  2465     <BLANKLINE>
       
  2466     import os
       
  2467     <BLANKLINE>
       
  2468     join = os.path.join
       
  2469     base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))
       
  2470     base = os.path.dirname(base)
       
  2471     <BLANKLINE>
       
  2472     import sys
       
  2473     sys.path[0:0] = [
       
  2474         join(base, 'parts/buildout'),
       
  2475         ]
       
  2476     <BLANKLINE>
       
  2477     <BLANKLINE>
       
  2478     import os
       
  2479     path = sys.path[0]
       
  2480     if os.environ.get('PYTHONPATH'):
       
  2481         path = os.pathsep.join([path, os.environ['PYTHONPATH']])
       
  2482     os.environ['BUILDOUT_ORIGINAL_PYTHONPATH'] = os.environ.get('PYTHONPATH', '')
       
  2483     os.environ['PYTHONPATH'] = path
       
  2484     import site # imports custom buildout-generated site.py
       
  2485     <BLANKLINE>
       
  2486     import zc.buildout.buildout
       
  2487     <BLANKLINE>
       
  2488     if __name__ == '__main__':
       
  2489         zc.buildout.buildout.main()
       
  2490     <BLANKLINE>
       
  2491 
       
  2492 
       
  2493 Note that, in the above two examples, the buildout script was installed
       
  2494 but not run.  To run the buildout, we'd have to run the installed
       
  2495 buildout script.
       
  2496 
       
  2497 If we have an existing buildout that already has a buildout.cfg, we'll
       
  2498 normally use the bootstrap command instead of init.  It will complain
       
  2499 if there isn't a configuration file:
       
  2500 
       
  2501     >>> sample_bootstrapped2 = tmpdir('sample-bootstrapped2')
       
  2502 
       
  2503     >>> print system(buildout
       
  2504     ...              +' -c'+os.path.join(sample_bootstrapped2, 'setup.cfg')
       
  2505     ...              +' bootstrap'),
       
  2506     While:
       
  2507       Initializing.
       
  2508     Error: Couldn't open /sample-bootstrapped2/setup.cfg
       
  2509 
       
  2510     >>> write(sample_bootstrapped2, 'setup.cfg',
       
  2511     ... """
       
  2512     ... [buildout]
       
  2513     ... parts =
       
  2514     ... """)
       
  2515 
       
  2516     >>> print system(buildout
       
  2517     ...              +' -c'+os.path.join(sample_bootstrapped2, 'setup.cfg')
       
  2518     ...              +' bootstrap'),
       
  2519     Creating directory '/sample-bootstrapped2/bin'.
       
  2520     Creating directory '/sample-bootstrapped2/parts'.
       
  2521     Creating directory '/sample-bootstrapped2/eggs'.
       
  2522     Creating directory '/sample-bootstrapped2/develop-eggs'.
       
  2523     Generated script '/sample-bootstrapped2/bin/buildout'.
       
  2524 
       
  2525 
       
  2526 Newest and Offline Modes
       
  2527 ------------------------
       
  2528 
       
  2529 By default buildout and recipes will try to find the newest versions
       
  2530 of distributions needed to satisfy requirements.  This can be very
       
  2531 time consuming, especially when incrementally working on setting up a
       
  2532 buildout or working on a recipe.  The buildout newest option can be
       
  2533 used to to suppress this.  If the newest option is set to false, then
       
  2534 new distributions won't be sought if an installed distribution meets
       
  2535 requirements.  The newest option can be set to false using the -N
       
  2536 command-line option.
       
  2537 
       
  2538 The offline option goes a bit further.  If the buildout offline option
       
  2539 is given a value of "true", the buildout and recipes that are aware of
       
  2540 the option will avoid doing network access.  This is handy when
       
  2541 running the buildout when not connected to the internet.  It also
       
  2542 makes buildouts run much faster. This option is typically set using
       
  2543 the buildout -o option.
       
  2544 
       
  2545 Preferring Final Releases
       
  2546 -------------------------
       
  2547 
       
  2548 Currently, when searching for new releases of your project's
       
  2549 dependencies, the newest available release is used.  This isn't usually
       
  2550 ideal, as you may get a development release or alpha releases not ready
       
  2551 to be widely used. You can request that final releases be preferred
       
  2552 using the ``prefer-final`` option in the buildout section::
       
  2553 
       
  2554   [buildout]
       
  2555   ...
       
  2556   prefer-final = true
       
  2557 
       
  2558 When the ``prefer-final`` option is set to true, then when searching for
       
  2559 new releases, final releases are preferred.  If there are final
       
  2560 releases that satisfy distribution requirements, then those releases
       
  2561 are used even if newer non-final releases are available.
       
  2562 
       
  2563 In buildout version 2, all final releases will be preferred by
       
  2564 default--that is ``prefer-final`` will also default to 'true'. You will
       
  2565 then need to use a 'false' value for ``prefer-final`` to get the newest
       
  2566 releases.
       
  2567 
       
  2568 A separate option controls the behavior of the build system itself.
       
  2569 When buildout looks for recipes, extensions, and for updates to itself,
       
  2570 it does prefer final releases by default, as of the 1.5.0 release.  The
       
  2571 ``accept-buildout-test-releases`` option will let you override this behavior.
       
  2572 However, it is typically changed by the --accept-buildout-test-releases
       
  2573 option to the bootstrap script, since bootstrapping is the first step to
       
  2574 selecting a buildout.
       
  2575 
       
  2576 Finding distributions
       
  2577 ---------------------
       
  2578 
       
  2579 By default, buildout searches the Python Package Index when looking
       
  2580 for distributions. You can, instead, specify your own index to search
       
  2581 using the `index` option::
       
  2582 
       
  2583   [buildout]
       
  2584   ...
       
  2585   index = http://index.example.com/
       
  2586 
       
  2587 This index, or the default of http://pypi.python.org/simple/ if no
       
  2588 index is specified, will always be searched for distributions unless
       
  2589 running buildout with options that prevent searching for
       
  2590 distributions. The latest version of the distribution that meets the
       
  2591 requirements of the buildout will always be used.
       
  2592 
       
  2593 You can also specify more locations to search for distributions using
       
  2594 the `find-links` option. All locations specified will be searched for
       
  2595 distributions along with the package index as described before.
       
  2596 
       
  2597 Locations can be urls::
       
  2598 
       
  2599   [buildout]
       
  2600   ...
       
  2601   find-links = http://download.zope.org/distribution/
       
  2602 
       
  2603 They can also be directories on disk::
       
  2604 
       
  2605   [buildout]
       
  2606   ...
       
  2607   find-links = /some/path
       
  2608 
       
  2609 Finally, they can also be direct paths to distributions::
       
  2610 
       
  2611   [buildout]
       
  2612   ...
       
  2613   find-links = /some/path/someegg-1.0.0-py2.3.egg
       
  2614 
       
  2615 Any number of locations can be specified in the `find-links` option::
       
  2616 
       
  2617   [buildout]
       
  2618   ...
       
  2619   find-links =
       
  2620       http://download.zope.org/distribution/
       
  2621       /some/otherpath
       
  2622       /some/path/someegg-1.0.0-py2.3.egg
       
  2623 
       
  2624 Dependency links
       
  2625 ----------------
       
  2626 
       
  2627 By default buildout will obey the setuptools dependency_links metadata
       
  2628 when it looks for dependencies. This behavior can be controlled with
       
  2629 the use-dependency-links buildout option::
       
  2630 
       
  2631   [buildout]
       
  2632   ...
       
  2633   use-dependency-links = false
       
  2634 
       
  2635 The option defaults to true. If you set it to false, then dependency
       
  2636 links are only looked for in the locations specified by find-links.
       
  2637 
       
  2638 Controlling the installation database
       
  2639 -------------------------------------
       
  2640 
       
  2641 The buildout installed option is used to specify the file used to save
       
  2642 information on installed parts.  This option is initialized to
       
  2643 ".installed.cfg", but it can be overridden in the configuration file
       
  2644 or on the command line:
       
  2645 
       
  2646     >>> write('buildout.cfg',
       
  2647     ... """
       
  2648     ... [buildout]
       
  2649     ... develop = recipes
       
  2650     ... parts = debug
       
  2651     ...
       
  2652     ... [debug]
       
  2653     ... recipe = recipes:debug
       
  2654     ... """)
       
  2655 
       
  2656     >>> print system(buildout+' buildout:installed=inst.cfg'),
       
  2657     Develop: '/sample-buildout/recipes'
       
  2658     Installing debug.
       
  2659     recipe recipes:debug
       
  2660 
       
  2661     >>> ls(sample_buildout)
       
  2662     -  b1.cfg
       
  2663     -  b2.cfg
       
  2664     -  base.cfg
       
  2665     d  bin
       
  2666     -  buildout.cfg
       
  2667     d  demo
       
  2668     d  develop-eggs
       
  2669     d  eggs
       
  2670     -  inst.cfg
       
  2671     d  parts
       
  2672     d  recipes
       
  2673 
       
  2674 The installation database can be disabled by supplying an empty
       
  2675 buildout installed option:
       
  2676 
       
  2677     >>> os.remove('inst.cfg')
       
  2678     >>> print system(buildout+' buildout:installed='),
       
  2679     Develop: '/sample-buildout/recipes'
       
  2680     Installing debug.
       
  2681     recipe recipes:debug
       
  2682 
       
  2683     >>> ls(sample_buildout)
       
  2684     -  b1.cfg
       
  2685     -  b2.cfg
       
  2686     -  base.cfg
       
  2687     d  bin
       
  2688     -  buildout.cfg
       
  2689     d  demo
       
  2690     d  develop-eggs
       
  2691     d  eggs
       
  2692     d  parts
       
  2693     d  recipes
       
  2694 
       
  2695 
       
  2696 Note that there will be no installation database if there are no parts:
       
  2697 
       
  2698     >>> write('buildout.cfg',
       
  2699     ... """
       
  2700     ... [buildout]
       
  2701     ... parts =
       
  2702     ... """)
       
  2703 
       
  2704     >>> print system(buildout+' buildout:installed=inst.cfg'),
       
  2705 
       
  2706     >>> ls(sample_buildout)
       
  2707     -  b1.cfg
       
  2708     -  b2.cfg
       
  2709     -  base.cfg
       
  2710     d  bin
       
  2711     -  buildout.cfg
       
  2712     d  demo
       
  2713     d  develop-eggs
       
  2714     d  eggs
       
  2715     d  parts
       
  2716     d  recipes
       
  2717 
       
  2718 Extensions
       
  2719 ----------
       
  2720 
       
  2721 A feature allows code to be loaded and run after
       
  2722 configuration files have been read but before the buildout has begun
       
  2723 any processing.  The intent is to allow special plugins such as
       
  2724 urllib2 request handlers to be loaded.
       
  2725 
       
  2726 To load an extension, we use the extensions option and list one or
       
  2727 more distribution requirements, on separate lines.  The distributions
       
  2728 named will be loaded and any ``zc.buildout.extension`` entry points found
       
  2729 will be called with the buildout as an argument.  When buildout
       
  2730 finishes processing, any ``zc.buildout.unloadextension`` entry points
       
  2731 found will be called with the buildout as an argument.
       
  2732 
       
  2733 Let's create a sample extension in our sample buildout created in the
       
  2734 previous section:
       
  2735 
       
  2736     >>> mkdir(sample_bootstrapped, 'demo')
       
  2737 
       
  2738     >>> write(sample_bootstrapped, 'demo', 'demo.py',
       
  2739     ... """
       
  2740     ... def ext(buildout):
       
  2741     ...     print 'ext', list(buildout)
       
  2742     ... def unload(buildout):
       
  2743     ...     print 'unload', list(buildout)
       
  2744     ... """)
       
  2745 
       
  2746     >>> write(sample_bootstrapped, 'demo', 'setup.py',
       
  2747     ... """
       
  2748     ... from setuptools import setup
       
  2749     ...
       
  2750     ... setup(
       
  2751     ...     name = "demo",
       
  2752     ...     entry_points = {
       
  2753     ...        'zc.buildout.extension': ['ext = demo:ext'],
       
  2754     ...        'zc.buildout.unloadextension': ['ext = demo:unload'],
       
  2755     ...        },
       
  2756     ...     )
       
  2757     ... """)
       
  2758 
       
  2759 Our extension just prints out the word 'demo', and lists the sections
       
  2760 found in the buildout passed to it.
       
  2761 
       
  2762 We'll update our buildout.cfg to list the demo directory as a develop
       
  2763 egg to be built:
       
  2764 
       
  2765     >>> write(sample_bootstrapped, 'buildout.cfg',
       
  2766     ... """
       
  2767     ... [buildout]
       
  2768     ... develop = demo
       
  2769     ... parts =
       
  2770     ... """)
       
  2771 
       
  2772     >>> os.chdir(sample_bootstrapped)
       
  2773     >>> print system(os.path.join(sample_bootstrapped, 'bin', 'buildout')),
       
  2774     Develop: '/sample-bootstrapped/demo'
       
  2775 
       
  2776 Now we can add the extensions option.  We were a bit tricky and ran
       
  2777 the buildout once with the demo develop egg defined but without the
       
  2778 extension option.  This is because extensions are loaded before the
       
  2779 buildout creates develop eggs. We needed to use a separate buildout
       
  2780 run to create the develop egg.  Normally, when eggs are loaded from
       
  2781 the network, we wouldn't need to do anything special.
       
  2782 
       
  2783     >>> write(sample_bootstrapped, 'buildout.cfg',
       
  2784     ... """
       
  2785     ... [buildout]
       
  2786     ... develop = demo
       
  2787     ... extensions = demo
       
  2788     ... parts =
       
  2789     ... """)
       
  2790 
       
  2791 We see that our extension is loaded and executed:
       
  2792 
       
  2793     >>> print system(os.path.join(sample_bootstrapped, 'bin', 'buildout')),
       
  2794     ext ['buildout']
       
  2795     Develop: '/sample-bootstrapped/demo'
       
  2796     unload ['buildout']
       
  2797 
       
  2798 Allow hosts
       
  2799 -----------
       
  2800 
       
  2801 On some environments the links visited by `zc.buildout` can be forbidden
       
  2802 by paranoiac firewalls. These URL might be on the chain of links
       
  2803 visited by `zc.buildout` wheter they are defined in the `find-links` option,
       
  2804 wheter they are defined by various eggs in their `url`, `download_url`,
       
  2805 `dependency_links` metadata.
       
  2806 
       
  2807 It is even harder to track that package_index works like a spider and
       
  2808 might visit links and go to other location.
       
  2809 
       
  2810 The `allow-hosts` option provides a way to prevent this, and
       
  2811 works exactly like the one provided in `easy_install`.
       
  2812 
       
  2813 You can provide a list of allowed host, together with wildcards::
       
  2814 
       
  2815     [buildout]
       
  2816     ...
       
  2817 
       
  2818     allow-hosts =
       
  2819         *.python.org
       
  2820         example.com
       
  2821 
       
  2822 All urls that does not match these hosts will not be visited.
       
  2823 
       
  2824 .. [#future_recipe_methods] In the future, additional methods may be
       
  2825        added. Older recipes with fewer methods will still be
       
  2826        supported.
       
  2827 
       
  2828 .. [#packaging_info] If we wanted to create a distribution from this
       
  2829        package, we would need specify much more information.  See the
       
  2830        `setuptools documentation
       
  2831        <http://peak.telecommunity.com/DevCenter/setuptools>`_.