|
1 Metadata-Version: 1.0 |
|
2 Name: zc.buildout |
|
3 Version: 1.5.2 |
|
4 Summary: System for managing development buildouts |
|
5 Home-page: http://pypi.python.org/pypi/zc.buildout |
|
6 Author: Jim Fulton |
|
7 Author-email: jim@zope.com |
|
8 License: ZPL 2.1 |
|
9 Description: ******** |
|
10 Buildout |
|
11 ******** |
|
12 |
|
13 .. contents:: |
|
14 |
|
15 The Buildout project provides support for creating applications, |
|
16 especially Python applications. It provides tools for assembling |
|
17 applications from multiple parts, Python or otherwise. An application |
|
18 may actually contain multiple programs, processes, and configuration |
|
19 settings. |
|
20 |
|
21 The word "buildout" refers to a description of a set of parts and the |
|
22 software to create and assemble them. It is often used informally to |
|
23 refer to an installed system based on a buildout definition. For |
|
24 example, if we are creating an application named "Foo", then "the Foo |
|
25 buildout" is the collection of configuration and application-specific |
|
26 software that allows an instance of the application to be created. We |
|
27 may refer to such an instance of the application informally as "a Foo |
|
28 buildout". |
|
29 |
|
30 To get a feel for some of the things you might use buildouts for, see |
|
31 the `Buildout examples`_. |
|
32 |
|
33 To lean more about using buildouts, see `Detailed Documentation`_. |
|
34 |
|
35 To see screencasts, talks, useful links and more documentation, visit |
|
36 the `Buildout website <http://www.buildout.org>`_. |
|
37 |
|
38 Recipes |
|
39 ******* |
|
40 |
|
41 Existing recipes include: |
|
42 |
|
43 `zc.recipe.egg <http://pypi.python.org/pypi/zc.recipe.egg>`_ |
|
44 The egg recipe installes one or more eggs, with their |
|
45 dependencies. It installs their console-script entry points with |
|
46 the needed eggs included in their paths. It is suitable for use with |
|
47 a "clean" Python: one without packages installed in site-packages. |
|
48 |
|
49 `z3c.recipe.scripts <http://pypi.python.org/pypi/z3c.recipe.scripts>`_ |
|
50 Like zc.recipe.egg, this recipe builds interpreter scripts and entry |
|
51 point scripts based on eggs. It can be used with a Python that has |
|
52 packages installed in site-packages, such as a system Python. The |
|
53 interpreter also has more features than the one offered by |
|
54 zc.recipe.egg. |
|
55 |
|
56 `zc.recipe.testrunner <http://pypi.python.org/pypi/zc.recipe.testrunner>`_ |
|
57 The testrunner egg creates a test runner script for one or |
|
58 more eggs. |
|
59 |
|
60 `zc.recipe.zope3checkout <http://pypi.python.org/pypi/zc.recipe.zope3checkout>`_ |
|
61 The zope3checkout recipe installs a Zope 3 checkout into a |
|
62 buildout. |
|
63 |
|
64 `zc.recipe.zope3instance <http://pypi.python.org/pypi/zc.recipe.zope3instance>`_ |
|
65 The zope3instance recipe sets up a Zope 3 instance. |
|
66 |
|
67 `zc.recipe.filestorage <http://pypi.python.org/pypi/zc.recipe.filestorage>`_ |
|
68 The filestorage recipe sets up a ZODB file storage for use in a |
|
69 Zope 3 instance created by the zope3instance recipe. |
|
70 |
|
71 Buildout examples |
|
72 ***************** |
|
73 |
|
74 Here are a few examples of what you can do with buildouts. We'll |
|
75 present these as a set of use cases. |
|
76 |
|
77 Try out an egg |
|
78 ============== |
|
79 |
|
80 Sometimes you want to try an egg (or eggs) that someone has released. |
|
81 You'd like to get a Python interpreter that lets you try things |
|
82 interactively or run sample scripts without having to do path |
|
83 manipulations. If you can and don't mind modifying your Python |
|
84 installation, you could use easy_install, otherwise, you could create |
|
85 a directory somewhere and create a buildout.cfg file in that directory |
|
86 containing:: |
|
87 |
|
88 [buildout] |
|
89 parts = mypython |
|
90 |
|
91 [mypython] |
|
92 recipe = zc.recipe.egg |
|
93 interpreter = mypython |
|
94 eggs = theegg |
|
95 |
|
96 where theegg is the name of the egg you want to try out. |
|
97 |
|
98 Run buildout in this directory. It will create a bin subdirectory |
|
99 that includes a mypython script. If you run mypython without any |
|
100 arguments you'll get an interactive interpreter with the egg in the |
|
101 path. If you run it with a script and script arguments, the script |
|
102 will run with the egg in its path. Of course, you can specify as many |
|
103 eggs as you want in the eggs option. |
|
104 |
|
105 If the egg provides any scripts (console_scripts entry points), those |
|
106 will be installed in your bin directory too. |
|
107 |
|
108 Work on a package |
|
109 ================= |
|
110 |
|
111 I often work on packages that are managed separately. They don't have |
|
112 scripts to be installed, but I want to be able to run their tests |
|
113 using the `zope.testing test runner |
|
114 <http://www.python.org/pypi/zope.testing>`_. In this kind of |
|
115 application, the program to be installed is the test runner. A good |
|
116 example of this is `zc.ngi <http://svn.zope.org/zc.ngi/trunk/>`_. |
|
117 |
|
118 Here I have a subversion project for the zc.ngi package. The software |
|
119 is in the src directory. The configuration file is very simple:: |
|
120 |
|
121 [buildout] |
|
122 develop = . |
|
123 parts = test |
|
124 |
|
125 [test] |
|
126 recipe = zc.recipe.testrunner |
|
127 eggs = zc.ngi |
|
128 |
|
129 I use the develop option to create a develop egg based on the current |
|
130 directory. I request a test script named "test" using the |
|
131 zc.recipe.testrunner recipe. In the section for the test script, I |
|
132 specify that I want to run the tests in the zc.ngi package. |
|
133 |
|
134 When I check out this project into a new sandbox, I run bootstrap.py |
|
135 to get setuptools and zc.buildout and to create bin/buildout. I run |
|
136 bin/buildout, which installs the test script, bin/test, which I can |
|
137 then use to run the tests. |
|
138 |
|
139 This is probably the most common type of buildout. |
|
140 |
|
141 If I need to run a previous version of zc.buildout, I use the |
|
142 `--version` option of the bootstrap.py script:: |
|
143 |
|
144 $ python bootstrap.py --version 1.1.3 |
|
145 |
|
146 The `zc.buildout project <http://svn.zope.org/zc.buildout/trunk>`_ |
|
147 is a slightly more complex example of this type of buildout. |
|
148 |
|
149 Install egg-based scripts |
|
150 ========================= |
|
151 |
|
152 A variation of the `Try out an egg`_ use case is to install scripts |
|
153 into your ~/bin directory (on Unix, of course). My ~/bin directory is |
|
154 a buildout with a configuration file that looks like:: |
|
155 |
|
156 |
|
157 [buildout] |
|
158 parts = foo bar |
|
159 bin-directory = . |
|
160 |
|
161 [foo] |
|
162 ... |
|
163 |
|
164 where foo and bar are packages with scripts that I want available. As |
|
165 I need new scripts, I can add additional sections. The bin-directory |
|
166 option specified that scripts should be installed into the current |
|
167 directory. |
|
168 |
|
169 Multi-program multi-machine systems |
|
170 =================================== |
|
171 |
|
172 Using an older prototype version of the buildout, we've build a number |
|
173 of systems involving multiple programs, databases, and machines. One |
|
174 typical example consists of: |
|
175 |
|
176 - Multiple Zope instances |
|
177 |
|
178 - Multiple ZEO servers |
|
179 |
|
180 - An LDAP server |
|
181 |
|
182 - Cache-invalidation and Mail delivery servers |
|
183 |
|
184 - Dozens of add-on packages |
|
185 |
|
186 - Multiple test runners |
|
187 |
|
188 - Multiple deployment modes, including dev, stage, and prod, |
|
189 with prod deployment over multiple servers |
|
190 |
|
191 Parts installed include: |
|
192 |
|
193 - Application software installs, including Zope, ZEO and LDAP |
|
194 software |
|
195 |
|
196 - Add-on packages |
|
197 |
|
198 - Bundles of configuration that define Zope, ZEO and LDAP instances |
|
199 |
|
200 - Utility scripts such as test runners, server-control |
|
201 scripts, cron jobs. |
|
202 |
|
203 Questions and Bug Reporting |
|
204 *************************** |
|
205 |
|
206 Please send questions and comments to the |
|
207 `distutils SIG mailing list <mailto://distutils-sig@python.org>`_. |
|
208 |
|
209 Report bugs using the `zc.buildout Launchpad Bug Tracker |
|
210 <https://launchpad.net/zc.buildout/+bugs>`_. |
|
211 |
|
212 System Python and zc.buildout 1.5 |
|
213 ********************************* |
|
214 |
|
215 The 1.5 line of zc.buildout introduced a number of changes. |
|
216 |
|
217 Problems |
|
218 ======== |
|
219 |
|
220 As usual, please send questions and comments to the `distutils SIG |
|
221 mailing list <mailto://distutils-sig@python.org>`_. Report bugs using |
|
222 the `zc.buildout Launchpad Bug Tracker |
|
223 <https://launchpad.net/zc.buildout/+bugs>`_. |
|
224 |
|
225 If problems are keeping you from your work, here's an easy way to |
|
226 revert to the old code temporarily: switch to a custom "emergency" |
|
227 bootstrap script, available from |
|
228 http://svn.zope.org/repos/main/zc.buildout/branches/1.4/bootstrap/bootstrap.py . |
|
229 |
|
230 This customized script will select zc.buildout 1.4.4 by default. |
|
231 zc.buildout 1.4.4 will not upgrade itself unless you explicitly specify |
|
232 a new version. It will also prefer older versions of zc.recipe.egg and |
|
233 some other common recipes. If you have trouble with other recipes, |
|
234 consider using a standard buildout "versions" section to specify older |
|
235 versions of these, as described in the Buildout documentation |
|
236 (http://pypi.python.org/pypi/zc.buildout#repeatable-buildouts-controlling-eggs-used). |
|
237 |
|
238 Working with a System Python |
|
239 ============================ |
|
240 |
|
241 While there are a number of new features available in zc.buildout 1.5, |
|
242 the biggest is that Buildout itself supports usage with a system Python. |
|
243 This can work if you follow a couple of simple rules. |
|
244 |
|
245 1. Use the new bootstrap.py (available from |
|
246 svn://svn.zope.org/repos/main/zc.buildout/trunk/bootstrap/bootstrap.py). |
|
247 |
|
248 2. Use buildout recipes that have been upgraded to work with zc.buildout 1.5 |
|
249 and higher. Specifically, they should use |
|
250 ``zc.buildout.easy_install.sitepackage_safe_scripts`` to generate |
|
251 their scripts, if any, rather than ``zc.buildout.easy_install.scripts``. |
|
252 See the `Recipes That Support a System Python`_ section below for more |
|
253 details on recipes that are available as of this writing, and |
|
254 `Updating Recipes to Support a System Python`_ for instructions on |
|
255 how to update a recipe. Note that you should generally only need to |
|
256 update recipes that generate scripts. |
|
257 |
|
258 You can then use ``include-site-packages = false`` and |
|
259 ``exec-sitecustomize = false`` buildout options to eliminate access to |
|
260 your Python's site packages and not execute its sitecustomize file, if |
|
261 it exists, respectively. |
|
262 |
|
263 Alternately, you can use the ``allowed-eggs-from-site-packages`` buildout |
|
264 option as a glob-aware whitelist of eggs that may come from site-packages. |
|
265 This value defaults to "*", accepting all eggs. |
|
266 |
|
267 It's important to note that recipes not upgraded for zc.buildout 1.5.0 |
|
268 should continue to work--just without internal support for a system Python. |
|
269 |
|
270 Using a system Python is inherently fragile. Using a clean, |
|
271 freshly-installed Python without customization in site-packages is more |
|
272 robust and repeatable. See some of the regression tests added to the |
|
273 1.5.0 line for the kinds of issues that you can encounter with a system |
|
274 Python, and see |
|
275 http://pypi.python.org/pypi/z3c.recipe.scripts#including-site-packages-and-sitecustomize |
|
276 for more discussion. |
|
277 |
|
278 However, using a system Python can be very convenient, and the |
|
279 zc.buildout code for this feature has been tested by many users already. |
|
280 Moreover, it has automated tests to exercise the problems that have been |
|
281 encountered and fixed. Many people rely on it. |
|
282 |
|
283 Recipes That Support a System Python |
|
284 ==================================== |
|
285 |
|
286 zc.recipe.egg continues to generate old-style scripts that are not safe |
|
287 for use with a system Python. This was done for backwards |
|
288 compatibility, because it is integral to so many buildouts and used as a |
|
289 dependency of so many other recipes. |
|
290 |
|
291 If you want to generate new-style scripts that do support system Python |
|
292 usage, use z3c.recipe.scripts instead |
|
293 (http://pypi.python.org/pypi/z3c.recipe.scripts). z3c.recipe.scripts has |
|
294 the same script and interpreter generation options as zc.recipe.egg, |
|
295 plus a few more for the new features mentioned above. In the simplest |
|
296 case, you should be able to simply change ``recipe = zc.recipe.egg`` to |
|
297 ``recipe = z3c.recipe.scripts`` in the pertinent sections of your |
|
298 buildout configuration and your generated scripts will work with a system |
|
299 Python. |
|
300 |
|
301 Other updated recipes include zc.recipe.testrunner 1.4.0 and |
|
302 z3c.recipe.tag 0.4.0. Others should be updated soon: see their change |
|
303 documents for details, or see `Updating Recipes to Support a System |
|
304 Python`_ for instructions on how to update recipes yourself. |
|
305 |
|
306 Templates for creating Python scripts with the z3c.recipe.filetemplate |
|
307 recipe can be easily changed to support a system Python. |
|
308 |
|
309 - If you don't care about supporting relative paths, simply using a |
|
310 generated interpreter with the eggs you want should be sufficient, as |
|
311 it was before. For instance, if the interpreter is named "py", use |
|
312 ``#!${buildout:bin-directory/py}`` or ``#!/usr/bin/env |
|
313 ${buildout:bin-directory/py}``). |
|
314 |
|
315 - If you do care about relative paths, (``relative-paths = true`` in |
|
316 your buildout configuration), then z3c.recipe.scripts does require a |
|
317 bit more changes, as is usual for the relative path support in that |
|
318 package. First, use z3c.recipe.scripts to generate a script or |
|
319 interpreter with the dependencies you want. This will create a |
|
320 directory in ``parts`` that has a site.py and sitecustomize.py. Then, |
|
321 begin your script as in the snippet below. The example assumes that |
|
322 the z3c.recipe.scripts generated were from a Buildout configuration |
|
323 section labeled "scripts": adjust accordingly. |
|
324 |
|
325 :: |
|
326 |
|
327 #!${buildout:executable} -S |
|
328 ${python-relative-path-setup} |
|
329 import sys |
|
330 sys.path.insert(0, ${scripts:parts-directory|path-repr}) |
|
331 import site |
|
332 |
|
333 Updating Recipes to Support a System Python |
|
334 =========================================== |
|
335 |
|
336 You should generally only need to update recipes that generate scripts. |
|
337 These recipes need to change from using ``zc.buildout.easy_install.scripts`` |
|
338 to be using ``zc.buildout.easy_install.sitepackage_safe_scripts``. |
|
339 The signatures of the two functions are different. Please compare:: |
|
340 |
|
341 def scripts( |
|
342 reqs, working_set, executable, dest, |
|
343 scripts=None, |
|
344 extra_paths=(), |
|
345 arguments='', |
|
346 interpreter=None, |
|
347 initialization='', |
|
348 relative_paths=False, |
|
349 ): |
|
350 |
|
351 def sitepackage_safe_scripts( |
|
352 dest, working_set, executable, site_py_dest, |
|
353 reqs=(), |
|
354 scripts=None, |
|
355 interpreter=None, |
|
356 extra_paths=(), |
|
357 initialization='', |
|
358 include_site_packages=False, |
|
359 exec_sitecustomize=False, |
|
360 relative_paths=False, |
|
361 script_arguments='', |
|
362 script_initialization='', |
|
363 ): |
|
364 |
|
365 In most cases, the arguments are merely reordered. The ``reqs`` |
|
366 argument is no longer required in order to make it easier to generate an |
|
367 interpreter alone. The ``arguments`` argument was renamed to |
|
368 ``script_arguments`` to clarify that it did not affect interpreter |
|
369 generation. |
|
370 |
|
371 The only new required argument is ``site_py_dest``. It must be the path |
|
372 to a directory in which the customized site.py and sitecustomize.py |
|
373 files will be written. A typical generation in a recipe will look like |
|
374 this. |
|
375 |
|
376 (In the recipe's __init__ method...) |
|
377 |
|
378 :: |
|
379 |
|
380 self.options = options |
|
381 b_options = buildout['buildout'] |
|
382 options['parts-directory'] = os.path.join( |
|
383 b_options['parts-directory'], self.name) |
|
384 |
|
385 (In the recipe's install method...) |
|
386 |
|
387 :: |
|
388 |
|
389 options = self.options |
|
390 generated = [] |
|
391 if not os.path.exists(options['parts-directory']): |
|
392 os.mkdir(options['parts-directory']) |
|
393 generated.append(options['parts-directory']) |
|
394 |
|
395 Then ``options['parts-directory']`` can be used for the ``site_py_dest`` |
|
396 value. |
|
397 |
|
398 If you want to support the other arguments (``include_site_packages``, |
|
399 ``exec_sitecustomize``, ``script_initialization``, as well as the |
|
400 ``allowed-eggs-from-site-packages`` option), you might want to look at |
|
401 some of the code in |
|
402 svn://svn.zope.org/repos/main/zc.buildout/trunk/z3c.recipe.scripts\_/src/z3c/recipe/scripts/scripts.py . |
|
403 You might even be able to adopt some of it by subclassing or delegating. |
|
404 The Scripts class in that file is the closest to what you might be used |
|
405 to from zc.recipe.egg. |
|
406 |
|
407 Important note for recipe authors: As of buildout 1.5.2, the code in |
|
408 recipes is *always run with the access to the site-packages as |
|
409 configured in the buildout section*. |
|
410 |
|
411 virtualenv |
|
412 ========== |
|
413 |
|
414 Using virtualenv (http://pypi.python.org/pypi/virtualenv) with the |
|
415 --no-site-packages option already provided a simple way of using a |
|
416 system Python. This is intended to continue to work, and some automated |
|
417 tests exist to demonstrate this. |
|
418 |
|
419 However, it is only supported to the degree that people have found it to |
|
420 work in the past. The existing Buildout tests for virtualenv are only |
|
421 for problems encountered previously. They are very far from |
|
422 comprehensive. |
|
423 |
|
424 Using Buildout with a system python has at least three advantages over |
|
425 using Buildout in conjunction with virtualenv. They may or may not be |
|
426 pertinent to your desired usage. |
|
427 |
|
428 - Unlike ``virtualenv --no-site-packages``, Buildout's support allows you |
|
429 to choose to let packages from your system Python be available to your |
|
430 software (see ``include-site-packages`` in |
|
431 http://pypi.python.org/pypi/z3c.recipe.scripts). |
|
432 |
|
433 You can even specify which eggs installed in your system Python can be |
|
434 allowed to fulfill some of your packages' dependencies (see |
|
435 ``allowed-eggs-from-site-packages`` in |
|
436 http://pypi.python.org/pypi/z3c.recipe.scripts). |
|
437 |
|
438 At the expense of some repeatability and platform dependency, this |
|
439 flexibility means that, for instance, you can rely on |
|
440 difficult-to-build eggs like lxml coming from your system Python. |
|
441 |
|
442 - Buildout's implementation has a full set of automated tests. |
|
443 |
|
444 - An integral Buildout implementation means fewer steps and fewer dependencies |
|
445 to work with a system Python. |
|
446 |
|
447 Detailed Documentation |
|
448 ********************** |
|
449 |
|
450 Buildouts |
|
451 ========= |
|
452 |
|
453 The word "buildout" refers to a description of a set of parts and the |
|
454 software to create and assemble them. It is often used informally to |
|
455 refer to an installed system based on a buildout definition. For |
|
456 example, if we are creating an application named "Foo", then "the Foo |
|
457 buildout" is the collection of configuration and application-specific |
|
458 software that allows an instance of the application to be created. We |
|
459 may refer to such an instance of the application informally as "a Foo |
|
460 buildout". |
|
461 |
|
462 This document describes how to define buildouts using buildout |
|
463 configuration files and recipes. There are three ways to set up the |
|
464 buildout software and create a buildout instance: |
|
465 |
|
466 1. Install the zc.buildout egg with easy_install and use the buildout |
|
467 script installed in a Python scripts area. |
|
468 |
|
469 2. Use the buildout bootstrap script to create a buildout that |
|
470 includes both the setuptools and zc.buildout eggs. This allows you |
|
471 to use the buildout software without modifying a Python install. |
|
472 The buildout script is installed into your buildout local scripts |
|
473 area. |
|
474 |
|
475 3. Use a buildout command from an already installed buildout to |
|
476 bootstrap a new buildout. (See the section on bootstraping later |
|
477 in this document.) |
|
478 |
|
479 Often, a software project will be managed in a software repository, |
|
480 such as a subversion repository, that includes some software source |
|
481 directories, buildout configuration files, and a copy of the buildout |
|
482 bootstrap script. To work on the project, one would check out the |
|
483 project from the repository and run the bootstrap script which |
|
484 installs setuptools and zc.buildout into the checkout as well as any |
|
485 parts defined. |
|
486 |
|
487 We have a sample buildout that we created using the bootstrap command |
|
488 of an existing buildout (method 3 above). It has the absolute minimum |
|
489 information. We have bin, develop-eggs, eggs and parts directories, |
|
490 and a configuration file: |
|
491 |
|
492 >>> ls(sample_buildout) |
|
493 d bin |
|
494 - buildout.cfg |
|
495 d develop-eggs |
|
496 d eggs |
|
497 d parts |
|
498 |
|
499 The bin directory contains scripts. |
|
500 |
|
501 >>> ls(sample_buildout, 'bin') |
|
502 - buildout |
|
503 |
|
504 >>> ls(sample_buildout, 'eggs') |
|
505 - setuptools-0.6-py2.4.egg |
|
506 - zc.buildout-1.0-py2.4.egg |
|
507 |
|
508 The develop-eggs directory is initially empty: |
|
509 |
|
510 >>> ls(sample_buildout, 'develop-eggs') |
|
511 |
|
512 The develop-eggs directory holds egg links for software being |
|
513 developed in the buildout. We separate develop-eggs and other eggs to |
|
514 allow eggs directories to be shared across multiple buildouts. For |
|
515 example, a common developer technique is to define a common eggs |
|
516 directory in their home that all non-develop eggs are stored in. This |
|
517 allows larger buildouts to be set up much more quickly and saves disk |
|
518 space. |
|
519 |
|
520 The parts directory just contains some helpers for the buildout script |
|
521 itself. |
|
522 |
|
523 >>> ls(sample_buildout, 'parts') |
|
524 d buildout |
|
525 |
|
526 The parts directory provides an area where recipes can install |
|
527 part data. For example, if we built a custom Python, we would |
|
528 install it in the part directory. Part data is stored in a |
|
529 sub-directory of the parts directory with the same name as the part. |
|
530 |
|
531 Buildouts are defined using configuration files. These are in the |
|
532 format defined by the Python ConfigParser module, with extensions |
|
533 that we'll describe later. By default, when a buildout is run, it |
|
534 looks for the file buildout.cfg in the directory where the buildout is |
|
535 run. |
|
536 |
|
537 The minimal configuration file has a buildout section that defines no |
|
538 parts: |
|
539 |
|
540 >>> cat(sample_buildout, 'buildout.cfg') |
|
541 [buildout] |
|
542 parts = |
|
543 |
|
544 A part is simply something to be created by a buildout. It can be |
|
545 almost anything, such as a Python package, a program, a directory, or |
|
546 even a configuration file. |
|
547 |
|
548 Recipes |
|
549 ------- |
|
550 |
|
551 A part is created by a recipe. Recipes are always installed as Python |
|
552 eggs. They can be downloaded from a package server, such as the |
|
553 Python Package Index, or they can be developed as part of a project |
|
554 using a "develop" egg. |
|
555 |
|
556 A develop egg is a special kind of egg that gets installed as an "egg |
|
557 link" that contains the name of a source directory. Develop eggs |
|
558 don't have to be packaged for distribution to be used and can be |
|
559 modified in place, which is especially useful while they are being |
|
560 developed. |
|
561 |
|
562 Let's create a recipe as part of the sample project. We'll create a |
|
563 recipe for creating directories. First, we'll create a recipes source |
|
564 directory for our local recipes: |
|
565 |
|
566 >>> mkdir(sample_buildout, 'recipes') |
|
567 |
|
568 and then we'll create a source file for our mkdir recipe: |
|
569 |
|
570 >>> write(sample_buildout, 'recipes', 'mkdir.py', |
|
571 ... """ |
|
572 ... import logging, os, zc.buildout |
|
573 ... |
|
574 ... class Mkdir: |
|
575 ... |
|
576 ... def __init__(self, buildout, name, options): |
|
577 ... self.name, self.options = name, options |
|
578 ... options['path'] = os.path.join( |
|
579 ... buildout['buildout']['directory'], |
|
580 ... options['path'], |
|
581 ... ) |
|
582 ... if not os.path.isdir(os.path.dirname(options['path'])): |
|
583 ... logging.getLogger(self.name).error( |
|
584 ... 'Cannot create %s. %s is not a directory.', |
|
585 ... options['path'], os.path.dirname(options['path'])) |
|
586 ... raise zc.buildout.UserError('Invalid Path') |
|
587 ... |
|
588 ... |
|
589 ... def install(self): |
|
590 ... path = self.options['path'] |
|
591 ... logging.getLogger(self.name).info( |
|
592 ... 'Creating directory %s', os.path.basename(path)) |
|
593 ... os.mkdir(path) |
|
594 ... return path |
|
595 ... |
|
596 ... def update(self): |
|
597 ... pass |
|
598 ... """) |
|
599 |
|
600 Currently, recipes must define 3 methods [#future_recipe_methods]_: |
|
601 |
|
602 - a constructor, |
|
603 |
|
604 - an install method, and |
|
605 |
|
606 - an update method. |
|
607 |
|
608 The constructor is responsible for updating a parts options to reflect |
|
609 data read from other sections. The buildout system keeps track of |
|
610 whether a part specification has changed. A part specification has |
|
611 changed if it's options, after adjusting for data read from other |
|
612 sections, has changed, or if the recipe has changed. Only the options |
|
613 for the part are considered. If data are read from other sections, |
|
614 then that information has to be reflected in the parts options. In |
|
615 the Mkdir example, the given path is interpreted relative to the |
|
616 buildout directory, and data from the buildout directory is read. The |
|
617 path option is updated to reflect this. If the directory option was |
|
618 changed in the buildout sections, we would know to update parts |
|
619 created using the mkdir recipe using relative path names. |
|
620 |
|
621 When buildout is run, it saves configuration data for installed parts |
|
622 in a file named ".installed.cfg". In subsequent runs, it compares |
|
623 part-configuration data stored in the .installed.cfg file and the |
|
624 part-configuration data loaded from the configuration files as |
|
625 modified by recipe constructors to decide if the configuration of a |
|
626 part has changed. If the configuration has changed, or if the recipe |
|
627 has changed, then the part is uninstalled and reinstalled. The |
|
628 buildout only looks at the part's options, so any data used to |
|
629 configure the part needs to be reflected in the part's options. It is |
|
630 the job of a recipe constructor to make sure that the options include |
|
631 all relevant data. |
|
632 |
|
633 Of course, parts are also uninstalled if they are no-longer used. |
|
634 |
|
635 The recipe defines a constructor that takes a buildout object, a part |
|
636 name, and an options dictionary. It saves them in instance attributes. |
|
637 If the path is relative, we'll interpret it as relative to the |
|
638 buildout directory. The buildout object passed in is a mapping from |
|
639 section name to a mapping of options for that section. The buildout |
|
640 directory is available as the directory option of the buildout |
|
641 section. We normalize the path and save it back into the options |
|
642 directory. |
|
643 |
|
644 The install method is responsible for creating the part. In this |
|
645 case, we need the path of the directory to create. We'll use a path |
|
646 option from our options dictionary. The install method logs what it's |
|
647 doing using the Python logging call. We return the path that we |
|
648 installed. If the part is uninstalled or reinstalled, then the path |
|
649 returned will be removed by the buildout machinery. A recipe install |
|
650 method is expected to return a string, or an iterable of strings |
|
651 containing paths to be removed if a part is uninstalled. For most |
|
652 recipes, this is all of the uninstall support needed. For more complex |
|
653 uninstallation scenarios use `Uninstall recipes`_. |
|
654 |
|
655 The update method is responsible for updating an already installed |
|
656 part. An empty method is often provided, as in this example, if parts |
|
657 can't be updated. An update method can return None, a string, or an |
|
658 iterable of strings. If a string or iterable of strings is returned, |
|
659 then the saved list of paths to be uninstalled is updated with the new |
|
660 information by adding any new files returned by the update method. |
|
661 |
|
662 We need to provide packaging information so that our recipe can be |
|
663 installed as a develop egg. The minimum information we need to specify |
|
664 [#packaging_info]_ is a name. For recipes, we also need to define the |
|
665 names of the recipe classes as entry points. Packaging information is |
|
666 provided via a setup.py script: |
|
667 |
|
668 >>> write(sample_buildout, 'recipes', 'setup.py', |
|
669 ... """ |
|
670 ... from setuptools import setup |
|
671 ... |
|
672 ... setup( |
|
673 ... name = "recipes", |
|
674 ... entry_points = {'zc.buildout': ['mkdir = mkdir:Mkdir']}, |
|
675 ... ) |
|
676 ... """) |
|
677 |
|
678 Our setup script defines an entry point. Entry points provide |
|
679 a way for an egg to define the services it provides. Here we've said |
|
680 that we define a zc.buildout entry point named mkdir. Recipe |
|
681 classes must be exposed as entry points in the zc.buildout group. we |
|
682 give entry points names within the group. |
|
683 |
|
684 We also need a README.txt for our recipes to avoid an annoying warning |
|
685 from distutils, on which setuptools and zc.buildout are based: |
|
686 |
|
687 >>> write(sample_buildout, 'recipes', 'README.txt', " ") |
|
688 |
|
689 Now let's update our buildout.cfg: |
|
690 |
|
691 >>> write(sample_buildout, 'buildout.cfg', |
|
692 ... """ |
|
693 ... [buildout] |
|
694 ... develop = recipes |
|
695 ... parts = data-dir |
|
696 ... |
|
697 ... [data-dir] |
|
698 ... recipe = recipes:mkdir |
|
699 ... path = mystuff |
|
700 ... """) |
|
701 |
|
702 Let's go through the changes one by one:: |
|
703 |
|
704 develop = recipes |
|
705 |
|
706 This tells the buildout to install a development egg for our recipes. |
|
707 Any number of paths can be listed. The paths can be relative or |
|
708 absolute. If relative, they are treated as relative to the buildout |
|
709 directory. They can be directory or file paths. If a file path is |
|
710 given, it should point to a Python setup script. If a directory path |
|
711 is given, it should point to a directory containing a setup.py file. |
|
712 Development eggs are installed before building any parts, as they may |
|
713 provide locally-defined recipes needed by the parts. |
|
714 |
|
715 :: |
|
716 |
|
717 parts = data-dir |
|
718 |
|
719 Here we've named a part to be "built". We can use any name we want |
|
720 except that different part names must be unique and recipes will often |
|
721 use the part name to decide what to do. |
|
722 |
|
723 :: |
|
724 |
|
725 [data-dir] |
|
726 recipe = recipes:mkdir |
|
727 path = mystuff |
|
728 |
|
729 |
|
730 When we name a part, we also create a section of the same |
|
731 name that contains part data. In this section, we'll define |
|
732 the recipe to be used to install the part. In this case, we also |
|
733 specify the path to be created. |
|
734 |
|
735 Let's run the buildout. We do so by running the build script in the |
|
736 buildout: |
|
737 |
|
738 >>> import os |
|
739 >>> os.chdir(sample_buildout) |
|
740 >>> buildout = os.path.join(sample_buildout, 'bin', 'buildout') |
|
741 >>> print system(buildout), |
|
742 Develop: '/sample-buildout/recipes' |
|
743 Installing data-dir. |
|
744 data-dir: Creating directory mystuff |
|
745 |
|
746 We see that the recipe created the directory, as expected: |
|
747 |
|
748 >>> ls(sample_buildout) |
|
749 - .installed.cfg |
|
750 d bin |
|
751 - buildout.cfg |
|
752 d develop-eggs |
|
753 d eggs |
|
754 d mystuff |
|
755 d parts |
|
756 d recipes |
|
757 |
|
758 In addition, .installed.cfg has been created containing information |
|
759 about the part we installed: |
|
760 |
|
761 >>> cat(sample_buildout, '.installed.cfg') |
|
762 [buildout] |
|
763 installed_develop_eggs = /sample-buildout/develop-eggs/recipes.egg-link |
|
764 parts = data-dir |
|
765 <BLANKLINE> |
|
766 [data-dir] |
|
767 __buildout_installed__ = /sample-buildout/mystuff |
|
768 __buildout_signature__ = recipes-c7vHV6ekIDUPy/7fjAaYjg== |
|
769 path = /sample-buildout/mystuff |
|
770 recipe = recipes:mkdir |
|
771 |
|
772 Note that the directory we installed is included in .installed.cfg. |
|
773 In addition, the path option includes the actual destination |
|
774 directory. |
|
775 |
|
776 If we change the name of the directory in the configuration file, |
|
777 we'll see that the directory gets removed and recreated: |
|
778 |
|
779 >>> write(sample_buildout, 'buildout.cfg', |
|
780 ... """ |
|
781 ... [buildout] |
|
782 ... develop = recipes |
|
783 ... parts = data-dir |
|
784 ... |
|
785 ... [data-dir] |
|
786 ... recipe = recipes:mkdir |
|
787 ... path = mydata |
|
788 ... """) |
|
789 |
|
790 >>> print system(buildout), |
|
791 Develop: '/sample-buildout/recipes' |
|
792 Uninstalling data-dir. |
|
793 Installing data-dir. |
|
794 data-dir: Creating directory mydata |
|
795 |
|
796 >>> ls(sample_buildout) |
|
797 - .installed.cfg |
|
798 d bin |
|
799 - buildout.cfg |
|
800 d develop-eggs |
|
801 d eggs |
|
802 d mydata |
|
803 d parts |
|
804 d recipes |
|
805 |
|
806 If any of the files or directories created by a recipe are removed, |
|
807 the part will be reinstalled: |
|
808 |
|
809 >>> rmdir(sample_buildout, 'mydata') |
|
810 >>> print system(buildout), |
|
811 Develop: '/sample-buildout/recipes' |
|
812 Uninstalling data-dir. |
|
813 Installing data-dir. |
|
814 data-dir: Creating directory mydata |
|
815 |
|
816 Error reporting |
|
817 --------------- |
|
818 |
|
819 If a user makes an error, an error needs to be printed and work needs |
|
820 to stop. This is accomplished by logging a detailed error message and |
|
821 then raising a (or an instance of a subclass of a) |
|
822 zc.buildout.UserError exception. Raising an error other than a |
|
823 UserError still displays the error, but labels it as a bug in the |
|
824 buildout software or recipe. In the sample above, of someone gives a |
|
825 non-existent directory to create the directory in: |
|
826 |
|
827 |
|
828 >>> write(sample_buildout, 'buildout.cfg', |
|
829 ... """ |
|
830 ... [buildout] |
|
831 ... develop = recipes |
|
832 ... parts = data-dir |
|
833 ... |
|
834 ... [data-dir] |
|
835 ... recipe = recipes:mkdir |
|
836 ... path = /xxx/mydata |
|
837 ... """) |
|
838 |
|
839 We'll get a user error, not a traceback. |
|
840 |
|
841 >>> print system(buildout), |
|
842 Develop: '/sample-buildout/recipes' |
|
843 data-dir: Cannot create /xxx/mydata. /xxx is not a directory. |
|
844 While: |
|
845 Installing. |
|
846 Getting section data-dir. |
|
847 Initializing part data-dir. |
|
848 Error: Invalid Path |
|
849 |
|
850 |
|
851 Recipe Error Handling |
|
852 --------------------- |
|
853 |
|
854 If an error occurs during installation, it is up to the recipe to |
|
855 clean up any system side effects, such as files created. Let's update |
|
856 the mkdir recipe to support multiple paths: |
|
857 |
|
858 >>> write(sample_buildout, 'recipes', 'mkdir.py', |
|
859 ... """ |
|
860 ... import logging, os, zc.buildout |
|
861 ... |
|
862 ... class Mkdir: |
|
863 ... |
|
864 ... def __init__(self, buildout, name, options): |
|
865 ... self.name, self.options = name, options |
|
866 ... |
|
867 ... # Normalize paths and check that their parent |
|
868 ... # directories exist: |
|
869 ... paths = [] |
|
870 ... for path in options['path'].split(): |
|
871 ... path = os.path.join(buildout['buildout']['directory'], path) |
|
872 ... if not os.path.isdir(os.path.dirname(path)): |
|
873 ... logging.getLogger(self.name).error( |
|
874 ... 'Cannot create %s. %s is not a directory.', |
|
875 ... options['path'], os.path.dirname(options['path'])) |
|
876 ... raise zc.buildout.UserError('Invalid Path') |
|
877 ... paths.append(path) |
|
878 ... options['path'] = ' '.join(paths) |
|
879 ... |
|
880 ... def install(self): |
|
881 ... paths = self.options['path'].split() |
|
882 ... for path in paths: |
|
883 ... logging.getLogger(self.name).info( |
|
884 ... 'Creating directory %s', os.path.basename(path)) |
|
885 ... os.mkdir(path) |
|
886 ... return paths |
|
887 ... |
|
888 ... def update(self): |
|
889 ... pass |
|
890 ... """) |
|
891 |
|
892 If there is an error creating a path, the install method will exit and |
|
893 leave previously created paths in place: |
|
894 |
|
895 >>> write(sample_buildout, 'buildout.cfg', |
|
896 ... """ |
|
897 ... [buildout] |
|
898 ... develop = recipes |
|
899 ... parts = data-dir |
|
900 ... |
|
901 ... [data-dir] |
|
902 ... recipe = recipes:mkdir |
|
903 ... path = foo bin |
|
904 ... """) |
|
905 |
|
906 >>> print system(buildout), # doctest: +ELLIPSIS |
|
907 Develop: '/sample-buildout/recipes' |
|
908 Uninstalling data-dir. |
|
909 Installing data-dir. |
|
910 data-dir: Creating directory foo |
|
911 data-dir: Creating directory bin |
|
912 While: |
|
913 Installing data-dir. |
|
914 <BLANKLINE> |
|
915 An internal error occurred due to a bug in either zc.buildout or in a |
|
916 recipe being used: |
|
917 Traceback (most recent call last): |
|
918 ... |
|
919 OSError: [Errno 17] File exists: '/sample-buildout/bin' |
|
920 |
|
921 We meant to create a directory bins, but typed bin. Now foo was |
|
922 left behind. |
|
923 |
|
924 >>> os.path.exists('foo') |
|
925 True |
|
926 |
|
927 If we fix the typo: |
|
928 |
|
929 >>> write(sample_buildout, 'buildout.cfg', |
|
930 ... """ |
|
931 ... [buildout] |
|
932 ... develop = recipes |
|
933 ... parts = data-dir |
|
934 ... |
|
935 ... [data-dir] |
|
936 ... recipe = recipes:mkdir |
|
937 ... path = foo bins |
|
938 ... """) |
|
939 |
|
940 >>> print system(buildout), # doctest: +ELLIPSIS |
|
941 Develop: '/sample-buildout/recipes' |
|
942 Installing data-dir. |
|
943 data-dir: Creating directory foo |
|
944 While: |
|
945 Installing data-dir. |
|
946 <BLANKLINE> |
|
947 An internal error occurred due to a bug in either zc.buildout or in a |
|
948 recipe being used: |
|
949 Traceback (most recent call last): |
|
950 ... |
|
951 OSError: [Errno 17] File exists: '/sample-buildout/foo' |
|
952 |
|
953 Now they fail because foo exists, because it was left behind. |
|
954 |
|
955 >>> remove('foo') |
|
956 |
|
957 Let's fix the recipe: |
|
958 |
|
959 >>> write(sample_buildout, 'recipes', 'mkdir.py', |
|
960 ... """ |
|
961 ... import logging, os, zc.buildout |
|
962 ... |
|
963 ... class Mkdir: |
|
964 ... |
|
965 ... def __init__(self, buildout, name, options): |
|
966 ... self.name, self.options = name, options |
|
967 ... |
|
968 ... # Normalize paths and check that their parent |
|
969 ... # directories exist: |
|
970 ... paths = [] |
|
971 ... for path in options['path'].split(): |
|
972 ... path = os.path.join(buildout['buildout']['directory'], path) |
|
973 ... if not os.path.isdir(os.path.dirname(path)): |
|
974 ... logging.getLogger(self.name).error( |
|
975 ... 'Cannot create %s. %s is not a directory.', |
|
976 ... options['path'], os.path.dirname(options['path'])) |
|
977 ... raise zc.buildout.UserError('Invalid Path') |
|
978 ... paths.append(path) |
|
979 ... options['path'] = ' '.join(paths) |
|
980 ... |
|
981 ... def install(self): |
|
982 ... paths = self.options['path'].split() |
|
983 ... created = [] |
|
984 ... try: |
|
985 ... for path in paths: |
|
986 ... logging.getLogger(self.name).info( |
|
987 ... 'Creating directory %s', os.path.basename(path)) |
|
988 ... os.mkdir(path) |
|
989 ... created.append(path) |
|
990 ... except: |
|
991 ... for d in created: |
|
992 ... os.rmdir(d) |
|
993 ... raise |
|
994 ... |
|
995 ... return paths |
|
996 ... |
|
997 ... def update(self): |
|
998 ... pass |
|
999 ... """) |
|
1000 |
|
1001 And put back the typo: |
|
1002 |
|
1003 >>> write(sample_buildout, 'buildout.cfg', |
|
1004 ... """ |
|
1005 ... [buildout] |
|
1006 ... develop = recipes |
|
1007 ... parts = data-dir |
|
1008 ... |
|
1009 ... [data-dir] |
|
1010 ... recipe = recipes:mkdir |
|
1011 ... path = foo bin |
|
1012 ... """) |
|
1013 |
|
1014 When we rerun the buildout: |
|
1015 |
|
1016 >>> print system(buildout), # doctest: +ELLIPSIS |
|
1017 Develop: '/sample-buildout/recipes' |
|
1018 Installing data-dir. |
|
1019 data-dir: Creating directory foo |
|
1020 data-dir: Creating directory bin |
|
1021 While: |
|
1022 Installing data-dir. |
|
1023 <BLANKLINE> |
|
1024 An internal error occurred due to a bug in either zc.buildout or in a |
|
1025 recipe being used: |
|
1026 Traceback (most recent call last): |
|
1027 ... |
|
1028 OSError: [Errno 17] File exists: '/sample-buildout/bin' |
|
1029 |
|
1030 .. Wait for the file to really disappear. My linux is weird. |
|
1031 |
|
1032 >>> wait_until("foo goes away", lambda : not os.path.exists('foo'), |
|
1033 ... timeout=200) |
|
1034 |
|
1035 we get the same error, but we don't get the directory left behind: |
|
1036 |
|
1037 >>> os.path.exists('foo') |
|
1038 False |
|
1039 |
|
1040 It's critical that recipes clean up partial effects when errors |
|
1041 occur. Because recipes most commonly create files and directories, |
|
1042 buildout provides a helper API for removing created files when an |
|
1043 error occurs. Option objects have a created method that can be called |
|
1044 to record files as they are created. If the install or update method |
|
1045 returns with an error, then any registered paths are removed |
|
1046 automatically. The method returns the files registered and can be |
|
1047 used to return the files created. Let's use this API to simplify the |
|
1048 recipe: |
|
1049 |
|
1050 >>> write(sample_buildout, 'recipes', 'mkdir.py', |
|
1051 ... """ |
|
1052 ... import logging, os, zc.buildout |
|
1053 ... |
|
1054 ... class Mkdir: |
|
1055 ... |
|
1056 ... def __init__(self, buildout, name, options): |
|
1057 ... self.name, self.options = name, options |
|
1058 ... |
|
1059 ... # Normalize paths and check that their parent |
|
1060 ... # directories exist: |
|
1061 ... paths = [] |
|
1062 ... for path in options['path'].split(): |
|
1063 ... path = os.path.join(buildout['buildout']['directory'], path) |
|
1064 ... if not os.path.isdir(os.path.dirname(path)): |
|
1065 ... logging.getLogger(self.name).error( |
|
1066 ... 'Cannot create %s. %s is not a directory.', |
|
1067 ... options['path'], os.path.dirname(options['path'])) |
|
1068 ... raise zc.buildout.UserError('Invalid Path') |
|
1069 ... paths.append(path) |
|
1070 ... options['path'] = ' '.join(paths) |
|
1071 ... |
|
1072 ... def install(self): |
|
1073 ... paths = self.options['path'].split() |
|
1074 ... for path in paths: |
|
1075 ... logging.getLogger(self.name).info( |
|
1076 ... 'Creating directory %s', os.path.basename(path)) |
|
1077 ... os.mkdir(path) |
|
1078 ... self.options.created(path) |
|
1079 ... |
|
1080 ... return self.options.created() |
|
1081 ... |
|
1082 ... def update(self): |
|
1083 ... pass |
|
1084 ... """) |
|
1085 |
|
1086 .. |
|
1087 |
|
1088 >>> remove(sample_buildout, 'recipes', 'mkdir.pyc') |
|
1089 |
|
1090 We returned by calling created, taking advantage of the fact that it |
|
1091 returns the registered paths. We did this for illustrative purposes. |
|
1092 It would be simpler just to return the paths as before. |
|
1093 |
|
1094 If we rerun the buildout, again, we'll get the error and no |
|
1095 directories will be created: |
|
1096 |
|
1097 >>> print system(buildout), # doctest: +ELLIPSIS |
|
1098 Develop: '/sample-buildout/recipes' |
|
1099 Installing data-dir. |
|
1100 data-dir: Creating directory foo |
|
1101 data-dir: Creating directory bin |
|
1102 While: |
|
1103 Installing data-dir. |
|
1104 <BLANKLINE> |
|
1105 An internal error occurred due to a bug in either zc.buildout or in a |
|
1106 recipe being used: |
|
1107 Traceback (most recent call last): |
|
1108 ... |
|
1109 OSError: [Errno 17] File exists: '/sample-buildout/bin' |
|
1110 |
|
1111 >>> os.path.exists('foo') |
|
1112 False |
|
1113 |
|
1114 Now, we'll fix the typo again and we'll get the directories we expect: |
|
1115 |
|
1116 >>> write(sample_buildout, 'buildout.cfg', |
|
1117 ... """ |
|
1118 ... [buildout] |
|
1119 ... develop = recipes |
|
1120 ... parts = data-dir |
|
1121 ... |
|
1122 ... [data-dir] |
|
1123 ... recipe = recipes:mkdir |
|
1124 ... path = foo bins |
|
1125 ... """) |
|
1126 |
|
1127 >>> print system(buildout), |
|
1128 Develop: '/sample-buildout/recipes' |
|
1129 Installing data-dir. |
|
1130 data-dir: Creating directory foo |
|
1131 data-dir: Creating directory bins |
|
1132 |
|
1133 >>> os.path.exists('foo') |
|
1134 True |
|
1135 >>> os.path.exists('bins') |
|
1136 True |
|
1137 |
|
1138 Configuration file syntax |
|
1139 ------------------------- |
|
1140 |
|
1141 As mentioned earlier, buildout configuration files use the format |
|
1142 defined by the Python ConfigParser module with extensions. The |
|
1143 extensions are: |
|
1144 |
|
1145 - option names are case sensitive |
|
1146 |
|
1147 - option values can use a substitution syntax, described below, to |
|
1148 refer to option values in specific sections. |
|
1149 |
|
1150 - option values can be appended or removed using the - and + |
|
1151 operators. |
|
1152 |
|
1153 The ConfigParser syntax is very flexible. Section names can contain |
|
1154 any characters other than newlines and right square braces ("]"). |
|
1155 Option names can contain any characters other than newlines, colons, |
|
1156 and equal signs, can not start with a space, and don't include |
|
1157 trailing spaces. |
|
1158 |
|
1159 It is likely that, in the future, some characters will be given |
|
1160 special buildout-defined meanings. This is already true of the |
|
1161 characters ":", "$", "%", "(", and ")". For now, it is a good idea to |
|
1162 keep section and option names simple, sticking to alphanumeric |
|
1163 characters, hyphens, and periods. |
|
1164 |
|
1165 Annotated sections |
|
1166 ------------------ |
|
1167 |
|
1168 When used with the `annotate` command, buildout displays annotated sections. |
|
1169 All sections are displayed, sorted alphabetically. For each section, |
|
1170 all key-value pairs are displayed, sorted alphabetically, along with |
|
1171 the origin of the value (file name or COMPUTED_VALUE, DEFAULT_VALUE, |
|
1172 COMMAND_LINE_VALUE). |
|
1173 |
|
1174 >>> print system(buildout+ ' annotate'), |
|
1175 ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE |
|
1176 <BLANKLINE> |
|
1177 Annotated sections |
|
1178 ================== |
|
1179 <BLANKLINE> |
|
1180 [buildout] |
|
1181 accept-buildout-test-releases= false |
|
1182 DEFAULT_VALUE |
|
1183 allow-hosts= * |
|
1184 DEFAULT_VALUE |
|
1185 allow-picked-versions= true |
|
1186 DEFAULT_VALUE |
|
1187 allowed-eggs-from-site-packages= * |
|
1188 DEFAULT_VALUE |
|
1189 bin-directory= bin |
|
1190 DEFAULT_VALUE |
|
1191 develop= recipes |
|
1192 /sample-buildout/buildout.cfg |
|
1193 develop-eggs-directory= develop-eggs |
|
1194 DEFAULT_VALUE |
|
1195 directory= /sample-buildout |
|
1196 COMPUTED_VALUE |
|
1197 eggs-directory= eggs |
|
1198 DEFAULT_VALUE |
|
1199 exec-sitecustomize= true |
|
1200 DEFAULT_VALUE |
|
1201 executable= ... |
|
1202 DEFAULT_VALUE |
|
1203 find-links= |
|
1204 DEFAULT_VALUE |
|
1205 include-site-packages= true |
|
1206 DEFAULT_VALUE |
|
1207 install-from-cache= false |
|
1208 DEFAULT_VALUE |
|
1209 installed= .installed.cfg |
|
1210 DEFAULT_VALUE |
|
1211 log-format= |
|
1212 DEFAULT_VALUE |
|
1213 log-level= INFO |
|
1214 DEFAULT_VALUE |
|
1215 newest= true |
|
1216 DEFAULT_VALUE |
|
1217 offline= false |
|
1218 DEFAULT_VALUE |
|
1219 parts= data-dir |
|
1220 /sample-buildout/buildout.cfg |
|
1221 parts-directory= parts |
|
1222 DEFAULT_VALUE |
|
1223 prefer-final= false |
|
1224 DEFAULT_VALUE |
|
1225 python= buildout |
|
1226 DEFAULT_VALUE |
|
1227 relative-paths= false |
|
1228 DEFAULT_VALUE |
|
1229 socket-timeout= |
|
1230 DEFAULT_VALUE |
|
1231 unzip= false |
|
1232 DEFAULT_VALUE |
|
1233 use-dependency-links= true |
|
1234 DEFAULT_VALUE |
|
1235 <BLANKLINE> |
|
1236 [data-dir] |
|
1237 path= foo bins |
|
1238 /sample-buildout/buildout.cfg |
|
1239 recipe= recipes:mkdir |
|
1240 /sample-buildout/buildout.cfg |
|
1241 <BLANKLINE> |
|
1242 |
|
1243 Variable substitutions |
|
1244 ---------------------- |
|
1245 |
|
1246 Buildout configuration files support variable substitution. |
|
1247 To illustrate this, we'll create an debug recipe to |
|
1248 allow us to see interactions with the buildout: |
|
1249 |
|
1250 >>> write(sample_buildout, 'recipes', 'debug.py', |
|
1251 ... """ |
|
1252 ... class Debug: |
|
1253 ... |
|
1254 ... def __init__(self, buildout, name, options): |
|
1255 ... self.buildout = buildout |
|
1256 ... self.name = name |
|
1257 ... self.options = options |
|
1258 ... |
|
1259 ... def install(self): |
|
1260 ... items = self.options.items() |
|
1261 ... items.sort() |
|
1262 ... for option, value in items: |
|
1263 ... print option, value |
|
1264 ... return () |
|
1265 ... |
|
1266 ... update = install |
|
1267 ... """) |
|
1268 |
|
1269 This recipe doesn't actually create anything. The install method |
|
1270 doesn't return anything, because it didn't create any files or |
|
1271 directories. |
|
1272 |
|
1273 We also have to update our setup script: |
|
1274 |
|
1275 >>> write(sample_buildout, 'recipes', 'setup.py', |
|
1276 ... """ |
|
1277 ... from setuptools import setup |
|
1278 ... entry_points = ( |
|
1279 ... ''' |
|
1280 ... [zc.buildout] |
|
1281 ... mkdir = mkdir:Mkdir |
|
1282 ... debug = debug:Debug |
|
1283 ... ''') |
|
1284 ... setup(name="recipes", entry_points=entry_points) |
|
1285 ... """) |
|
1286 |
|
1287 We've rearranged the script a bit to make the entry points easier to |
|
1288 edit. In particular, entry points are now defined as a configuration |
|
1289 string, rather than a dictionary. |
|
1290 |
|
1291 Let's update our configuration to provide variable substitution |
|
1292 examples: |
|
1293 |
|
1294 >>> write(sample_buildout, 'buildout.cfg', |
|
1295 ... """ |
|
1296 ... [buildout] |
|
1297 ... develop = recipes |
|
1298 ... parts = data-dir debug |
|
1299 ... log-level = INFO |
|
1300 ... |
|
1301 ... [debug] |
|
1302 ... recipe = recipes:debug |
|
1303 ... File 1 = ${data-dir:path}/file |
|
1304 ... File 2 = ${debug:File 1}/log |
|
1305 ... |
|
1306 ... [data-dir] |
|
1307 ... recipe = recipes:mkdir |
|
1308 ... path = mydata |
|
1309 ... """) |
|
1310 |
|
1311 We used a string-template substitution for File 1 and File 2. This |
|
1312 type of substitution uses the string.Template syntax. Names |
|
1313 substituted are qualified option names, consisting of a section name |
|
1314 and option name joined by a colon. |
|
1315 |
|
1316 Now, if we run the buildout, we'll see the options with the values |
|
1317 substituted. |
|
1318 |
|
1319 >>> print system(buildout), |
|
1320 Develop: '/sample-buildout/recipes' |
|
1321 Uninstalling data-dir. |
|
1322 Installing data-dir. |
|
1323 data-dir: Creating directory mydata |
|
1324 Installing debug. |
|
1325 File 1 /sample-buildout/mydata/file |
|
1326 File 2 /sample-buildout/mydata/file/log |
|
1327 recipe recipes:debug |
|
1328 |
|
1329 Note that the substitution of the data-dir path option reflects the |
|
1330 update to the option performed by the mkdir recipe. |
|
1331 |
|
1332 It might seem surprising that mydata was created again. This is |
|
1333 because we changed our recipes package by adding the debug module. |
|
1334 The buildout system didn't know if this module could effect the mkdir |
|
1335 recipe, so it assumed it could and reinstalled mydata. If we rerun |
|
1336 the buildout: |
|
1337 |
|
1338 >>> print system(buildout), |
|
1339 Develop: '/sample-buildout/recipes' |
|
1340 Updating data-dir. |
|
1341 Updating debug. |
|
1342 File 1 /sample-buildout/mydata/file |
|
1343 File 2 /sample-buildout/mydata/file/log |
|
1344 recipe recipes:debug |
|
1345 |
|
1346 We can see that mydata was not recreated. |
|
1347 |
|
1348 Note that, in this case, we didn't specify a log level, so |
|
1349 we didn't get output about what the buildout was doing. |
|
1350 |
|
1351 Section and option names in variable substitutions are only allowed to |
|
1352 contain alphanumeric characters, hyphens, periods and spaces. This |
|
1353 restriction might be relaxed in future releases. |
|
1354 |
|
1355 We can ommit the section name in a variable substitution to refer to |
|
1356 the current section. We can also use the special option, |
|
1357 _buildout_section_name_ to get the current section name. |
|
1358 |
|
1359 >>> write(sample_buildout, 'buildout.cfg', |
|
1360 ... """ |
|
1361 ... [buildout] |
|
1362 ... develop = recipes |
|
1363 ... parts = data-dir debug |
|
1364 ... log-level = INFO |
|
1365 ... |
|
1366 ... [debug] |
|
1367 ... recipe = recipes:debug |
|
1368 ... File 1 = ${data-dir:path}/file |
|
1369 ... File 2 = ${:File 1}/log |
|
1370 ... my_name = ${:_buildout_section_name_} |
|
1371 ... |
|
1372 ... [data-dir] |
|
1373 ... recipe = recipes:mkdir |
|
1374 ... path = mydata |
|
1375 ... """) |
|
1376 |
|
1377 >>> print system(buildout), |
|
1378 Develop: '/sample-buildout/recipes' |
|
1379 Uninstalling debug. |
|
1380 Updating data-dir. |
|
1381 Installing debug. |
|
1382 File 1 /sample-buildout/mydata/file |
|
1383 File 2 /sample-buildout/mydata/file/log |
|
1384 my_name debug |
|
1385 recipe recipes:debug |
|
1386 |
|
1387 Automatic part selection and ordering |
|
1388 ------------------------------------- |
|
1389 |
|
1390 When a section with a recipe is referred to, either through variable |
|
1391 substitution or by an initializing recipe, the section is treated as a |
|
1392 part and added to the part list before the referencing part. For |
|
1393 example, we can leave data-dir out of the parts list: |
|
1394 |
|
1395 >>> write(sample_buildout, 'buildout.cfg', |
|
1396 ... """ |
|
1397 ... [buildout] |
|
1398 ... develop = recipes |
|
1399 ... parts = debug |
|
1400 ... log-level = INFO |
|
1401 ... |
|
1402 ... [debug] |
|
1403 ... recipe = recipes:debug |
|
1404 ... File 1 = ${data-dir:path}/file |
|
1405 ... File 2 = ${debug:File 1}/log |
|
1406 ... |
|
1407 ... [data-dir] |
|
1408 ... recipe = recipes:mkdir |
|
1409 ... path = mydata |
|
1410 ... """) |
|
1411 |
|
1412 |
|
1413 It will still be treated as a part: |
|
1414 |
|
1415 >>> print system(buildout), |
|
1416 Develop: '/sample-buildout/recipes' |
|
1417 Uninstalling debug. |
|
1418 Updating data-dir. |
|
1419 Installing debug. |
|
1420 File 1 /sample-buildout/mydata/file |
|
1421 File 2 /sample-buildout/mydata/file/log |
|
1422 recipe recipes:debug |
|
1423 |
|
1424 >>> cat('.installed.cfg') # doctest: +ELLIPSIS |
|
1425 [buildout] |
|
1426 installed_develop_eggs = /sample-buildout/develop-eggs/recipes.egg-link |
|
1427 parts = data-dir debug |
|
1428 ... |
|
1429 |
|
1430 Note that the data-dir part is included *before* the debug part, |
|
1431 because the debug part refers to the data-dir part. Even if we list |
|
1432 the data-dir part after the debug part, it will be included before: |
|
1433 |
|
1434 >>> write(sample_buildout, 'buildout.cfg', |
|
1435 ... """ |
|
1436 ... [buildout] |
|
1437 ... develop = recipes |
|
1438 ... parts = debug data-dir |
|
1439 ... log-level = INFO |
|
1440 ... |
|
1441 ... [debug] |
|
1442 ... recipe = recipes:debug |
|
1443 ... File 1 = ${data-dir:path}/file |
|
1444 ... File 2 = ${debug:File 1}/log |
|
1445 ... |
|
1446 ... [data-dir] |
|
1447 ... recipe = recipes:mkdir |
|
1448 ... path = mydata |
|
1449 ... """) |
|
1450 |
|
1451 |
|
1452 It will still be treated as a part: |
|
1453 |
|
1454 >>> print system(buildout), |
|
1455 Develop: '/sample-buildout/recipes' |
|
1456 Updating data-dir. |
|
1457 Updating debug. |
|
1458 File 1 /sample-buildout/mydata/file |
|
1459 File 2 /sample-buildout/mydata/file/log |
|
1460 recipe recipes:debug |
|
1461 |
|
1462 >>> cat('.installed.cfg') # doctest: +ELLIPSIS |
|
1463 [buildout] |
|
1464 installed_develop_eggs = /sample-buildout/develop-eggs/recipes.egg-link |
|
1465 parts = data-dir debug |
|
1466 ... |
|
1467 |
|
1468 Extending sections (macros) |
|
1469 --------------------------- |
|
1470 |
|
1471 A section (other than the buildout section) can extend one or more |
|
1472 other sections using the ``<=`` option. Options from the referenced |
|
1473 sections are copied to the refering section *before* variable |
|
1474 substitution. This, together with the ability to refer to variables |
|
1475 of the current section allows sections to be used as macros. |
|
1476 |
|
1477 >>> write(sample_buildout, 'buildout.cfg', |
|
1478 ... """ |
|
1479 ... [buildout] |
|
1480 ... develop = recipes |
|
1481 ... parts = myfiles |
|
1482 ... log-level = INFO |
|
1483 ... |
|
1484 ... [debug] |
|
1485 ... recipe = recipes:debug |
|
1486 ... |
|
1487 ... [with_file1] |
|
1488 ... <= debug |
|
1489 ... file1 = ${:path}/file1 |
|
1490 ... color = red |
|
1491 ... |
|
1492 ... [with_file2] |
|
1493 ... <= debug |
|
1494 ... file2 = ${:path}/file2 |
|
1495 ... color = blue |
|
1496 ... |
|
1497 ... [myfiles] |
|
1498 ... <= with_file1 |
|
1499 ... with_file2 |
|
1500 ... path = mydata |
|
1501 ... """) |
|
1502 |
|
1503 >>> print system(buildout), |
|
1504 Develop: '/sample-buildout/recipes' |
|
1505 Uninstalling debug. |
|
1506 Uninstalling data-dir. |
|
1507 Installing myfiles. |
|
1508 color blue |
|
1509 file1 mydata/file1 |
|
1510 file2 mydata/file2 |
|
1511 path mydata |
|
1512 recipe recipes:debug |
|
1513 |
|
1514 In this example, the debug, with_file1 and with_file2 sections act as |
|
1515 macros. In particular, the variable substitutions are performed |
|
1516 relative to the myfiles section. |
|
1517 |
|
1518 Adding and removing options |
|
1519 --------------------------- |
|
1520 |
|
1521 We can append and remove values to an option by using the + and - |
|
1522 operators. |
|
1523 |
|
1524 This is illustrated below; first we define a base configuration. |
|
1525 |
|
1526 >>> write(sample_buildout, 'base.cfg', |
|
1527 ... """ |
|
1528 ... [buildout] |
|
1529 ... parts = part1 part2 part3 |
|
1530 ... |
|
1531 ... [part1] |
|
1532 ... recipe = |
|
1533 ... option = a1 a2 |
|
1534 ... |
|
1535 ... [part2] |
|
1536 ... recipe = |
|
1537 ... option = b1 b2 b3 b4 |
|
1538 ... |
|
1539 ... [part3] |
|
1540 ... recipe = |
|
1541 ... option = c1 c2 |
|
1542 ... |
|
1543 ... """) |
|
1544 |
|
1545 Extending this configuration, we can "adjust" the values set in the |
|
1546 base configuration file. |
|
1547 |
|
1548 >>> write(sample_buildout, 'extension1.cfg', |
|
1549 ... """ |
|
1550 ... [buildout] |
|
1551 ... extends = base.cfg |
|
1552 ... |
|
1553 ... # appending values |
|
1554 ... [part1] |
|
1555 ... option += a3 a4 |
|
1556 ... |
|
1557 ... # removing values |
|
1558 ... [part2] |
|
1559 ... option -= b1 b2 |
|
1560 ... |
|
1561 ... # alt. spelling |
|
1562 ... [part3] |
|
1563 ... option+=c3 c4 c5 |
|
1564 ... |
|
1565 ... # normal assignment |
|
1566 ... [part4] |
|
1567 ... option = h1 h2 |
|
1568 ... |
|
1569 ... """) |
|
1570 |
|
1571 An additional extension. |
|
1572 |
|
1573 >>> write(sample_buildout, 'extension2.cfg', |
|
1574 ... """ |
|
1575 ... [buildout] |
|
1576 ... extends = extension1.cfg |
|
1577 ... |
|
1578 ... # appending values |
|
1579 ... [part1] |
|
1580 ... option += a5 |
|
1581 ... |
|
1582 ... # removing values |
|
1583 ... [part2] |
|
1584 ... option -= b1 b2 b3 |
|
1585 ... |
|
1586 ... """) |
|
1587 |
|
1588 To verify that the options are adjusted correctly, we'll set up an |
|
1589 extension that prints out the options. |
|
1590 |
|
1591 >>> mkdir(sample_buildout, 'demo') |
|
1592 >>> write(sample_buildout, 'demo', 'demo.py', |
|
1593 ... """ |
|
1594 ... def ext(buildout): |
|
1595 ... print [part['option'] for name, part in buildout.items() \ |
|
1596 ... if name.startswith('part')] |
|
1597 ... """) |
|
1598 |
|
1599 >>> write(sample_buildout, 'demo', 'setup.py', |
|
1600 ... """ |
|
1601 ... from setuptools import setup |
|
1602 ... |
|
1603 ... setup( |
|
1604 ... name="demo", |
|
1605 ... entry_points={'zc.buildout.extension': ['ext = demo:ext']}, |
|
1606 ... ) |
|
1607 ... """) |
|
1608 |
|
1609 Set up a buildout configuration for this extension. |
|
1610 |
|
1611 >>> write(sample_buildout, 'buildout.cfg', |
|
1612 ... """ |
|
1613 ... [buildout] |
|
1614 ... develop = demo |
|
1615 ... parts = |
|
1616 ... """) |
|
1617 |
|
1618 >>> os.chdir(sample_buildout) |
|
1619 >>> print system(os.path.join(sample_buildout, 'bin', 'buildout')), |
|
1620 Develop: '/sample-buildout/demo' |
|
1621 Uninstalling myfiles. |
|
1622 Getting distribution for 'recipes'. |
|
1623 zip_safe flag not set; analyzing archive contents... |
|
1624 Got recipes 0.0.0. |
|
1625 warning: install_lib: 'build/lib' does not exist -- no Python modules to install |
|
1626 |
|
1627 Verify option values. |
|
1628 |
|
1629 >>> write(sample_buildout, 'buildout.cfg', |
|
1630 ... """ |
|
1631 ... [buildout] |
|
1632 ... develop = demo |
|
1633 ... extensions = demo |
|
1634 ... extends = extension2.cfg |
|
1635 ... """) |
|
1636 |
|
1637 >>> print system(os.path.join('bin', 'buildout')), |
|
1638 ['a1 a2/na3 a4/na5', 'b1 b2 b3 b4', 'c1 c2/nc3 c4 c5', 'h1 h2'] |
|
1639 Develop: '/sample-buildout/demo' |
|
1640 |
|
1641 Annotated sections output shows which files are responsible for which |
|
1642 operations. |
|
1643 |
|
1644 >>> print system(os.path.join('bin', 'buildout') + ' annotate'), |
|
1645 ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE |
|
1646 <BLANKLINE> |
|
1647 Annotated sections |
|
1648 ================== |
|
1649 ... |
|
1650 <BLANKLINE> |
|
1651 [part1] |
|
1652 option= a1 a2 |
|
1653 a3 a4 |
|
1654 a5 |
|
1655 /sample-buildout/base.cfg |
|
1656 += /sample-buildout/extension1.cfg |
|
1657 += /sample-buildout/extension2.cfg |
|
1658 recipe= |
|
1659 /sample-buildout/base.cfg |
|
1660 <BLANKLINE> |
|
1661 [part2] |
|
1662 option= b1 b2 b3 b4 |
|
1663 /sample-buildout/base.cfg |
|
1664 -= /sample-buildout/extension1.cfg |
|
1665 -= /sample-buildout/extension2.cfg |
|
1666 recipe= |
|
1667 /sample-buildout/base.cfg |
|
1668 <BLANKLINE> |
|
1669 [part3] |
|
1670 option= c1 c2 |
|
1671 c3 c4 c5 |
|
1672 /sample-buildout/base.cfg |
|
1673 += /sample-buildout/extension1.cfg |
|
1674 recipe= |
|
1675 /sample-buildout/base.cfg |
|
1676 <BLANKLINE> |
|
1677 [part4] |
|
1678 option= h1 h2 |
|
1679 /sample-buildout/extension1.cfg |
|
1680 |
|
1681 Cleanup. |
|
1682 |
|
1683 >>> os.remove(os.path.join(sample_buildout, 'base.cfg')) |
|
1684 >>> os.remove(os.path.join(sample_buildout, 'extension1.cfg')) |
|
1685 >>> os.remove(os.path.join(sample_buildout, 'extension2.cfg')) |
|
1686 |
|
1687 Multiple configuration files |
|
1688 ---------------------------- |
|
1689 |
|
1690 A configuration file can "extend" another configuration file. |
|
1691 Options are read from the other configuration file if they aren't |
|
1692 already defined by your configuration file. |
|
1693 |
|
1694 The configuration files your file extends can extend |
|
1695 other configuration files. The same file may be |
|
1696 used more than once although, of course, cycles aren't allowed. |
|
1697 |
|
1698 To see how this works, we use an example: |
|
1699 |
|
1700 >>> write(sample_buildout, 'buildout.cfg', |
|
1701 ... """ |
|
1702 ... [buildout] |
|
1703 ... extends = base.cfg |
|
1704 ... |
|
1705 ... [debug] |
|
1706 ... op = buildout |
|
1707 ... """) |
|
1708 |
|
1709 >>> write(sample_buildout, 'base.cfg', |
|
1710 ... """ |
|
1711 ... [buildout] |
|
1712 ... develop = recipes |
|
1713 ... parts = debug |
|
1714 ... |
|
1715 ... [debug] |
|
1716 ... recipe = recipes:debug |
|
1717 ... op = base |
|
1718 ... """) |
|
1719 |
|
1720 >>> print system(buildout), |
|
1721 Develop: '/sample-buildout/recipes' |
|
1722 Installing debug. |
|
1723 op buildout |
|
1724 recipe recipes:debug |
|
1725 |
|
1726 The example is pretty trivial, but the pattern it illustrates is |
|
1727 pretty common. In a more practical example, the base buildout might |
|
1728 represent a product and the extending buildout might be a |
|
1729 customization. |
|
1730 |
|
1731 Here is a more elaborate example. |
|
1732 |
|
1733 >>> other = tmpdir('other') |
|
1734 |
|
1735 >>> write(sample_buildout, 'buildout.cfg', |
|
1736 ... """ |
|
1737 ... [buildout] |
|
1738 ... extends = b1.cfg b2.cfg %(b3)s |
|
1739 ... |
|
1740 ... [debug] |
|
1741 ... op = buildout |
|
1742 ... """ % dict(b3=os.path.join(other, 'b3.cfg'))) |
|
1743 |
|
1744 >>> write(sample_buildout, 'b1.cfg', |
|
1745 ... """ |
|
1746 ... [buildout] |
|
1747 ... extends = base.cfg |
|
1748 ... |
|
1749 ... [debug] |
|
1750 ... op1 = b1 1 |
|
1751 ... op2 = b1 2 |
|
1752 ... """) |
|
1753 |
|
1754 >>> write(sample_buildout, 'b2.cfg', |
|
1755 ... """ |
|
1756 ... [buildout] |
|
1757 ... extends = base.cfg |
|
1758 ... |
|
1759 ... [debug] |
|
1760 ... op2 = b2 2 |
|
1761 ... op3 = b2 3 |
|
1762 ... """) |
|
1763 |
|
1764 >>> write(other, 'b3.cfg', |
|
1765 ... """ |
|
1766 ... [buildout] |
|
1767 ... extends = b3base.cfg |
|
1768 ... |
|
1769 ... [debug] |
|
1770 ... op4 = b3 4 |
|
1771 ... """) |
|
1772 |
|
1773 >>> write(other, 'b3base.cfg', |
|
1774 ... """ |
|
1775 ... [debug] |
|
1776 ... op5 = b3base 5 |
|
1777 ... """) |
|
1778 |
|
1779 >>> write(sample_buildout, 'base.cfg', |
|
1780 ... """ |
|
1781 ... [buildout] |
|
1782 ... develop = recipes |
|
1783 ... parts = debug |
|
1784 ... |
|
1785 ... [debug] |
|
1786 ... recipe = recipes:debug |
|
1787 ... name = base |
|
1788 ... """) |
|
1789 |
|
1790 >>> print system(buildout), |
|
1791 Develop: '/sample-buildout/recipes' |
|
1792 Uninstalling debug. |
|
1793 Installing debug. |
|
1794 name base |
|
1795 op buildout |
|
1796 op1 b1 1 |
|
1797 op2 b2 2 |
|
1798 op3 b2 3 |
|
1799 op4 b3 4 |
|
1800 op5 b3base 5 |
|
1801 recipe recipes:debug |
|
1802 |
|
1803 There are several things to note about this example: |
|
1804 |
|
1805 - We can name multiple files in an extends option. |
|
1806 |
|
1807 - We can reference files recursively. |
|
1808 |
|
1809 - Relative file names in extended options are interpreted relative to |
|
1810 the directory containing the referencing configuration file. |
|
1811 |
|
1812 Loading Configuration from URLs |
|
1813 ------------------------------- |
|
1814 |
|
1815 Configuration files can be loaded from URLs. To see how this works, |
|
1816 we'll set up a web server with some configuration files. |
|
1817 |
|
1818 >>> server_data = tmpdir('server_data') |
|
1819 |
|
1820 >>> write(server_data, "r1.cfg", |
|
1821 ... """ |
|
1822 ... [debug] |
|
1823 ... op1 = r1 1 |
|
1824 ... op2 = r1 2 |
|
1825 ... """) |
|
1826 |
|
1827 >>> write(server_data, "r2.cfg", |
|
1828 ... """ |
|
1829 ... [buildout] |
|
1830 ... extends = r1.cfg |
|
1831 ... |
|
1832 ... [debug] |
|
1833 ... op2 = r2 2 |
|
1834 ... op3 = r2 3 |
|
1835 ... """) |
|
1836 |
|
1837 >>> server_url = start_server(server_data) |
|
1838 |
|
1839 >>> write('client.cfg', |
|
1840 ... """ |
|
1841 ... [buildout] |
|
1842 ... develop = recipes |
|
1843 ... parts = debug |
|
1844 ... extends = %(url)s/r2.cfg |
|
1845 ... |
|
1846 ... [debug] |
|
1847 ... recipe = recipes:debug |
|
1848 ... name = base |
|
1849 ... """ % dict(url=server_url)) |
|
1850 |
|
1851 |
|
1852 >>> print system(buildout+ ' -c client.cfg'), |
|
1853 Develop: '/sample-buildout/recipes' |
|
1854 Uninstalling debug. |
|
1855 Installing debug. |
|
1856 name base |
|
1857 op1 r1 1 |
|
1858 op2 r2 2 |
|
1859 op3 r2 3 |
|
1860 recipe recipes:debug |
|
1861 |
|
1862 Here we specified a URL for the file we extended. The file we |
|
1863 downloaded, itself referred to a file on the server using a relative |
|
1864 URL reference. Relative references are interpreted relative to the |
|
1865 base URL when they appear in configuration files loaded via URL. |
|
1866 |
|
1867 We can also specify a URL as the configuration file to be used by a |
|
1868 buildout. |
|
1869 |
|
1870 >>> os.remove('client.cfg') |
|
1871 >>> write(server_data, 'remote.cfg', |
|
1872 ... """ |
|
1873 ... [buildout] |
|
1874 ... develop = recipes |
|
1875 ... parts = debug |
|
1876 ... extends = r2.cfg |
|
1877 ... |
|
1878 ... [debug] |
|
1879 ... recipe = recipes:debug |
|
1880 ... name = remote |
|
1881 ... """) |
|
1882 |
|
1883 >>> print system(buildout + ' -c ' + server_url + '/remote.cfg'), |
|
1884 While: |
|
1885 Initializing. |
|
1886 Error: Missing option: buildout:directory |
|
1887 |
|
1888 Normally, the buildout directory defaults to directory |
|
1889 containing a configuration file. This won't work for configuration |
|
1890 files loaded from URLs. In this case, the buildout directory would |
|
1891 normally be defined on the command line: |
|
1892 |
|
1893 >>> print system(buildout |
|
1894 ... + ' -c ' + server_url + '/remote.cfg' |
|
1895 ... + ' buildout:directory=' + sample_buildout |
|
1896 ... ), |
|
1897 Develop: '/sample-buildout/recipes' |
|
1898 Uninstalling debug. |
|
1899 Installing debug. |
|
1900 name remote |
|
1901 op1 r1 1 |
|
1902 op2 r2 2 |
|
1903 op3 r2 3 |
|
1904 recipe recipes:debug |
|
1905 |
|
1906 User defaults |
|
1907 ------------- |
|
1908 |
|
1909 If the file $HOME/.buildout/default.cfg, exists, it is read before |
|
1910 reading the configuration file. ($HOME is the value of the HOME |
|
1911 environment variable. The '/' is replaced by the operating system file |
|
1912 delimiter.) |
|
1913 |
|
1914 >>> old_home = os.environ['HOME'] |
|
1915 >>> home = tmpdir('home') |
|
1916 >>> mkdir(home, '.buildout') |
|
1917 >>> write(home, '.buildout', 'default.cfg', |
|
1918 ... """ |
|
1919 ... [debug] |
|
1920 ... op1 = 1 |
|
1921 ... op7 = 7 |
|
1922 ... """) |
|
1923 |
|
1924 >>> os.environ['HOME'] = home |
|
1925 >>> print system(buildout), |
|
1926 Develop: '/sample-buildout/recipes' |
|
1927 Uninstalling debug. |
|
1928 Installing debug. |
|
1929 name base |
|
1930 op buildout |
|
1931 op1 b1 1 |
|
1932 op2 b2 2 |
|
1933 op3 b2 3 |
|
1934 op4 b3 4 |
|
1935 op5 b3base 5 |
|
1936 op7 7 |
|
1937 recipe recipes:debug |
|
1938 |
|
1939 A buildout command-line argument, -U, can be used to suppress reading |
|
1940 user defaults: |
|
1941 |
|
1942 >>> print system(buildout + ' -U'), |
|
1943 Develop: '/sample-buildout/recipes' |
|
1944 Uninstalling debug. |
|
1945 Installing debug. |
|
1946 name base |
|
1947 op buildout |
|
1948 op1 b1 1 |
|
1949 op2 b2 2 |
|
1950 op3 b2 3 |
|
1951 op4 b3 4 |
|
1952 op5 b3base 5 |
|
1953 recipe recipes:debug |
|
1954 |
|
1955 >>> os.environ['HOME'] = old_home |
|
1956 |
|
1957 Log level |
|
1958 --------- |
|
1959 |
|
1960 We can control the level of logging by specifying a log level in out |
|
1961 configuration file. For example, so suppress info messages, we can |
|
1962 set the logging level to WARNING |
|
1963 |
|
1964 >>> write(sample_buildout, 'buildout.cfg', |
|
1965 ... """ |
|
1966 ... [buildout] |
|
1967 ... log-level = WARNING |
|
1968 ... extends = b1.cfg b2.cfg |
|
1969 ... """) |
|
1970 |
|
1971 >>> print system(buildout), |
|
1972 name base |
|
1973 op1 b1 1 |
|
1974 op2 b2 2 |
|
1975 op3 b2 3 |
|
1976 recipe recipes:debug |
|
1977 |
|
1978 Uninstall recipes |
|
1979 ----------------- |
|
1980 |
|
1981 As we've seen, when parts are installed, buildout keeps track of files |
|
1982 and directories that they create. When the parts are uninstalled these |
|
1983 files and directories are deleted. |
|
1984 |
|
1985 Sometimes more clean up is needed. For example, a recipe might add a |
|
1986 system service by calling chkconfig --add during installation. Later |
|
1987 during uninstallation, chkconfig --del will need to be called to |
|
1988 remove the system service. |
|
1989 |
|
1990 In order to deal with these uninstallation issues, you can register |
|
1991 uninstall recipes. Uninstall recipes are registered using the |
|
1992 'zc.buildout.uninstall' entry point. Parts specify uninstall recipes |
|
1993 using the 'uninstall' option. |
|
1994 |
|
1995 In comparison to regular recipes, uninstall recipes are much |
|
1996 simpler. They are simply callable objects that accept the name of the |
|
1997 part to be uninstalled and the part's options dictionary. Uninstall |
|
1998 recipes don't have access to the part itself since it maybe not be |
|
1999 able to be instantiated at uninstallation time. |
|
2000 |
|
2001 Here's a recipe that simulates installation of a system service, along |
|
2002 with an uninstall recipe that simulates removing the service. |
|
2003 |
|
2004 >>> write(sample_buildout, 'recipes', 'service.py', |
|
2005 ... """ |
|
2006 ... class Service: |
|
2007 ... |
|
2008 ... def __init__(self, buildout, name, options): |
|
2009 ... self.buildout = buildout |
|
2010 ... self.name = name |
|
2011 ... self.options = options |
|
2012 ... |
|
2013 ... def install(self): |
|
2014 ... print "chkconfig --add %s" % self.options['script'] |
|
2015 ... return () |
|
2016 ... |
|
2017 ... def update(self): |
|
2018 ... pass |
|
2019 ... |
|
2020 ... |
|
2021 ... def uninstall_service(name, options): |
|
2022 ... print "chkconfig --del %s" % options['script'] |
|
2023 ... """) |
|
2024 |
|
2025 To use these recipes we must register them using entry points. Make |
|
2026 sure to use the same name for the recipe and uninstall recipe. This is |
|
2027 required to let buildout know which uninstall recipe goes with which |
|
2028 recipe. |
|
2029 |
|
2030 >>> write(sample_buildout, 'recipes', 'setup.py', |
|
2031 ... """ |
|
2032 ... from setuptools import setup |
|
2033 ... entry_points = ( |
|
2034 ... ''' |
|
2035 ... [zc.buildout] |
|
2036 ... mkdir = mkdir:Mkdir |
|
2037 ... debug = debug:Debug |
|
2038 ... service = service:Service |
|
2039 ... |
|
2040 ... [zc.buildout.uninstall] |
|
2041 ... service = service:uninstall_service |
|
2042 ... ''') |
|
2043 ... setup(name="recipes", entry_points=entry_points) |
|
2044 ... """) |
|
2045 |
|
2046 Here's how these recipes could be used in a buildout: |
|
2047 |
|
2048 >>> write(sample_buildout, 'buildout.cfg', |
|
2049 ... """ |
|
2050 ... [buildout] |
|
2051 ... develop = recipes |
|
2052 ... parts = service |
|
2053 ... |
|
2054 ... [service] |
|
2055 ... recipe = recipes:service |
|
2056 ... script = /path/to/script |
|
2057 ... """) |
|
2058 |
|
2059 When the buildout is run the service will be installed |
|
2060 |
|
2061 >>> print system(buildout) |
|
2062 Develop: '/sample-buildout/recipes' |
|
2063 Uninstalling debug. |
|
2064 Installing service. |
|
2065 chkconfig --add /path/to/script |
|
2066 <BLANKLINE> |
|
2067 |
|
2068 The service has been installed. If the buildout is run again with no |
|
2069 changes, the service shouldn't be changed. |
|
2070 |
|
2071 >>> print system(buildout) |
|
2072 Develop: '/sample-buildout/recipes' |
|
2073 Updating service. |
|
2074 <BLANKLINE> |
|
2075 |
|
2076 Now we change the service part to trigger uninstallation and |
|
2077 re-installation. |
|
2078 |
|
2079 >>> write(sample_buildout, 'buildout.cfg', |
|
2080 ... """ |
|
2081 ... [buildout] |
|
2082 ... develop = recipes |
|
2083 ... parts = service |
|
2084 ... |
|
2085 ... [service] |
|
2086 ... recipe = recipes:service |
|
2087 ... script = /path/to/a/different/script |
|
2088 ... """) |
|
2089 |
|
2090 >>> print system(buildout) |
|
2091 Develop: '/sample-buildout/recipes' |
|
2092 Uninstalling service. |
|
2093 Running uninstall recipe. |
|
2094 chkconfig --del /path/to/script |
|
2095 Installing service. |
|
2096 chkconfig --add /path/to/a/different/script |
|
2097 <BLANKLINE> |
|
2098 |
|
2099 Now we remove the service part, and add another part. |
|
2100 |
|
2101 >>> write(sample_buildout, 'buildout.cfg', |
|
2102 ... """ |
|
2103 ... [buildout] |
|
2104 ... develop = recipes |
|
2105 ... parts = debug |
|
2106 ... |
|
2107 ... [debug] |
|
2108 ... recipe = recipes:debug |
|
2109 ... """) |
|
2110 |
|
2111 >>> print system(buildout) |
|
2112 Develop: '/sample-buildout/recipes' |
|
2113 Uninstalling service. |
|
2114 Running uninstall recipe. |
|
2115 chkconfig --del /path/to/a/different/script |
|
2116 Installing debug. |
|
2117 recipe recipes:debug |
|
2118 <BLANKLINE> |
|
2119 |
|
2120 Uninstall recipes don't have to take care of removing all the files |
|
2121 and directories created by the part. This is still done automatically, |
|
2122 following the execution of the uninstall recipe. An upshot is that an |
|
2123 uninstallation recipe can access files and directories created by a |
|
2124 recipe before they are deleted. |
|
2125 |
|
2126 For example, here's an uninstallation recipe that simulates backing up |
|
2127 a directory before it is deleted. It is designed to work with the |
|
2128 mkdir recipe introduced earlier. |
|
2129 |
|
2130 >>> write(sample_buildout, 'recipes', 'backup.py', |
|
2131 ... """ |
|
2132 ... import os |
|
2133 ... def backup_directory(name, options): |
|
2134 ... path = options['path'] |
|
2135 ... size = len(os.listdir(path)) |
|
2136 ... print "backing up directory %s of size %s" % (path, size) |
|
2137 ... """) |
|
2138 |
|
2139 It must be registered with the zc.buildout.uninstall entry |
|
2140 point. Notice how it is given the name 'mkdir' to associate it with |
|
2141 the mkdir recipe. |
|
2142 |
|
2143 >>> write(sample_buildout, 'recipes', 'setup.py', |
|
2144 ... """ |
|
2145 ... from setuptools import setup |
|
2146 ... entry_points = ( |
|
2147 ... ''' |
|
2148 ... [zc.buildout] |
|
2149 ... mkdir = mkdir:Mkdir |
|
2150 ... debug = debug:Debug |
|
2151 ... service = service:Service |
|
2152 ... |
|
2153 ... [zc.buildout.uninstall] |
|
2154 ... uninstall_service = service:uninstall_service |
|
2155 ... mkdir = backup:backup_directory |
|
2156 ... ''') |
|
2157 ... setup(name="recipes", entry_points=entry_points) |
|
2158 ... """) |
|
2159 |
|
2160 Now we can use it with a mkdir part. |
|
2161 |
|
2162 >>> write(sample_buildout, 'buildout.cfg', |
|
2163 ... """ |
|
2164 ... [buildout] |
|
2165 ... develop = recipes |
|
2166 ... parts = dir debug |
|
2167 ... |
|
2168 ... [dir] |
|
2169 ... recipe = recipes:mkdir |
|
2170 ... path = my_directory |
|
2171 ... |
|
2172 ... [debug] |
|
2173 ... recipe = recipes:debug |
|
2174 ... """) |
|
2175 |
|
2176 Run the buildout to install the part. |
|
2177 |
|
2178 >>> print system(buildout) |
|
2179 Develop: '/sample-buildout/recipes' |
|
2180 Uninstalling debug. |
|
2181 Installing dir. |
|
2182 dir: Creating directory my_directory |
|
2183 Installing debug. |
|
2184 recipe recipes:debug |
|
2185 <BLANKLINE> |
|
2186 |
|
2187 Now we remove the part from the configuration file. |
|
2188 |
|
2189 >>> write(sample_buildout, 'buildout.cfg', |
|
2190 ... """ |
|
2191 ... [buildout] |
|
2192 ... develop = recipes |
|
2193 ... parts = debug |
|
2194 ... |
|
2195 ... [debug] |
|
2196 ... recipe = recipes:debug |
|
2197 ... """) |
|
2198 |
|
2199 When the buildout is run the part is removed, and the uninstall recipe |
|
2200 is run before the directory is deleted. |
|
2201 |
|
2202 >>> print system(buildout) |
|
2203 Develop: '/sample-buildout/recipes' |
|
2204 Uninstalling dir. |
|
2205 Running uninstall recipe. |
|
2206 backing up directory /sample-buildout/my_directory of size 0 |
|
2207 Updating debug. |
|
2208 recipe recipes:debug |
|
2209 <BLANKLINE> |
|
2210 |
|
2211 Now we will return the registration to normal for the benefit of the |
|
2212 rest of the examples. |
|
2213 |
|
2214 >>> write(sample_buildout, 'recipes', 'setup.py', |
|
2215 ... """ |
|
2216 ... from setuptools import setup |
|
2217 ... entry_points = ( |
|
2218 ... ''' |
|
2219 ... [zc.buildout] |
|
2220 ... mkdir = mkdir:Mkdir |
|
2221 ... debug = debug:Debug |
|
2222 ... ''') |
|
2223 ... setup(name="recipes", entry_points=entry_points) |
|
2224 ... """) |
|
2225 |
|
2226 |
|
2227 Command-line usage |
|
2228 ------------------ |
|
2229 |
|
2230 A number of arguments can be given on the buildout command line. The |
|
2231 command usage is:: |
|
2232 |
|
2233 buildout [options and assignments] [command [command arguments]] |
|
2234 |
|
2235 The following options are supported: |
|
2236 |
|
2237 -h (or --help) |
|
2238 Print basic usage information. If this option is used, then all |
|
2239 other options are ignored. |
|
2240 |
|
2241 -c filename |
|
2242 The -c option can be used to specify a configuration file, rather than |
|
2243 buildout.cfg in the current directory. |
|
2244 |
|
2245 |
|
2246 -t socket_timeout |
|
2247 |
|
2248 Specify the socket timeout in seconds. |
|
2249 |
|
2250 -v |
|
2251 Increment the verbosity by 10. The verbosity is used to adjust |
|
2252 the logging level. The verbosity is subtracted from the numeric |
|
2253 value of the log-level option specified in the configuration file. |
|
2254 |
|
2255 -q |
|
2256 Decrement the verbosity by 10. |
|
2257 |
|
2258 -U |
|
2259 Don't read user-default configuration. |
|
2260 |
|
2261 -o |
|
2262 Run in off-line mode. This is equivalent to the assignment |
|
2263 buildout:offline=true. |
|
2264 |
|
2265 -O |
|
2266 Run in non-off-line mode. This is equivalent to the assignment |
|
2267 buildout:offline=false. This is the default buildout mode. The |
|
2268 -O option would normally be used to override a true offline |
|
2269 setting in a configuration file. |
|
2270 |
|
2271 -n |
|
2272 Run in newest mode. This is equivalent to the assignment |
|
2273 buildout:newest=true. With this setting, which is the default, |
|
2274 buildout will try to find the newest versions of distributions |
|
2275 available that satisfy its requirements. |
|
2276 |
|
2277 -N |
|
2278 Run in non-newest mode. This is equivalent to the assignment |
|
2279 buildout:newest=false. With this setting, buildout will not seek |
|
2280 new distributions if installed distributions satisfy it's |
|
2281 requirements. |
|
2282 |
|
2283 Assignments are of the form:: |
|
2284 |
|
2285 section_name:option_name=value |
|
2286 |
|
2287 Options and assignments can be given in any order. |
|
2288 |
|
2289 Here's an example: |
|
2290 |
|
2291 >>> write(sample_buildout, 'other.cfg', |
|
2292 ... """ |
|
2293 ... [buildout] |
|
2294 ... develop = recipes |
|
2295 ... parts = debug |
|
2296 ... installed = .other.cfg |
|
2297 ... log-level = WARNING |
|
2298 ... |
|
2299 ... [debug] |
|
2300 ... name = other |
|
2301 ... recipe = recipes:debug |
|
2302 ... """) |
|
2303 |
|
2304 Note that we used the installed buildout option to specify an |
|
2305 alternate file to store information about installed parts. |
|
2306 |
|
2307 >>> print system(buildout+' -c other.cfg debug:op1=foo -v'), |
|
2308 Develop: '/sample-buildout/recipes' |
|
2309 Installing debug. |
|
2310 name other |
|
2311 op1 foo |
|
2312 recipe recipes:debug |
|
2313 |
|
2314 Here we used the -c option to specify an alternate configuration file, |
|
2315 and the -v option to increase the level of logging from the default, |
|
2316 WARNING. |
|
2317 |
|
2318 Options can also be combined in the usual Unix way, as in: |
|
2319 |
|
2320 >>> print system(buildout+' -vcother.cfg debug:op1=foo'), |
|
2321 Develop: '/sample-buildout/recipes' |
|
2322 Updating debug. |
|
2323 name other |
|
2324 op1 foo |
|
2325 recipe recipes:debug |
|
2326 |
|
2327 Here we combined the -v and -c options with the configuration file |
|
2328 name. Note that the -c option has to be last, because it takes an |
|
2329 argument. |
|
2330 |
|
2331 >>> os.remove(os.path.join(sample_buildout, 'other.cfg')) |
|
2332 >>> os.remove(os.path.join(sample_buildout, '.other.cfg')) |
|
2333 |
|
2334 The most commonly used command is 'install' and it takes a list of |
|
2335 parts to install. if any parts are specified, only those parts are |
|
2336 installed. To illustrate this, we'll update our configuration and run |
|
2337 the buildout in the usual way: |
|
2338 |
|
2339 >>> write(sample_buildout, 'buildout.cfg', |
|
2340 ... """ |
|
2341 ... [buildout] |
|
2342 ... develop = recipes |
|
2343 ... parts = debug d1 d2 d3 |
|
2344 ... |
|
2345 ... [d1] |
|
2346 ... recipe = recipes:mkdir |
|
2347 ... path = d1 |
|
2348 ... |
|
2349 ... [d2] |
|
2350 ... recipe = recipes:mkdir |
|
2351 ... path = d2 |
|
2352 ... |
|
2353 ... [d3] |
|
2354 ... recipe = recipes:mkdir |
|
2355 ... path = d3 |
|
2356 ... |
|
2357 ... [debug] |
|
2358 ... recipe = recipes:debug |
|
2359 ... """) |
|
2360 |
|
2361 >>> print system(buildout), |
|
2362 Develop: '/sample-buildout/recipes' |
|
2363 Uninstalling debug. |
|
2364 Installing debug. |
|
2365 recipe recipes:debug |
|
2366 Installing d1. |
|
2367 d1: Creating directory d1 |
|
2368 Installing d2. |
|
2369 d2: Creating directory d2 |
|
2370 Installing d3. |
|
2371 d3: Creating directory d3 |
|
2372 |
|
2373 >>> ls(sample_buildout) |
|
2374 - .installed.cfg |
|
2375 - b1.cfg |
|
2376 - b2.cfg |
|
2377 - base.cfg |
|
2378 d bin |
|
2379 - buildout.cfg |
|
2380 d d1 |
|
2381 d d2 |
|
2382 d d3 |
|
2383 d demo |
|
2384 d develop-eggs |
|
2385 d eggs |
|
2386 d parts |
|
2387 d recipes |
|
2388 |
|
2389 >>> cat(sample_buildout, '.installed.cfg') |
|
2390 ... # doctest: +NORMALIZE_WHITESPACE |
|
2391 [buildout] |
|
2392 installed_develop_eggs = /sample-buildout/develop-eggs/recipes.egg-link |
|
2393 parts = debug d1 d2 d3 |
|
2394 <BLANKLINE> |
|
2395 [debug] |
|
2396 __buildout_installed__ = |
|
2397 __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== |
|
2398 recipe = recipes:debug |
|
2399 <BLANKLINE> |
|
2400 [d1] |
|
2401 __buildout_installed__ = /sample-buildout/d1 |
|
2402 __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== |
|
2403 path = /sample-buildout/d1 |
|
2404 recipe = recipes:mkdir |
|
2405 <BLANKLINE> |
|
2406 [d2] |
|
2407 __buildout_installed__ = /sample-buildout/d2 |
|
2408 __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== |
|
2409 path = /sample-buildout/d2 |
|
2410 recipe = recipes:mkdir |
|
2411 <BLANKLINE> |
|
2412 [d3] |
|
2413 __buildout_installed__ = /sample-buildout/d3 |
|
2414 __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== |
|
2415 path = /sample-buildout/d3 |
|
2416 recipe = recipes:mkdir |
|
2417 |
|
2418 Now we'll update our configuration file: |
|
2419 |
|
2420 >>> write(sample_buildout, 'buildout.cfg', |
|
2421 ... """ |
|
2422 ... [buildout] |
|
2423 ... develop = recipes |
|
2424 ... parts = debug d2 d3 d4 |
|
2425 ... |
|
2426 ... [d2] |
|
2427 ... recipe = recipes:mkdir |
|
2428 ... path = data2 |
|
2429 ... |
|
2430 ... [d3] |
|
2431 ... recipe = recipes:mkdir |
|
2432 ... path = data3 |
|
2433 ... |
|
2434 ... [d4] |
|
2435 ... recipe = recipes:mkdir |
|
2436 ... path = ${d2:path}-extra |
|
2437 ... |
|
2438 ... [debug] |
|
2439 ... recipe = recipes:debug |
|
2440 ... x = 1 |
|
2441 ... """) |
|
2442 |
|
2443 and run the buildout specifying just d3 and d4: |
|
2444 |
|
2445 >>> print system(buildout+' install d3 d4'), |
|
2446 Develop: '/sample-buildout/recipes' |
|
2447 Uninstalling d3. |
|
2448 Installing d3. |
|
2449 d3: Creating directory data3 |
|
2450 Installing d4. |
|
2451 d4: Creating directory data2-extra |
|
2452 |
|
2453 >>> ls(sample_buildout) |
|
2454 - .installed.cfg |
|
2455 - b1.cfg |
|
2456 - b2.cfg |
|
2457 - base.cfg |
|
2458 d bin |
|
2459 - buildout.cfg |
|
2460 d d1 |
|
2461 d d2 |
|
2462 d data2-extra |
|
2463 d data3 |
|
2464 d demo |
|
2465 d develop-eggs |
|
2466 d eggs |
|
2467 d parts |
|
2468 d recipes |
|
2469 |
|
2470 Only the d3 and d4 recipes ran. d3 was removed and data3 and data2-extra |
|
2471 were created. |
|
2472 |
|
2473 The .installed.cfg is only updated for the recipes that ran: |
|
2474 |
|
2475 >>> cat(sample_buildout, '.installed.cfg') |
|
2476 ... # doctest: +NORMALIZE_WHITESPACE |
|
2477 [buildout] |
|
2478 installed_develop_eggs = /sample-buildout/develop-eggs/recipes.egg-link |
|
2479 parts = debug d1 d2 d3 d4 |
|
2480 <BLANKLINE> |
|
2481 [debug] |
|
2482 __buildout_installed__ = |
|
2483 __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== |
|
2484 recipe = recipes:debug |
|
2485 <BLANKLINE> |
|
2486 [d1] |
|
2487 __buildout_installed__ = /sample-buildout/d1 |
|
2488 __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== |
|
2489 path = /sample-buildout/d1 |
|
2490 recipe = recipes:mkdir |
|
2491 <BLANKLINE> |
|
2492 [d2] |
|
2493 __buildout_installed__ = /sample-buildout/d2 |
|
2494 __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== |
|
2495 path = /sample-buildout/d2 |
|
2496 recipe = recipes:mkdir |
|
2497 <BLANKLINE> |
|
2498 [d3] |
|
2499 __buildout_installed__ = /sample-buildout/data3 |
|
2500 __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== |
|
2501 path = /sample-buildout/data3 |
|
2502 recipe = recipes:mkdir |
|
2503 <BLANKLINE> |
|
2504 [d4] |
|
2505 __buildout_installed__ = /sample-buildout/data2-extra |
|
2506 __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== |
|
2507 path = /sample-buildout/data2-extra |
|
2508 recipe = recipes:mkdir |
|
2509 |
|
2510 Note that the installed data for debug, d1, and d2 haven't changed, |
|
2511 because we didn't install those parts and that the d1 and d2 |
|
2512 directories are still there. |
|
2513 |
|
2514 Now, if we run the buildout without the install command: |
|
2515 |
|
2516 >>> print system(buildout), |
|
2517 Develop: '/sample-buildout/recipes' |
|
2518 Uninstalling d2. |
|
2519 Uninstalling d1. |
|
2520 Uninstalling debug. |
|
2521 Installing debug. |
|
2522 recipe recipes:debug |
|
2523 x 1 |
|
2524 Installing d2. |
|
2525 d2: Creating directory data2 |
|
2526 Updating d3. |
|
2527 Updating d4. |
|
2528 |
|
2529 We see the output of the debug recipe and that data2 was created. We |
|
2530 also see that d1 and d2 have gone away: |
|
2531 |
|
2532 >>> ls(sample_buildout) |
|
2533 - .installed.cfg |
|
2534 - b1.cfg |
|
2535 - b2.cfg |
|
2536 - base.cfg |
|
2537 d bin |
|
2538 - buildout.cfg |
|
2539 d data2 |
|
2540 d data2-extra |
|
2541 d data3 |
|
2542 d demo |
|
2543 d develop-eggs |
|
2544 d eggs |
|
2545 d parts |
|
2546 d recipes |
|
2547 |
|
2548 Alternate directory and file locations |
|
2549 -------------------------------------- |
|
2550 |
|
2551 The buildout normally puts the bin, eggs, and parts directories in the |
|
2552 directory in the directory containing the configuration file. You can |
|
2553 provide alternate locations, and even names for these directories. |
|
2554 |
|
2555 >>> alt = tmpdir('sample-alt') |
|
2556 |
|
2557 >>> write(sample_buildout, 'buildout.cfg', |
|
2558 ... """ |
|
2559 ... [buildout] |
|
2560 ... develop = recipes |
|
2561 ... parts = |
|
2562 ... develop-eggs-directory = %(developbasket)s |
|
2563 ... eggs-directory = %(basket)s |
|
2564 ... bin-directory = %(scripts)s |
|
2565 ... parts-directory = %(work)s |
|
2566 ... """ % dict( |
|
2567 ... developbasket = os.path.join(alt, 'developbasket'), |
|
2568 ... basket = os.path.join(alt, 'basket'), |
|
2569 ... scripts = os.path.join(alt, 'scripts'), |
|
2570 ... work = os.path.join(alt, 'work'), |
|
2571 ... )) |
|
2572 |
|
2573 >>> print system(buildout), |
|
2574 Creating directory '/sample-alt/scripts'. |
|
2575 Creating directory '/sample-alt/work'. |
|
2576 Creating directory '/sample-alt/basket'. |
|
2577 Creating directory '/sample-alt/developbasket'. |
|
2578 Develop: '/sample-buildout/recipes' |
|
2579 Uninstalling d4. |
|
2580 Uninstalling d3. |
|
2581 Uninstalling d2. |
|
2582 Uninstalling debug. |
|
2583 |
|
2584 >>> ls(alt) |
|
2585 d basket |
|
2586 d developbasket |
|
2587 d scripts |
|
2588 d work |
|
2589 |
|
2590 >>> ls(alt, 'developbasket') |
|
2591 - recipes.egg-link |
|
2592 |
|
2593 You can also specify an alternate buildout directory: |
|
2594 |
|
2595 >>> rmdir(alt) |
|
2596 >>> alt = tmpdir('sample-alt') |
|
2597 |
|
2598 >>> write(sample_buildout, 'buildout.cfg', |
|
2599 ... """ |
|
2600 ... [buildout] |
|
2601 ... directory = %(alt)s |
|
2602 ... develop = %(recipes)s |
|
2603 ... parts = |
|
2604 ... """ % dict( |
|
2605 ... alt=alt, |
|
2606 ... recipes=os.path.join(sample_buildout, 'recipes'), |
|
2607 ... )) |
|
2608 |
|
2609 >>> print system(buildout), |
|
2610 Creating directory '/sample-alt/bin'. |
|
2611 Creating directory '/sample-alt/parts'. |
|
2612 Creating directory '/sample-alt/eggs'. |
|
2613 Creating directory '/sample-alt/develop-eggs'. |
|
2614 Develop: '/sample-buildout/recipes' |
|
2615 |
|
2616 >>> ls(alt) |
|
2617 - .installed.cfg |
|
2618 d bin |
|
2619 d develop-eggs |
|
2620 d eggs |
|
2621 d parts |
|
2622 |
|
2623 >>> ls(alt, 'develop-eggs') |
|
2624 - recipes.egg-link |
|
2625 |
|
2626 Logging control |
|
2627 --------------- |
|
2628 |
|
2629 Three buildout options are used to control logging: |
|
2630 |
|
2631 log-level |
|
2632 specifies the log level |
|
2633 |
|
2634 verbosity |
|
2635 adjusts the log level |
|
2636 |
|
2637 log-format |
|
2638 allows an alternate logging for mat to be specified |
|
2639 |
|
2640 We've already seen the log level and verbosity. Let's look at an example |
|
2641 of changing the format: |
|
2642 |
|
2643 >>> write(sample_buildout, 'buildout.cfg', |
|
2644 ... """ |
|
2645 ... [buildout] |
|
2646 ... develop = recipes |
|
2647 ... parts = |
|
2648 ... log-level = 25 |
|
2649 ... verbosity = 5 |
|
2650 ... log-format = %(levelname)s %(message)s |
|
2651 ... """) |
|
2652 |
|
2653 Here, we've changed the format to include the log-level name, rather |
|
2654 than the logger name. |
|
2655 |
|
2656 We've also illustrated, with a contrived example, that the log level |
|
2657 can be a numeric value and that the verbosity can be specified in the |
|
2658 configuration file. Because the verbosity is subtracted from the log |
|
2659 level, we get a final log level of 20, which is the INFO level. |
|
2660 |
|
2661 >>> print system(buildout), |
|
2662 INFO Develop: '/sample-buildout/recipes' |
|
2663 |
|
2664 Predefined buildout options |
|
2665 --------------------------- |
|
2666 |
|
2667 Buildouts have a number of predefined options that recipes can use |
|
2668 and that users can override in their configuration files. To see |
|
2669 these, we'll run a minimal buildout configuration with a debug logging |
|
2670 level. One of the features of debug logging is that the configuration |
|
2671 database is shown. |
|
2672 |
|
2673 >>> write(sample_buildout, 'buildout.cfg', |
|
2674 ... """ |
|
2675 ... [buildout] |
|
2676 ... parts = |
|
2677 ... """) |
|
2678 |
|
2679 >>> print system(buildout+' -vv'), # doctest: +NORMALIZE_WHITESPACE |
|
2680 Installing 'zc.buildout', 'setuptools'. |
|
2681 We have a develop egg: zc.buildout X.X. |
|
2682 We have the best distribution that satisfies 'setuptools'. |
|
2683 Picked: setuptools = V.V |
|
2684 <BLANKLINE> |
|
2685 Configuration data: |
|
2686 [buildout] |
|
2687 accept-buildout-test-releases = false |
|
2688 allow-hosts = * |
|
2689 allow-picked-versions = true |
|
2690 allowed-eggs-from-site-packages = * |
|
2691 bin-directory = /sample-buildout/bin |
|
2692 develop-eggs-directory = /sample-buildout/develop-eggs |
|
2693 directory = /sample-buildout |
|
2694 eggs-directory = /sample-buildout/eggs |
|
2695 exec-sitecustomize = true |
|
2696 executable = python |
|
2697 find-links = |
|
2698 include-site-packages = true |
|
2699 install-from-cache = false |
|
2700 installed = /sample-buildout/.installed.cfg |
|
2701 log-format = |
|
2702 log-level = INFO |
|
2703 newest = true |
|
2704 offline = false |
|
2705 parts = |
|
2706 parts-directory = /sample-buildout/parts |
|
2707 prefer-final = false |
|
2708 python = buildout |
|
2709 relative-paths = false |
|
2710 socket-timeout = |
|
2711 unzip = false |
|
2712 use-dependency-links = true |
|
2713 verbosity = 20 |
|
2714 <BLANKLINE> |
|
2715 |
|
2716 All of these options can be overridden by configuration files or by |
|
2717 command-line assignments. We've discussed most of these options |
|
2718 already, but let's review them and touch on some we haven't discussed: |
|
2719 |
|
2720 allowed-eggs-from-site-packages |
|
2721 Sometimes you need or want to control what eggs from site-packages are |
|
2722 used. The allowed-eggs-from-site-packages option allows you to specify a |
|
2723 whitelist of project names that may be included from site-packages. You |
|
2724 can use globs to specify the value. It defaults to a single value of '*', |
|
2725 indicating that any package may come from site-packages. |
|
2726 |
|
2727 Here's a usage example:: |
|
2728 |
|
2729 [buildout] |
|
2730 ... |
|
2731 |
|
2732 allowed-eggs-from-site-packages = |
|
2733 demo |
|
2734 bigdemo |
|
2735 zope.* |
|
2736 |
|
2737 This option interacts with the ``include-site-packages`` option in the |
|
2738 following ways. |
|
2739 |
|
2740 If ``include-site-packages`` is true, then |
|
2741 ``allowed-eggs-from-site-packages`` filters what eggs from site-packages |
|
2742 may be chosen. Therefore, if ``allowed-eggs-from-site-packages`` is an |
|
2743 empty list, then no eggs from site-packages are chosen, but site-packages |
|
2744 will still be included at the end of path lists. |
|
2745 |
|
2746 If ``include-site-packages`` is false, the value of |
|
2747 ``allowed-eggs-from-site-packages`` is irrelevant. |
|
2748 |
|
2749 See the ``include-site-packages`` description for more information. |
|
2750 |
|
2751 bin-directory |
|
2752 The directory path where scripts are written. This can be a |
|
2753 relative path, which is interpreted relative to the directory |
|
2754 option. |
|
2755 |
|
2756 develop-eggs-directory |
|
2757 The directory path where development egg links are created for software |
|
2758 being created in the local project. This can be a relative path, |
|
2759 which is interpreted relative to the directory option. |
|
2760 |
|
2761 directory |
|
2762 The buildout directory. This is the base for other buildout file |
|
2763 and directory locations, when relative locations are used. |
|
2764 |
|
2765 eggs-directory |
|
2766 The directory path where downloaded eggs are put. It is common to share |
|
2767 this directory across buildouts. Eggs in this directory should |
|
2768 *never* be modified. This can be a relative path, which is |
|
2769 interpreted relative to the directory option. |
|
2770 |
|
2771 exec-sitecustomize |
|
2772 Normally the Python's real sitecustomize module is processed. |
|
2773 If you do not want it to be processed in order to increase the |
|
2774 repeatability of your buildout, set this value to 'false'. This will |
|
2775 be honored irrespective of the setting for include-site-packages. |
|
2776 This option will be honored by some recipes and not others. |
|
2777 z3c.recipe.scripts honors this and zc.recipe.egg does not, for |
|
2778 instance. |
|
2779 |
|
2780 executable |
|
2781 The Python executable used to run the buildout. See the python |
|
2782 option below. |
|
2783 |
|
2784 include-site-packages |
|
2785 You can choose not to have the site-packages of the underlying Python |
|
2786 available to your script or interpreter, in addition to the packages |
|
2787 from your eggs. This can increase repeatability for your buildout. |
|
2788 This option will be better used by some recipes than others. |
|
2789 z3c.recipe.scripts honors this fully and zc.recipe.egg only |
|
2790 partially, for instance. |
|
2791 |
|
2792 installed |
|
2793 The file path where information about the results of the previous |
|
2794 buildout run is written. This can be a relative path, which is |
|
2795 interpreted relative to the directory option. This file provides |
|
2796 an inventory of installed parts with information needed to decide |
|
2797 which if any parts need to be uninstalled. |
|
2798 |
|
2799 log-format |
|
2800 The format used for logging messages. |
|
2801 |
|
2802 log-level |
|
2803 The log level before verbosity adjustment |
|
2804 |
|
2805 parts |
|
2806 A white space separated list of parts to be installed. |
|
2807 |
|
2808 parts-directory |
|
2809 A working directory that parts can used to store data. |
|
2810 |
|
2811 python |
|
2812 The name of a section containing information about the default |
|
2813 Python interpreter. Recipes that need a installation |
|
2814 typically have options to tell them which Python installation to |
|
2815 use. By convention, if a section-specific option isn't used, the |
|
2816 option is looked for in the buildout section. The option must |
|
2817 point to a section with an executable option giving the path to a |
|
2818 Python executable. By default, the buildout section defines the |
|
2819 default Python as the Python used to run the buildout. |
|
2820 |
|
2821 relative-paths |
|
2822 The paths generated by zc.buildout are absolute by default, and this |
|
2823 option is ``false``. However, if you set this value to be ``true``, |
|
2824 bin/buildout will be generated with code that makes the paths relative. |
|
2825 Some recipes, such as zc.recipe.egg and z3c.recipe.scripts, honor this |
|
2826 value as well. |
|
2827 |
|
2828 unzip |
|
2829 By default, zc.buildout doesn't unzip zip-safe eggs ("unzip = false"). |
|
2830 This follows the policy followed by setuptools itself. Experience shows |
|
2831 this policy to to be inconvenient. Zipped eggs make debugging more |
|
2832 difficult and often import more slowly. You can include an unzip option in |
|
2833 the buildout section to change the default unzipping policy ("unzip = |
|
2834 true"). |
|
2835 |
|
2836 use-dependency-links |
|
2837 By default buildout will obey the setuptools dependency_links metadata |
|
2838 when it looks for dependencies. This behavior can be controlled with |
|
2839 the use-dependency-links buildout option:: |
|
2840 |
|
2841 [buildout] |
|
2842 ... |
|
2843 use-dependency-links = false |
|
2844 |
|
2845 The option defaults to true. If you set it to false, then dependency |
|
2846 links are only looked for in the locations specified by find-links. |
|
2847 |
|
2848 verbosity |
|
2849 A log-level adjustment. Typically, this is set via the -q and -v |
|
2850 command-line options. |
|
2851 |
|
2852 |
|
2853 Creating new buildouts and bootstrapping |
|
2854 ---------------------------------------- |
|
2855 |
|
2856 If zc.buildout is installed, you can use it to create a new buildout |
|
2857 with it's own local copies of zc.buildout and setuptools and with |
|
2858 local buildout scripts. |
|
2859 |
|
2860 >>> sample_bootstrapped = tmpdir('sample-bootstrapped') |
|
2861 |
|
2862 >>> print system(buildout |
|
2863 ... +' -c'+os.path.join(sample_bootstrapped, 'setup.cfg') |
|
2864 ... +' init'), |
|
2865 Creating '/sample-bootstrapped/setup.cfg'. |
|
2866 Creating directory '/sample-bootstrapped/bin'. |
|
2867 Creating directory '/sample-bootstrapped/parts'. |
|
2868 Creating directory '/sample-bootstrapped/eggs'. |
|
2869 Creating directory '/sample-bootstrapped/develop-eggs'. |
|
2870 Generated script '/sample-bootstrapped/bin/buildout'. |
|
2871 |
|
2872 Note that a basic setup.cfg was created for us. |
|
2873 |
|
2874 >>> ls(sample_bootstrapped) |
|
2875 d bin |
|
2876 d develop-eggs |
|
2877 d eggs |
|
2878 d parts |
|
2879 - setup.cfg |
|
2880 |
|
2881 >>> ls(sample_bootstrapped, 'bin') |
|
2882 - buildout |
|
2883 |
|
2884 >>> _ = (ls(sample_bootstrapped, 'eggs'), |
|
2885 ... ls(sample_bootstrapped, 'develop-eggs')) |
|
2886 - setuptools-0.6-py2.3.egg |
|
2887 - zc.buildout-1.0-py2.3.egg |
|
2888 |
|
2889 (We list both the eggs and develop-eggs directories because the |
|
2890 buildout or setuptools egg could be installed in the develop-eggs |
|
2891 directory if the original buildout had develop eggs for either |
|
2892 buildout or setuptools.) |
|
2893 |
|
2894 If relative-paths is ``true``, the buildout script uses relative paths. |
|
2895 |
|
2896 >>> write(sample_bootstrapped, 'setup.cfg', |
|
2897 ... ''' |
|
2898 ... [buildout] |
|
2899 ... relative-paths = true |
|
2900 ... parts = |
|
2901 ... ''') |
|
2902 |
|
2903 >>> print system(buildout |
|
2904 ... +' -c'+os.path.join(sample_bootstrapped, 'setup.cfg') |
|
2905 ... +' bootstrap'), |
|
2906 Generated script '/sample-bootstrapped/bin/buildout'. |
|
2907 |
|
2908 >>> buildout_script = join(sample_bootstrapped, 'bin', 'buildout') |
|
2909 >>> import sys |
|
2910 >>> if sys.platform.startswith('win'): |
|
2911 ... buildout_script += '-script.py' |
|
2912 >>> print open(buildout_script).read() # doctest: +ELLIPSIS |
|
2913 #!... -S |
|
2914 <BLANKLINE> |
|
2915 import os |
|
2916 <BLANKLINE> |
|
2917 join = os.path.join |
|
2918 base = os.path.dirname(os.path.abspath(os.path.realpath(__file__))) |
|
2919 base = os.path.dirname(base) |
|
2920 <BLANKLINE> |
|
2921 import sys |
|
2922 sys.path[0:0] = [ |
|
2923 join(base, 'parts/buildout'), |
|
2924 ] |
|
2925 <BLANKLINE> |
|
2926 <BLANKLINE> |
|
2927 import os |
|
2928 path = sys.path[0] |
|
2929 if os.environ.get('PYTHONPATH'): |
|
2930 path = os.pathsep.join([path, os.environ['PYTHONPATH']]) |
|
2931 os.environ['BUILDOUT_ORIGINAL_PYTHONPATH'] = os.environ.get('PYTHONPATH', '') |
|
2932 os.environ['PYTHONPATH'] = path |
|
2933 import site # imports custom buildout-generated site.py |
|
2934 <BLANKLINE> |
|
2935 import zc.buildout.buildout |
|
2936 <BLANKLINE> |
|
2937 if __name__ == '__main__': |
|
2938 zc.buildout.buildout.main() |
|
2939 <BLANKLINE> |
|
2940 |
|
2941 |
|
2942 Note that, in the above two examples, the buildout script was installed |
|
2943 but not run. To run the buildout, we'd have to run the installed |
|
2944 buildout script. |
|
2945 |
|
2946 If we have an existing buildout that already has a buildout.cfg, we'll |
|
2947 normally use the bootstrap command instead of init. It will complain |
|
2948 if there isn't a configuration file: |
|
2949 |
|
2950 >>> sample_bootstrapped2 = tmpdir('sample-bootstrapped2') |
|
2951 |
|
2952 >>> print system(buildout |
|
2953 ... +' -c'+os.path.join(sample_bootstrapped2, 'setup.cfg') |
|
2954 ... +' bootstrap'), |
|
2955 While: |
|
2956 Initializing. |
|
2957 Error: Couldn't open /sample-bootstrapped2/setup.cfg |
|
2958 |
|
2959 >>> write(sample_bootstrapped2, 'setup.cfg', |
|
2960 ... """ |
|
2961 ... [buildout] |
|
2962 ... parts = |
|
2963 ... """) |
|
2964 |
|
2965 >>> print system(buildout |
|
2966 ... +' -c'+os.path.join(sample_bootstrapped2, 'setup.cfg') |
|
2967 ... +' bootstrap'), |
|
2968 Creating directory '/sample-bootstrapped2/bin'. |
|
2969 Creating directory '/sample-bootstrapped2/parts'. |
|
2970 Creating directory '/sample-bootstrapped2/eggs'. |
|
2971 Creating directory '/sample-bootstrapped2/develop-eggs'. |
|
2972 Generated script '/sample-bootstrapped2/bin/buildout'. |
|
2973 |
|
2974 |
|
2975 Newest and Offline Modes |
|
2976 ------------------------ |
|
2977 |
|
2978 By default buildout and recipes will try to find the newest versions |
|
2979 of distributions needed to satisfy requirements. This can be very |
|
2980 time consuming, especially when incrementally working on setting up a |
|
2981 buildout or working on a recipe. The buildout newest option can be |
|
2982 used to to suppress this. If the newest option is set to false, then |
|
2983 new distributions won't be sought if an installed distribution meets |
|
2984 requirements. The newest option can be set to false using the -N |
|
2985 command-line option. |
|
2986 |
|
2987 The offline option goes a bit further. If the buildout offline option |
|
2988 is given a value of "true", the buildout and recipes that are aware of |
|
2989 the option will avoid doing network access. This is handy when |
|
2990 running the buildout when not connected to the internet. It also |
|
2991 makes buildouts run much faster. This option is typically set using |
|
2992 the buildout -o option. |
|
2993 |
|
2994 Preferring Final Releases |
|
2995 ------------------------- |
|
2996 |
|
2997 Currently, when searching for new releases of your project's |
|
2998 dependencies, the newest available release is used. This isn't usually |
|
2999 ideal, as you may get a development release or alpha releases not ready |
|
3000 to be widely used. You can request that final releases be preferred |
|
3001 using the ``prefer-final`` option in the buildout section:: |
|
3002 |
|
3003 [buildout] |
|
3004 ... |
|
3005 prefer-final = true |
|
3006 |
|
3007 When the ``prefer-final`` option is set to true, then when searching for |
|
3008 new releases, final releases are preferred. If there are final |
|
3009 releases that satisfy distribution requirements, then those releases |
|
3010 are used even if newer non-final releases are available. |
|
3011 |
|
3012 In buildout version 2, all final releases will be preferred by |
|
3013 default--that is ``prefer-final`` will also default to 'true'. You will |
|
3014 then need to use a 'false' value for ``prefer-final`` to get the newest |
|
3015 releases. |
|
3016 |
|
3017 A separate option controls the behavior of the build system itself. |
|
3018 When buildout looks for recipes, extensions, and for updates to itself, |
|
3019 it does prefer final releases by default, as of the 1.5.0 release. The |
|
3020 ``accept-buildout-test-releases`` option will let you override this behavior. |
|
3021 However, it is typically changed by the --accept-buildout-test-releases |
|
3022 option to the bootstrap script, since bootstrapping is the first step to |
|
3023 selecting a buildout. |
|
3024 |
|
3025 Finding distributions |
|
3026 --------------------- |
|
3027 |
|
3028 By default, buildout searches the Python Package Index when looking |
|
3029 for distributions. You can, instead, specify your own index to search |
|
3030 using the `index` option:: |
|
3031 |
|
3032 [buildout] |
|
3033 ... |
|
3034 index = http://index.example.com/ |
|
3035 |
|
3036 This index, or the default of http://pypi.python.org/simple/ if no |
|
3037 index is specified, will always be searched for distributions unless |
|
3038 running buildout with options that prevent searching for |
|
3039 distributions. The latest version of the distribution that meets the |
|
3040 requirements of the buildout will always be used. |
|
3041 |
|
3042 You can also specify more locations to search for distributions using |
|
3043 the `find-links` option. All locations specified will be searched for |
|
3044 distributions along with the package index as described before. |
|
3045 |
|
3046 Locations can be urls:: |
|
3047 |
|
3048 [buildout] |
|
3049 ... |
|
3050 find-links = http://download.zope.org/distribution/ |
|
3051 |
|
3052 They can also be directories on disk:: |
|
3053 |
|
3054 [buildout] |
|
3055 ... |
|
3056 find-links = /some/path |
|
3057 |
|
3058 Finally, they can also be direct paths to distributions:: |
|
3059 |
|
3060 [buildout] |
|
3061 ... |
|
3062 find-links = /some/path/someegg-1.0.0-py2.3.egg |
|
3063 |
|
3064 Any number of locations can be specified in the `find-links` option:: |
|
3065 |
|
3066 [buildout] |
|
3067 ... |
|
3068 find-links = |
|
3069 http://download.zope.org/distribution/ |
|
3070 /some/otherpath |
|
3071 /some/path/someegg-1.0.0-py2.3.egg |
|
3072 |
|
3073 Dependency links |
|
3074 ---------------- |
|
3075 |
|
3076 By default buildout will obey the setuptools dependency_links metadata |
|
3077 when it looks for dependencies. This behavior can be controlled with |
|
3078 the use-dependency-links buildout option:: |
|
3079 |
|
3080 [buildout] |
|
3081 ... |
|
3082 use-dependency-links = false |
|
3083 |
|
3084 The option defaults to true. If you set it to false, then dependency |
|
3085 links are only looked for in the locations specified by find-links. |
|
3086 |
|
3087 Controlling the installation database |
|
3088 ------------------------------------- |
|
3089 |
|
3090 The buildout installed option is used to specify the file used to save |
|
3091 information on installed parts. This option is initialized to |
|
3092 ".installed.cfg", but it can be overridden in the configuration file |
|
3093 or on the command line: |
|
3094 |
|
3095 >>> write('buildout.cfg', |
|
3096 ... """ |
|
3097 ... [buildout] |
|
3098 ... develop = recipes |
|
3099 ... parts = debug |
|
3100 ... |
|
3101 ... [debug] |
|
3102 ... recipe = recipes:debug |
|
3103 ... """) |
|
3104 |
|
3105 >>> print system(buildout+' buildout:installed=inst.cfg'), |
|
3106 Develop: '/sample-buildout/recipes' |
|
3107 Installing debug. |
|
3108 recipe recipes:debug |
|
3109 |
|
3110 >>> ls(sample_buildout) |
|
3111 - b1.cfg |
|
3112 - b2.cfg |
|
3113 - base.cfg |
|
3114 d bin |
|
3115 - buildout.cfg |
|
3116 d demo |
|
3117 d develop-eggs |
|
3118 d eggs |
|
3119 - inst.cfg |
|
3120 d parts |
|
3121 d recipes |
|
3122 |
|
3123 The installation database can be disabled by supplying an empty |
|
3124 buildout installed option: |
|
3125 |
|
3126 >>> os.remove('inst.cfg') |
|
3127 >>> print system(buildout+' buildout:installed='), |
|
3128 Develop: '/sample-buildout/recipes' |
|
3129 Installing debug. |
|
3130 recipe recipes:debug |
|
3131 |
|
3132 >>> ls(sample_buildout) |
|
3133 - b1.cfg |
|
3134 - b2.cfg |
|
3135 - base.cfg |
|
3136 d bin |
|
3137 - buildout.cfg |
|
3138 d demo |
|
3139 d develop-eggs |
|
3140 d eggs |
|
3141 d parts |
|
3142 d recipes |
|
3143 |
|
3144 |
|
3145 Note that there will be no installation database if there are no parts: |
|
3146 |
|
3147 >>> write('buildout.cfg', |
|
3148 ... """ |
|
3149 ... [buildout] |
|
3150 ... parts = |
|
3151 ... """) |
|
3152 |
|
3153 >>> print system(buildout+' buildout:installed=inst.cfg'), |
|
3154 |
|
3155 >>> ls(sample_buildout) |
|
3156 - b1.cfg |
|
3157 - b2.cfg |
|
3158 - base.cfg |
|
3159 d bin |
|
3160 - buildout.cfg |
|
3161 d demo |
|
3162 d develop-eggs |
|
3163 d eggs |
|
3164 d parts |
|
3165 d recipes |
|
3166 |
|
3167 Extensions |
|
3168 ---------- |
|
3169 |
|
3170 A feature allows code to be loaded and run after |
|
3171 configuration files have been read but before the buildout has begun |
|
3172 any processing. The intent is to allow special plugins such as |
|
3173 urllib2 request handlers to be loaded. |
|
3174 |
|
3175 To load an extension, we use the extensions option and list one or |
|
3176 more distribution requirements, on separate lines. The distributions |
|
3177 named will be loaded and any ``zc.buildout.extension`` entry points found |
|
3178 will be called with the buildout as an argument. When buildout |
|
3179 finishes processing, any ``zc.buildout.unloadextension`` entry points |
|
3180 found will be called with the buildout as an argument. |
|
3181 |
|
3182 Let's create a sample extension in our sample buildout created in the |
|
3183 previous section: |
|
3184 |
|
3185 >>> mkdir(sample_bootstrapped, 'demo') |
|
3186 |
|
3187 >>> write(sample_bootstrapped, 'demo', 'demo.py', |
|
3188 ... """ |
|
3189 ... def ext(buildout): |
|
3190 ... print 'ext', list(buildout) |
|
3191 ... def unload(buildout): |
|
3192 ... print 'unload', list(buildout) |
|
3193 ... """) |
|
3194 |
|
3195 >>> write(sample_bootstrapped, 'demo', 'setup.py', |
|
3196 ... """ |
|
3197 ... from setuptools import setup |
|
3198 ... |
|
3199 ... setup( |
|
3200 ... name = "demo", |
|
3201 ... entry_points = { |
|
3202 ... 'zc.buildout.extension': ['ext = demo:ext'], |
|
3203 ... 'zc.buildout.unloadextension': ['ext = demo:unload'], |
|
3204 ... }, |
|
3205 ... ) |
|
3206 ... """) |
|
3207 |
|
3208 Our extension just prints out the word 'demo', and lists the sections |
|
3209 found in the buildout passed to it. |
|
3210 |
|
3211 We'll update our buildout.cfg to list the demo directory as a develop |
|
3212 egg to be built: |
|
3213 |
|
3214 >>> write(sample_bootstrapped, 'buildout.cfg', |
|
3215 ... """ |
|
3216 ... [buildout] |
|
3217 ... develop = demo |
|
3218 ... parts = |
|
3219 ... """) |
|
3220 |
|
3221 >>> os.chdir(sample_bootstrapped) |
|
3222 >>> print system(os.path.join(sample_bootstrapped, 'bin', 'buildout')), |
|
3223 Develop: '/sample-bootstrapped/demo' |
|
3224 |
|
3225 Now we can add the extensions option. We were a bit tricky and ran |
|
3226 the buildout once with the demo develop egg defined but without the |
|
3227 extension option. This is because extensions are loaded before the |
|
3228 buildout creates develop eggs. We needed to use a separate buildout |
|
3229 run to create the develop egg. Normally, when eggs are loaded from |
|
3230 the network, we wouldn't need to do anything special. |
|
3231 |
|
3232 >>> write(sample_bootstrapped, 'buildout.cfg', |
|
3233 ... """ |
|
3234 ... [buildout] |
|
3235 ... develop = demo |
|
3236 ... extensions = demo |
|
3237 ... parts = |
|
3238 ... """) |
|
3239 |
|
3240 We see that our extension is loaded and executed: |
|
3241 |
|
3242 >>> print system(os.path.join(sample_bootstrapped, 'bin', 'buildout')), |
|
3243 ext ['buildout'] |
|
3244 Develop: '/sample-bootstrapped/demo' |
|
3245 unload ['buildout'] |
|
3246 |
|
3247 Allow hosts |
|
3248 ----------- |
|
3249 |
|
3250 On some environments the links visited by `zc.buildout` can be forbidden |
|
3251 by paranoiac firewalls. These URL might be on the chain of links |
|
3252 visited by `zc.buildout` wheter they are defined in the `find-links` option, |
|
3253 wheter they are defined by various eggs in their `url`, `download_url`, |
|
3254 `dependency_links` metadata. |
|
3255 |
|
3256 It is even harder to track that package_index works like a spider and |
|
3257 might visit links and go to other location. |
|
3258 |
|
3259 The `allow-hosts` option provides a way to prevent this, and |
|
3260 works exactly like the one provided in `easy_install`. |
|
3261 |
|
3262 You can provide a list of allowed host, together with wildcards:: |
|
3263 |
|
3264 [buildout] |
|
3265 ... |
|
3266 |
|
3267 allow-hosts = |
|
3268 *.python.org |
|
3269 example.com |
|
3270 |
|
3271 All urls that does not match these hosts will not be visited. |
|
3272 |
|
3273 .. [#future_recipe_methods] In the future, additional methods may be |
|
3274 added. Older recipes with fewer methods will still be |
|
3275 supported. |
|
3276 |
|
3277 .. [#packaging_info] If we wanted to create a distribution from this |
|
3278 package, we would need specify much more information. See the |
|
3279 `setuptools documentation |
|
3280 <http://peak.telecommunity.com/DevCenter/setuptools>`_. |
|
3281 |
|
3282 Always unzipping eggs |
|
3283 ===================== |
|
3284 |
|
3285 By default, zc.buildout doesn't unzip zip-safe eggs. |
|
3286 |
|
3287 >>> write('buildout.cfg', |
|
3288 ... ''' |
|
3289 ... [buildout] |
|
3290 ... parts = eggs |
|
3291 ... find-links = %(link_server)s |
|
3292 ... |
|
3293 ... [eggs] |
|
3294 ... recipe = zc.recipe.egg |
|
3295 ... eggs = demo |
|
3296 ... ''' % globals()) |
|
3297 |
|
3298 >>> _ = system(buildout) |
|
3299 >>> ls('eggs') |
|
3300 - demo-0.4c1-py2.4.egg |
|
3301 - demoneeded-1.2c1-py2.4.egg |
|
3302 d setuptools-0.6c8-py2.4.egg |
|
3303 - zc.buildout.egg-link |
|
3304 |
|
3305 This follows the policy followed by setuptools itself. Experience shows |
|
3306 this policy to to be inconvenient. Zipped eggs make debugging more |
|
3307 difficult and often import more slowly. |
|
3308 |
|
3309 You can include an unzip option in the buildout section to change the |
|
3310 default unzipping policy. |
|
3311 |
|
3312 >>> write('buildout.cfg', |
|
3313 ... ''' |
|
3314 ... [buildout] |
|
3315 ... parts = eggs |
|
3316 ... find-links = %(link_server)s |
|
3317 ... unzip = true |
|
3318 ... |
|
3319 ... [eggs] |
|
3320 ... recipe = zc.recipe.egg |
|
3321 ... eggs = demo |
|
3322 ... ''' % globals()) |
|
3323 |
|
3324 |
|
3325 >>> import os |
|
3326 >>> for name in os.listdir('eggs'): |
|
3327 ... if name.startswith('demo'): |
|
3328 ... remove('eggs', name) |
|
3329 |
|
3330 >>> _ = system(buildout) |
|
3331 >>> ls('eggs') |
|
3332 d demo-0.4c1-py2.4.egg |
|
3333 d demoneeded-1.2c1-py2.4.egg |
|
3334 d setuptools-0.6c8-py2.4.egg |
|
3335 - zc.buildout.egg-link |
|
3336 |
|
3337 Repeatable buildouts: controlling eggs used |
|
3338 =========================================== |
|
3339 |
|
3340 One of the goals of zc.buildout is to provide enough control to make |
|
3341 buildouts repeatable. It should be possible to check the buildout |
|
3342 configuration files for a project into a version control system and |
|
3343 later use the checked in files to get the same buildout, subject to |
|
3344 changes in the environment outside the buildout. |
|
3345 |
|
3346 An advantage of using Python eggs is that depenencies of eggs used are |
|
3347 automatically determined and used. The automatic inclusion of |
|
3348 depenent distributions is at odds with the goal of repeatable |
|
3349 buildouts. |
|
3350 |
|
3351 To support repeatable buildouts, a versions section can be created |
|
3352 with options for each distribution name whos version is to be fixed. |
|
3353 The section can then be specified via the buildout versions option. |
|
3354 |
|
3355 To see how this works, we'll create two versions of a recipe egg: |
|
3356 |
|
3357 >>> mkdir('recipe') |
|
3358 >>> write('recipe', 'recipe.py', |
|
3359 ... ''' |
|
3360 ... class Recipe: |
|
3361 ... def __init__(*a): pass |
|
3362 ... def install(self): |
|
3363 ... print 'recipe v1' |
|
3364 ... return () |
|
3365 ... update = install |
|
3366 ... ''') |
|
3367 |
|
3368 >>> write('recipe', 'setup.py', |
|
3369 ... ''' |
|
3370 ... from setuptools import setup |
|
3371 ... setup(name='spam', version='1', py_modules=['recipe'], |
|
3372 ... entry_points={'zc.buildout': ['default = recipe:Recipe']}, |
|
3373 ... ) |
|
3374 ... ''') |
|
3375 |
|
3376 >>> write('recipe', 'README', '') |
|
3377 |
|
3378 >>> print system(buildout+' setup recipe bdist_egg'), # doctest: +ELLIPSIS |
|
3379 Running setup script 'recipe/setup.py'. |
|
3380 ... |
|
3381 |
|
3382 >>> rmdir('recipe', 'build') |
|
3383 |
|
3384 >>> write('recipe', 'recipe.py', |
|
3385 ... ''' |
|
3386 ... class Recipe: |
|
3387 ... def __init__(*a): pass |
|
3388 ... def install(self): |
|
3389 ... print 'recipe v2' |
|
3390 ... return () |
|
3391 ... update = install |
|
3392 ... ''') |
|
3393 |
|
3394 >>> write('recipe', 'setup.py', |
|
3395 ... ''' |
|
3396 ... from setuptools import setup |
|
3397 ... setup(name='spam', version='2', py_modules=['recipe'], |
|
3398 ... entry_points={'zc.buildout': ['default = recipe:Recipe']}, |
|
3399 ... ) |
|
3400 ... ''') |
|
3401 |
|
3402 |
|
3403 >>> print system(buildout+' setup recipe bdist_egg'), # doctest: +ELLIPSIS |
|
3404 Running setup script 'recipe/setup.py'. |
|
3405 ... |
|
3406 |
|
3407 and we'll configure a buildout to use it: |
|
3408 |
|
3409 >>> write('buildout.cfg', |
|
3410 ... ''' |
|
3411 ... [buildout] |
|
3412 ... parts = foo |
|
3413 ... find-links = %s |
|
3414 ... |
|
3415 ... [foo] |
|
3416 ... recipe = spam |
|
3417 ... ''' % join('recipe', 'dist')) |
|
3418 |
|
3419 If we run the buildout, it will use version 2: |
|
3420 |
|
3421 >>> print system(buildout), |
|
3422 Getting distribution for 'spam'. |
|
3423 Got spam 2. |
|
3424 Installing foo. |
|
3425 recipe v2 |
|
3426 |
|
3427 We can specify a versions section that lists our recipe and name it in |
|
3428 the buildout section: |
|
3429 |
|
3430 >>> write('buildout.cfg', |
|
3431 ... ''' |
|
3432 ... [buildout] |
|
3433 ... parts = foo |
|
3434 ... find-links = %s |
|
3435 ... versions = release-1 |
|
3436 ... |
|
3437 ... [release-1] |
|
3438 ... spam = 1 |
|
3439 ... eggs = 2.2 |
|
3440 ... |
|
3441 ... [foo] |
|
3442 ... recipe = spam |
|
3443 ... ''' % join('recipe', 'dist')) |
|
3444 |
|
3445 Here we created a release-1 section listing the version 1 for the spam |
|
3446 distribution. We told the buildout to use it by specifying release-1 |
|
3447 as in the versions option. |
|
3448 |
|
3449 Now, if we run the buildout, we'll use version 1 of the spam recipe: |
|
3450 |
|
3451 >>> print system(buildout), |
|
3452 Getting distribution for 'spam==1'. |
|
3453 Got spam 1. |
|
3454 Uninstalling foo. |
|
3455 Installing foo. |
|
3456 recipe v1 |
|
3457 |
|
3458 Running the buildout in verbose mode will help us get information |
|
3459 about versions used. If we run the buildout in verbose mode without |
|
3460 specifying a versions section: |
|
3461 |
|
3462 >>> print system(buildout+' buildout:versions= -v'), # doctest: +ELLIPSIS |
|
3463 Installing 'zc.buildout', 'setuptools'. |
|
3464 We have a develop egg: zc.buildout 1.0.0. |
|
3465 We have the best distribution that satisfies 'setuptools'. |
|
3466 Picked: setuptools = 0.6 |
|
3467 Installing 'spam'. |
|
3468 We have the best distribution that satisfies 'spam'. |
|
3469 Picked: spam = 2. |
|
3470 Uninstalling foo. |
|
3471 Installing foo. |
|
3472 recipe v2 |
|
3473 |
|
3474 We'll get output that includes lines that tell us what versions |
|
3475 buildout chose a for us, like:: |
|
3476 |
|
3477 zc.buildout.easy_install.picked: spam = 2 |
|
3478 |
|
3479 This allows us to discover versions that are picked dynamically, so |
|
3480 that we can fix them in a versions section. |
|
3481 |
|
3482 If we run the buildout with the versions section: |
|
3483 |
|
3484 >>> print system(buildout+' -v'), # doctest: +ELLIPSIS |
|
3485 Installing 'zc.buildout', 'setuptools'. |
|
3486 We have a develop egg: zc.buildout 1.0.0. |
|
3487 We have the best distribution that satisfies 'setuptools'. |
|
3488 Picked: setuptools = 0.6 |
|
3489 Installing 'spam'. |
|
3490 We have the distribution that satisfies 'spam==1'. |
|
3491 Uninstalling foo. |
|
3492 Installing foo. |
|
3493 recipe v1 |
|
3494 |
|
3495 We won't get output for the spam distribution, which we didn't pick, |
|
3496 but we will get output for setuptools, which we didn't specify |
|
3497 versions for. |
|
3498 |
|
3499 You can request buildout to generate an error if it picks any |
|
3500 versions: |
|
3501 |
|
3502 >>> write('buildout.cfg', |
|
3503 ... ''' |
|
3504 ... [buildout] |
|
3505 ... parts = foo |
|
3506 ... find-links = %s |
|
3507 ... versions = release-1 |
|
3508 ... allow-picked-versions = false |
|
3509 ... |
|
3510 ... [release-1] |
|
3511 ... spam = 1 |
|
3512 ... eggs = 2.2 |
|
3513 ... |
|
3514 ... [foo] |
|
3515 ... recipe = spam |
|
3516 ... ''' % join('recipe', 'dist')) |
|
3517 |
|
3518 Using the download utility |
|
3519 ========================== |
|
3520 |
|
3521 The ``zc.buildout.download`` module provides a download utility that handles |
|
3522 the details of downloading files needed for a buildout run from the internet. |
|
3523 It downloads files to the local file system, using the download cache if |
|
3524 desired and optionally checking the downloaded files' MD5 checksum. |
|
3525 |
|
3526 We setup an HTTP server that provides a file we want to download: |
|
3527 |
|
3528 >>> server_data = tmpdir('sample_files') |
|
3529 >>> write(server_data, 'foo.txt', 'This is a foo text.') |
|
3530 >>> server_url = start_server(server_data) |
|
3531 |
|
3532 We also use a fresh directory for temporary files in order to make sure that |
|
3533 all temporary files have been cleaned up in the end: |
|
3534 |
|
3535 >>> import tempfile |
|
3536 >>> old_tempdir = tempfile.tempdir |
|
3537 >>> tempfile.tempdir = tmpdir('tmp') |
|
3538 |
|
3539 |
|
3540 Downloading without using the cache |
|
3541 ----------------------------------- |
|
3542 |
|
3543 If no download cache should be used, the download utility is instantiated |
|
3544 without any arguments: |
|
3545 |
|
3546 >>> from zc.buildout.download import Download |
|
3547 >>> download = Download() |
|
3548 >>> print download.cache_dir |
|
3549 None |
|
3550 |
|
3551 Downloading a file is achieved by calling the utility with the URL as an |
|
3552 argument. A tuple is returned that consists of the path to the downloaded copy |
|
3553 of the file and a boolean value indicating whether this is a temporary file |
|
3554 meant to be cleaned up during the same buildout run: |
|
3555 |
|
3556 >>> path, is_temp = download(server_url+'foo.txt') |
|
3557 >>> print path |
|
3558 /.../buildout-... |
|
3559 >>> cat(path) |
|
3560 This is a foo text. |
|
3561 |
|
3562 As we aren't using the download cache and haven't specified a target path |
|
3563 either, the download has ended up in a temporary file: |
|
3564 |
|
3565 >>> is_temp |
|
3566 True |
|
3567 |
|
3568 >>> import tempfile |
|
3569 >>> path.startswith(tempfile.gettempdir()) |
|
3570 True |
|
3571 |
|
3572 We are responsible for cleaning up temporary files behind us: |
|
3573 |
|
3574 >>> remove(path) |
|
3575 |
|
3576 When trying to access a file that doesn't exist, we'll get an exception: |
|
3577 |
|
3578 >>> try: download(server_url+'not-there') # doctest: +ELLIPSIS |
|
3579 ... except: print 'download error' |
|
3580 ... else: print 'woops' |
|
3581 download error |
|
3582 |
|
3583 Downloading a local file doesn't produce a temporary file but simply returns |
|
3584 the local file itself: |
|
3585 |
|
3586 >>> download(join(server_data, 'foo.txt')) |
|
3587 ('/sample_files/foo.txt', False) |
|
3588 |
|
3589 We can also have the downloaded file's MD5 sum checked: |
|
3590 |
|
3591 >>> try: from hashlib import md5 |
|
3592 ... except ImportError: from md5 import new as md5 |
|
3593 |
|
3594 >>> path, is_temp = download(server_url+'foo.txt', |
|
3595 ... md5('This is a foo text.').hexdigest()) |
|
3596 >>> is_temp |
|
3597 True |
|
3598 >>> remove(path) |
|
3599 |
|
3600 >>> download(server_url+'foo.txt', |
|
3601 ... md5('The wrong text.').hexdigest()) |
|
3602 Traceback (most recent call last): |
|
3603 ChecksumError: MD5 checksum mismatch downloading 'http://localhost/foo.txt' |
|
3604 |
|
3605 The error message in the event of an MD5 checksum mismatch for a local file |
|
3606 reads somewhat differently: |
|
3607 |
|
3608 >>> download(join(server_data, 'foo.txt'), |
|
3609 ... md5('This is a foo text.').hexdigest()) |
|
3610 ('/sample_files/foo.txt', False) |
|
3611 |
|
3612 >>> download(join(server_data, 'foo.txt'), |
|
3613 ... md5('The wrong text.').hexdigest()) |
|
3614 Traceback (most recent call last): |
|
3615 ChecksumError: MD5 checksum mismatch for local resource at '/sample_files/foo.txt'. |
|
3616 |
|
3617 Finally, we can download the file to a specified place in the file system: |
|
3618 |
|
3619 >>> target_dir = tmpdir('download-target') |
|
3620 >>> path, is_temp = download(server_url+'foo.txt', |
|
3621 ... path=join(target_dir, 'downloaded.txt')) |
|
3622 >>> print path |
|
3623 /download-target/downloaded.txt |
|
3624 >>> cat(path) |
|
3625 This is a foo text. |
|
3626 >>> is_temp |
|
3627 False |
|
3628 |
|
3629 Trying to download a file in offline mode will result in an error: |
|
3630 |
|
3631 >>> download = Download(cache=None, offline=True) |
|
3632 >>> download(server_url+'foo.txt') |
|
3633 Traceback (most recent call last): |
|
3634 UserError: Couldn't download 'http://localhost/foo.txt' in offline mode. |
|
3635 |
|
3636 As an exception to this rule, file system paths and URLs in the ``file`` |
|
3637 scheme will still work: |
|
3638 |
|
3639 >>> cat(download(join(server_data, 'foo.txt'))[0]) |
|
3640 This is a foo text. |
|
3641 >>> cat(download('file:' + join(server_data, 'foo.txt'))[0]) |
|
3642 This is a foo text. |
|
3643 |
|
3644 >>> remove(path) |
|
3645 |
|
3646 |
|
3647 Downloading using the download cache |
|
3648 ------------------------------------ |
|
3649 |
|
3650 In order to make use of the download cache, we need to configure the download |
|
3651 utility differently. To do this, we pass a directory path as the ``cache`` |
|
3652 attribute upon instantiation: |
|
3653 |
|
3654 >>> cache = tmpdir('download-cache') |
|
3655 >>> download = Download(cache=cache) |
|
3656 >>> print download.cache_dir |
|
3657 /download-cache/ |
|
3658 |
|
3659 Simple usage |
|
3660 ~~~~~~~~~~~~ |
|
3661 |
|
3662 When using the cache, a file will be stored in the cache directory when it is |
|
3663 first downloaded. The file system path returned by the download utility points |
|
3664 to the cached copy: |
|
3665 |
|
3666 >>> ls(cache) |
|
3667 >>> path, is_temp = download(server_url+'foo.txt') |
|
3668 >>> print path |
|
3669 /download-cache/foo.txt |
|
3670 >>> cat(path) |
|
3671 This is a foo text. |
|
3672 >>> is_temp |
|
3673 False |
|
3674 |
|
3675 Whenever the file is downloaded again, the cached copy is used. Let's change |
|
3676 the file on the server to see this: |
|
3677 |
|
3678 >>> write(server_data, 'foo.txt', 'The wrong text.') |
|
3679 >>> path, is_temp = download(server_url+'foo.txt') |
|
3680 >>> print path |
|
3681 /download-cache/foo.txt |
|
3682 >>> cat(path) |
|
3683 This is a foo text. |
|
3684 |
|
3685 If we specify an MD5 checksum for a file that is already in the cache, the |
|
3686 cached copy's checksum will be verified: |
|
3687 |
|
3688 >>> download(server_url+'foo.txt', md5('The wrong text.').hexdigest()) |
|
3689 Traceback (most recent call last): |
|
3690 ChecksumError: MD5 checksum mismatch for cached download |
|
3691 from 'http://localhost/foo.txt' at '/download-cache/foo.txt' |
|
3692 |
|
3693 Trying to access another file at a different URL which has the same base name |
|
3694 will result in the cached copy being used: |
|
3695 |
|
3696 >>> mkdir(server_data, 'other') |
|
3697 >>> write(server_data, 'other', 'foo.txt', 'The wrong text.') |
|
3698 >>> path, is_temp = download(server_url+'other/foo.txt') |
|
3699 >>> print path |
|
3700 /download-cache/foo.txt |
|
3701 >>> cat(path) |
|
3702 This is a foo text. |
|
3703 |
|
3704 Given a target path for the download, the utility will provide a copy of the |
|
3705 file at that location both when first downloading the file and when using a |
|
3706 cached copy: |
|
3707 |
|
3708 >>> remove(cache, 'foo.txt') |
|
3709 >>> ls(cache) |
|
3710 >>> write(server_data, 'foo.txt', 'This is a foo text.') |
|
3711 |
|
3712 >>> path, is_temp = download(server_url+'foo.txt', |
|
3713 ... path=join(target_dir, 'downloaded.txt')) |
|
3714 >>> print path |
|
3715 /download-target/downloaded.txt |
|
3716 >>> cat(path) |
|
3717 This is a foo text. |
|
3718 >>> is_temp |
|
3719 False |
|
3720 >>> ls(cache) |
|
3721 - foo.txt |
|
3722 |
|
3723 >>> remove(path) |
|
3724 >>> write(server_data, 'foo.txt', 'The wrong text.') |
|
3725 |
|
3726 >>> path, is_temp = download(server_url+'foo.txt', |
|
3727 ... path=join(target_dir, 'downloaded.txt')) |
|
3728 >>> print path |
|
3729 /download-target/downloaded.txt |
|
3730 >>> cat(path) |
|
3731 This is a foo text. |
|
3732 >>> is_temp |
|
3733 False |
|
3734 |
|
3735 In offline mode, downloads from any URL will be successful if the file is |
|
3736 found in the cache: |
|
3737 |
|
3738 >>> download = Download(cache=cache, offline=True) |
|
3739 >>> cat(download(server_url+'foo.txt')[0]) |
|
3740 This is a foo text. |
|
3741 |
|
3742 Local resources will be cached just like any others since download caches are |
|
3743 sometimes used to create source distributions: |
|
3744 |
|
3745 >>> remove(cache, 'foo.txt') |
|
3746 >>> ls(cache) |
|
3747 |
|
3748 >>> write(server_data, 'foo.txt', 'This is a foo text.') |
|
3749 >>> download = Download(cache=cache) |
|
3750 |
|
3751 >>> cat(download('file:' + join(server_data, 'foo.txt'), path=path)[0]) |
|
3752 This is a foo text. |
|
3753 >>> ls(cache) |
|
3754 - foo.txt |
|
3755 |
|
3756 >>> remove(cache, 'foo.txt') |
|
3757 |
|
3758 >>> cat(download(join(server_data, 'foo.txt'), path=path)[0]) |
|
3759 This is a foo text. |
|
3760 >>> ls(cache) |
|
3761 - foo.txt |
|
3762 |
|
3763 >>> remove(cache, 'foo.txt') |
|
3764 |
|
3765 However, resources with checksum mismatches will not be copied to the cache: |
|
3766 |
|
3767 >>> download(server_url+'foo.txt', md5('The wrong text.').hexdigest()) |
|
3768 Traceback (most recent call last): |
|
3769 ChecksumError: MD5 checksum mismatch downloading 'http://localhost/foo.txt' |
|
3770 >>> ls(cache) |
|
3771 |
|
3772 >>> remove(path) |
|
3773 |
|
3774 Finally, let's see what happens if the download cache to be used doesn't exist |
|
3775 as a directory in the file system yet: |
|
3776 |
|
3777 >>> Download(cache=join(cache, 'non-existent'))(server_url+'foo.txt') |
|
3778 Traceback (most recent call last): |
|
3779 UserError: The directory: |
|
3780 '/download-cache/non-existent' |
|
3781 to be used as a download cache doesn't exist. |
|
3782 |
|
3783 Using namespace sub-directories of the download cache |
|
3784 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
3785 |
|
3786 It is common to store cached copies of downloaded files within sub-directories |
|
3787 of the download cache to keep some degree of order. For example, zc.buildout |
|
3788 stores downloaded distributions in a sub-directory named "dist". Those |
|
3789 sub-directories are also known as namespaces. So far, we haven't specified any |
|
3790 namespaces to use, so the download utility stored files directly inside the |
|
3791 download cache. Let's use a namespace "test" instead: |
|
3792 |
|
3793 >>> download = Download(cache=cache, namespace='test') |
|
3794 >>> print download.cache_dir |
|
3795 /download-cache/test |
|
3796 |
|
3797 The namespace sub-directory hasn't been created yet: |
|
3798 |
|
3799 >>> ls(cache) |
|
3800 |
|
3801 Downloading a file now creates the namespace sub-directory and places a copy |
|
3802 of the file inside it: |
|
3803 |
|
3804 >>> path, is_temp = download(server_url+'foo.txt') |
|
3805 >>> print path |
|
3806 /download-cache/test/foo.txt |
|
3807 >>> ls(cache) |
|
3808 d test |
|
3809 >>> ls(cache, 'test') |
|
3810 - foo.txt |
|
3811 >>> cat(path) |
|
3812 This is a foo text. |
|
3813 >>> is_temp |
|
3814 False |
|
3815 |
|
3816 The next time we want to download that file, the copy from inside the cache |
|
3817 namespace is used. To see this clearly, we put a file with the same name but |
|
3818 different content both on the server and in the cache's root directory: |
|
3819 |
|
3820 >>> write(server_data, 'foo.txt', 'The wrong text.') |
|
3821 >>> write(cache, 'foo.txt', 'The wrong text.') |
|
3822 |
|
3823 >>> path, is_temp = download(server_url+'foo.txt') |
|
3824 >>> print path |
|
3825 /download-cache/test/foo.txt |
|
3826 >>> cat(path) |
|
3827 This is a foo text. |
|
3828 |
|
3829 >>> rmdir(cache, 'test') |
|
3830 >>> remove(cache, 'foo.txt') |
|
3831 >>> write(server_data, 'foo.txt', 'This is a foo text.') |
|
3832 |
|
3833 Using a hash of the URL as the filename in the cache |
|
3834 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
3835 |
|
3836 So far, the base name of the downloaded file read from the URL has been used |
|
3837 for the name of the cached copy of the file. This may not be desirable in some |
|
3838 cases, for example when downloading files from different locations that have |
|
3839 the same base name due to some naming convention, or if the file content |
|
3840 depends on URL parameters. In such cases, an MD5 hash of the complete URL may |
|
3841 be used as the filename in the cache: |
|
3842 |
|
3843 >>> download = Download(cache=cache, hash_name=True) |
|
3844 >>> path, is_temp = download(server_url+'foo.txt') |
|
3845 >>> print path |
|
3846 /download-cache/09f5793fcdc1716727f72d49519c688d |
|
3847 >>> cat(path) |
|
3848 This is a foo text. |
|
3849 >>> ls(cache) |
|
3850 - 09f5793fcdc1716727f72d49519c688d |
|
3851 |
|
3852 The path was printed just to illustrate matters; we cannot know the real |
|
3853 checksum since we don't know which port the server happens to listen at when |
|
3854 the test is run, so we don't actually know the full URL of the file. Let's |
|
3855 check that the checksum actually belongs to the particular URL used: |
|
3856 |
|
3857 >>> path.lower() == join(cache, md5(server_url+'foo.txt').hexdigest()).lower() |
|
3858 True |
|
3859 |
|
3860 The cached copy is used when downloading the file again: |
|
3861 |
|
3862 >>> write(server_data, 'foo.txt', 'The wrong text.') |
|
3863 >>> (path, is_temp) == download(server_url+'foo.txt') |
|
3864 True |
|
3865 >>> cat(path) |
|
3866 This is a foo text. |
|
3867 >>> ls(cache) |
|
3868 - 09f5793fcdc1716727f72d49519c688d |
|
3869 |
|
3870 If we change the URL, even in such a way that it keeps the base name of the |
|
3871 file the same, the file will be downloaded again this time and put in the |
|
3872 cache under a different name: |
|
3873 |
|
3874 >>> path2, is_temp = download(server_url+'other/foo.txt') |
|
3875 >>> print path2 |
|
3876 /download-cache/537b6d73267f8f4447586989af8c470e |
|
3877 >>> path == path2 |
|
3878 False |
|
3879 >>> path2.lower() == join(cache, md5(server_url+'other/foo.txt').hexdigest()).lower() |
|
3880 True |
|
3881 >>> cat(path) |
|
3882 This is a foo text. |
|
3883 >>> cat(path2) |
|
3884 The wrong text. |
|
3885 >>> ls(cache) |
|
3886 - 09f5793fcdc1716727f72d49519c688d |
|
3887 - 537b6d73267f8f4447586989af8c470e |
|
3888 |
|
3889 >>> remove(path) |
|
3890 >>> remove(path2) |
|
3891 >>> write(server_data, 'foo.txt', 'This is a foo text.') |
|
3892 |
|
3893 |
|
3894 Using the cache purely as a fall-back |
|
3895 ------------------------------------- |
|
3896 |
|
3897 Sometimes it is desirable to try downloading a file from the net if at all |
|
3898 possible, and use the cache purely as a fall-back option when a server is |
|
3899 down or if we are in offline mode. This mode is only in effect if a download |
|
3900 cache is configured in the first place: |
|
3901 |
|
3902 >>> download = Download(cache=cache, fallback=True) |
|
3903 >>> print download.cache_dir |
|
3904 /download-cache/ |
|
3905 |
|
3906 A downloaded file will be cached: |
|
3907 |
|
3908 >>> ls(cache) |
|
3909 >>> path, is_temp = download(server_url+'foo.txt') |
|
3910 >>> ls(cache) |
|
3911 - foo.txt |
|
3912 >>> cat(cache, 'foo.txt') |
|
3913 This is a foo text. |
|
3914 >>> is_temp |
|
3915 False |
|
3916 |
|
3917 If the file cannot be served, the cached copy will be used: |
|
3918 |
|
3919 >>> remove(server_data, 'foo.txt') |
|
3920 >>> try: Download()(server_url+'foo.txt') # doctest: +ELLIPSIS |
|
3921 ... except: print 'download error' |
|
3922 ... else: print 'woops' |
|
3923 download error |
|
3924 >>> path, is_temp = download(server_url+'foo.txt') |
|
3925 >>> cat(path) |
|
3926 This is a foo text. |
|
3927 >>> is_temp |
|
3928 False |
|
3929 |
|
3930 Similarly, if the file is served but we're in offline mode, we'll fall back to |
|
3931 using the cache: |
|
3932 |
|
3933 >>> write(server_data, 'foo.txt', 'The wrong text.') |
|
3934 >>> get(server_url+'foo.txt') |
|
3935 'The wrong text.' |
|
3936 |
|
3937 >>> offline_download = Download(cache=cache, offline=True, fallback=True) |
|
3938 >>> path, is_temp = offline_download(server_url+'foo.txt') |
|
3939 >>> print path |
|
3940 /download-cache/foo.txt |
|
3941 >>> cat(path) |
|
3942 This is a foo text. |
|
3943 >>> is_temp |
|
3944 False |
|
3945 |
|
3946 However, when downloading the file normally with the cache being used in |
|
3947 fall-back mode, the file will be downloaded from the net and the cached copy |
|
3948 will be replaced with the new content: |
|
3949 |
|
3950 >>> cat(download(server_url+'foo.txt')[0]) |
|
3951 The wrong text. |
|
3952 >>> cat(cache, 'foo.txt') |
|
3953 The wrong text. |
|
3954 |
|
3955 When trying to download a resource whose checksum does not match, the cached |
|
3956 copy will neither be used nor overwritten: |
|
3957 |
|
3958 >>> write(server_data, 'foo.txt', 'This is a foo text.') |
|
3959 >>> download(server_url+'foo.txt', md5('The wrong text.').hexdigest()) |
|
3960 Traceback (most recent call last): |
|
3961 ChecksumError: MD5 checksum mismatch downloading 'http://localhost/foo.txt' |
|
3962 >>> cat(cache, 'foo.txt') |
|
3963 The wrong text. |
|
3964 |
|
3965 |
|
3966 Configuring the download utility from buildout options |
|
3967 ------------------------------------------------------ |
|
3968 |
|
3969 The configuration options explained so far derive from the build logic |
|
3970 implemented by the calling code. Other options configure the download utility |
|
3971 for use in a particular project or buildout run; they are read from the |
|
3972 ``buildout`` configuration section. The latter can be passed directly as the |
|
3973 first argument to the download utility's constructor. |
|
3974 |
|
3975 The location of the download cache is specified by the ``download-cache`` |
|
3976 option: |
|
3977 |
|
3978 >>> download = Download({'download-cache': cache}, namespace='cmmi') |
|
3979 >>> print download.cache_dir |
|
3980 /download-cache/cmmi |
|
3981 |
|
3982 If the ``download-cache`` option specifies a relative path, it is understood |
|
3983 relative to the current working directory, or to the buildout directory if |
|
3984 that is given: |
|
3985 |
|
3986 >>> download = Download({'download-cache': 'relative-cache'}) |
|
3987 >>> print download.cache_dir |
|
3988 /sample-buildout/relative-cache/ |
|
3989 |
|
3990 >>> download = Download({'directory': join(sample_buildout, 'root'), |
|
3991 ... 'download-cache': 'relative-cache'}) |
|
3992 >>> print download.cache_dir |
|
3993 /sample-buildout/root/relative-cache/ |
|
3994 |
|
3995 Keyword parameters take precedence over the corresponding options: |
|
3996 |
|
3997 >>> download = Download({'download-cache': cache}, cache=None) |
|
3998 >>> print download.cache_dir |
|
3999 None |
|
4000 |
|
4001 Whether to assume offline mode can be inferred from either the ``offline`` or |
|
4002 the ``install-from-cache`` option. As usual with zc.buildout, these options |
|
4003 must assume one of the values 'true' and 'false': |
|
4004 |
|
4005 >>> download = Download({'offline': 'true'}) |
|
4006 >>> download.offline |
|
4007 True |
|
4008 |
|
4009 >>> download = Download({'offline': 'false'}) |
|
4010 >>> download.offline |
|
4011 False |
|
4012 |
|
4013 >>> download = Download({'install-from-cache': 'true'}) |
|
4014 >>> download.offline |
|
4015 True |
|
4016 |
|
4017 >>> download = Download({'install-from-cache': 'false'}) |
|
4018 >>> download.offline |
|
4019 False |
|
4020 |
|
4021 These two options are combined using logical 'or': |
|
4022 |
|
4023 >>> download = Download({'offline': 'true', 'install-from-cache': 'false'}) |
|
4024 >>> download.offline |
|
4025 True |
|
4026 |
|
4027 >>> download = Download({'offline': 'false', 'install-from-cache': 'true'}) |
|
4028 >>> download.offline |
|
4029 True |
|
4030 |
|
4031 The ``offline`` keyword parameter takes precedence over both the ``offline`` |
|
4032 and ``install-from-cache`` options: |
|
4033 |
|
4034 >>> download = Download({'offline': 'true'}, offline=False) |
|
4035 >>> download.offline |
|
4036 False |
|
4037 |
|
4038 >>> download = Download({'install-from-cache': 'false'}, offline=True) |
|
4039 >>> download.offline |
|
4040 True |
|
4041 |
|
4042 |
|
4043 Regressions |
|
4044 ----------- |
|
4045 |
|
4046 MD5 checksum calculation needs to be reliable on all supported systems, which |
|
4047 requires text files to be treated as binary to avoid implicit line-ending |
|
4048 conversions: |
|
4049 |
|
4050 >>> text = 'First line of text.\r\nSecond line.\r\n' |
|
4051 >>> f = open(join(server_data, 'foo.txt'), 'wb') |
|
4052 >>> f.write(text) |
|
4053 >>> f.close() |
|
4054 >>> path, is_temp = Download()(server_url+'foo.txt', md5(text).hexdigest()) |
|
4055 >>> remove(path) |
|
4056 |
|
4057 |
|
4058 Clean up |
|
4059 -------- |
|
4060 |
|
4061 We should have cleaned up all temporary files created by downloading things: |
|
4062 |
|
4063 >>> ls(tempfile.tempdir) |
|
4064 |
|
4065 Reset the global temporary directory: |
|
4066 |
|
4067 >>> tempfile.tempdir = old_tempdir |
|
4068 |
|
4069 Using a download cache |
|
4070 ====================== |
|
4071 |
|
4072 Normally, when distributions are installed, if any processing is |
|
4073 needed, they are downloaded from the internet to a temporary directory |
|
4074 and then installed from there. A download cache can be used to avoid |
|
4075 the download step. This can be useful to reduce network access and to |
|
4076 create source distributions of an entire buildout. |
|
4077 |
|
4078 The buildout download-cache option can be used to specify a directory |
|
4079 to be used as a download cache. |
|
4080 |
|
4081 In this example, we'll create a directory to hold the cache: |
|
4082 |
|
4083 >>> cache = tmpdir('cache') |
|
4084 |
|
4085 And set up a buildout that downloads some eggs: |
|
4086 |
|
4087 >>> write('buildout.cfg', |
|
4088 ... ''' |
|
4089 ... [buildout] |
|
4090 ... parts = eggs |
|
4091 ... download-cache = %(cache)s |
|
4092 ... find-links = %(link_server)s |
|
4093 ... |
|
4094 ... [eggs] |
|
4095 ... recipe = zc.recipe.egg |
|
4096 ... eggs = demo ==0.2 |
|
4097 ... ''' % globals()) |
|
4098 |
|
4099 We specified a link server that has some distributions available for |
|
4100 download: |
|
4101 |
|
4102 >>> print get(link_server), |
|
4103 <html><body> |
|
4104 <a href="bigdemo-0.1-py2.4.egg">bigdemo-0.1-py2.4.egg</a><br> |
|
4105 <a href="demo-0.1-py2.4.egg">demo-0.1-py2.4.egg</a><br> |
|
4106 <a href="demo-0.2-py2.4.egg">demo-0.2-py2.4.egg</a><br> |
|
4107 <a href="demo-0.3-py2.4.egg">demo-0.3-py2.4.egg</a><br> |
|
4108 <a href="demo-0.4c1-py2.4.egg">demo-0.4c1-py2.4.egg</a><br> |
|
4109 <a href="demoneeded-1.0.zip">demoneeded-1.0.zip</a><br> |
|
4110 <a href="demoneeded-1.1.zip">demoneeded-1.1.zip</a><br> |
|
4111 <a href="demoneeded-1.2c1.zip">demoneeded-1.2c1.zip</a><br> |
|
4112 <a href="extdemo-1.4.zip">extdemo-1.4.zip</a><br> |
|
4113 <a href="index/">index/</a><br> |
|
4114 <a href="other-1.0-py2.4.egg">other-1.0-py2.4.egg</a><br> |
|
4115 </body></html> |
|
4116 |
|
4117 |
|
4118 We'll enable logging on the link server so we can see what's going on: |
|
4119 |
|
4120 >>> get(link_server+'enable_server_logging') |
|
4121 GET 200 /enable_server_logging |
|
4122 '' |
|
4123 |
|
4124 We also specified a download cache. |
|
4125 |
|
4126 If we run the buildout, we'll see the eggs installed from the link |
|
4127 server as usual: |
|
4128 |
|
4129 >>> print system(buildout), |
|
4130 GET 200 / |
|
4131 GET 200 /demo-0.2-py2.4.egg |
|
4132 GET 200 /demoneeded-1.2c1.zip |
|
4133 Installing eggs. |
|
4134 Getting distribution for 'demo==0.2'. |
|
4135 Got demo 0.2. |
|
4136 Getting distribution for 'demoneeded'. |
|
4137 Got demoneeded 1.2c1. |
|
4138 Generated script '/sample-buildout/bin/demo'. |
|
4139 |
|
4140 We'll also get the download cache populated. The buildout doesn't put |
|
4141 files in the cache directly. It creates an intermediate directory, |
|
4142 dist: |
|
4143 |
|
4144 |
|
4145 >>> ls(cache) |
|
4146 d dist |
|
4147 |
|
4148 >>> ls(cache, 'dist') |
|
4149 - demo-0.2-py2.4.egg |
|
4150 - demoneeded-1.2c1.zip |
|
4151 |
|
4152 If we remove the installed eggs from eggs directory and re-run the buildout: |
|
4153 |
|
4154 >>> import os |
|
4155 >>> for f in os.listdir('eggs'): |
|
4156 ... if f.startswith('demo'): |
|
4157 ... remove('eggs', f) |
|
4158 |
|
4159 >>> print system(buildout), |
|
4160 GET 200 / |
|
4161 Updating eggs. |
|
4162 Getting distribution for 'demo==0.2'. |
|
4163 Got demo 0.2. |
|
4164 Getting distribution for 'demoneeded'. |
|
4165 Got demoneeded 1.2c1. |
|
4166 |
|
4167 We see that the distributions aren't downloaded, because they're |
|
4168 downloaded from the cache. |
|
4169 |
|
4170 Installing solely from a download cache |
|
4171 --------------------------------------- |
|
4172 |
|
4173 A download cache can be used as the basis of application source |
|
4174 releases. In an application source release, we want to distribute an |
|
4175 application that can be built without making any network accesses. In |
|
4176 this case, we distribute a buildout with download cache and tell the |
|
4177 buildout to install from the download cache only, without making |
|
4178 network accesses. The buildout install-from-cache option can be used |
|
4179 to signal that packages should be installed only from the download |
|
4180 cache. |
|
4181 |
|
4182 Let's remove our installed eggs and run the buildout with the |
|
4183 install-from-cache option set to true: |
|
4184 |
|
4185 >>> for f in os.listdir('eggs'): |
|
4186 ... if f.startswith('demo'): |
|
4187 ... remove('eggs', f) |
|
4188 |
|
4189 >>> write('buildout.cfg', |
|
4190 ... ''' |
|
4191 ... [buildout] |
|
4192 ... parts = eggs |
|
4193 ... download-cache = %(cache)s |
|
4194 ... install-from-cache = true |
|
4195 ... find-links = %(link_server)s |
|
4196 ... |
|
4197 ... [eggs] |
|
4198 ... recipe = zc.recipe.egg |
|
4199 ... eggs = demo |
|
4200 ... ''' % globals()) |
|
4201 |
|
4202 >>> print system(buildout), |
|
4203 Uninstalling eggs. |
|
4204 Installing eggs. |
|
4205 Getting distribution for 'demo'. |
|
4206 Got demo 0.2. |
|
4207 Getting distribution for 'demoneeded'. |
|
4208 Got demoneeded 1.2c1. |
|
4209 Generated script '/sample-buildout/bin/demo'. |
|
4210 |
|
4211 Caching extended configuration |
|
4212 ============================== |
|
4213 |
|
4214 As mentioned in the general buildout documentation, configuration files can |
|
4215 extend each other, including the ability to download configuration being |
|
4216 extended from a URL. If desired, zc.buildout caches downloaded configuration |
|
4217 in order to be able to use it when run offline. |
|
4218 |
|
4219 As we're going to talk about downloading things, let's start an HTTP server. |
|
4220 Also, all of the following will take place inside the sample buildout. |
|
4221 |
|
4222 >>> server_data = tmpdir('server_data') |
|
4223 >>> server_url = start_server(server_data) |
|
4224 >>> cd(sample_buildout) |
|
4225 |
|
4226 We also use a fresh directory for temporary files in order to make sure that |
|
4227 all temporary files have been cleaned up in the end: |
|
4228 |
|
4229 >>> import tempfile |
|
4230 >>> old_tempdir = tempfile.tempdir |
|
4231 >>> tempfile.tempdir = tmpdir('tmp') |
|
4232 |
|
4233 |
|
4234 Basic use of the extends cache |
|
4235 ------------------------------ |
|
4236 |
|
4237 We put some base configuration on a server and reference it from a sample |
|
4238 buildout: |
|
4239 |
|
4240 >>> write(server_data, 'base.cfg', """\ |
|
4241 ... [buildout] |
|
4242 ... parts = |
|
4243 ... foo = bar |
|
4244 ... """) |
|
4245 |
|
4246 >>> write('buildout.cfg', """\ |
|
4247 ... [buildout] |
|
4248 ... extends = %sbase.cfg |
|
4249 ... """ % server_url) |
|
4250 |
|
4251 When trying to run this buildout offline, we'll find that we cannot read all |
|
4252 of the required configuration: |
|
4253 |
|
4254 >>> print system(buildout + ' -o') |
|
4255 While: |
|
4256 Initializing. |
|
4257 Error: Couldn't download 'http://localhost/base.cfg' in offline mode. |
|
4258 |
|
4259 Trying the same online, we can: |
|
4260 |
|
4261 >>> print system(buildout) |
|
4262 Unused options for buildout: 'foo'. |
|
4263 |
|
4264 As long as we haven't said anything about caching downloaded configuration, |
|
4265 nothing gets cached. Offline mode will still cause the buildout to fail: |
|
4266 |
|
4267 >>> print system(buildout + ' -o') |
|
4268 While: |
|
4269 Initializing. |
|
4270 Error: Couldn't download 'http://localhost/base.cfg' in offline mode. |
|
4271 |
|
4272 Let's now specify a cache for base configuration files. This cache is |
|
4273 different from the download cache used by recipes for caching distributions |
|
4274 and other files; one might, however, use a namespace subdirectory of the |
|
4275 download cache for it. The configuration cache we specify will be created when |
|
4276 running buildout and the base.cfg file will be put in it (with the file name |
|
4277 being a hash of the complete URL): |
|
4278 |
|
4279 >>> mkdir('cache') |
|
4280 >>> write('buildout.cfg', """\ |
|
4281 ... [buildout] |
|
4282 ... extends = %sbase.cfg |
|
4283 ... extends-cache = cache |
|
4284 ... """ % server_url) |
|
4285 |
|
4286 >>> print system(buildout) |
|
4287 Unused options for buildout: 'foo'. |
|
4288 |
|
4289 >>> cache = join(sample_buildout, 'cache') |
|
4290 >>> ls(cache) |
|
4291 - 5aedc98d7e769290a29d654a591a3a45 |
|
4292 |
|
4293 >>> import os |
|
4294 >>> cat(cache, os.listdir(cache)[0]) |
|
4295 [buildout] |
|
4296 parts = |
|
4297 foo = bar |
|
4298 |
|
4299 We can now run buildout offline as it will read base.cfg from the cache: |
|
4300 |
|
4301 >>> print system(buildout + ' -o') |
|
4302 Unused options for buildout: 'foo'. |
|
4303 |
|
4304 The cache is being used purely as a fall-back in case we are offline or don't |
|
4305 have access to a configuration file to be downloaded. As long as we are |
|
4306 online, buildout attempts to download a fresh copy of each file even if a |
|
4307 cached copy of the file exists. To see this, we put different configuration in |
|
4308 the same place on the server and run buildout in offline mode so it takes |
|
4309 base.cfg from the cache: |
|
4310 |
|
4311 >>> write(server_data, 'base.cfg', """\ |
|
4312 ... [buildout] |
|
4313 ... parts = |
|
4314 ... bar = baz |
|
4315 ... """) |
|
4316 |
|
4317 >>> print system(buildout + ' -o') |
|
4318 Unused options for buildout: 'foo'. |
|
4319 |
|
4320 In online mode, buildout will download and use the modified version: |
|
4321 |
|
4322 >>> print system(buildout) |
|
4323 Unused options for buildout: 'bar'. |
|
4324 |
|
4325 Trying offline mode again, the new version will be used as it has been put in |
|
4326 the cache now: |
|
4327 |
|
4328 >>> print system(buildout + ' -o') |
|
4329 Unused options for buildout: 'bar'. |
|
4330 |
|
4331 Clean up: |
|
4332 |
|
4333 >>> rmdir(cache) |
|
4334 |
|
4335 |
|
4336 Specifying extends cache and offline mode |
|
4337 ----------------------------------------- |
|
4338 |
|
4339 Normally, the values of buildout options such as the location of a download |
|
4340 cache or whether to use offline mode are determined by first reading the |
|
4341 user's default configuration, updating it with the project's configuration and |
|
4342 finally applying command-line options. User and project configuration are |
|
4343 assembled by reading a file such as ``~/.buildout/default.cfg``, |
|
4344 ``buildout.cfg`` or a URL given on the command line, recursively (depth-first) |
|
4345 downloading any base configuration specified by the ``buildout:extends`` |
|
4346 option read from each of those config files, and finally evaluating each |
|
4347 config file to provide default values for options not yet read. |
|
4348 |
|
4349 This works fine for all options that do not influence how configuration is |
|
4350 downloaded in the first place. The ``extends-cache`` and ``offline`` options, |
|
4351 however, are treated differently from the procedure described in order to make |
|
4352 it simple and obvious to see where a particular configuration file came from |
|
4353 under any particular circumstances. |
|
4354 |
|
4355 - Offline and extends-cache settings are read from the two root config files |
|
4356 exclusively. Otherwise one could construct configuration files that, when |
|
4357 read, imply that they should have been read from a different source than |
|
4358 they have. Also, specifying the extends cache within a file that might have |
|
4359 to be taken from the cache before being read wouldn't make a lot of sense. |
|
4360 |
|
4361 - Offline and extends-cache settings given by the user's defaults apply to the |
|
4362 process of assembling the project's configuration. If no extends cache has |
|
4363 been specified by the user's default configuration, the project's root |
|
4364 config file must be available, be it from disk or from the net. |
|
4365 |
|
4366 - Offline mode turned on by the ``-o`` command line option is honoured from |
|
4367 the beginning even though command line options are applied to the |
|
4368 configuration last. If offline mode is not requested by the command line, it |
|
4369 may be switched on by either the user's or the project's config root. |
|
4370 |
|
4371 Extends cache |
|
4372 ~~~~~~~~~~~~~ |
|
4373 |
|
4374 Let's see the above rules in action. We create a new home directory for our |
|
4375 user and write user and project configuration that recursively extends online |
|
4376 bases, using different caches: |
|
4377 |
|
4378 >>> mkdir('home') |
|
4379 >>> mkdir('home', '.buildout') |
|
4380 >>> mkdir('cache') |
|
4381 >>> mkdir('user-cache') |
|
4382 >>> os.environ['HOME'] = join(sample_buildout, 'home') |
|
4383 >>> write('home', '.buildout', 'default.cfg', """\ |
|
4384 ... [buildout] |
|
4385 ... extends = fancy_default.cfg |
|
4386 ... extends-cache = user-cache |
|
4387 ... """) |
|
4388 >>> write('home', '.buildout', 'fancy_default.cfg', """\ |
|
4389 ... [buildout] |
|
4390 ... extends = %sbase_default.cfg |
|
4391 ... """ % server_url) |
|
4392 >>> write(server_data, 'base_default.cfg', """\ |
|
4393 ... [buildout] |
|
4394 ... foo = bar |
|
4395 ... offline = false |
|
4396 ... """) |
|
4397 |
|
4398 >>> write('buildout.cfg', """\ |
|
4399 ... [buildout] |
|
4400 ... extends = fancy.cfg |
|
4401 ... extends-cache = cache |
|
4402 ... """) |
|
4403 >>> write('fancy.cfg', """\ |
|
4404 ... [buildout] |
|
4405 ... extends = %sbase.cfg |
|
4406 ... """ % server_url) |
|
4407 >>> write(server_data, 'base.cfg', """\ |
|
4408 ... [buildout] |
|
4409 ... parts = |
|
4410 ... offline = false |
|
4411 ... """) |
|
4412 |
|
4413 Buildout will now assemble its configuration from all of these 6 files, |
|
4414 defaults first. The online resources end up in the respective extends caches: |
|
4415 |
|
4416 >>> print system(buildout) |
|
4417 Unused options for buildout: 'foo'. |
|
4418 |
|
4419 >>> ls('user-cache') |
|
4420 - 10e772cf422123ef6c64ae770f555740 |
|
4421 >>> cat('user-cache', os.listdir('user-cache')[0]) |
|
4422 [buildout] |
|
4423 foo = bar |
|
4424 offline = false |
|
4425 |
|
4426 >>> ls('cache') |
|
4427 - c72213127e6eb2208a3e1fc1dba771a7 |
|
4428 >>> cat('cache', os.listdir('cache')[0]) |
|
4429 [buildout] |
|
4430 parts = |
|
4431 offline = false |
|
4432 |
|
4433 If, on the other hand, the extends caches are specified in files that get |
|
4434 extended themselves, they won't be used for assembling the configuration they |
|
4435 belong to (user's or project's, resp.). The extends cache specified by the |
|
4436 user's defaults does, however, apply to downloading project configuration. |
|
4437 Let's rewrite the config files, clean out the caches and re-run buildout: |
|
4438 |
|
4439 >>> write('home', '.buildout', 'default.cfg', """\ |
|
4440 ... [buildout] |
|
4441 ... extends = fancy_default.cfg |
|
4442 ... """) |
|
4443 >>> write('home', '.buildout', 'fancy_default.cfg', """\ |
|
4444 ... [buildout] |
|
4445 ... extends = %sbase_default.cfg |
|
4446 ... extends-cache = user-cache |
|
4447 ... """ % server_url) |
|
4448 |
|
4449 >>> write('buildout.cfg', """\ |
|
4450 ... [buildout] |
|
4451 ... extends = fancy.cfg |
|
4452 ... """) |
|
4453 >>> write('fancy.cfg', """\ |
|
4454 ... [buildout] |
|
4455 ... extends = %sbase.cfg |
|
4456 ... extends-cache = cache |
|
4457 ... """ % server_url) |
|
4458 |
|
4459 >>> remove('user-cache', os.listdir('user-cache')[0]) |
|
4460 >>> remove('cache', os.listdir('cache')[0]) |
|
4461 |
|
4462 >>> print system(buildout) |
|
4463 Unused options for buildout: 'foo'. |
|
4464 |
|
4465 >>> ls('user-cache') |
|
4466 - 0548bad6002359532de37385bb532e26 |
|
4467 >>> cat('user-cache', os.listdir('user-cache')[0]) |
|
4468 [buildout] |
|
4469 parts = |
|
4470 offline = false |
|
4471 |
|
4472 >>> ls('cache') |
|
4473 |
|
4474 Clean up: |
|
4475 |
|
4476 >>> rmdir('user-cache') |
|
4477 >>> rmdir('cache') |
|
4478 |
|
4479 Offline mode and installation from cache |
|
4480 ----------------------------~~~~~~~~~~~~ |
|
4481 |
|
4482 If we run buildout in offline mode now, it will fail because it cannot get at |
|
4483 the remote configuration file needed by the user's defaults: |
|
4484 |
|
4485 >>> print system(buildout + ' -o') |
|
4486 While: |
|
4487 Initializing. |
|
4488 Error: Couldn't download 'http://localhost/base_default.cfg' in offline mode. |
|
4489 |
|
4490 Let's now successively turn on offline mode by different parts of the |
|
4491 configuration and see when buildout applies this setting in each case: |
|
4492 |
|
4493 >>> write('home', '.buildout', 'default.cfg', """\ |
|
4494 ... [buildout] |
|
4495 ... extends = fancy_default.cfg |
|
4496 ... offline = true |
|
4497 ... """) |
|
4498 >>> print system(buildout) |
|
4499 While: |
|
4500 Initializing. |
|
4501 Error: Couldn't download 'http://localhost/base_default.cfg' in offline mode. |
|
4502 |
|
4503 >>> write('home', '.buildout', 'default.cfg', """\ |
|
4504 ... [buildout] |
|
4505 ... extends = fancy_default.cfg |
|
4506 ... """) |
|
4507 >>> write('home', '.buildout', 'fancy_default.cfg', """\ |
|
4508 ... [buildout] |
|
4509 ... extends = %sbase_default.cfg |
|
4510 ... offline = true |
|
4511 ... """ % server_url) |
|
4512 >>> print system(buildout) |
|
4513 While: |
|
4514 Initializing. |
|
4515 Error: Couldn't download 'http://localhost/base.cfg' in offline mode. |
|
4516 |
|
4517 >>> write('home', '.buildout', 'fancy_default.cfg', """\ |
|
4518 ... [buildout] |
|
4519 ... extends = %sbase_default.cfg |
|
4520 ... """ % server_url) |
|
4521 >>> write('buildout.cfg', """\ |
|
4522 ... [buildout] |
|
4523 ... extends = fancy.cfg |
|
4524 ... offline = true |
|
4525 ... """) |
|
4526 >>> print system(buildout) |
|
4527 While: |
|
4528 Initializing. |
|
4529 Error: Couldn't download 'http://localhost/base.cfg' in offline mode. |
|
4530 |
|
4531 >>> write('buildout.cfg', """\ |
|
4532 ... [buildout] |
|
4533 ... extends = fancy.cfg |
|
4534 ... """) |
|
4535 >>> write('fancy.cfg', """\ |
|
4536 ... [buildout] |
|
4537 ... extends = %sbase.cfg |
|
4538 ... offline = true |
|
4539 ... """ % server_url) |
|
4540 >>> print system(buildout) |
|
4541 Unused options for buildout: 'foo'. |
|
4542 |
|
4543 The ``install-from-cache`` option is treated accordingly: |
|
4544 |
|
4545 >>> write('home', '.buildout', 'default.cfg', """\ |
|
4546 ... [buildout] |
|
4547 ... extends = fancy_default.cfg |
|
4548 ... install-from-cache = true |
|
4549 ... """) |
|
4550 >>> print system(buildout) |
|
4551 While: |
|
4552 Initializing. |
|
4553 Error: Couldn't download 'http://localhost/base_default.cfg' in offline mode. |
|
4554 |
|
4555 >>> write('home', '.buildout', 'default.cfg', """\ |
|
4556 ... [buildout] |
|
4557 ... extends = fancy_default.cfg |
|
4558 ... """) |
|
4559 >>> write('home', '.buildout', 'fancy_default.cfg', """\ |
|
4560 ... [buildout] |
|
4561 ... extends = %sbase_default.cfg |
|
4562 ... install-from-cache = true |
|
4563 ... """ % server_url) |
|
4564 >>> print system(buildout) |
|
4565 While: |
|
4566 Initializing. |
|
4567 Error: Couldn't download 'http://localhost/base.cfg' in offline mode. |
|
4568 |
|
4569 >>> write('home', '.buildout', 'fancy_default.cfg', """\ |
|
4570 ... [buildout] |
|
4571 ... extends = %sbase_default.cfg |
|
4572 ... """ % server_url) |
|
4573 >>> write('buildout.cfg', """\ |
|
4574 ... [buildout] |
|
4575 ... extends = fancy.cfg |
|
4576 ... install-from-cache = true |
|
4577 ... """) |
|
4578 >>> print system(buildout) |
|
4579 While: |
|
4580 Initializing. |
|
4581 Error: Couldn't download 'http://localhost/base.cfg' in offline mode. |
|
4582 |
|
4583 >>> write('buildout.cfg', """\ |
|
4584 ... [buildout] |
|
4585 ... extends = fancy.cfg |
|
4586 ... """) |
|
4587 >>> write('fancy.cfg', """\ |
|
4588 ... [buildout] |
|
4589 ... extends = %sbase.cfg |
|
4590 ... install-from-cache = true |
|
4591 ... """ % server_url) |
|
4592 >>> print system(buildout) |
|
4593 While: |
|
4594 Installing. |
|
4595 Checking for upgrades. |
|
4596 An internal error occurred ... |
|
4597 ValueError: install_from_cache set to true with no download cache |
|
4598 |
|
4599 |
|
4600 Clean up |
|
4601 -------- |
|
4602 |
|
4603 We should have cleaned up all temporary files created by downloading things: |
|
4604 |
|
4605 >>> ls(tempfile.tempdir) |
|
4606 |
|
4607 Reset the global temporary directory: |
|
4608 |
|
4609 >>> tempfile.tempdir = old_tempdir |
|
4610 |
|
4611 Using zc.buildout to run setup scripts |
|
4612 ====================================== |
|
4613 |
|
4614 zc buildout has a convenience command for running setup scripts. Why? |
|
4615 There are two reasons. If a setup script doesn't import setuptools, |
|
4616 you can't use any setuptools-provided commands, like bdist_egg. When |
|
4617 buildout runs a setup script, it arranges to import setuptools before |
|
4618 running the script so setuptools-provided commands are available. |
|
4619 |
|
4620 If you use a squeaky-clean Python to do your development, the setup |
|
4621 script that would import setuptools because setuptools isn't in the |
|
4622 path. Because buildout requires setuptools and knows where it has |
|
4623 installed a setuptools egg, it adds the setuptools egg to the Python |
|
4624 path before running the script. To run a setup script, use the |
|
4625 buildout setup command, passing the name of a script or a directory |
|
4626 containing a setup script and arguments to the script. Let's look at |
|
4627 an example: |
|
4628 |
|
4629 >>> mkdir('test') |
|
4630 >>> cd('test') |
|
4631 >>> write('setup.py', |
|
4632 ... ''' |
|
4633 ... from distutils.core import setup |
|
4634 ... setup(name='sample') |
|
4635 ... ''') |
|
4636 |
|
4637 We've created a super simple (stupid) setup script. Note that it |
|
4638 doesn't import setuptools. Let's try running it to create an egg. |
|
4639 We'll use the buildout script from our sample buildout: |
|
4640 |
|
4641 >>> print system(buildout+' setup'), |
|
4642 ... # doctest: +NORMALIZE_WHITESPACE |
|
4643 Error: The setup command requires the path to a setup script or |
|
4644 directory containing a setup script, and its arguments. |
|
4645 |
|
4646 Oops, we forgot to give the name of the setup script: |
|
4647 |
|
4648 >>> print system(buildout+' setup setup.py bdist_egg'), |
|
4649 ... # doctest: +ELLIPSIS |
|
4650 Running setup script 'setup.py'. |
|
4651 ... |
|
4652 |
|
4653 >>> ls('dist') |
|
4654 - sample-0.0.0-py2.5.egg |
|
4655 |
|
4656 Note that we can specify a directory name. This is often shorter and |
|
4657 preferred by the lazy :) |
|
4658 |
|
4659 >>> print system(buildout+' setup . bdist_egg'), # doctest: +ELLIPSIS |
|
4660 Running setup script './setup.py'. |
|
4661 ... |
|
4662 |
|
4663 Automatic Buildout Updates |
|
4664 ========================== |
|
4665 |
|
4666 When a buildout is run, one of the first steps performed is to check |
|
4667 for updates to either zc.buildout or setuptools. To demonstrate this, |
|
4668 we've created some "new releases" of buildout and setuptools in a |
|
4669 new_releases folder: |
|
4670 |
|
4671 >>> ls(new_releases) |
|
4672 d setuptools |
|
4673 - setuptools-99.99-py2.4.egg |
|
4674 d zc.buildout |
|
4675 - zc.buildout-100.0b1-pyN.N.egg |
|
4676 - zc.buildout-99.99-py2.4.egg |
|
4677 |
|
4678 Let's update the sample buildout.cfg to look in this area: |
|
4679 |
|
4680 >>> write(sample_buildout, 'buildout.cfg', |
|
4681 ... """ |
|
4682 ... [buildout] |
|
4683 ... find-links = %(new_releases)s |
|
4684 ... index = %(new_releases)s |
|
4685 ... parts = show-versions |
|
4686 ... develop = showversions |
|
4687 ... |
|
4688 ... [show-versions] |
|
4689 ... recipe = showversions |
|
4690 ... """ % dict(new_releases=new_releases)) |
|
4691 |
|
4692 We'll also include a recipe that echos the versions of setuptools and |
|
4693 zc.buildout used: |
|
4694 |
|
4695 >>> mkdir(sample_buildout, 'showversions') |
|
4696 |
|
4697 >>> write(sample_buildout, 'showversions', 'showversions.py', |
|
4698 ... """ |
|
4699 ... import pkg_resources |
|
4700 ... |
|
4701 ... class Recipe: |
|
4702 ... |
|
4703 ... def __init__(self, buildout, name, options): |
|
4704 ... pass |
|
4705 ... |
|
4706 ... def install(self): |
|
4707 ... for project in 'zc.buildout', 'setuptools': |
|
4708 ... req = pkg_resources.Requirement.parse(project) |
|
4709 ... print project, pkg_resources.working_set.find(req).version |
|
4710 ... return () |
|
4711 ... update = install |
|
4712 ... """) |
|
4713 |
|
4714 |
|
4715 >>> write(sample_buildout, 'showversions', 'setup.py', |
|
4716 ... """ |
|
4717 ... from setuptools import setup |
|
4718 ... |
|
4719 ... setup( |
|
4720 ... name = "showversions", |
|
4721 ... entry_points = {'zc.buildout': ['default = showversions:Recipe']}, |
|
4722 ... ) |
|
4723 ... """) |
|
4724 |
|
4725 |
|
4726 Now if we run the buildout, the buildout will upgrade itself to the |
|
4727 new versions found in new releases: |
|
4728 |
|
4729 >>> print system(buildout), |
|
4730 Getting distribution for 'zc.buildout'. |
|
4731 Got zc.buildout 99.99. |
|
4732 Getting distribution for 'setuptools'. |
|
4733 Got setuptools 99.99. |
|
4734 Upgraded: |
|
4735 zc.buildout version 99.99, |
|
4736 setuptools version 99.99; |
|
4737 restarting. |
|
4738 Generated script '/sample-buildout/bin/buildout'. |
|
4739 Develop: '/sample-buildout/showversions' |
|
4740 Installing show-versions. |
|
4741 zc.buildout 99.99 |
|
4742 setuptools 99.99 |
|
4743 |
|
4744 Notice that, even though we have a newer beta version of zc.buildout |
|
4745 available, the final "99.99" was selected. If you want to get non-final |
|
4746 versions, specify a specific version in your buildout's versions |
|
4747 section, you typically want to use the --accept-buildout-test-releases |
|
4748 option to the bootstrap script, which internally uses the |
|
4749 ``accept-buildout-test-releases = true`` discussed below. |
|
4750 |
|
4751 Our buildout script's site.py has been updated to use the new eggs: |
|
4752 |
|
4753 >>> cat(sample_buildout, 'parts', 'buildout', 'site.py') |
|
4754 ... # doctest: +NORMALIZE_WHITESPACE +ELLIPSIS |
|
4755 "... |
|
4756 def addsitepackages(known_paths): |
|
4757 """Add site packages, as determined by zc.buildout. |
|
4758 <BLANKLINE> |
|
4759 See original_addsitepackages, below, for the original version.""" |
|
4760 setuptools_path = '/sample-buildout/eggs/setuptools-99.99-pyN.N.egg' |
|
4761 sys.path.append(setuptools_path) |
|
4762 known_paths.add(os.path.normcase(setuptools_path)) |
|
4763 import pkg_resources |
|
4764 buildout_paths = [ |
|
4765 '/sample-buildout/eggs/zc.buildout-99.99-pyN.N.egg', |
|
4766 '/sample-buildout/eggs/setuptools-99.99-pyN.N.egg' |
|
4767 ] |
|
4768 for path in buildout_paths: |
|
4769 sitedir, sitedircase = makepath(path) |
|
4770 if not sitedircase in known_paths and os.path.exists(sitedir): |
|
4771 sys.path.append(sitedir) |
|
4772 known_paths.add(sitedircase) |
|
4773 pkg_resources.working_set.add_entry(sitedir) |
|
4774 sys.__egginsert = len(buildout_paths) # Support setuptools. |
|
4775 original_paths = [ |
|
4776 ... |
|
4777 ] |
|
4778 for path in original_paths: |
|
4779 if path == setuptools_path or path not in known_paths: |
|
4780 addsitedir(path, known_paths) |
|
4781 return known_paths |
|
4782 ... |
|
4783 |
|
4784 Now, let's recreate the sample buildout. If we specify constraints on |
|
4785 the versions of zc.buildout and setuptools (or distribute) to use, |
|
4786 running the buildout will install earlier versions of these packages: |
|
4787 |
|
4788 >>> write(sample_buildout, 'buildout.cfg', |
|
4789 ... """ |
|
4790 ... [buildout] |
|
4791 ... find-links = %(new_releases)s |
|
4792 ... index = %(new_releases)s |
|
4793 ... parts = show-versions |
|
4794 ... develop = showversions |
|
4795 ... zc.buildout-version = < 99 |
|
4796 ... setuptools-version = < 99 |
|
4797 ... distribute-version = < 99 |
|
4798 ... |
|
4799 ... [show-versions] |
|
4800 ... recipe = showversions |
|
4801 ... """ % dict(new_releases=new_releases)) |
|
4802 |
|
4803 Now we can see that we actually "upgrade" to an earlier version. |
|
4804 |
|
4805 >>> print system(buildout), |
|
4806 Upgraded: |
|
4807 zc.buildout version 1.0.0, |
|
4808 setuptools version 0.6; |
|
4809 restarting. |
|
4810 Develop: '/sample-buildout/showversions' |
|
4811 Updating show-versions. |
|
4812 zc.buildout 1.0.0 |
|
4813 setuptools 0.6 |
|
4814 |
|
4815 There are a number of cases, described below, in which the updates |
|
4816 don't happen. |
|
4817 |
|
4818 We won't upgrade in offline mode: |
|
4819 |
|
4820 >>> write(sample_buildout, 'buildout.cfg', |
|
4821 ... """ |
|
4822 ... [buildout] |
|
4823 ... find-links = %(new_releases)s |
|
4824 ... index = %(new_releases)s |
|
4825 ... parts = show-versions |
|
4826 ... develop = showversions |
|
4827 ... |
|
4828 ... [show-versions] |
|
4829 ... recipe = showversions |
|
4830 ... """ % dict(new_releases=new_releases)) |
|
4831 |
|
4832 >>> print system(buildout+' -o'), |
|
4833 Develop: '/sample-buildout/showversions' |
|
4834 Updating show-versions. |
|
4835 zc.buildout 1.0.0 |
|
4836 setuptools 0.6 |
|
4837 |
|
4838 Or in non-newest mode: |
|
4839 |
|
4840 >>> print system(buildout+' -N'), |
|
4841 Develop: '/sample-buildout/showversions' |
|
4842 Updating show-versions. |
|
4843 zc.buildout 1.0.0 |
|
4844 setuptools 0.6 |
|
4845 |
|
4846 We also won't upgrade if the buildout script being run isn't in the |
|
4847 buildout's bin directory. To see this we'll create a new buildout |
|
4848 directory: |
|
4849 |
|
4850 >>> sample_buildout2 = tmpdir('sample_buildout2') |
|
4851 >>> write(sample_buildout2, 'buildout.cfg', |
|
4852 ... """ |
|
4853 ... [buildout] |
|
4854 ... find-links = %(new_releases)s |
|
4855 ... index = %(new_releases)s |
|
4856 ... parts = |
|
4857 ... """ % dict(new_releases=new_releases)) |
|
4858 |
|
4859 >>> cd(sample_buildout2) |
|
4860 >>> print system(buildout), |
|
4861 Creating directory '/sample_buildout2/bin'. |
|
4862 Creating directory '/sample_buildout2/parts'. |
|
4863 Creating directory '/sample_buildout2/eggs'. |
|
4864 Creating directory '/sample_buildout2/develop-eggs'. |
|
4865 Getting distribution for 'zc.buildout'. |
|
4866 Got zc.buildout 99.99. |
|
4867 Getting distribution for 'setuptools'. |
|
4868 Got setuptools 99.99. |
|
4869 Not upgrading because not running a local buildout command. |
|
4870 |
|
4871 >>> ls('bin') |
|
4872 |
|
4873 As mentioned above, the ``accept-buildout-test-releases = true`` means that |
|
4874 newer non-final versions of these dependencies are preferred. Typically |
|
4875 users are not expected to actually manipulate this value. Instead, the |
|
4876 bootstrap script creates a buildout buildout script that passes in the |
|
4877 value as a command line override. This then results in the buildout |
|
4878 script being rewritten to remember the decision. |
|
4879 |
|
4880 We'll mimic this by passing the argument actually in the command line. |
|
4881 |
|
4882 >>> cd(sample_buildout) |
|
4883 >>> write(sample_buildout, 'buildout.cfg', |
|
4884 ... """ |
|
4885 ... [buildout] |
|
4886 ... find-links = %(new_releases)s |
|
4887 ... index = %(new_releases)s |
|
4888 ... parts = show-versions |
|
4889 ... develop = showversions |
|
4890 ... |
|
4891 ... [show-versions] |
|
4892 ... recipe = showversions |
|
4893 ... """ % dict(new_releases=new_releases)) |
|
4894 |
|
4895 >>> print system(buildout + |
|
4896 ... ' buildout:accept-buildout-test-releases=true'), |
|
4897 ... # doctest: +NORMALIZE_WHITESPACE |
|
4898 Getting distribution for 'zc.buildout'. |
|
4899 Got zc.buildout 100.0b1. |
|
4900 Upgraded: |
|
4901 zc.buildout version 100.0b1, |
|
4902 setuptools version 99.99; |
|
4903 restarting. |
|
4904 Generated script '/sample-buildout/bin/buildout'. |
|
4905 NOTE: Accepting early releases of build system packages. Rerun bootstrap |
|
4906 without --accept-buildout-test-releases (-t) to return to default |
|
4907 behavior. |
|
4908 Develop: '/sample-buildout/showversions' |
|
4909 Updating show-versions. |
|
4910 zc.buildout 100.0b1 |
|
4911 setuptools 99.99 |
|
4912 |
|
4913 The buildout script shows the change. |
|
4914 |
|
4915 >>> buildout_script = join(sample_buildout, 'bin', 'buildout') |
|
4916 >>> import sys |
|
4917 >>> if sys.platform.startswith('win'): |
|
4918 ... buildout_script += '-script.py' |
|
4919 >>> print open(buildout_script).read() # doctest: +ELLIPSIS |
|
4920 #... |
|
4921 sys.argv.insert(1, 'buildout:accept-buildout-test-releases=true') |
|
4922 print ('NOTE: Accepting early releases of build system packages. Rerun ' |
|
4923 'bootstrap without --accept-buildout-test-releases (-t) to return to ' |
|
4924 'default behavior.') |
|
4925 ... |
|
4926 |
|
4927 Debugging buildouts |
|
4928 =================== |
|
4929 |
|
4930 Buildouts can be pretty complex. When things go wrong, it isn't |
|
4931 always obvious why. Errors can occur due to problems in user input or |
|
4932 due to bugs in zc.buildout or recipes. When an error occurs, Python's |
|
4933 post-mortem debugger can be used to inspect the state of the buildout |
|
4934 or recipe code where the error occurred. To enable this, use the -D |
|
4935 option to the buildout. Let's create a recipe that has a bug: |
|
4936 |
|
4937 >>> mkdir(sample_buildout, 'recipes') |
|
4938 |
|
4939 >>> write(sample_buildout, 'recipes', 'mkdir.py', |
|
4940 ... """ |
|
4941 ... import os, zc.buildout |
|
4942 ... |
|
4943 ... class Mkdir: |
|
4944 ... |
|
4945 ... def __init__(self, buildout, name, options): |
|
4946 ... self.name, self.options = name, options |
|
4947 ... options['path'] = os.path.join( |
|
4948 ... buildout['buildout']['directory'], |
|
4949 ... options['path'], |
|
4950 ... ) |
|
4951 ... |
|
4952 ... def install(self): |
|
4953 ... directory = self.options['directory'] |
|
4954 ... os.mkdir(directory) |
|
4955 ... return directory |
|
4956 ... |
|
4957 ... def update(self): |
|
4958 ... pass |
|
4959 ... """) |
|
4960 |
|
4961 >>> write(sample_buildout, 'recipes', 'setup.py', |
|
4962 ... """ |
|
4963 ... from setuptools import setup |
|
4964 ... |
|
4965 ... setup(name = "recipes", |
|
4966 ... entry_points = {'zc.buildout': ['mkdir = mkdir:Mkdir']}, |
|
4967 ... ) |
|
4968 ... """) |
|
4969 |
|
4970 And create a buildout that uses it: |
|
4971 |
|
4972 >>> write(sample_buildout, 'buildout.cfg', |
|
4973 ... """ |
|
4974 ... [buildout] |
|
4975 ... develop = recipes |
|
4976 ... parts = data-dir |
|
4977 ... |
|
4978 ... [data-dir] |
|
4979 ... recipe = recipes:mkdir |
|
4980 ... path = mystuff |
|
4981 ... """) |
|
4982 |
|
4983 If we run the buildout, we'll get an error: |
|
4984 |
|
4985 >>> print system(buildout), |
|
4986 Develop: '/sample-buildout/recipes' |
|
4987 Installing data-dir. |
|
4988 While: |
|
4989 Installing data-dir. |
|
4990 Error: Missing option: data-dir:directory |
|
4991 |
|
4992 |
|
4993 If we want to debug the error, we can add the -D option. Here's we'll |
|
4994 supply some input: |
|
4995 |
|
4996 >>> print system(buildout+" -D", """\ |
|
4997 ... up |
|
4998 ... p self.options.keys() |
|
4999 ... q |
|
5000 ... """), |
|
5001 Develop: '/sample-buildout/recipes' |
|
5002 Installing data-dir. |
|
5003 > /zc/buildout/buildout.py(925)__getitem__() |
|
5004 -> raise MissingOption("Missing option: %s:%s" % (self.name, key)) |
|
5005 (Pdb) > /sample-buildout/recipes/mkdir.py(14)install() |
|
5006 -> directory = self.options['directory'] |
|
5007 (Pdb) ['path', 'recipe'] |
|
5008 (Pdb) While: |
|
5009 Installing data-dir. |
|
5010 Traceback (most recent call last): |
|
5011 File "/zc/buildout/buildout.py", line 1352, in main |
|
5012 getattr(buildout, command)(args) |
|
5013 File "/zc/buildout/buildout.py", line 383, in install |
|
5014 installed_files = self[part]._call(recipe.install) |
|
5015 File "/zc/buildout/buildout.py", line 961, in _call |
|
5016 return f() |
|
5017 File "/sample-buildout/recipes/mkdir.py", line 14, in install |
|
5018 directory = self.options['directory'] |
|
5019 File "/zc/buildout/buildout.py", line 925, in __getitem__ |
|
5020 raise MissingOption("Missing option: %s:%s" % (self.name, key)) |
|
5021 MissingOption: Missing option: data-dir:directory |
|
5022 <BLANKLINE> |
|
5023 Starting pdb: |
|
5024 |
|
5025 Testing Support |
|
5026 =============== |
|
5027 |
|
5028 The zc.buildout.testing module provides an API that can be used when |
|
5029 writing recipe tests. This API is documented below. Many examples of |
|
5030 using this API can be found in the zc.buildout, zc.recipe.egg, and |
|
5031 zc.recipe.testrunner tests. |
|
5032 |
|
5033 zc.buildout.testing.buildoutSetUp(test) |
|
5034 --------------------------------------- |
|
5035 |
|
5036 The buildoutSetup function can be used as a doctest setup function. |
|
5037 It creates a sample buildout that can be used by tests, changing the |
|
5038 current working directory to the sample_buildout. It also adds a |
|
5039 number of names to the test namespace: |
|
5040 |
|
5041 ``sample_buildout`` |
|
5042 This is the name of a buildout with a basic configuration. |
|
5043 |
|
5044 ``buildout`` |
|
5045 This is the path of the buildout script in the sample buildout. |
|
5046 |
|
5047 ``ls(*path)`` |
|
5048 List the contents of a directory. The directory path is provided as one or |
|
5049 more strings, to be joined with os.path.join. |
|
5050 |
|
5051 ``cat(*path)`` |
|
5052 Display the contents of a file. The file path is provided as one or |
|
5053 more strings, to be joined with os.path.join. |
|
5054 |
|
5055 On Windows, if the file doesn't exist, the function will try |
|
5056 adding a '-script.py' suffix. This helps to work around a |
|
5057 difference in script generation on windows. |
|
5058 |
|
5059 ``mkdir(*path)`` |
|
5060 Create a directory. The directory path is provided as one or |
|
5061 more strings, to be joined with os.path.join. |
|
5062 |
|
5063 ``rmdir(*path)`` |
|
5064 Remove a directory. The directory path is provided as one or |
|
5065 more strings, to be joined with os.path.join. |
|
5066 |
|
5067 ``remove(*path)`` |
|
5068 Remove a directory or file. The path is provided as one or |
|
5069 more strings, to be joined with os.path.join. |
|
5070 |
|
5071 ``tmpdir(name)`` |
|
5072 Create a temporary directory with the given name. The directory |
|
5073 will be automatically removed at the end of the test. The path of |
|
5074 the created directory is returned. |
|
5075 |
|
5076 Further, if the the normalize_path normlaizing substitution (see |
|
5077 below) is used, then any paths starting with this path will be |
|
5078 normalized to:: |
|
5079 |
|
5080 /name/restofpath |
|
5081 |
|
5082 No two temporary directories can be created with the same name. A |
|
5083 directory created with tmpdir can be removed with rmdir and recreated. |
|
5084 |
|
5085 Note that the sample_buildout directory is created by calling this |
|
5086 function. |
|
5087 |
|
5088 ``write(*path_and_contents)`` |
|
5089 Create a file. The file path is provided as one or more strings, |
|
5090 to be joined with os.path.join. The last argument is the file contents. |
|
5091 |
|
5092 ``system(command, input='')`` |
|
5093 Execute a system command with the given input passed to the |
|
5094 command's standard input. The output (error and regular output) |
|
5095 from the command is returned. |
|
5096 |
|
5097 ``get(url)`` |
|
5098 Get a web page. |
|
5099 |
|
5100 ``cd(*path)`` |
|
5101 Change to the given directory. The directory path is provided as one or |
|
5102 more strings, to be joined with os.path.join. |
|
5103 |
|
5104 The directory will be reset at the end of the test. |
|
5105 |
|
5106 ``join(*path)`` |
|
5107 A convenient reference to os.path.join. |
|
5108 |
|
5109 ``register_teardown(func)`` |
|
5110 Register a tear-down function. The function will be called with |
|
5111 no arguments at the end of the test. |
|
5112 |
|
5113 ``start_server(path)`` |
|
5114 Start a web server on the given path. The server will be shut |
|
5115 down at the end of the test. The server URL is returned. |
|
5116 |
|
5117 You can cause the server to start and stop logging it's output |
|
5118 using: |
|
5119 |
|
5120 >>> get(server_url+'enable_server_logging') |
|
5121 |
|
5122 and: |
|
5123 |
|
5124 >>> get(server_url+'disable_server_logging') |
|
5125 |
|
5126 This can be useful to see how buildout is interacting with a |
|
5127 server. |
|
5128 |
|
5129 |
|
5130 ``sdist(setup, dest)`` |
|
5131 Create a source distribution by running the given setup file and |
|
5132 placing the result in the given destination directory. If the |
|
5133 setup argument is a directory, the thge setup.py file in that |
|
5134 directory is used. |
|
5135 |
|
5136 ``bdist_egg(setup, executable, dest)`` |
|
5137 Create an egg by running the given setup file with the given |
|
5138 Python executable and placing the result in the given destination |
|
5139 directory. If the setup argument is a directory, then the |
|
5140 setup.py file in that directory is used. |
|
5141 |
|
5142 ``find_python(version)`` |
|
5143 Find a Python executable for the given version, where version is a |
|
5144 string like "2.4". |
|
5145 |
|
5146 This function uses the following strategy to find a Python of the |
|
5147 given version: |
|
5148 |
|
5149 - Look for an environment variable of the form PYTHON%(version)s. |
|
5150 |
|
5151 - On windows, look for \Pythonm%(version)s\python |
|
5152 |
|
5153 - on Unix, try running python%(version)s or just python to get the |
|
5154 executable |
|
5155 |
|
5156 ``zc.buildout.testing.buildoutTearDown(test)`` |
|
5157 ---------------------------------------------- |
|
5158 |
|
5159 Tear down everything set up by zc.buildout.testing.buildoutSetUp. Any |
|
5160 functions passed to register_teardown are called as well. |
|
5161 |
|
5162 ``install(project, destination)`` |
|
5163 --------------------------------- |
|
5164 |
|
5165 Install eggs for a given project into a destination. If the |
|
5166 destination is a test object, then the eggs directory of the |
|
5167 sample buildout (sample_buildout) defined by the test will be used. |
|
5168 Tests will use this to install the distributions for the packages |
|
5169 being tested (and their dependencies) into a sample buildout. The egg |
|
5170 to be used should already be loaded, by importing one of the modules |
|
5171 provided, before calling this function. |
|
5172 |
|
5173 ``install_develop(project, destination)`` |
|
5174 ----------------------------------------- |
|
5175 |
|
5176 Like install, but a develop egg is installed even if the current egg |
|
5177 if not a develop egg. |
|
5178 |
|
5179 ``Output normalization`` |
|
5180 ------------------------ |
|
5181 |
|
5182 Recipe tests often generate output that is dependent on temporary file |
|
5183 locations, operating system conventions or Python versions. To deal |
|
5184 with these dependencies, we often use |
|
5185 zope.testing.renormalizing.RENormalizing to normalize test output. |
|
5186 zope.testing.renormalizing.RENormalizing takes pairs of regular |
|
5187 expressions and substitutions. The zc.buildout.testing module provides |
|
5188 a few helpful variables that define regular-expression/substitution |
|
5189 pairs that you can pass to zope.testing.renormalizing.RENormalizing. |
|
5190 |
|
5191 |
|
5192 ``normalize_path`` |
|
5193 Converts tests paths, based on directories created with tmpdir(), |
|
5194 to simple paths. |
|
5195 |
|
5196 ``normalize_script`` |
|
5197 On Unix-like systems, scripts are implemented in single files |
|
5198 without suffixes. On windows, scripts are implemented with 2 |
|
5199 files, a -script.py file and a .exe file. This normalization |
|
5200 converts directory listings of Windows scripts to the form |
|
5201 generated on UNix-like systems. |
|
5202 |
|
5203 ``normalize_egg_py`` |
|
5204 Normalize Python version and platform indicators, if specified, in |
|
5205 egg names. |
|
5206 |
|
5207 Python API for egg and script installation |
|
5208 ========================================== |
|
5209 |
|
5210 The easy_install module provides some functions to provide support for |
|
5211 egg and script installation. It provides functionality at the python |
|
5212 level that is similar to easy_install, with a few exceptions: |
|
5213 |
|
5214 - By default, we look for new packages *and* the packages that |
|
5215 they depend on. This is somewhat like (and uses) the --upgrade |
|
5216 option of easy_install, except that we also upgrade required |
|
5217 packages. |
|
5218 |
|
5219 - If the highest-revision package satisfying a specification is |
|
5220 already present, then we don't try to get another one. This saves a |
|
5221 lot of search time in the common case that packages are pegged to |
|
5222 specific versions. |
|
5223 |
|
5224 - If there is a develop egg that satisfies a requirement, we don't |
|
5225 look for additional distributions. We always give preference to |
|
5226 develop eggs. |
|
5227 |
|
5228 - Distutils options for building extensions can be passed. |
|
5229 |
|
5230 Distribution installation |
|
5231 ------------------------- |
|
5232 |
|
5233 The easy_install module provides a function, install, for installing one |
|
5234 or more packages and their dependencies. The install function takes 2 |
|
5235 positional arguments: |
|
5236 |
|
5237 - An iterable of setuptools requirement strings for the distributions |
|
5238 to be installed, and |
|
5239 |
|
5240 - A destination directory to install to and to satisfy requirements |
|
5241 from. The destination directory can be None, in which case, no new |
|
5242 distributions are downloaded and there will be an error if the |
|
5243 needed distributions can't be found among those already installed. |
|
5244 |
|
5245 It supports a number of optional keyword arguments: |
|
5246 |
|
5247 links |
|
5248 A sequence of URLs, file names, or directories to look for |
|
5249 links to distributions. |
|
5250 |
|
5251 index |
|
5252 The URL of an index server, or almost any other valid URL. :) |
|
5253 |
|
5254 If not specified, the Python Package Index, |
|
5255 http://pypi.python.org/simple/, is used. You can specify an |
|
5256 alternate index with this option. If you use the links option and |
|
5257 if the links point to the needed distributions, then the index can |
|
5258 be anything and will be largely ignored. In the examples, here, |
|
5259 we'll just point to an empty directory on our link server. This |
|
5260 will make our examples run a little bit faster. |
|
5261 |
|
5262 executable |
|
5263 A path to a Python executable. Distributions will be installed |
|
5264 using this executable and will be for the matching Python version. |
|
5265 |
|
5266 path |
|
5267 A list of additional directories to search for locally-installed |
|
5268 distributions. |
|
5269 |
|
5270 always_unzip |
|
5271 A flag indicating that newly-downloaded distributions should be |
|
5272 directories even if they could be installed as zip files. |
|
5273 |
|
5274 working_set |
|
5275 An existing working set to be augmented with additional |
|
5276 distributions, if necessary to satisfy requirements. This allows |
|
5277 you to call install multiple times, if necessary, to gather |
|
5278 multiple sets of requirements. |
|
5279 |
|
5280 newest |
|
5281 A boolean value indicating whether to search for new distributions |
|
5282 when already-installed distributions meet the requirement. When |
|
5283 this is true, the default, and when the destination directory is |
|
5284 not None, then the install function will search for the newest |
|
5285 distributions that satisfy the requirements. |
|
5286 |
|
5287 versions |
|
5288 A dictionary mapping project names to version numbers to be used |
|
5289 when selecting distributions. This can be used to specify a set of |
|
5290 distribution versions independent of other requirements. |
|
5291 |
|
5292 use_dependency_links |
|
5293 A flag indicating whether to search for dependencies using the |
|
5294 setup dependency_links metadata or not. If true, links are searched |
|
5295 for using dependency_links in preference to other |
|
5296 locations. Defaults to true. |
|
5297 |
|
5298 include_site_packages |
|
5299 A flag indicating whether Python's non-standard-library packages should |
|
5300 be available for finding dependencies. Defaults to true. |
|
5301 |
|
5302 Paths outside of Python's standard library--or more precisely, those that |
|
5303 are not included when Python is started with the -S argument--are loosely |
|
5304 referred to as "site-packages" here. |
|
5305 |
|
5306 relative_paths |
|
5307 Adjust egg paths so they are relative to the script path. This |
|
5308 allows scripts to work when scripts and eggs are moved, as long as |
|
5309 they are both moved in the same way. |
|
5310 |
|
5311 The install method returns a working set containing the distributions |
|
5312 needed to meet the given requirements. |
|
5313 |
|
5314 We have a link server that has a number of eggs: |
|
5315 |
|
5316 >>> print get(link_server), |
|
5317 <html><body> |
|
5318 <a href="bigdemo-0.1-py2.4.egg">bigdemo-0.1-py2.4.egg</a><br> |
|
5319 <a href="demo-0.1-py2.4.egg">demo-0.1-py2.4.egg</a><br> |
|
5320 <a href="demo-0.2-py2.4.egg">demo-0.2-py2.4.egg</a><br> |
|
5321 <a href="demo-0.3-py2.4.egg">demo-0.3-py2.4.egg</a><br> |
|
5322 <a href="demo-0.4c1-py2.4.egg">demo-0.4c1-py2.4.egg</a><br> |
|
5323 <a href="demoneeded-1.0.zip">demoneeded-1.0.zip</a><br> |
|
5324 <a href="demoneeded-1.1.zip">demoneeded-1.1.zip</a><br> |
|
5325 <a href="demoneeded-1.2c1.zip">demoneeded-1.2c1.zip</a><br> |
|
5326 <a href="extdemo-1.4.zip">extdemo-1.4.zip</a><br> |
|
5327 <a href="index/">index/</a><br> |
|
5328 <a href="other-1.0-py2.4.egg">other-1.0-py2.4.egg</a><br> |
|
5329 </body></html> |
|
5330 |
|
5331 Let's make a directory and install the demo egg to it, using the demo: |
|
5332 |
|
5333 >>> dest = tmpdir('sample-install') |
|
5334 >>> import zc.buildout.easy_install |
|
5335 >>> ws = zc.buildout.easy_install.install( |
|
5336 ... ['demo==0.2'], dest, |
|
5337 ... links=[link_server], index=link_server+'index/') |
|
5338 |
|
5339 We requested version 0.2 of the demo distribution to be installed into |
|
5340 the destination server. We specified that we should search for links |
|
5341 on the link server and that we should use the (empty) link server |
|
5342 index directory as a package index. |
|
5343 |
|
5344 The working set contains the distributions we retrieved. |
|
5345 |
|
5346 >>> for dist in ws: |
|
5347 ... print dist |
|
5348 demo 0.2 |
|
5349 demoneeded 1.1 |
|
5350 |
|
5351 We got demoneeded because it was a dependency of demo. |
|
5352 |
|
5353 And the actual eggs were added to the eggs directory. |
|
5354 |
|
5355 >>> ls(dest) |
|
5356 - demo-0.2-py2.4.egg |
|
5357 - demoneeded-1.1-py2.4.egg |
|
5358 |
|
5359 If we remove the version restriction on demo, but specify a false |
|
5360 value for newest, no new distributions will be installed: |
|
5361 |
|
5362 >>> ws = zc.buildout.easy_install.install( |
|
5363 ... ['demo'], dest, links=[link_server], index=link_server+'index/', |
|
5364 ... newest=False) |
|
5365 >>> ls(dest) |
|
5366 - demo-0.2-py2.4.egg |
|
5367 - demoneeded-1.1-py2.4.egg |
|
5368 |
|
5369 If we leave off the newest option, we'll get an update for demo: |
|
5370 |
|
5371 >>> ws = zc.buildout.easy_install.install( |
|
5372 ... ['demo'], dest, links=[link_server], index=link_server+'index/') |
|
5373 >>> ls(dest) |
|
5374 - demo-0.2-py2.4.egg |
|
5375 - demo-0.3-py2.4.egg |
|
5376 - demoneeded-1.1-py2.4.egg |
|
5377 |
|
5378 Note that we didn't get the newest versions available. There were |
|
5379 release candidates for newer versions of both packages. By default, |
|
5380 final releases are preferred. We can change this behavior using the |
|
5381 prefer_final function: |
|
5382 |
|
5383 >>> zc.buildout.easy_install.prefer_final(False) |
|
5384 True |
|
5385 |
|
5386 The old setting is returned. |
|
5387 |
|
5388 >>> ws = zc.buildout.easy_install.install( |
|
5389 ... ['demo'], dest, links=[link_server], index=link_server+'index/') |
|
5390 >>> for dist in ws: |
|
5391 ... print dist |
|
5392 demo 0.4c1 |
|
5393 demoneeded 1.2c1 |
|
5394 |
|
5395 >>> ls(dest) |
|
5396 - demo-0.2-py2.4.egg |
|
5397 - demo-0.3-py2.4.egg |
|
5398 - demo-0.4c1-py2.4.egg |
|
5399 - demoneeded-1.1-py2.4.egg |
|
5400 - demoneeded-1.2c1-py2.4.egg |
|
5401 |
|
5402 Let's put the setting back to the default. |
|
5403 |
|
5404 >>> zc.buildout.easy_install.prefer_final(True) |
|
5405 False |
|
5406 |
|
5407 We can supply additional distributions. We can also supply |
|
5408 specifications for distributions that would normally be found via |
|
5409 dependencies. We might do this to specify a specific version. |
|
5410 |
|
5411 >>> ws = zc.buildout.easy_install.install( |
|
5412 ... ['demo', 'other', 'demoneeded==1.0'], dest, |
|
5413 ... links=[link_server], index=link_server+'index/') |
|
5414 |
|
5415 >>> for dist in ws: |
|
5416 ... print dist |
|
5417 demo 0.3 |
|
5418 other 1.0 |
|
5419 demoneeded 1.0 |
|
5420 |
|
5421 >>> ls(dest) |
|
5422 - demo-0.2-py2.4.egg |
|
5423 - demo-0.3-py2.4.egg |
|
5424 - demo-0.4c1-py2.4.egg |
|
5425 - demoneeded-1.0-py2.4.egg |
|
5426 - demoneeded-1.1-py2.4.egg |
|
5427 - demoneeded-1.2c1-py2.4.egg |
|
5428 d other-1.0-py2.4.egg |
|
5429 |
|
5430 We can request that eggs be unzipped even if they are zip safe. This |
|
5431 can be useful when debugging. (Note that Distribute will unzip eggs by |
|
5432 default, so if you are using Distribute, most or all eggs will already be |
|
5433 unzipped without this flag.) |
|
5434 |
|
5435 >>> rmdir(dest) |
|
5436 >>> dest = tmpdir('sample-install') |
|
5437 >>> ws = zc.buildout.easy_install.install( |
|
5438 ... ['demo'], dest, links=[link_server], index=link_server+'index/', |
|
5439 ... always_unzip=True) |
|
5440 |
|
5441 >>> ls(dest) |
|
5442 d demo-0.3-py2.4.egg |
|
5443 d demoneeded-1.1-py2.4.egg |
|
5444 |
|
5445 >>> rmdir(dest) |
|
5446 >>> dest = tmpdir('sample-install') |
|
5447 >>> ws = zc.buildout.easy_install.install( |
|
5448 ... ['demo'], dest, links=[link_server], index=link_server+'index/', |
|
5449 ... always_unzip=False) |
|
5450 |
|
5451 >>> ls(dest) |
|
5452 - demo-0.3-py2.4.egg |
|
5453 - demoneeded-1.1-py2.4.egg |
|
5454 |
|
5455 We can also set a default by calling the always_unzip function: |
|
5456 |
|
5457 >>> zc.buildout.easy_install.always_unzip(True) |
|
5458 False |
|
5459 |
|
5460 The old default is returned: |
|
5461 |
|
5462 >>> rmdir(dest) |
|
5463 >>> dest = tmpdir('sample-install') |
|
5464 >>> ws = zc.buildout.easy_install.install( |
|
5465 ... ['demo'], dest, links=[link_server], index=link_server+'index/') |
|
5466 |
|
5467 >>> ls(dest) |
|
5468 d demo-0.3-py2.4.egg |
|
5469 d demoneeded-1.1-py2.4.egg |
|
5470 |
|
5471 |
|
5472 >>> zc.buildout.easy_install.always_unzip(False) |
|
5473 True |
|
5474 |
|
5475 >>> rmdir(dest) |
|
5476 >>> dest = tmpdir('sample-install') |
|
5477 >>> ws = zc.buildout.easy_install.install( |
|
5478 ... ['demo'], dest, links=[link_server], index=link_server+'index/') |
|
5479 |
|
5480 >>> ls(dest) |
|
5481 - demo-0.3-py2.4.egg |
|
5482 - demoneeded-1.1-py2.4.egg |
|
5483 |
|
5484 >>> rmdir(dest) |
|
5485 >>> dest = tmpdir('sample-install') |
|
5486 >>> ws = zc.buildout.easy_install.install( |
|
5487 ... ['demo'], dest, links=[link_server], index=link_server+'index/', |
|
5488 ... always_unzip=True) |
|
5489 |
|
5490 >>> ls(dest) |
|
5491 d demo-0.3-py2.4.egg |
|
5492 d demoneeded-1.1-py2.4.egg |
|
5493 |
|
5494 Specifying version information independent of requirements |
|
5495 ---------------------------------------------------------- |
|
5496 |
|
5497 Sometimes it's useful to specify version information independent of |
|
5498 normal requirements specifications. For example, a buildout may need |
|
5499 to lock down a set of versions, without having to put put version |
|
5500 numbers in setup files or part definitions. If a dictionary is passed |
|
5501 to the install function, mapping project names to version numbers, |
|
5502 then the versions numbers will be used. |
|
5503 |
|
5504 >>> ws = zc.buildout.easy_install.install( |
|
5505 ... ['demo'], dest, links=[link_server], index=link_server+'index/', |
|
5506 ... versions = dict(demo='0.2', demoneeded='1.0')) |
|
5507 >>> [d.version for d in ws] |
|
5508 ['0.2', '1.0'] |
|
5509 |
|
5510 In this example, we specified a version for demoneeded, even though we |
|
5511 didn't define a requirement for it. The versions specified apply to |
|
5512 dependencies as well as the specified requirements. |
|
5513 |
|
5514 If we specify a version that's incompatible with a requirement, then |
|
5515 we'll get an error: |
|
5516 |
|
5517 >>> from zope.testing.loggingsupport import InstalledHandler |
|
5518 >>> handler = InstalledHandler('zc.buildout.easy_install') |
|
5519 >>> import logging |
|
5520 >>> logging.getLogger('zc.buildout.easy_install').propagate = False |
|
5521 |
|
5522 >>> ws = zc.buildout.easy_install.install( |
|
5523 ... ['demo >0.2'], dest, links=[link_server], |
|
5524 ... index=link_server+'index/', |
|
5525 ... versions = dict(demo='0.2', demoneeded='1.0')) |
|
5526 Traceback (most recent call last): |
|
5527 ... |
|
5528 IncompatibleVersionError: Bad version 0.2 |
|
5529 |
|
5530 >>> print handler |
|
5531 zc.buildout.easy_install DEBUG |
|
5532 Installing 'demo >0.2'. |
|
5533 zc.buildout.easy_install ERROR |
|
5534 The version, 0.2, is not consistent with the requirement, 'demo>0.2'. |
|
5535 |
|
5536 >>> handler.clear() |
|
5537 |
|
5538 If no versions are specified, a debugging message will be output |
|
5539 reporting that a version was picked automatically: |
|
5540 |
|
5541 >>> ws = zc.buildout.easy_install.install( |
|
5542 ... ['demo'], dest, links=[link_server], index=link_server+'index/', |
|
5543 ... ) |
|
5544 |
|
5545 >>> print handler |
|
5546 zc.buildout.easy_install DEBUG |
|
5547 Installing 'demo'. |
|
5548 zc.buildout.easy_install DEBUG |
|
5549 We have the best distribution that satisfies 'demo'. |
|
5550 zc.buildout.easy_install DEBUG |
|
5551 Picked: demo = 0.3 |
|
5552 zc.buildout.easy_install DEBUG |
|
5553 Getting required 'demoneeded' |
|
5554 zc.buildout.easy_install DEBUG |
|
5555 required by demo 0.3. |
|
5556 zc.buildout.easy_install DEBUG |
|
5557 We have the best distribution that satisfies 'demoneeded'. |
|
5558 zc.buildout.easy_install DEBUG |
|
5559 Picked: demoneeded = 1.1 |
|
5560 |
|
5561 >>> handler.uninstall() |
|
5562 >>> logging.getLogger('zc.buildout.easy_install').propagate = True |
|
5563 |
|
5564 We can request that we get an error if versions are picked: |
|
5565 |
|
5566 >>> zc.buildout.easy_install.allow_picked_versions(False) |
|
5567 True |
|
5568 |
|
5569 (The old setting is returned.) |
|
5570 |
|
5571 >>> ws = zc.buildout.easy_install.install( |
|
5572 ... ['demo'], dest, links=[link_server], index=link_server+'index/', |
|
5573 ... ) |
|
5574 Traceback (most recent call last): |
|
5575 ... |
|
5576 UserError: Picked: demo = 0.3 |
|
5577 |
|
5578 >>> zc.buildout.easy_install.allow_picked_versions(True) |
|
5579 False |
|
5580 |
|
5581 The function default_versions can be used to get and set default |
|
5582 version information to be used when no version information is passes. |
|
5583 If called with an argument, it sets the default versions: |
|
5584 |
|
5585 >>> zc.buildout.easy_install.default_versions(dict(demoneeded='1')) |
|
5586 {} |
|
5587 |
|
5588 It always returns the previous default versions. If called without an |
|
5589 argument, it simply returns the default versions without changing |
|
5590 them: |
|
5591 |
|
5592 >>> zc.buildout.easy_install.default_versions() |
|
5593 {'demoneeded': '1'} |
|
5594 |
|
5595 So with the default versions set, we'll get the requested version even |
|
5596 if the versions option isn't used: |
|
5597 |
|
5598 >>> ws = zc.buildout.easy_install.install( |
|
5599 ... ['demo'], dest, links=[link_server], index=link_server+'index/', |
|
5600 ... ) |
|
5601 |
|
5602 >>> [d.version for d in ws] |
|
5603 ['0.3', '1.0'] |
|
5604 |
|
5605 Of course, we can unset the default versions by passing an empty |
|
5606 dictionary: |
|
5607 |
|
5608 >>> zc.buildout.easy_install.default_versions({}) |
|
5609 {'demoneeded': '1'} |
|
5610 |
|
5611 >>> ws = zc.buildout.easy_install.install( |
|
5612 ... ['demo'], dest, links=[link_server], index=link_server+'index/', |
|
5613 ... ) |
|
5614 |
|
5615 >>> [d.version for d in ws] |
|
5616 ['0.3', '1.1'] |
|
5617 |
|
5618 Dependencies in Site Packages |
|
5619 ----------------------------- |
|
5620 |
|
5621 Paths outside of Python's standard library--or more precisely, those that are |
|
5622 not included when Python is started with the -S argument--are loosely referred |
|
5623 to as "site-packages" here. These site-packages are searched by default for |
|
5624 distributions. This can be disabled, so that, for instance, a system Python |
|
5625 can be used with buildout, cleaned of any packages installed by a user or |
|
5626 system package manager. |
|
5627 |
|
5628 The default behavior can be controlled and introspected using |
|
5629 zc.buildout.easy_install.include_site_packages. |
|
5630 |
|
5631 >>> zc.buildout.easy_install.include_site_packages() |
|
5632 True |
|
5633 |
|
5634 Here's an example of using a Python executable that includes our dependencies. |
|
5635 |
|
5636 Our "py_path" will have the "demoneeded," and "demo" packages available. |
|
5637 We'll simply be asking for "demoneeded" here, but without any external |
|
5638 index or links. |
|
5639 |
|
5640 >>> from zc.buildout.tests import create_sample_sys_install |
|
5641 >>> py_path, site_packages_path = make_py() |
|
5642 >>> create_sample_sys_install(site_packages_path) |
|
5643 |
|
5644 >>> example_dest = tmpdir('site-packages-example-install') |
|
5645 >>> workingset = zc.buildout.easy_install.install( |
|
5646 ... ['demoneeded'], example_dest, links=[], executable=py_path, |
|
5647 ... index=None) |
|
5648 >>> [dist.project_name for dist in workingset] |
|
5649 ['demoneeded'] |
|
5650 |
|
5651 That worked fine. Let's try again with site packages not allowed. We'll |
|
5652 change the policy by changing the default. Notice that the function for |
|
5653 changing the default value returns the previous value. |
|
5654 |
|
5655 >>> zc.buildout.easy_install.include_site_packages(False) |
|
5656 True |
|
5657 |
|
5658 >>> zc.buildout.easy_install.include_site_packages() |
|
5659 False |
|
5660 |
|
5661 >>> zc.buildout.easy_install.clear_index_cache() |
|
5662 >>> rmdir(example_dest) |
|
5663 >>> example_dest = tmpdir('site-packages-example-install') |
|
5664 >>> workingset = zc.buildout.easy_install.install( |
|
5665 ... ['demoneeded'], example_dest, links=[], executable=py_path, |
|
5666 ... index=None) |
|
5667 Traceback (most recent call last): |
|
5668 ... |
|
5669 MissingDistribution: Couldn't find a distribution for 'demoneeded'. |
|
5670 >>> zc.buildout.easy_install.clear_index_cache() |
|
5671 |
|
5672 Now we'll reset the default. |
|
5673 |
|
5674 >>> zc.buildout.easy_install.include_site_packages(True) |
|
5675 False |
|
5676 |
|
5677 >>> zc.buildout.easy_install.include_site_packages() |
|
5678 True |
|
5679 |
|
5680 Dependency links |
|
5681 ---------------- |
|
5682 |
|
5683 Setuptools allows metadata that describes where to search for package |
|
5684 dependencies. This option is called dependency_links. Buildout has its |
|
5685 own notion of where to look for dependencies, but it also uses the |
|
5686 setup tools dependency_links information if it's available. |
|
5687 |
|
5688 Let's demo this by creating an egg that specifies dependency_links. |
|
5689 |
|
5690 To begin, let's create a new egg repository. This repository hold a |
|
5691 newer version of the 'demoneeded' egg than the sample repository does. |
|
5692 |
|
5693 >>> repoloc = tmpdir('repo') |
|
5694 >>> from zc.buildout.tests import create_egg |
|
5695 >>> create_egg('demoneeded', '1.2', repoloc) |
|
5696 >>> link_server2 = start_server(repoloc) |
|
5697 |
|
5698 Turn on logging on this server so that we can see when eggs are pulled |
|
5699 from it. |
|
5700 |
|
5701 >>> get(link_server2 + 'enable_server_logging') |
|
5702 GET 200 /enable_server_logging |
|
5703 '' |
|
5704 |
|
5705 Now we can create an egg that specifies that its dependencies are |
|
5706 found on this server. |
|
5707 |
|
5708 >>> repoloc = tmpdir('repo2') |
|
5709 >>> create_egg('hasdeps', '1.0', repoloc, |
|
5710 ... install_requires = "'demoneeded'", |
|
5711 ... dependency_links = [link_server2]) |
|
5712 |
|
5713 Let's add the egg to another repository. |
|
5714 |
|
5715 >>> link_server3 = start_server(repoloc) |
|
5716 |
|
5717 Now let's install the egg. |
|
5718 |
|
5719 >>> example_dest = tmpdir('example-install') |
|
5720 >>> workingset = zc.buildout.easy_install.install( |
|
5721 ... ['hasdeps'], example_dest, |
|
5722 ... links=[link_server3], index=link_server3+'index/') |
|
5723 GET 200 / |
|
5724 GET 200 /demoneeded-1.2-pyN.N.egg |
|
5725 |
|
5726 The server logs show that the dependency was retrieved from the server |
|
5727 specified in the dependency_links. |
|
5728 |
|
5729 Now let's see what happens if we provide two different ways to retrieve |
|
5730 the dependencies. |
|
5731 |
|
5732 >>> rmdir(example_dest) |
|
5733 >>> example_dest = tmpdir('example-install') |
|
5734 >>> workingset = zc.buildout.easy_install.install( |
|
5735 ... ['hasdeps'], example_dest, index=link_server+'index/', |
|
5736 ... links=[link_server, link_server3]) |
|
5737 GET 200 / |
|
5738 GET 200 /demoneeded-1.2-pyN.N.egg |
|
5739 |
|
5740 Once again the dependency is fetched from the logging server even |
|
5741 though it is also available from the non-logging server. This is |
|
5742 because the version on the logging server is newer and buildout |
|
5743 normally chooses the newest egg available. |
|
5744 |
|
5745 If you wish to control where dependencies come from regardless of |
|
5746 dependency_links setup metadata use the 'use_dependency_links' option |
|
5747 to zc.buildout.easy_install.install(). |
|
5748 |
|
5749 >>> rmdir(example_dest) |
|
5750 >>> example_dest = tmpdir('example-install') |
|
5751 >>> workingset = zc.buildout.easy_install.install( |
|
5752 ... ['hasdeps'], example_dest, index=link_server+'index/', |
|
5753 ... links=[link_server, link_server3], |
|
5754 ... use_dependency_links=False) |
|
5755 |
|
5756 Notice that this time the dependency egg is not fetched from the |
|
5757 logging server. When you specify not to use dependency_links, eggs |
|
5758 will only be searched for using the links you explicitly provide. |
|
5759 |
|
5760 Another way to control this option is with the |
|
5761 zc.buildout.easy_install.use_dependency_links() function. This |
|
5762 function sets the default behavior for the zc.buildout.easy_install() |
|
5763 function. |
|
5764 |
|
5765 >>> zc.buildout.easy_install.use_dependency_links(False) |
|
5766 True |
|
5767 |
|
5768 The function returns its previous setting. |
|
5769 |
|
5770 >>> rmdir(example_dest) |
|
5771 >>> example_dest = tmpdir('example-install') |
|
5772 >>> workingset = zc.buildout.easy_install.install( |
|
5773 ... ['hasdeps'], example_dest, index=link_server+'index/', |
|
5774 ... links=[link_server, link_server3]) |
|
5775 |
|
5776 It can be overridden by passing a keyword argument to the install |
|
5777 function. |
|
5778 |
|
5779 >>> rmdir(example_dest) |
|
5780 >>> example_dest = tmpdir('example-install') |
|
5781 >>> workingset = zc.buildout.easy_install.install( |
|
5782 ... ['hasdeps'], example_dest, index=link_server+'index/', |
|
5783 ... links=[link_server, link_server3], |
|
5784 ... use_dependency_links=True) |
|
5785 GET 200 /demoneeded-1.2-pyN.N.egg |
|
5786 |
|
5787 To return the dependency_links behavior to normal call the function again. |
|
5788 |
|
5789 >>> zc.buildout.easy_install.use_dependency_links(True) |
|
5790 False |
|
5791 >>> rmdir(example_dest) |
|
5792 >>> example_dest = tmpdir('example-install') |
|
5793 >>> workingset = zc.buildout.easy_install.install( |
|
5794 ... ['hasdeps'], example_dest, index=link_server+'index/', |
|
5795 ... links=[link_server, link_server3]) |
|
5796 GET 200 /demoneeded-1.2-pyN.N.egg |
|
5797 |
|
5798 |
|
5799 Script generation |
|
5800 ----------------- |
|
5801 |
|
5802 The easy_install module provides support for creating scripts from eggs. |
|
5803 It provides two competing functions. One, ``scripts``, is a |
|
5804 well-established approach to generating reliable scripts with a "clean" |
|
5805 Python--e.g., one that does not have any packages in its site-packages. |
|
5806 The other, ``sitepackage_safe_scripts``, is newer, a bit trickier, and is |
|
5807 designed to work with a Python that has code in its site-packages, such |
|
5808 as a system Python. |
|
5809 |
|
5810 Both are similar to setuptools except that they provides facilities for |
|
5811 baking a script's path into the script. This has two advantages: |
|
5812 |
|
5813 - The eggs to be used by a script are not chosen at run time, making |
|
5814 startup faster and, more importantly, deterministic. |
|
5815 |
|
5816 - The script doesn't have to import pkg_resources because the logic that |
|
5817 pkg_resources would execute at run time is executed at script-creation |
|
5818 time. (There is an exception in ``sitepackage_safe_scripts`` if you |
|
5819 want to have your Python's site packages available, as discussed |
|
5820 below, but even in that case pkg_resources is only partially |
|
5821 activated, which can be a significant time savings.) |
|
5822 |
|
5823 |
|
5824 The ``scripts`` function |
|
5825 ~~~~~~~~~~~~~~~~~~~~~~~~ |
|
5826 |
|
5827 The ``scripts`` function is the first way to generate scripts that we'll |
|
5828 examine. It is the earlier approach that the package offered. Let's |
|
5829 create a destination directory for it to place them in: |
|
5830 |
|
5831 >>> bin = tmpdir('bin') |
|
5832 |
|
5833 Now, we'll use the scripts function to generate scripts in this directory |
|
5834 from the demo egg: |
|
5835 |
|
5836 >>> import sys |
|
5837 >>> scripts = zc.buildout.easy_install.scripts( |
|
5838 ... ['demo'], ws, sys.executable, bin) |
|
5839 |
|
5840 the four arguments we passed were: |
|
5841 |
|
5842 1. A sequence of distribution requirements. These are of the same |
|
5843 form as setuptools requirements. Here we passed a single |
|
5844 requirement, for the version 0.1 demo distribution. |
|
5845 |
|
5846 2. A working set, |
|
5847 |
|
5848 3. The Python executable to use, and |
|
5849 |
|
5850 3. The destination directory. |
|
5851 |
|
5852 The bin directory now contains a generated script: |
|
5853 |
|
5854 >>> ls(bin) |
|
5855 - demo |
|
5856 |
|
5857 The return value is a list of the scripts generated: |
|
5858 |
|
5859 >>> import os, sys |
|
5860 >>> if sys.platform == 'win32': |
|
5861 ... scripts == [os.path.join(bin, 'demo.exe'), |
|
5862 ... os.path.join(bin, 'demo-script.py')] |
|
5863 ... else: |
|
5864 ... scripts == [os.path.join(bin, 'demo')] |
|
5865 True |
|
5866 |
|
5867 Note that in Windows, 2 files are generated for each script. A script |
|
5868 file, ending in '-script.py', and an exe file that allows the script |
|
5869 to be invoked directly without having to specify the Python |
|
5870 interpreter and without having to provide a '.py' suffix. |
|
5871 |
|
5872 The demo script run the entry point defined in the demo egg: |
|
5873 |
|
5874 >>> cat(bin, 'demo') # doctest: +NORMALIZE_WHITESPACE |
|
5875 #!/usr/local/bin/python2.4 |
|
5876 <BLANKLINE> |
|
5877 import sys |
|
5878 sys.path[0:0] = [ |
|
5879 '/sample-install/demo-0.3-py2.4.egg', |
|
5880 '/sample-install/demoneeded-1.1-py2.4.egg', |
|
5881 ] |
|
5882 <BLANKLINE> |
|
5883 import eggrecipedemo |
|
5884 <BLANKLINE> |
|
5885 if __name__ == '__main__': |
|
5886 eggrecipedemo.main() |
|
5887 |
|
5888 Some things to note: |
|
5889 |
|
5890 - The demo and demoneeded eggs are added to the beginning of sys.path. |
|
5891 |
|
5892 - The module for the script entry point is imported and the entry |
|
5893 point, in this case, 'main', is run. |
|
5894 |
|
5895 Rather than requirement strings, you can pass tuples containing 3 |
|
5896 strings: |
|
5897 |
|
5898 - A script name, |
|
5899 |
|
5900 - A module, |
|
5901 |
|
5902 - An attribute expression for an entry point within the module. |
|
5903 |
|
5904 For example, we could have passed entry point information directly |
|
5905 rather than passing a requirement: |
|
5906 |
|
5907 >>> scripts = zc.buildout.easy_install.scripts( |
|
5908 ... [('demo', 'eggrecipedemo', 'main')], |
|
5909 ... ws, sys.executable, bin) |
|
5910 |
|
5911 >>> cat(bin, 'demo') # doctest: +NORMALIZE_WHITESPACE |
|
5912 #!/usr/local/bin/python2.4 |
|
5913 <BLANKLINE> |
|
5914 import sys |
|
5915 sys.path[0:0] = [ |
|
5916 '/sample-install/demo-0.3-py2.4.egg', |
|
5917 '/sample-install/demoneeded-1.1-py2.4.egg', |
|
5918 ] |
|
5919 <BLANKLINE> |
|
5920 import eggrecipedemo |
|
5921 <BLANKLINE> |
|
5922 if __name__ == '__main__': |
|
5923 eggrecipedemo.main() |
|
5924 |
|
5925 Passing entry-point information directly is handy when using eggs (or |
|
5926 distributions) that don't declare their entry points, such as |
|
5927 distributions that aren't based on setuptools. |
|
5928 |
|
5929 The interpreter keyword argument can be used to generate a script that can |
|
5930 be used to invoke the Python interactive interpreter with the path set |
|
5931 based on the working set. This generated script can also be used to |
|
5932 run other scripts with the path set on the working set: |
|
5933 |
|
5934 >>> scripts = zc.buildout.easy_install.scripts( |
|
5935 ... ['demo'], ws, sys.executable, bin, interpreter='py') |
|
5936 |
|
5937 |
|
5938 >>> ls(bin) |
|
5939 - demo |
|
5940 - py |
|
5941 |
|
5942 >>> if sys.platform == 'win32': |
|
5943 ... scripts == [os.path.join(bin, 'demo.exe'), |
|
5944 ... os.path.join(bin, 'demo-script.py'), |
|
5945 ... os.path.join(bin, 'py.exe'), |
|
5946 ... os.path.join(bin, 'py-script.py')] |
|
5947 ... else: |
|
5948 ... scripts == [os.path.join(bin, 'demo'), |
|
5949 ... os.path.join(bin, 'py')] |
|
5950 True |
|
5951 |
|
5952 The py script simply runs the Python interactive interpreter with |
|
5953 the path set: |
|
5954 |
|
5955 >>> cat(bin, 'py') # doctest: +NORMALIZE_WHITESPACE |
|
5956 #!/usr/local/bin/python2.4 |
|
5957 <BLANKLINE> |
|
5958 import sys |
|
5959 <BLANKLINE> |
|
5960 sys.path[0:0] = [ |
|
5961 '/sample-install/demo-0.3-pyN.N.egg', |
|
5962 '/sample-install/demoneeded-1.1-pyN.N.egg', |
|
5963 ] |
|
5964 <BLANKLINE> |
|
5965 _interactive = True |
|
5966 if len(sys.argv) > 1: |
|
5967 _options, _args = __import__("getopt").getopt(sys.argv[1:], 'ic:m:') |
|
5968 _interactive = False |
|
5969 for (_opt, _val) in _options: |
|
5970 if _opt == '-i': |
|
5971 _interactive = True |
|
5972 elif _opt == '-c': |
|
5973 exec _val |
|
5974 elif _opt == '-m': |
|
5975 sys.argv[1:] = _args |
|
5976 _args = [] |
|
5977 __import__("runpy").run_module( |
|
5978 _val, {}, "__main__", alter_sys=True) |
|
5979 <BLANKLINE> |
|
5980 if _args: |
|
5981 sys.argv[:] = _args |
|
5982 __file__ = _args[0] |
|
5983 del _options, _args |
|
5984 execfile(__file__) |
|
5985 <BLANKLINE> |
|
5986 if _interactive: |
|
5987 del _interactive |
|
5988 __import__("code").interact(banner="", local=globals()) |
|
5989 |
|
5990 If invoked with a script name and arguments, it will run that script, instead. |
|
5991 |
|
5992 >>> write('ascript', ''' |
|
5993 ... "demo doc" |
|
5994 ... print sys.argv |
|
5995 ... print (__name__, __file__, __doc__) |
|
5996 ... ''') |
|
5997 >>> print system(join(bin, 'py')+' ascript a b c'), |
|
5998 ['ascript', 'a', 'b', 'c'] |
|
5999 ('__main__', 'ascript', 'demo doc') |
|
6000 |
|
6001 For Python 2.5 and higher, you can also use the -m option to run a |
|
6002 module: |
|
6003 |
|
6004 >>> print system(join(bin, 'py')+' -m pdb'), |
|
6005 usage: pdb.py scriptfile [arg] ... |
|
6006 |
|
6007 >>> print system(join(bin, 'py')+' -m pdb what'), |
|
6008 Error: what does not exist |
|
6009 |
|
6010 An additional argument can be passed to define which scripts to install |
|
6011 and to provide script names. The argument is a dictionary mapping |
|
6012 original script names to new script names. |
|
6013 |
|
6014 >>> bin = tmpdir('bin2') |
|
6015 >>> scripts = zc.buildout.easy_install.scripts( |
|
6016 ... ['demo'], ws, sys.executable, bin, dict(demo='run')) |
|
6017 |
|
6018 >>> if sys.platform == 'win32': |
|
6019 ... scripts == [os.path.join(bin, 'run.exe'), |
|
6020 ... os.path.join(bin, 'run-script.py')] |
|
6021 ... else: |
|
6022 ... scripts == [os.path.join(bin, 'run')] |
|
6023 True |
|
6024 >>> ls(bin) |
|
6025 - run |
|
6026 |
|
6027 >>> print system(os.path.join(bin, 'run')), |
|
6028 3 1 |
|
6029 |
|
6030 The ``scripts`` function: Including extra paths in scripts |
|
6031 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
6032 |
|
6033 We can pass a keyword argument, extra paths, to cause additional paths |
|
6034 to be included in the a generated script: |
|
6035 |
|
6036 >>> foo = tmpdir('foo') |
|
6037 >>> scripts = zc.buildout.easy_install.scripts( |
|
6038 ... ['demo'], ws, sys.executable, bin, dict(demo='run'), |
|
6039 ... extra_paths=[foo]) |
|
6040 |
|
6041 >>> cat(bin, 'run') # doctest: +NORMALIZE_WHITESPACE |
|
6042 #!/usr/local/bin/python2.4 |
|
6043 <BLANKLINE> |
|
6044 import sys |
|
6045 sys.path[0:0] = [ |
|
6046 '/sample-install/demo-0.3-py2.4.egg', |
|
6047 '/sample-install/demoneeded-1.1-py2.4.egg', |
|
6048 '/foo', |
|
6049 ] |
|
6050 <BLANKLINE> |
|
6051 import eggrecipedemo |
|
6052 <BLANKLINE> |
|
6053 if __name__ == '__main__': |
|
6054 eggrecipedemo.main() |
|
6055 |
|
6056 The ``scripts`` function: Providing script arguments |
|
6057 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
6058 |
|
6059 An "argument" keyword argument can be used to pass arguments to an |
|
6060 entry point. The value passed is a source string to be placed between the |
|
6061 parentheses in the call: |
|
6062 |
|
6063 >>> scripts = zc.buildout.easy_install.scripts( |
|
6064 ... ['demo'], ws, sys.executable, bin, dict(demo='run'), |
|
6065 ... arguments='1, 2') |
|
6066 |
|
6067 >>> cat(bin, 'run') # doctest: +NORMALIZE_WHITESPACE |
|
6068 #!/usr/local/bin/python2.4 |
|
6069 import sys |
|
6070 sys.path[0:0] = [ |
|
6071 '/sample-install/demo-0.3-py2.4.egg', |
|
6072 '/sample-install/demoneeded-1.1-py2.4.egg', |
|
6073 ] |
|
6074 <BLANKLINE> |
|
6075 import eggrecipedemo |
|
6076 <BLANKLINE> |
|
6077 if __name__ == '__main__': |
|
6078 eggrecipedemo.main(1, 2) |
|
6079 |
|
6080 The ``scripts`` function: Passing initialization code |
|
6081 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
6082 |
|
6083 You can also pass script initialization code: |
|
6084 |
|
6085 >>> scripts = zc.buildout.easy_install.scripts( |
|
6086 ... ['demo'], ws, sys.executable, bin, dict(demo='run'), |
|
6087 ... arguments='1, 2', |
|
6088 ... initialization='import os\nos.chdir("foo")') |
|
6089 |
|
6090 >>> cat(bin, 'run') # doctest: +NORMALIZE_WHITESPACE |
|
6091 #!/usr/local/bin/python2.4 |
|
6092 import sys |
|
6093 sys.path[0:0] = [ |
|
6094 '/sample-install/demo-0.3-py2.4.egg', |
|
6095 '/sample-install/demoneeded-1.1-py2.4.egg', |
|
6096 ] |
|
6097 <BLANKLINE> |
|
6098 import os |
|
6099 os.chdir("foo") |
|
6100 <BLANKLINE> |
|
6101 import eggrecipedemo |
|
6102 <BLANKLINE> |
|
6103 if __name__ == '__main__': |
|
6104 eggrecipedemo.main(1, 2) |
|
6105 |
|
6106 The ``scripts`` function: Relative paths |
|
6107 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
6108 |
|
6109 Sometimes, you want to be able to move a buildout directory around and |
|
6110 have scripts still work without having to rebuild them. We can |
|
6111 control this using the relative_paths option to install. You need |
|
6112 to pass a common base directory of the scripts and eggs: |
|
6113 |
|
6114 >>> bo = tmpdir('bo') |
|
6115 >>> ba = tmpdir('ba') |
|
6116 >>> mkdir(bo, 'eggs') |
|
6117 >>> mkdir(bo, 'bin') |
|
6118 >>> mkdir(bo, 'other') |
|
6119 |
|
6120 >>> ws = zc.buildout.easy_install.install( |
|
6121 ... ['demo'], join(bo, 'eggs'), links=[link_server], |
|
6122 ... index=link_server+'index/') |
|
6123 |
|
6124 >>> scripts = zc.buildout.easy_install.scripts( |
|
6125 ... ['demo'], ws, sys.executable, join(bo, 'bin'), dict(demo='run'), |
|
6126 ... extra_paths=[ba, join(bo, 'bar')], |
|
6127 ... interpreter='py', |
|
6128 ... relative_paths=bo) |
|
6129 |
|
6130 >>> cat(bo, 'bin', 'run') # doctest: +NORMALIZE_WHITESPACE |
|
6131 #!/usr/local/bin/python2.4 |
|
6132 <BLANKLINE> |
|
6133 import os |
|
6134 <BLANKLINE> |
|
6135 join = os.path.join |
|
6136 base = os.path.dirname(os.path.abspath(os.path.realpath(__file__))) |
|
6137 base = os.path.dirname(base) |
|
6138 <BLANKLINE> |
|
6139 import sys |
|
6140 sys.path[0:0] = [ |
|
6141 join(base, 'eggs/demo-0.3-pyN.N.egg'), |
|
6142 join(base, 'eggs/demoneeded-1.1-pyN.N.egg'), |
|
6143 '/ba', |
|
6144 join(base, 'bar'), |
|
6145 ] |
|
6146 <BLANKLINE> |
|
6147 import eggrecipedemo |
|
6148 <BLANKLINE> |
|
6149 if __name__ == '__main__': |
|
6150 eggrecipedemo.main() |
|
6151 |
|
6152 Note that the extra path we specified that was outside the directory |
|
6153 passed as relative_paths wasn't converted to a relative path. |
|
6154 |
|
6155 Of course, running the script works: |
|
6156 |
|
6157 >>> print system(join(bo, 'bin', 'run')), |
|
6158 3 1 |
|
6159 |
|
6160 We specified an interpreter and its paths are adjusted too: |
|
6161 |
|
6162 >>> cat(bo, 'bin', 'py') # doctest: +NORMALIZE_WHITESPACE |
|
6163 #!/usr/local/bin/python2.4 |
|
6164 <BLANKLINE> |
|
6165 import os |
|
6166 <BLANKLINE> |
|
6167 join = os.path.join |
|
6168 base = os.path.dirname(os.path.abspath(os.path.realpath(__file__))) |
|
6169 base = os.path.dirname(base) |
|
6170 <BLANKLINE> |
|
6171 import sys |
|
6172 <BLANKLINE> |
|
6173 sys.path[0:0] = [ |
|
6174 join(base, 'eggs/demo-0.3-pyN.N.egg'), |
|
6175 join(base, 'eggs/demoneeded-1.1-pyN.N.egg'), |
|
6176 '/ba', |
|
6177 join(base, 'bar'), |
|
6178 ] |
|
6179 <BLANKLINE> |
|
6180 _interactive = True |
|
6181 if len(sys.argv) > 1: |
|
6182 _options, _args = __import__("getopt").getopt(sys.argv[1:], 'ic:m:') |
|
6183 _interactive = False |
|
6184 for (_opt, _val) in _options: |
|
6185 if _opt == '-i': |
|
6186 _interactive = True |
|
6187 elif _opt == '-c': |
|
6188 exec _val |
|
6189 elif _opt == '-m': |
|
6190 sys.argv[1:] = _args |
|
6191 _args = [] |
|
6192 __import__("runpy").run_module( |
|
6193 _val, {}, "__main__", alter_sys=True) |
|
6194 <BLANKLINE> |
|
6195 if _args: |
|
6196 sys.argv[:] = _args |
|
6197 __file__ = _args[0] |
|
6198 del _options, _args |
|
6199 execfile(__file__) |
|
6200 <BLANKLINE> |
|
6201 if _interactive: |
|
6202 del _interactive |
|
6203 __import__("code").interact(banner="", local=globals()) |
|
6204 |
|
6205 The ``sitepackage_safe_scripts`` function |
|
6206 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
6207 |
|
6208 The newer function for creating scripts is ``sitepackage_safe_scripts``. |
|
6209 It has the same basic functionality as the ``scripts`` function: it can |
|
6210 create scripts to run arbitrary entry points, and to run a Python |
|
6211 interpreter. The following are the differences from a user's |
|
6212 perspective. |
|
6213 |
|
6214 - It can be used safely with a Python that has packages installed itself, |
|
6215 such as a system-installed Python. |
|
6216 |
|
6217 - In contrast to the interpreter generated by the ``scripts`` method, which |
|
6218 supports only a small subset of the usual Python executable's options, |
|
6219 the interpreter generated by ``sitepackage_safe_scripts`` supports all |
|
6220 of them. This makes it possible to use as full Python replacement for |
|
6221 scripts that need the distributions specified in your buildout. |
|
6222 |
|
6223 - Both the interpreter and the entry point scripts allow you to include the |
|
6224 site packages, and/or the sitecustomize, of the Python executable, if |
|
6225 desired. |
|
6226 |
|
6227 It works by creating site.py and sitecustomize.py files that set up the |
|
6228 desired paths and initialization. These must be placed within an otherwise |
|
6229 empty directory. Typically this is in a recipe's parts directory. |
|
6230 |
|
6231 Here's the simplest example, building an interpreter script. |
|
6232 |
|
6233 >>> interpreter_dir = tmpdir('interpreter') |
|
6234 >>> interpreter_parts_dir = os.path.join( |
|
6235 ... interpreter_dir, 'parts', 'interpreter') |
|
6236 >>> interpreter_bin_dir = os.path.join(interpreter_dir, 'bin') |
|
6237 >>> mkdir(interpreter_bin_dir) |
|
6238 >>> mkdir(interpreter_dir, 'eggs') |
|
6239 >>> mkdir(interpreter_dir, 'parts') |
|
6240 >>> mkdir(interpreter_parts_dir) |
|
6241 |
|
6242 >>> ws = zc.buildout.easy_install.install( |
|
6243 ... ['demo'], join(interpreter_dir, 'eggs'), links=[link_server], |
|
6244 ... index=link_server+'index/') |
|
6245 >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( |
|
6246 ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, |
|
6247 ... interpreter='py') |
|
6248 |
|
6249 Depending on whether the machine being used is running Windows or not, this |
|
6250 produces either three or four files. In both cases, we have site.py and |
|
6251 sitecustomize.py generated in the parts/interpreter directory. For Windows, |
|
6252 we have py.exe and py-script.py; for other operating systems, we have py. |
|
6253 |
|
6254 >>> sitecustomize_path = os.path.join( |
|
6255 ... interpreter_parts_dir, 'sitecustomize.py') |
|
6256 >>> site_path = os.path.join(interpreter_parts_dir, 'site.py') |
|
6257 >>> interpreter_path = os.path.join(interpreter_bin_dir, 'py') |
|
6258 >>> if sys.platform == 'win32': |
|
6259 ... py_path = os.path.join(interpreter_bin_dir, 'py-script.py') |
|
6260 ... expected = [sitecustomize_path, |
|
6261 ... site_path, |
|
6262 ... os.path.join(interpreter_bin_dir, 'py.exe'), |
|
6263 ... py_path] |
|
6264 ... else: |
|
6265 ... py_path = interpreter_path |
|
6266 ... expected = [sitecustomize_path, site_path, py_path] |
|
6267 ... |
|
6268 >>> assert generated == expected, repr((generated, expected)) |
|
6269 |
|
6270 We didn't ask for any initialization, and we didn't ask to use the underlying |
|
6271 sitecustomization, so sitecustomize.py is empty. |
|
6272 |
|
6273 >>> cat(sitecustomize_path) |
|
6274 |
|
6275 The interpreter script is simple. It puts the directory with the |
|
6276 site.py and sitecustomize.py on the PYTHONPATH and (re)starts Python. |
|
6277 |
|
6278 >>> cat(py_path) |
|
6279 #!/usr/bin/python -S |
|
6280 import os |
|
6281 import sys |
|
6282 <BLANKLINE> |
|
6283 argv = [sys.executable] + sys.argv[1:] |
|
6284 environ = os.environ.copy() |
|
6285 path = '/interpreter/parts/interpreter' |
|
6286 if environ.get('PYTHONPATH'): |
|
6287 path = os.pathsep.join([path, environ['PYTHONPATH']]) |
|
6288 environ['PYTHONPATH'] = path |
|
6289 os.execve(sys.executable, argv, environ) |
|
6290 |
|
6291 The site.py file is a modified version of the underlying Python's site.py. |
|
6292 The most important modification is that it has a different version of the |
|
6293 addsitepackages function. It sets up the Python path, similarly to the |
|
6294 behavior of the function it replaces. The following shows the part that |
|
6295 buildout inserts, in the simplest case. |
|
6296 |
|
6297 >>> sys.stdout.write('#\n'); cat(site_path) |
|
6298 ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE |
|
6299 #... |
|
6300 def addsitepackages(known_paths): |
|
6301 """Add site packages, as determined by zc.buildout. |
|
6302 <BLANKLINE> |
|
6303 See original_addsitepackages, below, for the original version.""" |
|
6304 buildout_paths = [ |
|
6305 '/interpreter/eggs/demo-0.3-pyN.N.egg', |
|
6306 '/interpreter/eggs/demoneeded-1.1-pyN.N.egg' |
|
6307 ] |
|
6308 for path in buildout_paths: |
|
6309 sitedir, sitedircase = makepath(path) |
|
6310 if not sitedircase in known_paths and os.path.exists(sitedir): |
|
6311 sys.path.append(sitedir) |
|
6312 known_paths.add(sitedircase) |
|
6313 return known_paths |
|
6314 <BLANKLINE> |
|
6315 def original_addsitepackages(known_paths):... |
|
6316 |
|
6317 Here are some examples of the interpreter in use. |
|
6318 |
|
6319 >>> print call_py(interpreter_path, "print 16+26") |
|
6320 42 |
|
6321 <BLANKLINE> |
|
6322 >>> res = call_py(interpreter_path, "import sys; print sys.path") |
|
6323 >>> print res # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE |
|
6324 ['', |
|
6325 '/interpreter/parts/interpreter', |
|
6326 ..., |
|
6327 '/interpreter/eggs/demo-0.3-pyN.N.egg', |
|
6328 '/interpreter/eggs/demoneeded-1.1-pyN.N.egg'] |
|
6329 <BLANKLINE> |
|
6330 >>> clean_paths = eval(res.strip()) # This is used later for comparison. |
|
6331 |
|
6332 If you provide initialization, it goes in sitecustomize.py. |
|
6333 |
|
6334 >>> def reset_interpreter(): |
|
6335 ... # This is necessary because, in our tests, the timestamps of the |
|
6336 ... # .pyc files are not outdated when we want them to be. |
|
6337 ... rmdir(interpreter_bin_dir) |
|
6338 ... mkdir(interpreter_bin_dir) |
|
6339 ... rmdir(interpreter_parts_dir) |
|
6340 ... mkdir(interpreter_parts_dir) |
|
6341 ... |
|
6342 >>> reset_interpreter() |
|
6343 |
|
6344 >>> initialization_string = """\ |
|
6345 ... import os |
|
6346 ... os.environ['FOO'] = 'bar baz bing shazam'""" |
|
6347 >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( |
|
6348 ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, |
|
6349 ... interpreter='py', initialization=initialization_string) |
|
6350 >>> cat(sitecustomize_path) |
|
6351 import os |
|
6352 os.environ['FOO'] = 'bar baz bing shazam' |
|
6353 >>> print call_py(interpreter_path, "import os; print os.environ['FOO']") |
|
6354 bar baz bing shazam |
|
6355 <BLANKLINE> |
|
6356 |
|
6357 If you use relative paths, this affects the interpreter and site.py. (This is |
|
6358 again the UNIX version; the Windows version uses subprocess instead of |
|
6359 os.execve.) |
|
6360 |
|
6361 >>> reset_interpreter() |
|
6362 >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( |
|
6363 ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, |
|
6364 ... interpreter='py', relative_paths=interpreter_dir) |
|
6365 >>> cat(py_path) |
|
6366 #!/usr/bin/python -S |
|
6367 import os |
|
6368 import sys |
|
6369 <BLANKLINE> |
|
6370 join = os.path.join |
|
6371 base = os.path.dirname(os.path.abspath(os.path.realpath(__file__))) |
|
6372 base = os.path.dirname(base) |
|
6373 <BLANKLINE> |
|
6374 argv = [sys.executable] + sys.argv[1:] |
|
6375 environ = os.environ.copy() |
|
6376 path = join(base, 'parts/interpreter') |
|
6377 if environ.get('PYTHONPATH'): |
|
6378 path = os.pathsep.join([path, environ['PYTHONPATH']]) |
|
6379 environ['PYTHONPATH'] = path |
|
6380 os.execve(sys.executable, argv, environ) |
|
6381 |
|
6382 For site.py, we again show only the pertinent parts. Notice that the egg |
|
6383 paths join a base to a path, as with the use of this argument in the |
|
6384 ``scripts`` function. |
|
6385 |
|
6386 >>> sys.stdout.write('#\n'); cat(site_path) # doctest: +ELLIPSIS |
|
6387 #... |
|
6388 def addsitepackages(known_paths): |
|
6389 """Add site packages, as determined by zc.buildout. |
|
6390 <BLANKLINE> |
|
6391 See original_addsitepackages, below, for the original version.""" |
|
6392 join = os.path.join |
|
6393 base = os.path.dirname(os.path.abspath(os.path.realpath(__file__))) |
|
6394 base = os.path.dirname(base) |
|
6395 base = os.path.dirname(base) |
|
6396 buildout_paths = [ |
|
6397 join(base, 'eggs/demo-0.3-pyN.N.egg'), |
|
6398 join(base, 'eggs/demoneeded-1.1-pyN.N.egg') |
|
6399 ]... |
|
6400 |
|
6401 The paths resolve in practice as you would expect. |
|
6402 |
|
6403 >>> print call_py(interpreter_path, |
|
6404 ... "import sys, pprint; pprint.pprint(sys.path)") |
|
6405 ... # doctest: +ELLIPSIS |
|
6406 ['', |
|
6407 '/interpreter/parts/interpreter', |
|
6408 ..., |
|
6409 '/interpreter/eggs/demo-0.3-pyN.N.egg', |
|
6410 '/interpreter/eggs/demoneeded-1.1-pyN.N.egg'] |
|
6411 <BLANKLINE> |
|
6412 |
|
6413 The ``extra_paths`` argument affects the path in site.py. Notice that |
|
6414 /interpreter/other is added after the eggs. |
|
6415 |
|
6416 >>> reset_interpreter() |
|
6417 >>> mkdir(interpreter_dir, 'other') |
|
6418 >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( |
|
6419 ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, |
|
6420 ... interpreter='py', extra_paths=[join(interpreter_dir, 'other')]) |
|
6421 >>> sys.stdout.write('#\n'); cat(site_path) # doctest: +ELLIPSIS |
|
6422 #... |
|
6423 def addsitepackages(known_paths): |
|
6424 """Add site packages, as determined by zc.buildout. |
|
6425 <BLANKLINE> |
|
6426 See original_addsitepackages, below, for the original version.""" |
|
6427 buildout_paths = [ |
|
6428 '/interpreter/eggs/demo-0.3-pyN.N.egg', |
|
6429 '/interpreter/eggs/demoneeded-1.1-pyN.N.egg', |
|
6430 '/interpreter/other' |
|
6431 ]... |
|
6432 |
|
6433 >>> print call_py(interpreter_path, |
|
6434 ... "import sys, pprint; pprint.pprint(sys.path)") |
|
6435 ... # doctest: +ELLIPSIS |
|
6436 ['', |
|
6437 '/interpreter/parts/interpreter', |
|
6438 ..., |
|
6439 '/interpreter/eggs/demo-0.3-pyN.N.egg', |
|
6440 '/interpreter/eggs/demoneeded-1.1-pyN.N.egg', |
|
6441 '/interpreter/other'] |
|
6442 <BLANKLINE> |
|
6443 |
|
6444 The ``sitepackage_safe_scripts`` function: using site-packages |
|
6445 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
6446 |
|
6447 The ``sitepackage_safe_scripts`` function supports including site |
|
6448 packages. This has some advantages and some serious dangers. |
|
6449 |
|
6450 A typical reason to include site-packages is that it is easier to |
|
6451 install one or more dependencies in your Python than it is with |
|
6452 buildout. Some packages, such as lxml or Python PostgreSQL integration, |
|
6453 have dependencies that can be much easier to build and/or install using |
|
6454 other mechanisms, such as your operating system's package manager. By |
|
6455 installing some core packages into your Python's site-packages, this can |
|
6456 significantly simplify some application installations. |
|
6457 |
|
6458 However, doing this has a significant danger. One of the primary goals |
|
6459 of buildout is to provide repeatability. Some packages (one of the |
|
6460 better known Python openid packages, for instance) change their behavior |
|
6461 depending on what packages are available. If Python curl bindings are |
|
6462 available, these may be preferred by the library. If a certain XML |
|
6463 package is installed, it may be preferred by the library. These hidden |
|
6464 choices may cause small or large behavior differences. The fact that |
|
6465 they can be rarely encountered can actually make it worse: you forget |
|
6466 that this might be a problem, and debugging the differences can be |
|
6467 difficult. If you allow site-packages to be included in your buildout, |
|
6468 and the Python you use is not managed precisely by your application (for |
|
6469 instance, it is a system Python), you open yourself up to these |
|
6470 possibilities. Don't be unaware of the dangers. |
|
6471 |
|
6472 That explained, let's see how it works. If you don't use namespace packages, |
|
6473 this is very straightforward. |
|
6474 |
|
6475 >>> reset_interpreter() |
|
6476 >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( |
|
6477 ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, |
|
6478 ... interpreter='py', include_site_packages=True) |
|
6479 >>> sys.stdout.write('#\n'); cat(site_path) |
|
6480 ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE |
|
6481 #... |
|
6482 def addsitepackages(known_paths): |
|
6483 """Add site packages, as determined by zc.buildout. |
|
6484 <BLANKLINE> |
|
6485 See original_addsitepackages, below, for the original version.""" |
|
6486 setuptools_path = None |
|
6487 buildout_paths = [ |
|
6488 '/interpreter/eggs/demo-0.3-pyN.N.egg', |
|
6489 '/interpreter/eggs/demoneeded-1.1-pyN.N.egg' |
|
6490 ] |
|
6491 for path in buildout_paths: |
|
6492 sitedir, sitedircase = makepath(path) |
|
6493 if not sitedircase in known_paths and os.path.exists(sitedir): |
|
6494 sys.path.append(sitedir) |
|
6495 known_paths.add(sitedircase) |
|
6496 sys.__egginsert = len(buildout_paths) # Support distribute. |
|
6497 original_paths = [ |
|
6498 ... |
|
6499 ] |
|
6500 for path in original_paths: |
|
6501 if path == setuptools_path or path not in known_paths: |
|
6502 addsitedir(path, known_paths) |
|
6503 return known_paths |
|
6504 <BLANKLINE> |
|
6505 def original_addsitepackages(known_paths):... |
|
6506 |
|
6507 It simply adds the original paths using addsitedir after the code to add the |
|
6508 buildout paths. |
|
6509 |
|
6510 Here's an example of the new script in use. Other documents and tests in |
|
6511 this package give the feature a more thorough workout, but this should |
|
6512 give you an idea of the feature. |
|
6513 |
|
6514 >>> res = call_py(interpreter_path, "import sys; print sys.path") |
|
6515 >>> print res # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE |
|
6516 ['', |
|
6517 '/interpreter/parts/interpreter', |
|
6518 ..., |
|
6519 '/interpreter/eggs/demo-0.3-py2.4.egg', |
|
6520 '/interpreter/eggs/demoneeded-1.1-py2.4.egg', |
|
6521 ...] |
|
6522 <BLANKLINE> |
|
6523 |
|
6524 The clean_paths gathered earlier is a subset of this full list of paths. |
|
6525 |
|
6526 >>> full_paths = eval(res.strip()) |
|
6527 >>> len(clean_paths) < len(full_paths) |
|
6528 True |
|
6529 >>> set(os.path.normpath(p) for p in clean_paths).issubset( |
|
6530 ... os.path.normpath(p) for p in full_paths) |
|
6531 True |
|
6532 |
|
6533 Unfortunately, because of how setuptools namespace packages are implemented |
|
6534 differently for operating system packages (debs or rpms) as opposed to |
|
6535 standard setuptools installation, there's a slightly trickier dance if you |
|
6536 use them. To show this we'll needs some extra eggs that use namespaces. |
|
6537 We'll use the ``tellmy.fortune`` package, which we'll need to make an initial |
|
6538 call to another text fixture to create. |
|
6539 |
|
6540 >>> from zc.buildout.tests import create_sample_namespace_eggs |
|
6541 >>> namespace_eggs = tmpdir('namespace_eggs') |
|
6542 >>> create_sample_namespace_eggs(namespace_eggs) |
|
6543 |
|
6544 >>> reset_interpreter() |
|
6545 >>> ws = zc.buildout.easy_install.install( |
|
6546 ... ['demo', 'tellmy.fortune'], join(interpreter_dir, 'eggs'), |
|
6547 ... links=[link_server, namespace_eggs], index=link_server+'index/') |
|
6548 >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( |
|
6549 ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, |
|
6550 ... interpreter='py', include_site_packages=True) |
|
6551 >>> sys.stdout.write('#\n'); cat(site_path) |
|
6552 ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE |
|
6553 #... |
|
6554 def addsitepackages(known_paths): |
|
6555 """Add site packages, as determined by zc.buildout. |
|
6556 <BLANKLINE> |
|
6557 See original_addsitepackages, below, for the original version.""" |
|
6558 setuptools_path = '...setuptools...' |
|
6559 sys.path.append(setuptools_path) |
|
6560 known_paths.add(os.path.normcase(setuptools_path)) |
|
6561 import pkg_resources |
|
6562 buildout_paths = [ |
|
6563 '/interpreter/eggs/demo-0.3-pyN.N.egg', |
|
6564 '/interpreter/eggs/tellmy.fortune-1.0-pyN.N.egg', |
|
6565 '...setuptools...', |
|
6566 '/interpreter/eggs/demoneeded-1.1-pyN.N.egg' |
|
6567 ] |
|
6568 for path in buildout_paths: |
|
6569 sitedir, sitedircase = makepath(path) |
|
6570 if not sitedircase in known_paths and os.path.exists(sitedir): |
|
6571 sys.path.append(sitedir) |
|
6572 known_paths.add(sitedircase) |
|
6573 pkg_resources.working_set.add_entry(sitedir) |
|
6574 sys.__egginsert = len(buildout_paths) # Support distribute. |
|
6575 original_paths = [ |
|
6576 ... |
|
6577 ] |
|
6578 for path in original_paths: |
|
6579 if path == setuptools_path or path not in known_paths: |
|
6580 addsitedir(path, known_paths) |
|
6581 return known_paths |
|
6582 <BLANKLINE> |
|
6583 def original_addsitepackages(known_paths):... |
|
6584 |
|
6585 >>> print call_py(interpreter_path, "import sys; print sys.path") |
|
6586 ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE |
|
6587 ['', |
|
6588 '/interpreter/parts/interpreter', |
|
6589 ..., |
|
6590 '...setuptools...', |
|
6591 '/interpreter/eggs/demo-0.3-pyN.N.egg', |
|
6592 '/interpreter/eggs/tellmy.fortune-1.0-pyN.N.egg', |
|
6593 '/interpreter/eggs/demoneeded-1.1-pyN.N.egg', |
|
6594 ...] |
|
6595 |
|
6596 As you can see, the script now first imports pkg_resources. Then we |
|
6597 need to process egg files specially to look for namespace packages there |
|
6598 *before* we process process lines in .pth files that use the "import" |
|
6599 feature--lines that might be part of the setuptools namespace package |
|
6600 implementation for system packages, as mentioned above, and that must |
|
6601 come after processing egg namespaces. |
|
6602 |
|
6603 The most complex that this function gets is if you use namespace packages, |
|
6604 include site-packages, and use relative paths. For completeness, we'll look |
|
6605 at that result. |
|
6606 |
|
6607 >>> reset_interpreter() |
|
6608 >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( |
|
6609 ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, |
|
6610 ... interpreter='py', include_site_packages=True, |
|
6611 ... relative_paths=interpreter_dir) |
|
6612 >>> sys.stdout.write('#\n'); cat(site_path) |
|
6613 ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE |
|
6614 #... |
|
6615 def addsitepackages(known_paths): |
|
6616 """Add site packages, as determined by zc.buildout. |
|
6617 <BLANKLINE> |
|
6618 See original_addsitepackages, below, for the original version.""" |
|
6619 join = os.path.join |
|
6620 base = os.path.dirname(os.path.abspath(os.path.realpath(__file__))) |
|
6621 base = os.path.dirname(base) |
|
6622 base = os.path.dirname(base) |
|
6623 setuptools_path = '...setuptools...' |
|
6624 sys.path.append(setuptools_path) |
|
6625 known_paths.add(os.path.normcase(setuptools_path)) |
|
6626 import pkg_resources |
|
6627 buildout_paths = [ |
|
6628 join(base, 'eggs/demo-0.3-pyN.N.egg'), |
|
6629 join(base, 'eggs/tellmy.fortune-1.0-pyN.N.egg'), |
|
6630 '...setuptools...', |
|
6631 join(base, 'eggs/demoneeded-1.1-pyN.N.egg') |
|
6632 ] |
|
6633 for path in buildout_paths: |
|
6634 sitedir, sitedircase = makepath(path) |
|
6635 if not sitedircase in known_paths and os.path.exists(sitedir): |
|
6636 sys.path.append(sitedir) |
|
6637 known_paths.add(sitedircase) |
|
6638 pkg_resources.working_set.add_entry(sitedir) |
|
6639 sys.__egginsert = len(buildout_paths) # Support distribute. |
|
6640 original_paths = [ |
|
6641 ... |
|
6642 ] |
|
6643 for path in original_paths: |
|
6644 if path == setuptools_path or path not in known_paths: |
|
6645 addsitedir(path, known_paths) |
|
6646 return known_paths |
|
6647 <BLANKLINE> |
|
6648 def original_addsitepackages(known_paths):... |
|
6649 |
|
6650 >>> print call_py(interpreter_path, "import sys; print sys.path") |
|
6651 ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE |
|
6652 ['', |
|
6653 '/interpreter/parts/interpreter', |
|
6654 ..., |
|
6655 '...setuptools...', |
|
6656 '/interpreter/eggs/demo-0.3-pyN.N.egg', |
|
6657 '/interpreter/eggs/tellmy.fortune-1.0-pyN.N.egg', |
|
6658 '/interpreter/eggs/demoneeded-1.1-pyN.N.egg', |
|
6659 ...] |
|
6660 |
|
6661 The ``exec_sitecustomize`` argument does the same thing for the |
|
6662 sitecustomize module--it allows you to include the code from the |
|
6663 sitecustomize module in the underlying Python if you set the argument to |
|
6664 True. The z3c.recipe.scripts package sets up the full environment necessary |
|
6665 to demonstrate this piece. |
|
6666 |
|
6667 The ``sitepackage_safe_scripts`` function: writing scripts for entry points |
|
6668 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
6669 |
|
6670 All of the examples so far for this function have been creating |
|
6671 interpreters. The function can also write scripts for entry |
|
6672 points. They are almost identical to the scripts that we saw for the |
|
6673 ``scripts`` function except that they ``import site`` after setting the |
|
6674 sys.path to include our custom site.py and sitecustomize.py files. These |
|
6675 files then initialize the Python environment as we have already seen. Let's |
|
6676 see a simple example. |
|
6677 |
|
6678 >>> reset_interpreter() |
|
6679 >>> ws = zc.buildout.easy_install.install( |
|
6680 ... ['demo'], join(interpreter_dir, 'eggs'), links=[link_server], |
|
6681 ... index=link_server+'index/') |
|
6682 >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( |
|
6683 ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, |
|
6684 ... reqs=['demo']) |
|
6685 |
|
6686 As before, in Windows, 2 files are generated for each script. A script |
|
6687 file, ending in '-script.py', and an exe file that allows the script |
|
6688 to be invoked directly without having to specify the Python |
|
6689 interpreter and without having to provide a '.py' suffix. This is in addition |
|
6690 to the site.py and sitecustomize.py files that are generated as with our |
|
6691 interpreter examples above. |
|
6692 |
|
6693 >>> if sys.platform == 'win32': |
|
6694 ... demo_path = os.path.join(interpreter_bin_dir, 'demo-script.py') |
|
6695 ... expected = [sitecustomize_path, |
|
6696 ... site_path, |
|
6697 ... os.path.join(interpreter_bin_dir, 'demo.exe'), |
|
6698 ... demo_path] |
|
6699 ... else: |
|
6700 ... demo_path = os.path.join(interpreter_bin_dir, 'demo') |
|
6701 ... expected = [sitecustomize_path, site_path, demo_path] |
|
6702 ... |
|
6703 >>> assert generated == expected, repr((generated, expected)) |
|
6704 |
|
6705 The demo script runs the entry point defined in the demo egg: |
|
6706 |
|
6707 >>> cat(demo_path) # doctest: +NORMALIZE_WHITESPACE |
|
6708 #!/usr/local/bin/python2.4 -S |
|
6709 <BLANKLINE> |
|
6710 import sys |
|
6711 sys.path[0:0] = [ |
|
6712 '/interpreter/parts/interpreter', |
|
6713 ] |
|
6714 <BLANKLINE> |
|
6715 <BLANKLINE> |
|
6716 import os |
|
6717 path = sys.path[0] |
|
6718 if os.environ.get('PYTHONPATH'): |
|
6719 path = os.pathsep.join([path, os.environ['PYTHONPATH']]) |
|
6720 os.environ['BUILDOUT_ORIGINAL_PYTHONPATH'] = os.environ.get('PYTHONPATH', '') |
|
6721 os.environ['PYTHONPATH'] = path |
|
6722 import site # imports custom buildout-generated site.py |
|
6723 <BLANKLINE> |
|
6724 import eggrecipedemo |
|
6725 <BLANKLINE> |
|
6726 if __name__ == '__main__': |
|
6727 eggrecipedemo.main() |
|
6728 |
|
6729 >>> demo_call = join(interpreter_bin_dir, 'demo') |
|
6730 >>> if sys.platform == 'win32': |
|
6731 ... demo_call = '"%s"' % demo_call |
|
6732 >>> print system(demo_call) |
|
6733 3 1 |
|
6734 <BLANKLINE> |
|
6735 |
|
6736 There are a few differences from the ``scripts`` function. First, the |
|
6737 ``reqs`` argument (an iterable of string requirements or entry point |
|
6738 tuples) is a keyword argument here. We see that in the example above. |
|
6739 Second, the ``arguments`` argument is now named ``script_arguments`` to |
|
6740 try and clarify that it does not affect interpreters. While the |
|
6741 ``initialization`` argument continues to affect both the interpreters |
|
6742 and the entry point scripts, if you have initialization that is only |
|
6743 pertinent to the entry point scripts, you can use the |
|
6744 ``script_initialization`` argument. |
|
6745 |
|
6746 Let's see ``script_arguments`` and ``script_initialization`` in action. |
|
6747 |
|
6748 >>> reset_interpreter() |
|
6749 >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( |
|
6750 ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, |
|
6751 ... reqs=['demo'], script_arguments='1, 2', |
|
6752 ... script_initialization='import os\nos.chdir("foo")') |
|
6753 |
|
6754 >>> cat(demo_path) # doctest: +NORMALIZE_WHITESPACE |
|
6755 #!/usr/local/bin/python2.4 -S |
|
6756 import sys |
|
6757 sys.path[0:0] = [ |
|
6758 '/interpreter/parts/interpreter', |
|
6759 ] |
|
6760 <BLANKLINE> |
|
6761 import os |
|
6762 path = sys.path[0] |
|
6763 if os.environ.get('PYTHONPATH'): |
|
6764 path = os.pathsep.join([path, os.environ['PYTHONPATH']]) |
|
6765 os.environ['BUILDOUT_ORIGINAL_PYTHONPATH'] = os.environ.get('PYTHONPATH', '') |
|
6766 os.environ['PYTHONPATH'] = path |
|
6767 import site # imports custom buildout-generated site.py |
|
6768 import os |
|
6769 os.chdir("foo") |
|
6770 <BLANKLINE> |
|
6771 import eggrecipedemo |
|
6772 <BLANKLINE> |
|
6773 if __name__ == '__main__': |
|
6774 eggrecipedemo.main(1, 2) |
|
6775 |
|
6776 Handling custom build options for extensions provided in source distributions |
|
6777 ----------------------------------------------------------------------------- |
|
6778 |
|
6779 Sometimes, we need to control how extension modules are built. The |
|
6780 build function provides this level of control. It takes a single |
|
6781 package specification, downloads a source distribution, and builds it |
|
6782 with specified custom build options. |
|
6783 |
|
6784 The build function takes 3 positional arguments: |
|
6785 |
|
6786 spec |
|
6787 A package specification for a source distribution |
|
6788 |
|
6789 dest |
|
6790 A destination directory |
|
6791 |
|
6792 build_ext |
|
6793 A dictionary of options to be passed to the distutils build_ext |
|
6794 command when building extensions. |
|
6795 |
|
6796 It supports a number of optional keyword arguments: |
|
6797 |
|
6798 links |
|
6799 a sequence of URLs, file names, or directories to look for |
|
6800 links to distributions, |
|
6801 |
|
6802 index |
|
6803 The URL of an index server, or almost any other valid URL. :) |
|
6804 |
|
6805 If not specified, the Python Package Index, |
|
6806 http://pypi.python.org/simple/, is used. You can specify an |
|
6807 alternate index with this option. If you use the links option and |
|
6808 if the links point to the needed distributions, then the index can |
|
6809 be anything and will be largely ignored. In the examples, here, |
|
6810 we'll just point to an empty directory on our link server. This |
|
6811 will make our examples run a little bit faster. |
|
6812 |
|
6813 executable |
|
6814 A path to a Python executable. Distributions will be installed |
|
6815 using this executable and will be for the matching Python version. |
|
6816 |
|
6817 path |
|
6818 A list of additional directories to search for locally-installed |
|
6819 distributions. |
|
6820 |
|
6821 newest |
|
6822 A boolean value indicating whether to search for new distributions |
|
6823 when already-installed distributions meet the requirement. When |
|
6824 this is true, the default, and when the destination directory is |
|
6825 not None, then the install function will search for the newest |
|
6826 distributions that satisfy the requirements. |
|
6827 |
|
6828 versions |
|
6829 A dictionary mapping project names to version numbers to be used |
|
6830 when selecting distributions. This can be used to specify a set of |
|
6831 distribution versions independent of other requirements. |
|
6832 |
|
6833 |
|
6834 Our link server included a source distribution that includes a simple |
|
6835 extension, extdemo.c:: |
|
6836 |
|
6837 #include <Python.h> |
|
6838 #include <extdemo.h> |
|
6839 |
|
6840 static PyMethodDef methods[] = {}; |
|
6841 |
|
6842 PyMODINIT_FUNC |
|
6843 initextdemo(void) |
|
6844 { |
|
6845 PyObject *m; |
|
6846 m = Py_InitModule3("extdemo", methods, ""); |
|
6847 #ifdef TWO |
|
6848 PyModule_AddObject(m, "val", PyInt_FromLong(2)); |
|
6849 #else |
|
6850 PyModule_AddObject(m, "val", PyInt_FromLong(EXTDEMO)); |
|
6851 #endif |
|
6852 } |
|
6853 |
|
6854 The extension depends on a system-dependent include file, extdemo.h, |
|
6855 that defines a constant, EXTDEMO, that is exposed by the extension. |
|
6856 |
|
6857 We'll add an include directory to our sample buildout and add the |
|
6858 needed include file to it: |
|
6859 |
|
6860 >>> mkdir('include') |
|
6861 >>> write('include', 'extdemo.h', |
|
6862 ... """ |
|
6863 ... #define EXTDEMO 42 |
|
6864 ... """) |
|
6865 |
|
6866 Now, we can use the build function to create an egg from the source |
|
6867 distribution: |
|
6868 |
|
6869 >>> zc.buildout.easy_install.build( |
|
6870 ... 'extdemo', dest, |
|
6871 ... {'include-dirs': os.path.join(sample_buildout, 'include')}, |
|
6872 ... links=[link_server], index=link_server+'index/') |
|
6873 ['/sample-install/extdemo-1.4-py2.4-unix-i686.egg'] |
|
6874 |
|
6875 The function returns the list of eggs |
|
6876 |
|
6877 Now if we look in our destination directory, we see we have an extdemo egg: |
|
6878 |
|
6879 >>> ls(dest) |
|
6880 - demo-0.2-py2.4.egg |
|
6881 d demo-0.3-py2.4.egg |
|
6882 - demoneeded-1.0-py2.4.egg |
|
6883 d demoneeded-1.1-py2.4.egg |
|
6884 d extdemo-1.4-py2.4-unix-i686.egg |
|
6885 |
|
6886 Let's update our link server with a new version of extdemo: |
|
6887 |
|
6888 >>> update_extdemo() |
|
6889 >>> print get(link_server), |
|
6890 <html><body> |
|
6891 <a href="bigdemo-0.1-py2.4.egg">bigdemo-0.1-py2.4.egg</a><br> |
|
6892 <a href="demo-0.1-py2.4.egg">demo-0.1-py2.4.egg</a><br> |
|
6893 <a href="demo-0.2-py2.4.egg">demo-0.2-py2.4.egg</a><br> |
|
6894 <a href="demo-0.3-py2.4.egg">demo-0.3-py2.4.egg</a><br> |
|
6895 <a href="demo-0.4c1-py2.4.egg">demo-0.4c1-py2.4.egg</a><br> |
|
6896 <a href="demoneeded-1.0.zip">demoneeded-1.0.zip</a><br> |
|
6897 <a href="demoneeded-1.1.zip">demoneeded-1.1.zip</a><br> |
|
6898 <a href="demoneeded-1.2c1.zip">demoneeded-1.2c1.zip</a><br> |
|
6899 <a href="extdemo-1.4.zip">extdemo-1.4.zip</a><br> |
|
6900 <a href="extdemo-1.5.zip">extdemo-1.5.zip</a><br> |
|
6901 <a href="index/">index/</a><br> |
|
6902 <a href="other-1.0-py2.4.egg">other-1.0-py2.4.egg</a><br> |
|
6903 </body></html> |
|
6904 |
|
6905 The easy_install caches information about servers to reduce network |
|
6906 access. To see the update, we have to call the clear_index_cache |
|
6907 function to clear the index cache: |
|
6908 |
|
6909 >>> zc.buildout.easy_install.clear_index_cache() |
|
6910 |
|
6911 If we run build with newest set to False, we won't get an update: |
|
6912 |
|
6913 >>> zc.buildout.easy_install.build( |
|
6914 ... 'extdemo', dest, |
|
6915 ... {'include-dirs': os.path.join(sample_buildout, 'include')}, |
|
6916 ... links=[link_server], index=link_server+'index/', |
|
6917 ... newest=False) |
|
6918 ['/sample-install/extdemo-1.4-py2.4-linux-i686.egg'] |
|
6919 |
|
6920 >>> ls(dest) |
|
6921 - demo-0.2-py2.4.egg |
|
6922 d demo-0.3-py2.4.egg |
|
6923 - demoneeded-1.0-py2.4.egg |
|
6924 d demoneeded-1.1-py2.4.egg |
|
6925 d extdemo-1.4-py2.4-unix-i686.egg |
|
6926 |
|
6927 But if we run it with the default True setting for newest, then we'll |
|
6928 get an updated egg: |
|
6929 |
|
6930 >>> zc.buildout.easy_install.build( |
|
6931 ... 'extdemo', dest, |
|
6932 ... {'include-dirs': os.path.join(sample_buildout, 'include')}, |
|
6933 ... links=[link_server], index=link_server+'index/') |
|
6934 ['/sample-install/extdemo-1.5-py2.4-unix-i686.egg'] |
|
6935 |
|
6936 >>> ls(dest) |
|
6937 - demo-0.2-py2.4.egg |
|
6938 d demo-0.3-py2.4.egg |
|
6939 - demoneeded-1.0-py2.4.egg |
|
6940 d demoneeded-1.1-py2.4.egg |
|
6941 d extdemo-1.4-py2.4-unix-i686.egg |
|
6942 d extdemo-1.5-py2.4-unix-i686.egg |
|
6943 |
|
6944 The versions option also influences the versions used. For example, |
|
6945 if we specify a version for extdemo, then that will be used, even |
|
6946 though it isn't the newest. Let's clean out the destination directory |
|
6947 first: |
|
6948 |
|
6949 >>> import os |
|
6950 >>> for name in os.listdir(dest): |
|
6951 ... remove(dest, name) |
|
6952 |
|
6953 >>> zc.buildout.easy_install.build( |
|
6954 ... 'extdemo', dest, |
|
6955 ... {'include-dirs': os.path.join(sample_buildout, 'include')}, |
|
6956 ... links=[link_server], index=link_server+'index/', |
|
6957 ... versions=dict(extdemo='1.4')) |
|
6958 ['/sample-install/extdemo-1.4-py2.4-unix-i686.egg'] |
|
6959 |
|
6960 >>> ls(dest) |
|
6961 d extdemo-1.4-py2.4-unix-i686.egg |
|
6962 |
|
6963 Handling custom build options for extensions in develop eggs |
|
6964 ------------------------------------------------------------ |
|
6965 |
|
6966 The develop function is similar to the build function, except that, |
|
6967 rather than building an egg from a source directory containing a |
|
6968 setup.py script. |
|
6969 |
|
6970 The develop function takes 2 positional arguments: |
|
6971 |
|
6972 setup |
|
6973 The path to a setup script, typically named "setup.py", or a |
|
6974 directory containing a setup.py script. |
|
6975 |
|
6976 dest |
|
6977 The directory to install the egg link to |
|
6978 |
|
6979 It supports some optional keyword argument: |
|
6980 |
|
6981 build_ext |
|
6982 A dictionary of options to be passed to the distutils build_ext |
|
6983 command when building extensions. |
|
6984 |
|
6985 executable |
|
6986 A path to a Python executable. Distributions will be installed |
|
6987 using this executable and will be for the matching Python version. |
|
6988 |
|
6989 We have a local directory containing the extdemo source: |
|
6990 |
|
6991 >>> ls(extdemo) |
|
6992 - MANIFEST |
|
6993 - MANIFEST.in |
|
6994 - README |
|
6995 - extdemo.c |
|
6996 - setup.py |
|
6997 |
|
6998 Now, we can use the develop function to create a develop egg from the source |
|
6999 distribution: |
|
7000 |
|
7001 >>> zc.buildout.easy_install.develop( |
|
7002 ... extdemo, dest, |
|
7003 ... {'include-dirs': os.path.join(sample_buildout, 'include')}) |
|
7004 '/sample-install/extdemo.egg-link' |
|
7005 |
|
7006 The name of the egg link created is returned. |
|
7007 |
|
7008 Now if we look in our destination directory, we see we have an extdemo |
|
7009 egg link: |
|
7010 |
|
7011 >>> ls(dest) |
|
7012 d extdemo-1.4-py2.4-unix-i686.egg |
|
7013 - extdemo.egg-link |
|
7014 |
|
7015 And that the source directory contains the compiled extension: |
|
7016 |
|
7017 >>> ls(extdemo) |
|
7018 - MANIFEST |
|
7019 - MANIFEST.in |
|
7020 - README |
|
7021 d build |
|
7022 - extdemo.c |
|
7023 d extdemo.egg-info |
|
7024 - extdemo.so |
|
7025 - setup.py |
|
7026 |
|
7027 Download cache |
|
7028 -------------- |
|
7029 |
|
7030 Normally, when distributions are installed, if any processing is |
|
7031 needed, they are downloaded from the internet to a temporary directory |
|
7032 and then installed from there. A download cache can be used to avoid |
|
7033 the download step. This can be useful to reduce network access and to |
|
7034 create source distributions of an entire buildout. |
|
7035 |
|
7036 A download cache is specified by calling the download_cache |
|
7037 function. The function always returns the previous setting. If no |
|
7038 argument is passed, then the setting is unchanged. If an argument is |
|
7039 passed, the download cache is set to the given path, which must point |
|
7040 to an existing directory. Passing None clears the cache setting. |
|
7041 |
|
7042 To see this work, we'll create a directory and set it as the cache |
|
7043 directory: |
|
7044 |
|
7045 >>> cache = tmpdir('cache') |
|
7046 >>> zc.buildout.easy_install.download_cache(cache) |
|
7047 |
|
7048 We'll recreate our destination directory: |
|
7049 |
|
7050 >>> remove(dest) |
|
7051 >>> dest = tmpdir('sample-install') |
|
7052 |
|
7053 We'd like to see what is being fetched from the server, so we'll |
|
7054 enable server logging: |
|
7055 |
|
7056 >>> get(link_server+'enable_server_logging') |
|
7057 GET 200 /enable_server_logging |
|
7058 '' |
|
7059 |
|
7060 Now, if we install demo, and extdemo: |
|
7061 |
|
7062 >>> ws = zc.buildout.easy_install.install( |
|
7063 ... ['demo==0.2'], dest, |
|
7064 ... links=[link_server], index=link_server+'index/', |
|
7065 ... always_unzip=True) |
|
7066 GET 200 / |
|
7067 GET 404 /index/demo/ |
|
7068 GET 200 /index/ |
|
7069 GET 200 /demo-0.2-py2.4.egg |
|
7070 GET 404 /index/demoneeded/ |
|
7071 GET 200 /demoneeded-1.1.zip |
|
7072 |
|
7073 >>> zc.buildout.easy_install.build( |
|
7074 ... 'extdemo', dest, |
|
7075 ... {'include-dirs': os.path.join(sample_buildout, 'include')}, |
|
7076 ... links=[link_server], index=link_server+'index/') |
|
7077 GET 404 /index/extdemo/ |
|
7078 GET 200 /extdemo-1.5.zip |
|
7079 ['/sample-install/extdemo-1.5-py2.4-linux-i686.egg'] |
|
7080 |
|
7081 Not only will we get eggs in our destination directory: |
|
7082 |
|
7083 >>> ls(dest) |
|
7084 d demo-0.2-py2.4.egg |
|
7085 d demoneeded-1.1-py2.4.egg |
|
7086 d extdemo-1.5-py2.4-linux-i686.egg |
|
7087 |
|
7088 But we'll get distributions in the cache directory: |
|
7089 |
|
7090 >>> ls(cache) |
|
7091 - demo-0.2-py2.4.egg |
|
7092 - demoneeded-1.1.zip |
|
7093 - extdemo-1.5.zip |
|
7094 |
|
7095 The cache directory contains uninstalled distributions, such as zipped |
|
7096 eggs or source distributions. |
|
7097 |
|
7098 Let's recreate our destination directory and clear the index cache: |
|
7099 |
|
7100 >>> remove(dest) |
|
7101 >>> dest = tmpdir('sample-install') |
|
7102 >>> zc.buildout.easy_install.clear_index_cache() |
|
7103 |
|
7104 Now when we install the distributions: |
|
7105 |
|
7106 >>> ws = zc.buildout.easy_install.install( |
|
7107 ... ['demo==0.2'], dest, |
|
7108 ... links=[link_server], index=link_server+'index/', |
|
7109 ... always_unzip=True) |
|
7110 GET 200 / |
|
7111 GET 404 /index/demo/ |
|
7112 GET 200 /index/ |
|
7113 GET 404 /index/demoneeded/ |
|
7114 |
|
7115 >>> zc.buildout.easy_install.build( |
|
7116 ... 'extdemo', dest, |
|
7117 ... {'include-dirs': os.path.join(sample_buildout, 'include')}, |
|
7118 ... links=[link_server], index=link_server+'index/') |
|
7119 GET 404 /index/extdemo/ |
|
7120 ['/sample-install/extdemo-1.5-py2.4-linux-i686.egg'] |
|
7121 |
|
7122 >>> ls(dest) |
|
7123 d demo-0.2-py2.4.egg |
|
7124 d demoneeded-1.1-py2.4.egg |
|
7125 d extdemo-1.5-py2.4-linux-i686.egg |
|
7126 |
|
7127 Note that we didn't download the distributions from the link server. |
|
7128 |
|
7129 If we remove the restriction on demo, we'll download a newer version |
|
7130 from the link server: |
|
7131 |
|
7132 >>> ws = zc.buildout.easy_install.install( |
|
7133 ... ['demo'], dest, |
|
7134 ... links=[link_server], index=link_server+'index/', |
|
7135 ... always_unzip=True) |
|
7136 GET 200 /demo-0.3-py2.4.egg |
|
7137 |
|
7138 Normally, the download cache is the preferred source of downloads, but |
|
7139 not the only one. |
|
7140 |
|
7141 Installing solely from a download cache |
|
7142 --------------------------------------- |
|
7143 |
|
7144 A download cache can be used as the basis of application source |
|
7145 releases. In an application source release, we want to distribute an |
|
7146 application that can be built without making any network accesses. In |
|
7147 this case, we distribute a download cache and tell the easy_install |
|
7148 module to install from the download cache only, without making network |
|
7149 accesses. The install_from_cache function can be used to signal that |
|
7150 packages should be installed only from the download cache. The |
|
7151 function always returns the previous setting. Calling it with no |
|
7152 arguments returns the current setting without changing it: |
|
7153 |
|
7154 >>> zc.buildout.easy_install.install_from_cache() |
|
7155 False |
|
7156 |
|
7157 Calling it with a boolean value changes the setting and returns the |
|
7158 previous setting: |
|
7159 |
|
7160 >>> zc.buildout.easy_install.install_from_cache(True) |
|
7161 False |
|
7162 |
|
7163 Let's remove demo-0.3-py2.4.egg from the cache, clear the index cache, |
|
7164 recreate the destination directory, and reinstall demo: |
|
7165 |
|
7166 >>> for f in os.listdir(cache): |
|
7167 ... if f.startswith('demo-0.3-'): |
|
7168 ... remove(cache, f) |
|
7169 |
|
7170 >>> zc.buildout.easy_install.clear_index_cache() |
|
7171 >>> remove(dest) |
|
7172 >>> dest = tmpdir('sample-install') |
|
7173 |
|
7174 >>> ws = zc.buildout.easy_install.install( |
|
7175 ... ['demo'], dest, |
|
7176 ... links=[link_server], index=link_server+'index/', |
|
7177 ... always_unzip=True) |
|
7178 |
|
7179 >>> ls(dest) |
|
7180 d demo-0.2-py2.4.egg |
|
7181 d demoneeded-1.1-py2.4.egg |
|
7182 |
|
7183 This time, we didn't download from or even query the link server. |
|
7184 |
|
7185 .. Disable the download cache: |
|
7186 |
|
7187 >>> zc.buildout.easy_install.download_cache(None) |
|
7188 '/cache' |
|
7189 |
|
7190 >>> zc.buildout.easy_install.install_from_cache(False) |
|
7191 True |
|
7192 |
|
7193 Distribute Support |
|
7194 ================== |
|
7195 |
|
7196 Distribute is a drop-in replacement for Setuptools. |
|
7197 |
|
7198 zc.buildout is now compatible with Distribute 0.6. To use Distribute in your |
|
7199 buildout, you need use the ``--distribute`` option of the ``bootstrap.py`` |
|
7200 script:: |
|
7201 |
|
7202 $ python bootstrap.py --distribute |
|
7203 |
|
7204 This will download and install the latest Distribute 0.6 release in the |
|
7205 ``eggs`` directory, and use this version for the scripts that are created |
|
7206 in ``bin``. |
|
7207 |
|
7208 Notice that if you have a shared eggs directory, a buildout that uses |
|
7209 Distribute will not interfer with other buildouts that are based on Setuptools |
|
7210 and that are sharing the same eggs directory. |
|
7211 |
|
7212 Form more information about the Distribute project, see: |
|
7213 http://python-distribute.org |
|
7214 |
|
7215 |
|
7216 |
|
7217 Change History |
|
7218 ************** |
|
7219 |
|
7220 1.5.2 (2010-10-11) |
|
7221 ================== |
|
7222 |
|
7223 - changed metadata 'url' to pypi.python.org in order to solve |
|
7224 a temporary outage of buildout.org |
|
7225 |
|
7226 - IMPORTANT: For better backwards compatibility with the pre-1.5 line, |
|
7227 this release has two big changes from 1.5.0 and 1.5.1. |
|
7228 |
|
7229 - Buildout defaults to including site packages. |
|
7230 |
|
7231 - Buildout loads recipes and extensions with the same constraints to |
|
7232 site-packages that it builds eggs, instead of never allowing access |
|
7233 to site-packages. |
|
7234 |
|
7235 This means that the default configuration should better support |
|
7236 pre-existing use of system Python in recipes or builds. |
|
7237 |
|
7238 - To make it easier to detect the fact that buildout has set the PYTHONPATH, |
|
7239 BUILDOUT_ORIGINAL_PYTHONPATH is always set in the environment, even if |
|
7240 PYTHONPATH was not originally set. BUILDOUT_ORIGINAL_PYTHONPATH will |
|
7241 be an empty string if PYTHONPATH was not set. |
|
7242 |
|
7243 1.5.1 (2010-08-29) |
|
7244 ================== |
|
7245 |
|
7246 New features: |
|
7247 |
|
7248 - Scripts store the old PYTHONPATH in BUILDOUT_ORIGINAL_PYTHONPATH if it |
|
7249 existed, and store nothing in the value if it did not exist. This allows |
|
7250 code that does not want subprocesses to have the system-Python-protected |
|
7251 site.py to set the environment of the subprocess as it was originally. |
|
7252 |
|
7253 Bugs fixed: |
|
7254 |
|
7255 - https://bugs.launchpad.net/bugs/623590 : If include-site-packages were |
|
7256 true and versions were not set explicitly, system eggs were preferred |
|
7257 over newer released eggs. Fixed. |
|
7258 |
|
7259 1.5.0 (2010-08-23) |
|
7260 ================== |
|
7261 |
|
7262 New features: |
|
7263 |
|
7264 - zc.buildout supports Python 2.7. |
|
7265 |
|
7266 - By default, Buildout and the bootstrap script now prefer final versions of |
|
7267 Buildout, recipes, and extensions. This can be changed by using the |
|
7268 --accept-buildout-test-releases flag (or -t for short) when calling |
|
7269 bootstrap. This will hopefully allow beta releases of these items to |
|
7270 be more easily and safely made in the future. |
|
7271 |
|
7272 NOTE: dependencies of your own software are not affected by this new |
|
7273 behavior. Buildout continues to choose the newest available versions |
|
7274 of your dependencies regardless of whether they are final releases. To |
|
7275 prevent this, use the pre-existing switch ``prefer-final = true`` in |
|
7276 the [buildout] section of your configuration file (see |
|
7277 http://pypi.python.org/pypi/zc.buildout#preferring-final-releases) or |
|
7278 pin your versions using a versions section (see |
|
7279 http://pypi.python.org/pypi/zc.buildout#repeatable-buildouts-controlling-eggs-used). |
|
7280 |
|
7281 Bugs fixed: |
|
7282 |
|
7283 - You can now again use virtualenv with Buildout. The new features to let |
|
7284 buildout be used with a system Python are disabled in this configuration, |
|
7285 and the previous script generation behavior (1.4.3) is used, even if |
|
7286 the new function ``zc.buildout.easy_install.sitepackage_safe_scripts`` |
|
7287 is used. |
|
7288 |
|
7289 1.5.0b2 (2010-04-29) |
|
7290 ==================== |
|
7291 |
|
7292 This was a re-release of 1.4.3 in order to keep 1.5.0b1 release from hurting |
|
7293 workflows that combined virtualenv with zc.buildout. |
|
7294 |
|
7295 1.5.0b1 (2010-04-29) |
|
7296 ==================== |
|
7297 |
|
7298 New Features: |
|
7299 |
|
7300 - Added buildout:socket-timout option so that socket timeout can be configured |
|
7301 both from command line and from config files. (gotcha) |
|
7302 |
|
7303 - Buildout can be safely used with a system Python (or any Python with code |
|
7304 in site-packages), as long as you use (1) A fresh checkout, (2) the |
|
7305 new bootstrap.py, and (3) recipes that use the new |
|
7306 ``zc.buildout.easy_install.sitepackage_safe_scripts`` function to generate |
|
7307 scripts and interpreters. Many recipes will need to be updated to use |
|
7308 this new function. The scripts and interpreters generated by |
|
7309 ``zc.recipe.egg`` will continue to use the older function, not safe |
|
7310 with system Pythons. Use the ``z3c.recipe.scripts`` as a replacement. |
|
7311 |
|
7312 zc.recipe.egg is still a fully supported, and simpler, way of |
|
7313 generating scripts and interpreters if you are using a "clean" Python, |
|
7314 without code installed in site-packages. It keeps its previous behavior in |
|
7315 order to provide backwards compatibility. |
|
7316 |
|
7317 The z3c.recipe.scripts recipe allows you to control how you use the |
|
7318 code in site-packages. You can exclude it entirely (preferred); allow |
|
7319 eggs in it to fulfill package dependencies declared in setup.py and |
|
7320 buildout configuration; allow it to be available but not used to |
|
7321 fulfill dependencies declared in setup.py or buildout configuration; |
|
7322 or only allow certain eggs in site-packages to fulfill dependencies. |
|
7323 |
|
7324 - Added new function, ``zc.buildout.easy_install.sitepackage_safe_scripts``, |
|
7325 to generate scripts and interpreter. It produces a full-featured |
|
7326 interpreter (all command-line options supported) and the ability to |
|
7327 safely let scripts include site packages, such as with a system |
|
7328 Python. The ``z3c.recipe.scripts`` recipe uses this new function. |
|
7329 |
|
7330 - Improve bootstrap. |
|
7331 |
|
7332 * New options let you specify where to find ez_setup.py and where to find |
|
7333 a download cache. These options can keep bootstrap from going over the |
|
7334 network. |
|
7335 |
|
7336 * Another new option lets you specify where to put generated eggs. |
|
7337 |
|
7338 * The buildout script generated by bootstrap honors more of the settings |
|
7339 in the designated configuration file (e.g., buildout.cfg). |
|
7340 |
|
7341 * Correctly handle systems where pkg_resources is present but the rest of |
|
7342 setuptools is missing (like Ubuntu installs). |
|
7343 https://bugs.launchpad.net/zc.buildout/+bug/410528 |
|
7344 |
|
7345 - You can develop zc.buildout using Distribute instead of Setuptools. Use |
|
7346 the --distribute option on the dev.py script. (Releases should be tested |
|
7347 with both Distribute and Setuptools.) The tests for zc.buildout pass |
|
7348 with Setuptools and Python 2.4, 2.5, 2.6, and 2.7; and with Distribute and |
|
7349 Python 2.5, 2.6, and 2.7. Using zc.buildout with Distribute and Python 2.4 |
|
7350 is not recommended. |
|
7351 |
|
7352 - The ``distribute-version`` now works in the [buildout] section, mirroring |
|
7353 the ``setuptools-version`` option (this is for consistency; using the |
|
7354 general-purpose ``versions`` option is preferred). |
|
7355 |
|
7356 Bugs fixed: |
|
7357 |
|
7358 - Using Distribute with the ``allow-picked-versions = false`` buildout |
|
7359 option no longer causes an error. |
|
7360 |
|
7361 - The handling and documenting of default buildout options was normalized. |
|
7362 This means, among other things, that ``bin/buildout -vv`` and |
|
7363 ``bin/buildout annotate`` correctly list more of the options. |
|
7364 |
|
7365 - Installing a namespace package using a Python that already has a package |
|
7366 in the same namespace (e.g., in the Python's site-packages) failed in |
|
7367 some cases. It is now handled correctly. |
|
7368 |
|
7369 - Another variation of this error showed itself when at least two |
|
7370 dependencies were in a shared location like site-packages, and the |
|
7371 first one met the "versions" setting. The first dependency would be |
|
7372 added, but subsequent dependencies from the same location (e.g., |
|
7373 site-packages) would use the version of the package found in the |
|
7374 shared location, ignoring the version setting. This is also now |
|
7375 handled correctly. |
|
7376 |
|
7377 1.4.3 (2009-12-10) |
|
7378 ================== |
|
7379 |
|
7380 Bugs fixed: |
|
7381 |
|
7382 - Using pre-detected setuptools version for easy_installing tgz files. This |
|
7383 prevents a recursion error when easy_installing an upgraded "distribute" |
|
7384 tgz. Note that setuptools did not have this recursion problem solely |
|
7385 because it was packaged as an ``.egg``, which does not have to go through |
|
7386 the easy_install step. |
|
7387 |
|
7388 |
|
7389 1.4.2 (2009-11-01) |
|
7390 ================== |
|
7391 |
|
7392 New Feature: |
|
7393 |
|
7394 - Added a --distribute option to the bootstrap script, in order |
|
7395 to use Distribute rather than Setuptools. By default, Setuptools |
|
7396 is used. |
|
7397 |
|
7398 Bugs fixed: |
|
7399 |
|
7400 - While checking for new versions of setuptools and buildout itself, |
|
7401 compare requirement locations instead of requirement objects. |
|
7402 |
|
7403 - Incrementing didn't work properly when extending multiple files. |
|
7404 https://bugs.launchpad.net/zc.buildout/+bug/421022 |
|
7405 |
|
7406 - The download API computed MD5 checksums of text files wrong on Windows. |
|
7407 |
|
7408 1.4.1 (2009-08-27) |
|
7409 ================== |
|
7410 |
|
7411 New Feature: |
|
7412 |
|
7413 - Added a debug built-in recipe to make writing some tests easier. |
|
7414 |
|
7415 Bugs fixed: |
|
7416 |
|
7417 - (introduced in 1.4.0) option incrementing (-=) and decrementing (-=) |
|
7418 didn't work in the buildout section. |
|
7419 https://bugs.launchpad.net/zc.buildout/+bug/420463 |
|
7420 |
|
7421 - Option incrementing and decrementing didn't work for options |
|
7422 specified on the command line. |
|
7423 |
|
7424 - Scripts generated with relative-paths enabled couldn't be |
|
7425 symbolically linked to other locations and still work. |
|
7426 |
|
7427 - Scripts run using generated interpreters didn't have __file__ set correctly. |
|
7428 |
|
7429 - The standard Python -m option didn't work for custom interpreters. |
|
7430 |
|
7431 1.4.0 (2009-08-26) |
|
7432 ================== |
|
7433 |
|
7434 - When doing variable substitutions, you can omit the section name to |
|
7435 refer to a variable in the same section (e.g. ${:foo}). |
|
7436 |
|
7437 - When doing variable substitution, you can use the special option, |
|
7438 ``_buildout_section_name_`` to get the section name. This is most handy |
|
7439 for getting the current section name (e.g. ${:_buildout_section_name_}). |
|
7440 |
|
7441 - A new special option, ``<`` allows sections to be used as macros. |
|
7442 |
|
7443 - Added annotate command for annotated sections. Displays sections |
|
7444 key-value pairs along with the value origin. |
|
7445 |
|
7446 - Added a download API that handles the download cache, offline mode etc and |
|
7447 is meant to be reused by recipes. |
|
7448 |
|
7449 - Used the download API to allow caching of base configurations (specified by |
|
7450 the buildout section's 'extends' option). |
|
7451 |
|
7452 1.3.1 (2009-08-12) |
|
7453 ================== |
|
7454 |
|
7455 - Bug fixed: extras were ignored in some cases when versions were specified. |
|
7456 |
|
7457 1.3.0 (2009-06-22) |
|
7458 ================== |
|
7459 |
|
7460 - Better Windows compatibility in test infrastructure. |
|
7461 |
|
7462 - Now the bootstrap.py has an optional --version argument, |
|
7463 that can be used to force zc.buildout version to use. |
|
7464 |
|
7465 - ``zc.buildout.testing.buildoutSetUp`` installs a new handler in the |
|
7466 python root logging facility. This handler is now removed during |
|
7467 tear down as it might disturb other packages reusing buildout's |
|
7468 testing infrastructure. |
|
7469 |
|
7470 - fixed usage of 'relative_paths' keyword parameter on Windows |
|
7471 |
|
7472 - Added an unload entry point for extensions. |
|
7473 |
|
7474 - Fixed bug: when the relative paths option was used, relative paths |
|
7475 could be inserted into sys.path if a relative path was used to run |
|
7476 the generated script. |
|
7477 |
|
7478 1.2.1 (2009-03-18) |
|
7479 ================== |
|
7480 |
|
7481 - Refactored generation of relative egg paths to generate simpler code. |
|
7482 |
|
7483 1.2.0 (2009-03-17) |
|
7484 ================== |
|
7485 |
|
7486 - Added a relative_paths option to zc.buildout.easy_install.script to |
|
7487 generate egg paths relative to the script they're used in. |
|
7488 |
|
7489 1.1.2 (2009-03-16) |
|
7490 ================== |
|
7491 |
|
7492 - Added Python 2.6 support. Removed Python 2.3 support. |
|
7493 |
|
7494 - Fixed remaining deprecation warnings under Python 2.6, both when running |
|
7495 our tests and when using the package. |
|
7496 |
|
7497 - Switched from using os.popen* to subprocess.Popen, to avoid a deprecation |
|
7498 warning in Python 2.6. See: |
|
7499 |
|
7500 http://docs.python.org/library/subprocess.html#replacing-os-popen-os-popen2-os-popen3 |
|
7501 |
|
7502 - Made sure the 'redo_pyc' function and the doctest checkers work with Python |
|
7503 executable paths containing spaces. |
|
7504 |
|
7505 - Expand shell patterns when processing the list of paths in `develop`, e.g:: |
|
7506 |
|
7507 [buildout] |
|
7508 develop = ./local-checkouts/* |
|
7509 |
|
7510 - Conditionally import and use hashlib.md5 when it's available instead |
|
7511 of md5 module, which is deprecated in Python 2.6. |
|
7512 |
|
7513 - Added Jython support for bootstrap, development bootstrap |
|
7514 and zc.buildout support on Jython |
|
7515 |
|
7516 - Fixed a bug that would cause buildout to break while computing a |
|
7517 directory hash if it found a broken symlink (Launchpad #250573) |
|
7518 |
|
7519 1.1.1 (2008-07-28) |
|
7520 ================== |
|
7521 |
|
7522 - Fixed a bug that caused buildouts to fail when variable |
|
7523 substitutions are used to name standard directories, as in:: |
|
7524 |
|
7525 [buildout] |
|
7526 eggs-directory = ${buildout:directory}/develop-eggs |
|
7527 |
|
7528 1.1.0 (2008-07-19) |
|
7529 ================== |
|
7530 |
|
7531 - Added a buildout-level unzip option tp change the default policy for |
|
7532 unzipping zip-safe eggs. |
|
7533 |
|
7534 - Tracebacks are now printed for internal errors (as opposed to user |
|
7535 errors) even without the -D option. |
|
7536 |
|
7537 - pyc and pyo files are regenerated for installed eggs so that the |
|
7538 stored path in code objects matches the the install location. |
|
7539 |
|
7540 1.0.6 (2008-06-13) |
|
7541 ================== |
|
7542 |
|
7543 - Manually reverted the changeset for the fix for |
|
7544 https://bugs.launchpad.net/zc.buildout/+bug/239212 to verify thet the test |
|
7545 actually fails with the changeset: |
|
7546 http://svn.zope.org/zc.buildout/trunk/src/zc/buildout/buildout.py?rev=87309&r1=87277&r2=87309 |
|
7547 Thanks tarek for pointing this out. (seletz) |
|
7548 |
|
7549 - fixed the test for the += -= syntax in buildout.txt as the test |
|
7550 was actually wronng. The original implementation did a split/join |
|
7551 on whitespace, and later on that was corrected to respect the original |
|
7552 EOL setting, the test was not updated, though. (seletz) |
|
7553 |
|
7554 - added a test to verify against https://bugs.launchpad.net/zc.buildout/+bug/239212 |
|
7555 in allowhosts.txt (seletz) |
|
7556 |
|
7557 - further fixes for """AttributeError: Buildout instance has no |
|
7558 attribute '_logger'""" by providing reasonable defaults |
|
7559 within the Buildout constructor (related to the new 'allow-hosts' option) |
|
7560 (patch by Gottfried Ganssauge) (ajung) |
|
7561 |
|
7562 |
|
7563 1.0.5 (2008-06-10) |
|
7564 ================== |
|
7565 |
|
7566 - Fixed wrong split when using the += and -= syntax (mustapha) |
|
7567 |
|
7568 1.0.4 (2008-06-10) |
|
7569 ================== |
|
7570 |
|
7571 - Added the `allow-hosts` option (tarek) |
|
7572 |
|
7573 - Quote the 'executable' argument when trying to detect the python |
|
7574 version using popen4. (sidnei) |
|
7575 |
|
7576 - Quote the 'spec' argument, as in the case of installing an egg from |
|
7577 the buildout-cache, if the filename contains spaces it would fail (sidnei) |
|
7578 |
|
7579 - Extended configuration syntax to allow -= and += operators (malthe, mustapha). |
|
7580 |
|
7581 1.0.3 (2008-06-01) |
|
7582 ================== |
|
7583 |
|
7584 - fix for """AttributeError: Buildout instance has no attribute '_logger'""" |
|
7585 by providing reasonable defaults within the Buildout constructor. |
|
7586 (patch by Gottfried Ganssauge) (ajung) |
|
7587 |
|
7588 1.0.2 (2008-05-13) |
|
7589 ================== |
|
7590 |
|
7591 - More fixes for Windows. A quoted sh-bang is now used on Windows to make the |
|
7592 .exe files work with a Python executable in 'program files'. |
|
7593 |
|
7594 - Added "-t <timeout_in_seconds>" option for specifying the socket timeout. |
|
7595 (ajung) |
|
7596 |
|
7597 1.0.1 (2008-04-02) |
|
7598 ================== |
|
7599 |
|
7600 - Made easy_install.py's _get_version accept non-final releases of Python, |
|
7601 like 2.4.4c0. (hannosch) |
|
7602 |
|
7603 - Applied various patches for Windows (patch by Gottfried Ganssauge). (ajung) |
|
7604 |
|
7605 - Applied patch fixing rmtree issues on Windows (patch by |
|
7606 Gottfried Ganssauge). (ajung) |
|
7607 |
|
7608 1.0.0 (2008-01-13) |
|
7609 ================== |
|
7610 |
|
7611 - Added a French translation of the buildout tutorial. |
|
7612 |
|
7613 1.0.0b31 (2007-11-01) |
|
7614 ===================== |
|
7615 |
|
7616 Feature Changes |
|
7617 --------------- |
|
7618 |
|
7619 - Added a configuration option that allows buildouts to ignore |
|
7620 dependency_links metadata specified in setup. By default |
|
7621 dependency_links in setup are used in addition to buildout specified |
|
7622 find-links. This can make it hard to control where eggs come |
|
7623 from. Here's how to tell buildout to ignore URLs in |
|
7624 dependency_links:: |
|
7625 |
|
7626 [buildout] |
|
7627 use-dependency-links = false |
|
7628 |
|
7629 By default use-dependency-links is true, which matches the behavior |
|
7630 of previous versions of buildout. |
|
7631 |
|
7632 - Added a configuration option that causes buildout to error if a |
|
7633 version is picked. This is a nice safety belt when fixing all |
|
7634 versions is intended, especially when creating releases. |
|
7635 |
|
7636 Bugs Fixed |
|
7637 ---------- |
|
7638 |
|
7639 - 151820: Develop failed if the setup.py script imported modules in |
|
7640 the distribution directory. |
|
7641 |
|
7642 - Verbose logging of the develop command was omitting detailed |
|
7643 output. |
|
7644 |
|
7645 - The setup command wasn't documented. |
|
7646 |
|
7647 - The setup command failed if run in a directory without specifying a |
|
7648 configuration file. |
|
7649 |
|
7650 - The setup command raised a stupid exception if run without arguments. |
|
7651 |
|
7652 - When using a local find links or index, distributions weren't copied |
|
7653 to the download cache. |
|
7654 |
|
7655 - When installing from source releases, a version specification (via a |
|
7656 buildout versions section) for setuptools was ignored when deciding |
|
7657 which setuptools to use to build an egg from the source release. |
|
7658 |
|
7659 1.0.0b30 (2007-08-20) |
|
7660 ===================== |
|
7661 |
|
7662 Feature Changes |
|
7663 --------------- |
|
7664 |
|
7665 - Changed the default policy back to what it was to avoid breakage in |
|
7666 existing buildouts. Use:: |
|
7667 |
|
7668 [buildout] |
|
7669 prefer-final = true |
|
7670 |
|
7671 to get the new policy. The new policy will go into effect in |
|
7672 buildout 2. |
|
7673 |
|
7674 1.0.0b29 (2007-08-20) |
|
7675 ===================== |
|
7676 |
|
7677 Feature Changes |
|
7678 --------------- |
|
7679 |
|
7680 - Now, final distributions are prefered over non-final versions. If |
|
7681 both final and non-final versions satisfy a requirement, then the |
|
7682 final version will be used even if it is older. The normal way to |
|
7683 override this for specific packages is to specifically require a |
|
7684 non-final version, either specifically or via a lower bound. |
|
7685 |
|
7686 - There is a buildout prefer-final version that can be used with a |
|
7687 value of "false":: |
|
7688 |
|
7689 prefer-final = false |
|
7690 |
|
7691 To prefer newer versions, regardless of whether or not they are |
|
7692 final, buildout-wide. |
|
7693 |
|
7694 - The new simple Python index, http://cheeseshop.python.org/simple, is |
|
7695 used as the default index. This will provide better performance |
|
7696 than the human package index interface, |
|
7697 http://pypi.python.org/pypi. More importantly, it lists hidden |
|
7698 distributions, so buildouts with fixed distribution versions will be |
|
7699 able to find old distributions even if the distributions have been |
|
7700 hidden in the human PyPI interface. |
|
7701 |
|
7702 Bugs Fixed |
|
7703 ---------- |
|
7704 |
|
7705 - 126441: Look for default.cfg in the right place on Windows. |
|
7706 |
|
7707 1.0.0b28 (2007-07-05) |
|
7708 ===================== |
|
7709 |
|
7710 Bugs Fixed |
|
7711 ---------- |
|
7712 |
|
7713 - When requiring a specific version, buildout looked for new versions |
|
7714 even if that single version was already installed. |
|
7715 |
|
7716 1.0.0b27 (2007-06-20) |
|
7717 ===================== |
|
7718 |
|
7719 Bugs Fixed |
|
7720 ---------- |
|
7721 |
|
7722 - Scripts were generated incorrectly on Windows. This included the |
|
7723 buildout script itself, making buildout completely unusable. |
|
7724 |
|
7725 1.0.0b26 (2007-06-19) |
|
7726 ===================== |
|
7727 |
|
7728 Feature Changes |
|
7729 --------------- |
|
7730 |
|
7731 - Thanks to recent fixes in setuptools, I was able to change buildout |
|
7732 to use find-link and index information when searching extensions. |
|
7733 |
|
7734 Sadly, this work, especially the timing, was motivated my the need |
|
7735 to use alternate indexes due to performance problems in the cheese |
|
7736 shop (http://www.python.org/pypi/). I really home we can address |
|
7737 these performance problems soon. |
|
7738 |
|
7739 1.0.0b25 (2007-05-31) |
|
7740 ===================== |
|
7741 |
|
7742 Feature Changes |
|
7743 --------------- |
|
7744 |
|
7745 - buildout now changes to the buildout directory before running recipe |
|
7746 install and update methods. |
|
7747 |
|
7748 - Added a new init command for creating a new buildout. This creates |
|
7749 an empty configuration file and then bootstraps. |
|
7750 |
|
7751 - Except when using the new init command, it is now an error to run |
|
7752 buildout without a configuration file. |
|
7753 |
|
7754 - In verbose mode, when adding distributions to fulful requirements of |
|
7755 already-added distributions, we now show why the new distributions |
|
7756 are being added. |
|
7757 |
|
7758 - Changed the logging format to exclude the logger name for the |
|
7759 zc.buildout logger. This reduces noise in the output. |
|
7760 |
|
7761 - Clean up lots of messages, adding missing periods and adding quotes around |
|
7762 requirement strings and file paths. |
|
7763 |
|
7764 Bugs Fixed |
|
7765 ---------- |
|
7766 |
|
7767 - 114614: Buildouts could take a very long time if there were |
|
7768 dependency problems in large sets of pathologically interdependent |
|
7769 packages. |
|
7770 |
|
7771 - 59270: Buggy recipes can cause failures in later recipes via chdir |
|
7772 |
|
7773 - 61890: file:// urls don't seem to work in find-links |
|
7774 |
|
7775 setuptools requires that file urls that point to directories must |
|
7776 end in a "/". Added a workaround. |
|
7777 |
|
7778 - 75607: buildout should not run if it creates an empty buildout.cfg |
|
7779 |
|
7780 1.0.0b24 (2007-05-09) |
|
7781 ===================== |
|
7782 |
|
7783 Feature Changes |
|
7784 --------------- |
|
7785 |
|
7786 - Improved error reporting by showing which packages require other |
|
7787 packages that can't be found or that cause version conflicts. |
|
7788 |
|
7789 - Added an API for use by recipe writers to clean up created files |
|
7790 when recipe errors occur. |
|
7791 |
|
7792 - Log installed scripts. |
|
7793 |
|
7794 Bugs Fixed |
|
7795 ---------- |
|
7796 |
|
7797 - 92891: bootstrap crashes with recipe option in buildout section. |
|
7798 |
|
7799 - 113085: Buildout exited with a zero exist status when internal errors |
|
7800 occurred. |
|
7801 |
|
7802 |
|
7803 1.0.0b23 (2007-03-19) |
|
7804 ===================== |
|
7805 |
|
7806 Feature Changes |
|
7807 --------------- |
|
7808 |
|
7809 - Added support for download caches. A buildout can specify a cache |
|
7810 for distribution downloads. The cache can be shared among buildouts |
|
7811 to reduce network access and to support creating source |
|
7812 distributions for applications allowing install without network |
|
7813 access. |
|
7814 |
|
7815 - Log scripts created, as suggested in: |
|
7816 https://bugs.launchpad.net/zc.buildout/+bug/71353 |
|
7817 |
|
7818 Bugs Fixed |
|
7819 ---------- |
|
7820 |
|
7821 - It wasn't possible to give options on the command line for sections |
|
7822 not defined in a configuration file. |
|
7823 |
|
7824 1.0.0b22 (2007-03-15) |
|
7825 ===================== |
|
7826 |
|
7827 Feature Changes |
|
7828 --------------- |
|
7829 |
|
7830 - Improved error reporting and debugging support: |
|
7831 |
|
7832 - Added "logical tracebacks" that show functionally what the buildout |
|
7833 was doing when an error occurs. Don't show a Python traceback |
|
7834 unless the -D option is used. |
|
7835 |
|
7836 - Added a -D option that causes the buildout to print a traceback and |
|
7837 start the pdb post-mortem debugger when an error occurs. |
|
7838 |
|
7839 - Warnings are printed for unused options in the buildout section and |
|
7840 installed-part sections. This should make it easier to catch option |
|
7841 misspellings. |
|
7842 |
|
7843 - Changed the way the installed database (.installed.cfg) is handled |
|
7844 to avoid database corruption when a user breaks out of a buildout |
|
7845 with control-c. |
|
7846 |
|
7847 - Don't save an installed database if there are no installed parts or |
|
7848 develop egg links. |
|
7849 |
|
7850 1.0.0b21 (2007-03-06) |
|
7851 ===================== |
|
7852 |
|
7853 Feature Changes |
|
7854 --------------- |
|
7855 |
|
7856 - Added support for repeatable buildouts by allowing egg versions to |
|
7857 be specified in a versions section. |
|
7858 |
|
7859 - The easy_install module install and build functions now accept a |
|
7860 versions argument that supplied to mapping from project name to |
|
7861 version numbers. This can be used to fix version numbers for |
|
7862 required distributions and their depenencies. |
|
7863 |
|
7864 When a version isn't fixed, using either a versions option or using |
|
7865 a fixed version number in a requirement, then a debug log message is |
|
7866 emitted indicating the version picked. This is useful for setting |
|
7867 versions options. |
|
7868 |
|
7869 A default_versions function can be used to set a default value for |
|
7870 this option. |
|
7871 |
|
7872 - Adjusted the output for verbosity levels. Using a single -v option |
|
7873 no longer causes voluminous setuptools output. Uisng -vv and -vvv |
|
7874 now triggers extra setuptools output. |
|
7875 |
|
7876 - Added a remove testing helper function that removes files or directories. |
|
7877 |
|
7878 1.0.0b20 (2007-02-08) |
|
7879 ===================== |
|
7880 |
|
7881 Feature Changes |
|
7882 --------------- |
|
7883 |
|
7884 - Added a buildout newest option, to control whether the newest |
|
7885 distributions should be sought to meet requirements. This might |
|
7886 also provide a hint to recipes that don't deal with |
|
7887 distributions. For example, a recipe that manages subversion |
|
7888 checkouts might not update a checkout if newest is set to "false". |
|
7889 |
|
7890 - Added a *newest* keyword parameter to the |
|
7891 zc.buildout.easy_install.install and zc.buildout.easy_install.build |
|
7892 functions to control whether the newest distributions that meed |
|
7893 given requirements should be sought. If a false value is provided |
|
7894 for this parameter and already installed eggs meet the given |
|
7895 requirements, then no attempt will be made to search for newer |
|
7896 distributions. |
|
7897 |
|
7898 - The recipe-testing support setUp function now adds the name |
|
7899 *buildout* to the test namespace with a value that is the path to |
|
7900 the buildout script in the sample buildout. This allows tests to |
|
7901 use |
|
7902 |
|
7903 >>> print system(buildout), |
|
7904 |
|
7905 rather than: |
|
7906 |
|
7907 >>> print system(join('bin', 'buildout')), |
|
7908 |
|
7909 |
|
7910 Bugs Fixed |
|
7911 ---------- |
|
7912 |
|
7913 - Paths returned from update methods replaced lists of installed files |
|
7914 rather than augmenting them. |
|
7915 |
|
7916 1.0.0b19 (2007-01-24) |
|
7917 ===================== |
|
7918 |
|
7919 Bugs Fixed |
|
7920 ---------- |
|
7921 |
|
7922 - Explicitly specifying a Python executable failed if the output of |
|
7923 running Python with the -V option included a 2-digit (rather than a |
|
7924 3-digit) version number. |
|
7925 |
|
7926 1.0.0b18 (2007-01-22) |
|
7927 ===================== |
|
7928 |
|
7929 Feature Changes |
|
7930 --------------- |
|
7931 |
|
7932 - Added documentation for some previously undocumented features of the |
|
7933 easy_install APIs. |
|
7934 |
|
7935 - By popular demand, added a -o command-line option that is a short |
|
7936 hand for the assignment buildout:offline=true. |
|
7937 |
|
7938 Bugs Fixed |
|
7939 ---------- |
|
7940 |
|
7941 - When deciding whether recipe develop eggs had changed, buildout |
|
7942 incorrectly considered files in .svn and CVS directories. |
|
7943 |
|
7944 1.0.0b17 (2006-12-07) |
|
7945 ===================== |
|
7946 |
|
7947 Feature Changes |
|
7948 --------------- |
|
7949 |
|
7950 - Configuration files can now be loaded from URLs. |
|
7951 |
|
7952 Bugs Fixed |
|
7953 ---------- |
|
7954 |
|
7955 - https://bugs.launchpad.net/products/zc.buildout/+bug/71246 |
|
7956 |
|
7957 Buildout extensions installed as eggs couldn't be loaded in offline |
|
7958 mode. |
|
7959 |
|
7960 |
|
7961 1.0.0b16 (2006-12-07) |
|
7962 ===================== |
|
7963 |
|
7964 Feature Changes |
|
7965 --------------- |
|
7966 |
|
7967 - A new command-line argument, -U, suppresses reading user defaults. |
|
7968 |
|
7969 - You can now suppress use of an installed-part database |
|
7970 (e.g. .installed.cfg) by sprifying an empty value for the buildout |
|
7971 installed option. |
|
7972 |
|
7973 Bugs Fixed |
|
7974 ---------- |
|
7975 |
|
7976 - When the install command is used with a list of parts, only |
|
7977 those parts are supposed to be installed, but the buildout was also |
|
7978 building parts that those parts depended on. |
|
7979 |
|
7980 1.0.0b15 (2006-12-06) |
|
7981 ===================== |
|
7982 |
|
7983 Bugs Fixed |
|
7984 ---------- |
|
7985 |
|
7986 - Uninstall recipes weren't loaded correctly in cases where |
|
7987 no parts in the (new) configuration used the recipe egg. |
|
7988 |
|
7989 1.0.0b14 (2006-12-05) |
|
7990 ===================== |
|
7991 |
|
7992 Feature Changes |
|
7993 --------------- |
|
7994 |
|
7995 - Added uninstall recipes for dealing with complex uninstallation |
|
7996 scenarios. |
|
7997 |
|
7998 Bugs Fixed |
|
7999 ---------- |
|
8000 |
|
8001 - Automatic upgrades weren't performed on Windows due to a bug that |
|
8002 caused buildout to incorrectly determine that it wasn't running |
|
8003 locally in a buildout. |
|
8004 |
|
8005 - Fixed some spurious test failures on Windows. |
|
8006 |
|
8007 1.0.0b13 (2006-12-04) |
|
8008 ===================== |
|
8009 |
|
8010 Feature Changes |
|
8011 --------------- |
|
8012 |
|
8013 - Variable substitutions now reflect option data written by recipes. |
|
8014 |
|
8015 - A part referenced by a part in a parts list is now added to the parts |
|
8016 list before the referencing part. This means that you can omit |
|
8017 parts from the parts list if they are referenced by other parts. |
|
8018 |
|
8019 - Added a develop function to the easy_install module to aid in |
|
8020 creating develop eggs with custom build_ext options. |
|
8021 |
|
8022 - The build and develop functions in the easy_install module now |
|
8023 return the path of the egg or egg link created. |
|
8024 |
|
8025 - Removed the limitation that parts named in the install command can |
|
8026 only name configured parts. |
|
8027 |
|
8028 - Removed support ConfigParser-style variable substitutions |
|
8029 (e.g. %(foo)s). Only the string-template style of variable |
|
8030 (e.g. ${section:option}) substitutions will be supported. |
|
8031 Supporting both violates "there's only one way to do it". |
|
8032 |
|
8033 - Deprecated the buildout-section extendedBy option. |
|
8034 |
|
8035 Bugs Fixed |
|
8036 ---------- |
|
8037 |
|
8038 - We treat setuptools as a dependency of any distribution that |
|
8039 (declares that it) uses namespace packages, whether it declares |
|
8040 setuptools as a dependency or not. This wasn't working for eggs |
|
8041 intalled by virtue of being dependencies. |
|
8042 |
|
8043 |
|
8044 1.0.0b12 (2006-10-24) |
|
8045 ===================== |
|
8046 |
|
8047 Feature Changes |
|
8048 --------------- |
|
8049 |
|
8050 - Added an initialization argument to the |
|
8051 zc.buildout.easy_install.scripts function to include initialization |
|
8052 code in generated scripts. |
|
8053 |
|
8054 1.0.0b11 (2006-10-24) |
|
8055 ===================== |
|
8056 |
|
8057 Bugs Fixed |
|
8058 ---------- |
|
8059 |
|
8060 `67737 <https://launchpad.net/products/zc.buildout/+bug/67737>`_ |
|
8061 Verbose and quite output options caused errors when the |
|
8062 develop buildout option was used to create develop eggs. |
|
8063 |
|
8064 `67871 <https://launchpad.net/products/zc.buildout/+bug/67871>`_ |
|
8065 Installation failed if the source was a (local) unzipped |
|
8066 egg. |
|
8067 |
|
8068 `67873 <https://launchpad.net/products/zc.buildout/+bug/67873>`_ |
|
8069 There was an error in producing an error message when part names |
|
8070 passed to the install command weren't included in the |
|
8071 configuration. |
|
8072 |
|
8073 1.0.0b10 (2006-10-16) |
|
8074 ===================== |
|
8075 |
|
8076 Feature Changes |
|
8077 --------------- |
|
8078 |
|
8079 - Renamed the runsetup command to setup. (The old name still works.) |
|
8080 |
|
8081 - Added a recipe update method. Now install is only called when a part |
|
8082 is installed for the first time, or after an uninstall. Otherwise, |
|
8083 update is called. For backward compatibility, recipes that don't |
|
8084 define update methiods are still supported. |
|
8085 |
|
8086 - If a distribution defines namespace packages but fails to declare |
|
8087 setuptools as one of its dependencies, we now treat setuptools as an |
|
8088 implicit dependency. We generate a warning if the distribution |
|
8089 is a develop egg. |
|
8090 |
|
8091 - You can now create develop eggs for setup scripts that don't use setuptools. |
|
8092 |
|
8093 Bugs Fixed |
|
8094 ---------- |
|
8095 |
|
8096 - Egg links weren't removed when corresponding entries were removed |
|
8097 from develop sections. |
|
8098 |
|
8099 - Running a non-local buildout command (one not installed in the |
|
8100 buildout) ket to a hang if there were new versions of zc.buildout or |
|
8101 setuptools were available. Now we issue a warning and don't |
|
8102 upgrade. |
|
8103 |
|
8104 - When installing zip-safe eggs from local directories, the eggs were |
|
8105 moved, rather than copied, removing them from the source directory. |
|
8106 |
|
8107 1.0.0b9 (2006-10-02) |
|
8108 ==================== |
|
8109 |
|
8110 Bugs Fixed |
|
8111 ---------- |
|
8112 |
|
8113 Non-zip-safe eggs were not unzipped when they were installed. |
|
8114 |
|
8115 1.0.0b8 (2006-10-01) |
|
8116 ==================== |
|
8117 |
|
8118 Bugs Fixed |
|
8119 ---------- |
|
8120 |
|
8121 - Installing source distributions failed when using alternate Python |
|
8122 versions (depending on the versions of Python used.) |
|
8123 |
|
8124 - Installing eggs wasn't handled as efficiently as possible due to a |
|
8125 bug in egg URL parsing. |
|
8126 |
|
8127 - Fixed a bug in runsetup that caused setup scripts that introspected |
|
8128 __file__ to fail. |
|
8129 |
|
8130 1.0.0b7 |
|
8131 ======= |
|
8132 |
|
8133 Added a documented testing framework for use by recipes. Refactored |
|
8134 the buildout tests to use it. |
|
8135 |
|
8136 Added a runsetup command run a setup script. This is handy if, like |
|
8137 me, you don't install setuptools in your system Python. |
|
8138 |
|
8139 1.0.0b6 |
|
8140 ======= |
|
8141 |
|
8142 Fixed https://launchpad.net/products/zc.buildout/+bug/60582 |
|
8143 Use of extension options caused bootstrapping to fail if the eggs |
|
8144 directory didn't already exist. We no longer use extensions for |
|
8145 bootstrapping. There really isn't any reason to anyway. |
|
8146 |
|
8147 |
|
8148 1.0.0b5 |
|
8149 ======= |
|
8150 |
|
8151 Refactored to do more work in buildout and less work in easy_install. |
|
8152 This makes things go a little faster, makes errors a little easier to |
|
8153 handle, and allows extensions (like the sftp extension) to influence |
|
8154 more of the process. This was done to fix a problem in using the sftp |
|
8155 support. |
|
8156 |
|
8157 1.0.0b4 |
|
8158 ======= |
|
8159 |
|
8160 - Added an **experimental** extensions mechanism, mainly to support |
|
8161 adding sftp support to buildouts that need it. |
|
8162 |
|
8163 - Fixed buildout self-updating on Windows. |
|
8164 |
|
8165 1.0.0b3 |
|
8166 ======= |
|
8167 |
|
8168 - Added a help option (-h, --help) |
|
8169 |
|
8170 - Increased the default level of verbosity. |
|
8171 |
|
8172 - Buildouts now automatically update themselves to new versions of |
|
8173 zc.buildout and setuptools. |
|
8174 |
|
8175 - Added Windows support. |
|
8176 |
|
8177 - Added a recipe API for generating user errors. |
|
8178 |
|
8179 - No-longer generate a py_zc.buildout script. |
|
8180 |
|
8181 - Fixed some bugs in variable substitutions. |
|
8182 |
|
8183 The characters "-", "." and " ", weren't allowed in section or |
|
8184 option names. |
|
8185 |
|
8186 Substitutions with invalid names were ignored, which caused |
|
8187 missleading failures downstream. |
|
8188 |
|
8189 - Improved error handling. No longer show tracebacks for user errors. |
|
8190 |
|
8191 - Now require a recipe option (and therefore a section) for every part. |
|
8192 |
|
8193 - Expanded the easy_install module API to: |
|
8194 |
|
8195 - Allow extra paths to be provided |
|
8196 |
|
8197 - Specify explicit entry points |
|
8198 |
|
8199 - Specify entry-point arguments |
|
8200 |
|
8201 1.0.0b2 |
|
8202 ======= |
|
8203 |
|
8204 Added support for specifying some build_ext options when installing eggs |
|
8205 from source distributions. |
|
8206 |
|
8207 1.0.0b1 |
|
8208 ======= |
|
8209 |
|
8210 - Changed the bootstrapping code to only install setuptools and |
|
8211 zc.buildout. The bootstrap code no-longer runs the buildout itself. |
|
8212 This was to fix a bug that caused parts to be recreated |
|
8213 unnecessarily because the recipe signature in the initial buildout |
|
8214 reflected temporary locations for setuptools and zc.buildout. |
|
8215 |
|
8216 - Now create a minimal setup.py if it doesn't exist and issue a |
|
8217 warning that it is being created. |
|
8218 |
|
8219 - Fixed bug in saving installed configuration data. %'s and extra |
|
8220 spaces weren't quoted. |
|
8221 |
|
8222 1.0.0a1 |
|
8223 ======= |
|
8224 |
|
8225 Initial public version |
|
8226 |
|
8227 Download |
|
8228 ********************** |
|
8229 |
|
8230 Keywords: development build |
|
8231 Platform: UNKNOWN |
|
8232 Classifier: Intended Audience :: Developers |
|
8233 Classifier: License :: OSI Approved :: Zope Public License |
|
8234 Classifier: Topic :: Software Development :: Build Tools |
|
8235 Classifier: Topic :: Software Development :: Libraries :: Python Modules |