|
1 Buildouts |
|
2 ========= |
|
3 |
|
4 The word "buildout" refers to a description of a set of parts and the |
|
5 software to create and assemble them. It is often used informally to |
|
6 refer to an installed system based on a buildout definition. For |
|
7 example, if we are creating an application named "Foo", then "the Foo |
|
8 buildout" is the collection of configuration and application-specific |
|
9 software that allows an instance of the application to be created. We |
|
10 may refer to such an instance of the application informally as "a Foo |
|
11 buildout". |
|
12 |
|
13 This document describes how to define buildouts using buildout |
|
14 configuration files and recipes. There are three ways to set up the |
|
15 buildout software and create a buildout instance: |
|
16 |
|
17 1. Install the zc.buildout egg with easy_install and use the buildout |
|
18 script installed in a Python scripts area. |
|
19 |
|
20 2. Use the buildout bootstrap script to create a buildout that |
|
21 includes both the setuptools and zc.buildout eggs. This allows you |
|
22 to use the buildout software without modifying a Python install. |
|
23 The buildout script is installed into your buildout local scripts |
|
24 area. |
|
25 |
|
26 3. Use a buildout command from an already installed buildout to |
|
27 bootstrap a new buildout. (See the section on bootstraping later |
|
28 in this document.) |
|
29 |
|
30 Often, a software project will be managed in a software repository, |
|
31 such as a subversion repository, that includes some software source |
|
32 directories, buildout configuration files, and a copy of the buildout |
|
33 bootstrap script. To work on the project, one would check out the |
|
34 project from the repository and run the bootstrap script which |
|
35 installs setuptools and zc.buildout into the checkout as well as any |
|
36 parts defined. |
|
37 |
|
38 We have a sample buildout that we created using the bootstrap command |
|
39 of an existing buildout (method 3 above). It has the absolute minimum |
|
40 information. We have bin, develop-eggs, eggs and parts directories, |
|
41 and a configuration file: |
|
42 |
|
43 >>> ls(sample_buildout) |
|
44 d bin |
|
45 - buildout.cfg |
|
46 d develop-eggs |
|
47 d eggs |
|
48 d parts |
|
49 |
|
50 The bin directory contains scripts. |
|
51 |
|
52 >>> ls(sample_buildout, 'bin') |
|
53 - buildout |
|
54 |
|
55 >>> ls(sample_buildout, 'eggs') |
|
56 - setuptools-0.6-py2.4.egg |
|
57 - zc.buildout-1.0-py2.4.egg |
|
58 |
|
59 The develop-eggs directory is initially empty: |
|
60 |
|
61 >>> ls(sample_buildout, 'develop-eggs') |
|
62 |
|
63 The develop-eggs directory holds egg links for software being |
|
64 developed in the buildout. We separate develop-eggs and other eggs to |
|
65 allow eggs directories to be shared across multiple buildouts. For |
|
66 example, a common developer technique is to define a common eggs |
|
67 directory in their home that all non-develop eggs are stored in. This |
|
68 allows larger buildouts to be set up much more quickly and saves disk |
|
69 space. |
|
70 |
|
71 The parts directory just contains some helpers for the buildout script |
|
72 itself. |
|
73 |
|
74 >>> ls(sample_buildout, 'parts') |
|
75 d buildout |
|
76 |
|
77 The parts directory provides an area where recipes can install |
|
78 part data. For example, if we built a custom Python, we would |
|
79 install it in the part directory. Part data is stored in a |
|
80 sub-directory of the parts directory with the same name as the part. |
|
81 |
|
82 Buildouts are defined using configuration files. These are in the |
|
83 format defined by the Python ConfigParser module, with extensions |
|
84 that we'll describe later. By default, when a buildout is run, it |
|
85 looks for the file buildout.cfg in the directory where the buildout is |
|
86 run. |
|
87 |
|
88 The minimal configuration file has a buildout section that defines no |
|
89 parts: |
|
90 |
|
91 >>> cat(sample_buildout, 'buildout.cfg') |
|
92 [buildout] |
|
93 parts = |
|
94 |
|
95 A part is simply something to be created by a buildout. It can be |
|
96 almost anything, such as a Python package, a program, a directory, or |
|
97 even a configuration file. |
|
98 |
|
99 Recipes |
|
100 ------- |
|
101 |
|
102 A part is created by a recipe. Recipes are always installed as Python |
|
103 eggs. They can be downloaded from a package server, such as the |
|
104 Python Package Index, or they can be developed as part of a project |
|
105 using a "develop" egg. |
|
106 |
|
107 A develop egg is a special kind of egg that gets installed as an "egg |
|
108 link" that contains the name of a source directory. Develop eggs |
|
109 don't have to be packaged for distribution to be used and can be |
|
110 modified in place, which is especially useful while they are being |
|
111 developed. |
|
112 |
|
113 Let's create a recipe as part of the sample project. We'll create a |
|
114 recipe for creating directories. First, we'll create a recipes source |
|
115 directory for our local recipes: |
|
116 |
|
117 >>> mkdir(sample_buildout, 'recipes') |
|
118 |
|
119 and then we'll create a source file for our mkdir recipe: |
|
120 |
|
121 >>> write(sample_buildout, 'recipes', 'mkdir.py', |
|
122 ... """ |
|
123 ... import logging, os, zc.buildout |
|
124 ... |
|
125 ... class Mkdir: |
|
126 ... |
|
127 ... def __init__(self, buildout, name, options): |
|
128 ... self.name, self.options = name, options |
|
129 ... options['path'] = os.path.join( |
|
130 ... buildout['buildout']['directory'], |
|
131 ... options['path'], |
|
132 ... ) |
|
133 ... if not os.path.isdir(os.path.dirname(options['path'])): |
|
134 ... logging.getLogger(self.name).error( |
|
135 ... 'Cannot create %s. %s is not a directory.', |
|
136 ... options['path'], os.path.dirname(options['path'])) |
|
137 ... raise zc.buildout.UserError('Invalid Path') |
|
138 ... |
|
139 ... |
|
140 ... def install(self): |
|
141 ... path = self.options['path'] |
|
142 ... logging.getLogger(self.name).info( |
|
143 ... 'Creating directory %s', os.path.basename(path)) |
|
144 ... os.mkdir(path) |
|
145 ... return path |
|
146 ... |
|
147 ... def update(self): |
|
148 ... pass |
|
149 ... """) |
|
150 |
|
151 Currently, recipes must define 3 methods [#future_recipe_methods]_: |
|
152 |
|
153 - a constructor, |
|
154 |
|
155 - an install method, and |
|
156 |
|
157 - an update method. |
|
158 |
|
159 The constructor is responsible for updating a parts options to reflect |
|
160 data read from other sections. The buildout system keeps track of |
|
161 whether a part specification has changed. A part specification has |
|
162 changed if it's options, after adjusting for data read from other |
|
163 sections, has changed, or if the recipe has changed. Only the options |
|
164 for the part are considered. If data are read from other sections, |
|
165 then that information has to be reflected in the parts options. In |
|
166 the Mkdir example, the given path is interpreted relative to the |
|
167 buildout directory, and data from the buildout directory is read. The |
|
168 path option is updated to reflect this. If the directory option was |
|
169 changed in the buildout sections, we would know to update parts |
|
170 created using the mkdir recipe using relative path names. |
|
171 |
|
172 When buildout is run, it saves configuration data for installed parts |
|
173 in a file named ".installed.cfg". In subsequent runs, it compares |
|
174 part-configuration data stored in the .installed.cfg file and the |
|
175 part-configuration data loaded from the configuration files as |
|
176 modified by recipe constructors to decide if the configuration of a |
|
177 part has changed. If the configuration has changed, or if the recipe |
|
178 has changed, then the part is uninstalled and reinstalled. The |
|
179 buildout only looks at the part's options, so any data used to |
|
180 configure the part needs to be reflected in the part's options. It is |
|
181 the job of a recipe constructor to make sure that the options include |
|
182 all relevant data. |
|
183 |
|
184 Of course, parts are also uninstalled if they are no-longer used. |
|
185 |
|
186 The recipe defines a constructor that takes a buildout object, a part |
|
187 name, and an options dictionary. It saves them in instance attributes. |
|
188 If the path is relative, we'll interpret it as relative to the |
|
189 buildout directory. The buildout object passed in is a mapping from |
|
190 section name to a mapping of options for that section. The buildout |
|
191 directory is available as the directory option of the buildout |
|
192 section. We normalize the path and save it back into the options |
|
193 directory. |
|
194 |
|
195 The install method is responsible for creating the part. In this |
|
196 case, we need the path of the directory to create. We'll use a path |
|
197 option from our options dictionary. The install method logs what it's |
|
198 doing using the Python logging call. We return the path that we |
|
199 installed. If the part is uninstalled or reinstalled, then the path |
|
200 returned will be removed by the buildout machinery. A recipe install |
|
201 method is expected to return a string, or an iterable of strings |
|
202 containing paths to be removed if a part is uninstalled. For most |
|
203 recipes, this is all of the uninstall support needed. For more complex |
|
204 uninstallation scenarios use `Uninstall recipes`_. |
|
205 |
|
206 The update method is responsible for updating an already installed |
|
207 part. An empty method is often provided, as in this example, if parts |
|
208 can't be updated. An update method can return None, a string, or an |
|
209 iterable of strings. If a string or iterable of strings is returned, |
|
210 then the saved list of paths to be uninstalled is updated with the new |
|
211 information by adding any new files returned by the update method. |
|
212 |
|
213 We need to provide packaging information so that our recipe can be |
|
214 installed as a develop egg. The minimum information we need to specify |
|
215 [#packaging_info]_ is a name. For recipes, we also need to define the |
|
216 names of the recipe classes as entry points. Packaging information is |
|
217 provided via a setup.py script: |
|
218 |
|
219 >>> write(sample_buildout, 'recipes', 'setup.py', |
|
220 ... """ |
|
221 ... from setuptools import setup |
|
222 ... |
|
223 ... setup( |
|
224 ... name = "recipes", |
|
225 ... entry_points = {'zc.buildout': ['mkdir = mkdir:Mkdir']}, |
|
226 ... ) |
|
227 ... """) |
|
228 |
|
229 Our setup script defines an entry point. Entry points provide |
|
230 a way for an egg to define the services it provides. Here we've said |
|
231 that we define a zc.buildout entry point named mkdir. Recipe |
|
232 classes must be exposed as entry points in the zc.buildout group. we |
|
233 give entry points names within the group. |
|
234 |
|
235 We also need a README.txt for our recipes to avoid an annoying warning |
|
236 from distutils, on which setuptools and zc.buildout are based: |
|
237 |
|
238 >>> write(sample_buildout, 'recipes', 'README.txt', " ") |
|
239 |
|
240 Now let's update our buildout.cfg: |
|
241 |
|
242 >>> write(sample_buildout, 'buildout.cfg', |
|
243 ... """ |
|
244 ... [buildout] |
|
245 ... develop = recipes |
|
246 ... parts = data-dir |
|
247 ... |
|
248 ... [data-dir] |
|
249 ... recipe = recipes:mkdir |
|
250 ... path = mystuff |
|
251 ... """) |
|
252 |
|
253 Let's go through the changes one by one:: |
|
254 |
|
255 develop = recipes |
|
256 |
|
257 This tells the buildout to install a development egg for our recipes. |
|
258 Any number of paths can be listed. The paths can be relative or |
|
259 absolute. If relative, they are treated as relative to the buildout |
|
260 directory. They can be directory or file paths. If a file path is |
|
261 given, it should point to a Python setup script. If a directory path |
|
262 is given, it should point to a directory containing a setup.py file. |
|
263 Development eggs are installed before building any parts, as they may |
|
264 provide locally-defined recipes needed by the parts. |
|
265 |
|
266 :: |
|
267 |
|
268 parts = data-dir |
|
269 |
|
270 Here we've named a part to be "built". We can use any name we want |
|
271 except that different part names must be unique and recipes will often |
|
272 use the part name to decide what to do. |
|
273 |
|
274 :: |
|
275 |
|
276 [data-dir] |
|
277 recipe = recipes:mkdir |
|
278 path = mystuff |
|
279 |
|
280 |
|
281 When we name a part, we also create a section of the same |
|
282 name that contains part data. In this section, we'll define |
|
283 the recipe to be used to install the part. In this case, we also |
|
284 specify the path to be created. |
|
285 |
|
286 Let's run the buildout. We do so by running the build script in the |
|
287 buildout: |
|
288 |
|
289 >>> import os |
|
290 >>> os.chdir(sample_buildout) |
|
291 >>> buildout = os.path.join(sample_buildout, 'bin', 'buildout') |
|
292 >>> print system(buildout), |
|
293 Develop: '/sample-buildout/recipes' |
|
294 Installing data-dir. |
|
295 data-dir: Creating directory mystuff |
|
296 |
|
297 We see that the recipe created the directory, as expected: |
|
298 |
|
299 >>> ls(sample_buildout) |
|
300 - .installed.cfg |
|
301 d bin |
|
302 - buildout.cfg |
|
303 d develop-eggs |
|
304 d eggs |
|
305 d mystuff |
|
306 d parts |
|
307 d recipes |
|
308 |
|
309 In addition, .installed.cfg has been created containing information |
|
310 about the part we installed: |
|
311 |
|
312 >>> cat(sample_buildout, '.installed.cfg') |
|
313 [buildout] |
|
314 installed_develop_eggs = /sample-buildout/develop-eggs/recipes.egg-link |
|
315 parts = data-dir |
|
316 <BLANKLINE> |
|
317 [data-dir] |
|
318 __buildout_installed__ = /sample-buildout/mystuff |
|
319 __buildout_signature__ = recipes-c7vHV6ekIDUPy/7fjAaYjg== |
|
320 path = /sample-buildout/mystuff |
|
321 recipe = recipes:mkdir |
|
322 |
|
323 Note that the directory we installed is included in .installed.cfg. |
|
324 In addition, the path option includes the actual destination |
|
325 directory. |
|
326 |
|
327 If we change the name of the directory in the configuration file, |
|
328 we'll see that the directory gets removed and recreated: |
|
329 |
|
330 >>> write(sample_buildout, 'buildout.cfg', |
|
331 ... """ |
|
332 ... [buildout] |
|
333 ... develop = recipes |
|
334 ... parts = data-dir |
|
335 ... |
|
336 ... [data-dir] |
|
337 ... recipe = recipes:mkdir |
|
338 ... path = mydata |
|
339 ... """) |
|
340 |
|
341 >>> print system(buildout), |
|
342 Develop: '/sample-buildout/recipes' |
|
343 Uninstalling data-dir. |
|
344 Installing data-dir. |
|
345 data-dir: Creating directory mydata |
|
346 |
|
347 >>> ls(sample_buildout) |
|
348 - .installed.cfg |
|
349 d bin |
|
350 - buildout.cfg |
|
351 d develop-eggs |
|
352 d eggs |
|
353 d mydata |
|
354 d parts |
|
355 d recipes |
|
356 |
|
357 If any of the files or directories created by a recipe are removed, |
|
358 the part will be reinstalled: |
|
359 |
|
360 >>> rmdir(sample_buildout, 'mydata') |
|
361 >>> print system(buildout), |
|
362 Develop: '/sample-buildout/recipes' |
|
363 Uninstalling data-dir. |
|
364 Installing data-dir. |
|
365 data-dir: Creating directory mydata |
|
366 |
|
367 Error reporting |
|
368 --------------- |
|
369 |
|
370 If a user makes an error, an error needs to be printed and work needs |
|
371 to stop. This is accomplished by logging a detailed error message and |
|
372 then raising a (or an instance of a subclass of a) |
|
373 zc.buildout.UserError exception. Raising an error other than a |
|
374 UserError still displays the error, but labels it as a bug in the |
|
375 buildout software or recipe. In the sample above, of someone gives a |
|
376 non-existent directory to create the directory in: |
|
377 |
|
378 |
|
379 >>> write(sample_buildout, 'buildout.cfg', |
|
380 ... """ |
|
381 ... [buildout] |
|
382 ... develop = recipes |
|
383 ... parts = data-dir |
|
384 ... |
|
385 ... [data-dir] |
|
386 ... recipe = recipes:mkdir |
|
387 ... path = /xxx/mydata |
|
388 ... """) |
|
389 |
|
390 We'll get a user error, not a traceback. |
|
391 |
|
392 >>> print system(buildout), |
|
393 Develop: '/sample-buildout/recipes' |
|
394 data-dir: Cannot create /xxx/mydata. /xxx is not a directory. |
|
395 While: |
|
396 Installing. |
|
397 Getting section data-dir. |
|
398 Initializing part data-dir. |
|
399 Error: Invalid Path |
|
400 |
|
401 |
|
402 Recipe Error Handling |
|
403 --------------------- |
|
404 |
|
405 If an error occurs during installation, it is up to the recipe to |
|
406 clean up any system side effects, such as files created. Let's update |
|
407 the mkdir recipe to support multiple paths: |
|
408 |
|
409 >>> write(sample_buildout, 'recipes', 'mkdir.py', |
|
410 ... """ |
|
411 ... import logging, os, zc.buildout |
|
412 ... |
|
413 ... class Mkdir: |
|
414 ... |
|
415 ... def __init__(self, buildout, name, options): |
|
416 ... self.name, self.options = name, options |
|
417 ... |
|
418 ... # Normalize paths and check that their parent |
|
419 ... # directories exist: |
|
420 ... paths = [] |
|
421 ... for path in options['path'].split(): |
|
422 ... path = os.path.join(buildout['buildout']['directory'], path) |
|
423 ... if not os.path.isdir(os.path.dirname(path)): |
|
424 ... logging.getLogger(self.name).error( |
|
425 ... 'Cannot create %s. %s is not a directory.', |
|
426 ... options['path'], os.path.dirname(options['path'])) |
|
427 ... raise zc.buildout.UserError('Invalid Path') |
|
428 ... paths.append(path) |
|
429 ... options['path'] = ' '.join(paths) |
|
430 ... |
|
431 ... def install(self): |
|
432 ... paths = self.options['path'].split() |
|
433 ... for path in paths: |
|
434 ... logging.getLogger(self.name).info( |
|
435 ... 'Creating directory %s', os.path.basename(path)) |
|
436 ... os.mkdir(path) |
|
437 ... return paths |
|
438 ... |
|
439 ... def update(self): |
|
440 ... pass |
|
441 ... """) |
|
442 |
|
443 If there is an error creating a path, the install method will exit and |
|
444 leave previously created paths in place: |
|
445 |
|
446 >>> write(sample_buildout, 'buildout.cfg', |
|
447 ... """ |
|
448 ... [buildout] |
|
449 ... develop = recipes |
|
450 ... parts = data-dir |
|
451 ... |
|
452 ... [data-dir] |
|
453 ... recipe = recipes:mkdir |
|
454 ... path = foo bin |
|
455 ... """) |
|
456 |
|
457 >>> print system(buildout), # doctest: +ELLIPSIS |
|
458 Develop: '/sample-buildout/recipes' |
|
459 Uninstalling data-dir. |
|
460 Installing data-dir. |
|
461 data-dir: Creating directory foo |
|
462 data-dir: Creating directory bin |
|
463 While: |
|
464 Installing data-dir. |
|
465 <BLANKLINE> |
|
466 An internal error occurred due to a bug in either zc.buildout or in a |
|
467 recipe being used: |
|
468 Traceback (most recent call last): |
|
469 ... |
|
470 OSError: [Errno 17] File exists: '/sample-buildout/bin' |
|
471 |
|
472 We meant to create a directory bins, but typed bin. Now foo was |
|
473 left behind. |
|
474 |
|
475 >>> os.path.exists('foo') |
|
476 True |
|
477 |
|
478 If we fix the typo: |
|
479 |
|
480 >>> write(sample_buildout, 'buildout.cfg', |
|
481 ... """ |
|
482 ... [buildout] |
|
483 ... develop = recipes |
|
484 ... parts = data-dir |
|
485 ... |
|
486 ... [data-dir] |
|
487 ... recipe = recipes:mkdir |
|
488 ... path = foo bins |
|
489 ... """) |
|
490 |
|
491 >>> print system(buildout), # doctest: +ELLIPSIS |
|
492 Develop: '/sample-buildout/recipes' |
|
493 Installing data-dir. |
|
494 data-dir: Creating directory foo |
|
495 While: |
|
496 Installing data-dir. |
|
497 <BLANKLINE> |
|
498 An internal error occurred due to a bug in either zc.buildout or in a |
|
499 recipe being used: |
|
500 Traceback (most recent call last): |
|
501 ... |
|
502 OSError: [Errno 17] File exists: '/sample-buildout/foo' |
|
503 |
|
504 Now they fail because foo exists, because it was left behind. |
|
505 |
|
506 >>> remove('foo') |
|
507 |
|
508 Let's fix the recipe: |
|
509 |
|
510 >>> write(sample_buildout, 'recipes', 'mkdir.py', |
|
511 ... """ |
|
512 ... import logging, os, zc.buildout |
|
513 ... |
|
514 ... class Mkdir: |
|
515 ... |
|
516 ... def __init__(self, buildout, name, options): |
|
517 ... self.name, self.options = name, options |
|
518 ... |
|
519 ... # Normalize paths and check that their parent |
|
520 ... # directories exist: |
|
521 ... paths = [] |
|
522 ... for path in options['path'].split(): |
|
523 ... path = os.path.join(buildout['buildout']['directory'], path) |
|
524 ... if not os.path.isdir(os.path.dirname(path)): |
|
525 ... logging.getLogger(self.name).error( |
|
526 ... 'Cannot create %s. %s is not a directory.', |
|
527 ... options['path'], os.path.dirname(options['path'])) |
|
528 ... raise zc.buildout.UserError('Invalid Path') |
|
529 ... paths.append(path) |
|
530 ... options['path'] = ' '.join(paths) |
|
531 ... |
|
532 ... def install(self): |
|
533 ... paths = self.options['path'].split() |
|
534 ... created = [] |
|
535 ... try: |
|
536 ... for path in paths: |
|
537 ... logging.getLogger(self.name).info( |
|
538 ... 'Creating directory %s', os.path.basename(path)) |
|
539 ... os.mkdir(path) |
|
540 ... created.append(path) |
|
541 ... except: |
|
542 ... for d in created: |
|
543 ... os.rmdir(d) |
|
544 ... raise |
|
545 ... |
|
546 ... return paths |
|
547 ... |
|
548 ... def update(self): |
|
549 ... pass |
|
550 ... """) |
|
551 |
|
552 And put back the typo: |
|
553 |
|
554 >>> write(sample_buildout, 'buildout.cfg', |
|
555 ... """ |
|
556 ... [buildout] |
|
557 ... develop = recipes |
|
558 ... parts = data-dir |
|
559 ... |
|
560 ... [data-dir] |
|
561 ... recipe = recipes:mkdir |
|
562 ... path = foo bin |
|
563 ... """) |
|
564 |
|
565 When we rerun the buildout: |
|
566 |
|
567 >>> print system(buildout), # doctest: +ELLIPSIS |
|
568 Develop: '/sample-buildout/recipes' |
|
569 Installing data-dir. |
|
570 data-dir: Creating directory foo |
|
571 data-dir: Creating directory bin |
|
572 While: |
|
573 Installing data-dir. |
|
574 <BLANKLINE> |
|
575 An internal error occurred due to a bug in either zc.buildout or in a |
|
576 recipe being used: |
|
577 Traceback (most recent call last): |
|
578 ... |
|
579 OSError: [Errno 17] File exists: '/sample-buildout/bin' |
|
580 |
|
581 .. Wait for the file to really disappear. My linux is weird. |
|
582 |
|
583 >>> wait_until("foo goes away", lambda : not os.path.exists('foo'), |
|
584 ... timeout=200) |
|
585 |
|
586 we get the same error, but we don't get the directory left behind: |
|
587 |
|
588 >>> os.path.exists('foo') |
|
589 False |
|
590 |
|
591 It's critical that recipes clean up partial effects when errors |
|
592 occur. Because recipes most commonly create files and directories, |
|
593 buildout provides a helper API for removing created files when an |
|
594 error occurs. Option objects have a created method that can be called |
|
595 to record files as they are created. If the install or update method |
|
596 returns with an error, then any registered paths are removed |
|
597 automatically. The method returns the files registered and can be |
|
598 used to return the files created. Let's use this API to simplify the |
|
599 recipe: |
|
600 |
|
601 >>> write(sample_buildout, 'recipes', 'mkdir.py', |
|
602 ... """ |
|
603 ... import logging, os, zc.buildout |
|
604 ... |
|
605 ... class Mkdir: |
|
606 ... |
|
607 ... def __init__(self, buildout, name, options): |
|
608 ... self.name, self.options = name, options |
|
609 ... |
|
610 ... # Normalize paths and check that their parent |
|
611 ... # directories exist: |
|
612 ... paths = [] |
|
613 ... for path in options['path'].split(): |
|
614 ... path = os.path.join(buildout['buildout']['directory'], path) |
|
615 ... if not os.path.isdir(os.path.dirname(path)): |
|
616 ... logging.getLogger(self.name).error( |
|
617 ... 'Cannot create %s. %s is not a directory.', |
|
618 ... options['path'], os.path.dirname(options['path'])) |
|
619 ... raise zc.buildout.UserError('Invalid Path') |
|
620 ... paths.append(path) |
|
621 ... options['path'] = ' '.join(paths) |
|
622 ... |
|
623 ... def install(self): |
|
624 ... paths = self.options['path'].split() |
|
625 ... for path in paths: |
|
626 ... logging.getLogger(self.name).info( |
|
627 ... 'Creating directory %s', os.path.basename(path)) |
|
628 ... os.mkdir(path) |
|
629 ... self.options.created(path) |
|
630 ... |
|
631 ... return self.options.created() |
|
632 ... |
|
633 ... def update(self): |
|
634 ... pass |
|
635 ... """) |
|
636 |
|
637 .. |
|
638 |
|
639 >>> remove(sample_buildout, 'recipes', 'mkdir.pyc') |
|
640 |
|
641 We returned by calling created, taking advantage of the fact that it |
|
642 returns the registered paths. We did this for illustrative purposes. |
|
643 It would be simpler just to return the paths as before. |
|
644 |
|
645 If we rerun the buildout, again, we'll get the error and no |
|
646 directories will be created: |
|
647 |
|
648 >>> print system(buildout), # doctest: +ELLIPSIS |
|
649 Develop: '/sample-buildout/recipes' |
|
650 Installing data-dir. |
|
651 data-dir: Creating directory foo |
|
652 data-dir: Creating directory bin |
|
653 While: |
|
654 Installing data-dir. |
|
655 <BLANKLINE> |
|
656 An internal error occurred due to a bug in either zc.buildout or in a |
|
657 recipe being used: |
|
658 Traceback (most recent call last): |
|
659 ... |
|
660 OSError: [Errno 17] File exists: '/sample-buildout/bin' |
|
661 |
|
662 >>> os.path.exists('foo') |
|
663 False |
|
664 |
|
665 Now, we'll fix the typo again and we'll get the directories we expect: |
|
666 |
|
667 >>> write(sample_buildout, 'buildout.cfg', |
|
668 ... """ |
|
669 ... [buildout] |
|
670 ... develop = recipes |
|
671 ... parts = data-dir |
|
672 ... |
|
673 ... [data-dir] |
|
674 ... recipe = recipes:mkdir |
|
675 ... path = foo bins |
|
676 ... """) |
|
677 |
|
678 >>> print system(buildout), |
|
679 Develop: '/sample-buildout/recipes' |
|
680 Installing data-dir. |
|
681 data-dir: Creating directory foo |
|
682 data-dir: Creating directory bins |
|
683 |
|
684 >>> os.path.exists('foo') |
|
685 True |
|
686 >>> os.path.exists('bins') |
|
687 True |
|
688 |
|
689 Configuration file syntax |
|
690 ------------------------- |
|
691 |
|
692 As mentioned earlier, buildout configuration files use the format |
|
693 defined by the Python ConfigParser module with extensions. The |
|
694 extensions are: |
|
695 |
|
696 - option names are case sensitive |
|
697 |
|
698 - option values can use a substitution syntax, described below, to |
|
699 refer to option values in specific sections. |
|
700 |
|
701 - option values can be appended or removed using the - and + |
|
702 operators. |
|
703 |
|
704 The ConfigParser syntax is very flexible. Section names can contain |
|
705 any characters other than newlines and right square braces ("]"). |
|
706 Option names can contain any characters other than newlines, colons, |
|
707 and equal signs, can not start with a space, and don't include |
|
708 trailing spaces. |
|
709 |
|
710 It is likely that, in the future, some characters will be given |
|
711 special buildout-defined meanings. This is already true of the |
|
712 characters ":", "$", "%", "(", and ")". For now, it is a good idea to |
|
713 keep section and option names simple, sticking to alphanumeric |
|
714 characters, hyphens, and periods. |
|
715 |
|
716 Annotated sections |
|
717 ------------------ |
|
718 |
|
719 When used with the `annotate` command, buildout displays annotated sections. |
|
720 All sections are displayed, sorted alphabetically. For each section, |
|
721 all key-value pairs are displayed, sorted alphabetically, along with |
|
722 the origin of the value (file name or COMPUTED_VALUE, DEFAULT_VALUE, |
|
723 COMMAND_LINE_VALUE). |
|
724 |
|
725 >>> print system(buildout+ ' annotate'), |
|
726 ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE |
|
727 <BLANKLINE> |
|
728 Annotated sections |
|
729 ================== |
|
730 <BLANKLINE> |
|
731 [buildout] |
|
732 accept-buildout-test-releases= false |
|
733 DEFAULT_VALUE |
|
734 allow-hosts= * |
|
735 DEFAULT_VALUE |
|
736 allow-picked-versions= true |
|
737 DEFAULT_VALUE |
|
738 allowed-eggs-from-site-packages= * |
|
739 DEFAULT_VALUE |
|
740 bin-directory= bin |
|
741 DEFAULT_VALUE |
|
742 develop= recipes |
|
743 /sample-buildout/buildout.cfg |
|
744 develop-eggs-directory= develop-eggs |
|
745 DEFAULT_VALUE |
|
746 directory= /sample-buildout |
|
747 COMPUTED_VALUE |
|
748 eggs-directory= eggs |
|
749 DEFAULT_VALUE |
|
750 exec-sitecustomize= true |
|
751 DEFAULT_VALUE |
|
752 executable= ... |
|
753 DEFAULT_VALUE |
|
754 find-links= |
|
755 DEFAULT_VALUE |
|
756 include-site-packages= true |
|
757 DEFAULT_VALUE |
|
758 install-from-cache= false |
|
759 DEFAULT_VALUE |
|
760 installed= .installed.cfg |
|
761 DEFAULT_VALUE |
|
762 log-format= |
|
763 DEFAULT_VALUE |
|
764 log-level= INFO |
|
765 DEFAULT_VALUE |
|
766 newest= true |
|
767 DEFAULT_VALUE |
|
768 offline= false |
|
769 DEFAULT_VALUE |
|
770 parts= data-dir |
|
771 /sample-buildout/buildout.cfg |
|
772 parts-directory= parts |
|
773 DEFAULT_VALUE |
|
774 prefer-final= false |
|
775 DEFAULT_VALUE |
|
776 python= buildout |
|
777 DEFAULT_VALUE |
|
778 relative-paths= false |
|
779 DEFAULT_VALUE |
|
780 socket-timeout= |
|
781 DEFAULT_VALUE |
|
782 unzip= false |
|
783 DEFAULT_VALUE |
|
784 use-dependency-links= true |
|
785 DEFAULT_VALUE |
|
786 <BLANKLINE> |
|
787 [data-dir] |
|
788 path= foo bins |
|
789 /sample-buildout/buildout.cfg |
|
790 recipe= recipes:mkdir |
|
791 /sample-buildout/buildout.cfg |
|
792 <BLANKLINE> |
|
793 |
|
794 Variable substitutions |
|
795 ---------------------- |
|
796 |
|
797 Buildout configuration files support variable substitution. |
|
798 To illustrate this, we'll create an debug recipe to |
|
799 allow us to see interactions with the buildout: |
|
800 |
|
801 >>> write(sample_buildout, 'recipes', 'debug.py', |
|
802 ... """ |
|
803 ... class Debug: |
|
804 ... |
|
805 ... def __init__(self, buildout, name, options): |
|
806 ... self.buildout = buildout |
|
807 ... self.name = name |
|
808 ... self.options = options |
|
809 ... |
|
810 ... def install(self): |
|
811 ... items = self.options.items() |
|
812 ... items.sort() |
|
813 ... for option, value in items: |
|
814 ... print option, value |
|
815 ... return () |
|
816 ... |
|
817 ... update = install |
|
818 ... """) |
|
819 |
|
820 This recipe doesn't actually create anything. The install method |
|
821 doesn't return anything, because it didn't create any files or |
|
822 directories. |
|
823 |
|
824 We also have to update our setup script: |
|
825 |
|
826 >>> write(sample_buildout, 'recipes', 'setup.py', |
|
827 ... """ |
|
828 ... from setuptools import setup |
|
829 ... entry_points = ( |
|
830 ... ''' |
|
831 ... [zc.buildout] |
|
832 ... mkdir = mkdir:Mkdir |
|
833 ... debug = debug:Debug |
|
834 ... ''') |
|
835 ... setup(name="recipes", entry_points=entry_points) |
|
836 ... """) |
|
837 |
|
838 We've rearranged the script a bit to make the entry points easier to |
|
839 edit. In particular, entry points are now defined as a configuration |
|
840 string, rather than a dictionary. |
|
841 |
|
842 Let's update our configuration to provide variable substitution |
|
843 examples: |
|
844 |
|
845 >>> write(sample_buildout, 'buildout.cfg', |
|
846 ... """ |
|
847 ... [buildout] |
|
848 ... develop = recipes |
|
849 ... parts = data-dir debug |
|
850 ... log-level = INFO |
|
851 ... |
|
852 ... [debug] |
|
853 ... recipe = recipes:debug |
|
854 ... File 1 = ${data-dir:path}/file |
|
855 ... File 2 = ${debug:File 1}/log |
|
856 ... |
|
857 ... [data-dir] |
|
858 ... recipe = recipes:mkdir |
|
859 ... path = mydata |
|
860 ... """) |
|
861 |
|
862 We used a string-template substitution for File 1 and File 2. This |
|
863 type of substitution uses the string.Template syntax. Names |
|
864 substituted are qualified option names, consisting of a section name |
|
865 and option name joined by a colon. |
|
866 |
|
867 Now, if we run the buildout, we'll see the options with the values |
|
868 substituted. |
|
869 |
|
870 >>> print system(buildout), |
|
871 Develop: '/sample-buildout/recipes' |
|
872 Uninstalling data-dir. |
|
873 Installing data-dir. |
|
874 data-dir: Creating directory mydata |
|
875 Installing debug. |
|
876 File 1 /sample-buildout/mydata/file |
|
877 File 2 /sample-buildout/mydata/file/log |
|
878 recipe recipes:debug |
|
879 |
|
880 Note that the substitution of the data-dir path option reflects the |
|
881 update to the option performed by the mkdir recipe. |
|
882 |
|
883 It might seem surprising that mydata was created again. This is |
|
884 because we changed our recipes package by adding the debug module. |
|
885 The buildout system didn't know if this module could effect the mkdir |
|
886 recipe, so it assumed it could and reinstalled mydata. If we rerun |
|
887 the buildout: |
|
888 |
|
889 >>> print system(buildout), |
|
890 Develop: '/sample-buildout/recipes' |
|
891 Updating data-dir. |
|
892 Updating debug. |
|
893 File 1 /sample-buildout/mydata/file |
|
894 File 2 /sample-buildout/mydata/file/log |
|
895 recipe recipes:debug |
|
896 |
|
897 We can see that mydata was not recreated. |
|
898 |
|
899 Note that, in this case, we didn't specify a log level, so |
|
900 we didn't get output about what the buildout was doing. |
|
901 |
|
902 Section and option names in variable substitutions are only allowed to |
|
903 contain alphanumeric characters, hyphens, periods and spaces. This |
|
904 restriction might be relaxed in future releases. |
|
905 |
|
906 We can ommit the section name in a variable substitution to refer to |
|
907 the current section. We can also use the special option, |
|
908 _buildout_section_name_ to get the current section name. |
|
909 |
|
910 >>> write(sample_buildout, 'buildout.cfg', |
|
911 ... """ |
|
912 ... [buildout] |
|
913 ... develop = recipes |
|
914 ... parts = data-dir debug |
|
915 ... log-level = INFO |
|
916 ... |
|
917 ... [debug] |
|
918 ... recipe = recipes:debug |
|
919 ... File 1 = ${data-dir:path}/file |
|
920 ... File 2 = ${:File 1}/log |
|
921 ... my_name = ${:_buildout_section_name_} |
|
922 ... |
|
923 ... [data-dir] |
|
924 ... recipe = recipes:mkdir |
|
925 ... path = mydata |
|
926 ... """) |
|
927 |
|
928 >>> print system(buildout), |
|
929 Develop: '/sample-buildout/recipes' |
|
930 Uninstalling debug. |
|
931 Updating data-dir. |
|
932 Installing debug. |
|
933 File 1 /sample-buildout/mydata/file |
|
934 File 2 /sample-buildout/mydata/file/log |
|
935 my_name debug |
|
936 recipe recipes:debug |
|
937 |
|
938 Automatic part selection and ordering |
|
939 ------------------------------------- |
|
940 |
|
941 When a section with a recipe is referred to, either through variable |
|
942 substitution or by an initializing recipe, the section is treated as a |
|
943 part and added to the part list before the referencing part. For |
|
944 example, we can leave data-dir out of the parts list: |
|
945 |
|
946 >>> write(sample_buildout, 'buildout.cfg', |
|
947 ... """ |
|
948 ... [buildout] |
|
949 ... develop = recipes |
|
950 ... parts = debug |
|
951 ... log-level = INFO |
|
952 ... |
|
953 ... [debug] |
|
954 ... recipe = recipes:debug |
|
955 ... File 1 = ${data-dir:path}/file |
|
956 ... File 2 = ${debug:File 1}/log |
|
957 ... |
|
958 ... [data-dir] |
|
959 ... recipe = recipes:mkdir |
|
960 ... path = mydata |
|
961 ... """) |
|
962 |
|
963 |
|
964 It will still be treated as a part: |
|
965 |
|
966 >>> print system(buildout), |
|
967 Develop: '/sample-buildout/recipes' |
|
968 Uninstalling debug. |
|
969 Updating data-dir. |
|
970 Installing debug. |
|
971 File 1 /sample-buildout/mydata/file |
|
972 File 2 /sample-buildout/mydata/file/log |
|
973 recipe recipes:debug |
|
974 |
|
975 >>> cat('.installed.cfg') # doctest: +ELLIPSIS |
|
976 [buildout] |
|
977 installed_develop_eggs = /sample-buildout/develop-eggs/recipes.egg-link |
|
978 parts = data-dir debug |
|
979 ... |
|
980 |
|
981 Note that the data-dir part is included *before* the debug part, |
|
982 because the debug part refers to the data-dir part. Even if we list |
|
983 the data-dir part after the debug part, it will be included before: |
|
984 |
|
985 >>> write(sample_buildout, 'buildout.cfg', |
|
986 ... """ |
|
987 ... [buildout] |
|
988 ... develop = recipes |
|
989 ... parts = debug data-dir |
|
990 ... log-level = INFO |
|
991 ... |
|
992 ... [debug] |
|
993 ... recipe = recipes:debug |
|
994 ... File 1 = ${data-dir:path}/file |
|
995 ... File 2 = ${debug:File 1}/log |
|
996 ... |
|
997 ... [data-dir] |
|
998 ... recipe = recipes:mkdir |
|
999 ... path = mydata |
|
1000 ... """) |
|
1001 |
|
1002 |
|
1003 It will still be treated as a part: |
|
1004 |
|
1005 >>> print system(buildout), |
|
1006 Develop: '/sample-buildout/recipes' |
|
1007 Updating data-dir. |
|
1008 Updating debug. |
|
1009 File 1 /sample-buildout/mydata/file |
|
1010 File 2 /sample-buildout/mydata/file/log |
|
1011 recipe recipes:debug |
|
1012 |
|
1013 >>> cat('.installed.cfg') # doctest: +ELLIPSIS |
|
1014 [buildout] |
|
1015 installed_develop_eggs = /sample-buildout/develop-eggs/recipes.egg-link |
|
1016 parts = data-dir debug |
|
1017 ... |
|
1018 |
|
1019 Extending sections (macros) |
|
1020 --------------------------- |
|
1021 |
|
1022 A section (other than the buildout section) can extend one or more |
|
1023 other sections using the ``<=`` option. Options from the referenced |
|
1024 sections are copied to the refering section *before* variable |
|
1025 substitution. This, together with the ability to refer to variables |
|
1026 of the current section allows sections to be used as macros. |
|
1027 |
|
1028 >>> write(sample_buildout, 'buildout.cfg', |
|
1029 ... """ |
|
1030 ... [buildout] |
|
1031 ... develop = recipes |
|
1032 ... parts = myfiles |
|
1033 ... log-level = INFO |
|
1034 ... |
|
1035 ... [debug] |
|
1036 ... recipe = recipes:debug |
|
1037 ... |
|
1038 ... [with_file1] |
|
1039 ... <= debug |
|
1040 ... file1 = ${:path}/file1 |
|
1041 ... color = red |
|
1042 ... |
|
1043 ... [with_file2] |
|
1044 ... <= debug |
|
1045 ... file2 = ${:path}/file2 |
|
1046 ... color = blue |
|
1047 ... |
|
1048 ... [myfiles] |
|
1049 ... <= with_file1 |
|
1050 ... with_file2 |
|
1051 ... path = mydata |
|
1052 ... """) |
|
1053 |
|
1054 >>> print system(buildout), |
|
1055 Develop: '/sample-buildout/recipes' |
|
1056 Uninstalling debug. |
|
1057 Uninstalling data-dir. |
|
1058 Installing myfiles. |
|
1059 color blue |
|
1060 file1 mydata/file1 |
|
1061 file2 mydata/file2 |
|
1062 path mydata |
|
1063 recipe recipes:debug |
|
1064 |
|
1065 In this example, the debug, with_file1 and with_file2 sections act as |
|
1066 macros. In particular, the variable substitutions are performed |
|
1067 relative to the myfiles section. |
|
1068 |
|
1069 Adding and removing options |
|
1070 --------------------------- |
|
1071 |
|
1072 We can append and remove values to an option by using the + and - |
|
1073 operators. |
|
1074 |
|
1075 This is illustrated below; first we define a base configuration. |
|
1076 |
|
1077 >>> write(sample_buildout, 'base.cfg', |
|
1078 ... """ |
|
1079 ... [buildout] |
|
1080 ... parts = part1 part2 part3 |
|
1081 ... |
|
1082 ... [part1] |
|
1083 ... recipe = |
|
1084 ... option = a1 a2 |
|
1085 ... |
|
1086 ... [part2] |
|
1087 ... recipe = |
|
1088 ... option = b1 b2 b3 b4 |
|
1089 ... |
|
1090 ... [part3] |
|
1091 ... recipe = |
|
1092 ... option = c1 c2 |
|
1093 ... |
|
1094 ... """) |
|
1095 |
|
1096 Extending this configuration, we can "adjust" the values set in the |
|
1097 base configuration file. |
|
1098 |
|
1099 >>> write(sample_buildout, 'extension1.cfg', |
|
1100 ... """ |
|
1101 ... [buildout] |
|
1102 ... extends = base.cfg |
|
1103 ... |
|
1104 ... # appending values |
|
1105 ... [part1] |
|
1106 ... option += a3 a4 |
|
1107 ... |
|
1108 ... # removing values |
|
1109 ... [part2] |
|
1110 ... option -= b1 b2 |
|
1111 ... |
|
1112 ... # alt. spelling |
|
1113 ... [part3] |
|
1114 ... option+=c3 c4 c5 |
|
1115 ... |
|
1116 ... # normal assignment |
|
1117 ... [part4] |
|
1118 ... option = h1 h2 |
|
1119 ... |
|
1120 ... """) |
|
1121 |
|
1122 An additional extension. |
|
1123 |
|
1124 >>> write(sample_buildout, 'extension2.cfg', |
|
1125 ... """ |
|
1126 ... [buildout] |
|
1127 ... extends = extension1.cfg |
|
1128 ... |
|
1129 ... # appending values |
|
1130 ... [part1] |
|
1131 ... option += a5 |
|
1132 ... |
|
1133 ... # removing values |
|
1134 ... [part2] |
|
1135 ... option -= b1 b2 b3 |
|
1136 ... |
|
1137 ... """) |
|
1138 |
|
1139 To verify that the options are adjusted correctly, we'll set up an |
|
1140 extension that prints out the options. |
|
1141 |
|
1142 >>> mkdir(sample_buildout, 'demo') |
|
1143 >>> write(sample_buildout, 'demo', 'demo.py', |
|
1144 ... """ |
|
1145 ... def ext(buildout): |
|
1146 ... print [part['option'] for name, part in buildout.items() \ |
|
1147 ... if name.startswith('part')] |
|
1148 ... """) |
|
1149 |
|
1150 >>> write(sample_buildout, 'demo', 'setup.py', |
|
1151 ... """ |
|
1152 ... from setuptools import setup |
|
1153 ... |
|
1154 ... setup( |
|
1155 ... name="demo", |
|
1156 ... entry_points={'zc.buildout.extension': ['ext = demo:ext']}, |
|
1157 ... ) |
|
1158 ... """) |
|
1159 |
|
1160 Set up a buildout configuration for this extension. |
|
1161 |
|
1162 >>> write(sample_buildout, 'buildout.cfg', |
|
1163 ... """ |
|
1164 ... [buildout] |
|
1165 ... develop = demo |
|
1166 ... parts = |
|
1167 ... """) |
|
1168 |
|
1169 >>> os.chdir(sample_buildout) |
|
1170 >>> print system(os.path.join(sample_buildout, 'bin', 'buildout')), |
|
1171 Develop: '/sample-buildout/demo' |
|
1172 Uninstalling myfiles. |
|
1173 Getting distribution for 'recipes'. |
|
1174 zip_safe flag not set; analyzing archive contents... |
|
1175 Got recipes 0.0.0. |
|
1176 warning: install_lib: 'build/lib' does not exist -- no Python modules to install |
|
1177 |
|
1178 Verify option values. |
|
1179 |
|
1180 >>> write(sample_buildout, 'buildout.cfg', |
|
1181 ... """ |
|
1182 ... [buildout] |
|
1183 ... develop = demo |
|
1184 ... extensions = demo |
|
1185 ... extends = extension2.cfg |
|
1186 ... """) |
|
1187 |
|
1188 >>> print system(os.path.join('bin', 'buildout')), |
|
1189 ['a1 a2/na3 a4/na5', 'b1 b2 b3 b4', 'c1 c2/nc3 c4 c5', 'h1 h2'] |
|
1190 Develop: '/sample-buildout/demo' |
|
1191 |
|
1192 Annotated sections output shows which files are responsible for which |
|
1193 operations. |
|
1194 |
|
1195 >>> print system(os.path.join('bin', 'buildout') + ' annotate'), |
|
1196 ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE |
|
1197 <BLANKLINE> |
|
1198 Annotated sections |
|
1199 ================== |
|
1200 ... |
|
1201 <BLANKLINE> |
|
1202 [part1] |
|
1203 option= a1 a2 |
|
1204 a3 a4 |
|
1205 a5 |
|
1206 /sample-buildout/base.cfg |
|
1207 += /sample-buildout/extension1.cfg |
|
1208 += /sample-buildout/extension2.cfg |
|
1209 recipe= |
|
1210 /sample-buildout/base.cfg |
|
1211 <BLANKLINE> |
|
1212 [part2] |
|
1213 option= b1 b2 b3 b4 |
|
1214 /sample-buildout/base.cfg |
|
1215 -= /sample-buildout/extension1.cfg |
|
1216 -= /sample-buildout/extension2.cfg |
|
1217 recipe= |
|
1218 /sample-buildout/base.cfg |
|
1219 <BLANKLINE> |
|
1220 [part3] |
|
1221 option= c1 c2 |
|
1222 c3 c4 c5 |
|
1223 /sample-buildout/base.cfg |
|
1224 += /sample-buildout/extension1.cfg |
|
1225 recipe= |
|
1226 /sample-buildout/base.cfg |
|
1227 <BLANKLINE> |
|
1228 [part4] |
|
1229 option= h1 h2 |
|
1230 /sample-buildout/extension1.cfg |
|
1231 |
|
1232 Cleanup. |
|
1233 |
|
1234 >>> os.remove(os.path.join(sample_buildout, 'base.cfg')) |
|
1235 >>> os.remove(os.path.join(sample_buildout, 'extension1.cfg')) |
|
1236 >>> os.remove(os.path.join(sample_buildout, 'extension2.cfg')) |
|
1237 |
|
1238 Multiple configuration files |
|
1239 ---------------------------- |
|
1240 |
|
1241 A configuration file can "extend" another configuration file. |
|
1242 Options are read from the other configuration file if they aren't |
|
1243 already defined by your configuration file. |
|
1244 |
|
1245 The configuration files your file extends can extend |
|
1246 other configuration files. The same file may be |
|
1247 used more than once although, of course, cycles aren't allowed. |
|
1248 |
|
1249 To see how this works, we use an example: |
|
1250 |
|
1251 >>> write(sample_buildout, 'buildout.cfg', |
|
1252 ... """ |
|
1253 ... [buildout] |
|
1254 ... extends = base.cfg |
|
1255 ... |
|
1256 ... [debug] |
|
1257 ... op = buildout |
|
1258 ... """) |
|
1259 |
|
1260 >>> write(sample_buildout, 'base.cfg', |
|
1261 ... """ |
|
1262 ... [buildout] |
|
1263 ... develop = recipes |
|
1264 ... parts = debug |
|
1265 ... |
|
1266 ... [debug] |
|
1267 ... recipe = recipes:debug |
|
1268 ... op = base |
|
1269 ... """) |
|
1270 |
|
1271 >>> print system(buildout), |
|
1272 Develop: '/sample-buildout/recipes' |
|
1273 Installing debug. |
|
1274 op buildout |
|
1275 recipe recipes:debug |
|
1276 |
|
1277 The example is pretty trivial, but the pattern it illustrates is |
|
1278 pretty common. In a more practical example, the base buildout might |
|
1279 represent a product and the extending buildout might be a |
|
1280 customization. |
|
1281 |
|
1282 Here is a more elaborate example. |
|
1283 |
|
1284 >>> other = tmpdir('other') |
|
1285 |
|
1286 >>> write(sample_buildout, 'buildout.cfg', |
|
1287 ... """ |
|
1288 ... [buildout] |
|
1289 ... extends = b1.cfg b2.cfg %(b3)s |
|
1290 ... |
|
1291 ... [debug] |
|
1292 ... op = buildout |
|
1293 ... """ % dict(b3=os.path.join(other, 'b3.cfg'))) |
|
1294 |
|
1295 >>> write(sample_buildout, 'b1.cfg', |
|
1296 ... """ |
|
1297 ... [buildout] |
|
1298 ... extends = base.cfg |
|
1299 ... |
|
1300 ... [debug] |
|
1301 ... op1 = b1 1 |
|
1302 ... op2 = b1 2 |
|
1303 ... """) |
|
1304 |
|
1305 >>> write(sample_buildout, 'b2.cfg', |
|
1306 ... """ |
|
1307 ... [buildout] |
|
1308 ... extends = base.cfg |
|
1309 ... |
|
1310 ... [debug] |
|
1311 ... op2 = b2 2 |
|
1312 ... op3 = b2 3 |
|
1313 ... """) |
|
1314 |
|
1315 >>> write(other, 'b3.cfg', |
|
1316 ... """ |
|
1317 ... [buildout] |
|
1318 ... extends = b3base.cfg |
|
1319 ... |
|
1320 ... [debug] |
|
1321 ... op4 = b3 4 |
|
1322 ... """) |
|
1323 |
|
1324 >>> write(other, 'b3base.cfg', |
|
1325 ... """ |
|
1326 ... [debug] |
|
1327 ... op5 = b3base 5 |
|
1328 ... """) |
|
1329 |
|
1330 >>> write(sample_buildout, 'base.cfg', |
|
1331 ... """ |
|
1332 ... [buildout] |
|
1333 ... develop = recipes |
|
1334 ... parts = debug |
|
1335 ... |
|
1336 ... [debug] |
|
1337 ... recipe = recipes:debug |
|
1338 ... name = base |
|
1339 ... """) |
|
1340 |
|
1341 >>> print system(buildout), |
|
1342 Develop: '/sample-buildout/recipes' |
|
1343 Uninstalling debug. |
|
1344 Installing debug. |
|
1345 name base |
|
1346 op buildout |
|
1347 op1 b1 1 |
|
1348 op2 b2 2 |
|
1349 op3 b2 3 |
|
1350 op4 b3 4 |
|
1351 op5 b3base 5 |
|
1352 recipe recipes:debug |
|
1353 |
|
1354 There are several things to note about this example: |
|
1355 |
|
1356 - We can name multiple files in an extends option. |
|
1357 |
|
1358 - We can reference files recursively. |
|
1359 |
|
1360 - Relative file names in extended options are interpreted relative to |
|
1361 the directory containing the referencing configuration file. |
|
1362 |
|
1363 Loading Configuration from URLs |
|
1364 ------------------------------- |
|
1365 |
|
1366 Configuration files can be loaded from URLs. To see how this works, |
|
1367 we'll set up a web server with some configuration files. |
|
1368 |
|
1369 >>> server_data = tmpdir('server_data') |
|
1370 |
|
1371 >>> write(server_data, "r1.cfg", |
|
1372 ... """ |
|
1373 ... [debug] |
|
1374 ... op1 = r1 1 |
|
1375 ... op2 = r1 2 |
|
1376 ... """) |
|
1377 |
|
1378 >>> write(server_data, "r2.cfg", |
|
1379 ... """ |
|
1380 ... [buildout] |
|
1381 ... extends = r1.cfg |
|
1382 ... |
|
1383 ... [debug] |
|
1384 ... op2 = r2 2 |
|
1385 ... op3 = r2 3 |
|
1386 ... """) |
|
1387 |
|
1388 >>> server_url = start_server(server_data) |
|
1389 |
|
1390 >>> write('client.cfg', |
|
1391 ... """ |
|
1392 ... [buildout] |
|
1393 ... develop = recipes |
|
1394 ... parts = debug |
|
1395 ... extends = %(url)s/r2.cfg |
|
1396 ... |
|
1397 ... [debug] |
|
1398 ... recipe = recipes:debug |
|
1399 ... name = base |
|
1400 ... """ % dict(url=server_url)) |
|
1401 |
|
1402 |
|
1403 >>> print system(buildout+ ' -c client.cfg'), |
|
1404 Develop: '/sample-buildout/recipes' |
|
1405 Uninstalling debug. |
|
1406 Installing debug. |
|
1407 name base |
|
1408 op1 r1 1 |
|
1409 op2 r2 2 |
|
1410 op3 r2 3 |
|
1411 recipe recipes:debug |
|
1412 |
|
1413 Here we specified a URL for the file we extended. The file we |
|
1414 downloaded, itself referred to a file on the server using a relative |
|
1415 URL reference. Relative references are interpreted relative to the |
|
1416 base URL when they appear in configuration files loaded via URL. |
|
1417 |
|
1418 We can also specify a URL as the configuration file to be used by a |
|
1419 buildout. |
|
1420 |
|
1421 >>> os.remove('client.cfg') |
|
1422 >>> write(server_data, 'remote.cfg', |
|
1423 ... """ |
|
1424 ... [buildout] |
|
1425 ... develop = recipes |
|
1426 ... parts = debug |
|
1427 ... extends = r2.cfg |
|
1428 ... |
|
1429 ... [debug] |
|
1430 ... recipe = recipes:debug |
|
1431 ... name = remote |
|
1432 ... """) |
|
1433 |
|
1434 >>> print system(buildout + ' -c ' + server_url + '/remote.cfg'), |
|
1435 While: |
|
1436 Initializing. |
|
1437 Error: Missing option: buildout:directory |
|
1438 |
|
1439 Normally, the buildout directory defaults to directory |
|
1440 containing a configuration file. This won't work for configuration |
|
1441 files loaded from URLs. In this case, the buildout directory would |
|
1442 normally be defined on the command line: |
|
1443 |
|
1444 >>> print system(buildout |
|
1445 ... + ' -c ' + server_url + '/remote.cfg' |
|
1446 ... + ' buildout:directory=' + sample_buildout |
|
1447 ... ), |
|
1448 Develop: '/sample-buildout/recipes' |
|
1449 Uninstalling debug. |
|
1450 Installing debug. |
|
1451 name remote |
|
1452 op1 r1 1 |
|
1453 op2 r2 2 |
|
1454 op3 r2 3 |
|
1455 recipe recipes:debug |
|
1456 |
|
1457 User defaults |
|
1458 ------------- |
|
1459 |
|
1460 If the file $HOME/.buildout/default.cfg, exists, it is read before |
|
1461 reading the configuration file. ($HOME is the value of the HOME |
|
1462 environment variable. The '/' is replaced by the operating system file |
|
1463 delimiter.) |
|
1464 |
|
1465 >>> old_home = os.environ['HOME'] |
|
1466 >>> home = tmpdir('home') |
|
1467 >>> mkdir(home, '.buildout') |
|
1468 >>> write(home, '.buildout', 'default.cfg', |
|
1469 ... """ |
|
1470 ... [debug] |
|
1471 ... op1 = 1 |
|
1472 ... op7 = 7 |
|
1473 ... """) |
|
1474 |
|
1475 >>> os.environ['HOME'] = home |
|
1476 >>> print system(buildout), |
|
1477 Develop: '/sample-buildout/recipes' |
|
1478 Uninstalling debug. |
|
1479 Installing debug. |
|
1480 name base |
|
1481 op buildout |
|
1482 op1 b1 1 |
|
1483 op2 b2 2 |
|
1484 op3 b2 3 |
|
1485 op4 b3 4 |
|
1486 op5 b3base 5 |
|
1487 op7 7 |
|
1488 recipe recipes:debug |
|
1489 |
|
1490 A buildout command-line argument, -U, can be used to suppress reading |
|
1491 user defaults: |
|
1492 |
|
1493 >>> print system(buildout + ' -U'), |
|
1494 Develop: '/sample-buildout/recipes' |
|
1495 Uninstalling debug. |
|
1496 Installing debug. |
|
1497 name base |
|
1498 op buildout |
|
1499 op1 b1 1 |
|
1500 op2 b2 2 |
|
1501 op3 b2 3 |
|
1502 op4 b3 4 |
|
1503 op5 b3base 5 |
|
1504 recipe recipes:debug |
|
1505 |
|
1506 >>> os.environ['HOME'] = old_home |
|
1507 |
|
1508 Log level |
|
1509 --------- |
|
1510 |
|
1511 We can control the level of logging by specifying a log level in out |
|
1512 configuration file. For example, so suppress info messages, we can |
|
1513 set the logging level to WARNING |
|
1514 |
|
1515 >>> write(sample_buildout, 'buildout.cfg', |
|
1516 ... """ |
|
1517 ... [buildout] |
|
1518 ... log-level = WARNING |
|
1519 ... extends = b1.cfg b2.cfg |
|
1520 ... """) |
|
1521 |
|
1522 >>> print system(buildout), |
|
1523 name base |
|
1524 op1 b1 1 |
|
1525 op2 b2 2 |
|
1526 op3 b2 3 |
|
1527 recipe recipes:debug |
|
1528 |
|
1529 Uninstall recipes |
|
1530 ----------------- |
|
1531 |
|
1532 As we've seen, when parts are installed, buildout keeps track of files |
|
1533 and directories that they create. When the parts are uninstalled these |
|
1534 files and directories are deleted. |
|
1535 |
|
1536 Sometimes more clean up is needed. For example, a recipe might add a |
|
1537 system service by calling chkconfig --add during installation. Later |
|
1538 during uninstallation, chkconfig --del will need to be called to |
|
1539 remove the system service. |
|
1540 |
|
1541 In order to deal with these uninstallation issues, you can register |
|
1542 uninstall recipes. Uninstall recipes are registered using the |
|
1543 'zc.buildout.uninstall' entry point. Parts specify uninstall recipes |
|
1544 using the 'uninstall' option. |
|
1545 |
|
1546 In comparison to regular recipes, uninstall recipes are much |
|
1547 simpler. They are simply callable objects that accept the name of the |
|
1548 part to be uninstalled and the part's options dictionary. Uninstall |
|
1549 recipes don't have access to the part itself since it maybe not be |
|
1550 able to be instantiated at uninstallation time. |
|
1551 |
|
1552 Here's a recipe that simulates installation of a system service, along |
|
1553 with an uninstall recipe that simulates removing the service. |
|
1554 |
|
1555 >>> write(sample_buildout, 'recipes', 'service.py', |
|
1556 ... """ |
|
1557 ... class Service: |
|
1558 ... |
|
1559 ... def __init__(self, buildout, name, options): |
|
1560 ... self.buildout = buildout |
|
1561 ... self.name = name |
|
1562 ... self.options = options |
|
1563 ... |
|
1564 ... def install(self): |
|
1565 ... print "chkconfig --add %s" % self.options['script'] |
|
1566 ... return () |
|
1567 ... |
|
1568 ... def update(self): |
|
1569 ... pass |
|
1570 ... |
|
1571 ... |
|
1572 ... def uninstall_service(name, options): |
|
1573 ... print "chkconfig --del %s" % options['script'] |
|
1574 ... """) |
|
1575 |
|
1576 To use these recipes we must register them using entry points. Make |
|
1577 sure to use the same name for the recipe and uninstall recipe. This is |
|
1578 required to let buildout know which uninstall recipe goes with which |
|
1579 recipe. |
|
1580 |
|
1581 >>> write(sample_buildout, 'recipes', 'setup.py', |
|
1582 ... """ |
|
1583 ... from setuptools import setup |
|
1584 ... entry_points = ( |
|
1585 ... ''' |
|
1586 ... [zc.buildout] |
|
1587 ... mkdir = mkdir:Mkdir |
|
1588 ... debug = debug:Debug |
|
1589 ... service = service:Service |
|
1590 ... |
|
1591 ... [zc.buildout.uninstall] |
|
1592 ... service = service:uninstall_service |
|
1593 ... ''') |
|
1594 ... setup(name="recipes", entry_points=entry_points) |
|
1595 ... """) |
|
1596 |
|
1597 Here's how these recipes could be used in a buildout: |
|
1598 |
|
1599 >>> write(sample_buildout, 'buildout.cfg', |
|
1600 ... """ |
|
1601 ... [buildout] |
|
1602 ... develop = recipes |
|
1603 ... parts = service |
|
1604 ... |
|
1605 ... [service] |
|
1606 ... recipe = recipes:service |
|
1607 ... script = /path/to/script |
|
1608 ... """) |
|
1609 |
|
1610 When the buildout is run the service will be installed |
|
1611 |
|
1612 >>> print system(buildout) |
|
1613 Develop: '/sample-buildout/recipes' |
|
1614 Uninstalling debug. |
|
1615 Installing service. |
|
1616 chkconfig --add /path/to/script |
|
1617 <BLANKLINE> |
|
1618 |
|
1619 The service has been installed. If the buildout is run again with no |
|
1620 changes, the service shouldn't be changed. |
|
1621 |
|
1622 >>> print system(buildout) |
|
1623 Develop: '/sample-buildout/recipes' |
|
1624 Updating service. |
|
1625 <BLANKLINE> |
|
1626 |
|
1627 Now we change the service part to trigger uninstallation and |
|
1628 re-installation. |
|
1629 |
|
1630 >>> write(sample_buildout, 'buildout.cfg', |
|
1631 ... """ |
|
1632 ... [buildout] |
|
1633 ... develop = recipes |
|
1634 ... parts = service |
|
1635 ... |
|
1636 ... [service] |
|
1637 ... recipe = recipes:service |
|
1638 ... script = /path/to/a/different/script |
|
1639 ... """) |
|
1640 |
|
1641 >>> print system(buildout) |
|
1642 Develop: '/sample-buildout/recipes' |
|
1643 Uninstalling service. |
|
1644 Running uninstall recipe. |
|
1645 chkconfig --del /path/to/script |
|
1646 Installing service. |
|
1647 chkconfig --add /path/to/a/different/script |
|
1648 <BLANKLINE> |
|
1649 |
|
1650 Now we remove the service part, and add another part. |
|
1651 |
|
1652 >>> write(sample_buildout, 'buildout.cfg', |
|
1653 ... """ |
|
1654 ... [buildout] |
|
1655 ... develop = recipes |
|
1656 ... parts = debug |
|
1657 ... |
|
1658 ... [debug] |
|
1659 ... recipe = recipes:debug |
|
1660 ... """) |
|
1661 |
|
1662 >>> print system(buildout) |
|
1663 Develop: '/sample-buildout/recipes' |
|
1664 Uninstalling service. |
|
1665 Running uninstall recipe. |
|
1666 chkconfig --del /path/to/a/different/script |
|
1667 Installing debug. |
|
1668 recipe recipes:debug |
|
1669 <BLANKLINE> |
|
1670 |
|
1671 Uninstall recipes don't have to take care of removing all the files |
|
1672 and directories created by the part. This is still done automatically, |
|
1673 following the execution of the uninstall recipe. An upshot is that an |
|
1674 uninstallation recipe can access files and directories created by a |
|
1675 recipe before they are deleted. |
|
1676 |
|
1677 For example, here's an uninstallation recipe that simulates backing up |
|
1678 a directory before it is deleted. It is designed to work with the |
|
1679 mkdir recipe introduced earlier. |
|
1680 |
|
1681 >>> write(sample_buildout, 'recipes', 'backup.py', |
|
1682 ... """ |
|
1683 ... import os |
|
1684 ... def backup_directory(name, options): |
|
1685 ... path = options['path'] |
|
1686 ... size = len(os.listdir(path)) |
|
1687 ... print "backing up directory %s of size %s" % (path, size) |
|
1688 ... """) |
|
1689 |
|
1690 It must be registered with the zc.buildout.uninstall entry |
|
1691 point. Notice how it is given the name 'mkdir' to associate it with |
|
1692 the mkdir recipe. |
|
1693 |
|
1694 >>> write(sample_buildout, 'recipes', 'setup.py', |
|
1695 ... """ |
|
1696 ... from setuptools import setup |
|
1697 ... entry_points = ( |
|
1698 ... ''' |
|
1699 ... [zc.buildout] |
|
1700 ... mkdir = mkdir:Mkdir |
|
1701 ... debug = debug:Debug |
|
1702 ... service = service:Service |
|
1703 ... |
|
1704 ... [zc.buildout.uninstall] |
|
1705 ... uninstall_service = service:uninstall_service |
|
1706 ... mkdir = backup:backup_directory |
|
1707 ... ''') |
|
1708 ... setup(name="recipes", entry_points=entry_points) |
|
1709 ... """) |
|
1710 |
|
1711 Now we can use it with a mkdir part. |
|
1712 |
|
1713 >>> write(sample_buildout, 'buildout.cfg', |
|
1714 ... """ |
|
1715 ... [buildout] |
|
1716 ... develop = recipes |
|
1717 ... parts = dir debug |
|
1718 ... |
|
1719 ... [dir] |
|
1720 ... recipe = recipes:mkdir |
|
1721 ... path = my_directory |
|
1722 ... |
|
1723 ... [debug] |
|
1724 ... recipe = recipes:debug |
|
1725 ... """) |
|
1726 |
|
1727 Run the buildout to install the part. |
|
1728 |
|
1729 >>> print system(buildout) |
|
1730 Develop: '/sample-buildout/recipes' |
|
1731 Uninstalling debug. |
|
1732 Installing dir. |
|
1733 dir: Creating directory my_directory |
|
1734 Installing debug. |
|
1735 recipe recipes:debug |
|
1736 <BLANKLINE> |
|
1737 |
|
1738 Now we remove the part from the configuration file. |
|
1739 |
|
1740 >>> write(sample_buildout, 'buildout.cfg', |
|
1741 ... """ |
|
1742 ... [buildout] |
|
1743 ... develop = recipes |
|
1744 ... parts = debug |
|
1745 ... |
|
1746 ... [debug] |
|
1747 ... recipe = recipes:debug |
|
1748 ... """) |
|
1749 |
|
1750 When the buildout is run the part is removed, and the uninstall recipe |
|
1751 is run before the directory is deleted. |
|
1752 |
|
1753 >>> print system(buildout) |
|
1754 Develop: '/sample-buildout/recipes' |
|
1755 Uninstalling dir. |
|
1756 Running uninstall recipe. |
|
1757 backing up directory /sample-buildout/my_directory of size 0 |
|
1758 Updating debug. |
|
1759 recipe recipes:debug |
|
1760 <BLANKLINE> |
|
1761 |
|
1762 Now we will return the registration to normal for the benefit of the |
|
1763 rest of the examples. |
|
1764 |
|
1765 >>> write(sample_buildout, 'recipes', 'setup.py', |
|
1766 ... """ |
|
1767 ... from setuptools import setup |
|
1768 ... entry_points = ( |
|
1769 ... ''' |
|
1770 ... [zc.buildout] |
|
1771 ... mkdir = mkdir:Mkdir |
|
1772 ... debug = debug:Debug |
|
1773 ... ''') |
|
1774 ... setup(name="recipes", entry_points=entry_points) |
|
1775 ... """) |
|
1776 |
|
1777 |
|
1778 Command-line usage |
|
1779 ------------------ |
|
1780 |
|
1781 A number of arguments can be given on the buildout command line. The |
|
1782 command usage is:: |
|
1783 |
|
1784 buildout [options and assignments] [command [command arguments]] |
|
1785 |
|
1786 The following options are supported: |
|
1787 |
|
1788 -h (or --help) |
|
1789 Print basic usage information. If this option is used, then all |
|
1790 other options are ignored. |
|
1791 |
|
1792 -c filename |
|
1793 The -c option can be used to specify a configuration file, rather than |
|
1794 buildout.cfg in the current directory. |
|
1795 |
|
1796 |
|
1797 -t socket_timeout |
|
1798 |
|
1799 Specify the socket timeout in seconds. |
|
1800 |
|
1801 -v |
|
1802 Increment the verbosity by 10. The verbosity is used to adjust |
|
1803 the logging level. The verbosity is subtracted from the numeric |
|
1804 value of the log-level option specified in the configuration file. |
|
1805 |
|
1806 -q |
|
1807 Decrement the verbosity by 10. |
|
1808 |
|
1809 -U |
|
1810 Don't read user-default configuration. |
|
1811 |
|
1812 -o |
|
1813 Run in off-line mode. This is equivalent to the assignment |
|
1814 buildout:offline=true. |
|
1815 |
|
1816 -O |
|
1817 Run in non-off-line mode. This is equivalent to the assignment |
|
1818 buildout:offline=false. This is the default buildout mode. The |
|
1819 -O option would normally be used to override a true offline |
|
1820 setting in a configuration file. |
|
1821 |
|
1822 -n |
|
1823 Run in newest mode. This is equivalent to the assignment |
|
1824 buildout:newest=true. With this setting, which is the default, |
|
1825 buildout will try to find the newest versions of distributions |
|
1826 available that satisfy its requirements. |
|
1827 |
|
1828 -N |
|
1829 Run in non-newest mode. This is equivalent to the assignment |
|
1830 buildout:newest=false. With this setting, buildout will not seek |
|
1831 new distributions if installed distributions satisfy it's |
|
1832 requirements. |
|
1833 |
|
1834 Assignments are of the form:: |
|
1835 |
|
1836 section_name:option_name=value |
|
1837 |
|
1838 Options and assignments can be given in any order. |
|
1839 |
|
1840 Here's an example: |
|
1841 |
|
1842 >>> write(sample_buildout, 'other.cfg', |
|
1843 ... """ |
|
1844 ... [buildout] |
|
1845 ... develop = recipes |
|
1846 ... parts = debug |
|
1847 ... installed = .other.cfg |
|
1848 ... log-level = WARNING |
|
1849 ... |
|
1850 ... [debug] |
|
1851 ... name = other |
|
1852 ... recipe = recipes:debug |
|
1853 ... """) |
|
1854 |
|
1855 Note that we used the installed buildout option to specify an |
|
1856 alternate file to store information about installed parts. |
|
1857 |
|
1858 >>> print system(buildout+' -c other.cfg debug:op1=foo -v'), |
|
1859 Develop: '/sample-buildout/recipes' |
|
1860 Installing debug. |
|
1861 name other |
|
1862 op1 foo |
|
1863 recipe recipes:debug |
|
1864 |
|
1865 Here we used the -c option to specify an alternate configuration file, |
|
1866 and the -v option to increase the level of logging from the default, |
|
1867 WARNING. |
|
1868 |
|
1869 Options can also be combined in the usual Unix way, as in: |
|
1870 |
|
1871 >>> print system(buildout+' -vcother.cfg debug:op1=foo'), |
|
1872 Develop: '/sample-buildout/recipes' |
|
1873 Updating debug. |
|
1874 name other |
|
1875 op1 foo |
|
1876 recipe recipes:debug |
|
1877 |
|
1878 Here we combined the -v and -c options with the configuration file |
|
1879 name. Note that the -c option has to be last, because it takes an |
|
1880 argument. |
|
1881 |
|
1882 >>> os.remove(os.path.join(sample_buildout, 'other.cfg')) |
|
1883 >>> os.remove(os.path.join(sample_buildout, '.other.cfg')) |
|
1884 |
|
1885 The most commonly used command is 'install' and it takes a list of |
|
1886 parts to install. if any parts are specified, only those parts are |
|
1887 installed. To illustrate this, we'll update our configuration and run |
|
1888 the buildout in the usual way: |
|
1889 |
|
1890 >>> write(sample_buildout, 'buildout.cfg', |
|
1891 ... """ |
|
1892 ... [buildout] |
|
1893 ... develop = recipes |
|
1894 ... parts = debug d1 d2 d3 |
|
1895 ... |
|
1896 ... [d1] |
|
1897 ... recipe = recipes:mkdir |
|
1898 ... path = d1 |
|
1899 ... |
|
1900 ... [d2] |
|
1901 ... recipe = recipes:mkdir |
|
1902 ... path = d2 |
|
1903 ... |
|
1904 ... [d3] |
|
1905 ... recipe = recipes:mkdir |
|
1906 ... path = d3 |
|
1907 ... |
|
1908 ... [debug] |
|
1909 ... recipe = recipes:debug |
|
1910 ... """) |
|
1911 |
|
1912 >>> print system(buildout), |
|
1913 Develop: '/sample-buildout/recipes' |
|
1914 Uninstalling debug. |
|
1915 Installing debug. |
|
1916 recipe recipes:debug |
|
1917 Installing d1. |
|
1918 d1: Creating directory d1 |
|
1919 Installing d2. |
|
1920 d2: Creating directory d2 |
|
1921 Installing d3. |
|
1922 d3: Creating directory d3 |
|
1923 |
|
1924 >>> ls(sample_buildout) |
|
1925 - .installed.cfg |
|
1926 - b1.cfg |
|
1927 - b2.cfg |
|
1928 - base.cfg |
|
1929 d bin |
|
1930 - buildout.cfg |
|
1931 d d1 |
|
1932 d d2 |
|
1933 d d3 |
|
1934 d demo |
|
1935 d develop-eggs |
|
1936 d eggs |
|
1937 d parts |
|
1938 d recipes |
|
1939 |
|
1940 >>> cat(sample_buildout, '.installed.cfg') |
|
1941 ... # doctest: +NORMALIZE_WHITESPACE |
|
1942 [buildout] |
|
1943 installed_develop_eggs = /sample-buildout/develop-eggs/recipes.egg-link |
|
1944 parts = debug d1 d2 d3 |
|
1945 <BLANKLINE> |
|
1946 [debug] |
|
1947 __buildout_installed__ = |
|
1948 __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== |
|
1949 recipe = recipes:debug |
|
1950 <BLANKLINE> |
|
1951 [d1] |
|
1952 __buildout_installed__ = /sample-buildout/d1 |
|
1953 __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== |
|
1954 path = /sample-buildout/d1 |
|
1955 recipe = recipes:mkdir |
|
1956 <BLANKLINE> |
|
1957 [d2] |
|
1958 __buildout_installed__ = /sample-buildout/d2 |
|
1959 __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== |
|
1960 path = /sample-buildout/d2 |
|
1961 recipe = recipes:mkdir |
|
1962 <BLANKLINE> |
|
1963 [d3] |
|
1964 __buildout_installed__ = /sample-buildout/d3 |
|
1965 __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== |
|
1966 path = /sample-buildout/d3 |
|
1967 recipe = recipes:mkdir |
|
1968 |
|
1969 Now we'll update our configuration file: |
|
1970 |
|
1971 >>> write(sample_buildout, 'buildout.cfg', |
|
1972 ... """ |
|
1973 ... [buildout] |
|
1974 ... develop = recipes |
|
1975 ... parts = debug d2 d3 d4 |
|
1976 ... |
|
1977 ... [d2] |
|
1978 ... recipe = recipes:mkdir |
|
1979 ... path = data2 |
|
1980 ... |
|
1981 ... [d3] |
|
1982 ... recipe = recipes:mkdir |
|
1983 ... path = data3 |
|
1984 ... |
|
1985 ... [d4] |
|
1986 ... recipe = recipes:mkdir |
|
1987 ... path = ${d2:path}-extra |
|
1988 ... |
|
1989 ... [debug] |
|
1990 ... recipe = recipes:debug |
|
1991 ... x = 1 |
|
1992 ... """) |
|
1993 |
|
1994 and run the buildout specifying just d3 and d4: |
|
1995 |
|
1996 >>> print system(buildout+' install d3 d4'), |
|
1997 Develop: '/sample-buildout/recipes' |
|
1998 Uninstalling d3. |
|
1999 Installing d3. |
|
2000 d3: Creating directory data3 |
|
2001 Installing d4. |
|
2002 d4: Creating directory data2-extra |
|
2003 |
|
2004 >>> ls(sample_buildout) |
|
2005 - .installed.cfg |
|
2006 - b1.cfg |
|
2007 - b2.cfg |
|
2008 - base.cfg |
|
2009 d bin |
|
2010 - buildout.cfg |
|
2011 d d1 |
|
2012 d d2 |
|
2013 d data2-extra |
|
2014 d data3 |
|
2015 d demo |
|
2016 d develop-eggs |
|
2017 d eggs |
|
2018 d parts |
|
2019 d recipes |
|
2020 |
|
2021 Only the d3 and d4 recipes ran. d3 was removed and data3 and data2-extra |
|
2022 were created. |
|
2023 |
|
2024 The .installed.cfg is only updated for the recipes that ran: |
|
2025 |
|
2026 >>> cat(sample_buildout, '.installed.cfg') |
|
2027 ... # doctest: +NORMALIZE_WHITESPACE |
|
2028 [buildout] |
|
2029 installed_develop_eggs = /sample-buildout/develop-eggs/recipes.egg-link |
|
2030 parts = debug d1 d2 d3 d4 |
|
2031 <BLANKLINE> |
|
2032 [debug] |
|
2033 __buildout_installed__ = |
|
2034 __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== |
|
2035 recipe = recipes:debug |
|
2036 <BLANKLINE> |
|
2037 [d1] |
|
2038 __buildout_installed__ = /sample-buildout/d1 |
|
2039 __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== |
|
2040 path = /sample-buildout/d1 |
|
2041 recipe = recipes:mkdir |
|
2042 <BLANKLINE> |
|
2043 [d2] |
|
2044 __buildout_installed__ = /sample-buildout/d2 |
|
2045 __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== |
|
2046 path = /sample-buildout/d2 |
|
2047 recipe = recipes:mkdir |
|
2048 <BLANKLINE> |
|
2049 [d3] |
|
2050 __buildout_installed__ = /sample-buildout/data3 |
|
2051 __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== |
|
2052 path = /sample-buildout/data3 |
|
2053 recipe = recipes:mkdir |
|
2054 <BLANKLINE> |
|
2055 [d4] |
|
2056 __buildout_installed__ = /sample-buildout/data2-extra |
|
2057 __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== |
|
2058 path = /sample-buildout/data2-extra |
|
2059 recipe = recipes:mkdir |
|
2060 |
|
2061 Note that the installed data for debug, d1, and d2 haven't changed, |
|
2062 because we didn't install those parts and that the d1 and d2 |
|
2063 directories are still there. |
|
2064 |
|
2065 Now, if we run the buildout without the install command: |
|
2066 |
|
2067 >>> print system(buildout), |
|
2068 Develop: '/sample-buildout/recipes' |
|
2069 Uninstalling d2. |
|
2070 Uninstalling d1. |
|
2071 Uninstalling debug. |
|
2072 Installing debug. |
|
2073 recipe recipes:debug |
|
2074 x 1 |
|
2075 Installing d2. |
|
2076 d2: Creating directory data2 |
|
2077 Updating d3. |
|
2078 Updating d4. |
|
2079 |
|
2080 We see the output of the debug recipe and that data2 was created. We |
|
2081 also see that d1 and d2 have gone away: |
|
2082 |
|
2083 >>> ls(sample_buildout) |
|
2084 - .installed.cfg |
|
2085 - b1.cfg |
|
2086 - b2.cfg |
|
2087 - base.cfg |
|
2088 d bin |
|
2089 - buildout.cfg |
|
2090 d data2 |
|
2091 d data2-extra |
|
2092 d data3 |
|
2093 d demo |
|
2094 d develop-eggs |
|
2095 d eggs |
|
2096 d parts |
|
2097 d recipes |
|
2098 |
|
2099 Alternate directory and file locations |
|
2100 -------------------------------------- |
|
2101 |
|
2102 The buildout normally puts the bin, eggs, and parts directories in the |
|
2103 directory in the directory containing the configuration file. You can |
|
2104 provide alternate locations, and even names for these directories. |
|
2105 |
|
2106 >>> alt = tmpdir('sample-alt') |
|
2107 |
|
2108 >>> write(sample_buildout, 'buildout.cfg', |
|
2109 ... """ |
|
2110 ... [buildout] |
|
2111 ... develop = recipes |
|
2112 ... parts = |
|
2113 ... develop-eggs-directory = %(developbasket)s |
|
2114 ... eggs-directory = %(basket)s |
|
2115 ... bin-directory = %(scripts)s |
|
2116 ... parts-directory = %(work)s |
|
2117 ... """ % dict( |
|
2118 ... developbasket = os.path.join(alt, 'developbasket'), |
|
2119 ... basket = os.path.join(alt, 'basket'), |
|
2120 ... scripts = os.path.join(alt, 'scripts'), |
|
2121 ... work = os.path.join(alt, 'work'), |
|
2122 ... )) |
|
2123 |
|
2124 >>> print system(buildout), |
|
2125 Creating directory '/sample-alt/scripts'. |
|
2126 Creating directory '/sample-alt/work'. |
|
2127 Creating directory '/sample-alt/basket'. |
|
2128 Creating directory '/sample-alt/developbasket'. |
|
2129 Develop: '/sample-buildout/recipes' |
|
2130 Uninstalling d4. |
|
2131 Uninstalling d3. |
|
2132 Uninstalling d2. |
|
2133 Uninstalling debug. |
|
2134 |
|
2135 >>> ls(alt) |
|
2136 d basket |
|
2137 d developbasket |
|
2138 d scripts |
|
2139 d work |
|
2140 |
|
2141 >>> ls(alt, 'developbasket') |
|
2142 - recipes.egg-link |
|
2143 |
|
2144 You can also specify an alternate buildout directory: |
|
2145 |
|
2146 >>> rmdir(alt) |
|
2147 >>> alt = tmpdir('sample-alt') |
|
2148 |
|
2149 >>> write(sample_buildout, 'buildout.cfg', |
|
2150 ... """ |
|
2151 ... [buildout] |
|
2152 ... directory = %(alt)s |
|
2153 ... develop = %(recipes)s |
|
2154 ... parts = |
|
2155 ... """ % dict( |
|
2156 ... alt=alt, |
|
2157 ... recipes=os.path.join(sample_buildout, 'recipes'), |
|
2158 ... )) |
|
2159 |
|
2160 >>> print system(buildout), |
|
2161 Creating directory '/sample-alt/bin'. |
|
2162 Creating directory '/sample-alt/parts'. |
|
2163 Creating directory '/sample-alt/eggs'. |
|
2164 Creating directory '/sample-alt/develop-eggs'. |
|
2165 Develop: '/sample-buildout/recipes' |
|
2166 |
|
2167 >>> ls(alt) |
|
2168 - .installed.cfg |
|
2169 d bin |
|
2170 d develop-eggs |
|
2171 d eggs |
|
2172 d parts |
|
2173 |
|
2174 >>> ls(alt, 'develop-eggs') |
|
2175 - recipes.egg-link |
|
2176 |
|
2177 Logging control |
|
2178 --------------- |
|
2179 |
|
2180 Three buildout options are used to control logging: |
|
2181 |
|
2182 log-level |
|
2183 specifies the log level |
|
2184 |
|
2185 verbosity |
|
2186 adjusts the log level |
|
2187 |
|
2188 log-format |
|
2189 allows an alternate logging for mat to be specified |
|
2190 |
|
2191 We've already seen the log level and verbosity. Let's look at an example |
|
2192 of changing the format: |
|
2193 |
|
2194 >>> write(sample_buildout, 'buildout.cfg', |
|
2195 ... """ |
|
2196 ... [buildout] |
|
2197 ... develop = recipes |
|
2198 ... parts = |
|
2199 ... log-level = 25 |
|
2200 ... verbosity = 5 |
|
2201 ... log-format = %(levelname)s %(message)s |
|
2202 ... """) |
|
2203 |
|
2204 Here, we've changed the format to include the log-level name, rather |
|
2205 than the logger name. |
|
2206 |
|
2207 We've also illustrated, with a contrived example, that the log level |
|
2208 can be a numeric value and that the verbosity can be specified in the |
|
2209 configuration file. Because the verbosity is subtracted from the log |
|
2210 level, we get a final log level of 20, which is the INFO level. |
|
2211 |
|
2212 >>> print system(buildout), |
|
2213 INFO Develop: '/sample-buildout/recipes' |
|
2214 |
|
2215 Predefined buildout options |
|
2216 --------------------------- |
|
2217 |
|
2218 Buildouts have a number of predefined options that recipes can use |
|
2219 and that users can override in their configuration files. To see |
|
2220 these, we'll run a minimal buildout configuration with a debug logging |
|
2221 level. One of the features of debug logging is that the configuration |
|
2222 database is shown. |
|
2223 |
|
2224 >>> write(sample_buildout, 'buildout.cfg', |
|
2225 ... """ |
|
2226 ... [buildout] |
|
2227 ... parts = |
|
2228 ... """) |
|
2229 |
|
2230 >>> print system(buildout+' -vv'), # doctest: +NORMALIZE_WHITESPACE |
|
2231 Installing 'zc.buildout', 'setuptools'. |
|
2232 We have a develop egg: zc.buildout X.X. |
|
2233 We have the best distribution that satisfies 'setuptools'. |
|
2234 Picked: setuptools = V.V |
|
2235 <BLANKLINE> |
|
2236 Configuration data: |
|
2237 [buildout] |
|
2238 accept-buildout-test-releases = false |
|
2239 allow-hosts = * |
|
2240 allow-picked-versions = true |
|
2241 allowed-eggs-from-site-packages = * |
|
2242 bin-directory = /sample-buildout/bin |
|
2243 develop-eggs-directory = /sample-buildout/develop-eggs |
|
2244 directory = /sample-buildout |
|
2245 eggs-directory = /sample-buildout/eggs |
|
2246 exec-sitecustomize = true |
|
2247 executable = python |
|
2248 find-links = |
|
2249 include-site-packages = true |
|
2250 install-from-cache = false |
|
2251 installed = /sample-buildout/.installed.cfg |
|
2252 log-format = |
|
2253 log-level = INFO |
|
2254 newest = true |
|
2255 offline = false |
|
2256 parts = |
|
2257 parts-directory = /sample-buildout/parts |
|
2258 prefer-final = false |
|
2259 python = buildout |
|
2260 relative-paths = false |
|
2261 socket-timeout = |
|
2262 unzip = false |
|
2263 use-dependency-links = true |
|
2264 verbosity = 20 |
|
2265 <BLANKLINE> |
|
2266 |
|
2267 All of these options can be overridden by configuration files or by |
|
2268 command-line assignments. We've discussed most of these options |
|
2269 already, but let's review them and touch on some we haven't discussed: |
|
2270 |
|
2271 allowed-eggs-from-site-packages |
|
2272 Sometimes you need or want to control what eggs from site-packages are |
|
2273 used. The allowed-eggs-from-site-packages option allows you to specify a |
|
2274 whitelist of project names that may be included from site-packages. You |
|
2275 can use globs to specify the value. It defaults to a single value of '*', |
|
2276 indicating that any package may come from site-packages. |
|
2277 |
|
2278 Here's a usage example:: |
|
2279 |
|
2280 [buildout] |
|
2281 ... |
|
2282 |
|
2283 allowed-eggs-from-site-packages = |
|
2284 demo |
|
2285 bigdemo |
|
2286 zope.* |
|
2287 |
|
2288 This option interacts with the ``include-site-packages`` option in the |
|
2289 following ways. |
|
2290 |
|
2291 If ``include-site-packages`` is true, then |
|
2292 ``allowed-eggs-from-site-packages`` filters what eggs from site-packages |
|
2293 may be chosen. Therefore, if ``allowed-eggs-from-site-packages`` is an |
|
2294 empty list, then no eggs from site-packages are chosen, but site-packages |
|
2295 will still be included at the end of path lists. |
|
2296 |
|
2297 If ``include-site-packages`` is false, the value of |
|
2298 ``allowed-eggs-from-site-packages`` is irrelevant. |
|
2299 |
|
2300 See the ``include-site-packages`` description for more information. |
|
2301 |
|
2302 bin-directory |
|
2303 The directory path where scripts are written. This can be a |
|
2304 relative path, which is interpreted relative to the directory |
|
2305 option. |
|
2306 |
|
2307 develop-eggs-directory |
|
2308 The directory path where development egg links are created for software |
|
2309 being created in the local project. This can be a relative path, |
|
2310 which is interpreted relative to the directory option. |
|
2311 |
|
2312 directory |
|
2313 The buildout directory. This is the base for other buildout file |
|
2314 and directory locations, when relative locations are used. |
|
2315 |
|
2316 eggs-directory |
|
2317 The directory path where downloaded eggs are put. It is common to share |
|
2318 this directory across buildouts. Eggs in this directory should |
|
2319 *never* be modified. This can be a relative path, which is |
|
2320 interpreted relative to the directory option. |
|
2321 |
|
2322 exec-sitecustomize |
|
2323 Normally the Python's real sitecustomize module is processed. |
|
2324 If you do not want it to be processed in order to increase the |
|
2325 repeatability of your buildout, set this value to 'false'. This will |
|
2326 be honored irrespective of the setting for include-site-packages. |
|
2327 This option will be honored by some recipes and not others. |
|
2328 z3c.recipe.scripts honors this and zc.recipe.egg does not, for |
|
2329 instance. |
|
2330 |
|
2331 executable |
|
2332 The Python executable used to run the buildout. See the python |
|
2333 option below. |
|
2334 |
|
2335 include-site-packages |
|
2336 You can choose not to have the site-packages of the underlying Python |
|
2337 available to your script or interpreter, in addition to the packages |
|
2338 from your eggs. This can increase repeatability for your buildout. |
|
2339 This option will be better used by some recipes than others. |
|
2340 z3c.recipe.scripts honors this fully and zc.recipe.egg only |
|
2341 partially, for instance. |
|
2342 |
|
2343 installed |
|
2344 The file path where information about the results of the previous |
|
2345 buildout run is written. This can be a relative path, which is |
|
2346 interpreted relative to the directory option. This file provides |
|
2347 an inventory of installed parts with information needed to decide |
|
2348 which if any parts need to be uninstalled. |
|
2349 |
|
2350 log-format |
|
2351 The format used for logging messages. |
|
2352 |
|
2353 log-level |
|
2354 The log level before verbosity adjustment |
|
2355 |
|
2356 parts |
|
2357 A white space separated list of parts to be installed. |
|
2358 |
|
2359 parts-directory |
|
2360 A working directory that parts can used to store data. |
|
2361 |
|
2362 python |
|
2363 The name of a section containing information about the default |
|
2364 Python interpreter. Recipes that need a installation |
|
2365 typically have options to tell them which Python installation to |
|
2366 use. By convention, if a section-specific option isn't used, the |
|
2367 option is looked for in the buildout section. The option must |
|
2368 point to a section with an executable option giving the path to a |
|
2369 Python executable. By default, the buildout section defines the |
|
2370 default Python as the Python used to run the buildout. |
|
2371 |
|
2372 relative-paths |
|
2373 The paths generated by zc.buildout are absolute by default, and this |
|
2374 option is ``false``. However, if you set this value to be ``true``, |
|
2375 bin/buildout will be generated with code that makes the paths relative. |
|
2376 Some recipes, such as zc.recipe.egg and z3c.recipe.scripts, honor this |
|
2377 value as well. |
|
2378 |
|
2379 unzip |
|
2380 By default, zc.buildout doesn't unzip zip-safe eggs ("unzip = false"). |
|
2381 This follows the policy followed by setuptools itself. Experience shows |
|
2382 this policy to to be inconvenient. Zipped eggs make debugging more |
|
2383 difficult and often import more slowly. You can include an unzip option in |
|
2384 the buildout section to change the default unzipping policy ("unzip = |
|
2385 true"). |
|
2386 |
|
2387 use-dependency-links |
|
2388 By default buildout will obey the setuptools dependency_links metadata |
|
2389 when it looks for dependencies. This behavior can be controlled with |
|
2390 the use-dependency-links buildout option:: |
|
2391 |
|
2392 [buildout] |
|
2393 ... |
|
2394 use-dependency-links = false |
|
2395 |
|
2396 The option defaults to true. If you set it to false, then dependency |
|
2397 links are only looked for in the locations specified by find-links. |
|
2398 |
|
2399 verbosity |
|
2400 A log-level adjustment. Typically, this is set via the -q and -v |
|
2401 command-line options. |
|
2402 |
|
2403 |
|
2404 Creating new buildouts and bootstrapping |
|
2405 ---------------------------------------- |
|
2406 |
|
2407 If zc.buildout is installed, you can use it to create a new buildout |
|
2408 with it's own local copies of zc.buildout and setuptools and with |
|
2409 local buildout scripts. |
|
2410 |
|
2411 >>> sample_bootstrapped = tmpdir('sample-bootstrapped') |
|
2412 |
|
2413 >>> print system(buildout |
|
2414 ... +' -c'+os.path.join(sample_bootstrapped, 'setup.cfg') |
|
2415 ... +' init'), |
|
2416 Creating '/sample-bootstrapped/setup.cfg'. |
|
2417 Creating directory '/sample-bootstrapped/bin'. |
|
2418 Creating directory '/sample-bootstrapped/parts'. |
|
2419 Creating directory '/sample-bootstrapped/eggs'. |
|
2420 Creating directory '/sample-bootstrapped/develop-eggs'. |
|
2421 Generated script '/sample-bootstrapped/bin/buildout'. |
|
2422 |
|
2423 Note that a basic setup.cfg was created for us. |
|
2424 |
|
2425 >>> ls(sample_bootstrapped) |
|
2426 d bin |
|
2427 d develop-eggs |
|
2428 d eggs |
|
2429 d parts |
|
2430 - setup.cfg |
|
2431 |
|
2432 >>> ls(sample_bootstrapped, 'bin') |
|
2433 - buildout |
|
2434 |
|
2435 >>> _ = (ls(sample_bootstrapped, 'eggs'), |
|
2436 ... ls(sample_bootstrapped, 'develop-eggs')) |
|
2437 - setuptools-0.6-py2.3.egg |
|
2438 - zc.buildout-1.0-py2.3.egg |
|
2439 |
|
2440 (We list both the eggs and develop-eggs directories because the |
|
2441 buildout or setuptools egg could be installed in the develop-eggs |
|
2442 directory if the original buildout had develop eggs for either |
|
2443 buildout or setuptools.) |
|
2444 |
|
2445 If relative-paths is ``true``, the buildout script uses relative paths. |
|
2446 |
|
2447 >>> write(sample_bootstrapped, 'setup.cfg', |
|
2448 ... ''' |
|
2449 ... [buildout] |
|
2450 ... relative-paths = true |
|
2451 ... parts = |
|
2452 ... ''') |
|
2453 |
|
2454 >>> print system(buildout |
|
2455 ... +' -c'+os.path.join(sample_bootstrapped, 'setup.cfg') |
|
2456 ... +' bootstrap'), |
|
2457 Generated script '/sample-bootstrapped/bin/buildout'. |
|
2458 |
|
2459 >>> buildout_script = join(sample_bootstrapped, 'bin', 'buildout') |
|
2460 >>> import sys |
|
2461 >>> if sys.platform.startswith('win'): |
|
2462 ... buildout_script += '-script.py' |
|
2463 >>> print open(buildout_script).read() # doctest: +ELLIPSIS |
|
2464 #!... -S |
|
2465 <BLANKLINE> |
|
2466 import os |
|
2467 <BLANKLINE> |
|
2468 join = os.path.join |
|
2469 base = os.path.dirname(os.path.abspath(os.path.realpath(__file__))) |
|
2470 base = os.path.dirname(base) |
|
2471 <BLANKLINE> |
|
2472 import sys |
|
2473 sys.path[0:0] = [ |
|
2474 join(base, 'parts/buildout'), |
|
2475 ] |
|
2476 <BLANKLINE> |
|
2477 <BLANKLINE> |
|
2478 import os |
|
2479 path = sys.path[0] |
|
2480 if os.environ.get('PYTHONPATH'): |
|
2481 path = os.pathsep.join([path, os.environ['PYTHONPATH']]) |
|
2482 os.environ['BUILDOUT_ORIGINAL_PYTHONPATH'] = os.environ.get('PYTHONPATH', '') |
|
2483 os.environ['PYTHONPATH'] = path |
|
2484 import site # imports custom buildout-generated site.py |
|
2485 <BLANKLINE> |
|
2486 import zc.buildout.buildout |
|
2487 <BLANKLINE> |
|
2488 if __name__ == '__main__': |
|
2489 zc.buildout.buildout.main() |
|
2490 <BLANKLINE> |
|
2491 |
|
2492 |
|
2493 Note that, in the above two examples, the buildout script was installed |
|
2494 but not run. To run the buildout, we'd have to run the installed |
|
2495 buildout script. |
|
2496 |
|
2497 If we have an existing buildout that already has a buildout.cfg, we'll |
|
2498 normally use the bootstrap command instead of init. It will complain |
|
2499 if there isn't a configuration file: |
|
2500 |
|
2501 >>> sample_bootstrapped2 = tmpdir('sample-bootstrapped2') |
|
2502 |
|
2503 >>> print system(buildout |
|
2504 ... +' -c'+os.path.join(sample_bootstrapped2, 'setup.cfg') |
|
2505 ... +' bootstrap'), |
|
2506 While: |
|
2507 Initializing. |
|
2508 Error: Couldn't open /sample-bootstrapped2/setup.cfg |
|
2509 |
|
2510 >>> write(sample_bootstrapped2, 'setup.cfg', |
|
2511 ... """ |
|
2512 ... [buildout] |
|
2513 ... parts = |
|
2514 ... """) |
|
2515 |
|
2516 >>> print system(buildout |
|
2517 ... +' -c'+os.path.join(sample_bootstrapped2, 'setup.cfg') |
|
2518 ... +' bootstrap'), |
|
2519 Creating directory '/sample-bootstrapped2/bin'. |
|
2520 Creating directory '/sample-bootstrapped2/parts'. |
|
2521 Creating directory '/sample-bootstrapped2/eggs'. |
|
2522 Creating directory '/sample-bootstrapped2/develop-eggs'. |
|
2523 Generated script '/sample-bootstrapped2/bin/buildout'. |
|
2524 |
|
2525 |
|
2526 Newest and Offline Modes |
|
2527 ------------------------ |
|
2528 |
|
2529 By default buildout and recipes will try to find the newest versions |
|
2530 of distributions needed to satisfy requirements. This can be very |
|
2531 time consuming, especially when incrementally working on setting up a |
|
2532 buildout or working on a recipe. The buildout newest option can be |
|
2533 used to to suppress this. If the newest option is set to false, then |
|
2534 new distributions won't be sought if an installed distribution meets |
|
2535 requirements. The newest option can be set to false using the -N |
|
2536 command-line option. |
|
2537 |
|
2538 The offline option goes a bit further. If the buildout offline option |
|
2539 is given a value of "true", the buildout and recipes that are aware of |
|
2540 the option will avoid doing network access. This is handy when |
|
2541 running the buildout when not connected to the internet. It also |
|
2542 makes buildouts run much faster. This option is typically set using |
|
2543 the buildout -o option. |
|
2544 |
|
2545 Preferring Final Releases |
|
2546 ------------------------- |
|
2547 |
|
2548 Currently, when searching for new releases of your project's |
|
2549 dependencies, the newest available release is used. This isn't usually |
|
2550 ideal, as you may get a development release or alpha releases not ready |
|
2551 to be widely used. You can request that final releases be preferred |
|
2552 using the ``prefer-final`` option in the buildout section:: |
|
2553 |
|
2554 [buildout] |
|
2555 ... |
|
2556 prefer-final = true |
|
2557 |
|
2558 When the ``prefer-final`` option is set to true, then when searching for |
|
2559 new releases, final releases are preferred. If there are final |
|
2560 releases that satisfy distribution requirements, then those releases |
|
2561 are used even if newer non-final releases are available. |
|
2562 |
|
2563 In buildout version 2, all final releases will be preferred by |
|
2564 default--that is ``prefer-final`` will also default to 'true'. You will |
|
2565 then need to use a 'false' value for ``prefer-final`` to get the newest |
|
2566 releases. |
|
2567 |
|
2568 A separate option controls the behavior of the build system itself. |
|
2569 When buildout looks for recipes, extensions, and for updates to itself, |
|
2570 it does prefer final releases by default, as of the 1.5.0 release. The |
|
2571 ``accept-buildout-test-releases`` option will let you override this behavior. |
|
2572 However, it is typically changed by the --accept-buildout-test-releases |
|
2573 option to the bootstrap script, since bootstrapping is the first step to |
|
2574 selecting a buildout. |
|
2575 |
|
2576 Finding distributions |
|
2577 --------------------- |
|
2578 |
|
2579 By default, buildout searches the Python Package Index when looking |
|
2580 for distributions. You can, instead, specify your own index to search |
|
2581 using the `index` option:: |
|
2582 |
|
2583 [buildout] |
|
2584 ... |
|
2585 index = http://index.example.com/ |
|
2586 |
|
2587 This index, or the default of http://pypi.python.org/simple/ if no |
|
2588 index is specified, will always be searched for distributions unless |
|
2589 running buildout with options that prevent searching for |
|
2590 distributions. The latest version of the distribution that meets the |
|
2591 requirements of the buildout will always be used. |
|
2592 |
|
2593 You can also specify more locations to search for distributions using |
|
2594 the `find-links` option. All locations specified will be searched for |
|
2595 distributions along with the package index as described before. |
|
2596 |
|
2597 Locations can be urls:: |
|
2598 |
|
2599 [buildout] |
|
2600 ... |
|
2601 find-links = http://download.zope.org/distribution/ |
|
2602 |
|
2603 They can also be directories on disk:: |
|
2604 |
|
2605 [buildout] |
|
2606 ... |
|
2607 find-links = /some/path |
|
2608 |
|
2609 Finally, they can also be direct paths to distributions:: |
|
2610 |
|
2611 [buildout] |
|
2612 ... |
|
2613 find-links = /some/path/someegg-1.0.0-py2.3.egg |
|
2614 |
|
2615 Any number of locations can be specified in the `find-links` option:: |
|
2616 |
|
2617 [buildout] |
|
2618 ... |
|
2619 find-links = |
|
2620 http://download.zope.org/distribution/ |
|
2621 /some/otherpath |
|
2622 /some/path/someegg-1.0.0-py2.3.egg |
|
2623 |
|
2624 Dependency links |
|
2625 ---------------- |
|
2626 |
|
2627 By default buildout will obey the setuptools dependency_links metadata |
|
2628 when it looks for dependencies. This behavior can be controlled with |
|
2629 the use-dependency-links buildout option:: |
|
2630 |
|
2631 [buildout] |
|
2632 ... |
|
2633 use-dependency-links = false |
|
2634 |
|
2635 The option defaults to true. If you set it to false, then dependency |
|
2636 links are only looked for in the locations specified by find-links. |
|
2637 |
|
2638 Controlling the installation database |
|
2639 ------------------------------------- |
|
2640 |
|
2641 The buildout installed option is used to specify the file used to save |
|
2642 information on installed parts. This option is initialized to |
|
2643 ".installed.cfg", but it can be overridden in the configuration file |
|
2644 or on the command line: |
|
2645 |
|
2646 >>> write('buildout.cfg', |
|
2647 ... """ |
|
2648 ... [buildout] |
|
2649 ... develop = recipes |
|
2650 ... parts = debug |
|
2651 ... |
|
2652 ... [debug] |
|
2653 ... recipe = recipes:debug |
|
2654 ... """) |
|
2655 |
|
2656 >>> print system(buildout+' buildout:installed=inst.cfg'), |
|
2657 Develop: '/sample-buildout/recipes' |
|
2658 Installing debug. |
|
2659 recipe recipes:debug |
|
2660 |
|
2661 >>> ls(sample_buildout) |
|
2662 - b1.cfg |
|
2663 - b2.cfg |
|
2664 - base.cfg |
|
2665 d bin |
|
2666 - buildout.cfg |
|
2667 d demo |
|
2668 d develop-eggs |
|
2669 d eggs |
|
2670 - inst.cfg |
|
2671 d parts |
|
2672 d recipes |
|
2673 |
|
2674 The installation database can be disabled by supplying an empty |
|
2675 buildout installed option: |
|
2676 |
|
2677 >>> os.remove('inst.cfg') |
|
2678 >>> print system(buildout+' buildout:installed='), |
|
2679 Develop: '/sample-buildout/recipes' |
|
2680 Installing debug. |
|
2681 recipe recipes:debug |
|
2682 |
|
2683 >>> ls(sample_buildout) |
|
2684 - b1.cfg |
|
2685 - b2.cfg |
|
2686 - base.cfg |
|
2687 d bin |
|
2688 - buildout.cfg |
|
2689 d demo |
|
2690 d develop-eggs |
|
2691 d eggs |
|
2692 d parts |
|
2693 d recipes |
|
2694 |
|
2695 |
|
2696 Note that there will be no installation database if there are no parts: |
|
2697 |
|
2698 >>> write('buildout.cfg', |
|
2699 ... """ |
|
2700 ... [buildout] |
|
2701 ... parts = |
|
2702 ... """) |
|
2703 |
|
2704 >>> print system(buildout+' buildout:installed=inst.cfg'), |
|
2705 |
|
2706 >>> ls(sample_buildout) |
|
2707 - b1.cfg |
|
2708 - b2.cfg |
|
2709 - base.cfg |
|
2710 d bin |
|
2711 - buildout.cfg |
|
2712 d demo |
|
2713 d develop-eggs |
|
2714 d eggs |
|
2715 d parts |
|
2716 d recipes |
|
2717 |
|
2718 Extensions |
|
2719 ---------- |
|
2720 |
|
2721 A feature allows code to be loaded and run after |
|
2722 configuration files have been read but before the buildout has begun |
|
2723 any processing. The intent is to allow special plugins such as |
|
2724 urllib2 request handlers to be loaded. |
|
2725 |
|
2726 To load an extension, we use the extensions option and list one or |
|
2727 more distribution requirements, on separate lines. The distributions |
|
2728 named will be loaded and any ``zc.buildout.extension`` entry points found |
|
2729 will be called with the buildout as an argument. When buildout |
|
2730 finishes processing, any ``zc.buildout.unloadextension`` entry points |
|
2731 found will be called with the buildout as an argument. |
|
2732 |
|
2733 Let's create a sample extension in our sample buildout created in the |
|
2734 previous section: |
|
2735 |
|
2736 >>> mkdir(sample_bootstrapped, 'demo') |
|
2737 |
|
2738 >>> write(sample_bootstrapped, 'demo', 'demo.py', |
|
2739 ... """ |
|
2740 ... def ext(buildout): |
|
2741 ... print 'ext', list(buildout) |
|
2742 ... def unload(buildout): |
|
2743 ... print 'unload', list(buildout) |
|
2744 ... """) |
|
2745 |
|
2746 >>> write(sample_bootstrapped, 'demo', 'setup.py', |
|
2747 ... """ |
|
2748 ... from setuptools import setup |
|
2749 ... |
|
2750 ... setup( |
|
2751 ... name = "demo", |
|
2752 ... entry_points = { |
|
2753 ... 'zc.buildout.extension': ['ext = demo:ext'], |
|
2754 ... 'zc.buildout.unloadextension': ['ext = demo:unload'], |
|
2755 ... }, |
|
2756 ... ) |
|
2757 ... """) |
|
2758 |
|
2759 Our extension just prints out the word 'demo', and lists the sections |
|
2760 found in the buildout passed to it. |
|
2761 |
|
2762 We'll update our buildout.cfg to list the demo directory as a develop |
|
2763 egg to be built: |
|
2764 |
|
2765 >>> write(sample_bootstrapped, 'buildout.cfg', |
|
2766 ... """ |
|
2767 ... [buildout] |
|
2768 ... develop = demo |
|
2769 ... parts = |
|
2770 ... """) |
|
2771 |
|
2772 >>> os.chdir(sample_bootstrapped) |
|
2773 >>> print system(os.path.join(sample_bootstrapped, 'bin', 'buildout')), |
|
2774 Develop: '/sample-bootstrapped/demo' |
|
2775 |
|
2776 Now we can add the extensions option. We were a bit tricky and ran |
|
2777 the buildout once with the demo develop egg defined but without the |
|
2778 extension option. This is because extensions are loaded before the |
|
2779 buildout creates develop eggs. We needed to use a separate buildout |
|
2780 run to create the develop egg. Normally, when eggs are loaded from |
|
2781 the network, we wouldn't need to do anything special. |
|
2782 |
|
2783 >>> write(sample_bootstrapped, 'buildout.cfg', |
|
2784 ... """ |
|
2785 ... [buildout] |
|
2786 ... develop = demo |
|
2787 ... extensions = demo |
|
2788 ... parts = |
|
2789 ... """) |
|
2790 |
|
2791 We see that our extension is loaded and executed: |
|
2792 |
|
2793 >>> print system(os.path.join(sample_bootstrapped, 'bin', 'buildout')), |
|
2794 ext ['buildout'] |
|
2795 Develop: '/sample-bootstrapped/demo' |
|
2796 unload ['buildout'] |
|
2797 |
|
2798 Allow hosts |
|
2799 ----------- |
|
2800 |
|
2801 On some environments the links visited by `zc.buildout` can be forbidden |
|
2802 by paranoiac firewalls. These URL might be on the chain of links |
|
2803 visited by `zc.buildout` wheter they are defined in the `find-links` option, |
|
2804 wheter they are defined by various eggs in their `url`, `download_url`, |
|
2805 `dependency_links` metadata. |
|
2806 |
|
2807 It is even harder to track that package_index works like a spider and |
|
2808 might visit links and go to other location. |
|
2809 |
|
2810 The `allow-hosts` option provides a way to prevent this, and |
|
2811 works exactly like the one provided in `easy_install`. |
|
2812 |
|
2813 You can provide a list of allowed host, together with wildcards:: |
|
2814 |
|
2815 [buildout] |
|
2816 ... |
|
2817 |
|
2818 allow-hosts = |
|
2819 *.python.org |
|
2820 example.com |
|
2821 |
|
2822 All urls that does not match these hosts will not be visited. |
|
2823 |
|
2824 .. [#future_recipe_methods] In the future, additional methods may be |
|
2825 added. Older recipes with fewer methods will still be |
|
2826 supported. |
|
2827 |
|
2828 .. [#packaging_info] If we wanted to create a distribution from this |
|
2829 package, we would need specify much more information. See the |
|
2830 `setuptools documentation |
|
2831 <http://peak.telecommunity.com/DevCenter/setuptools>`_. |