|
1 Python API for egg and script installation |
|
2 ========================================== |
|
3 |
|
4 The easy_install module provides some functions to provide support for |
|
5 egg and script installation. It provides functionality at the python |
|
6 level that is similar to easy_install, with a few exceptions: |
|
7 |
|
8 - By default, we look for new packages *and* the packages that |
|
9 they depend on. This is somewhat like (and uses) the --upgrade |
|
10 option of easy_install, except that we also upgrade required |
|
11 packages. |
|
12 |
|
13 - If the highest-revision package satisfying a specification is |
|
14 already present, then we don't try to get another one. This saves a |
|
15 lot of search time in the common case that packages are pegged to |
|
16 specific versions. |
|
17 |
|
18 - If there is a develop egg that satisfies a requirement, we don't |
|
19 look for additional distributions. We always give preference to |
|
20 develop eggs. |
|
21 |
|
22 - Distutils options for building extensions can be passed. |
|
23 |
|
24 Distribution installation |
|
25 ------------------------- |
|
26 |
|
27 The easy_install module provides a function, install, for installing one |
|
28 or more packages and their dependencies. The install function takes 2 |
|
29 positional arguments: |
|
30 |
|
31 - An iterable of setuptools requirement strings for the distributions |
|
32 to be installed, and |
|
33 |
|
34 - A destination directory to install to and to satisfy requirements |
|
35 from. The destination directory can be None, in which case, no new |
|
36 distributions are downloaded and there will be an error if the |
|
37 needed distributions can't be found among those already installed. |
|
38 |
|
39 It supports a number of optional keyword arguments: |
|
40 |
|
41 links |
|
42 A sequence of URLs, file names, or directories to look for |
|
43 links to distributions. |
|
44 |
|
45 index |
|
46 The URL of an index server, or almost any other valid URL. :) |
|
47 |
|
48 If not specified, the Python Package Index, |
|
49 http://pypi.python.org/simple/, is used. You can specify an |
|
50 alternate index with this option. If you use the links option and |
|
51 if the links point to the needed distributions, then the index can |
|
52 be anything and will be largely ignored. In the examples, here, |
|
53 we'll just point to an empty directory on our link server. This |
|
54 will make our examples run a little bit faster. |
|
55 |
|
56 executable |
|
57 A path to a Python executable. Distributions will be installed |
|
58 using this executable and will be for the matching Python version. |
|
59 |
|
60 path |
|
61 A list of additional directories to search for locally-installed |
|
62 distributions. |
|
63 |
|
64 always_unzip |
|
65 A flag indicating that newly-downloaded distributions should be |
|
66 directories even if they could be installed as zip files. |
|
67 |
|
68 working_set |
|
69 An existing working set to be augmented with additional |
|
70 distributions, if necessary to satisfy requirements. This allows |
|
71 you to call install multiple times, if necessary, to gather |
|
72 multiple sets of requirements. |
|
73 |
|
74 newest |
|
75 A boolean value indicating whether to search for new distributions |
|
76 when already-installed distributions meet the requirement. When |
|
77 this is true, the default, and when the destination directory is |
|
78 not None, then the install function will search for the newest |
|
79 distributions that satisfy the requirements. |
|
80 |
|
81 versions |
|
82 A dictionary mapping project names to version numbers to be used |
|
83 when selecting distributions. This can be used to specify a set of |
|
84 distribution versions independent of other requirements. |
|
85 |
|
86 use_dependency_links |
|
87 A flag indicating whether to search for dependencies using the |
|
88 setup dependency_links metadata or not. If true, links are searched |
|
89 for using dependency_links in preference to other |
|
90 locations. Defaults to true. |
|
91 |
|
92 include_site_packages |
|
93 A flag indicating whether Python's non-standard-library packages should |
|
94 be available for finding dependencies. Defaults to true. |
|
95 |
|
96 Paths outside of Python's standard library--or more precisely, those that |
|
97 are not included when Python is started with the -S argument--are loosely |
|
98 referred to as "site-packages" here. |
|
99 |
|
100 relative_paths |
|
101 Adjust egg paths so they are relative to the script path. This |
|
102 allows scripts to work when scripts and eggs are moved, as long as |
|
103 they are both moved in the same way. |
|
104 |
|
105 The install method returns a working set containing the distributions |
|
106 needed to meet the given requirements. |
|
107 |
|
108 We have a link server that has a number of eggs: |
|
109 |
|
110 >>> print get(link_server), |
|
111 <html><body> |
|
112 <a href="bigdemo-0.1-py2.4.egg">bigdemo-0.1-py2.4.egg</a><br> |
|
113 <a href="demo-0.1-py2.4.egg">demo-0.1-py2.4.egg</a><br> |
|
114 <a href="demo-0.2-py2.4.egg">demo-0.2-py2.4.egg</a><br> |
|
115 <a href="demo-0.3-py2.4.egg">demo-0.3-py2.4.egg</a><br> |
|
116 <a href="demo-0.4c1-py2.4.egg">demo-0.4c1-py2.4.egg</a><br> |
|
117 <a href="demoneeded-1.0.zip">demoneeded-1.0.zip</a><br> |
|
118 <a href="demoneeded-1.1.zip">demoneeded-1.1.zip</a><br> |
|
119 <a href="demoneeded-1.2c1.zip">demoneeded-1.2c1.zip</a><br> |
|
120 <a href="extdemo-1.4.zip">extdemo-1.4.zip</a><br> |
|
121 <a href="index/">index/</a><br> |
|
122 <a href="other-1.0-py2.4.egg">other-1.0-py2.4.egg</a><br> |
|
123 </body></html> |
|
124 |
|
125 Let's make a directory and install the demo egg to it, using the demo: |
|
126 |
|
127 >>> dest = tmpdir('sample-install') |
|
128 >>> import zc.buildout.easy_install |
|
129 >>> ws = zc.buildout.easy_install.install( |
|
130 ... ['demo==0.2'], dest, |
|
131 ... links=[link_server], index=link_server+'index/') |
|
132 |
|
133 We requested version 0.2 of the demo distribution to be installed into |
|
134 the destination server. We specified that we should search for links |
|
135 on the link server and that we should use the (empty) link server |
|
136 index directory as a package index. |
|
137 |
|
138 The working set contains the distributions we retrieved. |
|
139 |
|
140 >>> for dist in ws: |
|
141 ... print dist |
|
142 demo 0.2 |
|
143 demoneeded 1.1 |
|
144 |
|
145 We got demoneeded because it was a dependency of demo. |
|
146 |
|
147 And the actual eggs were added to the eggs directory. |
|
148 |
|
149 >>> ls(dest) |
|
150 - demo-0.2-py2.4.egg |
|
151 - demoneeded-1.1-py2.4.egg |
|
152 |
|
153 If we remove the version restriction on demo, but specify a false |
|
154 value for newest, no new distributions will be installed: |
|
155 |
|
156 >>> ws = zc.buildout.easy_install.install( |
|
157 ... ['demo'], dest, links=[link_server], index=link_server+'index/', |
|
158 ... newest=False) |
|
159 >>> ls(dest) |
|
160 - demo-0.2-py2.4.egg |
|
161 - demoneeded-1.1-py2.4.egg |
|
162 |
|
163 If we leave off the newest option, we'll get an update for demo: |
|
164 |
|
165 >>> ws = zc.buildout.easy_install.install( |
|
166 ... ['demo'], dest, links=[link_server], index=link_server+'index/') |
|
167 >>> ls(dest) |
|
168 - demo-0.2-py2.4.egg |
|
169 - demo-0.3-py2.4.egg |
|
170 - demoneeded-1.1-py2.4.egg |
|
171 |
|
172 Note that we didn't get the newest versions available. There were |
|
173 release candidates for newer versions of both packages. By default, |
|
174 final releases are preferred. We can change this behavior using the |
|
175 prefer_final function: |
|
176 |
|
177 >>> zc.buildout.easy_install.prefer_final(False) |
|
178 True |
|
179 |
|
180 The old setting is returned. |
|
181 |
|
182 >>> ws = zc.buildout.easy_install.install( |
|
183 ... ['demo'], dest, links=[link_server], index=link_server+'index/') |
|
184 >>> for dist in ws: |
|
185 ... print dist |
|
186 demo 0.4c1 |
|
187 demoneeded 1.2c1 |
|
188 |
|
189 >>> ls(dest) |
|
190 - demo-0.2-py2.4.egg |
|
191 - demo-0.3-py2.4.egg |
|
192 - demo-0.4c1-py2.4.egg |
|
193 - demoneeded-1.1-py2.4.egg |
|
194 - demoneeded-1.2c1-py2.4.egg |
|
195 |
|
196 Let's put the setting back to the default. |
|
197 |
|
198 >>> zc.buildout.easy_install.prefer_final(True) |
|
199 False |
|
200 |
|
201 We can supply additional distributions. We can also supply |
|
202 specifications for distributions that would normally be found via |
|
203 dependencies. We might do this to specify a specific version. |
|
204 |
|
205 >>> ws = zc.buildout.easy_install.install( |
|
206 ... ['demo', 'other', 'demoneeded==1.0'], dest, |
|
207 ... links=[link_server], index=link_server+'index/') |
|
208 |
|
209 >>> for dist in ws: |
|
210 ... print dist |
|
211 demo 0.3 |
|
212 other 1.0 |
|
213 demoneeded 1.0 |
|
214 |
|
215 >>> ls(dest) |
|
216 - demo-0.2-py2.4.egg |
|
217 - demo-0.3-py2.4.egg |
|
218 - demo-0.4c1-py2.4.egg |
|
219 - demoneeded-1.0-py2.4.egg |
|
220 - demoneeded-1.1-py2.4.egg |
|
221 - demoneeded-1.2c1-py2.4.egg |
|
222 d other-1.0-py2.4.egg |
|
223 |
|
224 We can request that eggs be unzipped even if they are zip safe. This |
|
225 can be useful when debugging. (Note that Distribute will unzip eggs by |
|
226 default, so if you are using Distribute, most or all eggs will already be |
|
227 unzipped without this flag.) |
|
228 |
|
229 >>> rmdir(dest) |
|
230 >>> dest = tmpdir('sample-install') |
|
231 >>> ws = zc.buildout.easy_install.install( |
|
232 ... ['demo'], dest, links=[link_server], index=link_server+'index/', |
|
233 ... always_unzip=True) |
|
234 |
|
235 >>> ls(dest) |
|
236 d demo-0.3-py2.4.egg |
|
237 d demoneeded-1.1-py2.4.egg |
|
238 |
|
239 >>> rmdir(dest) |
|
240 >>> dest = tmpdir('sample-install') |
|
241 >>> ws = zc.buildout.easy_install.install( |
|
242 ... ['demo'], dest, links=[link_server], index=link_server+'index/', |
|
243 ... always_unzip=False) |
|
244 |
|
245 >>> ls(dest) |
|
246 - demo-0.3-py2.4.egg |
|
247 - demoneeded-1.1-py2.4.egg |
|
248 |
|
249 We can also set a default by calling the always_unzip function: |
|
250 |
|
251 >>> zc.buildout.easy_install.always_unzip(True) |
|
252 False |
|
253 |
|
254 The old default is returned: |
|
255 |
|
256 >>> rmdir(dest) |
|
257 >>> dest = tmpdir('sample-install') |
|
258 >>> ws = zc.buildout.easy_install.install( |
|
259 ... ['demo'], dest, links=[link_server], index=link_server+'index/') |
|
260 |
|
261 >>> ls(dest) |
|
262 d demo-0.3-py2.4.egg |
|
263 d demoneeded-1.1-py2.4.egg |
|
264 |
|
265 |
|
266 >>> zc.buildout.easy_install.always_unzip(False) |
|
267 True |
|
268 |
|
269 >>> rmdir(dest) |
|
270 >>> dest = tmpdir('sample-install') |
|
271 >>> ws = zc.buildout.easy_install.install( |
|
272 ... ['demo'], dest, links=[link_server], index=link_server+'index/') |
|
273 |
|
274 >>> ls(dest) |
|
275 - demo-0.3-py2.4.egg |
|
276 - demoneeded-1.1-py2.4.egg |
|
277 |
|
278 >>> rmdir(dest) |
|
279 >>> dest = tmpdir('sample-install') |
|
280 >>> ws = zc.buildout.easy_install.install( |
|
281 ... ['demo'], dest, links=[link_server], index=link_server+'index/', |
|
282 ... always_unzip=True) |
|
283 |
|
284 >>> ls(dest) |
|
285 d demo-0.3-py2.4.egg |
|
286 d demoneeded-1.1-py2.4.egg |
|
287 |
|
288 Specifying version information independent of requirements |
|
289 ---------------------------------------------------------- |
|
290 |
|
291 Sometimes it's useful to specify version information independent of |
|
292 normal requirements specifications. For example, a buildout may need |
|
293 to lock down a set of versions, without having to put put version |
|
294 numbers in setup files or part definitions. If a dictionary is passed |
|
295 to the install function, mapping project names to version numbers, |
|
296 then the versions numbers will be used. |
|
297 |
|
298 >>> ws = zc.buildout.easy_install.install( |
|
299 ... ['demo'], dest, links=[link_server], index=link_server+'index/', |
|
300 ... versions = dict(demo='0.2', demoneeded='1.0')) |
|
301 >>> [d.version for d in ws] |
|
302 ['0.2', '1.0'] |
|
303 |
|
304 In this example, we specified a version for demoneeded, even though we |
|
305 didn't define a requirement for it. The versions specified apply to |
|
306 dependencies as well as the specified requirements. |
|
307 |
|
308 If we specify a version that's incompatible with a requirement, then |
|
309 we'll get an error: |
|
310 |
|
311 >>> from zope.testing.loggingsupport import InstalledHandler |
|
312 >>> handler = InstalledHandler('zc.buildout.easy_install') |
|
313 >>> import logging |
|
314 >>> logging.getLogger('zc.buildout.easy_install').propagate = False |
|
315 |
|
316 >>> ws = zc.buildout.easy_install.install( |
|
317 ... ['demo >0.2'], dest, links=[link_server], |
|
318 ... index=link_server+'index/', |
|
319 ... versions = dict(demo='0.2', demoneeded='1.0')) |
|
320 Traceback (most recent call last): |
|
321 ... |
|
322 IncompatibleVersionError: Bad version 0.2 |
|
323 |
|
324 >>> print handler |
|
325 zc.buildout.easy_install DEBUG |
|
326 Installing 'demo >0.2'. |
|
327 zc.buildout.easy_install ERROR |
|
328 The version, 0.2, is not consistent with the requirement, 'demo>0.2'. |
|
329 |
|
330 >>> handler.clear() |
|
331 |
|
332 If no versions are specified, a debugging message will be output |
|
333 reporting that a version was picked automatically: |
|
334 |
|
335 >>> ws = zc.buildout.easy_install.install( |
|
336 ... ['demo'], dest, links=[link_server], index=link_server+'index/', |
|
337 ... ) |
|
338 |
|
339 >>> print handler |
|
340 zc.buildout.easy_install DEBUG |
|
341 Installing 'demo'. |
|
342 zc.buildout.easy_install DEBUG |
|
343 We have the best distribution that satisfies 'demo'. |
|
344 zc.buildout.easy_install DEBUG |
|
345 Picked: demo = 0.3 |
|
346 zc.buildout.easy_install DEBUG |
|
347 Getting required 'demoneeded' |
|
348 zc.buildout.easy_install DEBUG |
|
349 required by demo 0.3. |
|
350 zc.buildout.easy_install DEBUG |
|
351 We have the best distribution that satisfies 'demoneeded'. |
|
352 zc.buildout.easy_install DEBUG |
|
353 Picked: demoneeded = 1.1 |
|
354 |
|
355 >>> handler.uninstall() |
|
356 >>> logging.getLogger('zc.buildout.easy_install').propagate = True |
|
357 |
|
358 We can request that we get an error if versions are picked: |
|
359 |
|
360 >>> zc.buildout.easy_install.allow_picked_versions(False) |
|
361 True |
|
362 |
|
363 (The old setting is returned.) |
|
364 |
|
365 >>> ws = zc.buildout.easy_install.install( |
|
366 ... ['demo'], dest, links=[link_server], index=link_server+'index/', |
|
367 ... ) |
|
368 Traceback (most recent call last): |
|
369 ... |
|
370 UserError: Picked: demo = 0.3 |
|
371 |
|
372 >>> zc.buildout.easy_install.allow_picked_versions(True) |
|
373 False |
|
374 |
|
375 The function default_versions can be used to get and set default |
|
376 version information to be used when no version information is passes. |
|
377 If called with an argument, it sets the default versions: |
|
378 |
|
379 >>> zc.buildout.easy_install.default_versions(dict(demoneeded='1')) |
|
380 {} |
|
381 |
|
382 It always returns the previous default versions. If called without an |
|
383 argument, it simply returns the default versions without changing |
|
384 them: |
|
385 |
|
386 >>> zc.buildout.easy_install.default_versions() |
|
387 {'demoneeded': '1'} |
|
388 |
|
389 So with the default versions set, we'll get the requested version even |
|
390 if the versions option isn't used: |
|
391 |
|
392 >>> ws = zc.buildout.easy_install.install( |
|
393 ... ['demo'], dest, links=[link_server], index=link_server+'index/', |
|
394 ... ) |
|
395 |
|
396 >>> [d.version for d in ws] |
|
397 ['0.3', '1.0'] |
|
398 |
|
399 Of course, we can unset the default versions by passing an empty |
|
400 dictionary: |
|
401 |
|
402 >>> zc.buildout.easy_install.default_versions({}) |
|
403 {'demoneeded': '1'} |
|
404 |
|
405 >>> ws = zc.buildout.easy_install.install( |
|
406 ... ['demo'], dest, links=[link_server], index=link_server+'index/', |
|
407 ... ) |
|
408 |
|
409 >>> [d.version for d in ws] |
|
410 ['0.3', '1.1'] |
|
411 |
|
412 Dependencies in Site Packages |
|
413 ----------------------------- |
|
414 |
|
415 Paths outside of Python's standard library--or more precisely, those that are |
|
416 not included when Python is started with the -S argument--are loosely referred |
|
417 to as "site-packages" here. These site-packages are searched by default for |
|
418 distributions. This can be disabled, so that, for instance, a system Python |
|
419 can be used with buildout, cleaned of any packages installed by a user or |
|
420 system package manager. |
|
421 |
|
422 The default behavior can be controlled and introspected using |
|
423 zc.buildout.easy_install.include_site_packages. |
|
424 |
|
425 >>> zc.buildout.easy_install.include_site_packages() |
|
426 True |
|
427 |
|
428 Here's an example of using a Python executable that includes our dependencies. |
|
429 |
|
430 Our "py_path" will have the "demoneeded," and "demo" packages available. |
|
431 We'll simply be asking for "demoneeded" here, but without any external |
|
432 index or links. |
|
433 |
|
434 >>> from zc.buildout.tests import create_sample_sys_install |
|
435 >>> py_path, site_packages_path = make_py() |
|
436 >>> create_sample_sys_install(site_packages_path) |
|
437 |
|
438 >>> example_dest = tmpdir('site-packages-example-install') |
|
439 >>> workingset = zc.buildout.easy_install.install( |
|
440 ... ['demoneeded'], example_dest, links=[], executable=py_path, |
|
441 ... index=None) |
|
442 >>> [dist.project_name for dist in workingset] |
|
443 ['demoneeded'] |
|
444 |
|
445 That worked fine. Let's try again with site packages not allowed. We'll |
|
446 change the policy by changing the default. Notice that the function for |
|
447 changing the default value returns the previous value. |
|
448 |
|
449 >>> zc.buildout.easy_install.include_site_packages(False) |
|
450 True |
|
451 |
|
452 >>> zc.buildout.easy_install.include_site_packages() |
|
453 False |
|
454 |
|
455 >>> zc.buildout.easy_install.clear_index_cache() |
|
456 >>> rmdir(example_dest) |
|
457 >>> example_dest = tmpdir('site-packages-example-install') |
|
458 >>> workingset = zc.buildout.easy_install.install( |
|
459 ... ['demoneeded'], example_dest, links=[], executable=py_path, |
|
460 ... index=None) |
|
461 Traceback (most recent call last): |
|
462 ... |
|
463 MissingDistribution: Couldn't find a distribution for 'demoneeded'. |
|
464 >>> zc.buildout.easy_install.clear_index_cache() |
|
465 |
|
466 Now we'll reset the default. |
|
467 |
|
468 >>> zc.buildout.easy_install.include_site_packages(True) |
|
469 False |
|
470 |
|
471 >>> zc.buildout.easy_install.include_site_packages() |
|
472 True |
|
473 |
|
474 Dependency links |
|
475 ---------------- |
|
476 |
|
477 Setuptools allows metadata that describes where to search for package |
|
478 dependencies. This option is called dependency_links. Buildout has its |
|
479 own notion of where to look for dependencies, but it also uses the |
|
480 setup tools dependency_links information if it's available. |
|
481 |
|
482 Let's demo this by creating an egg that specifies dependency_links. |
|
483 |
|
484 To begin, let's create a new egg repository. This repository hold a |
|
485 newer version of the 'demoneeded' egg than the sample repository does. |
|
486 |
|
487 >>> repoloc = tmpdir('repo') |
|
488 >>> from zc.buildout.tests import create_egg |
|
489 >>> create_egg('demoneeded', '1.2', repoloc) |
|
490 >>> link_server2 = start_server(repoloc) |
|
491 |
|
492 Turn on logging on this server so that we can see when eggs are pulled |
|
493 from it. |
|
494 |
|
495 >>> get(link_server2 + 'enable_server_logging') |
|
496 GET 200 /enable_server_logging |
|
497 '' |
|
498 |
|
499 Now we can create an egg that specifies that its dependencies are |
|
500 found on this server. |
|
501 |
|
502 >>> repoloc = tmpdir('repo2') |
|
503 >>> create_egg('hasdeps', '1.0', repoloc, |
|
504 ... install_requires = "'demoneeded'", |
|
505 ... dependency_links = [link_server2]) |
|
506 |
|
507 Let's add the egg to another repository. |
|
508 |
|
509 >>> link_server3 = start_server(repoloc) |
|
510 |
|
511 Now let's install the egg. |
|
512 |
|
513 >>> example_dest = tmpdir('example-install') |
|
514 >>> workingset = zc.buildout.easy_install.install( |
|
515 ... ['hasdeps'], example_dest, |
|
516 ... links=[link_server3], index=link_server3+'index/') |
|
517 GET 200 / |
|
518 GET 200 /demoneeded-1.2-pyN.N.egg |
|
519 |
|
520 The server logs show that the dependency was retrieved from the server |
|
521 specified in the dependency_links. |
|
522 |
|
523 Now let's see what happens if we provide two different ways to retrieve |
|
524 the dependencies. |
|
525 |
|
526 >>> rmdir(example_dest) |
|
527 >>> example_dest = tmpdir('example-install') |
|
528 >>> workingset = zc.buildout.easy_install.install( |
|
529 ... ['hasdeps'], example_dest, index=link_server+'index/', |
|
530 ... links=[link_server, link_server3]) |
|
531 GET 200 / |
|
532 GET 200 /demoneeded-1.2-pyN.N.egg |
|
533 |
|
534 Once again the dependency is fetched from the logging server even |
|
535 though it is also available from the non-logging server. This is |
|
536 because the version on the logging server is newer and buildout |
|
537 normally chooses the newest egg available. |
|
538 |
|
539 If you wish to control where dependencies come from regardless of |
|
540 dependency_links setup metadata use the 'use_dependency_links' option |
|
541 to zc.buildout.easy_install.install(). |
|
542 |
|
543 >>> rmdir(example_dest) |
|
544 >>> example_dest = tmpdir('example-install') |
|
545 >>> workingset = zc.buildout.easy_install.install( |
|
546 ... ['hasdeps'], example_dest, index=link_server+'index/', |
|
547 ... links=[link_server, link_server3], |
|
548 ... use_dependency_links=False) |
|
549 |
|
550 Notice that this time the dependency egg is not fetched from the |
|
551 logging server. When you specify not to use dependency_links, eggs |
|
552 will only be searched for using the links you explicitly provide. |
|
553 |
|
554 Another way to control this option is with the |
|
555 zc.buildout.easy_install.use_dependency_links() function. This |
|
556 function sets the default behavior for the zc.buildout.easy_install() |
|
557 function. |
|
558 |
|
559 >>> zc.buildout.easy_install.use_dependency_links(False) |
|
560 True |
|
561 |
|
562 The function returns its previous setting. |
|
563 |
|
564 >>> rmdir(example_dest) |
|
565 >>> example_dest = tmpdir('example-install') |
|
566 >>> workingset = zc.buildout.easy_install.install( |
|
567 ... ['hasdeps'], example_dest, index=link_server+'index/', |
|
568 ... links=[link_server, link_server3]) |
|
569 |
|
570 It can be overridden by passing a keyword argument to the install |
|
571 function. |
|
572 |
|
573 >>> rmdir(example_dest) |
|
574 >>> example_dest = tmpdir('example-install') |
|
575 >>> workingset = zc.buildout.easy_install.install( |
|
576 ... ['hasdeps'], example_dest, index=link_server+'index/', |
|
577 ... links=[link_server, link_server3], |
|
578 ... use_dependency_links=True) |
|
579 GET 200 /demoneeded-1.2-pyN.N.egg |
|
580 |
|
581 To return the dependency_links behavior to normal call the function again. |
|
582 |
|
583 >>> zc.buildout.easy_install.use_dependency_links(True) |
|
584 False |
|
585 >>> rmdir(example_dest) |
|
586 >>> example_dest = tmpdir('example-install') |
|
587 >>> workingset = zc.buildout.easy_install.install( |
|
588 ... ['hasdeps'], example_dest, index=link_server+'index/', |
|
589 ... links=[link_server, link_server3]) |
|
590 GET 200 /demoneeded-1.2-pyN.N.egg |
|
591 |
|
592 |
|
593 Script generation |
|
594 ----------------- |
|
595 |
|
596 The easy_install module provides support for creating scripts from eggs. |
|
597 It provides two competing functions. One, ``scripts``, is a |
|
598 well-established approach to generating reliable scripts with a "clean" |
|
599 Python--e.g., one that does not have any packages in its site-packages. |
|
600 The other, ``sitepackage_safe_scripts``, is newer, a bit trickier, and is |
|
601 designed to work with a Python that has code in its site-packages, such |
|
602 as a system Python. |
|
603 |
|
604 Both are similar to setuptools except that they provides facilities for |
|
605 baking a script's path into the script. This has two advantages: |
|
606 |
|
607 - The eggs to be used by a script are not chosen at run time, making |
|
608 startup faster and, more importantly, deterministic. |
|
609 |
|
610 - The script doesn't have to import pkg_resources because the logic that |
|
611 pkg_resources would execute at run time is executed at script-creation |
|
612 time. (There is an exception in ``sitepackage_safe_scripts`` if you |
|
613 want to have your Python's site packages available, as discussed |
|
614 below, but even in that case pkg_resources is only partially |
|
615 activated, which can be a significant time savings.) |
|
616 |
|
617 |
|
618 The ``scripts`` function |
|
619 ~~~~~~~~~~~~~~~~~~~~~~~~ |
|
620 |
|
621 The ``scripts`` function is the first way to generate scripts that we'll |
|
622 examine. It is the earlier approach that the package offered. Let's |
|
623 create a destination directory for it to place them in: |
|
624 |
|
625 >>> bin = tmpdir('bin') |
|
626 |
|
627 Now, we'll use the scripts function to generate scripts in this directory |
|
628 from the demo egg: |
|
629 |
|
630 >>> import sys |
|
631 >>> scripts = zc.buildout.easy_install.scripts( |
|
632 ... ['demo'], ws, sys.executable, bin) |
|
633 |
|
634 the four arguments we passed were: |
|
635 |
|
636 1. A sequence of distribution requirements. These are of the same |
|
637 form as setuptools requirements. Here we passed a single |
|
638 requirement, for the version 0.1 demo distribution. |
|
639 |
|
640 2. A working set, |
|
641 |
|
642 3. The Python executable to use, and |
|
643 |
|
644 3. The destination directory. |
|
645 |
|
646 The bin directory now contains a generated script: |
|
647 |
|
648 >>> ls(bin) |
|
649 - demo |
|
650 |
|
651 The return value is a list of the scripts generated: |
|
652 |
|
653 >>> import os, sys |
|
654 >>> if sys.platform == 'win32': |
|
655 ... scripts == [os.path.join(bin, 'demo.exe'), |
|
656 ... os.path.join(bin, 'demo-script.py')] |
|
657 ... else: |
|
658 ... scripts == [os.path.join(bin, 'demo')] |
|
659 True |
|
660 |
|
661 Note that in Windows, 2 files are generated for each script. A script |
|
662 file, ending in '-script.py', and an exe file that allows the script |
|
663 to be invoked directly without having to specify the Python |
|
664 interpreter and without having to provide a '.py' suffix. |
|
665 |
|
666 The demo script run the entry point defined in the demo egg: |
|
667 |
|
668 >>> cat(bin, 'demo') # doctest: +NORMALIZE_WHITESPACE |
|
669 #!/usr/local/bin/python2.4 |
|
670 <BLANKLINE> |
|
671 import sys |
|
672 sys.path[0:0] = [ |
|
673 '/sample-install/demo-0.3-py2.4.egg', |
|
674 '/sample-install/demoneeded-1.1-py2.4.egg', |
|
675 ] |
|
676 <BLANKLINE> |
|
677 import eggrecipedemo |
|
678 <BLANKLINE> |
|
679 if __name__ == '__main__': |
|
680 eggrecipedemo.main() |
|
681 |
|
682 Some things to note: |
|
683 |
|
684 - The demo and demoneeded eggs are added to the beginning of sys.path. |
|
685 |
|
686 - The module for the script entry point is imported and the entry |
|
687 point, in this case, 'main', is run. |
|
688 |
|
689 Rather than requirement strings, you can pass tuples containing 3 |
|
690 strings: |
|
691 |
|
692 - A script name, |
|
693 |
|
694 - A module, |
|
695 |
|
696 - An attribute expression for an entry point within the module. |
|
697 |
|
698 For example, we could have passed entry point information directly |
|
699 rather than passing a requirement: |
|
700 |
|
701 >>> scripts = zc.buildout.easy_install.scripts( |
|
702 ... [('demo', 'eggrecipedemo', 'main')], |
|
703 ... ws, sys.executable, bin) |
|
704 |
|
705 >>> cat(bin, 'demo') # doctest: +NORMALIZE_WHITESPACE |
|
706 #!/usr/local/bin/python2.4 |
|
707 <BLANKLINE> |
|
708 import sys |
|
709 sys.path[0:0] = [ |
|
710 '/sample-install/demo-0.3-py2.4.egg', |
|
711 '/sample-install/demoneeded-1.1-py2.4.egg', |
|
712 ] |
|
713 <BLANKLINE> |
|
714 import eggrecipedemo |
|
715 <BLANKLINE> |
|
716 if __name__ == '__main__': |
|
717 eggrecipedemo.main() |
|
718 |
|
719 Passing entry-point information directly is handy when using eggs (or |
|
720 distributions) that don't declare their entry points, such as |
|
721 distributions that aren't based on setuptools. |
|
722 |
|
723 The interpreter keyword argument can be used to generate a script that can |
|
724 be used to invoke the Python interactive interpreter with the path set |
|
725 based on the working set. This generated script can also be used to |
|
726 run other scripts with the path set on the working set: |
|
727 |
|
728 >>> scripts = zc.buildout.easy_install.scripts( |
|
729 ... ['demo'], ws, sys.executable, bin, interpreter='py') |
|
730 |
|
731 |
|
732 >>> ls(bin) |
|
733 - demo |
|
734 - py |
|
735 |
|
736 >>> if sys.platform == 'win32': |
|
737 ... scripts == [os.path.join(bin, 'demo.exe'), |
|
738 ... os.path.join(bin, 'demo-script.py'), |
|
739 ... os.path.join(bin, 'py.exe'), |
|
740 ... os.path.join(bin, 'py-script.py')] |
|
741 ... else: |
|
742 ... scripts == [os.path.join(bin, 'demo'), |
|
743 ... os.path.join(bin, 'py')] |
|
744 True |
|
745 |
|
746 The py script simply runs the Python interactive interpreter with |
|
747 the path set: |
|
748 |
|
749 >>> cat(bin, 'py') # doctest: +NORMALIZE_WHITESPACE |
|
750 #!/usr/local/bin/python2.4 |
|
751 <BLANKLINE> |
|
752 import sys |
|
753 <BLANKLINE> |
|
754 sys.path[0:0] = [ |
|
755 '/sample-install/demo-0.3-pyN.N.egg', |
|
756 '/sample-install/demoneeded-1.1-pyN.N.egg', |
|
757 ] |
|
758 <BLANKLINE> |
|
759 _interactive = True |
|
760 if len(sys.argv) > 1: |
|
761 _options, _args = __import__("getopt").getopt(sys.argv[1:], 'ic:m:') |
|
762 _interactive = False |
|
763 for (_opt, _val) in _options: |
|
764 if _opt == '-i': |
|
765 _interactive = True |
|
766 elif _opt == '-c': |
|
767 exec _val |
|
768 elif _opt == '-m': |
|
769 sys.argv[1:] = _args |
|
770 _args = [] |
|
771 __import__("runpy").run_module( |
|
772 _val, {}, "__main__", alter_sys=True) |
|
773 <BLANKLINE> |
|
774 if _args: |
|
775 sys.argv[:] = _args |
|
776 __file__ = _args[0] |
|
777 del _options, _args |
|
778 execfile(__file__) |
|
779 <BLANKLINE> |
|
780 if _interactive: |
|
781 del _interactive |
|
782 __import__("code").interact(banner="", local=globals()) |
|
783 |
|
784 If invoked with a script name and arguments, it will run that script, instead. |
|
785 |
|
786 >>> write('ascript', ''' |
|
787 ... "demo doc" |
|
788 ... print sys.argv |
|
789 ... print (__name__, __file__, __doc__) |
|
790 ... ''') |
|
791 >>> print system(join(bin, 'py')+' ascript a b c'), |
|
792 ['ascript', 'a', 'b', 'c'] |
|
793 ('__main__', 'ascript', 'demo doc') |
|
794 |
|
795 For Python 2.5 and higher, you can also use the -m option to run a |
|
796 module: |
|
797 |
|
798 >>> print system(join(bin, 'py')+' -m pdb'), |
|
799 usage: pdb.py scriptfile [arg] ... |
|
800 |
|
801 >>> print system(join(bin, 'py')+' -m pdb what'), |
|
802 Error: what does not exist |
|
803 |
|
804 An additional argument can be passed to define which scripts to install |
|
805 and to provide script names. The argument is a dictionary mapping |
|
806 original script names to new script names. |
|
807 |
|
808 >>> bin = tmpdir('bin2') |
|
809 >>> scripts = zc.buildout.easy_install.scripts( |
|
810 ... ['demo'], ws, sys.executable, bin, dict(demo='run')) |
|
811 |
|
812 >>> if sys.platform == 'win32': |
|
813 ... scripts == [os.path.join(bin, 'run.exe'), |
|
814 ... os.path.join(bin, 'run-script.py')] |
|
815 ... else: |
|
816 ... scripts == [os.path.join(bin, 'run')] |
|
817 True |
|
818 >>> ls(bin) |
|
819 - run |
|
820 |
|
821 >>> print system(os.path.join(bin, 'run')), |
|
822 3 1 |
|
823 |
|
824 The ``scripts`` function: Including extra paths in scripts |
|
825 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
826 |
|
827 We can pass a keyword argument, extra paths, to cause additional paths |
|
828 to be included in the a generated script: |
|
829 |
|
830 >>> foo = tmpdir('foo') |
|
831 >>> scripts = zc.buildout.easy_install.scripts( |
|
832 ... ['demo'], ws, sys.executable, bin, dict(demo='run'), |
|
833 ... extra_paths=[foo]) |
|
834 |
|
835 >>> cat(bin, 'run') # doctest: +NORMALIZE_WHITESPACE |
|
836 #!/usr/local/bin/python2.4 |
|
837 <BLANKLINE> |
|
838 import sys |
|
839 sys.path[0:0] = [ |
|
840 '/sample-install/demo-0.3-py2.4.egg', |
|
841 '/sample-install/demoneeded-1.1-py2.4.egg', |
|
842 '/foo', |
|
843 ] |
|
844 <BLANKLINE> |
|
845 import eggrecipedemo |
|
846 <BLANKLINE> |
|
847 if __name__ == '__main__': |
|
848 eggrecipedemo.main() |
|
849 |
|
850 The ``scripts`` function: Providing script arguments |
|
851 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
852 |
|
853 An "argument" keyword argument can be used to pass arguments to an |
|
854 entry point. The value passed is a source string to be placed between the |
|
855 parentheses in the call: |
|
856 |
|
857 >>> scripts = zc.buildout.easy_install.scripts( |
|
858 ... ['demo'], ws, sys.executable, bin, dict(demo='run'), |
|
859 ... arguments='1, 2') |
|
860 |
|
861 >>> cat(bin, 'run') # doctest: +NORMALIZE_WHITESPACE |
|
862 #!/usr/local/bin/python2.4 |
|
863 import sys |
|
864 sys.path[0:0] = [ |
|
865 '/sample-install/demo-0.3-py2.4.egg', |
|
866 '/sample-install/demoneeded-1.1-py2.4.egg', |
|
867 ] |
|
868 <BLANKLINE> |
|
869 import eggrecipedemo |
|
870 <BLANKLINE> |
|
871 if __name__ == '__main__': |
|
872 eggrecipedemo.main(1, 2) |
|
873 |
|
874 The ``scripts`` function: Passing initialization code |
|
875 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
876 |
|
877 You can also pass script initialization code: |
|
878 |
|
879 >>> scripts = zc.buildout.easy_install.scripts( |
|
880 ... ['demo'], ws, sys.executable, bin, dict(demo='run'), |
|
881 ... arguments='1, 2', |
|
882 ... initialization='import os\nos.chdir("foo")') |
|
883 |
|
884 >>> cat(bin, 'run') # doctest: +NORMALIZE_WHITESPACE |
|
885 #!/usr/local/bin/python2.4 |
|
886 import sys |
|
887 sys.path[0:0] = [ |
|
888 '/sample-install/demo-0.3-py2.4.egg', |
|
889 '/sample-install/demoneeded-1.1-py2.4.egg', |
|
890 ] |
|
891 <BLANKLINE> |
|
892 import os |
|
893 os.chdir("foo") |
|
894 <BLANKLINE> |
|
895 import eggrecipedemo |
|
896 <BLANKLINE> |
|
897 if __name__ == '__main__': |
|
898 eggrecipedemo.main(1, 2) |
|
899 |
|
900 The ``scripts`` function: Relative paths |
|
901 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
902 |
|
903 Sometimes, you want to be able to move a buildout directory around and |
|
904 have scripts still work without having to rebuild them. We can |
|
905 control this using the relative_paths option to install. You need |
|
906 to pass a common base directory of the scripts and eggs: |
|
907 |
|
908 >>> bo = tmpdir('bo') |
|
909 >>> ba = tmpdir('ba') |
|
910 >>> mkdir(bo, 'eggs') |
|
911 >>> mkdir(bo, 'bin') |
|
912 >>> mkdir(bo, 'other') |
|
913 |
|
914 >>> ws = zc.buildout.easy_install.install( |
|
915 ... ['demo'], join(bo, 'eggs'), links=[link_server], |
|
916 ... index=link_server+'index/') |
|
917 |
|
918 >>> scripts = zc.buildout.easy_install.scripts( |
|
919 ... ['demo'], ws, sys.executable, join(bo, 'bin'), dict(demo='run'), |
|
920 ... extra_paths=[ba, join(bo, 'bar')], |
|
921 ... interpreter='py', |
|
922 ... relative_paths=bo) |
|
923 |
|
924 >>> cat(bo, 'bin', 'run') # doctest: +NORMALIZE_WHITESPACE |
|
925 #!/usr/local/bin/python2.4 |
|
926 <BLANKLINE> |
|
927 import os |
|
928 <BLANKLINE> |
|
929 join = os.path.join |
|
930 base = os.path.dirname(os.path.abspath(os.path.realpath(__file__))) |
|
931 base = os.path.dirname(base) |
|
932 <BLANKLINE> |
|
933 import sys |
|
934 sys.path[0:0] = [ |
|
935 join(base, 'eggs/demo-0.3-pyN.N.egg'), |
|
936 join(base, 'eggs/demoneeded-1.1-pyN.N.egg'), |
|
937 '/ba', |
|
938 join(base, 'bar'), |
|
939 ] |
|
940 <BLANKLINE> |
|
941 import eggrecipedemo |
|
942 <BLANKLINE> |
|
943 if __name__ == '__main__': |
|
944 eggrecipedemo.main() |
|
945 |
|
946 Note that the extra path we specified that was outside the directory |
|
947 passed as relative_paths wasn't converted to a relative path. |
|
948 |
|
949 Of course, running the script works: |
|
950 |
|
951 >>> print system(join(bo, 'bin', 'run')), |
|
952 3 1 |
|
953 |
|
954 We specified an interpreter and its paths are adjusted too: |
|
955 |
|
956 >>> cat(bo, 'bin', 'py') # doctest: +NORMALIZE_WHITESPACE |
|
957 #!/usr/local/bin/python2.4 |
|
958 <BLANKLINE> |
|
959 import os |
|
960 <BLANKLINE> |
|
961 join = os.path.join |
|
962 base = os.path.dirname(os.path.abspath(os.path.realpath(__file__))) |
|
963 base = os.path.dirname(base) |
|
964 <BLANKLINE> |
|
965 import sys |
|
966 <BLANKLINE> |
|
967 sys.path[0:0] = [ |
|
968 join(base, 'eggs/demo-0.3-pyN.N.egg'), |
|
969 join(base, 'eggs/demoneeded-1.1-pyN.N.egg'), |
|
970 '/ba', |
|
971 join(base, 'bar'), |
|
972 ] |
|
973 <BLANKLINE> |
|
974 _interactive = True |
|
975 if len(sys.argv) > 1: |
|
976 _options, _args = __import__("getopt").getopt(sys.argv[1:], 'ic:m:') |
|
977 _interactive = False |
|
978 for (_opt, _val) in _options: |
|
979 if _opt == '-i': |
|
980 _interactive = True |
|
981 elif _opt == '-c': |
|
982 exec _val |
|
983 elif _opt == '-m': |
|
984 sys.argv[1:] = _args |
|
985 _args = [] |
|
986 __import__("runpy").run_module( |
|
987 _val, {}, "__main__", alter_sys=True) |
|
988 <BLANKLINE> |
|
989 if _args: |
|
990 sys.argv[:] = _args |
|
991 __file__ = _args[0] |
|
992 del _options, _args |
|
993 execfile(__file__) |
|
994 <BLANKLINE> |
|
995 if _interactive: |
|
996 del _interactive |
|
997 __import__("code").interact(banner="", local=globals()) |
|
998 |
|
999 The ``sitepackage_safe_scripts`` function |
|
1000 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
1001 |
|
1002 The newer function for creating scripts is ``sitepackage_safe_scripts``. |
|
1003 It has the same basic functionality as the ``scripts`` function: it can |
|
1004 create scripts to run arbitrary entry points, and to run a Python |
|
1005 interpreter. The following are the differences from a user's |
|
1006 perspective. |
|
1007 |
|
1008 - It can be used safely with a Python that has packages installed itself, |
|
1009 such as a system-installed Python. |
|
1010 |
|
1011 - In contrast to the interpreter generated by the ``scripts`` method, which |
|
1012 supports only a small subset of the usual Python executable's options, |
|
1013 the interpreter generated by ``sitepackage_safe_scripts`` supports all |
|
1014 of them. This makes it possible to use as full Python replacement for |
|
1015 scripts that need the distributions specified in your buildout. |
|
1016 |
|
1017 - Both the interpreter and the entry point scripts allow you to include the |
|
1018 site packages, and/or the sitecustomize, of the Python executable, if |
|
1019 desired. |
|
1020 |
|
1021 It works by creating site.py and sitecustomize.py files that set up the |
|
1022 desired paths and initialization. These must be placed within an otherwise |
|
1023 empty directory. Typically this is in a recipe's parts directory. |
|
1024 |
|
1025 Here's the simplest example, building an interpreter script. |
|
1026 |
|
1027 >>> interpreter_dir = tmpdir('interpreter') |
|
1028 >>> interpreter_parts_dir = os.path.join( |
|
1029 ... interpreter_dir, 'parts', 'interpreter') |
|
1030 >>> interpreter_bin_dir = os.path.join(interpreter_dir, 'bin') |
|
1031 >>> mkdir(interpreter_bin_dir) |
|
1032 >>> mkdir(interpreter_dir, 'eggs') |
|
1033 >>> mkdir(interpreter_dir, 'parts') |
|
1034 >>> mkdir(interpreter_parts_dir) |
|
1035 |
|
1036 >>> ws = zc.buildout.easy_install.install( |
|
1037 ... ['demo'], join(interpreter_dir, 'eggs'), links=[link_server], |
|
1038 ... index=link_server+'index/') |
|
1039 >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( |
|
1040 ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, |
|
1041 ... interpreter='py') |
|
1042 |
|
1043 Depending on whether the machine being used is running Windows or not, this |
|
1044 produces either three or four files. In both cases, we have site.py and |
|
1045 sitecustomize.py generated in the parts/interpreter directory. For Windows, |
|
1046 we have py.exe and py-script.py; for other operating systems, we have py. |
|
1047 |
|
1048 >>> sitecustomize_path = os.path.join( |
|
1049 ... interpreter_parts_dir, 'sitecustomize.py') |
|
1050 >>> site_path = os.path.join(interpreter_parts_dir, 'site.py') |
|
1051 >>> interpreter_path = os.path.join(interpreter_bin_dir, 'py') |
|
1052 >>> if sys.platform == 'win32': |
|
1053 ... py_path = os.path.join(interpreter_bin_dir, 'py-script.py') |
|
1054 ... expected = [sitecustomize_path, |
|
1055 ... site_path, |
|
1056 ... os.path.join(interpreter_bin_dir, 'py.exe'), |
|
1057 ... py_path] |
|
1058 ... else: |
|
1059 ... py_path = interpreter_path |
|
1060 ... expected = [sitecustomize_path, site_path, py_path] |
|
1061 ... |
|
1062 >>> assert generated == expected, repr((generated, expected)) |
|
1063 |
|
1064 We didn't ask for any initialization, and we didn't ask to use the underlying |
|
1065 sitecustomization, so sitecustomize.py is empty. |
|
1066 |
|
1067 >>> cat(sitecustomize_path) |
|
1068 |
|
1069 The interpreter script is simple. It puts the directory with the |
|
1070 site.py and sitecustomize.py on the PYTHONPATH and (re)starts Python. |
|
1071 |
|
1072 >>> cat(py_path) |
|
1073 #!/usr/bin/python -S |
|
1074 import os |
|
1075 import sys |
|
1076 <BLANKLINE> |
|
1077 argv = [sys.executable] + sys.argv[1:] |
|
1078 environ = os.environ.copy() |
|
1079 path = '/interpreter/parts/interpreter' |
|
1080 if environ.get('PYTHONPATH'): |
|
1081 path = os.pathsep.join([path, environ['PYTHONPATH']]) |
|
1082 environ['PYTHONPATH'] = path |
|
1083 os.execve(sys.executable, argv, environ) |
|
1084 |
|
1085 The site.py file is a modified version of the underlying Python's site.py. |
|
1086 The most important modification is that it has a different version of the |
|
1087 addsitepackages function. It sets up the Python path, similarly to the |
|
1088 behavior of the function it replaces. The following shows the part that |
|
1089 buildout inserts, in the simplest case. |
|
1090 |
|
1091 >>> sys.stdout.write('#\n'); cat(site_path) |
|
1092 ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE |
|
1093 #... |
|
1094 def addsitepackages(known_paths): |
|
1095 """Add site packages, as determined by zc.buildout. |
|
1096 <BLANKLINE> |
|
1097 See original_addsitepackages, below, for the original version.""" |
|
1098 buildout_paths = [ |
|
1099 '/interpreter/eggs/demo-0.3-pyN.N.egg', |
|
1100 '/interpreter/eggs/demoneeded-1.1-pyN.N.egg' |
|
1101 ] |
|
1102 for path in buildout_paths: |
|
1103 sitedir, sitedircase = makepath(path) |
|
1104 if not sitedircase in known_paths and os.path.exists(sitedir): |
|
1105 sys.path.append(sitedir) |
|
1106 known_paths.add(sitedircase) |
|
1107 return known_paths |
|
1108 <BLANKLINE> |
|
1109 def original_addsitepackages(known_paths):... |
|
1110 |
|
1111 Here are some examples of the interpreter in use. |
|
1112 |
|
1113 >>> print call_py(interpreter_path, "print 16+26") |
|
1114 42 |
|
1115 <BLANKLINE> |
|
1116 >>> res = call_py(interpreter_path, "import sys; print sys.path") |
|
1117 >>> print res # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE |
|
1118 ['', |
|
1119 '/interpreter/parts/interpreter', |
|
1120 ..., |
|
1121 '/interpreter/eggs/demo-0.3-pyN.N.egg', |
|
1122 '/interpreter/eggs/demoneeded-1.1-pyN.N.egg'] |
|
1123 <BLANKLINE> |
|
1124 >>> clean_paths = eval(res.strip()) # This is used later for comparison. |
|
1125 |
|
1126 If you provide initialization, it goes in sitecustomize.py. |
|
1127 |
|
1128 >>> def reset_interpreter(): |
|
1129 ... # This is necessary because, in our tests, the timestamps of the |
|
1130 ... # .pyc files are not outdated when we want them to be. |
|
1131 ... rmdir(interpreter_bin_dir) |
|
1132 ... mkdir(interpreter_bin_dir) |
|
1133 ... rmdir(interpreter_parts_dir) |
|
1134 ... mkdir(interpreter_parts_dir) |
|
1135 ... |
|
1136 >>> reset_interpreter() |
|
1137 |
|
1138 >>> initialization_string = """\ |
|
1139 ... import os |
|
1140 ... os.environ['FOO'] = 'bar baz bing shazam'""" |
|
1141 >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( |
|
1142 ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, |
|
1143 ... interpreter='py', initialization=initialization_string) |
|
1144 >>> cat(sitecustomize_path) |
|
1145 import os |
|
1146 os.environ['FOO'] = 'bar baz bing shazam' |
|
1147 >>> print call_py(interpreter_path, "import os; print os.environ['FOO']") |
|
1148 bar baz bing shazam |
|
1149 <BLANKLINE> |
|
1150 |
|
1151 If you use relative paths, this affects the interpreter and site.py. (This is |
|
1152 again the UNIX version; the Windows version uses subprocess instead of |
|
1153 os.execve.) |
|
1154 |
|
1155 >>> reset_interpreter() |
|
1156 >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( |
|
1157 ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, |
|
1158 ... interpreter='py', relative_paths=interpreter_dir) |
|
1159 >>> cat(py_path) |
|
1160 #!/usr/bin/python -S |
|
1161 import os |
|
1162 import sys |
|
1163 <BLANKLINE> |
|
1164 join = os.path.join |
|
1165 base = os.path.dirname(os.path.abspath(os.path.realpath(__file__))) |
|
1166 base = os.path.dirname(base) |
|
1167 <BLANKLINE> |
|
1168 argv = [sys.executable] + sys.argv[1:] |
|
1169 environ = os.environ.copy() |
|
1170 path = join(base, 'parts/interpreter') |
|
1171 if environ.get('PYTHONPATH'): |
|
1172 path = os.pathsep.join([path, environ['PYTHONPATH']]) |
|
1173 environ['PYTHONPATH'] = path |
|
1174 os.execve(sys.executable, argv, environ) |
|
1175 |
|
1176 For site.py, we again show only the pertinent parts. Notice that the egg |
|
1177 paths join a base to a path, as with the use of this argument in the |
|
1178 ``scripts`` function. |
|
1179 |
|
1180 >>> sys.stdout.write('#\n'); cat(site_path) # doctest: +ELLIPSIS |
|
1181 #... |
|
1182 def addsitepackages(known_paths): |
|
1183 """Add site packages, as determined by zc.buildout. |
|
1184 <BLANKLINE> |
|
1185 See original_addsitepackages, below, for the original version.""" |
|
1186 join = os.path.join |
|
1187 base = os.path.dirname(os.path.abspath(os.path.realpath(__file__))) |
|
1188 base = os.path.dirname(base) |
|
1189 base = os.path.dirname(base) |
|
1190 buildout_paths = [ |
|
1191 join(base, 'eggs/demo-0.3-pyN.N.egg'), |
|
1192 join(base, 'eggs/demoneeded-1.1-pyN.N.egg') |
|
1193 ]... |
|
1194 |
|
1195 The paths resolve in practice as you would expect. |
|
1196 |
|
1197 >>> print call_py(interpreter_path, |
|
1198 ... "import sys, pprint; pprint.pprint(sys.path)") |
|
1199 ... # doctest: +ELLIPSIS |
|
1200 ['', |
|
1201 '/interpreter/parts/interpreter', |
|
1202 ..., |
|
1203 '/interpreter/eggs/demo-0.3-pyN.N.egg', |
|
1204 '/interpreter/eggs/demoneeded-1.1-pyN.N.egg'] |
|
1205 <BLANKLINE> |
|
1206 |
|
1207 The ``extra_paths`` argument affects the path in site.py. Notice that |
|
1208 /interpreter/other is added after the eggs. |
|
1209 |
|
1210 >>> reset_interpreter() |
|
1211 >>> mkdir(interpreter_dir, 'other') |
|
1212 >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( |
|
1213 ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, |
|
1214 ... interpreter='py', extra_paths=[join(interpreter_dir, 'other')]) |
|
1215 >>> sys.stdout.write('#\n'); cat(site_path) # doctest: +ELLIPSIS |
|
1216 #... |
|
1217 def addsitepackages(known_paths): |
|
1218 """Add site packages, as determined by zc.buildout. |
|
1219 <BLANKLINE> |
|
1220 See original_addsitepackages, below, for the original version.""" |
|
1221 buildout_paths = [ |
|
1222 '/interpreter/eggs/demo-0.3-pyN.N.egg', |
|
1223 '/interpreter/eggs/demoneeded-1.1-pyN.N.egg', |
|
1224 '/interpreter/other' |
|
1225 ]... |
|
1226 |
|
1227 >>> print call_py(interpreter_path, |
|
1228 ... "import sys, pprint; pprint.pprint(sys.path)") |
|
1229 ... # doctest: +ELLIPSIS |
|
1230 ['', |
|
1231 '/interpreter/parts/interpreter', |
|
1232 ..., |
|
1233 '/interpreter/eggs/demo-0.3-pyN.N.egg', |
|
1234 '/interpreter/eggs/demoneeded-1.1-pyN.N.egg', |
|
1235 '/interpreter/other'] |
|
1236 <BLANKLINE> |
|
1237 |
|
1238 The ``sitepackage_safe_scripts`` function: using site-packages |
|
1239 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
1240 |
|
1241 The ``sitepackage_safe_scripts`` function supports including site |
|
1242 packages. This has some advantages and some serious dangers. |
|
1243 |
|
1244 A typical reason to include site-packages is that it is easier to |
|
1245 install one or more dependencies in your Python than it is with |
|
1246 buildout. Some packages, such as lxml or Python PostgreSQL integration, |
|
1247 have dependencies that can be much easier to build and/or install using |
|
1248 other mechanisms, such as your operating system's package manager. By |
|
1249 installing some core packages into your Python's site-packages, this can |
|
1250 significantly simplify some application installations. |
|
1251 |
|
1252 However, doing this has a significant danger. One of the primary goals |
|
1253 of buildout is to provide repeatability. Some packages (one of the |
|
1254 better known Python openid packages, for instance) change their behavior |
|
1255 depending on what packages are available. If Python curl bindings are |
|
1256 available, these may be preferred by the library. If a certain XML |
|
1257 package is installed, it may be preferred by the library. These hidden |
|
1258 choices may cause small or large behavior differences. The fact that |
|
1259 they can be rarely encountered can actually make it worse: you forget |
|
1260 that this might be a problem, and debugging the differences can be |
|
1261 difficult. If you allow site-packages to be included in your buildout, |
|
1262 and the Python you use is not managed precisely by your application (for |
|
1263 instance, it is a system Python), you open yourself up to these |
|
1264 possibilities. Don't be unaware of the dangers. |
|
1265 |
|
1266 That explained, let's see how it works. If you don't use namespace packages, |
|
1267 this is very straightforward. |
|
1268 |
|
1269 >>> reset_interpreter() |
|
1270 >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( |
|
1271 ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, |
|
1272 ... interpreter='py', include_site_packages=True) |
|
1273 >>> sys.stdout.write('#\n'); cat(site_path) |
|
1274 ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE |
|
1275 #... |
|
1276 def addsitepackages(known_paths): |
|
1277 """Add site packages, as determined by zc.buildout. |
|
1278 <BLANKLINE> |
|
1279 See original_addsitepackages, below, for the original version.""" |
|
1280 setuptools_path = None |
|
1281 buildout_paths = [ |
|
1282 '/interpreter/eggs/demo-0.3-pyN.N.egg', |
|
1283 '/interpreter/eggs/demoneeded-1.1-pyN.N.egg' |
|
1284 ] |
|
1285 for path in buildout_paths: |
|
1286 sitedir, sitedircase = makepath(path) |
|
1287 if not sitedircase in known_paths and os.path.exists(sitedir): |
|
1288 sys.path.append(sitedir) |
|
1289 known_paths.add(sitedircase) |
|
1290 sys.__egginsert = len(buildout_paths) # Support distribute. |
|
1291 original_paths = [ |
|
1292 ... |
|
1293 ] |
|
1294 for path in original_paths: |
|
1295 if path == setuptools_path or path not in known_paths: |
|
1296 addsitedir(path, known_paths) |
|
1297 return known_paths |
|
1298 <BLANKLINE> |
|
1299 def original_addsitepackages(known_paths):... |
|
1300 |
|
1301 It simply adds the original paths using addsitedir after the code to add the |
|
1302 buildout paths. |
|
1303 |
|
1304 Here's an example of the new script in use. Other documents and tests in |
|
1305 this package give the feature a more thorough workout, but this should |
|
1306 give you an idea of the feature. |
|
1307 |
|
1308 >>> res = call_py(interpreter_path, "import sys; print sys.path") |
|
1309 >>> print res # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE |
|
1310 ['', |
|
1311 '/interpreter/parts/interpreter', |
|
1312 ..., |
|
1313 '/interpreter/eggs/demo-0.3-py2.4.egg', |
|
1314 '/interpreter/eggs/demoneeded-1.1-py2.4.egg', |
|
1315 ...] |
|
1316 <BLANKLINE> |
|
1317 |
|
1318 The clean_paths gathered earlier is a subset of this full list of paths. |
|
1319 |
|
1320 >>> full_paths = eval(res.strip()) |
|
1321 >>> len(clean_paths) < len(full_paths) |
|
1322 True |
|
1323 >>> set(os.path.normpath(p) for p in clean_paths).issubset( |
|
1324 ... os.path.normpath(p) for p in full_paths) |
|
1325 True |
|
1326 |
|
1327 Unfortunately, because of how setuptools namespace packages are implemented |
|
1328 differently for operating system packages (debs or rpms) as opposed to |
|
1329 standard setuptools installation, there's a slightly trickier dance if you |
|
1330 use them. To show this we'll needs some extra eggs that use namespaces. |
|
1331 We'll use the ``tellmy.fortune`` package, which we'll need to make an initial |
|
1332 call to another text fixture to create. |
|
1333 |
|
1334 >>> from zc.buildout.tests import create_sample_namespace_eggs |
|
1335 >>> namespace_eggs = tmpdir('namespace_eggs') |
|
1336 >>> create_sample_namespace_eggs(namespace_eggs) |
|
1337 |
|
1338 >>> reset_interpreter() |
|
1339 >>> ws = zc.buildout.easy_install.install( |
|
1340 ... ['demo', 'tellmy.fortune'], join(interpreter_dir, 'eggs'), |
|
1341 ... links=[link_server, namespace_eggs], index=link_server+'index/') |
|
1342 >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( |
|
1343 ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, |
|
1344 ... interpreter='py', include_site_packages=True) |
|
1345 >>> sys.stdout.write('#\n'); cat(site_path) |
|
1346 ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE |
|
1347 #... |
|
1348 def addsitepackages(known_paths): |
|
1349 """Add site packages, as determined by zc.buildout. |
|
1350 <BLANKLINE> |
|
1351 See original_addsitepackages, below, for the original version.""" |
|
1352 setuptools_path = '...setuptools...' |
|
1353 sys.path.append(setuptools_path) |
|
1354 known_paths.add(os.path.normcase(setuptools_path)) |
|
1355 import pkg_resources |
|
1356 buildout_paths = [ |
|
1357 '/interpreter/eggs/demo-0.3-pyN.N.egg', |
|
1358 '/interpreter/eggs/tellmy.fortune-1.0-pyN.N.egg', |
|
1359 '...setuptools...', |
|
1360 '/interpreter/eggs/demoneeded-1.1-pyN.N.egg' |
|
1361 ] |
|
1362 for path in buildout_paths: |
|
1363 sitedir, sitedircase = makepath(path) |
|
1364 if not sitedircase in known_paths and os.path.exists(sitedir): |
|
1365 sys.path.append(sitedir) |
|
1366 known_paths.add(sitedircase) |
|
1367 pkg_resources.working_set.add_entry(sitedir) |
|
1368 sys.__egginsert = len(buildout_paths) # Support distribute. |
|
1369 original_paths = [ |
|
1370 ... |
|
1371 ] |
|
1372 for path in original_paths: |
|
1373 if path == setuptools_path or path not in known_paths: |
|
1374 addsitedir(path, known_paths) |
|
1375 return known_paths |
|
1376 <BLANKLINE> |
|
1377 def original_addsitepackages(known_paths):... |
|
1378 |
|
1379 >>> print call_py(interpreter_path, "import sys; print sys.path") |
|
1380 ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE |
|
1381 ['', |
|
1382 '/interpreter/parts/interpreter', |
|
1383 ..., |
|
1384 '...setuptools...', |
|
1385 '/interpreter/eggs/demo-0.3-pyN.N.egg', |
|
1386 '/interpreter/eggs/tellmy.fortune-1.0-pyN.N.egg', |
|
1387 '/interpreter/eggs/demoneeded-1.1-pyN.N.egg', |
|
1388 ...] |
|
1389 |
|
1390 As you can see, the script now first imports pkg_resources. Then we |
|
1391 need to process egg files specially to look for namespace packages there |
|
1392 *before* we process process lines in .pth files that use the "import" |
|
1393 feature--lines that might be part of the setuptools namespace package |
|
1394 implementation for system packages, as mentioned above, and that must |
|
1395 come after processing egg namespaces. |
|
1396 |
|
1397 The most complex that this function gets is if you use namespace packages, |
|
1398 include site-packages, and use relative paths. For completeness, we'll look |
|
1399 at that result. |
|
1400 |
|
1401 >>> reset_interpreter() |
|
1402 >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( |
|
1403 ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, |
|
1404 ... interpreter='py', include_site_packages=True, |
|
1405 ... relative_paths=interpreter_dir) |
|
1406 >>> sys.stdout.write('#\n'); cat(site_path) |
|
1407 ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE |
|
1408 #... |
|
1409 def addsitepackages(known_paths): |
|
1410 """Add site packages, as determined by zc.buildout. |
|
1411 <BLANKLINE> |
|
1412 See original_addsitepackages, below, for the original version.""" |
|
1413 join = os.path.join |
|
1414 base = os.path.dirname(os.path.abspath(os.path.realpath(__file__))) |
|
1415 base = os.path.dirname(base) |
|
1416 base = os.path.dirname(base) |
|
1417 setuptools_path = '...setuptools...' |
|
1418 sys.path.append(setuptools_path) |
|
1419 known_paths.add(os.path.normcase(setuptools_path)) |
|
1420 import pkg_resources |
|
1421 buildout_paths = [ |
|
1422 join(base, 'eggs/demo-0.3-pyN.N.egg'), |
|
1423 join(base, 'eggs/tellmy.fortune-1.0-pyN.N.egg'), |
|
1424 '...setuptools...', |
|
1425 join(base, 'eggs/demoneeded-1.1-pyN.N.egg') |
|
1426 ] |
|
1427 for path in buildout_paths: |
|
1428 sitedir, sitedircase = makepath(path) |
|
1429 if not sitedircase in known_paths and os.path.exists(sitedir): |
|
1430 sys.path.append(sitedir) |
|
1431 known_paths.add(sitedircase) |
|
1432 pkg_resources.working_set.add_entry(sitedir) |
|
1433 sys.__egginsert = len(buildout_paths) # Support distribute. |
|
1434 original_paths = [ |
|
1435 ... |
|
1436 ] |
|
1437 for path in original_paths: |
|
1438 if path == setuptools_path or path not in known_paths: |
|
1439 addsitedir(path, known_paths) |
|
1440 return known_paths |
|
1441 <BLANKLINE> |
|
1442 def original_addsitepackages(known_paths):... |
|
1443 |
|
1444 >>> print call_py(interpreter_path, "import sys; print sys.path") |
|
1445 ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE |
|
1446 ['', |
|
1447 '/interpreter/parts/interpreter', |
|
1448 ..., |
|
1449 '...setuptools...', |
|
1450 '/interpreter/eggs/demo-0.3-pyN.N.egg', |
|
1451 '/interpreter/eggs/tellmy.fortune-1.0-pyN.N.egg', |
|
1452 '/interpreter/eggs/demoneeded-1.1-pyN.N.egg', |
|
1453 ...] |
|
1454 |
|
1455 The ``exec_sitecustomize`` argument does the same thing for the |
|
1456 sitecustomize module--it allows you to include the code from the |
|
1457 sitecustomize module in the underlying Python if you set the argument to |
|
1458 True. The z3c.recipe.scripts package sets up the full environment necessary |
|
1459 to demonstrate this piece. |
|
1460 |
|
1461 The ``sitepackage_safe_scripts`` function: writing scripts for entry points |
|
1462 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
1463 |
|
1464 All of the examples so far for this function have been creating |
|
1465 interpreters. The function can also write scripts for entry |
|
1466 points. They are almost identical to the scripts that we saw for the |
|
1467 ``scripts`` function except that they ``import site`` after setting the |
|
1468 sys.path to include our custom site.py and sitecustomize.py files. These |
|
1469 files then initialize the Python environment as we have already seen. Let's |
|
1470 see a simple example. |
|
1471 |
|
1472 >>> reset_interpreter() |
|
1473 >>> ws = zc.buildout.easy_install.install( |
|
1474 ... ['demo'], join(interpreter_dir, 'eggs'), links=[link_server], |
|
1475 ... index=link_server+'index/') |
|
1476 >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( |
|
1477 ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, |
|
1478 ... reqs=['demo']) |
|
1479 |
|
1480 As before, in Windows, 2 files are generated for each script. A script |
|
1481 file, ending in '-script.py', and an exe file that allows the script |
|
1482 to be invoked directly without having to specify the Python |
|
1483 interpreter and without having to provide a '.py' suffix. This is in addition |
|
1484 to the site.py and sitecustomize.py files that are generated as with our |
|
1485 interpreter examples above. |
|
1486 |
|
1487 >>> if sys.platform == 'win32': |
|
1488 ... demo_path = os.path.join(interpreter_bin_dir, 'demo-script.py') |
|
1489 ... expected = [sitecustomize_path, |
|
1490 ... site_path, |
|
1491 ... os.path.join(interpreter_bin_dir, 'demo.exe'), |
|
1492 ... demo_path] |
|
1493 ... else: |
|
1494 ... demo_path = os.path.join(interpreter_bin_dir, 'demo') |
|
1495 ... expected = [sitecustomize_path, site_path, demo_path] |
|
1496 ... |
|
1497 >>> assert generated == expected, repr((generated, expected)) |
|
1498 |
|
1499 The demo script runs the entry point defined in the demo egg: |
|
1500 |
|
1501 >>> cat(demo_path) # doctest: +NORMALIZE_WHITESPACE |
|
1502 #!/usr/local/bin/python2.4 -S |
|
1503 <BLANKLINE> |
|
1504 import sys |
|
1505 sys.path[0:0] = [ |
|
1506 '/interpreter/parts/interpreter', |
|
1507 ] |
|
1508 <BLANKLINE> |
|
1509 <BLANKLINE> |
|
1510 import os |
|
1511 path = sys.path[0] |
|
1512 if os.environ.get('PYTHONPATH'): |
|
1513 path = os.pathsep.join([path, os.environ['PYTHONPATH']]) |
|
1514 os.environ['BUILDOUT_ORIGINAL_PYTHONPATH'] = os.environ.get('PYTHONPATH', '') |
|
1515 os.environ['PYTHONPATH'] = path |
|
1516 import site # imports custom buildout-generated site.py |
|
1517 <BLANKLINE> |
|
1518 import eggrecipedemo |
|
1519 <BLANKLINE> |
|
1520 if __name__ == '__main__': |
|
1521 eggrecipedemo.main() |
|
1522 |
|
1523 >>> demo_call = join(interpreter_bin_dir, 'demo') |
|
1524 >>> if sys.platform == 'win32': |
|
1525 ... demo_call = '"%s"' % demo_call |
|
1526 >>> print system(demo_call) |
|
1527 3 1 |
|
1528 <BLANKLINE> |
|
1529 |
|
1530 There are a few differences from the ``scripts`` function. First, the |
|
1531 ``reqs`` argument (an iterable of string requirements or entry point |
|
1532 tuples) is a keyword argument here. We see that in the example above. |
|
1533 Second, the ``arguments`` argument is now named ``script_arguments`` to |
|
1534 try and clarify that it does not affect interpreters. While the |
|
1535 ``initialization`` argument continues to affect both the interpreters |
|
1536 and the entry point scripts, if you have initialization that is only |
|
1537 pertinent to the entry point scripts, you can use the |
|
1538 ``script_initialization`` argument. |
|
1539 |
|
1540 Let's see ``script_arguments`` and ``script_initialization`` in action. |
|
1541 |
|
1542 >>> reset_interpreter() |
|
1543 >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( |
|
1544 ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, |
|
1545 ... reqs=['demo'], script_arguments='1, 2', |
|
1546 ... script_initialization='import os\nos.chdir("foo")') |
|
1547 |
|
1548 >>> cat(demo_path) # doctest: +NORMALIZE_WHITESPACE |
|
1549 #!/usr/local/bin/python2.4 -S |
|
1550 import sys |
|
1551 sys.path[0:0] = [ |
|
1552 '/interpreter/parts/interpreter', |
|
1553 ] |
|
1554 <BLANKLINE> |
|
1555 import os |
|
1556 path = sys.path[0] |
|
1557 if os.environ.get('PYTHONPATH'): |
|
1558 path = os.pathsep.join([path, os.environ['PYTHONPATH']]) |
|
1559 os.environ['BUILDOUT_ORIGINAL_PYTHONPATH'] = os.environ.get('PYTHONPATH', '') |
|
1560 os.environ['PYTHONPATH'] = path |
|
1561 import site # imports custom buildout-generated site.py |
|
1562 import os |
|
1563 os.chdir("foo") |
|
1564 <BLANKLINE> |
|
1565 import eggrecipedemo |
|
1566 <BLANKLINE> |
|
1567 if __name__ == '__main__': |
|
1568 eggrecipedemo.main(1, 2) |
|
1569 |
|
1570 Handling custom build options for extensions provided in source distributions |
|
1571 ----------------------------------------------------------------------------- |
|
1572 |
|
1573 Sometimes, we need to control how extension modules are built. The |
|
1574 build function provides this level of control. It takes a single |
|
1575 package specification, downloads a source distribution, and builds it |
|
1576 with specified custom build options. |
|
1577 |
|
1578 The build function takes 3 positional arguments: |
|
1579 |
|
1580 spec |
|
1581 A package specification for a source distribution |
|
1582 |
|
1583 dest |
|
1584 A destination directory |
|
1585 |
|
1586 build_ext |
|
1587 A dictionary of options to be passed to the distutils build_ext |
|
1588 command when building extensions. |
|
1589 |
|
1590 It supports a number of optional keyword arguments: |
|
1591 |
|
1592 links |
|
1593 a sequence of URLs, file names, or directories to look for |
|
1594 links to distributions, |
|
1595 |
|
1596 index |
|
1597 The URL of an index server, or almost any other valid URL. :) |
|
1598 |
|
1599 If not specified, the Python Package Index, |
|
1600 http://pypi.python.org/simple/, is used. You can specify an |
|
1601 alternate index with this option. If you use the links option and |
|
1602 if the links point to the needed distributions, then the index can |
|
1603 be anything and will be largely ignored. In the examples, here, |
|
1604 we'll just point to an empty directory on our link server. This |
|
1605 will make our examples run a little bit faster. |
|
1606 |
|
1607 executable |
|
1608 A path to a Python executable. Distributions will be installed |
|
1609 using this executable and will be for the matching Python version. |
|
1610 |
|
1611 path |
|
1612 A list of additional directories to search for locally-installed |
|
1613 distributions. |
|
1614 |
|
1615 newest |
|
1616 A boolean value indicating whether to search for new distributions |
|
1617 when already-installed distributions meet the requirement. When |
|
1618 this is true, the default, and when the destination directory is |
|
1619 not None, then the install function will search for the newest |
|
1620 distributions that satisfy the requirements. |
|
1621 |
|
1622 versions |
|
1623 A dictionary mapping project names to version numbers to be used |
|
1624 when selecting distributions. This can be used to specify a set of |
|
1625 distribution versions independent of other requirements. |
|
1626 |
|
1627 |
|
1628 Our link server included a source distribution that includes a simple |
|
1629 extension, extdemo.c:: |
|
1630 |
|
1631 #include <Python.h> |
|
1632 #include <extdemo.h> |
|
1633 |
|
1634 static PyMethodDef methods[] = {}; |
|
1635 |
|
1636 PyMODINIT_FUNC |
|
1637 initextdemo(void) |
|
1638 { |
|
1639 PyObject *m; |
|
1640 m = Py_InitModule3("extdemo", methods, ""); |
|
1641 #ifdef TWO |
|
1642 PyModule_AddObject(m, "val", PyInt_FromLong(2)); |
|
1643 #else |
|
1644 PyModule_AddObject(m, "val", PyInt_FromLong(EXTDEMO)); |
|
1645 #endif |
|
1646 } |
|
1647 |
|
1648 The extension depends on a system-dependent include file, extdemo.h, |
|
1649 that defines a constant, EXTDEMO, that is exposed by the extension. |
|
1650 |
|
1651 We'll add an include directory to our sample buildout and add the |
|
1652 needed include file to it: |
|
1653 |
|
1654 >>> mkdir('include') |
|
1655 >>> write('include', 'extdemo.h', |
|
1656 ... """ |
|
1657 ... #define EXTDEMO 42 |
|
1658 ... """) |
|
1659 |
|
1660 Now, we can use the build function to create an egg from the source |
|
1661 distribution: |
|
1662 |
|
1663 >>> zc.buildout.easy_install.build( |
|
1664 ... 'extdemo', dest, |
|
1665 ... {'include-dirs': os.path.join(sample_buildout, 'include')}, |
|
1666 ... links=[link_server], index=link_server+'index/') |
|
1667 ['/sample-install/extdemo-1.4-py2.4-unix-i686.egg'] |
|
1668 |
|
1669 The function returns the list of eggs |
|
1670 |
|
1671 Now if we look in our destination directory, we see we have an extdemo egg: |
|
1672 |
|
1673 >>> ls(dest) |
|
1674 - demo-0.2-py2.4.egg |
|
1675 d demo-0.3-py2.4.egg |
|
1676 - demoneeded-1.0-py2.4.egg |
|
1677 d demoneeded-1.1-py2.4.egg |
|
1678 d extdemo-1.4-py2.4-unix-i686.egg |
|
1679 |
|
1680 Let's update our link server with a new version of extdemo: |
|
1681 |
|
1682 >>> update_extdemo() |
|
1683 >>> print get(link_server), |
|
1684 <html><body> |
|
1685 <a href="bigdemo-0.1-py2.4.egg">bigdemo-0.1-py2.4.egg</a><br> |
|
1686 <a href="demo-0.1-py2.4.egg">demo-0.1-py2.4.egg</a><br> |
|
1687 <a href="demo-0.2-py2.4.egg">demo-0.2-py2.4.egg</a><br> |
|
1688 <a href="demo-0.3-py2.4.egg">demo-0.3-py2.4.egg</a><br> |
|
1689 <a href="demo-0.4c1-py2.4.egg">demo-0.4c1-py2.4.egg</a><br> |
|
1690 <a href="demoneeded-1.0.zip">demoneeded-1.0.zip</a><br> |
|
1691 <a href="demoneeded-1.1.zip">demoneeded-1.1.zip</a><br> |
|
1692 <a href="demoneeded-1.2c1.zip">demoneeded-1.2c1.zip</a><br> |
|
1693 <a href="extdemo-1.4.zip">extdemo-1.4.zip</a><br> |
|
1694 <a href="extdemo-1.5.zip">extdemo-1.5.zip</a><br> |
|
1695 <a href="index/">index/</a><br> |
|
1696 <a href="other-1.0-py2.4.egg">other-1.0-py2.4.egg</a><br> |
|
1697 </body></html> |
|
1698 |
|
1699 The easy_install caches information about servers to reduce network |
|
1700 access. To see the update, we have to call the clear_index_cache |
|
1701 function to clear the index cache: |
|
1702 |
|
1703 >>> zc.buildout.easy_install.clear_index_cache() |
|
1704 |
|
1705 If we run build with newest set to False, we won't get an update: |
|
1706 |
|
1707 >>> zc.buildout.easy_install.build( |
|
1708 ... 'extdemo', dest, |
|
1709 ... {'include-dirs': os.path.join(sample_buildout, 'include')}, |
|
1710 ... links=[link_server], index=link_server+'index/', |
|
1711 ... newest=False) |
|
1712 ['/sample-install/extdemo-1.4-py2.4-linux-i686.egg'] |
|
1713 |
|
1714 >>> ls(dest) |
|
1715 - demo-0.2-py2.4.egg |
|
1716 d demo-0.3-py2.4.egg |
|
1717 - demoneeded-1.0-py2.4.egg |
|
1718 d demoneeded-1.1-py2.4.egg |
|
1719 d extdemo-1.4-py2.4-unix-i686.egg |
|
1720 |
|
1721 But if we run it with the default True setting for newest, then we'll |
|
1722 get an updated egg: |
|
1723 |
|
1724 >>> zc.buildout.easy_install.build( |
|
1725 ... 'extdemo', dest, |
|
1726 ... {'include-dirs': os.path.join(sample_buildout, 'include')}, |
|
1727 ... links=[link_server], index=link_server+'index/') |
|
1728 ['/sample-install/extdemo-1.5-py2.4-unix-i686.egg'] |
|
1729 |
|
1730 >>> ls(dest) |
|
1731 - demo-0.2-py2.4.egg |
|
1732 d demo-0.3-py2.4.egg |
|
1733 - demoneeded-1.0-py2.4.egg |
|
1734 d demoneeded-1.1-py2.4.egg |
|
1735 d extdemo-1.4-py2.4-unix-i686.egg |
|
1736 d extdemo-1.5-py2.4-unix-i686.egg |
|
1737 |
|
1738 The versions option also influences the versions used. For example, |
|
1739 if we specify a version for extdemo, then that will be used, even |
|
1740 though it isn't the newest. Let's clean out the destination directory |
|
1741 first: |
|
1742 |
|
1743 >>> import os |
|
1744 >>> for name in os.listdir(dest): |
|
1745 ... remove(dest, name) |
|
1746 |
|
1747 >>> zc.buildout.easy_install.build( |
|
1748 ... 'extdemo', dest, |
|
1749 ... {'include-dirs': os.path.join(sample_buildout, 'include')}, |
|
1750 ... links=[link_server], index=link_server+'index/', |
|
1751 ... versions=dict(extdemo='1.4')) |
|
1752 ['/sample-install/extdemo-1.4-py2.4-unix-i686.egg'] |
|
1753 |
|
1754 >>> ls(dest) |
|
1755 d extdemo-1.4-py2.4-unix-i686.egg |
|
1756 |
|
1757 Handling custom build options for extensions in develop eggs |
|
1758 ------------------------------------------------------------ |
|
1759 |
|
1760 The develop function is similar to the build function, except that, |
|
1761 rather than building an egg from a source directory containing a |
|
1762 setup.py script. |
|
1763 |
|
1764 The develop function takes 2 positional arguments: |
|
1765 |
|
1766 setup |
|
1767 The path to a setup script, typically named "setup.py", or a |
|
1768 directory containing a setup.py script. |
|
1769 |
|
1770 dest |
|
1771 The directory to install the egg link to |
|
1772 |
|
1773 It supports some optional keyword argument: |
|
1774 |
|
1775 build_ext |
|
1776 A dictionary of options to be passed to the distutils build_ext |
|
1777 command when building extensions. |
|
1778 |
|
1779 executable |
|
1780 A path to a Python executable. Distributions will be installed |
|
1781 using this executable and will be for the matching Python version. |
|
1782 |
|
1783 We have a local directory containing the extdemo source: |
|
1784 |
|
1785 >>> ls(extdemo) |
|
1786 - MANIFEST |
|
1787 - MANIFEST.in |
|
1788 - README |
|
1789 - extdemo.c |
|
1790 - setup.py |
|
1791 |
|
1792 Now, we can use the develop function to create a develop egg from the source |
|
1793 distribution: |
|
1794 |
|
1795 >>> zc.buildout.easy_install.develop( |
|
1796 ... extdemo, dest, |
|
1797 ... {'include-dirs': os.path.join(sample_buildout, 'include')}) |
|
1798 '/sample-install/extdemo.egg-link' |
|
1799 |
|
1800 The name of the egg link created is returned. |
|
1801 |
|
1802 Now if we look in our destination directory, we see we have an extdemo |
|
1803 egg link: |
|
1804 |
|
1805 >>> ls(dest) |
|
1806 d extdemo-1.4-py2.4-unix-i686.egg |
|
1807 - extdemo.egg-link |
|
1808 |
|
1809 And that the source directory contains the compiled extension: |
|
1810 |
|
1811 >>> ls(extdemo) |
|
1812 - MANIFEST |
|
1813 - MANIFEST.in |
|
1814 - README |
|
1815 d build |
|
1816 - extdemo.c |
|
1817 d extdemo.egg-info |
|
1818 - extdemo.so |
|
1819 - setup.py |
|
1820 |
|
1821 Download cache |
|
1822 -------------- |
|
1823 |
|
1824 Normally, when distributions are installed, if any processing is |
|
1825 needed, they are downloaded from the internet to a temporary directory |
|
1826 and then installed from there. A download cache can be used to avoid |
|
1827 the download step. This can be useful to reduce network access and to |
|
1828 create source distributions of an entire buildout. |
|
1829 |
|
1830 A download cache is specified by calling the download_cache |
|
1831 function. The function always returns the previous setting. If no |
|
1832 argument is passed, then the setting is unchanged. If an argument is |
|
1833 passed, the download cache is set to the given path, which must point |
|
1834 to an existing directory. Passing None clears the cache setting. |
|
1835 |
|
1836 To see this work, we'll create a directory and set it as the cache |
|
1837 directory: |
|
1838 |
|
1839 >>> cache = tmpdir('cache') |
|
1840 >>> zc.buildout.easy_install.download_cache(cache) |
|
1841 |
|
1842 We'll recreate our destination directory: |
|
1843 |
|
1844 >>> remove(dest) |
|
1845 >>> dest = tmpdir('sample-install') |
|
1846 |
|
1847 We'd like to see what is being fetched from the server, so we'll |
|
1848 enable server logging: |
|
1849 |
|
1850 >>> get(link_server+'enable_server_logging') |
|
1851 GET 200 /enable_server_logging |
|
1852 '' |
|
1853 |
|
1854 Now, if we install demo, and extdemo: |
|
1855 |
|
1856 >>> ws = zc.buildout.easy_install.install( |
|
1857 ... ['demo==0.2'], dest, |
|
1858 ... links=[link_server], index=link_server+'index/', |
|
1859 ... always_unzip=True) |
|
1860 GET 200 / |
|
1861 GET 404 /index/demo/ |
|
1862 GET 200 /index/ |
|
1863 GET 200 /demo-0.2-py2.4.egg |
|
1864 GET 404 /index/demoneeded/ |
|
1865 GET 200 /demoneeded-1.1.zip |
|
1866 |
|
1867 >>> zc.buildout.easy_install.build( |
|
1868 ... 'extdemo', dest, |
|
1869 ... {'include-dirs': os.path.join(sample_buildout, 'include')}, |
|
1870 ... links=[link_server], index=link_server+'index/') |
|
1871 GET 404 /index/extdemo/ |
|
1872 GET 200 /extdemo-1.5.zip |
|
1873 ['/sample-install/extdemo-1.5-py2.4-linux-i686.egg'] |
|
1874 |
|
1875 Not only will we get eggs in our destination directory: |
|
1876 |
|
1877 >>> ls(dest) |
|
1878 d demo-0.2-py2.4.egg |
|
1879 d demoneeded-1.1-py2.4.egg |
|
1880 d extdemo-1.5-py2.4-linux-i686.egg |
|
1881 |
|
1882 But we'll get distributions in the cache directory: |
|
1883 |
|
1884 >>> ls(cache) |
|
1885 - demo-0.2-py2.4.egg |
|
1886 - demoneeded-1.1.zip |
|
1887 - extdemo-1.5.zip |
|
1888 |
|
1889 The cache directory contains uninstalled distributions, such as zipped |
|
1890 eggs or source distributions. |
|
1891 |
|
1892 Let's recreate our destination directory and clear the index cache: |
|
1893 |
|
1894 >>> remove(dest) |
|
1895 >>> dest = tmpdir('sample-install') |
|
1896 >>> zc.buildout.easy_install.clear_index_cache() |
|
1897 |
|
1898 Now when we install the distributions: |
|
1899 |
|
1900 >>> ws = zc.buildout.easy_install.install( |
|
1901 ... ['demo==0.2'], dest, |
|
1902 ... links=[link_server], index=link_server+'index/', |
|
1903 ... always_unzip=True) |
|
1904 GET 200 / |
|
1905 GET 404 /index/demo/ |
|
1906 GET 200 /index/ |
|
1907 GET 404 /index/demoneeded/ |
|
1908 |
|
1909 >>> zc.buildout.easy_install.build( |
|
1910 ... 'extdemo', dest, |
|
1911 ... {'include-dirs': os.path.join(sample_buildout, 'include')}, |
|
1912 ... links=[link_server], index=link_server+'index/') |
|
1913 GET 404 /index/extdemo/ |
|
1914 ['/sample-install/extdemo-1.5-py2.4-linux-i686.egg'] |
|
1915 |
|
1916 >>> ls(dest) |
|
1917 d demo-0.2-py2.4.egg |
|
1918 d demoneeded-1.1-py2.4.egg |
|
1919 d extdemo-1.5-py2.4-linux-i686.egg |
|
1920 |
|
1921 Note that we didn't download the distributions from the link server. |
|
1922 |
|
1923 If we remove the restriction on demo, we'll download a newer version |
|
1924 from the link server: |
|
1925 |
|
1926 >>> ws = zc.buildout.easy_install.install( |
|
1927 ... ['demo'], dest, |
|
1928 ... links=[link_server], index=link_server+'index/', |
|
1929 ... always_unzip=True) |
|
1930 GET 200 /demo-0.3-py2.4.egg |
|
1931 |
|
1932 Normally, the download cache is the preferred source of downloads, but |
|
1933 not the only one. |
|
1934 |
|
1935 Installing solely from a download cache |
|
1936 --------------------------------------- |
|
1937 |
|
1938 A download cache can be used as the basis of application source |
|
1939 releases. In an application source release, we want to distribute an |
|
1940 application that can be built without making any network accesses. In |
|
1941 this case, we distribute a download cache and tell the easy_install |
|
1942 module to install from the download cache only, without making network |
|
1943 accesses. The install_from_cache function can be used to signal that |
|
1944 packages should be installed only from the download cache. The |
|
1945 function always returns the previous setting. Calling it with no |
|
1946 arguments returns the current setting without changing it: |
|
1947 |
|
1948 >>> zc.buildout.easy_install.install_from_cache() |
|
1949 False |
|
1950 |
|
1951 Calling it with a boolean value changes the setting and returns the |
|
1952 previous setting: |
|
1953 |
|
1954 >>> zc.buildout.easy_install.install_from_cache(True) |
|
1955 False |
|
1956 |
|
1957 Let's remove demo-0.3-py2.4.egg from the cache, clear the index cache, |
|
1958 recreate the destination directory, and reinstall demo: |
|
1959 |
|
1960 >>> for f in os.listdir(cache): |
|
1961 ... if f.startswith('demo-0.3-'): |
|
1962 ... remove(cache, f) |
|
1963 |
|
1964 >>> zc.buildout.easy_install.clear_index_cache() |
|
1965 >>> remove(dest) |
|
1966 >>> dest = tmpdir('sample-install') |
|
1967 |
|
1968 >>> ws = zc.buildout.easy_install.install( |
|
1969 ... ['demo'], dest, |
|
1970 ... links=[link_server], index=link_server+'index/', |
|
1971 ... always_unzip=True) |
|
1972 |
|
1973 >>> ls(dest) |
|
1974 d demo-0.2-py2.4.egg |
|
1975 d demoneeded-1.1-py2.4.egg |
|
1976 |
|
1977 This time, we didn't download from or even query the link server. |
|
1978 |
|
1979 .. Disable the download cache: |
|
1980 |
|
1981 >>> zc.buildout.easy_install.download_cache(None) |
|
1982 '/cache' |
|
1983 |
|
1984 >>> zc.buildout.easy_install.install_from_cache(False) |
|
1985 True |