1 Python API for egg and script installation

     2 ==========================================

     3 

     4 The easy_install module provides some functions to provide support for

     5 egg and script installation.  It provides functionality at the python

     6 level that is similar to easy_install, with a few exceptions:

     7 

     8 - By default, we look for new packages *and* the packages that

     9   they depend on.  This is somewhat like (and uses) the --upgrade

    10   option of easy_install, except that we also upgrade required

    11   packages.

    12 

    13 - If the highest-revision package satisfying a specification is

    14   already present, then we don't try to get another one.  This saves a

    15   lot of search time in the common case that packages are pegged to

    16   specific versions.

    17 

    18 - If there is a develop egg that satisfies a requirement, we don't

    19   look for additional distributions.  We always give preference to

    20   develop eggs.

    21 

    22 - Distutils options for building extensions can be passed.

    23 

    24 Distribution installation

    25 -------------------------

    26 

    27 The easy_install module provides a function, install, for installing one

    28 or more packages and their dependencies.  The install function takes 2

    29 positional arguments:

    30 

    31 - An iterable of setuptools requirement strings for the distributions

    32   to be installed, and

    33 

    34 - A destination directory to install to and to satisfy requirements

    35   from.  The destination directory can be None, in which case, no new

    36   distributions are downloaded and there will be an error if the

    37   needed distributions can't be found among those already installed.

    38 

    39 It supports a number of optional keyword arguments:

    40 

    41 links

    42    A sequence of URLs, file names, or directories to look for

    43    links to distributions.

    44 

    45 index

    46    The URL of an index server, or almost any other valid URL. :)

    47 

    48    If not specified, the Python Package Index,

    49    http://pypi.python.org/simple/, is used.  You can specify an

    50    alternate index with this option.  If you use the links option and

    51    if the links point to the needed distributions, then the index can

    52    be anything and will be largely ignored.  In the examples, here,

    53    we'll just point to an empty directory on our link server.  This

    54    will make our examples run a little bit faster.

    55 

    56 executable

    57    A path to a Python executable.  Distributions will be installed

    58    using this executable and will be for the matching Python version.

    59 

    60 path

    61    A list of additional directories to search for locally-installed

    62    distributions.

    63 

    64 always_unzip

    65    A flag indicating that newly-downloaded distributions should be

    66    directories even if they could be installed as zip files.

    67 

    68 working_set

    69    An existing working set to be augmented with additional

    70    distributions, if necessary to satisfy requirements.  This allows

    71    you to call install multiple times, if necessary, to gather

    72    multiple sets of requirements.

    73 

    74 newest

    75    A boolean value indicating whether to search for new distributions

    76    when already-installed distributions meet the requirement.  When

    77    this is true, the default, and when the destination directory is

    78    not None, then the install function will search for the newest

    79    distributions that satisfy the requirements.

    80 

    81 versions

    82    A dictionary mapping project names to version numbers to be used

    83    when selecting distributions.  This can be used to specify a set of

    84    distribution versions independent of other requirements.

    85 

    86 use_dependency_links

    87    A flag indicating whether to search for dependencies using the

    88    setup dependency_links metadata or not. If true, links are searched

    89    for using dependency_links in preference to other

    90    locations. Defaults to true.

    91 

    92 include_site_packages

    93     A flag indicating whether Python's non-standard-library packages should

    94     be available for finding dependencies.  Defaults to true.

    95 

    96     Paths outside of Python's standard library--or more precisely, those that

    97     are not included when Python is started with the -S argument--are loosely

    98     referred to as "site-packages" here.

    99 

   100 relative_paths

   101    Adjust egg paths so they are relative to the script path.  This

   102    allows scripts to work when scripts and eggs are moved, as long as

   103    they are both moved in the same way.

   104 

   105 The install method returns a working set containing the distributions

   106 needed to meet the given requirements.

   107 

   108 We have a link server that has a number of eggs:

   109 

   110     >>> print get(link_server),

   111     <html><body>

   112     <a href="bigdemo-0.1-py2.4.egg">bigdemo-0.1-py2.4.egg</a><br>

   113     <a href="demo-0.1-py2.4.egg">demo-0.1-py2.4.egg</a><br>

   114     <a href="demo-0.2-py2.4.egg">demo-0.2-py2.4.egg</a><br>

   115     <a href="demo-0.3-py2.4.egg">demo-0.3-py2.4.egg</a><br>

   116     <a href="demo-0.4c1-py2.4.egg">demo-0.4c1-py2.4.egg</a><br>

   117     <a href="demoneeded-1.0.zip">demoneeded-1.0.zip</a><br>

   118     <a href="demoneeded-1.1.zip">demoneeded-1.1.zip</a><br>

   119     <a href="demoneeded-1.2c1.zip">demoneeded-1.2c1.zip</a><br>

   120     <a href="extdemo-1.4.zip">extdemo-1.4.zip</a><br>

   121     <a href="index/">index/</a><br>

   122     <a href="other-1.0-py2.4.egg">other-1.0-py2.4.egg</a><br>

   123     </body></html>

   124 

   125 Let's make a directory and install the demo egg to it, using the demo:

   126 

   127     >>> dest = tmpdir('sample-install')

   128     >>> import zc.buildout.easy_install

   129     >>> ws = zc.buildout.easy_install.install(

   130     ...     ['demo==0.2'], dest,

   131     ...     links=[link_server], index=link_server+'index/')

   132 

   133 We requested version 0.2 of the demo distribution to be installed into

   134 the destination server.  We specified that we should search for links

   135 on the link server and that we should use the (empty) link server

   136 index directory as a package index.

   137 

   138 The working set contains the distributions we retrieved.

   139 

   140     >>> for dist in ws:

   141     ...     print dist

   142     demo 0.2

   143     demoneeded 1.1

   144 

   145 We got demoneeded because it was a dependency of demo.

   146 

   147 And the actual eggs were added to the eggs directory.

   148 

   149     >>> ls(dest)

   150     -  demo-0.2-py2.4.egg

   151     -  demoneeded-1.1-py2.4.egg

   152 

   153 If we remove the version restriction on demo, but specify a false

   154 value for newest, no new distributions will be installed:

   155 

   156     >>> ws = zc.buildout.easy_install.install(

   157     ...     ['demo'], dest, links=[link_server], index=link_server+'index/',

   158     ...     newest=False)

   159     >>> ls(dest)

   160     -  demo-0.2-py2.4.egg

   161     -  demoneeded-1.1-py2.4.egg

   162 

   163 If we leave off the newest option, we'll get an update for demo:

   164 

   165     >>> ws = zc.buildout.easy_install.install(

   166     ...     ['demo'], dest, links=[link_server], index=link_server+'index/')

   167     >>> ls(dest)

   168     -  demo-0.2-py2.4.egg

   169     -  demo-0.3-py2.4.egg

   170     -  demoneeded-1.1-py2.4.egg

   171 

   172 Note that we didn't get the newest versions available.  There were

   173 release candidates for newer versions of both packages. By default,

   174 final releases are preferred.  We can change this behavior using the

   175 prefer_final function:

   176 

   177     >>> zc.buildout.easy_install.prefer_final(False)

   178     True

   179 

   180 The old setting is returned.

   181 

   182     >>> ws = zc.buildout.easy_install.install(

   183     ...     ['demo'], dest, links=[link_server], index=link_server+'index/')

   184     >>> for dist in ws:

   185     ...     print dist

   186     demo 0.4c1

   187     demoneeded 1.2c1

   188 

   189     >>> ls(dest)

   190     -  demo-0.2-py2.4.egg

   191     -  demo-0.3-py2.4.egg

   192     -  demo-0.4c1-py2.4.egg

   193     -  demoneeded-1.1-py2.4.egg

   194     -  demoneeded-1.2c1-py2.4.egg

   195 

   196 Let's put the setting back to the default.

   197 

   198     >>> zc.buildout.easy_install.prefer_final(True)

   199     False

   200 

   201 We can supply additional distributions.  We can also supply

   202 specifications for distributions that would normally be found via

   203 dependencies.  We might do this to specify a specific version.

   204 

   205     >>> ws = zc.buildout.easy_install.install(

   206     ...     ['demo', 'other', 'demoneeded==1.0'], dest,

   207     ...     links=[link_server], index=link_server+'index/')

   208 

   209     >>> for dist in ws:

   210     ...     print dist

   211     demo 0.3

   212     other 1.0

   213     demoneeded 1.0

   214 

   215     >>> ls(dest)

   216     -  demo-0.2-py2.4.egg

   217     -  demo-0.3-py2.4.egg

   218     -  demo-0.4c1-py2.4.egg

   219     -  demoneeded-1.0-py2.4.egg

   220     -  demoneeded-1.1-py2.4.egg

   221     -  demoneeded-1.2c1-py2.4.egg

   222     d  other-1.0-py2.4.egg

   223 

   224 We can request that eggs be unzipped even if they are zip safe.  This

   225 can be useful when debugging.  (Note that Distribute will unzip eggs by

   226 default, so if you are using Distribute, most or all eggs will already be

   227 unzipped without this flag.)

   228 

   229     >>> rmdir(dest)

   230     >>> dest = tmpdir('sample-install')

   231     >>> ws = zc.buildout.easy_install.install(

   232     ...     ['demo'], dest, links=[link_server], index=link_server+'index/',

   233     ...     always_unzip=True)

   234 

   235     >>> ls(dest)

   236     d  demo-0.3-py2.4.egg

   237     d  demoneeded-1.1-py2.4.egg

   238 

   239     >>> rmdir(dest)

   240     >>> dest = tmpdir('sample-install')

   241     >>> ws = zc.buildout.easy_install.install(

   242     ...     ['demo'], dest, links=[link_server], index=link_server+'index/',

   243     ...     always_unzip=False)

   244 

   245     >>> ls(dest)

   246     -  demo-0.3-py2.4.egg

   247     -  demoneeded-1.1-py2.4.egg

   248 

   249 We can also set a default by calling the always_unzip function:

   250 

   251     >>> zc.buildout.easy_install.always_unzip(True)

   252     False

   253 

   254 The old default is returned:

   255 

   256     >>> rmdir(dest)

   257     >>> dest = tmpdir('sample-install')

   258     >>> ws = zc.buildout.easy_install.install(

   259     ...     ['demo'], dest, links=[link_server], index=link_server+'index/')

   260 

   261     >>> ls(dest)

   262     d  demo-0.3-py2.4.egg

   263     d  demoneeded-1.1-py2.4.egg

   264 

   265 

   266     >>> zc.buildout.easy_install.always_unzip(False)

   267     True

   268 

   269     >>> rmdir(dest)

   270     >>> dest = tmpdir('sample-install')

   271     >>> ws = zc.buildout.easy_install.install(

   272     ...     ['demo'], dest, links=[link_server], index=link_server+'index/')

   273 

   274     >>> ls(dest)

   275     -  demo-0.3-py2.4.egg

   276     -  demoneeded-1.1-py2.4.egg

   277 

   278     >>> rmdir(dest)

   279     >>> dest = tmpdir('sample-install')

   280     >>> ws = zc.buildout.easy_install.install(

   281     ...     ['demo'], dest, links=[link_server], index=link_server+'index/',

   282     ...     always_unzip=True)

   283 

   284     >>> ls(dest)

   285     d  demo-0.3-py2.4.egg

   286     d  demoneeded-1.1-py2.4.egg

   287 

   288 Specifying version information independent of requirements

   289 ----------------------------------------------------------

   290 

   291 Sometimes it's useful to specify version information independent of

   292 normal requirements specifications.  For example, a buildout may need

   293 to lock down a set of versions, without having to put put version

   294 numbers in setup files or part definitions.  If a dictionary is passed

   295 to the install function, mapping project names to version numbers,

   296 then the versions numbers will be used.

   297 

   298     >>> ws = zc.buildout.easy_install.install(

   299     ...     ['demo'], dest, links=[link_server], index=link_server+'index/',

   300     ...     versions = dict(demo='0.2', demoneeded='1.0'))

   301     >>> [d.version for d in ws]

   302     ['0.2', '1.0']

   303 

   304 In this example, we specified a version for demoneeded, even though we

   305 didn't define a requirement for it.  The versions specified apply to

   306 dependencies as well as the specified requirements.

   307 

   308 If we specify a version that's incompatible with a requirement, then

   309 we'll get an error:

   310 

   311     >>> from zope.testing.loggingsupport import InstalledHandler

   312     >>> handler = InstalledHandler('zc.buildout.easy_install')

   313     >>> import logging

   314     >>> logging.getLogger('zc.buildout.easy_install').propagate = False

   315 

   316     >>> ws = zc.buildout.easy_install.install(

   317     ...     ['demo >0.2'], dest, links=[link_server],

   318     ...     index=link_server+'index/',

   319     ...     versions = dict(demo='0.2', demoneeded='1.0'))

   320     Traceback (most recent call last):

   321     ...

   322     IncompatibleVersionError: Bad version 0.2

   323 

   324     >>> print handler

   325     zc.buildout.easy_install DEBUG

   326       Installing 'demo >0.2'.

   327     zc.buildout.easy_install ERROR

   328       The version, 0.2, is not consistent with the requirement, 'demo>0.2'.

   329 

   330     >>> handler.clear()

   331 

   332 If no versions are specified, a debugging message will be output

   333 reporting that a version was picked automatically:

   334 

   335     >>> ws = zc.buildout.easy_install.install(

   336     ...     ['demo'], dest, links=[link_server], index=link_server+'index/',

   337     ...     )

   338 

   339     >>> print handler

   340     zc.buildout.easy_install DEBUG

   341       Installing 'demo'.

   342     zc.buildout.easy_install DEBUG

   343       We have the best distribution that satisfies 'demo'.

   344     zc.buildout.easy_install DEBUG

   345       Picked: demo = 0.3

   346     zc.buildout.easy_install DEBUG

   347       Getting required 'demoneeded'

   348     zc.buildout.easy_install DEBUG

   349         required by demo 0.3.

   350     zc.buildout.easy_install DEBUG

   351       We have the best distribution that satisfies 'demoneeded'.

   352     zc.buildout.easy_install DEBUG

   353       Picked: demoneeded = 1.1

   354 

   355     >>> handler.uninstall()

   356     >>> logging.getLogger('zc.buildout.easy_install').propagate = True

   357 

   358 We can request that we get an error if versions are picked:

   359 

   360     >>> zc.buildout.easy_install.allow_picked_versions(False)

   361     True

   362 

   363 (The old setting is returned.)

   364 

   365     >>> ws = zc.buildout.easy_install.install(

   366     ...     ['demo'], dest, links=[link_server], index=link_server+'index/',

   367     ...     )

   368     Traceback (most recent call last):

   369     ...

   370     UserError: Picked: demo = 0.3

   371 

   372     >>> zc.buildout.easy_install.allow_picked_versions(True)

   373     False

   374 

   375 The function default_versions can be used to get and set default

   376 version information to be used when no version information is passes.

   377 If called with an argument, it sets the default versions:

   378 

   379     >>> zc.buildout.easy_install.default_versions(dict(demoneeded='1'))

   380     {}

   381 

   382 It always returns the previous default versions.  If called without an

   383 argument, it simply returns the default versions without changing

   384 them:

   385 

   386     >>> zc.buildout.easy_install.default_versions()

   387     {'demoneeded': '1'}

   388 

   389 So with the default versions set, we'll get the requested version even

   390 if the versions option isn't used:

   391 

   392     >>> ws = zc.buildout.easy_install.install(

   393     ...     ['demo'], dest, links=[link_server], index=link_server+'index/',

   394     ...     )

   395 

   396     >>> [d.version for d in ws]

   397     ['0.3', '1.0']

   398 

   399 Of course, we can unset the default versions by passing an empty

   400 dictionary:

   401 

   402     >>> zc.buildout.easy_install.default_versions({})

   403     {'demoneeded': '1'}

   404 

   405     >>> ws = zc.buildout.easy_install.install(

   406     ...     ['demo'], dest, links=[link_server], index=link_server+'index/',

   407     ...     )

   408 

   409     >>> [d.version for d in ws]

   410     ['0.3', '1.1']

   411 

   412 Dependencies in Site Packages

   413 -----------------------------

   414 

   415 Paths outside of Python's standard library--or more precisely, those that are

   416 not included when Python is started with the -S argument--are loosely referred

   417 to as "site-packages" here.  These site-packages are searched by default for

   418 distributions.  This can be disabled, so that, for instance, a system Python

   419 can be used with buildout, cleaned of any packages installed by a user or

   420 system package manager.

   421 

   422 The default behavior can be controlled and introspected using

   423 zc.buildout.easy_install.include_site_packages.

   424 

   425     >>> zc.buildout.easy_install.include_site_packages()

   426     True

   427 

   428 Here's an example of using a Python executable that includes our dependencies.

   429 

   430 Our "py_path" will have the "demoneeded," and "demo" packages available.

   431  We'll simply be asking for "demoneeded" here, but without any external

   432  index or links.

   433 

   434     >>> from zc.buildout.tests import create_sample_sys_install

   435     >>> py_path, site_packages_path = make_py()

   436     >>> create_sample_sys_install(site_packages_path)

   437 

   438     >>> example_dest = tmpdir('site-packages-example-install')

   439     >>> workingset = zc.buildout.easy_install.install(

   440     ...     ['demoneeded'], example_dest, links=[], executable=py_path,

   441     ...     index=None)

   442     >>> [dist.project_name for dist in workingset]

   443     ['demoneeded']

   444 

   445 That worked fine.  Let's try again with site packages not allowed.  We'll

   446 change the policy by changing the default.  Notice that the function for

   447 changing the default value returns the previous value.

   448 

   449     >>> zc.buildout.easy_install.include_site_packages(False)

   450     True

   451 

   452     >>> zc.buildout.easy_install.include_site_packages()

   453     False

   454 

   455     >>> zc.buildout.easy_install.clear_index_cache()

   456     >>> rmdir(example_dest)

   457     >>> example_dest = tmpdir('site-packages-example-install')

   458     >>> workingset = zc.buildout.easy_install.install(

   459     ...     ['demoneeded'], example_dest, links=[], executable=py_path,

   460     ...     index=None)

   461     Traceback (most recent call last):

   462         ...

   463     MissingDistribution: Couldn't find a distribution for 'demoneeded'.

   464     >>> zc.buildout.easy_install.clear_index_cache()

   465 

   466 Now we'll reset the default.

   467 

   468     >>> zc.buildout.easy_install.include_site_packages(True)

   469     False

   470 

   471     >>> zc.buildout.easy_install.include_site_packages()

   472     True

   473 

   474 Dependency links

   475 ----------------

   476 

   477 Setuptools allows metadata that describes where to search for package

   478 dependencies. This option is called dependency_links. Buildout has its

   479 own notion of where to look for dependencies, but it also uses the

   480 setup tools dependency_links information if it's available.

   481 

   482 Let's demo this by creating an egg that specifies dependency_links.

   483 

   484 To begin, let's create a new egg repository. This repository hold a

   485 newer version of the 'demoneeded' egg than the sample repository does.

   486 

   487     >>> repoloc = tmpdir('repo')

   488     >>> from zc.buildout.tests import create_egg

   489     >>> create_egg('demoneeded', '1.2', repoloc)

   490     >>> link_server2 = start_server(repoloc)

   491 

   492 Turn on logging on this server so that we can see when eggs are pulled

   493 from it.

   494 

   495     >>> get(link_server2 + 'enable_server_logging')

   496     GET 200 /enable_server_logging

   497     ''

   498 

   499 Now we can create an egg that specifies that its dependencies are

   500 found on this server.

   501 

   502     >>> repoloc = tmpdir('repo2')

   503     >>> create_egg('hasdeps', '1.0', repoloc,

   504     ...            install_requires = "'demoneeded'",

   505     ...            dependency_links = [link_server2])

   506 

   507 Let's add the egg to another repository.

   508 

   509     >>> link_server3 = start_server(repoloc)

   510 

   511 Now let's install the egg.

   512 

   513     >>> example_dest = tmpdir('example-install')

   514     >>> workingset = zc.buildout.easy_install.install(

   515     ...     ['hasdeps'], example_dest,

   516     ...     links=[link_server3], index=link_server3+'index/')

   517     GET 200 /

   518     GET 200 /demoneeded-1.2-pyN.N.egg

   519 

   520 The server logs show that the dependency was retrieved from the server

   521 specified in the dependency_links.

   522 

   523 Now let's see what happens if we provide two different ways to retrieve

   524 the dependencies.

   525 

   526     >>> rmdir(example_dest)

   527     >>> example_dest = tmpdir('example-install')

   528     >>> workingset = zc.buildout.easy_install.install(

   529     ...     ['hasdeps'], example_dest, index=link_server+'index/',

   530     ...     links=[link_server, link_server3])

   531     GET 200 /

   532     GET 200 /demoneeded-1.2-pyN.N.egg

   533 

   534 Once again the dependency is fetched from the logging server even

   535 though it is also available from the non-logging server. This is

   536 because the version on the logging server is newer and buildout

   537 normally chooses the newest egg available.

   538 

   539 If you wish to control where dependencies come from regardless of

   540 dependency_links setup metadata use the 'use_dependency_links' option

   541 to zc.buildout.easy_install.install().

   542 

   543     >>> rmdir(example_dest)

   544     >>> example_dest = tmpdir('example-install')

   545     >>> workingset = zc.buildout.easy_install.install(

   546     ...     ['hasdeps'], example_dest, index=link_server+'index/',

   547     ...     links=[link_server, link_server3],

   548     ...     use_dependency_links=False)

   549 

   550 Notice that this time the dependency egg is not fetched from the

   551 logging server. When you specify not to use dependency_links, eggs

   552 will only be searched for using the links you explicitly provide.

   553 

   554 Another way to control this option is with the

   555 zc.buildout.easy_install.use_dependency_links() function. This

   556 function sets the default behavior for the zc.buildout.easy_install()

   557 function.

   558 

   559     >>> zc.buildout.easy_install.use_dependency_links(False)

   560     True

   561 

   562 The function returns its previous setting.

   563 

   564     >>> rmdir(example_dest)

   565     >>> example_dest = tmpdir('example-install')

   566     >>> workingset = zc.buildout.easy_install.install(

   567     ...     ['hasdeps'], example_dest, index=link_server+'index/',

   568     ...     links=[link_server, link_server3])

   569 

   570 It can be overridden by passing a keyword argument to the install

   571 function.

   572 

   573     >>> rmdir(example_dest)

   574     >>> example_dest = tmpdir('example-install')

   575     >>> workingset = zc.buildout.easy_install.install(

   576     ...     ['hasdeps'], example_dest, index=link_server+'index/',

   577     ...     links=[link_server, link_server3],

   578     ...	    use_dependency_links=True)

   579     GET 200 /demoneeded-1.2-pyN.N.egg

   580 

   581 To return the dependency_links behavior to normal call the function again.

   582 

   583     >>> zc.buildout.easy_install.use_dependency_links(True)

   584     False

   585     >>> rmdir(example_dest)

   586     >>> example_dest = tmpdir('example-install')

   587     >>> workingset = zc.buildout.easy_install.install(

   588     ...     ['hasdeps'], example_dest, index=link_server+'index/',

   589     ...     links=[link_server, link_server3])

   590     GET 200 /demoneeded-1.2-pyN.N.egg

   591 

   592 

   593 Script generation

   594 -----------------

   595 

   596 The easy_install module provides support for creating scripts from eggs.

   597 It provides two competing functions.  One, ``scripts``, is a

   598 well-established approach to generating reliable scripts with a "clean"

   599 Python--e.g., one that does not have any packages in its site-packages.

   600 The other, ``sitepackage_safe_scripts``, is newer, a bit trickier, and is

   601 designed to work with a Python that has code in its site-packages, such

   602 as a system Python.

   603 

   604 Both are similar to setuptools except that they provides facilities for

   605 baking a script's path into the script.  This has two advantages:

   606 

   607 - The eggs to be used by a script are not chosen at run time, making

   608   startup faster and, more importantly, deterministic.

   609 

   610 - The script doesn't have to import pkg_resources because the logic that

   611   pkg_resources would execute at run time is executed at script-creation

   612   time.  (There is an exception in ``sitepackage_safe_scripts`` if you

   613   want to have your Python's site packages available, as discussed

   614   below, but even in that case pkg_resources is only partially

   615   activated, which can be a significant time savings.)

   616 

   617 

   618 The ``scripts`` function

   619 ~~~~~~~~~~~~~~~~~~~~~~~~

   620 

   621 The ``scripts`` function is the first way to generate scripts that we'll

   622 examine. It is the earlier approach that the package offered.  Let's

   623 create a destination directory for it to place them in:

   624 

   625     >>> bin = tmpdir('bin')

   626 

   627 Now, we'll use the scripts function to generate scripts in this directory

   628 from the demo egg:

   629 

   630     >>> import sys

   631     >>> scripts = zc.buildout.easy_install.scripts(

   632     ...     ['demo'], ws, sys.executable, bin)

   633 

   634 the four arguments we passed were:

   635 

   636 1. A sequence of distribution requirements.  These are of the same

   637    form as setuptools requirements.  Here we passed a single

   638    requirement, for the version 0.1 demo distribution.

   639 

   640 2. A working set,

   641 

   642 3. The Python executable to use, and

   643 

   644 3. The destination directory.

   645 

   646 The bin directory now contains a generated script:

   647 

   648     >>> ls(bin)

   649     -  demo

   650 

   651 The return value is a list of the scripts generated:

   652 

   653     >>> import os, sys

   654     >>> if sys.platform == 'win32':

   655     ...     scripts == [os.path.join(bin, 'demo.exe'),

   656     ...                 os.path.join(bin, 'demo-script.py')]

   657     ... else:

   658     ...     scripts == [os.path.join(bin, 'demo')]

   659     True

   660 

   661 Note that in Windows, 2 files are generated for each script.  A script

   662 file, ending in '-script.py', and an exe file that allows the script

   663 to be invoked directly without having to specify the Python

   664 interpreter and without having to provide a '.py' suffix.

   665 

   666 The demo script run the entry point defined in the demo egg:

   667 

   668     >>> cat(bin, 'demo') # doctest: +NORMALIZE_WHITESPACE

   669     #!/usr/local/bin/python2.4

   670     <BLANKLINE>

   671     import sys

   672     sys.path[0:0] = [

   673       '/sample-install/demo-0.3-py2.4.egg',

   674       '/sample-install/demoneeded-1.1-py2.4.egg',

   675       ]

   676     <BLANKLINE>

   677     import eggrecipedemo

   678     <BLANKLINE>

   679     if __name__ == '__main__':

   680         eggrecipedemo.main()

   681 

   682 Some things to note:

   683 

   684 - The demo and demoneeded eggs are added to the beginning of sys.path.

   685 

   686 - The module for the script entry point is imported and the entry

   687   point, in this case, 'main', is run.

   688 

   689 Rather than requirement strings, you can pass tuples containing 3

   690 strings:

   691 

   692   - A script name,

   693 

   694   - A module,

   695 

   696   - An attribute expression for an entry point within the module.

   697 

   698 For example, we could have passed entry point information directly

   699 rather than passing a requirement:

   700 

   701     >>> scripts = zc.buildout.easy_install.scripts(

   702     ...     [('demo', 'eggrecipedemo', 'main')],

   703     ...     ws, sys.executable, bin)

   704 

   705     >>> cat(bin, 'demo') # doctest: +NORMALIZE_WHITESPACE

   706     #!/usr/local/bin/python2.4

   707     <BLANKLINE>

   708     import sys

   709     sys.path[0:0] = [

   710       '/sample-install/demo-0.3-py2.4.egg',

   711       '/sample-install/demoneeded-1.1-py2.4.egg',

   712       ]

   713     <BLANKLINE>

   714     import eggrecipedemo

   715     <BLANKLINE>

   716     if __name__ == '__main__':

   717         eggrecipedemo.main()

   718 

   719 Passing entry-point information directly is handy when using eggs (or

   720 distributions) that don't declare their entry points, such as

   721 distributions that aren't based on setuptools.

   722 

   723 The interpreter keyword argument can be used to generate a script that can

   724 be used to invoke the Python interactive interpreter with the path set

   725 based on the working set.  This generated script can also be used to

   726 run other scripts with the path set on the working set:

   727 

   728     >>> scripts = zc.buildout.easy_install.scripts(

   729     ...     ['demo'], ws, sys.executable, bin, interpreter='py')

   730 

   731 

   732     >>> ls(bin)

   733     -  demo

   734     -  py

   735 

   736     >>> if sys.platform == 'win32':

   737     ...     scripts == [os.path.join(bin, 'demo.exe'),

   738     ...                 os.path.join(bin, 'demo-script.py'),

   739     ...                 os.path.join(bin, 'py.exe'),

   740     ...                 os.path.join(bin, 'py-script.py')]

   741     ... else:

   742     ...     scripts == [os.path.join(bin, 'demo'),

   743     ...                 os.path.join(bin, 'py')]

   744     True

   745 

   746 The py script simply runs the Python interactive interpreter with

   747 the path set:

   748 

   749     >>> cat(bin, 'py') # doctest: +NORMALIZE_WHITESPACE

   750     #!/usr/local/bin/python2.4

   751     <BLANKLINE>

   752     import sys

   753     <BLANKLINE>

   754     sys.path[0:0] = [

   755       '/sample-install/demo-0.3-pyN.N.egg',

   756       '/sample-install/demoneeded-1.1-pyN.N.egg',

   757       ]

   758     <BLANKLINE>

   759     _interactive = True

   760     if len(sys.argv) > 1:

   761         _options, _args = __import__("getopt").getopt(sys.argv[1:], 'ic:m:')

   762         _interactive = False

   763         for (_opt, _val) in _options:

   764             if _opt == '-i':

   765                 _interactive = True

   766             elif _opt == '-c':

   767                 exec _val

   768             elif _opt == '-m':

   769                 sys.argv[1:] = _args

   770                 _args = []

   771                 __import__("runpy").run_module(

   772                      _val, {}, "__main__", alter_sys=True)

   773     <BLANKLINE>

   774         if _args:

   775             sys.argv[:] = _args

   776             __file__ = _args[0]

   777             del _options, _args

   778             execfile(__file__)

   779     <BLANKLINE>

   780     if _interactive:

   781         del _interactive

   782         __import__("code").interact(banner="", local=globals())

   783 

   784 If invoked with a script name and arguments, it will run that script, instead.

   785 

   786     >>> write('ascript', '''

   787     ... "demo doc"

   788     ... print sys.argv

   789     ... print (__name__, __file__, __doc__)

   790     ... ''')

   791     >>> print system(join(bin, 'py')+' ascript a b c'),

   792     ['ascript', 'a', 'b', 'c']

   793     ('__main__', 'ascript', 'demo doc')

   794 

   795 For Python 2.5 and higher, you can also use the -m option to run a

   796 module:

   797 

   798     >>> print system(join(bin, 'py')+' -m pdb'),

   799     usage: pdb.py scriptfile [arg] ...

   800 

   801     >>> print system(join(bin, 'py')+' -m pdb what'),

   802     Error: what does not exist

   803 

   804 An additional argument can be passed to define which scripts to install

   805 and to provide script names. The argument is a dictionary mapping

   806 original script names to new script names.

   807 

   808     >>> bin = tmpdir('bin2')

   809     >>> scripts = zc.buildout.easy_install.scripts(

   810     ...    ['demo'], ws, sys.executable, bin, dict(demo='run'))

   811 

   812     >>> if sys.platform == 'win32':

   813     ...     scripts == [os.path.join(bin, 'run.exe'),

   814     ...                 os.path.join(bin, 'run-script.py')]

   815     ... else:

   816     ...     scripts == [os.path.join(bin, 'run')]

   817     True

   818     >>> ls(bin)

   819     -  run

   820 

   821     >>> print system(os.path.join(bin, 'run')),

   822     3 1

   823 

   824 The ``scripts`` function: Including extra paths in scripts

   825 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

   826 

   827 We can pass a keyword argument, extra paths, to cause additional paths

   828 to be included in the a generated script:

   829 

   830     >>> foo = tmpdir('foo')

   831     >>> scripts = zc.buildout.easy_install.scripts(

   832     ...    ['demo'], ws, sys.executable, bin, dict(demo='run'),

   833     ...    extra_paths=[foo])

   834 

   835     >>> cat(bin, 'run') # doctest: +NORMALIZE_WHITESPACE

   836     #!/usr/local/bin/python2.4

   837     <BLANKLINE>

   838     import sys

   839     sys.path[0:0] = [

   840       '/sample-install/demo-0.3-py2.4.egg',

   841       '/sample-install/demoneeded-1.1-py2.4.egg',

   842       '/foo',

   843       ]

   844     <BLANKLINE>

   845     import eggrecipedemo

   846     <BLANKLINE>

   847     if __name__ == '__main__':

   848         eggrecipedemo.main()

   849 

   850 The ``scripts`` function: Providing script arguments

   851 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

   852 

   853 An "argument" keyword argument can be used to pass arguments to an

   854 entry point.  The value passed is a source string to be placed between the

   855 parentheses in the call:

   856 

   857     >>> scripts = zc.buildout.easy_install.scripts(

   858     ...    ['demo'], ws, sys.executable, bin, dict(demo='run'),

   859     ...    arguments='1, 2')

   860 

   861     >>> cat(bin, 'run') # doctest: +NORMALIZE_WHITESPACE

   862     #!/usr/local/bin/python2.4

   863     import sys

   864     sys.path[0:0] = [

   865       '/sample-install/demo-0.3-py2.4.egg',

   866       '/sample-install/demoneeded-1.1-py2.4.egg',

   867       ]

   868     <BLANKLINE>

   869     import eggrecipedemo

   870     <BLANKLINE>

   871     if __name__ == '__main__':

   872         eggrecipedemo.main(1, 2)

   873 

   874 The ``scripts`` function: Passing initialization code

   875 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

   876 

   877 You can also pass script initialization code:

   878 

   879     >>> scripts = zc.buildout.easy_install.scripts(

   880     ...    ['demo'], ws, sys.executable, bin, dict(demo='run'),

   881     ...    arguments='1, 2',

   882     ...    initialization='import os\nos.chdir("foo")')

   883 

   884     >>> cat(bin, 'run') # doctest: +NORMALIZE_WHITESPACE

   885     #!/usr/local/bin/python2.4

   886     import sys

   887     sys.path[0:0] = [

   888       '/sample-install/demo-0.3-py2.4.egg',

   889       '/sample-install/demoneeded-1.1-py2.4.egg',

   890       ]

   891     <BLANKLINE>

   892     import os

   893     os.chdir("foo")

   894     <BLANKLINE>

   895     import eggrecipedemo

   896     <BLANKLINE>

   897     if __name__ == '__main__':

   898         eggrecipedemo.main(1, 2)

   899 

   900 The ``scripts`` function: Relative paths

   901 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

   902 

   903 Sometimes, you want to be able to move a buildout directory around and

   904 have scripts still work without having to rebuild them.  We can

   905 control this using the relative_paths option to install.  You need

   906 to pass a common base directory of the scripts and eggs:

   907 

   908     >>> bo = tmpdir('bo')

   909     >>> ba = tmpdir('ba')

   910     >>> mkdir(bo, 'eggs')

   911     >>> mkdir(bo, 'bin')

   912     >>> mkdir(bo, 'other')

   913 

   914     >>> ws = zc.buildout.easy_install.install(

   915     ...     ['demo'], join(bo, 'eggs'), links=[link_server],

   916     ...     index=link_server+'index/')

   917 

   918     >>> scripts = zc.buildout.easy_install.scripts(

   919     ...    ['demo'], ws, sys.executable, join(bo, 'bin'), dict(demo='run'),

   920     ...    extra_paths=[ba, join(bo, 'bar')],

   921     ...    interpreter='py',

   922     ...    relative_paths=bo)

   923 

   924     >>> cat(bo, 'bin', 'run') # doctest: +NORMALIZE_WHITESPACE

   925     #!/usr/local/bin/python2.4

   926     <BLANKLINE>

   927     import os

   928     <BLANKLINE>

   929     join = os.path.join

   930     base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))

   931     base = os.path.dirname(base)

   932     <BLANKLINE>

   933     import sys

   934     sys.path[0:0] = [

   935       join(base, 'eggs/demo-0.3-pyN.N.egg'),

   936       join(base, 'eggs/demoneeded-1.1-pyN.N.egg'),

   937       '/ba',

   938       join(base, 'bar'),

   939       ]

   940     <BLANKLINE>

   941     import eggrecipedemo

   942     <BLANKLINE>

   943     if __name__ == '__main__':

   944         eggrecipedemo.main()

   945 

   946 Note that the extra path we specified that was outside the directory

   947 passed as relative_paths wasn't converted to a relative path.

   948 

   949 Of course, running the script works:

   950 

   951     >>> print system(join(bo, 'bin', 'run')),

   952     3 1

   953 

   954 We specified an interpreter and its paths are adjusted too:

   955 

   956     >>> cat(bo, 'bin', 'py') # doctest: +NORMALIZE_WHITESPACE

   957     #!/usr/local/bin/python2.4

   958     <BLANKLINE>

   959     import os

   960     <BLANKLINE>

   961     join = os.path.join

   962     base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))

   963     base = os.path.dirname(base)

   964     <BLANKLINE>

   965     import sys

   966     <BLANKLINE>

   967     sys.path[0:0] = [

   968       join(base, 'eggs/demo-0.3-pyN.N.egg'),

   969       join(base, 'eggs/demoneeded-1.1-pyN.N.egg'),

   970       '/ba',

   971       join(base, 'bar'),

   972       ]

   973     <BLANKLINE>

   974     _interactive = True

   975     if len(sys.argv) > 1:

   976         _options, _args = __import__("getopt").getopt(sys.argv[1:], 'ic:m:')

   977         _interactive = False

   978         for (_opt, _val) in _options:

   979             if _opt == '-i':

   980                 _interactive = True

   981             elif _opt == '-c':

   982                 exec _val

   983             elif _opt == '-m':

   984                 sys.argv[1:] = _args

   985                 _args = []

   986                 __import__("runpy").run_module(

   987                      _val, {}, "__main__", alter_sys=True)

   988     <BLANKLINE>

   989         if _args:

   990             sys.argv[:] = _args

   991             __file__ = _args[0]

   992             del _options, _args

   993             execfile(__file__)

   994     <BLANKLINE>

   995     if _interactive:

   996         del _interactive

   997         __import__("code").interact(banner="", local=globals())

   998 

   999 The ``sitepackage_safe_scripts`` function

  1000 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  1001 

  1002 The newer function for creating scripts is ``sitepackage_safe_scripts``.

  1003 It has the same basic functionality as the ``scripts`` function: it can

  1004 create scripts to run arbitrary entry points, and to run a Python

  1005 interpreter.  The following are the differences from a user's

  1006 perspective.

  1007 

  1008 - It can be used safely with a Python that has packages installed itself,

  1009   such as a system-installed Python.

  1010 

  1011 - In contrast to the interpreter generated by the ``scripts`` method, which

  1012   supports only a small subset of the usual Python executable's options,

  1013   the interpreter generated by ``sitepackage_safe_scripts`` supports all

  1014   of them. This makes it possible to use as full Python replacement for

  1015   scripts that need the distributions specified in your buildout.

  1016 

  1017 - Both the interpreter and the entry point scripts allow you to include the

  1018   site packages, and/or the sitecustomize, of the Python executable, if

  1019   desired.

  1020 

  1021 It works by creating site.py and sitecustomize.py files that set up the

  1022 desired paths and initialization.  These must be placed within an otherwise

  1023 empty directory.  Typically this is in a recipe's parts directory.

  1024 

  1025 Here's the simplest example, building an interpreter script.

  1026 

  1027     >>> interpreter_dir = tmpdir('interpreter')

  1028     >>> interpreter_parts_dir = os.path.join(

  1029     ...     interpreter_dir, 'parts', 'interpreter')

  1030     >>> interpreter_bin_dir = os.path.join(interpreter_dir, 'bin')

  1031     >>> mkdir(interpreter_bin_dir)

  1032     >>> mkdir(interpreter_dir, 'eggs')

  1033     >>> mkdir(interpreter_dir, 'parts')

  1034     >>> mkdir(interpreter_parts_dir)

  1035 

  1036     >>> ws = zc.buildout.easy_install.install(

  1037     ...     ['demo'], join(interpreter_dir, 'eggs'), links=[link_server],

  1038     ...     index=link_server+'index/')

  1039     >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(

  1040     ...     interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,

  1041     ...     interpreter='py')

  1042 

  1043 Depending on whether the machine being used is running Windows or not, this

  1044 produces either three or four files.  In both cases, we have site.py and

  1045 sitecustomize.py generated in the parts/interpreter directory.  For Windows,

  1046 we have py.exe and py-script.py; for other operating systems, we have py.

  1047 

  1048     >>> sitecustomize_path = os.path.join(

  1049     ...     interpreter_parts_dir, 'sitecustomize.py')

  1050     >>> site_path = os.path.join(interpreter_parts_dir, 'site.py')

  1051     >>> interpreter_path = os.path.join(interpreter_bin_dir, 'py')

  1052     >>> if sys.platform == 'win32':

  1053     ...     py_path = os.path.join(interpreter_bin_dir, 'py-script.py')

  1054     ...     expected = [sitecustomize_path,

  1055     ...                 site_path,

  1056     ...                 os.path.join(interpreter_bin_dir, 'py.exe'),

  1057     ...                 py_path]

  1058     ... else:

  1059     ...     py_path = interpreter_path

  1060     ...     expected = [sitecustomize_path, site_path, py_path]

  1061     ...

  1062     >>> assert generated == expected, repr((generated, expected))

  1063 

  1064 We didn't ask for any initialization, and we didn't ask to use the underlying

  1065 sitecustomization, so sitecustomize.py is empty.

  1066 

  1067     >>> cat(sitecustomize_path)

  1068 

  1069 The interpreter script is simple.  It puts the directory with the

  1070 site.py and sitecustomize.py on the PYTHONPATH and (re)starts Python.

  1071 

  1072     >>> cat(py_path)

  1073     #!/usr/bin/python -S

  1074     import os

  1075     import sys

  1076     <BLANKLINE>

  1077     argv = [sys.executable] + sys.argv[1:]

  1078     environ = os.environ.copy()

  1079     path = '/interpreter/parts/interpreter'

  1080     if environ.get('PYTHONPATH'):

  1081         path = os.pathsep.join([path, environ['PYTHONPATH']])

  1082     environ['PYTHONPATH'] = path

  1083     os.execve(sys.executable, argv, environ)

  1084 

  1085 The site.py file is a modified version of the underlying Python's site.py.

  1086 The most important modification is that it has a different version of the

  1087 addsitepackages function.  It sets up the Python path, similarly to the

  1088 behavior of the function it replaces.  The following shows the part that

  1089 buildout inserts, in the simplest case.

  1090 

  1091     >>> sys.stdout.write('#\n'); cat(site_path)

  1092     ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE

  1093     #...

  1094     def addsitepackages(known_paths):

  1095         """Add site packages, as determined by zc.buildout.

  1096     <BLANKLINE>

  1097         See original_addsitepackages, below, for the original version."""

  1098         buildout_paths = [

  1099             '/interpreter/eggs/demo-0.3-pyN.N.egg',

  1100             '/interpreter/eggs/demoneeded-1.1-pyN.N.egg'

  1101             ]

  1102         for path in buildout_paths:

  1103             sitedir, sitedircase = makepath(path)

  1104             if not sitedircase in known_paths and os.path.exists(sitedir):

  1105                 sys.path.append(sitedir)

  1106                 known_paths.add(sitedircase)

  1107         return known_paths

  1108     <BLANKLINE>

  1109     def original_addsitepackages(known_paths):...

  1110 

  1111 Here are some examples of the interpreter in use.

  1112 

  1113     >>> print call_py(interpreter_path, "print 16+26")

  1114     42

  1115     <BLANKLINE>

  1116     >>> res = call_py(interpreter_path, "import sys; print sys.path")

  1117     >>> print res # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE

  1118     ['',

  1119      '/interpreter/parts/interpreter',

  1120      ...,

  1121      '/interpreter/eggs/demo-0.3-pyN.N.egg',

  1122      '/interpreter/eggs/demoneeded-1.1-pyN.N.egg']

  1123     <BLANKLINE>

  1124     >>> clean_paths = eval(res.strip()) # This is used later for comparison.

  1125 

  1126 If you provide initialization, it goes in sitecustomize.py.

  1127 

  1128     >>> def reset_interpreter():

  1129     ...     # This is necessary because, in our tests, the timestamps of the

  1130     ...     # .pyc files are not outdated when we want them to be.

  1131     ...     rmdir(interpreter_bin_dir)

  1132     ...     mkdir(interpreter_bin_dir)

  1133     ...     rmdir(interpreter_parts_dir)

  1134     ...     mkdir(interpreter_parts_dir)

  1135     ...

  1136     >>> reset_interpreter()