Skip to main content

zc.buildout recipes to compile and install software or python packages and generate scripts or configuration files.

Project description

Introduction

This is a collection of recipe which can be use inside or outside a minitage environment. What is interresting in using them in minitage is that you ll have all your system dependencies in the build environment automaticly.

The egg has those entry point:
  • cmmi: install configure/make/make install softwares

  • fetch: fetch something, somewhere, with git, http, frp, static, hg, svn or bzr.

  • egg: install python eggs / packages ‘setuptoolisables’

  • printer: print or dump to a file all versions needed to achieve eggs requirements (versions.cfg made easy)

  • scripts: install scripts from an egg and install egg dependencies if they are not already in the cache

  • wsgi: Make a Python paste configuration file eatable by mod_wsgi with all the eggs dependencies you need.

The reasons why i have rewrite yet another buildout recipes builder are:
  • Support for downloading stuff

  • Do not rely on easy_install dependency system

  • Support on the fly patchs for eggs and other distribution.

  • Support multiple hooks at each stage of the build system.

  • Support for distutils

  • Robust offline mode

  • We like pypi, but offer a mode to scan for eggs without need to check the index,

  • Support malformed or not indexed distributions. In other terms, we provide an url, and the recipe builds it, that’s all.

  • All recipes must support automaticly minitage dependencies and rpath linking.

You can browse the code on minitage’s following resources:

You can migrate your buldouts without any effort with buildout.minitagificator:

minitage.recipe.scripts

Abstract

  • This recipe intends to install eggs and python software and on top of installed stuff, generating scripts and envrionment files.

  • This recipe inherit from minitage;recipe:egg.

  • Its heavilly inspired by zc.recipe.eggs* and try to completly replace it whereas be API compatbile.

  • You can use it in conjunction with the buildout.minitagificator extension which monkey patch zc.buildout to use minitage recipes.

  • What we can do that zc.recipe.egg wouldnt do, either at all or not in the way we want to:

    • All scripts support initialisation code

    • The ‘scripts’ egg metadata is also handled

  • You can use it as you would use minitage.recipe:egg, use patch facility and etc.

  • Ths recipe is also declared under those entry points:

    • minitage.recipe:eggs

    • minitage.recipe:script

Specific options

  • All the shared options and the options from minitage.recipe:egg +

  • scripts

    • Scripts to generate, if empty, generate scripts for all the working set.

    • If your egg have an old ‘scripts’ metadata, and old scripts where you want wrappers to be generated, just add the egg name to the scripts entry.

    • If you want to rename a script, just enter something like entrypoint|scriptname=NewName:

      s=NewName
  • zap If you do not want to a script, just enter a line separated list of not wanted scripts

  • entry-points

    A list of entry-point identifiers of the form::

    name=module:attrs

    where name is a script name, module is a dotted name resolving to a module name, and attrs is a dotted name resolving to a callable object within a module. This option is useful when working with distributions that don’t declare entry points, such as distributions not written to work with setuptools.

  • interpreter

    The name of a script to generate that allow access to the Python interpreter with the PYTHONPATH set with all the working set entries.

  • dependent-scripts

    compatibility option, has no effect

  • arguments

    Specify some arguments to be passed to entry points as Python source.

  • initialization

    Python code to run prior to call the entry point

Detailled documentation

Let’s create a buildout configuration file:

>>> rmdir(tempdir)
>>> mkdir(tempdir)
>>> cd(tempdir)
>>> a = [mkdir(d) for d in ('eggs', 'develop-eggs', 'bin', 'src')]
>>> install_develop_eggs(['minitage.recipe.scripts'])
>>> install_eggs_from_pathes(['zc.buildout'], sys.path)
>>> touch('buildout.cfg')
>>> sh('buildout -o bootstrap')
buildout -o bootstrap...
>>> index_url = start_server(os.path.sep.join(tempdir))

Initializing test env.

>>> if os.path.exists('foo'): rmdir(dl)
>>> mkdir('dl')
>>> if os.path.exists('foo'): rmdir(foo)
>>> mkdir('foo')
>>> mkdir('foo/src/toto')
>>> touch('foo/setup.py', data="""
... from setuptools import setup, find_packages
... setup(name='foo', version='1.0',
...     packages=find_packages('src'),
...     package_dir = {'': 'src'},
...     include_package_data=True,
...     scripts=['src/toto/toto.py'],
...     entry_points={'console_scripts': ['s=toto.toto:f']},
...     )
... """)
>>> touch('foo/src/toto/__init__.py')
>>> touch('foo/src/toto/toto.py', data="""
... def f():
...     print "foo"
... if __name__ == '__main__' :
...     print 'called'
...
... """)
>>> noecho = [os.remove(d) for d in os.listdir('.') if '.tar.gz' in d]
>>> os.chdir('foo')
>>> sh('python setup.py sdist')
p...
>>> noecho = [shutil.copy(os.path.join('dist', d), os.path.join('..', d)) for d in os.listdir('dist')]
>>> os.chdir('..')

Generating all scripts

Thus by not specifying any scripts entry in the buildout part.

>>> data = """
... [buildout]
... download-cache=${buildout:directory}/dl
... parts = part
... [part]
... recipe=minitage.recipe.scripts
... find-links=%(index)s
... eggs=foo
... """%{'index': index_url}
>>> touch('buildout.cfg', data=data)
>>> noecho = [remove(os.path.join('eggs', egg)) for egg in os.listdir('eggs') if 'foo' in egg]
>>> sh('bin/buildout -vvvvv install')
b...
minitage.recipe: Got foo 1.0.
minitage.recipe: Picked: foo = 1.0
minitage.recipe: All egg dependencies seem to be installed!
minitage.recipe: Generated scripts: 's', 'toto.py'...

Look at what have been generated.

>>> cat('bin', 'toto.py')
#!...
# ! GENERATED BY minitage.recipe !
import os
import sys
import subprocess...
sys.path[0:0] = ['/tmp/buildout.test/eggs/foo-1.0-py....egg' ]...
# EXEC ORGINAL CODE WITHOUT SHEBANG
__doc__  = 'I am generated by minitage.recipe.script recipe'...
os.environ['PYTHONPATH'] = ':'.join(sys.path + os.environ.get('PYTHONPATH', '').split(':'))
sys.argv.pop(0)
sys.exit(
    subprocess.Popen(
        [sys.executable, '/tmp/buildout.test/eggs/foo-1.0-py....egg/EGG-INFO/scripts/toto.py']+sys.argv,
        env=os.environ
    ).wait()
)...
>>> cat('bin', 's')
#!...
#!!! #GENERATED VIA MINITAGE.recipe !!!...
import sys...
sys.path[0:0] = [ '/tmp/buildout.test/eggs/foo-1.0-py....egg' ]...
import toto.toto...
if __name__ == '__main__':
    toto.toto.f()...

Selecting scripts to install

Installing only s.

>>> data = """
... [buildout]
... download-cache=${buildout:directory}/dl
... parts = part
... [part]
... recipe=minitage.recipe.scripts
... find-links=%(index)s
... scripts =
...     s
... eggs=foo
... """%{'index': index_url}
>>> touch('buildout.cfg', data=data)
>>> sh('bin/buildout -vvvvv install')
b...
minitage.recipe: Generated scripts: 's'....

Installing only toto.py.

>>> data = """
... [buildout]
... download-cache=${buildout:directory}/dl
... parts = part
... [part]
... recipe=minitage.recipe.scripts
... find-links=%(index)s
... scripts =
...     toto.py
... eggs=foo
... """%{'index': index_url}
>>> touch('buildout.cfg', data=data)
>>> sh('bin/buildout -vvvvv install')
b...
minitage.recipe: Generated scripts: 'toto.py'....

Declaring entry-points

We ll add an entry point ‘t’ to be generated.

>>> data = """
... [buildout]
... download-cache=${buildout:directory}/dl
... parts = part
... [part]
... recipe=minitage.recipe.scripts
... find-links=%(index)s
... entry-points=t=toto.toto:f
... eggs=foo
... """%{'index': index_url}
>>> touch('buildout.cfg', data=data)
>>> sh('bin/buildout -vvvvv install')
b...
minitage.recipe: Generated scripts: 't'....

Adding initialization code

What about adding environment variables for gis env.

>>> data = """
... [buildout]
... download-cache=${buildout:directory}/dl
... parts = part
... [part]
... recipe=minitage.recipe.scripts
... find-links=%(index)s
... entry-points=t=toto.toto:f
... eggs=foo
... initialization = import os;os.environ.set('GDAL', 'TRUE')
... """%{'index': index_url}
>>> touch('buildout.cfg', data=data)
>>> sh('bin/buildout -vvvvv install')
b...
>>> "import os;os.environ.set('GDAL', 'TRUE')" in open(os.path.join('bin', 't')).read()
True

Adding arguments

What about adding arguments to our launchers.

>>> data = """
... [buildout]
... download-cache=${buildout:directory}/dl
... parts = part
... [part]
... recipe=minitage.recipe.scripts
... find-links=%(index)s
... eggs = foo
... entry-points=t=toto.toto:f
... arguments = ['a', 'b']
... """%{'index': index_url}
>>> touch('buildout.cfg', data=data)
>>> sh('bin/buildout -vvvvv install')
b...
>>> "toto.toto.f(['a', 'b'])" in open(os.path.join('bin', 't')).read()
True

Generating a python interpreter

Here is how you can generate a specific python interpreter will all the environement of the working set.

>>> data = """
... [buildout]
... download-cache=${buildout:directory}/dl
... parts = part
... [part]
... recipe=minitage.recipe.scripts
... find-links=%(index)s
... interpreter = mypy
... arguments = ['a', 'b']
... eggs=foo
... """%{'index': index_url}
>>> touch('buildout.cfg', data=data)
>>> sh('bin/buildout -vvvvv install')
b...
minitage.recipe: Generated scripts: 'mypy'....
>>> cat('bin', 'mypy')
#!...
#!!! #GENERATED VIA MINITAGE.recipe !!!...
sys.path[0:0] = [ '/tmp/buildout.test/eggs/foo-1.0-py....egg' ]...
if _interactive:
    import code
    code.interact(banner="", local=globals())...

Generating an envrionment file

Here is how you can generate a specific envrionment file that you can source from to get the PYTHONPATH populated with eggs that you have configured.

>>> data = """
... [buildout]
... download-cache=${buildout:directory}/dl
... parts = part
... [part]
... recipe=minitage.recipe.scripts
... find-links=%(index)s
... env-file = mypy
... eggs=foo
... """%{'index': index_url}
>>> touch('buildout.cfg', data=data)
>>> sh('bin/buildout -vvvvv install')
b...
minitage.recipe: Generated scripts: '/tmp/buildout.test/bin/mypy'....
>>> cat('bin', 'mypy')
#!/usr/bin/env sh
<BLANKLINE>
PYTHONPATH="/tmp/buildout.test/eggs/foo-1.0-py....egg:$PYTHONPATH"
export PYTHONPATH
<BLANKLINE>
<BLANKLINE>

CHANGELOG

1.40

  • fix a special case where all scripts were filtered out

1.39

  • do not preerve PYTHONPATH on env.file generation

1.37

  • Decorator helper

1.36

  • fix develop link

1.35

  • splitted out from minitage.recipe

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page