Metadata-Version: 1.0
Name: keas.build
Version: 0.1.7
Summary: A Build System
Home-page: http://pypi.python.org/pypi/keas.build
Author: Stephan Richter and the Zope Community
Author-email: zope-dev@zope.org
License: ZPL 2.1
Description: A set of tools for managing multi-package software deployments with buildout.
        
        
        ==========
        keas.build
        ==========
        
        Manage large multi-package projects with ease
        =============================================
        
        ``keas.build`` is a command line tool for quickly creating new
        eggs and buildout configurations for an application that has multiple
        interdependant packages.  As an example, consider a web application
        that helps you manage your twitter followers, called Twollo (short
        for twitter followers), you may have several different python packages
        which handle different aspects of your application:
        
        - ``twollo.integration`` - a package for managing integration with
        twitter
        
        - ``twollo.web`` - a web front end for the ``twollo.integration``
        package
        
        - ``twollo.rest`` - a rest API for working with
        ``twollo.integration``
        
        - ``twollo.utils`` - a package filled with random utilities and
        stuff.
        
        Using ``keas.build`` you will be able to manage the lifecycle of these
        python packages and any number of deployment configurations
        coherently.  Specifically, ``keas.build`` will:
        
        - Allow you to define a Project, which is a collection of
        interdependent eggs that generally get released together.
        
        - Automatically create new egg releases of each package when necessary.
        
        - Upload new eggs to a private egg repository.
        
        - Generate versioned buildout configuration files that combine the
        eggs properly.
        
        - Upload the buildout configuration files to a private configuration server.
        
        - Upload dependent buildout configuration files to a private configuration
        server (by checking the extends= chain)
        
        Installation
        ============
        
        Installing a released version of keas.build
        -------------------------------------------
        
        You can use ``easy_install`` to get the latest release::
        
        $ easy_install keas.build
        
        Installing a developer version of keas.build
        --------------------------------------------
        
        Checkout the code::
        
        $ svn checkout svn://svn.zope.org/repos/main/keas.build/trunk keas.build
        $ cd keas.build
        
        Run the bootstrap script and buildout::
        
        $ python bootstrap.py
        $ ./bin/buildout
        
        Running the ``build-package`` script
        ------------------------------------
        
        Once installation is complete, you should be able to run the
        ``build-package`` script. (Found in *./bin/build-package* with developer
        installations)::
        
        $ build-package
        Usage: build-package [options]
        
        Options:
        -h, --help            show this help message and exit
        -c FILE, --config-file=FILE
        The file containing the configuration of the project.
        -q, --quiet           When specified, no messages are displayed.
        -v, --verbose         When specified, debug information is created.
        -d, --use-defaults    When specified, no user input is required and the defaults are used.
        -o, --offline-mode    When set, no server commands are executed.
        -n, --next-version    When set, the system guesses the next version to generate.
        -b BRANCH, --use-branch=BRANCH
        When specified, this branch will be always used.
        --no-upload           When set, the generated configuration files are not uploaded.
        --no-branch-update    When set, the branch is not updated with a new version after a release is created.
        
        Getting Started
        ===============
        
        Assumptions
        -----------
        
        First we need to start with some assumptions that keas.build has about
        the way packages are laid out. When developing multiple packages in
        tandem, it often makes sense to have your subversion repository laid
        out like so::
        
        SVNROOT/packages/
        branches/
        Twollo-0.x/
        ...
        Twollo-1.x/
        twollo.integration/
        twollo.rest/
        twollo.utils/
        twollo.web/
        tags/
        twollo.integration-1.2/
        twollo.integration-1.3/
        twollo.web-1.7/
        twollo.web-1.8/
        twollo.web-1.9/
        ...
        trunk/
        twollo.integration/
        twollo.rest/
        twollo.utils/
        twollo.web/
        
        The important thing to note is that each package does *not* have its
        own branches/ tags/ trunk/ directories, but rather there is just one
        set of the entire "project."
        
        Defining a Project configuration file
        --------------------------------------
        
        Before you can really do anything with the ``build-package`` script,
        you have to define a configuration file. Project configuration files
        use the INI [#ini]_ file format.  Every project configuration file
        must have a ``[build]`` section.  The project configuration file for
        the Twollo would look something like this::
        
        
        # Twollo.cfg
        
        [build]
        name = Twollo  #this has nothing to do with the twollo package namespace
        version = +
        template = Twollo-Release-Template.cfg
        package-index = https://build.twollo.com/eggs/
        package-index-username = someuser
        package-index-password = somepass
        buildout-server = https://build.twollo.com/buildouts/
        buildout-server-username = someuser
        buildout-server-password = somepass
        svn-repos = https://svn.twollo.com/svn/packages/
        svn-repos-username = somesvnuser
        svn-repos-password = somepass
        packages = twollo.integration
        twollo.web
        twollo.rest
        twollo.utils
        
        Let's go over each of the settings in the ``build`` section of
        *Twollo.cfg*.
        
        - **name** - This is the name of the project. It can be anything you
        want and has nothing to do with the packages that make up the
        project.  The name will be part of the generation buildout
        configuration files.
        
        - **version** - This is the version to use when making a new release
        of the Project.  The version number becomes part of the filename for
        the generated buildout configuration files.
        
        - Using **+** as the version will simply increment the version
        number of the project from the versions that have already been
        released.
        
        - **template** - This is a base buildout configuration file to use for
        all deployments.  When a new Project release is created, the
        ``[versions]`` section will automatically be updated with the
        correct versions of each of the ``twollow.*`` packages.  More on
        this later.
        
        - **tag-layout** - Choose from ``flat`` or ``subfolder``
        
        - **flat** Tags will be created in svn as /tags/package-version
        This is the default setting.
        
        - **subfolder** Tags will be created in svn as /tags/package/version
        
        - **upload-type** - Choose from ``internal`` or ``setup.py``
        
        - **internal** Upload packages to a WebDAV enabled web server using the
        below credetials. This is the default setting.
        (actually does a ``python setup.py sdist`` and uploads the result)
        
        - **setup.py** Executes ``python setup.py sdist register upload``,
        does nothing else as this command should take care of the upload.
        
        - **package-index** - The url to a WebDAV [#webdav]_ enabled web
        server where generated eggs for each of the ``twollow.*`` packages
        should be uploaded. Used for upload only if ``upload-type`` is ``internal``.
        Also used to check/get existing versions of packages.
        
        - **package-index-username** - The username for accessing the WebDAV
        server
        
        - **package-index-password** - The password for accessing the WebDAV
        server
        
        - **buildout-upload-type** - Choose from ``webdav``, ``local`` or ``mypypi``
        
        - **webdav** Upload generated buildout files to the url specified by
        ``buildout-server`` with the WebDAV protocol.
        
        - **local** Just generate buildout files, don't upload them.
        If ``buildout-server`` is given buildout files will be copied to that
        folder.
        
        - **mypypi** Upload generated buildout files to the url specified by
        ``buildout-server``. The url should point to the mypypi upload page.
        (Something like http://yourhost/++projects++/)
        
        - **buildout-server** - The url to a WebDAV enabled web
        server where generated buildout files should be uploaded.
        If ``buildout-upload-type`` is ``local`` this is a path on the local
        filesystem. Buildout files wil be copied to this folder.
        If not given, the process stops after releasing the packages.
        
        - **buildout-server-username** - The username for accessing the WebDAV
        server
        
        - **buildout-server-password** - The password for accessing the WebDAV
        server
        
        - **svn-repos** - The url for the subversion repository where all the
        source code lives, including release tags.
        
        - **svn-repos-username** - The username for the url repository.
        Use the command line option ``--force-svnauth`` to force all svn operations
        to use this credential.
        Otherwise cached authentication will be used.
        
        - **svn-repos-password** - The password for the url repository.
        
        - **packages** - a list of packages that are part of the project.
        These are the packages that live in the svn repository and that
        should be released in conjunction with each other.
        
        Defining a Release Template
        ---------------------------
        
        As we saw in the previous section, *Twollo.cfg* refers to a file
        called *Twollo-Release-Template.cfg*.  This is just a base buildout
        configuration.  For the Twollo project, it might look like this::
        
        # Twollo-Release-Template.cfg
        
        [buildout]
        extends = http://download.zope.org/zope3.4/3.4.0/versions.cfg
        parts = test
        find-links = https://build.twollo.com/eggs/
        
        [test]
        recipe = zc.recipe.testrunner
        eggs = twollo.web
        twollo.integration
        twollo.utils
        twollo.rest
        
        [twollo-app]
        recipe = zc.zope3recipes:app
        servers = zserver
        site.zcml = <include package="twollo.web" file="app.zcml" />
        eggs = twollo.web
        
        [zope3]
        location =
        
        When a new release of the Twollo project is made, a ``[versions]``
        section will be added to this configuration file with all the
        correct ``twollow.*`` versions pinned down.
        
        Defining Multiple Deployment Configurations
        -------------------------------------------
        
        Each time you release a Project, you may want to generate different
        buildout configuration files for all the different deployment
        environments you might have.  For example, you may have three
        different environments: Development, QA, and Production. These are
        called variants.  Each environment may need to have the application
        run on different ports, at different log levels, or have other small
        differences.
        
        We can easily generate additional configuration variants by adding
        addtional sections to the *Twollo.cfg* file::
        
        # Twollo.cfg
        
        [Development]
        template = Twollo-Instance-Template.cfg
        port = 9080
        logdir = /opt/twollo/dev/logs
        install-dir = /opt/twollo/dev
        loglevel = debug
        cache-size = 1000
        
        [QA]
        template = Twollo-Instance-Template.cfg
        port = 9082
        logdir = /opt/twollo/qa/logs
        install-dir = /opt/twollo/qa
        loglevel = info
        cache-size = 1000
        
        [Production]
        template = Twollo-Instance-Template.cfg
        port = 8080
        logdir = /var/log/twollo
        install-dir = /opt/twollo/
        loglevel = warn
        cache-size = 200000
        
        We can then have a single *Twollo-Instance-Template.cfg* file that
        uses python's built in string templating to access the variables we
        set in *Twollo.cfg*.  For the Twollo project, it might look like this::
        
        # Twollo-Instance-Template.cfg
        
        [buildout]
        parts += twollo
        directory = %(install-dir)s
        
        [database]
        recipe = zc.recipe.filestorage
        
        [twollo]
        recipe = zc.zope3recipes:instance
        application = twollo-app
        zope.conf =
        
        <zodb>
        cache-size %(cache-size)s
        <filestorage>
        path ${database:path}
        </filestorage>
        </zodb>
        
        <server>
        type WSGI-HTTP
        address %(port)s
        </server>
        
        <eventlog>
        level %(loglevel)s
        <logfile>
        formatter zope.exceptions.log.Formatter
        path %(logdir)s/twollo.log
        </logfile>
        </eventlog>
        
        <accesslog>
        <logfile>
        level info
        path %(logdir)s/twollo-access.log
        </logfile>
        </accesslog>
        
        Releasing a Project
        -------------------
        
        Once you have created all the necessary configuration files, you can
        make your first project release.  This is where the ``build-package``
        script comes in to play.  The first time you run the ``build-package``
        script, the only option you need to pass in will be the configuration
        file.
        
        The ``build-package`` script will prompt you for version information
        about each of the packages it will be releasing as part of the
        Twollo.cfg project.  Your first interaction with the script might look
        like this::
        
        $ build-package -c Twollo.cfg --quiet
        Version for `twollo.integration` : 1.0.0
        The release twollo.integration-1.0.0 does not exist.
        Do you want to create it? yes/no [yes]: yes
        Version for `twollo.rest` : 1.0.0
        The release twollo.rest-1.0.0 does not exist.
        Do you want to create it? yes/no [yes]: yes
        Version for `twollo.utils` : 1.0.0
        The release twollo.utils-1.0.0 does not exist.
        Do you want to create it? yes/no [yes]: yes
        Version for `twollo.web` : 1.0.0
        The release twollo.web-1.0.0 does not exist.
        Do you want to create it? yes/no [yes]: yes
        
        The next time you make a release, you can set the ``-n`` flag for
        ``build-package`` to automatically guess the next version that should
        be released.  It does this by first looking for all the release tags
        of a given package and finding the last changed revision for the trunk
        of a given package.  If any code for the given package was changed
        since the last time it was released, it will bump the most minor
        version number automatically.  If no change has occured, it will
        choose the latest existing release.
        
        You can also use the ``-d`` flag to make ``build-package`` not prompt
        you before creating a new release.
        
        If you need to make a new release from a particular branch, you can
        use the ``-b`` option.  For example, if bug fixes have been made to the
        Twollo-1.x branch, we can create a new release using code from this
        branch like this::
        
        $ build-package -c Twollo.cfg -nb Twollo-1.x
        
        When the new package versions are calculated, they will be versioned
        along the 1.x line, even if you have since created 2.x releases, by
        analyzing the name of the branch.
        
        Caveat when using ``-n`` and ``-d`` on a branch with a branchname ending
        with the version number is that you'll need to have the package versions
        matching the branch version.
        E.g. having a branch: ``branches/twollo-1.9`` will suppose packages like
        ``twollow.web-1.9.x`` and ``twollow.utils-1.9.x`` and so on.
        You should be aware of this also when releasing packages from the trunk.
        Most probably you'll drive development on the trunk and branch out for a
        stable. In this case package versions on the branch should be kept inline.
        
        Installing a Released Project
        -----------------------------
        
        ``keas.build`` also comes with a very simple installation script that
        can be used to quickly install any variant of the released project::
        
        $ install --help
        Usage: install [options]
        
        Options:
        -h, --help            show this help message and exit
        -u URL, --url=URL     The base URL at which the releases can be found.
        -p PROJECT, --project=PROJECT
        The name of the project to be installed.
        -V VARIANT, --variant=VARIANT
        The variant of the project to be installed.
        -v VERSION, --version=VERSION
        The version of the project to be installed.
        -l, --latest          When specified, the latest version will be chosen.
        --username=USER       The username needed to access the site.
        --password=PASSWORD   The password needed to access the site.
        -b PATH, --buildout-path=PATH
        The path to the buildout executable.
        --quiet               When specified, no messages are displayed.
        --verbose             When specified, debug information is created.
        
        For example, to install the latest QA version of the ``Twollo`` project,
        you would run:
        
        $ install -u https://build.twollo.com/buildouts/ -p Twollo -V QA --latest
        
        Creating Helper Scripts
        -----------------------
        
        Sometimes it can be a pain to remember what all the command line
        options are that you need to pass for building your project.
        Fortunately, it is really easy to create helper scripts that just set
        some defaults for you.
        
        For example, to create a build-twollo script, you would add the
        following to a buildout configuration file::
        
        [build-twollo]
        recipe = zc.recipe.egg
        eggs = keas.build
        scripts = build=build-twollo
        initialization =
        sys.argv[1:1] = ['-c', 'Twollo.cfg']
        
        As another example, you could create an install-twollo-dev script that
        automatically installs the latest development release::
        
        [install-twollo-dev]
        recipe = zc.recipe.egg
        eggs = keas.build
        scripts = install=install-twollo-dev
        initialization =
        sys.argv[1:1] = ['-u', 'http://build.twollo.com/buildouts/',
        ' --username', 'someuser', '--password', 'somepass',
        '-p', 'Twollo', '-V', 'Development', '--latest']
        
        The possibilities are endless!
        
        Footnotes
        =========
        
        .. [#ini] See http://en.wikipedia.org/wiki/INI_file
        .. [#webdav] See http://en.wikipedia.org/wiki/WebDAV
        
        
        CHANGES
        =======
        
        0.1.7 (2010-04-26)
        ------------------
        
        - BUGFIX: the dependent config file gathering killed the version pins
        in the master config file
        
        - Improvement: PYPI-like simple index support when checking package versions
        
        - Improvement: Check dependent configs, upload all to the server.
        
        - Improvement: Add ``--force-version`` option.
        
        - Improvement: Add version to svn log comment. That makes life easier (at least
        with TortoiseSVN)
        
        - Improvement: Add ``--force-svnauth`` option.
        
        - Improvement: Add ``--directory`` option to ``install``.
        
        0.1.6 (2009-11-2)
        ------------------
        
        - Improvement: When determining released versions of a branch, support
        branch names ending with .x, as in MyProject-0.3.x
        
        0.1.5 (2009-10-16)
        ------------------
        
        - Improvement: add username and password on install to the URL that buildout
        gets for the .cfg file
        Let's hope that buildout does not leave that password around.
        
        - Improvement(?) or fix: remove twisted dependency that was used in deploy
        for SSH
        
        - Improvement: removed dependency on lxml.  Now we just use pythons
        built-in xml library.
        
        - Improvement: added ``buildout-upload-type`` option.
        See doc for further info
        
        - Bug Fix: multiline template option values crashed on parsing
        
        - Bug Fix: re did not like non-text parameters coming from BeautifulSoup
        
        - Improvement: added the ``--timeout`` option to install
        
        - Improvement: added mypypi buildout file upload support
        
        - Bug Fix: revert to checkout of the complete source tree
        
        0.1.4 (2009-10-01)
        ------------------
        
        - Bug Fix: installer script would crash when -u option was missing a
        trailing /
        
        - Bug Fix: installer script would crash when no variants were found.
        
        - Improvement: Building a package will no longer checkout the entire
        branch just to update the setup.py file with the new version
        number.  Instead only the top level directory is checked out.
        
        - Bug Fix: project build script would crash when variant
        configuration was missing information required by its template. Now
        a helpful error message is printed and no files will be uploaded
        until all have been created successfully.
        
        - Bug Fix: Typing Ctrl+c on the command line no longer causes a
        KeyboardInterrupt traceback to be spit out.
        
        - Bug Fix: when running the installer, if the buildout command prompts
        the user for input, the installer will no longer eat up the prompt.
        
        0.1.3 (2009-09-30)
        ------------------
        
        - Initial public release.
        
        0.1.1 (internal)
        ----------------
        
        - Bug Fix: If a project variant is specified for which no release have
        been made, the build script now exits nicely with a human readable
        error message.
        
        0.1.0 (internal)
        ----------------
        
        - Initial release.
        
Keywords: zope3 release build
Platform: UNKNOWN
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Zope Public License
Classifier: Programming Language :: Python
Classifier: Natural Language :: English
Classifier: Operating System :: OS Independent
Classifier: Framework :: Buildout
