Metadata-Version: 1.0
Name: options
Version: 1.0
Summary: Container for flexible class, instance, and function call options
Home-page: https://bitbucket.org/jeunice/options
Author: Jonathan Eunice
Author-email: jonathan.eunice@gmail.com
License: UNKNOWN
Description: A module that helps encapsulate option and configuration data using a
        multi-layer stacking (a.k.a. nested context) model.
        
        Classes are expected to define default option values. When instances are
        created, they can be instantiated with "override" values. For any option that
        the instances doesn't override, the class default "shines through" and remains
        in effect. Similarly, individual method calls can set transient values that
        apply just for the duration of that call. If the call doesn't set a value, the
        instance value applies. If the instance didn't set a
        value, the class default applies. Python's ``with`` statement can be used to
        tweak options for essentially arbitrary duration.
        
        This layered or stacked approach is particularly helpful for highly
        functional classes that aim for "reasonable" or "intelligent" defaults and
        behaviors, that allow users to override those defaults at any time, and that
        aim for a simple, unobtrusive API. It can also be used to provide flexible
        option handling for functions.
        
        This option-handling pattern is based on delegation rather than inheritance.
        It's described in `this StackOverflow.com discussion of "configuration sprawl" 
        <http://stackoverflow.com/questions/11702437/where-to-keep-options-values-paths-to-important-files-etc/11703813#11703813>`_.
        
        Unfortunately, it's a bit hard to demonstrate the virtues of this approach with
        simple code. Python already supports flexible function arguments, including
        variable number of arguments (``*args``) and optional keyword arguments
        (``**kwargs``). Combined with object inheritance, base Python features already
        cover a large number of use cases and requirements. But when you have a large
        number of configuration and instance variables, and when you might want to
        temporarily override either class or instance settings, things get dicey. This
        messy, complicated space is where ``options`` truly begins to shine.
        
        Usage
        =====
        
        ::
        
            from options import Options, attrs
            
            class Shape(object):
            
                options = Options(
                    name   = None,
                    color  = 'white',
                    height = 10,
                    width  = 10,
                )
                
                def __init__(self, **kwargs):
                    self.options = Shape.options.push(kwargs)
                
                def draw(self, **kwargs):
                    opts = self.options.push(kwargs)
                    print attrs(opts)
        
            one = Shape(name='one')
            one.draw()
            one.draw(color='red')
            one.draw(color='green', width=22)
            
        yielding::
        
            color='white', width=10, name='one', height=10
            color='red', width=10, name='one', height=10
            color='green', width=22, name='one', height=10
        
        So far we could do this with instance variables and standard arguments. It
        might look a bit like this::
        
            class ClassicShape(object):
        
                def __init__(self, name=None, color='white', height=10, width=10):
                    self.name   = name
                    self.color  = color
                    self.height = height
                    self.width  = width
        
        but when we got to the ``draw`` method, things would be quite a bit messier.::
        
                def draw(self, **kwargs):
                    name   = kwargs.get('name',   self.name)
                    color  = kwargs.get('color',  self.color)
                    height = kwargs.get('height', self.height)
                    width  = kwargs.get('width',  self.width)
                    print "color='{}', width={}, name='{}', height={}".format(color, width, name, height)
                
        One problem here is that we broke apart the values provided to ``__init__()`` into
        separate instance variables, now we need to re-assemble them into something unified.
        And we need to explicitly choose between the ``**kwargs`` and the instance variables.
        It gets repetitive, and is not pretty. Another classic alternative, using
        native keyword arguments, is no
        better::
        
                def draw2(self, name=None, color=None, height=None, width=None):
                    name   = name   or self.name
                    color  = color  or self.color
                    height = height or self.height
                    width  = width  or self.width
                    print "color='{}', width={}, name='{}', height={}".format(color, width, name, height)
        
        If we add just a few more instance variables, we have the `Mr. Creosote
        <http://en.wikipedia.org/wiki/Mr_Creosote>`_ of class design on our hands. Not
        good. Things get worse if we want to set default values for all shapes in the
        class. We have to rework every method that uses values, the ``__init__`` method,
        *et cetera*. We've entered "just one more wafer-thin mint..." territory.
        
        But with ``options``, it's easy::
        
            Shape.options.set(color='blue')
            one.draw()
            one.draw(height=100)
            one.draw(height=44, color='yellow')
            
        yields::
        
            color='blue', width=10, name='one', height=10
            color='blue', width=10, name='one', height=100
            color='yellow', width=10, name='one', height=44
        
        In one line, we reset the default for all ``Shape`` objects.
        
        The more options and settings a class has, the more unwieldy the class and
        instance variable approach becomes, and the more desirable the delegation
        alternative. Inheritance is a great software pattern for many kinds of data and
        program structures, but it's a bad pattern for complex option and configuration
        handling. For richly featured classes, the delegation pattern ``options`` proves
        simpler. Supporting even a large number of options requires almost no additional
        code and imposes no additional complexity or failure modes. By consolidating
        options into one place, and by allowing neat, attribute-style access, everything
        is kept tidy. We can add new options or methods with confidence::
        
            def is_tall(self, **kwargs):
                opts = self.options.push(kwargs)
                return opts.height > 100
        
        Under the covers, ``options`` uses a variation on the ``ChainMap`` data
        structure (a multi-layer dictionary) to provide its option stacking. Every
        option set is stacked on top of previously set option sets, with lower-level
        values shining through if they're not set at higher levels. This stacking or
        overlay model resembles how local and global variables are managed in many
        programming languages.
        
        Magic Parameters
        ================
        
        Python's ``*args`` variable-number of arguments and ``**kwargs`` keyword
        arguments are sometimes called "magic" arguments. ``options`` takes this up a
        notch, allowing setters much like Python's ``property`` function or
        ``@property`` decorator. This allows arguments to be interpreted on the fly.
        This is useful, for instance, to provide relative rather than just absolute
        values. As an example, say that we added this code after ``Shape.options`` was
        defined::
        
            options.magic(
                height = lambda v, cur: cur.height + int(v) if isinstance(v, str) else v,
                width  = lambda v, cur: cur.width  + int(v) if isinstance(v, str) else v
            )
            
        Now, in addition to absolute ``height`` and ``width`` parameters which are
        provided by specifying ``int`` (integer/numeric) values, your module
        auto-magically supports relative parameters for ``height`` and ``width``.::
        
            one.draw(width='+200')
            
        yields::
        
            color='blue', width=210, name='one', height=10
            
        This can be as fancy as you like, defining an entire domain-specific expression language.
        But even small functions can give you a great bump in expressive power. For example,
        add this and you get full relative arithmetic capability (``+``, ``-``, ``*``, and ``/``)::
        
            def relmath(value, currently):
                if isinstance(value, str):
                    if value.startswith('*'):
                        return currently * int(value[1:])
                    elif value.startswith('/'):
                        return currently / int(value[1:])
                    else:
                        return currently + int(value)
                else:
                    return value
            
            ...
            
            options.magic(
                height = lambda v, cur: relmath(v, cur.height),
                width  = lambda v, cur: relmath(v, cur.width)
            )
        
        Then::
        
            one.draw(width='*4', height='/2')
        
        yields::
        
            color='blue', width=40, name='one', height=5
            
        Magically interpreted parameters are the sort of thing that one doesn't need
        very often or for every parameter--but when they're useful, they're *enormously*
        useful and highly leveraged, leading to much simpler, much higher function APIs.
        We call them 'magical' here because of the "auto-magical" interpretation, but
        they are really just analogs of Python object properties. The magic function is
        basically a "setter" for a dictionary element.
        
        Design Considerations
        =====================
        
        In general, classes will define a set of methods that are "outwards
        facing"--methods called by external code when consuming the class's
        functionality. Those methods should generally expose their options through
        ``**kwargs``, creating a local variable (say ``opts``) that represents the sum
        of all options in use--the full stack of call, instance, and class options,
        including any present magical interpretations.
        
        Internal class methods--the sort that are not generally called by external code,
        and that by Python convention are often prefixed by an underscore (``_``)--these
        generally do not need ``**kwargs``. They should accept their options as a
        single variable (say ``opts`` again) that the externally-facing methods will
        provide.
        
        For example, if ``options`` didn't provide the nice formatting function ``attrs``,
        we might have designed our own::
        
            def _attrs(self, opts):
                nicekeys = [ k for k in opts.keys() if not k.startswith('_') ]
                return ', '.join([ "{}={}".format(k, repr(opts[k])) for k in nicekeys ])
           
            def draw(self, **kwargs):
                opts = self.options.push(kwargs)
                print self._attrs(opts)
                
        ``draw()``, being the outward-facing API, accepts general arguments and
        manages their stacking (by ``push``ing ``kwargs`` onto the instance options).
        When the internal ``_attrs()`` method is called, it is handed a pre-digested
        ``opts`` package of options.
        
        A nice side-effect of making this distinction: Whenever you see a method with
        ``**kwargs``, you know it's outward-facing. When you see a method with just
        ``opts``, you know it's internal.
        
        Objects defined with ``options`` make excellent "callables."
        Define the ``__call__`` method, and you have a very nice analog of
        function calls.
        
        ``options`` has broad utility, but it's not for every class or module. It best
        suits high-level front-end APIs that multiplex lots of potential functionality,
        and wish/need to do it in a clean/simple way. Classes for which the set of
        instance variables is small, or functions/methods for which the set of
        known/possible parameters is limited--these work just fine with classic Python
        calling conventions. For those, ``options`` is overkill. "Horses for courses."
        
        Setting and Unsetting
        =====================
        
        Using ``options``, objects often become "entry points" that represent both
        a set of capabilities and a set of configurations for how that functionality
        will be used. As a result, you may want to be able to set the object's
        values directly, without referencing their underlying ``options``. It's
        convenient to add a ``set()`` method, then use it, as follows::
        
            def set(self, **kwargs):
                self.options.set(**kwargs)
                
            one.set(width='*10', color='orange')
            one.draw()
            
        yields::
        
            color='orange', width=100, name='one', height=10
        
        ``one.set()`` is now the equivalent of ``one.options.set()``. Or continue using
        the ``options`` attribute explicitly, if you prefer that.
        
        Values can also be unset.::
        
            from options import Unset
        
            one.set(color=Unset)
            one.draw()
            
        yields::
        
            color='blue', width=100, name='one', height=10
            
        Because ``'blue'`` was the color to which ``Shape`` had be most recently set.
        With the color of the instance unset, the color of the class shines through.
        
        **NOTA BENE** while options are ideally accessed with an attribute notion,
        the preferred of setting options is through method calls: ``set()`` if
        accessing directly, or ``push()`` if stacking values as part of a method call.
        These perform the interpretation and unsetting magic;
        straight assignment does not. In the future, ``options`` may provide an
        equivalent ``__setattr__()`` method to allow assignment--but not yet.
        
        Leftovers
        =========
        
        ``options`` expects you to define all feasible and legitimate options at the
        class level, and to give them reasonable defaults.
        
        None of the initial settings ever have magic applied. Much of the
        expected interpretation "magic" will be relative settings, and relative settings
        require a baseline value. The top level is expected and demanded to provide a
        reasonable baseline.
        
        Any options set "further down" such as when an instance is created or a method
        called should set keys that were already-defined at the class level.
        
        However, there are cases where "extra" ``**kwargs`` values may be provided and
        make sense. Your object might be a very high level entry point, for example,
        representing very large buckets of functionality, with many options. Some of
        those options are relevant to the current instance, while others are intended as
        pass-throughs for lower-level modules/objects. This may seem a doubly rarefied
        case--and it is, relatively speaking. But it does happen, and when you need
        multi-level processing, it's really, really super amazingly handy to
        have it.
        
        ``options`` supports this in its core ``push()`` method by taking the values
        that are known to be part of the class's options, and deleting those from
        ``kwargs``. Any values left over in the ``kwargs`` ``dict`` are either errors,
        or intended for other recipients.
        
        As yet, there is no automatic check for leftovers.
        
        The Magic APIs
        ==============
        
        The callables (usually functions, lambda expressions, static methods, or methods) called
        to preform magical interpretation can be called with 1, 2, or 3 parameters.
        ``options`` inquires as to how many parameters the callable accepts. If it
        accepts only 1, it will be the value passed in. Cleanups like "convert to upper case"
        can be done, but no relative interpretation. If it accepts 2 arguments,
        it will be called with the value and the current option mapping, in that order.
        (NB this order reverses the way you may think logical. Caution advised.) If the
        callable requires 3 parameters, it will be ``None``, value, current mapping. This
        supports method calls, though has the defect of not really
        passing in the current instance.
        
        A decorator form, ``magical()`` is also supported. It must be given the
        name of the key exactly::
        
            @options.magical('name')
            def capitalize_name(self, v, cur):
                return ' '.join(w.capitalize() for w in v.split())
        
        The net is that you can provide just about any kind of callable.
        But the meta-programming of the magic interpretation API could use a little work.
        
        Subclassing
        ===========
        
        Subclass options may differ from superclass options. Usually they will share
        many options, but some may be added, and others removed. To modify the set of
        available options, the subclass defines its options with the ``add()`` method to
        the superclass options. This creates a layered
        effect, just like ``push()`` for an instance. The difference is, ``push()`` does
        not allow new options (keys) to be defined; ``add()`` does. It is also possible to
        assign the special null object ``Prohibited``, which will disallow instances of the
        subclass from setting those values.::
        
            options = Superclass.options.add(
                func   = None,
                prefix = Prohibited,  # was available in superclass, but not here
                suffix = Prohibited,  # ditto
            )
            
        Because some of the "additions" can be prohibitions (i.e. removing
        particular options from being set or used), this is "adding to" the superclass's
        options in the sense of "adding a layer onto" rather than strict "adding
        options."
        
        An alternative is to copy (or restate) the superclass's options. That suits
        cases where the subclass is highly independent, and where changes to the
        superclass's options should not effect the subclass's options.
        With ``add()``, they remain linked in the same way as instances and classes are.
        
        Transients
        ==========
        
        Some options do not make sense as permanent values--they are needed only as
        transient settings in the context of individual calls. The special null value
        ``Transient`` can be assigned as an option value to signal this.
        
        Flat Arguments
        ==============
        
        Sometimes it's more elegant to provide some arguments as flat, sequential values
        rather than by keyword. In this case, use the ``addflat()`` method::
            
            def __init__(self, *args, **kwargs):
                self.options = Quoter.options.push(kwargs)
                self.options.addflat(args, ['prefix', 'suffix'])
                
        to consume optional ``prefix`` and ``suffix`` flat arguments. This makes the following
        equivalent::
        
            q1 = Quoter('[', ']')
            q2 = Quoter(prefix='[', suffix=']')
        
        An explicit ``addflat()`` method is provided not as much for Zen of Python
        reasons ("Explicit is better than implicit."), but because flat arguments are
        commonly combined with abbreviation/shorthand conventions, which may require
        some logic to implement. For example, if only a ``prefix`` is given as a flat
        argument, you may want to use the same value to implicitly set the ``suffix``.
        To this end, addflat returns the set of keys that it consumed::
        
                if args:
                    used = self.options.addflat(args, ['prefix', 'suffix'])
                    if 'suffix' not in used:
                        self.options.suffix = self.options.prefix
        
        Related Work
        ============
        
        A huge amount of work, both in Python and beyond, has gone into
        the effective management of configuration information. 
        
         * Program defaults. Values pre-established by developers, often
           as ``ALL_UPPERCASE_IDENTIFIERS`` or as keyword default to 
           functions.
        
         * Configuration file format parsers/formatters.
           Huge amounts of the INI, JSON, XML, and YAML specifications and
           toolchains, for example, are configuration-related. 
           There are many. `anyconfig <https://pypi.python.org/pypi/anyconfig>`_
           is perhaps of interest for its flexibility.
           You could
           probably lump into this group binary data marshalling schemes
           such as ``pickle``.
        
         * Command-line argument parsers. These are all about taking configuration
           information from the command line. 
           `argh <https://pypi.python.org/pypi/argh>`_ is one I particularly
           like for its simple, declarative nature.
           (`aaargh <https://pypi.python.org/pypi/aaargh>`_ is similar.)
        
         * System and environment introspection. The best known of these
           would be ``sys.argv`` and
           ``s.environ`` to get command line arguments and the values
           of operating system environment variables (especially when running
           on Unixy platforms). But any code that asks "Where am I running?"
           or "What is my IP address?" or otherwise inspects its current
           execution environment and configures itself accordingly
           is doing a form of configuration discovery.
        
         * Attribute-accessible dictionary objects. It is incredibly easy to
           create simple versions of this idea in Python--and rather tricky
           to create robust, full-featured versions. Caveat emptor.  `stuf
           <https://pypi.python.org/pypi/stuf>`_ and `treedict
           <https://pypi.python.org/pypi/treedict>`_ are cream-of-the-crop
           implementations of this idea. I have not tried `confetti
           <https://pypi.python.org/pypi/confetti>`_ or 
           `Yaco <https://pypi.python.org/pypi/Yaco>`_, but they look like
           variations on the same theme.
        
         * The portion of Web frameworks concerned with getting and setting 
           cookies, URL query and hash attributes, form variables, and/or 
           HTML5 local stroage. Not that 
           these are pariticularly secure, scalable, or robust sources...but they're 
           important configuration information nonetheless.
        
         * While slighly afield, database interface modules are often used for
           querying configuration information from SQL or NoSQL databases.
        
         * Some object metaprogramming systems. That's a mouthful, right?
           Well some modules implement metaclasses that change the basic
           behavior of objects. `value <https://pypi.python.org/pypi/value>`_
           for example provides very common-sense treatment of object
           instantiation with out all the Javaesque 
           ``self.x = x; self.y = y; self.z = z`` repetition.
           ``options`` similarly redesigns how parameters should be passed
           and object values stored.
        
         * Combomatics. Many configuration-related modules combine two
           or more of these approaches. E.g. 
           `yconf <https://pypi.python.org/pypi/yconf>`_
           combines YAML config file parsing with ``argparse`` command
           line parsing. In the future, ``options`` will also follow
           this path. There's no need to take programmer time and 
           attention for several different low-level
           configuration-related tasks.
        
         * Dependency injection frameworks are all about providing configuration
           information from outside code modules. They tend to be rather
           abstract and have a high "activation energy," but the more complex
           and composed-of-many-different-components your system is, the
           more valuable the "DI pattern" becomes.
        
         * Other things. 
           `conflib <https://pypi.python.org/pypi/conflib>`_, uses dictionary
           updates to stack
           default, global, and 
           local settings; it also provides a measure of validation.
        
        
        This diversity, while occasionally frustrating, makes some sense.
        Configruation data, after all, is just "state," and "managing state"
        is pretty much what all computing is about.
        Pretty much every program has to do it. That so many use so
        many different, home-grown ways is why there's such a good opportunity.
        
        `Flask's documentation <http://flask.pocoo.org/docs/config/#configuring-from-files>`_ is a real-world example of how "spread everywhere" this can be,
        with some data coming from the program, some from files, some from
        environment variables, some from Web-JSON, etc.
        
        Notes
        =====
        
         * This is a work in progress. The underlying techniques have been successfully
           used in multiple projects, but it remains in an evolving state as a
           standalone module. The API may change over time. Swim at your own risk.
           
         * Open question: Could "magic" parameter processing be
           improved with a properties-based approach akin to that of `basicproperty <http://pypi.python.org/pypi/basicproperty>`_,
           `propertylib <http://pypi.python.org/pypi/propertylib>`_,
           `classproperty <http://pypi.python.org/pypi/classproperty>`_, and `realproperty <http://pypi.python.org/pypi/rwproperty>`_.
           
         * Open question: Should "magic" parameter setters be allow to change
           multiple options at once? A use case for this: "Abbreviation"
           options that combine multiple changes into one compact option. These would
           probably not have stored values themselves. It would require setting the
           "dependent" option values via side-effect rather than functional return values.
           
         * The author, `Jonathan Eunice <mailto:jonathan.eunice@gmail.com>`_ or
           `@jeunice on Twitter <http://twitter.com/jeunice>`_
           welcomes your comments and suggestions.
        
        Recent Changes
        ==============
        
         * Commenced automated multi-version testing with
           `pytest <http://pypi.python.org/pypi/pytest>`_
           and `tox <http://pypi.python.org/pypi/tox>`_. Now
           successfully packaged for, and tested against, Python 2.6, 2.7, 3.2, and 3.3.
           
         * Options is now packaged for, and tested against, PyPy 1.9 (based on 2.7.2).
           The underlying ``stuf`` module and ``orderedstuf`` class is not
           certified for PyPy, and it exhibits a bug with file objects on PyPy.
           ``options`` works around this bug, and tests fine on PyPy. Still, 
           buyer beware. 
           
         * Versions subsequent to 0.200 require a late-model version of ``stuf`` to
           avoid a problem its earlier iterations had with file objects. Versions after 0.320
           depend on ``stuf`` for ``chainstuf``, rather than the ``otherstuf`` sidecar.
        
         * Now packaged as a package, not a set of modules. ``six`` module now required only for testing.
         
         * API for ``push()`` and ``addflat()`` cleaned up to explicitly delink those methods.
         
        Installation
        ============
        
        ::
        
            pip install options
        
        To ``easy_install`` under a specific Python version (3.3 in this example)::
        
            python3.3 -m easy_install options
            
        (You may need to prefix these with "sudo " to authorize installation.)
        
Keywords: options config configuration parameters arguments
Platform: UNKNOWN
Classifier: Development Status :: 4 - Beta
Classifier: Operating System :: OS Independent
Classifier: License :: OSI Approved :: BSD License
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 2.6
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.2
Classifier: Programming Language :: Python :: 3.3
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Programming Language :: Python :: Implementation :: PyPy
Classifier: Topic :: Software Development :: Libraries :: Python Modules
