Metadata-Version: 1.0
Name: swab
Version: 0.1.1
Summary: Swab: Simple WSGI A/B testing
Home-page: UNKNOWN
Author: Oliver Cope
Author-email: oliver@redgecko.org
License: BSD
Description: Simple WSGI A/B testing.
        
        Usage
        -----
        
        1. Define an experiment::
        
                from swab import Swab
                s = Swab('/tmp/.swab-test-data')
                s.addexperiment('button-size', ['default', 'larger'], 'order-completed')
        
            Here we have an experiment called 'button-text', for testing two sizes of
            button on a form.
            
            The two variants the experment are called 'small' and 'large'. The first
            listed is always taken as the control variant, and results will be
            presented in terms of how much better or worse the other variants have
            performed compared with the control
        
            The goal associated with the test is called 'order-completed'.
        
        2. Take a WSGI application::
        
                from swab import record_goal, show_variant
        
                def get_button_style(environ):
                    if show_variant('button-size', environ) == 'small':
                        return 'font-size: 100%'
                    else:
                        return 'font-size: 150%'
        
                def myapp(environ, start_response):
        
                    if environ['REQUEST_METHOD'] == 'GET':
                        start_response('200 OK', [])
                        return ['''
                            <html>
                                <body>
                                    <form method="POST">
                                        <button style='%s' type="submit">Complete your order</button>
                                    </form>
                                </body>
                            </html>''' % (get_button_style(environ),)
                        ]
        
                    if environ['REQUEST_METHOD'] == 'POST':
                        swab.record_goal('order-completed', environ)
        
        3. Wrap it in the swab middleware::
        
                app = s.middleware(myapp)
        
        4. Let it run, and after a while browse to http://<server>/swab/results to see
        which variant is winning.
        
        Testing
        --------
        
        To test your competing pages, start Swab in debug mode::
        
        	swab = Swab('/tmp/.swab-test-data', debug=True)
        
        Then append '?swab.<experiment-name>=<variant-name>' to URLs to force one
        variant or another to be shown.
        
        Disable debug mode in production to avoid people messing about with your
        figures.
        
        Basic design
        ------------
        
        Each visitor is assigned an identity which is persisted by means of a cookie.
        The identity is a base64 encoded randomly generated byte sequence. This
        identity is used as a seed for a RNG, which is used to switch visitors into
        test groups.
        
        Every time a test is shown (triggered by the ``show_variant`` function), a line
        is entered into a file at ``<datadir>/<experiment>/<variant>/__all__``
        
        Every time a goal is recorded (triggered by the ``record_goal`` function), a
        line is entered into a file at ``<datadir>/<experiment>/<variant>/<goal>``
        
        Each log line has the format ``<timestamp>:<identity>\n``.
        
        No file locking is used: it is assumed that this will be run on a system where
        each line is smaller than the fs blocksize, allowing us to avoid this overhead.
        The lines may become interleaved, but there should be no risk of corruption
        even with multiple simultaneous writes. See
        http://www.perlmonks.org/?node_id=486488 for a discussion of the issue.
        
        Statistical analysis
        --------------------
        
        The `further reading`_ section contains links to the various articles that
        helped me implement the algorithms and statistical formulae used in Swab.
        However I don't have a background in maths or stats and I would welcome
        contributions or corrections from anyone with a deeper understanding
        of these tests.
        
        
        TODO
        ----
        
            - Allow a set percentage of users to be tipped into each test version
            - Filter search engine spiders out from tests
            - Investigate multivariate testing
            - Allow users to be excluded from tests *after* being shown a test variant (this would js-based exclusion of bots, eg http://www.kalzumeus.com/2010/06/07/detecting-bots-in-javascrip/)
        
        Further reading
        ---------------
        
            - http://www.slideshare.net/patio11/ab-testing-framework-design-3296257/download
            - http://20bits.com/articles/statistical-analysis-and-ab-testing/.
            - http://elem.com/~btilly/effective-ab-testing/
            - http://www.bingocardcreator.com/abingo/resources
            - http://blog.joshbaker.com/2008/10/20/multivariate-testing-fractional-factorial-full-factorial-taguchi/
        
        (c) 2010 Oliver Cope. For licensing information please see LICENSE.txt
        
        
        Version 0.1.1
        
          * Bugfix for ZeroDivisionErrors when no data has been collected
        
        Version 0.1
        
          * Initial release
        
        
Platform: UNKNOWN
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Web Environment
Classifier: License :: OSI Approved :: BSD License
Classifier: Programming Language :: Python
