.. ref-tutorial:

========
Tutorial
========


Quick Start
===========

A simple setup looks like::

    from littleworkers import Pool

    # Define your commands.
    commands = [
        'ls -al',
        'cd /tmp && mkdir foo',
        'date',
        'echo "Hello There."',
        'sleep 2 && echo "Done."'
    ]

    # Setup a pool. Since I have two cores, I'll use two workers.
    lil = Pool(workers=2)

    # Run!
    lil.run(commands)


Philosophy
==========

littleworkers shines when you just want to parallelize something without a lot
of fuss & when you care more about the data/commands to be run.

* Tiny source
* Easy to queue a set of actions
* Works with any runnable commands
* Uses processes
* Non-blocking

Seriously, it's not a replacement for threading or multiprocessing if your
application needs to share a ton of data with the children.


Extension
=========

littleworkers was designed to be extended, so most customizations should be
possible without forking the code. Instead, you should simple subclass ``Pool``
& extend/override the method. You can find the details of each method in the
:doc:`api` docs.


Example Customizations
======================

You want the stdout back::

    import subprocess
    from littleworkers import Pool


    class MyPool(Pool):
        def __init__(self, *args, **kwargs):
            super(MyPool, self).__init__(*args, **kwargs)
            self.collected_output = []

        def create_process(self, command):
            logging.debug("Starting process to handle command '%s'." % command)
            return subprocess.Popen(command, shell=True, stdout=subprocess.PIPE)

        def remove_from_pool(self, pid):
            self.collected_output.append(self.pool[pid].stdout.read())
            return super(MyPool, self).remove_from_pool(pid)

You want to use a ``Queue`` instead of the default ``list``::

    from Queue import Queue, Empty
    from littleworkers import Pool


    class QueuePool(Pool):
        def __init__(self, *args, **kwargs):
            super(QueuePool, self).__init__(*args, **kwargs)
            self.commands = Queue()

        def prepare_commands(self, commands):
            for command in commands:
                self.commands.put(command)

        def command_count(self):
            return self.commands.qsize()

        def next_command(self):
            try:
                return self.commands.get()
            except Empty:
                return None

You want to setup a callback::

    from littleworkers import Pool

    codes = []

    def track(proc):
        codes.append("%s returned status %s" % (proc.pid, proc.returncode))

    commands = [
        'sleep 1',
        'busted_command --here',
        'sleep 1',
    ]
    lil.run(commands, callback=track)
