.. _topics-install:

=======================
How to Install Merengue
=======================

This document will get you up and running with Merengue.

.. admonition:: Conventions

    We need to assume an operating system in order to describe the installation
    of the system packages required for Merengue. For these directions, we
    will assume the use of  *Ubuntu OS*. If you are not using *Ubuntu OS*,
    you will need to adapt the following ``apt-get`` commands to your
    specific operating system.


Previous Dependencies
=====================

* ``Python 2.5+``: The Python programming language.

.. code-block:: bash

    $ sudo apt-get install python2.5

* ``python-setuptools``: Required for installing Python packages.

.. code-block:: bash

    $ sudo apt-get install python-setuptools

* ``ffmpeg``: FFmpeg is a complete, cross-platform solution to record, convert, and stream audio and video. It includes libavcodec - the leading audio/video codec library. 

.. code-block:: bash

    $ sudo apt-get install ffmpeg libavcodec-unstripped-52 libavdevice-unstripped-52 libavformat-unstripped-52 libavutil-unstripped-49 libpostproc-unstripped-51 libswscale-unstripped-0 yamdi

* ``gettext``: A utility for managing translations catalogs.

    $ sudo apt-get install gettext

PIL Installation
----------------

The `Python Imaging Library`_ (PIL) is usually one of the hardest components to
install in a Python environment from Python packages, because it requires several system
packages to properly compile all of the image manipulation libraries.

To install PIL correctly (for an Ubuntu/Debian environment), install the
following dependencies:

.. code-block:: bash

    $ sudo apt-get install libfreetype6-dev
    $ sudo apt-get install python-tk
    $ sudo apt-get install tcl8.5-dev
    $ sudo apt-get install tk8.5-dev
    $ sudo apt-get install liblcms1-dev liblcms-utils

.. _`Python Imaging Library`: http://www.pythonware.com/products/pil/

Other Recommended Dependencies
------------------------------

* ``ipython``: a Python shell with autocomplete, help, etc.

.. code-block:: bash

    $ apt-get install ipython


Installing with easy_install
============================

The easiest way to install Merengue is to use easy_install. All dependencies will be
installed automatically.

Use the following command:

.. code-block:: bash

    $ sudo easy_install merengue

Go to the `project creation`_ section to continue.

.. _`project creation`: #merengue-project-creation


Manual Installation
===================

Installing Django
-----------------

Merengue uses Django 1.3. You can install
it with these instructions:

.. code-block:: bash

    $ wget --trust-server-names http://www.djangoproject.com/download/1.3/tarball/
    $ tar xzvf Django-1.3.tar.gz
    $ cd Django-1.3
    $ sudo python setup.py install

Test to see if you have the correct Django version:

.. code-block:: bash

    $ python -c "import django; print django.VERSION"
    (1, 3, 0, 'final', 0)

Downloading Merengue
--------------------

Download the Merengue source from a tarball or from the subversion repository:

.. code-block:: bash

    $ svn checkout https://svn.merengueproject.org/trunk/merengueproj
    $ cd merengueproj

Installing Python Dependencies
------------------------------

The `requirements.txt`_ file lists the Python packages required by Merengue. You
will need to install these dependencies via Python eggs. You can use ``easy_install``
to install each egg separately, or use ``pip``  to install all of the required
packages via the following command:

.. code-block:: bash

    $ sudo pip install -r requirements.txt

.. _`requirements.txt`: http://dev.merengueproject.org/browser/trunk/merengueproj/requirements.txt

Installing Merengue
-------------------

Execute ``setup.py`` to complete the manual install process:

.. code-block:: bash

    $ sudo python setup.py install


Merengue Project Creation
=========================

When beginning a Merengue project, you can begin your development in one of two
modes: production mode or development mode.

In production mode, your project will contain a "frozen" version of Merengue
(the code will be copied). So, if you are using Merengue subversion code, any
new (i.e. following) code updates will not affect your project.

In development mode, your project will contain symbolic links back to the
Merengue code in the subversion repository. In this case, your project will be
affected by any code updates to the Merengue subversion repository (including any
new features and fixes that have been implemented, as well as any new errors or
changes that may cause compatibility problems with your code).

* To create a project in *production mode*:

.. code-block:: bash

    $ merengue-admin.py startproject myproject


* To create a project in *development mode*:

.. code-block:: bash

    $ merengue-admin.py startproject myproject --develop


Language(s) Configuration
-------------------------

In Merengue, language configuration should be done before database creation
because data models are created with translatable fields (the number of columns
created is equal to the number of configured languages).

The ``LANGUAGE_CODE`` and ``LANGUAGES`` parameters must be defined in the
configuration file ``myproject/settings.py``:

    .. code-block:: python

        LANGUAGE_CODE = 'es'

        LANGUAGES = (
            ('es', ugettext('Español')),
            ('en', ugettext('English')),
            ('fr', ugettext('Français')),
        )

.. admonition:: More Information

    See the `django-transmeta`_ application for more information about
    translatable models.

.. _`django-transmeta`: http://code.google.com/p/django-transmeta/

Database Installation and Configuration
---------------------------------------

The database connection must be defined in the configuration file by setting
the ``DATABASE_*`` parameters. You can use all `database engines supported by Django.`_

.. _`database engines supported by Django.`: http://docs.djangoproject.com/en/1.3/ref/settings/#database-engine


.. admonition:: PostgreSQL Assumption

    The following instructions assume the use of PostgreSQL as the database engine.
    Other databases can be used, but the steps below (specifically,
    the database commands) will have to be adapted for use with your database.

Let's suppose that you were configuring a PostgreSQL database with the
following parameters:

    .. code-block:: python

        DATABASE = {
            'default': {
                'ENGINE': 'django.db.backends.postgresql_psycopg2',
                'NAME': 'myproject',
                'USER': 'myprojectuser',
                'PASSWORD': 'password',
                'HOST': '',
                'PORT': '',
            }
        }

Install the  database and the appropriate python dependencies: ``postgresql``, ``python-psycopg2``.

* ``postgresql``: the object-relational database system that we will use.

    .. code-block:: bash

        $ apt-get install postgresql

* ``python-psycopg``: the python interface to the PostgreSQL database.

    .. code-block:: bash

        $ apt-get install python-psycopg2


Creating the Project Database
-----------------------------

.. admonition:: Note

    We assume that your user has superadmin permissions in PostgreSQL.

The PostgreSQL database and user is created with these instructions:

.. code-block:: bash

    $ createuser myprojectuser
    $ createdb --owner=myprojectuser myproject

We have to permit connections to the database from the local computer. Edit
/etc/postgresql/X.X/main/pg_hba.conf and add the following line (not at
the end):

.. code-block:: bash

    local myproject myprojectuser trust
    local test_myproject myprojectuser trust # necessary for tests

Reload pg_hba.conf in PostgreSQL server with the following command:

.. code-block:: bash
 
    $ /etc/init.d/postgresql-X.X reload

Restart PostgreSQL and check your user access with this command:

.. code-block:: bash

    $ psql myproject -U myprojectuser

Building Initial Merengue Models
--------------------------------

You will need to build the initial Merengue database models with the following two commands:

.. code-block:: bash

    $ python manage.py syncdb
    $ python manage.py migrate

You can ignore the creation of the superuser because a user named ``admin`` with
the password ``admin`` will be created automatically.


Collecting Media from Installed Django Applications
---------------------------------------------------

Some Django applications use media files that Merengue needs. To include these
files in your project, execute the command:

.. code-block:: bash

    $ python manage.py sync_apps_media --link

.. admonition:: Note

    You can remove the ``--link`` option if you want Merengue to copy the media
    files instead of linking to them.


View the Site
-------------

Go to your project directory and test the Django development server to make sure
it starts without problems:

.. code-block:: bash

    $ python manage.py runserver

Open the web browser and enter the following URL: http://localhost:8000/admin/ to see
if the admin web interface is running successfully.

Use the admin username and password below to access the admin interface:

.. code-block:: bash

    user: admin
    password: admin

To see the public site, enter the URL: http://localhost:8000/


Installation with GIS
=====================

By default, the GIS features are disabled in Merengue.

To add GIS support to your Merengue project, you will need to
perform some additional installation and configuration instructions. This section can
be skipped if your project does not require GIS features.


Settings Changes
----------------

To use GIS, change one line in your project ``settings.py`` file:

.. code-block:: python

    USE_GIS = True


PostGIS dependencies
--------------------

.. admonition:: Note

    Again, we assume that you are using PostgreSQL as your database backend as
    well as PostGIS for GIS extensions. If you are using another database, you
    have to adapt the following instructions.
    
GeoDjango install instructions are online at http://geodjango.org/docs/install.html

In addition to the dependencies listed above, you should install the following
package:

* ``geopy``: makes it easy for developers to locate the coordinates of
  addresses, cities, countries, and landmarks across the globe.

.. code-block:: bash

    $ easy_install geopy


GIS Extensions Installation
---------------------------

With Ubuntu 10.04, all of the GIS extensions for postgresql-8.4 can be installed
with the following commands:

.. code-block:: bash

    $ sudo apt-get install binutils libgdal1-1.6.0 postgresql-8.4-postgis python-psycopg2 python-setuptools
    $ export PG_CONFIG='/usr/lib/postgresql/8.4/bin/pg_config'

If you are using Ubuntu with a version newer than 8.04 and older than 10.04, then
install all of the GIS extensions for postgresql-8.3 with the commands below:

.. code-block:: bash

    $ sudo apt-get install binutils libgdal1-1.5.0 postgresql-8.3-postgis python-psycopg2 python-setuptools
    $ export PG_CONFIG='/usr/lib/postgresql/8.3/bin/pg_config'


.. admonition:: Note

    If you are using postgresql-8.4 with a version of Ubuntu earlier than 10.04
    and there exists no ``postgresql-8.4-postgis`` package, then use
    ``postgresql-8.3`` packages along with the following options.  Use option
    ``-p 5433`` with ``psql``, ``createuser``, and ``createdb``. Configure your
    Merengue project to use that port (5433) with the database.

If your distribution does not have the package postgis it will be necessary to
compile it:

1. Install the PostgreSQL source headers:

.. code-block:: bash

    $ sudo apt-get install postgresql-server-dev-X.X
    $ export PG_CONFIG='/usr/lib/postgresql/X.X/bin/pg_config'

2. Compile the `GEOS`_ library:

.. code-block:: bash

    $ cd resources
    $ tar xjf geos-x.x.x.tar.bz2
    $ cd geos-x.x.x
    $ ./configure && make
    $ sudo make install

3. Compile the `PROJ.4`_ library:

.. code-block:: bash

    $ cd resources
    $ tar xzf proj-x.x.x.tar.gz
    $ cd proj-x.x.x/nad
    $ tar xzf ../../proj-datumgrid-x.x.tar.gz
    $ cd ..
    $ ./configure
    $ make
    $ sudo make install
    $ cd ..

4. Compile `PostGIS`_:

.. code-block:: bash

    $ tar xzf postgis-x.x.x.tar.gz
    $ cd postgis-x.x.x
    $ ./configure --with-pgsql=$PG_CONFIG --datadir=`$PG_CONFIG --sharedir`
    $ make
    $ sudo make install
    $ cd ..

5. Compile the `GDAL`_ library:

.. code-block:: bash

    $ tar xzf gdal-x.x.x.tar.gz
    $ cd gdal-x.x.x
    $ ./configure
    $ make # Go get some coffee, this takes a while.
    $ sudo make install
    $ cd ..

6. Update all dynamic libraries:

.. code-block:: bash

    $ sudo ldconfig -v

.. admonition:: Note

    Make sure that the file ``/etc/ld.so.conf`` is located in the directory
    ``/usr/local/lib``

.. _`GEOS`: http://trac.osgeo.org/geos/
.. _`PROJ.4`: http://download.osgeo.org/proj/
.. _`PostGIS`: http://postgis.refractions.net/
.. _`GDAL`: http://trac.osgeo.org/gdal/


Changes in Database Configuration
---------------------------------

Create the postgis template database:

.. code-block:: bash

    $ createdb -E UTF8 template_postgis
    $ createlang -d template_postgis plpgsql

* In Ubuntu (version 10.04 or greater):

.. code-block:: bash

    $ PG_GIS_TEMPLATES=/usr/share/postgresql/8.4/contrib/
    $ psql -d template_postgis -f $PG_GIS_TEMPLATES/postgis.sql
    $ psql -d template_postgis -f $PG_GIS_TEMPLATES/spatial_ref_sys.sql
    $ psql -d template_postgis -c "GRANT ALL ON geometry_columns TO PUBLIC;"
    $ psql -d template_postgis -c "GRANT ALL ON spatial_ref_sys TO PUBLIC;"

* In Ubuntu (versions greater than 8.04 and less than 10.04 -- those using the postgres-8.3-postgis package):

.. code-block:: bash

    $ PG_GIS_TEMPLATES=/usr/share/postgresql-8.3-postgis/
    $ psql -d template_postgis -f $PG_GIS_TEMPLATES/lwpostgis.sql
    $ psql -d template_postgis -f $PG_GIS_TEMPLATES/spatial_ref_sys.sql
    $ psql -d template_postgis -c "GRANT ALL ON geometry_columns TO PUBLIC;"
    $ psql -d template_postgis -c "GRANT ALL ON spatial_ref_sys TO PUBLIC;"

.. admonition:: Note

     Remember that if you have postgresql-8.3 and 8.4 running at the
     same time you need to add ``-p 5433`` option to the ``psql``, ``createdb``, and
     ``createlang`` commands.

* In any other case:

.. code-block:: bash

    $ psql -d template_postgis -f `$PG_CONFIG --sharedir`/lwpostgis.sql
    $ psql -d template_postgis -f `$PG_CONFIG --sharedir`/spatial_ref_sys.sql
    $ psql -d template_postgis -c "GRANT ALL ON geometry_columns TO PUBLIC;"
    $ psql -d template_postgis -c "GRANT ALL ON spatial_ref_sys TO PUBLIC;"

Creating the Project Database
-----------------------------

.. admonition:: Note

     If you used the previous (non-GIS) installation instructions, you will need
     to re-create the project database, because that database will not
     work with GIS enabled. Before continuing, you will need to delete the
     previous database.

Database creation changes slightly because the project database has to use
``template_postgis`` as the database template (with all postgis extensions
included).

.. code-block:: bash

    $ createuser myproject
    $ createdb -T template_postgis --owner=myproject myproject


Upgrading Merengue in Development Mode
======================================

If your Merengue project is configured for development mode, your models may
change when you update the Merengue code. To upgrade your models after a code
update, you can use the South ``migrate`` command as follows:

.. code-block:: bash

    $ python manage.py migrate

This ``migrate`` command does not affect the installed plugins in the
project. In order to perform the database migrations for plugins, you will need to execute
the ``migrate_plugins`` command:

.. code-block:: bash

    $ python manage.py migrate_plugins

Also, we recommend following the `changes`_ on the Merengue `skel project`_. This
documentation may be helpful in incorporating these changes into your own project.

.. _`changes`: https://tracpub.yaco.es/merengue/log/trunk/merengueproj/projskel
.. _`skel project`: http://tracpub.yaco.es/merengue/browser/trunk/merengueproj/projskel


Database Backup
===============

A database backup (stored in SQL) may be created with the following command:

.. code-block:: bash

    $ python manage.py backupdb

.. note::

    When using PostgreSQL as the database backend, you will need to install the
    ``pexpect`` python package.

.. _topics-deploying:

==================
Deploying Merengue
==================

Since Merengue is based on Django, we recommend the reading of the `Deploying Django`_
documentation before deploying Merengue.

The recommended deployment configuration for Merengue is to use nginx with WSGI.
However, other Django deployment configurations can be used.

This section includes some points to consider when planning a Merengue site deployment.

Serve Media Files with Web Server
====================================

To help keep Merengue plugins and apps portable and reusable, Merengue plugins (and some
Merengue apps) can contain media files located in a ``media`` directory inside
each plugin directory. Merengue has implemented the ``merengue.views.static.serve``
view that virtualizes the media directory specified as ``MEDIA_ROOT``. Thus,
the Django development server will work correctly.

However, if you want to set the configuration so media files are served by the
web server, you must collect all media files and store them below ``MEDIA_ROOT``.
This is done with:

.. code-block:: bash

    $ python manage.py sync_plugins_media --link --all

The ``--link`` option can be removed if you would rather copy the media files
(as opposed to symlinking them).

Also, some Django applications include their own media files. These media files
must be collected via the command:

.. code-block:: bash

    $ python manage.py sync_apps_media --link

.. admonition:: More information

    See :ref:`media files in plugins <topics-plugins-media-files>` for more information about this.


Example: A Merengue Configuration with nginx and WSGI
=====================================================

In this section, we will show the demo.merengueproject.org configuration. This
setup uses the ``nginx`` server with the ``uWSGI`` module. See `Running uWSGI behind Nginx`_
for more information..

.. _`Running uWSGI behind Nginx`: http://projects.unbit.it/uwsgi/wiki/RunOnNginx

This is an example nginx configuration file::

    http {
        server {
            listen       80;
            server_name  demo.merengueprojectorg.org;

            access_log  logs/demo.merengue.access.log main;

            location /media/ {
                alias /home/demomerengue/demomerengueprojectorg/media/;
            }

            location /admin_media/ {
                alias /home/demomerengue/django_src/django/contrib/admin/media/;
            }

            location / {
                uwsgi_pass unix:///home/demomerengue/sock/uwsgi.sock;
                include uwsgi_params;
                uwsgi_param UWSGI_SCRIPT wsgi;
            }
        }
    }

The ``include uwsgi_params`` line includes this file::

    uwsgi_param  QUERY_STRING       $query_string;
    uwsgi_param  REQUEST_METHOD     $request_method;
    uwsgi_param  CONTENT_TYPE       $content_type;
    uwsgi_param  CONTENT_LENGTH     $content_length;

    uwsgi_param  REQUEST_URI        $request_uri;
    uwsgi_param  PATH_INFO          $document_uri;
    uwsgi_param  DOCUMENT_ROOT      $document_root;
    uwsgi_param  SERVER_PROTOCOL    $server_protocol;

    uwsgi_param  REMOTE_ADDR        $remote_addr;
    uwsgi_param  REMOTE_PORT        $remote_port;
    uwsgi_param  SERVER_PORT        $server_port;
    uwsgi_param  SERVER_NAME        $server_name;

Finally, for starting the uWSGI server, you need a script similar to the one
below (see the `demomerengue script`_ for a more complete example):

.. _`demomerengue script`: http://dev.merengueproject.org/browser/sites/demomerengueprojectorg/conf/demo/scripts/demomerengue

.. code-block:: bash

    BASEDIR="/home/demomerengue"
    PIDFILE="$BASEDIR/pid/demomerengue.pid"
    VIRTUALENV=$BASEDIR/virtualenv/
    SOCKET="$BASEDIR/sock/uwsgi.sock"
    PROJECTNAME="demomerengueprojectorg"
    LOGFILE="$BASEDIR/logs/demomerengue.uwsgi.log"

    cd $BASEDIR
    . $BASEDIR/virtualenv/bin/activate
    $BASEDIR/bin/uwsgi -p 2 -C -M 4 -A 4 -m -s $SOCKET -H $VIRTUALENV $PROJECTNAME.wsgi --pythonpath $BASEDIR/$PROJECT/ --pidfile $PIDFILE -d $LOGFILE

Optimizing Merengue
===================

Merengue is quite flexible and optimizable and allows you to improve a lot
your site performance. See :ref:`optimizing a Merengue installation <topics-optimization>`
for more information.

.. _`Deploying Django`: http://docs.djangoproject.com/en/dev/howto/deployment/