.. _topics-install:

=======================
How to Install Merengue
=======================

This document will get you up and running with Merengue.

.. admonition:: Conventions

    We need to assume an operating system in order to describe the installation
    of the system packages required for Merengue. For these directions, we
    will assume the use of  *Ubuntu OS*. If you are not using *Ubuntu OS*,
    you will need to adapt the following ``apt-get`` commands to your
    specific operating system.


Previous Dependencies
=====================

* ``Python``: The Python programming language. Supported 2.5, 2.6 and 2.7 versions. Python 3 not supported.

.. code-block:: bash

    $ sudo apt-get install python

* ``python-setuptools``: Required for installing Python packages.

.. code-block:: bash

    $ sudo apt-get install python-setuptools

* ``ffmpeg``: FFmpeg is a complete, cross-platform solution to record, convert, and stream audio and video. It includes libavcodec - the leading audio/video codec library. 

.. code-block:: bash

    $ sudo apt-get install ffmpeg libavcodec-unstripped-52 libavdevice-unstripped-52 libavformat-unstripped-52 libavutil-unstripped-49 libpostproc-unstripped-51 libswscale-unstripped-0 yamdi

* ``gettext``: A utility for managing translations catalogs.

.. code-block:: bash

    $ sudo apt-get install gettext

PIL Installation
----------------

The `Python Imaging Library`_ (PIL) is usually one of the hardest components to
install in a Python environment from Python packages, because it requires several system
packages to properly compile all of the image manipulation libraries.

To install PIL correctly (for an Ubuntu/Debian environment), install the
following package:

.. code-block:: bash

    $ sudo apt-get install python-imaging

By the way, you may try to install PIL dependences separately and install PIL from egg package:

.. code-block:: bash

    $ sudo apt-get install libfreetype6-dev
    $ sudo apt-get install python-tk
    $ sudo apt-get install tcl8.5-dev
    $ sudo apt-get install tk8.5-dev
    $ sudo apt-get install liblcms1-dev liblcms-utils

.. _`Python Imaging Library`: http://www.pythonware.com/products/pil/

Other Recommended Dependencies
------------------------------

* ``ipython``: a Python shell with autocomplete, help, etc.

.. code-block:: bash

    $ apt-get install ipython


Installing with easy_install
============================

The easiest way to install Merengue is to use easy_install. All dependencies will be
installed automatically.

Use the following command:

.. code-block:: bash

    $ sudo easy_install merengue

Go to the `project creation`_ section to continue.

.. _`project creation`: #merengue-project-creation


Development Installation
========================

In development installation Merengue will be installed from subversion
repository. In this case, your project will be affected by any code updates to
the Merengue subversion repository (including any new features and fixes that
have been implemented, as well as any new errors or changes that may cause
compatibility problems with your code).

Installing Django
-----------------

Merengue uses Django 1.3. You can install
it with these instructions:

.. code-block:: bash

    $ wget --trust-server-names http://www.djangoproject.com/download/1.3/tarball/
    $ tar xzvf Django-1.3.tar.gz
    $ cd Django-1.3
    $ sudo python setup.py install

Test to see if you have the correct Django version:

.. code-block:: bash

    $ python -c "import django; print django.VERSION"
    (1, 3, 0, 'final', 0)

Downloading Merengue
--------------------

Download the Merengue source from a tarball or from the subversion repository:

.. code-block:: bash

    $ svn checkout https://svn.merengueproject.org/trunk/merengueproj
    $ cd merengueproj

Installing Python Dependencies
------------------------------

The `requirements.txt`_ file lists the Python packages required by Merengue. You
will need to install these dependencies via Python eggs. You can use ``easy_install``
to install each egg separately, or use ``pip``  to install all of the required
packages via the following command:

.. code-block:: bash

    $ sudo pip install -r requirements.txt

.. _`requirements.txt`: http://dev.merengueproject.org/browser/trunk/merengueproj/requirements.txt

Installing Merengue
-------------------

Execute ``setup.py`` to complete the manual install process:

.. code-block:: bash

    $ sudo python setup.py develop

.. admonition:: Note

    ``develop`` command in ``setup.py`` will create symlinks in your
    ``site-packages`` directory. Then you can update your Merengue subversion
    repository and the installed package will be updated too.


Merengue Project Creation
=========================

When beginning a Merengue project, you can begin your development in one of two
modes: production mode or development mode.

By default, your project will contain symbolic links back to the
Merengue code. In this case, your project will be affected by any code updates
to the Merengue package (i.e. doing a ``easy_install -U Merengue``).

.. code-block:: bash

    $ merengue-admin.py startproject myproject

If you want to freeze Merengue in your project, you can use the ``--copy``
option and the Merengue code will be copied instead of linked. So, any new (i.e.
following) code updates will not affect your project.

Language(s) Configuration
-------------------------

In Merengue, language configuration should be done before database creation
because data models are created with translatable fields (the number of columns
created is equal to the number of configured languages).

The ``LANGUAGE_CODE`` and ``LANGUAGES`` parameters must be defined in the
configuration file ``myproject/settings.py``:

    .. code-block:: python

        LANGUAGE_CODE = 'es'

        LANGUAGES = (
            ('es', ugettext('Español')),
            ('en', ugettext('English')),
            ('fr', ugettext('Français')),
        )

.. admonition:: More Information

    See the `django-transmeta`_ application for more information about
    translatable models.

.. _`django-transmeta`: http://code.google.com/p/django-transmeta/

Database Installation and Configuration
---------------------------------------

The database connection must be defined in the configuration file by setting
the ``DATABASE`` setting. You can use all `database engines supported by Django.`_

We will include in this documentation examples of configuring both PostgreSQL
and SQLite databases.

.. _`database engines supported by Django.`: http://docs.djangoproject.com/en/1.3/ref/settings/#database-engine

SQLite
~~~~~~

You can quickly setup a SQLite database by simply using the following
parameters into the settings.py file:

    .. code-block:: python

        DATABASE = {
            'default': {
                'ENGINE': 'django.db.backends.sqlite3',
                'NAME': 'database.db',
                'USER': '',
                'PASSWORD': '',
                'HOST': '',
                'PORT': '',
            }
        }

where ``NAME`` is the path of the database. 

.. admonition:: Note

    SQLite is not a recommended database for put your site in production, but
    it is only useful in the developing process.

PostgreSQL
~~~~~~~~~~

Let's suppose that you were configuring a PostgreSQL database with the
following parameters:

    .. code-block:: python

        DATABASE = {
            'default': {
                'ENGINE': 'django.db.backends.postgresql_psycopg2',
                'NAME': 'myproject',
                'USER': 'myprojectuser',
                'PASSWORD': 'password',
                'HOST': '',
                'PORT': '',
            }
        }

Install the  database and the appropriate python dependencies: ``postgresql``, ``python-psycopg2``.

* ``postgresql``: the object-relational database system that we will use.

    .. code-block:: bash

        $ apt-get install postgresql

* ``python-psycopg``: the python interface to the PostgreSQL database.

    .. code-block:: bash

        $ apt-get install python-psycopg2


Now you have to creating the project Database

.. admonition:: Note

    We assume that your user has superadmin permissions in PostgreSQL.

The PostgreSQL database and user is created with these instructions:

.. code-block:: bash

    $ createuser myprojectuser
    $ createdb --owner=myprojectuser myproject

We have to permit connections to the database from the local computer. Edit
/etc/postgresql/X.X/main/pg_hba.conf and add the following line (not at
the end):

.. code-block:: bash

    local myproject myprojectuser trust
    local test_myproject myprojectuser trust # necessary for tests

Reload pg_hba.conf in PostgreSQL server with the following command:

.. code-block:: bash
 
    $ /etc/init.d/postgresql-X.X reload

Restart PostgreSQL and check your user access with this command:

.. code-block:: bash

    $ psql myproject -U myprojectuser

Building Initial Merengue Models
--------------------------------

You will need to build the initial Merengue database models with the following two commands:

.. code-block:: bash

    $ python manage.py syncdb
    $ python manage.py migrate

You can ignore the creation of the superuser because a user named ``admin`` with
the password ``admin`` will be created automatically.


Collecting Media from Installed Django Applications
---------------------------------------------------

Some Django applications use media files that Merengue needs. To include these
files in your project, execute the command:

.. code-block:: bash

    $ python manage.py sync_apps_media --link

.. admonition:: Note

    You can remove the ``--link`` option if you want Merengue to copy the media
    files instead of linking to them.


View the Site
-------------

Go to your project directory and test the Django development server to make sure
it starts without problems:

.. code-block:: bash

    $ python manage.py runserver

Open the web browser and enter the following URL: http://localhost:8000/admin/ to see
if the admin web interface is running successfully.

Use the admin username and password below to access the admin interface:

.. code-block:: bash

    user: admin
    password: admin

To see the public site, enter the URL: http://localhost:8000/


Installation with GIS
=====================

By default, the GIS features are disabled in Merengue.

To add GIS support to your Merengue project, you will need to
perform some additional installation and configuration instructions. This section can
be skipped if your project does not require GIS features.


Settings Changes
----------------

To use GIS, change one line in your project ``settings.py`` file:

.. code-block:: python

    USE_GIS = True


PostGIS dependencies
--------------------

.. admonition:: Note

    Again, we assume that you are using PostgreSQL as your database backend as
    well as PostGIS for GIS extensions. If you are using another database, you
    have to adapt the following instructions.
    
GeoDjango install instructions are online at http://geodjango.org/docs/install.html

In addition to the dependencies listed above, you should install the following
package:

* ``geopy``: makes it easy for developers to locate the coordinates of
  addresses, cities, countries, and landmarks across the globe.

.. code-block:: bash

    $ easy_install geopy


GIS Extensions Installation
---------------------------

With Ubuntu 10.04, all of the GIS extensions for postgresql-8.4 can be installed
with the following commands:

.. code-block:: bash

    $ sudo apt-get install binutils libgdal1-1.6.0 postgresql-8.4-postgis python-psycopg2 python-setuptools
    $ export PG_CONFIG='/usr/lib/postgresql/8.4/bin/pg_config'

If you are using Ubuntu with a version newer than 8.04 and older than 10.04, then
install all of the GIS extensions for postgresql-8.3 with the commands below:

.. code-block:: bash

    $ sudo apt-get install binutils libgdal1-1.5.0 postgresql-8.3-postgis python-psycopg2 python-setuptools
    $ export PG_CONFIG='/usr/lib/postgresql/8.3/bin/pg_config'


.. admonition:: Note

    If you are using postgresql-8.4 with a version of Ubuntu earlier than 10.04
    and there exists no ``postgresql-8.4-postgis`` package, then use
    ``postgresql-8.3`` packages along with the following options.  Use option
    ``-p 5433`` with ``psql``, ``createuser``, and ``createdb``. Configure your
    Merengue project to use that port (5433) with the database.

If your distribution does not have the package postgis it will be necessary to
compile it:

1. Install the PostgreSQL source headers:

.. code-block:: bash

    $ sudo apt-get install postgresql-server-dev-X.X
    $ export PG_CONFIG='/usr/lib/postgresql/X.X/bin/pg_config'

2. Compile the `GEOS`_ library:

.. code-block:: bash

    $ cd resources
    $ tar xjf geos-x.x.x.tar.bz2
    $ cd geos-x.x.x
    $ ./configure && make
    $ sudo make install

3. Compile the `PROJ.4`_ library:

.. code-block:: bash

    $ cd resources
    $ tar xzf proj-x.x.x.tar.gz
    $ cd proj-x.x.x/nad
    $ tar xzf ../../proj-datumgrid-x.x.tar.gz
    $ cd ..
    $ ./configure
    $ make
    $ sudo make install
    $ cd ..

4. Compile `PostGIS`_:

.. code-block:: bash

    $ tar xzf postgis-x.x.x.tar.gz
    $ cd postgis-x.x.x
    $ ./configure --with-pgsql=$PG_CONFIG --datadir=`$PG_CONFIG --sharedir`
    $ make
    $ sudo make install
    $ cd ..

5. Compile the `GDAL`_ library:

.. code-block:: bash

    $ tar xzf gdal-x.x.x.tar.gz
    $ cd gdal-x.x.x
    $ ./configure
    $ make # Go get some coffee, this takes a while.
    $ sudo make install
    $ cd ..

6. Update all dynamic libraries:

.. code-block:: bash

    $ sudo ldconfig -v

.. admonition:: Note

    Make sure that the line ``/usr/local/lib`` is included in the file
    ``/etc/ld.so.conf``

.. _`GEOS`: http://trac.osgeo.org/geos/
.. _`PROJ.4`: http://download.osgeo.org/proj/
.. _`PostGIS`: http://postgis.refractions.net/
.. _`GDAL`: http://trac.osgeo.org/gdal/


Changes in Database Configuration
---------------------------------

Create the postgis template database:

.. code-block:: bash

    $ createdb -E UTF8 template_postgis
    $ createlang -d template_postgis plpgsql

* In Ubuntu (version 10.04 or greater):

.. code-block:: bash

    $ PG_GIS_TEMPLATES=/usr/share/postgresql/8.4/contrib/
    $ psql -d template_postgis -f $PG_GIS_TEMPLATES/postgis.sql
    $ psql -d template_postgis -f $PG_GIS_TEMPLATES/spatial_ref_sys.sql
    $ psql -d template_postgis -c "GRANT ALL ON geometry_columns TO PUBLIC;"
    $ psql -d template_postgis -c "GRANT ALL ON spatial_ref_sys TO PUBLIC;"

* In Ubuntu (versions greater than 8.04 and less than 10.04 -- those using the postgres-8.3-postgis package):

.. code-block:: bash

    $ PG_GIS_TEMPLATES=/usr/share/postgresql-8.3-postgis/
    $ psql -d template_postgis -f $PG_GIS_TEMPLATES/lwpostgis.sql
    $ psql -d template_postgis -f $PG_GIS_TEMPLATES/spatial_ref_sys.sql
    $ psql -d template_postgis -c "GRANT ALL ON geometry_columns TO PUBLIC;"
    $ psql -d template_postgis -c "GRANT ALL ON spatial_ref_sys TO PUBLIC;"

.. admonition:: Note

     Remember that if you have postgresql-8.3 and 8.4 running at the
     same time you need to add ``-p 5433`` option to the ``psql``, ``createdb``, and
     ``createlang`` commands.

* In any other case:

.. code-block:: bash

    $ psql -d template_postgis -f `$PG_CONFIG --sharedir`/lwpostgis.sql
    $ psql -d template_postgis -f `$PG_CONFIG --sharedir`/spatial_ref_sys.sql
    $ psql -d template_postgis -c "GRANT ALL ON geometry_columns TO PUBLIC;"
    $ psql -d template_postgis -c "GRANT ALL ON spatial_ref_sys TO PUBLIC;"

Creating the Project Database
-----------------------------

.. admonition:: Note

     If you used the previous (non-GIS) installation instructions, you will need
     to re-create the project database, because that database will not
     work with GIS enabled. Before continuing, you will need to delete the
     previous database.

Database creation changes slightly because the project database has to use
``template_postgis`` as the database template (with all postgis extensions
included).

.. code-block:: bash

    $ createuser myproject
    $ createdb -T template_postgis --owner=myproject myproject


Upgrading Merengue
==================

To upgrade your models after a Merengue update, you can use the South
``migrate`` command as follows:

.. code-block:: bash

    $ python manage.py migrate

This ``migrate`` command does not affect the installed plugins in the
project. In order to perform the database migrations for plugins, you will need to execute
the ``migrate_plugins`` command:

.. code-block:: bash

    $ python manage.py migrate_plugins

Sometimes new Merengue versions can include new applications will may need
to install media files. The best way to ensure your system have all the media
files you can execute this command:

.. code-block:: bash

    $ python manage.py sync_apps_media --link

Also, we recommend following the `changes`_ on the Merengue `skel project`_. This
documentation may be helpful in incorporating these changes into your own project.

.. _`changes`: https://tracpub.yaco.es/merengue/log/trunk/merengueproj/projskel
.. _`skel project`: http://tracpub.yaco.es/merengue/browser/trunk/merengueproj/projskel


Database Backup
===============

A database backup (stored in SQL) may be created with the following command:

.. code-block:: bash

    $ python manage.py backupdb

.. note::

    When using PostgreSQL as the database backend, you will need to install the
    ``pexpect`` python package.

.. _topics-deploying:

==================
Deploying Merengue
==================

Since Merengue is based on Django, we recommend the reading of the `Deploying Django`_
documentation before deploying Merengue.

The recommended deployment configuration for Merengue is to use nginx with WSGI.
However, other Django deployment configurations can be used.

This section includes some points to consider when planning a Merengue site deployment.

Serve Media Files with Web Server
====================================

To help keep Merengue plugins and apps portable and reusable, Merengue plugins (and some
Merengue apps) can contain media files located in a ``media`` directory inside
each plugin directory. Merengue has implemented the ``merengue.views.static.serve``
view that virtualizes the media directory specified as ``MEDIA_ROOT``. Thus,
the Django development server will work correctly.

However, if you want to set the configuration so media files are served by the
web server, you must collect all media files and store them below ``MEDIA_ROOT``.
This is done with:

.. code-block:: bash

    $ python manage.py sync_plugins_media --link --all

The ``--link`` option can be removed if you would rather copy the media files
(as opposed to symlinking them).

Also, some Django applications include their own media files. These media files
must be collected via the command:

.. code-block:: bash

    $ python manage.py sync_apps_media --link

.. admonition:: More information

    See :ref:`media files in plugins <topics-plugins-media-files>` for more information about this.


Example: A Merengue Configuration with nginx and WSGI
=====================================================

In this section, we will show the demo.merengueproject.org configuration. This
setup uses the ``nginx`` server with the ``uWSGI`` module. See `Running uWSGI behind Nginx`_
for more information.

.. _`Running uWSGI behind Nginx`: http://projects.unbit.it/uwsgi/wiki/RunOnNginx

This is an example nginx configuration file (valid for ``nginx-1.0.X`` compiled
from source code)::

    server {
        listen       80;
        server_name  yoursite.yourdomain.org;

        access_log  logs/yoursite.yourdomain.access.log;

        location /media/ {
            expires max;
            alias /home/user/yoursite/media/;
        }

        location /admin_media/ {
            expires max;
            alias /home/user/yoursite/media/admin/;
        }

        location / {
            include uwsgi_params;
            uwsgi_pass unix:///home/user/uwsgi.sock;
            uwsgi_param UWSGI_SCRIPT wsgi;
        }
    }

.. admonition:: Assumptions

    We assume that ``/home/user/yoursite`` is a typical Merengue project, created with the
    ``merengue-admin.py startproject`` command, per the installation docs.
    Also we assume that ``virtualenv`` was used to install the Merengue package,
    and that the virtualenv environment is located at the path ``/home/user/venvs/merengue``.

Finally, for starting the uWSGI server, you need a script similar to this:

.. code-block:: bash

    #!/bin/bash
    PROJECTDIR="/home/user"
    PIDFILE="$PROJECTDIR/demomerengue.pid"
    VIRTUALENV=$PROJECTDIR/venvs/merengue
    SOCKET="$PROJECTDIR/uwsgi.sock"
    PROJECTNAME="yoursite"
    LOGFILE="$PROJECTDIR/demomerengue.uwsgi.log"

    export LC_ALL=es_ES.UTF-8

    function start {
      echo "Starting Merengue website..."
      uwsgi -p 4 -C -M 4 -A 4 -m --no-orphans -s $SOCKET -H $VIRTUALENV --pythonpath $PROJECTDIR \
      --pythonpath $PROJECTDIR/$PROJECTNAME --pidfile $PIDFILE -d $LOGFILE
    }

    function stop {
        echo "Stopping Merengue website..."
        kill -9 `cat $PIDFILE`
    }

    case "$1" in
        start)
            start
        ;;

        stop)
            stop
        ;;

        restart)
            stop
            start
        ;;

        *)
            echo "Usage: yoursitectl {start|stop|restart}"
            exit 1
        ;;
    esac

    exit 0

.. admonition:: Note

    The above instructions will launch four uwsgi worker processes in parallel
    (``-p 4`` argument). The number of workers depends on your site load. By
    the way, if you have more than one working process, you will need to use
    Memcache as the cache backend to get the invalidation cache working properly in
    all of the processes. See :ref:`configuring caching <topics-optimization-configuring-caching>`
    for more information.

Finally, create a ``wsgi.py`` file inside your Merengue project
directory with the following content:

.. code-block:: python

    import os
    import sys
    import django.core.handlers.wsgi

    os.environ['DJANGO_SETTINGS_MODULE'] = 'yoursite.settings'

    proj_dir = os.path.abspath(os.path.dirname(__file__))
    sys.path.append(os.path.join(proj_dir, 'apps'))
    sys.path.append(os.path.join(proj_dir, 'merengue', 'apps'))

    application = django.core.handlers.wsgi.WSGIHandler()


Optimizing Merengue
===================

Merengue is quite flexible and optimizable and allows you to significantly improve
your site's performance. See :ref:`optimizing a Merengue installation <topics-optimization>`
for more information.

.. _`Deploying Django`: http://docs.djangoproject.com/en/dev/howto/deployment/
