Installation Instructions for MiModD and Additional Software
************************************************************

.. Note::
   The most recent version of this document can be found online at:
   http://mimodd.readthedocs.org/

MiModD runs under Linux and Mac OS X (10.8 through 10.10).

Additional requirements are listed separately for the different installation 
schemes below.

Available installation schemes for MiModD:

- as a standalone command line package with optional post-install integration 
  into Galaxy for a browser-based graphical tool interface. 
  
  This is the recommended scheme for most users.
  
- from the Galaxy Tool Shed directly into any existing Galaxy.

- for OS X and certain Linux distributions we are offering preconfigured 
  bundles of MiModD and Galaxy at
  https://sourceforge.net/projects/mimodd/files/Platform-specific%20bundled%20distributions/, 
  that let you experience MiModD without any installation - just run a fully 
  functional Galaxy with MiModD from the extracted download archive.
  
In addition, MiModD can use SnpEff (http://snpeff.sourceforge.net) to 
annotate variants. Installation of SnpEff is optional and possible also after 
installing MiModD. Please refer to our instructions for `installing and 
integrating SnpEff <http://mimodd.readthedocs.org/en/latest/install_snpeff.html>`__ 
with MiModD for details.


Standard Installation
=====================

This installation scheme is fast and flexible - it gives full access to MiModD 
as a command line package with minimal installation overhead, while the 
installation can, easily and at any time, be `integrated with a local Galaxy 
installation 
<http://mimodd.readthedocs.org/en/latest/install_galaxy.html#integration-with-mimodd>`__. 

For OS X users, in particular, it also minimizes installation requirements 
through the use of precompiled installation archives.

In summary, unless you have a specific reason to prefer the `Galaxy Tool Shed 
Installation`_ scheme below, follow the instructions below to install MiModD.

.. admonition:: Requirements

   OS X - installation from precompiled wheel archive:
   
   - zlib library version 1.2.5+
   - Python 3.2+
   
   Linux and other platforms - installation from source:
   
   - a C/C++ compiler (tested with gcc/g++ and Clang)
   - zlib library version 1.2.5+ including header files
   - Python 3.2+ including the Python C header files

   See `Installation of required software 
   <http://mimodd.readthedocs.org/en/latest/install_dependencies.html>`_ for 
   more details.

Open a terminal window and try executing (possibly with superuser 
rights, i.e., with ``sudo`` prepended)::

  pip3 install MiModD
    
or alternatively::

  python3 -m pip install MiModD

.. Note::

   Either command will use the *pip* installation tool to install the MiModD 
   package from its online repository into your system's Python 3 library. 
   
   If your Python does not have *pip* pre-installed, you should `download 
   get-pip.py <https://bootstrap.pypa.io/get-pip.py>`__, then install pip for 
   your Python 3 with (this may require superuser rights again):

     ``python3 <DOWNLOAD_DIR>/get-pip.py``
  
   , where you should replace ``<DOWNLOAD_DIR>`` with the path to your 
   downloaded file.
   
   If you are behind a proxy you may have to append 
   ``--proxy PROXY_ADDRESS:PORT_NUMBER`` to the command.
 
By default, this will install MiModD from source on Linux systems and from a
precompiled wheel archive on compatible versions of OS X.

Depending on your environment and needs you may wish to append some of the 
following options to the above command (for a full list of pip installation 
options type ``pip3 help install`` or read the `online documentation <http://pip.readthedocs.org>`__):
    
``--proxy PROXY_ADDRESS:PORT_NUMBER``
    for installation from behind a proxy
    
``--user``
    to install MiModD relative to the current user's home directory
    
    This option is ideal for single-user installs and does not require 
    write permission in system-wide installation directories.
     
``--install-option="--install-scripts=<EXECUTABLE_DIR>"``
    to install the *mimodd* executable into <EXECUTABLE_DIR>
    
    If this option is not provided, the location of the executable is 
    system-dependent and will be determined automatically during the 
    installation process. This option will not work when installing from a 
    precompiled wheel archive (the default installation on OS X).

``--no-use-wheel``
    to force installation from source even if a wheel file is available for 
    your system
    
    This option is relevant only for OS X, for which we provide precompiled 
    wheel installation files, and in the hopefully rare event that 
    the wheel file that pip chooses to install on your system turns out to be 
    incompatible with your platform architecture.
    
    Note that a source installation has an extended list of requirements (see 
    above) compared to a wheel installation.

After the installation has finished, try running::

  mimodd
  
from the command line. If this displays general package information and help, 
you can proceed directly to `configuring MiModD for your system`_.


.. _path finding:

.. admonition:: Adding *mimodd* to $PATH
   
   If you get a *command not found* or similar error message with the 
   ``mimodd`` command, you need to make sure that your system can discover the 
   executable file from the $PATH environment variable.

   For this, you first need to locate the executable file. Unless MiModD was 
   installed from a precompiled archive, the installer output should contain 
   the path to the executable on one of its last lines reading:
   
   *changing mode of <EXECUTABLE_DIR>/mimodd to 755*
   
   , where *<EXECUTABLE_DIR>* will be replaced with the actual installation 
   directory.
   
   If you do not see this line, you can locate the MiModD package by executing:
   
   ``python3 -m MiModD.config``
     
   , which will report the package location in the top part of its output.
   
   Then truncate this (possibly long) path from the right side up to the 
   right-most occurence of ``lib/`` and replace the ``lib/`` with ``bin/`` to obtain the 
   path to the executable. So for, e.g., a reported package path of
   
   ``/usr/local/lib/python3.4/dist-packages/MiModD`` 
   
   the executable path is
   
   ``/usr/local/bin/``
   
   After locating the executable you can either:
   
   - append the directory to your $PATH variable (`here 
     <http://www.cyberciti.biz/faq/unix-linux-adding-path>`__ is a link to a 
     nice and comprehensive explanation of how to do this) or
   
   - copy or symlink the executable to a different directory already in $PATH 
     (use ``echo $PATH`` to see which directories are included).
     
     As an example, here is how you would create, for a *mimodd* executable 
     that got installed into a directory */home/me/bin*, a symbolic link in 
     */usr/local/bin* (very likely this would require a ``sudo`` prepended):
     
     ``ln -s /home/me/bin/mimodd /usr/local/bin/mimodd``


------------------------------------------


Galaxy Tool Shed Installation
=============================

.. admonition:: Requirements

   - a C/C++ compiler (tested with gcc/g++ and Clang)

Direct installation from Tool Sheds is an increasingly popular way of adding 
new tools to existing instances of Galaxy. Its main advantage is that Tool Shed 
installs are encapsulated in and managed by Galaxy and do not require changes 
to system directories. This approach is often chosen by administrators of 
larger Galaxy servers for which system stability is a primary 
concern. 

MiModD is available from the `main Galaxy Tool Shed 
<https://toolshed.g2.bx.psu.edu>`__. For one-click installation of all 
components use the `suite_mimodd_0_1_5 
<https://toolshed.g2.bx.psu.edu/view/wolma/suite_mimodd_0_1_5>`__ installer.

.. Note::

   The Tool Shed Installation scheme never uses precompiled installation 
   archives, but instead compiles all dependencies (including Python3) from 
   source so you will need a C/C++ compiler even on OS X systems.
   
The installation process is the same as for any other Tool Shed tool, and is 
described `here 
<https://wiki.galaxyproject.org/Admin/Tools/AddToolFromToolShedTutorial>`__.

.. Note::

   After the installation, the complete Galaxy interface for all MiModD tools 
   will be added to your Galaxy's Tools bar, but the order of the tools will 
   be more or less random. This is one of the disadvantages of the Tool Shed 
   installation scheme and you will have to manually edit your Galaxy's 
   ``integrated_tool_panel.xml`` file to sort the items on the Tool bar.

Like for the standard installation, you should configure MiModD before using 
the package. However, for a Tool Shed installation of MiModD we recommend the 
environment variable-based configuration described below.


------------------------------------------


Configuring MiModD for your system
==================================

.. Note::

    You can change all settings at any time, but you should definitely set
    them according to your system specifications before you first start using
    MiModD.

The simplest way to configure MiModD is from the command line using its own 
config subcommand.

Alternatively, you can change settings through a special environmental variable 
recognized by MiModD. This configuration through an environment variable is 
convenient in situations in which you do not have easy access to the *mimodd* 
command, as is the case if you have installed the package from the Galaxy Tool 
Shed.

.. _config:

The config subcommand
---------------------

From the command line use::

    mimodd config
    
to see the current configuration settings of MiModD.

For a freshly installed copy of MiModD the output should be similar to this::

    CURRENT MIMODD SETTINGS
    -----------------------
    PARAMETER : VALUE
    .......................
    TMPFILES_PATH : /var/tmp/
    MULTITHREADING_LEVEL : 4
    MAX_MEMORY : 2
    SNPEFF_PATH : not available

The ``config`` subcommand is also the standard tool to **change** the settings. 
Its general invocation pattern for this purpose is::

  mimodd config [--tmpfiles PATH] [--snpeff PATH] [-t THREADS] [-m MEMORY]
  
So to set a new *TMPFILES_PATH* and to set *MAX_MEM* to 8 GB, you could use::

  mimodd config --tmpfiles /var/tmp/mimodd_tmp -m 8

Depending on your installation of MiModD, changes to the configuration file may 
require superuser rights, so you will have to prepend ``sudo`` to the above.
Proceed to `Configuration parameters`_ to learn more about the settings that 
can be changed by the options.


Configuration through an environment variable
---------------------------------------------

As an alternative to command line configuration, you can provide new settings 
for the `configuration parameters`_ also through the MiModD-specific 
environment variable ``$MIMODD_CONFIG_UPDATE``.

The value of this variable has to be set to a ``:``-separated list of 
*parameter=value* entries, for example::

  export MIMODD_CONFIG_UPDATE=MAX_MEMORY=8:MULTITHREADING_LEVEL=4
  
The presence of the environment variable will be detected by MiModD at its next 
run from the same terminal, and will be used to update its internal config 
file accordingly.

You can also pass the variable exclusively to Galaxy by prepending 
it to the start script and **this is the recommended approach to configure a 
Galaxy Tool Shed installation of MiModD**::

  MIMODD_CONFIG_UPDATE=MULTITHREADING_LEVEL=3:TMPFILES_PATH=home/galaxy/tmp/ sh run.sh

With this, the MiModD config file will get updated when the first MiModD tool 
gets executed from within the Galaxy instance.

.. Note::
   
   Changes made through $MIMODD_CONFIG_UPDATE are persistent so the variable 
   has to be found only once.
   
   To store the change *mimodd* needs to be executed with write privileges for 
   the MiModD package directory. This should not be an issue with Tool Shed 
   installations of MiModD, but can make this configuration scheme problematic 
   in combination with Standard Installations in system directories.


Configuration parameters
------------------------

listed as pairs of

PARAMETER : ``config options``
    where PARAMETER can be used in the $MIMODD_CONFIG_UPDATE 
    variable and the *config options* are used to set the same parameter using 
    the ``mimodd config`` command.


TMPFILES_PATH : ``--tmpfiles``, ``--tmpfiles-path``
    the directory in which MiModD will store temporary files [default: /var/tmp]

    In a typical analysis pipeline, MiModD may produce several GB of data
    in this directory and remove them automatically again when the data is not 
    any longer needed. Under exceptional circumstances, however, MiModD might 
    fail to delete data files, so this directory is the first place you should 
    look at if you notice reduced disk space. Also, any users of MiModD will 
    require write permission in this directory.

MULTITHREADING_LEVEL : ``-t``, ``--threads``
    the maximum number of cores that a single MiModD command is supposed to use 
    at any time [default: 4]
    
    Many MiModD commands take advantage of multiprocessing to speed up analyses.
    These commands try to respect this setting if possible although some may 
    use slightly more than their allocated CPU share.
    
MAX_MEMORY : ``-m``, ``--memory``
    the maximum memory in GB that any single MiModD command should use [default: 2]
    
    WGS data files are often very large and some MiModD tools can operate on 
    them more efficiently when allowed to hold a larger portion of such files 
    in memory at once. Most of the time MiModD will consume less than 1 GB 
    though.
    The setting is adhered to relatively strictly by most tools with the 
    exception of the SNAP aligner-based tools *mimodd snap index*, *mimodd 
    snap*, *mimodd snap_batch* and the Galaxy *SNAP Read Alignment* tool.
    Due to the nature of their underlying alignment algorithm these tools will 
    require a fixed amount of memory that depends on the size of the reference 
    genome and which may be significantly more than the configured setting.

.. Tip::

    MULTITHREADING\_LEVEL and MAX\_MEMORY will have a big effect on the
    performance of MiModD, but also on the responsiveness of your system
    during execution of MiModD commannds. As a rule of thumb, if you do not 
    have special requirements, we recommend to set both parameters to between 
    50 and 75 % of the available resources on your system, i.e., if you have 
    8 threads and 16 GB of RAM on your system, you might set 
    MULTITHREADING\_LEVEL to 4-6 and MAX\_MEM to 8-12.
    
    See also the section on `MiModD Hardware Requirements 
    <http://mimodd.readthedocs.org/en/latest/hardware.html>`__ in the User 
    Guide for more notes on performance.

SNPEFF_PATH : ``--snpeff``, ``--snpeff-path``
    the path to the optional SnpEff variant annotation tool
    
    This parameter should be set to the directory that you installed SnpEff 
    into (e.g., ~/snpEff if you followed the recommended installation steps). 
    If you do not have SnpEff installed, keep the default setting, to have all 
    SnpEff-dependent functionality of MiModD deactivated.

