==========================
 Docutils Front-End Tools
==========================

:Author: David Goodger
:Contact: goodger@users.sourceforge.net
:Revision: $Revision: 1.39 $
:Date: $Date: 2003/06/16 17:31:37 $
:Copyright: This document has been placed in the public domain.

.. contents::


Introduction
============

Once the Docutils package is unpacked, you will discover a "``tools``"
directory containing several front ends for common Docutils
processing.  Rather than a single all-purpose program, Docutils has
many small front ends, each specialized for a specific "Reader" (which
knows how to interpret a file in context), a "Parser" (which
understands the syntax of the text), and a "Writer" (which knows how
to generate a specific data format).  Most front ends have common
options and the same command-line usage pattern::

    toolname [options] [<source> [<destination]]

The exceptions are buildhtml.py_ and pep2html.py_.  See html.py_ for
concrete examples.  Each tool has a "``--help``" option which lists
the `command-line options`_ and arguments it supports.  Processing can
also be customized with `configuration files`_.

The two arguments, "source" and "destination", are optional.  If only
one argument (source) is specified, the standard output (stdout) is
used for the destination.  If no arguments are specified, the standard
input (stdin) is used for the source as well.


Getting Help
------------

First, try the "``--help``" option each front-end tool has.

Users who have questions or need assistance with Docutils or
reStructuredText should `post a message`_ to the `Docutils-Users
mailing list`_.  The `Docutils project web site`_ has more
information.

.. _post a message: mailto:docutils-users@lists.sourceforge.net
.. _Docutils-Users mailing list:
   http://lists.sourceforge.net/lists/listinfo/docutils-users
.. _Docutils project web site: http://docutils.sourceforge.net/


The Tools
=========

buildhtml.py
------------

:Readers: Standalone, PEP
:Parser: reStructuredText
:Writers: HTML, PEP/HTML

Use ``buildhtml.py`` to generate .html from all the .txt files
(including PEPs) in each <directory> given, and their subdirectories
too.  (Use the ``--local`` option to skip subdirectories.)

Usage::

    buildhtml.py [options] [<directory> ...]

After unpacking the Docutils package, the following shell commands
will generate HTML for all included documentation::

    cd docutils/tools
    buildhtml.py ..

For official releases, the directory may be called "docutils-X.Y",
where "X.Y" is the release version.  Alternatively::

    cd docutils
    tools/buildhtml.py --config=tools/docutils.conf

The current directory (and all subdirectories) is chosen by default if
no directory is named.  Some files may generate system messages
(tools/test.txt contains intentional errors); use the ``--quiet``
option to suppress all warnings.  The ``--config`` option ensures that
the correct stylesheets, templates, and settings are in place
(``./docutils.conf`` is picked up automatically).  Command-line
options may be used to override config file settings or replace them
altogether.


html.py
-------

:Reader: Standalone
:Parser: reStructuredText
:Writer: HTML

The ``html.py`` front end reads standalone reStructuredText source
files and produces HTML 4 (XHTML 1) output compatible with modern
browsers.  For example, to process a reStructuredText file
"``test.txt``" into HTML::

    html.py test.txt test.html

In fact, there *is* a "``test.txt``" file in the "``tools``"
directory.  It contains "at least one example of each reStructuredText
construct", including intentional errors.  Use it to put the system
through its paces and compare input to output.

Now open the "``test.html``" file in your favorite browser to see the
results.  To get a footer with a link to the source file, date & time
of processing, and links to the Docutils projects, add some options::

    html.py -stg test.txt test.html


Stylesheets
```````````

``html.py`` inserts into the generated HTML a link to a cascading
stylesheet, defaulting to "``default.css``" (override with a
"``--stylesheet``" or "``--stylesheet-path``" command-line option or
with configuration file settings).  The
"``tools/stylesheets/default.css``" stylesheet is provided for basic
use.  To experiment with styles, rather than editing the default
stylesheet (which will be updated as the project evolves), it is
recommended to use an "``@import``" statement to create a "wrapper"
stylesheet.  For example, a "``my.css``" stylesheet could contain the
following::

    @import url(default.css);

    h1, h2, h3, h4, h5, h6, p.topic-title {
      font-family: sans-serif }

Generate HTML with the following command::

    html.py -stg --stylesheet my.css test.txt test.html

When viewed in a browser, the new "wrapper" stylesheet will change the
typeface family of titles to "sans serif", typically Helvetica or
Arial.  Other styles will not be affected.  Styles in wrapper
stylesheets override styles in imported stylesheets, enabling
incremental experimentation.


pep.py
------

:Reader: PEP
:Parser: reStructuredText
:Writer: PEP/HTML

``pep.py`` reads a new-style PEP (marked up with reStructuredText) and
produces HTML.  It requires a template file and a stylesheet.  By
default, it makes use of a "``pep-html-template``" file and a
"``default.css``" stylesheet in the current directory, but these can
be overridden by command-line options or configuration files.  The
"``tools/stylesheets/pep.css``" stylesheet is intended specifically
for PEP use.

The "``docutils.conf``" `configuration file`_ in the "``spec``"
directory of Docutils contains a default setup for use in processing
the PEP files there (``spec/pep-*.txt``) into HTML.  It specifies a
default template (``tools/pep-html-template``) and a default
stylesheet (``tools/stylesheets/pep.css``).  See Stylesheets_ above
for more information.


pep2html.py
-----------

:Reader: PEP
:Parser: reStructuredText
:Writer: PEP/HTML

``pep2html.py`` is a modified version of the original script by
Fredrik Lundh, with support for Docutils added.  It reads the
beginning of a PEP text file to determine the format (old-style
indented or new-style reStructuredText) and processes accordingly.
Since it does not use the Docutils front end mechanism (the common
command-line options are not supported), it must be configured using
`configuration files`_.  The template and stylesheet requirements of
``pep2html.py`` are the same as those of `pep.py`_ above.

Arguments to ``pep2html.py`` may be a list of PEP numbers or .txt
files.  If no arguments are given, all files of the form
"``pep-*.txt``" are processed.


rst2latex.py
------------

:Reader: Standalone
:Parser: reStructuredText
:Writer: LaTeX2e

The ``rst2latex.py`` front end reads standalone reStructuredText
source files and produces LaTeX2e output. For example, to process a
reStructuredText file "``test.txt``" into LaTeX::

    rst2latex.py test.txt test.tex

The output file "``test.tex``" should then be processed with ``latex``
or ``pdflatex`` to get a typeset document.

Some limitations and difference apply:

- Gif,jpg or png images are not handled, when processed with
  ``latex``, use ``pdflatex`` instead.
- Only Latin-1 is tested up to now.
- The generated file includes a file ``style.tex``, which allows the
  inclusion of special packages or changes to settings made in the
  header.
- Not all constructs are possible (e.g. multirow/multicoumn entries in
  tables are not).


docutils-xml.py
---------------

:Reader: Standalone
:Parser: reStructuredText
:Writer: XML (Docutils native)

The ``docutils-xml.py`` front end produces Docutils-native XML output.
This can be transformed with standard XML tools such as XSLT
processors into arbitrary final forms.


publish.py
----------

:Reader: Standalone
:Parser: reStructuredText
:Writer: Pseudo-XML

``publish.py`` is used for debugging the Docutils Reader to Transform
to Writer pipeline.  It produces a compact pretty-printed
"pseudo-XML", where nesting is indicated by indentation (no end-tags).
External attributes for all elements are output, and internal
attributes for any leftover "pending" elements are also given.


quicktest.py
------------

:Reader: N/A
:Parser: reStructuredText
:Writer: N/A

The ``quicktest.py`` tool is used for testing the reStructuredText
parser.  It does not use a Docutils Reader or Writer or the standard
Docutils command-line options.  Rather, it does its own I/O and calls
the parser directly.  No transforms are applied to the parsed
document.  Various forms output are possible:

- Pretty-printed pseudo-XML (default)
- Test data (Python list of input and pseudo-XML output strings;
  useful for creating new test cases)
- Pretty-printed native XML
- Raw native XML (with or without a stylesheet reference)



Customization
=============

Command-Line Options
--------------------

Each front-end tool supports command-line options for one-off
customization.  For persistent customization, use `configuration
files`_.

Use the "--help" option on each of the front ends to list the
command-line options it supports.  Command-line options and their
corresponding configuration file entry names are listed in
`Configuration File Entries`_ below.


.. _configuration file:

Configuration Files
-------------------

Configuration files are used for persistent customization; they can be
set once and take effect every time you use a front-end tool.

By default, Docutils checks the following places for configuration
files, in the following order:

1. ``/etc/docutils.conf``: This is a system-wide configuration file,
   applicable to all Docutils processing on the system.

2. ``./docutils.conf``: This is a project-specific configuration file,
   located in the current directory.  The Docutils front end has to be
   executed from the directory containing this configuration file for
   it to take effect (note that this may have nothing to do with the
   location of the source files).  Settings in the project-specific
   configuration file will override corresponding settings in the
   system-wide file.

3. ``~/.docutils``: This is a user-specific configuration file,
   located in the user's home directory.  Settings in this file will
   override corresponding settings in both the system-wide and
   project-specific configuration files.

If more than one configuration file is found, all will be read but
later entries will override earlier ones.  For example, a "stylesheet"
entry in a user-specific configuration file will override a
"stylesheet" entry in the system-wide file.

In addition, a configuration file may be explicitly specified with the
"--config" command-line option.  This configuration file is read after
the three implicit ones listed above.

Configuration files use the standard ConfigParser.py_ Python_ module.
From its documentation:

    The configuration file consists of sections, lead by a "[section]"
    header and followed by "name: value" entries, with continuations
    in the style of `RFC 822`_; "name=value" is also accepted.  Note
    that leading whitespace is removed from values.  The optional
    values can contain format strings which refer to other values in
    the same section, or values in a special DEFAULT section.
    Additional defaults can be provided upon initialization and
    retrieval.  Lines beginning with "#" or ";" are ignored and may be
    used to provide comments.

Docutils currently only uses an "[options]" section; all other
sections are ignored.

.. Note:: The configuration file format may change in the future.

Configuration entry names correspond to internal option attributes.
Underscores ("_") and hyphens ("-") can be used interchangably in
entry names.  The correspondence between entry names and command-line
options is listed in `Configuration File Entries`_ below.

.. _ConfigParser.py:
   http://www.python.org/doc/current/lib/module-ConfigParser.html
.. _Python: http://www.python.org/
.. _RFC 822: http://www.rfc-editor.org/rfc/rfc822.txt


Configuration File Entries
--------------------------

Listed below are the Docutils runtime settings.  Most may be specified
in `configuration files`_, where hyphens may be used in place of
underscores.  Some knowledge of Python_ is assumed for some
attributes.

attribution
    (HTML Writer.)  Format for block quote attributions: one of "dash"
    (em-dash prefix), "parentheses"/"parens", or "none".

    Default: "dash".  Options: ``--attribution``.

compact_lists
    (HTML Writer.)  Remove extra vertical whitespace between items of
    bullet lists and enumerated lists, when list items are "simple"
    (i.e., all items each contain one paragraph and/or one "simple"
    sublist only).

    Default: enabled (1).  Options: ``--compact-lists,
    --no-compact-lists``.

config
    Path to a configuration file to read (if it exists)
    [#pwd]_. Settings may override defaults and earlier settings.
    This is only effective as a command-line option; setting it in a
    config file has no effect.

    Filesystem path settings contained within the config file will be
    interpreted relative to the config file's location (*not* relative
    to the current working directory).

    Default: None.  Options: ``--config``.

datestamp
    Include a time/datestamp in the document footer.  Contains a
    format string for Python's ``time.strftime``.  See the `time
    module documentation`__.

    Default: None.  Options: ``--date, -d, --time, -t,
    --no-datestamp``.

    __ http://www.python.org/doc/current/lib/module-time.html

debug
    Report debug-level system messages.

    Default: don't (None).  Options: ``--debug, --no-debug``.

docinfo_xform
    (Standalone Reader.)  Enable or disable the bibliographic field
    list transform (docutils.transforms.frontmatter.DocInfo).

    Default: enabled (1).  Options: ``--no-doc-info``.

doctitle_xform

    (Standalone Reader.)  Enable or disable the promotion of a lone
    top-level section title to document title (and subsequent section
    title to document subtitle promotion;
    docutils.transforms.frontmatter.DocTitle).

    Default: enabled ().  Options: ``--no-doc-title``.

doctype_declaration
    (XML Writer.)  Generate XML with a DOCTYPE declaration.

    Default: do (1).  Options: ``--no-doctype``.

dump_internals
    At the end of processing, write all internal attributes of the
    document (``document.__dict__``) to stderr.

    Default: don't (None).  Options: ``--dump-internals`` (hidden, for
    development use only).

dump_pseudo_xml
    At the end of processing, write the pseudo-XML representation of
    the document to stderr.

    Default: don't (None).  Options: ``--dump-pseudo-xml`` (hidden,
    for development use only).

dump_settings
    At the end of processing, write all Docutils settings to stderr.

    Default: don't (None).  Options: ``--dump-settings`` (hidden, for
    development use only).

dump_transforms
    At the end of processing, write a list of all transforms applied
    to the document to stderr.

    Default: don't (None).  Options: ``--dump-transforms`` (hidden,
    for development use only).

embed_stylesheet
    (HTML Writer.)  Embed the stylesheet in the output HTML file.  The
    stylesheet file must be accessible during processing.  The
    stylesheet is embedded inside a comment, so it must not contain
    the text "``--``" (two hyphens).

    Default: link, don't embed (None).  Options: ``--embed-stylesheet,
    --link-stylesheet``.

error_encoding
    The text encoding for error output.

    Default: "ascii".  Options: ``--error-encoding, -e``.

error_encoding_error_handler
    The encoding error handler for unencodable characters in error
    output.  Acceptable values are the same as for the "error"
    parameter of Python's ``encode`` string method.

    Default: "backslashreplace" for Python 2.3 and later; "replace"
    otherwise.  Options: ``--error-encoding-error-handler,
    --error-encoding, -e``.

exit_level
    A system message level threshold; non-halting system messages at
    or above this level will produce a non-zero exit status at normal
    exit.  Exit status is the maximum system message level plus 10 (11
    for INFO, etc.).

    Default: disabled (5).  Options: ``--exit``.

expose_internals
    List of internal attribues to expose as external attributes (with
    "internal:" namespace prefix).

    Default: don't (None).  Options: ``--expose-internal-attribute``
    (hidden, for development use only).

footnote_backlinks
    Enable or disable backlinks from footnotes and citations to their
    references.

    Default: enabled (1).  Options: ``--footnote-backlinks,
    --no-footnote-backlinks``.

footnote_references
    (HTML Writer.)  Format for footnote references, one of
    "superscript" or "brackets".

    Default: "superscript"; "brackets" in PEP/HTML Writer.  Options:
    ``--footnote-references``.

generator
    Include a "Generated by Docutils" credit and link in the document
    footer.

    Default: off (None).  Options: ``--generator, -g,
    --no-generator``.

halt_level
    The threshold at or above which system messages are converted to
    exceptions, halting execution immediately.

    Default: severe (4).  Options: ``--halt, --strict``.

indents
    (XML Writer.)  Generate XML with indents and newlines.

    Default: don't (None).  Options: ``--indents``.

input_encoding
    The text encoding for input.

    Default: auto-detect (None).  Options: ``--input-encoding, -i``.

language_code
    `ISO 639`_ 2-letter language code (3-letter codes used only if no
    2-letter code exists).

    Default: English ("en").  Options: ``--language, -l``.

newlines
    (XML Writer.)  Generate XML with newlines before and after tags.

    Default: don't (None).  Options: ``--newlines``.

no_random
    (PEP/HTML Writer.)  Workaround for platforms which core-dump on
    "``import random``".

    Default: random enabled (None).  Options: ``--no-random``
    (hidden).

output_encoding
    The text encoding for output.

    Default: "UTF-8".  Options: ``--output-encoding, -o``.

output_encoding_error_handler
    The encoding error handler for unencodable characters in output.
    Acceptable values are the same as for the "error" parameter of
    Python's ``encode`` string method.

    Default: "strict".  Options: ``--output-encoding-error-handler,
    --output-encoding, -o``.

pep_home
    (PEP/HTML Writer.)  Home URL prefix for PEPs.

    Default: current directory (".").  Options: ``--pep-home``.

pep_references
    (reStructuredText Parser.)  Recognize and link to PEP references
    (like "PEP 258").

    Default: disabled (None); enabled (1) in PEP Reader.  Options:
    ``--pep-references``.

pep_stylesheet
    (PEP/HTML Writer.)  CSS stylesheet URL, used verbatim.  Overrides
    HTML stylesheet (``--stylesheet``).

    Default: None.  Options: ``--pep-stylesheet``.

pep_stylesheet_path
    (PEP/HTML Writer.)  Path to CSS stylesheet [#pwd]_.  Path is
    adjusted relative to the output HTML file.  Overrides HTML
    stylesheet (``--stylesheet``) and PEP stylesheet
    (``--pep-stylesheet``).

    Default: None.  Options: ``--pep-stylesheet-path``.

pep_template
    (PEP/HTML Writer.)  Path to PEP template file [#pwd]_.

    Default: "pep-html-template" (in current directory).  Options:
    ``--pep-template``.

python_home
    (PEP/HTML Writer.)  Python's home URL.

    Default: parent directory ("..").  Options: ``--python-home``.

recurse
    (``buildhtml.py`` front end.)  Recursively scan subdirectories.

    Default: recurse (1).  Options: ``--recurse, --local``.

report_level
    Verbosity threshold at or above which system messages are
    reported.

    Default: warning (2).  Options: ``--report, -r, --verbose, -v,
    --quiet, -q``.

rfc_references
    (reStructuredText Parser.)  Recognize and link to RFC references
    (like "RFC 822").

    Default: disabled (None); enabled (1) in PEP Reader.  Options:
    ``--rfc-references``.

silent
    (``buildhtml.py`` front end.)  Work silently (no progress
    messages).  Independent of "report_level".

    Default: show progress (None).  Options: ``--silent``.

source_link
    Include a "View document source" link in the document footer.  URL
    will be relative to the destination.

    Default: don't (None).  Options: ``--source-link, -s,
    --no-source-link``.

source_url
    An explicit URL for a "View document source" link, used verbatim.

    Default: compute if source_link (None).  Options: ``--source-uri,
    --no-source-link``.

stylesheet
    (HTML Writer.)  CSS stylesheet URL, used verbatim.  Overridden by
    "stylesheet_path" URL option (``--stylesheet-path``).

    Default: "default.css".  Options: ``--stylesheet``.

stylesheet_path
    (HTML Writer.)  Path to CSS stylesheet [#pwd]_.  Overrides
    "stylesheet" URL option (``--stylesheet``).  Path is adjusted
    relative to the output HTML file.

    Default: None.  Options: ``--stylesheet``.

tab_width
    (reStructuredText Parser.)  Number of spaces for hard tab
    expansion.

    Default: 8.  Options: ``--tab-width``.

toc_backlinks
    Enable backlinks from section titles to table of contents entries
    ("entry"), to the top of the TOC ("top"), or disable ("none").

    Default: "entry".  Options: ``--toc-entry-backlinks,
    --toc-top-backlinks, --no-toc-backlinks``.

trim-footnote-reference-space
    (reStructuredText Parser.)  Remove spaces before footnote
    references.

    Default: don't (None).  Options:
    ``--trim-footnote-reference-space``.

warning_stream
    Path to a file for the output of system messages (warnings)
    [#pwd]_.

    Default: stderr (None).  Options: ``--warnings``.

xml_declaration
    (XML and HTML Writers.)  Generate XML with an XML declaration.

    .. Caution:: The XML declaration carries text encoding
       information, without which standard tools may be unable to read
       the generated XML.

    Default: do (1).  Options: ``--no-xml-declaration``.


For Internal Use Only
`````````````````````

Setting these in a config file has no effect.


_directories
    (``buildhtml.py`` front end.)  List of paths to source
    directories, set from positional arguments.

    Default: current working directory (None).  No command-line
    options.

_disable_config
    Prevent standard configuration files from being read.

    Default: config files enabled (None).  No command-line options.

_destination
    Path to output destination, set from positional arguments.

    Default: stdout (None).  No command-line options.

_source
    Path to input source, set from positional arguments.

    Default: stdin (None).  No command-line options.

.. _ISO 639: http://lcweb.loc.gov/standards/iso639-2/englangn.html

.. [#pwd] Path relative to the working directory of the process at
   launch.


..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End:
