# Chapter 14: Using the rst3 plugin

Contents

- Chapter 14: Using the rst3 plugin
  - Overview
  - Comparison with previous rst plugins
  - Options
    - Options supporting the http plugin
    - Options that set command names
  - Headline Commands
  - Using doc parts
  - Setting defaults
  - The code-block directive
  - Required cascading style sheets
  - Notes about rST markup
  - Examples
  - Theory of operation
  - Controlling the rst3 plugin from scripts
  - Acknowledgements

## Overview

The rst3 plugin creates output files from Leo outlines containing reStructuredText (rST) ReStructuredText is a simple and powerful text formatting markup language. Outlines are a natural way to organize rST (or any text).

This plugin adds the Write Restructured Text command to Leo's Edit menu. The Write Restructured Text command searches the selected outline looking for rst root nodes whose headline have the form @rst <filename>. The plugin then creates the named file in various ways depending which rst3 options are in effect.

By default, the rst3 plugin creates rST headings automatically from outlines, so the higher-level nodes in the outline correspond to higher-level sections in the output. Creating rST headings automatically relieves you from one of the most tedious chores in creating rST markup.

To use this plugin effectively, Python's docutils module must be installed. The rst3 plugin will use the SilverCity syntax coloring package if it installed.

This plugin sends .htm, .html or .tex files to the docutils module for further processing. Docutils generates HTML files or LaTeX files depending on the file's extension. HTML files generated by docutils refer to three .css (cascading style sheet) files that should exist in the same directory as the generated HTML file. You can control the formatting of the HTML file by altering these .css files. See Required cascading style sheets for more details.

## Comparison with previous rst plugins

The rst3 plugin provides several important benefits compared to previous rst plugins:

1. This plugin allows much more flexible control over its operations using rst3 options. You can set options in @settings trees, in headlines and in body text. Nodes inherit most rst3 options from ancestor nodes much like nodes inherit Leo directives. See Options.

The most important option is the code_mode option. This option specifies whether to process the body text of a node as rst markup (rst mode) or source code that is to be converted to rST markup (code mode). Unlike previous versions of this plugin, any node can be processed in any mode.

2. Headlines can set any option. This turns out to be very convenient. See Headline commands for details. Nodes whose headlines start with @rst-options are treated specially. The body text of such rst options nodes should contain nothing but lines of the form:

    <option>=<value>

3. You can set any rst3 options in the body text of any node using doc parts of the form:

    @ @rst-options
    list of rst3 options, one per line
    @c

Such option doc parts allows you to specify rst3 options even with source code files.

4. You can embed rST markup in the body text of any node using doc parts of the form:

    @ @rst-markup
    any rST markup
    @c

Such markup doc parts allow you to fine-tune the formatting of your source files. See Using doc parts for full details.

All these features create a wonderful environment for using rST. There is complete integration of Leo outlines with rST markup. You can embed both rst3 options and rST markup in any node, even node in derived files.

## Options

You can set any option in any node, so you have complete control over how this plugin processes each node. The following is a list of options that control how the rst3 plugin formats each particular node.

code_mode (default: False)
  True: process nodes in code mode. False: process nodes in rst mode. In code mode, the rst3 plugin automatically creates rST markup to display the body text as an rst code-block. In rst mode the rst3 simply copies the body text of the node to the output. That is, the plugin assumes that body text is already valid rST markup.
doc_only_mode (default: False)
  True: process nodes in doc_only mode. False: process nodes in rst mode or code_mode, depending on options. In doc_only mode, the rst3 plugin outputs only regular doc parts and @ @rst-markup doc parts. Headlines create section in doc_only mode only if a) the node contains a doc part or b) the show_organizer_nodes option is in effect. As always can use @ @rst-options for per-node control of the process, especially node-by-node control of the show_organizer_nodes option.
default_path (default: '')
  The path to be prepended to filenames given in root nodes.
generate_rst (default: True)
  A master switch. True: generate rST markup for rST sections and rST code-blocks. False: don't generate rST markup and ignore @ @rst-markup doc parts.
number_code_lines (default: True)
  Controls whether to number code lines in code mode. This option has no effect in rst mode.
show_doc_parts_as_paragraphs (default: False)
  True: Move doc parts outside of the code-block directive in code mode. This option has no effect in rst mode. Cool: Any rST markup in doc parts included as the result of this option will be rendered properly.
show_headlines (default: True)
  True: automatically generate rST sections from headlines. Applies to both code mode and rst mode. The level of the node in the outline determines the level of the section underling in the rST markup. Higher-level headlines in the outline correspond to higher-level section headings; lower-level headlines in the outline correspond to lower-level section headings.
show_leo_directives (default: True)
  True: include Leo directives in code mode. This option has no effect in rst mode.
show_markup_doc_parts (default: False)
  True: include markup doc parts in code mode. This option has no effect in rst mode.
show_options_doc_parts (default: False)
  True: include options doc parts in code mode. This option has no effect in rst mode.
show_options_nodes (default: False)
  True: show @rst-options nodes.
show_organizer_nodes (default: True)
  True: generate rST sections for nodes that do not contain body text. This option has no effect unless the rST section would otherwise be written.
strip_at_file_prefixes (default: True)
  True: remove @auto, @file, @nosent and @thin from the start of headlines.
stylesheet_name (default: 'default.css')
  The name of the stylesheet passed to docutils.
stylesheet_path (default: '')
  The directory containing the stylesheet passed to docutils.
underline_characters (default: '''#=+*^~"'`-:><_''')
  The underlining characters to be used to specify rST sections. The first character is reserved so you can specify the top-level section explicitly.
verbose (default: True)
  True: write informational messages.
write_intermediate_file (default: False)
  True: writes intermediate files before sending them to docutils. This option only applies to .htm, .html and .tex files. The name of the intermediate file has the name of the output file with .txt appended. This option has effect only if the generate_rst option is True.
publish-argv-for-missing-stylesheets (Default: '')

  The arguments to be passed to docutils.core.Publisher().publish() when no stylesheet is in effect. This is a string that represents a comma-separated list of strings: For example, the option:

      publish-argv-for-missing-stylesheets=--language=de,--documentclass=report,--use-latex-toc

  results in the call:

      publish(['--language=de','--documentclass=report','--use-latex-toc'])

### Options supporting the http plugin

The following options are for the use of Bernhard Mulder's http plugin. The http plugin creates an http server running on a local port, typically 8080. When the http plugin is running you will see a purple message in the log window that looks something like this:

    http serving enabled on port 8080, version 0.91

To use the http plugin, start a web browser and enter this url:

    http://localhost:8080/

You will see a a top level page containing one link for every open .leo file. Clicking on link will cause the http server to pass a new page to the browser. You can use the browser's refresh button to update the top-level view in the browser after you have opened or closed files.

Important: See the docstring for the http plugin for information on configuring the plugin. Some of the following rst3 settings must match values of settings for the http plugin. Here are the rst3 options that support the http plugin:

http_server_support (default: False)

  A master switch: none of the following options have any effect unless this option is True. If True, the rst3 plugin does the following:

  - Writes node markers in the rst output for use by the http plugin. Node markers are rst named hyperlink targets. By default they look like:

        .. _http-node-marker-N

    where N is a unique node number.

  - Adds additional information to all nodes of the tree being formatted using Leo's unknownAttributes mechanism.

http_attributename (default: 'rst_http_attribute')

  The name of the attribute name written to the unknownAttributes attribute of each each outline node in the rst root tree. The default is 'rst_http_attribute'; it should match the following setting of the http plugin:

      @string rst_http_attributename = 'rst_http_attribute'

  This option has no effect unless the http_server_support option is True.

clear_http_attributes (default: False)
  If True the rst3 plugin initially clears the fields specified by http_attributename. This option has no effect unless the http_server_support option is True.
node_begin_marker (default: 'http-node-marker-')
  The string used for node markers. This option has no effect unless the http_server_support option is True.

### Options that set command names

The following options specify the 'spelling' of headline commands. The option_prefix and option_prefixes command also define the spelling of special doc parts.

You can change these to make them shorter or to avoid conflicts with headlines in your Leo files. The list below merely gives the default value for each setting.

code_prefix: '@rst-code'

ignore_headline_prefix: '@rst-no-head'

ignore_headlines_prefix: '@rst-no-headlines'

ignore_prefix_tree: '@rst-ignore'

ignore_node_prefix: '@rst-ignore-node'

ignore_tree_prefix: '@rst-ignore-tree'

option_prefix: '@rst-option'

options_prefix: '@rst-options'

preformat_prefix: '@rst-preformat'

rst_prefix: '@rst'

show_headline_prefix: '@rst-head'

## Headline Commands

It is often convenient to set options in headlines. This is done with the following headline commands:

@rst TEXT
  Enter rst mode. Create a section called TEXT provided the show_headlines option is set.
@rst-code TEXT
  Enter code mode. Create a section called TEXT provided the show_headlines option is set.
@rst-ignore-node TEXT
  Suppress all output from a single node. TEXT is ignored. Has no effect on descendant nodes and does not change any rst3 option.
@rst-ignore-tree TEXT and @rst-ignore TEXT
  Suppress all output from the node and its descendants. TEXT is ignored. Does not change any rst3 formatting option.
@rst-no-head TEXT
  Suppress the generation of an rST section for this node only. Does not affect descendant nodes. Has no effect on descendant nodes and does not change any rst3 option.
@rst-no-headlines TEXT
  Set the show_headlines option to False. As a result, TEXT is ignored.
@rst-option <option> = <value>
  Set a single option to the given value. The default value is True.
@rst-options TEXT
  Set zero or more options from the body text of the node. TEXT is ignored. The entire body text is treated as if it were in an @ @rst-options doc part.
@rst-preformat TEXT
  Format the entire body text of the node as if the entire text preformatted. TEXT is ignored. In effect, a line containing '::' is added to the start of the text, and all following lines of the text are indented. This option has no effect on descendant nodes.

Notes:

- Several of these commands affect only the node in which they appear. Such commands set internal settings variables only: they have no effect on the visible rst3 options.
- If a headline generates an rST section, the section name does not include the headline command. Furthermore, no rST section is generated if the TEXT is empty.

## Using doc parts

Recall that in Leo a doc part starts with the '@' directive and continues until the end of body text or until the '@c' directive is seen. For example:

    @ This is a comment in a doc part.
    Doc parts can span multiple lines.
    The next line ends the doc part
    @c

Leo converts doc parts into comments using whatever comment delimiters are in effect.

Option doc parts are doc parts that start with @ @rst-options. All other lines of such doc parts should be of the form name=value. (rST comments lines starting with '..' are allowed). For example:

    @ @rst-options
    .. This comment line is ignored.
    show_headlines=False
    show_leo_directives=False
    verbose=True
    @c

The rst3 plugin sets options from option doc parts, even in code mode.

Markup doc parts are doc parts that start with @ @rst-markup. For example:

    @ @rst-markup
    .. contents::

The rst3 plugin replaces markup doc parts by the markup they contain, even in code mode.

Option and markup doc parts are especially useful in code mode. They allow you to specify options or insert rST markup directly from with derived files. This makes your source files self contained: there is no need to add other nodes to make the rst3 plugin happy.

A cool feature: In code mode, rST markup in ordinary doc parts will be rendered properly when the show_doc_parts_as_paragraphs option is in effect. Important: Regardless of the show_doc_parts_as_paragraphs option, doc parts have no special significance in rst mode. That is, the entire doc part from the opening '@' to the closing '@c and everything in between are treated normal rST markup.

## Setting defaults

You can set the defaults for any rst3 option in several ways:

1. By setting options in the root node using @ @rst-options doc parts. For example, the root of the file that generated this documentation contains:

    @ @rst-options
    code_mode=False
    doc_mode_only=False
    generate_rst=True
    show_organizer_nodes=True
    show_headlines=True
    show_leo_directives=True
    verbose=True
    @c

2. By setting options in @settings trees. To do this, you must prefix the option name shown in this documentation with the prefix 'rst3_'. For example:

    @settings
        @bool rst3_write_intermediate_file = True

## The code-block directive

This plugin defines a code-block rST directive. The primary purpose of this directive is to show formatted source code.

In rst mode you can insert the code-block directive like any other rST markup. The rst3 plugin generates code-block directives when handling nodes in code mode.

This directive takes one argument, a language name. Like this:

    .. code-block:: Python

        import leoPlugins
        import leoGlobals as g

This directive syntax colors the code in the indented code block that follows the directive. The result looks like this if the SilverCity syntax coloring module has been installed.

import leoPlugins
import leoGlobals as g

Otherwise, the output looks like this:

    import leoPlugins
    import leoGlobals as g

See the tree called "Scripting chapter using the rst plugin" in LeoDocs.leo for many examples of how to use code-blocks.

## Required cascading style sheets

HTML files generated by the rst3 plugin assume that three .css (cascading style sheet) files exist in the same directory. For the HTML output to look good the following .css files should exist:

- default.css is the default style sheet that docutils expects to exist.
- leo_rst.css contains some style improvements based on Gunnar Schwant's DocFactory.
- silver_city.css is the style sheet that controls the syntax highlighting generated by SilverCity.

The latter two style sheets are imported at the end of the default.css.

Important: You can use cascading style sheets to do things that otherwise wouldn't be possible with "plain" rST. For instance, the background color of this page was specified in a body style.

## Notes about rST markup

The plugin reserves the '#' character for your own use so that you can specify an rST headline explicitly. For example,:

    #####
    Title
    #####

You would typically put such a title in the rst root node. Otherwise, you should let the rst3 plugin automatically generate rST sections from headlines.

To create a table of contents (TOC) put:

    .. contents:: Table of Contents

on its own line wherever you want the TOC to appear. This line can appear in body text in rst mode, or in an @ @rst-markup doc part in code mode.

## Examples

The file ListManagerDocs.html is an impressive example of the kind of output that can be generated relatively easily using this plugin.

The source for ListManagerDocs.html is wxListManager.leo. Important: wxListManager.leo was written for the rst2 plugin; it could be greatly simplified if adapted for the rst3 plugin.

The files in LeoDocs.leo under the node 'Leo's HTML Users Guide' contain examples of using the rst3 plugin.

This documentation was created using the rst3 plugin. The source code for this documentation is in LeoDocs.leo. The source code for all the rst plugins is in LeoPlugins.leo.

## Theory of operation

The code for the rst3 plugin is more complex than usual. Fortunately, the overall organization is straightforward.

defaultOptionsDict
  This dictionary represents each rst3 option. To add another option, just add another entry to this dictionary. Keys are the option name, including the rst3_ prefix. Values are the default value of the option. The hard values in the dictionary may be changed as the result of @settings entries.
processTree
  processTree is the top-level code that handles one rst root node. It calls preprocessTree to create the tnodeOptionDict ivar. processTree then calls either writeNormalTree or writeSpecialTree depending on whether text will be sent to docutils for further processing. These two methods handle mundane details of opening an closing files. Both writeNormalTree and writeSpecialTree call writeTree to do the actual work.
tnodeOptionDict
  The entries in this dictionary represents the options that are set in one particular node. The keys of tnodeOptionDict are tnodes, the values are anonymous dictionaries. These anonymous inner dictionaries contain the options that are explicitly set at each tnode (and thus each position). Preprocessing the tree this way ensures that each node (headline and body text) is parsed exactly once.
writeTree
  writeTree first calls scanAllOptions, which has the effect of initializing all options. writeTree then calls writeNode for each node that will be processed by the rst3 plugin. Options may cause the plugin to skip a node or an entire subtree.
writeNode
  writeNode first calls scanAllOptions to compute the options that are in effect for that single node. Once options have computed, processing the node is straightforward. writeNode calls writeBody and writeHeadline to do the real work. These methods generate or skip text based on various options.
scanAllOptions

  scanAllOptions recreates the optionsDict ivar to represent all the options in effect for for the particular node being processed by writeNode. Client code gets these options by calling the getOption method.

  scanAllOptions first inits all options from settings, then updates those options using the anonymous dictionaries contained in the tnodeOptionsDict. scanAllOptions works like g.scanAllDirectives, but the code is much simpler.

## Controlling the rst3 plugin from scripts

A new method has been added to the rst3 plugin to make it more easily to drive the plugin from scripts:

    def writeNodeToString (self,p=None,ext=None)

writeNodeToString scans p's tree (p defaults to presently selected node) looking for @rst nodes. When the first @rst node is found, writeNodeToString processes the node as usual, with the following changes:

- @rst need not be followed by a filename; any filename and its extension are ignored.
- Only the `ext` argument to writeNodeToString determines the type of output produced. The valid values for the `ext` argument are None (for rst output), '.html', '.pdf', and '.tex'.
- Instead of writing the result to a file, writeNodeToString returns the tuple (p,s), where p is the node whose tree produced the output, and s is the output itself.
- writeNodeToString returns after processing at most one @rst node.

Scripts can easily use writeNodeToString to convert @rst trees into various kinds of output. For example, here is the body of @button rst->html in LeoDocs.leo:

    import leoPlugins
    rst3 = leoPlugins.getPluginModule('rst3')
    if rst3:
        controller = rst3.controllers.get(c)
        if controller:
            p,s = controller.writeNodeToString(ext='.html')
            print '*' * 40,p
            print s

Notes:

- This script scans the presently selected tree for @rst nodes, just like the @button rst script does. In particular, if the presently selected tree does not contain an @rst node the search continues in parent trees. When an @rst node is found, it converts the node (and descendants) to html and returns p, the found @rst node and s, the html itself.
- The script shown above merely prints the output, but many other actions could be taken. In particular, because writeNodeToString returns p it would be possible to write a script that would scan multiple @rst nodes.
- None and '.tex' are also valid values for the ext argument to writeNodeToString. None converts the @rst tree to a single reStructuredText string, as shown in @button rst->rst.
- There is some support for ext='.pdf', but this is experimental code. Expect crashes.

## Acknowledgements

Josef Dalcolmo wrote the initial rst plugin. Timo Honkasalo, Bernhard Mulder, Paul Paterson, Kent Tenney and Steve Zatz made contributions to the rst and rst2 plugins. Edward K. Ream designed and wrote the rst3 plugin.