# Appendices

Contents

- Appendices
  - Bugs
  - Errors while tangling
  - Errors while untangling
  - Errors while reading @file nodes
  - Errors while writing @file nodes
  - Format of .leo files
  - Format of derived files
  - Unicode reference

## Bugs

The following bugs can not be fixed:

1. The `Untangle` command has no way of updating a section whose name has been changed in the derived file.

   Because of the `@unit` directive, there is not even a way to issue a meaningful warning.

2. The `Tangle` command treats `@c` like `@code` in `CWEB` mode.

   I recommend changing `@c` to `@< c @>=` throughout your `CWEB` sources and adding a reference to `@< c @>` in all roots of `CWEB` files.

## Errors while tangling

The following error messages may be generated by the `Tangle` commands.

`Can not re-open temp file`
  A file error occurred while trying to reopen the temporary file used during tangling.
`Can not rename temporary file name`
  A file error occurred while trying to change the name of the temporary file used during tangling.
`@code expects the header: [text of header] to contain a section name`
  An @code directive appeared in body text whose header does not contain a section name.
`@directive not valid here`
  An @directive was seen somewhere other than the start of a line.
`Halting Tangle: too many errors`
  Tangle detected more than 20 errors.
`Invalid recursive reference of << section name >>`
  A section was defined in terms of itself, either directly or indirectly. The trace shows the chain of section definitions that resulted in the illegal definition.
`This message is followed by a walkback of the section names that recursively reference the section.`

  The walkback looks like this:

      called from << section name >>
      called from << section name >>
      ...

`Multiple parts not allowed for << section name >>`

  Sections can be defined in several parts in two ways:

  1. Using `<< section name >>=` in several places with the same section name.
  2. Using several `@code` directives within the same body text.

  As a precaution against mistakenly defining a section in more than one place, it is invalid to use @code in different nodes to define multiple parts for the same section. In particular, this error may arise when using cloned nodes. This error may always be eliminated by using `<< section name >>=` instead of `@code`.

`No file written because of errors`
  Tangle did not write a file because errors were found.
`Run on comment`
  A C-language comment was not properly terminated.
`Run on file name in @root directive`
  The file name in an @root directive was not terminated with the proper delimiter.
`Run on section name`
  A section name was not properly terminated before the end of the line in which it started.
`Run on string`
  A C-language string or character constant was not properly terminated.
`Section definition not valid here.`
  Something that looks like a section definition was seen in the middle of a line.
`Sections nested too deeply`
  A section was defined using more than 100 levels of section definitions (!) You could easily create an outline containing every computer program ever written in less than 50 levels.
`The outline contains no roots`
  The selected outline contained no @root directive.
`Undefined section: <<< section name >>>`
  A reference to an undefined section was encountered.
`Unexpected @directive while putting code`
  Tangle outputs 2-character WEB control code in a comment. This message is given if we find such comments in a code definition.
`Warning: possible duplicate definition of: <<< section name >>>`
  (The text of the duplicate definition follows.) The section may have been defined in more than one place.
`Warning: <<< section name >>> has been defined but not used`
  The indicated section appears in the outline but is never referenced.

## Errors while untangling

The following error messages may be generated by the Untangle commands.

Incompatible definitions of <<< section name >>>
  Two expansions of `<< section name >>` were different in the derived file. This typically arises when the programmer changes one of the expansions but not the other.
Missing root part
  No end sentinel line was found for a part of the expansion of the code in the `@root` node. This is likely the result of adding, deleting or altering a sentinel line.
`Missing root section`
  No end sentinel line was found for the expansion of the code in the `@root` node. This is likely the result of adding, deleting or altering a sentinel line.
`Missing sentinel line for <<< section name >>>`
  The end sentinel name for `<< section name >>` was expected but not found. This is likely the result of adding, deleting or altering a sentinel line.
`Unterminated section: <<< section name >>>`
  The end of the file was reached before encountering the end sentinel line for `<< section name >>`. This is likely the result of adding, deleting or altering a sentinel line.

## Errors while reading @file nodes

The following are all serious errors, meaning that the data in a derived file has been corrupted:

    Bad @+leosentinel in <filename>
    Bad @delims
    Bad attribute field in @+node
    Bad child index in @+node
    File may have damaged sentinels!
    Ignoring <sentinel kind> sentinel.  Expecting <sentinel kind>
    Ignoring unexpected @+leo sentinel
    Missing <sentinel kind> sentinel
    Missing @file in root @node sentinel
    Outline corrupted: different nodes have same clone index!
    Replacing body text of orphan <node name>
    Unexpected end of file. Expecting <sentinel kind> sentinel
    Unknown sentinel: <sentinel line>

You should restore the information from a backup .leo file using the `Read Outline Only` commands, followed by a `Write @file Nodes` command

## Errors while writing @file nodes

Errors while writing `@file` nodes are harmless. No information is lost because all information is written to the .leo file rather than the derived file.

`bad @delims directive`
  A node contains an ill-formed `@delims` directive.
`@ignore node <headline>`
  The body text of headline contains an `@ignore` directive.
`orphan node: <headline>`
  The node is referenced by no ancestor node, and no `@others` directive applies to it.
`@others already expanded in: <headline>`
  The node contains more than one `@others` directive.
`Rename failed: no file created! (file may be read-only)`
  Leo's `Save` command writes derived files to a temporary file, and replaces the derived file by the temporary files only if the two files are different. A problem with the file system prevented the temporary file from being renamed.

`Path does not exist: <filename>`

`undefined section: <section name> referenced from: <headline>`
  The node given by headline contains a reference to section name but no section definition node for section name exists in the descendants of headline.

## Format of .leo files

This technical information may be of use to those wanting to process Leo files with special-purpose filters. Leo's uses XML for its file format. The following sections describe this format in detail. Important: The actual read/write code in leoFileCommands.py is the authoritative guide. When in doubt about what Leo actually writes, look at an actual .leo file in another editor.

Here are the elements that may appear in Leo files. These elements must appear in this order.

`<?xml>`

  Leo files start with the following line:

      <?xml version="1.0" encoding="UTF-8"?>

`<?xml-stylesheet>`

  An xml-stylesheet line is option. For example:

      <?xml-stylesheet ekr_stylesheet?>

`<leo_file>`
  The `<leo_file>` element opens an element that contains the entire file. `</leo_file>` ends the file.
`<leo_header>`

  The `<leo_header>` element specifies version information and other information that affects how Leo parses the file. For example:

      <leo_header file_format="2" tnodes="0" max_tnode_index="5725" clone_windows="0"/>

  The `file_format` attribute gives the 'major' format number. It is '2' for all 4.x versions of Leo. The `tnodes` and `clone_windows` attributes are no longer used. The `max_tnode_index` attribute is the largest tnode index.

`<globals>`

  The `globals` element specifies information relating to the entire file. For example:

      <globals body_outline_ratio="0.50">
          <global_window_position top="27" left="27" height="472" width="571"/>
          <global_log_window_position top="183" left="446" height="397" width="534"/>
      </globals>

  - The `body_outline_ratio attribute` specifies the ratio of the height of the body pane to the total height of the Leo window. It initializes the position of the splitter separating the outline pane from the body pane.
  - The `global_window_position` and `global_log_window_position` elements specify the position of the Leo window and Log window in global coordinates:
`<preferences>`
  This element is vestigial. Leo ignores the `<preferences>` element when reading. Leo writes an empty `<preferences>` element.
`<find_panel_settings>`
  This element is vestigial. Leo ignores the `<find_panel_settings>` element when reading. Leo writes an empty `<find_panel_settings>` element.
`<clone_windows>`
  This element is vestigial. Leo ignores the `<clone_windows>` element when reading. Leo no longer writes `<clone_windows>` elements.
`<vnodes>`
  A single `<vnodes>` element contains nested `<v>` elements. `<v>` elements correspond to vnodes. The nesting of `<v>` elements indicates outline structure in the obvious way.
`<v>`

  The `<v>` element represents a single vnode and has the following form:

      <v...><vh>sss</vh> (zero or more nested v elements) </v>

  The `<vh>` element specifies the headline text. `sss` is the headline text encoded with the usual XML escapes. As shown above, a `<v>` element may contain nested `<v>` elements. This nesting indicates outline structure in the obvious way. Zero or more of the following attributes may appear in `<v>` elements:

      t=name.timestamp.n
      a="xxx"

  The `t="Tnnn"` attribute specifies the tnode associated with this vnode. The `a="xxx"` attribute specifies vnode attributes. The `xxx` denotes one or more upper-case letters whose meanings are as follows:

      C       The vnode is a clone. (Not used in 4.x)
      E       The vnode is expanded so its children are visible.
      M       The vnode is marked.
      T       The vnode is the top visible node.
      V       The vnode is the current vnode.

  For example, `a="EM"` specifies that the vnode is expanded and is marked.

  New in 4.0:

  - `<v>` elements corresponding to `@file` nodes now contain `tnodeList` attributes. The `tnodeList` attribute allows Leo to recreate the order in which nodes should appear in the outline. The `tnodeList` attribute is a list of gnx's: global node indices. See `Format of derived files (4.x)` for the format of gnx's.
  - Plugins and scripts may add attributes to `<v>` and `<t>` elements. See Chapter 13: Writing plugins for details.
`<tnodes>`
  A single `<tnodes>` element contains a non-nested list of `<t>` elements.
`<t>`

  The `<t>` element represents a single tnode and has this form:

      <t tx="Tnnn"><tb>sss</tb></t>

  The `tx` attribute is required. The `t` attribute of `<v>` elements refer to this `tx` attribute. The `<tb>` element specifies the body text. `sss` is the headline text encoded with the usual XML escapes.

  New in 4.0: Plugins and scripts may add attributes to `<v>` and `<t>` elements. See Chapter 13: Writing plugins for details.

## Format of derived files

This section describe the format of derived files. Leo's sentinel lines are comments, and this section describes those comments.

The format of derived files produced by Leo 4.0 and later meets the following goals:

- The minimum of sentinels needed to properly recreate the outline.
- A robust way of telling whether newlines belong to sentinels or not. Prior to Leo 4.0, deleting blank lines before and after sentinel lines could corrupt the outline structure of the derived file.
- A minimum of intrusion and ugliness.

New in Leo 4.2: Files derived from `@thin` use gnx's in `@+node` and `@-node` sentinels. Such gnx's permanently and uniquely identify nodes. Gnx's have the form:

    id.yyyymmddhhmmss
    id.yyyymmddhhmmss.n

The second form is used if two gnx's would otherwise be identical.

- `id` is a string unique to a developer, e.g., a cvs id.
- `yyyymmddhhmmss` is the node's creation date.
- `n` is an integer.

Here are the sentinels used by Leo, in alphabetical order. Unless otherwise noted, the documentation applies to all versions of Leo. In the following discussion, `gnx` denotes a gnx as described above.

@<<

  A sentinel of the form `@<<section_name>>` represents a section reference.

  If the reference does not end the line, the sentinel line ending the expansion is followed by the remainder of the reference line. This allows the Read code to recreate the reference line exactly.

@@

  The `@@` sentinel represents any line starting with `@` in body text except `@``*whitespace*, ``@doc` and `@others`. Examples:

      @@nocolor
      @@pagewidth 80
      @@tabwidth 4
      @@code

@afterref (Leo 4.0 and later)
  Marks non-whitespace text appearing after a section references.
@+all (Leo 4.0 and later)
  Marks the start of text generated by the `@all` directive.
@-all (Leo 4.0 and later)
  Marks the end of text generated by the `@all` directive.
@at and @doc

  The `@+doc` and `@-doc` sentinels delimit doc parts within a node that starts with `@doc`. These sentinels are nested within `@body` directives. Similarly, `@+at` and `@-at` sentinels delimit doc parts within a node that start with `@` whitespace. We use the following trailing whitespace convention to determine where putDocPart has inserted line breaks:

      A line in a doc part is followed by an inserted newline
      if and only if the newline if preceded by whitespace.

  To make this convention work, Leo's write code deletes the trailing whitespace of all lines that are followed by a "real" newline.

  Leo 4.0 and later: The `@+doc` and `@+at` sentinels now contain the whitespace that follows the corresponding `@doc` or `@` directives.

@+body (Leo 3.x only)
  Marks the start of body text.
@-body (Leo 3.x only)
  Marks the end of body text.
@delims
  The `@delims` sentinel inserts `@@delims` sentinels into the derived file. The new delimiter strings continue in effect until the next `@@delims` sentinel in the derived file or until the end of the derived file. Adding, deleting or changing `@@delim` sentinels will destroy Leo's ability to read the derived file. Mistakes in using the `@delims` directives have no effect on Leo, though such mistakes will thoroughly mess up a derived file as far as compilers, HTML renderers, etc. are concerned.
@+leo

  Marks the start of any derived file. This sentinel has the form:

      <opening_delim>@leo<closing_delim>

  The read code uses single-line comments if `<closing_delim>` is empty. The write code generates single-line comments if possible.

  New in Leo 4.0: The `@+leo` sentinel contains other information following `@leo`. For example:

      <opening_delim>@leo-ver=4-thin<closing_delim>

@-leo
  Marks the end of the Leo file. Nothing but whitespace should follow this directive.
@+middle (Leo 4.0 and later)
  Marks the start of intermediate nodes between the node that references a section and the node that defines the section. Typically no such sentinels are needed: most sections are defined in a direct child of the referencing node.
@-middle (Leo 4.0 and later)
  Marks the end of intermediate nodes between the node that references a section and the node that defines the section.
@+node and @-node

  Mark the start and end of a node.

  Leo 4.2 and later:: `@+node` and `@-node` sentinels use gnx's as described above:

      @+node:gnx:<headline>
      @-node:gnx:<headline>

  Leo generates these sentinels only for nodes containing body text. Leo no longer generates other `@+node` sentinels to indicate outline structure. As a result, there is no longer any need for `@+body` sentinels.

  Before Leo 4.2:: `@+node` and `@-node` sentinels used child indices, and status fields as described below:

      @+node:<child_index>:<status_fields>:<headline>
      @-node::<child_index>:<status_fields>:<headline>

  `<child_index>` is a number from 1 to n indicating the index of the node in the list of its parent's children. `<status_field>` is the cloneIndex field of the form: "C=nnn", where nnn is an immutable clone index. `<headline>` contains headline text, not reference text.

@nl (Leo 4.0 and later)
  Adds a newline to the body text.
@nonl (Leo 4.0 and later)
  Suppresses a newline from the body text.
@others
  The `@+others` sentinel indicates the start of the expansion of an `@+others` directive, which continues until the matching `@-others` sentinel. `@others` sentinels are nested within `@body` sentinels; the expansion of the `@others` directive always occurs within the body text of some node.
@verbatim
  `@verbatim` indicates that the next line of the derived file is not a sentinel. This escape convention allows body text to contain lines that would otherwise be considered sentinel lines.
@@verbatimAfterRef

  `@verbatimAfterRef` is generated when a comment following a section reference would otherwise be treated as a sentinel. In Python code, an example would be:

      << ref >> #+others

  This sentinel is required only when blank lines are suppressed between sentinel lines.

## Unicode reference

Leo uses unicode internally for all strings.

1. Leo converts headline and body text to unicode when reading .leo files and derived files. Both .leo files and derived files may specify their encoding. The default is `utf-8`. If the encoding used in a derived file is not "utf-8" it is represented in the `@+leo` sentinel line. For example:

       #@+leo-encoding=iso-8859-1.

   The `utf-8` encoding is a "lossless" encoding (it can represent all unicode code points), so converting to and from utf-8 plain strings will never cause a problem. When reading or writing a character not in a "lossy" encoding, Leo converts such characters to '?' and issues a warning.

2. When writing .leo files and derived files Leo uses the same encoding used to read the file, again with `utf-8` used as a default.

3. leoSettings.leo contains the following Unicode settings, with the defaults as shown:

       default_derived_file_encoding = UTF-8
       new_leo_file_encoding = UTF-8

   These control the default encodings used when writing derived files and .leo files. Changing the `new_leo_file_encoding` setting is not recommended. See the comments in leoSettings.leo. You may set `default_derived_file_encoding` to anything that makes sense for you.

4. The `@encoding` directive specifies the encoding used in a derived file. You can't mix encodings in a single derived file.