<?xml version="1.0" encoding="UTF-8"?>

<!-- ====================================================================== -->
<!-- = chapter-7.xml                                                      = -->
<!-- = afnix programmer's guide chapter 7                                 = -->
<!-- ====================================================================== -->
<!-- = This  program  is  free  software; you  can redistribute it and/or = -->
<!-- = modify it provided that this copyright notice is kept intact.      = -->
<!-- = This program is distributed in the hope that it will be useful but = -->
<!-- = without  any  warranty;  without  even  the  implied  warranty  of = -->
<!-- = merchantability or fitness for  a  particular purpose. In no event = -->
<!-- = shall  the  copyright  holder be liable for any  direct, indirect, = -->
<!-- = incidental  or special  damages arising  in any way out of the use = -->
<!-- = of this software.                                                  = -->
<!-- ====================================================================== -->
<!-- = copyright (c) 2005-2007 - amaury darsch                            = -->
<!-- ====================================================================== -->

<chapter volume="1" number="7">
  <title>Regular expressions</title>

  <p>
  This chapter covers the <afnix/> regular expressions
  or <em>regex</em> syntax and programming use. The <afnix/>
  regex is an original implementation with its own syntax
  and execution model. 
  </p>

  <!-- regular expression syntax -->
  <section>
    <title>Regular expression syntax</title>

    <p>
    <afnix/> implements a regular expression engine via a special
    <code>Regex</code> object. A regular expression can be built
    implicitly or explicitly with the use of the <code>Regex</code>
    object. The regex syntax uses the <code>[</code> and
    <code>]</code> characters as block delimiters. When used in a
    source file, the lexical analyzer automatically recognizes a
    regex and built the object accordingly. In other word, the
    regex system is builtin in the <afnix/> language. The following
    example shows two equivalent way to define the same regex
    expression.
    </p>

    <example>
    # syntax builtin regex
    (== [$d+] 2000)         # true
    # explicit builtin regex
    (== (Regex "$d+") 2000) # true
    </example>

    <p>
    In its first form, the <code>[</code> and <code>]</code>
    characters are used as syntax delimiters. The lexical analyzer
    automatically recognizes this token as a regex and built
    the equivalent <code>Regex</code> object. The second form is the
    explicit construction of the <code>Regex</code> object. Note also
    that the <code>[</code> and <code>]</code> characters are also
    used as regex block delimiters.
    </p>

    <!-- regex characters and meta-characters -->
    <subsect>
      <title>Regex characters and meta-characters</title>

      <p>
      Any character, except the one used as operators can be used in a
      regex. The <code>$</code> character is used as a meta-character
      -- or control character -- to represent a particular set of
      characters. For example, <code>[hello world]</code> is a
      regex which match only the <code>"hello world"</code>
      string. The <code>[$d+]</code> regex matches one or more
      digits. The following meta characters are builtin in the
      regex engine. 
      </p>

      <table>
       <title>Regular expression meta-characters</title>
       <thead>
         <tr>
           <td>Character</td><td>Description</td>
         </tr>
       </thead>
       <tbody>
         <tr><td>$a</td><td>matches any letter or digit</td></tr>
	 <tr><td>$b</td><td>matches any blank characters</td></tr>
	 <tr><td>$d</td><td>matches any digit</td></tr>
	 <tr><td>$e</td><td>matches eol, cr and eof</td></tr>
	 <tr><td>$l</td><td>matches any lower case letter</td></tr>
	 <tr><td>$n</td><td>matches eol or cr</td></tr>
	 <tr><td>$s</td><td>matches any letter</td></tr>
	 <tr><td>$u</td><td>matches any upper case letter</td></tr>
	 <tr><td>$w</td><td>matches any afnix word constituent</td></tr>
	 <tr><td>$x</td><td>matches any hexadecimal characters</td></tr>
       </tbody>
      </table>

      <p>
      The uppercase version is the complement of the corresponding
      lowercase character set.
      </p>

      <table>
       <title>Regular expression meta-characters complement</title>
       <thead>
         <tr>
           <td>Character</td><td>Description</td>
         </tr>
       </thead>
       <tbody>
         <tr><td>$A</td><td>any character except letter or digit</td></tr>
	 <tr><td>$B</td><td>any character except blank characters</td></tr>
	 <tr><td>$D</td><td>any character except digit</td></tr>
	 <tr><td>$E</td><td>any character except eol, cr and eof</td></tr>
	 <tr><td>$L</td><td>any character except lower case letter</td></tr>
	 <tr><td>$N</td><td>any character except eol or cr</td></tr>
	 <tr><td>$S</td><td>any character except letter</td></tr>
	 <tr><td>$U</td><td>any character except upper case letter</td></tr>
	 <tr><td>$W</td><td>any character except word constituent</td></tr>
	 <tr><td>$X</td><td>any character except hex characters</td></tr>
       </tbody>
      </table>

      <p>
      A character which follows a <code>$</code> character and that is
      not a meta character is treated as a normal character. For
      example <code>$[</code> is the <code>[</code> character. A
      quoted string can be used to define character matching which
      could otherwise be interpreted as control characters or
      operator. A quoted string also interprets standard
      <em>escaped sequences</em> but not meta characters.
      </p>

      <example>
      (== [$d+]   2000) # true
      (== ["$d+"] 2000) # false
      </example>
    </subsect>

    <!-- regex character set -->
    <subsect>
      <title>Regex character set</title>

      <p>
      A character set is defined with the <code>&lt;</code> and
      <code>&gt;</code> characters. Any enclosed character defines a
      character set. Note that meta characters are also interpreted
      inside a character set. For example, <code>&lt;$d+-&gt;</code>
      represents any digit or a plus or minus. If the first character
      is the <code>^</code> character in the character set, the
      character set is complemented with regards to its definition.
      </p>
    </subsect>

    <!-- regex block and operations -->
    <subsect>
      <title>Regex blocks and operators</title>

      <p>
      The <code>[</code> and <code>]</code> characters are the
      regex sub-expressions delimiters. When used at the top level of
      a regex definition, they can identify an implicit object. Their
      use at the top level for explicit construction is optional. The
      following example is strictly equivalent. 
      </p>

      <example>
      # simple real number check
      const real-1 (Regex "$d*.$d+")
      # another way with [] characters
      const real-2 (Regex "[$d*.$d+]")
      </example>

      <p>
      Sub-expressions can be nested -- that's their role -- and
      combined with operators. There is no limit in the nesting
      level.
      </p>

      <example>
      # pair of digit testing
      (== [$d$d[$d$d]+] 2000)  # true
      (== [$d$d[$d$d]+] 20000) # false
      </example>

      <p>
      The following unary operators can be used with single character,
      control characters and sub-expressions.
      </p>

      <table>
       <title>Regular expression operators</title>
       <thead>
         <tr>
           <td>Operator</td><td>Description</td>
         </tr>
       </thead>
       <tbody>
         <tr><td>*</td><td>match 0 or more times</td></tr>
         <tr><td>+</td><td>match 1 or more times</td></tr>
         <tr><td>?</td><td>match 0 or 1 time</td></tr>
         <tr><td>|</td><td>alternation</td></tr>
       </tbody>
      </table>

      <p>
      Alternation is an operator which work with a secondary
      expression. Care should be taken when writing the right
      sub-expression. For example the following regex
      <code>[$d|hello]</code> is equivalent to
      <code>[[$d|h]ello]</code>. In other word, the minimal first
      sub-expression is used when compiling the regex. 
      </p>
    </subsect>

    <!-- grouping -->
    <subsect>
      <title>Grouping</title>

      <p>
      Groups of sub-expressions are created with the <code>(</code>
      and <code>)</code> characters. When a group is matched, the
      resulting sub-string is placed on a stack and can be used
      later. In this respect, the regex engine can be used to extract
      sub-strings. The following example extracts the month, day and
      year from a particular date format:
      <code>[($d$d):($d$d):($d$d$d$d)]</code>. This regex assumes a
      date in the form <em>mm:dd:yyyy</em>.
      </p>

      <example>
      if (== (const re [($d$d):($d$d)]) "12:31") {
        trans hr (re:get 0)
	trans mn (re:get 1)
      }
      </example>

      <p>
      Grouping is the mechanism to retrieve sub-strings when a match
      is successful. If the regex is bound to a symbol, the
      <code>get</code> method can be used to get the sub-string by
      index.
      </p>
    </subsect>
  </section>

  <!-- regex object -->
  <section>
    <title>Regex object</title>

    <p>
    Although a regex can be built implicitly, the <code>Regex</code>
    object can also be used to build a new regex. The argument is
    a string which is compiled during the object construction.
    </p>

    <!-- literal object -->
    <subsect>
      <title>Literal object</title>
      
      <p>
      A <code>Regex</code> object is a literal object. This means that
      the <code>to-string</code> method is available and that a call
      to the <code>println</code> special form will work directly.
      </p>

      <example>
      const   re (Regex "$d+")
      println re           # $d+
      println re:to-string # [$d+]
      </example>
    </subsect>

    <!-- regex operators -->
    <subsect>
      <title>Regex operators</title>

      <p>
      The <code>==</code> and <code>!=</code> operators are the
      primary operators to perform a regex match. The <code>==</code>
      operator returns <code>true</code> if the regex matches the
      string argument from the beginning to the end of string. Such
      operator implies the begin and end of string anchoring. The
      <code>&lt;</code> operator returns true if the regex matches the
      string or a substring or the string argument.
      </p>
    </subsect>

    <!-- regex methods -->
    <subsect>
      <title>Regex methods</title>

      <p>
      The primary regex method is the <code>get</code> method which
      returns by index the sub-string when a group has been
      matched. The <code>length</code> method returns the number of
      group match.
      </p>

      <example>
      if (== (const re [($d$d):($d$d)]) "12:31") {
        re:length # 2
	re:get 0  # 12
	re:get 1  # 31
      }
      </example>

      <p>
      The <code>match</code> method returns the first string which is
      matched by the regex.
      </p>

      <example>
      const regex [$d+]
      regex:match "Happy new year 2000" # 2000
      </example>

      <p>
      The <code>replace</code> method any occurrence of the matching
      string with the string argument.
      </p>

      <example>
      const regex [$d+]
      regex:replace "Hello year 2000" "3000" # hello year 3000
      </example>
    </subsect>

    <!-- argument conversion -->
    <subsect>
      <title>Argument conversion</title>
      
      <p>
      The use of the <code>Regex</code> operators implies that the
      arguments are evaluated as literal object. For this reason, an
      implicit string conversion is made during such operator
      call. For example, passing the integer <code>12</code> or the
      string <code>"12"</code> is strictly equivalent. Care should be
      taken when using this implicit conversion with real numbers.
      </p>
    </subsect>
  </section>
</chapter>
