<!--  vim: set sw=2 sts=2 et ft=docbk:

  Part of the A-A-P recipe executive: Variables and Scopes

  Copyright (C) 2002-2003 Stichting NLnet Labs
  Permission to copy and use this file is specified in the file COPYING.
  If this file is missing you can find it here: http://www.a-a-p.org/COPYING

-->

<bridgehead>Using Variables</bridgehead>

<para>
A variable value is normally a string.  The meaning of the value depends on
where it is used.  For example, "*" is interpreted as a wildcard when a
variable is used where a file name is expected.  Thus the wildcard is not
expanded in the assignment.  If you need it, use a
<link linkend="python-expression">Python expression</link> to expand
wildcards.
</para>

<para>
When using Python you can assign any type of value to a variable.  Only a few
types are supported for variables used in &Aap; commands:
  <informaltable frame='none'>
    <tgroup cols='2'>
      <colspec colwidth="200"/>
      <tbody>
        <row>
        <entry>String</entry>
        <entry>&nbsp;</entry>
        </row>
        <row>
        <entry>Integer or Long</entry>
        <entry>converted to a string</entry>
        </row>
        <row>
        <entry>ExpandVar object</entry>
        <entry>used for delayed expansion of variables</entry>
        </row>
      </tbody>
    </tgroup>
  </informaltable>
</para>

<para>
In Python code the ExpandVar object needs to be expanded before you can use
the value it contains.  Use the
<link linkend="python-var2string">var2string()</link> function for that.
</para>

<para>
Normally using $VAR gets what you want.  &Aap; will use the kind of quoting
expected and add attributes when needed.  This depends on the place where the
variable is used.  However, when you want something else, this can be
specified:
</para>
<para>
  <informaltable frame='none'>
    <tgroup cols='2'>
      <colspec colwidth="100"/>
      <tbody>
        <row>
        <entry>$var</entry>
        <entry>depends on where it's used</entry>
        </row>
        <row>
        <entry>&nbsp;</entry>
        <entry>&nbsp;</entry>
        </row>

        <row>
        <entry>$?var</entry>
        <entry>when the variable is not set or defined use an empty string
                instead of generating an error</entry>
        </row>
        <row>
        <entry>&nbsp;</entry>
        <entry>&nbsp;</entry>
        </row>

        <row>
        <entry>$-var</entry>
        <entry>without attributes (may collapse white space)</entry>
        </row>
        <row>
        <entry>$+var</entry>
        <entry>with attributes</entry>
        </row>
        <row>
        <entry>&nbsp;</entry>
        <entry>&nbsp;</entry>
        </row>
        
        <row>
        <entry>$*var </entry>
        <entry>use rc-style expansion (may collapse white space)</entry>
        </row>
        <row>
        <entry>&nbsp;</entry>
        <entry>&nbsp;</entry>
        </row>

        <row>
        <entry>$/var</entry>
        <entry>slashes changed to backslashes (for MS-Windows commands)</entry>
        </row>
        <row>
        <entry>&nbsp;</entry>
        <entry>&nbsp;</entry>
        </row>

        <row>
        <entry>$=var</entry>
        <entry>no quotes or backslashes</entry>
        </row>
        <row>
        <entry>$'var</entry>
        <entry>aap quoted (using ' and/or " where required, no backslashes)</entry>
        </row>
        <row>
        <entry>$"var</entry>
        <entry>quoted with " (doubled for a literal ")</entry>
        </row>
        <row>
        <entry>$\var</entry>
        <entry>special characters escaped with a backslash</entry>
        </row>
        <row>
        <entry>$!var</entry>
        <entry>depends on the shell, either like $'var or $"var</entry>
        </row>
      </tbody>
    </tgroup>
  </informaltable>

</para>

<para>
In most places $var is expanded as $+'var (with attributes, using ' and " for
quoting).  The exceptions are:

  <informaltable frame='none'>
    <tgroup cols='3'>
      <colspec colwidth="150"/>
      <colspec colwidth="100"/>
      <tbody>
        <row>
        <entry>:sys</entry>
        <entry>$-!var</entry>
        <entry>no attributes, shell quoting</entry>
        </row>
        <row>
        <entry>$n in $(v[$n])</entry>
        <entry>$-=var</entry>
        <entry>no attributes, no quoting</entry>
        </row>
        <row>
        <entry>:del</entry>
        <entry>$-'var</entry>
        <entry>no attributes, normal quoting</entry>
        </row>
      </tbody>
    </tgroup>
  </informaltable>
</para>

<para>
The quoted variables don't handle the backslash as a special character.  This
is useful for MS-Windows file names.  Example:
</para>

<programlisting>
        prog : "dir\file 1.c"
                :print $'source
</programlisting>

<para>
Results in:  "dir\file 1.c"
</para>

<para>
Be careful with using "$\var" and quotes, you may not always get what you
wanted.
</para>


<bridgehead>RC-style expansion</bridgehead>

<para>
RC-style expansion means that each item in a variable is concatenated to the
item immediately before and after the variable.  Example:
</para>

<programlisting>
        var = one two three
        :print dir/$*var
</programlisting>

<para>
Results in:
</para>

<literallayout>        dir/one dir/two dir/three </literallayout>

<para>
For the expansion the variable is used as a list of white-separated items.
Quotes can be used to include white space in an item.
Use double quotes around a single quote and single quotes around a double
quote.  Escaping the meaning of quotes with a backslash is not supported.
</para>

<para>
When using rc-style expansion of a variable that is empty, the result is
empty.  Example:
</para>

<programlisting>
        aa = 
        :print bla$*aa
</programlisting>

<para>
This prints an empty line.  The "bla" is omitted, because the rc-style
expansion has zero items.
</para>

<para>
When concatenating variables and using rc-style expansion, the attributes of
the last variable overrule the identical attributes of a previous one.
</para>

<programlisting>
        v1 = foo {check = 1}
        v2 = bar {check = 2}
        vv = $*v1$v2
</programlisting>

<para>
Is equivalent to:
</para>
<programlisting>
        vv = foobar{check = 1}{check = 2}
</programlisting>

<para>
When using rc-style expansion, quotes will not be kept as they are, but
removed and re-inserted where used or necessary.  Example:
</para>

<programlisting>
        foo: "file 1.c" foo.c
                :print "dir/$*source"
</programlisting>

<para>
Results in:
</para>

<para>
<literallayout>         "dir/file 1.c" "dir/foo.c" </literallayout>
</para>



<bridgehead>Variable Indexing</bridgehead>

<para>
To get one item out of a variable that is a list of items, use an index number
in square brackets.  Parenthesis or curly braces must be used around the
variable name and the index.  The first item is indexed with zero.  Example:
</para>

<programlisting>
        BAR = beer coffee cola
        :print $(BAR[0])

        BAR_ONE = $(BAR[2])
        :print $BAR_ONE
</programlisting>

<para>
Results in:
</para>

<para>
<literallayout>        beer
        cola </literallayout>
</para>

<para>
Using an index for which no item exists gives an empty result.  When $MESSAGE
includes "warning" a message is printed about this.
</para>


<bridgehead>Using Scopes</bridgehead>

<para>
A dot is considered part of the variable name.  It separates the scope name
from the variable name within that scope.  However, a trailing dot is not part
of the variable name, so that this works:
</para>

<programlisting>
          :print $result.     # print the result
</programlisting>

<para>
  In Python code you need to explicitly specify the scope name.  
  When no scope name is given only the local scope is used.
  To get the equivalent of an &Aap; command that does not specify a scope, you
  need to use the "_no" scope in Python.
  The same example as above but now with a Python expression looks like this:
</para>

<programlisting>
          :print `_no.result`.      # print the result
</programlisting>


<bridgehead>Predefined Scopes</bridgehead>

<para>
Aap defines a scope for each recipe and each block of commands.
</para>

<para>
A user may also define a specific scope, see below.  These scope names must
start with an alphabetical name.  Scope names starting with an underscore are
used for predefined scopes.
</para>

<para>
Each time a block of commands is executed a new scope is created.  Thus when
executing the commands for a dependency a second time, its scope will not
contain items from the first time.
</para>

<para>
A variable may exist in several scopes with a different value.  To specify
which scope is to be used, a scope name is prepended before the variable name,
using a dot to separate the two.
</para>

<para>
These scope specifiers can be used to access a specific scope:

  <informaltable frame='none'>
    <tgroup cols='2'>
      <colspec colwidth="100"/>
      <tbody>
        <row>
        <entry>_recipe</entry>
        <entry>The current recipe.  Useful in build commands that are
                    defined in the recipe.</entry>
        </row>
        <row>
        <entry>_top</entry>
        <entry>The toplevel recipe.  This can be regarded as the global
                    scope.</entry>
        </row>
        <row>
        <entry>_default</entry>
        <entry>The scope of default values, after the defaults settings
                    have been done, but before reading user or system startup
                    recipes.  Cannot be used in the recipe that sets the
                    default settings.</entry>
        </row>
        <row>
        <entry>_start</entry>
        <entry>The scope of startup values, as it was before reading the
                    toplevel recipe.  Cannot be used in the recipe that sets
                    the default settings and in the startup recipes.</entry>
        </row>
        <row>
        <entry>_arg</entry>
        <entry>The scope of variables set on the command line.
                Can be used to obtain the values set when &Aap; was executed
                or arguments of the <link
                  linkend="cmd-execute">:execute</link> command.
                Although the scope is writable, thus you can mess it up...
                    </entry>
        </row>
        <row>
        <entry>_parent</entry>
        <entry>The parent recipe.  Only valid in a child recipe.</entry>
        </row>
        <row>
        <entry>_caller</entry>
        <entry>The scope of the command block that invoked the current
                    command block.  Can only be used in command blocks of
                    dependencies, rules, actions and 
                    <link linkend="cmd-tree">:tree</link>.</entry>
        </row>
      </tbody>
    </tgroup>
  </informaltable>
</para>

<para>
These scope specifiers can be used to search scopes to find a variable.  The
first scope in which the variable exists is used.

  <informaltable frame='none'>
    <tgroup cols='2'>
      <colspec colwidth="100"/>
      <tbody>
        <row>
        <entry>_no</entry>
        <entry>No scope, equal to leaving out the scope specifier in
                    recipe commands, but required in Python commands.
                    First looks in the current scope, then "_stack" and then
                    "_tree".</entry>
        </row>
        <row>
        <entry>_stack</entry>
        <entry>Uses the scope of the command block that invoked the
                    current scope, the command blocks that invoked that scope,
                    and further up the call stack.  Excludes the toplevel.
                    Can only be used in command blocks of dependencies, rules
                    and actions.</entry>
        </row>
        <row>
        <entry>_tree</entry>
        <entry>Uses the scope of the current recipe, its parent, the
                    parent of its parent, etc., up to the toplevel.
                    In the toplevel recipe it is equal to "_top".</entry>
        </row>
        <row>
        <entry>_up</entry>
        <entry>First uses "_stack" and then "_tree", but excludes the
                    current scope.</entry>
        </row>
      </tbody>
    </tgroup>
  </informaltable>
</para>

<para>
These are the scopes searched for a variable with the "_up" scope when it is
used in a build command block:
<orderedlist>
<listitem><para>
  Invoking command blocks.  That is, the scope of the command block that
  invoked the current command block with an
  <link linkend="cmd-update">:update</link> command or because of a dependency.
  Then the scope of the command block that invoked that command block, and so
  on.  This excludes the toplevel.
</para></listitem>
<listitem><para>
 The recipe in which the command block was defined, its parent and so on.
 This goes on until and including the toplevel.
</para></listitem>
</orderedlist>

This is used both for reading a variable and assigning a new value.  It is an
error when assigning a new value to a variable that does not exist.
</para>

<para>
For the commands of an action the sequence is slightly different:
<orderedlist>
<listitem><para>
  Invoking command blocks.  That is, the scope of the command block that
  invoked the action with a
  <link linkend="cmd-do">:do</link> command.
  Then the scope of the command block that invoked that command block, and so
  on.  This excludes the toplevel.
</para></listitem>
<listitem><para>
 The recipe in which the ":do" command was invoked, its parent and so on.
 This goes on until and including the toplevel.
</para></listitem>
<listitem><para>
 The recipe in which the action was defined, its parent and so on.
 This goes on until the toplevel.
</para></listitem>
</orderedlist>
</para>

<para>
The "_no" scope is used for a variable in recipe commands without a specified
scope.  Thus these two are equivalent:
</para>

<programlisting>
          :print $foobar
          :print $_no.foobar
</programlisting>

<para>
But in Python commands a variable without a specified scope is always in the
local scope.  You must use "_no" to get the same effect:
</para>

<programlisting>
          foo = $bar         # finds "bar" in local or "_up" scope
          @foo = bar         # finds "bar" in local scope only
          @foo = _no.bar     # finds "bar" in local or "_up" scope
</programlisting>

<para>
When reading a variable with the "_no" scope it is first looked up in the
local scope.  If it does not exist, the "_stack" and "_tree" scopes are used,
as explained above.
</para>

<para>
When writing a variable without a specified scope it is always put in the
local scope.  A specific situation where this may lead to an unexpected result
is appending:
</para>

<programlisting>
          foo += something
</programlisting>

<para>
This is equivalent to:
</para>

<programlisting>
          foo = $foo something
</programlisting>

<para>
This obtains the value of "foo" from the first scope where it is defined, but
it is set in the current scope.  To change the variable where it is defined
use the "_no" scope explicitly:
</para>

<programlisting>
          _no.foo += something
</programlisting>


<bridgehead>User Scopes</bridgehead>

<para>
The user can define a new scope by assigning a value to a variable, using the
scope name:
</para>

<programlisting>
        s_debug.foo = xxx
</programlisting>

<para>
This creates the scope "s_debug" if it didn't exist yet.  The variable "foo"
within that scope is assigned the value "xxx".
</para>

<para>
The scope name must start with an alphabetic character.  Following characters
may be letters, digits and the underscore.
</para>

<para>
A user defined scope is only used when explicitly specified.  The "_no" and
"_up" scopes do not use it.
</para>

<para>
The scope can be accessed from everywhere, except recipes that create a new
toplevel scope have their own set of user defined scopes.  That is when using
<link linkend="cmd-execute">:execute</link> or 
":child {nopass}".  ":execute {pass}" and
<link linkend="cmd-child">:child</link> do share the
user scopes.
</para>

<para>
There cannot be a scope name and a variable with the same name.  This applies
to variables in ALL scopes!  Thus when you have a scope "foo" in one place,
you cannot use the variable "foo" anywhere else.  The only exception is that
you can use the variable "foo" in scopes that have been abandoned when the
user scope "foo" is created, but that is tricky.
</para>

<para>
Recommendation: Let user scope names start with "s_".
</para>

<para>
A user scope can be specified for a dependency:
<programlisting>
        s_foo.OPTIMIZE = 4
        ...
        foo : {scope = s_foo} foo.c
            :do build $source
</programlisting>
</para>

<para>
A user scope can be specified for a rule:
<programlisting>
        :rule %.a : {scope = s_aaa} %.b
</programlisting>
</para>

<para>
A user scope can be specified for an action:
<programlisting>
        :do foobar {scope = s_some} foo.bar
</programlisting>
</para>


<bridgehead>Variables In Build Commands</bridgehead>

<para>
A dependency and a rule can have a list of commands.  For these commands the
following variables are available:

  <informaltable frame='none'>
    <tgroup cols='2'>
      <colspec colwidth="150"/>
      <tbody>
        <row>
        <entry>$source</entry>
        <entry>The list of input files as a string.</entry>
        </row>
        <row>
        <entry>$source_list</entry>
        <entry>The list of input files as a Python list.</entry>
        </row>
        <row>
        <entry>$source_dl</entry>
        <entry>Only for use in Python commands: A list of
                        dictionaries, each input item is one entry.</entry>
        </row>
        <row>
        <entry>$depend</entry>
        <entry>The list of dependencies (source files plus virtual
                        dependencies) as a string.</entry>
        </row>
        <row>
        <entry>$depend_list</entry>
        <entry>The list of dependencies (source files plus virtual
                        dependencies) as a Python list.</entry>
        </row>
        <row>
        <entry>$depend_dl</entry>
        <entry>Only for use in Python commands: A list of
                        dictionaries, each dependency item is one
                        entry.</entry>
        </row>
        <row>
        <entry>$target</entry>
        <entry>The list of output files as a string.</entry>
        </row>
        <row>
        <entry>$target_list</entry>
        <entry>The list of output files as a Python list.</entry>
        </row>
        <row>
        <entry>$target_dl</entry>
        <entry>Only for use in Python commands: A list of
                        dictionaries, each output item is one entry.</entry>
        </row>
        <row>
        <entry>$buildtarget</entry>
        <entry>The name of the target for which the commands are
                        executed.  It is one of the items in $target.</entry>
        </row>
        <row>
        <entry>$match</entry>
        <entry>For a rule: the string that matched with %</entry>
        </row>
      </tbody>
    </tgroup>
  </informaltable>
</para>

<para>
Example:
</para>

<programlisting>
        doit {virtual}:
            :print building $target
        prog : "main file.c" doit
            :print building $target from $source
</programlisting>

<para>
Results in:

<literallayout>          building doit{virtual=1}
          building prog from "main file.c" </literallayout>
</para>

<para>
Note that quoting of expanded $var depends on the command used.
</para>

<para>
The Python lists $source_list and $target_list can be used to loop over each
item.  Example:
</para>

<programlisting>
        $OUT : foo.txt
            @for item in target_list:
                :print $source > $item
</programlisting>

<para>
Note the difference between $source and $depend: $source only contains real
files, $depend also contains virtual dependencies.
</para>

<para>
The list of dictionaries can be used to access the attributes of each item.
Each dictionary has an entry "name", which is the (file) name of the item.
Other entries are attributes.  Example:
</para>

<programlisting>
        prog : file.c {check = md5}
            @print source_dl[0]["name"], source_dl[0]["check"]
</programlisting>

<para>
Results in:  file.c  md5
</para>
