@c -*-texinfo-*-
@node Engine Internal Workings, Rendering Loop, Threads in CS, Internals
@section The Engine Internals Explained
@cindex Engine Internals

@noindent
@emph{Written by Jorrit Tyberghein,
@email{jorrit.tyberghein@@gmail.com}. Last updated 30 April 2004.}

This section explains, in some detail, how the engine works internally.
This information is mainly for people who are interested in learning
how the Crystal Space engine works internally and for people who want
to contribute to the engine proper. The sources for the 3D engine can be found
in @file{CS/plugins/engine/3d}.

Note: This discussion is specific to how the engine works in relation
to the new renderer. The old renderer is still in Crystal Space at this moment
but this discussion does not bear upon it.

@subheading Engine Data Structures

The following sections explain the various data structures that
are used in the engine. Many of these data structures are subclasses of
@samp{csObject}, which means that you can use them in your game to attach
other game specific objects.

@subheading Engine Data Structures: csEngine

At the core of everything there is the @samp{csEngine} class which implements
the publicly viewable @samp{iEngine} interface.  The engine class implements
the following interfaces:

@itemize @bullet
@item
@code{iEngine}: The main entry point for the engine.
@item
@code{iObject}: Allows attachment of game-specific data.
@end itemize

The engine class maintains the following lists of different types of objects
or operations:

@itemize @bullet
@item
@code{sectors}: Class @samp{csSector}, interface @samp{iSector}. As explained
in other parts of this manual, a sector is simply an infinite region of space.
There is always at least one sector.

@item
@code{mesh_factories}: Class @samp{csMeshFactoryWrapper}, interface
@samp{iMeshFactoryWrapper}. Mesh factory wrappers contain templates for making
objects. A factory is not visible itself but you can create mesh objects
from a factory and place them into sectors. Note that children of hierarchical
mesh factories are not in this list.

@item
@code{meshes}: Class @samp{csMeshWrapper}, interface @samp{iMeshWrapper}. These
are the actual mesh objects. Mesh objects are normally placed in sectors but
every mesh will also be present in this list so that the engine can perform
maintenance on them. A mesh object that is not in any sector, but only
in this list, will not be visible. Note that children of hierarchical
mesh objects are not in this list.

@item
@code{collections}: Class @samp{csCollection}, interface @samp{iCollection}. A
collection is a simple container for other objects (meshes, lights, etc.).
This can be useful for scripts.

@item
@code{camera_positions}: Class @samp{csCameraPosition}, interface
@samp{iCameraPosition}. If you define start positions in your map file, then
they will be placed into this list.

@item
@code{regions}: Class @samp{csRegion}, interface @samp{iRegion}.  Regions
provide a way to organize your objects.  Every object (mesh, factory, material,
light, sector, etc.) can be placed into a region. Note that a region is not
a geometrical concept. You can use one region to contain all materials,
and another to contain all mesh objects, for example. On the other hand, it is
probably most useful to have regions that represent geometrical areas which
you can load and unload from the engine (useful for making huge maps).

@item
@code{textures}: Class @samp{csTextureWrapper}, interface
@samp{iTextureWrapper}.  A texture is a processed image that can be used by the
renderer.

@item
@code{materials}: Class @samp{csMaterialWrapper}, interface
@samp{iMaterialWrapper}.  A material is a surface attribute for a mesh object.
A material usually uses at least one texture but it is also possible to have
materials that do not utilize textures at all.

@item
@code{halos}: Class @samp{csLightHalo}, interface @samp{iHalo}.  Halos and lens
flares are in this list.  This list contains all halos that were visible during
frame rendering. After the entire frame has been rendered the engine uses
this list to draw all halos that were visible.

@item
@code{shared_variables}: Class @samp{csSharedVariable}, interface
@samp{iSharedVariable}. The engine maintains a list of variables that can
be used for the engine sequence manager and also for controlling @sc{lod}
(level of detail).

@item
@code{cleanup}: This is an array that keeps track of items that the engine must
clean up (delete) when the engine itself is destroyed.

@item
@code{render_priorities}: This is a list of all render priorities supported by
the engine. Render priorities control in which order objects
in a sector must be rendered.

@item
@code{want_to_die}: This is a set of all @samp{iMeshWrapper} objects
that want to die the next frame. The purpose of this lists is for mesh
objects to schedule their own destruction (this can be useful for an
explosion mesh object for example). The mesh object cannot destroy itself
since it would then remove its own instance and this would be problematic.
@end itemize

In addition, the engine also maintains the following important variables:

@itemize @bullet
@item
@code{Strings}: Interface @samp{iStringSet}.  The string set interface is used
for converting strings to numerical identifiers.  The renderer and shader
systems uses these identifiers for tagging types of items since comparison of
numerical identifiers is faster than comparison of string identifiers.

@item
@code{cache_mgr}: Interface @samp{iCacheManager}.  This is the default cache
manager maintained by the engine.  The cache manager is responsible for storing
data that is hard to compute.  Mesh objects and other plugins may use the cache
manager to store, for example, lightmaps and other stuff.

@item
@code{defaultRenderLoop}: Interface @samp{iRenderLoop}. The default renderloop
controls how objects are rendered in a sector in combination with shaders.
More on that later. Note that sectors can override the default renderloop.

@item
@code{renderLoopManager}: Interface @samp{csRenderLoopManager}.  The renderloop
manager is responsible for creating renderloops.

@item
@code{render_context}: Interface @samp{iTextureHandle}.  This is used while
rendering on a texture (i.e. when using @code{iGraphics3D::SetRenderTarget()}
is in effect).

@item
@code{default_shader}: Interface @samp{iShader}.  This is the default shader
that is used on all materials which do not otherwise specify a shader.

@item
@code{top_clipper}: Class @samp{csRenderView}.  During rendering of a frame
this variable points at the top level clipper.  More on how @samp{csRenderView}
works later.
@end itemize

@subheading Engine Data Structures: csSector

A sector is one of the most important data structures in the engine.
It is a container for other objects (meshes and lights).
The sector class implements the following interfaces:

@itemize @bullet
@item
@code{iSector}: The main entry point for the sector.
@item
@code{iObject}: Allows attachment of game-specific data.
@end itemize

The sector maintains the following lists:

@itemize @bullet
@item
@code{meshes}: Interface @samp{iMeshWrapper}.  This is a list of all mesh
objects that are visible in this sector.  Note that a mesh can be in multiple
sectors at the same time (if it is crossing a portal boundary).

@item
@code{camera_meshes}: Interface @samp{iMeshWrapper}.  This list contains all
meshes that have the @samp{CS_ENTITY_CAMERA} flag set.  These meshes have to be
treated a bit differently (more on that later).

@item
@code{portal_meshes}: Interface @samp{iMeshWrapper}.  This is a set of meshes
that have portals which leave from this sector.  By using this set you can
quickly find all sectors that are reachable from this sector (through portals).

@item
@code{lights}: Interface @samp{iLight}.  This is a list of all lights in this
sector.

@item
@code{visibleMeshCache}: Class @samp{csSector::visibleMeshCacheHolder}.  This
is a special list of caches for visibility.  More on this later.

@item
@code{usedMeshLists}: Class @samp{csRenderMeshList}.  This is a list of data
structures which need to be cleaned up when the sector itself is destroyed.
More on this later.
@end itemize

In addition, the sector also maintains the following important variables:

@itemize @bullet
@item
@code{dynamic_ambient_color}: Class @samp{csColor}.  This is the current
dynamic ambient color that is added to all mesh objects in this sector.

@item
@code{renderloop}: Interface @samp{iRenderLoop}.  The optional renderloop for
this sector.  If null, then the engine's renderloop will be used.

@item
@code{culler}: Interface @samp{iVisibilityCuller}.  Every sector has a
visibility culler; usually an instance of either @dfn{Frustvis} or
@dfn{Dynavis}.  Frustvis is the default culler.

@item
@code{lights::kdtree}: Class @samp{csKDTree}.  The @dfn{kdtree} helps to find
lights based upon some box in the sector.  Every light is added to this tree
with a @dfn{bbox} (bounding box) that is calculated from the center of the
light and the influence radius.
@end itemize

@subheading Engine Data Structures: csMeshFactoryWrapper

A @samp{csMeshFactoryWrapper} is a wrapper class which the engine uses to keep
track of @code{iMeshObjectFactory} instances (these represent the real mesh
object factories), and also stores some engine-specific information.
The mesh factory wrapper class implements the following interfaces:

@itemize @bullet
@item
@code{iMeshFactoryWrapper}: The main entry point for the factory wrapper.
@item
@code{iObject}: Allows attachment of game-specific data.
@end itemize

A mesh factory wrapper maintains the following important variables:

@itemize @bullet
@item
@code{meshFact}: Interface @samp{iMeshObjectFactory}.  This is the actual mesh
object factory represented by this wrapper.

@item
@code{parent}: Interface @samp{iMeshFactoryWrapper}.  Mesh factories can be
hierarchical.  If they are, then this variable will point to the parent.
Otherwise it is null.

@item
@code{transform}: Class @samp{csReversibleTransform}.  This transform is only
used if this factory is a child of another factory.  In that case it is the
relative position of this child relative to the parent.  This is used when
instantiating mesh objects from a hierarchical factory to set up the initial
transforms.

@item
@code{children}: Class @samp{csMeshFactoryFactoryList}.  If this is a
hierarchical mesh factory, then the children are listed here.

@item
@code{static_lod}: Class @samp{csStaticLODFactoryMesh}.  If this is a
hierarchical factory with static @sc{lod} (level of detail), then this field
contains more information about that.
@end itemize

@subheading Engine Data Structures: csMeshWrapper

A @samp{csMeshWrapper} is a wrapper class for the engine which allows it to
keep track of an @samp{iMeshObject} instance (real mesh objects), and also
stores some engine-specific information.  The mesh wrapper class implements the
following interfaces:

@itemize @bullet
@item
@code{iMeshWrapper}: The main entry point for the mesh object wrapper.
@item
@code{iObject}: Allows attachment of game-specific data.
@item
@code{iVisibilityObject}: Used by the visibility culling plugin.  Mesh wrappers
can be registered with the visibility culler that lives in a sector.
@item
@code{iImposter}: Not yet implemented, but will be used for an imposter system.
@end itemize

A mesh object wrapper maintains the following important variables:

@itemize @bullet
@item
@code{meshobj}: Interface @samp{iMeshObject}. This is the actual mesh object
represented by this wrapper.

@item
@code{Parent} and @code{csParent}: Interface @samp{iMeshWrapper} and class
@samp{csMeshWrapper}.  If the mesh is part of a hierarchical mesh, then these
fields will point at the parent mesh.

@item
@code{children}: Class @samp{csMeshMeshList}.  This is a list of children if
this is a hierarchical mesh object.

@item
@code{factory}: Interface @samp{iMeshFactoryWrapper}.  Points a the factory
from which this mesh originated (optional).

@item
@code{movable}: Class @samp{csMovable}.  This important field contains
information about where the mesh actually resides.  It maintains a list of
sectors (a mesh can reside in multiple sectors at once) and a current position
and transform.  It is also aware of the hierarchical nature of meshes so that
you can ask the real position and the position relative to the parent.

@item
@code{zbufMode} Class @samp{csZBufMode}.  The z-buffer mode to use for
rendering this object (zfill, zuse, ztest, etc.).

@item
@code{render_priority}: Type @samp{int}.  This is a number indicating this
mesh's render priority.

@item
@code{static_lod}: Class @samp{csStaticLODMesh}.  If this mesh is the parent
for a static @sc{lod} (level of detail) mesh, then this structure will contain
more information about that.

@item
@code{light_info}: Interface @samp{iLightingInfo}.  Some meshes implement this
interface.  The engine very often requires a pointer to the mesh's
implementation of this interface.  As an optimization, it therefore maintains a
pointer to that interface here in order to avoid @code{scfQueryInterface},
which is somewhat time-consuming.

@item
@code{shadow_receiver}: Interface @samp{iShadowReceiver}.  Some meshes
implement this interface.  Again, as an optimization, a pointer to this
information is maintained.

@item
@code{portal_container}: Interface @samp{iPortalContainer}.  A portal container
is a special type of mesh which is maintained by the engine itself and which
contains portals to other sectors.  This interface can also be obtained from
the mesh object via @code{scfQueryInterface}, but it is also maintained
here as an optimization.

@item
@code{relevant_lights}: Interface @samp{iLight}.  This is used for dynamic
lighting calculation, and will contain a list of all lights that currently
affect this mesh.

@item
@code{relevant_lights_ref}: Class @samp{csMeshWrapper::LightRef}.  This is a
mirror of @samp{relevant_lights}.  It uses @samp{csWeakRef} as a trick for
detecting if a light has been removed or changed.  (Weak references are
automatically set to null as soon as the object to which the weak reference
points is deleted.)

@item
Several imposter variables: not yet implemented.
@end itemize

@subheading Engine Data Structures: csRegion

A region is a collection of objects.  It utilizes @samp{csObject} to hold all
objects that are in the region.  The region class implements the following
interfaces:

@itemize @bullet
@item
@code{iRegion}: The main entry point for the region.
@item
@code{iObject}: Allows attachment of game-specific data.
@end itemize

@subheading Engine Data Structures: csTextureWrapper

A texture wrapper is an engine object which keeps track of the real
textures from the renderers (@samp{iTextureHandle}).
The texture wrapper class implements the following interfaces:

@itemize @bullet
@item
@code{iTextureWrapper}: The main entry point for the texture wrapper.
@item
@code{iObject}: Allows attachment of game-specific data.
@end itemize

A texture wrapper maintains the following important variables:

@itemize @bullet
@item
@code{image}: Interface @samp{iImage}.  This is the object which contains the
original image data from disk.  After @dfn{preparing} this texture to make it
suitable the renderer, the image is released (unless keep_image is true).

@item
@code{keep_image}: Type @samp{bool}.  To prevent @samp{image} from being
cleared, set this to true.  Note that this increases memory usage of a single
texture, often dramatically, so only do this if the original image data will be
needed at a later time.

@item
@code{handle}: Interface @samp{iTextureHandle}.  This is the texture handle as
returned by the renderer (via the @code{iTextureManager} interface).
@end itemize

@subheading Engine Data Structures: csMaterialWrapper

A material wrapper is an engine object which keeps track of the real
materials from the renderers (@samp{iMaterialHandle}).
The material wrapper class implements the following interfaces:

@itemize @bullet
@item
@code{iMaterialWrapper}: The main entry point for the material wrapper.
@item
@code{iObject}: Allows attachment of game-specific data.
@end itemize

A material wrapper maintains the following important variables:

@itemize @bullet
@item
@code{material}: Interface @samp{iMaterial}.  This is the original
representation of the material from which the @code{iMaterialHandle} was
generated.

@item
@code{matEngine}: Interface @samp{iMaterialEngine}.  This points to the
engine-specific material class (@samp{csMaterial}, which implements both
@samp{iMaterial} and @samp{iMaterialEngine}).  The reason for the presence of
this variable is that @samp{iMaterial} is an interface that is not specific to
the engine so it is not possible to maintain engine-specific information there
(such as a reference to an @samp{iTextureWrapper}).  Instead, such information
is maintained in the @samp{iMaterialEngine} interface.

@item
@code{handle}: Interface @samp{iMaterialHandle}.  This is the actual material
handle as returned by the renderer (via the @samp{iTextureManager} interface).
@end itemize

@subheading Engine Data Structures: csMovable

The movable class is used by the @samp{csMeshWrapper} class to keep track
of the mesh's current location.
The movable class implements the following interfaces:

@itemize @bullet
@item
@code{iMovable}: The main entry point for the movable.
@end itemize

A movable maintains the following important variables:

@itemize @bullet
@item
@code{obj}: Class @samp{csReversibleTransform}.  The world-to-object transform
for the mesh represented by this movable.  For this transform @dfn{This}
represents object space and @dfn{Other} represents world space.

@item
@code{is_identity}: Type @samp{bool}.  If the @samp{obj} transform is an
identity transform (i.e. neither moves nor rotates the object) then it is
possible to perform various optimizations.

@item
@code{sectors}: Class @samp{csMovableSectorList}.  This is a list of sectors in
which the mesh object resides In many cases a mesh will only reside in only one
sector, but it is possible for a mesh to be in several sectors at once if the
mesh happens to cross a portal boundary.  Note that the application is
responsible for adding a mesh to sectors.  The engine will never do so.

@item
@code{listeners}: Interface @samp{iMovableListener}.  This is an array of
listeners (pieces of code) interested in knowing when the movable changes (i.e.
when the mesh moves).

@item
@code{parent}: Interface @samp{iMovable}.  If the mesh is being used in a
hierarchical mesh object, then this will point to the movable of the parent
mesh.

@item
@code{object}: Class @samp{csMeshWrapper}. This is the mesh for the movable.

@item
@code{updatenr}: Type @samp{long}.  One way to get notified about movable
updates is to register as a listener.  The listener will be called whenever the
movable is updated.  Another way is to keep track of the current
@samp{updatenr}.  This number is updated every time the movable is updated.
@end itemize

@subheading Engine Data Structures: csLight

The light class is the representation of a light. A light lives in a sector.
The light class implements the following interfaces:

@itemize @bullet
@item
@code{iLight}: The main entry point for the light.
@item
@code{iObject}: Allows attachment of game-specific data.
@end itemize

A light maintains the following important variables:

@itemize @bullet
@item
@code{light_id}: Type @samp{char*}.  This is a 16-byte string (@sc{md5} hash)
which is used for caching light data.  It is, hopefully, unique for a given
light since it is calculated from various light settings such as the position,
radius, sector, etc.

@item
@code{childnode}: Class @samp{csKDTreeChild}.  Lights are represented in a
@sc{kd}-tree that lives in the sector (see above).  This child object is the
representation of the light in that tree.

@item
@code{sector}: Interface @samp{iSector}. The current sector of the light.

@item
@code{center}: Class @samp{csVector3}. The current position of the light.

@item
@code{color}: Class @samp{csColor}. The current color of the light.

@item
@code{halo}: Class @samp{csHalo}.  If the light has a halo, then this field
contains a pointer to that structure.

@item
@code{dynamic_type}: Type @samp{int}.  This represents the type of the light
(dynamic, static, etc.).

@item
@code{attenunation}: Type @samp{int}.  This is the kind of attenunation that
the light uses.

@item
@code{attenuationvec}: Class @samp{csVector3}.  If the attenuation is @sc{clq}
based, then this will be the vector.

@item
@code{influenceRadius}: Type @samp{float}.  This is the influence radius of the
light.  Beyond this distance the light has almost no effect.

@item
@code{lightnr}: Type @samp{uint32}.  This number is used to detect light
changes.  It will be updated whenever the light is moved or otherwise changes.

@item
@code{light_cb_vector}: Interface @samp{iLightCallback}.  This is another way
to get notified of light changes.

@item
@code{lightinginfos}: Interface @samp{iLightingInfo}.  This is a hash set with
lighting @dfn{info}'s which this light is currently affecting.  When the light
color changes (without the light moving) the light class simply notifies every
element of this set.
@end itemize

@subheading Engine Data Structures: csPortalContainer

This is the representation of a portal container. It
is the only mesh object that is implemented directly in the engine itself.
A portal container is a collection of portals (i.e. every polygon in the
portal container is a portal to another sector).
The portal container class implements the following interfaces:

@itemize @bullet
@item
@code{iMeshObject}: A portal container is a mesh object.
@item
@code{iPortalContainer}: The portal container-specific @sc{api} (this
is the state interface of the mesh object).
@item
@code{iShadowReceiver}: A portal container can receive shadows (and pass
them along to the destination sectors).
@end itemize

A portal container maintains the following important variables:

@itemize @bullet
@item
@code{portals}: Class @samp{csPortal}.  This is a list of all portals in the
portal container.

@item
@code{data_nr}: Type @samp{uint32}.  This number is increased with each
significant change to the portal container.

@item
@code{vertices}: Class @samp{csVector3}.  This is an array of vertices that are
used by the portals in this container.  These vertices are in object space.

@item
@code{world_vertices}: Class @samp{csVector3}.  This is an array of vertices in
world space.

@item
@code{camera_vertices}: Class @samp{csVector3}.  This is an array of vertices
in camera space.

@item
@code{meshwrapper}: Class @samp{csMeshWrapper}.  A pointer back to the mesh
wrapper containing this portal container.
@end itemize

@subheading Engine Data Structures: csRenderView

This is a temporary data structure that is used during rendering. It will
be discussed in more detail in the section about drawing a frame. Here is a
brief summary of the class.
The renderview class implements the following interfaces:

@itemize @bullet
@item
@code{iRenderView}: The main entry point for the render view.
@end itemize

The renderview class maintains the following important variables:

@itemize @bullet
@item
@code{context_id}: Type @samp{uint32}.  This is a field that is incremented
with each new @code{csRenderView} instance (and also when a new render context
is created), and is used so the engine can detect if two render contexts are
the same (comparing on pointers is not valid since render contexts are reused).

@item
@code{ctxt}: Class @samp{csRenderContext}.  This is the current render context.
There is one @samp{csRenderView} instance that is used through the entire
rendering of a frame but every time a portal is traversed a new
@samp{csRenderContext} instance is created.

@item
@code{original_camera}: Interface @samp{iCamera}.  This points at the original
camera which was used when starting the rendering (i.e. the top level camera).
Normally, this camera will be constant unless there is a space warping portal
(a portal that changes space; for example, a mirror).  In that case a new
camera is created for the new render context.
@end itemize

@subheading Drawing a Frame

When rendering a frame the base entry point is @code{iEngine::Draw()}. However,
most applications do not call this function directly. Most applications will
invoke @code{iView::Draw()} instead.

@code{iEngine::Draw()} accepts two arguments: the camera to use, and the
2D clipping area. When this function is called then engine will perform the
following steps in the order shown:

@itemize @bullet
@item
@samp{current_framenumber} is incremented so that the engine and other
plugins can check the current frame number to update per-frame caches, for
example.

@item
@code{csEngine::ControlMeshes()} is called. This function checks
all meshes that want to die (the @samp{csEngine::want_to_die} set) and destroys
them.

@item
A @samp{csRenderView} instance is created and initializated with the
current camera and clipper.

@item
A few @code{iGraphics3D} methods are invoked to ensure that the clipper is
set up correctly, among other things.

@item
The engine checks if the sector has a renderloop. If so, it will use it.
Otherwise it will use the default renderloop in the engine.
It will call @code{iRenderLoop::Draw()}, which will be described below.  The
upshot of this method invocation is the recursive drawing of everything
visible.

@item
Halos are drawn.

@item
The clipper in @code{iGraphics3D} is reset.
@end itemize

@code{iRenderLoop::Draw()} is called for every sector in the recursion.  It
will first be invoked by @code{csEngine::Draw()}, and it will also be called
again whenever portal traversal occurs (see below).  The following operations
are performed by this function in the order shown:

@itemize @bullet
@item
@code{csRenderView::SetupClipPlanes()} is invoked. This function will
set up the clipping planes for the current 2D portal boundary (2D clipper) and
the current camera (view frustum). The far plane and near clipping planes
are also added if needed.

@item
Many of the shaders depend on @dfn{shader variables}.
A stack of shader variables is maintained at each level. At the lowest
level there is the renderloop. Here the variables from the shader manager
itself are pushed onto the stack.  Presently, there are no variables at this
level, but in the future one can imagine variables such as the current frame
number, and other globals.  @code{iShaderManager::PushVariables()} performs
this operation.

@item
@code{csSector::PrepareDraw()} is invoked. This function will first make
sure the visibility culler plugin for the sector is present. Then, it will
set the current sector on the @samp{csRenderView}. Next, it will call all
@dfn{callbacks} that are interested in hearing when this sector is traversed.
One of the most important things this function does however, is move all
meshes that have the @samp{CS_ENTITY_CAMERA} flag set so that their center
point is at the camera position.

@item
It loops over all steps
in this renderloop and perform them one by one. Each renderstep will
render all meshes in the sector using some shader type (more on this later).
This is the place where the meshes in this sector are actually rendered.
It is also during rendering of meshes that portals are actually traversed.
Note that, typically, only one renderstep will have portal traversal enabled.

@item
After doing all rendering of this sector the shader variables that were added
here are popped from the stack.

@item
All halos in this sector are added to the engine queue so that the
engine can draw them after render recursion ends.
@end itemize

What actually occurs during invocation of the rendersteps depends upon how the
renderloop is set up, and upon how many rendersteps there are.  What follows is
an explanation of the default case of a single render step that uses the
@emph{old-renderer compatibility} shader type (this is the renderloop that is
created by @code{iEngine::CreateDefaultRenderLoop()}).  This render step is
designed to be compatible with old renderer style maps that use lightmaps
without no special shaders.  It uses @samp{CS_ZBUF_MESH} for this render step
which means that the z-buf mode of the mesh itself will be used.  It also
disables Z-offset since there will only be one step.  Z-offset is used to make
sure that, when using multi-texturing, the polygons end up at exactly the same
Z-distance (i.e. no z fighting occurs).  The first step is always rendered
using disabled z-offset.  Subsequent steps should enable it to make sure they
will render nicely on the previously rendered base.  This render step also
enables portal traversal.

@code{iRenderStep::Perform()} resides in the generic render step plugin which
is located in the source tree at @file{CS/plugins/engine/renderloop/stdsteps}.
This function performs the following tasks in the order shown:

@itemize @bullet
@item
Obtains a list of all visible meshes (via @code{iSector::GetVisibleMeshes()})
from the sector.  A later section will go into this step in more detail.

@item
After getting a @samp{csRenderMeshList} with all visible meshes, they are
sorted based upon render priority (and for some render priorities it will also
sort on distance from the camera---i.e. alpha render priority).  In addition
it will also sort on shaders so that meshes using the same shader are put
together (this is more efficient for the renderer).

@item
Shader variables are added to the variable stack (remember
we first had the shader variables from the shader manager itself, now we
add another level of variables on that stack).  Presently, the only variables
added are the fog parameters of the current sector.

@item
The sorted and visible meshes are rendered. It will collect
all meshes with the same shader and then call @code{RenderMeshes()}.
@code{RenderMeshes()} will first set the current settings for this step
(like z-offset). Then it will get the number of passes for the given
shader and render all meshes with that shader with the given pass. Note
that the shader will also add variables to the stack at this point; in
particular variables about the current object to camera transformation and
the current material.

@item
If @samp{portaltraversal} is enabled and the mesh is a portal container (this
is indicated by @samp{csRenderMesh} having a portal field that is not null),
then the portal will be traversed by invoking
@code{iPortalContainer::Draw()}. More on this later, but this is the important
place where the recursion through portals occurs.

@item
The settings for this step are restored.
@end itemize

@subheading Drawing a Frame: Getting the Visible Meshes

One very important function that is called from within
@code{iRenderStep::Perform()} is @code{iSector::GetVisibleMeshes()}.  This
function is responsible for finding all meshes that are visible.  To calculate
this the sector uses the visibility culler that is in every sector.  At
this moment we have two possible implementations for that visibility culler:
@dfn{Frustvis} and @dfn{Dynavis}.  Frustvis performs frustum culling alone.
This means that every object that is in the view frustum is considered visible.
Dynavis does frustum culling and also attempts to find out which objects are
visible and which are not by using an occlusion buffer.  There is another
chapter explaining the differences a bit more and also how Dynavis works
internally.  The description here is only a short synopsis.
@code{GetVisibleMeshes()} performs the following actions in the order shown:

@itemize @bullet
@item
In every sector a list of visible mesh lists is maintained.  The reason for
this is that sometimes @code{GetVisibleMeshes()} is called multiple times in
one frame and it is desirable to avoid duplicating the visibility culling,
since that can be expensive (especially in the case of Dynavis).  Therefore, a
cache of visible mesh lists is kept, and each entry in that cache (i.e. every
list of meshes in that cache) has an associated current frame number and
context identification number.  These two numbers are used to check if the
cached list can be used for the given frame and context.  If there is no such
cached list then proceed to the next step.  Otherwise return the cached list
that matches the frame and context identification.

@item
If there is no matching list in the cache, then a slot in the list of cached
mesh lists which was for another frame is re-used.  It is known that the chosen
list will never be used again since that frame has already been rendered (it is
in the past).  So, if such an entry is found, the list can be re-used, and
re-calculation via the visibility culler can be performed (by calling
@code{iVisibilityCuller::VisTest()}.

@item
If no such dead entry is found, a new entry is added to the cache and is
populated with the visible meshes by again calling
@code{iVisibilityCuller::VisTest()}.
@end itemize

@code{iVisibilityCuller::VisTest()} internally uses a @sc{kd}-tree to traverse
the objects in the sector and, for all visible objects, it invokes the
@samp{iVisibilityCullerListener} callback that was provided as a parameter of
the @code{VisTest()} call.  More detail about the actual implementation of
@code{VisTest()} for both Frustvis and Dynavis will be presented in later
sections.

The callback's @code{ObjectVisible()} is a function that resides in the sector
(the culler itself is implemented in another plugin).  @code{ObjectVisible()}
will mark the given object as visible by adding all @samp{csRenderMesh}
instances as returned by the visible @samp{iMeshObject} (through the
@code{iMeshObject::GetRenderMeshes()} invocation) to the list of visible meshes
which is being built.  In addition, it will also do something special if the
mesh is a hierarchical mesh that uses static @sc{lod}.  In such a case, it will
calculate which of the children should be visible for the given distance to the
camera (and other potential factors), and will mark that child as visible too.

It is important to note that there is an important difference between normal
hierarchical meshes and static @sc{lod} meshes with regards to the visibility
culler.  Both are implemented using hierarchical meshes but, in case of a
normal hierarchical mesh, all children are registered with the visibility
culler as separate meshes.  This means that every mesh in a hierarchical mesh
is considered a separate entity for the visibility culler, and is also tested
for visibility on its own (so a child of a hierarchical mesh can be visible
even if the parent is not visible).  For a static @sc{lod} mesh the situation
is completely different.  In that case, the children are not registered with
the visibility culler but, instead, the mesh object will mark the correct child
as visible when it, itself, is visible.  This means that you must always make
sure that, with static @sc{lod}, the parent is a mesh that is at least as big
as the biggest child.  Usually, a @samp{nullmesh} with an appropriate box is
used for this purpose.

@subheading Drawing a Frame: Traversing Portals

While performing the renderstep, it will try traverse portals if that
renderstep has portal traversal enabled.  This will proceed to
@code{iPortalContainer::Draw()}, which will perform the following actions in
the order shown:

@itemize @bullet
@item
@code{Prepare()} is invoked in order to ensure that the portal container is
ready for use.  This function will initialize the portal container, compress
the vertices (remove double vertices), and calculate the bounding box and
sphere.

@item
@code{WorldToCamera()} is invoked.  This will take the world space vertices and
planes and transform them to camera space.

@item
The portal and far plane are set up, if needed.

@item
During the call to @code{GetRenderMeshes()} that occurred earlier (when the
renderloop called @code{iSector->GetVisibleMeshes()}) it tested if the portal
container object needed to be clipped to the frustum or near plane.  The result
of that clip test was stored in the variables @code{clip_plane},
@code{clip_portal}, and @code{clip_z_plane}.  If clipping is required then, at
ths stage, it will loop over every portal and clip it before traversal.  If no
clipping is required, it will simply loop over all portals and traverse them.

@item
Traversing through a portal is done with @code{DrawOnePortal()}.  This function
will first set up various things for fog and floating portals (like calling
@code{iGraphics3D::OpenPortal()}, for instance), and then call
@code{csPortal::Draw()}.

@item
@code{csPortal::Draw()} will first make sure the sector is present by calling
@code{CompleteSector()}.  This function invokes missing sector callbacks if the
sector is missing and if such callbacks have been registered.  This gives the
application a chance to load the destination sector.  If this fails then portal
traversal will not occur.

@item
@code{csPortal::Draw()} will also make sure portal recursion is not occurring
too deeply the same sector.  This is done to avoid infinite loops caused by
space warping portals that warp back in space so that the same portal is
visible again.

@item
The @samp{csRenderView} for the current portal is updated (i.e. clipper, fog
information, recursion level, last portal, previous sector, clip plane, etc.).

@item
If this is a space warping portal (a portal that changes the camera transform),
then a new camera is created.  @@@@@@ TODO Continue here...
@end itemize

@@@@@@TODO: explain Frustvis::VisTest()@*
@@@@@@TODO: explain Dynavis::VisTest()@*
@@@@@@TODO: explain the operations of one mesh object in the above context@*
