/*!
 \mainpage

\htmlonly
<H1>Extended TCL Commands</H1>

The list of extended TCL commands can be found <a href="tcl/tclindex.html">here</a>.
\endhtmlonly

<H1>Introduction</H1>
	<p>The Generic API is divided into two layers, the TCL layer and the C++ layer.  the TCL layer is what the end-user "sees" -- they write code which the TCL layer executes.  The C++ layer, on the other hand, is available only indirectly.

<p>TCL provides a mechanism for extending the command-set by writing functions in C or C++.  These functions are loaded into TCL as a shared object using the package requires or load command.  When this is done, the extended commands become available.  The C++ layer consists of these extended commands and the memory space which they use to store and manipulate data.

<p>In order to create C or C++ functions which TCL can understand, one must write them in a special format.  In LDAS, this is handled by a program known as SWIG (Simplified Wrapper and Interface Generator).  The SWIG program uses 1 or more SWIG interface files to create a set of C functions which TCL can understand.  These functions are written to a source file (.c or .cc) which is then compiled into the shared object.

	<p>This document is divided into 6 areas:
	<p><ol>
	  <li><A href="#pointers">Pointers in TCL</A></li>
	  <li><A href="#exceptions">Exception Handling & Error Messages</A></li>
	  <li><A href="#registry">Object Registry</A></li>
	  <li><A href="#apistore">API Store</A></li>
	  <li><A href="#ilwdelements">ILWD Elements</A></li>
	  <li><A href="#commands">Command Overview</A></li>
	</ol>
	
	<br><hr>
	<A name="pointers"></A>
	<h2>1. Pointers in TCL</h2>
	<p>The generic API creates a variety of objects in the C++ layer which need to be manipulated by TCL.  In order to do so, each of these objects must have a unique label.  The pointer value of the object in the C++ layer is used as the label.  This value is conveted by SWIG into an ascii representation of the pointer with the format:
	
	<p>_xxxxxxxx_classname_p

	<p>where 'xxxxxxxx' is a hexadecimal number representing the pointer value (it may have fewer than 8 characters) and 'classname' is the name of the class.  For example, a  pointer to an ObjectSpace TCP Socket class (os_tcp_socket) may appear as:

	<p>_80afc18_os_tcp_socket_p

	<p><hr>
	<A name="exceptions"></A>
	<h2>2. Exception Handling & Error Messages</h2>

	Error messages contain 3 parts:
	
	<ol>
	  <li>Error label - This consists of a short message with spaces replaced by underscores (for example, 'invalid_compression').  This is meant to identify the specific error which has occurred.</li>
	  <li>Optional error information - Any extra information about the error</li>
	  <li>Error Location - The filename and line number (in the C++ code) at which the exception occurred.</li>
	</ol>
	
	<p>The general format for an error message is:
	<pre>
    ldas_exception: error information [filename:line]
	
    or    

    ldas_exception [filename:line]
	</pre>
	  The colon after the label is omitted if there is no extra error information.

	<p>Each exception which is thrown by a generic API command will consist of one or more error messages, concatenated together.  The label and information (if it exists) for the most recent message will be returned by the function.  For example:
	<pre>
    illegal_format: 'asci' format unknown

    or

    invalid_element
	</pre>

	<p>The remaining error messages and location information can be obtained via the 'errorInfo' TCL variable.  For example, a 'puts $errorInfo' command might return:
	<pre>
    illegal_format: 'asci' format unknown [genericAPI.cc:1045]
        invoked from within
    "set e1 [putElement $bstring asci]"
    ...

    or

    illegal_element [LdasElement.cc:415]
    invalid_attribute: 'formatt' attribute unknown [LdasElement.cc:719]
        invoked from within
    "set e1 [putElement "&lt;real_4 formatt='ascii'&gt;1.34&lt;/real_4&gt;"]"
	...
	</pre>
	
	<h3>Special Exceptions</h3>
	
	<ul type=disc>
		<li>unexpected_exception - This exception can be thrown by ANY generic API command.  This occurs when an exception is thrown which was not expected.  These exceptions typically occur from the implementation (for example, the standard libraries) which does not utilize throw specifications for all of the standard library functions.  In practice, this should not occur.  However, I am catching them just in case (it's better than aborting the program).</li>
	</ul>

	<p><hr>
	<A name="registry"></A>
	<h2>3. Object Registry</h2>

	<p>The generic API instantiates many objects in the C++ layer.  Pointers to these objects are returned to the TCL layer, where they are used as parameters in other generic API function calls.  This creates two problems:
	<ol>
	  <li>The TCL user may not destruct these objects when they are no longer needed; this will result in a memory leak.</li>
	  <li>2) The TCL user may destruct the object, but then try to access the object again with the pointer value they have.  This will result in a segmentation fault.</li>
	</ol>

	<p>In order to prevent this from happening, the Generic API uses a utility class called 'ObjectRegistry'.

	<p>The ObjectRegistry is a templated class which keeps track of dynamically instantiated objects.  Whenever the generic api creates an object, the pointer to this object is registered with the 'ObjectRegistry' class.  When an object is destructed, the reference is removed from the class.  Every time a method is called which takes a pointer to an object as a parameter, the code checks to see whether or not this object exists in the object registry.  If the object does not exist, then an exception is thrown ( e.g., 'invalid_socket' or 'invalid_server' ).  If the object does exist, then the method continues.  

	<p>The ObjectRegsitry object is instantiated as a global variable in the genericAPI.cc file.  When the genericAPI shared object is loaded, this object is instantiated.  When the shared object is unloaded, the destructor for this object is called.  The ObjectRegistry destructor destructs all of its referenced objects.

	<p>The <code>reset</code> extended TCL command causes all ObjectRegistry objects to destruct their contents.  This ensures a 'clean slate' and restores the API to its original state.  This will also reset other loaded API's which have registered themselves with this API.
	<p>The <code>save</code> command writes all ObjectRegistry objects to a file.
	<p>The <code>restore</code> command reads ObjectRegistry objects from a file.

	<p><hr>
	<A name="apistore"></A>
	<h2>4. API Store</h2>
	<p>The Generic API has three commands which are supposed to affect not only itself, but any other API's which have been loaded into the TCL interpreter.  These commands are <code>reset</code>, <code>save</code>, and <code>restore</code>.

	<p>In order for these commands to execute properly, the Generic API must know about all of the loaded API's.  This is done through the Api Store.

	<p>The Api Store is a C++ class (<code>ApiStore</code>) which stores "proxies" (<code>ApiProxy</code> objects) which contain information about a loaded API.  A proxy stores:
	<ol>
	  <li>The name of the API</li>
	  <li>The address of the API's reset function.</li>
	  <li>The address of the API's save function.</li>
	  <li>The address of the API's restore function.</li>
	</ol>
		The function addresses may be NULL, in which case they are never called.

		<p>When an API is loaded, it registers its proxy with the API store.  The Generic API <code>reset</code>, <code>save</code>, and <code>restore</code> functions then access the API store in order to call the appropriate function in the other API's.  The API is registered by simply creating a static instance of an <code>ApiProxy</code> object in the shared object, for example:
	  <pre>
namespace
{
    Registry::ApiProxy frameApiProxy( 
        "Frame", &amp;resetFrameApi, &amp;saveFrameApi, &amp;restoreFrameApi );
}
	</pre>

	<p><hr>
	<A name="ilwdelements"></A>
	<h2>5. ILWD Elements</h2>
	<p>The Internal Light-Weight Data Format is a subset of XML.  It is designed to consist of the minimum number of elements needed to move data through sockets and between the TCL and C++ layers in the LDAS API's.  It is primarily meant to be a machine oriented data format, however it is also meant to be viewable in ASCII form without requiring a special browser.

	<p>There are four different types of ILWD elements:
	<ol>
	  <li><A ref="#arrayelement">Arrays</A> - An Array of numerical data</li>
	  <li><A ref="#stringelement">Strings</A> - A string or list of strings</li>
	  <li><A ref="#containerelement">Containers</A> - Container for other ILWD elements</li>
	  <li><A ref="#externalelement">External</A> - Non-parsed data</li>
	</ol>
	
	<A name="arrayelement"></A>
	<h4>Array Elements</h4>
	Array elements are used to store multi-dimensional numerical data.  They possess the following tag names:
	<ol type=a>
	  <li><B>CHAR</B> - signed byte</li>
	  <li><B>CHAR_U</B> - unsigned byte</li>
	  <li><B>INT_2S</B> - 2 byte signed integer</li>
	  <li><B>INT_2U</B> - 2 byte unsigned integer</li>
	  <li><B>INT_4S</B> - 4 byte signed integer</li>
	  <li><B>INT_4U</B> - 4 byte unsigned integer</li>
	  <li><B>INT_8S</B> - 8 byte signed integer</li>
	  <li><B>INT_8U</B> - 8 byte unsigned integer</li>
	  <li><B>REAL_4</B> - 4 byte IEEE 754 floating point number</li>
	  <li><B>REAL_8</B> - 8 byte IEEE 754 floating point number</li>
	</ol>
	
	<p>Array elements also possess the following attributes:
	<ol type=a>
	  <li>name = "name1:name2:name3:..." - the name or names to be associated with this data.</li>
	  <li>ndim = "integer" - The number of dimensions in the rawdata.  This must be greater-than or equal-to 1.  This defaults to 1 if not present.</li>
	  <li>dims = "integer, integer, ..." - ndim comma delimited integers telling the number of elements in each dimension.  If ndim > 1, then this attribute is required, otherwise it defaults to 1.</li>
	  <li>units = "unit1, unti2, ... " - ndim comma delimited unit names specifying the units, if any, for each dimension.  Defaults to an empty string if not present.</li>
	  <li>mdorder = "f77 | c" - indicates whether a multidimensional data set is to be incremented fastest on the first index (f77) or the last index (c); the default is "c".</li>
	  <li>format = "ascii | base64 | binary" - The encoded format of the data; if "ascii" then compression is not allowed and each number is whitespace or comma delimited.  The default is "ascii".</li>
	  <li>compression = "none | gzip[0-9] } gzip" - A string specifying the level of compression used for binary or base64 formats.  The values "none" & "gzip0" are equivalent.  The values "gzip" & "gzip6" are equivalent.  The default is "none".</li>
	  <li>byteorder = "little | big" - Indicates whether data is stored in little-endian or big-endian order.  The default is little-endian.</li>
	  <li>bytes = "integer" - The number of bytes of data between the start and end tags.  This is required if the format is binary and compression is non-zero.</li>
	  <li>comment = "arbitrary text string" - optional comment, defaults to an empty string if not present.</li>
	</ol>

	<A name="stringelement"></A>
	<h4>String Element</h4>
	<p>A string element is used to store a list of ASCII strings.  The tag for a string element is <B>lstring</B>.  

	<p>If there is more than one string in the element, the strings are delimited by a "\," (backslash-comma) character sequence.  Backslashes appearing within a string must be escaped with another backslash.
	  
	<p>Strings possess the following attributes:
	<ol type=a>
	  <li>name = "name1:name2:name3:..." - the name or names to be associated with this data.</li>
	  <li>dims = "integer" - The number of strings contained within the element.  Defaults to 1 if not present.</li>
	  <li>format = "ascii | base64 | binary" - The encoded format of the data; if "ascii" then compression is not allowed and strings will be delimited by "\,".  In this case, backslashes appearing in the string must be escaped with another backslash.  If the format is "base64", the strings will be delimited by "\," and THEN converted into base64.  The "binary" format is equivalent to "ascii" for this element (except that "binary" can be compressed).  The default is "ascii".</li>
	  <li>compression = "none | gzip[0-9] } gzip" - A string specifying the level of compression used for binary or base64 formats.  The values "none" & "gzip0" are equivalent.  The values "gzip" & "gzip6" are equivalent.  The default is "none".</li>
	  <li>size = "integer" - The number of characers of data, this includes delimiter characters.</li>
	  <li>bytes = "integer" - The number of bytes of data between the start and end tags.  This is used only if the format is binary and compression is non-zero.</li>
	  <li>comment = "arbitrary text string" - optional comment, defaults to an empty string if not present.</li>
	</ol>

	<A name="containerelement"></A>
	<h4>Container Element</h4>
	
	<p>A container element is used to store other elements, including other containers.  The tag for a container element is <B>ilwd</B>,

	  <p>A container possesses the following attributes:
	<ol type=a>
	  <li>name = "name1:name2:name3:..." - the name or names to be associated with this data.</li>
	  <li>size = "integer" - The number of elements within the container.  This defaults to 1 if not present.  A size of zero is allowed.</li>
	  <li>comment = "arbitrary text string" - optional comment, defaults to an empty string if not present.</li>
	</ol>

	<A name="externalelement"></A>
	<h4>External Element</h4>

	<p>An external element is used to hold information which is not in the ILWD format.  The contents of an external element is completeley ignored when parsing.  The "size" attribute is used to skip over the data.  The tag for an external element is <B>external</B>.

	  <p>An external element possesses the followin attributes:
	<ol type=a>
	  <li>name = "name1:name2:name3:..." - the name or names to be associated with this data.</li>
	  <li>format = "ascii | base64 | binary" - The encoded format of the data; if "ascii" then compression is not allowed and each number is whitespace or comma delimited.  The default is "ascii".</li>
	  <li>compression = "none | gzip[0-9] } gzip" - A string specifying the level of compression used for binary or base64 formats.  The values "none" & "gzip0" are equivalent.  The values "gzip" & "gzip6" are equivalent.  The default is "none".</li>
	  <li>size = "integer" - The number of bytes of data contained in the external element.</li>
	  <li>bytes = "integer" - The number of bytes of data between the start and end tags.  This is used only if the format is binary and compression is non-zero.</li>
	  <li>comment = "arbitrary text string" - optional comment, defaults to an empty string if not present.</li>
	  <li>parser = "none | ascii | xml" - The parser to use on the contained data.</li>
	</ol>

	<p><hr>
	<A name="commands"></A>
	<h2>6. Command Overview</h2>	
	
	<ol type=a>
	  <li><A href="#binary">Binary Commands</A></li>
	  <li><A href="#element">Element Commands</A></li>
	  <li><A href="#socket">Socket Commands</A></li>
	  <li><A href="#utility">Utility Commands</A></li>
	</ol>

	<A name="binary"></A>
	<h3>a. Binary Commands</h3>

	<p>The Generic API <A href="tcl/Binary.html">binary commands</A> implement the following features:

    <ul type=disc>
        <li>Transfer of Binary objects between the TCL and C++ layers.
        <li>Transfer of Binary objects through sockets.
        <li>Manipulation of Binary objects in the C++ layer.
	</ul>

	<A name="element"></A>
	<h3>b. Element Commands</h3>

	<p>The Generic API <A href="tcl/Element.html">element commands</A> implement the following features:

	<ul type=disc>
        <li>Transfer of ILWD objects between the TCL and C++ layers.
        <li>Transfer of ILWD objects through sockets.
        <li>Manipulation/Querying of Elements in the C++ layer.
        <li>ILWD File I/O.
	</ul>
	
	<h4>ILWD format & compression</h4>
	<p>One of the responsibilities of the Generic API is the manipulation of data in the Internal Light-Weight Data (ILWD) format.  The purpose of the ILWD format is to store information in a way which enables the user to easily view and interpret the data.  The format chosen for ILWD is very similar to XML.
	
	<p>When an ILWD object is written in "XML" form, it can have three different formats:
	<ul>
	  <li>ASCII - The element's data will be written using only printable characters.  Numerical data will be written in decimal form (e.g. 10 45 -139, etc.).</li>
	  <li>Base64 - The element's data will be converted into Base-64 and then printed.</li>
	  <li>Binary - The element's data will be written as binary, and therefore may contain non-printable characters.  Note that this defies the XML specification.</li>
	</ul>
	In addition to these formats, the user can also specify a compression type and level.  Currently the only supported types are:
	<ul>
	  <li>none - The data is not compressed.</li>
	  <li>gzip0-gzip9 - The data is compressed using the gzip protocol at the specified compression level.  The level is optional, and if the user does not supply it a default level of 6 is used.</li>
	</ul>
	If the write format chosen by the user is ASCII, then the only valid compression level is "none".
	<p>When ILWD data is stored as an object in memory, it is stored in its uncompressed binary form.  For example, the data for an array element such as &lt;real_4&gt; will be stored as a normal C array of type REAL_4 (REAL_4 is normally a float).  When it is time to convert the element to XML form (for example, when writing to a file) the user specifies what format and compression he or she wants.
	<p>However, sometimes the user may want to write a container object where the contained elements have different formats.  For example, we might have:

&lt;ilwd name='strain' size='2'&gt;<br>
    &lt;int_4u name='time' comment='GPS time in seconds' format='ascii'&gt;45981475&lt;/int_4u&gt;<br>
    &lt;real_8 name='data' dims='1024' format='base64' compression='gzip'&gt;A5Bas391 ... &lt;/real_8&gt;<br>
&lt;/ilwd&gt;<br>

In this example the user way want to be able to read the time at which the data was taken, but they may not need to have the data in readable form.

<p>In order to support this, each element object can also store a write format and compression.  These specify the default format and compression which the element should use when writing itself.  These attributes are set via the <code>setWriteFormat</code> command.  The current defaults can be obtained via the <code>getWriteFormat</code> command.  In order to write an element using the default format or compression, the user can use a format of "user" and a compression of "user".  These are the defaults for the commands which write an element in XML form (<code>getElement</code>, <code>writeElement</code>, and <code>sendElementAscii</code>).  When an element is read into memory from its XML form, the format and compression used become the defaults.  Functions which do this are <code>putElement</code>, <code>readElement</code>, and <code>recvElementAscii</code>.

Along with the data, the element stores a default write format and compression.  

	<A name="socket"></A>
	<h3>c. Socket Commands</h3>
	
	<p>The Generic API <A href="tcl/Socket.html">socket commands</A> implement the following features:

	<ul type=disc>
        <li>Socket Creation/Destruction.
        <li>Socket Connection.
        <li>Socket Querying.
    </ul>

	<A name="utility"></A>
	<h3>d. Utility Commands</h3>
	
	<p>The Generic API <A href="tcl/Utility.html">utility commands</A> implement the following features:

	<ul type=disc>
        <li>API Manipulation.
        <li>Thread Manipulation/Querying.
	</ul>
*/
//=======================================================================
/*! \namespace GenericAPI
\brief General purpose elements.

Elements of this namespace are to be of general use by other high
level APIs.

 */
//=======================================================================
/*! \namespace Registry
\brief Object Registry
 */
