Tests configuration file
************************







The tests configuration file looks like a INI file. There is two required
sections : the Main section and the Data section. You'll need some additional
sections to define groups and tests. Note that some options found in a
configuration file may be overriden using the command line tools.

A sample configuration file is available in the *examples* directory of the
distribution. Another one is available in `sample/tester.ini`_. 


The Main section
````````````````
The main section defines how to run all tests together. It's a kind of
*runtests* configuration, supporting the following options :

+------------+-----+---------------------------------------------------------+
| name       | req | description                                             |
+============+=====+=========================================================+
| load       | no  | comma separated list of modules to preload, usually to  |
|            |     | register additional checkers, reporters...              |
+------------+-----+---------------------------------------------------------+
| threads    | no  | number of threads to use by runtests (default to 1)     |
+------------+-----+---------------------------------------------------------+
| verbosity  | no  | integer giving the execution verbosity (default to 0)   |
+------------+-----+---------------------------------------------------------+
| includes   | no  | A comma separated list of configuration file to include |
|            |     | path can be either absolute or relative and containts   |
|            |     | environment variable or home directory resolution.      |
|            |     | Value defined in the including file overwrite those in  |
|            |     | the included. The Value defined in the first elements   |
|            |     | of the list are overwrite by those in the last. Missing |
|            |     | file and recursive include are ignored                  |
+------------+-----+---------------------------------------------------------+
| includes   |     | Same as includes but all section define in the included |
| -test      | no  | file will be marked as text.                            |
|            |     |                                                         |
+------------+-----+---------------------------------------------------------+

The main section can contain any other variables which will be used as a base
for all test definitions. See below for more explanation.

Note that only test listed in the **tests** variable will be executed by 
default.


The Data section
````````````````
The data section describes how to store execution data. It has the following
options :

+------------+-----+--------------------------------------------------------+
| name       | req |   description                                          |
+============+=====+========================================================+
| location   | yes | output file or directory (depends on the mode)         |
+------------+-----+--------------------------------------------------------+
| mode       | no  | execution mode (daily, hourly, default to daily)       |
+------------+-----+--------------------------------------------------------+
| encoding   | no  | encoding to use in the data file (default to UTF-8)    |
+------------+-----+--------------------------------------------------------+

There should not be anything else in the data section (actually it would be
ignored). 


Group definitions
`````````````````
A group defines some variables common to a group of packages. Exemple groups are
Python_package and Zope_package for instance. A new group is defined by adding a
section with the name of the group and by adding group's variables in this
section.


Test definitions
````````````````
A test is defined by the following options :

+-----------------+-----+-----------------------------------------------------+
| name            | req |   description                                       |
+=================+=====+=====================================================+
| groups          | no  | comma separated list of groups this test belongs to |
+-----------------+-----+-----------------------------------------------------+
| repository      | yes | the repository type followed by the source          |
|                 |     | repository identifier (eg hg:/home/src/pylint/ )    |
+-----------------+-----+-----------------------------------------------------+
| path            | yes | the path of the package in the repository           |
+-----------------+-----+-----------------------------------------------------+
| is_test         | no  | Mark this section as a test. default to 0 except    |
|                 |     | for section includes throught includes-tests.       |
|                 |     | Can be set to 0 or 1 to force test execution.       |
+-----------------+-----+-----------------------------------------------------+
| tag             | no  | an optional tag used to retreive the package from   |
|                 |     | the repository                                      |
+-----------------+-----+-----------------------------------------------------+
| checks          | yes | comma separated list of checks to apply             |
+-----------------+-----+-----------------------------------------------------+
| dependencies    | no  | comma separated list of tests or package repository |
|                 |     | ids needed to be installed to run this test         |
+-----------------+-----+-----------------------------------------------------+
| max_cpu_time    | no  | The maximum CPU Time in second to be used by this   |
|                 |     | test.                                               |
+-----------------+-----+-----------------------------------------------------+
| max_time        | no  | The maximum Real Time in second this will run       |
|                 |     | test.                                               |
+-----------------+-----+-----------------------------------------------------+
| max_memory      | no  | The maximum Memory in bytes the test can allocate   |
+-----------------+-----+-----------------------------------------------------+
| max_retrieve    | no  | The delay in second while the test try to abort     |
|                 |     | nicely (reporting the error and cleaning up the     |
|                 |     | environement before it's killed. default to 60.     |
+-----------------+-----+-----------------------------------------------------+
| preprocessors   | no  | list to preprocessors to execute when this test is  |
|                 |     | used as a dependancy                                |
+-----------------+-----+-----------------------------------------------------+

All the limit value can be express with unit (s, min, h, d) or (KB, MB, GB)
This unit can be combined . eg : 3h 42min 56 second
space and coma are ignored 


additional option can be provide to pass usefull information to the report:
+----------------------+-----+------------------------------------------------+
| name                 | req |   description                                  |
+======================+=====+================================================+
| repository_view_url  | no  | The base URL to view repository file           |
+----------------------+-----+------------------------------------------------+
| repository_view_root | no  | the repository root in the url view            |
|                      |     | (if required)                                  |
+----------------------+-----+------------------------------------------------+
+
A test is built from some global, command lines, group and specific
variables. The best way to explain this is to explain the algorithm used :

1. get extras main variables

2. get groups variables, if the test belong to one ore more group

3. get test specific variables

4. extract test, checkers and preprocessors options and consider everything
   else as environment variables.

if the same variable is found in different places, the latest readen is kept.


Giving options to preprocessors and checkers
````````````````````````````````````````````
You can give options to preprocessors and checkers in the configuration file by
preceding the option name by the preprocessor or checker name. For instance, you
can give the treshold option to the python_lint checker by adding ::

  python_lint_treshold = 4

or you can tell you want to use the setup_install preprocessor before the
python_unittest check by adding ::

  setup_install_match_checks=python_unittest

Options specifics to preprocessors and checkers are described in their
respective section.


.. _`sample/tester.ini` : samples/tester.ini
