diff options
Diffstat (limited to 'share/cmake-3.17/Help/manual/ctest.1.rst')
| -rw-r--r-- | share/cmake-3.17/Help/manual/ctest.1.rst | 1579 |
1 files changed, 0 insertions, 1579 deletions
diff --git a/share/cmake-3.17/Help/manual/ctest.1.rst b/share/cmake-3.17/Help/manual/ctest.1.rst deleted file mode 100644 index 6503f0e..0000000 --- a/share/cmake-3.17/Help/manual/ctest.1.rst +++ /dev/null @@ -1,1579 +0,0 @@ -.. cmake-manual-description: CTest Command-Line Reference - -ctest(1) -******** - -.. contents:: - -Synopsis -======== - -.. parsed-literal:: - - ctest [<options>] - ctest --build-and-test <path-to-source> <path-to-build> - --build-generator <generator> [<options>...] - [--build-options <opts>...] [--test-command <command> [<args>...]] - ctest {-D <dashboard> | -M <model> -T <action> | -S <script> | -SP <script>} - [-- <dashboard-options>...] - -Description -=========== - -The **ctest** executable is the CMake test driver program. -CMake-generated build trees created for projects that use the -:command:`enable_testing` and :command:`add_test` commands have testing support. -This program will run the tests and report results. - -Options -======= - -``-C <cfg>, --build-config <cfg>`` - Choose configuration to test. - - Some CMake-generated build trees can have multiple build - configurations in the same tree. This option can be used to specify - which one should be tested. Example configurations are ``Debug`` and - ``Release``. - -``--progress`` - Enable short progress output from tests. - - When the output of **ctest** is being sent directly to a terminal, the - progress through the set of tests is reported by updating the same line - rather than printing start and end messages for each test on new lines. - This can significantly reduce the verbosity of the test output. - Test completion messages are still output on their own line for failed - tests and the final test summary will also still be logged. - - This option can also be enabled by setting the environment variable - :envvar:`CTEST_PROGRESS_OUTPUT`. - -``-V,--verbose`` - Enable verbose output from tests. - - Test output is normally suppressed and only summary information is - displayed. This option will show all test output. - -``-VV,--extra-verbose`` - Enable more verbose output from tests. - - Test output is normally suppressed and only summary information is - displayed. This option will show even more test output. - -``--debug`` - Displaying more verbose internals of CTest. - - This feature will result in a large number of output that is mostly - useful for debugging dashboard problems. - -``--output-on-failure`` - Output anything outputted by the test program if the test should fail. - This option can also be enabled by setting the - :envvar:`CTEST_OUTPUT_ON_FAILURE` environment variable - -``-F`` - Enable failover. - - This option allows CTest to resume a test set execution that was - previously interrupted. If no interruption occurred, the ``-F`` option - will have no effect. - -``-j <jobs>, --parallel <jobs>`` - Run the tests in parallel using the given number of jobs. - - This option tells CTest to run the tests in parallel using given - number of jobs. This option can also be set by setting the - :envvar:`CTEST_PARALLEL_LEVEL` environment variable. - - This option can be used with the :prop_test:`PROCESSORS` test property. - - See `Label and Subproject Summary`_. - -``--resource-spec-file <file>`` - Run CTest with :ref:`resource allocation <ctest-resource-allocation>` enabled, - using the - :ref:`resource specification file <ctest-resource-specification-file>` - specified in ``<file>``. - - When ``ctest`` is run as a `Dashboard Client`_ this sets the - ``ResourceSpecFile`` option of the `CTest Test Step`_. - -``--test-load <level>`` - While running tests in parallel (e.g. with ``-j``), try not to start - tests when they may cause the CPU load to pass above a given threshold. - - When ``ctest`` is run as a `Dashboard Client`_ this sets the - ``TestLoad`` option of the `CTest Test Step`_. - -``-Q,--quiet`` - Make CTest quiet. - - This option will suppress all the output. The output log file will - still be generated if the ``--output-log`` is specified. Options such - as ``--verbose``, ``--extra-verbose``, and ``--debug`` are ignored - if ``--quiet`` is specified. - -``-O <file>, --output-log <file>`` - Output to log file. - - This option tells CTest to write all its output to a ``<file>`` log file. - -``-N,--show-only[=<format>]`` - Disable actual execution of tests. - - This option tells CTest to list the tests that would be run but not - actually run them. Useful in conjunction with the ``-R`` and ``-E`` - options. - - ``<format>`` can be one of the following values. - - ``human`` - Human-friendly output. This is not guaranteed to be stable. - This is the default. - - ``json-v1`` - Dump the test information in JSON format. - See `Show as JSON Object Model`_. - -``-L <regex>, --label-regex <regex>`` - Run tests with labels matching regular expression. - - This option tells CTest to run only the tests whose labels match the - given regular expression. - -``-R <regex>, --tests-regex <regex>`` - Run tests matching regular expression. - - This option tells CTest to run only the tests whose names match the - given regular expression. - -``-E <regex>, --exclude-regex <regex>`` - Exclude tests matching regular expression. - - This option tells CTest to NOT run the tests whose names match the - given regular expression. - -``-LE <regex>, --label-exclude <regex>`` - Exclude tests with labels matching regular expression. - - This option tells CTest to NOT run the tests whose labels match the - given regular expression. - -``-FA <regex>, --fixture-exclude-any <regex>`` - Exclude fixtures matching ``<regex>`` from automatically adding any tests to - the test set. - - If a test in the set of tests to be executed requires a particular fixture, - that fixture's setup and cleanup tests would normally be added to the test set - automatically. This option prevents adding setup or cleanup tests for fixtures - matching the ``<regex>``. Note that all other fixture behavior is retained, - including test dependencies and skipping tests that have fixture setup tests - that fail. - -``-FS <regex>, --fixture-exclude-setup <regex>`` - Same as ``-FA`` except only matching setup tests are excluded. - -``-FC <regex>, --fixture-exclude-cleanup <regex>`` - Same as ``-FA`` except only matching cleanup tests are excluded. - -``-D <dashboard>, --dashboard <dashboard>`` - Execute dashboard test. - - This option tells CTest to act as a CDash client and perform a - dashboard test. All tests are ``<Mode><Test>``, where ``<Mode>`` can be - ``Experimental``, ``Nightly``, and ``Continuous``, and ``<Test>`` can be - ``Start``, ``Update``, ``Configure``, ``Build``, ``Test``, - ``Coverage``, and ``Submit``. - - See `Dashboard Client`_. - -``-D <var>:<type>=<value>`` - Define a variable for script mode. - - Pass in variable values on the command line. Use in conjunction - with ``-S`` to pass variable values to a dashboard script. Parsing ``-D`` - arguments as variable values is only attempted if the value - following ``-D`` does not match any of the known dashboard types. - -``-M <model>, --test-model <model>`` - Sets the model for a dashboard. - - This option tells CTest to act as a CDash client where the ``<model>`` - can be ``Experimental``, ``Nightly``, and ``Continuous``. - Combining ``-M`` and ``-T`` is similar to ``-D``. - - See `Dashboard Client`_. - -``-T <action>, --test-action <action>`` - Sets the dashboard action to perform. - - This option tells CTest to act as a CDash client and perform some - action such as ``start``, ``build``, ``test`` etc. See - `Dashboard Client Steps`_ for the full list of actions. - Combining ``-M`` and ``-T`` is similar to ``-D``. - - See `Dashboard Client`_. - -``-S <script>, --script <script>`` - Execute a dashboard for a configuration. - - This option tells CTest to load in a configuration script which sets - a number of parameters such as the binary and source directories. - Then CTest will do what is required to create and run a dashboard. - This option basically sets up a dashboard and then runs ``ctest -D`` - with the appropriate options. - - See `Dashboard Client`_. - -``-SP <script>, --script-new-process <script>`` - Execute a dashboard for a configuration. - - This option does the same operations as ``-S`` but it will do them in a - separate process. This is primarily useful in cases where the - script may modify the environment and you do not want the modified - environment to impact other ``-S`` scripts. - - See `Dashboard Client`_. - -``-I [Start,End,Stride,test#,test#|Test file], --tests-information`` - Run a specific number of tests by number. - - This option causes CTest to run tests starting at number ``Start``, - ending at number ``End``, and incrementing by ``Stride``. Any additional - numbers after ``Stride`` are considered individual test numbers. ``Start``, - ``End``, or ``Stride`` can be empty. Optionally a file can be given that - contains the same syntax as the command line. - -``-U, --union`` - Take the Union of ``-I`` and ``-R``. - - When both ``-R`` and ``-I`` are specified by default the intersection of - tests are run. By specifying ``-U`` the union of tests is run instead. - -``--rerun-failed`` - Run only the tests that failed previously. - - This option tells CTest to perform only the tests that failed during - its previous run. When this option is specified, CTest ignores all - other options intended to modify the list of tests to run (``-L``, ``-R``, - ``-E``, ``-LE``, ``-I``, etc). In the event that CTest runs and no tests - fail, subsequent calls to CTest with the ``--rerun-failed`` option will run - the set of tests that most recently failed (if any). - -``--repeat <mode>:<n>`` - Run tests repeatedly based on the given ``<mode>`` up to ``<n>`` times. - The modes are: - - ``until-fail`` - Require each test to run ``<n>`` times without failing in order to pass. - This is useful in finding sporadic failures in test cases. - - ``until-pass`` - Allow each test to run up to ``<n>`` times in order to pass. - Repeats tests if they fail for any reason. - This is useful in tolerating sporadic failures in test cases. - - ``after-timeout`` - Allow each test to run up to ``<n>`` times in order to pass. - Repeats tests only if they timeout. - This is useful in tolerating sporadic timeouts in test cases - on busy machines. - -``--repeat-until-fail <n>`` - Equivalent to ``--repeat until-fail:<n>``. - -``--max-width <width>`` - Set the max width for a test name to output. - - Set the maximum width for each test name to show in the output. - This allows the user to widen the output to avoid clipping the test - name which can be very annoying. - -``--interactive-debug-mode [0|1]`` - Set the interactive mode to ``0`` or ``1``. - - This option causes CTest to run tests in either an interactive mode - or a non-interactive mode. On Windows this means that in - non-interactive mode, all system debug pop up windows are blocked. - In dashboard mode (``Experimental``, ``Nightly``, ``Continuous``), the default - is non-interactive. When just running tests not for a dashboard the - default is to allow popups and interactive debugging. - -``--no-label-summary`` - Disable timing summary information for labels. - - This option tells CTest not to print summary information for each - label associated with the tests run. If there are no labels on the - tests, nothing extra is printed. - - See `Label and Subproject Summary`_. - -``--no-subproject-summary`` - Disable timing summary information for subprojects. - - This option tells CTest not to print summary information for each - subproject associated with the tests run. If there are no subprojects on the - tests, nothing extra is printed. - - See `Label and Subproject Summary`_. - -``--build-and-test`` -See `Build and Test Mode`_. - -``--test-output-size-passed <size>`` - Limit the output for passed tests to ``<size>`` bytes. - -``--test-output-size-failed <size>`` - Limit the output for failed tests to ``<size>`` bytes. - -``--overwrite`` - Overwrite CTest configuration option. - - By default CTest uses configuration options from configuration file. - This option will overwrite the configuration option. - -``--force-new-ctest-process`` - Run child CTest instances as new processes. - - By default CTest will run child CTest instances within the same - process. If this behavior is not desired, this argument will - enforce new processes for child CTest processes. - -``--schedule-random`` - Use a random order for scheduling tests. - - This option will run the tests in a random order. It is commonly - used to detect implicit dependencies in a test suite. - -``--submit-index`` - Legacy option for old Dart2 dashboard server feature. - Do not use. - -``--timeout <seconds>`` - Set the default test timeout. - - This option effectively sets a timeout on all tests that do not - already have a timeout set on them via the :prop_test:`TIMEOUT` - property. - -``--stop-time <time>`` - Set a time at which all tests should stop running. - - Set a real time of day at which all tests should timeout. Example: - ``7:00:00 -0400``. Any time format understood by the curl date parser - is accepted. Local time is assumed if no timezone is specified. - -``--print-labels`` - Print all available test labels. - - This option will not run any tests, it will simply print the list of - all labels associated with the test set. - -``--no-tests=<[error|ignore]>`` - Regard no tests found either as error or ignore it. - - If no tests were found, the default behavior of CTest is to always log an - error message but to return an error code in script mode only. This option - unifies the behavior of CTest by either returning an error code if no tests - were found or by ignoring it. - -.. include:: OPTIONS_HELP.txt - -.. _`Label and Subproject Summary`: - -Label and Subproject Summary -============================ - -CTest prints timing summary information for each ``LABEL`` and subproject -associated with the tests run. The label time summary will not include labels -that are mapped to subprojects. - -When the :prop_test:`PROCESSORS` test property is set, CTest will display a -weighted test timing result in label and subproject summaries. The time is -reported with `sec*proc` instead of just `sec`. - -The weighted time summary reported for each label or subproject ``j`` -is computed as:: - - Weighted Time Summary for Label/Subproject j = - sum(raw_test_time[j,i] * num_processors[j,i], i=1...num_tests[j]) - - for labels/subprojects j=1...total - -where: - -* ``raw_test_time[j,i]``: Wall-clock time for the ``i`` test - for the ``j`` label or subproject -* ``num_processors[j,i]``: Value of the CTest :prop_test:`PROCESSORS` property - for the ``i`` test for the ``j`` label or subproject -* ``num_tests[j]``: Number of tests associated with the ``j`` label or subproject -* ``total``: Total number of labels or subprojects that have at least one test run - -Therefore, the weighted time summary for each label or subproject represents -the amount of time that CTest gave to run the tests for each label or -subproject and gives a good representation of the total expense of the tests -for each label or subproject when compared to other labels or subprojects. - -For example, if ``SubprojectA`` showed ``100 sec*proc`` and ``SubprojectB`` showed -``10 sec*proc``, then CTest allocated approximately 10 times the CPU/core time -to run the tests for ``SubprojectA`` than for ``SubprojectB`` (e.g. so if effort -is going to be expended to reduce the cost of the test suite for the whole -project, then reducing the cost of the test suite for ``SubprojectA`` would -likely have a larger impact than effort to reduce the cost of the test suite -for ``SubprojectB``). - -.. _`Build and Test Mode`: - -Build and Test Mode -=================== - -CTest provides a command-line signature to configure (i.e. run cmake on), -build, and/or execute a test:: - - ctest --build-and-test <path-to-source> <path-to-build> - --build-generator <generator> - [<options>...] - [--build-options <opts>...] - [--test-command <command> [<args>...]] - -The configure and test steps are optional. The arguments to this command line -are the source and binary directories. The ``--build-generator`` option *must* -be provided to use ``--build-and-test``. If ``--test-command`` is specified -then that will be run after the build is complete. Other options that affect -this mode include: - -``--build-target`` - Specify a specific target to build. - - If left out the ``all`` target is built. - -``--build-nocmake`` - Run the build without running cmake first. - - Skip the cmake step. - -``--build-run-dir`` - Specify directory to run programs from. - - Directory where programs will be after it has been compiled. - -``--build-two-config`` - Run CMake twice. - -``--build-exe-dir`` - Specify the directory for the executable. - -``--build-generator`` - Specify the generator to use. See the :manual:`cmake-generators(7)` manual. - -``--build-generator-platform`` - Specify the generator-specific platform. - -``--build-generator-toolset`` - Specify the generator-specific toolset. - -``--build-project`` - Specify the name of the project to build. - -``--build-makeprogram`` - Specify the explicit make program to be used by CMake when configuring and - building the project. Only applicable for Make and Ninja based generators. - -``--build-noclean`` - Skip the make clean step. - -``--build-config-sample`` - A sample executable to use to determine the configuration that - should be used. e.g. ``Debug``, ``Release`` etc. - -``--build-options`` - Additional options for configuring the build (i.e. for CMake, not for - the build tool). Note that if this is specified, the ``--build-options`` - keyword and its arguments must be the last option given on the command - line, with the possible exception of ``--test-command``. - -``--test-command`` - The command to run as the test step with the ``--build-and-test`` option. - All arguments following this keyword will be assumed to be part of the - test command line, so it must be the last option given. - -``--test-timeout`` - The time limit in seconds - -.. _`Dashboard Client`: - -Dashboard Client -================ - -CTest can operate as a client for the `CDash`_ software quality dashboard -application. As a dashboard client, CTest performs a sequence of steps -to configure, build, and test software, and then submits the results to -a `CDash`_ server. The command-line signature used to submit to `CDash`_ is:: - - ctest (-D <dashboard> | -M <model> -T <action> | -S <script> | -SP <script>) - [-- <dashboard-options>...] - -Options for Dashboard Client include: - -``--group <group>`` - Specify what group you'd like to submit results to - - Submit dashboard to specified group instead of default one. By - default, the dashboard is submitted to Nightly, Experimental, or - Continuous group, but by specifying this option, the group can be - arbitrary. - - This replaces the deprecated option ``--track``. - Despite the name change its behavior is unchanged. - -``-A <file>, --add-notes <file>`` - Add a notes file with submission. - - This option tells CTest to include a notes file when submitting - dashboard. - -``--tomorrow-tag`` - ``Nightly`` or ``Experimental`` starts with next day tag. - - This is useful if the build will not finish in one day. - -``--extra-submit <file>[;<file>]`` - Submit extra files to the dashboard. - - This option will submit extra files to the dashboard. - -``--http1.0`` - Submit using `HTTP 1.0`. - - This option will force CTest to use `HTTP 1.0` to submit files to the - dashboard, instead of `HTTP 1.1`. - -``--no-compress-output`` - Do not compress test output when submitting. - - This flag will turn off automatic compression of test output. Use - this to maintain compatibility with an older version of CDash which - doesn't support compressed test output. - -Dashboard Client Steps ----------------------- - -CTest defines an ordered list of testing steps of which some or all may -be run as a dashboard client: - -``Start`` - Start a new dashboard submission to be composed of results recorded - by the following steps. - See the `CTest Start Step`_ section below. - -``Update`` - Update the source tree from its version control repository. - Record the old and new versions and the list of updated source files. - See the `CTest Update Step`_ section below. - -``Configure`` - Configure the software by running a command in the build tree. - Record the configuration output log. - See the `CTest Configure Step`_ section below. - -``Build`` - Build the software by running a command in the build tree. - Record the build output log and detect warnings and errors. - See the `CTest Build Step`_ section below. - -``Test`` - Test the software by loading a ``CTestTestfile.cmake`` - from the build tree and executing the defined tests. - Record the output and result of each test. - See the `CTest Test Step`_ section below. - -``Coverage`` - Compute coverage of the source code by running a coverage - analysis tool and recording its output. - See the `CTest Coverage Step`_ section below. - -``MemCheck`` - Run the software test suite through a memory check tool. - Record the test output, results, and issues reported by the tool. - See the `CTest MemCheck Step`_ section below. - -``Submit`` - Submit results recorded from other testing steps to the - software quality dashboard server. - See the `CTest Submit Step`_ section below. - -Dashboard Client Modes ----------------------- - -CTest defines three modes of operation as a dashboard client: - -``Nightly`` - This mode is intended to be invoked once per day, typically at night. - It enables the ``Start``, ``Update``, ``Configure``, ``Build``, ``Test``, - ``Coverage``, and ``Submit`` steps by default. Selected steps run even - if the ``Update`` step reports no changes to the source tree. - -``Continuous`` - This mode is intended to be invoked repeatedly throughout the day. - It enables the ``Start``, ``Update``, ``Configure``, ``Build``, ``Test``, - ``Coverage``, and ``Submit`` steps by default, but exits after the - ``Update`` step if it reports no changes to the source tree. - -``Experimental`` - This mode is intended to be invoked by a developer to test local changes. - It enables the ``Start``, ``Configure``, ``Build``, ``Test``, ``Coverage``, - and ``Submit`` steps by default. - -Dashboard Client via CTest Command-Line ---------------------------------------- - -CTest can perform testing on an already-generated build tree. -Run the ``ctest`` command with the current working directory set -to the build tree and use one of these signatures:: - - ctest -D <mode>[<step>] - ctest -M <mode> [ -T <step> ]... - -The ``<mode>`` must be one of the above `Dashboard Client Modes`_, -and each ``<step>`` must be one of the above `Dashboard Client Steps`_. - -CTest reads the `Dashboard Client Configuration`_ settings from -a file in the build tree called either ``CTestConfiguration.ini`` -or ``DartConfiguration.tcl`` (the names are historical). The format -of the file is:: - - # Lines starting in '#' are comments. - # Other non-blank lines are key-value pairs. - <setting>: <value> - -where ``<setting>`` is the setting name and ``<value>`` is the -setting value. - -In build trees generated by CMake, this configuration file is -generated by the :module:`CTest` module if included by the project. -The module uses variables to obtain a value for each setting -as documented with the settings below. - -.. _`CTest Script`: - -Dashboard Client via CTest Script ---------------------------------- - -CTest can perform testing driven by a :manual:`cmake-language(7)` -script that creates and maintains the source and build tree as -well as performing the testing steps. Run the ``ctest`` command -with the current working directory set outside of any build tree -and use one of these signatures:: - - ctest -S <script> - ctest -SP <script> - -The ``<script>`` file must call :ref:`CTest Commands` commands -to run testing steps explicitly as documented below. The commands -obtain `Dashboard Client Configuration`_ settings from their -arguments or from variables set in the script. - -Dashboard Client Configuration -============================== - -The `Dashboard Client Steps`_ may be configured by named -settings as documented in the following sections. - -.. _`CTest Start Step`: - -CTest Start Step ----------------- - -Start a new dashboard submission to be composed of results recorded -by the following steps. - -In a `CTest Script`_, the :command:`ctest_start` command runs this step. -Arguments to the command may specify some of the step settings. -The command first runs the command-line specified by the -``CTEST_CHECKOUT_COMMAND`` variable, if set, to initialize the source -directory. - -Configuration settings include: - -``BuildDirectory`` - The full path to the project build tree. - - * `CTest Script`_ variable: :variable:`CTEST_BINARY_DIRECTORY` - * :module:`CTest` module variable: :variable:`PROJECT_BINARY_DIR` - -``SourceDirectory`` - The full path to the project source tree. - - * `CTest Script`_ variable: :variable:`CTEST_SOURCE_DIRECTORY` - * :module:`CTest` module variable: :variable:`PROJECT_SOURCE_DIR` - -.. _`CTest Update Step`: - -CTest Update Step ------------------ - -In a `CTest Script`_, the :command:`ctest_update` command runs this step. -Arguments to the command may specify some of the step settings. - -Configuration settings to specify the version control tool include: - -``BZRCommand`` - ``bzr`` command-line tool to use if source tree is managed by Bazaar. - - * `CTest Script`_ variable: :variable:`CTEST_BZR_COMMAND` - * :module:`CTest` module variable: none - -``BZRUpdateOptions`` - Command-line options to the ``BZRCommand`` when updating the source. - - * `CTest Script`_ variable: :variable:`CTEST_BZR_UPDATE_OPTIONS` - * :module:`CTest` module variable: none - -``CVSCommand`` - ``cvs`` command-line tool to use if source tree is managed by CVS. - - * `CTest Script`_ variable: :variable:`CTEST_CVS_COMMAND` - * :module:`CTest` module variable: ``CVSCOMMAND`` - -``CVSUpdateOptions`` - Command-line options to the ``CVSCommand`` when updating the source. - - * `CTest Script`_ variable: :variable:`CTEST_CVS_UPDATE_OPTIONS` - * :module:`CTest` module variable: ``CVS_UPDATE_OPTIONS`` - -``GITCommand`` - ``git`` command-line tool to use if source tree is managed by Git. - - * `CTest Script`_ variable: :variable:`CTEST_GIT_COMMAND` - * :module:`CTest` module variable: ``GITCOMMAND`` - - The source tree is updated by ``git fetch`` followed by - ``git reset --hard`` to the ``FETCH_HEAD``. The result is the same - as ``git pull`` except that any local modifications are overwritten. - Use ``GITUpdateCustom`` to specify a different approach. - -``GITInitSubmodules`` - If set, CTest will update the repository's submodules before updating. - - * `CTest Script`_ variable: :variable:`CTEST_GIT_INIT_SUBMODULES` - * :module:`CTest` module variable: ``CTEST_GIT_INIT_SUBMODULES`` - -``GITUpdateCustom`` - Specify a custom command line (as a semicolon-separated list) to run - in the source tree (Git work tree) to update it instead of running - the ``GITCommand``. - - * `CTest Script`_ variable: :variable:`CTEST_GIT_UPDATE_CUSTOM` - * :module:`CTest` module variable: ``CTEST_GIT_UPDATE_CUSTOM`` - -``GITUpdateOptions`` - Command-line options to the ``GITCommand`` when updating the source. - - * `CTest Script`_ variable: :variable:`CTEST_GIT_UPDATE_OPTIONS` - * :module:`CTest` module variable: ``GIT_UPDATE_OPTIONS`` - -``HGCommand`` - ``hg`` command-line tool to use if source tree is managed by Mercurial. - - * `CTest Script`_ variable: :variable:`CTEST_HG_COMMAND` - * :module:`CTest` module variable: none - -``HGUpdateOptions`` - Command-line options to the ``HGCommand`` when updating the source. - - * `CTest Script`_ variable: :variable:`CTEST_HG_UPDATE_OPTIONS` - * :module:`CTest` module variable: none - -``P4Client`` - Value of the ``-c`` option to the ``P4Command``. - - * `CTest Script`_ variable: :variable:`CTEST_P4_CLIENT` - * :module:`CTest` module variable: ``CTEST_P4_CLIENT`` - -``P4Command`` - ``p4`` command-line tool to use if source tree is managed by Perforce. - - * `CTest Script`_ variable: :variable:`CTEST_P4_COMMAND` - * :module:`CTest` module variable: ``P4COMMAND`` - -``P4Options`` - Command-line options to the ``P4Command`` for all invocations. - - * `CTest Script`_ variable: :variable:`CTEST_P4_OPTIONS` - * :module:`CTest` module variable: ``CTEST_P4_OPTIONS`` - -``P4UpdateCustom`` - Specify a custom command line (as a semicolon-separated list) to run - in the source tree (Perforce tree) to update it instead of running - the ``P4Command``. - - * `CTest Script`_ variable: none - * :module:`CTest` module variable: ``CTEST_P4_UPDATE_CUSTOM`` - -``P4UpdateOptions`` - Command-line options to the ``P4Command`` when updating the source. - - * `CTest Script`_ variable: :variable:`CTEST_P4_UPDATE_OPTIONS` - * :module:`CTest` module variable: ``CTEST_P4_UPDATE_OPTIONS`` - -``SVNCommand`` - ``svn`` command-line tool to use if source tree is managed by Subversion. - - * `CTest Script`_ variable: :variable:`CTEST_SVN_COMMAND` - * :module:`CTest` module variable: ``SVNCOMMAND`` - -``SVNOptions`` - Command-line options to the ``SVNCommand`` for all invocations. - - * `CTest Script`_ variable: :variable:`CTEST_SVN_OPTIONS` - * :module:`CTest` module variable: ``CTEST_SVN_OPTIONS`` - -``SVNUpdateOptions`` - Command-line options to the ``SVNCommand`` when updating the source. - - * `CTest Script`_ variable: :variable:`CTEST_SVN_UPDATE_OPTIONS` - * :module:`CTest` module variable: ``SVN_UPDATE_OPTIONS`` - -``UpdateCommand`` - Specify the version-control command-line tool to use without - detecting the VCS that manages the source tree. - - * `CTest Script`_ variable: :variable:`CTEST_UPDATE_COMMAND` - * :module:`CTest` module variable: ``<VCS>COMMAND`` - when ``UPDATE_TYPE`` is ``<vcs>``, else ``UPDATE_COMMAND`` - -``UpdateOptions`` - Command-line options to the ``UpdateCommand``. - - * `CTest Script`_ variable: :variable:`CTEST_UPDATE_OPTIONS` - * :module:`CTest` module variable: ``<VCS>_UPDATE_OPTIONS`` - when ``UPDATE_TYPE`` is ``<vcs>``, else ``UPDATE_OPTIONS`` - -``UpdateType`` - Specify the version-control system that manages the source - tree if it cannot be detected automatically. - The value may be ``bzr``, ``cvs``, ``git``, ``hg``, - ``p4``, or ``svn``. - - * `CTest Script`_ variable: none, detected from source tree - * :module:`CTest` module variable: ``UPDATE_TYPE`` if set, - else ``CTEST_UPDATE_TYPE`` - -.. _`UpdateVersionOnly`: - -``UpdateVersionOnly`` - Specify that you want the version control update command to only - discover the current version that is checked out, and not to update - to a different version. - - * `CTest Script`_ variable: :variable:`CTEST_UPDATE_VERSION_ONLY` - -.. _`UpdateVersionOverride`: - -``UpdateVersionOverride`` - Specify the current version of your source tree. - - When this variable is set to a non-empty string, CTest will report the value - you specified rather than using the update command to discover the current - version that is checked out. Use of this variable supersedes - ``UpdateVersionOnly``. Like ``UpdateVersionOnly``, using this variable tells - CTest not to update the source tree to a different version. - - * `CTest Script`_ variable: :variable:`CTEST_UPDATE_VERSION_OVERRIDE` - -Additional configuration settings include: - -``NightlyStartTime`` - In the ``Nightly`` dashboard mode, specify the "nightly start time". - With centralized version control systems (``cvs`` and ``svn``), - the ``Update`` step checks out the version of the software as of - this time so that multiple clients choose a common version to test. - This is not well-defined in distributed version-control systems so - the setting is ignored. - - * `CTest Script`_ variable: :variable:`CTEST_NIGHTLY_START_TIME` - * :module:`CTest` module variable: ``NIGHTLY_START_TIME`` if set, - else ``CTEST_NIGHTLY_START_TIME`` - -.. _`CTest Configure Step`: - -CTest Configure Step --------------------- - -In a `CTest Script`_, the :command:`ctest_configure` command runs this step. -Arguments to the command may specify some of the step settings. - -Configuration settings include: - -``ConfigureCommand`` - Command-line to launch the software configuration process. - It will be executed in the location specified by the - ``BuildDirectory`` setting. - - * `CTest Script`_ variable: :variable:`CTEST_CONFIGURE_COMMAND` - * :module:`CTest` module variable: :variable:`CMAKE_COMMAND` - followed by :variable:`PROJECT_SOURCE_DIR` - -``LabelsForSubprojects`` - Specify a semicolon-separated list of labels that will be treated as - subprojects. This mapping will be passed on to CDash when configure, test or - build results are submitted. - - * `CTest Script`_ variable: :variable:`CTEST_LABELS_FOR_SUBPROJECTS` - * :module:`CTest` module variable: ``CTEST_LABELS_FOR_SUBPROJECTS`` - - See `Label and Subproject Summary`_. - -.. _`CTest Build Step`: - -CTest Build Step ----------------- - -In a `CTest Script`_, the :command:`ctest_build` command runs this step. -Arguments to the command may specify some of the step settings. - -Configuration settings include: - -``DefaultCTestConfigurationType`` - When the build system to be launched allows build-time selection - of the configuration (e.g. ``Debug``, ``Release``), this specifies - the default configuration to be built when no ``-C`` option is - given to the ``ctest`` command. The value will be substituted into - the value of ``MakeCommand`` to replace the literal string - ``${CTEST_CONFIGURATION_TYPE}`` if it appears. - - * `CTest Script`_ variable: :variable:`CTEST_CONFIGURATION_TYPE` - * :module:`CTest` module variable: ``DEFAULT_CTEST_CONFIGURATION_TYPE``, - initialized by the :envvar:`CMAKE_CONFIG_TYPE` environment variable - -``LabelsForSubprojects`` - Specify a semicolon-separated list of labels that will be treated as - subprojects. This mapping will be passed on to CDash when configure, test or - build results are submitted. - - * `CTest Script`_ variable: :variable:`CTEST_LABELS_FOR_SUBPROJECTS` - * :module:`CTest` module variable: ``CTEST_LABELS_FOR_SUBPROJECTS`` - - See `Label and Subproject Summary`_. - -``MakeCommand`` - Command-line to launch the software build process. - It will be executed in the location specified by the - ``BuildDirectory`` setting. - - * `CTest Script`_ variable: :variable:`CTEST_BUILD_COMMAND` - * :module:`CTest` module variable: ``MAKECOMMAND``, - initialized by the :command:`build_command` command - -``UseLaunchers`` - For build trees generated by CMake using one of the - :ref:`Makefile Generators` or the :generator:`Ninja` - generator, specify whether the - ``CTEST_USE_LAUNCHERS`` feature is enabled by the - :module:`CTestUseLaunchers` module (also included by the - :module:`CTest` module). When enabled, the generated build - system wraps each invocation of the compiler, linker, or - custom command line with a "launcher" that communicates - with CTest via environment variables and files to report - granular build warning and error information. Otherwise, - CTest must "scrape" the build output log for diagnostics. - - * `CTest Script`_ variable: :variable:`CTEST_USE_LAUNCHERS` - * :module:`CTest` module variable: ``CTEST_USE_LAUNCHERS`` - -.. _`CTest Test Step`: - -CTest Test Step ---------------- - -In a `CTest Script`_, the :command:`ctest_test` command runs this step. -Arguments to the command may specify some of the step settings. - -Configuration settings include: - -``ResourceSpecFile`` - Specify a - :ref:`resource specification file <ctest-resource-specification-file>`. See - :ref:`ctest-resource-allocation` for more information. - -``LabelsForSubprojects`` - Specify a semicolon-separated list of labels that will be treated as - subprojects. This mapping will be passed on to CDash when configure, test or - build results are submitted. - - * `CTest Script`_ variable: :variable:`CTEST_LABELS_FOR_SUBPROJECTS` - * :module:`CTest` module variable: ``CTEST_LABELS_FOR_SUBPROJECTS`` - - See `Label and Subproject Summary`_. - -``TestLoad`` - While running tests in parallel (e.g. with ``-j``), try not to start - tests when they may cause the CPU load to pass above a given threshold. - - * `CTest Script`_ variable: :variable:`CTEST_TEST_LOAD` - * :module:`CTest` module variable: ``CTEST_TEST_LOAD`` - -``TimeOut`` - The default timeout for each test if not specified by the - :prop_test:`TIMEOUT` test property. - - * `CTest Script`_ variable: :variable:`CTEST_TEST_TIMEOUT` - * :module:`CTest` module variable: ``DART_TESTING_TIMEOUT`` - -.. _`CTest Coverage Step`: - -CTest Coverage Step -------------------- - -In a `CTest Script`_, the :command:`ctest_coverage` command runs this step. -Arguments to the command may specify some of the step settings. - -Configuration settings include: - -``CoverageCommand`` - Command-line tool to perform software coverage analysis. - It will be executed in the location specified by the - ``BuildDirectory`` setting. - - * `CTest Script`_ variable: :variable:`CTEST_COVERAGE_COMMAND` - * :module:`CTest` module variable: ``COVERAGE_COMMAND`` - -``CoverageExtraFlags`` - Specify command-line options to the ``CoverageCommand`` tool. - - * `CTest Script`_ variable: :variable:`CTEST_COVERAGE_EXTRA_FLAGS` - * :module:`CTest` module variable: ``COVERAGE_EXTRA_FLAGS`` - - These options are the first arguments passed to ``CoverageCommand``. - -.. _`CTest MemCheck Step`: - -CTest MemCheck Step -------------------- - -In a `CTest Script`_, the :command:`ctest_memcheck` command runs this step. -Arguments to the command may specify some of the step settings. - -Configuration settings include: - -``MemoryCheckCommand`` - Command-line tool to perform dynamic analysis. Test command lines - will be launched through this tool. - - * `CTest Script`_ variable: :variable:`CTEST_MEMORYCHECK_COMMAND` - * :module:`CTest` module variable: ``MEMORYCHECK_COMMAND`` - -``MemoryCheckCommandOptions`` - Specify command-line options to the ``MemoryCheckCommand`` tool. - They will be placed prior to the test command line. - - * `CTest Script`_ variable: :variable:`CTEST_MEMORYCHECK_COMMAND_OPTIONS` - * :module:`CTest` module variable: ``MEMORYCHECK_COMMAND_OPTIONS`` - -``MemoryCheckType`` - Specify the type of memory checking to perform. - - * `CTest Script`_ variable: :variable:`CTEST_MEMORYCHECK_TYPE` - * :module:`CTest` module variable: ``MEMORYCHECK_TYPE`` - -``MemoryCheckSanitizerOptions`` - Specify options to sanitizers when running with a sanitize-enabled build. - - * `CTest Script`_ variable: :variable:`CTEST_MEMORYCHECK_SANITIZER_OPTIONS` - * :module:`CTest` module variable: ``MEMORYCHECK_SANITIZER_OPTIONS`` - -``MemoryCheckSuppressionFile`` - Specify a file containing suppression rules for the - ``MemoryCheckCommand`` tool. It will be passed with options - appropriate to the tool. - - * `CTest Script`_ variable: :variable:`CTEST_MEMORYCHECK_SUPPRESSIONS_FILE` - * :module:`CTest` module variable: ``MEMORYCHECK_SUPPRESSIONS_FILE`` - -Additional configuration settings include: - -``BoundsCheckerCommand`` - Specify a ``MemoryCheckCommand`` that is known to be command-line - compatible with Bounds Checker. - - * `CTest Script`_ variable: none - * :module:`CTest` module variable: none - -``PurifyCommand`` - Specify a ``MemoryCheckCommand`` that is known to be command-line - compatible with Purify. - - * `CTest Script`_ variable: none - * :module:`CTest` module variable: ``PURIFYCOMMAND`` - -``ValgrindCommand`` - Specify a ``MemoryCheckCommand`` that is known to be command-line - compatible with Valgrind. - - * `CTest Script`_ variable: none - * :module:`CTest` module variable: ``VALGRIND_COMMAND`` - -``ValgrindCommandOptions`` - Specify command-line options to the ``ValgrindCommand`` tool. - They will be placed prior to the test command line. - - * `CTest Script`_ variable: none - * :module:`CTest` module variable: ``VALGRIND_COMMAND_OPTIONS`` - -``DrMemoryCommand`` - Specify a ``MemoryCheckCommand`` that is known to be a command-line - compatible with DrMemory. - - * `CTest Script`_ variable: none - * :module:`CTest` module variable: ``DRMEMORY_COMMAND`` - -``DrMemoryCommandOptions`` - Specify command-line options to the ``DrMemoryCommand`` tool. - They will be placed prior to the test command line. - - * `CTest Script`_ variable: none - * :module:`CTest` module variable: ``DRMEMORY_COMMAND_OPTIONS`` - -.. _`CTest Submit Step`: - -CTest Submit Step ------------------ - -In a `CTest Script`_, the :command:`ctest_submit` command runs this step. -Arguments to the command may specify some of the step settings. - -Configuration settings include: - -``BuildName`` - Describe the dashboard client platform with a short string. - (Operating system, compiler, etc.) - - * `CTest Script`_ variable: :variable:`CTEST_BUILD_NAME` - * :module:`CTest` module variable: ``BUILDNAME`` - -``CDashVersion`` - Legacy option. Not used. - - * `CTest Script`_ variable: none, detected from server - * :module:`CTest` module variable: ``CTEST_CDASH_VERSION`` - -``CTestSubmitRetryCount`` - Specify a number of attempts to retry submission on network failure. - - * `CTest Script`_ variable: none, - use the :command:`ctest_submit` ``RETRY_COUNT`` option. - * :module:`CTest` module variable: ``CTEST_SUBMIT_RETRY_COUNT`` - -``CTestSubmitRetryDelay`` - Specify a delay before retrying submission on network failure. - - * `CTest Script`_ variable: none, - use the :command:`ctest_submit` ``RETRY_DELAY`` option. - * :module:`CTest` module variable: ``CTEST_SUBMIT_RETRY_DELAY`` - -``CurlOptions`` - Specify a semicolon-separated list of options to control the - Curl library that CTest uses internally to connect to the - server. Possible options are ``CURLOPT_SSL_VERIFYPEER_OFF`` - and ``CURLOPT_SSL_VERIFYHOST_OFF``. - - * `CTest Script`_ variable: :variable:`CTEST_CURL_OPTIONS` - * :module:`CTest` module variable: ``CTEST_CURL_OPTIONS`` - -``DropLocation`` - Legacy option. When ``SubmitURL`` is not set, it is constructed from - ``DropMethod``, ``DropSiteUser``, ``DropSitePassword``, ``DropSite``, and - ``DropLocation``. - - * `CTest Script`_ variable: :variable:`CTEST_DROP_LOCATION` - * :module:`CTest` module variable: ``DROP_LOCATION`` if set, - else ``CTEST_DROP_LOCATION`` - -``DropMethod`` - Legacy option. When ``SubmitURL`` is not set, it is constructed from - ``DropMethod``, ``DropSiteUser``, ``DropSitePassword``, ``DropSite``, and - ``DropLocation``. - - * `CTest Script`_ variable: :variable:`CTEST_DROP_METHOD` - * :module:`CTest` module variable: ``DROP_METHOD`` if set, - else ``CTEST_DROP_METHOD`` - -``DropSite`` - Legacy option. When ``SubmitURL`` is not set, it is constructed from - ``DropMethod``, ``DropSiteUser``, ``DropSitePassword``, ``DropSite``, and - ``DropLocation``. - - * `CTest Script`_ variable: :variable:`CTEST_DROP_SITE` - * :module:`CTest` module variable: ``DROP_SITE`` if set, - else ``CTEST_DROP_SITE`` - -``DropSitePassword`` - Legacy option. When ``SubmitURL`` is not set, it is constructed from - ``DropMethod``, ``DropSiteUser``, ``DropSitePassword``, ``DropSite``, and - ``DropLocation``. - - * `CTest Script`_ variable: :variable:`CTEST_DROP_SITE_PASSWORD` - * :module:`CTest` module variable: ``DROP_SITE_PASSWORD`` if set, - else ``CTEST_DROP_SITE_PASWORD`` - -``DropSiteUser`` - Legacy option. When ``SubmitURL`` is not set, it is constructed from - ``DropMethod``, ``DropSiteUser``, ``DropSitePassword``, ``DropSite``, and - ``DropLocation``. - - * `CTest Script`_ variable: :variable:`CTEST_DROP_SITE_USER` - * :module:`CTest` module variable: ``DROP_SITE_USER`` if set, - else ``CTEST_DROP_SITE_USER`` - -``IsCDash`` - Legacy option. Not used. - - * `CTest Script`_ variable: :variable:`CTEST_DROP_SITE_CDASH` - * :module:`CTest` module variable: ``CTEST_DROP_SITE_CDASH`` - -``ScpCommand`` - Legacy option. Not used. - - * `CTest Script`_ variable: :variable:`CTEST_SCP_COMMAND` - * :module:`CTest` module variable: ``SCPCOMMAND`` - -``Site`` - Describe the dashboard client host site with a short string. - (Hostname, domain, etc.) - - * `CTest Script`_ variable: :variable:`CTEST_SITE` - * :module:`CTest` module variable: ``SITE``, - initialized by the :command:`site_name` command - -``SubmitURL`` - The ``http`` or ``https`` URL of the dashboard server to send the submission - to. - - * `CTest Script`_ variable: :variable:`CTEST_SUBMIT_URL` - * :module:`CTest` module variable: ``SUBMIT_URL`` if set, - else ``CTEST_SUBMIT_URL`` - -``TriggerSite`` - Legacy option. Not used. - - * `CTest Script`_ variable: :variable:`CTEST_TRIGGER_SITE` - * :module:`CTest` module variable: ``TRIGGER_SITE`` if set, - else ``CTEST_TRIGGER_SITE`` - -.. _`Show as JSON Object Model`: - -Show as JSON Object Model -========================= - -When the ``--show-only=json-v1`` command line option is given, the test -information is output in JSON format. Version 1.0 of the JSON object -model is defined as follows: - -``kind`` - The string "ctestInfo". - -``version`` - A JSON object specifying the version components. Its members are - - ``major`` - A non-negative integer specifying the major version component. - ``minor`` - A non-negative integer specifying the minor version component. - -``backtraceGraph`` - JSON object representing backtrace information with the - following members: - - ``commands`` - List of command names. - ``files`` - List of file names. - ``nodes`` - List of node JSON objects with members: - - ``command`` - Index into the ``commands`` member of the ``backtraceGraph``. - ``file`` - Index into the ``files`` member of the ``backtraceGraph``. - ``line`` - Line number in the file where the backtrace was added. - ``parent`` - Index into the ``nodes`` member of the ``backtraceGraph`` - representing the parent in the graph. - -``tests`` - A JSON array listing information about each test. Each entry - is a JSON object with members: - - ``name`` - Test name. - ``config`` - Configuration that the test can run on. - Empty string means any config. - ``command`` - List where the first element is the test command and the - remaining elements are the command arguments. - ``backtrace`` - Index into the ``nodes`` member of the ``backtraceGraph``. - ``properties`` - Test properties. - Can contain keys for each of the supported test properties. - -.. _`ctest-resource-allocation`: - -Resource Allocation -=================== - -CTest provides a mechanism for tests to specify the resources that they need -in a fine-grained way, and for users to specify the resources availiable on -the running machine. This allows CTest to internally keep track of which -resources are in use and which are free, scheduling tests in a way that -prevents them from trying to claim resources that are not available. - -When the resource allocation feature is used, CTest will not oversubscribe -resources. For example, if a resource has 8 slots, CTest will not run tests -that collectively use more than 8 slots at a time. This has the effect of -limiting how many tests can run at any given time, even if a high ``-j`` -argument is used, if those tests all use some slots from the same resource. -In addition, it means that a single test that uses more of a resource than is -available on a machine will not run at all (and will be reported as -``Not Run``). - -A common use case for this feature is for tests that require the use of a GPU. -Multiple tests can simultaneously allocate memory from a GPU, but if too many -tests try to do this at once, some of them will fail to allocate, resulting in -a failed test, even though the test would have succeeded if it had the memory -it needed. By using the resource allocation feature, each test can specify how -much memory it requires from a GPU, allowing CTest to schedule tests in a way -that running several of these tests at once does not exhaust the GPU's memory -pool. - -Please note that CTest has no concept of what a GPU is or how much memory it -has, nor does it have any way of communicating with a GPU to retrieve this -information or perform any memory management. CTest simply keeps track of a -list of abstract resource types, each of which has a certain number of slots -available for tests to use. Each test specifies the number of slots that it -requires from a certain resource, and CTest then schedules them in a way that -prevents the total number of slots in use from exceeding the listed capacity. -When a test is executed, and slots from a resource are allocated to that test, -tests may assume that they have exclusive use of those slots for the duration -of the test's process. - -The CTest resource allocation feature consists of two inputs: - -* The :ref:`resource specification file <ctest-resource-specification-file>`, - described below, which describes the resources available on the system. -* The :prop_test:`RESOURCE_GROUPS` property of tests, which describes the - resources required by the test. - -When CTest runs a test, the resources allocated to that test are passed in the -form of a set of -:ref:`environment variables <ctest-resource-environment-variables>` as -described below. Using this information to decide which resource to connect to -is left to the test writer. - -The ``RESOURCE_GROUPS`` property tells CTest what resources a test expects -to use grouped in a way meaningful to the test. The test itself must read -the :ref:`environment variables <ctest-resource-environment-variables>` to -determine which resources have been allocated to each group. For example, -each group may correspond to a process the test will spawn when executed. - -Note that even if a test specifies a ``RESOURCE_GROUPS`` property, it is still -possible for that to test to run without any resource allocation (and without -the corresponding -:ref:`environment variables <ctest-resource-environment-variables>`) -if the user does not pass a resource specification file. Passing this file, -either through the ``--resource-spec-file`` command-line argument or the -``RESOURCE_SPEC_FILE`` argument to :command:`ctest_test`, is what activates the -resource allocation feature. Tests should check the -``CTEST_RESOURCE_GROUP_COUNT`` environment variable to find out whether or not -resource allocation is activated. This variable will always (and only) be -defined if resource allocation is activated. If resource allocation is not -activated, then the ``CTEST_RESOURCE_GROUP_COUNT`` variable will not exist, -even if it exists for the parent ``ctest`` process. If a test absolutely must -have resource allocation, then it can return a failing exit code or use the -:prop_test:`SKIP_RETURN_CODE` or :prop_test:`SKIP_REGULAR_EXPRESSION` -properties to indicate a skipped test. - -.. _`ctest-resource-specification-file`: - -Resource Specification File ---------------------------- - -The resource specification file is a JSON file which is passed to CTest, either -on the :manual:`ctest(1)` command line as ``--resource-spec-file``, or as the -``RESOURCE_SPEC_FILE`` argument of :command:`ctest_test`. The resource -specification file must be a JSON object. All examples in this document assume -the following resource specification file: - -.. code-block:: json - - { - "version": { - "major": 1, - "minor": 0 - }, - "local": [ - { - "gpus": [ - { - "id": "0", - "slots": 2 - }, - { - "id": "1", - "slots": 4 - }, - { - "id": "2", - "slots": 2 - }, - { - "id": "3" - } - ], - "crypto_chips": [ - { - "id": "card0", - "slots": 4 - } - ] - } - ] - } - -The members are: - -``version`` - An object containing a ``major`` integer field and a ``minor`` integer field. - Currently, the only supported version is major ``1``, minor ``0``. Any other - value is an error. - -``local`` - A JSON array of resource sets present on the system. Currently, this array - is restricted to being of size 1. - - Each array element is a JSON object with members whose names are equal to the - desired resource types, such as ``gpus``. These names must start with a - lowercase letter or an underscore, and subsequent characters can be a - lowercase letter, a digit, or an underscore. Uppercase letters are not - allowed, because certain platforms have case-insensitive environment - variables. See the `Environment Variables`_ section below for - more information. It is recommended that the resource type name be the plural - of a noun, such as ``gpus`` or ``crypto_chips`` (and not ``gpu`` or - ``crypto_chip``.) - - Please note that the names ``gpus`` and ``crypto_chips`` are just examples, - and CTest does not interpret them in any way. You are free to make up any - resource type you want to meet your own requirements. - - The value for each resource type is a JSON array consisting of JSON objects, - each of which describe a specific instance of the specified resource. These - objects have the following members: - - ``id`` - A string consisting of an identifier for the resource. Each character in - the identifier can be a lowercase letter, a digit, or an underscore. - Uppercase letters are not allowed. - - Identifiers must be unique within a resource type. However, they do not - have to be unique across resource types. For example, it is valid to have a - ``gpus`` resource named ``0`` and a ``crypto_chips`` resource named ``0``, - but not two ``gpus`` resources both named ``0``. - - Please note that the IDs ``0``, ``1``, ``2``, ``3``, and ``card0`` are just - examples, and CTest does not interpret them in any way. You are free to - make up any IDs you want to meet your own requirements. - - ``slots`` - An optional unsigned number specifying the number of slots available on the - resource. For example, this could be megabytes of RAM on a GPU, or - cryptography units available on a cryptography chip. If ``slots`` is not - specified, a default value of ``1`` is assumed. - -In the example file above, there are four GPUs with ID's 0 through 3. GPU 0 has -2 slots, GPU 1 has 4, GPU 2 has 2, and GPU 3 has a default of 1 slot. There is -also one cryptography chip with 4 slots. - -``RESOURCE_GROUPS`` Property ----------------------------- - -See :prop_test:`RESOURCE_GROUPS` for a description of this property. - -.. _`ctest-resource-environment-variables`: - -Environment Variables ---------------------- - -Once CTest has decided which resources to allocate to a test, it passes this -information to the test executable as a series of environment variables. For -each example below, we will assume that the test in question has a -:prop_test:`RESOURCE_GROUPS` property of -``2,gpus:2;gpus:4,gpus:1,crypto_chips:2``. - -The following variables are passed to the test process: - -.. envvar:: CTEST_RESOURCE_GROUP_COUNT - - The total number of groups specified by the :prop_test:`RESOURCE_GROUPS` - property. For example: - - * ``CTEST_RESOURCE_GROUP_COUNT=3`` - - This variable will only be defined if :manual:`ctest(1)` has been given a - ``--resource-spec-file``, or if :command:`ctest_test` has been given a - ``RESOURCE_SPEC_FILE``. If no resource specification file has been given, - this variable will not be defined. - -.. envvar:: CTEST_RESOURCE_GROUP_<num> - - The list of resource types allocated to each group, with each item - separated by a comma. ``<num>`` is a number from zero to - ``CTEST_RESOURCE_GROUP_COUNT`` minus one. ``CTEST_RESOURCE_GROUP_<num>`` - is defined for each ``<num>`` in this range. For example: - - * ``CTEST_RESOURCE_GROUP_0=gpus`` - * ``CTEST_RESOURCE_GROUP_1=gpus`` - * ``CTEST_RESOURCE_GROUP_2=crypto_chips,gpus`` - -.. envvar:: CTEST_RESOURCE_GROUP_<num>_<resource-type> - - The list of resource IDs and number of slots from each ID allocated to each - group for a given resource type. This variable consists of a series of - pairs, each pair separated by a semicolon, and with the two items in the pair - separated by a comma. The first item in each pair is ``id:`` followed by the - ID of a resource of type ``<resource-type>``, and the second item is - ``slots:`` followed by the number of slots from that resource allocated to - the given group. For example: - - * ``CTEST_RESOURCE_GROUP_0_GPUS=id:0,slots:2`` - * ``CTEST_RESOURCE_GROUP_1_GPUS=id:2,slots:2`` - * ``CTEST_RESOURCE_GROUP_2_GPUS=id:1,slots:4;id:3,slots:1`` - * ``CTEST_RESOURCE_GROUP_2_CRYPTO_CHIPS=id:card0,slots:2`` - - In this example, group 0 gets 2 slots from GPU ``0``, group 1 gets 2 slots - from GPU ``2``, and group 2 gets 4 slots from GPU ``1``, 1 slot from GPU - ``3``, and 2 slots from cryptography chip ``card0``. - - ``<num>`` is a number from zero to ``CTEST_RESOURCE_GROUP_COUNT`` minus one. - ``<resource-type>`` is the name of a resource type, converted to uppercase. - ``CTEST_RESOURCE_GROUP_<num>_<resource-type>`` is defined for the product - of each ``<num>`` in the range listed above and each resource type listed in - ``CTEST_RESOURCE_GROUP_<num>``. - - Because some platforms have case-insensitive names for environment variables, - the names of resource types may not clash in a case-insensitive environment. - Because of this, for the sake of simplicity, all resource types must be - listed in all lowercase in the - :ref:`resource specification file <ctest-resource-specification-file>` and - in the :prop_test:`RESOURCE_GROUPS` property, and they are converted to all - uppercase in the ``CTEST_RESOURCE_GROUP_<num>_<resource-type>`` environment - variable. - -See Also -======== - -.. include:: LINKS.txt - -.. _`CDash`: http://cdash.org/ |