summaryrefslogtreecommitdiff
path: root/share/cmake-3.17/Help/manual/ctest.1.rst
diff options
context:
space:
mode:
Diffstat (limited to 'share/cmake-3.17/Help/manual/ctest.1.rst')
-rw-r--r--share/cmake-3.17/Help/manual/ctest.1.rst1579
1 files changed, 1579 insertions, 0 deletions
diff --git a/share/cmake-3.17/Help/manual/ctest.1.rst b/share/cmake-3.17/Help/manual/ctest.1.rst
new file mode 100644
index 0000000..6503f0e
--- /dev/null
+++ b/share/cmake-3.17/Help/manual/ctest.1.rst
@@ -0,0 +1,1579 @@
+.. cmake-manual-description: CTest Command-Line Reference
+
+ctest(1)
+********
+
+.. contents::
+
+Synopsis
+========
+
+.. parsed-literal::
+
+ ctest [<options>]
+ ctest --build-and-test <path-to-source> <path-to-build>
+ --build-generator <generator> [<options>...]
+ [--build-options <opts>...] [--test-command <command> [<args>...]]
+ ctest {-D <dashboard> | -M <model> -T <action> | -S <script> | -SP <script>}
+ [-- <dashboard-options>...]
+
+Description
+===========
+
+The **ctest** executable is the CMake test driver program.
+CMake-generated build trees created for projects that use the
+:command:`enable_testing` and :command:`add_test` commands have testing support.
+This program will run the tests and report results.
+
+Options
+=======
+
+``-C <cfg>, --build-config <cfg>``
+ Choose configuration to test.
+
+ Some CMake-generated build trees can have multiple build
+ configurations in the same tree. This option can be used to specify
+ which one should be tested. Example configurations are ``Debug`` and
+ ``Release``.
+
+``--progress``
+ Enable short progress output from tests.
+
+ When the output of **ctest** is being sent directly to a terminal, the
+ progress through the set of tests is reported by updating the same line
+ rather than printing start and end messages for each test on new lines.
+ This can significantly reduce the verbosity of the test output.
+ Test completion messages are still output on their own line for failed
+ tests and the final test summary will also still be logged.
+
+ This option can also be enabled by setting the environment variable
+ :envvar:`CTEST_PROGRESS_OUTPUT`.
+
+``-V,--verbose``
+ Enable verbose output from tests.
+
+ Test output is normally suppressed and only summary information is
+ displayed. This option will show all test output.
+
+``-VV,--extra-verbose``
+ Enable more verbose output from tests.
+
+ Test output is normally suppressed and only summary information is
+ displayed. This option will show even more test output.
+
+``--debug``
+ Displaying more verbose internals of CTest.
+
+ This feature will result in a large number of output that is mostly
+ useful for debugging dashboard problems.
+
+``--output-on-failure``
+ Output anything outputted by the test program if the test should fail.
+ This option can also be enabled by setting the
+ :envvar:`CTEST_OUTPUT_ON_FAILURE` environment variable
+
+``-F``
+ Enable failover.
+
+ This option allows CTest to resume a test set execution that was
+ previously interrupted. If no interruption occurred, the ``-F`` option
+ will have no effect.
+
+``-j <jobs>, --parallel <jobs>``
+ Run the tests in parallel using the given number of jobs.
+
+ This option tells CTest to run the tests in parallel using given
+ number of jobs. This option can also be set by setting the
+ :envvar:`CTEST_PARALLEL_LEVEL` environment variable.
+
+ This option can be used with the :prop_test:`PROCESSORS` test property.
+
+ See `Label and Subproject Summary`_.
+
+``--resource-spec-file <file>``
+ Run CTest with :ref:`resource allocation <ctest-resource-allocation>` enabled,
+ using the
+ :ref:`resource specification file <ctest-resource-specification-file>`
+ specified in ``<file>``.
+
+ When ``ctest`` is run as a `Dashboard Client`_ this sets the
+ ``ResourceSpecFile`` option of the `CTest Test Step`_.
+
+``--test-load <level>``
+ While running tests in parallel (e.g. with ``-j``), try not to start
+ tests when they may cause the CPU load to pass above a given threshold.
+
+ When ``ctest`` is run as a `Dashboard Client`_ this sets the
+ ``TestLoad`` option of the `CTest Test Step`_.
+
+``-Q,--quiet``
+ Make CTest quiet.
+
+ This option will suppress all the output. The output log file will
+ still be generated if the ``--output-log`` is specified. Options such
+ as ``--verbose``, ``--extra-verbose``, and ``--debug`` are ignored
+ if ``--quiet`` is specified.
+
+``-O <file>, --output-log <file>``
+ Output to log file.
+
+ This option tells CTest to write all its output to a ``<file>`` log file.
+
+``-N,--show-only[=<format>]``
+ Disable actual execution of tests.
+
+ This option tells CTest to list the tests that would be run but not
+ actually run them. Useful in conjunction with the ``-R`` and ``-E``
+ options.
+
+ ``<format>`` can be one of the following values.
+
+ ``human``
+ Human-friendly output. This is not guaranteed to be stable.
+ This is the default.
+
+ ``json-v1``
+ Dump the test information in JSON format.
+ See `Show as JSON Object Model`_.
+
+``-L <regex>, --label-regex <regex>``
+ Run tests with labels matching regular expression.
+
+ This option tells CTest to run only the tests whose labels match the
+ given regular expression.
+
+``-R <regex>, --tests-regex <regex>``
+ Run tests matching regular expression.
+
+ This option tells CTest to run only the tests whose names match the
+ given regular expression.
+
+``-E <regex>, --exclude-regex <regex>``
+ Exclude tests matching regular expression.
+
+ This option tells CTest to NOT run the tests whose names match the
+ given regular expression.
+
+``-LE <regex>, --label-exclude <regex>``
+ Exclude tests with labels matching regular expression.
+
+ This option tells CTest to NOT run the tests whose labels match the
+ given regular expression.
+
+``-FA <regex>, --fixture-exclude-any <regex>``
+ Exclude fixtures matching ``<regex>`` from automatically adding any tests to
+ the test set.
+
+ If a test in the set of tests to be executed requires a particular fixture,
+ that fixture's setup and cleanup tests would normally be added to the test set
+ automatically. This option prevents adding setup or cleanup tests for fixtures
+ matching the ``<regex>``. Note that all other fixture behavior is retained,
+ including test dependencies and skipping tests that have fixture setup tests
+ that fail.
+
+``-FS <regex>, --fixture-exclude-setup <regex>``
+ Same as ``-FA`` except only matching setup tests are excluded.
+
+``-FC <regex>, --fixture-exclude-cleanup <regex>``
+ Same as ``-FA`` except only matching cleanup tests are excluded.
+
+``-D <dashboard>, --dashboard <dashboard>``
+ Execute dashboard test.
+
+ This option tells CTest to act as a CDash client and perform a
+ dashboard test. All tests are ``<Mode><Test>``, where ``<Mode>`` can be
+ ``Experimental``, ``Nightly``, and ``Continuous``, and ``<Test>`` can be
+ ``Start``, ``Update``, ``Configure``, ``Build``, ``Test``,
+ ``Coverage``, and ``Submit``.
+
+ See `Dashboard Client`_.
+
+``-D <var>:<type>=<value>``
+ Define a variable for script mode.
+
+ Pass in variable values on the command line. Use in conjunction
+ with ``-S`` to pass variable values to a dashboard script. Parsing ``-D``
+ arguments as variable values is only attempted if the value
+ following ``-D`` does not match any of the known dashboard types.
+
+``-M <model>, --test-model <model>``
+ Sets the model for a dashboard.
+
+ This option tells CTest to act as a CDash client where the ``<model>``
+ can be ``Experimental``, ``Nightly``, and ``Continuous``.
+ Combining ``-M`` and ``-T`` is similar to ``-D``.
+
+ See `Dashboard Client`_.
+
+``-T <action>, --test-action <action>``
+ Sets the dashboard action to perform.
+
+ This option tells CTest to act as a CDash client and perform some
+ action such as ``start``, ``build``, ``test`` etc. See
+ `Dashboard Client Steps`_ for the full list of actions.
+ Combining ``-M`` and ``-T`` is similar to ``-D``.
+
+ See `Dashboard Client`_.
+
+``-S <script>, --script <script>``
+ Execute a dashboard for a configuration.
+
+ This option tells CTest to load in a configuration script which sets
+ a number of parameters such as the binary and source directories.
+ Then CTest will do what is required to create and run a dashboard.
+ This option basically sets up a dashboard and then runs ``ctest -D``
+ with the appropriate options.
+
+ See `Dashboard Client`_.
+
+``-SP <script>, --script-new-process <script>``
+ Execute a dashboard for a configuration.
+
+ This option does the same operations as ``-S`` but it will do them in a
+ separate process. This is primarily useful in cases where the
+ script may modify the environment and you do not want the modified
+ environment to impact other ``-S`` scripts.
+
+ See `Dashboard Client`_.
+
+``-I [Start,End,Stride,test#,test#|Test file], --tests-information``
+ Run a specific number of tests by number.
+
+ This option causes CTest to run tests starting at number ``Start``,
+ ending at number ``End``, and incrementing by ``Stride``. Any additional
+ numbers after ``Stride`` are considered individual test numbers. ``Start``,
+ ``End``, or ``Stride`` can be empty. Optionally a file can be given that
+ contains the same syntax as the command line.
+
+``-U, --union``
+ Take the Union of ``-I`` and ``-R``.
+
+ When both ``-R`` and ``-I`` are specified by default the intersection of
+ tests are run. By specifying ``-U`` the union of tests is run instead.
+
+``--rerun-failed``
+ Run only the tests that failed previously.
+
+ This option tells CTest to perform only the tests that failed during
+ its previous run. When this option is specified, CTest ignores all
+ other options intended to modify the list of tests to run (``-L``, ``-R``,
+ ``-E``, ``-LE``, ``-I``, etc). In the event that CTest runs and no tests
+ fail, subsequent calls to CTest with the ``--rerun-failed`` option will run
+ the set of tests that most recently failed (if any).
+
+``--repeat <mode>:<n>``
+ Run tests repeatedly based on the given ``<mode>`` up to ``<n>`` times.
+ The modes are:
+
+ ``until-fail``
+ Require each test to run ``<n>`` times without failing in order to pass.
+ This is useful in finding sporadic failures in test cases.
+
+ ``until-pass``
+ Allow each test to run up to ``<n>`` times in order to pass.
+ Repeats tests if they fail for any reason.
+ This is useful in tolerating sporadic failures in test cases.
+
+ ``after-timeout``
+ Allow each test to run up to ``<n>`` times in order to pass.
+ Repeats tests only if they timeout.
+ This is useful in tolerating sporadic timeouts in test cases
+ on busy machines.
+
+``--repeat-until-fail <n>``
+ Equivalent to ``--repeat until-fail:<n>``.
+
+``--max-width <width>``
+ Set the max width for a test name to output.
+
+ Set the maximum width for each test name to show in the output.
+ This allows the user to widen the output to avoid clipping the test
+ name which can be very annoying.
+
+``--interactive-debug-mode [0|1]``
+ Set the interactive mode to ``0`` or ``1``.
+
+ This option causes CTest to run tests in either an interactive mode
+ or a non-interactive mode. On Windows this means that in
+ non-interactive mode, all system debug pop up windows are blocked.
+ In dashboard mode (``Experimental``, ``Nightly``, ``Continuous``), the default
+ is non-interactive. When just running tests not for a dashboard the
+ default is to allow popups and interactive debugging.
+
+``--no-label-summary``
+ Disable timing summary information for labels.
+
+ This option tells CTest not to print summary information for each
+ label associated with the tests run. If there are no labels on the
+ tests, nothing extra is printed.
+
+ See `Label and Subproject Summary`_.
+
+``--no-subproject-summary``
+ Disable timing summary information for subprojects.
+
+ This option tells CTest not to print summary information for each
+ subproject associated with the tests run. If there are no subprojects on the
+ tests, nothing extra is printed.
+
+ See `Label and Subproject Summary`_.
+
+``--build-and-test``
+See `Build and Test Mode`_.
+
+``--test-output-size-passed <size>``
+ Limit the output for passed tests to ``<size>`` bytes.
+
+``--test-output-size-failed <size>``
+ Limit the output for failed tests to ``<size>`` bytes.
+
+``--overwrite``
+ Overwrite CTest configuration option.
+
+ By default CTest uses configuration options from configuration file.
+ This option will overwrite the configuration option.
+
+``--force-new-ctest-process``
+ Run child CTest instances as new processes.
+
+ By default CTest will run child CTest instances within the same
+ process. If this behavior is not desired, this argument will
+ enforce new processes for child CTest processes.
+
+``--schedule-random``
+ Use a random order for scheduling tests.
+
+ This option will run the tests in a random order. It is commonly
+ used to detect implicit dependencies in a test suite.
+
+``--submit-index``
+ Legacy option for old Dart2 dashboard server feature.
+ Do not use.
+
+``--timeout <seconds>``
+ Set the default test timeout.
+
+ This option effectively sets a timeout on all tests that do not
+ already have a timeout set on them via the :prop_test:`TIMEOUT`
+ property.
+
+``--stop-time <time>``
+ Set a time at which all tests should stop running.
+
+ Set a real time of day at which all tests should timeout. Example:
+ ``7:00:00 -0400``. Any time format understood by the curl date parser
+ is accepted. Local time is assumed if no timezone is specified.
+
+``--print-labels``
+ Print all available test labels.
+
+ This option will not run any tests, it will simply print the list of
+ all labels associated with the test set.
+
+``--no-tests=<[error|ignore]>``
+ Regard no tests found either as error or ignore it.
+
+ If no tests were found, the default behavior of CTest is to always log an
+ error message but to return an error code in script mode only. This option
+ unifies the behavior of CTest by either returning an error code if no tests
+ were found or by ignoring it.
+
+.. include:: OPTIONS_HELP.txt
+
+.. _`Label and Subproject Summary`:
+
+Label and Subproject Summary
+============================
+
+CTest prints timing summary information for each ``LABEL`` and subproject
+associated with the tests run. The label time summary will not include labels
+that are mapped to subprojects.
+
+When the :prop_test:`PROCESSORS` test property is set, CTest will display a
+weighted test timing result in label and subproject summaries. The time is
+reported with `sec*proc` instead of just `sec`.
+
+The weighted time summary reported for each label or subproject ``j``
+is computed as::
+
+ Weighted Time Summary for Label/Subproject j =
+ sum(raw_test_time[j,i] * num_processors[j,i], i=1...num_tests[j])
+
+ for labels/subprojects j=1...total
+
+where:
+
+* ``raw_test_time[j,i]``: Wall-clock time for the ``i`` test
+ for the ``j`` label or subproject
+* ``num_processors[j,i]``: Value of the CTest :prop_test:`PROCESSORS` property
+ for the ``i`` test for the ``j`` label or subproject
+* ``num_tests[j]``: Number of tests associated with the ``j`` label or subproject
+* ``total``: Total number of labels or subprojects that have at least one test run
+
+Therefore, the weighted time summary for each label or subproject represents
+the amount of time that CTest gave to run the tests for each label or
+subproject and gives a good representation of the total expense of the tests
+for each label or subproject when compared to other labels or subprojects.
+
+For example, if ``SubprojectA`` showed ``100 sec*proc`` and ``SubprojectB`` showed
+``10 sec*proc``, then CTest allocated approximately 10 times the CPU/core time
+to run the tests for ``SubprojectA`` than for ``SubprojectB`` (e.g. so if effort
+is going to be expended to reduce the cost of the test suite for the whole
+project, then reducing the cost of the test suite for ``SubprojectA`` would
+likely have a larger impact than effort to reduce the cost of the test suite
+for ``SubprojectB``).
+
+.. _`Build and Test Mode`:
+
+Build and Test Mode
+===================
+
+CTest provides a command-line signature to configure (i.e. run cmake on),
+build, and/or execute a test::
+
+ ctest --build-and-test <path-to-source> <path-to-build>
+ --build-generator <generator>
+ [<options>...]
+ [--build-options <opts>...]
+ [--test-command <command> [<args>...]]
+
+The configure and test steps are optional. The arguments to this command line
+are the source and binary directories. The ``--build-generator`` option *must*
+be provided to use ``--build-and-test``. If ``--test-command`` is specified
+then that will be run after the build is complete. Other options that affect
+this mode include:
+
+``--build-target``
+ Specify a specific target to build.
+
+ If left out the ``all`` target is built.
+
+``--build-nocmake``
+ Run the build without running cmake first.
+
+ Skip the cmake step.
+
+``--build-run-dir``
+ Specify directory to run programs from.
+
+ Directory where programs will be after it has been compiled.
+
+``--build-two-config``
+ Run CMake twice.
+
+``--build-exe-dir``
+ Specify the directory for the executable.
+
+``--build-generator``
+ Specify the generator to use. See the :manual:`cmake-generators(7)` manual.
+
+``--build-generator-platform``
+ Specify the generator-specific platform.
+
+``--build-generator-toolset``
+ Specify the generator-specific toolset.
+
+``--build-project``
+ Specify the name of the project to build.
+
+``--build-makeprogram``
+ Specify the explicit make program to be used by CMake when configuring and
+ building the project. Only applicable for Make and Ninja based generators.
+
+``--build-noclean``
+ Skip the make clean step.
+
+``--build-config-sample``
+ A sample executable to use to determine the configuration that
+ should be used. e.g. ``Debug``, ``Release`` etc.
+
+``--build-options``
+ Additional options for configuring the build (i.e. for CMake, not for
+ the build tool). Note that if this is specified, the ``--build-options``
+ keyword and its arguments must be the last option given on the command
+ line, with the possible exception of ``--test-command``.
+
+``--test-command``
+ The command to run as the test step with the ``--build-and-test`` option.
+ All arguments following this keyword will be assumed to be part of the
+ test command line, so it must be the last option given.
+
+``--test-timeout``
+ The time limit in seconds
+
+.. _`Dashboard Client`:
+
+Dashboard Client
+================
+
+CTest can operate as a client for the `CDash`_ software quality dashboard
+application. As a dashboard client, CTest performs a sequence of steps
+to configure, build, and test software, and then submits the results to
+a `CDash`_ server. The command-line signature used to submit to `CDash`_ is::
+
+ ctest (-D <dashboard> | -M <model> -T <action> | -S <script> | -SP <script>)
+ [-- <dashboard-options>...]
+
+Options for Dashboard Client include:
+
+``--group <group>``
+ Specify what group you'd like to submit results to
+
+ Submit dashboard to specified group instead of default one. By
+ default, the dashboard is submitted to Nightly, Experimental, or
+ Continuous group, but by specifying this option, the group can be
+ arbitrary.
+
+ This replaces the deprecated option ``--track``.
+ Despite the name change its behavior is unchanged.
+
+``-A <file>, --add-notes <file>``
+ Add a notes file with submission.
+
+ This option tells CTest to include a notes file when submitting
+ dashboard.
+
+``--tomorrow-tag``
+ ``Nightly`` or ``Experimental`` starts with next day tag.
+
+ This is useful if the build will not finish in one day.
+
+``--extra-submit <file>[;<file>]``
+ Submit extra files to the dashboard.
+
+ This option will submit extra files to the dashboard.
+
+``--http1.0``
+ Submit using `HTTP 1.0`.
+
+ This option will force CTest to use `HTTP 1.0` to submit files to the
+ dashboard, instead of `HTTP 1.1`.
+
+``--no-compress-output``
+ Do not compress test output when submitting.
+
+ This flag will turn off automatic compression of test output. Use
+ this to maintain compatibility with an older version of CDash which
+ doesn't support compressed test output.
+
+Dashboard Client Steps
+----------------------
+
+CTest defines an ordered list of testing steps of which some or all may
+be run as a dashboard client:
+
+``Start``
+ Start a new dashboard submission to be composed of results recorded
+ by the following steps.
+ See the `CTest Start Step`_ section below.
+
+``Update``
+ Update the source tree from its version control repository.
+ Record the old and new versions and the list of updated source files.
+ See the `CTest Update Step`_ section below.
+
+``Configure``
+ Configure the software by running a command in the build tree.
+ Record the configuration output log.
+ See the `CTest Configure Step`_ section below.
+
+``Build``
+ Build the software by running a command in the build tree.
+ Record the build output log and detect warnings and errors.
+ See the `CTest Build Step`_ section below.
+
+``Test``
+ Test the software by loading a ``CTestTestfile.cmake``
+ from the build tree and executing the defined tests.
+ Record the output and result of each test.
+ See the `CTest Test Step`_ section below.
+
+``Coverage``
+ Compute coverage of the source code by running a coverage
+ analysis tool and recording its output.
+ See the `CTest Coverage Step`_ section below.
+
+``MemCheck``
+ Run the software test suite through a memory check tool.
+ Record the test output, results, and issues reported by the tool.
+ See the `CTest MemCheck Step`_ section below.
+
+``Submit``
+ Submit results recorded from other testing steps to the
+ software quality dashboard server.
+ See the `CTest Submit Step`_ section below.
+
+Dashboard Client Modes
+----------------------
+
+CTest defines three modes of operation as a dashboard client:
+
+``Nightly``
+ This mode is intended to be invoked once per day, typically at night.
+ It enables the ``Start``, ``Update``, ``Configure``, ``Build``, ``Test``,
+ ``Coverage``, and ``Submit`` steps by default. Selected steps run even
+ if the ``Update`` step reports no changes to the source tree.
+
+``Continuous``
+ This mode is intended to be invoked repeatedly throughout the day.
+ It enables the ``Start``, ``Update``, ``Configure``, ``Build``, ``Test``,
+ ``Coverage``, and ``Submit`` steps by default, but exits after the
+ ``Update`` step if it reports no changes to the source tree.
+
+``Experimental``
+ This mode is intended to be invoked by a developer to test local changes.
+ It enables the ``Start``, ``Configure``, ``Build``, ``Test``, ``Coverage``,
+ and ``Submit`` steps by default.
+
+Dashboard Client via CTest Command-Line
+---------------------------------------
+
+CTest can perform testing on an already-generated build tree.
+Run the ``ctest`` command with the current working directory set
+to the build tree and use one of these signatures::
+
+ ctest -D <mode>[<step>]
+ ctest -M <mode> [ -T <step> ]...
+
+The ``<mode>`` must be one of the above `Dashboard Client Modes`_,
+and each ``<step>`` must be one of the above `Dashboard Client Steps`_.
+
+CTest reads the `Dashboard Client Configuration`_ settings from
+a file in the build tree called either ``CTestConfiguration.ini``
+or ``DartConfiguration.tcl`` (the names are historical). The format
+of the file is::
+
+ # Lines starting in '#' are comments.
+ # Other non-blank lines are key-value pairs.
+ <setting>: <value>
+
+where ``<setting>`` is the setting name and ``<value>`` is the
+setting value.
+
+In build trees generated by CMake, this configuration file is
+generated by the :module:`CTest` module if included by the project.
+The module uses variables to obtain a value for each setting
+as documented with the settings below.
+
+.. _`CTest Script`:
+
+Dashboard Client via CTest Script
+---------------------------------
+
+CTest can perform testing driven by a :manual:`cmake-language(7)`
+script that creates and maintains the source and build tree as
+well as performing the testing steps. Run the ``ctest`` command
+with the current working directory set outside of any build tree
+and use one of these signatures::
+
+ ctest -S <script>
+ ctest -SP <script>
+
+The ``<script>`` file must call :ref:`CTest Commands` commands
+to run testing steps explicitly as documented below. The commands
+obtain `Dashboard Client Configuration`_ settings from their
+arguments or from variables set in the script.
+
+Dashboard Client Configuration
+==============================
+
+The `Dashboard Client Steps`_ may be configured by named
+settings as documented in the following sections.
+
+.. _`CTest Start Step`:
+
+CTest Start Step
+----------------
+
+Start a new dashboard submission to be composed of results recorded
+by the following steps.
+
+In a `CTest Script`_, the :command:`ctest_start` command runs this step.
+Arguments to the command may specify some of the step settings.
+The command first runs the command-line specified by the
+``CTEST_CHECKOUT_COMMAND`` variable, if set, to initialize the source
+directory.
+
+Configuration settings include:
+
+``BuildDirectory``
+ The full path to the project build tree.
+
+ * `CTest Script`_ variable: :variable:`CTEST_BINARY_DIRECTORY`
+ * :module:`CTest` module variable: :variable:`PROJECT_BINARY_DIR`
+
+``SourceDirectory``
+ The full path to the project source tree.
+
+ * `CTest Script`_ variable: :variable:`CTEST_SOURCE_DIRECTORY`
+ * :module:`CTest` module variable: :variable:`PROJECT_SOURCE_DIR`
+
+.. _`CTest Update Step`:
+
+CTest Update Step
+-----------------
+
+In a `CTest Script`_, the :command:`ctest_update` command runs this step.
+Arguments to the command may specify some of the step settings.
+
+Configuration settings to specify the version control tool include:
+
+``BZRCommand``
+ ``bzr`` command-line tool to use if source tree is managed by Bazaar.
+
+ * `CTest Script`_ variable: :variable:`CTEST_BZR_COMMAND`
+ * :module:`CTest` module variable: none
+
+``BZRUpdateOptions``
+ Command-line options to the ``BZRCommand`` when updating the source.
+
+ * `CTest Script`_ variable: :variable:`CTEST_BZR_UPDATE_OPTIONS`
+ * :module:`CTest` module variable: none
+
+``CVSCommand``
+ ``cvs`` command-line tool to use if source tree is managed by CVS.
+
+ * `CTest Script`_ variable: :variable:`CTEST_CVS_COMMAND`
+ * :module:`CTest` module variable: ``CVSCOMMAND``
+
+``CVSUpdateOptions``
+ Command-line options to the ``CVSCommand`` when updating the source.
+
+ * `CTest Script`_ variable: :variable:`CTEST_CVS_UPDATE_OPTIONS`
+ * :module:`CTest` module variable: ``CVS_UPDATE_OPTIONS``
+
+``GITCommand``
+ ``git`` command-line tool to use if source tree is managed by Git.
+
+ * `CTest Script`_ variable: :variable:`CTEST_GIT_COMMAND`
+ * :module:`CTest` module variable: ``GITCOMMAND``
+
+ The source tree is updated by ``git fetch`` followed by
+ ``git reset --hard`` to the ``FETCH_HEAD``. The result is the same
+ as ``git pull`` except that any local modifications are overwritten.
+ Use ``GITUpdateCustom`` to specify a different approach.
+
+``GITInitSubmodules``
+ If set, CTest will update the repository's submodules before updating.
+
+ * `CTest Script`_ variable: :variable:`CTEST_GIT_INIT_SUBMODULES`
+ * :module:`CTest` module variable: ``CTEST_GIT_INIT_SUBMODULES``
+
+``GITUpdateCustom``
+ Specify a custom command line (as a semicolon-separated list) to run
+ in the source tree (Git work tree) to update it instead of running
+ the ``GITCommand``.
+
+ * `CTest Script`_ variable: :variable:`CTEST_GIT_UPDATE_CUSTOM`
+ * :module:`CTest` module variable: ``CTEST_GIT_UPDATE_CUSTOM``
+
+``GITUpdateOptions``
+ Command-line options to the ``GITCommand`` when updating the source.
+
+ * `CTest Script`_ variable: :variable:`CTEST_GIT_UPDATE_OPTIONS`
+ * :module:`CTest` module variable: ``GIT_UPDATE_OPTIONS``
+
+``HGCommand``
+ ``hg`` command-line tool to use if source tree is managed by Mercurial.
+
+ * `CTest Script`_ variable: :variable:`CTEST_HG_COMMAND`
+ * :module:`CTest` module variable: none
+
+``HGUpdateOptions``
+ Command-line options to the ``HGCommand`` when updating the source.
+
+ * `CTest Script`_ variable: :variable:`CTEST_HG_UPDATE_OPTIONS`
+ * :module:`CTest` module variable: none
+
+``P4Client``
+ Value of the ``-c`` option to the ``P4Command``.
+
+ * `CTest Script`_ variable: :variable:`CTEST_P4_CLIENT`
+ * :module:`CTest` module variable: ``CTEST_P4_CLIENT``
+
+``P4Command``
+ ``p4`` command-line tool to use if source tree is managed by Perforce.
+
+ * `CTest Script`_ variable: :variable:`CTEST_P4_COMMAND`
+ * :module:`CTest` module variable: ``P4COMMAND``
+
+``P4Options``
+ Command-line options to the ``P4Command`` for all invocations.
+
+ * `CTest Script`_ variable: :variable:`CTEST_P4_OPTIONS`
+ * :module:`CTest` module variable: ``CTEST_P4_OPTIONS``
+
+``P4UpdateCustom``
+ Specify a custom command line (as a semicolon-separated list) to run
+ in the source tree (Perforce tree) to update it instead of running
+ the ``P4Command``.
+
+ * `CTest Script`_ variable: none
+ * :module:`CTest` module variable: ``CTEST_P4_UPDATE_CUSTOM``
+
+``P4UpdateOptions``
+ Command-line options to the ``P4Command`` when updating the source.
+
+ * `CTest Script`_ variable: :variable:`CTEST_P4_UPDATE_OPTIONS`
+ * :module:`CTest` module variable: ``CTEST_P4_UPDATE_OPTIONS``
+
+``SVNCommand``
+ ``svn`` command-line tool to use if source tree is managed by Subversion.
+
+ * `CTest Script`_ variable: :variable:`CTEST_SVN_COMMAND`
+ * :module:`CTest` module variable: ``SVNCOMMAND``
+
+``SVNOptions``
+ Command-line options to the ``SVNCommand`` for all invocations.
+
+ * `CTest Script`_ variable: :variable:`CTEST_SVN_OPTIONS`
+ * :module:`CTest` module variable: ``CTEST_SVN_OPTIONS``
+
+``SVNUpdateOptions``
+ Command-line options to the ``SVNCommand`` when updating the source.
+
+ * `CTest Script`_ variable: :variable:`CTEST_SVN_UPDATE_OPTIONS`
+ * :module:`CTest` module variable: ``SVN_UPDATE_OPTIONS``
+
+``UpdateCommand``
+ Specify the version-control command-line tool to use without
+ detecting the VCS that manages the source tree.
+
+ * `CTest Script`_ variable: :variable:`CTEST_UPDATE_COMMAND`
+ * :module:`CTest` module variable: ``<VCS>COMMAND``
+ when ``UPDATE_TYPE`` is ``<vcs>``, else ``UPDATE_COMMAND``
+
+``UpdateOptions``
+ Command-line options to the ``UpdateCommand``.
+
+ * `CTest Script`_ variable: :variable:`CTEST_UPDATE_OPTIONS`
+ * :module:`CTest` module variable: ``<VCS>_UPDATE_OPTIONS``
+ when ``UPDATE_TYPE`` is ``<vcs>``, else ``UPDATE_OPTIONS``
+
+``UpdateType``
+ Specify the version-control system that manages the source
+ tree if it cannot be detected automatically.
+ The value may be ``bzr``, ``cvs``, ``git``, ``hg``,
+ ``p4``, or ``svn``.
+
+ * `CTest Script`_ variable: none, detected from source tree
+ * :module:`CTest` module variable: ``UPDATE_TYPE`` if set,
+ else ``CTEST_UPDATE_TYPE``
+
+.. _`UpdateVersionOnly`:
+
+``UpdateVersionOnly``
+ Specify that you want the version control update command to only
+ discover the current version that is checked out, and not to update
+ to a different version.
+
+ * `CTest Script`_ variable: :variable:`CTEST_UPDATE_VERSION_ONLY`
+
+.. _`UpdateVersionOverride`:
+
+``UpdateVersionOverride``
+ Specify the current version of your source tree.
+
+ When this variable is set to a non-empty string, CTest will report the value
+ you specified rather than using the update command to discover the current
+ version that is checked out. Use of this variable supersedes
+ ``UpdateVersionOnly``. Like ``UpdateVersionOnly``, using this variable tells
+ CTest not to update the source tree to a different version.
+
+ * `CTest Script`_ variable: :variable:`CTEST_UPDATE_VERSION_OVERRIDE`
+
+Additional configuration settings include:
+
+``NightlyStartTime``
+ In the ``Nightly`` dashboard mode, specify the "nightly start time".
+ With centralized version control systems (``cvs`` and ``svn``),
+ the ``Update`` step checks out the version of the software as of
+ this time so that multiple clients choose a common version to test.
+ This is not well-defined in distributed version-control systems so
+ the setting is ignored.
+
+ * `CTest Script`_ variable: :variable:`CTEST_NIGHTLY_START_TIME`
+ * :module:`CTest` module variable: ``NIGHTLY_START_TIME`` if set,
+ else ``CTEST_NIGHTLY_START_TIME``
+
+.. _`CTest Configure Step`:
+
+CTest Configure Step
+--------------------
+
+In a `CTest Script`_, the :command:`ctest_configure` command runs this step.
+Arguments to the command may specify some of the step settings.
+
+Configuration settings include:
+
+``ConfigureCommand``
+ Command-line to launch the software configuration process.
+ It will be executed in the location specified by the
+ ``BuildDirectory`` setting.
+
+ * `CTest Script`_ variable: :variable:`CTEST_CONFIGURE_COMMAND`
+ * :module:`CTest` module variable: :variable:`CMAKE_COMMAND`
+ followed by :variable:`PROJECT_SOURCE_DIR`
+
+``LabelsForSubprojects``
+ Specify a semicolon-separated list of labels that will be treated as
+ subprojects. This mapping will be passed on to CDash when configure, test or
+ build results are submitted.
+
+ * `CTest Script`_ variable: :variable:`CTEST_LABELS_FOR_SUBPROJECTS`
+ * :module:`CTest` module variable: ``CTEST_LABELS_FOR_SUBPROJECTS``
+
+ See `Label and Subproject Summary`_.
+
+.. _`CTest Build Step`:
+
+CTest Build Step
+----------------
+
+In a `CTest Script`_, the :command:`ctest_build` command runs this step.
+Arguments to the command may specify some of the step settings.
+
+Configuration settings include:
+
+``DefaultCTestConfigurationType``
+ When the build system to be launched allows build-time selection
+ of the configuration (e.g. ``Debug``, ``Release``), this specifies
+ the default configuration to be built when no ``-C`` option is
+ given to the ``ctest`` command. The value will be substituted into
+ the value of ``MakeCommand`` to replace the literal string
+ ``${CTEST_CONFIGURATION_TYPE}`` if it appears.
+
+ * `CTest Script`_ variable: :variable:`CTEST_CONFIGURATION_TYPE`
+ * :module:`CTest` module variable: ``DEFAULT_CTEST_CONFIGURATION_TYPE``,
+ initialized by the :envvar:`CMAKE_CONFIG_TYPE` environment variable
+
+``LabelsForSubprojects``
+ Specify a semicolon-separated list of labels that will be treated as
+ subprojects. This mapping will be passed on to CDash when configure, test or
+ build results are submitted.
+
+ * `CTest Script`_ variable: :variable:`CTEST_LABELS_FOR_SUBPROJECTS`
+ * :module:`CTest` module variable: ``CTEST_LABELS_FOR_SUBPROJECTS``
+
+ See `Label and Subproject Summary`_.
+
+``MakeCommand``
+ Command-line to launch the software build process.
+ It will be executed in the location specified by the
+ ``BuildDirectory`` setting.
+
+ * `CTest Script`_ variable: :variable:`CTEST_BUILD_COMMAND`
+ * :module:`CTest` module variable: ``MAKECOMMAND``,
+ initialized by the :command:`build_command` command
+
+``UseLaunchers``
+ For build trees generated by CMake using one of the
+ :ref:`Makefile Generators` or the :generator:`Ninja`
+ generator, specify whether the
+ ``CTEST_USE_LAUNCHERS`` feature is enabled by the
+ :module:`CTestUseLaunchers` module (also included by the
+ :module:`CTest` module). When enabled, the generated build
+ system wraps each invocation of the compiler, linker, or
+ custom command line with a "launcher" that communicates
+ with CTest via environment variables and files to report
+ granular build warning and error information. Otherwise,
+ CTest must "scrape" the build output log for diagnostics.
+
+ * `CTest Script`_ variable: :variable:`CTEST_USE_LAUNCHERS`
+ * :module:`CTest` module variable: ``CTEST_USE_LAUNCHERS``
+
+.. _`CTest Test Step`:
+
+CTest Test Step
+---------------
+
+In a `CTest Script`_, the :command:`ctest_test` command runs this step.
+Arguments to the command may specify some of the step settings.
+
+Configuration settings include:
+
+``ResourceSpecFile``
+ Specify a
+ :ref:`resource specification file <ctest-resource-specification-file>`. See
+ :ref:`ctest-resource-allocation` for more information.
+
+``LabelsForSubprojects``
+ Specify a semicolon-separated list of labels that will be treated as
+ subprojects. This mapping will be passed on to CDash when configure, test or
+ build results are submitted.
+
+ * `CTest Script`_ variable: :variable:`CTEST_LABELS_FOR_SUBPROJECTS`
+ * :module:`CTest` module variable: ``CTEST_LABELS_FOR_SUBPROJECTS``
+
+ See `Label and Subproject Summary`_.
+
+``TestLoad``
+ While running tests in parallel (e.g. with ``-j``), try not to start
+ tests when they may cause the CPU load to pass above a given threshold.
+
+ * `CTest Script`_ variable: :variable:`CTEST_TEST_LOAD`
+ * :module:`CTest` module variable: ``CTEST_TEST_LOAD``
+
+``TimeOut``
+ The default timeout for each test if not specified by the
+ :prop_test:`TIMEOUT` test property.
+
+ * `CTest Script`_ variable: :variable:`CTEST_TEST_TIMEOUT`
+ * :module:`CTest` module variable: ``DART_TESTING_TIMEOUT``
+
+.. _`CTest Coverage Step`:
+
+CTest Coverage Step
+-------------------
+
+In a `CTest Script`_, the :command:`ctest_coverage` command runs this step.
+Arguments to the command may specify some of the step settings.
+
+Configuration settings include:
+
+``CoverageCommand``
+ Command-line tool to perform software coverage analysis.
+ It will be executed in the location specified by the
+ ``BuildDirectory`` setting.
+
+ * `CTest Script`_ variable: :variable:`CTEST_COVERAGE_COMMAND`
+ * :module:`CTest` module variable: ``COVERAGE_COMMAND``
+
+``CoverageExtraFlags``
+ Specify command-line options to the ``CoverageCommand`` tool.
+
+ * `CTest Script`_ variable: :variable:`CTEST_COVERAGE_EXTRA_FLAGS`
+ * :module:`CTest` module variable: ``COVERAGE_EXTRA_FLAGS``
+
+ These options are the first arguments passed to ``CoverageCommand``.
+
+.. _`CTest MemCheck Step`:
+
+CTest MemCheck Step
+-------------------
+
+In a `CTest Script`_, the :command:`ctest_memcheck` command runs this step.
+Arguments to the command may specify some of the step settings.
+
+Configuration settings include:
+
+``MemoryCheckCommand``
+ Command-line tool to perform dynamic analysis. Test command lines
+ will be launched through this tool.
+
+ * `CTest Script`_ variable: :variable:`CTEST_MEMORYCHECK_COMMAND`
+ * :module:`CTest` module variable: ``MEMORYCHECK_COMMAND``
+
+``MemoryCheckCommandOptions``
+ Specify command-line options to the ``MemoryCheckCommand`` tool.
+ They will be placed prior to the test command line.
+
+ * `CTest Script`_ variable: :variable:`CTEST_MEMORYCHECK_COMMAND_OPTIONS`
+ * :module:`CTest` module variable: ``MEMORYCHECK_COMMAND_OPTIONS``
+
+``MemoryCheckType``
+ Specify the type of memory checking to perform.
+
+ * `CTest Script`_ variable: :variable:`CTEST_MEMORYCHECK_TYPE`
+ * :module:`CTest` module variable: ``MEMORYCHECK_TYPE``
+
+``MemoryCheckSanitizerOptions``
+ Specify options to sanitizers when running with a sanitize-enabled build.
+
+ * `CTest Script`_ variable: :variable:`CTEST_MEMORYCHECK_SANITIZER_OPTIONS`
+ * :module:`CTest` module variable: ``MEMORYCHECK_SANITIZER_OPTIONS``
+
+``MemoryCheckSuppressionFile``
+ Specify a file containing suppression rules for the
+ ``MemoryCheckCommand`` tool. It will be passed with options
+ appropriate to the tool.
+
+ * `CTest Script`_ variable: :variable:`CTEST_MEMORYCHECK_SUPPRESSIONS_FILE`
+ * :module:`CTest` module variable: ``MEMORYCHECK_SUPPRESSIONS_FILE``
+
+Additional configuration settings include:
+
+``BoundsCheckerCommand``
+ Specify a ``MemoryCheckCommand`` that is known to be command-line
+ compatible with Bounds Checker.
+
+ * `CTest Script`_ variable: none
+ * :module:`CTest` module variable: none
+
+``PurifyCommand``
+ Specify a ``MemoryCheckCommand`` that is known to be command-line
+ compatible with Purify.
+
+ * `CTest Script`_ variable: none
+ * :module:`CTest` module variable: ``PURIFYCOMMAND``
+
+``ValgrindCommand``
+ Specify a ``MemoryCheckCommand`` that is known to be command-line
+ compatible with Valgrind.
+
+ * `CTest Script`_ variable: none
+ * :module:`CTest` module variable: ``VALGRIND_COMMAND``
+
+``ValgrindCommandOptions``
+ Specify command-line options to the ``ValgrindCommand`` tool.
+ They will be placed prior to the test command line.
+
+ * `CTest Script`_ variable: none
+ * :module:`CTest` module variable: ``VALGRIND_COMMAND_OPTIONS``
+
+``DrMemoryCommand``
+ Specify a ``MemoryCheckCommand`` that is known to be a command-line
+ compatible with DrMemory.
+
+ * `CTest Script`_ variable: none
+ * :module:`CTest` module variable: ``DRMEMORY_COMMAND``
+
+``DrMemoryCommandOptions``
+ Specify command-line options to the ``DrMemoryCommand`` tool.
+ They will be placed prior to the test command line.
+
+ * `CTest Script`_ variable: none
+ * :module:`CTest` module variable: ``DRMEMORY_COMMAND_OPTIONS``
+
+.. _`CTest Submit Step`:
+
+CTest Submit Step
+-----------------
+
+In a `CTest Script`_, the :command:`ctest_submit` command runs this step.
+Arguments to the command may specify some of the step settings.
+
+Configuration settings include:
+
+``BuildName``
+ Describe the dashboard client platform with a short string.
+ (Operating system, compiler, etc.)
+
+ * `CTest Script`_ variable: :variable:`CTEST_BUILD_NAME`
+ * :module:`CTest` module variable: ``BUILDNAME``
+
+``CDashVersion``
+ Legacy option. Not used.
+
+ * `CTest Script`_ variable: none, detected from server
+ * :module:`CTest` module variable: ``CTEST_CDASH_VERSION``
+
+``CTestSubmitRetryCount``
+ Specify a number of attempts to retry submission on network failure.
+
+ * `CTest Script`_ variable: none,
+ use the :command:`ctest_submit` ``RETRY_COUNT`` option.
+ * :module:`CTest` module variable: ``CTEST_SUBMIT_RETRY_COUNT``
+
+``CTestSubmitRetryDelay``
+ Specify a delay before retrying submission on network failure.
+
+ * `CTest Script`_ variable: none,
+ use the :command:`ctest_submit` ``RETRY_DELAY`` option.
+ * :module:`CTest` module variable: ``CTEST_SUBMIT_RETRY_DELAY``
+
+``CurlOptions``
+ Specify a semicolon-separated list of options to control the
+ Curl library that CTest uses internally to connect to the
+ server. Possible options are ``CURLOPT_SSL_VERIFYPEER_OFF``
+ and ``CURLOPT_SSL_VERIFYHOST_OFF``.
+
+ * `CTest Script`_ variable: :variable:`CTEST_CURL_OPTIONS`
+ * :module:`CTest` module variable: ``CTEST_CURL_OPTIONS``
+
+``DropLocation``
+ Legacy option. When ``SubmitURL`` is not set, it is constructed from
+ ``DropMethod``, ``DropSiteUser``, ``DropSitePassword``, ``DropSite``, and
+ ``DropLocation``.
+
+ * `CTest Script`_ variable: :variable:`CTEST_DROP_LOCATION`
+ * :module:`CTest` module variable: ``DROP_LOCATION`` if set,
+ else ``CTEST_DROP_LOCATION``
+
+``DropMethod``
+ Legacy option. When ``SubmitURL`` is not set, it is constructed from
+ ``DropMethod``, ``DropSiteUser``, ``DropSitePassword``, ``DropSite``, and
+ ``DropLocation``.
+
+ * `CTest Script`_ variable: :variable:`CTEST_DROP_METHOD`
+ * :module:`CTest` module variable: ``DROP_METHOD`` if set,
+ else ``CTEST_DROP_METHOD``
+
+``DropSite``
+ Legacy option. When ``SubmitURL`` is not set, it is constructed from
+ ``DropMethod``, ``DropSiteUser``, ``DropSitePassword``, ``DropSite``, and
+ ``DropLocation``.
+
+ * `CTest Script`_ variable: :variable:`CTEST_DROP_SITE`
+ * :module:`CTest` module variable: ``DROP_SITE`` if set,
+ else ``CTEST_DROP_SITE``
+
+``DropSitePassword``
+ Legacy option. When ``SubmitURL`` is not set, it is constructed from
+ ``DropMethod``, ``DropSiteUser``, ``DropSitePassword``, ``DropSite``, and
+ ``DropLocation``.
+
+ * `CTest Script`_ variable: :variable:`CTEST_DROP_SITE_PASSWORD`
+ * :module:`CTest` module variable: ``DROP_SITE_PASSWORD`` if set,
+ else ``CTEST_DROP_SITE_PASWORD``
+
+``DropSiteUser``
+ Legacy option. When ``SubmitURL`` is not set, it is constructed from
+ ``DropMethod``, ``DropSiteUser``, ``DropSitePassword``, ``DropSite``, and
+ ``DropLocation``.
+
+ * `CTest Script`_ variable: :variable:`CTEST_DROP_SITE_USER`
+ * :module:`CTest` module variable: ``DROP_SITE_USER`` if set,
+ else ``CTEST_DROP_SITE_USER``
+
+``IsCDash``
+ Legacy option. Not used.
+
+ * `CTest Script`_ variable: :variable:`CTEST_DROP_SITE_CDASH`
+ * :module:`CTest` module variable: ``CTEST_DROP_SITE_CDASH``
+
+``ScpCommand``
+ Legacy option. Not used.
+
+ * `CTest Script`_ variable: :variable:`CTEST_SCP_COMMAND`
+ * :module:`CTest` module variable: ``SCPCOMMAND``
+
+``Site``
+ Describe the dashboard client host site with a short string.
+ (Hostname, domain, etc.)
+
+ * `CTest Script`_ variable: :variable:`CTEST_SITE`
+ * :module:`CTest` module variable: ``SITE``,
+ initialized by the :command:`site_name` command
+
+``SubmitURL``
+ The ``http`` or ``https`` URL of the dashboard server to send the submission
+ to.
+
+ * `CTest Script`_ variable: :variable:`CTEST_SUBMIT_URL`
+ * :module:`CTest` module variable: ``SUBMIT_URL`` if set,
+ else ``CTEST_SUBMIT_URL``
+
+``TriggerSite``
+ Legacy option. Not used.
+
+ * `CTest Script`_ variable: :variable:`CTEST_TRIGGER_SITE`
+ * :module:`CTest` module variable: ``TRIGGER_SITE`` if set,
+ else ``CTEST_TRIGGER_SITE``
+
+.. _`Show as JSON Object Model`:
+
+Show as JSON Object Model
+=========================
+
+When the ``--show-only=json-v1`` command line option is given, the test
+information is output in JSON format. Version 1.0 of the JSON object
+model is defined as follows:
+
+``kind``
+ The string "ctestInfo".
+
+``version``
+ A JSON object specifying the version components. Its members are
+
+ ``major``
+ A non-negative integer specifying the major version component.
+ ``minor``
+ A non-negative integer specifying the minor version component.
+
+``backtraceGraph``
+ JSON object representing backtrace information with the
+ following members:
+
+ ``commands``
+ List of command names.
+ ``files``
+ List of file names.
+ ``nodes``
+ List of node JSON objects with members:
+
+ ``command``
+ Index into the ``commands`` member of the ``backtraceGraph``.
+ ``file``
+ Index into the ``files`` member of the ``backtraceGraph``.
+ ``line``
+ Line number in the file where the backtrace was added.
+ ``parent``
+ Index into the ``nodes`` member of the ``backtraceGraph``
+ representing the parent in the graph.
+
+``tests``
+ A JSON array listing information about each test. Each entry
+ is a JSON object with members:
+
+ ``name``
+ Test name.
+ ``config``
+ Configuration that the test can run on.
+ Empty string means any config.
+ ``command``
+ List where the first element is the test command and the
+ remaining elements are the command arguments.
+ ``backtrace``
+ Index into the ``nodes`` member of the ``backtraceGraph``.
+ ``properties``
+ Test properties.
+ Can contain keys for each of the supported test properties.
+
+.. _`ctest-resource-allocation`:
+
+Resource Allocation
+===================
+
+CTest provides a mechanism for tests to specify the resources that they need
+in a fine-grained way, and for users to specify the resources availiable on
+the running machine. This allows CTest to internally keep track of which
+resources are in use and which are free, scheduling tests in a way that
+prevents them from trying to claim resources that are not available.
+
+When the resource allocation feature is used, CTest will not oversubscribe
+resources. For example, if a resource has 8 slots, CTest will not run tests
+that collectively use more than 8 slots at a time. This has the effect of
+limiting how many tests can run at any given time, even if a high ``-j``
+argument is used, if those tests all use some slots from the same resource.
+In addition, it means that a single test that uses more of a resource than is
+available on a machine will not run at all (and will be reported as
+``Not Run``).
+
+A common use case for this feature is for tests that require the use of a GPU.
+Multiple tests can simultaneously allocate memory from a GPU, but if too many
+tests try to do this at once, some of them will fail to allocate, resulting in
+a failed test, even though the test would have succeeded if it had the memory
+it needed. By using the resource allocation feature, each test can specify how
+much memory it requires from a GPU, allowing CTest to schedule tests in a way
+that running several of these tests at once does not exhaust the GPU's memory
+pool.
+
+Please note that CTest has no concept of what a GPU is or how much memory it
+has, nor does it have any way of communicating with a GPU to retrieve this
+information or perform any memory management. CTest simply keeps track of a
+list of abstract resource types, each of which has a certain number of slots
+available for tests to use. Each test specifies the number of slots that it
+requires from a certain resource, and CTest then schedules them in a way that
+prevents the total number of slots in use from exceeding the listed capacity.
+When a test is executed, and slots from a resource are allocated to that test,
+tests may assume that they have exclusive use of those slots for the duration
+of the test's process.
+
+The CTest resource allocation feature consists of two inputs:
+
+* The :ref:`resource specification file <ctest-resource-specification-file>`,
+ described below, which describes the resources available on the system.
+* The :prop_test:`RESOURCE_GROUPS` property of tests, which describes the
+ resources required by the test.
+
+When CTest runs a test, the resources allocated to that test are passed in the
+form of a set of
+:ref:`environment variables <ctest-resource-environment-variables>` as
+described below. Using this information to decide which resource to connect to
+is left to the test writer.
+
+The ``RESOURCE_GROUPS`` property tells CTest what resources a test expects
+to use grouped in a way meaningful to the test. The test itself must read
+the :ref:`environment variables <ctest-resource-environment-variables>` to
+determine which resources have been allocated to each group. For example,
+each group may correspond to a process the test will spawn when executed.
+
+Note that even if a test specifies a ``RESOURCE_GROUPS`` property, it is still
+possible for that to test to run without any resource allocation (and without
+the corresponding
+:ref:`environment variables <ctest-resource-environment-variables>`)
+if the user does not pass a resource specification file. Passing this file,
+either through the ``--resource-spec-file`` command-line argument or the
+``RESOURCE_SPEC_FILE`` argument to :command:`ctest_test`, is what activates the
+resource allocation feature. Tests should check the
+``CTEST_RESOURCE_GROUP_COUNT`` environment variable to find out whether or not
+resource allocation is activated. This variable will always (and only) be
+defined if resource allocation is activated. If resource allocation is not
+activated, then the ``CTEST_RESOURCE_GROUP_COUNT`` variable will not exist,
+even if it exists for the parent ``ctest`` process. If a test absolutely must
+have resource allocation, then it can return a failing exit code or use the
+:prop_test:`SKIP_RETURN_CODE` or :prop_test:`SKIP_REGULAR_EXPRESSION`
+properties to indicate a skipped test.
+
+.. _`ctest-resource-specification-file`:
+
+Resource Specification File
+---------------------------
+
+The resource specification file is a JSON file which is passed to CTest, either
+on the :manual:`ctest(1)` command line as ``--resource-spec-file``, or as the
+``RESOURCE_SPEC_FILE`` argument of :command:`ctest_test`. The resource
+specification file must be a JSON object. All examples in this document assume
+the following resource specification file:
+
+.. code-block:: json
+
+ {
+ "version": {
+ "major": 1,
+ "minor": 0
+ },
+ "local": [
+ {
+ "gpus": [
+ {
+ "id": "0",
+ "slots": 2
+ },
+ {
+ "id": "1",
+ "slots": 4
+ },
+ {
+ "id": "2",
+ "slots": 2
+ },
+ {
+ "id": "3"
+ }
+ ],
+ "crypto_chips": [
+ {
+ "id": "card0",
+ "slots": 4
+ }
+ ]
+ }
+ ]
+ }
+
+The members are:
+
+``version``
+ An object containing a ``major`` integer field and a ``minor`` integer field.
+ Currently, the only supported version is major ``1``, minor ``0``. Any other
+ value is an error.
+
+``local``
+ A JSON array of resource sets present on the system. Currently, this array
+ is restricted to being of size 1.
+
+ Each array element is a JSON object with members whose names are equal to the
+ desired resource types, such as ``gpus``. These names must start with a
+ lowercase letter or an underscore, and subsequent characters can be a
+ lowercase letter, a digit, or an underscore. Uppercase letters are not
+ allowed, because certain platforms have case-insensitive environment
+ variables. See the `Environment Variables`_ section below for
+ more information. It is recommended that the resource type name be the plural
+ of a noun, such as ``gpus`` or ``crypto_chips`` (and not ``gpu`` or
+ ``crypto_chip``.)
+
+ Please note that the names ``gpus`` and ``crypto_chips`` are just examples,
+ and CTest does not interpret them in any way. You are free to make up any
+ resource type you want to meet your own requirements.
+
+ The value for each resource type is a JSON array consisting of JSON objects,
+ each of which describe a specific instance of the specified resource. These
+ objects have the following members:
+
+ ``id``
+ A string consisting of an identifier for the resource. Each character in
+ the identifier can be a lowercase letter, a digit, or an underscore.
+ Uppercase letters are not allowed.
+
+ Identifiers must be unique within a resource type. However, they do not
+ have to be unique across resource types. For example, it is valid to have a
+ ``gpus`` resource named ``0`` and a ``crypto_chips`` resource named ``0``,
+ but not two ``gpus`` resources both named ``0``.
+
+ Please note that the IDs ``0``, ``1``, ``2``, ``3``, and ``card0`` are just
+ examples, and CTest does not interpret them in any way. You are free to
+ make up any IDs you want to meet your own requirements.
+
+ ``slots``
+ An optional unsigned number specifying the number of slots available on the
+ resource. For example, this could be megabytes of RAM on a GPU, or
+ cryptography units available on a cryptography chip. If ``slots`` is not
+ specified, a default value of ``1`` is assumed.
+
+In the example file above, there are four GPUs with ID's 0 through 3. GPU 0 has
+2 slots, GPU 1 has 4, GPU 2 has 2, and GPU 3 has a default of 1 slot. There is
+also one cryptography chip with 4 slots.
+
+``RESOURCE_GROUPS`` Property
+----------------------------
+
+See :prop_test:`RESOURCE_GROUPS` for a description of this property.
+
+.. _`ctest-resource-environment-variables`:
+
+Environment Variables
+---------------------
+
+Once CTest has decided which resources to allocate to a test, it passes this
+information to the test executable as a series of environment variables. For
+each example below, we will assume that the test in question has a
+:prop_test:`RESOURCE_GROUPS` property of
+``2,gpus:2;gpus:4,gpus:1,crypto_chips:2``.
+
+The following variables are passed to the test process:
+
+.. envvar:: CTEST_RESOURCE_GROUP_COUNT
+
+ The total number of groups specified by the :prop_test:`RESOURCE_GROUPS`
+ property. For example:
+
+ * ``CTEST_RESOURCE_GROUP_COUNT=3``
+
+ This variable will only be defined if :manual:`ctest(1)` has been given a
+ ``--resource-spec-file``, or if :command:`ctest_test` has been given a
+ ``RESOURCE_SPEC_FILE``. If no resource specification file has been given,
+ this variable will not be defined.
+
+.. envvar:: CTEST_RESOURCE_GROUP_<num>
+
+ The list of resource types allocated to each group, with each item
+ separated by a comma. ``<num>`` is a number from zero to
+ ``CTEST_RESOURCE_GROUP_COUNT`` minus one. ``CTEST_RESOURCE_GROUP_<num>``
+ is defined for each ``<num>`` in this range. For example:
+
+ * ``CTEST_RESOURCE_GROUP_0=gpus``
+ * ``CTEST_RESOURCE_GROUP_1=gpus``
+ * ``CTEST_RESOURCE_GROUP_2=crypto_chips,gpus``
+
+.. envvar:: CTEST_RESOURCE_GROUP_<num>_<resource-type>
+
+ The list of resource IDs and number of slots from each ID allocated to each
+ group for a given resource type. This variable consists of a series of
+ pairs, each pair separated by a semicolon, and with the two items in the pair
+ separated by a comma. The first item in each pair is ``id:`` followed by the
+ ID of a resource of type ``<resource-type>``, and the second item is
+ ``slots:`` followed by the number of slots from that resource allocated to
+ the given group. For example:
+
+ * ``CTEST_RESOURCE_GROUP_0_GPUS=id:0,slots:2``
+ * ``CTEST_RESOURCE_GROUP_1_GPUS=id:2,slots:2``
+ * ``CTEST_RESOURCE_GROUP_2_GPUS=id:1,slots:4;id:3,slots:1``
+ * ``CTEST_RESOURCE_GROUP_2_CRYPTO_CHIPS=id:card0,slots:2``
+
+ In this example, group 0 gets 2 slots from GPU ``0``, group 1 gets 2 slots
+ from GPU ``2``, and group 2 gets 4 slots from GPU ``1``, 1 slot from GPU
+ ``3``, and 2 slots from cryptography chip ``card0``.
+
+ ``<num>`` is a number from zero to ``CTEST_RESOURCE_GROUP_COUNT`` minus one.
+ ``<resource-type>`` is the name of a resource type, converted to uppercase.
+ ``CTEST_RESOURCE_GROUP_<num>_<resource-type>`` is defined for the product
+ of each ``<num>`` in the range listed above and each resource type listed in
+ ``CTEST_RESOURCE_GROUP_<num>``.
+
+ Because some platforms have case-insensitive names for environment variables,
+ the names of resource types may not clash in a case-insensitive environment.
+ Because of this, for the sake of simplicity, all resource types must be
+ listed in all lowercase in the
+ :ref:`resource specification file <ctest-resource-specification-file>` and
+ in the :prop_test:`RESOURCE_GROUPS` property, and they are converted to all
+ uppercase in the ``CTEST_RESOURCE_GROUP_<num>_<resource-type>`` environment
+ variable.
+
+See Also
+========
+
+.. include:: LINKS.txt
+
+.. _`CDash`: http://cdash.org/