diff options
| -rw-r--r-- | changelog/2626.doc | 1 | ||||
| -rw-r--r-- | doc/en/mark.rst | 1 | ||||
| -rw-r--r-- | doc/en/skipping.rst | 1 | ||||
| -rw-r--r-- | doc/en/unittest.rst | 135 | ||||
| -rw-r--r-- | doc/en/usage.rst | 7 |
5 files changed, 94 insertions, 51 deletions
diff --git a/changelog/2626.doc b/changelog/2626.doc new file mode 100644 index 000000000..d244ca12c --- /dev/null +++ b/changelog/2626.doc @@ -0,0 +1 @@ +Explicitly document which pytest features work with ``unittest``. diff --git a/doc/en/mark.rst b/doc/en/mark.rst index ab9546d31..0b0e072a0 100644 --- a/doc/en/mark.rst +++ b/doc/en/mark.rst @@ -10,6 +10,7 @@ By using the ``pytest.mark`` helper you can easily set metadata on your test functions. There are some builtin markers, for example: +* :ref:`skip <skip>` - always skip a test function * :ref:`skipif <skipif>` - skip a test function if a certain condition is met * :ref:`xfail <xfail>` - produce an "expected failure" outcome if a certain condition is met diff --git a/doc/en/skipping.rst b/doc/en/skipping.rst index f3c292a18..8690035a3 100644 --- a/doc/en/skipping.rst +++ b/doc/en/skipping.rst @@ -27,6 +27,7 @@ corresponding to the "short" letters shown in the test progress:: (See :ref:`how to change command line options defaults`) .. _skipif: +.. _skip: .. _`condition booleans`: Skipping test functions diff --git a/doc/en/unittest.rst b/doc/en/unittest.rst index 56721ff9b..92b9e653e 100644 --- a/doc/en/unittest.rst +++ b/doc/en/unittest.rst @@ -2,58 +2,77 @@ .. _`unittest.TestCase`: .. _`unittest`: -Support for unittest.TestCase / Integration of fixtures -===================================================================== - -.. _`unittest.py style`: http://docs.python.org/library/unittest.html - -``pytest`` has support for running Python `unittest.py style`_ tests. -It's meant for leveraging existing unittest-style projects -to use pytest features. Concretely, pytest will automatically -collect ``unittest.TestCase`` subclasses and their ``test`` methods in -test files. It will invoke typical setup/teardown methods and -generally try to make test suites written to run on unittest, to also -run using ``pytest``. We assume here that you are familiar with writing -``unittest.TestCase`` style tests and rather focus on -integration aspects. - -Note that this is meant as a provisional way of running your test code -until you fully convert to pytest-style tests. To fully take advantage of -:ref:`fixtures <fixture>`, :ref:`parametrization <parametrize>` and -:ref:`hooks <writing-plugins>` you should convert (tools like `unittest2pytest -<https://pypi.python.org/pypi/unittest2pytest/>`__ are helpful). -Also, not all 3rd party pluging are expected to work best with -``unittest.TestCase`` style tests. - -Usage -------------------------------------------------------------------- - -After :ref:`installation` type:: - - pytest - -and you should be able to run your unittest-style tests if they -are contained in ``test_*`` modules. If that works for you then -you can make use of most :ref:`pytest features <features>`, for example -``--pdb`` debugging in failures, using :ref:`plain assert-statements <assert>`, -:ref:`more informative tracebacks <tbreportdemo>`, stdout-capturing or -distributing tests to multiple CPUs via the ``-nNUM`` option if you -installed the ``pytest-xdist`` plugin. Please refer to -the general ``pytest`` documentation for many more examples. +unittest.TestCase Support +========================= -.. note:: +``pytest`` supports running Python ``unittest``-based tests out of the box. +It's meant for leveraging existing ``unittest``-based test suites +to use pytest as a test runner and also allow to incrementally adapt +the test suite to take full advantage of pytest's features. + +To run an existing ``unittest``-style test suite using ``pytest``, type:: + + pytest tests + + +pytest will automatically collect ``unittest.TestCase`` subclasses and +their ``test`` methods in ``test_*.py`` or ``*_test.py`` files. + +Almost all ``unittest`` features are supported: + +* ``@unittest.skip`` style decorators; +* ``setUp/tearDown``; +* ``setUpClass/tearDownClass()``; + +.. _`load_tests protocol`: https://docs.python.org/3/library/unittest.html#load-tests-protocol +.. _`setUpModule/tearDownModule`: https://docs.python.org/3/library/unittest.html#setupmodule-and-teardownmodule +.. _`subtests`: https://docs.python.org/3/library/unittest.html#distinguishing-test-iterations-using-subtests + +Up to this point pytest does not have support for the following features: + +* `load_tests protocol`_; +* `setUpModule/tearDownModule`_; +* `subtests`_; + +Benefits out of the box +----------------------- + +By running your test suite with pytest you can make use of several features, +in most cases without having to modify existing code: + +* Obtain :ref:`more informative tracebacks <tbreportdemo>`; +* :ref:`stdout and stderr <captures>` capturing; +* :ref:`Test selection options <select-tests>` using ``-k`` and ``-m`` flags; +* :ref:`maxfail`; +* :ref:`--pdb <pdb-option>` command-line option for debugging on test failures + (see :ref:`note <pdb-unittest-note>` below); +* Distribute tests to multiple CPUs using the `pytest-xdist <http://pypi.python.org/pypi/pytest-xdist>`_ plugin; +* Use :ref:`plain assert-statements <assert>` instead of ``self.assert*`` functions (`unittest2pytest + <https://pypi.python.org/pypi/unittest2pytest/>`__ is immensely helpful in this); - Running tests from ``unittest.TestCase`` subclasses with ``--pdb`` will - disable tearDown and cleanup methods for the case that an Exception - occurs. This allows proper post mortem debugging for all applications - which have significant logic in their tearDown machinery. However, - supporting this feature has the following side effect: If people - overwrite ``unittest.TestCase`` ``__call__`` or ``run``, they need to - to overwrite ``debug`` in the same way (this is also true for standard - unittest). -Mixing pytest fixtures into unittest.TestCase style tests ------------------------------------------------------------ +pytest features in ``unittest.TestCase`` subclasses +--------------------------------------------------- + +The following pytest features work in ``unittest.TestCase`` subclasses: + +* :ref:`Marks <mark>`: :ref:`skip <skip>`, :ref:`skipif <skipif>`, :ref:`xfail <xfail>`; +* :ref:`Auto-use fixtures <mixing-fixtures>`; + +The following pytest features **do not** work, and probably +never will due to different design philosophies: + +* :ref:`Fixtures <fixture>` (except for ``autouse`` fixtures, see :ref:`below <mixing-fixtures>`); +* :ref:`Parametrization <parametrize>`; +* :ref:`Custom hooks <writing-plugins>`; + + +Third party plugins may or may not work well, depending on the plugin and the test suite. + +.. _mixing-fixtures: + +Mixing pytest fixtures into ``unittest.TestCase`` subclasses using marks +------------------------------------------------------------------------ Running your unittest with ``pytest`` allows you to use its :ref:`fixture mechanism <fixture>` with ``unittest.TestCase`` style @@ -143,8 +162,8 @@ share the same ``self.db`` instance which was our intention when writing the class-scoped fixture function above. -autouse fixtures and accessing other fixtures -------------------------------------------------------------------- +Using autouse fixtures and accessing other fixtures +--------------------------------------------------- Although it's usually better to explicitly declare use of fixtures you need for a given test, you may sometimes want to have fixtures that are @@ -165,6 +184,7 @@ creation of a per-test temporary directory:: import unittest class MyTest(unittest.TestCase): + @pytest.fixture(autouse=True) def initdir(self, tmpdir): tmpdir.chdir() # change to pytest-provided temporary directory @@ -200,3 +220,16 @@ was executed ahead of the ``test_method``. You can also gradually move away from subclassing from ``unittest.TestCase`` to *plain asserts* and then start to benefit from the full pytest feature set step by step. + +.. _pdb-unittest-note: + +.. note:: + + Running tests from ``unittest.TestCase`` subclasses with ``--pdb`` will + disable tearDown and cleanup methods for the case that an Exception + occurs. This allows proper post mortem debugging for all applications + which have significant logic in their tearDown machinery. However, + supporting this feature has the following side effect: If people + overwrite ``unittest.TestCase`` ``__call__`` or ``run``, they need to + to overwrite ``debug`` in the same way (this is also true for standard + unittest). diff --git a/doc/en/usage.rst b/doc/en/usage.rst index e19415892..9360a8ed2 100644 --- a/doc/en/usage.rst +++ b/doc/en/usage.rst @@ -41,6 +41,8 @@ Getting help on version, option names, environment variables pytest -h | --help # show help on command line and config file options +.. _maxfail: + Stopping after the first (or N) failures --------------------------------------------------- @@ -49,6 +51,8 @@ To stop the testing process after the first (N) failures:: pytest -x # stop after first failure pytest --maxfail=2 # stop after two failures +.. _select-tests: + Specifying tests / selecting tests --------------------------------------------------- @@ -135,6 +139,9 @@ with Ctrl+C to find out where the tests are *hanging*. By default no output will be shown (because KeyboardInterrupt is caught by pytest). By using this option you make sure a trace is shown. + +.. _pdb-option: + Dropping to PDB_ (Python Debugger) on failures ----------------------------------------------- |