.. _plugins: Working with plugins and conftest files ======================================= ``pytest`` implements all aspects of configuration, collection, running and reporting by calling `well specified hooks`_. Virtually any Python module can be registered as a plugin. It can implement any number of hook functions (usually two or three) which all have a ``pytest_`` prefix, making hook functions easy to distinguish and find. There are three basic location types: * `builtin plugins`_: loaded from pytest's internal ``_pytest`` directory. * `external plugins`_: modules discovered through `setuptools entry points`_ * `conftest.py plugins`_: modules auto-discovered in test directories .. _`pytest/plugin`: http://bitbucket.org/hpk42/pytest/src/tip/pytest/plugin/ .. _`conftest.py plugins`: .. _`conftest.py`: .. _`localplugin`: .. _`conftest`: conftest.py: local per-directory plugins ---------------------------------------- local ``conftest.py`` plugins contain directory-specific hook implementations. Session and test running activities will invoke all hooks defined in ``conftest.py`` files closer to the root of the filesystem. Example: Assume the following layout and content of files:: a/conftest.py: def pytest_runtest_setup(item): # called for running each test in 'a' directory print ("setting up", item) a/test_sub.py: def test_sub(): pass test_flat.py: def test_flat(): pass Here is how you might run it:: py.test test_flat.py # will not show "setting up" py.test a/test_sub.py # will show "setting up" .. Note:: If you have ``conftest.py`` files which do not reside in a python package directory (i.e. one containing an ``__init__.py``) then "import conftest" can be ambiguous because there might be other ``conftest.py`` files as well on your PYTHONPATH or ``sys.path``. It is thus good practise for projects to either put ``conftest.py`` under a package scope or to never import anything from a conftest.py file. .. _`external plugins`: .. _`extplugins`: Installing External Plugins / Searching --------------------------------------- Installing a plugin happens through any usual Python installation tool, for example:: pip install pytest-NAME pip uninstall pytest-NAME If a plugin is installed, ``pytest`` automatically finds and integrates it, there is no need to activate it. We have a :doc:`page listing all 3rd party plugins and their status against the latest py.test version ` and here is a little annotated list for some popular plugins: .. _`django`: https://www.djangoproject.com/ * `pytest-django `_: write tests for `django`_ apps, using pytest integration. * `pytest-twisted `_: write tests for `twisted `_ apps, starting a reactor and processing deferreds from test functions. * `pytest-capturelog `_: to capture and assert about messages from the logging module * `pytest-cov `_: coverage reporting, compatible with distributed testing * `pytest-xdist `_: to distribute tests to CPUs and remote hosts, to run in boxed mode which allows to survive segmentation faults, to run in looponfailing mode, automatically re-running failing tests on file changes, see also :ref:`xdist` * `pytest-instafail `_: to report failures while the test run is happening. * `pytest-bdd `_ and `pytest-konira `_ to write tests using behaviour-driven testing. * `pytest-timeout `_: to timeout tests based on function marks or global definitions. * `pytest-cache `_: to interactively re-run failing tests and help other plugins to store test run information across invocations. * `pytest-pep8 `_: a ``--pep8`` option to enable PEP8 compliance checking. * `oejskit `_: a plugin to run javascript unittests in life browsers To see a complete list of all plugins with their latest testing status against different py.test and Python versions, please visit `pytest-plugs `_. You may also discover more plugins through a `pytest- pypi.python.org search`_. .. _`available installable plugins`: .. _`pytest- pypi.python.org search`: http://pypi.python.org/pypi?%3Aaction=search&term=pytest-&submit=search External plugin development --------------------------- Pytest community cares about pytest users, in particular, it's crucial for the community to keep all pieces of the pytest ecosystem well supported. External plugins are important for pytest users, as they implement many useful, and sometimes critical features which the pytest core does not implement. To simplify the plugin development and/or support, it was decided to create teams(companies) on popular code hosting services (at the moment it's github and bitbucket): * ``_: probably the most popular hosting for pytest plugins the name ``pytest-dev`` is choosen because ``pytest`` was already taken by some unknown person, who's not active on the github or just doesn't check the email used to register that organization. * ``_: pytest is hosted on the bitbucket, also a lot of pytest plugins are hosted there. More code hosting services can be added in the future. Community encourages pytest plugin developers to move their plugins under those organizations (eg transfer ownership), paying with a better support, faster feedback, better discoverability, etc. Generic workflow of the ownership transfer looks like: * If not a member already, developer asks for a membership in the `pytest plugin developers community` organizations listed above, sending an email to ``_. * Developer transfers the ownership of his plugin repository to the organization listed above. * Existing organization members will make sure proper write permissions are set up for developer who transfered the ownership to the organization. Writing a plugin by looking at examples --------------------------------------- .. _`Distribute`: http://pypi.python.org/pypi/distribute .. _`setuptools`: http://pypi.python.org/pypi/setuptools If you want to write a plugin, there are many real-life examples you can copy from: * a custom collection example plugin: :ref:`yaml plugin` * around 20 `builtin plugins`_ which provide pytest's own functionality * many `external plugins`_ providing additional features All of these plugins implement the documented `well specified hooks`_ to extend and add functionality. .. _`setuptools entry points`: Making your plugin installable by others ---------------------------------------- If you want to make your plugin externally available, you may define a so-called entry point for your distribution so that ``pytest`` finds your plugin module. Entry points are a feature that is provided by `setuptools`_ or `Distribute`_. pytest looks up the ``pytest11`` entrypoint to discover its plugins and you can thus make your plugin available by defining it in your setuptools/distribute-based setup-invocation: .. sourcecode:: python # sample ./setup.py file from setuptools import setup setup( name="myproject", packages = ['myproject'] # the following makes a plugin available to pytest entry_points = { 'pytest11': [ 'name_of_plugin = myproject.pluginmodule', ] }, ) If a package is installed this way, ``pytest`` will load ``myproject.pluginmodule`` as a plugin which can define `well specified hooks`_. .. _`pluginorder`: Plugin discovery order at tool startup -------------------------------------- ``pytest`` loads plugin modules at tool startup in the following way: * by loading all builtin plugins * by loading all plugins registered through `setuptools entry points`_. * by pre-scanning the command line for the ``-p name`` option and loading the specified plugin before actual command line parsing. * by loading all :file:`conftest.py` files as inferred by the command line invocation: - if no test paths are specified use current dir as a test path - if exists, load ``conftest.py`` and ``test*/conftest.py`` relative to the directory part of the first test path. Note that pytest does not find ``conftest.py`` files in deeper nested sub directories at tool startup. It is usually a good idea to keep your conftest.py file in the top level test or project root directory. * by recursively loading all plugins specified by the ``pytest_plugins`` variable in ``conftest.py`` files Requiring/Loading plugins in a test module or conftest file ----------------------------------------------------------- You can require plugins in a test module or a conftest file like this:: pytest_plugins = "name1", "name2", When the test module or conftest plugin is loaded the specified plugins will be loaded as well. You can also use dotted path like this:: pytest_plugins = "myapp.testsupport.myplugin" which will import the specified module as a ``pytest`` plugin. Accessing another plugin by name -------------------------------- If a plugin wants to collaborate with code from another plugin it can obtain a reference through the plugin manager like this: .. sourcecode:: python plugin = config.pluginmanager.getplugin("name_of_plugin") If you want to look at the names of existing plugins, use the ``--traceconfig`` option. .. _`findpluginname`: Finding out which plugins are active ------------------------------------ If you want to find out which plugins are active in your environment you can type:: py.test --traceconfig and will get an extended test header which shows activated plugins and their names. It will also print local plugins aka :ref:`conftest.py ` files when they are loaded. .. _`cmdunregister`: Deactivating / unregistering a plugin by name --------------------------------------------- You can prevent plugins from loading or unregister them:: py.test -p no:NAME This means that any subsequent try to activate/load the named plugin will it already existing. See :ref:`findpluginname` for how to obtain the name of a plugin. .. _`builtin plugins`: pytest default plugin reference =============================== You can find the source code for the following plugins in the `pytest repository `_. .. autosummary:: _pytest.assertion _pytest.capture _pytest.config _pytest.doctest _pytest.genscript _pytest.helpconfig _pytest.junitxml _pytest.mark _pytest.monkeypatch _pytest.nose _pytest.pastebin _pytest.pdb _pytest.pytester _pytest.python _pytest.recwarn _pytest.resultlog _pytest.runner _pytest.main _pytest.skipping _pytest.terminal _pytest.tmpdir _pytest.unittest .. _`well specified hooks`: pytest hook reference ===================== Hook specification and validation --------------------------------- ``pytest`` calls hook functions to implement initialization, running, test execution and reporting. When ``pytest`` loads a plugin it validates that each hook function conforms to its respective hook specification. Each hook function name and its argument names need to match a hook specification. However, a hook function may accept *fewer* parameters by simply not specifying them. If you mistype argument names or the hook name itself you get an error showing the available arguments. Initialization, command line and configuration hooks ---------------------------------------------------- .. currentmodule:: _pytest.hookspec .. autofunction:: pytest_load_initial_conftests .. autofunction:: pytest_cmdline_preparse .. autofunction:: pytest_cmdline_parse .. autofunction:: pytest_namespace .. autofunction:: pytest_addoption .. autofunction:: pytest_cmdline_main .. autofunction:: pytest_configure .. autofunction:: pytest_unconfigure Generic "runtest" hooks ----------------------- All runtest related hooks receive a :py:class:`pytest.Item` object. .. autofunction:: pytest_runtest_protocol .. autofunction:: pytest_runtest_setup .. autofunction:: pytest_runtest_call .. autofunction:: pytest_runtest_teardown .. autofunction:: pytest_runtest_makereport For deeper understanding you may look at the default implementation of these hooks in :py:mod:`_pytest.runner` and maybe also in :py:mod:`_pytest.pdb` which interacts with :py:mod:`_pytest.capture` and its input/output capturing in order to immediately drop into interactive debugging when a test failure occurs. The :py:mod:`_pytest.terminal` reported specifically uses the reporting hook to print information about a test run. Collection hooks ---------------- ``pytest`` calls the following hooks for collecting files and directories: .. autofunction:: pytest_ignore_collect .. autofunction:: pytest_collect_directory .. autofunction:: pytest_collect_file For influencing the collection of objects in Python modules you can use the following hook: .. autofunction:: pytest_pycollect_makeitem .. autofunction:: pytest_generate_tests After collection is complete, you can modify the order of items, delete or otherwise amend the test items: .. autofunction:: pytest_collection_modifyitems Reporting hooks --------------- Session related reporting hooks: .. autofunction:: pytest_collectstart .. autofunction:: pytest_itemcollected .. autofunction:: pytest_collectreport .. autofunction:: pytest_deselected And here is the central hook for reporting about test execution: .. autofunction:: pytest_runtest_logreport Debugging/Interaction hooks --------------------------- There are few hooks which can be used for special reporting or interaction with exceptions: .. autofunction:: pytest_internalerror .. autofunction:: pytest_keyboard_interrupt .. autofunction:: pytest_exception_interact Declaring new hooks ------------------------ Plugins and ``conftest.py`` files may declare new hooks that can then be implemented by other plugins in order to alter behaviour or interact with the new plugin: .. autofunction:: pytest_addhooks Hooks are usually declared as do-nothing functions that contain only documentation describing when the hook will be called and what return values are expected. For an example, see `newhooks.py`_ from :ref:`xdist`. .. _`newhooks.py`: https://bitbucket.org/hpk42/pytest-xdist/src/52082f70e7dd04b00361091b8af906c60fd6700f/xdist/newhooks.py?at=default Using hooks from 3rd party plugins ------------------------------------- Using new hooks from plugins as explained above might be a little tricky because the standard `Hook specification and validation`_ mechanism: if you depend on a plugin that is not installed, validation will fail and the error message will not make much sense to your users. One approach is to defer the hook implementation to a new plugin instead of declaring the hook functions directly in your plugin module, for example:: # contents of myplugin.py class DeferPlugin(object): """Simple plugin to defer pytest-xdist hook functions.""" def pytest_testnodedown(self, node, error): """standard xdist hook function. """ def pytest_configure(config): if config.pluginmanager.hasplugin('xdist'): config.pluginmanager.register(DeferPlugin()) This has the added benefit of allowing you to conditionally install hooks depending on which plugins are installed. hookwrapper: executing around other hooks ------------------------------------------------- .. currentmodule:: _pytest.core .. versionadded:: 2.7 (experimental) pytest plugins can implement hook wrappers which which wrap the execution of other hook implementations. A hook wrapper is a generator function which yields exactly once. When pytest invokes hooks it first executes hook wrappers and passes the same arguments as to the regular hooks. At the yield point of the hook wrapper pytest will execute the next hook implementations and return their result to the yield point in the form of a :py:class:`CallOutcome` instance which encapsulates a result or exception info. The yield point itself will thus typically not raise exceptions (unless there are bugs). Here is an example definition of a hook wrapper:: import pytest @pytest.mark.hookwrapper def pytest_pyfunc_call(pyfuncitem): # do whatever you want before the next hook executes outcome = yield # outcome.excinfo may be None or a (cls, val, tb) tuple res = outcome.get_result() # will raise if outcome was exception # postprocess result Note that hook wrappers don't return results themselves, they merely perform tracing or other side effects around the actual hook implementations. If the result of the underlying hook is a mutable object, they may modify that result, however. Reference of objects involved in hooks ====================================== .. autoclass:: _pytest.config.Config() :members: .. autoclass:: _pytest.config.Parser() :members: .. autoclass:: _pytest.main.Node() :members: .. autoclass:: _pytest.main.Collector() :members: :show-inheritance: .. autoclass:: _pytest.main.Item() :members: :show-inheritance: .. autoclass:: _pytest.python.Module() :members: :show-inheritance: .. autoclass:: _pytest.python.Class() :members: :show-inheritance: .. autoclass:: _pytest.python.Function() :members: :show-inheritance: .. autoclass:: _pytest.runner.CallInfo() :members: .. autoclass:: _pytest.runner.TestReport() :members: .. autoclass:: _pytest.core.CallOutcome() :members: