summaryrefslogtreecommitdiff
path: root/bench
AgeCommit message (Collapse)Author
2020-11-01unittest: fix quadratic behavior in collection of unittests using ↵Ran Benita
setUpClass/setup_method This is similar to 50114d4731876dae; I missed that unittest does the same thing.
2020-10-23python: fix quadratic behavior in collection of items using xunit fixturesRan Benita
Since commit 0f918b1a9dd14a2d046 pytest uses auto-generated autouse pytest fixtures for the xunit fixtures {setup,teardown}_{module,class,method,function}. All of these fixtures were given the same name. Unfortunately, pytest fixture lookup for a name works by grabbing all of the fixtures globally declared with a name and filtering to only those which match the specific node. So each xunit-using item iterates over a list (of fixturedefs) of a size of all previous same-xunit-using items, i.e. quadratic. Fixing this properly to use a better data structure is likely to take some effort, but we can avoid the immediate problem by just using a different name for each item's autouse fixture, so it only matches itself. A benchmark is added to demonstrate the issue. It is still way too slow after the fix and possibly still quadratic, but for a different reason which is another matter. Running --collect-only, before (snipped): 202533232 function calls (201902604 primitive calls) in 86.379 seconds ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 85.688 85.688 main.py:320(pytest_collection) 1 0.000 0.000 85.688 85.688 main.py:567(perform_collect) 80557/556 0.021 0.000 85.050 0.153 {method 'extend' of 'list' objects} 85001/15001 0.166 0.000 85.045 0.006 main.py:785(genitems) 10002 0.050 0.000 84.219 0.008 runner.py:455(collect_one_node) 10002 0.049 0.000 83.763 0.008 runner.py:340(pytest_make_collect_report) 10002 0.079 0.000 83.668 0.008 runner.py:298(from_call) 10002 0.019 0.000 83.541 0.008 runner.py:341(<lambda>) 5001 0.184 0.000 81.922 0.016 python.py:412(collect) 5000 0.020 0.000 81.072 0.016 python.py:842(collect) 30003 0.118 0.000 78.478 0.003 python.py:218(pytest_pycollect_makeitem) 30000 0.190 0.000 77.957 0.003 python.py:450(_genfunctions) 40001 0.081 0.000 76.664 0.002 nodes.py:183(from_parent) 30000 0.087 0.000 76.629 0.003 python.py:1595(from_parent) 40002 0.092 0.000 76.583 0.002 nodes.py:102(_create) 30000 0.305 0.000 76.404 0.003 python.py:1533(__init__) 15000 0.132 0.000 74.765 0.005 fixtures.py:1439(getfixtureinfo) 15000 0.165 0.000 73.664 0.005 fixtures.py:1492(getfixtureclosure) 15000 0.044 0.000 57.584 0.004 fixtures.py:1653(getfixturedefs) 30000 18.840 0.001 57.540 0.002 fixtures.py:1668(_matchfactories) 37507500 31.352 0.000 38.700 0.000 nodes.py:76(ischildnode) 15000 10.464 0.001 15.806 0.001 fixtures.py:1479(_getautousenames) 112930587/112910019 7.333 0.000 7.339 0.000 {built-in method builtins.len} After: 51890333 function calls (51259706 primitive calls) in 27.306 seconds ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 26.783 26.783 main.py:320(pytest_collection) 1 0.000 0.000 26.783 26.783 main.py:567(perform_collect) 80557/556 0.020 0.000 26.108 0.047 {method 'extend' of 'list' objects} 85001/15001 0.151 0.000 26.103 0.002 main.py:785(genitems) 10002 0.047 0.000 25.324 0.003 runner.py:455(collect_one_node) 10002 0.045 0.000 24.888 0.002 runner.py:340(pytest_make_collect_report) 10002 0.069 0.000 24.805 0.002 runner.py:298(from_call) 10002 0.017 0.000 24.690 0.002 runner.py:341(<lambda>) 5001 0.168 0.000 23.150 0.005 python.py:412(collect) 5000 0.019 0.000 22.223 0.004 python.py:858(collect) 30003 0.101 0.000 19.818 0.001 python.py:218(pytest_pycollect_makeitem) 30000 0.161 0.000 19.368 0.001 python.py:450(_genfunctions) 30000 0.302 0.000 18.236 0.001 python.py:1611(from_parent) 40001 0.084 0.000 18.051 0.000 nodes.py:183(from_parent) 40002 0.116 0.000 17.967 0.000 nodes.py:102(_create) 30000 0.308 0.000 17.770 0.001 python.py:1549(__init__) 15000 0.117 0.000 16.111 0.001 fixtures.py:1439(getfixtureinfo) 15000 0.134 0.000 15.135 0.001 fixtures.py:1492(getfixtureclosure) 15000 9.320 0.001 14.738 0.001 fixtures.py:1479(_getautousenames)
2019-07-09Add rudimentary mypy type checkingRan Benita
Add a very lax mypy configuration, add it to tox -e linting, and fix/ignore the few errors that come up. The idea is to get it running before diving in too much. This enables: - Progressively adding type annotations and enabling more strict options, which will improve the codebase (IMO). - Annotating the public API in-line, and eventually exposing it to library users who use type checkers (with a py.typed file). Though, none of this is done yet. Refs https://github.com/pytest-dev/pytest/issues/3342.
2019-06-03pre-commit run reorder-python-imports --all-filesAnthony Sottile
2019-06-03pre-commit run fix-encoding-pragma --all-filesAnthony Sottile
2019-05-14Use fix-encoding-pragma pre-commit hookAnthony Sottile
2019-05-06Use exec directlyAnthony Sottile
2019-04-12pre-commit autoupdateAnthony Sottile
2019-03-18Fix bench/bench.py without argsDaniel Hahler
Fixes: > File "…/Vcs/pytest/src/_pytest/config/__init__.py", line 60, in main > config = _prepareconfig(args, plugins) > File "…/Vcs/pytest/src/_pytest/config/__init__.py", line 179, in _prepareconfig > raise TypeError(msg.format(args, type(args))) > TypeError: `args` parameter expected to be a list or tuple of strings, got: 'empty.py' (type: <class 'str'>)
2019-03-01Run isortDaniel Hahler
2018-10-25Apply reorder-python-imports to all filesAnthony Sottile
2018-10-08Upgrade pre-commit hooksAnthony Sottile
2018-08-22py.builtin.exec_ => six.exec_Anthony Sottile
2018-05-23sort out flake8 issues and unicode name usageRonny Pfannschmidt
2018-05-23run blackRonny Pfannschmidt
2018-05-23fix line endingsRonny Pfannschmidt
2014-10-06simplify internal pytester machineryholger krekel
2014-03-14- turn on capturing before early conftest loading and make terminal writerholger krekel
use the original stream. - avoid resetting capture FDs/sys.stdout for each test by keeping capturing always turned on and looking at snapshotted capturing data during runtest and collection phases.
2014-01-18replace py.test module references with pytestJurko Gospodnetić
The only remaining 'py.test' references are: * those referring to the 'py.test' executable * those in code explicitly testing py.test/pytest module compatibility * those in old CHANGES documentation * those in documentation generated based on external data * those in seemingly unfinished & unmaintained Japanese documentation Minor stylistic changes and typo corrections made to documentation next to several applied py.test --> pytest content changes.
2013-12-09make bench.py accept an optional script name and add a slow "manyparam" testholger krekel
2013-12-07add a skip benchmark file (from issue400).holger krekel
2013-10-03simplify the implementation of NodeKeywords getting rid of __ descriptors ↵holger krekel
appearing there.
2013-08-06argcomplete: FastFileCompleter that doesn't call bash in subprocess, strip ↵Anthon van der Neut
prefix dir ``` timeit result for 10000 iterations of expanding '/d' (lowered the count in the code afterwards) # 2.7.5 3.3.2 # FilesCompleter 75.1109 69.2116 # FastFilesCompleter 0.7383 1.0760 ``` - does not display prefix dir (like bash, not like compgen), py.test /usr/<TAB> does not show /usr/bin/ but bin/
2010-11-23python3 fixesBenjamin Peterson
2010-11-18fix bare "py.test" runs without a directory by not defaulting to ↵holger krekel
--doctest-modules which will virtually import everything
2010-10-31introduce "-q" option which decreases verbosity and basically leads to a ↵holger krekel
unittest/nosetest-style "." output add it in an ini file like this: [pytest] addargs=-q and you get that by default.
2010-10-15adding a small bench script to see where time is spend in the hook architectureholger krekel
--HG-- branch : trunk