aboutsummaryrefslogtreecommitdiff
path: root/crosperf/crosperf_unittest.py
AgeCommit message (Collapse)Author
2020-05-26crosperf: introduce option "compress_results" for result directoryzhizhouy
Perf data and report file can occupy large disk space when we copy and store them in the crosperf results directory. This patch introduces a new field for crosperf so that we create a tarball for it to save space. By default compress_result is set to True. BUG=chromium:1079048 TEST=Passed unittest, tested with simple examples. Change-Id: I70db6f3f70dc33bacfccd8833a4327c494b4d6b5 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2212432 Reviewed-by: George Burgess <gbiv@chromium.org> Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com>
2020-01-30crosperf: migration to python 3Zhizhou Yang
This patch migrates crosperf and its utils to python 3. TEST=Passed presubmit check; tested with simple experiment locally. BUG=chromium:1011676 Change-Id: Ib2a9f9c7cf6a1bb1d0b42a1dd3d9e3cbb4d70a36 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2003796 Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Mike Frysinger <vapier@chromium.org> Reviewed-by: Caroline Tice <cmtice@chromium.org> Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
2019-11-13crosperf: fix unittest error without prodaccessZhizhou Yang
After locking mechanism changed, crosperf checks if locks_dir is set or not and by default uses X20 location. This requires prodaccess thus unittests will fail without prodaccess if locks_dir is not explicitly set. This patch set all experiment files in unittests to use /tmp for locks_dir. Also fixed some lint error for deprecated functions. TEST=Passed all unittest. BUG=None Change-Id: Iff4623aad9632842010e5efb85a1e75a38bcf73e Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1912767 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com>
2019-10-27crosperf: Add option to not download debug symbolsZhizhou Yang
This patch introduces a new option --download_debug to crosperf. We will only download debug symbols when this option is set to True. This will save space when perf test users doesn't need it. TEST=passed unittests. BUG=chromium:1010294 Change-Id: Ic80958377f1c25cb04e3b7b56b40fafcada6d2d8 Reviewed-on: https://chromium-review.googlesource.com/1865958 Tested-by: Zhizhou Yang <zhizhouy@google.com> Commit-Ready: Zhizhou Yang <zhizhouy@google.com> Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org> Reviewed-by: Denis Nikitin <denik@chromium.org>
2019-09-24crosperf: Fix cros lint warningsDenis Nikitin
Fixed python lint warnings in crosperf scripts. BUG=chromium:993035 TEST=`repo upload` works w/o --no-verify Change-Id: I3dd7b0ea956458e57e8bd77c19a5b8cb13e1d350 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1821449 Tested-by: Denis Nikitin <denik@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org>
2019-09-24crosperf: Add top_interval argument to experimentDenis Nikitin
Added the following optional argument in the global settings: "top_interval" - Run top command in the background of a benchmark with interval of sampling specified in seconds (float type). With zero value don't run top (default value). Top output is stored in <resultsdir>/top.log. BUG=chromium:966514 TEST=unittests and HW tests on eve, kevin64, scarlet passed. Change-Id: I87b187c85912140fc0c16e224580b60061c8bd94 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1817121 Tested-by: Denis Nikitin <denik@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org>
2019-09-16crosperf: Setup CPU frequencyDenis Nikitin
Added "cpu_freq_pct" in global settings specifying percent of the max frequency which will be set on DUT during benchmark run. The actual value will be chosen from available frequencies less than or equal to the calculated number. Feature requires scaling_available_frequencies. On the platforms not supporting scaling_available_frequencies "cpu_freq_pct" will be ignored. BUG=chromium:966514 TEST=Unittest and local HW tests on veyron, scarlet and kevin64 passed. Change-Id: I175e2836ecf02fcf6b93d196926c7801d7f016c0 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1803780 Reviewed-by: George Burgess <gbiv@chromium.org> Tested-by: Denis Nikitin <denik@chromium.org> Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org>
2019-09-10crosperf: Add dut config arguments to experimentDenis Nikitin
In global settings added optional arguments: "cooldown_time" - wait time prior running a benchmark (default: 0), "cooldown_temp" - temperature threshold for waiting (default: 40), "governor" - CPU governor (default: performance), "cpu_usage" - Restrict CPU usage to specific configurations (default: all). "turbostat" argument is moved from benchmark to global settings. Current CL does not apply configurations. Instead it just propagates parameters to SuiteRunner class through "dut_config" dictionary. BUG=chromium:966514 TEST=Unitest and local HW tests passed. Change-Id: I1b2a65883e5176fdde49c9858ebe62a097df89cb Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1778515 Tested-by: Denis Nikitin <denik@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org>
2019-08-09crosperf: Print traceback instead of exception type.Tiancong Wang
I found that the error message of crosperf is sometimes frustrating that when you trigger an exception somewhere in the crosperf tool, it will always return the exception type. Sometimes it's really hard to find out where the exception/error happens in your code. I think it might make sense to handle all exceptions with a try...except block, to make the error message always appear at last, so raising the original exception prints whole traceback, instead of dropping it to general SystemExit. BUG=None TEST=Tested locally, and added unittest. Change-Id: Id772af444bf3788011c74504c71939d2514f1752 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1743146 Commit-Queue: Tiancong Wang <tcwang@google.com> Tested-by: Tiancong Wang <tcwang@google.com> Reviewed-by: George Burgess <gbiv@chromium.org>
2019-08-02crosperf: Add "intel_pstate" and "turbostat" argsDenis Nikitin
In global setting added "intel_pstate" option. Type: string. Values: active, passive, no_hwp, ''(equivalent to active). Default ''. In benchmark setting added "turbostat" option. Type: boolean. Default True. BUG=chromium:966514 TEST=tested on samus, eve, rammus Change-Id: I1533c6d6586aab5acf3665bce68920a0ab3092bc Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1726635 Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org> Commit-Queue: Sean Abraham <seanabraham@chromium.org> Reviewed-by: Sean Abraham <seanabraham@chromium.org> Tested-by: Sean Abraham <seanabraham@chromium.org>
2019-07-22crosperf: introduce skylab field and pass to experiment and labelZhizhou Yang
This is the initial patch to migrate crosperf from autotest to skylab for lab runs. This patch introduces skylab field in experiment, parsed it in settings_factory and restructured label class and experiment class so that further steps can use skylab field. TEST=Passed all unit tests BUG=chromium:984790 Change-Id: I1104fea739977c522938790012f4f3f522f9d65f Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1705234 Reviewed-by: Caroline Tice <cmtice@chromium.org> Tested-by: Zhizhou Yang <zhizhouy@google.com> Auto-Submit: Zhizhou Yang <zhizhouy@google.com> Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org>
2019-03-15crosperf: Feature to ignore min and max value in resultsZhizhou Yang
This patch provides an option for user to ignore min and max value in the results when generating report. User can use this feature by specifying `ignore_min_max: True` (which by default is False) in experiment file. When values count in a single test is smaller than 3, we automatically ignore this option for it, and print out a warning. BUG=chromium:938758 TEST=Tested with examples in general/cwp mode report generation. Passed all unit tests. Change-Id: I36a4c4d99836c201cdd2f2f9f2a4b1a4ffdaa47d Reviewed-on: https://chromium-review.googlesource.com/1521054 Commit-Ready: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Zhizhou Yang <zhizhouy@google.com>
2019-03-14crosperf: Add unittest for disable ASLR CLTiancong Wang
In crrev.com/c/1398522, we disabled ASLR in crosperf by default. In that CL, there's no unittest support and other crosperf unittest might fail because of the CL. This patch is to fix the errors in unittest and also add unittests for the newly added functionalities. BUG=chromium:930332 TEST=Unittests pass Change-Id: I332604245ea2ee1100a498ce021530572e7609ba Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1497036 Reviewed-by: Caroline Tice <cmtice@chromium.org> Tested-by: Tiancong Wang <tcwang@google.com>
2018-11-16crosperf: Passing dso and weight to result_cache.pyZhizhou Yang
To generate and collect dso sample count, this patch passes dso and weight into result_cache.py, by setting them as members of Benchmark object. Meantime, fixed all related unit tests. BUG=chromium:902785 TEST=Passed all unit tests and sample tests locally. Change-Id: I483a00d199d48fd5e77d882d7ac81bf93b6940df Reviewed-on: https://chromium-review.googlesource.com/1338344 Commit-Ready: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Caroline Tice <cmtice@chromium.org>
2017-04-08[toolchain-utils] Fix remaining lint errors in toolchain-utils.Caroline Tice
In addition to fixing the lint errors, this also fixes the Python formatting issues (ran tc_pyformat on nearly all the files). BUG=chromium:570450 TEST=Ran all crosperf & bisect tool unit tests. Ran afe_lock_machine.py (check machine status) Ran full crosperf test (octane, speedometer, BootPerf) on alex. Change-Id: Ic86f9192801ac67769f3de30f1c5f0d203ce0831 Reviewed-on: https://chromium-review.googlesource.com/471886 Commit-Ready: Caroline Tice <cmtice@chromium.org> Tested-by: Caroline Tice <cmtice@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2016-09-13crosperf+cros_utils: Swap to #!/usr/bin/env.George Burgess IV
`cros lint` was recently updated to complain about using #!/usr/bin/python{,2,3} directly. Instead, it prefers /usr/bin/env python{2,3} now. BUG=None TEST=./run_tests.sh passes; linter seems much happier with crosperf. Change-Id: I9b88a7af1f8e03b7c870781c3d8dc1b872a3dfc3 Reviewed-on: https://chrome-internal-review.googlesource.com/286341 Commit-Ready: Luis Lozano <llozano@chromium.org> Tested-by: Luis Lozano <llozano@chromium.org> Reviewed-by: Luis Lozano <llozano@chromium.org>
2016-04-14[crosperf] Fix more lint errors.Caroline Tice
BUG=chromium:567921 Test=Tested in my directory. Change-Id: Ibe128dd030887e0372380fb8f95a5e37ee63feeb Reviewed-on: https://chrome-internal-review.googlesource.com/255303 Commit-Ready: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com> Reviewed-by: Yunlian Jiang <yunlian@google.com>
2016-02-01toolchain-utils: remove uses of deprecated-module 'optparse'.Rahul Chaudhry
BUG=chromium:570458 TEST='crosperf/run_tests.sh' passes. Change-Id: I0e239a91abad72920c0d5e6e70b43683ecccb868 Reviewed-on: https://chrome-internal-review.googlesource.com/246640 Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com> Tested-by: Rahul Chaudhry <rahulchaudhry@google.com> Reviewed-by: Luis Lozano <llozano@chromium.org>
2015-12-17crosperf: fix lint warnings.Yunlian Jiang
BUG=chromium:570454 TEST=crosperf_unittest.py is lint clean. Change-Id: Ie33b95ec0cda61749b021e863ee65a47b491bb48 Reviewed-on: https://chrome-internal-review.googlesource.com/242520 Commit-Ready: Yunlian Jiang <yunlian@google.com> Tested-by: Yunlian Jiang <yunlian@google.com> Reviewed-by: Caroline Tice <cmtice@google.com>
2015-12-16Run pyformat on all the toolchain-utils files.Luis Lozano
This gets rid of a lot of lint issues. Ran by doing this: for f in *.py; do echo -n "$f " ; if [ -x $f ]; then pyformat -i --remove_trailing_comma --yapf --force_quote_type=double $f ; else pyformat -i --remove_shebang --remove_trailing_comma --yapf --force_quote_type=double $f ; fi ; done BUG=chromium:567921 TEST=Ran simple crosperf run. Change-Id: I59778835fdaa5f706d2e1765924389f9e97433d1 Reviewed-on: https://chrome-internal-review.googlesource.com/242031 Reviewed-by: Luis Lozano <llozano@chromium.org> Commit-Queue: Luis Lozano <llozano@chromium.org> Tested-by: Luis Lozano <llozano@chromium.org> Reviewed-by: Yunlian Jiang <yunlian@google.com>
2015-12-11Crosperf: Fix many broken unittests.Caroline Tice
Apparently people have been making changes and not running the unittests; many of the unittests were suffering from bit-rot. This CL fixes most of the broken unittests (there are still 3 large ones being worked on -- that will be in another CL). BUG=chromium:567883 TEST=Verified that these unittests work now. Change-Id: I0252a276a035894e70e04c61339cdba148c9bbfd Reviewed-on: https://chrome-internal-review.googlesource.com/241513 Commit-Ready: Luis Lozano <llozano@chromium.org> Tested-by: Luis Lozano <llozano@chromium.org> Reviewed-by: Luis Lozano <llozano@chromium.org>
2015-06-24Make crosperf unit tests executable.cmtice
BUG=None TEST=None Change-Id: I77e305e4e8711c86116838fd77fcc93fac2c7dc1 Reviewed-on: https://chrome-internal-review.googlesource.com/219922 Reviewed-by: Caroline Tice <cmtice@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2015-04-27Fix various unittest failures.cmtice
Over the past few months some of the unittests have not kept up with code changes and have stopped working. This CL fixes that. BUG=None TEST=Ran all the unittests by hand. Change-Id: I4fdd09846ae92ce2d45ec478a39ff3ece30b4579 Reviewed-on: https://chrome-internal-review.googlesource.com/214134 Reviewed-by: Luis Lozano <llozano@chromium.org> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-07-03Adding/updating a few more unittests for Crosperf.cmtice
BUG=None Test=Ran all regression tests. Ran 'normal' crosperf. Change-Id: If1dcafee39f10c20c935908c466fbae8fdb23a35 Reviewed-on: https://chrome-internal-review.googlesource.com/166485 Reviewed-by: Han Shen <shenhan@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
generated by cgit v1.2.3 (git 2.25.1) at 2024-07-11 06:46:18 +0000