aboutsummaryrefslogtreecommitdiff
path: root/crosperf/suite_runner.py
AgeCommit message (Collapse)Author
2022-10-06crosperf: find sshwatcher relative to chromeos_rootBob Haarman
The root of the ChromiumOS source tree is not always in the same location relative to where the crosperf scripts are. This makes the logic for finding sshwatcher.go not work everywhere. To fix this, this change uses the label.chromeos_root path that is present in crosperf runs to find sshwatcher.go. BUG=b:235119397 TEST=run crosperf in standalone toolchain-utils as well as inside ChromiumOS tree to see that sshwatcher.go is found in both cases. Change-Id: I981a55d3cfd2638da0bf320b86ebc5601101ce3f Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3935652 Tested-by: Bob Haarman <inglorion@chromium.org> Reviewed-by: Denis Nikitin <denik@chromium.org> Commit-Queue: Bob Haarman <inglorion@chromium.org>
2022-10-03crosperf: use sshwatcher to set up port forwarding to DUTsBob Haarman
Some DUTs (notably those on gTransit) require port forwarding to be accessible from inside the chroot. This CL makes crosperf's SuitRunner use sshwatcher to set up the required forwarding. BUG=b:235119397 TEST=suite_runner_unittest, then try it out for real Change-Id: Idad2abf40713e2d0aa3b81fdb4762a22b638400e Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3928354 Reviewed-by: Denis Nikitin <denik@chromium.org> Reviewed-by: Jordan Abrahams-Whitehead <ajordanr@google.com> Commit-Queue: Bob Haarman <inglorion@chromium.org> Tested-by: Bob Haarman <inglorion@chromium.org>
2022-09-30crosperf: fix pylint errors in suite_runner.pyBob Haarman
crosperf/suit_runner.py uses .format(), which we no longer want. This converts the code to use f-strings instead. BUG=None TEST=See that pylint no longer complains about .format() Change-Id: I7085f2da5659843d2501eb2f19137b64dff7b106 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3928353 Tested-by: Bob Haarman <inglorion@chromium.org> Reviewed-by: Jordan Abrahams-Whitehead <ajordanr@google.com> Reviewed-by: Denis Nikitin <denik@chromium.org> Commit-Queue: Bob Haarman <inglorion@chromium.org>
2022-09-13Update license boilerplate text in source code filesMike Frysinger
Normally we don't do this, but enough changes have accumulated that we're doing a tree-wide one-off update of the name & style. BUG=chromium:1098010 TEST=`repo upload` works Change-Id: Icb42e5012a87920c2cd13b666fb3e55e7e4fb3b8 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3891080 Auto-Submit: Mike Frysinger <vapier@chromium.org> Tested-by: Mike Frysinger <vapier@chromium.org> Commit-Queue: George Burgess <gbiv@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org>
2022-09-07remove `from __future__ import ...` directivesGeorge Burgess IV
These are only useful when we're running code in a Python 2.7 interpreter. Since we no longer support python2, drop these. BUG=b:244644217 TEST=run_tests_for.py shows no new failures Change-Id: Ief9a12b87a560ab38ca71668636874bcb434a0b3 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3877339 Reviewed-by: Ryan Beltran <ryanbeltran@chromium.org> Commit-Queue: George Burgess <gbiv@chromium.org> Reviewed-by: Jordan Abrahams-Whitehead <ajordanr@google.com> Tested-by: George Burgess <gbiv@chromium.org>
2022-09-07Autoformat all Python codeGeorge Burgess IV
This autoformats all Python code with our new Python formatter, `black`. BUG=b:244644217 TEST=None Change-Id: I15ee49233d98fb6295c0c53c129bbf8e78e0d9ff Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3877337 Tested-by: George Burgess <gbiv@chromium.org> Reviewed-by: Jordan Abrahams-Whitehead <ajordanr@google.com> Commit-Queue: George Burgess <gbiv@chromium.org>
2022-04-21toolchain_utils: s/Cr OS/CrOS/gGeorge Burgess IV
Result of running `sed -ri 's/Chrom(ium|e) OS/Chrom\1OS/g' $(find -type f)`. BUG=None TEST=None Change-Id: I59be92537aa19bc989f52b585e307e76dbde401b Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3600147 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Commit-Queue: George Burgess <gbiv@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2021-04-14replaces skylab with crosfleetChristopher Di Bella
BUG=chromium:1187326 TEST=Tested by hand on chrotomation.mtv Change-Id: I7e0bd2a99e85f288fb3b1b9ff52c8e38d25df245 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2773632 Reviewed-by: Caroline Tice <cmtice@chromium.org> Commit-Queue: Caroline Tice <cmtice@chromium.org> Tested-by: Caroline Tice <cmtice@chromium.org>
2020-04-23crosperf: add support to run tast benchmarkszhizhouy
Crosperf used to only support autotest server/client tests, and was limiting us from running and collecting tast benchmarks. This patch introduces a simple support for crosperf to run tast. Note that currently we do not support passing arguments or running tast with local changes. It basically launches `tast run <ip> <tast_test>` and collects chart json result and parses it. BUG=chromium:1071937 TEST=tested with tast.platform.ReportDiskUsage Change-Id: I1980a48c4ca8e02548e5313d32b8c5ddcf5e3e60 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2160128 Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Auto-Submit: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: George Burgess <gbiv@chromium.org>
2020-02-14toolchain-utils: fix cros lint and some random errorsZhizhou Yang
We have enabled --py3 for cros lint and thus reveals some new linting errors in migrated scripts. This patch fixes them and some other random errors. BUG=chromium:1011676 TEST=Passed unittests. Change-Id: If129e01c21845e1b944a2b64e50e9fed7138c845 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2055972 Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: George Burgess <gbiv@chromium.org> Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
2020-02-07crosperf: introduce new test_that and skylab run for suite_runnerZhizhou Yang
With client tests wrapped up by crosperf_Wrapper server tests, we need a categorizing for all tests lunched by suite runner. BUG=chromium:984790 TEST=Passed unittest and simple experiment. Change-Id: Ib7dc848cf5c18c3d79db059163ed1217fd7d318f Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1977040 Reviewed-by: Caroline Tice <cmtice@chromium.org> Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
2020-01-30crosperf: migration to python 3Zhizhou Yang
This patch migrates crosperf and its utils to python 3. TEST=Passed presubmit check; tested with simple experiment locally. BUG=chromium:1011676 Change-Id: Ib2a9f9c7cf6a1bb1d0b42a1dd3d9e3cbb4d70a36 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2003796 Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Mike Frysinger <vapier@chromium.org> Reviewed-by: Caroline Tice <cmtice@chromium.org> Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
2019-12-14crosperf: fix skylab telemetry_Crosperf tests launching commandZhizhou Yang
Current command generated from suite_runner for telemetry_Crosperf run is not correct, and this patch will fix it. Also added an unittest to verify it in the future. It is unfortunate that we currently still cannot run test with args successfully in skylab, but according to the args passed to skylab, this change should work. TEST=Checked command line to launch skylab create-test. BUG=None Change-Id: I05f0f4c60e04d19e2a2637192023829d4dd9f48b Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1963355 Reviewed-by: George Burgess <gbiv@chromium.org> Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
2019-12-11crosperf: migrate device setup functions for telemetry_CrosperfZhizhou Yang
This is a patch working together with crrev.com/c/1949606 to migrate device setup functions from suite_runner to autotest telmetry_Crosperf. In this patch, we modified the logic of different suite types along with skylab mode for suite_runner. In result_cache, we handle the wait time log file to accumulate cool down time per machine. We also move intel_pstate kernel updating code into SetupDevice() since it doesn't increase overhead by checking every iteration. TEST=Passed all unittests; tested with different benchmark modes (Note that currently skylab server tests cannot be tested because of regression: crbug.com/984103). BUG=chromium:1020655 Cq-Depend: chromium:1949606 Change-Id: If8e5099d096d2568d4f54584a8fcfd3c0f99c8f8 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1949626 Reviewed-by: George Burgess <gbiv@chromium.org> Reviewed-by: Denis Nikitin <denik@chromium.org> Tested-by: Zhizhou Yang <zhizhouy@google.com> Auto-Submit: Zhizhou Yang <zhizhouy@google.com> Commit-Queue: Denis Nikitin <denik@chromium.org>
2019-12-04crosperf: move enable_aslr and turbostat option into dut_configZhizhou Yang
This patch moves enable_aslr and turbostat into dut_config dictionary, since they are all device setup steps, and it make code cleaner and easier to migrate to telemetry_Crosperf. TEST=Passed unittests. BUG=chromium:1020655 Change-Id: I29a649e76591e206efc19bb2b8a8df9aead16575 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1947903 Reviewed-by: Denis Nikitin <denik@chromium.org> Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
2019-12-03crosperf: Make StopUI/StartUI more robustDenis Nikitin
There was a bug when crosperf failed due to StopUI failure. This happened when in a preceding run crosperf exits with an exception in DeviceSetup (for example Keyboard interrupt) and fails to call StartUI. Current change fixes this problem and includes a unittest case testing the exception case. BUG=None TEST=unittest and HW test on DUT with stopped ui pass Change-Id: Id6c69aebefe21b12ec7ee3a7c7f9dff92d143908 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1941036 Commit-Queue: Denis Nikitin <denik@chromium.org> Tested-by: Denis Nikitin <denik@chromium.org> Reviewed-by: Zhizhou Yang <zhizhouy@google.com>
2019-11-25crosperf: fix the skylab launch tests modeZhizhou Yang
Crosperf provides a mode to use `skylab create-test` to launch tests when user specify 'skylab: true' in experiment file. Since Skylab has been updated in these months, the command line itself changes. This patch fixes it: 1) bb mode becomes deprecated, do not use it. 2) Task id becomes a recipe id, we need to parse it instead of swarming. 3) Real result location can be found in the json output of `wait-task` from key "child-result". TEST=Passed unittest, tested with simple experiment file. BUG=chromium:984790 Change-Id: Iee114b1e6c623bb64a0ca784df9c2ba6d95e3d07 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1916542 Reviewed-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Caroline Tice <cmtice@chromium.org> Tested-by: Zhizhou Yang <zhizhouy@google.com> Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
2019-11-12crosperf: migrate all device setup code to a separate utils fileZhizhou Yang
This patch extracts all the device setup code which interacts with DUT to a single utils file, and be put into a wrapper class. This will help migrating all related code to telemetry_Crosperf for skylab runs. BUG=chromium:1020655 TEST=Passed all unittests; tested with simple experiment on kevin. Change-Id: I2edcd7bb2d8cd0255d3ae6d380a5983c24427d98 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1895500 Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Denis Nikitin <denik@chromium.org> Commit-Queue: Zhizhou Yang <zhizhouy@google.com>
2019-11-05crosperf: Improve cooldown functionalityDenis Nikitin
"Restart UI" procedure is changed. DUT configuration including cooldown in SuiteRunner is now performed while UI is off. Start UI after DUT configuration is done. The change showed decrease of cooldown waiting time by a factor of 10. Added Total experiment duration time in the report along with total cooldown wait time. Cooldown wait time is now printed per machine (with parallel execution on multiple machines single cooldown time could be misleading). To collect data per machine cooldown_time attribute is moved from SuiteRunner to CrosMachine class. BUG=chromium:966514 TEST=unittests and HW tests on kevin64 passed. Change-Id: Ib3e145cd986ca7242d97b0aeeb7795ba066753fc Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1897236 Tested-by: Denis Nikitin <denik@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org>
2019-10-09crosperf: gsutil location changedZhizhou Yang
Most recently, the checkout location of depot_tools in chromeos has been changed, which will break Crosperf at image downloading. This patch fixes it. BUG=chromium:1012904 TEST=Passed unittests. Change-Id: I60b8de6aa3a34faa8be06aedc8ff169da993d5a5 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1848858 Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Denis Nikitin <denik@chromium.org> Reviewed-by: Tiancong Wang <tcwang@google.com> Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org>
2019-09-24crosperf: Add top_interval argument to experimentDenis Nikitin
Added the following optional argument in the global settings: "top_interval" - Run top command in the background of a benchmark with interval of sampling specified in seconds (float type). With zero value don't run top (default value). Top output is stored in <resultsdir>/top.log. BUG=chromium:966514 TEST=unittests and HW tests on eve, kevin64, scarlet passed. Change-Id: I87b187c85912140fc0c16e224580b60061c8bd94 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1817121 Tested-by: Denis Nikitin <denik@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org>
2019-09-17crosperf: Fix thermal failure on veyronDenis Nikitin
Added the flag to ignore command error when requesting thermal data from DUT. It appeared that some devices (like veyron-minnie) have sensors with not readable temperature. BUG=None TEST=added unittest case and ran HW test on local veyron-minnie. Change-Id: I1edb12fce7a784a31f6fb93267c942d308fb2891 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1808021 Tested-by: Denis Nikitin <denik@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org>
2019-09-16crosperf: Setup CPU frequencyDenis Nikitin
Added "cpu_freq_pct" in global settings specifying percent of the max frequency which will be set on DUT during benchmark run. The actual value will be chosen from available frequencies less than or equal to the calculated number. Feature requires scaling_available_frequencies. On the platforms not supporting scaling_available_frequencies "cpu_freq_pct" will be ignored. BUG=chromium:966514 TEST=Unittest and local HW tests on veyron, scarlet and kevin64 passed. Change-Id: I175e2836ecf02fcf6b93d196926c7801d7f016c0 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1803780 Reviewed-by: George Burgess <gbiv@chromium.org> Tested-by: Denis Nikitin <denik@chromium.org> Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org>
2019-09-13crosperf: Wait for CPU cooldown and setup governor.Denis Nikitin
Based on dut_config['cooldown_temp'] and dut_config['cooldown_time'] wait for CPU to cooldown to a temperature 'cooldown_temp' or until timeout 'cooldown_time' expires. While waiting change CPU governor to a power saving mode. With 'cooldown_time' = 0 don't wait. Setup CPU governor based on dut_config['governor'] for a benchmark run. Additionally function helper DutWrapper is added in SuiteRunner which simplifies remote DUT calls. BUG=chromium:966514 TEST=Unitest and local HW tests passed. Change-Id: I2b946b570e7a8d8c4e2321f8bb8de3be2c3162d0 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1790131 Tested-by: Denis Nikitin <denik@chromium.org> Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org>
2019-09-10crosperf: Setup CPU configuration based on 'cpu_usage'.Denis Nikitin
Based on dut_config['cpu_usage'] setup DUT to use big or little cores only on ARM platform. Ignore 'cpu_usage' if setup is invalid for targeted platform. For example 'big_only" on Intel or ARM w/o big/little support. BUG=chromium:966514 TEST=Unitest and local HW tests passed. Cq-Depend: chromium:1778515 Change-Id: Ida457a1ceaf20c4dcbb8d40334a2423836999574 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1783849 Reviewed-by: Caroline Tice <cmtice@chromium.org> Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org> Tested-by: Denis Nikitin <denik@chromium.org>
2019-09-10crosperf: Add dut config arguments to experimentDenis Nikitin
In global settings added optional arguments: "cooldown_time" - wait time prior running a benchmark (default: 0), "cooldown_temp" - temperature threshold for waiting (default: 40), "governor" - CPU governor (default: performance), "cpu_usage" - Restrict CPU usage to specific configurations (default: all). "turbostat" argument is moved from benchmark to global settings. Current CL does not apply configurations. Instead it just propagates parameters to SuiteRunner class through "dut_config" dictionary. BUG=chromium:966514 TEST=Unitest and local HW tests passed. Change-Id: I1b2a65883e5176fdde49c9858ebe62a097df89cb Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1778515 Tested-by: Denis Nikitin <denik@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org>
2019-09-03crosperf: update skylab command lineZhizhou Yang
Recently there are some big changes to options of skylab command, this patch updates the command to the newest version. BUG=chromium:984790 TEST=Tested with different crosperf tests. Change-Id: I6ed81c00372668914f79b66c1e18d980c073593d Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1775533 Reviewed-by: Caroline Tice <cmtice@chromium.org> Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Auto-Submit: Zhizhou Yang <zhizhouy@google.com> Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org>
2019-08-02crosperf: Add "intel_pstate" and "turbostat" argsDenis Nikitin
In global setting added "intel_pstate" option. Type: string. Values: active, passive, no_hwp, ''(equivalent to active). Default ''. In benchmark setting added "turbostat" option. Type: boolean. Default True. BUG=chromium:966514 TEST=tested on samus, eve, rammus Change-Id: I1533c6d6586aab5acf3665bce68920a0ab3092bc Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1726635 Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org> Commit-Queue: Sean Abraham <seanabraham@chromium.org> Reviewed-by: Sean Abraham <seanabraham@chromium.org> Tested-by: Sean Abraham <seanabraham@chromium.org>
2019-07-22crosperf: Init workflow to use skylab to run testsZhizhou Yang
This patch provides init workflow to use skylab to run both original test_that test (aka. client test) and telemetry_Crosperf test. This patch also removes telemetry_run since no one uses it any more. This patch will: 1) Launch skylab to create test to run on specific remote(s). 2) Wait until test finishes, and download results from gs location. 3) Skipping afelock system for lab machines. This patch is missing a lot of important concepts for our final goal, marking them as TODO in the patch. TEST=passed all unittest. BUG=chromium:984790 Change-Id: I7d0d15e67e4610d42ff84aa3fe2f1f1ac90fc1bb Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1707928 Reviewed-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Caroline Tice <cmtice@chromium.org> Tested-by: Zhizhou Yang <zhizhouy@google.com> Auto-Submit: Zhizhou Yang <zhizhouy@google.com> Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org>
2019-07-10crosperf: Recover intel_pstate performance setupDenis Nikitin
Revert back performance governor intel_pstate setup. Change the order of no_turbo setup which should follow the intel_pstate governor setup. BUG=chromium:966514 TEST=verified on samus, eve, rammus, kefka. Change-Id: I40e6930bc62151cafe1a95f7f9d05c138e6d7663 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1687384 Tested-by: Denis Nikitin <denik@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org>
2019-05-28crosperf: Stop setting cpu governor.Manoj Gupta
Performance governor makes data noisy for high-end machines while powersave makes speedometer2 timeout. Let's try with the defaults for now and decide a better strategy later. BUG=chromium:966514 TEST=unit tests pass. Change-Id: I12e0f1ce7a0eed52f07cd3e1272d4280b93ef3b0 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1632422 Tested-by: Manoj Gupta <manojgupta@chromium.org> Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org> Commit-Queue: Manoj Gupta <manojgupta@chromium.org>
2019-05-24crosperf: Set governor to powersave.Manoj Gupta
This is to improve stability in performance data. BUG=chromium:966514 TEST=powersave governor works. Change-Id: I0bb0f8349c69437f154bcdac29fa64d2664fb6ef Reviewed-on: https://chromium-review.googlesource.com/1627301 Commit-Ready: Manoj Gupta <manojgupta@chromium.org> Tested-by: Manoj Gupta <manojgupta@chromium.org> Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org>
2019-05-22crosperf: Add check core online in PinGovernerFrequency().Tiancong Wang
Due to Hyper-Threading is disabled, not all cpus in /sys/devices/system/cpu/cpu* are online on the x86 DUTs. Our PinGovernerFrequency() functions need to write performance to scaling_governor and it fails if the cpu is offline. This patches adds checks before writing to make sure the core is not offline. BUG=None TEST=Local tests. Change-Id: I5dee6851848c18ff9caac0fb5bf5a6a18741a3c8 Reviewed-on: https://chromium-review.googlesource.com/1618095 Commit-Ready: Tiancong Wang <tcwang@google.com> Tested-by: Tiancong Wang <tcwang@google.com> Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2019-04-25crosperf: Use --fast to prevent copying sysinfo for telemetry_CrosperfZhizhou Yang
We don't need sysinfo collected by perf from DUT. --fast will stop copying sysinfo but was orignially disabled for profiling runs because of a conflict in autotest profiler. Now that we don't use autotest profiler any more, this patch will turn on --fast for telemetry_Crosperf no matter profiler_args are present or not. BUG=chromium:956174 TEST=Passed unit tests; Tested both with profiler_args and without, checked the sysinfo directory and it was empty. Change-Id: Id99db48d6ae9043309c5113d0a6d9afd6e3c095b Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1583018 Reviewed-by: Luis Lozano <llozano@chromium.org> Tested-by: Zhizhou Yang <zhizhouy@google.com>
2019-02-12crosperf: Disable ASLR in crosperf by defaultTiancong Wang
Disabling Address Space Layout Randomization (ASLR) can reduce much variance when measuring performance of benchmark from run to run. This CL will disable ASLR in crosperf by default and add a flag '--enable_aslr' for users to enable it for other testing purpose. BUG=chromium:930332 TEST=Tested locally for several boards and several benchmarks Change-Id: Icfbe477a99ef9a68b503e4979cf2ff47d07a128f Reviewed-on: https://chromium-review.googlesource.com/1398522 Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com> Tested-by: Tiancong Wang <tcwang@google.com> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2018-05-04crosperf: Disable turbo only if enabled.Manoj Gupta
Write to no_turbo file only if the field is set to 0. BUG=chromium:839125 TEST=No more errors Change-Id: If5de67606557c21107843837afd035cf32545475 Reviewed-on: https://chromium-review.googlesource.com/1042866 Commit-Ready: Manoj Gupta <manojgupta@chromium.org> Tested-by: Manoj Gupta <manojgupta@chromium.org> Reviewed-by: Caroline Tice <cmtice@chromium.org>
2018-05-03crosperf: Ignore error when disabling turbo.Manoj Gupta
Some boards do not permit changing tubo config /sys/devices/system/cpu/intel_pstate/no_turbo Ignore the error for now so that tests can continue. Also update some formatting suggested by tc_format. BUG=chromium:839125 TEST=No more errors. Change-Id: I3a709ea2638d783dd3a3ebf70eba8a6dbc4df53a Reviewed-on: https://chromium-review.googlesource.com/1040773 Commit-Queue: Manoj Gupta <manojgupta@chromium.org> Tested-by: Manoj Gupta <manojgupta@chromium.org> Trybot-Ready: Manoj Gupta <manojgupta@chromium.org> Reviewed-by: Luis Lozano <llozano@chromium.org>
2018-03-07crosperf: restart ui instead of reboot machine.Yunlian Jiang
We restart machine to try to get a clean state for testing. But the restart beeping is anoying, as a workaround, we restart ui instead of reboot machine. BUG=None TEST=unittest passes. Change-Id: Idec234e86af661213206f954eda1fb0967c74024 Reviewed-on: https://chromium-review.googlesource.com/953864 Commit-Ready: Yunlian Jiang <yunlian@chromium.org> Tested-by: Yunlian Jiang <yunlian@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2017-04-08[toolchain-utils] Fix remaining lint errors in toolchain-utils.Caroline Tice
In addition to fixing the lint errors, this also fixes the Python formatting issues (ran tc_pyformat on nearly all the files). BUG=chromium:570450 TEST=Ran all crosperf & bisect tool unit tests. Ran afe_lock_machine.py (check machine status) Ran full crosperf test (octane, speedometer, BootPerf) on alex. Change-Id: Ic86f9192801ac67769f3de30f1c5f0d203ce0831 Reviewed-on: https://chromium-review.googlesource.com/471886 Commit-Ready: Caroline Tice <cmtice@chromium.org> Tested-by: Caroline Tice <cmtice@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2017-03-17Set scaling governor to performance in crosperf.Manoj Gupta
Elm seems to have stable numbers when only governor is set. Disable turbo is intel pstate thermal driver is used. Tested on squawks and lumpy. Includes some minor tc_pyformat changes. BUG=chromium:700190 TEST=Tested on elm, squawks, lumpy, kevin, falco and veyron_jaq. Change-Id: Ib368234b3bde5e54ff9b53ae3766e7384bd1df5f Reviewed-on: https://chromium-review.googlesource.com/454299 Commit-Ready: Manoj Gupta <manojgupta@chromium.org> Tested-by: Manoj Gupta <manojgupta@chromium.org> Reviewed-by: Caroline Tice <cmtice@chromium.org>
2017-01-24Fix frequency pinning on crosperf.Manoj Gupta
Crosperf was pinning frequency to maximum based on cpu0 information. This does not work as intended for arm Big-Little cores. Instead do it for each cpu based on its own information. BUG=chromium:680742 TEST=Tested on elm, kevin, falco, samus, peach_pit and veyron_jaq. Change-Id: I77fc76d8f1e94563fe12b3555bf08511fdbfdc32 Reviewed-on: https://chromium-review.googlesource.com/428186 Commit-Ready: Manoj Gupta <manojgupta@chromium.org> Tested-by: Manoj Gupta <manojgupta@chromium.org> Reviewed-by: Luis Lozano <llozano@chromium.org>
2017-01-11Fix test_that run for non-telemtry tests.Manoj Gupta
Do not add the autotest_dir argument to test_that for non-telemetry tests. Minor change: Remove a leftover print from download_images unit test. BUG: chromium:679001 TEST:crosperf unit tests Change-Id: I0ff5365902af40097b61ae57d7c0d8d43ea308a8 Reviewed-on: https://chromium-review.googlesource.com/426073 Reviewed-by: Caroline Tice <cmtice@chromium.org> Commit-Queue: Manoj Gupta <manojgupta@chromium.org> Tested-by: Manoj Gupta <manojgupta@chromium.org> Trybot-Ready: Manoj Gupta <manojgupta@chromium.org>
2017-01-11Fix some lint warning in suite_runner*.Luis A. Lozano
BUG=chromium:632113 TEST=run_tests Change-Id: I9fe070420153f5ae175db86961c95b1686ea7c8f Reviewed-on: https://chromium-review.googlesource.com/421107 Reviewed-by: Caroline Tice <cmtice@chromium.org> Commit-Queue: Luis Lozano <llozano@chromium.org> Tested-by: Luis Lozano <llozano@chromium.org>
2016-11-10Fix crosperf crashes when running non-telemetry test on R53 or older images.Manoj Gupta
Do not call decrease wait time for non-telemetry tests. BUG: chromium:663435 TEST:crosperf unit tests Change-Id: Ia13eb8f258ab8fc09fdfbdb4b93a6745e06b44b1 Reviewed-on: https://chrome-internal-review.googlesource.com/303736 Commit-Ready: Manoj Gupta <manojgupta@google.com> Tested-by: Manoj Gupta <manojgupta@google.com> Reviewed-by: Caroline Tice <cmtice@google.com>
2016-11-07Download autotest files with crosperf and add autotest path to test_thatManoj Gupta
when running non-telemetry tests BUG: chromium:647429 TEST:crosperf unit tests, sample experiment file that runs WebGl_Aquarium Change-Id: I067e350fee36596ce269b954773c39a3868632aa Reviewed-on: https://chrome-internal-review.googlesource.com/302296 Commit-Ready: Manoj Gupta <manojgupta@google.com> Tested-by: Manoj Gupta <manojgupta@google.com> Reviewed-by: Caroline Tice <cmtice@google.com>
2016-11-01Update formatting of files before committing changes for autotest. No ↵Manoj Gupta
functional change. BUG: chromium:647429 TEST: crosperf unit tests Change-Id: I827f9ec4d00a0d8c1ae8a9930d7832289e694dc8 Reviewed-on: https://chrome-internal-review.googlesource.com/301457 Commit-Ready: Manoj Gupta <manojgupta@google.com> Tested-by: Manoj Gupta <manojgupta@google.com> Reviewed-by: Luis Lozano <llozano@chromium.org>
2016-10-05crosperf: fix the error unexpected keyword argument 'command'Yunlian Jiang
BUG=none TEST=the error is gone. Change-Id: I8dad69d2b45ed821be0c7be77e07b33cf0663eb9 Reviewed-on: https://chrome-internal-review.googlesource.com/293515 Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Yunlian Jiang <yunlian@google.com> Tested-by: Yunlian Jiang <yunlian@google.com>
2016-10-04crosperf: Decrease 10 seconds delay for pagecyclers.Yunlian Jiang
New telemetry adds 10 seconds delay after rendering a page. We do not need that so this CL changes the 10 secnods delay to 2 seconds. BUG=None TEST=the 10 seconds delay is changed to 2 seconds.. Change-Id: I5c835acdf4068fba1aceb0d2a5a45e7113a558d3 Reviewed-on: https://chrome-internal-review.googlesource.com/291818 Reviewed-by: Ting-Yuan Huang <laszio@google.com> Commit-Queue: Yunlian Jiang <yunlian@google.com> Tested-by: Yunlian Jiang <yunlian@google.com>
2016-09-22Revert "[crosperf] Remove --autotest_dir arg from non-telemetry tests."Caroline Tice
This reverts commit 70462462d54070a2440fccb506dcea2ea076cc05. Upon further discussion I realize this is the wrong fix for this problem. Change-Id: I7e9c94e763378e4834858acd826985c55bfcea4d Reviewed-on: https://chrome-internal-review.googlesource.com/289636 Commit-Ready: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com> Reviewed-by: Yunlian Jiang <yunlian@google.com>
2016-09-16[crosperf] Remove --autotest_dir arg from non-telemetry tests.Caroline Tice
Some autotests that are not telemetry tests will fail if we pass the --autotest_dir arg. This CL removes that arg from non-telemetry tests. BUG=chromium:647429 TEST=ran graphics_WebGLAquarium ok. Change-Id: I00c79a6dc8cc4642b5b9dd342a92df91d0cb9c2c Reviewed-on: https://chrome-internal-review.googlesource.com/287597 Commit-Ready: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com> Reviewed-by: Yunlian Jiang <yunlian@google.com>
generated by cgit v1.2.3 (git 2.25.1) at 2024-07-11 13:47:25 +0000