aboutsummaryrefslogtreecommitdiff
path: root/crosperf
AgeCommit message (Collapse)Author
2020-12-08default_remotes: fix a typoJian Cai
Change "tigger" to "tiger". Also update a unit test. BUG=chromium:1156409 TEST=None. Change-Id: I2dd394ccdfa79632e637cbefddf559e95296c8c1 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2580398 Reviewed-by: Tiancong Wang <tcwang@google.com> Tested-by: Jian Cai <jiancai@google.com>
2020-12-04default_remotes: Update toolchain DUT namesJian Cai
Replace veyron_minnie with a newer model as it reaches end of life. BUG=b:170402899 TEST=None Change-Id: I0caf06b84b465dcc88a3fdd9dc7d19f13fcd11b9 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2574734 Reviewed-by: Tiancong Wang <tcwang@google.com> Tested-by: Jian Cai <jiancai@google.com>
2020-11-09crosperf: Fix BadChecksum failure on coralDenis Nikitin
Ignore "core id", "apicid", "initial apicid" fields from cpuinfo in machine checksum calculation. The values may differ on the same type of machines. Add more descriptive output for the BadChecksum error. Unittest is updated accordingly. BUG=chromium:1145386 TEST=./run_tests.sh in crosperf. Change-Id: Ifcc91fb70f02c41d77787fbb665741bc130152c0 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2523398 Tested-by: Denis Nikitin <denik@chromium.org> Reviewed-by: Bob Haarman <inglorion@chromium.org> Commit-Queue: Denis Nikitin <denik@chromium.org>
2020-10-13crosperf: refactor chrome src searchDenis Nikitin
Replace multiple levels of if/else of the chrome src search with a loop. Add a missing search path .cache/distfiles/chrome-src. Add checks for the Chrome source location and raise RuntimeError if it does not exist. BUG=None TEST=Tested with crosperf and distfiles/target/chrome-src-internal. Change-Id: I66dd7c802a4e80238e4d625586198bdf573621dc Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2453595 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Reviewed-by: Caroline Tice <cmtice@chromium.org> Tested-by: Denis Nikitin <denik@chromium.org> Commit-Queue: George Burgess <gbiv@chromium.org>
2020-10-04crosperf: Update path search for chrome sourcesDenis Nikitin
Chrome source location has recently changed and path search in crosperf is updated accordingly. BUG=None TEST=run crosperf after repo sync and ebuild prepare of chrome. Change-Id: I715b3a9bf4892a993181b0be5864de94f651afe2 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2446895 Commit-Queue: Denis Nikitin <denik@chromium.org> Commit-Queue: Manoj Gupta <manojgupta@chromium.org> Tested-by: Denis Nikitin <denik@chromium.org> Auto-Submit: Denis Nikitin <denik@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2020-09-25fix formatting/lint issues pointed out by repohooksBob Haarman
Previous changes resulted in some complaints about formatting and Python 3 compatibility from the repo hooks. This change fixes those. BUG=None TEST=repo upload --cbr . # check that it no longer complains Change-Id: I99cc51dcb8d499d59b7b47817f4cef8fa6ba5059 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2419831 Tested-by: Bob Haarman <inglorion@chromium.org> Reviewed-by: Manoj Gupta (OoO) <manojgupta@chromium.org>
2020-09-25rephrase some language in code and commentsBob Haarman
This rephrases some code and comments to avoid unclear and non-inclusive wording. For the remaining terms in unblocked_terms.txt, comments are added to explain why these terms are present in the code. BUG=chromium:1099035 TEST=repo hooks Change-Id: I985613d811836b6fb6351eadcf3e08a3290db97f Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2419656 Tested-by: Bob Haarman <inglorion@chromium.org> Reviewed-by: Manoj Gupta (OoO) <manojgupta@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org>
2020-09-18crosperf: Add verification of per-process profilesDenis Nikitin
Verification is very helpful when we record per-process perf profile and collect top statistics. Using verification we can make sure that perf profiles are collected from the hottest functions in top statistics. BUG=chromium:1098098 TEST=unittests and crosperf with per-process profiling are passing. Change-Id: Ib79b070babcdea38932ba709ac3e14fa3311f842 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2407223 Reviewed-by: Tiancong Wang <tcwang@google.com> Reviewed-by: Caroline Tice <cmtice@chromium.org> Commit-Queue: Denis Nikitin <denik@chromium.org> Tested-by: Denis Nikitin <denik@chromium.org>
2020-07-31default_remotes: Update toolchain DUT namesTiancong Wang
BUG=b:153872557 TEST=None Change-Id: Ibc9f16d253ee9bd113fc654846462e647dc0594c Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2327953 Tested-by: Tiancong Wang <tcwang@google.com> Reviewed-by: George Burgess <gbiv@chromium.org> Commit-Queue: George Burgess <gbiv@chromium.org> Auto-Submit: Tiancong Wang <tcwang@google.com>
2020-06-20crosperf: Save cpuinfo and top logsDenis Nikitin
Copy cpuinfo and top logs into results directory. BUG=None TEST=tested on local DUT. Change-Id: Ic868894d681db4a27f78e378945e1e565d4a5cc7 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2255007 Reviewed-by: Zhizhou Yang <zhizhouy@google.com> Commit-Queue: Denis Nikitin <denik@chromium.org> Tested-by: Denis Nikitin <denik@chromium.org>
2020-06-05crosperf: Include PID in topstatsDenis Nikitin
Top statistics was showing commands which could combined multiple processes. To include PID in topstats we need to split the commands into separate processes. The process PID is appended to the command name in topstats. The top chrome process is the renderer process running the benchmark. The list size depends on the number of non-chrome processes which we limit to 5. For example with 10 chrome processes in the top with 10 following non-chrome processes the list will show 15 entries. BUG=None TEST=Tested on eve, bob and cheza. Change-Id: Ibf1e61c8cb522aba13cd51a590bb7e24597f66a6 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2227626 Reviewed-by: George Burgess <gbiv@chromium.org> Tested-by: Denis Nikitin <denik@chromium.org>
2020-05-28results_cache: also `return` instead of `raise`ingGeorge Burgess IV
Hasty CLs are hasty. What can I say? ¯\_(ツ)_/¯ Zhizhou says we want to 'succeed' in this case with a log message, though. BUG=chromium:1079048 TEST=Unittests Change-Id: I70ea71dae42d7bee069d7311e14059061c5fa0a9 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2220215 Reviewed-by: George Burgess <gbiv@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2020-05-28crosperf: fix a bug where we tar the world in results_cacheGeorge Burgess IV
This consumes potentially tons of disk. Also we'll raise in this situation from now on, since it never seems to be what we want. BUG=chromium:1079048 TEST=Unittests Change-Id: Ie1933b5ff0b939777d7e1d6d35dd5a29039ca07a Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2220291 Reviewed-by: Zhizhou Yang <zhizhouy@google.com> Tested-by: George Burgess <gbiv@chromium.org>
2020-05-28crosperf: fix compress dir for tast testszhizhouy
Tast results directory has different name from test_that results, causing a bug when compress_results option is on. This CL fixes it. BUG=chromium:1079048 TEST=Tested with tast and test_that benchmarks in crosperf Change-Id: I06e6615c748aeeb2fc4576f4e1bcfdad93526962 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2218541 Reviewed-by: George Burgess <gbiv@chromium.org> Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com>
2020-05-26crosperf: introduce option "compress_results" for result directoryzhizhouy
Perf data and report file can occupy large disk space when we copy and store them in the crosperf results directory. This patch introduces a new field for crosperf so that we create a tarball for it to save space. By default compress_result is set to True. BUG=chromium:1079048 TEST=Passed unittest, tested with simple examples. Change-Id: I70db6f3f70dc33bacfccd8833a4327c494b4d6b5 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2212432 Reviewed-by: George Burgess <gbiv@chromium.org> Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com>
2020-05-14toolchain_utils: Make DUT config in nightly crosperf defaultDenis Nikitin
Remove experiment arguments reducing performance noise from the buildbot_test_toolchains script and make the values default in experiment settings. Add descriptions about default values. BUG=None TEST=Tested on local DUT. Change-Id: I5fa3d5bf36a2ca37da713b42b49c1f2ad4f08c98 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2197549 Tested-by: Denis Nikitin <denik@chromium.org> Reviewed-by: Zhizhou Yang <zhizhouy@google.com>
2020-05-04toolchain-utils: Update the xbuddy lookup for hyphens in board names.Caroline Tice
BUG=chromium:1077841 TEST=tested manually with Crosperf & veyron_minnie & it worked. Change-Id: Ied842f9109ea81d751c2fc5d676d547ef5122271 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2180299 Reviewed-by: Bob Haarman <inglorion@chromium.org> Commit-Queue: Caroline Tice <cmtice@chromium.org> Tested-by: Caroline Tice <cmtice@chromium.org>
2020-05-01crosperf: fix --helpGeorge Burgess IV
This is a format string; without this, --help raises ValueError: unsupported format character ',' (0x2c) at index 311 BUG=None TEST=crosperf --help Change-Id: I5fbdb996888156f8a1531c2b140c3f734be7edff Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2173547 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2020-05-01toolchain-utils: Add platform.ReportDiskUsage to toolchain nightly perf testszhizhouy
Adding the tast test platform.ReportDiskUsage when using 'all_toolchain_perf' benchmark set. This will make toolchain nightly test launch this tast test together with other telemetry benchmarks and show image size changes. BUG=chromium:1071937 TEST=Run experiment with all_toolchain_perf Change-Id: If05893e4ad129f29276a14893024641d03ec1b95 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2161773 Reviewed-by: George Burgess <gbiv@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com>
2020-04-23crosperf: add support to run tast benchmarkszhizhouy
Crosperf used to only support autotest server/client tests, and was limiting us from running and collecting tast benchmarks. This patch introduces a simple support for crosperf to run tast. Note that currently we do not support passing arguments or running tast with local changes. It basically launches `tast run <ip> <tast_test>` and collects chart json result and parses it. BUG=chromium:1071937 TEST=tested with tast.platform.ReportDiskUsage Change-Id: I1980a48c4ca8e02548e5313d32b8c5ddcf5e3e60 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2160128 Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Auto-Submit: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: George Burgess <gbiv@chromium.org>
2020-03-26crosperf: raise error at exit when benchmarks fail to runzhizhouy
Crosperf always returns 0 no matter benchmarks fail or experiment interrupted in the middle. Thus we cannot tell if a run succeeded or not with the exit status of it and it makes our nightly test failures hard to find. In this patch, I changed the behavior of crosperf return value: 1) Crosperf will not generate any report or send email if terminated or all benchmarks fail. It raises RuntimeError stating all benchmarks failing in the end. 2) Crosperf will generate report if benchmarks (but not all) fail, and will raise RuntimeError stating benchmarks partially failing. 3) Crosperf will also copy results json files to local results directory for further info. BUG=chromium:1063703 TEST=Passed all unittests, tested with different failure situations. Change-Id: I998bad51cd7301b9451645d22e8734963bc01aed Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2119231 Reviewed-by: Caroline Tice <cmtice@chromium.org> Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
2020-03-05crosperf: do not include graphics_WebGLAquarium in all_toolchain_perfZhizhou Yang
Currently due to crbug.com/1057755, graphics_WebGLAquarium will fail and take 1 hour to timeout. We do not want to waste 2 * 5 iterations which means 10 hours in our nightly test everyday waiting for it to fail. Will re-enable it when the bug is fixed. BUG=chromium:1057755 TEST=None Change-Id: I8da4d60b19e95d00be202f206d0d581bc426adf4 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2088536 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
2020-02-14toolchain-utils: fix cros lint and some random errorsZhizhou Yang
We have enabled --py3 for cros lint and thus reveals some new linting errors in migrated scripts. This patch fixes them and some other random errors. BUG=chromium:1011676 TEST=Passed unittests. Change-Id: If129e01c21845e1b944a2b64e50e9fed7138c845 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2055972 Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: George Burgess <gbiv@chromium.org> Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
2020-02-07crosperf: introduce new test_that and skylab run for suite_runnerZhizhou Yang
With client tests wrapped up by crosperf_Wrapper server tests, we need a categorizing for all tests lunched by suite runner. BUG=chromium:984790 TEST=Passed unittest and simple experiment. Change-Id: Ib7dc848cf5c18c3d79db059163ed1217fd7d318f Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1977040 Reviewed-by: Caroline Tice <cmtice@chromium.org> Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
2020-01-30crosperf: migration to python 3Zhizhou Yang
This patch migrates crosperf and its utils to python 3. TEST=Passed presubmit check; tested with simple experiment locally. BUG=chromium:1011676 Change-Id: Ib2a9f9c7cf6a1bb1d0b42a1dd3d9e3cbb4d70a36 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2003796 Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Mike Frysinger <vapier@chromium.org> Reviewed-by: Caroline Tice <cmtice@chromium.org> Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
2020-01-24crosperf: remove AFE server interactionsZhizhou Yang
This patch removes afe locking mode for crosperf, since: 1) All lab machines has been migrated to skylab. 2) Interaction with afe depends on python 2 code in autotest. BUG=chromium:1044764 TEST=Passed unittest and simple experiment run. Change-Id: Ie0a250548d739f32bbc08619f637380104df6c28 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2016008 Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Auto-Submit: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: George Burgess <gbiv@chromium.org>
2020-01-16crosperf: fix pylint for crosperfZhizhou Yang
This is the first step of python 3 migration for crosperf TEST=Passed presubmit for all .py files in crosperf BUG=chromium:1011676 Change-Id: I615d0323c81ba688c8ce2aa0cac307035d0b882b Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2001696 Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Auto-Submit: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: George Burgess <gbiv@chromium.org>
2019-12-14crosperf: fix skylab telemetry_Crosperf tests launching commandZhizhou Yang
Current command generated from suite_runner for telemetry_Crosperf run is not correct, and this patch will fix it. Also added an unittest to verify it in the future. It is unfortunate that we currently still cannot run test with args successfully in skylab, but according to the args passed to skylab, this change should work. TEST=Checked command line to launch skylab create-test. BUG=None Change-Id: I05f0f4c60e04d19e2a2637192023829d4dd9f48b Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1963355 Reviewed-by: George Burgess <gbiv@chromium.org> Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
2019-12-12Crosperf: Do not image DUT in Skylab modeZhizhou Yang
We should not image DUT with certain image when the test is going to be launched via 'skylab create-test'. This is because the DUT may be occupied by other tasks before Crosperf acutally get it assigned, and the first step after the test starts is to flash image. This patch is a temproray solution to ensure no bad behavior happens on the DUT before testing. TEST=Tested wth skylab run, w/o caching. BUG=chromium:984790 Change-Id: Ie2f439d5777813905bdca5f4a2d1e401df2dc83e Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1952172 Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Caroline Tice <cmtice@chromium.org> Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
2019-12-11crosperf: migrate device setup functions for telemetry_CrosperfZhizhou Yang
This is a patch working together with crrev.com/c/1949606 to migrate device setup functions from suite_runner to autotest telmetry_Crosperf. In this patch, we modified the logic of different suite types along with skylab mode for suite_runner. In result_cache, we handle the wait time log file to accumulate cool down time per machine. We also move intel_pstate kernel updating code into SetupDevice() since it doesn't increase overhead by checking every iteration. TEST=Passed all unittests; tested with different benchmark modes (Note that currently skylab server tests cannot be tested because of regression: crbug.com/984103). BUG=chromium:1020655 Cq-Depend: chromium:1949606 Change-Id: If8e5099d096d2568d4f54584a8fcfd3c0f99c8f8 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1949626 Reviewed-by: George Burgess <gbiv@chromium.org> Reviewed-by: Denis Nikitin <denik@chromium.org> Tested-by: Zhizhou Yang <zhizhouy@google.com> Auto-Submit: Zhizhou Yang <zhizhouy@google.com> Commit-Queue: Denis Nikitin <denik@chromium.org>
2019-12-05crosperf: Filter-in exp_avg_surface_fps in rendering.desktopDenis Nikitin
Rendering.desktop benchmark which was added into crosperf does not return results in avg_surface_fps field. Until recently it worked fine. There is an alternative field in histograms.json "exp_avg_surface_fps" holding the data. Add exp_avg_surface_fps in default-telemetry-results.json to filter-in the benchmark field in crosperf results. BUG=chromium:1030475 TEST=crosperf with rendering.desktop returns data on chell and lulu Change-Id: Ia1d25f7f1894ca559f1e14e032fcf274dc4df49d Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1951336 Reviewed-by: George Burgess <gbiv@chromium.org> Tested-by: Denis Nikitin <denik@chromium.org>
2019-12-04crosperf: move enable_aslr and turbostat option into dut_configZhizhou Yang
This patch moves enable_aslr and turbostat into dut_config dictionary, since they are all device setup steps, and it make code cleaner and easier to migrate to telemetry_Crosperf. TEST=Passed unittests. BUG=chromium:1020655 Change-Id: I29a649e76591e206efc19bb2b8a8df9aead16575 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1947903 Reviewed-by: Denis Nikitin <denik@chromium.org> Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
2019-12-03crosperf: Make StopUI/StartUI more robustDenis Nikitin
There was a bug when crosperf failed due to StopUI failure. This happened when in a preceding run crosperf exits with an exception in DeviceSetup (for example Keyboard interrupt) and fails to call StartUI. Current change fixes this problem and includes a unittest case testing the exception case. BUG=None TEST=unittest and HW test on DUT with stopped ui pass Change-Id: Id6c69aebefe21b12ec7ee3a7c7f9dff92d143908 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1941036 Commit-Queue: Denis Nikitin <denik@chromium.org> Tested-by: Denis Nikitin <denik@chromium.org> Reviewed-by: Zhizhou Yang <zhizhouy@google.com>
2019-11-25crosperf: fix the skylab launch tests modeZhizhou Yang
Crosperf provides a mode to use `skylab create-test` to launch tests when user specify 'skylab: true' in experiment file. Since Skylab has been updated in these months, the command line itself changes. This patch fixes it: 1) bb mode becomes deprecated, do not use it. 2) Task id becomes a recipe id, we need to parse it instead of swarming. 3) Real result location can be found in the json output of `wait-task` from key "child-result". TEST=Passed unittest, tested with simple experiment file. BUG=chromium:984790 Change-Id: Iee114b1e6c623bb64a0ca784df9c2ba6d95e3d07 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1916542 Reviewed-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Caroline Tice <cmtice@chromium.org> Tested-by: Zhizhou Yang <zhizhouy@google.com> Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
2019-11-13crosperf: fix unittest error without prodaccessZhizhou Yang
After locking mechanism changed, crosperf checks if locks_dir is set or not and by default uses X20 location. This requires prodaccess thus unittests will fail without prodaccess if locks_dir is not explicitly set. This patch set all experiment files in unittests to use /tmp for locks_dir. Also fixed some lint error for deprecated functions. TEST=Passed all unittest. BUG=None Change-Id: Iff4623aad9632842010e5efb85a1e75a38bcf73e Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1912767 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com>
2019-11-12crosperf: Update xbuddy location.Manoj Gupta
xbuddy location has changed to chromite. So adjust crosperf accordingly. BUG=b:144248787 TEST=translate_xbuddy works. Change-Id: Id7b7542d485f906fdcad7918f916b42286ce961a Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1911381 Tested-by: Manoj Gupta <manojgupta@chromium.org> Reviewed-by: Luis Lozano <llozano@chromium.org>
2019-11-12crosperf: migrate all device setup code to a separate utils fileZhizhou Yang
This patch extracts all the device setup code which interacts with DUT to a single utils file, and be put into a wrapper class. This will help migrating all related code to telemetry_Crosperf for skylab runs. BUG=chromium:1020655 TEST=Passed all unittests; tested with simple experiment on kevin. Change-Id: I2edcd7bb2d8cd0255d3ae6d380a5983c24427d98 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1895500 Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Denis Nikitin <denik@chromium.org> Commit-Queue: Zhizhou Yang <zhizhouy@google.com>
2019-11-05crosperf: Improve cooldown functionalityDenis Nikitin
"Restart UI" procedure is changed. DUT configuration including cooldown in SuiteRunner is now performed while UI is off. Start UI after DUT configuration is done. The change showed decrease of cooldown waiting time by a factor of 10. Added Total experiment duration time in the report along with total cooldown wait time. Cooldown wait time is now printed per machine (with parallel execution on multiple machines single cooldown time could be misleading). To collect data per machine cooldown_time attribute is moved from SuiteRunner to CrosMachine class. BUG=chromium:966514 TEST=unittests and HW tests on kevin64 passed. Change-Id: Ib3e145cd986ca7242d97b0aeeb7795ba066753fc Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1897236 Tested-by: Denis Nikitin <denik@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org>
2019-11-04crosperf: fix a download image bugZhizhou Yang
This is a fix to a bug introduced by https://chromium-review.googlesource.com/1865958. GetXbuddyPath() function is also called in experiment_file and it's for re-writing the fields to output. So at that time, we don't need to really check whether downloading debug again. BUG=chromium:1010294 TEST=tested with kevin locally. Change-Id: Ic7daf6d1242f012bdef803faca355df0365f2cdf Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1895979 Tested-by: Zhizhou Yang <zhizhouy@google.com> Auto-Submit: Zhizhou Yang <zhizhouy@google.com> Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org> Reviewed-by: Denis Nikitin <denik@chromium.org>
2019-10-27crosperf: Add option to not download debug symbolsZhizhou Yang
This patch introduces a new option --download_debug to crosperf. We will only download debug symbols when this option is set to True. This will save space when perf test users doesn't need it. TEST=passed unittests. BUG=chromium:1010294 Change-Id: Ic80958377f1c25cb04e3b7b56b40fafcada6d2d8 Reviewed-on: https://chromium-review.googlesource.com/1865958 Tested-by: Zhizhou Yang <zhizhouy@google.com> Commit-Ready: Zhizhou Yang <zhizhouy@google.com> Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org> Reviewed-by: Denis Nikitin <denik@chromium.org>
2019-10-16crosperf: Update top stats and cooldown reportDenis Nikitin
Redirect top statistics from benchmark runs into a separate file topstats.log under results_dir directory. Fix "highest 5" usages to show highest usages of a command (instead of a process) per snapshot. Improve mechanism of calculation chrome high CPU load when benchmark is running. Add Cooldown wait time into email report. Fix minor cros lint warnings to unblock repo upload. BUG=chromium:966514 TEST=unittests and HW tests on eve passed. Change-Id: I3999efd554cb5a3b27a2ce3fddb2f20714b434fd Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1856818 Tested-by: Denis Nikitin <denik@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org>
2019-10-15toolchain-utils: change naming related to AFE lockZhizhou Yang
After introducing the new locking mechanism, the behavior of AFELockManager totally changed. This file changed the naming of all related code in toochain-utils. This patch also changed the behavior of locks_dir option, deprecated the use_file_lock option explicitly. TEST=Tested with different DUT types. BUG=chromium:1006434 Change-Id: Ib15efce54ec4d4c5c2a18fecca3f350248462035 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1863530 Reviewed-by: Caroline Tice <cmtice@chromium.org> Reviewed-by: Denis Nikitin <denik@chromium.org> Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com>
2019-10-09crosperf: gsutil location changedZhizhou Yang
Most recently, the checkout location of depot_tools in chromeos has been changed, which will break Crosperf at image downloading. This patch fixes it. BUG=chromium:1012904 TEST=Passed unittests. Change-Id: I60b8de6aa3a34faa8be06aedc8ff169da993d5a5 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1848858 Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Denis Nikitin <denik@chromium.org> Reviewed-by: Tiancong Wang <tcwang@google.com> Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org>
2019-10-09crosperf: Change video crosbolt benchmark to Tast versionKeiichi Watanabe
Since video autotests are already ported to Tast and stopped running, replace crosbolt benchmark with Tast version. BUG=chromium:953553 TEST=None Change-Id: I09a44027ab2dc338679d90f4aa8ad362f160852e Signed-off-by: Keiichi Watanabe <keiichiw@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/1801410 Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com> Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org> Reviewed-by: Zhizhou Yang <zhizhouy@google.com>
2019-09-28crosperf: Print top5 commands into logDenis Nikitin
Enable top in nightly tests and print top 5 commands with high CPU usage into log. BUG=chromium:966514 TEST=unittests and HW tests on eve passed. Change-Id: I0efd7f0875829c4423e998ec2fbec7436f7ae148 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1829309 Tested-by: Denis Nikitin <denik@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org>
2019-09-27toolchain-utils: remove AFE local server codeZhizhou Yang
We used to have local server for AFE locking mechanism to lock non-lab machines. But this local server was not supported since long time ago. This patch removes all local server related code from AFE locking. Note that there will be another CLs to deal with issues in http://crbug.com/1006434#c1 TEST=Passed crosperf local tests with different type of machines. BUG=chromium:1006434 Change-Id: I47dbca73edd3e493496fd13ba0b3de705dc11513 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1822029 Reviewed-by: Caroline Tice <cmtice@chromium.org> Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com>
2019-09-26crosperf: Add cpu temp/freq in cwp reportDenis Nikitin
Result organizer with CWP has a different control flow and didn't include cpu frequency and temperature data. Current change adds missing data into the Full table of the report file. BUG=None TEST=tested on veyron with cwp experiment file. Change-Id: I67447ab016cee8265c1937b236cc5c38330576e9 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1827918 Tested-by: Denis Nikitin <denik@chromium.org> Reviewed-by: Zhizhou Yang <zhizhouy@google.com>
2019-09-26crosperf: mark a run as failed if samples collected is 0Zhizhou Yang
There is situation that in cwp mode, test runs correctly but not samples are collected. For this case, we want to mark the run as failure so that we could have correct speedup calculation and user can notice it. BUG=chromium:927554 TEST=Passed unit tests Change-Id: I4d4f9bd25159e8d3d0b14a84f7f566593e1438ac Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1594288 Tested-by: Zhizhou Yang <zhizhouy@google.com> Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Auto-Submit: Zhizhou Yang <zhizhouy@google.com> Commit-Queue: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
2019-09-24crosperf: Fix cros lint warningsDenis Nikitin
Fixed python lint warnings in crosperf scripts. BUG=chromium:993035 TEST=`repo upload` works w/o --no-verify Change-Id: I3dd7b0ea956458e57e8bd77c19a5b8cb13e1d350 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1821449 Tested-by: Denis Nikitin <denik@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org>
2019-09-24crosperf: add per tag story summary when parsing histogramsJian Cai
Add support to collecting results based on story tags in histograms. This allows us to remove the workaround for loading.desktop and deprecate chartjson. BUG=chromium:1000497 TEST=local tests. Change-Id: I8249d00b3d2d5d761c8a3bc24ef2fd95a3e1115b Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1783837 Tested-by: Jian Cai <jiancai@google.com> Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org> Commit-Queue: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com> Reviewed-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Tiancong Wang <tcwang@google.com>
generated by cgit v1.2.3 (git 2.25.1) at 2024-07-05 09:57:18 +0000