Age | Commit message (Collapse) | Author |
|
BUG=b:153872557
TEST=None
Change-Id: Ibc9f16d253ee9bd113fc654846462e647dc0594c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2327953
Tested-by: Tiancong Wang <tcwang@google.com>
Reviewed-by: George Burgess <gbiv@chromium.org>
Commit-Queue: George Burgess <gbiv@chromium.org>
Auto-Submit: Tiancong Wang <tcwang@google.com>
|
|
Copy cpuinfo and top logs into results directory.
BUG=None
TEST=tested on local DUT.
Change-Id: Ic868894d681db4a27f78e378945e1e565d4a5cc7
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2255007
Reviewed-by: Zhizhou Yang <zhizhouy@google.com>
Commit-Queue: Denis Nikitin <denik@chromium.org>
Tested-by: Denis Nikitin <denik@chromium.org>
|
|
Top statistics was showing commands which could combined multiple
processes. To include PID in topstats we need to split the commands into
separate processes.
The process PID is appended to the command name in topstats. The top
chrome process is the renderer process running the benchmark.
The list size depends on the number of non-chrome processes which we
limit to 5. For example with 10 chrome processes in the top with 10 following
non-chrome processes the list will show 15 entries.
BUG=None
TEST=Tested on eve, bob and cheza.
Change-Id: Ibf1e61c8cb522aba13cd51a590bb7e24597f66a6
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2227626
Reviewed-by: George Burgess <gbiv@chromium.org>
Tested-by: Denis Nikitin <denik@chromium.org>
|
|
Hasty CLs are hasty. What can I say? ¯\_(ツ)_/¯ Zhizhou says we want to
'succeed' in this case with a log message, though.
BUG=chromium:1079048
TEST=Unittests
Change-Id: I70ea71dae42d7bee069d7311e14059061c5fa0a9
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2220215
Reviewed-by: George Burgess <gbiv@chromium.org>
Tested-by: George Burgess <gbiv@chromium.org>
|
|
This consumes potentially tons of disk. Also we'll raise in this
situation from now on, since it never seems to be what we want.
BUG=chromium:1079048
TEST=Unittests
Change-Id: Ie1933b5ff0b939777d7e1d6d35dd5a29039ca07a
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2220291
Reviewed-by: Zhizhou Yang <zhizhouy@google.com>
Tested-by: George Burgess <gbiv@chromium.org>
|
|
Tast results directory has different name from test_that results,
causing a bug when compress_results option is on. This CL fixes it.
BUG=chromium:1079048
TEST=Tested with tast and test_that benchmarks in crosperf
Change-Id: I06e6615c748aeeb2fc4576f4e1bcfdad93526962
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2218541
Reviewed-by: George Burgess <gbiv@chromium.org>
Commit-Queue: Zhizhou Yang <zhizhouy@google.com>
Tested-by: Zhizhou Yang <zhizhouy@google.com>
|
|
Perf data and report file can occupy large disk space when we copy and
store them in the crosperf results directory. This patch introduces a
new field for crosperf so that we create a tarball for it to save space.
By default compress_result is set to True.
BUG=chromium:1079048
TEST=Passed unittest, tested with simple examples.
Change-Id: I70db6f3f70dc33bacfccd8833a4327c494b4d6b5
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2212432
Reviewed-by: George Burgess <gbiv@chromium.org>
Commit-Queue: Zhizhou Yang <zhizhouy@google.com>
Tested-by: Zhizhou Yang <zhizhouy@google.com>
|
|
Remove experiment arguments reducing performance noise from the
buildbot_test_toolchains script and make the values default in
experiment settings.
Add descriptions about default values.
BUG=None
TEST=Tested on local DUT.
Change-Id: I5fa3d5bf36a2ca37da713b42b49c1f2ad4f08c98
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2197549
Tested-by: Denis Nikitin <denik@chromium.org>
Reviewed-by: Zhizhou Yang <zhizhouy@google.com>
|
|
BUG=chromium:1077841
TEST=tested manually with Crosperf & veyron_minnie & it worked.
Change-Id: Ied842f9109ea81d751c2fc5d676d547ef5122271
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2180299
Reviewed-by: Bob Haarman <inglorion@chromium.org>
Commit-Queue: Caroline Tice <cmtice@chromium.org>
Tested-by: Caroline Tice <cmtice@chromium.org>
|
|
This is a format string; without this, --help raises
ValueError: unsupported format character ',' (0x2c) at index 311
BUG=None
TEST=crosperf --help
Change-Id: I5fbdb996888156f8a1531c2b140c3f734be7edff
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2173547
Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
Tested-by: George Burgess <gbiv@chromium.org>
|
|
Adding the tast test platform.ReportDiskUsage when using
'all_toolchain_perf' benchmark set. This will make toolchain nightly
test launch this tast test together with other telemetry benchmarks and
show image size changes.
BUG=chromium:1071937
TEST=Run experiment with all_toolchain_perf
Change-Id: If05893e4ad129f29276a14893024641d03ec1b95
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2161773
Reviewed-by: George Burgess <gbiv@chromium.org>
Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
Commit-Queue: Zhizhou Yang <zhizhouy@google.com>
Tested-by: Zhizhou Yang <zhizhouy@google.com>
|
|
Crosperf used to only support autotest server/client tests, and was
limiting us from running and collecting tast benchmarks.
This patch introduces a simple support for crosperf to run tast. Note
that currently we do not support passing arguments or running tast with
local changes. It basically launches `tast run <ip> <tast_test>` and
collects chart json result and parses it.
BUG=chromium:1071937
TEST=tested with tast.platform.ReportDiskUsage
Change-Id: I1980a48c4ca8e02548e5313d32b8c5ddcf5e3e60
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2160128
Commit-Queue: Zhizhou Yang <zhizhouy@google.com>
Tested-by: Zhizhou Yang <zhizhouy@google.com>
Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
Reviewed-by: George Burgess <gbiv@chromium.org>
|
|
Crosperf always returns 0 no matter benchmarks fail or experiment
interrupted in the middle. Thus we cannot tell if a run succeeded or not
with the exit status of it and it makes our nightly test failures hard
to find.
In this patch, I changed the behavior of crosperf return value:
1) Crosperf will not generate any report or send email if terminated or
all benchmarks fail. It raises RuntimeError stating all benchmarks
failing in the end.
2) Crosperf will generate report if benchmarks (but not all) fail, and
will raise RuntimeError stating benchmarks partially failing.
3) Crosperf will also copy results json files to local results directory
for further info.
BUG=chromium:1063703
TEST=Passed all unittests, tested with different failure situations.
Change-Id: I998bad51cd7301b9451645d22e8734963bc01aed
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2119231
Reviewed-by: Caroline Tice <cmtice@chromium.org>
Commit-Queue: Zhizhou Yang <zhizhouy@google.com>
Tested-by: Zhizhou Yang <zhizhouy@google.com>
Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
|
|
Currently due to crbug.com/1057755, graphics_WebGLAquarium will fail and
take 1 hour to timeout. We do not want to waste 2 * 5 iterations which
means 10 hours in our nightly test everyday waiting for it to fail.
Will re-enable it when the bug is fixed.
BUG=chromium:1057755
TEST=None
Change-Id: I8da4d60b19e95d00be202f206d0d581bc426adf4
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2088536
Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
Commit-Queue: Zhizhou Yang <zhizhouy@google.com>
Tested-by: Zhizhou Yang <zhizhouy@google.com>
Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
|
|
We have enabled --py3 for cros lint and thus reveals some new linting
errors in migrated scripts. This patch fixes them and some other random
errors.
BUG=chromium:1011676
TEST=Passed unittests.
Change-Id: If129e01c21845e1b944a2b64e50e9fed7138c845
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2055972
Tested-by: Zhizhou Yang <zhizhouy@google.com>
Reviewed-by: George Burgess <gbiv@chromium.org>
Commit-Queue: Zhizhou Yang <zhizhouy@google.com>
Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
|
|
With client tests wrapped up by crosperf_Wrapper server tests, we need a
categorizing for all tests lunched by suite runner.
BUG=chromium:984790
TEST=Passed unittest and simple experiment.
Change-Id: Ib7dc848cf5c18c3d79db059163ed1217fd7d318f
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1977040
Reviewed-by: Caroline Tice <cmtice@chromium.org>
Commit-Queue: Zhizhou Yang <zhizhouy@google.com>
Tested-by: Zhizhou Yang <zhizhouy@google.com>
Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
|
|
This patch migrates crosperf and its utils to python 3.
TEST=Passed presubmit check; tested with simple experiment locally.
BUG=chromium:1011676
Change-Id: Ib2a9f9c7cf6a1bb1d0b42a1dd3d9e3cbb4d70a36
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2003796
Tested-by: Zhizhou Yang <zhizhouy@google.com>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Caroline Tice <cmtice@chromium.org>
Commit-Queue: Zhizhou Yang <zhizhouy@google.com>
Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
|
|
This patch removes afe locking mode for crosperf, since:
1) All lab machines has been migrated to skylab.
2) Interaction with afe depends on python 2 code in autotest.
BUG=chromium:1044764
TEST=Passed unittest and simple experiment run.
Change-Id: Ie0a250548d739f32bbc08619f637380104df6c28
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2016008
Commit-Queue: Zhizhou Yang <zhizhouy@google.com>
Tested-by: Zhizhou Yang <zhizhouy@google.com>
Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
Reviewed-by: George Burgess <gbiv@chromium.org>
|
|
This is the first step of python 3 migration for crosperf
TEST=Passed presubmit for all .py files in crosperf
BUG=chromium:1011676
Change-Id: I615d0323c81ba688c8ce2aa0cac307035d0b882b
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2001696
Commit-Queue: Zhizhou Yang <zhizhouy@google.com>
Tested-by: Zhizhou Yang <zhizhouy@google.com>
Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
Reviewed-by: George Burgess <gbiv@chromium.org>
|
|
Current command generated from suite_runner for telemetry_Crosperf run
is not correct, and this patch will fix it. Also added an unittest to
verify it in the future.
It is unfortunate that we currently still cannot run test with args
successfully in skylab, but according to the args passed to skylab, this
change should work.
TEST=Checked command line to launch skylab create-test.
BUG=None
Change-Id: I05f0f4c60e04d19e2a2637192023829d4dd9f48b
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1963355
Reviewed-by: George Burgess <gbiv@chromium.org>
Commit-Queue: Zhizhou Yang <zhizhouy@google.com>
Tested-by: Zhizhou Yang <zhizhouy@google.com>
Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
|
|
We should not image DUT with certain image when the test is going to be
launched via 'skylab create-test'. This is because the DUT may be
occupied by other tasks before Crosperf acutally get it assigned, and
the first step after the test starts is to flash image.
This patch is a temproray solution to ensure no bad behavior happens on
the DUT before testing.
TEST=Tested wth skylab run, w/o caching.
BUG=chromium:984790
Change-Id: Ie2f439d5777813905bdca5f4a2d1e401df2dc83e
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1952172
Tested-by: Zhizhou Yang <zhizhouy@google.com>
Reviewed-by: Caroline Tice <cmtice@chromium.org>
Commit-Queue: Zhizhou Yang <zhizhouy@google.com>
Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
|
|
This is a patch working together with crrev.com/c/1949606 to migrate
device setup functions from suite_runner to autotest telmetry_Crosperf.
In this patch, we modified the logic of different suite types along with
skylab mode for suite_runner.
In result_cache, we handle the wait time log file to accumulate cool
down time per machine.
We also move intel_pstate kernel updating code into SetupDevice() since
it doesn't increase overhead by checking every iteration.
TEST=Passed all unittests; tested with different benchmark modes (Note
that currently skylab server tests cannot be tested because of
regression: crbug.com/984103).
BUG=chromium:1020655
Cq-Depend: chromium:1949606
Change-Id: If8e5099d096d2568d4f54584a8fcfd3c0f99c8f8
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1949626
Reviewed-by: George Burgess <gbiv@chromium.org>
Reviewed-by: Denis Nikitin <denik@chromium.org>
Tested-by: Zhizhou Yang <zhizhouy@google.com>
Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
Commit-Queue: Denis Nikitin <denik@chromium.org>
|
|
Rendering.desktop benchmark which was added into crosperf does not
return results in avg_surface_fps field. Until recently it worked fine.
There is an alternative field in histograms.json "exp_avg_surface_fps"
holding the data.
Add exp_avg_surface_fps in default-telemetry-results.json to filter-in
the benchmark field in crosperf results.
BUG=chromium:1030475
TEST=crosperf with rendering.desktop returns data on chell and lulu
Change-Id: Ia1d25f7f1894ca559f1e14e032fcf274dc4df49d
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1951336
Reviewed-by: George Burgess <gbiv@chromium.org>
Tested-by: Denis Nikitin <denik@chromium.org>
|
|
This patch moves enable_aslr and turbostat into dut_config dictionary,
since they are all device setup steps, and it make code cleaner and
easier to migrate to telemetry_Crosperf.
TEST=Passed unittests.
BUG=chromium:1020655
Change-Id: I29a649e76591e206efc19bb2b8a8df9aead16575
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1947903
Reviewed-by: Denis Nikitin <denik@chromium.org>
Commit-Queue: Zhizhou Yang <zhizhouy@google.com>
Tested-by: Zhizhou Yang <zhizhouy@google.com>
Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
|
|
There was a bug when crosperf failed due to StopUI failure.
This happened when in a preceding run crosperf exits with an exception
in DeviceSetup (for example Keyboard interrupt) and fails to call
StartUI.
Current change fixes this problem and includes a unittest case testing
the exception case.
BUG=None
TEST=unittest and HW test on DUT with stopped ui pass
Change-Id: Id6c69aebefe21b12ec7ee3a7c7f9dff92d143908
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1941036
Commit-Queue: Denis Nikitin <denik@chromium.org>
Tested-by: Denis Nikitin <denik@chromium.org>
Reviewed-by: Zhizhou Yang <zhizhouy@google.com>
|
|
Crosperf provides a mode to use `skylab create-test` to launch tests
when user specify 'skylab: true' in experiment file.
Since Skylab has been updated in these months, the command line itself
changes. This patch fixes it:
1) bb mode becomes deprecated, do not use it.
2) Task id becomes a recipe id, we need to parse it instead of swarming.
3) Real result location can be found in the json output of `wait-task`
from key "child-result".
TEST=Passed unittest, tested with simple experiment file.
BUG=chromium:984790
Change-Id: Iee114b1e6c623bb64a0ca784df9c2ba6d95e3d07
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1916542
Reviewed-by: Zhizhou Yang <zhizhouy@google.com>
Reviewed-by: Caroline Tice <cmtice@chromium.org>
Tested-by: Zhizhou Yang <zhizhouy@google.com>
Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
|
|
After locking mechanism changed, crosperf checks if locks_dir is set or
not and by default uses X20 location. This requires prodaccess thus
unittests will fail without prodaccess if locks_dir is not explicitly
set. This patch set all experiment files in unittests to use /tmp for
locks_dir.
Also fixed some lint error for deprecated functions.
TEST=Passed all unittest.
BUG=None
Change-Id: Iff4623aad9632842010e5efb85a1e75a38bcf73e
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1912767
Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
Commit-Queue: Zhizhou Yang <zhizhouy@google.com>
Tested-by: Zhizhou Yang <zhizhouy@google.com>
|
|
xbuddy location has changed to chromite.
So adjust crosperf accordingly.
BUG=b:144248787
TEST=translate_xbuddy works.
Change-Id: Id7b7542d485f906fdcad7918f916b42286ce961a
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1911381
Tested-by: Manoj Gupta <manojgupta@chromium.org>
Reviewed-by: Luis Lozano <llozano@chromium.org>
|
|
This patch extracts all the device setup code which interacts with DUT
to a single utils file, and be put into a wrapper class. This will help
migrating all related code to telemetry_Crosperf for skylab runs.
BUG=chromium:1020655
TEST=Passed all unittests; tested with simple experiment on kevin.
Change-Id: I2edcd7bb2d8cd0255d3ae6d380a5983c24427d98
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1895500
Tested-by: Zhizhou Yang <zhizhouy@google.com>
Reviewed-by: Zhizhou Yang <zhizhouy@google.com>
Reviewed-by: Denis Nikitin <denik@chromium.org>
Commit-Queue: Zhizhou Yang <zhizhouy@google.com>
|
|
"Restart UI" procedure is changed.
DUT configuration including cooldown in SuiteRunner is now performed
while UI is off. Start UI after DUT configuration is done.
The change showed decrease of cooldown waiting time by a factor of 10.
Added Total experiment duration time in the report along with total
cooldown wait time.
Cooldown wait time is now printed per machine (with parallel execution
on multiple machines single cooldown time could be misleading).
To collect data per machine cooldown_time attribute is moved from
SuiteRunner to CrosMachine class.
BUG=chromium:966514
TEST=unittests and HW tests on kevin64 passed.
Change-Id: Ib3e145cd986ca7242d97b0aeeb7795ba066753fc
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1897236
Tested-by: Denis Nikitin <denik@chromium.org>
Reviewed-by: George Burgess <gbiv@chromium.org>
|
|
This is a fix to a bug introduced by
https://chromium-review.googlesource.com/1865958. GetXbuddyPath()
function is also called in experiment_file and it's for re-writing the
fields to output. So at that time, we don't need to really check whether
downloading debug again.
BUG=chromium:1010294
TEST=tested with kevin locally.
Change-Id: Ic7daf6d1242f012bdef803faca355df0365f2cdf
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1895979
Tested-by: Zhizhou Yang <zhizhouy@google.com>
Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org>
Reviewed-by: Denis Nikitin <denik@chromium.org>
|
|
This patch introduces a new option --download_debug to crosperf.
We will only download debug symbols when this option is set to True.
This will save space when perf test users doesn't need it.
TEST=passed unittests.
BUG=chromium:1010294
Change-Id: Ic80958377f1c25cb04e3b7b56b40fafcada6d2d8
Reviewed-on: https://chromium-review.googlesource.com/1865958
Tested-by: Zhizhou Yang <zhizhouy@google.com>
Commit-Ready: Zhizhou Yang <zhizhouy@google.com>
Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org>
Reviewed-by: Denis Nikitin <denik@chromium.org>
|
|
Redirect top statistics from benchmark runs into a separate file
topstats.log under results_dir directory.
Fix "highest 5" usages to show highest usages of a command (instead of
a process) per snapshot.
Improve mechanism of calculation chrome high CPU load when benchmark is
running.
Add Cooldown wait time into email report.
Fix minor cros lint warnings to unblock repo upload.
BUG=chromium:966514
TEST=unittests and HW tests on eve passed.
Change-Id: I3999efd554cb5a3b27a2ce3fddb2f20714b434fd
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1856818
Tested-by: Denis Nikitin <denik@chromium.org>
Reviewed-by: George Burgess <gbiv@chromium.org>
|
|
After introducing the new locking mechanism, the behavior of
AFELockManager totally changed. This file changed the naming of all
related code in toochain-utils.
This patch also changed the behavior of locks_dir option, deprecated the
use_file_lock option explicitly.
TEST=Tested with different DUT types.
BUG=chromium:1006434
Change-Id: Ib15efce54ec4d4c5c2a18fecca3f350248462035
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1863530
Reviewed-by: Caroline Tice <cmtice@chromium.org>
Reviewed-by: Denis Nikitin <denik@chromium.org>
Commit-Queue: Zhizhou Yang <zhizhouy@google.com>
Tested-by: Zhizhou Yang <zhizhouy@google.com>
|
|
Most recently, the checkout location of depot_tools in chromeos has been
changed, which will break Crosperf at image downloading. This patch
fixes it.
BUG=chromium:1012904
TEST=Passed unittests.
Change-Id: I60b8de6aa3a34faa8be06aedc8ff169da993d5a5
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1848858
Commit-Queue: Zhizhou Yang <zhizhouy@google.com>
Tested-by: Zhizhou Yang <zhizhouy@google.com>
Reviewed-by: Denis Nikitin <denik@chromium.org>
Reviewed-by: Tiancong Wang <tcwang@google.com>
Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org>
|
|
Since video autotests are already ported to Tast and stopped running,
replace crosbolt benchmark with Tast version.
BUG=chromium:953553
TEST=None
Change-Id: I09a44027ab2dc338679d90f4aa8ad362f160852e
Signed-off-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/1801410
Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org>
Reviewed-by: Zhizhou Yang <zhizhouy@google.com>
|
|
Enable top in nightly tests and print top 5 commands with high CPU usage
into log.
BUG=chromium:966514
TEST=unittests and HW tests on eve passed.
Change-Id: I0efd7f0875829c4423e998ec2fbec7436f7ae148
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1829309
Tested-by: Denis Nikitin <denik@chromium.org>
Reviewed-by: George Burgess <gbiv@chromium.org>
|
|
We used to have local server for AFE locking mechanism to lock non-lab
machines. But this local server was not supported since long time ago.
This patch removes all local server related code from AFE locking.
Note that there will be another CLs to deal with issues in
http://crbug.com/1006434#c1
TEST=Passed crosperf local tests with different type of machines.
BUG=chromium:1006434
Change-Id: I47dbca73edd3e493496fd13ba0b3de705dc11513
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1822029
Reviewed-by: Caroline Tice <cmtice@chromium.org>
Commit-Queue: Zhizhou Yang <zhizhouy@google.com>
Tested-by: Zhizhou Yang <zhizhouy@google.com>
|
|
Result organizer with CWP has a different control flow and didn't
include cpu frequency and temperature data.
Current change adds missing data into the Full table of the report
file.
BUG=None
TEST=tested on veyron with cwp experiment file.
Change-Id: I67447ab016cee8265c1937b236cc5c38330576e9
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1827918
Tested-by: Denis Nikitin <denik@chromium.org>
Reviewed-by: Zhizhou Yang <zhizhouy@google.com>
|
|
There is situation that in cwp mode, test runs correctly but not samples
are collected. For this case, we want to mark the run as failure so that
we could have correct speedup calculation and user can notice it.
BUG=chromium:927554
TEST=Passed unit tests
Change-Id: I4d4f9bd25159e8d3d0b14a84f7f566593e1438ac
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1594288
Tested-by: Zhizhou Yang <zhizhouy@google.com>
Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org>
Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
Commit-Queue: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
|
|
Fixed python lint warnings in crosperf scripts.
BUG=chromium:993035
TEST=`repo upload` works w/o --no-verify
Change-Id: I3dd7b0ea956458e57e8bd77c19a5b8cb13e1d350
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1821449
Tested-by: Denis Nikitin <denik@chromium.org>
Reviewed-by: George Burgess <gbiv@chromium.org>
|
|
Add support to collecting results based on story tags in
histograms. This allows us to remove the workaround for loading.desktop
and deprecate chartjson.
BUG=chromium:1000497
TEST=local tests.
Change-Id: I8249d00b3d2d5d761c8a3bc24ef2fd95a3e1115b
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1783837
Tested-by: Jian Cai <jiancai@google.com>
Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org>
Commit-Queue: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
Reviewed-by: Zhizhou Yang <zhizhouy@google.com>
Reviewed-by: Tiancong Wang <tcwang@google.com>
|
|
In this patch, we modified the behavior of crosperf locking system. We
no longer need to specify whether to use file locking or afe locking,
instead locking methods are all automatically detected.
We also introduced a new leasing mechanism for skylab duts.
Now, if a dut:
- in afe: use afe lock on it
- in skylab: lease via skylab command
- local: use file lock on it
When crosperf finishes or dies, we will also try to unlock those locked
by us.
There will be another CL to change the naming of afe_lcok to general
locks and remove unused afe local server.
BUG=chromium:984790
TEST=Tested crosperf with DUTs in Skylab quota pool, afe and locally.
Change-Id: I0a9bf4a16f54cfa0e8af077765fbe5a0a39e7c88
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1793911
Reviewed-by: Caroline Tice <cmtice@chromium.org>
Tested-by: Zhizhou Yang <zhizhouy@google.com>
Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org>
Commit-Queue: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
|
|
Added the following optional argument in the global settings:
"top_interval" - Run top command in the background of a benchmark with
interval of sampling specified in seconds (float type).
With zero value don't run top (default value).
Top output is stored in <resultsdir>/top.log.
BUG=chromium:966514
TEST=unittests and HW tests on eve, kevin64, scarlet passed.
Change-Id: I87b187c85912140fc0c16e224580b60061c8bd94
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1817121
Tested-by: Denis Nikitin <denik@chromium.org>
Reviewed-by: George Burgess <gbiv@chromium.org>
|
|
Added the flag to ignore command error when requesting thermal data from
DUT.
It appeared that some devices (like veyron-minnie) have sensors with not
readable temperature.
BUG=None
TEST=added unittest case and ran HW test on local veyron-minnie.
Change-Id: I1edb12fce7a784a31f6fb93267c942d308fb2891
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1808021
Tested-by: Denis Nikitin <denik@chromium.org>
Reviewed-by: George Burgess <gbiv@chromium.org>
|
|
Added "cpu_freq_pct" in global settings specifying percent of the max
frequency which will be set on DUT during benchmark run. The actual
value will be chosen from available frequencies less than or equal to
the calculated number.
Feature requires scaling_available_frequencies. On the platforms not
supporting scaling_available_frequencies "cpu_freq_pct" will be ignored.
BUG=chromium:966514
TEST=Unittest and local HW tests on veyron, scarlet and kevin64 passed.
Change-Id: I175e2836ecf02fcf6b93d196926c7801d7f016c0
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1803780
Reviewed-by: George Burgess <gbiv@chromium.org>
Tested-by: Denis Nikitin <denik@chromium.org>
Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org>
|
|
Added mock patches for time.time() and time.sleep()
in experiment_runner_unittest.py.
Unittest execution time dropped from 12s to <1s.
BUG=chromium:989571
TEST=cd <path-to-crosperf>; ./run_tests.sh
Change-Id: I60af5305cd022b9a5854c444cfa9c50cbe0a22c0
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1801423
Tested-by: Denis Nikitin <denik@chromium.org>
Reviewed-by: Zhizhou Yang <zhizhouy@google.com>
Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org>
|
|
Based on dut_config['cooldown_temp'] and dut_config['cooldown_time']
wait for CPU to cooldown to a temperature 'cooldown_temp' or until
timeout 'cooldown_time' expires.
While waiting change CPU governor to a power saving mode.
With 'cooldown_time' = 0 don't wait.
Setup CPU governor based on dut_config['governor'] for a benchmark run.
Additionally function helper DutWrapper is added in SuiteRunner
which simplifies remote DUT calls.
BUG=chromium:966514
TEST=Unitest and local HW tests passed.
Change-Id: I2b946b570e7a8d8c4e2321f8bb8de3be2c3162d0
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1790131
Tested-by: Denis Nikitin <denik@chromium.org>
Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org>
Reviewed-by: George Burgess <gbiv@chromium.org>
|
|
Based on dut_config['cpu_usage'] setup DUT to use
big or little cores only on ARM platform.
Ignore 'cpu_usage' if setup is invalid for targeted platform.
For example 'big_only" on Intel or ARM w/o big/little support.
BUG=chromium:966514
TEST=Unitest and local HW tests passed.
Cq-Depend: chromium:1778515
Change-Id: Ida457a1ceaf20c4dcbb8d40334a2423836999574
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1783849
Reviewed-by: Caroline Tice <cmtice@chromium.org>
Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org>
Tested-by: Denis Nikitin <denik@chromium.org>
|
|
In global settings added optional arguments:
"cooldown_time" - wait time prior running a benchmark (default: 0),
"cooldown_temp" - temperature threshold for waiting (default: 40),
"governor" - CPU governor (default: performance),
"cpu_usage" - Restrict CPU usage to specific configurations (default: all).
"turbostat" argument is moved from benchmark to global settings.
Current CL does not apply configurations.
Instead it just propagates parameters to SuiteRunner class
through "dut_config" dictionary.
BUG=chromium:966514
TEST=Unitest and local HW tests passed.
Change-Id: I1b2a65883e5176fdde49c9858ebe62a097df89cb
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1778515
Tested-by: Denis Nikitin <denik@chromium.org>
Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org>
|