Age | Commit message (Collapse) | Author |
|
This patch migrates crosperf and its utils to python 3.
TEST=Passed presubmit check; tested with simple experiment locally.
BUG=chromium:1011676
Change-Id: Ib2a9f9c7cf6a1bb1d0b42a1dd3d9e3cbb4d70a36
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2003796
Tested-by: Zhizhou Yang <zhizhouy@google.com>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Caroline Tice <cmtice@chromium.org>
Commit-Queue: Zhizhou Yang <zhizhouy@google.com>
Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
|
|
Result organizer with CWP has a different control flow and didn't
include cpu frequency and temperature data.
Current change adds missing data into the Full table of the report
file.
BUG=None
TEST=tested on veyron with cwp experiment file.
Change-Id: I67447ab016cee8265c1937b236cc5c38330576e9
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1827918
Tested-by: Denis Nikitin <denik@chromium.org>
Reviewed-by: Zhizhou Yang <zhizhouy@google.com>
|
|
Check for cpustats.log file in results_dir and extract
data with CPU frequncy and temperature.
Calculate avg/min/max values from the measurements and
add to result keyvals.
The parser attempts to read turbostat.log first (which comes from
Intel devices along with cpustats.log) and only if it doesn't exist
or empty switches to cpustats.log (usually ARM devices).
BUG=chromium:966514
TEST=tested locally on veyron, scarlet, kevin64 (arm), eve (intel)
Change-Id: I12991b39a0f1acb6197ca663a6876b0f013d2026
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1753164
Reviewed-by: George Burgess <gbiv@chromium.org>
Tested-by: Denis Nikitin <denik@chromium.org>
|
|
Check for turbostat.log file in results directory and if file exists
parse the context and extract measurements of CPU Frequency and
temperature for ALL cores (not each).
Keys added to keyvals: 'cpufreq_min', 'cpufreq_max', 'cpufreq_avg',
'cputemp'.
BUG=chromium:966514
TEST=tested all crosbolt perf benchmarks
Change-Id: If90d26664c70ccb73750f17573b89933fdb048f4
Reviewed-on: https://chromium-review.googlesource.com/1709824
Tested-by: Denis Nikitin <denik@chromium.org>
Commit-Ready: Denis Nikitin <denik@chromium.org>
Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org>
Reviewed-by: Zhizhou Yang <zhizhouy@google.com>
Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
|
|
Pylint throws a bunch of warnings on results_cache.py and
results_organizer.py (other source files not checked).
The change fixes most warnings EXCEPT docstring which would require more
elaboration.
BUG=none
TEST=presubmit passess
Change-Id: I4e48d94c9b430d8b224e70e9267abe3795bff7cc
Reviewed-on: https://chromium-review.googlesource.com/1719770
Tested-by: Denis Nikitin <denik@chromium.org>
Commit-Ready: Denis Nikitin <denik@chromium.org>
Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org>
Reviewed-by: George Burgess <gbiv@chromium.org>
|
|
What we collected from benchmark run actually are samples from perf
tool, so "cpu cycles" is not a accurate name.
BUG=chromium:936573
TEST=Tested with cwp and general mode on eve; Passed all unittests.
Change-Id: I35533cea0987c4e1b112498cc1b0271eaab665ae
Reviewed-on: https://chromium-review.googlesource.com/1495963
Commit-Ready: Zhizhou Yang <zhizhouy@google.com>
Tested-by: Zhizhou Yang <zhizhouy@google.com>
Reviewed-by: Caroline Tice <cmtice@chromium.org>
|
|
TODO: unit tests needed.
This patch trys to generate a new type of report when cwp_dso is
specified. The report will:
1) Generate a new summary table at benchmark level with weighted cpu
cycle data.
2) Do not generate perf table.
3) Generate a full table with only cpu cycles in it.
TEST=generated correct report with a normal run and two cwp_dso runs:
one succeeded run and one with failures.
BUG=chromium:902785
Change-Id: I0c62000c6690c66d50a7a4de3cfc280853b134e2
Reviewed-on: https://chromium-review.googlesource.com/1376571
Commit-Ready: Zhizhou Yang <zhizhouy@google.com>
Tested-by: Zhizhou Yang <zhizhouy@google.com>
Reviewed-by: Caroline Tice <cmtice@chromium.org>
|
|
page_cycler_v2.typical_25 is deprecated (and flaky!). The
recommended replacement is loading.desktop@@typical. This CL updates
our nightly performance test infrastructure to handle
loading.desktop@@typical.
BUG=chromium:916340
TEST=Tested with nightly run on chrotomation2.
Change-Id: I6366b3a8e6a2926ad759a209cfded890e3109810
Reviewed-on: https://chromium-review.googlesource.com/1382826
Commit-Ready: Caroline Tice <cmtice@chromium.org>
Tested-by: Caroline Tice <cmtice@chromium.org>
Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
|
|
Sometimes telemetry tests do not fail but results are empty.
In that case, mark the run as failed explicitly.
BUG=chromium:729634
TEST=Tested manually on failing lummpy nightly tests. The generated report shows
tests as failed as expected.
Change-Id: Ia52af887e81ed630d7578f9abe93f9656c70512b
Reviewed-on: https://chromium-review.googlesource.com/526392
Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
Tested-by: Manoj Gupta <manojgupta@chromium.org>
Commit-Queue: Manoj Gupta <manojgupta@chromium.org>
Trybot-Ready: Manoj Gupta <manojgupta@chromium.org>
|
|
If a single benchmark run contains multiple iterations (like
BootPerfServer), then only the first internal iteration was getting the
correct pass/fail status, so even though everything passed the report
title would claim 1 pass and 9 fails (for 10 internal iterations).
This fixes that.
BUG=chromium:426960
TEST=Ran BootPerfServer with change. Now shows correct number of
passes/fails for the iterations.
Change-Id: I9638b39c94692fd754698e64d89cb22c10e9d479
Reviewed-on: https://chromium-review.googlesource.com/470547
Commit-Ready: Caroline Tice <cmtice@chromium.org>
Tested-by: Caroline Tice <cmtice@chromium.org>
Reviewed-by: George Burgess <gbiv@chromium.org>
Reviewed-by: Yunlian Jiang <yunlian@chromium.org>
|
|
In addition to fixing the lint errors, this also fixes the Python
formatting issues (ran tc_pyformat on nearly all the files).
BUG=chromium:570450
TEST=Ran all crosperf & bisect tool unit tests.
Ran afe_lock_machine.py (check machine status)
Ran full crosperf test (octane, speedometer, BootPerf) on alex.
Change-Id: Ic86f9192801ac67769f3de30f1c5f0d203ce0831
Reviewed-on: https://chromium-review.googlesource.com/471886
Commit-Ready: Caroline Tice <cmtice@chromium.org>
Tested-by: Caroline Tice <cmtice@chromium.org>
Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
|
|
As part of playing with results_report (so I could make it less
ChromeOS-dependent), I started refactoring results_organizer, which is
only used by results_report.
This was the result. The goal was to make it easier to understand what,
exactly, results_organizer is trying to do.
BUG=chromium:641098
TEST=./test_all.sh passes.
Change-Id: I8e53175fbc1ded6f23631550223905b3321376ca
Reviewed-on: https://chrome-internal-review.googlesource.com/280815
Commit-Ready: George Burgess <gbiv@google.com>
Tested-by: George Burgess <gbiv@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
|
|
This gets rid of a lot of lint issues.
Ran by doing this:
for f in *.py; do echo -n "$f " ; if [ -x $f ]; then pyformat -i
--remove_trailing_comma --yapf --force_quote_type=double $f ; else
pyformat -i --remove_shebang --remove_trailing_comma --yapf
--force_quote_type=double $f ; fi ; done
BUG=chromium:567921
TEST=Ran simple crosperf run.
Change-Id: I59778835fdaa5f706d2e1765924389f9e97433d1
Reviewed-on: https://chrome-internal-review.googlesource.com/242031
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
|
|
BUG=chromium:567921
TEST=it is lint clean now.
Change-Id: I0cf9445f5db822462c53d75a138e984070319714
Reviewed-on: https://chrome-internal-review.googlesource.com/241429
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
|
|
In some weird cases, telemetry benchmarks incorrectly
return empty performance results. This was causing an invalid
key error while generating the json report.
BUG=chromium:551105
TEST=Forced generation of empty results and tested we dont
have the dictionary missing key issue.
Change-Id: I32803c573f1d268e010313292b99c0017a5dff92
Reviewed-on: https://chrome-internal-review.googlesource.com/237516
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Caroline Tice <cmtice@google.com>
|
|
This CL updates the cache mechanism to store the machine spec
info (the checksum string) in the cache, as well as storing the
machine information in the json reports (to go into the dashboard
archive). It also updates the json reports to always include the
full set of result fields for a test, and it verifies that the
old cache directory exists before trying to remove it.
BUG=None
TEST=Tested in my directory, with --cache_only=True, --rerun=True and
--json_report=True
Change-Id: Ib12bae4ca10141de0b972e3a7aeee83c7b59049b
Reviewed-on: https://chrome-internal-review.googlesource.com/229765
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
|
|
Crosperf's help was completely out of date and incorrect. Fixed
that. Also, I cleaned up the options, removed some that were no
longer appropriate or not ever used, and made some options global-only.
Also removed all references to pyauto. We no longer use or run pyauto
tests.
Option changes: Completely removed outlier_range (it was not actually doing
anything), key_results_only (applied only to pyauto tests), use_test_that
(not needed now that we're not using pyauto) and md5sum (never used).
Made the following options global-only: rm_chroot_tmp, perf_args, logging_level,
show_all_results, board. Removed 'build' from global settings and made
it an image-only setting (it made no sense as a global setting).
I also adjusted the unittests to work with these changes.
BUG=None
TEST=I tested all the options in all their various positions; I tested with and
wtihout caching; and I tested local and official images. I also ran the unittests. Everything works.
Change-Id: I7653663f65c0a3363a1904c0220f1066e72c7f9d
Reviewed-on: https://chrome-internal-review.googlesource.com/160758
Reviewed-by: Bhaskar Janakiraman <bjanakiraman@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
|
|
The current implementation of the option that masks unimportant results
discards the uninteresting results before caching the test results. After
further thought, that was not the right design choice. Among other things it
can cause confusing results, such as seen in issue 357346. We should always
cache all the results, and do the result filtering during report generation.
This CL makes that change.
BUG=357346,357343
TEST=Ran crosperf tests with and without cache hits, and with and without entreies in the json file. It all seems to work as expected.
Change-Id: I778e5614c73bf751ebaa2d4606af636275247c60
Reviewed-on: https://chrome-internal-review.googlesource.com/159108
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
|
|
BUG=None
TEST=run crosperf with pyauto/telemetry test with/without cache.
all pass.
Change-Id: If07ac020a9107a79d5780a58fd6dcc924d07f07f
Reviewed-on: https://gerrit-int.chromium.org/36594
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-on: https://gerrit-int.chromium.org/39241
Reviewed-by: Yunlian Jiang <yunlian@google.com>
|
|
Got rid of stale copies of some tools like "crosperf" and moved all
files under v14 directory (that came from perforce) into the top
directory.
BUG=None
TEST=None
Change-Id: I408d17a36ceb00e74db71403d2351fd466a14f8e
Reviewed-on: https://gerrit-int.chromium.org/33887
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Luis Lozano <llozano@chromium.org>
|
|
|
|
|