aboutsummaryrefslogtreecommitdiff
path: root/crosperf
AgeCommit message (Collapse)Author
2019-05-28crosperf: Stop setting cpu governor.Manoj Gupta
Performance governor makes data noisy for high-end machines while powersave makes speedometer2 timeout. Let's try with the defaults for now and decide a better strategy later. BUG=chromium:966514 TEST=unit tests pass. Change-Id: I12e0f1ce7a0eed52f07cd3e1272d4280b93ef3b0 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1632422 Tested-by: Manoj Gupta <manojgupta@chromium.org> Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org> Commit-Queue: Manoj Gupta <manojgupta@chromium.org>
2019-05-24crosperf: Set governor to powersave.Manoj Gupta
This is to improve stability in performance data. BUG=chromium:966514 TEST=powersave governor works. Change-Id: I0bb0f8349c69437f154bcdac29fa64d2664fb6ef Reviewed-on: https://chromium-review.googlesource.com/1627301 Commit-Ready: Manoj Gupta <manojgupta@chromium.org> Tested-by: Manoj Gupta <manojgupta@chromium.org> Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org>
2019-05-22crosperf: Add check core online in PinGovernerFrequency().Tiancong Wang
Due to Hyper-Threading is disabled, not all cpus in /sys/devices/system/cpu/cpu* are online on the x86 DUTs. Our PinGovernerFrequency() functions need to write performance to scaling_governor and it fails if the cpu is offline. This patches adds checks before writing to make sure the core is not offline. BUG=None TEST=Local tests. Change-Id: I5dee6851848c18ff9caac0fb5bf5a6a18741a3c8 Reviewed-on: https://chromium-review.googlesource.com/1618095 Commit-Ready: Tiancong Wang <tcwang@google.com> Tested-by: Tiancong Wang <tcwang@google.com> Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2019-04-25crosperf: Use --fast to prevent copying sysinfo for telemetry_CrosperfZhizhou Yang
We don't need sysinfo collected by perf from DUT. --fast will stop copying sysinfo but was orignially disabled for profiling runs because of a conflict in autotest profiler. Now that we don't use autotest profiler any more, this patch will turn on --fast for telemetry_Crosperf no matter profiler_args are present or not. BUG=chromium:956174 TEST=Passed unit tests; Tested both with profiler_args and without, checked the sysinfo directory and it was empty. Change-Id: Id99db48d6ae9043309c5113d0a6d9afd6e3c095b Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1583018 Reviewed-by: Luis Lozano <llozano@chromium.org> Tested-by: Zhizhou Yang <zhizhouy@google.com>
2019-04-18crosperf: generate perf report with correct debug filesZhizhou Yang
This patch fixes the issue in chromium:946588. This patch makes perf report no longer use hard code debug directories. There are several different situations: 1) When running tests on a downloaded image, it will download debug.tgz from gs, extract it to debug_files in /tmp. Options --symfs and --vmlinux will depend on this directory, and throw a warning to user that --kallsyms cannot be applied. 2) If running with downloaded image and debug.tgz could not work, then we will try to use local build, but give user a warning that it may not match real symbols well. 3) When running tests with local build, try to find debug info from /build/$board directory. Thus, this patch added a new field in label, called 'debug_path', if this is manually set in experiment file, then crosperf will directly use the location. Downloading of debug.tgz will only happen when perf_args is set in global settings. TEST=Passed all unit tests, tested with eve and sand. BUG=chromium:946588 Change-Id: I7f35d1216d912c8526d5501748f951face1273aa Reviewed-on: https://chromium-review.googlesource.com/1561780 Commit-Ready: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2019-04-08Fix 'result' variable referenced before assignment issueZhizhou Yang
This issue is caused by lacking of coverage in code. When we are parsing json result, value/values doesn't exist in its filed_dict for a key, this issue will be hitten. TEST=Eliminated the error caused by this issue. BUG=chromium:929519 Change-Id: I81e4c0cf5f92c2dca9662ba86629355b3b048089 Reviewed-on: https://chromium-review.googlesource.com/1553681 Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2019-04-05crosperf: make run_tests use our new test runnerGeorge Burgess IV
This is three small changes in one. The main attraction is no functional change. The benefits we see are: A. less cluttered output B. total wall time of test running is down from 18s to 13s on my machine, since the test runner spawns all tests in parallel The second change is that we now set PYTHONPATH for linting. Otherwise, we'll see errors about bad imports. Finally, this fixes a test. It appears that the parameters of this assertRaises function have been accidentally flipped, though that works without issue outside of the chroot. To understand why, consider that assertRaises has two forms that act completely differently: self.assertRaises(Exception, foo) Which causes self.assertRaises to run foo(), and expect that foo() raises an Exception. Additionally, it can be used like so: with self.assertRaises(Exception): foo() Which causes assertRaises to intercept any Exception that the block under it may throw. The difference between the in-chroot and out-of-chroot Python is simple: outside of the chroot, the default value for the second arg to assertRaises -- not counting `self` -- is None. Inside of the chroot, they have a special, hidden sentinel value that it defaults to. So, *outside* of the chroot, `self.assertRaises(Exception)` is equivalent to `self.assertRaises(Exception, None)`, so the latter returns a context object that tries to catch an exception. Inside of the chroot, `self.assertRaises(Exception, None)` is not equivalent to `self.assertRaises(Exception)`, and `None` is called as though it was a function. Since the removed code was `self.assertRaises(foo, None)`, outside of the chroot, we'd just be returned a context object that we dropped on the floor. Inside of the chroot, `self.assertRaises(foo, None)` would try to call `None()`, fail, then try to say `issubclass(exception_that_calling_none_raised, foo)`, but foo is a function, so `issubclass` would raise on its own. For those keeping score at home, the chroot's behavior was introduced in upstream Python in 7f71e04cb510c24be337a22350324dc8a28e9775, which landed in the 2.7.10 release. It wasn't until 2.7.11 that the revert 049060c249a83b69c4841ed37b7f4303f9ad7dd7 took effect. For more, the bug was https://bugs.python.org/issue24134 BUG=None TEST=./run_tests.sh Change-Id: I58c398caafde03242ed3ca7530bf9a819fae99e2 Reviewed-on: https://chromium-review.googlesource.com/1548399 Commit-Ready: George Burgess <gbiv@chromium.org> Tested-by: George Burgess <gbiv@chromium.org> Reviewed-by: Caroline Tice <cmtice@chromium.org>
2019-03-22crosperf: Include score into the result of speedometer2.Tiancong Wang
Add RunsPerMinute__summary to the result of speedometer2. Now it has both total execution time and overall score. BUG=None TEST=Tested locally. Change-Id: I9f20153a40ac965e0fcf55e979b52e5ab53d07cd Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1534039 Reviewed-by: Luis Lozano <llozano@chromium.org> Tested-by: Tiancong Wang <tcwang@google.com> Commit-Queue: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
2019-03-19crosperf: Fix the name of amean column for weighted resultsZhizhou Yang
This fixes the naming of amean column for cwp mode, should be "Weighted Samples Amean" other than just "Amean". BUG=None TEST=Tested with both cwp and general mode. Change-Id: I66c0cce41da4aab2cfc65e867c0e0335e32029ad Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1529231 Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Caroline Tice <cmtice@chromium.org>
2019-03-15crosperf: Feature to ignore min and max value in resultsZhizhou Yang
This patch provides an option for user to ignore min and max value in the results when generating report. User can use this feature by specifying `ignore_min_max: True` (which by default is False) in experiment file. When values count in a single test is smaller than 3, we automatically ignore this option for it, and print out a warning. BUG=chromium:938758 TEST=Tested with examples in general/cwp mode report generation. Passed all unit tests. Change-Id: I36a4c4d99836c201cdd2f2f9f2a4b1a4ffdaa47d Reviewed-on: https://chromium-review.googlesource.com/1521054 Commit-Ready: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Zhizhou Yang <zhizhouy@google.com>
2019-03-14crosperf: Add unittest for disable ASLR CLTiancong Wang
In crrev.com/c/1398522, we disabled ASLR in crosperf by default. In that CL, there's no unittest support and other crosperf unittest might fail because of the CL. This patch is to fix the errors in unittest and also add unittests for the newly added functionalities. BUG=chromium:930332 TEST=Unittests pass Change-Id: I332604245ea2ee1100a498ce021530572e7609ba Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1497036 Reviewed-by: Caroline Tice <cmtice@chromium.org> Tested-by: Tiancong Wang <tcwang@google.com>
2019-03-08Swap READMEs to markdownGeorge Burgess IV
Suggested by vapier@ in I7daf1cbe93f8809e469698d7772a6c68baf9e194 This ignores README.chromium, since I'm unsure if that's a special name that some tool somewhere relies on. I skimmed over all of these in gitiles (as you can probably tell by the 35 patch sets :) ) and fixed the issues that jumped out at me. I'm sure that some remain somewhere, but I think I got the majority of them. An attempt has been made to make all files roughly follow the rules here: https://github.com/google/styleguide/blob/gh-pages/docguide/style.md Note that not all of them have super meaningful introductions with ToCs/etc, but the rest should be relatively consistent with that guide. BUG=None TEST=Manual inspection Change-Id: I158048d972d49966e3b8ff1e07e5429928edb339 Reviewed-on: https://chromium-review.googlesource.com/1504893 Commit-Ready: George Burgess <gbiv@chromium.org> Tested-by: George Burgess <gbiv@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org>
2019-03-07crosperf: do not put key 'samples' in result.keyval when test failedZhizhou Yang
This patch fixed two issues: 1) When determining how many iterations pass/fail, we originally check if samples exists or not in result.keyval. So it always comes up with "all pass" even if test fails and samples equals to 0. 2) Stddev will become large when we include 0s in samples. After this change, the value in failed cell with be 'None', and not be calculated in stddev and avg. BUG=chromium:938582 TEST=Tested with runs with failures, for cwp and general mode. Passed all unittests. Change-Id: If14eff40631be1080f6713f2bfb631843cb2ab62 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1504632 Reviewed-by: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com>
2019-03-06crosperf: enabling result caching for cwp modeZhizhou Yang
This patch provides support for result caching when cwp_dso is specified. This patch introduced cwp_dso into ResultsCache class, so that cwp_dso info will be stored while caching. When populating from cache, samples will be re-calculated with the perf.data file that has been cached before. This patch also: 1) Fixed return value of GetSamples(). 2) Not need to pass weight to result. 3) Added unit tests. BUG=chromium:905854 TEST=Passed all unit tests; Tested with --rerun=True/False, both on cwp and general mode, to make sure cache works. Change-Id: I297fa0aaad781b4a665a4a29bb6fdb4bca0d790b Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1501458 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Reviewed-by: Caroline Tice <cmtice@chromium.org> Tested-by: Zhizhou Yang <zhizhouy@google.com>
2019-03-06crosperf: replace cpu cycles in report with samplesZhizhou Yang
What we collected from benchmark run actually are samples from perf tool, so "cpu cycles" is not a accurate name. BUG=chromium:936573 TEST=Tested with cwp and general mode on eve; Passed all unittests. Change-Id: I35533cea0987c4e1b112498cc1b0271eaab665ae Reviewed-on: https://chromium-review.googlesource.com/1495963 Commit-Ready: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Caroline Tice <cmtice@chromium.org>
2019-02-12crosperf: Disable ASLR in crosperf by defaultTiancong Wang
Disabling Address Space Layout Randomization (ASLR) can reduce much variance when measuring performance of benchmark from run to run. This CL will disable ASLR in crosperf by default and add a flag '--enable_aslr' for users to enable it for other testing purpose. BUG=chromium:930332 TEST=Tested locally for several boards and several benchmarks Change-Id: Icfbe477a99ef9a68b503e4979cf2ff47d07a128f Reviewed-on: https://chromium-review.googlesource.com/1398522 Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com> Tested-by: Tiancong Wang <tcwang@google.com> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2019-02-08crosperf: in cwp mode, first check whether iterations are the sameZhizhou Yang
In cwp mode, we ask user to set same iterations for different benchmark runs, but if user did not do so, he will spend too long running all benchmarks and then get an error message in the end. This patch checks whether iterations in experiment file are the same at the time we construct benchmark. Also the patch fixed a small test case mismatch in weight check so that all unit tests can pass. Fixed format issues from pylint. BUG=chromium:929390 TEST=All unit tests passed. Change-Id: I0eae51205e8c0fa80131c5b79c4773e88c5b0915 Reviewed-on: https://chromium-review.googlesource.com/1459854 Commit-Ready: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Caroline Tice <cmtice@chromium.org>
2019-01-24crosperf: allow weight in cwp approximation larger than 1Zhizhou Yang
This patch changed the check on the weight in cwp approximation mode. The weight should be a float no less than 0. BUG=chromium:924593 TEST=Tested with a simple experiment on eve. Change-Id: Ia5ef804b5f11e36f6b5066fba6f40855bd716d2f Reviewed-on: https://chromium-review.googlesource.com/1431272 Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Luis Lozano <llozano@chromium.org>
2018-12-29crosperf: Fix bugs and nit in report generationZhizhou Yang
Fixed two parts in source code to make unit test pass: 1) Without unit, cpu cycles still should multiply by weight 2) Do not try to access cwp_dso from experiment in BenchmarkResult object, because there is method to generate report without running experiment. One nit to change the name of TableGenerator used for CPU cycle report. TEST=passed all unit tests in crosperf BUG=chromium:902785 CQ-DEPEND=CL:1376571 Change-Id: I8308b0287d60e77542ac60f5ecd5734907e51e01 Reviewed-on: https://chromium-review.googlesource.com/1389035 Commit-Ready: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Luis Lozano <llozano@chromium.org>
2018-12-28crosperf: Generate report for cwp approximationZhizhou Yang
TODO: unit tests needed. This patch trys to generate a new type of report when cwp_dso is specified. The report will: 1) Generate a new summary table at benchmark level with weighted cpu cycle data. 2) Do not generate perf table. 3) Generate a full table with only cpu cycles in it. TEST=generated correct report with a normal run and two cwp_dso runs: one succeeded run and one with failures. BUG=chromium:902785 Change-Id: I0c62000c6690c66d50a7a4de3cfc280853b134e2 Reviewed-on: https://chromium-review.googlesource.com/1376571 Commit-Ready: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Caroline Tice <cmtice@chromium.org>
2018-12-20toolchain-utils: Add loading.desktop@@typical to nightly tests.Caroline Tice
page_cycler_v2.typical_25 is deprecated (and flaky!). The recommended replacement is loading.desktop@@typical. This CL updates our nightly performance test infrastructure to handle loading.desktop@@typical. BUG=chromium:916340 TEST=Tested with nightly run on chrotomation2. Change-Id: I6366b3a8e6a2926ad759a209cfded890e3109810 Reviewed-on: https://chromium-review.googlesource.com/1382826 Commit-Ready: Caroline Tice <cmtice@chromium.org> Tested-by: Caroline Tice <cmtice@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2018-12-11crosperf: fix cpu_cycles in result_cacheZhizhou Yang
When cwp_dso is specified, we want to introduce cpu_cycles as a key into keyvals in Result object. This patch fixes the issue that cpu_cycles should be a number with unit. BUG=chromium:913086 TEST=passes webrtc benchmark with story tag Change-Id: I166bef15eeed2f7ae414b665843c8fd1f69b2580 Reviewed-on: https://chromium-review.googlesource.com/1368750 Commit-Ready: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Caroline Tice <cmtice@chromium.org>
2018-12-06crosperf: make cwp_dso only work for run_local=FalseZhizhou Yang
We do not want to run cwp_dso experiment with run_local to be True. Note that run_local is set to True by default. BUG=chromium:902785 TEST=Passed all unit tests Change-Id: I11688f34b3c96ac91d3a75ec645bdb5430bef69e Reviewed-on: https://chromium-review.googlesource.com/1363472 Commit-Ready: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Caroline Tice <cmtice@chromium.org>
2018-11-17crosperf: collect CPU cycles for CWP approximationZhizhou Yang
This patch added functionality to collect CPU cycles in Result object for each benchmark run. It will work when cwp_dso is specified. BUG=chromium:902785 TEST=Tested locally with telemetry_Crosperf with cwp_dso Change-Id: I15e187220876084eb696ffd8f3b01304ac4b96b6 Reviewed-on: https://chromium-review.googlesource.com/1340722 Commit-Ready: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Caroline Tice <cmtice@chromium.org>
2018-11-16crosperf: Passing dso and weight to result_cache.pyZhizhou Yang
To generate and collect dso sample count, this patch passes dso and weight into result_cache.py, by setting them as members of Benchmark object. Meantime, fixed all related unit tests. BUG=chromium:902785 TEST=Passed all unit tests and sample tests locally. Change-Id: I483a00d199d48fd5e77d882d7ac81bf93b6940df Reviewed-on: https://chromium-review.googlesource.com/1338344 Commit-Ready: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Caroline Tice <cmtice@chromium.org>
2018-11-15crosperf: do not allow more than one story tags in test_argsZhizhou Yang
This patch added a check at stage of parsing benchmark, to ensure that story tag only appear once in test_args. This is because more than one story tags will cause an implicit failure when running benchmark on DUT and crosperf cannot catch it. BUG=chromium:902466 TEST=Passed unit tests and a simple test Change-Id: Ic8e4aa19928911fb4ed03529ab89471856cec05c Reviewed-on: https://chromium-review.googlesource.com/1330010 Commit-Ready: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Caroline Tice <cmtice@chromium.org>
2018-11-13crosperf: introduce cwp_dso and weight for CWP approximationZhizhou Yang
This patch does initial work to support CWP approximation in crosperf. It introduceds a global option cwp_dso, which is used to specify the type of DSO to use for approximation; an option weight in benchmark setting, which decides the weight of each benchmark in test. This patch also contains some value checks for these two options. BUG=chromium:902785 TEST=Passed all unit tests and sample tests locally. Change-Id: I565c4baf0630ce6c1b62ad0398d05a5f336aac6e Reviewed-on: https://chromium-review.googlesource.com/1323473 Commit-Ready: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Caroline Tice <cmtice@chromium.org>
2018-11-09Crosperf: Treat benchmarks with story- test_args as different benchmarksZhizhou Yang
The bug refers to https://crbug.com/902466. This patch modified the behavior of crosperf on determining duplicate benchmarks. When there are two blocks with same benchmark name in experiment file, we will append them to benchmark list first. After that, we will rename the benchmark if it has test_args: 'story-filter' or 'story-tag-filter'. The new benchmark name will be passed to experiment so that we will treat them as two different runs and get separate tables on the report. BUG=chromium:902466 TEST=Passed all up-to-date unittests, tested and verified with one experiment file containing following benchmark settings: benchmark: webrtc { iterations: 3 test_args: --story-filter=datachannel suite: telemetry_Crosperf } benchmark: webrtc { iterations: 3 test_args: --story-tag-filter=smoothness suite: telemetry_Crosperf } Change-Id: Id733273a5f9f43d149407055c9c0da3b761ddeef Reviewed-on: https://chromium-review.googlesource.com/1321415 Commit-Ready: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Caroline Tice <cmtice@chromium.org>
2018-11-07Crosperf: Update board list for crosperf unit testZhizhou Yang
This patch fixes a failed unit test in crosperf, by updating the board list for test_get_default_remotes(). BUG=chromium:902578 TEST=Passed all unit tests. Change-Id: I1d3e615228ef2fd3b52be500a5b769652551c286 Reviewed-on: https://chromium-review.googlesource.com/1321416 Commit-Ready: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Caroline Tice <cmtice@chromium.org>
2018-11-05crosperf: move all_crosbolt_perf benchmark to telemetry_crosperf suiteZhizhou Yang
This patch fixes the issue that all_crosbolt_perf cannot run correctly with crosperf. The construction of this benchmark should sit in telemetry suite. BUG=chromium:901568 TEST=ran all_crosbolt_perf benchmark with crosperf on local DUT Change-Id: I94018cd7ad8348238085270b92c5a7e44ae4bbaa Reviewed-on: https://chromium-review.googlesource.com/1316708 Commit-Ready: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Caroline Tice <cmtice@chromium.org>
2018-09-16Remove lumpy and parrot which eliminated from lab.Luis A. Lozano
BUG=none TEST=none Change-Id: I5c84f765e7a958e5d7055f850d7da8fa7203b384 Reviewed-on: https://chromium-review.googlesource.com/1227060 Commit-Ready: Luis Lozano <llozano@chromium.org> Tested-by: Luis Lozano <llozano@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2018-08-21crosperf: remove deprecated testsKuo-Hsin Yang
Remove the json grammar to parse the result for deprecated tests dromaeo.*, sunspider and and robohornet_pro. These tests are already removed from telemetry tests. BUG=chromium:867332 TEST=none Change-Id: I06e6e99b51181e30410000223aca1114641cace2 Reviewed-on: https://chromium-review.googlesource.com/1182715 Commit-Ready: Vovo Yang <vovoy@chromium.org> Tested-by: Vovo Yang <vovoy@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2018-07-06crosperf: remove smoothness.tough_webgl_casesTing-Yuan Huang
It is no longer available. TEST=none BUG=none Change-Id: I25437887ec2bd608ba90430a4753e4010ab8a2f5 Reviewed-on: https://chromium-review.googlesource.com/1128364 Commit-Ready: Ting-Yuan Huang <laszio@chromium.org> Tested-by: Ting-Yuan Huang <laszio@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Reviewed-by: Yunlian Jiang <yunlian@chromium.org>
2018-06-05update _estimated_stddev for speedometer2Yunlian Jiang
This updates _estimated_stddev for speedometer2. The value is collected by running speedometer2 on kevin and samus to get the maximum stddev/mean. BUG=chromium:849321 TEST=None Change-Id: If5efa962451a4f221eef4c33a408e88c7d3fee43 Reviewed-on: https://chromium-review.googlesource.com/1087347 Commit-Ready: Yunlian Jiang <yunlian@chromium.org> Tested-by: Yunlian Jiang <yunlian@chromium.org> Reviewed-by: Caroline Tice <cmtice@chromium.org>
2018-06-05add speedometer2 result formatYunlian Jiang
This adds the speedometer2 result format to default-telemetry-results.json. BUG=chromium:849321 TEST=run crosperf and it showed the result for speedometer2. Change-Id: I8b824b6c1739a032bb15a706cee96d1f45c444ab Reviewed-on: https://chromium-review.googlesource.com/1086292 Commit-Ready: Yunlian Jiang <yunlian@chromium.org> Tested-by: Yunlian Jiang <yunlian@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2018-06-02crosperf: add speedometer2 to toolchain perf tests.Rahul Chaudhry
BUG=None TEST=None Change-Id: I73a8893e892b2941ba1170b1606e4a17744265c5 Reviewed-on: https://chromium-review.googlesource.com/1082954 Commit-Ready: Rahul Chaudhry <rahulchaudhry@chromium.org> Tested-by: Rahul Chaudhry <rahulchaudhry@chromium.org> Reviewed-by: Luis Lozano <llozano@chromium.org>
2018-05-04crosperf: Disable turbo only if enabled.Manoj Gupta
Write to no_turbo file only if the field is set to 0. BUG=chromium:839125 TEST=No more errors Change-Id: If5de67606557c21107843837afd035cf32545475 Reviewed-on: https://chromium-review.googlesource.com/1042866 Commit-Ready: Manoj Gupta <manojgupta@chromium.org> Tested-by: Manoj Gupta <manojgupta@chromium.org> Reviewed-by: Caroline Tice <cmtice@chromium.org>
2018-05-03crosperf: Ignore error when disabling turbo.Manoj Gupta
Some boards do not permit changing tubo config /sys/devices/system/cpu/intel_pstate/no_turbo Ignore the error for now so that tests can continue. Also update some formatting suggested by tc_format. BUG=chromium:839125 TEST=No more errors. Change-Id: I3a709ea2638d783dd3a3ebf70eba8a6dbc4df53a Reviewed-on: https://chromium-review.googlesource.com/1040773 Commit-Queue: Manoj Gupta <manojgupta@chromium.org> Tested-by: Manoj Gupta <manojgupta@chromium.org> Trybot-Ready: Manoj Gupta <manojgupta@chromium.org> Reviewed-by: Luis Lozano <llozano@chromium.org>
2018-05-02toolchain-utils: Update list of toolchain boards.Manoj Gupta
Add the new boards to the list of toolchain boards so that their status can be queried. BUG=chromium:837973 TEST=None Change-Id: I9565b2d38d3e2c95f1c6b6564b148315e9d182f2 Reviewed-on: https://chromium-review.googlesource.com/1038438 Tested-by: Manoj Gupta <manojgupta@chromium.org> Reviewed-by: Caroline Tice <cmtice@chromium.org>
2018-04-10crosperf: filter result for jetstream and cros_ui_smoothnessYunlian Jiang
This CL removes most of data point for jetstream and cros_ui_smooothness benchmark from the final report of crosperf. It only keeps the summary numbers. BUG=none TEST=run crosperf, only summary data were shown for these two benchmarks. Change-Id: I4482e848700ee9555ad713a07095fe9db89f7747 Reviewed-on: https://chromium-review.googlesource.com/1003386 Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com> Tested-by: Yunlian Jiang <yunlian@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2018-04-03crosperf: remove three benchmarks from test suiteYunlian Jiang
Three benchamrks were deleted in upstream, we need to remove them from our test suite. BUG=chromium:828463 TEST=uniitest passes. Change-Id: I5c92ca86217dc4bac79a75cba82dab4a793e0b3a Reviewed-on: https://chromium-review.googlesource.com/992879 Commit-Ready: Yunlian Jiang <yunlian@chromium.org> Tested-by: Yunlian Jiang <yunlian@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2018-03-08Replacing top_25_smooth with cros_ui_smoothness in crosperfKuo-Hsin Yang
top_25_smooth was replaced in CL:903246. BUG=chromium:818100 TEST=None Change-Id: I5bfe5659b06192237b0a985abb03b99139928e17 Reviewed-on: https://chromium-review.googlesource.com/954809 Commit-Ready: Vovo Yang <vovoy@chromium.org> Tested-by: Vovo Yang <vovoy@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Reviewed-by: Chung-yih Wang <cywang@google.com>
2018-03-07crosperf: restart ui instead of reboot machine.Yunlian Jiang
We restart machine to try to get a clean state for testing. But the restart beeping is anoying, as a workaround, we restart ui instead of reboot machine. BUG=None TEST=unittest passes. Change-Id: Idec234e86af661213206f954eda1fb0967c74024 Reviewed-on: https://chromium-review.googlesource.com/953864 Commit-Ready: Yunlian Jiang <yunlian@chromium.org> Tested-by: Yunlian Jiang <yunlian@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2018-01-29default_remotes: remove daisy devices that are out of service.Rahul Chaudhry
BUG=chromium:806365 TEST=None Change-Id: I5dc6b13ff184bb95dad8a4f37014d906a09a74dd Reviewed-on: https://chromium-review.googlesource.com/891661 Commit-Ready: Rahul Chaudhry <rahulchaudhry@chromium.org> Tested-by: Rahul Chaudhry <rahulchaudhry@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2017-11-30Remove droameo benchmarks from predefined sets.Luis A. Lozano
They were renamed (grouped) in chromium:777664 but, instead of using the new name, we are going to just remove them from the set. CWP data indicates these benchmarks are not representative. BUG=None TEST=None Change-Id: I034961d4f98eda0cb6ab7e706ac7309afbf23f69 Reviewed-on: https://chromium-review.googlesource.com/802714 Commit-Ready: Luis Lozano <llozano@chromium.org> Tested-by: Luis Lozano <llozano@chromium.org> Reviewed-by: Ting-Yuan Huang <laszio@chromium.org>
2017-10-31Remove startup.cold.blank_page as Chrome perf team removed it.Dean Liao
startup.cold.blank_page was removed by Chrome perf team: https://chromium-review.googlesource.com/c/chromium/src/+/644107 https://bugs.chromium.org/p/chromium/issues/detail?id=575762&desc=2 BUG=chromium:779484 TEST=./run_tests.sh Change-Id: Ib4cd3c2eaed2f496b8e25ab286919895adaa7571 Reviewed-on: https://chromium-review.googlesource.com/742778 Commit-Ready: Shuo-Peng Liao <deanliao@chromium.org> Tested-by: Shuo-Peng Liao <deanliao@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2017-10-05crosperf: Add new keys used in page cycler benchmarks.Manoj Gupta
croperf wasn't able to generate reports for page cycler since the names of tracked keys have changed. BUG=chromium:770802 TEST=Crosperf shows the results again. Change-Id: I02561bf577a342d9a1f26ece19b011edbdf0776f Reviewed-on: https://chromium-review.googlesource.com/701260 Commit-Ready: Manoj Gupta <manojgupta@chromium.org> Tested-by: Manoj Gupta <manojgupta@chromium.org> Reviewed-by: Yunlian Jiang <yunlian@chromium.org>
2017-09-29crosperf: Fix experiment_factory_unittest.Manoj Gupta
Remove x86-alex to fix crosperf unit tests. BUG=None TEST=crosperf unit tests pass. Change-Id: Ia494bb201272416b728cdd2c11d9bca712530b11 Reviewed-on: https://chromium-review.googlesource.com/693004 Commit-Ready: Manoj Gupta <manojgupta@chromium.org> Tested-by: Manoj Gupta <manojgupta@chromium.org> Reviewed-by: Yunlian Jiang <yunlian@chromium.org>
2017-09-07crosperf: remove x86-alex machines from default_remotesYunlian Jiang
We are no longer using x86-alex machines, so we can remove them from default_remotes. This also fixes the nightly job Checking lab machines. BUG=None TEST=None Change-Id: I22e252e63b9f6f1cf46b1d5bb83c13af71fd8382 Reviewed-on: https://chromium-review.googlesource.com/655481 Commit-Ready: Yunlian Jiang <yunlian@chromium.org> Tested-by: Yunlian Jiang <yunlian@chromium.org> Reviewed-by: Caroline Tice <cmtice@chromium.org>
2017-07-27crosperf: remove duplicated benchmark in all_crosbolt_perfYunlian Jiang
This removes the duplicated benchmark video_PlaybackPerf.h264 in all_crosbolt_perf BUG=None TEST=the duplicated test was removed. Change-Id: Id4822fc1b82986cba2847b2e32ef825e90969c58 Reviewed-on: https://chromium-review.googlesource.com/590359 Commit-Ready: Yunlian Jiang <yunlian@chromium.org> Tested-by: Yunlian Jiang <yunlian@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
generated by cgit v1.2.3 (git 2.25.1) at 2024-07-25 20:12:49 +0000