aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2014-07-02Add regression test for suite_runner.pycmtice
BUG=None TEST=Ran regression test. Ran normal crosperf. Change-Id: I35c5a83d691e0a2a9272422539afdc740e7828f6 Reviewed-on: https://chrome-internal-review.googlesource.com/166169 Reviewed-by: Han Shen <shenhan@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-06-24Update nightly tests (and weekly report stuff) to use official builds.cmtice
This CL changes the nightly tests to download and use the official image for our 'vanilla' image, rather than building the vanilla image ourselves. It also updates the code that creates & copies the tar files of the images, for the weekly reports, to look for and copy the official images. BUG=None TEST=Tested the changes by hand. Also let them run for a couple of nights for our nightly tests. Change-Id: I8c3215aab7550dc9f148ed79095295f702141017 Reviewed-on: https://chrome-internal-review.googlesource.com/166477 Reviewed-by: Luis Lozano <llozano@chromium.org> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-06-24Adds a second build of host packages.Han Shen
Adds a second build of host packages and do some testing, which mimics the ways chromiumos-sdk cbuildbot does. So actually we perform 2 builds of all host packages - 1. build new toolchain using old toolchain and build all host packages using the newly built toolchain 2. build the new toolchain again but using new toolchain built in step 1, and build all host packages using the newly built toolchain TEST=locally BUG=None Change-Id: Ic35092893b10860ca1112a1abb3356a279d36869 Reviewed-on: https://chrome-internal-review.googlesource.com/163325 Reviewed-by: Luis Lozano <llozano@chromium.org> Commit-Queue: Han Shen <shenhan@google.com> Tested-by: Han Shen <shenhan@google.com>
2014-06-07Add script for generating weekly comparison report.cmtice
BUG=None TEST=Ran this script multiple times from the role account, with the nightly tests. Change-Id: Ifbf723a77388908f6fe9c1b1816a20aa5c424f26 Reviewed-on: https://chrome-internal-review.googlesource.com/164978 Reviewed-by: Caroline Tice <cmtice@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-06-05Update test_toolchains.py to save tar files of the images used.cmtice
This is for generating a weekly report that compares the nightly test results. All of the nightly test results are cached, but in order to find the correct cache results, you need the image from which the results were created. Due to disk space, some of the chroots will be deleted before the weekly report gets generated. Therefore, this modification tars up the test & vanilla image directories and stores them separately, where they can be accessed for generating the weekly report. BUG=None TEST=Used this successfully last night for nightly test runs. Change-Id: I01b457a402c3649d7904a478b37bad60b3395300 Reviewed-on: https://chrome-internal-review.googlesource.com/164879 Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-06-05Update and expand machine_manager_unittest.cmtice
Expand machine_manager_unittest.py to test all functions in machine_manager.py. Modify MachineManager class to optionally take a command_executer and logger as arguments; also to make command executer and logger class member attributes (rather than external objects), to simplify testing. BUG=None TEST=Ran unittest to make sure it works; ran all the other unittests; ran several 'real' crosperf runs to verify that the changes didn't break crosperf. Change-Id: I29798457cb1f7572cb339722b572b9d95b21b989 Reviewed-on: https://chrome-internal-review.googlesource.com/165256 Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-06-02Add --cache_only global setting to crosperf.cmtice
The CL adds a new setting, --cache_only, which causes crosperf to ONLY report test results that it finds in the cache (it will NOT rerun any tests, even if no cache results are found). This is partly for implementing a weekly summary report, where we want to compare the nightly results from the past week, but not actually rerun anything. BUG=None TEST=I tested this option while generating weekly test reports, and it worked as it is supposed to. Change-Id: Icb1f5343f76ce999bebd0810bb86ab64a7577313 Reviewed-on: https://chrome-internal-review.googlesource.com/164890 Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-05-31Fix problems found when running benchmark_run_unittest.cmtice
When the changes to benchmark_run_unittest got committed, somehow the changes to logger.py did not get committed (adding Mock functions to logger). Also, MockMachineManger.AcquireMachine needs an extra parameter, to match change to MachineManager.AcquireMachine. This CL fixes both of those problems. BUG=None TEST=Ran unittests successfully after these changes. Change-Id: I7ae35241ee62ddb484728ffc5c26ea14bc1f7c7f Reviewed-on: https://chrome-internal-review.googlesource.com/164889 Reviewed-by: Yunlian Jiang <yunlian@google.com> Tested-by: Caroline Tice <cmtice@google.com> Commit-Queue: Caroline Tice <cmtice@google.com>
2014-05-26Rename chromite.buildbot -> chromite.cbuildbot.Don Garrett
BUG=None TEST=None CQ-DEPEND=CL:199664 Change-Id: I6b2b119960cf4368a5cbc594523b4d44407e91f9 Reviewed-on: https://chrome-internal-review.googlesource.com/163786 Reviewed-by: Don Garrett <dgarrett@google.com> Commit-Queue: Don Garrett <dgarrett@google.com> Tested-by: Don Garrett <dgarrett@google.com>
2014-05-21Fix bug, parameter name masked module name.cmtice
New 'logger' parameter name in __init__ function masked module name. This CL renames the parameter to something else. BUG=None TEST=Ran crosperf with the fix and it worked. Change-Id: I19b5e452d50ce9937662ec3ee36ebb79747985f5 Reviewed-on: https://chrome-internal-review.googlesource.com/164135 Reviewed-by: Luis Lozano <llozano@chromium.org> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-05-21Better handling of "not identical machines" failure.cmtice
Nightly tests, especially on the x86-generic boxes, fail frequently because the DUTs have different board images on them, so Crosperf decides the machines are not identical and refuses to run the tests. With this CL, if the machine_manager finds that the machines fail the "identical" test, it will try to push the same image onto all the machines, and then check them again to see if they are the same. It only tries this once; if they fail the check the second time around, it is still a fatal failure. This should eliminate many of the unnecessary failures in our nightly tests. This CL also fixes a small bug in the auto-delete script (this fix has been running for a while on mobiletc-prebuild, but never got committed). BUG=None TEST=Ran several iterations where I forced the first "identical" check to fail. The changes worked. Change-Id: Ied2a55e5d3e2789e58a503aef03269888954b579 Reviewed-on: https://chrome-internal-review.googlesource.com/163334 Reviewed-by: Luis Lozano <llozano@chromium.org> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-05-21Add --skip_chroot_upgrade flag to build_packages.cmtice
This CL adds the '--skip_chroot_upgrade' flag to all calls to build_packages that clobber CFLAGS, LDFLAGS, etc. This is to prevent the call to build_packages from deciding to try to rebuild the compiler itself (which is not the right place to do that). BUG=None TEST=Tested adding flag by hand to call to build_packages that used to fail; it passed with the flag. Change-Id: Ia0152f76f8a25cf0488a4a72c4add7e4b3182feb Reviewed-on: https://chrome-internal-review.googlesource.com/164050 Reviewed-by: Han Shen <shenhan@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-05-20Add unit test for experiment_runner.pycmtice
BUG=None TEST=Ran unittest. Change-Id: I13b2822f57f4201c94215c603a60f12755230f71 Reviewed-on: https://chrome-internal-review.googlesource.com/163833 Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-05-14Update & complete benchmark_run_unittest.pycmtice
Add tests for all the methods in BenchmarkRun. BUG=Non TEST=Ran unittest. Change-Id: I9046f010ce0e48ba8d897585fec2f95732df9cd3 Reviewed-on: https://chrome-internal-review.googlesource.com/162988 Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-05-12Expand experiment_factory_unittest.py to be more complete.cmtice
This CL adds more testing to experiment_factory_unittest.py, and it fixes one small bug in experiment_factory.py. BUG=None TEST=Ran unittest. Change-Id: I9fd13c05d793bb71accf3e19d361dff79f7110e9 Reviewed-on: https://chrome-internal-review.googlesource.com/163149 Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-05-08Correctly test failures while building boards.Han Shen
TEST=None BUG=None Change-Id: Iaae09d30214d6f8bb8b069d5fc4009a20cb0e6d8 Reviewed-on: https://chrome-internal-review.googlesource.com/162912 Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Han Shen <shenhan@google.com> Tested-by: Han Shen <shenhan@google.com>
2014-05-08Add 3 unittests, for test_flag.py, config.py and benchmark.pycmtice
BUG=None TEST=Ran unittests. Change-Id: I2c421f8322e58173932656da9ae7a5af1bf3febd Reviewed-on: https://chrome-internal-review.googlesource.com/162886 Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-05-08Add unittesting for download_images.py (and translate_xbuddy.py).cmtice
This re-structures the code in download_images.py, for easier unit testing, and adds unit tests for download_images. The download_images_buildid_test.py tests translate_xbuddy.py as well. BUG=None TEST=Ran crosperf tests for official & trybot images with new download_images.py file; Ran both unittest files. Change-Id: Ib6ea17d5bd8ed8c87c403a7b1eef49237e5414b8 Reviewed-on: https://chrome-internal-review.googlesource.com/162866 Reviewed-by: Caroline Tice <cmtice@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-05-03Add support to build mutiple boards for binutils/gcc.Han Shen
Now you can specify multiple boards in one command line like this ./build_tool.py --board='x86-alex,lumpy' TEST=None BUG=None Change-Id: Idac899bb225ce51eab922f5ab0b08b36ce7b0e20 Reviewed-on: https://chrome-internal-review.googlesource.com/161923 Reviewed-by: Luis Lozano <llozano@chromium.org> Commit-Queue: Han Shen <shenhan@google.com> Tested-by: Han Shen <shenhan@google.com>
2014-04-30Fix bug to now allow mounting chrome_src from inside chroot.cmtice
Crosperf was checking to see if chrome_src was inside the chroot, and if so was not mounting it. This patch fixes that problem. BUG=None TEST=Tested with my new chroot (which doesn't have chrome source in 'normal' place). Change-Id: Ide3273ca73aecffa28fa8d19a53f300eb957c29e Reviewed-on: https://chrome-internal-review.googlesource.com/162009 Reviewed-by: Luis Lozano <llozano@chromium.org> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-04-29Fix small bugs with non-autotest Telemetry; other cleanups.cmtice
This CL fixes various small problems found during recent testing: - Force show_all_results to be true for 'pure' telemetry case (the results filtering needs the autotest results to work properly) - Fix bug that was inserting extra 'src' into Chrome src path sometimes - Since non-autotest Telemetry and non-Telemetry-Crosperf tests can't handle 'perf' make it a Fatal error to call them with perf args. - Add missing field to example experiment file. - Add status output messages to downlaod_images.py for non-verbose logging. BUG=None TEST=Tested all these fixes on multiple test runs. Change-Id: If8209356c695dc8b21f8627399a1cbd4a858df23 Reviewed-on: https://chrome-internal-review.googlesource.com/161759 Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-04-25Abort deleting the directory if deletion of any of its chromeos subtree fails.Han Shen
TEST=None BUG=None Change-Id: I4dfd2f67d8bc7f6f329b58800c148a923e78eace Reviewed-on: https://chrome-internal-review.googlesource.com/161678 Reviewed-by: Luis Lozano <llozano@chromium.org> Commit-Queue: Han Shen <shenhan@google.com> Tested-by: Han Shen <shenhan@google.com>
2014-04-24Tell users where results files are stored.cmtice
This updates the output messages to actually tell users where the results files are stored, especially the perf.data and perf.report files. BUG=None TEST=Ran Crosperf with these changes. Change-Id: I4030d8fbf47c0a313e0ab349da9c4d201f946d11 Reviewed-on: https://chrome-internal-review.googlesource.com/161486 Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-04-24Add necessary output to auto_delete_nightly_test_data script and return with ↵Han Shen
meaning exit code. The script did not have any output in case of a success run, which means we do not get emails about this cron job. Also the exit code was always 0. Fixed by adding print statements and return correct exit code. TEST=None BUG=None Change-Id: If06a4d57a94387494f9c1b30fea765f66e6e0f56 Reviewed-on: https://chrome-internal-review.googlesource.com/161302 Reviewed-by: Caroline Tice <cmtice@google.com> Commit-Queue: Han Shen <shenhan@google.com> Tested-by: Han Shen <shenhan@google.com>
2014-04-23Update and standardize example experiment files.cmtice
This cleans up the example experiment files, makes sure they are all up to date, and that they use consisten wording and examples throughout. It also renames some of them. BUG=None TEST=None Change-Id: I1bbf9e364a0dfae7cfb880011f2c140e1a347e98 Reviewed-on: https://chrome-internal-review.googlesource.com/161386 Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-04-22Use CrosRunCommand to find if cherrypy is available. Fixed bug in CrosRunCommandLuis Lozano
Call CrosRunCommand instead of calling ssh directly to find if cherrypy is available. Also fix bug in CrosRunCommand. The retval returned was always 0. BUG=None TEST=Reproduced with small test cases. Change-Id: I277416b1809eb7287a75bd224b4154f442132a52 Reviewed-on: https://chrome-internal-review.googlesource.com/161123 Reviewed-by: Han Shen <shenhan@google.com> Tested-by: Luis Lozano <llozano@chromium.org> Commit-Queue: Caroline Tice <cmtice@google.com>
2014-04-22Fixed command to figure out if cherrypi is available on the DUTLuis Lozano
Figured out why we sometimes we were not using cros flash for imaging the machines. It turns out the ssh command used was incorrect and (did not specify an empty file for the known hosts) so it will sometimes will fail when connecting to the DUT. There was another problem because the code assumed a return value of non-zero meant cherrypi was not available. We know also check for error code 255 which means the ssh itself failed (not the remote command). BUG=None TEST=Imaged a machine Change-Id: I7c5cef859b73bc9f2a777825914a1c1bb162704b Reviewed-on: https://chrome-internal-review.googlesource.com/161009 Reviewed-by: Han Shen <shenhan@google.com> Commit-Queue: Luis Lozano <llozano@chromium.org> Tested-by: Luis Lozano <llozano@chromium.org>
2014-04-21Clean up Crosperf options and help.cmtice
Crosperf's help was completely out of date and incorrect. Fixed that. Also, I cleaned up the options, removed some that were no longer appropriate or not ever used, and made some options global-only. Also removed all references to pyauto. We no longer use or run pyauto tests. Option changes: Completely removed outlier_range (it was not actually doing anything), key_results_only (applied only to pyauto tests), use_test_that (not needed now that we're not using pyauto) and md5sum (never used). Made the following options global-only: rm_chroot_tmp, perf_args, logging_level, show_all_results, board. Removed 'build' from global settings and made it an image-only setting (it made no sense as a global setting). I also adjusted the unittests to work with these changes. BUG=None TEST=I tested all the options in all their various positions; I tested with and wtihout caching; and I tested local and official images. I also ran the unittests. Everything works. Change-Id: I7653663f65c0a3363a1904c0220f1066e72c7f9d Reviewed-on: https://chrome-internal-review.googlesource.com/160758 Reviewed-by: Bhaskar Janakiraman <bjanakiraman@google.com> Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-04-18Various minor crosperf cleanups.cmtice
- Add results summary information for peacekeeper.html to json file. - Don't add section for 'perf table' to html report if there is no perf report. - Remove the color box for the gmean speedup (there's no good way to label it and it's confusing unless you know what it is). BUG=None TEST=Ran Crosperf with these changes. It all seems to work correctly. Change-Id: I2bf5bdfc2667ae665767374d10672d2d19696267 Reviewed-on: https://chrome-internal-review.googlesource.com/160896 Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-04-17Rename bootstrap_compiler.py to build_tool.py.Han Shen
TEST=None BUG=None Change-Id: I33d914575c17f4c2b6535ecdbdc474759e9f8f56 Reviewed-on: https://chrome-internal-review.googlesource.com/160841 Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Han Shen <shenhan@google.com> Tested-by: Han Shen <shenhan@google.com>
2014-04-17Do not generate silly ssh warning while checking for cherrypiLuis Lozano
BUG=None TEST=Tested in nightly testing Change-Id: Ia46a0cbb34890a0cb1222f16157c7c98179ec033 Reviewed-on: https://chrome-internal-review.googlesource.com/160769 Reviewed-by: Caroline Tice <cmtice@google.com> Reviewed-by: Luis Lozano <llozano@chromium.org> Commit-Queue: Luis Lozano <llozano@chromium.org> Tested-by: Luis Lozano <llozano@chromium.org>
2014-04-17Add support for building gcc/binutils for a specific board for ↵Han Shen
bootstrap_compiler.py. TEST=None BUG=None Change-Id: I11edfd930b3c4d1985731a6912dec64e941ba03f Reviewed-on: https://chrome-internal-review.googlesource.com/150959 Reviewed-by: Han Shen <shenhan@google.com> Commit-Queue: Han Shen <shenhan@google.com> Tested-by: Han Shen <shenhan@google.com>
2014-04-12Download official & trybot images; treat as local images for tests.cmtice
Currently we cannot get md5sums for official & trybot images, which causes multiple problems: Crosperf re-images the DUT every time it runs a test for an image, even if the image is already on the DUT; if crosperf is running an experiment with multiple official or trybot images, it can get confused about which results belong to which image; caching does not work properly without md5sums. To fix all of these problems, this CL changes Crosperf to download the official or trybot images into the /tmp directory in the chroot, and from there it can treat them as local images (getting md5sums etc). In order to download them, it first has to translate the xbuddy syntax (which can contain aliases such as 'lumpy/dev-latest') into the actual image name (e.g. lumpy-release/R36-5727.0.0). This translation is done by a new script, translate_xbuddy.py, which must be run from the toolchain-utils directory inside the chroot. BUG=356279,356474,356476 TEST=Tested with official and trybot images, using both correct designations and xbuddy aliases; tested with the image missing from the chroot and with it already in /tmp; tested with and without caching. Also tested with the experiment file included in the 3 bugs. Change-Id: I917b5520604b7d4851db2c8123165e81b866da2b Reviewed-on: https://chrome-internal-review.googlesource.com/159465 Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-04-11Update unittests to all pass.cmtice
Fix the parameters to the various unittests so they match recent changes and the unittests all pass again. BUG=None TEST=I ran all the unittests with the changes. Change-Id: I083b5127a2ade8f1dbaf2bb173d82183871cb7c7 Reviewed-on: https://chrome-internal-review.googlesource.com/159915 Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-04-08Always write verbose logs to files.cmtice
The option that reduces the amount of screen output (--logging_level = 'quiet' or 'average') was also reducing what got written to the log files. This fixes that. With this CL, even though the amount of screen output is reduced, what gets written to the log files is always the same (verbose). This CL also makes "average" the default logging level. BUG=None TEST=I tested all three logging levels, saving the results in different log directories and compared the log files afterwards, to make sure the log files contained the same amount of logging, regardless of the screen logging level. Change-Id: I6982499d67c080afa835e01ab4fa2282dd992052 Reviewed-on: https://chrome-internal-review.googlesource.com/159654 Reviewed-by: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com> Commit-Queue: Caroline Tice <cmtice@google.com>
2014-04-05Remove 'Completed' column; Remove 'retval' row from summary table.cmtice
The "Completed" column causes confusion because tests that fail are still considered to have completed. This CL removes the 'Completed' column, and instead posts the number of iterations that passed or failed, for each test image, in the table header, beside the image name. It also removes the 'retval' row from the summary table, since the retval will either be '0' if a test iteration passes or '1' if it fails, and seeing the average of all the retvals is useless, unless they all pass or fail anyway. The retval row is still there in the full table, which shows all the results for each iteration. BUG=357340 TEST=I ran crosperf with this change in all configurations I could think of and it seems to work. Change-Id: I0e64b9ed016b0a4604814bac49442848cd27f5d3 Reviewed-on: https://chrome-internal-review.googlesource.com/159113 Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-04-04Check for 'significant' results at report generation instead of caching.cmtice
The current implementation of the option that masks unimportant results discards the uninteresting results before caching the test results. After further thought, that was not the right design choice. Among other things it can cause confusing results, such as seen in issue 357346. We should always cache all the results, and do the result filtering during report generation. This CL makes that change. BUG=357346,357343 TEST=Ran crosperf tests with and without cache hits, and with and without entreies in the json file. It all seems to work as expected. Change-Id: I778e5614c73bf751ebaa2d4606af636275247c60 Reviewed-on: https://chrome-internal-review.googlesource.com/159108 Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-04-04Add OWNERS file at the top level directoryLuis Lozano
BUG=None TEST=None Change-Id: I23a6270cb1c7176d0e12eb8d7f999814c6beaa10 Reviewed-on: https://chrome-internal-review.googlesource.com/159319 Reviewed-by: Han Shen <shenhan@google.com> Commit-Queue: Luis Lozano <llozano@chromium.org> Tested-by: Luis Lozano <llozano@chromium.org> Reviewed-by: Bhaskar Janakiraman <bjanakiraman@google.com>
2014-04-03Add page_cycler.bloat to Toolchain team recommended test set.cmtice
BUG=None TEST=Ran test with this change. Change-Id: I89aab58c8890a7c3f36707d2ffe7422fbfa17547 Reviewed-on: https://chrome-internal-review.googlesource.com/158585 Reviewed-by: Yunlian Jiang <yunlian@google.com> Tested-by: Caroline Tice <cmtice@google.com> Commit-Queue: Caroline Tice <cmtice@google.com>
2014-03-26Update Sheriff email to point to new web page.cmtice
Update the weekly sheriff rotation email to point the sheriff to the new team Sheriff Duty Details page. BUG=None TEST=None Change-Id: I0278186dbf1f004869e4956e546117cd6efa85c7 Reviewed-on: https://chrome-internal-review.googlesource.com/158315 Reviewed-by: Luis Lozano <llozano@chromium.org> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-03-26Fix syntax error with using some official builds.cmtice
This CL fixes a bug with the syntax for specifying some official builds with xbuddy. BUG=None TEST=Tested this version with the older specifications as well as the new version that caused problems. Change-Id: Ifa189f554ff2aa45b125716d51da1d9755fb9ddc Reviewed-on: https://chrome-internal-review.googlesource.com/158255 Reviewed-by: Han Shen <shenhan@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-03-25Update to use "working" version of perf for now.cmtice
This is a temporary fix, until we can figure out why the 'normal' version of 'perf' is not properly symbolizing ChromeOS perf data. This CL commits a version of the 'perf' binary (based on perf 3.13) that does properly symbolize ChromeOS perf data files. It also updates crosperf to copy this working perf binary into the chroot (in /tmp) and to use it when generating the perf report. Work to track down and fix the real problem is ongoing, but in the meantime this allows Crosperf users to get meaningful perf reports for ChromeOS. BUG=None TEST=Tested Crosperf using this patch and generating perf reports. Change-Id: Ia8dd157b88d37a897eb9becf8fa444de41e88873 Reviewed-on: https://chrome-internal-review.googlesource.com/157756 Reviewed-by: Caroline Tice <cmtice@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-03-19Fix error that wrote logs to 'verbose' subdirectory.cmtice
The recent commit that introduced logging levels accidentally left in a line that caused log files to be written to a subdirectory named 'verbose' rather than to 'logs'. This fixes that problem. BUG=None TEST=Tested this will all 3 levels of logging; it works properly. Change-Id: I0587f242108a1e3762f4161e88f4e6ac119a367b Reviewed-on: https://chrome-internal-review.googlesource.com/157420 Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com> Reviewed-by: Han Shen <shenhan@google.com>
2014-03-19Various small fixes for nightly infrastructure tests.cmtice
- Change the number of iterations for tests from 1 to 3. - Increase the timeout for image_chromeos.py, so 'cros flash', which takes a bit longer than cros_image_to_target.sh did, doesn't get killed just as its finishing. - Comment out smoothness.top_25 for now, since it's broken and/or hanging. BUG=None TEST=Successfully ran nightly infrastructure tests with these changes. Change-Id: I77b663b1910fe12ef927afdf15d88a7c1e82d491 Reviewed-on: https://chrome-internal-review.googlesource.com/157419 Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-03-14Add --logging_level option to crosperf.cmtice
Add --logging_level={verbose, average, quiet} option to crosperf, optionally reducing the amount of logging output. "verbose" gives the same output Crosperf always gave; "quiet" gives minimal output, and "average" is in the middle. The default for now is "verbose". Doing this also involved adding logging levels to the command_executer, and to image_chromeos (again, with the default being "verbose"). BUG=None TEST=Tested all three levels of logging, with and without having to reimage the DUT. It all worked properly. Change-Id: Icf8c9a6831fe25202adcb624c6c7e1d1a7ac25a5 Reviewed-on: https://chrome-internal-review.googlesource.com/156883 Reviewed-by: Caroline Tice <cmtice@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-03-14Fix Crosperf's ETA bug.cmtice
Sometimes crosperf shows a negative number for ETA. This fixes that. BUG=None TEST=Force the bug to happen; tested with and without the fix. Change-Id: I0c8c909a944ad3cf25b8c5416f93ddc0b860a03d Reviewed-on: https://chrome-internal-review.googlesource.com/157125 Reviewed-by: Han Shen <shenhan@google.com> Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-03-06Fix bug introduced with the --show_all_results flag, using cache.cmtice
The call to self._ProcessResults is supposed to take the show_all_results flag as an argument. This was overlooked in the read-it-out-of-the-cache path when this change was first introduced. This CL fixes that. BUG=None TEST=I tested cached and non-cached runs, with and without the --show_all_results option. Everything worked as it should. Change-Id: Ibf8f206406b0a4535d48b598709e62135289f993 Reviewed-on: https://chrome-internal-review.googlesource.com/156367 Reviewed-by: Yunlian Jiang <yunlian@google.com> Tested-by: Caroline Tice <cmtice@google.com> Commit-Queue: Caroline Tice <cmtice@google.com>
2014-03-05Fix problem with looking for 'perf' in the wrong directory.cmtice
BUG=None TEST=Tested with old and new chroots; seems to work. Change-Id: Id1a7fc9d6b1dea0b96fe51032bb25d4722dd8b48 Reviewed-on: https://chrome-internal-review.googlesource.com/156381 Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-02-20Add 'build' settings option and xbuddy image format.cmtice
This patch updates Crosperf to add the 'build' tag in the experiment file, and to allow that field to contain xbuddy syntax for using trybot and/or official builds in the test runs. It also adds a bit more checking to make sure we have everything necessary for running 'cros flash' before attempting to use it. BUG=None TEST=I have run this using an experiment file that compares a local image, a trybot image and an official image against each other. It all worked. Change-Id: Ia896799061508fb5718a3201b1599d8622de0b3f Reviewed-on: https://chrome-internal-review.googlesource.com/154097 Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-02-13Use 'scaling_max_freq' when 'scaling_available_frequencies' is not available.Han Shen
In recently arm images, there is no such 'scaling_available_frequencies' file, which leads failure of crosperf. Fixed this by choosing 'scaling_max_freq'. TEST=tested on daisy BUG=None Change-Id: I433e161495421c57854234fd0e7f7e54406bb9e8 Reviewed-on: https://chrome-internal-review.googlesource.com/154579 Reviewed-by: Caroline Tice <cmtice@google.com> Commit-Queue: Han Shen <shenhan@google.com> Tested-by: Han Shen <shenhan@google.com>
generated by cgit v1.2.3 (git 2.25.1) at 2024-07-11 14:24:51 +0000