aboutsummaryrefslogtreecommitdiff
path: root/test_toolchains.py
AgeCommit message (Collapse)Author
2016-10-24Handle exceptions in crosperf.Manoj Gupta
Change error message in buildbot. Catch exceptions in crosperf exceution to do a graceful exit. BUG=chromium:622875 TEST=ran run_tests Change-Id: I03093ab27edcc85e318ad43097ab8cd316792dcb Reviewed-on: https://chrome-internal-review.googlesource.com/299275 Commit-Ready: Manoj Gupta <manojgupta@google.com> Tested-by: Manoj Gupta <manojgupta@google.com> Reviewed-by: Caroline Tice <cmtice@google.com> Reviewed-by: Luis Lozano <llozano@chromium.org>
2016-01-27toolchain-utils: strip argv[0] before calling setup_chromeos.Main()Rahul Chaudhry
This is a followup to https://chrome-internal-review.googlesource.com/#/c/245868 test_toolchains.py directly calls setup_chromeos.Main(). setup_chromeos.py was changed recently to use argparse instead of optparse. Remove the name of the script from the arguments to Main() to keep argparse happy. BUG=None TEST='nightly_wrapper.sh lumpy ...' proceeds with no error during argument parsing. Change-Id: I30e8e6f1372676126b0af08125467096ea866266 Reviewed-on: https://chrome-internal-review.googlesource.com/246266 Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com> Tested-by: Rahul Chaudhry <rahulchaudhry@google.com> Reviewed-by: Caroline Tice <cmtice@google.com>
2016-01-26toolchain-utils: strip argv[0] before calling Main().Rahul Chaudhry
https://chrome-internal-review.googlesource.com/#/c/244221 fixed some cros lint errors, and in the process, modified test_toolchains.py to use argparse instead of optparse. The flags parsing semantics of these modules seem to differ when they see an argument which is not a flag. As a result, "nightly_wrapper.sh" nightly cron jobs are failing with error: "unrecognized arguments: /usr/local/google/crostc/toolchain-utils/test_toolchains.py" This CL strips the argv[0] before calling Main(), since the name of the script is not really an argument and doesn't need to be parsed. BUG=None TEST='nightly_wrapper.sh lumpy ...' proceeds with no error during argument parsing. Change-Id: Idf13ab457170bb051d4d0693a887795d0c16918d Reviewed-on: https://chrome-internal-review.googlesource.com/245868 Commit-Ready: Luis Lozano <llozano@chromium.org> Tested-by: Luis Lozano <llozano@chromium.org> Reviewed-by: Luis Lozano <llozano@chromium.org>
2016-01-22Fix cros lint errors.Caroline Tice
Also move deprecated scripts to the 'deprecated' directory. BUG=chromiumos:570464 TEST=tested scripts to make sure they still work. Change-Id: I3442a86d898104591233a0849ea0bafb52ecf1f7 Reviewed-on: https://chrome-internal-review.googlesource.com/244221 Commit-Ready: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com> Reviewed-by: Yunlian Jiang <yunlian@google.com>
2015-12-16Run pyformat on all the toolchain-utils files.Luis Lozano
This gets rid of a lot of lint issues. Ran by doing this: for f in *.py; do echo -n "$f " ; if [ -x $f ]; then pyformat -i --remove_trailing_comma --yapf --force_quote_type=double $f ; else pyformat -i --remove_shebang --remove_trailing_comma --yapf --force_quote_type=double $f ; fi ; done BUG=chromium:567921 TEST=Ran simple crosperf run. Change-Id: I59778835fdaa5f706d2e1765924389f9e97433d1 Reviewed-on: https://chrome-internal-review.googlesource.com/242031 Reviewed-by: Luis Lozano <llozano@chromium.org> Commit-Queue: Luis Lozano <llozano@chromium.org> Tested-by: Luis Lozano <llozano@chromium.org> Reviewed-by: Yunlian Jiang <yunlian@google.com>
2015-11-05Make 'label' names more meaningful in nightly tests.Caroline Tice
The nightly crosperf tests currently use the label 'test_image' for the images that we build or that the trybot built. This is not a very informative name. This CL tries to make the names a least a little bit more useful. BUG=None TEST=Tested by hand in role account directory tree. Change-Id: Ib88f711ec96ec5cb71efebae79a76e6e8ebc784a Reviewed-on: https://chrome-internal-review.googlesource.com/237796 Commit-Ready: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com> Reviewed-by: Luis Lozano <llozano@chromium.org>
2015-09-14Make new scheduler the default.Han Shen
This CL also add a new option '--noschedv2' to use the original scheduler in case of bugs in the new scheduler. Change-Id: I11ddefe7e7c1162019e9ad72ed3cef0b962b4d06 Reviewed-on: https://chrome-internal-review.googlesource.com/230484 Commit-Ready: Han Shen <shenhan@google.com> Tested-by: Han Shen <shenhan@google.com> Reviewed-by: Caroline Tice <cmtice@google.com>
2015-09-04Start generating json archive files for nightly tests.Caroline Tice
This turns on the option for the nightly tests to generate json files, which get stored in a pending archives directory and can later be picked up, converted to ColumnIO and stored on a colossus server. It also fixes a bug where the nightly test was looking for a .boto file in /home/mobiletc-prebuild instead of /usr/local/google/home/mobiletc-prebuild. BUG=None TEST=None Change-Id: Id413097686d8e72499d141ca7b8d2e131a993cb9 Reviewed-on: https://chrome-internal-review.googlesource.com/228860 Commit-Ready: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com> Reviewed-by: Luis Lozano <llozano@chromium.org>
2015-08-28Add a new option '--schedv2' so later invocation to crosperf uses the new ↵Han Shen
scheduler. This option is added to buildbot_test_toolchains.py & test_toolchains.py. TEST=None Change-Id: I8f31d0dd7b4b1db6af35759dd476f0f55e224366 Reviewed-on: https://chrome-internal-review.googlesource.com/228645 Commit-Ready: Han Shen <shenhan@google.com> Tested-by: Han Shen <shenhan@google.com> Reviewed-by: Caroline Tice <cmtice@google.com>
2015-08-15Remove temporary workaround.Caroline Tice
A good version of Chrome (we think) has made it into ChromeOS now, so this will remove the temporary workaround for the lumpy tests. If the lumpy tests work with this removed, then we will remove it for the buildbot tests. BUG=None TEST=None Change-Id: I84526b06185176a3225ba52b3e3990d0d30ef170 Reviewed-on: https://chrome-internal-review.googlesource.com/227158 Reviewed-by: Luis Lozano <llozano@chromium.org> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2015-08-12Temporarily use good external Chrome source tree.Caroline Tice
This is to work around a problem with the current version of Chrome inside ChromeOS (telemetry is broken in that version of Chrome). This CL makes the nightly tests use an external Chrome source tree that has been verified to work properly. This will be un-done, once ChromeOS gets a good version of Chrome. BUG=None TEST=Tested crosperf on chrotomation2 using this. Change-Id: I41caef9bb1b1973c83745e3962a5c36c489d7514 Reviewed-on: https://chrome-internal-review.googlesource.com/226695 Reviewed-by: Luis Lozano <llozano@chromium.org> Commit-Queue: Luis Lozano <llozano@chromium.org> Tested-by: Luis Lozano <llozano@chromium.org>
2015-07-01Add 'retries' to crosperf for nightly non-buildbot tests.cmtice
BUG=None TEST=None Change-Id: Id991ed0d5e23d363940d65aeb49f4e4fbbd2051e Reviewed-on: https://chrome-internal-review.googlesource.com/221000 Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2015-06-30Update all_toolchain_perf to use new recommended set.cmtice
Update the all_toolchain_perf alias to use the new recommended set of tests. Also update the default test values to include default values for the newly recommended tests. Also update the nightly tests to use the newly recommended set. BUG=None TEST=Tested new alias with crosperf. Change-Id: I603f866723646a2d5b6d86fbe1de3c4617e09046 Reviewed-on: https://chrome-internal-review.googlesource.com/220385 Reviewed-by: Han Shen <shenhan@google.com> Tested-by: Caroline Tice <cmtice@google.com> Commit-Queue: Caroline Tice <cmtice@google.com>
2015-06-23Remove temporary workaround for nightly tests locking.cmtice
Due to issues with mobiletc-prebuild not being able to access the afe locking mechanisms, the nightly tests were temporarily using the old file locking mechanism. The issues with the role account using the afe locks has been resolved, so this CL reverts the nightly tests to using the afe machine locking mechanism. BUG=None TEST=Ran crosperf tests on chrotomation2 from role account using afe locking mechanism. Change-Id: I243c28d80718c2e98913ad49d50bf5b46ff82758 Reviewed-on: https://chrome-internal-review.googlesource.com/219905 Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2015-06-19Fix typo in file. Committing as obvious.cmtice
Change-Id: I68fc37b0eaf3ff5bec53abe7e11b8227644685e9 Reviewed-on: https://chrome-internal-review.googlesource.com/219726 Reviewed-by: Caroline Tice <cmtice@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2015-06-12Update nightly tests to use old file locking mechanism for now.cmtice
There are some issue with the role account using the AFE server locking mechanism, so we will temporarily revert the nightly tests to use the old file locking mechanism. This CL depends on the CL that re-enables the file locking mechanism (with a new flag). BUG=None TEST=None CQ-DEPEND=CL:219096 Change-Id: If525e4fcf292d3c57aa9c69a1b796914bd6d434c Reviewed-on: https://chrome-internal-review.googlesource.com/219088 Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2015-05-29Fix typo.cmtice
BUG=None TEST=None Change-Id: If450b820bc218fbd02190982b0854c4a421e55c2 Reviewed-on: https://chrome-internal-review.googlesource.com/217620 Reviewed-by: Caroline Tice <cmtice@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2015-05-28Add missing import statement.cmtice
BUG=None TEST=Testing by hand in role account. Change-Id: If7657ae15bea5faee891e5ae9959082196d77e32 Reviewed-on: https://chrome-internal-review.googlesource.com/217513 Reviewed-by: Luis Lozano <llozano@chromium.org> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2015-05-27Fix typo from previous commit.cmtice
Remove leftover colon from end of line that's no longer a conditional check. BUG=None TEST=None Change-Id: I3cbcc5f18280adb3044386df205de7309c7a7855 Reviewed-on: https://chrome-internal-review.googlesource.com/217440 Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2015-05-27Update nightly tests to use mail-sheriff program for reports.cmtice
This updates the nightly buildbot and local tester programs to tell crosperf to put its results in a special directory, and to not send email. After crosperf finishes, it looks for the email information in the special directory and sends out then email using the mail-sheriff program. BUG=None TEST=Currently running buildbot test. Will not commit the change until the test passes. Since the test takes several hours to run I wanted to put this out for review now. Change-Id: Idd3d137e4a6e144b677690fc6775580dfac56af1 Reviewed-on: https://chrome-internal-review.googlesource.com/217286 Reviewed-by: Luis Lozano <llozano@chromium.org> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2015-02-19Only tar chromiumos_test_image.bin file, instead of all files under images ↵Han Shen
directory. Change-Id: I27ca96a5e347de6d2e5ae280cfd37a7669056679 Reviewed-on: https://chrome-internal-review.googlesource.com/199736 Reviewed-by: Caroline Tice <cmtice@google.com> Commit-Queue: Han Shen <shenhan@google.com> Tested-by: Han Shen <shenhan@google.com>
2015-02-14Mark buildbot_test_toolchains.py, test_toolchains.py, and weekly_report.py ↵Rahul Chaudhry
executable. BUG=None TEST=None Change-Id: Ia49aef079414d7072a57dc14d9b0d99fa3a75d40 Reviewed-on: https://chrome-internal-review.googlesource.com/198670 Reviewed-by: Caroline Tice <cmtice@google.com> Commit-Queue: Rahul Chaudhry <rahulchaudhry@google.com> Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
2014-08-14Add AFDO to nightly builds.cmtice
This adds an option, '--use_afdo' to build_chromeos.py, along with a list of boards for which it is valid. The flag will cause build_chromeos.py to add 'USE=afdo_use' to calls to build_packages, if the board is appropriate. This CL also updates test_toolchains.py to pass --use_afdo to build_chromeos, if appropriate. BUG=None TEST=Tried it in my own directory and it worked. Change-Id: I0a2056f86783dbd30a2fd650729c888fd1df1f67 Reviewed-on: https://chrome-internal-review.googlesource.com/169174 Reviewed-by: Luis Lozano <llozano@chromium.org> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-07-08Minor fixes to weekly report generation.cmtice
Update test_toolchains.py to clean out old (stale) data before copying current image tar files to reports directory. Update weekly_report.py to generate report for previous 6 days, depending on when report is run, rather than being hard-coded to start the report on a particular day. BUG=Non TEST=Ran test_toolchains.py as part of nightly test; it worked properly. Ran weekly_report.py and it did the right thing. Change-Id: Id173ccaa3146c76e19636501984bab11a41f8568 Reviewed-on: https://chrome-internal-review.googlesource.com/168465 Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-06-24Update nightly tests (and weekly report stuff) to use official builds.cmtice
This CL changes the nightly tests to download and use the official image for our 'vanilla' image, rather than building the vanilla image ourselves. It also updates the code that creates & copies the tar files of the images, for the weekly reports, to look for and copy the official images. BUG=None TEST=Tested the changes by hand. Also let them run for a couple of nights for our nightly tests. Change-Id: I8c3215aab7550dc9f148ed79095295f702141017 Reviewed-on: https://chrome-internal-review.googlesource.com/166477 Reviewed-by: Luis Lozano <llozano@chromium.org> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-06-05Update test_toolchains.py to save tar files of the images used.cmtice
This is for generating a weekly report that compares the nightly test results. All of the nightly test results are cached, but in order to find the correct cache results, you need the image from which the results were created. Due to disk space, some of the chroots will be deleted before the weekly report gets generated. Therefore, this modification tars up the test & vanilla image directories and stores them separately, where they can be accessed for generating the weekly report. BUG=None TEST=Used this successfully last night for nightly test runs. Change-Id: I01b457a402c3649d7904a478b37bad60b3395300 Reviewed-on: https://chrome-internal-review.googlesource.com/164879 Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-03-19Various small fixes for nightly infrastructure tests.cmtice
- Change the number of iterations for tests from 1 to 3. - Increase the timeout for image_chromeos.py, so 'cros flash', which takes a bit longer than cros_image_to_target.sh did, doesn't get killed just as its finishing. - Comment out smoothness.top_25 for now, since it's broken and/or hanging. BUG=None TEST=Successfully ran nightly infrastructure tests with these changes. Change-Id: I77b663b1910fe12ef927afdf15d88a7c1e82d491 Reviewed-on: https://chrome-internal-review.googlesource.com/157419 Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-01-15Make sure we have correct .boto file and permissions for runningcmtice
Telemetry tests inside chroot. BUG=None TEST=Ran nightly infrastructure tests with these changes and they ran and passed. Change-Id: I5c469958f13fc037b8f219b4db542193caf342ca Reviewed-on: https://chrome-internal-review.googlesource.com/151652 Reviewed-by: Caroline Tice <cmtice@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2013-11-06Make changes to run Telemetry tests in nightly infrastructure tests.cmtice
BUG=None TEST=Tested this running the nightly infrastructure cron jobs. Change-Id: I6cccee222b074e9410b45df6750c097430207746 Reviewed-on: https://chrome-internal-review.googlesource.com/147515 Reviewed-by: Luis Lozano <llozano@chromium.org> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2013-10-03Remove (at least temporarily) the --minilayout argument for checking out ↵Caroline Tice
ChromeOS, so the nightly infrastructure tests can build. BUG=None TEST=None Change-Id: I7a6c68142e0c9af106a047986a95a545ece6a094 Reviewed-on: https://chrome-internal-review.googlesource.com/145917 Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2013-08-26add vanilla_image option to build_chromeos.pyYunlian Jiang
This adds vanilla_iamge option to build_chromeos.py. With this option, it runs build_packages with --usepkg. In the test_toolchains.py, it uses this option when building vanilla image. BUG=None TEST=None Change-Id: I54f85e60ccae833ab6682476e19365a51df5ef29 Reviewed-on: https://gerrit-int.chromium.org/43088 Reviewed-by: Luis Lozano <llozano@chromium.org> Commit-Queue: Yunlian Jiang <yunlian@google.com> Tested-by: Yunlian Jiang <yunlian@google.com>
2013-03-15Cleaned up directory after copy of tools from perforce directoryLuis Lozano
Got rid of stale copies of some tools like "crosperf" and moved all files under v14 directory (that came from perforce) into the top directory. BUG=None TEST=None Change-Id: I408d17a36ceb00e74db71403d2351fd466a14f8e Reviewed-on: https://gerrit-int.chromium.org/33887 Tested-by: Luis Lozano <llozano@chromium.org> Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Luis Lozano <llozano@chromium.org>
generated by cgit v1.2.3 (git 2.25.1) at 2024-07-06 07:58:00 +0000