aboutsummaryrefslogtreecommitdiff
path: root/auto_delete_nightly_test_data.py
AgeCommit message (Collapse)Author
2020-12-10toolchain-utils: Remove 'target' from distfiles path (for cleanup).upstream-mirror-mastermirror-chromium-masterCaroline Tice
'target' is no longer part of the path from distfiles to chrome-src, so this CL fixes the path inour cleanup script. BUG=None TEST=tested script by hand. Change-Id: I8a69b0d153a301b61c5176e44d1ad65dc8e467cd Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2585652 Auto-Submit: Caroline Tice <cmtice@chromium.org> Tested-by: Caroline Tice <cmtice@chromium.org> Commit-Queue: Manoj Gupta <manojgupta@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2020-09-25fix formatting/lint issues pointed out by repohooksBob Haarman
Previous changes resulted in some complaints about formatting and Python 3 compatibility from the repo hooks. This change fixes those. BUG=None TEST=repo upload --cbr . # check that it no longer complains Change-Id: I99cc51dcb8d499d59b7b47817f4cef8fa6ba5059 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2419831 Tested-by: Bob Haarman <inglorion@chromium.org> Reviewed-by: Manoj Gupta (OoO) <manojgupta@chromium.org>
2020-09-25rephrase some language in code and commentsBob Haarman
This rephrases some code and comments to avoid unclear and non-inclusive wording. For the remaining terms in unblocked_terms.txt, comments are added to explain why these terms are present in the code. BUG=chromium:1099035 TEST=repo hooks Change-Id: I985613d811836b6fb6351eadcf3e08a3290db97f Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2419656 Tested-by: Bob Haarman <inglorion@chromium.org> Reviewed-by: Manoj Gupta (OoO) <manojgupta@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org>
2020-08-06Update cleanup script to get chrome temp files too.Caroline Tice
Final patch didn't get uploaded last time; here it is. BUG=None TEST=Tested updaed script on chrotomation3. Change-Id: I8abccb8c5ed98b7086dc03bc05bd3d75ae8b6734 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2341096 Reviewed-by: George Burgess <gbiv@chromium.org> Commit-Queue: Caroline Tice <cmtice@chromium.org> Tested-by: Caroline Tice <cmtice@chromium.org>
2020-08-06Update cleanup script to get chrome temp files too.Caroline Tice
BUG=None TEST=Tested updated script on chrotomation3 Change-Id: Ic7c684e2a6fcb52596cbdc9f308e0ca1fbb6f05b Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2340400 Commit-Queue: Caroline Tice <cmtice@chromium.org> Tested-by: Caroline Tice <cmtice@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org>
2020-05-15Update tmp cleanup on chrotomation3Denis Nikitin
Remove "*-llvm-next-nightly" images from chroot /tmp. Updated CleanChromeOsImageFiles which now deletes directories with rmtree. This should fix the problem with cron job auto delete. BUG=None TEST=Tested with --dry_run on chrotomation3. Change-Id: If8458540e009c50dec829f14c47581fdd7be305e Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2202728 Tested-by: Denis Nikitin <denik@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2020-04-29Fix a typoManoj Gupta
s/chromeos_chroot/chromeos_root in AbandonCLs. BUG=chromium:1067029 TEST=unit tests Change-Id: Ie3727aaf3b10e196f2abe4a9d90db2f6766a42a5 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2173246 Tested-by: Manoj Gupta <manojgupta@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org>
2020-04-29toolchain-utils: Auto abandon old CLsManoj Gupta
Abandon CLs without any update that are older than 2 weeks. Ignoring days to preserve intentionally since sometimes CLs can be older than several days. BUG=chromium:1067029 TEST=unit tests Change-Id: Ie0f377a32ccf40331221c579401b5aaa2e84cf6c Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2147820 Tested-by: Manoj Gupta <manojgupta@chromium.org> Reviewed-by: Tiancong Wang <tcwang@google.com>
2020-02-12toolchain-utils: Partially port scripts to python 3Zhizhou Yang
This patch ports some still-in-use python scripts under root directory of toolchain-utils to python 3. BUG=chromium:1011676 TEST=Passed unittests and tested with manually launching. Change-Id: Id6066944780a7204fe4746cd271f41ac20f2274d Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2049103 Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Auto-Submit: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: George Burgess <gbiv@chromium.org>
2020-02-07toolchain-utils: move weekday testing into nightly-testing dir.Luis Lozano
BUG=None TEST=None. Change-Id: I99b9c1c09bcf7d6aaddd79f38e9ea8105f192eb0 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2043954 Reviewed-by: George Burgess <gbiv@chromium.org> Reviewed-by: Zhizhou Yang <zhizhouy@google.com> Commit-Queue: Luis Lozano <llozano@chromium.org> Tested-by: Luis Lozano <llozano@chromium.org> Auto-Submit: Luis Lozano <llozano@chromium.org>
2019-04-11test data cleanup: also delete CrAU_* filesGeorge Burgess IV
These are currently consuming 97GB of disk on chrotomation2, and should probably be cleaned up regularly. BUG=None TEST=None Change-Id: I85bcf38b19a75653d089baa0aa85d661b8d8dd14 Reviewed-on: https://chromium-review.googlesource.com/1562354 Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com> Tested-by: George Burgess <gbiv@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2017-12-27crosperf: Cleanup files downloaded with -tryjob suffix.Manoj Gupta
Add -tryjob suffix to cleanup list. Also remove whole directory not accessed in last day instead of trying to delete individual files/directories. This makes it easier to see only the preserved directories later on. + Minor tc_format related changes. BUG=None TEST=Directories to be deleted show up in dry run.y Change-Id: I711a8d9200686be56fb4f081682d76207dea368b Reviewed-on: https://chromium-review.googlesource.com/844897 Commit-Ready: Manoj Gupta <manojgupta@chromium.org> Tested-by: Manoj Gupta <manojgupta@chromium.org> Reviewed-by: Yunlian Jiang <yunlian@chromium.org>
2017-01-24Cleanup stale files from chrotomation2.Manoj Gupta
Remove images and autotest_files directories left in chromeos chroot *-pfq directories. Slight refactoring to avoid code duplication for release and pfq directories. Minor mandatory formatting updates. BUG=chromium:656755 TEST=Old files show up for removal in dry run Change-Id: I91230c0696148847dc408a3047f468c663b8ab8d Reviewed-on: https://chromium-review.googlesource.com/431268 Commit-Ready: Manoj Gupta <manojgupta@chromium.org> Tested-by: Manoj Gupta <manojgupta@chromium.org> Reviewed-by: Caroline Tice <cmtice@chromium.org>
2016-12-07Cleanup stale files on chrotomation2.Manoj Gupta
Remove cros-update* and autotest_files directories left in chromeos chroot. Improve auto_delete script dry run to include chroot tmp cleanup. Minor mandatory formatting updates. BUG=chromium:656755 TEST=Old files show up for removal in dry run Change-Id: I9d190f3220fc68a8f2cb437c08ae9506960f4983 Reviewed-on: https://chrome-internal-review.googlesource.com/310057 Commit-Ready: Manoj Gupta <manojgupta@google.com> Tested-by: Manoj Gupta <manojgupta@google.com> Reviewed-by: Caroline Tice <cmtice@google.com>
2016-01-22Fix cros lint errors.Caroline Tice
Also move deprecated scripts to the 'deprecated' directory. BUG=chromiumos:570464 TEST=tested scripts to make sure they still work. Change-Id: I3442a86d898104591233a0849ea0bafb52ecf1f7 Reviewed-on: https://chrome-internal-review.googlesource.com/244221 Commit-Ready: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com> Reviewed-by: Yunlian Jiang <yunlian@google.com>
2015-12-16crosperf: clean lint warnings.Yunlian Jiang
BUG=chromium:567921 TEST=these three files are lint warning free. Change-Id: I23f9120283fc82958abcc16f47cd29d5800bd181 Reviewed-on: https://chrome-internal-review.googlesource.com/242485 Commit-Ready: Yunlian Jiang <yunlian@google.com> Tested-by: Yunlian Jiang <yunlian@google.com> Reviewed-by: Caroline Tice <cmtice@google.com>
2015-12-16Run pyformat on all the toolchain-utils files.Luis Lozano
This gets rid of a lot of lint issues. Ran by doing this: for f in *.py; do echo -n "$f " ; if [ -x $f ]; then pyformat -i --remove_trailing_comma --yapf --force_quote_type=double $f ; else pyformat -i --remove_shebang --remove_trailing_comma --yapf --force_quote_type=double $f ; fi ; done BUG=chromium:567921 TEST=Ran simple crosperf run. Change-Id: I59778835fdaa5f706d2e1765924389f9e97433d1 Reviewed-on: https://chrome-internal-review.googlesource.com/242031 Reviewed-by: Luis Lozano <llozano@chromium.org> Commit-Queue: Luis Lozano <llozano@chromium.org> Tested-by: Luis Lozano <llozano@chromium.org> Reviewed-by: Yunlian Jiang <yunlian@google.com>
2015-12-15crosperf: RunCommand should return one type of object.Luis Lozano
Cleaned up the interfaces for the RunCommand routines. These were returning different types (int or tuple) depending on the value of the return_ouput parameter. Returning different unrelated types from a routine is bad practice. Linter complains about this with several warnings like this: "Attempting to unpack a non-sequence defined at line XY of utils.command_executer" BUG=chromium:566256 TEST=ran crosperf with a example experiment file Ran run_tests. Change-Id: Ibb83ab9322c87558077fc4937ef5c0686bbe5417 Reviewed-on: https://chrome-internal-review.googlesource.com/241459 Commit-Ready: Luis Lozano <llozano@chromium.org> Tested-by: Luis Lozano <llozano@chromium.org> Reviewed-by: Han Shen <shenhan@google.com>
2015-11-05Make cleaning of nightly data less verbose.Luis Lozano
The nightly process for clean up of old data is too verbose and makes the summary report difficult to follow. BUG=None TEST=None Change-Id: I3b1e18c0d913e47d4281617970a7e9a2ea0d9eb1 Reviewed-on: https://chrome-internal-review.googlesource.com/237839 Commit-Ready: Luis Lozano <llozano@chromium.org> Tested-by: Luis Lozano <llozano@chromium.org> Reviewed-by: Han Shen <shenhan@google.com>
2015-03-11Fix typo that was causing some files to not be found/deleted.cmtice
BUG=None TEST=Tested command by hand. Change-Id: Ie61fcd802fe91ca02f361fe3665001409e4b185f Reviewed-on: https://chrome-internal-review.googlesource.com/207315 Reviewed-by: Rahul Chaudhry <rahulchaudhry@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2015-02-19Added a function that cleans temporaries/images under crostc/chromeos.Han Shen
Change-Id: Ifa92fc519afadcf10297773a2d42e27e1c9c0f9d Reviewed-on: https://chrome-internal-review.googlesource.com/199617 Reviewed-by: Luis Lozano <llozano@chromium.org> Commit-Queue: Han Shen <shenhan@google.com> Tested-by: Han Shen <shenhan@google.com>
2015-02-05Update command_executer.py to allow a silent execution mode,cmtice
where it does not attempt to write any logs. Update auto_delete_nightly_test_data.py to use the command_executer in silent mode (so it doesn't run out of space writing logs on the disk it is trying to clean up). Change-Id: Id1c2c5f171a2d3854d3b53516ead31050da647b4 Reviewed-on: https://chrome-internal-review.googlesource.com/195877 Reviewed-by: Han Shen <shenhan@google.com> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-05-21Better handling of "not identical machines" failure.cmtice
Nightly tests, especially on the x86-generic boxes, fail frequently because the DUTs have different board images on them, so Crosperf decides the machines are not identical and refuses to run the tests. With this CL, if the machine_manager finds that the machines fail the "identical" test, it will try to push the same image onto all the machines, and then check them again to see if they are the same. It only tries this once; if they fail the check the second time around, it is still a fatal failure. This should eliminate many of the unnecessary failures in our nightly tests. This CL also fixes a small bug in the auto-delete script (this fix has been running for a while on mobiletc-prebuild, but never got committed). BUG=None TEST=Ran several iterations where I forced the first "identical" check to fail. The changes worked. Change-Id: Ied2a55e5d3e2789e58a503aef03269888954b579 Reviewed-on: https://chrome-internal-review.googlesource.com/163334 Reviewed-by: Luis Lozano <llozano@chromium.org> Commit-Queue: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com>
2014-04-25Abort deleting the directory if deletion of any of its chromeos subtree fails.Han Shen
TEST=None BUG=None Change-Id: I4dfd2f67d8bc7f6f329b58800c148a923e78eace Reviewed-on: https://chrome-internal-review.googlesource.com/161678 Reviewed-by: Luis Lozano <llozano@chromium.org> Commit-Queue: Han Shen <shenhan@google.com> Tested-by: Han Shen <shenhan@google.com>
2014-04-24Add necessary output to auto_delete_nightly_test_data script and return with ↵Han Shen
meaning exit code. The script did not have any output in case of a success run, which means we do not get emails about this cron job. Also the exit code was always 0. Fixed by adding print statements and return correct exit code. TEST=None BUG=None Change-Id: If06a4d57a94387494f9c1b30fea765f66e6e0f56 Reviewed-on: https://chrome-internal-review.googlesource.com/161302 Reviewed-by: Caroline Tice <cmtice@google.com> Commit-Queue: Han Shen <shenhan@google.com> Tested-by: Han Shen <shenhan@google.com>
2013-12-16Add a script to delete nightly test data that are old.Han Shen
TEST=tested on tc-build.hot Change-Id: I6b6bc298b56c6ac79cbbd66f8168bc234580c884 Reviewed-on: https://chrome-internal-review.googlesource.com/150264 Reviewed-by: Yunlian Jiang <yunlian@google.com> Commit-Queue: Han Shen <shenhan@google.com> Tested-by: Han Shen <shenhan@google.com>
generated by cgit v1.2.3 (git 2.25.1) at 2024-07-06 01:34:00 +0000