aboutsummaryrefslogtreecommitdiff
path: root/llvm_tools
AgeCommit message (Collapse)Author
2022-01-14patch_sync: Refactor shared main codeJordan R Abrahams
This commit attempts to remove duplicated code between Android and CrOS, as the complexity between this duplicated code exceeded the scope of the main file. Also adds display_patches code. This is to pretty print what patches are getting applied and to where. Only enabled if --verbose is on. Also does some minor function cleanup in version_control.rs. BUG=b:209493133 TEST=cargo test TEST=patch_sync transpose --dry-run --verbose <...> Change-Id: I63410160ff5159f4c079ac1cc189674fe3fc02a0 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3379481 Reviewed-by: George Burgess <gbiv@chromium.org> Tested-by: Jordan Abrahams-Whitehead <ajordanr@google.com> Commit-Queue: Jordan Abrahams-Whitehead <ajordanr@google.com>
2022-01-14patch_sync: Fix PatchDictSchema fieldsJordan R Abrahams
These were out of alphabetical order, and serde_json would write them in the incorrect order to the PATCHES.json file. Additionally, the `platforms` field may end up being null after some discussion with the Android team. This converts platforms to an Optional accordingly. BUG=b:209493133 TEST=patch_sync transpose <...> Change-Id: I7c0b2e984d713698a0ed53e7f36b38bef1a49d1c Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3379480 Reviewed-by: George Burgess <gbiv@chromium.org> Tested-by: Jordan Abrahams-Whitehead <ajordanr@google.com> Commit-Queue: Jordan Abrahams-Whitehead <ajordanr@google.com>
2022-01-13patch_sync: Commit featuresJordan R Abrahams
This adds several important version control features: * Changes branches on set up. * Cleans up any changes made, even if upload fails. * Sends commits for review (enabled via CLI). * Enables CQ for CrOS. BUG=b:209493133 TEST=cargo test Change-Id: I8ab2650aae301c08fd80358162a285e46d44e3e1 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3379479 Reviewed-by: George Burgess <gbiv@chromium.org> Commit-Queue: Jordan Abrahams-Whitehead <ajordanr@google.com> Tested-by: Jordan Abrahams-Whitehead <ajordanr@google.com>
2022-01-13patch_sync: Add better command debuggingJordan R Abrahams
This commit allows the git commands to print out what the command was that they failed at. This is pretty helpful in debugging what went wrong. We don't capture the repo output ever, so that can remained piped to the terminal. But we need to do some trickery for the git cmd. BUG=b:209493133 TEST=None Change-Id: I389257fef1e3bf394fb4013588df6c78e83b733a Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3379478 Reviewed-by: George Burgess <gbiv@chromium.org> Reviewed-by: Michael Benfield <mbenfield@google.com> Tested-by: Jordan Abrahams-Whitehead <ajordanr@google.com> Commit-Queue: Jordan Abrahams-Whitehead <ajordanr@google.com>
2022-01-08patch_sync: Filter patches by platformJordan R Abrahams
Because some patch collections may contain patches which don't apply to our given platform, we don't want to attempt to lookup their hashes. If we do, we may encounter a file-not-found error, as those patches haven't been ported to this platform. Also fixes a bug where no_commit was inverted. BUG=b:209493133 TEST=N/A Change-Id: I8b86133dfc6361919174f9f9b392d756b4304d2b Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3364792 Reviewed-by: George Burgess <gbiv@chromium.org> Tested-by: Jordan Abrahams-Whitehead <ajordanr@google.com> Commit-Queue: Jordan Abrahams-Whitehead <ajordanr@google.com>
2022-01-04patch_sync: Add show subcommandJordan R Abrahams
This allows users to view the combined PATCHES.json, also known as the source of truth. Also makes some changes so that serializing the patches vec to a string is more natural. BUG=b:209493133 TEST=cargo test TEST=patch_sync show -s <cros_checkout> <android_checkout> Change-Id: Ic2054c0c3c7e0bb03b698339b75e9ccc6dd68c74 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3355361 Reviewed-by: Michael Benfield <mbenfield@google.com> Tested-by: Jordan Abrahams <ajordanr@google.com> Commit-Queue: Jordan Abrahams <ajordanr@google.com>
2022-01-04patch_sync: Add patch_parsing moduleJordan R Abrahams
At present, the patch_sync code does nothing as the base version only sets up the code for future reviews. This adds in the ability to parse the PATCHES.json files, check the differences across past versions, and transpose those patches to another repo. This parsing assumes the PATCHES.json now use the uniform schema, which the AOSP currently does, but CrOS may not by the time this commit lands. BUG=b:209493133 TEST=cargo build TEST=cargo test Change-Id: I8de01eae3d9555dbb000378516a5bf38e29fea8e Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3339405 Reviewed-by: George Burgess <gbiv@chromium.org> Tested-by: Jordan Abrahams <ajordanr@google.com> Commit-Queue: Jordan Abrahams <ajordanr@google.com>
2022-01-04patch_sync: Base code, version control managingJordan R Abrahams
This is the base code addition for the patch_sync project. This project attempts to synchronize LLVM patches and ChromeOS. This code is not meant to run, but simply be the starting point so that we break up the large code dump. This code is meant to compile, but is not meant to be run. It only adds the version control code which manages `repo` interactions. Additionally, this adds the full Cargo.toml (at least for the current patch stack). BUG=b:209493133 TEST=cargo build Change-Id: I903edba6e633ec53a7fa4d2e7b0c2aac1094933a Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3339404 Reviewed-by: George Burgess <gbiv@chromium.org> Tested-by: Jordan Abrahams <ajordanr@google.com> Commit-Queue: Jordan Abrahams <ajordanr@google.com>
2021-12-22llvm_tools: Update get_upstream_patch schemaJordan R Abrahams
The get_upstream_patch.py program is using the old schema for PATCHES.json, and appending with that old schema to the new format. This commit fixes this issue. Additionally, this ensures that the keys are sorted on write to prevent line-diffs from getting out of sync. BUG=b:209493133 TEST=get_upstream_patch $args --platform chromiumos --platform android Change-Id: I0f96843b0134daa27574a94f583efdd27f394a3f Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3352574 Reviewed-by: Christopher Di Bella <cjdb@google.com> Commit-Queue: Jordan Abrahams <ajordanr@google.com> Tested-by: Jordan Abrahams <ajordanr@google.com>
2021-12-21llvm_tools: Catch dup SHAs in get_upstream_patchJordan R Abrahams
If a duplicate SHA is listed in get_upstream_patch.py, we need to report this as an error and not do anything. This commit adds this functionality, as well as some structures to make adding this feature easier. BUG=b:187795686 TEST=get_upstream_patch.py $args --sha $SHA --sha $SHA Change-Id: I56f1159d5bdd34c52efc00e991b7f1333b7ba3f7 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3351166 Commit-Queue: Christopher Di Bella <cjdb@google.com> Reviewed-by: Christopher Di Bella <cjdb@google.com> Commit-Queue: Jordan Abrahams <ajordanr@google.com> Tested-by: Jordan Abrahams <ajordanr@google.com>
2021-11-15llvm_tools: Make skip_dependencies optionalRyan Beltran
This CL makes new skip_denedencies arg in get_upstream patch default to False so that the new behavior introduced in CL:3255665 is disabled by default. This is intended to fix a failure in the nightly revert checker that was preventing cherry_picks. BUG=b:206461485 TEST=Local test of get_upstream_patch.py, and nightly_revert_checker_test.py Change-Id: Ia7586b69f9aefb5ca820a60eed593d45a4b9a5bb Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3282951 Reviewed-by: Christopher Di Bella <cjdb@google.com> Commit-Queue: Christopher Di Bella <cjdb@google.com> Tested-by: Christopher Di Bella <cjdb@google.com> Auto-Submit: Ryan Beltran <ryanbeltran@chromium.org>
2021-11-04llvm_tools: adds a way to skip dependencies when pulling from PhabChristopher Di Bella
Phabricator has a way to list dependencies, which Arcanist will pull in succession to make sure the patch is correct. This is fine until some of the patches land in main: Arcanist then gets confused and doesn't always correctly apply the patches. This patch makes it possible to skip dependencies by adding a new flag to our upstream patch getter. The current design is only going to permit `--skip_dependencies` when there's exactly one `--differential` patch, since `--skip_dependencies` is a global option. If we're finding this to be a common nuisance, then we can look at redesigning the feature to allow lists of differentials with skipped dependencies. ``` ./get_upstream_patch.py --differential D12345 --skip_dependencies ./get_upstream_patch.py --cl 12345 --differential D45678 --skip_dependencies ./get_upstream_patch.py --cl 12345 --skip-dependencies ./get_upstream_patch.py --differential D12345 \ --differential D45678 \ --skip_dependencies ``` BUG=b:204779256 TEST=Tested locally Change-Id: I89032cba82c17c9f844bfac3cba330d173826ffd Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3255665 Commit-Queue: Christopher Di Bella <cjdb@google.com> Tested-by: Christopher Di Bella <cjdb@google.com> Auto-Submit: Christopher Di Bella <cjdb@google.com> Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org>
2021-10-28Replace non-inclusive languageJordan R Abrahams
Wherever possible, this commit removes non-inclusive languages as per COIL policy. Some of this language exists in dead code, so this commit also removes confirmed dead code. Also some minor lint fixes to get the code past the lint checks. BUG=b:204057159 TEST=Presubmit tests Change-Id: I299bfbe5dfcbe761efc5dcb29caea2337abe3f30 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3243940 Tested-by: Jordan R Abrahams <ajordanr@google.com> Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org>
2021-10-27llvm_tools: fix yapf warningsRyan Beltran
This CL applies fixes automatically generated by yapf during presubmit hook as well as a few manual fixes: * Removing erroneous `verbose` argument from call to RunTryJobs * Replacing / with parenthsis for long lines * Shortened some variable names to fix lines that were too long BUG=None TEST=rerean unit tests Change-Id: Ic6ed4bb74f067dd5c67991481caf75d8829bf86e Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3248930 Tested-by: Ryan Beltran <ryanbeltran@chromium.org> Auto-Submit: Ryan Beltran <ryanbeltran@chromium.org> Commit-Queue: Manoj Gupta <manojgupta@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2021-10-27llvm_tools: Add lldb_server to updated_packagesRyan Beltran
This CL adds dev-util/lldb-server to the list of packages updated by various tools. Additionally, it updates the default packages in update_chromeos_llvm_hash.py so that we don't have to manually supply them anymore, and refactors other scripts to use that list instead of maintaining their own. BUG=b:196602751 TEST=`python3 -m unittest discover -s . -p '*_unittest.py'` Change-Id: I0856c8f854dc2cbd0f2f21b71db96d3ef000caa8 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3248929 Tested-by: Ryan Beltran <ryanbeltran@chromium.org> Auto-Submit: Ryan Beltran <ryanbeltran@chromium.org> Commit-Queue: Manoj Gupta <manojgupta@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2021-09-20Clean up cros-logging-import pylint attributesJaques Clapauch
BUG=b:200542097 TEST=None Change-Id: I82f4011d499b323a4eceda2949e103dc1559299f Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3171413 Commit-Queue: Jaques Clapauch <jaquesc@google.com> Commit-Queue: Chris McDonald <cjmcdonald@chromium.org> Commit-Queue: George Burgess <gbiv@chromium.org> Tested-by: Jaques Clapauch <jaquesc@google.com> Auto-Submit: Jaques Clapauch <jaquesc@google.com> Reviewed-by: Chris McDonald <cjmcdonald@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org>
2021-08-11llvm_tools: update llvm major in non-llvm ebuildsRyan Beltran
This CL implements updating the llvm major version in non-llvm ebuilds. An example of the current behavior (which uprevs the major version only for the LLVM ebuild) can be seen here: https://crrev.com/c/3080644/2 BUG=b:196193631 TEST=./update_chromeos_llvm_hash_unittest.py Change-Id: Icf82f0a8a4a01b255d0eeb242fe139aaa6957920 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3088319 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Commit-Queue: Ryan Beltran <ryanbeltran@chromium.org> Tested-by: Ryan Beltran <ryanbeltran@chromium.org>
2021-08-09llvm_tools: more informative message when checking out llvm-project-copyBob Haarman
We keep a clone of the LLVM repository, which we create if it did not already exist. This takes a while, so we inform the user. However, the message did not specify where we are cloning to or why. This change makes the message take the form: Checking out LLVM to '<path>' so that we can map between commit hashes and revision numbers. This may take a while, but only has to be done once. BUG=b:192699914 TEST=ran llvm_tools/get_llvm_hash.py with a few parameters Change-Id: I86d83f03ab293c5cfb3014e80175c674740a2818 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3078689 Tested-by: Bob Haarman <inglorion@chromium.org> Reviewed-by: Jordan R Abrahams <ajordanr@google.com> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2021-08-04llvm_tools: use upstream revert_checkerGeorge Burgess IV
Sourcing this properly from LLVM is awkward, since at the least, we'd need to keep an LLVM git repo around. On top of that, we'd need to come up with some way to pin this at a specific version (since the evolution of LLVM ideally shouldn't impact the functionality of bits here). Rather than dealing with all of that, let's just copy this from the repo and declare victory. Changes should be super rare anyway. :) BUG=chromium:194731505 TEST=./nightly_revert_checker_test.py Change-Id: I6179cae57d01bee7cae7883c93f71cb9c2f1f3d1 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3068861 Reviewed-by: Bob Haarman <inglorion@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2021-08-03llvm_tools: add newline at the end of PATCHES.jsonJian Cai
This appeases the "file needs a trailing newline" lint warning when running `repo upload` with local changes created by get_upstream_patch.py. BUG=b:194873857 TEST=`repo upload` passed. Change-Id: Icc4bf60009182290f2d0beb10f5ee7fefe14e044 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3068076 Reviewed-by: George Burgess <gbiv@chromium.org> Tested-by: Jian Cai <jiancai@google.com>
2021-08-03git_llvm_rev: prefer upstream/* branches over non-upstream/ branchesGeorge Burgess IV
It seems that LLVM has an old precedent of `release/${version}` branches, but has swapped to naming new branches `upstream/release/${version}`. This includes a retroactive addition of `upstream/release/${version}` for old versions, which makes the `ValueError` here get raised. Prefer the newer, shinier names. BUG=b:194731505 TEST=./git_llvm_rev_test.py Change-Id: If170c004eae0edf1a69792207e5c681deb10e50b Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3068860 Reviewed-by: Bob Haarman <inglorion@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2021-07-23Revert "[llvm_tools] automatically uprevs sys-devel/gcc whenever LLVM_NEXT ↵Manoj Gupta
is changed" This reverts commit a5725194cabd4244fd04c106c3a8d8b0d246b751. Reason for revert: breaks llvm-next testing. b/194493350 Original change's description: > [llvm_tools] automatically uprevs sys-devel/gcc whenever LLVM_NEXT is changed > > BUG=b:192996912 > TEST=update_chromeos_llvm_hash_unittest.py verified via CQ > > Change-Id: Id06f054fdc4400c9b9af48c5383172192cb9a8a3 > Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3021576 > Tested-by: Christopher Di Bella <cjdb@google.com> > Auto-Submit: Christopher Di Bella <cjdb@google.com> > Reviewed-by: Luis Lozano <llozano@chromium.org> > Commit-Queue: Christopher Di Bella <cjdb@google.com> Bug: b:192996912, b:194493350 Change-Id: I3a8e70bc20f09d92d594c1821b90157b6b5aba18 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3048754 Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
2021-07-14[llvm_tools] automatically uprevs sys-devel/gcc whenever LLVM_NEXT is changedChristopher Di Bella
BUG=b:192996912 TEST=update_chromeos_llvm_hash_unittest.py verified via CQ Change-Id: Id06f054fdc4400c9b9af48c5383172192cb9a8a3 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3021576 Tested-by: Christopher Di Bella <cjdb@google.com> Auto-Submit: Christopher Di Bella <cjdb@google.com> Reviewed-by: Luis Lozano <llozano@chromium.org> Commit-Queue: Christopher Di Bella <cjdb@google.com>
2021-05-25llvm_tools: detect cros checkout for default chroot_pathRyan Beltran
This CL improves the assumptions made when selecting the default chroot path in update_chromeos_llvm_hash.py. Previously, it assumed the chrome source tree was installed to ~/chromiumos which didn't work for people with multiple chroots or with different naming. Now we detect whether or not we are inside of the cros source tree and use that cros checkout if so (otherwise defaulting back to old logic). BUG=None TEST=./update_chromeos_llvm_hash.py produced CL:2912507 without specifiying chroot_path manually Change-Id: Idb6dfc18a6f7a95c36eba4209dc43e08e373d6e3 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2912956 Commit-Queue: Ryan Beltran <ryanbeltran@chromium.org> Tested-by: Ryan Beltran <ryanbeltran@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2021-05-25llvm_tools: Fix lint warnings in llvm update toolsRyan Beltran
This CL fixes some glint errors that were causing repo upload to reqiure a no-verify flag, it also includes yapf formatting. BUG=None TEST=Reran unit tests in affected files Change-Id: Ifa9827f204d2b7f1973a901722e0f5390ef850aa Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2912511 Commit-Queue: Ryan Beltran <ryanbeltran@chromium.org> Tested-by: Ryan Beltran <ryanbeltran@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2021-05-25llvm_tools: llvm version detection for roll toolsRyan Beltran
This CL implements version detection for update_chromeos_llvm_hash so that uprevs will automatically get the correct major version in the llvm ebuild. BUG=b:186802799 TEST=Updated unit tests and ran tools manually Change-Id: Id3a21ff7f708ce38303e27202428fb3bb519bafe Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2910173 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Commit-Queue: Ryan Beltran <ryanbeltran@chromium.org> Tested-by: Ryan Beltran <ryanbeltran@chromium.org>
2021-05-15llvm_tool: make either --sha or --differential required.Jian Cai
This fixed the following error when only --sha or --differential is used. TypeError: can only concatenate list (not "NoneType") to list BUG=b:187554374 TEST=get_upstream_patch.py --chroot_path <chroot> --start llvm --sha <sha> TEST=get_upstream_patch.py --chroot_path <chroot> --start llvm --differential <rev> TEST=get_upstream_patch.py --chroot_path <chroot> --start llvm Change-Id: Ie50e1c081a6a41859d7e4db7a2d815e48b84fa3d Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2898172 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Tested-by: Jian Cai <jiancai@google.com>
2021-05-10llvm_tool: support differential reviews in the cherry-pick tool.Jian Cai
Rename cherrypick_cl.py to get_llvm_upstream.py, and add --differential option for creating local patches based on differential reviews. This should help testing experimental Clang/LLVM patches on ChromeOS. BUG=chromium:1202311 TEST=Successfully run the script with --sha $SHA --differential $DIFFERENTIAL_REVISION Change-Id: I7bd1704d1079352db52fc4690ef453436fe854de Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2861111 Tested-by: Jian Cai <jiancai@google.com> Reviewed-by: George Burgess <gbiv@chromium.org>
2021-04-28llvm_tools: refactor automatic LLVM bisectionJian Cai
This removes update_all_tryjobs_with_auto.py and move a function to its only user module. Also stopped raising errors when a build is not ready for "cros buildresult" to avoid confusion. BUG=chromium:1151055 TEST=local and CQ tests. Change-Id: I1591d17f1fe76cf6fb223c18ab0c96349982f53c Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2551918 Tested-by: Jian Cai <jiancai@google.com> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2021-04-28llvm_tools: fix LLVM bisection tool.Jian Cai
The changes the LLVM bisection tool to adpat to the new output of "cros tryjob --json". BUG=chromium:1203504 TEST=Successfully launched the tool. Change-Id: I1d6676fb966aee68ce7af177a65de0358a73baf2 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2855895 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Tested-by: Jian Cai <jiancai@google.com>
2021-03-10nightly_revert_checker: update branch nameGeorge Burgess IV
Android updated this branch name, so we have to reflect that here. BUG=b:182277561 TEST=None Change-Id: Ia709901bf8544eb751bc4c736f9e351640c52329 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2746504 Reviewed-by: Pirama Arumuga Nainar <pirama@google.com> Reviewed-by: George Burgess <gbiv@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2021-02-24fixes failure when a commit has already been cherry-pickedChristopher Di Bella
cherrypick_cl.py fails whenever a commit is already present, but the revert checker should simply note this and move on. BUG=chromium:1181376 TEST=nightly_revert_checker_test.py Change-Id: I04500fef98a95607df3cb23bf0a901d3ba5062a7 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2716971 Auto-Submit: Christopher Di Bella <cjdb@google.com> Commit-Queue: Manoj Gupta <manojgupta@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Tested-by: Manoj Gupta <manojgupta@chromium.org>
2021-02-12adds cherry-picking to the nightly revert checkerChristopher Di Bella
BUG=chromium:1085465 TEST=nightly_revert_checker_test.py Change-Id: I8069bb3f6ca8d07f54568b85373910a192d11ea5 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2680123 Auto-Submit: Christopher Di Bella <cjdb@google.com> Tested-by: Manoj Gupta <manojgupta@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org>
2021-01-28llvm_tools: make the lexan 4c uploader ignore old clang(s)George Burgess IV
Lexan saw a crash from clang 9.0.0 uploaded to their bucket. There's ~nothing we care to do about these crashes, and 4c even fails to parse the header of them. Seems best to warn & ignore these cases automatically (if an actual _user_ wants this functionality, 4c can easily be fixed, but I doubt anyone cares) No test is provided since submit_test_case has quite a few side-effects. Doesn't seem worth splitting it or mocking a lot to add an automated test for this peephole. BUG=None TEST=manually ran on chrotomation Change-Id: Iae131913e7b9494015262e41af8ad4193c9d62f6 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2657875 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2021-01-21llvm_tools: Fix bisection with main branchManoj Gupta
1. Use 'origin/main' when checking out llvm repo first time. 2. Use 'cros/main' when checking out chromiumos-overlay. BUG=None TEST=local bisection run Change-Id: Ie4b2d6fcd85c730ddeec9053bb7cfc13235cca34 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2643561 Reviewed-by: George Burgess <gbiv@chromium.org> Tested-by: Manoj Gupta <manojgupta@chromium.org>
2020-12-07llvm_tools: update upstream branch names to point to `main`George Burgess IV
LLVM recently renamed their `master` branch to `main` upstream. This changes our tooling to search for the latter name instead. BUG=None TEST=Unittests; ran `git llvm-rev --sha HEAD` in an up-to-date llvm repo. Change-Id: Icdc6dac48fda7697d18d0d77935e46670273bf84 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2577643 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2020-11-21llvm_tools: pass cl number as str to avoid type errorBob Haarman
llvm_bisection.py invokes the Gerrit CLI to abandon CLs created as part of the bisection. It passes a CL number to the command. Before this change, the CL number was passed as an int, which leads to the following error: Traceback (most recent call last): File "./auto_llvm_bisection.py", line 103, in main bisection_ret = llvm_bisection.main(args_output) File "/usr/local/google/home/inglorion/chromiumos/src/third_party/toolchain-utils/llvm_tools/llvm_bisection.py", line 364, in main subprocess.check_output([gerrit, 'abandon', build['cl']], File "/usr/lib/python3.8/subprocess.py", line 411, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "/usr/lib/python3.8/subprocess.py", line 489, in run with Popen(*popenargs, **kwargs) as process: File "/usr/lib/python3.8/subprocess.py", line 854, in __init__ self._execute_child(args, executable, preexec_fn, close_fds, File "/usr/lib/python3.8/subprocess.py", line 1637, in _execute_child self.pid = _posixsubprocess.fork_exec( TypeError: expected str, bytes or os.PathLike object, not int With this change, we convert the value to an str first, avoiding the type error. BUG=chromium:1151325 TEST=run auto_llvm_bisection.py Change-Id: I2b151c76757067f536712f69b14d6be549ee7c75 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2551789 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Reviewed-by: Jian Cai <jiancai@google.com> Tested-by: Bob Haarman <inglorion@chromium.org>
2020-10-09llvm_tools: add exist_ok=True to state dir creationGeorge Burgess IV
we just want to make sure that the state file directory exists here. BUG=None TEST=Ran Change-Id: I563821a357e2863c0a71570cfcb7319401a03778 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2460837 Tested-by: George Burgess <gbiv@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2020-10-08llvm_tools: use gsutil.py instead of gsutilGeorge Burgess IV
chrotomation.mtv has gsutil.py from depot_tools, and no pre-installed gsutil. seems easiest to just add another dependency on gsutil.py. BUG=None TEST=Ran on chrotomation Change-Id: I9833c0fb1e11d31b3a4c17375e0f296ff5a83a8e Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2462043 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2020-10-07llvm_tools: add a lexan crash autouploaderGeorge Burgess IV
This CL adds an autouploader for crashes that land in Lexan's bucket. This autouploader uploads them to 4c, much like how `bisect_clang_crashes.py` does. The intent is to run this regularly on chrotomation. BUG=None TEST=Ran it Change-Id: I7cfbe463d89994f6ed3f750c9e8277e1fad0738e Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2451306 Reviewed-by: Jian Cai <jiancai@google.com> Tested-by: George Burgess <gbiv@chromium.org>
2020-10-02llvm_tools: abandon CLs after completing LLVM bisection.Jian Cai
Abandon CLs created for bisection if LLVM bisection successfully found the root cause. BUG=chromium:1081457 TEST=Verified locally. Change-Id: I3702c38432fbaf87f7df62418c0b29cbe4ca722a Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2382420 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org> Tested-by: Jian Cai <jiancai@google.com>
2020-10-02llvm_tools: refactor LLVM bisection toolJian Cai
BUG=chromium:1081457 TEST=Verified locally. Change-Id: Ic662a7bb697efb920a83255d3da87a0031e694ed Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2371502 Tested-by: Jian Cai <jiancai@google.com> Reviewed-by: George Burgess <gbiv@chromium.org>
2020-09-21llvm_tools: add a script to fetch uploaded versions of LLVMGeorge Burgess IV
Along with sdk tarballs, the SDK builder publishes manifest files containing the versions of packages contained by a particular tarball. This script is intended to extract these versions, so we can more confidently answer the question "when did this LLVM change _actually_ take effect?" Example usage: ``` [ub] /m/s/g/c/s/t/t/llvm_tools [ 10ms ] ~> ./fetch_cros_sdk_rolls.py cros-sdk-2020.08.28.002952.tar.xz.Manifest: 11.0_pre394483_p20200618-r11 cros-sdk-2020.09.01.145913.tar.xz.Manifest: 11.0_pre394483_p20200618-r11 cros-sdk-2020.09.02.121932.tar.xz.Manifest: 11.0_pre394483_p20200618-r11 cros-sdk-2020.09.03.075651.tar.xz.Manifest: 11.0_pre394483_p20200618-r11 cros-sdk-2020.09.04.035039.tar.xz.Manifest: 11.0_pre394483_p20200618-r11 cros-sdk-2020.09.05.000203.tar.xz.Manifest: 11.0_pre394483_p20200618-r11 cros-sdk-2020.09.05.193017.tar.xz.Manifest: 11.0_pre394483_p20200618-r11 cros-sdk-2020.09.06.145537.tar.xz.Manifest: 11.0_pre394483_p20200618-r11 cros-sdk-2020.09.07.100426.tar.xz.Manifest: 11.0_pre394483_p20200618-r11 cros-sdk-2020.09.08.234732.tar.xz.Manifest: 11.0_pre394483_p20200618-r11 cros-sdk-2020.09.09.204657.tar.xz.Manifest: 11.0_pre394483_p20200618-r11 cros-sdk-2020.09.11.112254.tar.xz.Manifest: 11.0_pre399094_p20200824-r1 cros-sdk-2020.09.14.044644.tar.xz.Manifest: 11.0_pre399094_p20200824-r2 cros-sdk-2020.09.15.034200.tar.xz.Manifest: 11.0_pre399094_p20200824-r2 cros-sdk-2020.09.15.235650.tar.xz.Manifest: 11.0_pre399094_p20200824-r2 cros-sdk-2020.09.16.195018.tar.xz.Manifest: 11.0_pre399094_p20200824-r3 cros-sdk-2020.09.17.182131.tar.xz.Manifest: 11.0_pre399094_p20200824-r3 cros-sdk-2020.09.18.133717.tar.xz.Manifest: 11.0_pre399094_p20200824-r3 cros-sdk-2020.09.19.085133.tar.xz.Manifest: 11.0_pre399094_p20200824-r3 cros-sdk-2020.09.20.033151.tar.xz.Manifest: 11.0_pre399094_p20200824-r3 ``` This is literally a trivial wrapper around 2 gsutil commands and very lite JSON wrangling, so I didn't include a test. Happy to add one if people feel strongly that it's necessary. BUG=None TEST=Ran the script Change-Id: If22a5faf89e601709a3304473bab2dba71ccd64c Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2422663 Tested-by: George Burgess <gbiv@chromium.org> Reviewed-by: Tiancong Wang <tcwang@google.com>
2020-09-17toolchain-utils: Update README.md for llvm_tools.Caroline Tice
This corrects an error, a typo, and an omission I discovered in the README.md instructions. BUG=None TEST=None Change-Id: I19c46d87bf326c9ab320dc7bb23b4a8dae5573fe Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2416246 Auto-Submit: Caroline Tice <cmtice@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Commit-Queue: Caroline Tice <cmtice@chromium.org> Tested-by: Caroline Tice <cmtice@chromium.org>
2020-09-15llvm_tools: save intermediate state even if a step failsGeorge Burgess IV
When uploading multiple crashes, a network error may happen (or anything else, really). This isn't going to save us from SIGKILLs or power-offs (it's impossible to save us 100% from either of those, since there's always going to be a race between writing the JSON file and submitting the test-case), but it should keep us from submitting duplicate crash reports in most cases. This also has us sort results from `gsutil ls`, since that may not always print things in a deterministic order. BUG=None TEST=Ran the script; ^C'ed it in the middle. Change-Id: I9695d83db6fb8161dc6fce16b13980c8eacf219d Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2412482 Reviewed-by: Jian Cai <jiancai@google.com> Tested-by: George Burgess <gbiv@chromium.org>
2020-09-15llvm_tools: tweak script verbosityGeorge Burgess IV
At the moment, this script is silent except for when gsutil sneakily prints file copying status to stdout/stderr. Since we're already configuring `logging` here, be a bit more descriptive about our current status. BUG=None TEST=Ran the script. Change-Id: Ibcdb0f6cd488e70b9f810a9ca615556a7d672948 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2412481 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2020-08-03llvm_tools: add a tool to bisect clang crashesJian Cai
Add a tool that downloads clang crash diagnoses from chrome-toolchain-artifacts/clang-crash-dignoses and send them to 4c server for bisection. BUG=chromium:1056904 TEST=local tests Change-Id: I75081259705c236a20173fc591d080c6b0504cc5 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2325008 Tested-by: Jian Cai <jiancai@google.com> Reviewed-by: George Burgess <gbiv@chromium.org>
2020-07-29llvm_tools: clarify that syncing all of llvm is only done onceGeorge Burgess IV
There was some confusion about the intended frequency of LLVM syncs. Adding a bit to the "syncing" message should hopefully clear that up. BUG=None TEST=None Change-Id: I2454267602d9eaab79073882b172d61b3cb98daa Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2321565 Reviewed-by: Luis Lozano <llozano@chromium.org> Tested-by: Luis Lozano <llozano@chromium.org>
2020-07-28llvm_tools: check for uncommited changes before cherry-picksJian Cai
Exit when uncommited changes in chromiumos-overlay repo is detected. BUG=chromium:1057428 TEST=local tests. Change-Id: I6dacfb67028036b415672c8fba622b49c4ae8b76 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2321560 Tested-by: Jian Cai <jiancai@google.com> Reviewed-by: George Burgess <gbiv@chromium.org> Reviewed-by: Luis Lozano <llozano@chromium.org>
2020-07-27llvm_tools: remove future importGeorge Burgess IV
Now that we're guaranteed to be on py3, this isn't needed. Mostly doing this to test a crbug feature. Fixed: 1109998 BUG=chromium:1109998 TEST=None Change-Id: I1f5d0c9c7c1297e9281be07ced4127e42546b3c8 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2321109 Tested-by: George Burgess <gbiv@chromium.org> Reviewed-by: Denis Nikitin <denik@chromium.org>
generated by cgit v1.2.3 (git 2.25.1) at 2024-06-27 18:11:59 +0000