aboutsummaryrefslogtreecommitdiff
path: root/compiler_wrapper
AgeCommit message (Collapse)Author
2021-03-09Merging 15 commit(s) from Chromium's toolchain-utilsPirama Arumuga Nainar
Merged commit digest: 028b14d toolchain-utils: Remove 'target' from distfiles path (for cleanup). bd1004a toolchain-utils: Fix hyphen/underscore issue for nightly tests. d5d49d2 afdo_metadata: Publish the new kernel profiles d796e2c default_remotes: fix a typo 9d35abc llvm_tools: update upstream branch names to point to `main` 369072a default_remotes: Update toolchain DUT names 4a8085e compiler_wrapper: move llvm-next options to clang options 2c27f0a llvm_tools: pass cl number as str to avoid type error 2421372 afdo_metadata: Publish the new kernel profiles cb138b5 afdo_metadata: Publish the new kernel profiles 8c34e96 crosperf: Fix BadChecksum failure on coral 6da42e5 afdo_metadata: Publish the new kernel profiles 84b5c8f compiler_wrapper: report parent processes' info in warnings tarballs 49187a5 afdo_metadata: Publish the new kernel profiles 602351c update_kernel_afdo: use gsutil.py instead of gsutil Change-Id: I6d94b7821404b85366f1f1c0b0f3fd8ecec8ded4
2020-11-26compiler_wrapper: move llvm-next options to clang optionsinglorion
The new Clang version has a number of new warnings which we disable. This CL makes those apply to llvm (current) as well as llvm-next. BUG=None TEST=go test Change-Id: I1f4929580953003d21676e7e72af4577592fb217 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2561725 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Tested-by: Manoj Gupta <manojgupta@chromium.org> Commit-Queue: Manoj Gupta <manojgupta@chromium.org>
2020-10-23compiler_wrapper: report parent processes' info in warnings tarballsGeorge Burgess IV
The bug I'm chasing down right now depends on environment variables, and could be due to a `make` invocation in one of N places. Having this context available to me is useful. It seriously pollutes the JSON we dump, but use of `jq` that I've put in the doc about this makes that a non-issue. BUG=None TEST=Ran on stress-ng. parent process info was dumped. Change-Id: Ib4d3b6c9887b4cf2a38472ef299c81af27e9a617 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2481829 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2020-10-21Merging 84 commit(s) from Chromium's toolchain-utilsPirama Arumuga Nainar
Merged commit digest: efb75cf Handle clangPostFlags for Android 356e7cf afdo_metadata: Publish the new kernel profiles 8373fd4 Revert "compiler_wrapper: enable -mharden-sls=all on aarch64" 5672f28 crosperf: refactor chrome src search aac9c23 toolchain_utils: replace gsutil with gsutil.py 2159420 llvm_tools: add exist_ok=True to state dir creation fd598ff rust_uprev: emerge rust instead of ebuilding it 8ecb53f rust_uprev: emerge arm-none-eabi-gcc & do it in parallel 6868239 rust_uprev: check for files in gs:// before downloading them a874714 rust_uprev: support -rN ebuilds 7624e2d llvm_tools: use gsutil.py instead of gsutil 44e5787 rust_tools: Fix the tool using real Rust uprev experiences. fff1c26 afdo_metadata: Publish the new kernel profiles 6d9e457 llvm_tools: add a lexan crash autouploader 92d7005 toolchain-utils: create pending_archives if needed d8187c7 toolchain-utils: Sync compiler wrapper changes with llvm changes. d184b9a crosperf: Update path search for chrome sources b9a4299 llvm_tools: abandon CLs after completing LLVM bisection. fb8fd5b compiler_wrapper: enable -mharden-sls=all on aarch64 089db67 llvm_tools: refactor LLVM bisection tool ab646e1 afdo_metadata: Update kernel AFDO profiles 8223d16 fix formatting/lint issues pointed out by repohooks 4f7eb71 rephrase some language in code and comments 8a9125c toolchain-utils: Update compiler wrapper to disable warning. e2cbc18 add unblocked_terms.txt to automatically check language 780b6e4 llvm_tools: add a script to fetch uploaded versions of LLVM 9ba2a4f compiler_wrapper: search compiler executable in $PATH 7b73dcd crosperf: Add verification of per-process profiles 18422a4 bisection: exit after updating kernel 4edc81e crosperf: Add support to run tests with ssh forwarding 38df970 toolchain-utils: Update README.md for llvm_tools. 563cdd9 llvm_tools: save intermediate state even if a step fails c1e7b46 llvm_tools: tweak script verbosity 29a3f18 afdo_metadata: Publish the new kernel profiles. 2e734a5 crosperf: replace statistics stdev with pstdev 5d21b27 afdo_metadata: Publish the new kernel profiles eaed177 afdo_tools: Update update_kernel_afdo to check previous branch 4d7093d cros_utils: clean up naming 98de93b afdo_metadata: Publish the new kernel profiles 40733c3 rust_tools: Provide a big hammer to do everything to uprev Rust 243ce37 compiler_wrapper: remove some flags from unsupport clang flags 20f7df6 crosperf: remove uses of numpy 7c7161d rust_tools: Refactor to create and remove Rust versions independently 59defeb afdo_metadata: Publish the new kernel profiles 0b767f9 pgo_tools: Add a monitor for LLVM PGO profile freshness 3dac9fa rust_tools: Add a tool to automatically generate a Rust uprev 1713d25 compiler_wrapper: use crashArtifactsDir to apply clang opts 0a377f4 compiler_wrapper: dump tricium clang-tidy crash information 2953a41 Update cleanup script to get chrome temp files too. 811b580 Update cleanup script to get chrome temp files too. 9f99023 afdo_metadata: Publish the new kernel profiles. 9726f50 llvm_tools: add a tool to bisect clang crashes f30ddd4 default_remotes: Update toolchain DUT names d1a9a25 compiler_wrapper: add -fexperimental-new-pass-manager e1db855 llvm_tools: clarify that syncing all of llvm is only done once 0869ed0 llvm_tools: check for uncommited changes before cherry-picks d81dc93 afdo_tools: Update script to sort profiles by timestamp 9259683 afdo_metadata: Update kernel AFDO profiles 4f29d44 llvm_tools: remove future import e8ef273 compiler_wrapper: Updated instructions for updating the wrapper 120da3f rust_tools: add a monitor for upstream gentoo commits + rust releases cb46500 compiler_wrapper: add -clang-tidy-flag support 6c2e5c4 afdo_metadata: Publish the new kernel profiles 7b9d2ba Put reference filters on 'git ls-remote'. 916add2 compiler_wrapper: rename update_compiler_wrapper.sh 94fd622 compiler_wrapper: remove -grecord-gcc-switches 6cfbb3d afdo_metadata: Update kernel AFDO profiles 8caccf6 afdo_metadata: Update kernel AFDO profiles edc59f7 presubmits: respect CHROMEOS_ROOT_DIRECTORY when entering the chroot d6f4dfa presubmits: add types to check-presubmit.py. cab4264 Update AFDO kernel profiles. f226ba1 Update owners file fab4f3c cros_util: fix up naming 8078daa cros_utils: clean up unused file/code dc996b9 compiler_wrapper: redirect clang crash dignoses 0c1730a debug_info_test: fix up naming ad5e320 debug_info_test: replace whitelist with allowlist 536aff5 llvm_tools: fix up naming efb96d1 android_merge_from_upstream: fix up naming 74a6b09 afdo_tools: add update_kernel_afdo script 163efaa wrapper: add support for Tricium clang-tidy 5cbe70b afdo_metadata: Publish the new kernel profiles 647df11 crosperf: Save cpuinfo and top logs 83785c3 wrapper: fork() before exec()ing Change-Id: Ib885bb07621423f91584f6a4fd02fe998bf45141
2020-10-21Handle clangPostFlags for AndroidPirama Arumuga Nainar
BUG=None TEST=None Change-Id: I55bf76106eb1635c0bb4b936eaf3b96eef1dbb44 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2490653 Reviewed-by: George Burgess <gbiv@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2020-10-13Revert "compiler_wrapper: enable -mharden-sls=all on aarch64"Jian Cai
This reverts commit fb8fd5be362ec7a52dc3be33859d31b0553c1038. Enabling the option caused 4.4 based arm64 devices failed to boot. Disable this option until the issue is fixed. BUG=chromium:1096302 TEST=Verified locally. Change-Id: I02d5a72423f3cf0097edff7ace85b23caf3ff554 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2469305 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Tested-by: Jian Cai <jiancai@google.com>
2020-10-05toolchain-utils: Sync compiler wrapper changes with llvm changes.Caroline Tice
This updates the compiler wrapper flags to match recent updates to compiler wrapper in sys-devel/llvm (for llvm-next). BUG=chromium:1133468 TEST=None Change-Id: Ibab9f0e1d0e35db3ba838e70a812efe2d5137a0a Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2448871 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Commit-Queue: Caroline Tice <cmtice@chromium.org> Tested-by: Caroline Tice <cmtice@chromium.org>
2020-10-02compiler_wrapper: enable -mharden-sls=all on aarch64Jian Cai
Migrate straight-line speculation vulnerability. Link: http://lists.llvm.org/pipermail/llvm-dev/2020-June/142109.html BUG=chromium:1096302 TEST=https://docs.google.com/spreadsheets/d/1EZbTYUdq2Y6Jpgy5PrCACB3GrInFy0ahI40IPexBZDM/edit#gid=336625829 Change-Id: I35e8f3cce53b86159f1705429774ceb803264f7d Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2436936 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Tested-by: Jian Cai <jiancai@google.com>
2020-09-22toolchain-utils: Update compiler wrapper to disable warning.Caroline Tice
This CL updates toolchain-utils to match recent compiler wrapper changes in sys-devel/llvm/files. It's purpose is to disable the new warning -Wcompound-token-split-by-space. Also update golden tests appropriately. BUG=chromium:1130080 TEST=Previous CL already tessted/committed. Change-Id: I261250dc7ed6f1be7c3aa4652d793c924e1dc18d Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2423619 Commit-Queue: Caroline Tice <cmtice@chromium.org> Commit-Queue: George Burgess <gbiv@chromium.org> Tested-by: Caroline Tice <cmtice@chromium.org> Auto-Submit: Caroline Tice <cmtice@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org>
2020-09-18compiler_wrapper: search compiler executable in $PATHJian Cai
Search the compiler executable in the directories named by PATH environment variable and add its parent directory as PYTHONPATH to locate bisect_driver.py BUG=chromium:1121018 TEST=verified locally Change-Id: If1c397bedfc03b929d73ae490c1bb8f11b4edea5 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2415130 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Tested-by: Jian Cai <jiancai@google.com>
2020-08-13compiler_wrapper: remove some flags from unsupport clang flagsJian Cai
LLVM implemented some flags that used to not be supported. Remove these flags from unsupported flags in the wrapper. BUG=chromium:1112929 TEST=tested locally. Change-Id: If91df0d35ac3db926067d3e0c5e213f6c5e3f390 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2346956 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Tested-by: Jian Cai <jiancai@google.com>
2020-08-10compiler_wrapper: use crashArtifactsDir to apply clang optsGeorge Burgess IV
Since we now have crashArtifactsDir as a more abstract concept, it seems good to apply '-fcrash-diagnostics-dir=' based on that, rather than having it as an explicit clang flag. BUG=chromium:1113442 TEST=FIXME Change-Id: I147779d7cf8806b3bc712c2998446158e87ec967 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2343975 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2020-08-10compiler_wrapper: dump tricium clang-tidy crash informationGeorge Burgess IV
This CL has us dump extra info when we observe a crash while using `WITH_TIDY=tricium`. In particular: - We'll now try to generate a -E'ed source file, and stash that in a subdirectory of our general crash_diagnostics directory. - Our JSON object that represents clang-tidy output has metadata about the aforementioned crash file, if it was generated. BUG=chromium:1113442 TEST=`go test`; ran locally with a crashy clang-tidy Change-Id: I577e5ec8e990af3b7cf1d8a9fea347582d5b227b Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2343974 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2020-07-30compiler_wrapper: add -fexperimental-new-pass-managerJian Cai
LLVM is currently build with DENABLE_EXPERIMENTAL_NEW_PASS_MANAGER=ON on Chrome OS. This is however not reflected in clang crash diagnoses. Adding -fexperimental-new-pass-manager to help with reproducibility on 4c. BUG=chromium:1111103 TEST=CQ tests. Change-Id: Ie01431dff397e982469e29c9132e21374354762d Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2327414 Reviewed-by: George Burgess <gbiv@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Tested-by: Jian Cai <jiancai@google.com>
2020-07-24compiler_wrapper: Updated instructions for updating the wrapperinglorion
This change adds some more details on how to test and commit changes to the compiler wrapper. BUG=None TEST=Followed the steps locally to verify that they work. Change-Id: I5a3d644e92fd94ed3792a59a30f39db08053390d Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2305371 Commit-Queue: Bob Haarman <inglorion@chromium.org> Tested-by: Bob Haarman <inglorion@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org> Reviewed-by: Jian Cai <jiancai@google.com>
2020-07-22compiler_wrapper: add -clang-tidy-flag supportGeorge Burgess IV
This CL adds support for specifying compiler flags that're meant to be passed only to clang-tidy. This allows for easier local experimentation with clang-tidy by users. BUG=chromium:982944 TEST=`WITH_TIDY=1 emerge-amd64-generic biod` with custom cflags; cq+1 on crrev.com/c/2308774 Change-Id: I654559d14a78ade4920fe37757fb5b249a997290 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2309152 Reviewed-by: Caroline Tice <cmtice@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2020-07-18compiler_wrapper: rename update_compiler_wrapper.shJian Cai
Rename the script to install_compiler_wrapper.sh to avoid confusion with chromiumos-overlay/sys-devel/llvm/files/update_compiler_wrapper.sh. BUG=None. Test=None. Change-Id: I39bbc3dc4a63fa480c394f2e92c1723033cdc321 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2304726 Tested-by: Jian Cai <jiancai@google.com> Reviewed-by: Bob Haarman <inglorion@chromium.org>
2020-07-14compiler_wrapper: remove -grecord-gcc-switchesBob Haarman
-grecord-gcc-switches causes the compiler and compiler flags to be stored in the produced debug information. This causes Goma builds to be non-deterministic, because the compiler path differs depending on whether the compiler was run locally or on Goma. This, in turn, causes cache pollution and needless work. Since we are not in fact using the information stored by -grecord-gcc-switches, this change disables it. We can re-enable it when we want to start using the information, provided we come up with another way to avoid the issues described earlier. BUG=chromium:1103065 TEST=Build an object file in chromeos-chrome and check that info is absent Change-Id: Ib8022a3a1baf2e4b5d3d9d1453deb67f539baff4 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2290624 Commit-Queue: Bob Haarman <inglorion@chromium.org> Tested-by: Bob Haarman <inglorion@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org> Reviewed-by: Tiancong Wang <tcwang@google.com>
2020-06-26compiler_wrapper: redirect clang crash dignosesJian Cai
Redirect all diagnoses of clang crashes to a dedicated directory using fcrash-diagnostics-dir so we can upload these files to the cloud. The flag was added to clangPostFlags to overwite redirection done on package level. BUG=chromium:1056904 TEST=None Change-Id: I78bfc77d7f3af8556b4308f2e71407d627627747 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2251038 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Commit-Queue: Jian Cai <jiancai@google.com> Tested-by: Jian Cai <jiancai@google.com>
2020-06-24wrapper: add support for Tricium clang-tidyGeorge Burgess IV
This adds a special `WITH_TIDY` mode that's specifically for use with Tricium. Crucially, this has us dump diagnostics as YAML, and stash a fair amount of information about each clang-tidy invocation in the same place where we dump YAML. These bits are intended to be used by the script added in I54ecc88d38faa4bfd502d632d3fd5c74734dabc0. BUG=chromium:1035951 TEST=Ran on all platform2 packages `emerge`able on amd64-generic. Change-Id: I63ef06dc6ddc016ebb6ba0c4a0cea8320fef7415 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2245785 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2020-06-20wrapper: fork() before exec()ingGeorge Burgess IV
This CL is a wild guess that tries to work around what appears to be an incredibly rare Go runtime bug (https://github.com/golang/go/issues/24484). As mentioned on the chromium bug, apparently execve is taking a while... somehow, and `pthread_create` fails with `EAGAIN` concurrently. So the intent here is to `fork()` + `exec()`. `fork()` will a) dispose of the thread that's trying to `pthread_create` b) isolate the `exec` in its own process, so the parent's runtime can spin up as many threads as it likes in peace. This slows down an 'ideal' kernel build (aka a build where ccache is a 100% hit rate, so the compiler wrapper's overheads are very pronounced) by ~4% in my measurements. BUG=chromium:1095528 TEST=sdk tryjob; CQ Change-Id: I73e1b238738b74c85def4709c09610f586fb04e5 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2252552 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2020-06-19Merging 2 commit(s) from Chromium's toolchain-utilsandroid-r-beta-3android-r-beta-2George Burgess IV
Merged commit digest: 2efe72e wrapper: print android-llvm@ when using the Android wrapper. 576a953 wrapper: tweak "toolchain doesn't exist" checks in the updater Bug: 159291805 Change-Id: Ie6a97f95668431be4a90636ff237567f1c77ffbd
2020-06-19wrapper: print android-llvm@ when using the Android wrapper.George Burgess IV
Otherwise, the wrapper might say: ``` Internal error. Please report to chromeos-toolchain@google.com. ``` to Android developers in some cases. BUG=b:159291805 TEST=`go test` Change-Id: I9a96068f02a2c6ce5e251c8754e267dfb1a30db1 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2252553 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2020-06-19wrapper: tweak "toolchain doesn't exist" checks in the updaterGeorge Burgess IV
When stderr is connected to a tty, equery will output things if it fails due to being unable to find a package. When it's not, equery will output nothing on failure. BUG=None TEST=./update_compiler_wrapper.sh Change-Id: I7d16d6a531f1f28fc0157873176d41f6881cd04b Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2252550 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2020-06-16Symlink to android_llvm_next_flags.go in toolchain/llvm_androidPirama Arumuga Nainar
This is a downstream/Android-only change to symlink to android_llvm_next_flags.go in toolchain/llvm_android. It, as the name suggests, has custom flags for Android's LLVM-next compiler. This file is maintained in the llvm_android project to track configurations for the Android toolchain in one project. Test: compiler_wrapper/build.py for Android and validate that flags are passed correctly. Change-Id: I7a535aa6b39119864024495c33427504cbcb3c64
2020-06-15wrapper: allow Android to specify their own llvm-flagsGeorge Burgess IV
Given the discussion on https://android-review.googlesource.com/c/platform/external/toolchain-utils/+/1330695, it sounds like our Android friends want to be able to tweak llvm-next flags without going through chromium-review + having to pull their changes back into Android. This CL allows them to make changes to their llvm-next flags entirely locally, and isolates CrOS' llvm-next flags from Android's. BUG=None TEST=Built the wrapper for CrOS and Android. Change-Id: If1bcfe762a4f368db9816557752651edc877f497 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2247003 Reviewed-by: Pirama Arumuga Nainar <pirama@google.com> Reviewed-by: Tiancong Wang <tcwang@google.com> Tested-by: George Burgess <gbiv@chromium.org>
2020-06-11[android wrapper] Write warning report to stdout on AndroidPirama Arumuga Nainar
Write the warning report to stdout (surrounded by a tag <LLVM_NEXT_ERROR_REPORT>). Currently OUT_DIR only gets passed to local compiles, and so remote compiles can't do the double build. Even if they write to /tmp, RBE or Goma won't know to copy them as part of the output. Packaging the warnings reports is a problem as well since the additional build step needs extra surgery in soong (it's not possible to add a dangling, optional build step in Ninja). Writing the reports to stdout conveniently solves both these issues. We can just scrape the reports with a script. Originall reviewed at https://android-review.googlesource.com/c/platform/external/toolchain-utils/+/1328433/2. BUG=None TEST=build master with r391452 and check progress past new errors. Change-Id: I5f1f71faba002836067a82f6aa4b26d5ba8b7b99 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2239371 Commit-Queue: Pirama Arumuga Nainar <pirama@google.com> Tested-by: Pirama Arumuga Nainar <pirama@google.com> Reviewed-by: George Burgess <gbiv@chromium.org>
2020-06-11[compiler-wrapper] Handle double build for clang-tidyPirama Arumuga Nainar
For clang-tidy, the equivalent of -Werror is --warnings-as-errors (or -warnings-as-errors). The error message will also have "warnings-as-errors". So, 1. Trigger double-build if "warnings-as-errors" is in *stdout*. Clang-tidy will also fail if clang issues a Werror diagnostic. In this case, we'd get "clang-diagnostic" in stdout. Do a re-build in both these cases. 2. Remove flags containing "-warnings-as-errors" in the rerun. Unlike -Werror/Wno-error, clang-tidy doesn't have a -no-warnings-as-errors to override a prior -warnings-as-errors. Existing double-build processing will disable -Werror flags to Clang. Originally reviewed at https://android-review.googlesource.com/c/platform/external/toolchain-utils/+/1322200. BUG=None TEST=go test Change-Id: I18de245972da81e0ac3600f9098b71cec82e9e96 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2239370 Commit-Queue: Pirama Arumuga Nainar <pirama@google.com> Tested-by: Pirama Arumuga Nainar <pirama@google.com> Reviewed-by: George Burgess <gbiv@chromium.org>
2020-06-02compiler_wrapper: add a `static` optionGeorge Burgess IV
If this option is specified, we'll build a fully static binary, rather than having it link with libc. If unspecified, our behavior shouldn't differ from what we have today. BUG=chromium:1087435 TEST=CQ Change-Id: If3939336e4e10d0279501f56ffbed8272a5f546c Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2219362 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2020-05-04Revert "wrapper: Temporarily disable -Wmax-tokens"Tiancong Wang
This reverts commit 3b2358092e00ab7904a873b516ac6aac832c2277. According to crrev.com/c/2171058, the fix in chromium could work. Will wait till the patch submitted in chromium and CrOS uprevs to merge the change into CrOS. Reason for revert: The workaround is not needed anymore. Original change's description: > wrapper: Temporarily disable -Wmax-tokens > > As reported in crbug.com/1072209, -Wmax-tokens is causing latest > chrome fail to build. While we are blocked by Chrome uprevs, we > want to update the compiler by disabling the warning, until it's > fixed properly on upstream Chrome. > > BUG=chromium:1072209 > TEST=Local testing in CrOS > > Change-Id: I082dd342a802e80250d0fca7c4b15afb4e972e7e > Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2161770 > Commit-Queue: Tiancong Wang <tcwang@google.com> > Tested-by: Tiancong Wang <tcwang@google.com> > Reviewed-by: George Burgess <gbiv@chromium.org> Bug: chromium:1072209 Change-Id: I904fb3c4f7d125579667bef675be654a5465fad9 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2173949 Tested-by: Tiancong Wang <tcwang@google.com> Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Commit-Queue: Tiancong Wang <tcwang@google.com>
2020-04-22wrapper: Temporarily disable -Wmax-tokensTiancong Wang
As reported in crbug.com/1072209, -Wmax-tokens is causing latest chrome fail to build. While we are blocked by Chrome uprevs, we want to update the compiler by disabling the warning, until it's fixed properly on upstream Chrome. BUG=chromium:1072209 TEST=Local testing in CrOS Change-Id: I082dd342a802e80250d0fca7c4b15afb4e972e7e Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2161770 Commit-Queue: Tiancong Wang <tcwang@google.com> Tested-by: Tiancong Wang <tcwang@google.com> Reviewed-by: George Burgess <gbiv@chromium.org>
2020-04-08compiler_wrapper: assume /dev/null is a configure testGeorge Burgess IV
Linux does a lot of "is this flag supported" testing by passing /dev/null as an input file. Since this code is pretty heuristic-y anyway, let's see if we can get by with calling that a `configure`-like file. :) BUG=None TEST=emerge-eve chromeos-kernel-4_4; no more fatal warnings Change-Id: Ie1cd80b36e93a9269280345a1df006baa356a9f6 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2142404 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2020-04-03compiler_wrapper: make the update script not die on missing packagesGeorge Burgess IV
Fresh chroots might not have all of these GCCs emerged. Just `continue` if that's not the case. This also fixes failing lints. BUG=None TEST=Ran the script Change-Id: I57a2ca72dfbaea324ddb66423294e1b4dd8989e1 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2080660 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2020-04-01compiler_wrapper: disable -Wno-error on conftest-y filesGeorge Burgess IV
There are some ebuilds that (unfortunately) run configure steps inside of their regular `make` invocations. Rather than trying to fix those as they pop up, add a heuristic to our -Wno-error bits. This is similar to a heuristic goma added a while back when experimenting with full builds of CrOS using goma. BUG=None TEST=unittests Change-Id: I23efdc8ad5ea29621b105040bc33786896f6e486 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2132722 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2020-04-01compiler_wrapper: s/strings.Contains/bytes.Contains/George Burgess IV
Rather than turning all of stderr into a string here, we can turn the thing we're searching for into a []byte. Should shave off a few cycles, and is just as readable. BUG=None TEST=unittests Change-Id: Ib4cc916ada11e0d8e286775b15dd93998c8403aa Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2132721 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2020-04-01compiler_wrapper: don't write -Werror files on still-failed buildsGeorge Burgess IV
Not `return`ing here was troublesome for our monitoring infra, since it meant that _any_ clang error that mentioned '-Werror', with this support turned on, would turn into a json file. With configure steps and the like, this probably isn't the behavior that we want. BUG=None TEST=unittests Change-Id: I0669c5bd9473b687c34c23f284433a21e8a1d88c Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2132233 Commit-Queue: George Burgess <gbiv@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2020-03-20gcc: Add tigerlake to corei7 mappingEd Baker
GCC 4.9 does not support new architecture flags such as skylake or tigerlake. Bug=b:141363149 TEST=Verify unit tests pass cd ${chromium_os}/src/third_party/toolchain-utils export PYTHONPATH=$(readlink -f .):$PYTHONPATH ./run_tests_for.py ./compiler_wrapper/* Change-Id: I63a74cb8fa6d2b40a9890a1e61f5be7917321975 Signed-off-by: Edward Baker <edward.baker@intel.com> Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1959943 Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2020-03-13compiler_wrapper: Stop setting CCACHE_BASEDIRManoj Gupta
CCACHE_BASEDIR used to be set to sysroot so that ccache can reuse compilation results across different boards. This is not useful on builders and also less useful for board builds since we set closer to target CFLAGS now. It also breaks use of -fmacro-prefix-map option in clang/gcc. So rip it out and simplify the rest of code. BUG=chromium:983759 TEST=go test Change-Id: Ia99ca994c976ee1137c3d904a78f7e25931c70f6 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2103170 Commit-Queue: Manoj Gupta <manojgupta@chromium.org> Tested-by: Manoj Gupta <manojgupta@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org>
2020-03-13compiler_wrapper: Add "-fcommon" to clang invocationsManoj Gupta
Clang ToT has switched to "-fno-common" as default to match GCC 10 which breaks many packages. Upgrading to ToT versions of many packages does not fix the problem as packages upstream have not fixed them yet. Pass "-fcommon" in compiler wrapper to workaround this for now. This does not change the current clang behavior which defaults to "-fcommon" so this is a NFC. BUG=chromium:1060413 TEST=packages build with ToT clang; NFC for current clang Change-Id: I5a40e502db201615f45a7e4720105bebef2f2b0a Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2102905 Reviewed-by: George Burgess <gbiv@chromium.org> Commit-Queue: Manoj Gupta <manojgupta@chromium.org> Tested-by: Manoj Gupta <manojgupta@chromium.org>
2020-03-10wrapper: resolve /proc/self/cwd only when seen.George Burgess IV
This wrapper may run on Mac, which doesn't have /proc/self/cwd available. This also adds a test to be sure this selective Readlink() behavior continues to work. BUG=None TEST=SDK Tryjob; tests pass Change-Id: I6f6aaeb7ea45ab8b3d68299422f0594a689ecfcd Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2095763 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Reviewed-by: Tiancong Wang <tcwang@google.com> Tested-by: George Burgess <gbiv@chromium.org>
2020-03-05wrapper: make a test consider the caller's umaskGeorge Burgess IV
Apparently my umask on my glinux system is 027 outside of the chroot, so this test fails for me. Mask out any bits that can't be set due to our umask. BUG=None TEST=go test Change-Id: I683a1ab315479e277d8b9b1fd3b61c96818bc055 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2088433 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2020-03-05wrapper: correctly Join() paths for AndroidGeorge Burgess IV
filepath.Join(".", "foo") folds to "foo", which is fine as long as we're not Exec()ing the result of that Join(). If we do, we might execute a binary we weren't expecting to (e.g., one looked up from ${PATH}, rather than the one at ${CWD}). BUG=None TEST=`go test` Change-Id: Ifbd247c2e489c786d0a0d3342b5d1b61a7320796 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2088432 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2020-03-04compiler-wrapper: Update clang-tidy flags to match Android.Caroline Tice
The flags that we use to generate clang tidy warnings have gotten out of sync with the flags the Android team uses. This CL fixes that. BUG=None TEST=None Change-Id: I766af02fee111e2e2233612e8352de1fb0802c3b Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2088431 Reviewed-by: George Burgess <gbiv@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Commit-Queue: Caroline Tice <cmtice@chromium.org> Tested-by: Caroline Tice <cmtice@chromium.org>
2020-03-04compiler_wrapper: Stop disabling two warnings.Manoj Gupta
Chrome OS code now is clean of following warnings: "-Wreorder-init-list" and "-return-stack-address" so stop disabling them. BUG=chromium:1057384 TEST=warnings tested by CL:1898670 Change-Id: I568cf93a9e836ba936ea976312e9a78b5c473157 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2081707 Tested-by: Manoj Gupta <manojgupta@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org>
2020-03-01compiler_wrapper: pre-buffer stdin properly.George Burgess IV
This CL makes our compiler wrapper pre-buffer all stdin when clang is expected to depend on it. Otherwise, clang might not read all of the stdin it's given, so future invocations of clang will only be handed partial stdin. BUG=chromium:1052532 TEST=emerged the kernel; sdk tryjob Change-Id: I6660e6333a1e8cf4649ef38de5c91d0b700fe1b5 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2080852 Tested-by: George Burgess <gbiv@chromium.org> Reviewed-by: Tiancong Wang <tcwang@google.com>
2020-02-28When double-build-with-Wno-error, add -Wno-error=foo for -Werror=fooPirama Arumuga Nainar
This will gracefully handle cases where a warning enforced with -Werror got stricter in upstream. This change also needs updates to golden testdata. No additional tests added since this is covered by existing tests. Originally reviewed in https://android-review.googlesource.com/c/1244796. BUG=b:149836702 TEST=go test Change-Id: I94624deb258b15e131387377b99bfb1747dca1a1 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2079548 Tested-by: Pirama Arumuga Nainar <pirama@google.com> Reviewed-by: George Burgess <gbiv@chromium.org>
2020-02-28Change behavior of doubleBuildWithWNoError for AndroidPirama Arumuga Nainar
"Double build with -Wno-error" is always on in the Android useLlvmNext wrapper and always off in the production wrapper. The warnings JSON is written to $OUT_DIR/warnings_reports. NFC for the ChromeOS configs. Originally reviewed in https://android-review.googlesource.com/c/1242126. BUG=b:149836702 TEST=go test Change-Id: I5f1a43524772d991b5c64422189643cde61c3826 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2079547 Tested-by: Pirama Arumuga Nainar <pirama@google.com> Reviewed-by: George Burgess <gbiv@chromium.org>
2020-02-25compiler_wrapper: fix write() racesGeorge Burgess IV
If we make this function write its data atomically to the RUSAGE file, we can have all compilations log to this file, which would allow us to trivially log all of the build info for a given tryjob. BUG=chromium:1054897 TEST=nocturne-llvm-next-toolchain-tryjob. Local --nousepkg build_packages produced a file with 0 torn lines. Change-Id: Ic6b014af11e812111fa21a56133bbe1b6b31f954 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2066710 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Commit-Queue: George Burgess <gbiv@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2020-02-14toolchain-utils: fix python3 errors in go scriptsZhizhou Yang
There are several places go scrtips launching and generating python related code. This patch makes sure they are python 3 compatible. BUG=chromium:1042452, chromium:1011676 TEST=Passed all python and go tests. Change-Id: Ie531ceedb14ca2af94bd26f59ba4cab399ef1e6a Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2054081 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
2020-02-13compiler_wrapper: Use syscall.exec on platforms other than Chrome OSTiancong Wang
In crbug.com/1000863, it's reported that golang exec don't play well on Chrome OS, portage sandbox to be exact. That's when we start to use libc's exec. However, the wrapper is also used on non-Chrome OS platforms, such as Android, and linking against libc has no benefit, and might be the root cause of a recent bug. This patch adds code to selectively use Golang's or libc's exec, depending on the platform. BUG=chromium:1000863 BUG=b:144783188 TEST=Build the wrapper locally with Chrome OS and Android configurations Change-Id: Ifd6fa8223205536450b65f728060d7c5556d7619 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2051669 Tested-by: Tiancong Wang <tcwang@google.com> Commit-Queue: Tiancong Wang <tcwang@google.com> Reviewed-by: George Burgess <gbiv@chromium.org>
generated by cgit v1.2.3 (git 2.25.1) at 2024-06-29 15:14:33 +0000