diff options
| author | jonathanmetzman <31354670+jonathanmetzman@users.noreply.github.com> | 2021-03-03 14:57:29 -0800 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2021-03-03 14:57:29 -0800 |
| commit | 2cc07015ef3ca912b7286185d848fccba34d0af9 (patch) | |
| tree | e10745c509d24b68f362322a019cbd5e614183ab /infra | |
| parent | 196cc1dc35ca7ca4fc0a4f837821f8b9df0a24e0 (diff) | |
| download | oss-fuzz-2cc07015ef3ca912b7286185d848fccba34d0af9.tar.gz | |
[base-clang] Reduce build time by ~65% (#5280)
Reduce build time by doing the following:
1. Building the second stage clang build with a clang binary we download
from chromium.
2. Changing NPROC to be half of the cores instead of assuming it's 16
cores. This still addresses the OOM when building on GCB but speeds up
local building.
3. Don't install recommended packages and use --depth 1 when possible
(very minor improvements compared to the above).
In all this reduces local build time of base-clang from 32 minutes
to 11 minutes.
Because build times are reduced, it will be easier to
iteratively develop changes needed for #5170
Diffstat (limited to 'infra')
| -rwxr-xr-x | infra/base-images/base-clang/checkout_build_install_llvm.sh | 18 |
1 files changed, 8 insertions, 10 deletions
diff --git a/infra/base-images/base-clang/checkout_build_install_llvm.sh b/infra/base-images/base-clang/checkout_build_install_llvm.sh index c5d97d5bf..6d9fee7bf 100755 --- a/infra/base-images/base-clang/checkout_build_install_llvm.sh +++ b/infra/base-images/base-clang/checkout_build_install_llvm.sh @@ -15,10 +15,13 @@ # ################################################################################ -NPROC=16 # See issue #4270. The compiler crashes on GCB instance with 32 vCPUs. +# See issue #4270. The compiler crashes on GCB instance with 32 vCPUs, so when +# we compile on GCB we want 16 cores. But locally we want more (so use nproc / +# 2). +NPROC=$(expr $(nproc) / 2) LLVM_DEP_PACKAGES="build-essential make cmake ninja-build git python3 g++-multilib binutils-dev" -apt-get install -y $LLVM_DEP_PACKAGES +apt-get install -y $LLVM_DEP_PACKAGES --no-install-recommends # Checkout CHECKOUT_RETRIES=10 @@ -60,7 +63,7 @@ function cmake_llvm { # Use chromium's clang revision mkdir $SRC/chromium_tools cd $SRC/chromium_tools -git clone https://chromium.googlesource.com/chromium/src/tools/clang +git clone https://chromium.googlesource.com/chromium/src/tools/clang --depth 1 cd clang LLVM_SRC=$SRC/llvm-project @@ -89,11 +92,9 @@ fi git -C $LLVM_SRC checkout $LLVM_REVISION echo "Using LLVM revision: $LLVM_REVISION" -# Build & install. We build clang in two stages because gcc can't build a -# static version of libcxxabi -# (see https://github.com/google/oss-fuzz/issues/2164). +# Build & install. mkdir -p $WORK/llvm-stage2 $WORK/llvm-stage1 -cd $WORK/llvm-stage1 +python3 $SRC/chromium_tools/clang/scripts/update.py --output-dir $WORK/llvm-stage1 TARGET_TO_BUILD= case $(uname -m) in @@ -111,9 +112,6 @@ esac PROJECTS_TO_BUILD="libcxx;libcxxabi;compiler-rt;clang;lld" -cmake_llvm -ninja -j $NPROC - cd $WORK/llvm-stage2 export CC=$WORK/llvm-stage1/bin/clang export CXX=$WORK/llvm-stage1/bin/clang++ |