aboutsummaryrefslogtreecommitdiff
path: root/checkbuild.py
AgeCommit message (Collapse)Author
2019-08-29Fix `checkbuild.py --help`.Dan Albert
This does have the rather unfortunate behavior of requiring bootstrap to complete before the help can be displayed, but there isn't really a way around that. Test: ./checkbuild.py --help Bug: None Change-Id: If9b76166836bb4f0774629855a8cfc6ab4f64535
2018-11-14Move the bootstrapping code into its own library.Dan Albert
The ndk library is used by builds other than the NDK (binutils at the very least), so we need to make the bootstrapping code accessible to other builds. Test: ./checkbuild.py Bug: None Change-Id: I8efa3e8f871b7e1ee2de787a9151891b1b646d96
2018-11-13Use the bootstrap Python for external scripts.Dan Albert
These scripts make use of the NDK package, which is not going to maintain compatibility with Python 2. Test: ./checkbuild.py Bug: None Change-Id: Ia8327bb1146dd9a5418d637a19cecbb607c4746a
2018-11-12Break bootstrap dependencies on the ndk package.Dan Albert
The ndk package is Python 3 now, whereas the bootstrap may be run by Python 2. Rather than trying to maintain Python 2 compatibility in modules used by bootstrapping code (which would include ndk.paths, preventing us from taking advantage of pathlib), move all the bootstrap code directly into the bootstrapping script, and break dependencies on the ndk package. Temporarily there's a lot of duplication, but it's mostly the path handling code and that will be branching significantly soon since it can now make use of pathlib. Test: ./checkbuild.py Bug: None Change-Id: I5c12b304bc3a4b69aa71fe987b77449efa23718e
2018-11-09Bootstrap checkbuild.py with Python 3.Dan Albert
This allows us to implement checkbuild.py and its associated pieces exclusively in Python 3 even though it isn't available on the build servers. The version currently available in external/python/cpython3 is 3.6, so this gets us features even as new as type annotations and f-strings. On my P920 the bootstrapping step takes about a minute. When bootstrapping is complete a .bootstrapped file is placed in the out directory to avoid rebuilding Python every time. We can't go through our existing Python 2 as an intermediate step because we can't build the SSL (and therefore the hashlib) module for that version of Python because it's not compatible with whatever is on these machines (which might be BoringSSL?). Test: ./checkbuild.py Bug: None Change-Id: Ie01a586cdbfaf22da3025aabbc3ed931480b7742
2018-11-09Move checkbuild.py into the ndk package.Dan Albert
In preparation for adding a bootstrap phase to build Python 3. Test: ./checkbuild.py Bug: None Change-Id: Ic752f1c30ff03c972de3d1f19709b07082867345
2018-11-09Remove duplicate build module.Dan Albert
Test: ./checkbuild.py Bug: None Change-Id: I70bc424f123f41cf4c0b50dba6cabd9c3d49301d
2018-10-29Merge changes from topic "change-layout"Treehugger Robot
* changes: Put the new toolchain back in to toolchains/llvm. Fix pylint errors.
2018-10-25Remove ndk-depends.Elliott Hughes
Judging by the lack of interest on the web, it seems like this probably isn't much used. ReLinker (https://github.com/KeepSafe/ReLinker) seems like a better solution to native library loading problems anyway. Rather than rewrite ndk-depends in Python now, let's see if anyone actually still has a reasonable use for it. (Folks who just want readelf/llvm-readobj should probably just be using those already.) Bug: http://b/22085867 Test: builds Change-Id: I938ddd55586c4ff8768bc7b5e0b356f1c31abed1
2018-10-25Put the new toolchain back in to toolchains/llvm.Dan Albert
Moving this to an entirely separate directory will likely be too disruptive. While not very difficult to fix in any given build system, there's a significant number of third-party projects that do not even use standalone toolchains. Those few of them using Clang can be kept working for the time being (they'll still break when we remove $NDK/sysroot if they're using `--sysroot` like they should be rather than `-isystem`). This also saves us a small amount of space in the transition period. Although we still need to duplicate binutils, libc++, the sysroot, etc, we can at least dedup Clang. Test: ./checkbuild.py && ./run_tests.py Bug: https://github.com/android-ndk/ndk/issues/780 Change-Id: I5760e55240fc3ff0ed2fe5385b8dca9394327dd1
2018-10-25Fix pylint errors.Dan Albert
Test: pylint checkbuild.py Bug: None Change-Id: Iefd28ac54fbb7200fb7f6c708d1a9339c3c09090
2018-10-24ndk-stack: switch to python.Elliott Hughes
We've started to see cases (https://issuetracker.google.com/117306194) where addr2line chokes on DWARF information, so it's probably time to switch to llvm-symbolizer. While doing so, we may as well just switch to a small python script. This changes the output format, because the old format was significantly less useful. In addition to causing ridiculously long lines, it didn't handle inlining. This new script simply outputs whatever llvm-symbolizer has to say, on as many lines as llvm-symbolizer thinks are necessary to accurately represent the situation, rather than trying to parse and reformat it. This script also copies the `Abort message:` line, which is often the most useful line in the tombstone. Why not reuse the platform's `stack`? Because making that work and sharing the code seems like a lot of work for no immediate benefit: the NDK's needs seem a lot simpler, and the obvious improvements one could make to ndk-stack (automatically finding the symbols given an ndk-build/cmake project) aren't in the platform tool anyway. A python script as short as this hopefully encourages folks to roll their own better tools that fit in with however they keep track of their published builds. Example input: 10-24 19:19:57.714 5340 5340 F DEBUG : *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** 10-24 19:19:57.714 5340 5340 F DEBUG : Build fingerprint: 'Android/hikey960/hikey960:Q/PI/enh10051658:userdebug/test-keys' 10-24 19:19:57.714 5340 5340 F DEBUG : Revision: '0' 10-24 19:19:57.714 5340 5340 F DEBUG : ABI: 'arm' 10-24 19:19:57.714 5340 5340 F DEBUG : pid: 5336, tid: 5336, name: crasher >>> crasher <<< 10-24 19:19:57.714 5340 5340 F DEBUG : signal 35 (<debuggerd signal>), code -1 (SI_QUEUE), fault addr -------- 10-24 19:19:57.715 5340 5340 F DEBUG : Abort message: 'attempted to close file descriptor 3, expected to be unowned, actually owned by FILE* 0xe970400c' 10-24 19:19:57.715 5340 5340 F DEBUG : r0 00000000 r1 000014d8 r2 00000023 r3 ffa73890 10-24 19:19:57.715 5340 5340 F DEBUG : r4 000014d8 r5 ffa738a4 r6 e98efc70 r7 0000016b 10-24 19:19:57.715 5340 5340 F DEBUG : r8 ffa73688 r9 00000014 r10 000014d8 r11 00000000 10-24 19:19:57.715 5340 5340 F DEBUG : ip ffa73890 sp ffa73680 lr e9874737 pc e987474a 10-24 19:19:57.719 5340 5340 F DEBUG : 10-24 19:19:57.719 5340 5340 F DEBUG : backtrace: 10-24 19:19:57.719 5340 5340 F DEBUG : #00 pc 0002e74a /system/lib/libc.so (fdsan_error(char const*, ...)+266) 10-24 19:19:57.719 5340 5340 F DEBUG : #01 pc 0002e553 /system/lib/libc.so (android_fdsan_close_with_tag+386) 10-24 19:19:57.719 5340 5340 F DEBUG : #02 pc 0002ec37 /system/lib/libc.so (close+6) 10-24 19:19:57.719 5340 5340 F DEBUG : #03 pc 00001c93 /system/bin/crasher (do_action+786) 10-24 19:19:57.719 5340 5340 F DEBUG : #04 pc 00002a35 /system/bin/crasher (main+56) 10-24 19:19:57.719 5340 5340 F DEBUG : #05 pc 0009695d /system/lib/libc.so (__libc_init+60) 10-24 19:19:57.719 5340 5340 F DEBUG : #06 pc 0000180f /system/bin/crasher (_start_main+38) 10-24 19:19:57.719 5340 5340 F DEBUG : #07 pc 00000306 <anonymous:e9c56000> Example output: ********** Crash dump: ********** Build fingerprint: 'Android/hikey960/hikey960:Q/PI/enh10051658:userdebug/test-keys' Abort message: 'attempted to close file descriptor 3, expected to be unowned, actually owned by FILE* 0xe970400c' #00 0x0002e74a /system/lib/libc.so (fdsan_error(char const*, ...)+266) fdsan_error(char const*, ...) bionic/libc/private/bionic_inline_raise.h:0:3 #01 0x0002e553 /system/lib/libc.so (android_fdsan_close_with_tag+386) android_fdsan_close_with_tag bionic/libc/bionic/fdsan.cpp:0:7 #02 0x0002ec37 /system/lib/libc.so (close+6) close bionic/libc/bionic/fdsan.cpp:381:12 #03 0x00001c93 /system/bin/crasher (do_action+786) #04 0x00002a35 /system/bin/crasher (main+56) #05 0x0009695d /system/lib/libc.so (__libc_init+60) __libc_init bionic/libc/bionic/libc_init_dynamic.cpp:136:8 #06 0x0000180f /system/bin/crasher (_start_main+38) #07 0x00000306 <anonymous:e9c56000> Bug: http://b/22085867 Test: ran manually Change-Id: I80b77f13c32a92adb972191b7b9548249e58b3ce
2018-10-23Merge "Move ndk-gdb out of HostTools."Treehugger Robot
2018-10-23Move ndk-gdb out of HostTools.Dan Albert
Test: ./checkbuild.py ndk-gdb Test: ./checkbuild.py Bug: None Change-Id: Ib4cfa782a27a551115e54204205f484e80990d7b
2018-10-22Update Vulkan validation for split repo layoutCody Northrop
Test: python checkbuild.py Test: python build.py --installdir /tmp/foo Change-Id: Ieb0ad8abed64e96585def2acb9049bed282c5695
2018-10-19Merge changes Idec4db48,Ie0a74e2b,I094d3e0fDan Albert
* changes: Update r19 changelog. Master is now r20. Move the actual changelog into docs/changelogs.
2018-10-18Move the actual changelog into docs/changelogs.Dan Albert
This makes it easier for us to maintain the changelog when there's an active release branch. We make all our changes in master but cherry-pick to the release branch, so the changelogs for rN and rN+1 are the same. Test: ./checkbuild.py changelog Bug: None Change-Id: I094d3e0f8ee0559fc1a6b910f6c32df13830e97f
2018-10-18Use libcompiler_rt-extras with libgcc.Dan Albert
Test: ./checkbuild.py && ./run_tests.py Test: Manual inspection of the output libgcc.a script Bug: https://github.com/android-ndk/ndk/issues/294 Change-Id: I4822b99a716b8e0b1235fb27cf5f038573541f10
2018-10-17Move system_libs into meta.Dan Albert
This also moves to autogenerating this list based on the available sysroot, since we've apparently forgotten to add a few things. Test: ./checkbuild.py Test: Manual test to check that the warning still appears for items not in the list Bug: https://github.com/android-ndk/ndk/issues/801 Change-Id: I9b4320fdac260f2ce3cb9151a3078be2d990b56d
2018-10-12Merge "Install yasm to the new toolchain."Treehugger Robot
2018-10-12Build CRT objects using just-built toolchain.Dan Albert
Test: ./checkbuild.py && ./run_tests.py Bug: None Change-Id: If3e4268e9be53847b4c1923fac4a3424b0205f61
2018-10-12Install yasm to the new toolchain.Dan Albert
Test: ./checkbuild.py && ./run_tests.py --filter yasm Bug: None Change-Id: I2574c904a10b84227984c94b3c7f2c158b6edea2
2018-10-11Update to clang r339409b.Dan Albert
Test: ./checkbuild.py && ./run_tests.py Bug: None Change-Id: Ie84481b9bbc2d7b5483b7850e72aab752c2a0a01
2018-10-05Merge changes Ic9d213a7,Ib08c795cTreehugger Robot
* changes: Revert "Revert "Cleanup more build_support uses in checkbuild.py."" Revert "Revert "Refactor the build/install module interface.""
2018-10-05Revert "Revert "Cleanup more build_support uses in checkbuild.py.""Dan Albert
This reverts commit d3091ab582c75c9bc5522497695204254c2be60b. Test: ./checkbuild.py Bug: None Change-Id: Ic9d213a72301059a5fdcf5b7ab3195f40a59bf25
2018-10-05Revert "Revert "Refactor the build/install module interface.""Dan Albert
This reverts commit c93d87ee5a193bc2a8ba40809415602083f3cf90. Test: ./checkbuild.py Test: ./checkbuild.py --system windows64 Bug: None Change-Id: Ib08c795c7a86b0bb507283b28793ace940e2cd9d
2018-10-04Pass -gcc-toolchain in the GCC wrappers.Dan Albert
Test: arm-linux-androideabi-gcc -v foo.c is only as broken as r17's Bug: https://github.com/android-ndk/ndk/issues/805 Change-Id: Id4eab102b4ef2736d048f97dac5786ea197152e3
2018-10-02Revert "Refactor the build/install module interface."Dan Albert
This reverts commit e1db8f930120fc3e712f9c306819f6a3b4153f89. Reason for revert: Broke the Windows build Change-Id: I05510d5669b35d227f54a36bb6aae611507eec81
2018-10-02Revert "Cleanup more build_support uses in checkbuild.py."Dan Albert
This reverts commit f770b37d1a10c76b9cd347a3853894585507826d. Reason for revert: Other patch broke Windows build. Reverting to avoid merge conflict. Change-Id: I34a43b7773e71cd21d4d34aa6fd52db05bfcbc0b
2018-09-28Cleanup more build_support uses in checkbuild.py.Dan Albert
Test: ./checkbuild.py Bug: None Change-Id: I01cb28e522b9ee62900c058db3c63b6f33ee9f9a
2018-09-28Refactor the build/install module interface.Dan Albert
This change factors out out_dir, dist_dir, and args into a BuildContext object. This allows us to clean up many of the path-related interfaces in the build system. This also makes it easier for us to guarantee safe access to dependencies. Since modules now need some initialization, we can no longer trivial construct a new instance of a module to make use of it. Instead, we go through the module list contained by the BuildContext object and gate that access to only modules listed in the module's dependency list. Test: ./checkbuild.py Bug: None Change-Id: Idbf650e4606c0e370aaa6ac3393296b55d95e6a5
2018-09-27Split the toolchain module in two.Dan Albert
The libc++ module needs to use a mostly-built toolchain to build itself, so we need to separate these stages. Test: ./checkbuild.py Bug: None Change-Id: I4ac482b5ee7645f3e8e3135659d91ef60a5dfe8b
2018-09-26Fix the notice group for the toolchain module.Dan Albert
Test: ./checkbuild.py Bug: None Change-Id: Iacf6c715bd7ba6a1e08c05e28935e1066cc4e7b6
2018-09-26Build libc++ using the just-built NDK.Dan Albert
The saves us the pain of the awkward dance we sometimes need to do to update prebuilts/ndk before we can update ndk-build. libc++ will now be built against the same toolchains and sysroots as the tests that use it. Test: ./checkbuild.py libc++ Test: ./checkbuild.py --system windows64 libc++ Bug: None Change-Id: I797441ccf221b5ac45471f88f923e17f4e0ebeb4
2018-09-26Build the Linux NDK alongside Windows.Dan Albert
We want to migrate to building libc++ (and similar components) using the just-built NDK. To do this, we need to have a just-built NDK that matches the current host rather than the host we're building for. Build the Linux NDK before building the Windows NDK to facilitate this. To handle this, there is now an out directory per host (i.e. out/linux, out/windows64). A symlink will be created linking out/android-ndk-rN-canary to the NDK for the given host since that's how we're accustomed to accessing it, but each NDK will be installed in its host directory. When Windows is targeted, Linux dist artifacts (intermediate artifacts that really shouldn't be in dist any more, plus logs) are installed to the Linux out directory rather than out/dist, since that need only contain Windows artifacts. As it is much more common to build a Linux NDK than a Windows NDK for local development, the Windows build will reuse the existing Linux NDK if available. `--skip-deps` will cause the Linux NDK to not be built (note that deps need to be built the first time the Windows NDK is built so the Windows NDK can build its own deps). Test: ./checkbuild.py --system linux Test: ./checkbuild.py --system windows64 Test: ./checkbuild.py --system windows64 --skip-deps ndk-build Bug: None Change-Id: I81c87bde928d80bb01c9e0aaff9a8937e3177ad1
2018-09-26Remove dev-cleanup.sh.Dan Albert
Doesn't seem like we make any of the mistakes that made this necessary any more. Test: None of the things cleaned by the script exist any more except the tests. The test runner performs its own cleanups. Bug: None Change-Id: I89a4c4bebb5bc26761d4f7cc11afca3ab83e0e6e
2018-09-25Install target specific Clang wrappers.Dan Albert
These wrappers can be used to target the given architecture/API combination without needing to specify -target. These obviate the need for standalone toolchains. We can't remove the code that does this in make_standalone_toolchain.py since it installs `clang` (no triple or version) and `$TRIPLE-clang` (no version) scripts, which we can't assume any defaults for in the toolchain directory. Ideally these would be symlinks rather than wrapper scripts to avoid the unnecessary indirection (Clang will infer its default target based on argv[0]), but the SDK manager can't install symlinks and Windows only allows administrators to create them. Test: Bug: Change-Id: Ib9ce6a7e7dbd57985fc2f113c12ce3b5a456752c
2018-09-25Enable the new toolchain layout.Dan Albert
Test: ./checkbuild.py Bug: https://github.com/android-ndk/ndk/issues/780 Change-Id: I127d66f4ad0f8d802f00b90f00c3d3bc51fa7270
2018-09-24Update to clang-r339409.Dan Albert
Test: ./checkbuild.py && ./run_tests.py Bug: https://github.com/android-ndk/ndk/issues/780 Change-Id: I5059882ecab898929979ae5d7e2ead9349eb24c1
2018-09-24Add a TODO to the Toolchain module.Dan Albert
We can save some space here. Test: None Bug: None Change-Id: I1342a5f16531e06d6f5e60333e91c794983fd7f8
2018-09-13Escape command-line in checkbuild logRyan Prichard
Turn: bash /x/ndk/out/android-ndk-r19-canary/build/ndk-build -j112 V=1 APP_ABI=arm64-v8a armeabi-v7a x86 x86_64 APP_MODULES=c++_shared c++_static ... into: bash /x/ndk/out/android-ndk-r19-canary/build/ndk-build -j112 V=1 'APP_ABI=arm64-v8a armeabi-v7a x86 x86_64' 'APP_MODULES=c++_shared c++_static' ... pipes.quote is the Python 2.7 equivalent of Python 3.3's shlex.quote: https://stackoverflow.com/questions/26790916/python-3-backward-compatability-shlex-quote-vs-pipes-quote Python 3.5 also has pipes.quote. Bug: none Test: ./checkbuild.py Change-Id: Ifa16f000ac0ffa61ee23592f4cf9bff9c61238ae
2018-09-06Merge "Build CRT objects with noexecstack"Treehugger Robot
2018-08-30Install the Just Works toolchain.Dan Albert
Install a copy (we can't remove the old paths until gradle has adapted to the new layout) of the toolchain/sysroot/etc to $NDK/toolchain such that it can be invoked without the build system needing to juggle --sysroot/-gcc-toolchain/etc. This follows the second approach described on https://github.com/android-ndk/ndk/issues/780. That is it installs a full copy of the toolchain rather than installing scripts to refer to the old toolchain during the transition period. Test: ./checkbuild.py && ./run_tests.py Bug: https://github.com/android-ndk/ndk/issues/780 Change-Id: I8d8f10ea6c95378937ac54e440c5e1a9cb3bbeaa
2018-08-30Add support for disabling a module.Dan Albert
This doesn't make it unbuildable, but will not build the module unless it is explicitly named (or depended upon). This gives us a way to add module definitions for things that we don't necessarily want to ship yet, but are useful for local testing. Test: Disable gdbserver, `./checkbuild.py`, see it not in the list, `./checkbuild.py gdbserver`, see that it is. Bug: None Change-Id: Ie87dbdc575db2ee27d8c21e677658531a824479b
2018-08-29Build CRT objects with noexecstackRyan Prichard
Either -Wa,--noexecstack or -Wl,-z,noexecstack should be sufficient by itself. Add a test to verify that NDK binaries can be created using Gold's --warn-execstack without a warning. Bug: https://github.com/android-ndk/ndk/issues/779 Test: checkbuild.py && run_tests.py Change-Id: Ief462056b8751b5cdb39a8d7a7cb86d392c4f288
2018-08-09Merge "Explicitly build c++_static."Treehugger Robot
2018-08-09Explicitly build c++_static.Dan Albert
Test: ./checkbuild.py libc++ Bug: https://github.com/android-ndk/ndk/issues/748 Change-Id: I744b500a2e532698617b83db28f1eba4f87f3102
2018-08-08Install "GCC" wrappers that call Clang.Dan Albert
These are provided to ease porting to new NDKs for projects that are not actually sensitive to changes in compiler, just changes to compiler install path. Test: ./checkbuild.py && path/to/arm-linux-androideabi-gcc --help Bug: https://github.com/android-ndk/ndk/issues/758 Change-Id: I8becbf764c37800b5670f412f63302c638252674
2018-06-21Use the just-built sysroot for building libc++.Dan Albert
The libc++ headers now make use of <android/ndk-version.h>, which is generated as part of our build rather than as a part of the platform prebuilts. Test: ./checkbuild.py && ./run_tests.py Bug: None Change-Id: Idf434e396af0ed3d3131007480fba24ded85e0ab
2018-05-22Don't explicitly link libdl in linker scripts.Dan Albert
The driver already does this. Test: ./checkbuild.py && ./run_tests.py Bug: https://github.com/android-ndk/ndk/issues/692 Change-Id: I903dc0c440ebef5a08dcc37cef390b745297e939
generated by cgit v1.2.3 (git 2.25.1) at 2024-07-10 09:56:21 +0000