diff options
Diffstat (limited to 'binary_search_tool')
70 files changed, 5479 insertions, 0 deletions
diff --git a/binary_search_tool/.gitignore b/binary_search_tool/.gitignore new file mode 100644 index 00000000..c4977a33 --- /dev/null +++ b/binary_search_tool/.gitignore @@ -0,0 +1,7 @@ +log +*.pyc +working_set.txt +objects.txt +.binary_search_state.py.state* +binary_search_state.py.state + diff --git a/binary_search_tool/MAINTENANCE b/binary_search_tool/MAINTENANCE new file mode 100644 index 00000000..8e5b3c61 --- /dev/null +++ b/binary_search_tool/MAINTENANCE @@ -0,0 +1,115 @@ +This document is for future maintainers of the binary search/bisection tools. + +Authors: + * Original Tool: asharif@, llozano@, cmtice@ + * Updates after May 2016: cburden@ + * chromeos-toolchain@ + +The following are good reference materials on how the tool works: + * Ahmad's original presentation: + https://goto.google.com/zxdfyi + + * Bisection tool update design doc: + https://goto.google.com/zcwei + + * Bisection tool webpage: + https://goto.google.com/ruwpyi + + * Compiler wrapper webpage: + https://goto.google.com/xossn + + +TESTING: +All unit tests live under the ./test directory. However, these tests +specifically test binary_search_state.py, binary_search_perforce.py, bisect.py. +These unit tests will not test the specific logic for ChromeOS/Android +bisection. To test the ChromeOS/Android bisectors, use the common/hash_test.sh +test. This is a simple test case that just checks the hashes of files on your +file system. This means you won't have to find a specific compiler error for +the bisector to triage in order to test each bisector. + +TODO: +The bisection tool (I believe) is in a fairly good state. So these are mostly +wishlist items and things that could use some improvement. + + 1. Get rid of binary_search_perforce.py. This file is mostly legacy code and + the majority of it isn't even used to bisect object files. The file was + originally intended to bisect CLs, and binary_search_state.py just reused + the binary searching logic from it. Maybe just extract the binary searching + logic from binary_search_perforce.py and put it in its own module in + cros_utils? + + 2. Cleanup unit tests in ./test. These tests are a little hacked together, + and are all under one test suite. Maybe consider organizing them across + multiple directories. + + 3. Create a "checkout setup" system for bisection. Currently if you want to + bisect, you have to run scripts/edit sources in this repo. Ideally these + scripts would be static, and if you wanted to bisect/make changes you would + "checkout" or copy all the scripts to a working directory and have a unique + working directory for each bisection. Credits to Luis for this idea =) + + 4. Make all scripts relative to each other. Currently all scripts enforce the + idea that their cwd will be ./binary_search_tool/. But it would be less + confusing to have each script relative to each other. There's quite a few + stackoverflow topics on how to do this best, but each one has some sort of + downside or flaw. + + 5. Overall modularize code more, especially in binary_search_state.py + +DESIGN EXPLANATIONS: +Some of the design decisions are a bit difficult to understand from just reading +the code unfortunately. I will attempt to clear up the major offenders of this: + + 1. common.py's argument dictionary: + binary_search_state.py and bisect.py both have to have near identical + arguments in order to support argument overriding in bisect.py. However + they do have to be slightly different. Mainly, bisect.py needs to have no + default values for arguments (so it can determine what's being overriden). + + In order to reduce huge amounts of code duplication for the argument + building, we put argument building in common.py. That way both modules + can reference the arguments, and they can have different configurations + across both. + + 2. Compiler wrapper: + The compiler wrapper is called before all compiler calls. It exists to + trick whatever build system (make, emerge, etc.) into thinking our + bisection is just a normal build, when really we're doing some tricks. + + The biggest benefit the compiler wrapper gives is: knowing for sure which + files are actually generated by the compiler during bisection setup, and + potentially being able to skip compilations while triaging (speeding up the + triaging process significantly). + + 3. The weird options for the --verify, --verbose, --file_args, etc. arguments: + Some of the arguments for the bisection tool have a weird set of options + for the AddArgument method (nargs, const, default, StrToBool). This is so + we can make argument overriding workable. These options allow the following + functionality for a boolean argument (using --prune as an example): + * --prune (prune set to True) + * <not given> (prune set to False) + * --prune=True (prune set to True) + * --prune=False (prune set to False) + + The first two are easy to implement (action='store_true'), but the last two + are why the extra weird arguments are required. Now, why would we want the + last two? Imagine if the Android bisector set --prune=True as a default + argument. With just the first two options above it would be impossible for + the user to override prune and set it to False. So the user needs the + --prune=False option. See the argparse documentation for more details. + + 4. General binary searching logic/pruning logic: + binary_search_state.py will enumerate all items into a list. The binary + search will find the *first* bad item (starting with lowest index). + Everything to the left of the "current" index is switched to good, + everything to right of the "current" index is switched to bad. Once a bad + item is found, it's put at the very end of the list. + + If prune is set, the tool will continuing searching until all bad items are + found (instead of stopping after the first one). If the tool finds the same + item twice, that means no more bad items exist. This is because the item + was found, said item was put at the end of the list, and it was found + again. Because the binary search logic finds the bad item with the lowest + index, this means nothing in between the start of the list and the end of + the list is bad (thus no more bad items remain). diff --git a/binary_search_tool/README.bisect b/binary_search_tool/README.bisect new file mode 100644 index 00000000..e6185e8a --- /dev/null +++ b/binary_search_tool/README.bisect @@ -0,0 +1,213 @@ + +bisect.py is a wrapper around the general purpose binary_search_state.py. It +provides a user friendly interface for bisecting various compilation errors. +The 2 currently provided methods of bisecting are ChromeOS package and object +bisection. Each method defines a default set of options to pass to +binary_search_state.py and allow the user to override these defaults (see +the "Overriding" section). + +** NOTE ** +All commands, examples, scripts, etc. are to be run from your chroot unless +stated otherwise. + +Bisection Methods: + +1) ChromeOS Package: + This method will bisect across all packages in a ChromeOS repository and find + the offending packages (according to your test script). This method takes the + following arguments: + + board: The board to bisect on. For example: daisy, falco, etc. + remote: The IP address of the physical machine you're using to test with. + + By default the ChromeOS package method will do a simple interactive test that + pings the machine and prompts the user if the machine is good. + + a) Setup: + The ChromeOS package method requires that you have three build trees: + + /build/${board}.bad - The build tree for your "bad" build + /build/${board}.good - The build tree for your "good" build + /build/${board}.work - A full copy of /build/${board}.bad + + b) Cleanup: + bisect.py does most cleanup for you, the only thing required by the user is + to cleanup all built images and the three build trees made in /build/ + + c) Default Arguments: + --get_initial_items='cros_pkg/get_initial_items.sh' + --switch_to_good='cros_pkg/switch_to_good.sh' + --switch_to_bad='cros_pkg/switch_to_bad.sh' + --test_setup_script='cros_pkg/test_setup.sh' + --test_script='cros_pkg/interactive_test.sh' + --incremental + --prune + --file_args + + d) Additional Documentation: + See ./cros_pkg/README.cros_pkg_triage for full documentation of ChromeOS + package bisection. + + e) Examples: + i) Basic interactive test package bisection, on daisy board: + ./bisect.py package daisy 172.17.211.184 + + ii) Basic boot test package bisection, on daisy board: + ./bisect.py package daisy 172.17.211.184 -t cros_pkg/boot_test.sh + +2) ChromeOS Object: + This method will bisect across all objects in a ChromeOS package and find + the offending objects (according to your test script). This method takes the + following arguments: + + board: The board to bisect on. For example: daisy, falco, etc. + remote: The IP address of the physical machine you're using to test with. + package: The package to bisect with. For example: chromeos-chrome + dir: (Optional) the directory for your good/bad build trees. Defaults to + $BISECT_DIR or /tmp/sysroot_bisect. This value will set $BISECT_DIR + for all bisecting scripts. + + By default the ChromeOS object method will do a simple interactive test that + pings the machine and prompts the user if the machine is good. + + a) Setup: + The ChromeOS package method requires that you populate your good and bad set + of objects. sysroot_wrapper will automatically detect the BISECT_STAGE + variable and use this to populate emerged objects. Here is an example: + + # Defaults to /tmp/sysroot_bisect + export BISECT_DIR="/path/to/where/you/want/to/store/builds/" + + export BISECT_STAGE="POPULATE_GOOD" + ./switch_to_good_compiler.sh + emerge-${board} -C ${package_to_bisect} + emerge-${board} ${package_to_bisect} + + export BISECT_STAGE="POPULATE_BAD" + ./switch_to_bad_compiler.sh + emerge-${board} -C {package_to_bisect} + emerge-${board} ${package_to_bisect} + + b) Cleanup: + The user must clean up all built images and the populated object files. + + c) Default Arguments: + --get_initial_items='sysroot_wrapper/get_initial_items.sh' + --switch_to_good='sysroot_wrapper/switch_to_good.sh' + --switch_to_bad='sysroot_wrapper/switch_to_bad.sh' + --test_setup_script='sysroot_wrapper/test_setup.sh' + --test_script='sysroot_wrapper/interactive_test.sh' + --noincremental + --prune + --file_args + + d) Additional Documentation: + See ./sysroot_wrapper/README for full documentation of ChromeOS object file + bisecting. + + e) Examples: + i) Basic interactive test object bisection, on daisy board for + cryptohome package: + ./bisect.py object daisy 172.17.211.184 cryptohome + + ii) Basic boot test package bisection, on daisy board for cryptohome + package: + ./bisect.py object daisy 172.17.211.184 cryptohome \ + --test_script=sysroot_wrapper/boot_test.sh + +3) Android object: + NOTE: Because this isn't a ChromeOS bisection tool, the concept of a + chroot doesn't exist. Just run this tool from a normal shell. + + This method will bisect across all objects in the Android source tree and + find the offending objects (according to your test script). This method takes + the following arguments: + + android_src: The location of your android source tree + num_jobs: (Optional) The number of jobs to pass to make. This is dependent + on how many cores your machine has. A good number is probably + somewhere around 5 to 10. + device_id: (Optional) The serial code for the device you are testing on. + This is used to determine which device should be used in case + multiple devices are plugged into your computer. You can get + serial code for your device by running "adb devices". + dir: (Optional) the directory for your good/bad build trees. Defaults to + $BISECT_DIR or ~/ANDROID_BISECT/. This value will set $BISECT_DIR + for all bisecting scripts. + + By default the Android object method will do a simple interactive test that + pings the machine and prompts the user if the machine is good. + + a) Setup: + The Android object method requires that you populate your good and bad set + of objects. The Android compiler wrapper will automatically detect the + BISECT_STAGE variable and use this to populate emerged objects. Here is an + example: + + # Defaults to ~/ANDROID_BISECT/ + export BISECT_DIR="/path/to/where/you/want/to/store/builds/" + + export BISECT_STAGE="POPULATE_GOOD" + # Install the "good" compiler + ./switch_to_good_compiler.sh + make clean + make -j <your_preferred_number_of_jobs> + + export BISECT_STAGE="POPULATE_BAD" + # Install the "bad" compiler + ./switch_to_bad_compiler.sh + make clean + make -j <your_preferred_number_of_jobs> + + b) Cleanup: + The user must clean up all built images and the populated object files. + + c) Default Arguments: + --get_initial_items='android/get_initial_items.sh' + --switch_to_good='android/switch_to_good.sh' + --switch_to_bad='android/switch_to_bad.sh' + --test_setup_script='android/test_setup.sh' + --test_script='android/interactive_test.sh' + --incremental + --prune + --file_args + + d) Additional Documentation: + See ./android/README.android for full documentation of Android object file + bisecting. + + e) Examples: + i) Basic interactive test android bisection, where the android source is + at ~/android_src: + ./bisect.py android ~/android_src + + ii) Basic boot test android bisection, where the android source is at + ~/android_src, and 10 jobs will be used to build android: + ./bisect.py android ~/android_src --num_jobs=10 \ + --test_script=sysroot_wrapper/boot_test.sh + +Resuming: + bisect.py and binary_search_state.py offer the ability to resume a bisection + in case it was interrupted by a SIGINT, power failure, etc. Every time the + tool completes a bisection iteration its state is saved to disk (usually to + the file "./bisect.py.state"). If passed the --resume option, the tool + it will automatically detect the state file and resume from the last + completed iteration. + +Overriding: + You can run ./bisect.py --help or ./binary_search_state.py --help for a full + list of arguments that can be overriden. Here are a couple of examples: + + Example 1 (do boot test instead of interactive test): + ./bisect.py package daisy 172.17.211.182 --test_script=cros_pkg/boot_test.sh + + Example 2 (do package bisector system test instead of interactive test, this + is used to test the bisecting tool itself -- see comments in + hash_test.sh for more details): + ./bisect.py package daisy 172.17.211.182 \ + --test_script=common/hash_test.sh --test_setup_script="" + + Example 3 (enable verbose mode, disable pruning, and disable verification): + ./bisect.py package daisy 172.17.211.182 \ + --verbose --prune=False --verify=False + diff --git a/binary_search_tool/__init__.py b/binary_search_tool/__init__.py new file mode 100644 index 00000000..8b137891 --- /dev/null +++ b/binary_search_tool/__init__.py @@ -0,0 +1 @@ + diff --git a/binary_search_tool/android/README.android b/binary_search_tool/android/README.android new file mode 100644 index 00000000..9e518f60 --- /dev/null +++ b/binary_search_tool/android/README.android @@ -0,0 +1,186 @@ + +binary_search_state.py is a general binary search triage tool that +performs a binary search on a set of things to try to identify which +thing or thing(s) in the set is 'bad'. binary_search_state.py assumes +that the user has two sets, one where everything is known to be good, +and one which contains at least one bad item. binary_search_state.py +then copies items from the good and bad sets into a working set and +tests the result (good or bad). binary_search_state.py requires that +a set of scripts be supplied to it for any particular job. For more +information on binary_search_state.py, see + +https://sites.google.com/a/google.com/chromeos-toolchain-team-home2/home/team-tools-and-scripts/binary-searcher-tool-for-triage + +This particular set of scripts is designed to work wtih +binary_search_state.py in order to find the bad object or set of +bad objects in an Android build. + + +QUICKSTART: + +After setting up your 2 build trees (see Prerequisites section), do the +following: + + - Decide which test script to use (boot_test.sh or + interactive_test.sh) + - Get the serial number for the Android device you will use for testing. + - Run the following: + + $ cd <android_src> + $ source build/envsetup.sh + $ lunch <android_device_lunch_combo> + $ cd <path_to_toolchain_utils>/binary_search_tool/ + $ NUM_JOBS=10 ANDROID_SERIAL=<device_serial> \ + ./android/setup.sh <android_src> + + If you chose the boot test, then: + TEST_SCRIPT=android/boot_test.sh + + If you chose the interactive test, then: + TEST_SCRIPT=android/interactive_test.sh + + Finally, run the binary search tool: + + $ python ./binary_search_state.py \ + --get_initial_items=android/get_initial_items.sh \ + --switch_to_good=android/switch_to_good.sh \ + --switch_to_bad=android/switch_to_bad.sh \ + --test_setup_script=android/test_setup.sh \ + --test_script=$TEST_SCRIPT \ + --file_args \ + --prune + + Once you have completely finished doing the binary search/triage, + run the cleanup script: + + $ android/cleanup.sh + + + +FILES AND SCRIPTS: + + Check the header comments for each script for more in depth documentation. + + boot_test.sh - One of two possible test scripts used to determine + if the Android image built from the objects is good + or bad. This script tests to see if the image + booted, and requires no user intervention. + + cleanup.sh - This is called after the binary search tool completes. This + script will clean up the common.sh file generated by setup.sh + + get_initial_items.sh - This script is used to determine all Android objects + that will be bisected. + + test_setup.sh - This script will build and flash your image to the + Android device. If the flash fails, this script will + help the user troubleshoot by trying to flash again or + by asking the user to manually flash it. + + interactive_test.sh - One of two possible scripts used to determine + if the Android image built from the objects + is good or bad. This script requires user + interaction to determine if the image is + good or bad. + + setup.sh - This is the first script the user should call, after + taking care of the prerequisites. It sets up the + environment appropriately for running the Android + object binary search triage, and it generates the + necessary common script (see below). + + switch_to_bad.sh - This script is used to link objects from the + 'bad' build tree into the work area. + + switch_to_good.sh - This script is used to link objects from the + 'good' build tree into the work area. + + +GENERATED SCRIPTS: + + common.sh - contains basic environment variable definitions for + this binary search triage session. + +ASSUMPTIONS: + +- There are two different Android builds, for the same board/lunch combo with + the same set of generated object files. One build creates a good working + Android image and the other does not. + +- The toolchain bug you are tracking down is not related to the linker. If the + linker is broken or generates bad code, this tool is unlikely to help you. + + +PREREQUISITES FOR USING THESE SCRIPTS: + + Step 1: Decide where to store each build tree + By default, each build tree is stored in "~/ANDROID_BISECT". However you + can override this by exporting BISECT_DIR set to whatever directory you + please. Keep in mind these build trees take dozens of gigabytes each. + + Step 2: Setup your android build environment + 1. `cd <android_src>` + 2. `source build/envsetup.sh` + 3. `lunch <android_device_lunch_combo>` + + Step 3: Populate the good build tree + 1. `make clean` + 2. `export BISECT_STAGE=POPULATE_GOOD` + 3. Install your "good" toolchain in Android, this will most likely be + the toolchain that comes preinstalled with the Android source. + 4. Build all of Android: `make -j10`. The "-j" parameter depends on how + many cores your machine has. See Android documentation for more details. + + Step 4: Populate the bad build tree + 1. `make clean` + 2. `export BISECT_STAGE=POPULATE_BAD` + 3. Install your "bad" toolchain in Android. + 4. Build all of Android again. + + Step 5: Run the android setup script + 1. `cd <path_to_toolchain_utils>/binary_search_tool/` + 2. `NUM_JOBS=<jobs> ANDROID_SERIAL=<android_serial_num> \ + android/setup.sh <android_src>` + + WARNING: It's important that you leave the full "out/" directory in your + Android source alone after Step 4. The binary search tool will + use this directory as a skeleton to build each test image while + triaging. + +USING THESE SCRIPTS FOR BINARY TRIAGE OF OBJECTS: + +To use these scripts, you must first run setup.sh, passing it the path to your +Android source directory. setup.sh will do the following: + + - Verify that your build trees are set up correctly (with good, bad). + - Verify that each build tree has the same contents. + - Verify that the android build environment (lunch, etc.) are setup in your + current shell. + - Create the common.sh file that the other scripts passed to the + binary triage tool will need. + + +This set of scripts comes with two alternate test scripts. One test +script, boot_test.sh, just checks to make sure that the image +booted (wait for device to boot to home screen) and assumes that is enough. +The other test script, interactive_test.sh, is interactive and asks YOU +to tell it whether the image on the android device is ok or not (it +prompts you and waits for a response). + + +Once you have run setup.sh (and decided which test script you +want to use) run the binary triage tool using these scripts to +isolate/identify the bad object: + +./binary_search_state.py \ + --get_initial_items=android/get_initial_items.sh \ + --switch_to_good=android/switch_to_good.sh \ + --switch_to_bad=android/switch_to_bad.sh \ + --test_setup_script=android/test_setup.sh \ + --test_script=android/boot_test.sh \ # could use interactive_test.sh instead + --prune + + +After you have finished running the tool and have identified the bad +object(s), you will want to run the cleanup script (android/cleanup.sh). + diff --git a/binary_search_tool/android/boot_test.sh b/binary_search_tool/android/boot_test.sh new file mode 100755 index 00000000..dc871601 --- /dev/null +++ b/binary_search_tool/android/boot_test.sh @@ -0,0 +1,61 @@ +#!/bin/bash -u +# +# Copyright 2016 Google Inc. All Rights Reserved. +# +# This script pings the android device to determine if it successfully booted. +# +# This script is intended to be used by binary_search_state.py, as +# part of the binary search triage on the Android source tree. It +# waits for the test setup script to build and install the image, then checks +# if image boots or not. It should return '0' if the test succeeds +# (the image is 'good'); '1' if the test fails (the image is 'bad'); and '125' +# if it could not determine (does not apply in this case). +# + +source android/common.sh + +# Check if boot animation has stopped and trim whitespace +is_booted() +{ + # Wait for boot animation to stop and trim whitespace + status=`adb shell getprop init.svc.bootanim | tr -d '[:space:]'` + [[ "$status" == "stopped" ]] + return $? +} + +# Wait for device to boot, retry every 1s +# WARNING: Do not run without timeout command, could run forever +wait_for_boot() +{ + while ! is_booted + do + sleep 1 + done +} + +echo "Waiting 60 seconds for device to come online..." +timeout 60 adb wait-for-device +retval=$? + +if [[ ${retval} -eq 0 ]]; then + echo "Android image has been built and installed." +else + echo "Device failed to reboot within 60 seconds." + exit 1 +fi + +echo "Waiting 60 seconds for device to finish boot..." +# Spawn subshell that will timeout in 60 seconds +# Feed to cat so that timeout will recognize it as a command +# (timeout only works for commands/programs, not functions) +timeout 60 cat <(wait_for_boot) +retval=$? + +if [[ ${retval} -eq 0 ]]; then + echo "Android device fully booted!" +else + echo "Device failed to fully boot within 60 seconds." + exit 1 +fi + +exit ${retval} diff --git a/binary_search_tool/android/cleanup.sh b/binary_search_tool/android/cleanup.sh new file mode 100755 index 00000000..c89c337d --- /dev/null +++ b/binary_search_tool/android/cleanup.sh @@ -0,0 +1,11 @@ +#!/bin/bash +# +# Copyright 2016 Google Inc. All Rights Reserved. +# +# This script is part of the Android binary search triage process. +# It should be the last script called by the user, after the user has +# successfully run the bisection tool and found their bad items. This script +# will perform all necessary cleanup for the bisection tool. +# + +rm android/common.sh diff --git a/binary_search_tool/android/get_initial_items.sh b/binary_search_tool/android/get_initial_items.sh new file mode 100755 index 00000000..2a1eda3a --- /dev/null +++ b/binary_search_tool/android/get_initial_items.sh @@ -0,0 +1,14 @@ +#!/bin/bash -u +# +# Copyright 2016 Google Inc. All Rights Reserved. +# +# This script is intended to be used by binary_search_state.py, as +# part of the binary search triage on the Android source tree. This script +# generates the list of current Android object files, that is then used +# for doing the binary search. +# + +source android/common.sh + +cat ${BISECT_GOOD_BUILD}/_LIST + diff --git a/binary_search_tool/android/interactive_test.sh b/binary_search_tool/android/interactive_test.sh new file mode 100755 index 00000000..e506b236 --- /dev/null +++ b/binary_search_tool/android/interactive_test.sh @@ -0,0 +1,39 @@ +#!/bin/bash -u +# +# Copyright 2016 Google Inc. All Rights Reserved. +# +# This script pings the android device to determine if it successfully booted. +# It then asks the user if the image is good or not, allowing the user to +# conduct whatever tests the user wishes, and waiting for a response. +# +# This script is intended to be used by binary_search_state.py, as +# part of the binary search triage on the Android source tree. It +# waits for the test setup script to build and install the image, then asks the +# user if the image is good or not. It should return '0' if the test succeeds +# (the image is 'good'); '1' if the test fails (the image is 'bad'); and '125' +# if it could not determine (does not apply in this case). +# + +source android/common.sh + +echo "Waiting 60 seconds for device to boot..." +timeout 60 adb wait-for-device +retval=$? + +if [[ ${retval} -eq 0 ]]; then + echo "Android image has been built and installed." +else + echo "Device failed to reboot within 60 seconds." + exit 1 +fi + +while true; do + read -p "Is this a good Android image?" yn + case $yn in + [Yy]* ) exit 0;; + [Nn]* ) exit 1;; + * ) echo "Please answer yes or no.";; + esac +done + +exit 125 diff --git a/binary_search_tool/android/setup.sh b/binary_search_tool/android/setup.sh new file mode 100755 index 00000000..7f8ba0e9 --- /dev/null +++ b/binary_search_tool/android/setup.sh @@ -0,0 +1,147 @@ +#!/bin/bash -u +# +# Copyright 2016 Google Inc. All Rights Reserved. +# +# This script is part of the Android binary search triage process. +# It should be the first script called by the user, after the user has set up +# the two necessary build tree directories (see the prerequisites section of +# README.android). +# +# WARNING: +# Before running this script make sure you have setup the Android build +# environment in this shell (i.e. successfully run 'lunch'). +# +# This script takes three arguments. The first argument must be the path of +# the android source tree being tested. The second (optional) argument is the +# device ID for fastboot/adb so the test device can be uniquely indentified in +# case multiple phones are plugged in. The final (optional) argument is the +# number of jobs that various programs can use for parallelism +# (make, xargs, etc.). There is also the argument for bisection directory, but +# this is not strictly an argument for just this script (as it should be set +# during the POPULATE_GOOD and POPULATE_BAD steps, see README.android for +# details). +# +# Example call: +# ANDROID_SERIAL=002ee16b1558a3d3 NUM_JOBS=10 android/setup.sh ~/android +# +# This will setup the bisector for Nexus5X, using 10 jobs, where the android +# source lives at ~/android. +# +# NOTE: ANDROID_SERIAL is actually an option used by ADB. You can also simply +# do 'export ANDROID_SERIAL=<device_id>' and the bisector will still work. +# Furthermore, if your device is the only Android device plugged in you can +# ignore ANDROID_SERIAL. +# +# This script sets all necessary environment variables, and ensures the +# environment for the binary search triage process is setup properly. In +# addition, this script generates common.sh, which generates enviroment +# variables used by the other scripts in the package binary search triage process. +# + +# +# Positional arguments +# + +ANDROID_SRC=$1 + +# +# Optional arguments +# + +# If DEVICE_ID is not null export this as ANDROID_SERIAL for use by adb +# If DEVICE_ID is null then leave null +DEVICE_ID=${ANDROID_SERIAL:+"export ANDROID_SERIAL=${ANDROID_SERIAL} "} + +NUM_JOBS=${NUM_JOBS:-"1"} +BISECT_ANDROID_DIR=${BISECT_DIR:-~/ANDROID_BISECT} + +# +# Set up basic variables. +# + +GOOD_BUILD=${BISECT_ANDROID_DIR}/good +BAD_BUILD=${BISECT_ANDROID_DIR}/bad +WORK_BUILD=${ANDROID_SRC} + +# +# Verify that the necessary directories exist. +# + +if [[ ! -d ${GOOD_BUILD} ]] ; then + echo "Error: ${GOOD_BUILD} does not exist." + exit 1 +fi + +if [[ ! -d ${BAD_BUILD} ]] ; then + echo "Error: ${BAD_BUILD} does not exist." + exit 1 +fi + +if [[ ! -d ${WORK_BUILD} ]] ; then + echo "Error: ${WORK_BUILD} does not exist." + exit 1 +fi + +# +# Verify that good/bad object lists are the same +# + +good_list=`mktemp` +bad_list=`mktemp` +sort ${GOOD_BUILD}/_LIST > good_list +sort ${BAD_BUILD}/_LIST > bad_list + +diff good_list bad_list +diff_result=$? +rm good_list bad_list + +if [ ${diff_result} -ne 0 ]; then + echo "Error: good and bad object lists differ." + echo "diff exited with non-zero status: ${diff_result}" + exit 1 +fi + +# +# Ensure android build environment is setup +# +# ANDROID_PRODUCT_OUT is only set once lunch is successfully executed. Fail if +# ANDROID_PRODUCT_OUT is unset. +# + +if [ -z ${ANDROID_PRODUCT_OUT+0} ]; then + echo "Error: Android build environment is not setup." + echo "cd to ${ANDROID_SRC} and do the following:" + echo " source build/envsetup.sh" + echo " lunch <device_lunch_combo>" + exit 1 +fi + +# +# Create common.sh file, containing appropriate environment variables. +# + +COMMON_FILE="android/common.sh" + +cat <<-EOF > ${COMMON_FILE} + +BISECT_ANDROID_DIR=${BISECT_ANDROID_DIR} + +BISECT_ANDROID_SRC=${ANDROID_SRC} +BISECT_NUM_JOBS=${NUM_JOBS} + +BISECT_GOOD_BUILD=${GOOD_BUILD} +BISECT_BAD_BUILD=${BAD_BUILD} +BISECT_WORK_BUILD=${WORK_BUILD} + +BISECT_GOOD_SET=${GOOD_BUILD}/_LIST +BISECT_BAD_SET=${BAD_BUILD}/_LIST + +${DEVICE_ID} + +export BISECT_STAGE="TRIAGE" + +EOF + +chmod 755 ${COMMON_FILE} + +exit 0 diff --git a/binary_search_tool/android/switch_to_bad.sh b/binary_search_tool/android/switch_to_bad.sh new file mode 100755 index 00000000..f746b628 --- /dev/null +++ b/binary_search_tool/android/switch_to_bad.sh @@ -0,0 +1,42 @@ +#!/bin/bash -u +# +# Copyright 2016 Google Inc. All Rights Reserved. +# +# This script is intended to be used by binary_search_state.py, as +# part of the binary search triage on the Android source tree. This script +# symlinks a list of object files from the 'bad' build tree into the working +# build tree, for testing. +# +# It is highly recommended to not use --noincremental with these scripts. If the +# switch scripts are given non incremental sets of GOOD/BAD objects, make will +# not be able to do an incremental build and will take much longer to build. +# + + +source android/common.sh + +OBJ_LIST_FILE=$1 + +# Symlink from BAD obj to working tree. +SWITCH_CMD="ln -f ${BISECT_BAD_BUILD}/{} {}; touch {};" + +overall_status=0 + +# Check that number of arguments == 1 +if [ $# -eq 1 ] ; then + # Run symlink once per input line, ignore empty lines. + # Have ${BISECT_NUM_JOBS} processes running concurrently. + # Pass to "sh" to allow multiple commands to be executed. + xargs -P ${BISECT_NUM_JOBS} -a ${OBJ_LIST_FILE} -r -l -I '{}' \ + sh -c "${SWITCH_CMD}" +else + echo "ERROR:" + echo "Please run the binary search tool with --file_args" + echo "Android has too many files to be passed as command line arguments" + echo "The binary search tool will now exit..." + exit 1 +fi +overall_status=$? + + +exit ${overall_status} diff --git a/binary_search_tool/android/switch_to_good.sh b/binary_search_tool/android/switch_to_good.sh new file mode 100755 index 00000000..1c046c3f --- /dev/null +++ b/binary_search_tool/android/switch_to_good.sh @@ -0,0 +1,41 @@ +#!/bin/bash -u +# +# Copyright 2016 Google Inc. All Rights Reserved. +# +# This script is intended to be used by binary_search_state.py, as +# part of the binary search triage on the Android source tree. This script +# symlinks a list of object files from the 'good' build tree into the working +# build tree, for testing. +# +# It is highly recommended to not use --noincremental with these scripts. If the +# switch scripts are given non incremental sets of GOOD/BAD objects, make will +# not be able to do an incremental build and will take much longer to build. +# + +source android/common.sh + +OBJ_LIST_FILE=$1 + +# Symlink from GOOD obj to working tree. +SWITCH_CMD="ln -f ${BISECT_GOOD_BUILD}/{} {}; touch {};" + +overall_status=0 + +# Check that number of arguments == 1 +if [ $# -eq 1 ] ; then + # Run symlink once per input line, ignore empty lines. + # Have ${BISECT_NUM_JOBS} processes running concurrently. + # Pass to "sh" to allow multiple commands to be executed. + xargs -P ${BISECT_NUM_JOBS} -a ${OBJ_LIST_FILE} -r -l -I '{}' \ + sh -c "${SWITCH_CMD}" +else + echo "ERROR:" + echo "Please run the binary search tool with --file_args" + echo "Android has too many files to be passed as command line arguments" + echo "The binary search tool will now exit..." + exit 1 +fi +overall_status=$? + + +exit ${overall_status} diff --git a/binary_search_tool/android/test_setup.sh b/binary_search_tool/android/test_setup.sh new file mode 100755 index 00000000..26f8ec22 --- /dev/null +++ b/binary_search_tool/android/test_setup.sh @@ -0,0 +1,130 @@ +#!/bin/bash +# +# Copyright 2016 Google Inc. All Rights Reserved. +# +# This is the test setup script for generating an Android image based off the +# current working build tree. make is called to relink the object files and +# generate the new Android image to be flashed. The device is then rebooted into +# bootloader mode and fastboot is used to flash the new image. The device is +# then rebooted so the user's test script can run. +# +# This script is intended to be used by binary_search_state.py, as +# part of the binary search triage on the Android source tree. It should +# return '0' if the setup succeeds; and '1' if the setup fails (the image +# could not build or be flashed). +# + +source android/common.sh + +manual_flash() +{ + echo + echo "Please manually flash the built image to your device." + echo "To do so follow these steps:" + echo " 1. Boot your device into fastboot mode." + echo " 2. cd to '${BISECT_ANDROID_SRC}'" + echo " 2. Run 'source build/envsetup.sh'" + echo " 3. Run 'lunch'" + echo " 4. Run '${ADB_DEVICE}fastboot flashall -w'" + echo "Or see the following link for more in depth steps:" + echo "https://source.android.com/source/running.html" + echo + while true; do + sleep 1 + read -p "Was the flashing of the image successful? " choice + case $choice in + [Yy]*) return 0;; + [Nn]*) return 1;; + *) echo "Please answer y or n.";; + esac + done +} + +auto_flash() +{ + echo + echo "Please ensure your Android device is on and in fastboot mode so" + echo "fastboot flash may run." + echo + sleep 1 + read -p $'Press enter to continue and retry the flashing' notused + + echo " ${ADB_DEVICE}fastboot flashall -w" + fastboot flashall -w +} + +flash() +{ + echo + echo "FLASHING" + echo "Rebooting device into fastboot mode." + echo " ${ADB_DEVICE}adb reboot bootloader" + adb reboot bootloader + + echo + echo "Waiting for device to reach fastboot mode." + echo "(will timeout after 60 seconds)" + # fastboot will block indefinitely until device comes online. + # Grab random variable to test if device is online. + # If takes >60s then we error out and ask the user for help. + timeout 60 fastboot getvar 0 2>/dev/null + fastboot_flash_status=$? + + if [[ ${fastboot_flash_status} -eq 0 ]]; then + echo + echo "Flashing image." + echo " ${ADB_DEVICE}fastboot flashall -w" + fastboot flashall -w + fastboot_flash_status=$? + fi + + while [[ ${fastboot_flash_status} -ne 0 ]] ; do + echo + echo "fastboot flash has failed! From here you can:" + echo "1. Debug and/or flash manually" + echo "2. Retry flashing automatically" + echo "3. Abort this installation and skip this image" + echo "4. Abort this installation and mark test as failed" + sleep 1 + read -p "Which method would you like to do? " choice + case $choice in + 1) manual_flash && break;; + 2) auto_flash && break;; + 3) return 125;; + 4) return 1;; + *) echo "Please answer 1, 2, 3, or 4.";; + esac + done +} + +# Number of jobs make will use. Can be customized and played with. +MAKE_JOBS=${BISECT_NUM_JOBS} + +# Set ADB_DEVICE to "ANDROID_SERIAL=${ANDROID_SERIAL}" or "" if device id not +# set. This is used for debugging info so users can confirm which device +# commands are being sent to. +ADB_DEVICE=${ANDROID_SERIAL:+"ANDROID_SERIAL=${ANDROID_SERIAL} "} + +echo +echo "INSTALLATION BEGIN" +echo + +cd ${BISECT_ANDROID_SRC} + +echo "BUILDING IMAGE" + +make -j ${MAKE_JOBS} +make_status=$? + +exit_val=0 +if [[ ${make_status} -eq 0 ]]; then + flash + exit_val=$? +else + echo "ERROR:" + echo "make returned a non-zero status: ${make_status}. Skipping image..." + exit_val=1 +fi + + +exit ${exit_val} diff --git a/binary_search_tool/binary_search_perforce.py b/binary_search_tool/binary_search_perforce.py new file mode 100755 index 00000000..7ac2fba6 --- /dev/null +++ b/binary_search_tool/binary_search_perforce.py @@ -0,0 +1,447 @@ +#!/usr/bin/python2 +"""Module of binary serch for perforce.""" +from __future__ import print_function + +import math +import argparse +import os +import re +import sys +import tempfile + +from cros_utils import command_executer +from cros_utils import logger + +verbose = True + + +def _GetP4ClientSpec(client_name, p4_paths): + p4_string = '' + for p4_path in p4_paths: + if ' ' not in p4_path: + p4_string += ' -a %s' % p4_path + else: + p4_string += " -a \"" + (' //' + client_name + '/').join(p4_path) + "\"" + + return p4_string + + +def GetP4Command(client_name, p4_port, p4_paths, checkoutdir, p4_snapshot=''): + command = '' + + if p4_snapshot: + command += 'mkdir -p ' + checkoutdir + for p4_path in p4_paths: + real_path = p4_path[1] + if real_path.endswith('...'): + real_path = real_path.replace('/...', '') + command += ( + '; mkdir -p ' + checkoutdir + '/' + os.path.dirname(real_path)) + command += ('&& rsync -lr ' + p4_snapshot + '/' + real_path + ' ' + + checkoutdir + '/' + os.path.dirname(real_path)) + return command + + command += ' export P4CONFIG=.p4config' + command += ' && mkdir -p ' + checkoutdir + command += ' && cd ' + checkoutdir + command += ' && cp ${HOME}/.p4config .' + command += ' && chmod u+w .p4config' + command += " && echo \"P4PORT=" + p4_port + "\" >> .p4config" + command += " && echo \"P4CLIENT=" + client_name + "\" >> .p4config" + command += (' && g4 client ' + _GetP4ClientSpec(client_name, p4_paths)) + command += ' && g4 sync ' + command += ' && cd -' + return command + + +class BinarySearchPoint(object): + """Class of binary search point.""" + + def __init__(self, revision, status, tag=None): + self.revision = revision + self.status = status + self.tag = tag + + +class BinarySearcher(object): + """Class of binary searcher.""" + + def __init__(self, logger_to_set=None): + self.sorted_list = [] + self.index_log = [] + self.status_log = [] + self.skipped_indices = [] + self.current = 0 + self.points = {} + self.lo = 0 + self.hi = 0 + if logger_to_set is not None: + self.logger = logger_to_set + else: + self.logger = logger.GetLogger() + + def SetSortedList(self, sorted_list): + assert len(sorted_list) > 0 + self.sorted_list = sorted_list + self.index_log = [] + self.hi = len(sorted_list) - 1 + self.lo = 0 + self.points = {} + for i in range(len(self.sorted_list)): + bsp = BinarySearchPoint(self.sorted_list[i], -1, 'Not yet done.') + self.points[i] = bsp + + def SetStatus(self, status, tag=None): + message = ('Revision: %s index: %d returned: %d' % + (self.sorted_list[self.current], self.current, status)) + self.logger.LogOutput(message, print_to_console=verbose) + assert status == 0 or status == 1 or status == 125 + self.index_log.append(self.current) + self.status_log.append(status) + bsp = BinarySearchPoint(self.sorted_list[self.current], status, tag) + self.points[self.current] = bsp + + if status == 125: + self.skipped_indices.append(self.current) + + if status == 0 or status == 1: + if status == 0: + self.lo = self.current + 1 + elif status == 1: + self.hi = self.current + self.logger.LogOutput('lo: %d hi: %d\n' % (self.lo, self.hi)) + self.current = (self.lo + self.hi) / 2 + + if self.lo == self.hi: + message = ('Search complete. First bad version: %s' + ' at index: %d' % (self.sorted_list[self.current], self.lo)) + self.logger.LogOutput(message) + return True + + for index in range(self.lo, self.hi): + if index not in self.skipped_indices: + return False + self.logger.LogOutput( + 'All skipped indices between: %d and %d\n' % (self.lo, self.hi), + print_to_console=verbose) + return True + + # Does a better job with chromeos flakiness. + def GetNextFlakyBinary(self): + t = (self.lo, self.current, self.hi) + q = [t] + while len(q): + element = q.pop(0) + if element[1] in self.skipped_indices: + # Go top + to_add = (element[0], (element[0] + element[1]) / 2, element[1]) + q.append(to_add) + # Go bottom + to_add = (element[1], (element[1] + element[2]) / 2, element[2]) + q.append(to_add) + else: + self.current = element[1] + return + assert len(q), 'Queue should never be 0-size!' + + def GetNextFlakyLinear(self): + current_hi = self.current + current_lo = self.current + while True: + if current_hi < self.hi and current_hi not in self.skipped_indices: + self.current = current_hi + break + if current_lo >= self.lo and current_lo not in self.skipped_indices: + self.current = current_lo + break + if current_lo < self.lo and current_hi >= self.hi: + break + + current_hi += 1 + current_lo -= 1 + + def GetNext(self): + self.current = (self.hi + self.lo) / 2 + # Try going forward if current is skipped. + if self.current in self.skipped_indices: + self.GetNextFlakyBinary() + + # TODO: Add an estimated time remaining as well. + message = ('Estimated tries: min: %d max: %d\n' % + (1 + math.log(self.hi - self.lo, 2), + self.hi - self.lo - len(self.skipped_indices))) + self.logger.LogOutput(message, print_to_console=verbose) + message = ('lo: %d hi: %d current: %d version: %s\n' % + (self.lo, self.hi, self.current, self.sorted_list[self.current])) + self.logger.LogOutput(message, print_to_console=verbose) + self.logger.LogOutput(str(self), print_to_console=verbose) + return self.sorted_list[self.current] + + def SetLoRevision(self, lo_revision): + self.lo = self.sorted_list.index(lo_revision) + + def SetHiRevision(self, hi_revision): + self.hi = self.sorted_list.index(hi_revision) + + def GetAllPoints(self): + to_return = '' + for i in range(len(self.sorted_list)): + to_return += ('%d %d %s\n' % (self.points[i].status, i, + self.points[i].revision)) + + return to_return + + def __str__(self): + to_return = '' + to_return += 'Current: %d\n' % self.current + to_return += str(self.index_log) + '\n' + revision_log = [] + for index in self.index_log: + revision_log.append(self.sorted_list[index]) + to_return += str(revision_log) + '\n' + to_return += str(self.status_log) + '\n' + to_return += 'Skipped indices:\n' + to_return += str(self.skipped_indices) + '\n' + to_return += self.GetAllPoints() + return to_return + + +class RevisionInfo(object): + """Class of reversion info.""" + + def __init__(self, date, client, description): + self.date = date + self.client = client + self.description = description + self.status = -1 + + +class VCSBinarySearcher(object): + """Class of VCS binary searcher.""" + + def __init__(self): + self.bs = BinarySearcher() + self.rim = {} + self.current_ce = None + self.checkout_dir = None + self.current_revision = None + + def Initialize(self): + pass + + def GetNextRevision(self): + pass + + def CheckoutRevision(self, revision): + pass + + def SetStatus(self, status): + pass + + def Cleanup(self): + pass + + def SetGoodRevision(self, revision): + if revision is None: + return + assert revision in self.bs.sorted_list + self.bs.SetLoRevision(revision) + + def SetBadRevision(self, revision): + if revision is None: + return + assert revision in self.bs.sorted_list + self.bs.SetHiRevision(revision) + + +class P4BinarySearcher(VCSBinarySearcher): + """Class of P4 binary searcher.""" + + def __init__(self, p4_port, p4_paths, test_command): + VCSBinarySearcher.__init__(self) + self.p4_port = p4_port + self.p4_paths = p4_paths + self.test_command = test_command + self.checkout_dir = tempfile.mkdtemp() + self.ce = command_executer.GetCommandExecuter() + self.client_name = 'binary-searcher-$HOSTNAME-$USER' + self.job_log_root = '/home/asharif/www/coreboot_triage/' + self.changes = None + + def Initialize(self): + self.Cleanup() + command = GetP4Command(self.client_name, self.p4_port, self.p4_paths, 1, + self.checkout_dir) + self.ce.RunCommand(command) + command = 'cd %s && g4 changes ...' % self.checkout_dir + _, out, _ = self.ce.RunCommandWOutput(command) + self.changes = re.findall(r'Change (\d+)', out) + change_infos = re.findall(r'Change (\d+) on ([\d/]+) by ' + r"([^\s]+) ('[^']*')", out) + for change_info in change_infos: + ri = RevisionInfo(change_info[1], change_info[2], change_info[3]) + self.rim[change_info[0]] = ri + # g4 gives changes in reverse chronological order. + self.changes.reverse() + self.bs.SetSortedList(self.changes) + + def SetStatus(self, status): + self.rim[self.current_revision].status = status + return self.bs.SetStatus(status) + + def GetNextRevision(self): + next_revision = self.bs.GetNext() + self.current_revision = next_revision + return next_revision + + def CleanupCLs(self): + if not os.path.isfile(self.checkout_dir + '/.p4config'): + command = 'cd %s' % self.checkout_dir + command += ' && cp ${HOME}/.p4config .' + command += " && echo \"P4PORT=" + self.p4_port + "\" >> .p4config" + command += " && echo \"P4CLIENT=" + self.client_name + "\" >> .p4config" + self.ce.RunCommand(command) + command = 'cd %s' % self.checkout_dir + command += '; g4 changes -c %s' % self.client_name + _, out, _ = self.ce.RunCommandWOUTPUOT(command) + changes = re.findall(r'Change (\d+)', out) + if len(changes) != 0: + command = 'cd %s' % self.checkout_dir + for change in changes: + command += '; g4 revert -c %s' % change + self.ce.RunCommand(command) + + def CleanupClient(self): + command = 'cd %s' % self.checkout_dir + command += '; g4 revert ...' + command += '; g4 client -d %s' % self.client_name + self.ce.RunCommand(command) + + def Cleanup(self): + self.CleanupCLs() + self.CleanupClient() + + def __str__(self): + to_return = '' + for change in self.changes: + ri = self.rim[change] + if ri.status == -1: + to_return = '%s\t%d\n' % (change, ri.status) + else: + to_return += ('%s\t%d\t%s\t%s\t%s\t%s\t%s\t%s\n' % + (change, ri.status, ri.date, ri.client, ri.description, + self.job_log_root + change + '.cmd', + self.job_log_root + change + '.out', + self.job_log_root + change + '.err')) + return to_return + + +class P4GCCBinarySearcher(P4BinarySearcher): + """Class of P4 gcc binary searcher.""" + + # TODO: eventually get these patches from g4 instead of creating them manually + def HandleBrokenCLs(self, current_revision): + cr = int(current_revision) + problematic_ranges = [] + problematic_ranges.append([44528, 44539]) + problematic_ranges.append([44528, 44760]) + problematic_ranges.append([44335, 44882]) + command = 'pwd' + for pr in problematic_ranges: + if cr in range(pr[0], pr[1]): + patch_file = '/home/asharif/triage_tool/%d-%d.patch' % (pr[0], pr[1]) + f = open(patch_file) + patch = f.read() + f.close() + files = re.findall('--- (//.*)', patch) + command += '; cd %s' % self.checkout_dir + for f in files: + command += '; g4 open %s' % f + command += '; patch -p2 < %s' % patch_file + self.current_ce.RunCommand(command) + + def CheckoutRevision(self, current_revision): + job_logger = logger.Logger( + self.job_log_root, current_revision, True, subdir='') + self.current_ce = command_executer.GetCommandExecuter(job_logger) + + self.CleanupCLs() + # Change the revision of only the gcc part of the toolchain. + command = ('cd %s/gcctools/google_vendor_src_branch/gcc ' + '&& g4 revert ...; g4 sync @%s' % + (self.checkout_dir, current_revision)) + self.current_ce.RunCommand(command) + + self.HandleBrokenCLs(current_revision) + + +def Main(argv): + """The main function.""" + # Common initializations + ### command_executer.InitCommandExecuter(True) + ce = command_executer.GetCommandExecuter() + + parser = argparse.ArgumentParser() + parser.add_argument( + '-n', + '--num_tries', + dest='num_tries', + default='100', + help='Number of tries.') + parser.add_argument( + '-g', + '--good_revision', + dest='good_revision', + help='Last known good revision.') + parser.add_argument( + '-b', + '--bad_revision', + dest='bad_revision', + help='Last known bad revision.') + parser.add_argument( + '-s', '--script', dest='script', help='Script to run for every version.') + options = parser.parse_args(argv) + # First get all revisions + p4_paths = ['//depot2/gcctools/google_vendor_src_branch/gcc/gcc-4.4.3/...', + '//depot2/gcctools/google_vendor_src_branch/binutils/' + 'binutils-2.20.1-mobile/...', + '//depot2/gcctools/google_vendor_src_branch/' + 'binutils/binutils-20100303/...'] + p4gccbs = P4GCCBinarySearcher('perforce2:2666', p4_paths, '') + + # Main loop: + terminated = False + num_tries = int(options.num_tries) + script = os.path.expanduser(options.script) + + try: + p4gccbs.Initialize() + p4gccbs.SetGoodRevision(options.good_revision) + p4gccbs.SetBadRevision(options.bad_revision) + while not terminated and num_tries > 0: + current_revision = p4gccbs.GetNextRevision() + + # Now run command to get the status + ce = command_executer.GetCommandExecuter() + command = '%s %s' % (script, p4gccbs.checkout_dir) + status = ce.RunCommand(command) + message = ('Revision: %s produced: %d status\n' % + (current_revision, status)) + logger.GetLogger().LogOutput(message, print_to_console=verbose) + terminated = p4gccbs.SetStatus(status) + num_tries -= 1 + logger.GetLogger().LogOutput(str(p4gccbs), print_to_console=verbose) + + if not terminated: + logger.GetLogger().LogOutput( + 'Tries: %d expired.' % num_tries, print_to_console=verbose) + logger.GetLogger().LogOutput(str(p4gccbs.bs), print_to_console=verbose) + except (KeyboardInterrupt, SystemExit): + logger.GetLogger().LogOutput('Cleaning up...') + finally: + logger.GetLogger().LogOutput(str(p4gccbs.bs), print_to_console=verbose) + status = p4gccbs.Cleanup() + + +if __name__ == '__main__': + Main(sys.argv[1:]) diff --git a/binary_search_tool/binary_search_state.py b/binary_search_tool/binary_search_state.py new file mode 100755 index 00000000..a10e90b9 --- /dev/null +++ b/binary_search_tool/binary_search_state.py @@ -0,0 +1,598 @@ +#!/usr/bin/python2 +"""The binary search wrapper.""" + +from __future__ import print_function + +import argparse +import contextlib +import errno +import math +import os +import pickle +import sys +import tempfile +import time + +# Adds cros_utils to PYTHONPATH +import common + +# Now we do import from cros_utils +from cros_utils import command_executer +from cros_utils import logger + +import binary_search_perforce + +GOOD_SET_VAR = 'BISECT_GOOD_SET' +BAD_SET_VAR = 'BISECT_BAD_SET' + +STATE_FILE = '%s.state' % sys.argv[0] +HIDDEN_STATE_FILE = os.path.join( + os.path.dirname(STATE_FILE), '.%s' % os.path.basename(STATE_FILE)) + + +class Error(Exception): + """The general binary search tool error class.""" + pass + + +@contextlib.contextmanager +def SetFile(env_var, items): + """Generate set files that can be used by switch/test scripts. + + Generate temporary set file (good/bad) holding contents of good/bad items for + the current binary search iteration. Store the name of each file as an + environment variable so all child processes can access it. + + This function is a contextmanager, meaning it's meant to be used with the + "with" statement in Python. This is so cleanup and setup happens automatically + and cleanly. Execution of the outer "with" statement happens at the "yield" + statement. + + Args: + env_var: What environment variable to store the file name in. + items: What items are in this set. + """ + with tempfile.NamedTemporaryFile() as f: + os.environ[env_var] = f.name + f.write('\n'.join(items)) + f.flush() + yield + + +class BinarySearchState(object): + """The binary search state class.""" + + def __init__(self, get_initial_items, switch_to_good, switch_to_bad, + test_setup_script, test_script, incremental, prune, iterations, + prune_iterations, verify, file_args, verbose): + """BinarySearchState constructor, see Run for full args documentation.""" + self.get_initial_items = get_initial_items + self.switch_to_good = switch_to_good + self.switch_to_bad = switch_to_bad + self.test_setup_script = test_setup_script + self.test_script = test_script + self.incremental = incremental + self.prune = prune + self.iterations = iterations + self.prune_iterations = prune_iterations + self.verify = verify + self.file_args = file_args + self.verbose = verbose + + self.l = logger.GetLogger() + self.ce = command_executer.GetCommandExecuter() + + self.resumed = False + self.prune_cycles = 0 + self.search_cycles = 0 + self.binary_search = None + self.all_items = None + self.PopulateItemsUsingCommand(self.get_initial_items) + self.currently_good_items = set([]) + self.currently_bad_items = set([]) + self.found_items = set([]) + self.known_good = set([]) + + self.start_time = time.time() + + def SwitchToGood(self, item_list): + """Switch given items to "good" set.""" + if self.incremental: + self.l.LogOutput( + 'Incremental set. Wanted to switch %s to good' % str(item_list), + print_to_console=self.verbose) + incremental_items = [ + item for item in item_list if item not in self.currently_good_items + ] + item_list = incremental_items + self.l.LogOutput( + 'Incremental set. Actually switching %s to good' % str(item_list), + print_to_console=self.verbose) + + if not item_list: + return + + self.l.LogOutput( + 'Switching %s to good' % str(item_list), print_to_console=self.verbose) + self.RunSwitchScript(self.switch_to_good, item_list) + self.currently_good_items = self.currently_good_items.union(set(item_list)) + self.currently_bad_items.difference_update(set(item_list)) + + def SwitchToBad(self, item_list): + """Switch given items to "bad" set.""" + if self.incremental: + self.l.LogOutput( + 'Incremental set. Wanted to switch %s to bad' % str(item_list), + print_to_console=self.verbose) + incremental_items = [ + item for item in item_list if item not in self.currently_bad_items + ] + item_list = incremental_items + self.l.LogOutput( + 'Incremental set. Actually switching %s to bad' % str(item_list), + print_to_console=self.verbose) + + if not item_list: + return + + self.l.LogOutput( + 'Switching %s to bad' % str(item_list), print_to_console=self.verbose) + self.RunSwitchScript(self.switch_to_bad, item_list) + self.currently_bad_items = self.currently_bad_items.union(set(item_list)) + self.currently_good_items.difference_update(set(item_list)) + + def RunSwitchScript(self, switch_script, item_list): + """Pass given items to switch script. + + Args: + switch_script: path to switch script + item_list: list of all items to be switched + """ + if self.file_args: + with tempfile.NamedTemporaryFile() as f: + f.write('\n'.join(item_list)) + f.flush() + command = '%s %s' % (switch_script, f.name) + ret, _, _ = self.ce.RunCommandWExceptionCleanup( + command, print_to_console=self.verbose) + else: + command = '%s %s' % (switch_script, ' '.join(item_list)) + try: + ret, _, _ = self.ce.RunCommandWExceptionCleanup( + command, print_to_console=self.verbose) + except OSError as e: + if e.errno == errno.E2BIG: + raise Error('Too many arguments for switch script! Use --file_args') + else: + raise + assert ret == 0, 'Switch script %s returned %d' % (switch_script, ret) + + def TestScript(self): + """Run test script and return exit code from script.""" + command = self.test_script + ret, _, _ = self.ce.RunCommandWExceptionCleanup(command) + return ret + + def TestSetupScript(self): + """Run test setup script and return exit code from script.""" + if not self.test_setup_script: + return 0 + + command = self.test_setup_script + ret, _, _ = self.ce.RunCommandWExceptionCleanup(command) + return ret + + def DoVerify(self): + """Verify correctness of test environment. + + Verify that a "good" set of items produces a "good" result and that a "bad" + set of items produces a "bad" result. To be run directly before running + DoSearch. If verify is False this step is skipped. + """ + if not self.verify: + return + + self.l.LogOutput('VERIFICATION') + self.l.LogOutput('Beginning tests to verify good/bad sets\n') + + self._OutputProgress('Verifying items from GOOD set\n') + with SetFile(GOOD_SET_VAR, self.all_items), SetFile(BAD_SET_VAR, []): + self.l.LogOutput('Resetting all items to good to verify.') + self.SwitchToGood(self.all_items) + status = self.TestSetupScript() + assert status == 0, 'When reset_to_good, test setup should succeed.' + status = self.TestScript() + assert status == 0, 'When reset_to_good, status should be 0.' + + self._OutputProgress('Verifying items from BAD set\n') + with SetFile(GOOD_SET_VAR, []), SetFile(BAD_SET_VAR, self.all_items): + self.l.LogOutput('Resetting all items to bad to verify.') + self.SwitchToBad(self.all_items) + status = self.TestSetupScript() + # The following assumption is not true; a bad image might not + # successfully push onto a device. + # assert status == 0, 'When reset_to_bad, test setup should succeed.' + if status == 0: + status = self.TestScript() + assert status == 1, 'When reset_to_bad, status should be 1.' + + def DoSearch(self): + """Perform full search for bad items. + + Perform full search until prune_iterations number of bad items are found. + """ + while (True and len(self.all_items) > 1 and + self.prune_cycles < self.prune_iterations): + terminated = self.DoBinarySearch() + self.prune_cycles += 1 + if not terminated: + break + # Prune is set. + prune_index = self.binary_search.current + + # If found item is last item, no new items can be found + if prune_index == len(self.all_items) - 1: + self.l.LogOutput('First bad item is the last item. Breaking.') + self.l.LogOutput('Bad items are: %s' % self.all_items[-1]) + break + + # If already seen item we have no new bad items to find, finish up + if self.all_items[prune_index] in self.found_items: + self.l.LogOutput( + 'Found item already found before: %s.' % + self.all_items[prune_index], + print_to_console=self.verbose) + self.l.LogOutput('No more bad items remaining. Done searching.') + self.l.LogOutput('Bad items are: %s' % ' '.join(self.found_items)) + break + + new_all_items = list(self.all_items) + # Move prune item to the end of the list. + new_all_items.append(new_all_items.pop(prune_index)) + self.found_items.add(new_all_items[-1]) + + # Everything below newly found bad item is now known to be a good item. + # Take these good items out of the equation to save time on the next + # search. We save these known good items so they are still sent to the + # switch_to_good script. + if prune_index: + self.known_good.update(new_all_items[:prune_index]) + new_all_items = new_all_items[prune_index:] + + self.l.LogOutput( + 'Old list: %s. New list: %s' % (str(self.all_items), + str(new_all_items)), + print_to_console=self.verbose) + + if not self.prune: + self.l.LogOutput('Not continuning further, --prune is not set') + break + # FIXME: Do we need to Convert the currently good items to bad + self.PopulateItemsUsingList(new_all_items) + + def DoBinarySearch(self): + """Perform single iteration of binary search.""" + # If in resume mode don't reset search_cycles + if not self.resumed: + self.search_cycles = 0 + else: + self.resumed = False + + terminated = False + while self.search_cycles < self.iterations and not terminated: + self.SaveState() + self.OutputIterationProgress() + + self.search_cycles += 1 + [bad_items, good_items] = self.GetNextItems() + + with SetFile(GOOD_SET_VAR, good_items), SetFile(BAD_SET_VAR, bad_items): + # TODO: bad_items should come first. + self.SwitchToGood(good_items) + self.SwitchToBad(bad_items) + status = self.TestSetupScript() + if status == 0: + status = self.TestScript() + terminated = self.binary_search.SetStatus(status) + + if terminated: + self.l.LogOutput('Terminated!', print_to_console=self.verbose) + if not terminated: + self.l.LogOutput('Ran out of iterations searching...') + self.l.LogOutput(str(self), print_to_console=self.verbose) + return terminated + + def PopulateItemsUsingCommand(self, command): + """Update all_items and binary search logic from executable. + + This method is mainly required for enumerating the initial list of items + from the get_initial_items script. + + Args: + command: path to executable that will enumerate items. + """ + ce = command_executer.GetCommandExecuter() + _, out, _ = ce.RunCommandWExceptionCleanup( + command, return_output=True, print_to_console=self.verbose) + all_items = out.split() + self.PopulateItemsUsingList(all_items) + + def PopulateItemsUsingList(self, all_items): + """Update all_items and binary searching logic from list. + + Args: + all_items: new list of all_items + """ + self.all_items = all_items + self.binary_search = binary_search_perforce.BinarySearcher( + logger_to_set=self.l) + self.binary_search.SetSortedList(self.all_items) + + def SaveState(self): + """Save state to STATE_FILE. + + SaveState will create a new unique, hidden state file to hold data from + object. Then atomically overwrite the STATE_FILE symlink to point to the + new data. + + Raises: + Error if STATE_FILE already exists but is not a symlink. + """ + ce, l = self.ce, self.l + self.ce, self.l, self.binary_search.logger = None, None, None + old_state = None + + _, path = tempfile.mkstemp(prefix=HIDDEN_STATE_FILE, dir='.') + with open(path, 'wb') as f: + pickle.dump(self, f) + + if os.path.exists(STATE_FILE): + if os.path.islink(STATE_FILE): + old_state = os.readlink(STATE_FILE) + else: + raise Error(('%s already exists and is not a symlink!\n' + 'State file saved to %s' % (STATE_FILE, path))) + + # Create new link and atomically overwrite old link + temp_link = '%s.link' % HIDDEN_STATE_FILE + os.symlink(path, temp_link) + os.rename(temp_link, STATE_FILE) + + if old_state: + os.remove(old_state) + + self.ce, self.l, self.binary_search.logger = ce, l, l + + @classmethod + def LoadState(cls): + """Create BinarySearchState object from STATE_FILE.""" + if not os.path.isfile(STATE_FILE): + return None + try: + bss = pickle.load(file(STATE_FILE)) + bss.l = logger.GetLogger() + bss.ce = command_executer.GetCommandExecuter() + bss.binary_search.logger = bss.l + bss.start_time = time.time() + + # Set resumed to be True so we can enter DoBinarySearch without the method + # resetting our current search_cycles to 0. + bss.resumed = True + + # Set currently_good_items and currently_bad_items to empty so that the + # first iteration after resuming will always be non-incremental. This is + # just in case the environment changes, the user makes manual changes, or + # a previous switch_script corrupted the environment. + bss.currently_good_items = set([]) + bss.currently_bad_items = set([]) + + binary_search_perforce.verbose = bss.verbose + return bss + except StandardError: + return None + + def RemoveState(self): + """Remove STATE_FILE and its symlinked data from file system.""" + if os.path.exists(STATE_FILE): + if os.path.islink(STATE_FILE): + real_file = os.readlink(STATE_FILE) + os.remove(real_file) + os.remove(STATE_FILE) + + def GetNextItems(self): + """Get next items for binary search based on result of the last test run.""" + border_item = self.binary_search.GetNext() + index = self.all_items.index(border_item) + + next_bad_items = self.all_items[:index + 1] + next_good_items = self.all_items[index + 1:] + list(self.known_good) + + return [next_bad_items, next_good_items] + + def ElapsedTimeString(self): + """Return h m s format of elapsed time since execution has started.""" + diff = int(time.time() - self.start_time) + seconds = diff % 60 + minutes = (diff / 60) % 60 + hours = diff / (60 * 60) + + seconds = str(seconds).rjust(2) + minutes = str(minutes).rjust(2) + hours = str(hours).rjust(2) + + return '%sh %sm %ss' % (hours, minutes, seconds) + + def _OutputProgress(self, progress_text): + """Output current progress of binary search to console and logs. + + Args: + progress_text: The progress to display to the user. + """ + progress = ('\n***** PROGRESS (elapsed time: %s) *****\n' + '%s' + '************************************************') + progress = progress % (self.ElapsedTimeString(), progress_text) + self.l.LogOutput(progress) + + def OutputIterationProgress(self): + out = ('Search %d of estimated %d.\n' + 'Prune %d of max %d.\n' + 'Current bad items found:\n' + '%s\n') + out = out % (self.search_cycles + 1, + math.ceil(math.log(len(self.all_items), 2)), + self.prune_cycles + 1, self.prune_iterations, + ', '.join(self.found_items)) + self._OutputProgress(out) + + def __str__(self): + ret = '' + ret += 'all: %s\n' % str(self.all_items) + ret += 'currently_good: %s\n' % str(self.currently_good_items) + ret += 'currently_bad: %s\n' % str(self.currently_bad_items) + ret += str(self.binary_search) + return ret + + +class MockBinarySearchState(BinarySearchState): + """Mock class for BinarySearchState.""" + + def __init__(self, **kwargs): + # Initialize all arguments to None + default_kwargs = { + 'get_initial_items': 'echo "1"', + 'switch_to_good': None, + 'switch_to_bad': None, + 'test_setup_script': None, + 'test_script': None, + 'incremental': True, + 'prune': False, + 'iterations': 50, + 'prune_iterations': 100, + 'verify': True, + 'file_args': False, + 'verbose': False + } + default_kwargs.update(kwargs) + super(MockBinarySearchState, self).__init__(**default_kwargs) + + +def _CanonicalizeScript(script_name): + """Return canonical path to script. + + Args: + script_name: Relative or absolute path to script + + Returns: + Canonicalized script path + """ + script_name = os.path.expanduser(script_name) + if not script_name.startswith('/'): + return os.path.join('.', script_name) + + +def Run(get_initial_items, + switch_to_good, + switch_to_bad, + test_script, + test_setup_script=None, + iterations=50, + prune=False, + noincremental=False, + file_args=False, + verify=True, + prune_iterations=100, + verbose=False, + resume=False): + """Run binary search tool. Equivalent to running through terminal. + + Args: + get_initial_items: Script to enumerate all items being binary searched + switch_to_good: Script that will take items as input and switch them to good + set + switch_to_bad: Script that will take items as input and switch them to bad + set + test_script: Script that will determine if the current combination of good + and bad items make a "good" or "bad" result. + test_setup_script: Script to do necessary setup (building, compilation, + etc.) for test_script. + iterations: How many binary search iterations to run before exiting. + prune: If False the binary search tool will stop when the first bad item is + found. Otherwise then binary search tool will continue searching + until all bad items are found (or prune_iterations is reached). + noincremental: Whether to send "diffs" of good/bad items to switch scripts. + file_args: If True then arguments to switch scripts will be a file name + containing a newline separated list of the items to switch. + verify: If True, run tests to ensure initial good/bad sets actually + produce a good/bad result. + prune_iterations: Max number of bad items to search for. + verbose: If True will print extra debug information to user. + resume: If True will resume using STATE_FILE. + + Returns: + 0 for success, error otherwise + """ + if resume: + bss = BinarySearchState.LoadState() + if not bss: + logger.GetLogger().LogOutput( + '%s is not a valid binary_search_tool state file, cannot resume!' % + STATE_FILE) + return 1 + else: + switch_to_good = _CanonicalizeScript(switch_to_good) + switch_to_bad = _CanonicalizeScript(switch_to_bad) + if test_setup_script: + test_setup_script = _CanonicalizeScript(test_setup_script) + test_script = _CanonicalizeScript(test_script) + get_initial_items = _CanonicalizeScript(get_initial_items) + incremental = not noincremental + + binary_search_perforce.verbose = verbose + + bss = BinarySearchState(get_initial_items, switch_to_good, switch_to_bad, + test_setup_script, test_script, incremental, prune, + iterations, prune_iterations, verify, file_args, + verbose) + bss.DoVerify() + + try: + bss.DoSearch() + bss.RemoveState() + logger.GetLogger().LogOutput('Total execution time: %s' % + bss.ElapsedTimeString()) + except Error as e: + logger.GetLogger().LogError(e) + return 1 + + return 0 + + +def Main(argv): + """The main function.""" + # Common initializations + + parser = argparse.ArgumentParser() + common.BuildArgParser(parser) + logger.GetLogger().LogOutput(' '.join(argv)) + options = parser.parse_args(argv) + + if not (options.get_initial_items and options.switch_to_good and + options.switch_to_bad and options.test_script) and not options.resume: + parser.print_help() + return 1 + + if options.resume: + logger.GetLogger().LogOutput('Resuming from %s' % STATE_FILE) + if len(argv) > 1: + logger.GetLogger().LogOutput(('Note: resuming from previous state, ' + 'ignoring given options and loading saved ' + 'options instead.')) + + # Get dictionary of all options + args = vars(options) + return Run(**args) + + +if __name__ == '__main__': + sys.exit(Main(sys.argv[1:])) diff --git a/binary_search_tool/bisect.py b/binary_search_tool/bisect.py new file mode 100755 index 00000000..d5a8b710 --- /dev/null +++ b/binary_search_tool/bisect.py @@ -0,0 +1,398 @@ +#!/usr/bin/python2 +"""The unified package/object bisecting tool.""" + +from __future__ import print_function + +import abc +import argparse +import os +import sys +from argparse import RawTextHelpFormatter + +import common + +from cros_utils import command_executer +from cros_utils import logger + +import binary_search_state + + +class Bisector(object): + """The abstract base class for Bisectors.""" + + # Make Bisector an abstract class + __metaclass__ = abc.ABCMeta + + def __init__(self, options, overrides=None): + """Constructor for Bisector abstract base class + + Args: + options: positional arguments for specific mode (board, remote, etc.) + overrides: optional dict of overrides for argument defaults + """ + self.options = options + self.overrides = overrides + if not overrides: + self.overrides = {} + self.logger = logger.GetLogger() + self.ce = command_executer.GetCommandExecuter() + + def _PrettyPrintArgs(self, args, overrides): + """Output arguments in a nice, human readable format + + Will print and log all arguments for the bisecting tool and make note of + which arguments have been overridden. + + Example output: + ./bisect.py package daisy 172.17.211.184 -I "" -t cros_pkg/my_test.sh + Performing ChromeOS Package bisection + Method Config: + board : daisy + remote : 172.17.211.184 + + Bisection Config: (* = overridden) + get_initial_items : cros_pkg/get_initial_items.sh + switch_to_good : cros_pkg/switch_to_good.sh + switch_to_bad : cros_pkg/switch_to_bad.sh + * test_setup_script : + * test_script : cros_pkg/my_test.sh + prune : True + noincremental : False + file_args : True + + Args: + args: The args to be given to binary_search_state.Run. This represents + how the bisection tool will run (with overridden arguments already + added in). + overrides: The dict of overriden arguments provided by the user. This is + provided so the user can be told which arguments were + overriden and with what value. + """ + # Output method config (board, remote, etc.) + options = vars(self.options) + out = '\nPerforming %s bisection\n' % self.method_name + out += 'Method Config:\n' + max_key_len = max([len(str(x)) for x in options.keys()]) + for key in sorted(options): + val = options[key] + key_str = str(key).rjust(max_key_len) + val_str = str(val) + out += ' %s : %s\n' % (key_str, val_str) + + # Output bisection config (scripts, prune, etc.) + out += '\nBisection Config: (* = overridden)\n' + max_key_len = max([len(str(x)) for x in args.keys()]) + # Print args in common._ArgsDict order + args_order = [x['dest'] for x in common.GetArgsDict().itervalues()] + compare = lambda x, y: cmp(args_order.index(x), args_order.index(y)) + + for key in sorted(args, cmp=compare): + val = args[key] + key_str = str(key).rjust(max_key_len) + val_str = str(val) + changed_str = '*' if key in overrides else ' ' + + out += ' %s %s : %s\n' % (changed_str, key_str, val_str) + + out += '\n' + self.logger.LogOutput(out) + + def ArgOverride(self, args, overrides, pretty_print=True): + """Override arguments based on given overrides and provide nice output + + Args: + args: dict of arguments to be passed to binary_search_state.Run (runs + dict.update, causing args to be mutated). + overrides: dict of arguments to update args with + pretty_print: if True print out args/overrides to user in pretty format + """ + args.update(overrides) + if pretty_print: + self._PrettyPrintArgs(args, overrides) + + @abc.abstractmethod + def PreRun(self): + pass + + @abc.abstractmethod + def Run(self): + pass + + @abc.abstractmethod + def PostRun(self): + pass + + +class BisectPackage(Bisector): + """The class for package bisection steps.""" + + cros_pkg_setup = 'cros_pkg/setup.sh' + cros_pkg_cleanup = 'cros_pkg/%s_cleanup.sh' + + def __init__(self, options, overrides): + super(BisectPackage, self).__init__(options, overrides) + self.method_name = 'ChromeOS Package' + self.default_kwargs = { + 'get_initial_items': 'cros_pkg/get_initial_items.sh', + 'switch_to_good': 'cros_pkg/switch_to_good.sh', + 'switch_to_bad': 'cros_pkg/switch_to_bad.sh', + 'test_setup_script': 'cros_pkg/test_setup.sh', + 'test_script': 'cros_pkg/interactive_test.sh', + 'noincremental': False, + 'prune': True, + 'file_args': True + } + self.setup_cmd = ('%s %s %s' % (self.cros_pkg_setup, self.options.board, + self.options.remote)) + self.ArgOverride(self.default_kwargs, self.overrides) + + def PreRun(self): + ret, _, _ = self.ce.RunCommandWExceptionCleanup( + self.setup_cmd, print_to_console=True) + if ret: + self.logger.LogError('Package bisector setup failed w/ error %d' % ret) + return 1 + return 0 + + def Run(self): + return binary_search_state.Run(**self.default_kwargs) + + def PostRun(self): + cmd = self.cros_pkg_cleanup % self.options.board + ret, _, _ = self.ce.RunCommandWExceptionCleanup(cmd, print_to_console=True) + if ret: + self.logger.LogError('Package bisector cleanup failed w/ error %d' % ret) + return 1 + + self.logger.LogOutput(('Cleanup successful! To restore the bisection ' + 'environment run the following:\n' + ' cd %s; %s') % (os.getcwd(), self.setup_cmd)) + return 0 + + +class BisectObject(Bisector): + """The class for object bisection steps.""" + + sysroot_wrapper_setup = 'sysroot_wrapper/setup.sh' + sysroot_wrapper_cleanup = 'sysroot_wrapper/cleanup.sh' + + def __init__(self, options, overrides): + super(BisectObject, self).__init__(options, overrides) + self.method_name = 'ChromeOS Object' + self.default_kwargs = { + 'get_initial_items': 'sysroot_wrapper/get_initial_items.sh', + 'switch_to_good': 'sysroot_wrapper/switch_to_good.sh', + 'switch_to_bad': 'sysroot_wrapper/switch_to_bad.sh', + 'test_setup_script': 'sysroot_wrapper/test_setup.sh', + 'test_script': 'sysroot_wrapper/interactive_test.sh', + 'noincremental': False, + 'prune': True, + 'file_args': True + } + self.options = options + if options.dir: + os.environ['BISECT_DIR'] = options.dir + self.options.dir = os.environ.get('BISECT_DIR', '/tmp/sysroot_bisect') + self.setup_cmd = ('%s %s %s %s' % (self.sysroot_wrapper_setup, + self.options.board, self.options.remote, + self.options.package)) + + self.ArgOverride(self.default_kwargs, overrides) + + def PreRun(self): + ret, _, _ = self.ce.RunCommandWExceptionCleanup( + self.setup_cmd, print_to_console=True) + if ret: + self.logger.LogError('Object bisector setup failed w/ error %d' % ret) + return 1 + + os.environ['BISECT_STAGE'] = 'TRIAGE' + return 0 + + def Run(self): + return binary_search_state.Run(**self.default_kwargs) + + def PostRun(self): + cmd = self.sysroot_wrapper_cleanup + ret, _, _ = self.ce.RunCommandWExceptionCleanup(cmd, print_to_console=True) + if ret: + self.logger.LogError('Object bisector cleanup failed w/ error %d' % ret) + return 1 + self.logger.LogOutput(('Cleanup successful! To restore the bisection ' + 'environment run the following:\n' + ' cd %s; %s') % (os.getcwd(), self.setup_cmd)) + return 0 + + +class BisectAndroid(Bisector): + """The class for Android bisection steps.""" + + android_setup = 'android/setup.sh' + android_cleanup = 'android/cleanup.sh' + default_dir = os.path.expanduser('~/ANDROID_BISECT') + + def __init__(self, options, overrides): + super(BisectAndroid, self).__init__(options, overrides) + self.method_name = 'Android' + self.default_kwargs = { + 'get_initial_items': 'android/get_initial_items.sh', + 'switch_to_good': 'android/switch_to_good.sh', + 'switch_to_bad': 'android/switch_to_bad.sh', + 'test_setup_script': 'android/test_setup.sh', + 'test_script': 'android/interactive_test.sh', + 'prune': True, + 'file_args': True, + 'noincremental': False, + } + self.options = options + if options.dir: + os.environ['BISECT_DIR'] = options.dir + self.options.dir = os.environ.get('BISECT_DIR', self.default_dir) + + num_jobs = "NUM_JOBS='%s'" % self.options.num_jobs + device_id = '' + if self.options.device_id: + device_id = "ANDROID_SERIAL='%s'" % self.options.device_id + + self.setup_cmd = ('%s %s %s %s' % (num_jobs, device_id, self.android_setup, + self.options.android_src)) + + self.ArgOverride(self.default_kwargs, overrides) + + def PreRun(self): + ret, _, _ = self.ce.RunCommandWExceptionCleanup( + self.setup_cmd, print_to_console=True) + if ret: + self.logger.LogError('Android bisector setup failed w/ error %d' % ret) + return 1 + + os.environ['BISECT_STAGE'] = 'TRIAGE' + return 0 + + def Run(self): + return binary_search_state.Run(**self.default_kwargs) + + def PostRun(self): + cmd = self.android_cleanup + ret, _, _ = self.ce.RunCommandWExceptionCleanup(cmd, print_to_console=True) + if ret: + self.logger.LogError('Android bisector cleanup failed w/ error %d' % ret) + return 1 + self.logger.LogOutput(('Cleanup successful! To restore the bisection ' + 'environment run the following:\n' + ' cd %s; %s') % (os.getcwd(), self.setup_cmd)) + return 0 + + +def Run(bisector): + log = logger.GetLogger() + + log.LogOutput('Setting up Bisection tool') + ret = bisector.PreRun() + if ret: + return ret + + log.LogOutput('Running Bisection tool') + ret = bisector.Run() + if ret: + return ret + + log.LogOutput('Cleaning up Bisection tool') + ret = bisector.PostRun() + if ret: + return ret + + return 0 + + +_HELP_EPILOG = """ +Run ./bisect.py {method} --help for individual method help/args + +------------------ + +See README.bisect for examples on argument overriding + +See below for full override argument reference: +""" + + +def Main(argv): + override_parser = argparse.ArgumentParser( + add_help=False, + argument_default=argparse.SUPPRESS, + usage='bisect.py {mode} [options]') + common.BuildArgParser(override_parser, override=True) + + epilog = _HELP_EPILOG + override_parser.format_help() + parser = argparse.ArgumentParser( + epilog=epilog, formatter_class=RawTextHelpFormatter) + subparsers = parser.add_subparsers( + title='Bisect mode', + description=('Which bisection method to ' + 'use. Each method has ' + 'specific setup and ' + 'arguments. Please consult ' + 'the README for more ' + 'information.')) + + parser_package = subparsers.add_parser('package') + parser_package.add_argument('board', help='Board to target') + parser_package.add_argument('remote', help='Remote machine to test on') + parser_package.set_defaults(handler=BisectPackage) + + parser_object = subparsers.add_parser('object') + parser_object.add_argument('board', help='Board to target') + parser_object.add_argument('remote', help='Remote machine to test on') + parser_object.add_argument('package', help='Package to emerge and test') + parser_object.add_argument( + '--dir', + help=('Bisection directory to use, sets ' + '$BISECT_DIR if provided. Defaults to ' + 'current value of $BISECT_DIR (or ' + '/tmp/sysroot_bisect if $BISECT_DIR is ' + 'empty).')) + parser_object.set_defaults(handler=BisectObject) + + parser_android = subparsers.add_parser('android') + parser_android.add_argument('android_src', help='Path to android source tree') + parser_android.add_argument( + '--dir', + help=('Bisection directory to use, sets ' + '$BISECT_DIR if provided. Defaults to ' + 'current value of $BISECT_DIR (or ' + '~/ANDROID_BISECT/ if $BISECT_DIR is ' + 'empty).')) + parser_android.add_argument( + '-j', + '--num_jobs', + type=int, + default=1, + help=('Number of jobs that make and various ' + 'scripts for bisector can spawn. Setting ' + 'this value too high can freeze up your ' + 'machine!')) + parser_android.add_argument( + '--device_id', + default='', + help=('Device id for device used for testing. ' + 'Use this if you have multiple Android ' + 'devices plugged into your machine.')) + parser_android.set_defaults(handler=BisectAndroid) + + options, remaining = parser.parse_known_args(argv) + if remaining: + overrides = override_parser.parse_args(remaining) + overrides = vars(overrides) + else: + overrides = {} + + subcmd = options.handler + del options.handler + + bisector = subcmd(options, overrides) + return Run(bisector) + + +if __name__ == '__main__': + os.chdir(os.path.dirname(__file__)) + sys.exit(Main(sys.argv[1:])) diff --git a/binary_search_tool/bisect_driver.py b/binary_search_tool/bisect_driver.py new file mode 100644 index 00000000..0b3fb1d4 --- /dev/null +++ b/binary_search_tool/bisect_driver.py @@ -0,0 +1,334 @@ +# Copyright 2016 Google Inc. All Rights Reserved. +# +# This script is used to help the compiler wrapper in the Android build system +# bisect for bad object files. +"""Utilities for bisection of Android object files. + +This module contains a set of utilities to allow bisection between +two sets (good and bad) of object files. Mostly used to find compiler +bugs. + +Reference page: +https://sites.google.com/a/google.com/chromeos-toolchain-team-home2/home/team-tools-and-scripts/bisecting-chromeos-compiler-problems/bisection-compiler-wrapper + +Design doc: +https://docs.google.com/document/d/1yDgaUIa2O5w6dc3sSTe1ry-1ehKajTGJGQCbyn0fcEM +""" + +from __future__ import print_function + +import contextlib +import fcntl +import os +import shutil +import subprocess +import sys + +VALID_MODES = ['POPULATE_GOOD', 'POPULATE_BAD', 'TRIAGE'] +GOOD_CACHE = 'good' +BAD_CACHE = 'bad' +LIST_FILE = os.path.join(GOOD_CACHE, '_LIST') + +CONTINUE_ON_MISSING = os.environ.get('BISECT_CONTINUE_ON_MISSING', None) == '1' +WRAPPER_SAFE_MODE = os.environ.get('BISECT_WRAPPER_SAFE_MODE', None) == '1' + + +class Error(Exception): + """The general compiler wrapper error class.""" + pass + + +@contextlib.contextmanager +def lock_file(path, mode): + """Lock file and block if other process has lock on file. + + Acquire exclusive lock for file. Only blocks other processes if they attempt + to also acquire lock through this method. If only reading (modes 'r' and 'rb') + then the lock is shared (i.e. many reads can happen concurrently, but only one + process may write at a time). + + This function is a contextmanager, meaning it's meant to be used with the + "with" statement in Python. This is so cleanup and setup happens automatically + and cleanly. Execution of the outer "with" statement happens at the "yield" + statement. Execution resumes after the yield when the outer "with" statement + ends. + + Args: + path: path to file being locked + mode: mode to open file with ('w', 'r', etc.) + """ + with open(path, mode) as f: + # Share the lock if just reading, make lock exclusive if writing + if f.mode == 'r' or f.mode == 'rb': + lock_type = fcntl.LOCK_SH + else: + lock_type = fcntl.LOCK_EX + + try: + fcntl.lockf(f, lock_type) + yield f + f.flush() + except: + raise + finally: + fcntl.lockf(f, fcntl.LOCK_UN) + + +def log_to_file(path, execargs, link_from=None, link_to=None): + """Common logging function. + + Log current working directory, current execargs, and a from-to relationship + between files. + """ + with lock_file(path, 'a') as log: + log.write('cd: %s; %s\n' % (os.getcwd(), ' '.join(execargs))) + if link_from and link_to: + log.write('%s -> %s\n' % (link_from, link_to)) + + +def exec_and_return(execargs): + """Execute process and return. + + Execute according to execargs and return immediately. Don't inspect + stderr or stdout. + """ + return subprocess.call(execargs) + + +def which_cache(obj_file): + """Determine which cache an object belongs to. + + The binary search tool creates two files for each search iteration listing + the full set of bad objects and full set of good objects. We use this to + determine where an object file should be linked from (good or bad). + """ + bad_set_file = os.environ.get('BISECT_BAD_SET') + ret = subprocess.call(['grep', '-x', '-q', obj_file, bad_set_file]) + if ret == 0: + return BAD_CACHE + else: + return GOOD_CACHE + + +def makedirs(path): + """Try to create directories in path.""" + try: + os.makedirs(path) + except os.error: + if not os.path.isdir(path): + raise + + +def get_obj_path(execargs): + """Get the object path for the object file in the list of arguments. + + Returns: + Absolute object path from execution args (-o argument). If no object being + outputted or output doesn't end in ".o" then return empty string. + """ + try: + i = execargs.index('-o') + except ValueError: + return '' + + obj_path = execargs[i + 1] + if not obj_path.endswith(('.o',)): + # TODO: what suffixes do we need to contemplate + # TODO: add this as a warning + # TODO: need to handle -r compilations + return '' + + return os.path.abspath(obj_path) + + +def get_dep_path(execargs): + """Get the dep file path for the dep file in the list of arguments. + + Returns: + Absolute path of dependency file path from execution args (-o argument). If + no dependency being outputted then return empty string. + """ + if '-MD' not in execargs and '-MMD' not in execargs: + return '' + + # If -MF given this is the path of the dependency file. Otherwise the + # dependency file is the value of -o but with a .d extension + if '-MF' in execargs: + i = execargs.index('-MF') + dep_path = execargs[i + 1] + return os.path.abspath(dep_path) + + full_obj_path = get_obj_path(execargs) + if not full_obj_path: + return '' + + return full_obj_path[:-2] + '.d' + + +def get_dwo_path(execargs): + """Get the dwo file path for the dwo file in the list of arguments. + + Returns: + Absolute dwo file path from execution args (-gsplit-dwarf argument) If no + dwo file being outputted then return empty string. + """ + if '-gsplit-dwarf' not in execargs: + return '' + + full_obj_path = get_obj_path(execargs) + if not full_obj_path: + return '' + + return full_obj_path[:-2] + '.dwo' + + +def in_object_list(obj_name, list_filename): + """Check if object file name exist in file with object list.""" + if not obj_name: + return False + + with lock_file(list_filename, 'r') as list_file: + for line in list_file: + if line.strip() == obj_name: + return True + + return False + + +def get_side_effects(execargs): + """Determine side effects generated by compiler + + Returns: + List of paths of objects that the compiler generates as side effects. + """ + side_effects = [] + + # Cache dependency files + full_dep_path = get_dep_path(execargs) + if full_dep_path: + side_effects.append(full_dep_path) + + # Cache dwo files + full_dwo_path = get_dwo_path(execargs) + if full_dwo_path: + side_effects.append(full_dwo_path) + + return side_effects + + +def cache_file(execargs, bisect_dir, cache, abs_file_path): + """Cache compiler output file (.o/.d/.dwo).""" + # os.path.join fails with absolute paths, use + instead + bisect_path = os.path.join(bisect_dir, cache) + abs_file_path + bisect_path_dir = os.path.dirname(bisect_path) + makedirs(bisect_path_dir) + pop_log = os.path.join(bisect_dir, cache, '_POPULATE_LOG') + log_to_file(pop_log, execargs, abs_file_path, bisect_path) + + try: + if os.path.exists(abs_file_path): + shutil.copy2(abs_file_path, bisect_path) + except Exception: + print('Could not cache file %s' % abs_file_path, file=sys.stderr) + raise + + +def restore_file(bisect_dir, cache, abs_file_path): + """Restore file from cache (.o/.d/.dwo).""" + # os.path.join fails with absolute paths, use + instead + cached_path = os.path.join(bisect_dir, cache) + abs_file_path + if os.path.exists(cached_path): + if os.path.exists(abs_file_path): + os.remove(abs_file_path) + os.link(cached_path, abs_file_path) + else: + raise Error(('%s is missing from %s cache! Unsure how to proceed. Make ' + 'will now crash.' % (cache, cached_path))) + + +def bisect_populate(execargs, bisect_dir, population_name): + """Add necessary information to the bisect cache for the given execution. + + Extract the necessary information for bisection from the compiler + execution arguments and put it into the bisection cache. This + includes copying the created object file, adding the object + file path to the cache list and keeping a log of the execution. + + Args: + execargs: compiler execution arguments. + bisect_dir: bisection directory. + population_name: name of the cache being populated (good/bad). + """ + retval = exec_and_return(execargs) + if retval: + return retval + + full_obj_path = get_obj_path(execargs) + # If not a normal compiler call then just exit + if not full_obj_path: + return + + cache_file(execargs, bisect_dir, population_name, full_obj_path) + + population_dir = os.path.join(bisect_dir, population_name) + with lock_file(os.path.join(population_dir, '_LIST'), 'a') as object_list: + object_list.write('%s\n' % full_obj_path) + + for side_effect in get_side_effects(execargs): + cache_file(execargs, bisect_dir, population_name, side_effect) + + +def bisect_triage(execargs, bisect_dir): + full_obj_path = get_obj_path(execargs) + obj_list = os.path.join(bisect_dir, LIST_FILE) + + # If the output isn't an object file just call compiler + if not full_obj_path: + return exec_and_return(execargs) + + # If this isn't a bisected object just call compiler + # This shouldn't happen! + if not in_object_list(full_obj_path, obj_list): + if CONTINUE_ON_MISSING: + log_file = os.path.join(bisect_dir, '_MISSING_CACHED_OBJ_LOG') + log_to_file(log_file, execargs, '? compiler', full_obj_path) + return exec_and_return(execargs) + else: + raise Error(('%s is missing from cache! To ignore export ' + 'BISECT_CONTINUE_ON_MISSING=1. See documentation for more ' + 'details on this option.' % full_obj_path)) + + cache = which_cache(full_obj_path) + + # If using safe WRAPPER_SAFE_MODE option call compiler and overwrite the + # result from the good/bad cache. This option is safe and covers all compiler + # side effects, but is very slow! + if WRAPPER_SAFE_MODE: + retval = exec_and_return(execargs) + if retval: + return retval + os.remove(full_obj_path) + restore_file(bisect_dir, cache, full_obj_path) + return + + # Generate compiler side effects. Trick Make into thinking compiler was + # actually executed. + for side_effect in get_side_effects(execargs): + restore_file(bisect_dir, cache, side_effect) + + # If generated object file happened to be pruned/cleaned by Make then link it + # over from cache again. + if not os.path.exists(full_obj_path): + restore_file(bisect_dir, cache, full_obj_path) + + +def bisect_driver(bisect_stage, bisect_dir, execargs): + """Call appropriate bisection stage according to value in bisect_stage.""" + if bisect_stage == 'POPULATE_GOOD': + bisect_populate(execargs, bisect_dir, GOOD_CACHE) + elif bisect_stage == 'POPULATE_BAD': + bisect_populate(execargs, bisect_dir, BAD_CACHE) + elif bisect_stage == 'TRIAGE': + bisect_triage(execargs, bisect_dir) + else: + raise ValueError('wrong value for BISECT_STAGE: %s' % bisect_stage) diff --git a/binary_search_tool/common.py b/binary_search_tool/common.py new file mode 100644 index 00000000..945270be --- /dev/null +++ b/binary_search_tool/common.py @@ -0,0 +1,261 @@ +"""Common config and logic for binary search tool + +This module serves two main purposes: + 1. Programatically include the utils module in PYTHONPATH + 2. Create the argument parsing shared between binary_search_state.py and + bisect.py + +The argument parsing is handled by populating _ArgsDict with all arguments. +_ArgsDict is required so that binary_search_state.py and bisect.py can share +the argument parsing, but treat them slightly differently. For example, +bisect.py requires that all argument defaults are suppressed so that overriding +can occur properly (i.e. only options that are explicitly entered by the user +end up in the resultant options dictionary). + +ArgumentDict inherits OrderedDict in order to preserve the order the args are +created so the help text is made properly. +""" + +from __future__ import print_function + +import collections +import os +import sys + +# Programatically adding utils python path to PYTHONPATH +if os.path.isabs(sys.argv[0]): + utils_pythonpath = os.path.abspath('{0}/..'.format( + os.path.dirname(sys.argv[0]))) +else: + wdir = os.getcwd() + utils_pythonpath = os.path.abspath('{0}/{1}/..'.format(wdir, os.path.dirname( + sys.argv[0]))) +sys.path.append(utils_pythonpath) + + +class ArgumentDict(collections.OrderedDict): + """Wrapper around OrderedDict, represents CLI arguments for program. + + AddArgument enforces the following layout: + { + ['-n', '--iterations'] : { + 'dest': 'iterations', + 'type': int, + 'help': 'Number of iterations to try in the search.', + 'default': 50 + } + [arg_name1, arg_name2, ...] : { + arg_option1 : arg_option_val1, + ... + }, + ... + } + """ + _POSSIBLE_OPTIONS = ['action', 'nargs', 'const', 'default', 'type', 'choices', + 'required', 'help', 'metavar', 'dest'] + + def AddArgument(self, *args, **kwargs): + """Add argument to ArgsDict, has same signature as argparse.add_argument + + Emulates the the argparse.add_argument method so the internal OrderedDict + can be safely and easily populated. Each call to this method will have a 1-1 + corresponding call to argparse.add_argument once BuildArgParser is called. + + Args: + *args: The names for the argument (-V, --verbose, etc.) + **kwargs: The options for the argument, corresponds to the args of + argparse.add_argument + + Returns: + None + + Raises: + TypeError: if args is empty or if option in kwargs is not a valid + option for argparse.add_argument. + """ + if len(args) == 0: + raise TypeError('Argument needs at least one name') + + for key in kwargs: + if key not in self._POSSIBLE_OPTIONS: + raise TypeError('Invalid option "%s" for argument %s' % (key, args[0])) + + self[args] = kwargs + + +_ArgsDict = ArgumentDict() + + +def GetArgsDict(): + """_ArgsDict singleton method""" + if not _ArgsDict: + _BuildArgsDict(_ArgsDict) + return _ArgsDict + + +def BuildArgParser(parser, override=False): + """Add all arguments from singleton ArgsDict to parser. + + Will take argparse parser and add all arguments in ArgsDict. Will ignore + the default and required options if override is set to True. + + Args: + parser: type argparse.ArgumentParser, will call add_argument for every item + in _ArgsDict + override: True if being called from bisect.py. Used to say that default and + required options are to be ignored + + Returns: + None + """ + ArgsDict = GetArgsDict() + + # Have no defaults when overriding + for arg_names, arg_options in ArgsDict.iteritems(): + if override: + arg_options = arg_options.copy() + arg_options.pop('default', None) + arg_options.pop('required', None) + + parser.add_argument(*arg_names, **arg_options) + + +def StrToBool(str_in): + if str_in.lower() in ['true', 't', '1']: + return True + if str_in.lower() in ['false', 'f', '0']: + return False + + raise AttributeError('%s is not a valid boolean string' % str_in) + + +def _BuildArgsDict(args): + """Populate ArgumentDict with all arguments""" + args.AddArgument( + '-n', + '--iterations', + dest='iterations', + type=int, + help='Number of iterations to try in the search.', + default=50) + args.AddArgument( + '-i', + '--get_initial_items', + dest='get_initial_items', + help=('Script to run to get the initial objects. ' + 'If your script requires user input ' + 'the --verbose option must be used')) + args.AddArgument( + '-g', + '--switch_to_good', + dest='switch_to_good', + help=('Script to run to switch to good. ' + 'If your switch script requires user input ' + 'the --verbose option must be used')) + args.AddArgument( + '-b', + '--switch_to_bad', + dest='switch_to_bad', + help=('Script to run to switch to bad. ' + 'If your switch script requires user input ' + 'the --verbose option must be used')) + args.AddArgument( + '-I', + '--test_setup_script', + dest='test_setup_script', + help=('Optional script to perform building, flashing, ' + 'and other setup before the test script runs.')) + args.AddArgument( + '-t', + '--test_script', + dest='test_script', + help=('Script to run to test the ' + 'output after packages are built.')) + # No input (evals to False), + # --prune (evals to True), + # --prune=False, + # --prune=True + args.AddArgument( + '-p', + '--prune', + dest='prune', + nargs='?', + const=True, + default=False, + type=StrToBool, + metavar='bool', + help=('If True, continue until all bad items are found. ' + 'Defaults to False.')) + # No input (evals to False), + # --noincremental (evals to True), + # --noincremental=False, + # --noincremental=True + args.AddArgument( + '-c', + '--noincremental', + dest='noincremental', + nargs='?', + const=True, + default=False, + type=StrToBool, + metavar='bool', + help=('If True, don\'t propagate good/bad changes ' + 'incrementally. Defaults to False.')) + # No input (evals to False), + # --file_args (evals to True), + # --file_args=False, + # --file_args=True + args.AddArgument( + '-f', + '--file_args', + dest='file_args', + nargs='?', + const=True, + default=False, + type=StrToBool, + metavar='bool', + help=('Whether to use a file to pass arguments to scripts. ' + 'Defaults to False.')) + # No input (evals to True), + # --verify (evals to True), + # --verify=False, + # --verify=True + args.AddArgument( + '--verify', + dest='verify', + nargs='?', + const=True, + default=True, + type=StrToBool, + metavar='bool', + help=('Whether to run verify iterations before searching. ' + 'Defaults to True.')) + args.AddArgument( + '-N', + '--prune_iterations', + dest='prune_iterations', + type=int, + help='Number of prune iterations to try in the search.', + default=100) + # No input (evals to False), + # --verbose (evals to True), + # --verbose=False, + # --verbose=True + args.AddArgument( + '-V', + '--verbose', + dest='verbose', + nargs='?', + const=True, + default=False, + type=StrToBool, + metavar='bool', + help='If True, print full output to console.') + args.AddArgument( + '-r', + '--resume', + dest='resume', + action='store_true', + help=('Resume bisection tool execution from state file.' + 'Useful if the last bisection was terminated ' + 'before it could properly finish.')) diff --git a/binary_search_tool/common/boot_test.sh b/binary_search_tool/common/boot_test.sh new file mode 100755 index 00000000..8f6d9a7d --- /dev/null +++ b/binary_search_tool/common/boot_test.sh @@ -0,0 +1,22 @@ +#!/bin/bash -u +# +# Copyright 2016 Google Inc. All Rights Reserved. +# +# This script pings the chromebook to determine if it has successfully booted. +# +# This script is intended to be used by binary_search_state.py, as +# part of the binary search triage on ChromeOS package and object files. +# It waits for the test setup script to build and install the image, then pings +# the machine. It should return '0' if the test succeeds (the image booted); '1' +# if the test fails (the image did not boot); and '125' if it could not +# determine (does not apply in this case). +# + +source common/common.sh + +# Send 3 pings and wait 3 seconds for any responsed (then timeout). +ping -c 3 -W 3 ${BISECT_REMOTE} +retval=$? + + +exit $retval diff --git a/binary_search_tool/common/hash_test.sh b/binary_search_tool/common/hash_test.sh new file mode 100755 index 00000000..5450988e --- /dev/null +++ b/binary_search_tool/common/hash_test.sh @@ -0,0 +1,57 @@ +#!/bin/bash -u +# +# Copyright 2016 Google Inc. All Rights Reserved. +# +# This script is intended to be used by binary_search_state.py. It is to +# be used for testing/development of the binary search triage tool +# itself. It waits for the test setup script to build and install the +# image, then checks the hashes in the provided file. +# If the real hashes match the checksum hashes, then the image is 'good', +# otherwise it is 'bad'. This allows the rest of the bisecting tool +# to run without requiring help from the user (as it would if we were +# dealing with a real 'bad' image). +# + +# +# Initialize the value below before using this script!!! +# +# Make an md5sum of all the files you want to check. For example if you want +# file1, file2, and file3 to be found as bad items: +# +# md5sum file1 file2 file3 > checksum.out +# +# (Make sure you are hashing the files from your good build and that the hashes +# from good to bad build differ) +# +# Then set HASHES_FILE to be the path to 'checksum.out' +# In this example, file1, file2, file3 will be found as the bad files +# because their hashes won't match when from the bad build tree. This is +# assuming that the hashes between good/bad builds change. It is suggested to +# build good and bad builds at different optimization levels to help ensure +# each item has a different hash. +# +# WARNING: +# Make sure paths to all files are absolute paths or relative to +# binary_search_state.py +# +# cros_pkg bisector example: +# 1. Build good packages with -O1, bad packages with -O2 +# 2. cros_pkg/switch_to_good.sh pkg1 pkg2 pkg3 +# 3. md5sum pkg1 pkg2 pkg3 > checksum.out.cros_pkg +# 4. Set HASHES_FILE to be checksum.out.cros_pkg +# 5. Run the bisector with this test script +# +# +HASHES_FILE= + +if [[ -z "${HASHES_FILE}" || ! -f "${HASHES_FILE}" ]]; +then + echo "ERROR: HASHES_FILE must be intialized in common/hash_test.sh" + exit 3 +fi + +md5sum -c --status ${HASHES_FILE} +md5_result=$? + + +exit $md5_result diff --git a/binary_search_tool/common/interactive_test.sh b/binary_search_tool/common/interactive_test.sh new file mode 100755 index 00000000..8773dd12 --- /dev/null +++ b/binary_search_tool/common/interactive_test.sh @@ -0,0 +1,37 @@ +#!/bin/bash -u +# +# Copyright 2016 Google Inc. All Rights Reserved. +# +# This script pings the chromebook to determine if it successfully booted. +# It then asks the user if the image is good or not, allowing the user to +# conduct whatever tests the user wishes, and waiting for a response. +# +# This script is intended to be used by binary_search_state.py, as +# part of the binary search triage on ChromeOS package and object files. It +# waits for the test setup script to build and install the image, then asks the +# user if the image is good or not. It should return '0' if the test succeeds +# (the image is 'good'); '1' if the test fails (the image is 'bad'); and '125' +# if it could not determine (does not apply in this case). +# + +source common/common.sh + +ping -c 3 -W 3 ${BISECT_REMOTE} +retval=$? + +if [[ ${retval} -eq 0 ]]; then + echo "ChromeOS image has been built and installed on ${BISECT_REMOTE}." +else + exit 1 +fi + +while true; do + read -p "Is this a good ChromeOS image?" yn + case $yn in + [Yy]* ) exit 0;; + [Nn]* ) exit 1;; + * ) echo "Please answer yes or no.";; + esac +done + +exit 125 diff --git a/binary_search_tool/common/test_setup.sh b/binary_search_tool/common/test_setup.sh new file mode 100755 index 00000000..c4f5f698 --- /dev/null +++ b/binary_search_tool/common/test_setup.sh @@ -0,0 +1,166 @@ +#!/bin/bash +# +# Copyright 2016 Google Inc. All Rights Reserved. +# +# This is a generic ChromeOS package/image test setup script. It is meant to +# be used for either the object file or package bisection tools. This script +# does one of the following depending on what ${BISECT_MODE} is set to: +# +# 1) ${BISECT_MODE} is PACKAGE_MODE: +# build_image is called and generates a new ChromeOS image using whatever +# packages are currently in the build tree. This image is then pushed to the +# remote machine using flash over ethernet (or usb flash if ethernet flash +# fails). +# +# 2) ${BISECT_MODE} is OBJECT_MODE: +# emerge is called for ${BISECT_PACKAGE} and generates a build for said +# package. This package is then deployed to the remote machine and the machine +# is rebooted. If deploying fails then a new ChromeOS image is built from +# scratch and pushed to the machine like in PACKAGE_MODE. +# +# This script is intended to be used by binary_search_state.py, as +# part of the binary search triage on ChromeOS objects and packages. It should +# return '0' if the setup succeeds; and '1' if the setup fails (the image +# could not build or be flashed). +# + +export PYTHONUNBUFFERED=1 + +source common/common.sh + +usb_flash() +{ + echo + echo "Insert a usb stick into the current machine" + echo "Note: The cros flash will take time and doesn't give much output." + echo " Be patient. If your usb access light is flashing it's working." + sleep 1 + read -p "Press enter to continue" notused + + cros flash --board=${BISECT_BOARD} --clobber-stateful usb:// ~/trunk/src/build/images/${BISECT_BOARD}/latest/chromiumos_test_image.bin + + echo + echo "Flash to usb complete!" + echo "Plug the usb into your chromebook and install the image." + echo "Refer to the ChromiumOS Developer's Handbook for more details." + echo "http://www.chromium.org/chromium-os/developer-guide#TOC-Boot-from-your-USB-disk" + while true; do + sleep 1 + read -p "Was the installation of the image successful? " choice + case $choice in + [Yy]*) return 0;; + [Nn]*) return 1;; + *) echo "Please answer y or n.";; + esac + done +} + +ethernet_flash() +{ + echo + echo "Please ensure your Chromebook is up and running Chrome so" + echo "cros flash may run." + echo "If your Chromebook has a broken image you can try:" + echo "1. Rebooting your Chromebook 6 times to install the last working image" + echo "2. Alternatively, running the following command on the Chromebook" + echo " will also rollback to the last working image:" + echo " 'update_engine_client --rollback --nopowerwash --reboot'" + echo "3. Flashing a new image through USB" + echo + sleep 1 + read -p $'Press enter to continue and retry the ethernet flash' notused + cros flash --board=${BISECT_BOARD} --clobber-stateful ${BISECT_REMOTE} ~/trunk/src/build/images/${BISECT_BOARD}/latest/chromiumos_test_image.bin +} + +reboot() +{ + ret_val=0 + pushd ~/trunk/src/scripts > /dev/null + set -- --remote=${BISECT_REMOTE} + . ./common.sh || ret_val=1 + . ./remote_access.sh || ret_val=1 + TMP=$(mktemp -d) + FLAGS "$@" || ret_val=1 + remote_access_init || ret_val=1 + remote_reboot || ret_val=1 + popd > /dev/null + + return $ret_val +} + +echo +echo "INSTALLATION BEGIN" +echo + +if [[ "${BISECT_MODE}" == "OBJECT_MODE" ]]; then + echo "EMERGING ${BISECT_PACKAGE}" + CLEAN_DELAY=0 emerge-${BISECT_BOARD} -C ${BISECT_PACKAGE} + emerge-${BISECT_BOARD} ${BISECT_PACKAGE} + emerge_status=$? + + if [[ ${emerge_status} -ne 0 ]] ; then + echo "emerging ${BISECT_PACKAGE} returned a non-zero status: $emerge_status" + exit 1 + fi + + echo + echo "DEPLOYING" + echo "cros deploy ${BISECT_REMOTE} ${BISECT_PACKAGE}" + cros deploy ${BISECT_REMOTE} ${BISECT_PACKAGE} --log-level=info + deploy_status=$? + + if [[ ${deploy_status} -eq 0 ]] ; then + echo "Deploy successful. Rebooting device..." + reboot + if [[ $? -ne 0 ]] ; then + echo + echo "Could not automatically reboot device!" + read -p "Please manually reboot device and press enter to continue" notused + fi + exit 0 + fi + + echo "Deploy failed! Trying build_image/cros flash instead..." + echo +fi + +echo "BUILDING IMAGE" +pushd ~/trunk/src/scripts +./build_image test --board=${BISECT_BOARD} --noenable_rootfs_verification --noeclean +build_status=$? +popd + +if [[ ${build_status} -eq 0 ]] ; then + echo + echo "FLASHING" + echo "Pushing built image onto device." + echo "cros flash --board=${BISECT_BOARD} --clobber-stateful ${BISECT_REMOTE} ~/trunk/src/build/images/${BISECT_BOARD}/latest/chromiumos_test_image.bin" + cros flash --board=${BISECT_BOARD} --clobber-stateful ${BISECT_REMOTE} ~/trunk/src/build/images/${BISECT_BOARD}/latest/chromiumos_test_image.bin + cros_flash_status=$? + while [[ ${cros_flash_status} -ne 0 ]] ; do + while true; do + echo + echo "cros flash has failed! From here you can:" + echo "1. Flash through USB" + echo "2. Retry flashing over ethernet" + echo "3. Abort this installation and skip this image" + echo "4. Abort this installation and mark test as failed" + sleep 1 + read -p "Which method would you like to do? " choice + case $choice in + 1) usb_flash && break;; + 2) ethernet_flash && break;; + 3) exit 125;; + 4) exit 1;; + *) echo "Please answer 1, 2, 3, or 4.";; + esac + done + + cros_flash_status=$? + done +else + echo "build_image returned a non-zero status: ${build_status}" + exit 1 +fi + +exit 0 diff --git a/binary_search_tool/compiler_wrapper.py b/binary_search_tool/compiler_wrapper.py new file mode 100755 index 00000000..3d6403a2 --- /dev/null +++ b/binary_search_tool/compiler_wrapper.py @@ -0,0 +1,63 @@ +#!/usr/bin/python2 +"""Prototype compiler wrapper. + +Only tested with: gcc, g++, clang, clang++ +Installation instructions: + 1. Rename compiler from <compiler_name> to <compiler_name>.real + 2. Create symlink from this script (compiler_wrapper.py), and name it + <compiler_name>. compiler_wrapper.py can live anywhere as long as it is + executable. + +Reference page: +https://sites.google.com/a/google.com/chromeos-toolchain-team-home2/home/team-tools-and-scripts/bisecting-chromeos-compiler-problems/bisection-compiler-wrapper + +Design doc: +https://docs.google.com/document/d/1yDgaUIa2O5w6dc3sSTe1ry-1ehKajTGJGQCbyn0fcEM +""" + +from __future__ import print_function + +import os +import shlex +import sys + +import bisect_driver + +WRAPPED = '%s.real' % sys.argv[0] +BISECT_STAGE = os.environ.get('BISECT_STAGE') +DEFAULT_BISECT_DIR = os.path.expanduser('~/ANDROID_BISECT') +BISECT_DIR = os.environ.get('BISECT_DIR') or DEFAULT_BISECT_DIR + + +def ProcessArgFile(arg_file): + args = [] + # Read in entire file at once and parse as if in shell + with open(arg_file, 'rb') as f: + args.extend(shlex.split(f.read())) + + return args + + +def Main(_): + if not os.path.islink(sys.argv[0]): + print("Compiler wrapper can't be called directly!") + return 1 + + execargs = [WRAPPED] + sys.argv[1:] + + if BISECT_STAGE not in bisect_driver.VALID_MODES or '-o' not in execargs: + os.execv(WRAPPED, [WRAPPED] + sys.argv[1:]) + + # Handle @file argument syntax with compiler + for idx, _ in enumerate(execargs): + # @file can be nested in other @file arguments, use While to re-evaluate + # the first argument of the embedded file. + while execargs[idx][0] == '@': + args_in_file = ProcessArgFile(execargs[idx][1:]) + execargs = execargs[0:idx] + args_in_file + execargs[idx + 1:] + + bisect_driver.bisect_driver(BISECT_STAGE, BISECT_DIR, execargs) + + +if __name__ == '__main__': + sys.exit(Main(sys.argv[1:])) diff --git a/binary_search_tool/cros_pkg/README.cros_pkg_triage b/binary_search_tool/cros_pkg/README.cros_pkg_triage new file mode 100644 index 00000000..5e285008 --- /dev/null +++ b/binary_search_tool/cros_pkg/README.cros_pkg_triage @@ -0,0 +1,185 @@ + +binary_search_state.py is a general binary search triage tool that +performs a binary search on a set of things to try to identify which +thing or thing(s) in the set is 'bad'. binary_search_state.py assumes +that the user has two sets, one where everything is known to be good, +ane one which contains at least one bad item. binary_search_state.py +then copies items from the good and bad sets into a working set and +tests the result (good or bad). binary_search_state.py requires that +a set of scripts be supplied to it for any particular job. For more +information on binary_search_state.py, see + +https://sites.google.com/a/google.com/chromeos-toolchain-team-home2/home/team-tools-and-scripts/binary-searcher-tool-for-triage + +This particular set of scripts is designed to work wtih +binary_search_state.py in order to find the bad package or set of +packages in a ChromeOS build. + + +QUICKSTART: + +After setting up your 3 build trees (see Prerequisites section), do the +following: + + - Decide which test script to use (boot_test.sh or + interactive_test.sh) + - Get the IP name or address of the chromebook you will use for testing. + - Do the following inside your chroot: + + $ cd ~/trunk/src/third_party/toolchain_utils/binary_search_tool + $ ./cros_pkg/setup.sh <board-to-test> <IP-name-or-address-of-chromebook> + + If you chose the boot test, then: + + $ python ./binary_search_state.py \ + --get_initial_items=cros_pkg/get_initial_items.sh \ + --switch_to_good=cros_pkg/switch_to_good.sh \ + --switch_to_bad=cros_pkg/switch_to_bad.sh \ + --test_setup_script=cros_pkg/test_setup.sh \ + --test_script=cros_pkg/boot_test.sh \ + --file_args \ + --prune + + Otherwise, if you chose the interactive test, then: + + $ python ./binary_search_state.py \ + --get_initial_items=cros_pkg/get_initial_items.sh \ + --switch_to_good=cros_pkg/switch_to_good.sh \ + --switch_to_bad=cros_pkg/switch_to_bad.sh \ + --test_setup_script=cros_pkg/test_setup.sh \ + --test_script=cros_pkg/interactive_test.sh \ + --file_args \ + --prune + + Once you have completely finished doing the binary search/triage, + run the genereated cleanup script, to restore your chroot to the state + it was in before you ran the setup.sh script: + + $ cros_pkg/${BOARD}_cleanup.sh + + + +FILES AND SCRIPTS: + + boot_test.sh - One of two possible test scripts used to determine + if the ChromeOS image built from the packages is good + or bad. This script tests to see if the image + booted, and requires no user intervention. + + create_cleanup_script.py - This is called by setup.sh, to + generate ${BOARD}_cleanup.sh, + which is supposed to be run by the user + after the binary search triage process is + finished, to undo the changes made by + setup.sh and return everything + to its original state. + + get_initial_items.sh - This script is used to determine the current + set of ChromeOS packages. + + test_setup.sh - This script will build and flash your image to the + remote machine. If the flash fails, this script will + help the user troubleshoot by flashing through usb or + by retrying the flash over ethernet. + + interactive_test.sh - One of two possible scripts used to determine + if the ChromeOS image built from the packages + is good or bad. This script requires user + interaction to determine if the image is + good or bad. + + setup.sh - This is the first script the user should call, after + taking care of the prerequisites. It sets up the + environment appropriately for running the ChromeOS + package binary search triage, and it generates two + necessary scripts (see below). + + switch_to_bad.sh - This script is used to copy packages from the + 'bad' build tree into the work area. + + switch_to_good.sh - This script is used to copy packages from the + good' build tree into the work area. + + +GENERATED SCRIPTS: + + common.sh - contains basic environment variable definitions for + this binary search triage session. + + ${BOARD}_cleanup.sh - script to undo all the changes made by + running setup.sh, and returning + everything to its original state. The user + should manually run this script once the + binary search triage process is over. + +ASSUMPTIONS: + +- There are two different ChromeOS builds, for the same board, with the + same set of ChromeOS packages. One build creates a good working ChromeOS + image and the other does not. + +- You have saved the complete build trees for both the good and bad builds. + + +PREREQUISITES FOR USING THESE SCRIPTS (inside the chroot): + +- The "good" build tree, for the board, is in /build/${board}.good + (e.g. /build/lumpy.good or /build/daisy.good). + +- The "bad" build tree is in /build/${board}.bad + (e.g. /build/lumpy.bad or /build/daisy.bad). + +- You made a complete copy of the "bad" build tree , and put it in + /build/${board}.work (e.g. /build/lumpy.work or /build/daisy.work. + The easiest way to do this is to use something similar to the + following set of commands (this example assumes the board is + 'lumpy'): + + $ cd /build + $ sudo tar -cvf lumpy.bad.tar lumpy.bad + $ sudo mv lumpy.bad lumpy.work + $ sudo tar -xvf lumpy.bad.tar + + +USING THESE SCRIPTS FOR BINARY TRIAGE OF PACKAGES: + +To use these scripts, you must first run setup.sh, passing it two +arguments (in order): the board for which you are building the image; +and the name or ip address of the chromebook you want to use for +testing your chromeos images. setup.sh will do the following: + + - Verify that your build trees are set up correctly (with good, bad + and work). + - Create a soft link for /build/${board} pointing to the work build + tree. + - Create the common.sh file that the other scripts passed to the + binary triage tool will need. + - Create a cleanup script, ${board}_cleanup.sh, for you to + run after you are done with the binary triages, to undo all of these + various changes that setup.sh did. + + +This set of scripts comes with two alternate test scripts. One test +script, boot_test.sh, just checks to make sure that the image +booted (i.e. responds to ping) and assumes that is enough. The other +test script, interactive_test.sh, is interactive and asks YOU +to tell it whether the image on the chromebook is ok or not (it +prompts you and waits for a response). + + +Once you have run setup.sh (and decided which test script you +want to use) run the binary triage tool using these scripts to +isolate/identify the bad package: + +~/trunk/src/third_party/toolchain_utils/binary_search_tool/binary_search_state.py \ + --get_initial_items=cros_pkg/get_initial_items.sh \ + --switch_to_good=cros_pkg/switch_to_good.sh \ + --switch_to_bad=cros_pkg/switch_to_bad.sh \ + --test_setup_script=cros_pkg/test_setup.sh \ + --test_script=cros_pkg/boots_test.sh \ # could use interactive_test.sh instead + --prune + + +After you have finished running the tool and have identified the bad +package(s), you will want to run the cleanup script that setup.sh +generated (cros_pkg/${BOARD}_cleanup.sh). diff --git a/binary_search_tool/cros_pkg/boot_test.sh b/binary_search_tool/cros_pkg/boot_test.sh new file mode 120000 index 00000000..9a345617 --- /dev/null +++ b/binary_search_tool/cros_pkg/boot_test.sh @@ -0,0 +1 @@ +../common/boot_test.sh
\ No newline at end of file diff --git a/binary_search_tool/cros_pkg/create_cleanup_script.py b/binary_search_tool/cros_pkg/create_cleanup_script.py new file mode 100755 index 00000000..32a1f160 --- /dev/null +++ b/binary_search_tool/cros_pkg/create_cleanup_script.py @@ -0,0 +1,114 @@ +#!/usr/bin/python2 +# +# Copyright 2015 Google Inc. All Rights Reserved +"""The script to generate a cleanup script after setup.sh. + +This script takes a set of flags, telling it what setup.sh changed +during the set up process. Based on the values of the input flags, it +generates a cleanup script, named ${BOARD}_cleanup.sh, which will +undo the changes made by setup.sh, returning everything to its +original state. +""" + +from __future__ import print_function + +import argparse +import sys + + +def Usage(parser, msg): + print('ERROR: ' + msg) + parser.print_help() + sys.exit(1) + + +def Main(argv): + """Generate a script to undo changes done by setup.sh + + The script setup.sh makes a change that needs to be + undone, namely it creates a soft link making /build/${board} point + to /build/${board}.work. To do this, it had to see if + /build/${board} already existed, and if so, whether it was a real + tree or a soft link. If it was soft link, it saved the old value + of the link, then deleted it and created the new link. If it was + a real tree, it renamed the tree to /build/${board}.save, and then + created the new soft link. If the /build/${board} did not + previously exist, then it just created the new soft link. + + This function takes arguments that tell it exactly what setup.sh + actually did, then generates a script to undo those exact changes. + """ + + parser = argparse.ArgumentParser() + parser.add_argument( + '--board', + dest='board', + required=True, + help='Chromeos board for packages/image.') + + parser.add_argument( + '--old_tree_missing', + dest='tree_existed', + action='store_false', + help='Did /build/${BOARD} exist.', + default=True) + + parser.add_argument( + '--renamed_tree', + dest='renamed_tree', + action='store_true', + help='Was /build/${BOARD} saved & renamed.', + default=False) + + parser.add_argument( + '--old_link', + dest='old_link', + help=('The original build tree soft link.')) + + options = parser.parse_args(argv[1:]) + + if options.old_link or options.renamed_tree: + if not options.tree_existed: + Usage(parser, 'If --tree_existed is False, cannot have ' + '--renamed_tree or --old_link') + + if options.old_link and options.renamed_tree: + Usage(parser, '--old_link and --renamed_tree are incompatible options.') + + if options.tree_existed: + if not options.old_link and not options.renamed_tree: + Usage(parser, 'If --tree_existed is True, then must have either ' + '--old_link or --renamed_tree') + + out_filename = 'cros_pkg/' + options.board + '_cleanup.sh' + + with open(out_filename, 'w') as out_file: + out_file.write('#!/bin/bash\n\n') + # First, remove the 'new' soft link. + out_file.write('sudo rm /build/%s\n' % options.board) + if options.tree_existed: + if options.renamed_tree: + # Old build tree existed and was a real tree, so it got + # renamed. Move the renamed tree back to the original tree. + out_file.write('sudo mv /build/%s.save /build/%s\n' % + (options.board, options.board)) + else: + # Old tree existed and was already a soft link. Re-create the + # original soft link. + original_link = options.old_link + if original_link[0] == "'": + original_link = original_link[1:] + if original_link[-1] == "'": + original_link = original_link[:-1] + out_file.write('sudo ln -s %s /build/%s\n' % (original_link, + options.board)) + out_file.write('\n') + # Remove common.sh file + out_file.write('rm common/common.sh\n') + + return 0 + + +if __name__ == '__main__': + retval = Main(sys.argv) + sys.exit(retval) diff --git a/binary_search_tool/cros_pkg/get_initial_items.sh b/binary_search_tool/cros_pkg/get_initial_items.sh new file mode 100755 index 00000000..49ca3d18 --- /dev/null +++ b/binary_search_tool/cros_pkg/get_initial_items.sh @@ -0,0 +1,16 @@ +#!/bin/bash -u +# +# Copyright 2015 Google Inc. All Rights Reserved. +# +# This script is intended to be used by binary_search_state.py, as +# part of the binary search triage on ChromeOS packages. This script +# generates the list of current ChromeOS packages, that is then used +# for doing the binary search. +# + +source common/common.sh + +cd ${GOOD_BUILD}/packages +find . -name "*.tbz2" + + diff --git a/binary_search_tool/cros_pkg/interactive_test.sh b/binary_search_tool/cros_pkg/interactive_test.sh new file mode 120000 index 00000000..18fe3958 --- /dev/null +++ b/binary_search_tool/cros_pkg/interactive_test.sh @@ -0,0 +1 @@ +../common/interactive_test.sh
\ No newline at end of file diff --git a/binary_search_tool/cros_pkg/setup.sh b/binary_search_tool/cros_pkg/setup.sh new file mode 100755 index 00000000..ae31fa82 --- /dev/null +++ b/binary_search_tool/cros_pkg/setup.sh @@ -0,0 +1,123 @@ +#!/bin/bash -u +# +# Copyright 2015 Google Inc. All Rights Reserved. +# +# This script is part of the ChromeOS package binary search triage process. +# It should be the first script called by the user, after the user has set up +# the three necessary build tree directories (see the prerequisites section of +# README.cros_pkg_triage). +# +# This script requires two arguments. The first argument must be the name of +# the board for which this work is being done (e.g. 'daisy', 'lumpy','parrot', +# etc.). The second argument must be the name or IP address of the chromebook +# on which the ChromeOS images will be pushed and tested. +# +# This script sets up a soft link definining /build/${board} to point +# to the working build tree, for the binary search triags process. In +# addition, this script generates two other scripts, common.sh, +# which generates enviroment variables used by the other scripts in the +# package binary search triage process; and ${board}_cleanup.sh, +# which undoes the various changes that this script performs, returning the +# user's environment to its original state. +# + +# Set up basic variables. + +BOARD=$1 +REMOTE=$2 + +GOOD_BUILD=/build/${BOARD}.good +BAD_BUILD=/build/${BOARD}.bad +WORK_BUILD=/build/${BOARD}.work + +# +# Verify that the necessary directories exist. +# + +if [[ ! -d ${GOOD_BUILD} ]] ; then + echo "Error: ${GOOD_BUILD} does not exist." + exit 1 +fi + +if [[ ! -d ${BAD_BUILD} ]] ; then + echo "Error: ${BAD_BUILD} does not exist." + exit 1 +fi + +if [[ ! -d ${WORK_BUILD} ]] ; then + echo "Error: ${WORK_BUILD} does not exist." + exit 1 +fi + +# +# Check to see if /build/${BOARD} already exists and if so, in what state. +# Set appropriate flags & values, in order to be able to undo these changes +# in ${board}_cleanup.sh. If it's a soft link, remove it; if it's a +# read tree, rename it. +# + +build_tree_existed=0 +build_tree_was_soft_link=0 +build_tree_renamed=0 +build_tree_link="" + +if [[ -d "/build/${BOARD}" ]] ; then + build_tree_existed=1 + if [[ -L "/build/${BOARD}" ]] ; then + build_tree_was_soft_link=1 + build_tree_link=`readlink /build/${BOARD}` + sudo rm /build/${BOARD} + else + build_tree_renamed=1 + sudo mv /build/${BOARD} /build/${BOARD}.save + fi +fi + +# Make "working' tree the default board tree (set up soft link) + +sudo ln -s /build/${BOARD}.work /build/${BOARD} + +# +# Create common.sh file, containing appropriate environment variables. +# + +COMMON_FILE="common/common.sh" + +cat <<-EOF > ${COMMON_FILE} + +BISECT_BOARD=${BOARD} +BISECT_REMOTE=${REMOTE} +BISECT_MODE="PACKAGE_MODE" + +GOOD_BUILD=/build/${BOARD}.good +BAD_BUILD=/build/${BOARD}.bad +WORK_BUILD=/build/${BOARD}.work + +EOF + +chmod 755 ${COMMON_FILE} + +# +# Create clean-up script, calling create_cleanup_script.py with +# the appropriate flags. +# + +if [[ ${build_tree_existed} -eq 0 ]] ; then + + python cros_pkg/create_cleanup_script.py --board=${BOARD} \ + --old_tree_missing + +elif [[ ${build_tree_was_soft_link} -eq 0 ]] ; then + + python cros_pkg/create_cleanup_script.py --board=${BOARD} \ + --renamed_tree + +else + + python cros_pkg/create_cleanup_script.py --board=${BOARD} \ + --old_link="'${build_tree_link}'" +fi + +chmod 755 cros_pkg/${BOARD}_cleanup.sh + +exit 0 diff --git a/binary_search_tool/cros_pkg/switch_to_bad.sh b/binary_search_tool/cros_pkg/switch_to_bad.sh new file mode 100755 index 00000000..126425f4 --- /dev/null +++ b/binary_search_tool/cros_pkg/switch_to_bad.sh @@ -0,0 +1,46 @@ +#!/bin/bash -u +# +# Copyright 2015 Google Inc. All Rights Reserved. +# +# This script is intended to be used by binary_search_state.py, as +# part of the binary search triage on ChromeOS packages. This script +# copies a list of packages from the 'bad' build tree into the working +# build tree, for testing. +# + +source common/common.sh + +pushd ${WORK_BUILD} + +PKG_LIST_FILE=$1 + +overall_status=0 + +if [[ -f ${PKG_LIST_FILE} ]] ; then + + # Read every line, and handle case where last line has no newline + while read pkg || [[ -n "$pkg" ]]; + do + sudo cp ${BAD_BUILD}/packages/$pkg ${WORK_BUILD}/packages/$pkg + status=$? + if [[ ${status} -ne 0 ]] ; then + echo "Failed to copy ${pkg} to work build tree." + overall_status=2 + fi + done < ${PKG_LIST_FILE} +else + + for o in "$@" + do + sudo cp ${BAD_BUILD}/packages/$o ${WORK_BUILD}/packages/$o + status=$? + if [[ ${status} -ne 0 ]] ; then + echo "Failed to copy ${pkg} to work build tree." + overall_status=2 + fi + done +fi + +popd + +exit ${overall_status} diff --git a/binary_search_tool/cros_pkg/switch_to_good.sh b/binary_search_tool/cros_pkg/switch_to_good.sh new file mode 100755 index 00000000..a9095e99 --- /dev/null +++ b/binary_search_tool/cros_pkg/switch_to_good.sh @@ -0,0 +1,46 @@ +#!/bin/bash -u +# +# Copyright 2015 Google Inc. All Rights Reserved. +# +# This script is intended to be used by binary_search_state.py, as +# part of the binary search triage on ChromeOS packages. This script +# copies a list of packages from the 'good' build tree into the working +# build tree, for testing. +# + +source common/common.sh + +pushd ${WORK_BUILD} + +PKG_LIST_FILE=$1 + +overall_status=0 + +if [[ -f ${PKG_LIST_FILE} ]] ; then + + # Read every line, and handle case where last line has no newline + while read pkg || [[ -n "$pkg" ]]; + do + sudo cp ${GOOD_BUILD}/packages/$pkg ${WORK_BUILD}/packages/$pkg + status=$? + if [[ ${status} -ne 0 ]] ; then + echo "Failed to copy ${pkg} to work build tree." + overall_status=2 + fi + done < ${PKG_LIST_FILE} +else + + for o in "$@" + do + sudo cp ${GOOD_BUILD}/packages/$o ${WORK_BUILD}/packages/$o + status=$? + if [[ ${status} -ne 0 ]] ; then + echo "Failed to copy ${pkg} to work build tree." + overall_status=2 + fi + done +fi + +popd + +exit ${overall_status} diff --git a/binary_search_tool/cros_pkg/test_setup.sh b/binary_search_tool/cros_pkg/test_setup.sh new file mode 120000 index 00000000..39e715f6 --- /dev/null +++ b/binary_search_tool/cros_pkg/test_setup.sh @@ -0,0 +1 @@ +../common/test_setup.sh
\ No newline at end of file diff --git a/binary_search_tool/ndk/DO_BISECTION.sh b/binary_search_tool/ndk/DO_BISECTION.sh new file mode 100755 index 00000000..298d5747 --- /dev/null +++ b/binary_search_tool/ndk/DO_BISECTION.sh @@ -0,0 +1,92 @@ +#!/bin/bash +# +# Copyright 2016 Google Inc. All Rights Reserved. +# +# This is an example script to show users the steps for bisecting an NDK +# application for Android. Our example is the Teapot app that comes bundled with +# the NDK as a sample app. +# +# Our Teapot app only has 12 or so object files generated per build. Bisection +# for just 12 object files is overkill, but this bisection process easily scales +# to thousands of object files (as seen with the Android source). +# +# Setup: +# 1. Install NDK (make sure it is in your PATH) +# 2. Install compiler_wrapper.py +# 3. Connect an arm7 device (tested with Nexus 5X) +# a. See README for supporting other device archs +# +# Tested in bash on Linux. + +# Set CWD to where this script lives +pushd "$(dirname "$0")" + +# If Teapot dir already exists remove it. +if [[ -d Teapot ]]; then + rm -rf Teapot +fi + +# Unzip our repository we'll be testing with. +tar -xzf Teapot.tar.gz + +# Apply small setup patch. This patch makes a small change to the build system +# to make this bisecting example a little easier. It inserts the option to only +# build for an arm7. See the patch file for details. +# (This patch file was generated with git, -p1 will remove the a/ and b/) +patch -p1 -i PATCH1 + +# We want all of our cached files to be stored in ~/NDK_EXAMPLE_BISECT +# Remove directory if already exists +export BISECT_DIR=~/NDK_EXAMPLE_BISECT +if [[ -d ${BISECT_DIR} ]]; then + rm -rf ${BISECT_DIR} +fi + +# We will now take our normal "good compiler" and do a full build of the app. We +# need to clean before building. This ensures that all objects are generated and +# can be cached. +pushd Teapot +export BISECT_STAGE=POPULATE_GOOD +./gradlew clean +./gradlew installArm7Debug +popd + +# Inserting "compiler error". Really this is just a patch that inserts a simple +# error in the code, but this is used to simulate our compiler error. This patch +# will simply cause the app to crash as soon as it starts. See the patch file +# for details. +# (This patch file was generated with git, -p1 will remove the a/ and b/) +patch -p1 -i PATCH2 + +# Now that we have installed our bad compiler (i.e. applied the above patch that +# acts like a compiler error), we want to enumerate and cache all objects +# generated by this "bad compiler". So again, we clean the build tree so that +# all objects are regenerated and can be cached. +pushd Teapot +export BISECT_STAGE=POPULATE_BAD +./gradlew clean +./gradlew installArm7Debug +popd + +# Now ~/NDK_EXAMPLE_BISECT holds the caches for both good and bad compiler +# outputs. We will now use these to bisect our problem. We should find that +# TeapotRenderer.o is the bad file (because this is where PATCH2 inserted the +# "compiler error"). + +# Tell the compiler wrapper to not cache outputs, and instead begin bisecting. +export BISECT_STAGE=TRIAGE + +# Run the actual bisection tool. This will automatically narrow down which +# object file has the error. The test_setup.sh script will rebuild our app +# with gradle, and boot_test.sh will ping the device to see if the app crashed +# or not. +cd .. +./binary_search_state.py \ + --get_initial_items=ndk/get_initial_items.sh \ + --switch_to_good=ndk/switch_to_good.sh \ + --switch_to_bad=ndk/switch_to_bad.sh \ + --test_setup_script=ndk/test_setup.sh \ + --test_script=ndk/boot_test.sh \ + --file_args + +popd diff --git a/binary_search_tool/ndk/PATCH1 b/binary_search_tool/ndk/PATCH1 new file mode 100644 index 00000000..eddf61cf --- /dev/null +++ b/binary_search_tool/ndk/PATCH1 @@ -0,0 +1,40 @@ +From 93395bf49f856abac5ab06d4bcaa7cdbf76a77fc Mon Sep 17 00:00:00 2001 +From: Cassidy Burden <cburden@google.com> +Date: Tue, 9 Aug 2016 09:38:41 -0700 +Subject: [PATCH] FOR BINARY SEARCH TOOL: Add arm7 target + +Add arm7 target to build.gradle file. This is so the bisection tool only +has to triage the object files generated for our specific device. +Without this target we would have to binary search across object files +meant for x86 targets (that we can't even test on our ARM device). +--- + Teapot/app/build.gradle | 7 ++++++- + 1 file changed, 6 insertions(+), 1 deletion(-) + +diff --git a/Teapot/app/build.gradle b/Teapot/app/build.gradle +index 78cf54d..c322114 100644 +--- a/Teapot/app/build.gradle ++++ b/Teapot/app/build.gradle +@@ -29,7 +29,7 @@ model { + cppFlags.addAll(['-I' + "${ndkDir}/sources/android/cpufeatures", + '-I' + file('src/main/jni/ndk_helper')]) + cppFlags.addAll(['-std=c++11', '-Wall', +- '-fno-exceptions', '-fno-rtti']) ++ '-fno-exceptions', '-fno-rtti', '-gsplit-dwarf']) + ldLibs.addAll(['android', 'log', 'EGL', 'GLESv2','atomic']) + } + sources { +@@ -51,5 +51,10 @@ model { + proguardFiles.add(file('proguard-rules.txt')) + } + } ++ productFlavors{ ++ create("arm7") { ++ ndk.abiFilters.add("armeabi-v7a") ++ } ++ } + } + } +-- +2.8.0.rc3.226.g39d4020 + diff --git a/binary_search_tool/ndk/PATCH2 b/binary_search_tool/ndk/PATCH2 new file mode 100644 index 00000000..9fcf45d0 --- /dev/null +++ b/binary_search_tool/ndk/PATCH2 @@ -0,0 +1,34 @@ +From 960134fb87a194595f2a0a36290be7961e12b946 Mon Sep 17 00:00:00 2001 +From: Cassidy Burden <cburden@google.com> +Date: Tue, 9 Aug 2016 09:46:27 -0700 +Subject: [PATCH] FOR BISECTION TOOL: Insert error + +Insert error into code that will cause crash. This is the "compiler +error" that we will be triaging. We will be pretending the compiler +mistakenly inserted a nullptr where it shouldn't have. + +This error causes the app to immediately crash upon starting. This makes +it very easy to automatically test the app through adb. Not all compiler +problems will be this easy to test, and may require manual testing from +you (the user). See android/interactive_test.sh for an example on +manual testing from the user. +--- + Teapot/app/src/main/jni/TeapotRenderer.cpp | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/Teapot/app/src/main/jni/TeapotRenderer.cpp b/Teapot/app/src/main/jni/TeapotRenderer.cpp +index 7cafdb3..75cadbf 100644 +--- a/Teapot/app/src/main/jni/TeapotRenderer.cpp ++++ b/Teapot/app/src/main/jni/TeapotRenderer.cpp +@@ -58,7 +58,7 @@ void TeapotRenderer::Init() { + num_vertices_ = sizeof(teapotPositions) / sizeof(teapotPositions[0]) / 3; + int32_t stride = sizeof(TEAPOT_VERTEX); + int32_t index = 0; +- TEAPOT_VERTEX* p = new TEAPOT_VERTEX[num_vertices_]; ++ TEAPOT_VERTEX* p = nullptr; //new TEAPOT_VERTEX[num_vertices_]; + for (int32_t i = 0; i < num_vertices_; ++i) { + p[i].pos[0] = teapotPositions[index]; + p[i].pos[1] = teapotPositions[index + 1]; +-- +2.8.0.rc3.226.g39d4020 + diff --git a/binary_search_tool/ndk/README b/binary_search_tool/ndk/README new file mode 100644 index 00000000..324d1391 --- /dev/null +++ b/binary_search_tool/ndk/README @@ -0,0 +1,84 @@ + +This is an example bisection for an NDK build system. This example specifically +bisects the sample NDK Teapot app. All steps (setup and otherwise) for bisection +can be found in DO_BISECTION.sh. This shell script is meant to show the process +required to bisect a compiler problem in an arbitrary NDK app build system. + +There are three necessary setup steps to run this example: + + 1. Install the NDK (known to work with r12b) + a. See here for NDK: https://developer.android.com/ndk/index.html + b. Go here for older NDK downloads: https://github.com/android-ndk/ndk/wiki + + 2. Install the compiler wrapper provided with this repo. See + compiler_wrapper.py for more details. + a. Essentially you must go into the NDK source (or where you build system + stores its toolchain), rename your compilers to <compiler>.real, and + create a symlink pointing to compiler_wrapper.py named <compiler> + (where your compiler used to be). + b. If you're using the toolchains that come with the NDK they live at: + <ndk_path>/toolchains/<arch>/prebuilt/<host>/bin + example: + <ndk_path>/toolchains/llvm/prebuilt/linux-x86_64/bin/clang + + 3. Plug in an Arm7 compatible Android device with usb debugging enabled. + a. This bisection example was tested with a Nexus 5X + b. It should be relatively simple to change the example to work with other + types of devices. Just change the scripts, and change PATCH1 to use a + different build flavor (like x86). See below for more details. + +This example contains two patches: + + PATCH1 - This is the necessary changes to the build system to make the + bisection easier. More specifically, it adds an arm7 build flavor to gradle. + By default, this project will build objects for all possible architectures and + package them into one big apk. These object files meant for another + architecture just sit there and don't actually execute. By adding a build + flavor for arm7, our compiler wrapper won't try to bisect object files meant + for another device. + + PATCH2 - This patch is what inserts the "compiler error". This is a simple + nullptr error in one of the source files, but it is meant to mimic bad code + generation. The result of the error is the app simply crashes immediately + after starting. + +Using another device architecture: + + If we want to bisect for an x86-64 device we first need to provide a arch + specific build flavor in our app/build.gradle file: + + create("x86-64") { + ndk.abiFilters.add("x86_64") + } + + We want to add this under the same "productFlavors" section that our arm7 + build flavor is in (see PATCH1). Now we should have the "installx86-64Debug" + task in our build system. We can use this to build and install an x86-64 + version of our app. + + Now we want to change our test_setup.sh script to run our new gradle task: + ./gradlew installx86-64Debug + + Keep in mind, these specific build flavors are not required. If your build + system makes these device specific builds difficult to implement, the + bisection tool will function perfectly fine without them. However, the + downside of not having targetting a single architecture is the bisection will + take longer (as it will need to search across more object files). + +Additional Documentation: + These are internal Google documents, if you are a developer external to + Google please ask whoever gave you this sample for access or copies to the + documentation. If you cannot gain access, the various READMEs paired with the + bisector should help you. + + * Ahmad's original presentation: + https://goto.google.com/zxdfyi + + * Bisection tool update design doc: + https://goto.google.com/zcwei + + * Bisection tool webpage: + https://goto.google.com/ruwpyi + + * Compiler wrapper webpage: + https://goto.google.com/xossn diff --git a/binary_search_tool/ndk/Teapot.tar.gz b/binary_search_tool/ndk/Teapot.tar.gz Binary files differnew file mode 100644 index 00000000..87faf54b --- /dev/null +++ b/binary_search_tool/ndk/Teapot.tar.gz diff --git a/binary_search_tool/ndk/boot_test.sh b/binary_search_tool/ndk/boot_test.sh new file mode 100755 index 00000000..b8c34aa5 --- /dev/null +++ b/binary_search_tool/ndk/boot_test.sh @@ -0,0 +1,27 @@ +#!/bin/bash -u +# +# Copyright 2016 Google Inc. All Rights Reserved. +# +# This script checks the android device to determine if the app is currently +# running. For our specific test case we will be checking if the Teapot app +# has crashed. +# +# This script is intended to be used by binary_search_state.py, as +# part of the binary search triage on the Android NDK apps. It +# waits for the test setup script to build and install the app, then checks if +# app boots or not. It should return '0' if the test succeeds +# (the image is 'good'); '1' if the test fails (the image is 'bad'); and '125' +# if it could not determine (does not apply in this case). +# + +echo "Starting Teapot app..." +adb shell am start -n com.sample.teapot/com.sample.teapot.TeapotNativeActivity +sleep 3 + +echo "Checking if Teapot app crashed..." +adb shell ps | grep com.sample.teapot + +retval=$? + + +exit ${retval} diff --git a/binary_search_tool/ndk/get_initial_items.sh b/binary_search_tool/ndk/get_initial_items.sh new file mode 100755 index 00000000..bc2d05cd --- /dev/null +++ b/binary_search_tool/ndk/get_initial_items.sh @@ -0,0 +1,12 @@ +#!/bin/bash -u +# +# Copyright 2016 Google Inc. All Rights Reserved. +# +# This script is intended to be used by binary_search_state.py, as +# part of the binary search triage on the Android NDK apps. This script +# generates the list of object files to be bisected. This list is generated +# by the compiler wrapper during the POPULATE_GOOD and POPULATE_BAD stages. +# + +cat ${BISECT_DIR}/good/_LIST + diff --git a/binary_search_tool/ndk/switch_to_bad.sh b/binary_search_tool/ndk/switch_to_bad.sh new file mode 120000 index 00000000..0172bce5 --- /dev/null +++ b/binary_search_tool/ndk/switch_to_bad.sh @@ -0,0 +1 @@ +switch_to_good.sh
\ No newline at end of file diff --git a/binary_search_tool/ndk/switch_to_good.sh b/binary_search_tool/ndk/switch_to_good.sh new file mode 100755 index 00000000..cb8d5fd9 --- /dev/null +++ b/binary_search_tool/ndk/switch_to_good.sh @@ -0,0 +1,46 @@ +#!/bin/bash -u +# +# Copyright 2016 Google Inc. All Rights Reserved. +# +# This script is intended to be used by binary_search_state.py, as +# part of the binary search triage on Android NDK apps. This script simply +# deletes all given objects, signaling gradle to execute a recompilation of said +# object files. +# + +# Input is a file, with newline seperated list of files we will be switching +OBJ_LIST_FILE=$1 + +# Check that number of arguments == 1 +if [ $# -ne 1 ] ; then + echo "ERROR:" + echo "Got multiple inputs to switch script!" + echo "Run binary_search_state.py with --file_args" + exit 1 +fi + +# Remove any file that's being switched. This is because Gradle only recompiles +# if: +# 1. The resultant object file doesn't exist +# 2. The hash of the source file has changed +# +# Because we have no reliable way to edit the source file, we instead remove the +# object file and have the compiler wrapper insert the file from the appropriate +# cache (good or bad). +# +# Not entirely relevant to the Teapot example, but something to consider: +# This removing strategy has the side effect that all switched items cause the +# invocation of the compiler wrapper, which can add up and slow the build +# process. With Android's source tree, Make checks the timestamp of the object +# file. So we symlink in the appropriate file and touch it to tell Make it needs +# to be relinked. This avoids having to call the compiler wrapper in the +# majority of cases. +# +# However, a similar construct doesn't seem to exist in Gradle. It may be better +# to add a build target to Gradle that will always relink all given object +# files. This way we can avoid calling the compiler wrapper while Triaging and +# save some time. Not really necessary + +cat $OBJ_LIST_FILE | xargs rm +exit 0 + diff --git a/binary_search_tool/ndk/test_setup.sh b/binary_search_tool/ndk/test_setup.sh new file mode 100755 index 00000000..477bcb21 --- /dev/null +++ b/binary_search_tool/ndk/test_setup.sh @@ -0,0 +1,27 @@ +#!/bin/bash +# +# Copyright 2016 Google Inc. All Rights Reserved. +# +# This is the setup script for generating and installing the ndk app. +# +# This script is intended to be used by binary_search_state.py, as +# part of the binary search triage on the Android source tree. It should +# return '0' if the setup succeeds; and '1' if the setup fails (the image +# could not build or be flashed). +# + +echo +echo "INSTALLATION BEGIN" +echo + +# This normally shouldn't be hardcoded, but this is a sample bisection. +# Also keep in mind that the bisection tool mandates all paths are +# relative to binary_search_state.py +cd ndk/Teapot + +echo "BUILDING APP" + +./gradlew installArm7Debug +gradle_status=$? + +exit ${gradle_status} diff --git a/binary_search_tool/sysroot_wrapper/README b/binary_search_tool/sysroot_wrapper/README new file mode 100644 index 00000000..599d700d --- /dev/null +++ b/binary_search_tool/sysroot_wrapper/README @@ -0,0 +1,28 @@ +This is a set of scripts to use when triaging compiler problem by using +the bisecting functionality included in the sysroot_wrapper.hardened. +The only script that you need to create for your triaging problem is the +test_script.sh (The ones in this directory are here only as an example). + +Before running the binary searcher tool you will need to run the setup script: + +./sysroot_wrapper/setup.sh ${board} ${remote_ip} ${package} + +This setup script will ensure your $BISECT_DIR is properly populated and +generate a common variable script for the convenience of the scripts in +./sysroot_wrapper + +To run the binary searcher tool with these scripts, execute it like this: + +./binary_search_state.py --get_initial_items=./sysroot_wrapper/get_initial_items.sh --switch_to_good=./sysroot_wrapper/switch_to_good.sh --switch_to_bad=./sysroot_wrapper/switch_to_bad.sh --test_script=./sysroot_wrapper/test_script.sh --noincremental --file_args 2>&1 | tee /tmp/binary_search.log + +Finally once done you will want to run the cleanup script: + +./sysroot_wrapper/cleanup.sh + +For more information on how to use the sysroot_wrapper to do object file +triaging see: + +https://sites.google.com/a/google.com/chromeos-toolchain-team-home2/home/team-tools-and-scripts/bisecting-compiler-problems + + + diff --git a/binary_search_tool/sysroot_wrapper/boot_test.sh b/binary_search_tool/sysroot_wrapper/boot_test.sh new file mode 120000 index 00000000..9a345617 --- /dev/null +++ b/binary_search_tool/sysroot_wrapper/boot_test.sh @@ -0,0 +1 @@ +../common/boot_test.sh
\ No newline at end of file diff --git a/binary_search_tool/sysroot_wrapper/cleanup.sh b/binary_search_tool/sysroot_wrapper/cleanup.sh new file mode 100755 index 00000000..5066d638 --- /dev/null +++ b/binary_search_tool/sysroot_wrapper/cleanup.sh @@ -0,0 +1,11 @@ +#!/bin/bash +# +# Copyright 2016 Google Inc. All Rights Reserved. +# +# This script is part of the ChromeOS object binary search triage process. +# It should be the last script called by the user, after the user has +# successfully run the bisection tool and found their bad items. This script +# will perform all necessary cleanup for the bisection tool. +# + +rm common/common.sh diff --git a/binary_search_tool/sysroot_wrapper/get_initial_items.sh b/binary_search_tool/sysroot_wrapper/get_initial_items.sh new file mode 100755 index 00000000..c1beb972 --- /dev/null +++ b/binary_search_tool/sysroot_wrapper/get_initial_items.sh @@ -0,0 +1,5 @@ +#!/bin/bash -u + +source common/common.sh + +cat ${bisect_dir}/good/_LIST diff --git a/binary_search_tool/sysroot_wrapper/glibc_test_script.sh b/binary_search_tool/sysroot_wrapper/glibc_test_script.sh new file mode 100755 index 00000000..58413ad1 --- /dev/null +++ b/binary_search_tool/sysroot_wrapper/glibc_test_script.sh @@ -0,0 +1,49 @@ +#!/bin/bash -u + +# This is an example execution script. +# This script changes with the problem you are trying to fix. +# This particular script was used to triage a problem where a glibc +# compiled with a new compiler would expose a problem in piglit. +# Note it returns 0 only when the installation of the image succeeded +# (ie: the machine booted after installation) + +source common/common.sh + +#export BISECT_STAGE=TRIAGE +echo "BISECT_STAGE=${BISECT_STAGE}" + +echo "State of sets" +wc -l ${bisect_dir}/*_SET + +board=x86-alex +DUT=172.17.186.180 + +echo "Cleaning up" +{ sudo emerge -C cross-i686-pc-linux-gnu/glibc || exit 125; } &>> /tmp/glibc_triage.log + +echo "Building" +{ sudo -E emerge cross-i686-pc-linux-gnu/glibc || exit 125; } &>> /tmp/glibc_triage.log + +echo "Building image" +{ /home/llozano/trunk/src/scripts/build_image --board=${board} test || exit 125; } &>> /tmp/glibc_triage.log + +echo "Installing image" +cros flash ${DUT} latest &> /tmp/tmp_cros_flash_result.log + +cat /tmp/tmp_cros_flash_result.log >> /tmp/cros_flash_result.log + +grep "Cros Flash completed successfully" /tmp/tmp_cros_flash_result.log || exit 125 + +echo "Trying piglit" + +echo "export DISPLAY=:0.0; echo \$DISPLAY; /usr/local/piglit/lib/piglit/bin/glx-close-display -auto" > /tmp/repro.sh +SSH_OPTS="-oUserKnownHostsFile=/dev/null -oStrictHostKeyChecking=no -oServerAliveInterval=10 -i /var/cache/chromeos-cache/distfiles/target/./chrome-src/src/third_party/chromite/ssh_keys/testing_rsa" +scp ${SSH_OPTS} /tmp/repro.sh root@${DUT}:/tmp + +# notice the bash -l here. Otherwise the DISPLAY cannot be set +( ssh ${SSH_OPTS} root@${DUT} -- /bin/bash -l /tmp/repro.sh ) > /tmp/result +grep pass /tmp/result || { echo "PIGLIT FAILED"; exit 1; } + +echo "PIGLIT PASSED" + +exit 0 diff --git a/binary_search_tool/sysroot_wrapper/interactive_test.sh b/binary_search_tool/sysroot_wrapper/interactive_test.sh new file mode 120000 index 00000000..18fe3958 --- /dev/null +++ b/binary_search_tool/sysroot_wrapper/interactive_test.sh @@ -0,0 +1 @@ +../common/interactive_test.sh
\ No newline at end of file diff --git a/binary_search_tool/sysroot_wrapper/setup.sh b/binary_search_tool/sysroot_wrapper/setup.sh new file mode 100755 index 00000000..f5907f59 --- /dev/null +++ b/binary_search_tool/sysroot_wrapper/setup.sh @@ -0,0 +1,73 @@ +#!/bin/bash -u +# +# Copyright 2016 Google Inc. All Rights Reserved. +# +# This script is part of the ChromeOS object binary search triage process. +# It should be the first script called by the user, after the user has set up +# the two necessary build tree directories (see sysroot_wrapper/README). +# +# This script requires three arguments. The first argument must be the name of +# the board for which this work is being done (e.g. 'daisy', 'lumpy','parrot', +# etc.). The second argument must be the name or IP address of the chromebook +# on which the ChromeOS images will be pushed and tested. The third argument +# must be the name of the package being bisected (e.g. 'chromeos-chrome', +# 'cryptohome', etc.). +# +# This script generates common/common.sh, which generates enviroment variables +# used by the other scripts in the object file binary search triage process. +# + +# Set up basic variables. +bisect_dir=${BISECT_DIR:-/tmp/sysroot_bisect} + +BOARD=$1 +REMOTE=$2 +PACKAGE=$3 + +GOOD_BUILD=${bisect_dir}/good +BAD_BUILD=${bisect_dir}/bad +GOOD_LIST=${GOOD_BUILD}/_LIST +BAD_LIST=${BAD_BUILD}/_LIST + +# +# Verify that the necessary directories exist. +# + +if [[ ! -d ${GOOD_BUILD} ]] ; then + echo "Error: ${GOOD_BUILD} does not exist." + exit 1 +fi + +if [[ ! -d ${BAD_BUILD} ]] ; then + echo "Error: ${BAD_BUILD} does not exist." + exit 1 +fi + +if [[ ! -e ${GOOD_LIST} ]] ; then + echo "Error: ${GOOD_LIST} does not exist." + exit 1 +fi + +if [[ ! -e ${BAD_LIST} ]] ; then + echo "Error: ${BAD_LIST} does not exist." + exit 1 +fi + +COMMON_FILE="common/common.sh" + +cat <<-EOF > ${COMMON_FILE} + +BISECT_BOARD=${BOARD} +BISECT_REMOTE=${REMOTE} +BISECT_PACKAGE=${PACKAGE} +BISECT_MODE="OBJECT_MODE" + +bisect_dir=${bisect_dir} + +export BISECT_STAGE=TRIAGE + +EOF + +chmod 755 ${COMMON_FILE} + +exit 0 diff --git a/binary_search_tool/sysroot_wrapper/switch_to_bad.sh b/binary_search_tool/sysroot_wrapper/switch_to_bad.sh new file mode 100755 index 00000000..32f96780 --- /dev/null +++ b/binary_search_tool/sysroot_wrapper/switch_to_bad.sh @@ -0,0 +1,9 @@ +#!/bin/bash -u + +source common/common.sh + +# Remove file, signaling to emerge that it needs to be rebuilt. The compiler +# wrapper will insert the correct object file based on $BISECT_BAD_SET +cat $1 | sudo xargs rm -f + +exit 0 diff --git a/binary_search_tool/sysroot_wrapper/switch_to_good.sh b/binary_search_tool/sysroot_wrapper/switch_to_good.sh new file mode 100755 index 00000000..f59b278d --- /dev/null +++ b/binary_search_tool/sysroot_wrapper/switch_to_good.sh @@ -0,0 +1,9 @@ +#!/bin/bash -u + +source common/common.sh + +# Remove file, signaling to emerge that it needs to be rebuilt. The compiler +# wrapper will insert the correct object file based on $BISECT_GOOD_SET +cat $1 | sudo xargs rm -f + +exit 0 diff --git a/binary_search_tool/sysroot_wrapper/test_script.sh b/binary_search_tool/sysroot_wrapper/test_script.sh new file mode 100755 index 00000000..2629a187 --- /dev/null +++ b/binary_search_tool/sysroot_wrapper/test_script.sh @@ -0,0 +1,34 @@ +#!/bin/bash -u + +# This is an example execution script. +# This script changes with the problem you are trying to fix. +# This particular script was used to triage a problem where the kernel +# would not boot while migrating to GCC 4.9. +# Note it returns 0 only when the installation of the image succeeded +# (ie: the machine booted after installation) + +source common/common.sh + +export BISECT_STAGE=TRIAGE +echo "BISECT_STAGE=${BISECT_STAGE}" + +echo "State of sets" +wc -l ${bisect_dir}/*_SET + +echo "Cleaning up" +{ /usr/bin/sudo rm -rf /build/falco/var/cache/portage/sys-kernel && emerge-falco -C sys-kernel/chromeos-kernel-3_8-3.8.11-r96 || exit 125; } &>> /tmp/kernel_triage.log + +echo "Building" +{ /usr/local/bin/emerge-falco =sys-kernel/chromeos-kernel-3_8-3.8.11-r96 || exit 125; } &>> /tmp/kernel_triage.log + +echo "Building image" +{ /home/llozano/trunk/src/scripts/build_image --board=falco test || exit 125; } &>> /tmp/kernel_triage.log + +echo "Installing image" +cros flash 172.17.187.150 latest &> /tmp/tmp_cros_flash_result.log + +cat /tmp/tmp_cros_flash_result.log >> /tmp/cros_flash_result.log + +grep "Cros Flash completed successfully" /tmp/tmp_cros_flash_result.log || exit 1 + +exit 0 diff --git a/binary_search_tool/sysroot_wrapper/test_setup.sh b/binary_search_tool/sysroot_wrapper/test_setup.sh new file mode 120000 index 00000000..39e715f6 --- /dev/null +++ b/binary_search_tool/sysroot_wrapper/test_setup.sh @@ -0,0 +1 @@ +../common/test_setup.sh
\ No newline at end of file diff --git a/binary_search_tool/sysroot_wrapper/testing_test.py b/binary_search_tool/sysroot_wrapper/testing_test.py new file mode 100755 index 00000000..2f7bc4c3 --- /dev/null +++ b/binary_search_tool/sysroot_wrapper/testing_test.py @@ -0,0 +1,37 @@ +#!/usr/bin/python2 +"""Test for sysroot_wrapper bisector. + +All files in bad_files will be determined to be bad. This test was made for +chromeos-chrome built for a daisy board, if you are using another package you +will need to change the base_path accordingly. +""" + +from __future__ import print_function + +import subprocess +import sys +import os + +base_path = ('/var/cache/chromeos-chrome/chrome-src-internal/src/out_daisy/' + 'Release/obj/') +bad_files = [ + os.path.join(base_path, 'base/base.cpu.o'), + os.path.join(base_path, 'base/base.version.o'), + os.path.join(base_path, 'apps/apps.launcher.o') +] + +bisect_dir = os.environ.get('BISECT_DIR', '/tmp/sysroot_bisect') + + +def Main(_): + for test_file in bad_files: + test_file = test_file.strip() + cmd = ['grep', test_file, os.path.join(bisect_dir, 'BAD_SET')] + ret = subprocess.call(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE) + if not ret: + return 1 + return 0 + + +if __name__ == '__main__': + sys.exit(Main(sys.argv[1:])) diff --git a/binary_search_tool/test/__init__.py b/binary_search_tool/test/__init__.py new file mode 100644 index 00000000..8b137891 --- /dev/null +++ b/binary_search_tool/test/__init__.py @@ -0,0 +1 @@ + diff --git a/binary_search_tool/test/binary_search_tool_tester.py b/binary_search_tool/test/binary_search_tool_tester.py new file mode 100755 index 00000000..775c1715 --- /dev/null +++ b/binary_search_tool/test/binary_search_tool_tester.py @@ -0,0 +1,421 @@ +#!/usr/bin/python2 + +# Copyright 2012 Google Inc. All Rights Reserved. +"""Tests for bisecting tool.""" + +from __future__ import print_function + +__author__ = 'shenhan@google.com (Han Shen)' + +import os +import random +import sys +import unittest + +from cros_utils import command_executer +from binary_search_tool import binary_search_state +from binary_search_tool import bisect + +import common +import gen_obj + + +def GenObj(): + obj_num = random.randint(100, 1000) + bad_obj_num = random.randint(obj_num / 100, obj_num / 20) + if bad_obj_num == 0: + bad_obj_num = 1 + gen_obj.Main(['--obj_num', str(obj_num), '--bad_obj_num', str(bad_obj_num)]) + + +def CleanObj(): + os.remove(common.OBJECTS_FILE) + os.remove(common.WORKING_SET_FILE) + print('Deleted "{0}" and "{1}"'.format(common.OBJECTS_FILE, + common.WORKING_SET_FILE)) + + +class BisectTest(unittest.TestCase): + """Tests for bisect.py""" + + def setUp(self): + with open('./is_setup', 'w'): + pass + + try: + os.remove(binary_search_state.STATE_FILE) + except OSError: + pass + + def tearDown(self): + try: + os.remove('./is_setup') + os.remove(os.readlink(binary_search_state.STATE_FILE)) + os.remove(binary_search_state.STATE_FILE) + except OSError: + pass + + class FullBisector(bisect.Bisector): + """Test bisector to test bisect.py with""" + + def __init__(self, options, overrides): + super(BisectTest.FullBisector, self).__init__(options, overrides) + + def PreRun(self): + GenObj() + return 0 + + def Run(self): + return binary_search_state.Run(get_initial_items='./gen_init_list.py', + switch_to_good='./switch_to_good.py', + switch_to_bad='./switch_to_bad.py', + test_script='./is_good.py', + prune=True, + file_args=True) + + def PostRun(self): + CleanObj() + return 0 + + def test_full_bisector(self): + ret = bisect.Run(self.FullBisector({}, {})) + self.assertEquals(ret, 0) + self.assertFalse(os.path.exists(common.OBJECTS_FILE)) + self.assertFalse(os.path.exists(common.WORKING_SET_FILE)) + + def check_output(self): + _, out, _ = command_executer.GetCommandExecuter().RunCommandWOutput( + ('grep "Bad items are: " logs/binary_search_tool_tester.py.out | ' + 'tail -n1')) + ls = out.splitlines() + self.assertEqual(len(ls), 1) + line = ls[0] + + _, _, bad_ones = line.partition('Bad items are: ') + bad_ones = bad_ones.split() + expected_result = common.ReadObjectsFile() + + # Reconstruct objects file from bad_ones and compare + actual_result = [0] * len(expected_result) + for bad_obj in bad_ones: + actual_result[int(bad_obj)] = 1 + + self.assertEqual(actual_result, expected_result) + + +class BisectingUtilsTest(unittest.TestCase): + """Tests for bisecting tool.""" + + def setUp(self): + """Generate [100-1000] object files, and 1-5% of which are bad ones.""" + GenObj() + + with open('./is_setup', 'w'): + pass + + try: + os.remove(binary_search_state.STATE_FILE) + except OSError: + pass + + def tearDown(self): + """Cleanup temp files.""" + CleanObj() + + try: + os.remove(os.readlink(binary_search_state.STATE_FILE)) + except OSError: + pass + + cleanup_list = ['./is_setup', binary_search_state.STATE_FILE, + 'noinc_prune_bad', 'noinc_prune_good'] + for f in cleanup_list: + if os.path.exists(f): + os.remove(f) + + def runTest(self): + ret = binary_search_state.Run(get_initial_items='./gen_init_list.py', + switch_to_good='./switch_to_good.py', + switch_to_bad='./switch_to_bad.py', + test_script='./is_good.py', + prune=True, + file_args=True) + self.assertEquals(ret, 0) + self.check_output() + + def test_arg_parse(self): + args = ['--get_initial_items', './gen_init_list.py', '--switch_to_good', + './switch_to_good.py', '--switch_to_bad', './switch_to_bad.py', + '--test_script', './is_good.py', '--prune', '--file_args'] + ret = binary_search_state.Main(args) + self.assertEquals(ret, 0) + self.check_output() + + def test_test_setup_script(self): + os.remove('./is_setup') + with self.assertRaises(AssertionError): + ret = binary_search_state.Run(get_initial_items='./gen_init_list.py', + switch_to_good='./switch_to_good.py', + switch_to_bad='./switch_to_bad.py', + test_script='./is_good.py', + prune=True, + file_args=True) + + ret = binary_search_state.Run(get_initial_items='./gen_init_list.py', + switch_to_good='./switch_to_good.py', + switch_to_bad='./switch_to_bad.py', + test_script='./is_good.py', + test_setup_script='./test_setup.py', + prune=True, + file_args=True) + self.assertEquals(ret, 0) + self.check_output() + + def test_bad_test_setup_script(self): + with self.assertRaises(AssertionError): + binary_search_state.Run(get_initial_items='./gen_init_list.py', + switch_to_good='./switch_to_good.py', + switch_to_bad='./switch_to_bad.py', + test_script='./is_good.py', + test_setup_script='./test_setup_bad.py', + prune=True, + file_args=True) + + def test_bad_save_state(self): + state_file = binary_search_state.STATE_FILE + hidden_state_file = os.path.basename(binary_search_state.HIDDEN_STATE_FILE) + + with open(state_file, 'w') as f: + f.write('test123') + + bss = binary_search_state.MockBinarySearchState() + with self.assertRaises(binary_search_state.Error): + bss.SaveState() + + with open(state_file, 'r') as f: + self.assertEquals(f.read(), 'test123') + + os.remove(state_file) + + # Cleanup generated save state that has no symlink + files = os.listdir(os.getcwd()) + save_states = [x for x in files if x.startswith(hidden_state_file)] + _ = [os.remove(x) for x in save_states] + + def test_save_state(self): + state_file = binary_search_state.STATE_FILE + + bss = binary_search_state.MockBinarySearchState() + bss.SaveState() + self.assertTrue(os.path.exists(state_file)) + first_state = os.readlink(state_file) + + bss.SaveState() + second_state = os.readlink(state_file) + self.assertTrue(os.path.exists(state_file)) + self.assertTrue(second_state != first_state) + self.assertFalse(os.path.exists(first_state)) + + bss.RemoveState() + self.assertFalse(os.path.islink(state_file)) + self.assertFalse(os.path.exists(second_state)) + + def test_load_state(self): + test_items = [1, 2, 3, 4, 5] + + bss = binary_search_state.MockBinarySearchState() + bss.all_items = test_items + bss.currently_good_items = set([1, 2, 3]) + bss.currently_bad_items = set([4, 5]) + bss.SaveState() + + bss = None + + bss2 = binary_search_state.MockBinarySearchState.LoadState() + self.assertEquals(bss2.all_items, test_items) + self.assertEquals(bss2.currently_good_items, set([])) + self.assertEquals(bss2.currently_bad_items, set([])) + + def test_tmp_cleanup(self): + bss = binary_search_state.MockBinarySearchState( + get_initial_items='echo "0\n1\n2\n3"', + switch_to_good='./switch_tmp.py', + file_args=True) + bss.SwitchToGood(['0', '1', '2', '3']) + + tmp_file = None + with open('tmp_file', 'r') as f: + tmp_file = f.read() + os.remove('tmp_file') + + self.assertFalse(os.path.exists(tmp_file)) + ws = common.ReadWorkingSet() + for i in range(3): + self.assertEquals(ws[i], 42) + + def test_verify_fail(self): + bss = binary_search_state.MockBinarySearchState( + get_initial_items='./gen_init_list.py', + switch_to_good='./switch_to_bad.py', + switch_to_bad='./switch_to_good.py', + test_script='./is_good.py', + prune=True, + file_args=True, + verify=True) + with self.assertRaises(AssertionError): + bss.DoVerify() + + def test_early_terminate(self): + bss = binary_search_state.MockBinarySearchState( + get_initial_items='./gen_init_list.py', + switch_to_good='./switch_to_good.py', + switch_to_bad='./switch_to_bad.py', + test_script='./is_good.py', + prune=True, + file_args=True, + iterations=1) + bss.DoSearch() + self.assertFalse(bss.found_items) + + def test_no_prune(self): + bss = binary_search_state.MockBinarySearchState( + get_initial_items='./gen_init_list.py', + switch_to_good='./switch_to_good.py', + switch_to_bad='./switch_to_bad.py', + test_script='./is_good.py', + test_setup_script='./test_setup.py', + prune=False, + file_args=True) + bss.DoSearch() + self.assertEquals(len(bss.found_items), 1) + + bad_objs = common.ReadObjectsFile() + found_obj = int(bss.found_items.pop()) + self.assertEquals(bad_objs[found_obj], 1) + + def test_set_file(self): + binary_search_state.Run(get_initial_items='./gen_init_list.py', + switch_to_good='./switch_to_good_set_file.py', + switch_to_bad='./switch_to_bad_set_file.py', + test_script='./is_good.py', + prune=True, + file_args=True, + verify=True) + self.check_output() + + def test_noincremental_prune(self): + ret = binary_search_state.Run( + get_initial_items='./gen_init_list.py', + switch_to_good='./switch_to_good_noinc_prune.py', + switch_to_bad='./switch_to_bad_noinc_prune.py', + test_script='./is_good_noinc_prune.py', + test_setup_script='./test_setup.py', + prune=True, + noincremental=True, + file_args=True, + verify=False) + self.assertEquals(ret, 0) + self.check_output() + + def check_output(self): + _, out, _ = command_executer.GetCommandExecuter().RunCommandWOutput( + ('grep "Bad items are: " logs/binary_search_tool_tester.py.out | ' + 'tail -n1')) + ls = out.splitlines() + self.assertEqual(len(ls), 1) + line = ls[0] + + _, _, bad_ones = line.partition('Bad items are: ') + bad_ones = bad_ones.split() + expected_result = common.ReadObjectsFile() + + # Reconstruct objects file from bad_ones and compare + actual_result = [0] * len(expected_result) + for bad_obj in bad_ones: + actual_result[int(bad_obj)] = 1 + + self.assertEqual(actual_result, expected_result) + + +class BisectStressTest(unittest.TestCase): + """Stress tests for bisecting tool.""" + + def test_every_obj_bad(self): + amt = 25 + gen_obj.Main(['--obj_num', str(amt), '--bad_obj_num', str(amt)]) + ret = binary_search_state.Run(get_initial_items='./gen_init_list.py', + switch_to_good='./switch_to_good.py', + switch_to_bad='./switch_to_bad.py', + test_script='./is_good.py', + prune=True, + file_args=True, + verify=False) + self.assertEquals(ret, 0) + self.check_output() + + def test_every_index_is_bad(self): + amt = 25 + for i in range(amt): + obj_list = ['0'] * amt + obj_list[i] = '1' + obj_list = ','.join(obj_list) + gen_obj.Main(['--obj_list', obj_list]) + ret = binary_search_state.Run(get_initial_items='./gen_init_list.py', + switch_to_good='./switch_to_good.py', + switch_to_bad='./switch_to_bad.py', + test_setup_script='./test_setup.py', + test_script='./is_good.py', + prune=True, + file_args=True) + self.assertEquals(ret, 0) + self.check_output() + + def check_output(self): + _, out, _ = command_executer.GetCommandExecuter().RunCommandWOutput( + ('grep "Bad items are: " logs/binary_search_tool_tester.py.out | ' + 'tail -n1')) + ls = out.splitlines() + self.assertEqual(len(ls), 1) + line = ls[0] + + _, _, bad_ones = line.partition('Bad items are: ') + bad_ones = bad_ones.split() + expected_result = common.ReadObjectsFile() + + # Reconstruct objects file from bad_ones and compare + actual_result = [0] * len(expected_result) + for bad_obj in bad_ones: + actual_result[int(bad_obj)] = 1 + + self.assertEqual(actual_result, expected_result) + + +def Main(argv): + num_tests = 2 + if len(argv) > 1: + num_tests = int(argv[1]) + + suite = unittest.TestSuite() + for _ in range(0, num_tests): + suite.addTest(BisectingUtilsTest()) + suite.addTest(BisectingUtilsTest('test_arg_parse')) + suite.addTest(BisectingUtilsTest('test_test_setup_script')) + suite.addTest(BisectingUtilsTest('test_bad_test_setup_script')) + suite.addTest(BisectingUtilsTest('test_bad_save_state')) + suite.addTest(BisectingUtilsTest('test_save_state')) + suite.addTest(BisectingUtilsTest('test_load_state')) + suite.addTest(BisectingUtilsTest('test_tmp_cleanup')) + suite.addTest(BisectingUtilsTest('test_verify_fail')) + suite.addTest(BisectingUtilsTest('test_early_terminate')) + suite.addTest(BisectingUtilsTest('test_no_prune')) + suite.addTest(BisectingUtilsTest('test_set_file')) + suite.addTest(BisectingUtilsTest('test_noincremental_prune')) + suite.addTest(BisectTest('test_full_bisector')) + suite.addTest(BisectStressTest('test_every_obj_bad')) + suite.addTest(BisectStressTest('test_every_index_is_bad')) + runner = unittest.TextTestRunner() + runner.run(suite) + + +if __name__ == '__main__': + Main(sys.argv) diff --git a/binary_search_tool/test/common.py b/binary_search_tool/test/common.py new file mode 100755 index 00000000..baac9434 --- /dev/null +++ b/binary_search_tool/test/common.py @@ -0,0 +1,41 @@ +#!/usr/bin/python2 +"""Common utility functions.""" + +DEFAULT_OBJECT_NUMBER = 1238 +DEFAULT_BAD_OBJECT_NUMBER = 23 +OBJECTS_FILE = 'objects.txt' +WORKING_SET_FILE = 'working_set.txt' + + +def ReadWorkingSet(): + working_set = [] + f = open(WORKING_SET_FILE, 'r') + for l in f: + working_set.append(int(l)) + f.close() + return working_set + + +def WriteWorkingSet(working_set): + f = open(WORKING_SET_FILE, 'w') + for o in working_set: + f.write('{0}\n'.format(o)) + f.close() + + +def ReadObjectsFile(): + objects_file = [] + f = open(OBJECTS_FILE, 'r') + for l in f: + objects_file.append(int(l)) + f.close() + return objects_file + + +def ReadObjectIndex(filename): + object_index = [] + f = open(filename, 'r') + for o in f: + object_index.append(int(o)) + f.close() + return object_index diff --git a/binary_search_tool/test/gen_init_list.py b/binary_search_tool/test/gen_init_list.py new file mode 100755 index 00000000..4a79a1b1 --- /dev/null +++ b/binary_search_tool/test/gen_init_list.py @@ -0,0 +1,22 @@ +#!/usr/bin/python2 +"""Prints out index for every object file, starting from 0.""" + +from __future__ import print_function + +import sys + +from cros_utils import command_executer +import common + + +def Main(): + ce = command_executer.GetCommandExecuter() + _, l, _ = ce.RunCommandWOutput( + 'cat {0} | wc -l'.format(common.OBJECTS_FILE), print_to_console=False) + for i in range(0, int(l)): + print(i) + + +if __name__ == '__main__': + Main() + sys.exit(0) diff --git a/binary_search_tool/test/gen_obj.py b/binary_search_tool/test/gen_obj.py new file mode 100755 index 00000000..265729d2 --- /dev/null +++ b/binary_search_tool/test/gen_obj.py @@ -0,0 +1,97 @@ +#!/usr/bin/python2 +"""Script to generate a list of object files. + +0 represents a good object file. +1 represents a bad object file. +""" + +from __future__ import print_function + +import argparse +import os +import random +import sys + +import common + + +def Main(argv): + """Generates a list, the value of each element is 0 or 1. + + The number of 1s in the list is specified by bad_obj_num. + The others are all 0s. The total number of 0s and 1s is specified by obj_num. + + Args: + argv: argument from command line + + Returns: + 0 always. + """ + parser = argparse.ArgumentParser() + parser.add_argument( + '-n', + '--obj_num', + dest='obj_num', + default=common.DEFAULT_OBJECT_NUMBER, + help=('Number of total objects.')) + parser.add_argument( + '-b', + '--bad_obj_num', + dest='bad_obj_num', + default=common.DEFAULT_BAD_OBJECT_NUMBER, + help=('Number of bad objects. Must be great than or ' + 'equal to zero and less than total object ' + 'number.')) + parser.add_argument( + '-o', + '--obj_list', + dest='obj_list', + default='', + help=('List of comma seperated objects to generate. ' + 'A 0 means the object is good, a 1 means the ' + 'object is bad.')) + options = parser.parse_args(argv) + + obj_num = int(options.obj_num) + bad_obj_num = int(options.bad_obj_num) + bad_to_gen = int(options.bad_obj_num) + obj_list = options.obj_list + if not obj_list: + obj_list = [] + for i in range(obj_num): + if bad_to_gen > 0 and random.randint(1, obj_num) <= bad_obj_num: + obj_list.append(1) + bad_to_gen -= 1 + else: + obj_list.append(0) + while bad_to_gen > 0: + t = random.randint(0, obj_num - 1) + if obj_list[t] == 0: + obj_list[t] = 1 + bad_to_gen -= 1 + else: + obj_list = obj_list.split(',') + + if os.path.isfile(common.OBJECTS_FILE): + os.remove(common.OBJECTS_FILE) + if os.path.isfile(common.WORKING_SET_FILE): + os.remove(common.WORKING_SET_FILE) + + f = open(common.OBJECTS_FILE, 'w') + w = open(common.WORKING_SET_FILE, 'w') + for i in obj_list: + f.write('{0}\n'.format(i)) + w.write('{0}\n'.format(i)) + f.close() + + obj_num = len(obj_list) + bad_obj_num = obj_list.count('1') + print('Generated {0} object files, with {1} bad ones.'.format(obj_num, + bad_obj_num)) + + return 0 + + +if __name__ == '__main__': + retval = Main(sys.argv[1:]) + sys.exit(retval) diff --git a/binary_search_tool/test/is_good.py b/binary_search_tool/test/is_good.py new file mode 100755 index 00000000..bfe9cc32 --- /dev/null +++ b/binary_search_tool/test/is_good.py @@ -0,0 +1,24 @@ +#!/usr/bin/python2 +"""Check to see if the working set produces a good executable.""" + +from __future__ import print_function + +import os +import sys + +import common + + +def Main(): + if not os.path.exists('./is_setup'): + return 1 + working_set = common.ReadWorkingSet() + for w in working_set: + if w == 1: + return 1 ## False, linking failure + return 0 + + +if __name__ == '__main__': + retval = Main() + sys.exit(retval) diff --git a/binary_search_tool/test/is_good_noinc_prune.py b/binary_search_tool/test/is_good_noinc_prune.py new file mode 100755 index 00000000..5aafd6c2 --- /dev/null +++ b/binary_search_tool/test/is_good_noinc_prune.py @@ -0,0 +1,46 @@ +#!/usr/bin/python2 +"""Check to see if the working set produces a good executable. + +This test script is made for the noincremental-prune test. This makes sure +that, after pruning starts (>1 bad item is found), that the number of args sent +to the switch scripts is equals to the actual number of items (i.e. checking +that noincremental always holds). +""" + +from __future__ import print_function + +import os +import sys + +import common + + +def Main(): + working_set = common.ReadWorkingSet() + + with open('noinc_prune_good', 'r') as good_args: + num_good_args = len(good_args.readlines()) + + with open('noinc_prune_bad', 'r') as bad_args: + num_bad_args = len(bad_args.readlines()) + + num_args = num_good_args + num_bad_args + if num_args != len(working_set): + print('Only %d args, expected %d' % (num_args, len(working_set))) + print('%d good args, %d bad args' % (num_good_args, num_bad_args)) + return 3 + + os.remove('noinc_prune_bad') + os.remove('noinc_prune_good') + + if not os.path.exists('./is_setup'): + return 1 + for w in working_set: + if w == 1: + return 1 ## False, linking failure + return 0 + + +if __name__ == '__main__': + retval = Main() + sys.exit(retval) diff --git a/binary_search_tool/test/switch_tmp.py b/binary_search_tool/test/switch_tmp.py new file mode 100755 index 00000000..165004ed --- /dev/null +++ b/binary_search_tool/test/switch_tmp.py @@ -0,0 +1,34 @@ +#!/usr/bin/python2 +"""Change portions of the object files to good. + +This file is a test switch script. Used only for the test test_tmp_cleanup. +The "portion" is defined by the file (which is passed as the only argument to +this script) content. Every line in the file is an object index, which will be +set to good (mark as 42). +""" + +from __future__ import print_function + +import sys + +import common + + +def Main(argv): + working_set = common.ReadWorkingSet() + object_index = common.ReadObjectIndex(argv[1]) + + # Random number so the results can be checked + for oi in object_index: + working_set[int(oi)] = 42 + + common.WriteWorkingSet(working_set) + with open('tmp_file', 'w') as f: + f.write(argv[1]) + + return 0 + + +if __name__ == '__main__': + retval = Main(sys.argv) + sys.exit(retval) diff --git a/binary_search_tool/test/switch_to_bad.py b/binary_search_tool/test/switch_to_bad.py new file mode 100755 index 00000000..b8602421 --- /dev/null +++ b/binary_search_tool/test/switch_to_bad.py @@ -0,0 +1,27 @@ +#!/usr/bin/python2 +"""Switch part of the objects file in working set to (possible) bad ones.""" + +from __future__ import print_function + +import sys + +import common + + +def Main(argv): + """Switch part of the objects file in working set to (possible) bad ones.""" + working_set = common.ReadWorkingSet() + objects_file = common.ReadObjectsFile() + object_index = common.ReadObjectIndex(argv[1]) + + for oi in object_index: + working_set[oi] = objects_file[oi] + + common.WriteWorkingSet(working_set) + + return 0 + + +if __name__ == '__main__': + retval = Main(sys.argv) + sys.exit(retval) diff --git a/binary_search_tool/test/switch_to_bad_noinc_prune.py b/binary_search_tool/test/switch_to_bad_noinc_prune.py new file mode 100755 index 00000000..87bf1584 --- /dev/null +++ b/binary_search_tool/test/switch_to_bad_noinc_prune.py @@ -0,0 +1,42 @@ +#!/usr/bin/python2 +"""Switch part of the objects file in working set to (possible) bad ones. + +The "portion" is defined by the file (which is passed as the only argument to +this script) content. Every line in the file is an object index, which will be +set to good (mark as 0). + +This switch script is made for the noincremental-prune test. This makes sure +that, after pruning starts (>1 bad item is found), that the number of args sent +to the switch scripts is equals to the actual number of items (i.e. checking +that noincremental always holds). + +Warning: This switch script assumes the --file_args option +""" + +from __future__ import print_function + +import shutil +import sys + +import common + + +def Main(argv): + """Switch part of the objects file in working set to (possible) bad ones.""" + working_set = common.ReadWorkingSet() + objects_file = common.ReadObjectsFile() + object_index = common.ReadObjectIndex(argv[1]) + + for oi in object_index: + working_set[oi] = objects_file[oi] + + shutil.copy(argv[1], './noinc_prune_bad') + + common.WriteWorkingSet(working_set) + + return 0 + + +if __name__ == '__main__': + retval = Main(sys.argv) + sys.exit(retval) diff --git a/binary_search_tool/test/switch_to_bad_set_file.py b/binary_search_tool/test/switch_to_bad_set_file.py new file mode 100755 index 00000000..f535fdfd --- /dev/null +++ b/binary_search_tool/test/switch_to_bad_set_file.py @@ -0,0 +1,37 @@ +#!/usr/bin/python2 +"""Switch part of the objects file in working set to (possible) bad ones. + +This script is meant to be specifically used with the set_file test. This uses +the set files generated by binary_search_state to do the switching. +""" + +from __future__ import print_function + +import os +import sys + +import common + + +def Main(_): + """Switch part of the objects file in working set to (possible) bad ones.""" + working_set = common.ReadWorkingSet() + objects_file = common.ReadObjectsFile() + + if not os.path.exists(os.environ['BISECT_BAD_SET']): + print('Bad set file does not exist!') + return 1 + + object_index = common.ReadObjectIndex(os.environ['BISECT_BAD_SET']) + + for oi in object_index: + working_set[int(oi)] = objects_file[oi] + + common.WriteWorkingSet(working_set) + + return 0 + + +if __name__ == '__main__': + retval = Main(sys.argv) + sys.exit(retval) diff --git a/binary_search_tool/test/switch_to_good.py b/binary_search_tool/test/switch_to_good.py new file mode 100755 index 00000000..68e9633f --- /dev/null +++ b/binary_search_tool/test/switch_to_good.py @@ -0,0 +1,30 @@ +#!/usr/bin/python2 +"""Change portions of the object files to good. + +The "portion" is defined by the file (which is passed as the only argument to +this script) content. Every line in the file is an object index, which will be +set to good (mark as 0). +""" + +from __future__ import print_function + +import sys + +import common + + +def Main(argv): + working_set = common.ReadWorkingSet() + object_index = common.ReadObjectIndex(argv[1]) + + for oi in object_index: + working_set[int(oi)] = 0 + + common.WriteWorkingSet(working_set) + + return 0 + + +if __name__ == '__main__': + retval = Main(sys.argv) + sys.exit(retval) diff --git a/binary_search_tool/test/switch_to_good_noinc_prune.py b/binary_search_tool/test/switch_to_good_noinc_prune.py new file mode 100755 index 00000000..c5e78e45 --- /dev/null +++ b/binary_search_tool/test/switch_to_good_noinc_prune.py @@ -0,0 +1,40 @@ +#!/usr/bin/python2 +"""Change portions of the object files to good. + +The "portion" is defined by the file (which is passed as the only argument to +this script) content. Every line in the file is an object index, which will be +set to good (mark as 0). + +This switch script is made for the noincremental-prune test. This makes sure +that, after pruning starts (>1 bad item is found), that the number of args sent +to the switch scripts is equals to the actual number of items (i.e. checking +that noincremental always holds). + +Warning: This switch script assumes the --file_args option +""" + +from __future__ import print_function + +import shutil +import sys + +import common + + +def Main(argv): + working_set = common.ReadWorkingSet() + object_index = common.ReadObjectIndex(argv[1]) + + for oi in object_index: + working_set[int(oi)] = 0 + + shutil.copy(argv[1], './noinc_prune_good') + + common.WriteWorkingSet(working_set) + + return 0 + + +if __name__ == '__main__': + retval = Main(sys.argv) + sys.exit(retval) diff --git a/binary_search_tool/test/switch_to_good_set_file.py b/binary_search_tool/test/switch_to_good_set_file.py new file mode 100755 index 00000000..83777af0 --- /dev/null +++ b/binary_search_tool/test/switch_to_good_set_file.py @@ -0,0 +1,39 @@ +#!/usr/bin/python2 +"""Change portions of the object files to good. + +The "portion" is defined by the file (which is passed as the only argument to +this script) content. Every line in the file is an object index, which will be +set to good (mark as 0). + +This script is meant to be specifically used with the set_file test. This uses +the set files generated by binary_search_state to do the switching. +""" + +from __future__ import print_function + +import os +import sys + +import common + + +def Main(_): + working_set = common.ReadWorkingSet() + + if not os.path.exists(os.environ['BISECT_GOOD_SET']): + print('Good set file does not exist!') + return 1 + + object_index = common.ReadObjectIndex(os.environ['BISECT_GOOD_SET']) + + for oi in object_index: + working_set[int(oi)] = 0 + + common.WriteWorkingSet(working_set) + + return 0 + + +if __name__ == '__main__': + retval = Main(sys.argv) + sys.exit(retval) diff --git a/binary_search_tool/test/test_setup.py b/binary_search_tool/test/test_setup.py new file mode 100755 index 00000000..3fb5a23c --- /dev/null +++ b/binary_search_tool/test/test_setup.py @@ -0,0 +1,19 @@ +#!/usr/bin/python2 +"""Emulate running of test setup script, is_good.py should fail without this.""" + +from __future__ import print_function + +import sys + + +def Main(): + # create ./is_setup + with open('./is_setup', 'w'): + pass + + return 0 + + +if __name__ == '__main__': + retval = Main() + sys.exit(retval) diff --git a/binary_search_tool/test/test_setup_bad.py b/binary_search_tool/test/test_setup_bad.py new file mode 100755 index 00000000..8d72763e --- /dev/null +++ b/binary_search_tool/test/test_setup_bad.py @@ -0,0 +1,15 @@ +#!/usr/bin/python2 +"""Emulate test setup that fails (i.e. failed flash to device)""" + +from __future__ import print_function + +import sys + + +def Main(): + return 1 ## False, flashing failure + + +if __name__ == '__main__': + retval = Main() + sys.exit(retval) |