binary search tool: Add android bisector documentation

Change-Id: I8f9c954accdac769d2a696f31fb1d90d50264eaa
Reviewed-on: https://chrome-internal-review.googlesource.com/272011
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>

2 files changed, 257 insertions, 0 deletions

diff --git a/binary_search_tool/README.bisect b/binary_search_tool/README.bisect
index 5e0ecc76..19bcb74b 100644
--- a/binary_search_tool/README.bisect
+++ b/binary_search_tool/README.bisect
@@ -115,6 +115,77 @@ Bisection Methods:
         ./bisect.py object daisy 172.17.211.184 cryptohome \
         --test_script=sysroot_wrapper/boot_test.sh
 
+3) Android object:
+  NOTE: Because this isn't a ChromeOS bisection tool, the concept of a
+        chroot doesn't exist. Just run this tool from a normal shell.
+
+  This method will bisect across all objects in the Android source tree and
+  find the offending objects (according to your test script). This method takes
+  the following arguments:
+
+    android_src: The location of your android source tree
+    num_jobs: (Optional) The number of jobs to pass to make. This is dependent
+              on how many cores your machine has. A good number is probably
+	      somewhere around 5 to 10.
+    device_id: (Optional) The serial code for the device you are testing on.
+               This is used to determine which device should be used in case
+	       multiple devices are plugged into your computer. You can get
+	       serial code for your device by running "adb devices".
+    dir: (Optional) the directory for your good/bad build trees. Defaults to
+         $BISECT_DIR or ~/ANDROID_BISECT/. This value will set $BISECT_DIR
+         for all bisecting scripts.
+
+  By default the Android object method will do a simple interactive test that
+  pings the machine and prompts the user if the machine is good.
+
+  a) Setup:
+    The Android object method requires that you populate your good and bad set
+    of objects. The Android compiler wrapper will automatically detect the
+    BISECT_STAGE variable and use this to populate emerged objects. Here is an
+    example:
+
+      # Defaults to ~/ANDROID_BISECT/
+      export BISECT_DIR="/path/to/where/you/want/to/store/builds/"
+
+      export BISECT_STAGE="POPULATE_GOOD"
+      # Install the "good" compiler
+      ./switch_to_good_compiler.sh
+      make clean
+      make -j <your_preferred_number_of_jobs>
+
+      export BISECT_STAGE="POPULATE_BAD"
+      # Install the "bad" compiler
+      ./switch_to_bad_compiler.sh
+      make clean
+      make -j <your_preferred_number_of_jobs>
+
+  b) Cleanup:
+    The user must clean up all built images and the populated object files.
+
+  c) Default Arguments:
+    --get_initial_items='android/get_initial_items.sh'
+    --switch_to_good='android/switch_to_good.sh'
+    --switch_to_bad='android/switch_to_bad.sh'
+    --install_script='android/install.sh'
+    --test_script='android/interactive_test.sh'
+    --incremental
+    --prune
+    --file_args
+
+  d) Additional Documentation:
+    See ./android/README.android for full documentation of Android object file
+    bisecting.
+
+  e) Examples:
+    i)  Basic interactive test android bisection, where the android source is
+        at ~/android_src:
+        ./bisect.py android ~/android_src
+
+    ii) Basic boot test android bisection, where the android source is at
+        ~/android_src, and 10 jobs will be used to build android:
+        ./bisect.py android ~/android_src --num_jobs=10 \
+        --test_script=sysroot_wrapper/boot_test.sh
+
 Resuming:
   bisect.py and binary_search_state.py offer the ability to resume a bisection
   in case it was interrupted by a SIGINT, power failure, etc. Every time the
diff --git a/binary_search_tool/android/README.android b/binary_search_tool/android/README.android
new file mode 100644
index 00000000..04402322
--- /dev/null
+++ b/binary_search_tool/android/README.android
@@ -0,0 +1,186 @@
+
+binary_search_state.py is a general binary search triage tool that
+performs a binary search on a set of things to try to identify which
+thing or thing(s) in the set is 'bad'.  binary_search_state.py assumes
+that the user has two sets, one where everything is known to be good,
+and one which contains at least one bad item.  binary_search_state.py
+then copies items from the good and bad sets into a working set and
+tests the result (good or bad).  binary_search_state.py requires that
+a set of scripts be supplied to it for any particular job.  For more
+information on binary_search_state.py, see
+
+https://sites.google.com/a/google.com/chromeos-toolchain-team-home2/home/team-tools-and-scripts/binary-searcher-tool-for-triage
+
+This particular set of scripts is designed to work wtih
+binary_search_state.py in order to find the bad object or set of
+bad objects in an Android build.
+
+
+QUICKSTART:
+
+After setting up your 2 build trees (see Prerequisites section), do the
+following:
+
+ - Decide which test script to use (boot_test.sh or
+   interactive_test.sh)
+ - Get the serial number for the Android device you will use for testing.
+ - Run the following:
+
+   $ cd <android_src>
+   $ source build/envsetup.sh
+   $ lunch <android_device_lunch_combo>
+   $ cd <path_to_toolchain_utils>/binary_search_tool/
+   $ NUM_JOBS=10 ANDROID_SERIAL=<device_serial> \
+         ./android/setup.sh <android_src>
+
+   If you chose the boot test, then:
+   TEST_SCRIPT=android/boot_test.sh
+
+   If you chose the interactive test, then:
+   TEST_SCRIPT=android/interactive_test.sh
+
+   Finally, run the binary search tool:
+
+   $ python ./binary_search_state.py \
+       --get_initial_items=android/get_initial_items.sh \
+       --switch_to_good=android/switch_to_good.sh \
+       --switch_to_bad=android/switch_to_bad.sh \
+       --install_script=android/install.sh \
+       --test_script=$TEST_SCRIPT \
+       --file_args \
+       --prune
+
+   Once you have completely finished doing the binary search/triage,
+   run the cleanup script:
+
+   $ android/cleanup.sh
+
+
+
+FILES AND SCRIPTS:
+
+  Check the header comments for each script for more in depth documentation.
+
+  boot_test.sh - One of two possible test scripts used to determine
+                 if the Android image built from the objects is good
+                 or bad. This script tests to see if the image
+                 booted, and requires no user intervention.
+
+  cleanup.sh - This is called after the binary search tool completes. This
+               script will clean up the common.sh file generated by setup.sh
+
+  get_initial_items.sh - This script is used to determine all Android objects
+                         that will be bisected.
+
+  install.sh - This script will build and flash your image to the
+               Android device. If the flash fails, this script will
+               help the user troubleshoot by trying to flash again or
+               by asking the user to manually flash it.
+
+  interactive_test.sh - One of two possible scripts used to determine
+                        if the Android image built from the objects
+                        is good or bad.  This script requires user
+                        interaction to determine if the image is
+                        good or bad.
+
+  setup.sh - This is the first script the user should call, after
+             taking care of the prerequisites.  It sets up the
+             environment appropriately for running the Android
+             object binary search triage, and it generates the
+             necessary common script (see below).
+
+  switch_to_bad.sh - This script is used to link objects from the
+                     'bad' build tree into the work area.
+
+  switch_to_good.sh - This script is used to link objects from the
+                      'good' build tree into the work area.
+
+
+GENERATED SCRIPTS:
+
+  common.sh  - contains basic environment variable definitions for
+               this binary search triage session.
+
+ASSUMPTIONS:
+
+- There are two different Android builds, for the same board/lunch combo with
+  the same set of generated object files.  One build creates a good working
+  Android image and the other does not.
+
+- The toolchain bug you are tracking down is not related to the linker. If the
+  linker is broken or generates bad code, this tool is unlikely to help you.
+
+
+PREREQUISITES FOR USING THESE SCRIPTS:
+
+  Step 1: Decide where to store each build tree
+    By default, each build tree is stored in "~/ANDROID_BISECT". However you
+    can override this by exporting BISECT_DIR set to whatever directory you
+    please. Keep in mind these build trees take dozens of gigabytes each.
+
+  Step 2: Setup your android build environment
+    1. `cd <android_src>`
+    2. `source build/envsetup.sh`
+    3. `lunch <android_device_lunch_combo>`
+
+  Step 3: Populate the good build tree
+    1. `make clean`
+    2. `export BISECT_STAGE=POPULATE_GOOD`
+    3. Install your "good" toolchain in Android, this will most likely be
+       the toolchain that comes preinstalled with the Android source.
+    4. Build all of Android: `make -j10`. The "-j" parameter depends on how
+       many cores your machine has. See Android documentation for more details.
+
+  Step 4: Populate the bad build tree
+    1. `make clean`
+    2. `export BISECT_STAGE=POPULATE_BAD`
+    3. Install your "bad" toolchain in Android.
+    4. Build all of Android again.
+
+  Step 5: Run the android setup script
+    1. `cd <path_to_toolchain_utils>/binary_search_tool/`
+    2. `NUM_JOBS=<jobs> ANDROID_SERIAL=<android_serial_num> \
+        android/setup.sh <android_src>`
+
+   WARNING: It's important that you leave the full "out/" directory in your
+            Android source alone after Step 4. The binary search tool will
+            use this directory as a skeleton to build each test image while
+            triaging.
+
+USING THESE SCRIPTS FOR BINARY TRIAGE OF OBJECTS:
+
+To use these scripts, you must first run setup.sh, passing it the path to your
+Android source directory. setup.sh will do the following:
+
+   - Verify that your build trees are set up correctly (with good, bad).
+   - Verify that each build tree has the same contents.
+   - Verify that the android build environment (lunch, etc.) are setup in your
+     current shell.
+   - Create the common.sh file that the other scripts passed to the
+     binary triage tool will need.
+
+
+This set of scripts comes with two alternate test scripts.  One test
+script, boot_test.sh, just checks to make sure that the image
+booted (wait for device to boot to home screen) and assumes that is enough.
+The other test script, interactive_test.sh, is interactive and asks YOU
+to tell it whether the image on the android device is ok or not (it
+prompts you and waits for a response).
+
+
+Once you have run setup.sh (and decided which test script you
+want to use) run the binary triage tool using these scripts to
+isolate/identify the bad object:
+
+./binary_search_state.py \
+ --get_initial_items=android/get_initial_items.sh \
+ --switch_to_good=android/switch_to_good.sh \
+ --switch_to_bad=android/switch_to_bad.sh \
+ --install_script=android/install.sh \
+ --test_script=android/boot_test.sh \  # could use interactive_test.sh instead
+ --prune
+
+
+After you have finished running the tool and have identified the bad
+object(s), you will want to run the cleanup script (android/cleanup.sh).
+