binary search tool: Move root path of cros_pkg execution, rename scripts

Update cros_pkg scripts and bisect.py so that the package bisector needs
to be run from binary_search_tool/ instead of
binary_search_tool/cros_pkg. This fits with how sysroot_wrapper
implements its scripts. Also rename all cros_pkg scripts to remove
cros_pkg prefix and remove undo_eclean.py.

TEST=Run unit tests and run interactive/testing cros_pkg test

Change-Id: I2781319934b704b91346745ae2d4d916fee35d02
Reviewed-on: https://chrome-internal-review.googlesource.com/266365
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>

12 files changed, 106 insertions, 154 deletions

diff --git a/binary_search_tool/bisect.py b/binary_search_tool/bisect.py
index 06be35fc..e8bd5ec7 100755
--- a/binary_search_tool/bisect.py
+++ b/binary_search_tool/bisect.py
@@ -49,14 +49,14 @@ class Bisector(object):
 class BisectPackage(Bisector):
   """The class for package bisection steps."""
 
-  cros_pkg_setup = './cros_pkg_setup.sh'
-  cros_pkg_cleanup = './cros_pkg_%s_cleanup.sh'
+  cros_pkg_setup = 'cros_pkg/setup.sh'
+  cros_pkg_cleanup = 'cros_pkg/%s_cleanup.sh'
   default_kwargs = {
-      'get_initial_items': './cros_pkg_get_initial_items.sh',
-      'switch_to_good': './cros_pkg_switch_to_good.sh',
-      'switch_to_bad': './cros_pkg_switch_to_bad.sh',
-      'install_script': './cros_pkg_install.sh',
-      'test_script': './cros_pkg_interactive_test.sh',
+      'get_initial_items': 'cros_pkg/get_initial_items.sh',
+      'switch_to_good': 'cros_pkg/switch_to_good.sh',
+      'switch_to_bad': 'cros_pkg/switch_to_bad.sh',
+      'install_script': 'cros_pkg/install.sh',
+      'test_script': 'cros_pkg/interactive_test.sh',
       'noincremental': False,
       'prune': True,
       'file_args': True
@@ -66,7 +66,6 @@ class BisectPackage(Bisector):
     super(BisectPackage, self).__init__(options)
 
   def PreRun(self):
-    os.chdir('./cros_pkg')
     cmd = ('%s %s %s' %
            (self.cros_pkg_setup, self.options.board, self.options.remote))
     ret, _, _ = self.ce.RunCommandWExceptionCleanup(cmd, print_to_console=True)
diff --git a/binary_search_tool/cros_pkg/README.cros_pkg_triage b/binary_search_tool/cros_pkg/README.cros_pkg_triage
index 78715b59..0e3c0e03 100644
--- a/binary_search_tool/cros_pkg/README.cros_pkg_triage
+++ b/binary_search_tool/cros_pkg/README.cros_pkg_triage
@@ -21,97 +21,96 @@ QUICKSTART:
 After setting up your 3 build trees (see Prerequisites section), do the
 following:
 
- - Decide which test script to use (cros_pkg_boot_test.sh or
-   cros_pkg_interactive_test.sh)
+ - Decide which test script to use (boot_test.sh or
+   interactive_test.sh)
  - Get the IP name or address of the chromebook you will use for testing.
  - Do the following inside your chroot:
 
-   $ cd ~/trunk/src/third_party/toolchain_utils/binary_search_tool/cros_pkg
-   $ ./cros_pkg_setup.sh <board-to-test> \
-                         <IP-name-or-address-of-chromebook>
+   $ cd ~/trunk/src/third_party/toolchain_utils/binary_search_tool
+   $ ./cros_pkg/setup.sh <board-to-test> <IP-name-or-address-of-chromebook>
 
    If you chose the boot test, then:
 
-   $ python ../binary_search_state.py \
-       --get_initial_items=cros_pkg_get_initial_items.sh \
-       --switch_to_good=cros_pkg_switch_to_good.sh \
-       --switch_to_bad=cros_pkg_switch_to_bad.sh \
-       --install_script=cros_pkg_install.sh \
-       --test_script=cros_pkg_boot_test.sh \
+   $ python ./binary_search_state.py \
+       --get_initial_items=cros_pkg/get_initial_items.sh \
+       --switch_to_good=cros_pkg/switch_to_good.sh \
+       --switch_to_bad=cros_pkg/switch_to_bad.sh \
+       --install_script=cros_pkg/install.sh \
+       --test_script=cros_pkg/boot_test.sh \
        --file_args \
        --prune
 
    Otherwise, if you chose the interactive test, then:
 
-   $ python ../binary_search_state.py \
-       --get_initial_items=cros_pkg_get_initial_items.sh \
-       --switch_to_good=cros_pkg_switch_to_good.sh \
-       --switch_to_bad=cros_pkg_switch_to_bad.sh \
-       --install_script=cros_pkg_install.sh \
-       --test_script=cros_pkg_interactive_test.sh \
+   $ python ./binary_search_state.py \
+       --get_initial_items=cros_pkg/get_initial_items.sh \
+       --switch_to_good=cros_pkg/switch_to_good.sh \
+       --switch_to_bad=cros_pkg/switch_to_bad.sh \
+       --install_script=cros_pkg/install.sh \
+       --test_script=cros_pkg/interactive_test.sh \
        --file_args \
        --prune
 
    Once you have completely finished doing the binary search/triage,
    run the genereated cleanup script, to restore your chroot to the state
-   it was in before you ran the cros_pkg_setup.sh script:
+   it was in before you ran the setup.sh script:
 
-   $ ./cros_pkg_${BOARD}_cleanup.sh
+   $ cros_pkg/${BOARD}_cleanup.sh
 
 
 
 FILES AND SCRIPTS:
 
-  cros_pkg_boot_test.sh - One of two possible test scripts used to determine
-                          if the ChromeOS image built from the packages is good
-                          or bad.  This script tests to see if the image
-                          booted, and requires no user intervention.
+  boot_test.sh - One of two possible test scripts used to determine
+                 if the ChromeOS image built from the packages is good
+                 or bad.  This script tests to see if the image
+                 booted, and requires no user intervention.
 
-  cros_pkg_create_cleanup_script.py - This is called by cros_pkg_setup.sh, to
-                                      generate cros_pkg_${BOARD}_cleanup.sh,
-                                      which is supposed to be run by the user
-                                      after the binary search triage process is
-                                      finished, to undo the changes made by
-                                      cros_pkg_setup.sh and return everything
-                                      to its original state.
+  create_cleanup_script.py - This is called by setup.sh, to
+                             generate ${BOARD}_cleanup.sh,
+                             which is supposed to be run by the user
+                             after the binary search triage process is
+                             finished, to undo the changes made by
+                             setup.sh and return everything
+                             to its original state.
 
-  cros_pkg_get_initial_items.sh - This script is used to determine the current
-                                  set of ChromeOS packages.
+  get_initial_items.sh - This script is used to determine the current
+                         set of ChromeOS packages.
 
-  cros_pkg_install.sh - This script will build and flash your image to the
-                        remote machine. If the flash fails, this script will
-			help the user troubleshoot by flashing through usb or
-			by retrying the flash over ethernet.
+  install.sh - This script will build and flash your image to the
+               remote machine. If the flash fails, this script will
+               help the user troubleshoot by flashing through usb or
+               by retrying the flash over ethernet.
 
-  cros_pkg_interactive_test.sh - One of two possible scripts used to determine
-                                 if the ChromeOS image built from the packages
-                                 is good or bad.  This script requires user
-                                 interaction to determine if the image is
-                                 good or bad.
+  interactive_test.sh - One of two possible scripts used to determine
+                        if the ChromeOS image built from the packages
+                        is good or bad.  This script requires user
+                        interaction to determine if the image is
+                        good or bad.
 
-  cros_pkg_setup.sh - This is the first script the user should call, after
-                      taking care of the prerequisites.  It sets up the
-                      environment appropriately for running the ChromeOS
-                      package binary search triage, and it generates two
-                      necessary scripts (see below).
+  setup.sh - This is the first script the user should call, after
+             taking care of the prerequisites.  It sets up the
+             environment appropriately for running the ChromeOS
+             package binary search triage, and it generates two
+             necessary scripts (see below).
 
-  cros_pkg_switch_to_bad.sh - This script is used to copy packages from the
-                              'bad' build tree into the work area.
+  switch_to_bad.sh - This script is used to copy packages from the
+                     'bad' build tree into the work area.
 
-  cros_pkg_switch_to_good.sh - This script is used to copy packages from the
-                               'good' build tree into the work area.
+  switch_to_good.sh - This script is used to copy packages from the
+                      good' build tree into the work area.
 
 
 GENERATED SCRIPTS:
 
-  cros_pkg_common.sh  - contains basic environment variable definitions for
-                        this binary search triage session.
+  common.sh  - contains basic environment variable definitions for
+               this binary search triage session.
 
-  cros_pkg_${BOARD}_cleanup.sh - script to undo all the changes made by
-                                 running cros_pkg_setup.sh, and returning
-                                 everything to its original state. The user
-                                 should manually run this script once the
-                                 binary search triage process is over.
+  ${BOARD}_cleanup.sh - script to undo all the changes made by
+                        running setup.sh, and returning
+                        everything to its original state. The user
+                        should manually run this script once the
+                        binary search triage process is over.
 
 ASSUMPTIONS:
 
@@ -144,43 +143,43 @@ PREREQUISITES FOR USING THESE SCRIPTS (inside the chroot):
 
 USING THESE SCRIPTS FOR BINARY TRIAGE OF PACKAGES:
 
-To use these scripts, you must first run cros_pkg_setup.sh, passing it two
+To use these scripts, you must first run setup.sh, passing it two
 arguments (in order): the board for which you are building the image;
 and the name or ip address of the chromebook you want to use for
-testing your chromeos images.  cros_pkg_setup.sh will do the following:
+testing your chromeos images.  setup.sh will do the following:
 
    - Verify that your build trees are set up correctly (with good, bad
      and work).
    - Create a soft link for /build/${board} pointing to the work build
      tree.
-   - Create the cros_pkg_common.sh file that the other scripts passed to the
+   - Create the common.sh file that the other scripts passed to the
      binary triage tool will need.
-   - Create a cleanup script, cros_pkg_${board}_cleanup.sh, for you to
+   - Create a cleanup script, ${board}_cleanup.sh, for you to
      run after you are done with the binary triages, to undo all of these
-     various changes that cros_pkg_setup.sh did.
+     various changes that setup.sh did.
 
 
 This set of scripts comes with two alternate test scripts.  One test
-script, cros_pkg_boot_test.sh, just checks to make sure that the image
+script, boot_test.sh, just checks to make sure that the image
 booted (i.e. responds to ping) and assumes that is enough.  The other
-test script, cros_pkg_interactive_test.sh, is interactive and asks YOU
+test script, interactive_test.sh, is interactive and asks YOU
 to tell it whether the image on the chromebook is ok or not (it
 prompts you and waits for a response).
 
 
-Once you have run cros_pkg_setup.sh (and decided which test script you
+Once you have run setup.sh (and decided which test script you
 want to use) run the binary triage tool using these scripts to
 isolate/identify the bad package:
 
 ~/trunk/src/third_party/toolchain_utils/binary_search_tool/binary_search_state.py \
- --get_initial_items=cros_pkg_get_initial_items.sh \
- --switch_to_good=cros_pkg_switch_to_good.sh \
- --switch_to_bad=cros_pkg_switch_to_bad.sh \
- --install_script=cros_pkg_install.sh \
- --test_script=cros_pkg_boots_test.sh \  # could use cros_pkg_interactive_test.sh instead
+ --get_initial_items=cros_pkg/get_initial_items.sh \
+ --switch_to_good=cros_pkg/switch_to_good.sh \
+ --switch_to_bad=cros_pkg/switch_to_bad.sh \
+ --install_script=cros_pkg/install.sh \
+ --test_script=cros_pkg/boots_test.sh \  # could use interactive_test.sh instead
  --prune
 
 
 After you have finished running the tool and have identified the bad
-package(s), you will want to run the cleanup script that cros_pkg_setup.sh
-generated (cros_pkg_${BOARD}_cleanup.sh).
+package(s), you will want to run the cleanup script that setup.sh
+generated (cros_pkg/${BOARD}_cleanup.sh).
diff --git a/binary_search_tool/cros_pkg/cros_pkg_boot_test.sh b/binary_search_tool/cros_pkg/boot_test.sh
index e95745eb..4ccb05c6 100755
--- a/binary_search_tool/cros_pkg/cros_pkg_boot_test.sh
+++ b/binary_search_tool/cros_pkg/boot_test.sh
@@ -12,7 +12,7 @@
 # apply in this case).
 #
 
-source cros_pkg_common.sh
+source cros_pkg/common.sh
 
 # Send 3 pings and wait 3 seconds for any responsed (then timeout).
 ping -c 3 -W 3 ${REMOTE}
diff --git a/binary_search_tool/cros_pkg/cros_pkg_create_cleanup_script.py b/binary_search_tool/cros_pkg/create_cleanup_script.py
index a4770a8c..45a90260 100755
--- a/binary_search_tool/cros_pkg/cros_pkg_create_cleanup_script.py
+++ b/binary_search_tool/cros_pkg/create_cleanup_script.py
@@ -1,12 +1,12 @@
 #!/usr/bin/python2
 #
 #  Copyright 2015 Google Inc. All Rights Reserved
-"""The script to generate a cleanup script after cros_pkg_setup.sh.
+"""The script to generate a cleanup script after setup.sh.
 
-This script takes a set of flags, telling it what cros_pkg_setup.sh changed
+This script takes a set of flags, telling it what setup.sh changed
 during the set up process. Based on the values of the input flags, it
-generates a cleanup script, named cros_pkg_${BOARD}_cleanup.sh, which will
-undo the changes made by cros_pkg_setup.sh, returning everything to its
+generates a cleanup script, named ${BOARD}_cleanup.sh, which will
+undo the changes made by setup.sh, returning everything to its
 original state.
 """
 
@@ -23,9 +23,9 @@ def Usage(parser, msg):
 
 
 def Main(argv):
-  """Generate a script to undo changes done by cros_pkg_setup.sh
+  """Generate a script to undo changes done by setup.sh
 
-    The script cros_pkg_setup.sh makes a change that needs to be
+    The script setup.sh makes a change that needs to be
     undone, namely it creates a soft link making /build/${board} point
     to /build/${board}.work.  To do this, it had to see if
     /build/${board} already existed, and if so, whether it was a real
@@ -35,9 +35,8 @@ def Main(argv):
     created the new soft link.  If the /build/${board} did not
     previously exist, then it just created the new soft link.
 
-    This function takes arguments that tell it exactly what cros_pkg_setup.sh
+    This function takes arguments that tell it exactly what setup.sh
     actually did, then generates a script to undo those exact changes.
-
   """
 
   parser = argparse.ArgumentParser()
@@ -77,7 +76,7 @@ def Main(argv):
       Usage(parser, 'If --tree_existed is True, then must have either '
             '--old_link or --renamed_tree')
 
-  out_filename = 'cros_pkg_' + options.board + '_cleanup.sh'
+  out_filename = 'cros_pkg/' + options.board + '_cleanup.sh'
 
   with open(out_filename, 'w') as out_file:
     out_file.write('#!/bin/bash\n\n')
@@ -100,8 +99,8 @@ def Main(argv):
         out_file.write('sudo ln -s %s /build/%s\n' % (original_link,
                                                       options.board))
     out_file.write('\n')
-    # Remove cros_pkg_common.sh file
-    out_file.write('rm cros_pkg_common.sh\n')
+    # Remove common.sh file
+    out_file.write('rm cros_pkg/common.sh\n')
 
   return 0
 
diff --git a/binary_search_tool/cros_pkg/cros_pkg_undo_eclean.py b/binary_search_tool/cros_pkg/cros_pkg_undo_eclean.py
deleted file mode 100755
index a08651ed..00000000
--- a/binary_search_tool/cros_pkg/cros_pkg_undo_eclean.py
+++ /dev/null
@@ -1,45 +0,0 @@
-#!/usr/bin/python2
-#
-# Copyright 2015 Google Inc. All Rights Reserved.
-"""Comment out eclean invocations from input script.
-
-This script takes a single argument, the name of a file (normally expected to
-be 'build_image'), which should be a shell script. It then creates a new
-output file, named <input_file>.edited, and it copies each line from the
-input file to the output file.  If the line from the input file contains the
-string 'eclean', it prepends a '#' to the line before copying it to the
-output file, in effect commenting out any lines that contain 'eclean'.
-"""
-
-from __future__ import print_function
-
-import sys
-import os
-
-
-def Main(args):
-
-  if args:
-    filename = args[0]
-    if not os.path.exists(filename):
-      return 1
-  else:
-    return 1
-
-  outname = filename + '.edited'
-  with open(filename, 'r') as input_file:
-    lines = input_file.readlines()
-    with open(outname, 'w') as out_file:
-      for line in lines:
-        if line.find('eclean') >= 0:
-          out_line = '# ' + line
-        else:
-          out_line = line
-        out_file.write(out_line)
-
-  return 0
-
-
-if __name__ == '__main__':
-  retval = Main(sys.argv[1:])
-  sys.exit(retval)
diff --git a/binary_search_tool/cros_pkg/cros_pkg_get_initial_items.sh b/binary_search_tool/cros_pkg/get_initial_items.sh
index c1ca033b..6c87e4f2 100755
--- a/binary_search_tool/cros_pkg/cros_pkg_get_initial_items.sh
+++ b/binary_search_tool/cros_pkg/get_initial_items.sh
@@ -8,7 +8,7 @@
 # for doing the binary search.
 #
 
-source cros_pkg_common.sh
+source cros_pkg/common.sh
 
 cd ${GOOD_BUILD}/packages
 find . -name "*.tbz2"
diff --git a/binary_search_tool/cros_pkg/cros_pkg_install.sh b/binary_search_tool/cros_pkg/install.sh
index ac42d152..f12534d6 100755
--- a/binary_search_tool/cros_pkg/cros_pkg_install.sh
+++ b/binary_search_tool/cros_pkg/install.sh
@@ -16,7 +16,7 @@
 
 export PYTHONUNBUFFERED=1
 
-source cros_pkg_common.sh
+source cros_pkg/common.sh
 
 usb_flash()
 {
diff --git a/binary_search_tool/cros_pkg/cros_pkg_interactive_test.sh b/binary_search_tool/cros_pkg/interactive_test.sh
index 5aa19f44..011c6d8d 100755
--- a/binary_search_tool/cros_pkg/cros_pkg_interactive_test.sh
+++ b/binary_search_tool/cros_pkg/interactive_test.sh
@@ -14,7 +14,7 @@
 # not determine (does not apply in this case).
 #
 
-source cros_pkg_common.sh
+source cros_pkg/common.sh
 
 ping -c 3 -W 3 ${REMOTE}
 retval=$?
diff --git a/binary_search_tool/cros_pkg/cros_pkg_setup.sh b/binary_search_tool/cros_pkg/setup.sh
index 99cb802d..b3bee64c 100755
--- a/binary_search_tool/cros_pkg/cros_pkg_setup.sh
+++ b/binary_search_tool/cros_pkg/setup.sh
@@ -14,9 +14,9 @@
 #
 # This script sets up a soft link definining /build/${board} to point
 # to the working build tree, for the binary search triags process.  In
-# addition, this script generates two other scripts, cros_pkg_common.sh,
+# addition, this script generates two other scripts, common.sh,
 # which generates enviroment variables used by the other scripts in the
-# package binary search triage process; and cros_pkg_${board}_cleanup.sh,
+# package binary search triage process; and ${board}_cleanup.sh,
 # which undoes the various changes that this script performs, returning the
 # user's environment to its original state.
 #
@@ -52,7 +52,7 @@ fi
 #
 # Check to see if /build/${BOARD} already exists and if so, in what state.
 # Set appropriate flags & values, in order to be able to undo these changes
-# in cros_pkg_${board_cleanup.sh.  If it's a soft link, remove it; if it's a
+# in ${board}_cleanup.sh. If it's a soft link, remove it; if it's a
 # read tree, rename it.
 #
 
@@ -78,10 +78,10 @@ fi
 sudo ln -s /build/${BOARD}.work /build/${BOARD}
 
 #
-# Create cros_pkg_common.sh file, containing appropriate environment variables.
+# Create common.sh file, containing appropriate environment variables.
 #
 
-COMMON_FILE="cros_pkg_common.sh"
+COMMON_FILE="cros_pkg/common.sh"
 
 cat <<-EOF > ${COMMON_FILE}
 
@@ -97,26 +97,26 @@ EOF
 chmod 755 ${COMMON_FILE}
 
 #
-# Create clean-up script, calling cros_pkg_create_cleanup_script.py with
+# Create clean-up script, calling create_cleanup_script.py with
 # the appropriate flags.
 #
 
 if [[ ${build_tree_existed} -eq 0 ]] ; then
 
-    python cros_pkg_create_cleanup_script.py --board=${BOARD} \
+    python cros_pkg/create_cleanup_script.py --board=${BOARD} \
         --old_tree_missing
 
 elif [[ ${build_tree_was_soft_link} -eq 0 ]] ; then
 
-    python cros_pkg_create_cleanup_script.py --board=${BOARD} \
+    python cros_pkg/create_cleanup_script.py --board=${BOARD} \
         --renamed_tree
 
 else
 
-    python cros_pkg_create_cleanup_script.py --board=${BOARD} \
+    python cros_pkg/create_cleanup_script.py --board=${BOARD} \
         --old_link="'${build_tree_link}'"
 fi
 
-chmod 755 cros_pkg_${BOARD}_cleanup.sh
+chmod 755 cros_pkg/${BOARD}_cleanup.sh
 
 exit 0
diff --git a/binary_search_tool/cros_pkg/cros_pkg_switch_to_bad.sh b/binary_search_tool/cros_pkg/switch_to_bad.sh
index 8f07bf30..c971270a 100755
--- a/binary_search_tool/cros_pkg/cros_pkg_switch_to_bad.sh
+++ b/binary_search_tool/cros_pkg/switch_to_bad.sh
@@ -8,7 +8,7 @@
 # build tree, for testing.
 #
 
-source cros_pkg_common.sh
+source cros_pkg/common.sh
 
 pushd ${WORK_BUILD}
 
diff --git a/binary_search_tool/cros_pkg/cros_pkg_switch_to_good.sh b/binary_search_tool/cros_pkg/switch_to_good.sh
index 3d64d25a..c5bfa8b9 100755
--- a/binary_search_tool/cros_pkg/cros_pkg_switch_to_good.sh
+++ b/binary_search_tool/cros_pkg/switch_to_good.sh
@@ -8,7 +8,7 @@
 # build tree, for testing.
 #
 
-source cros_pkg_common.sh
+source cros_pkg/common.sh
 
 pushd ${WORK_BUILD}
 
diff --git a/binary_search_tool/cros_pkg/cros_pkg_testing_test.sh b/binary_search_tool/cros_pkg/testing_test.sh
index dc2edea9..c20b78bb 100755
--- a/binary_search_tool/cros_pkg/cros_pkg_testing_test.sh
+++ b/binary_search_tool/cros_pkg/testing_test.sh
@@ -13,7 +13,7 @@
 # dealing with a real 'bad' image).
 #
 
-source cros_pkg_common.sh
+source cros_pkg/common.sh
 
 #
 #Initialize the value below before using this script!!!
@@ -27,7 +27,7 @@ HASH=''
 
 if [ -z "${HASH}" ]
 then
-    echo "ERROR: HASH must be intialized in cros_pkg_testing_test.sh"
+    echo "ERROR: HASH must be intialized in testing_test.sh"
     exit 3
 fi