toolchain-utils: port binary_search_tool to python3

This patch migrates bisect tool in toolchain-utils to python 3.

BUG=chromium:1011676
TEST=Passed all unittests and run_bisect_tests.py

Change-Id: Ia6dd48d927eddcbb2118058f63b33be843d3eb7a
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2042219
Tested-by: Zhizhou Yang <zhizhouy@google.com>
Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
Reviewed-by: George Burgess <gbiv@chromium.org>

28 files changed, 147 insertions, 158 deletions

diff --git a/binary_search_tool/MAINTENANCE b/binary_search_tool/MAINTENANCE
index 8e5b3c61..8f96ff10 100644
--- a/binary_search_tool/MAINTENANCE
+++ b/binary_search_tool/MAINTENANCE
@@ -1,3 +1,7 @@
+# Copyright 2020 The Chromium OS Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
 This document is for future maintainers of the binary search/bisection tools.
 
 Authors:
@@ -21,7 +25,9 @@ The following are good reference materials on how the tool works:
 
 TESTING:
 All unit tests live under the ./test directory. However, these tests
-specifically test binary_search_state.py, binary_search_perforce.py, bisect.py.
+specifically test binary_search_state.py, binary_search_perforce.py,
+run_bisect.py.
+
 These unit tests will not test the specific logic for ChromeOS/Android
 bisection. To test the ChromeOS/Android bisectors, use the common/hash_test.sh
 test. This is a simple test case that just checks the hashes of files on your
@@ -62,10 +68,11 @@ Some of the design decisions are a bit difficult to understand from just reading
 the code unfortunately. I will attempt to clear up the major offenders of this:
 
   1. common.py's argument dictionary:
-     binary_search_state.py and bisect.py both have to have near identical
-     arguments in order to support argument overriding in bisect.py. However
-     they do have to be slightly different. Mainly, bisect.py needs to have no
-     default values for arguments (so it can determine what's being overriden).
+     binary_search_state.py and run_bisect.py both have to have near identical
+     arguments in order to support argument overriding in run_bisect.py. However
+     they do have to be slightly different. Mainly, run_bisect.py needs to have
+     no default values for arguments (so it can determine what's being
+     overriden).
 
      In order to reduce huge amounts of code duplication for the argument
      building, we put argument building in common.py. That way both modules
diff --git a/binary_search_tool/README.bisect.md b/binary_search_tool/README.bisect.md
index 74715ca0..bd9e0f14 100644
--- a/binary_search_tool/README.bisect.md
+++ b/binary_search_tool/README.bisect.md
@@ -1,6 +1,6 @@
-# `bisect.py`
+# `run_bisect.py`
 
-`bisect.py` is a wrapper around the general purpose
+`run_bisect.py` is a wrapper around the general purpose
 `binary_search_state.py`. It provides a user friendly interface for
 bisecting various compilation errors.  The 2 currently provided
 methods of bisecting are ChromeOS package and object bisection. Each
@@ -33,8 +33,8 @@ pings the machine and prompts the user if the machine is good.
     /build/${board}.work - A full copy of /build/${board}.bad
     ```
 
-1.  Cleanup: bisect.py does most cleanup for you, the only thing required by the
-    user is to cleanup all built images and the three build trees made in
+1.  Cleanup: run_bisect.py does most cleanup for you, the only thing required by
+    the user is to cleanup all built images and the three build trees made in
     `/build/`
 
 1.  Default Arguments:
@@ -58,13 +58,13 @@ pings the machine and prompts the user if the machine is good.
     1.  Basic interactive test package bisection, on daisy board:
 
         ```
-        ./bisect.py package daisy 172.17.211.184
+        ./run_bisect.py package daisy 172.17.211.184
         ```
 
     2.  Basic boot test package bisection, on daisy board:
 
         ```
-        ./bisect.py package daisy 172.17.211.184 -t cros_pkg/boot_test.sh
+        ./run_bisect.py package daisy 172.17.211.184 -t cros_pkg/boot_test.sh
         ```
 
 ### ChromeOS Object
@@ -125,10 +125,10 @@ pings the machine and prompts the user if the machine is good.
 1.  Examples:
 
     1.  Basic interactive test object bisection, on daisy board for cryptohome
-        package: `./bisect.py object daisy 172.17.211.184 cryptohome`
+        package: `./run_bisect.py object daisy 172.17.211.184 cryptohome`
 
     2.  Basic boot test package bisection, on daisy board for cryptohome
-        package: `./bisect.py object daisy 172.17.211.184 cryptohome
+        package: `./run_bisect.py object daisy 172.17.211.184 cryptohome
         --test_script=sysroot_wrapper/boot_test.sh`
 
 ### Android object
@@ -202,16 +202,17 @@ the following arguments:
 1.  Examples:
 
     1.  Basic interactive test android bisection, where the android source is at
-        ~/android_src: `./bisect.py android ~/android_src`
+        ~/android_src: `./run_bisect.py android ~/android_src`
 
     2. Basic boot test android bisection, where the android source is at
-       `~/android_src`, and 10 jobs will be used to build android: `./bisect.py
+       `~/android_src`, and 10 jobs will be used to build android:
+       `./run_bisect.py
        android ~/android_src --num_jobs=10
        --test_script=sysroot_wrapper/boot_test.sh`
 
 ### Resuming
 
-`bisect.py` and `binary_search_state.py` offer the
+`run_bisect.py` and `binary_search_state.py` offer the
 ability to resume a bisection in case it was interrupted by a
 SIGINT, power failure, etc. Every time the tool completes a
 bisection iteration its state is saved to disk (usually to the file
@@ -221,14 +222,14 @@ completed iteration.
 
 ### Overriding
 
-You can run `./bisect.py --help` or `./binary_search_state.py
+You can run `./run_bisect.py --help` or `./binary_search_state.py
 --help` for a full list of arguments that can be overriden. Here are
 a couple of examples:
 
 Example 1 (do boot test instead of interactive test):
 
 ```
-./bisect.py package daisy 172.17.211.182 --test_script=cros_pkg/boot_test.sh
+./run_bisect.py package daisy 172.17.211.182 --test_script=cros_pkg/boot_test.sh
 ```
 
 Example 2 (do package bisector system test instead of interactive test, this
@@ -236,13 +237,13 @@ Example 2 (do package bisector system test instead of interactive test, this
            hash_test.sh for more details):
 
 ```
-./bisect.py package daisy 172.17.211.182 \
+./run_bisect.py package daisy 172.17.211.182 \
     --test_script=common/hash_test.sh --test_setup_script=""
 ```
 
 Example 3 (enable verbose mode, disable pruning, and disable verification):
 
 ```
-./bisect.py package daisy 172.17.211.182
+./run_bisect.py package daisy 172.17.211.182
       --verbose --prune=False --verify=False
 ```
diff --git a/binary_search_tool/README.pass_bisect.md b/binary_search_tool/README.pass_bisect.md
index d03a563b..eb0f12a6 100644
--- a/binary_search_tool/README.pass_bisect.md
+++ b/binary_search_tool/README.pass_bisect.md
@@ -41,7 +41,7 @@ them.
     bisection:
 
     ```
-    ./bisect.py android PATH_TO_ANDROID_HOME_DIR
+    ./run_bisect.py android PATH_TO_ANDROID_HOME_DIR
                 --pass_bisect=’android/generate_cmd.sh’
                 --prune=False
                 --ir_diff
diff --git a/binary_search_tool/binary_search_perforce.py b/binary_search_tool/binary_search_perforce.py
index 18517e64..5065f709 100755
--- a/binary_search_tool/binary_search_perforce.py
+++ b/binary_search_tool/binary_search_perforce.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 # Copyright 2020 The Chromium OS Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
@@ -106,7 +106,7 @@ class BinarySearcherForPass(object):
       True if we find the bad pass/transformation, or cannot find bad one after
       decreasing to the first pass/transformation. Otherwise False.
     """
-    assert status == 0 or status == 1 or status == 125
+    assert status in (0, 1, 125), status
 
     if self.current == 0:
       message = ('Runtime error occurs before first pass/transformation. '
@@ -161,7 +161,7 @@ class BinarySearcher(object):
     message = ('Revision: %s index: %d returned: %d' %
                (self.sorted_list[self.current], self.current, status))
     self.logger.LogOutput(message, print_to_console=verbose)
-    assert status == 0 or status == 1 or status == 125
+    assert status in (0, 1, 125), status
     self.index_log.append(self.current)
     self.status_log.append(status)
     bsp = BinarySearchPoint(self.sorted_list[self.current], status, tag)
@@ -170,7 +170,7 @@ class BinarySearcher(object):
     if status == 125:
       self.skipped_indices.append(self.current)
 
-    if status == 0 or status == 1:
+    if status in (0, 1):
       if status == 0:
         self.lo = self.current + 1
       elif status == 1:
@@ -415,9 +415,8 @@ class P4GCCBinarySearcher(P4BinarySearcher):
     for pr in problematic_ranges:
       if cr in range(pr[0], pr[1]):
         patch_file = '/home/asharif/triage_tool/%d-%d.patch' % (pr[0], pr[1])
-        f = open(patch_file)
-        patch = f.read()
-        f.close()
+        with open(patch_file, encoding='utf-8') as f:
+          patch = f.read()
         files = re.findall('--- (//.*)', patch)
         command += '; cd %s' % self.checkout_dir
         for f in files:
@@ -507,7 +506,7 @@ def Main(argv):
     logger.GetLogger().LogOutput('Cleaning up...')
   finally:
     logger.GetLogger().LogOutput(str(p4gccbs.bs), print_to_console=verbose)
-    status = p4gccbs.Cleanup()
+    p4gccbs.Cleanup()
 
 
 if __name__ == '__main__':
diff --git a/binary_search_tool/binary_search_state.py b/binary_search_tool/binary_search_state.py
index e5b6b5ef..1ddd65ce 100755
--- a/binary_search_tool/binary_search_state.py
+++ b/binary_search_tool/binary_search_state.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 # Copyright 2020 The Chromium OS Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
@@ -22,9 +22,9 @@ import tempfile
 import time
 
 # Adds cros_utils to PYTHONPATH
-import binary_search_perforce
-import common
-import pass_mapping
+from binary_search_tool import binary_search_perforce
+from binary_search_tool import common
+from binary_search_tool import pass_mapping
 
 # Now we do import from cros_utils
 from cros_utils import command_executer
@@ -38,11 +38,6 @@ HIDDEN_STATE_FILE = os.path.join(
     os.path.dirname(STATE_FILE), '.%s' % os.path.basename(STATE_FILE))
 
 
-class Error(Exception):
-  """The general binary search tool error class."""
-  pass
-
-
 @contextlib.contextmanager
 def SetFile(env_var, items):
   """Generate set files that can be used by switch/test scripts.
@@ -60,7 +55,7 @@ def SetFile(env_var, items):
     env_var: What environment variable to store the file name in.
     items: What items are in this set.
   """
-  with tempfile.NamedTemporaryFile() as f:
+  with tempfile.NamedTemporaryFile('w', encoding='utf-8') as f:
     os.environ[env_var] = f.name
     f.write('\n'.join(items))
     f.flush()
@@ -162,7 +157,7 @@ class BinarySearchState(object):
       item_list: list of all items to be switched
     """
     if self.file_args:
-      with tempfile.NamedTemporaryFile() as f:
+      with tempfile.NamedTemporaryFile('w', encoding='utf-8') as f:
         f.write('\n'.join(item_list))
         f.flush()
         command = '%s %s' % (switch_script, f.name)
@@ -175,9 +170,8 @@ class BinarySearchState(object):
             command, print_to_console=self.verbose)
       except OSError as e:
         if e.errno == errno.E2BIG:
-          raise Error('Too many arguments for switch script! Use --file_args')
-        else:
-          raise
+          raise RuntimeError('Too many arguments for switch script! Use '
+                             '--file_args')
     assert ret == 0, 'Switch script %s returned %d' % (switch_script, ret)
 
   def TestScript(self):
@@ -390,7 +384,7 @@ class BinarySearchState(object):
           break
         pass_num += 1
         last_pass = l
-    if limit != -1 and pass_num != limit:
+    if limit not in (-1, pass_num):
       raise ValueError('[Error] While building, limit number does not match.')
     return pass_num, self.CollectPassName(last_pass)
 
@@ -615,7 +609,7 @@ class BinarySearchState(object):
     new data.
 
     Raises:
-      Error if STATE_FILE already exists but is not a symlink.
+      OSError if STATE_FILE already exists but is not a symlink.
     """
     ce, l = self.ce, self.l
     self.ce, self.l, self.binary_search.logger = None, None, None
@@ -629,8 +623,8 @@ class BinarySearchState(object):
       if os.path.islink(STATE_FILE):
         old_state = os.readlink(STATE_FILE)
       else:
-        raise Error(('%s already exists and is not a symlink!\n'
-                     'State file saved to %s' % (STATE_FILE, path)))
+        raise OSError(('%s already exists and is not a symlink!\n'
+                       'State file saved to %s' % (STATE_FILE, path)))
 
     # Create new link and atomically overwrite old link
     temp_link = '%s.link' % HIDDEN_STATE_FILE
@@ -693,8 +687,8 @@ class BinarySearchState(object):
     """Return h m s format of elapsed time since execution has started."""
     diff = int(time.time() - self.start_time)
     seconds = diff % 60
-    minutes = (diff / 60) % 60
-    hours = diff / (60 * 60)
+    minutes = (diff // 60) % 60
+    hours = diff // (60 * 60)
 
     seconds = str(seconds).rjust(2)
     minutes = str(minutes).rjust(2)
@@ -878,16 +872,12 @@ def Run(get_initial_items,
                             verify, file_args, verbose)
     bss.DoVerify()
 
-  try:
-    bss.DoSearchBadItems()
-    if pass_bisect:
-      bss.DoSearchBadPass()
-    bss.RemoveState()
-    logger.GetLogger().LogOutput(
-        'Total execution time: %s' % bss.ElapsedTimeString())
-  except Error as e:
-    logger.GetLogger().LogError(e)
-    return 1
+  bss.DoSearchBadItems()
+  if pass_bisect:
+    bss.DoSearchBadPass()
+  bss.RemoveState()
+  logger.GetLogger().LogOutput(
+      'Total execution time: %s' % bss.ElapsedTimeString())
 
   return 0
 
diff --git a/binary_search_tool/common.py b/binary_search_tool/common.py
index 1d033a44..85cd478b 100644
--- a/binary_search_tool/common.py
+++ b/binary_search_tool/common.py
@@ -8,14 +8,14 @@
 This module serves two main purposes:
   1. Programatically include the utils module in PYTHONPATH
   2. Create the argument parsing shared between binary_search_state.py and
-     bisect.py
+     run_bisect.py
 
 The argument parsing is handled by populating _ArgsDict with all arguments.
-_ArgsDict is required so that binary_search_state.py and bisect.py can share
-the argument parsing, but treat them slightly differently. For example,
-bisect.py requires that all argument defaults are suppressed so that overriding
-can occur properly (i.e. only options that are explicitly entered by the user
-end up in the resultant options dictionary).
+_ArgsDict is required so that binary_search_state.py and run_bisect.py can
+share the argument parsing, but treat them slightly differently. For example,
+run_bisect.py requires that all argument defaults are suppressed so that
+overriding can occur properly (i.e. only options that are explicitly entered
+by the user end up in the resultant options dictionary).
 
 ArgumentDict inherits OrderedDict in order to preserve the order the args are
 created so the help text is made properly.
@@ -109,8 +109,8 @@ def BuildArgParser(parser, override=False):
   Args:
     parser: type argparse.ArgumentParser, will call add_argument for every item
             in _ArgsDict
-    override: True if being called from bisect.py. Used to say that default and
-              required options are to be ignored
+    override: True if being called from run_bisect.py. Used to say that default
+              and required options are to be ignored
 
   Returns:
     None
@@ -118,7 +118,7 @@ def BuildArgParser(parser, override=False):
   ArgsDict = GetArgsDict()
 
   # Have no defaults when overriding
-  for arg_names, arg_options in ArgsDict.iteritems():
+  for arg_names, arg_options in ArgsDict.items():
     if override:
       arg_options = arg_options.copy()
       arg_options.pop('default', None)
diff --git a/binary_search_tool/compiler_wrapper.py b/binary_search_tool/compiler_wrapper.py
index 32f0f9e2..8755ed42 100755
--- a/binary_search_tool/compiler_wrapper.py
+++ b/binary_search_tool/compiler_wrapper.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 # Copyright 2020 The Chromium OS Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
@@ -26,7 +26,7 @@ import os
 import shlex
 import sys
 
-import bisect_driver
+from binary_search_tool import bisect_driver
 
 WRAPPED = '%s.real' % sys.argv[0]
 BISECT_STAGE = os.environ.get('BISECT_STAGE')
diff --git a/binary_search_tool/cros_pkg/create_cleanup_script.py b/binary_search_tool/cros_pkg/create_cleanup_script.py
index 1a7d97b8..62ee38f1 100755
--- a/binary_search_tool/cros_pkg/create_cleanup_script.py
+++ b/binary_search_tool/cros_pkg/create_cleanup_script.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 # Copyright 2020 The Chromium OS Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
@@ -87,7 +87,7 @@ def Main(argv):
 
   out_filename = 'cros_pkg/' + options.board + '_cleanup.sh'
 
-  with open(out_filename, 'w') as out_file:
+  with open(out_filename, 'w', encoding='utf-8') as out_file:
     out_file.write('#!/bin/bash\n\n')
     # First, remove the 'new' soft link.
     out_file.write('sudo rm /build/%s\n' % options.board)
diff --git a/binary_search_tool/bisect.py b/binary_search_tool/run_bisect.py
index 6e19dd69..ef1048bb 100755
--- a/binary_search_tool/bisect.py
+++ b/binary_search_tool/run_bisect.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 # Copyright 2020 The Chromium OS Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
@@ -14,17 +14,14 @@ from argparse import RawTextHelpFormatter
 import os
 import sys
 
-import six
-
-import binary_search_state
-import common
+from binary_search_tool import binary_search_state
+from binary_search_tool import common
 
 from cros_utils import command_executer
 from cros_utils import logger
 
 
-@six.add_metaclass(abc.ABCMeta)
-class Bisector(object):
+class Bisector(object, metaclass=abc.ABCMeta):
   """The abstract base class for Bisectors."""
 
   def __init__(self, options, overrides=None):
@@ -48,7 +45,7 @@ class Bisector(object):
     which arguments have been overridden.
 
     Example output:
-      ./bisect.py package daisy 172.17.211.184 -I "" -t cros_pkg/my_test.sh
+      ./run_bisect.py package daisy 172.17.211.184 -I "" -t cros_pkg/my_test.sh
       Performing ChromeOS Package bisection
       Method Config:
         board : daisy
@@ -87,7 +84,7 @@ class Bisector(object):
     out += '\nBisection Config: (* = overridden)\n'
     max_key_len = max([len(str(x)) for x in args.keys()])
     # Print args in common._ArgsDict order
-    args_order = [x['dest'] for x in common.GetArgsDict().itervalues()]
+    args_order = [x['dest'] for x in common.GetArgsDict().values()]
     for key in sorted(args, key=args_order.index):
       val = args[key]
       key_str = str(key).rjust(max_key_len)
@@ -308,7 +305,7 @@ def Run(bisector):
 
 
 _HELP_EPILOG = """
-Run ./bisect.py {method} --help for individual method help/args
+Run ./run_bisect.py {method} --help for individual method help/args
 
 ------------------
 
@@ -322,7 +319,7 @@ def Main(argv):
   override_parser = argparse.ArgumentParser(
       add_help=False,
       argument_default=argparse.SUPPRESS,
-      usage='bisect.py {mode} [options]')
+      usage='run_bisect.py {mode} [options]')
   common.BuildArgParser(override_parser, override=True)
 
   epilog = _HELP_EPILOG + override_parser.format_help()
diff --git a/binary_search_tool/run_bisect_tests.py b/binary_search_tool/run_bisect_tests.py
index 2de58ac1..9172d678 100755
--- a/binary_search_tool/run_bisect_tests.py
+++ b/binary_search_tool/run_bisect_tests.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 # Copyright 2020 The Chromium OS Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
diff --git a/binary_search_tool/sysroot_wrapper/testing_test.py b/binary_search_tool/sysroot_wrapper/testing_test.py
index e860c000..b5ceec1f 100755
--- a/binary_search_tool/sysroot_wrapper/testing_test.py
+++ b/binary_search_tool/sysroot_wrapper/testing_test.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 # Copyright 2020 The Chromium OS Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
diff --git a/binary_search_tool/test/binary_search_tool_test.py b/binary_search_tool/test/binary_search_tool_test.py
index 43437c91..ca9313b5 100755
--- a/binary_search_tool/test/binary_search_tool_test.py
+++ b/binary_search_tool/test/binary_search_tool_test.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 # Copyright 2020 The Chromium OS Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
@@ -18,10 +18,10 @@ import unittest
 
 from cros_utils import command_executer
 from binary_search_tool import binary_search_state
-from binary_search_tool import bisect
+from binary_search_tool import run_bisect
 
-import common
-import gen_obj
+from binary_search_tool.test import common
+from binary_search_tool.test import gen_obj
 
 
 def GenObj():
@@ -40,10 +40,10 @@ def CleanObj():
 
 
 class BisectTest(unittest.TestCase):
-  """Tests for bisect.py"""
+  """Tests for run_bisect.py"""
 
   def setUp(self):
-    with open('./is_setup', 'w'):
+    with open('./is_setup', 'w', encoding='utf-8'):
       pass
 
     try:
@@ -59,8 +59,8 @@ class BisectTest(unittest.TestCase):
     except OSError:
       pass
 
-  class FullBisector(bisect.Bisector):
-    """Test bisector to test bisect.py with"""
+  class FullBisector(run_bisect.Bisector):
+    """Test bisector to test run_bisect.py with"""
 
     def __init__(self, options, overrides):
       super(BisectTest.FullBisector, self).__init__(options, overrides)
@@ -83,14 +83,14 @@ class BisectTest(unittest.TestCase):
       return 0
 
   def test_full_bisector(self):
-    ret = bisect.Run(self.FullBisector({}, {}))
+    ret = run_bisect.Run(self.FullBisector({}, {}))
     self.assertEqual(ret, 0)
     self.assertFalse(os.path.exists(common.OBJECTS_FILE))
     self.assertFalse(os.path.exists(common.WORKING_SET_FILE))
 
   def check_output(self):
     _, out, _ = command_executer.GetCommandExecuter().RunCommandWOutput(
-        ('grep "Bad items are: " logs/binary_search_tool_tester.py.out | '
+        ('grep "Bad items are: " logs/binary_search_tool_test.py.out | '
          'tail -n1'))
     ls = out.splitlines()
     self.assertEqual(len(ls), 1)
@@ -115,7 +115,7 @@ class BisectingUtilsTest(unittest.TestCase):
     """Generate [100-1000] object files, and 1-5% of which are bad ones."""
     GenObj()
 
-    with open('./is_setup', 'w'):
+    with open('./is_setup', 'w', encoding='utf-8'):
       pass
 
     try:
@@ -198,14 +198,14 @@ class BisectingUtilsTest(unittest.TestCase):
     state_file = binary_search_state.STATE_FILE
     hidden_state_file = os.path.basename(binary_search_state.HIDDEN_STATE_FILE)
 
-    with open(state_file, 'w') as f:
+    with open(state_file, 'w', encoding='utf-8') as f:
       f.write('test123')
 
     bss = binary_search_state.MockBinarySearchState()
     with self.assertRaises(binary_search_state.Error):
       bss.SaveState()
 
-    with open(state_file, 'r') as f:
+    with open(state_file, 'r', encoding='utf-8') as f:
       self.assertEqual(f.read(), 'test123')
 
     os.remove(state_file)
@@ -257,7 +257,7 @@ class BisectingUtilsTest(unittest.TestCase):
     bss.SwitchToGood(['0', '1', '2', '3'])
 
     tmp_file = None
-    with open('tmp_file', 'r') as f:
+    with open('tmp_file', 'r', encoding='utf-8') as f:
       tmp_file = f.read()
     os.remove('tmp_file')
 
@@ -333,7 +333,7 @@ class BisectingUtilsTest(unittest.TestCase):
 
   def check_output(self):
     _, out, _ = command_executer.GetCommandExecuter().RunCommandWOutput(
-        ('grep "Bad items are: " logs/binary_search_tool_tester.py.out | '
+        ('grep "Bad items are: " logs/binary_search_tool_test.py.out | '
          'tail -n1'))
     ls = out.splitlines()
     self.assertEqual(len(ls), 1)
@@ -356,7 +356,7 @@ class BisectingUtilsPassTest(BisectingUtilsTest):
 
   def check_pass_output(self, pass_name, pass_num, trans_num):
     _, out, _ = command_executer.GetCommandExecuter().RunCommandWOutput(
-        ('grep "Bad pass: " logs/binary_search_tool_tester.py.out | '
+        ('grep "Bad pass: " logs/binary_search_tool_test.py.out | '
          'tail -n1'))
     ls = out.splitlines()
     self.assertEqual(len(ls), 1)
@@ -367,7 +367,7 @@ class BisectingUtilsPassTest(BisectingUtilsTest):
 
     _, out, _ = command_executer.GetCommandExecuter().RunCommandWOutput(
         ('grep "Bad transformation number: '
-         '" logs/binary_search_tool_tester.py.out | '
+         '" logs/binary_search_tool_test.py.out | '
          'tail -n1'))
     ls = out.splitlines()
     self.assertEqual(len(ls), 1)
@@ -513,7 +513,7 @@ class BisectStressTest(unittest.TestCase):
 
   def check_output(self):
     _, out, _ = command_executer.GetCommandExecuter().RunCommandWOutput(
-        ('grep "Bad items are: " logs/binary_search_tool_tester.py.out | '
+        ('grep "Bad items are: " logs/binary_search_tool_test.py.out | '
          'tail -n1'))
     ls = out.splitlines()
     self.assertEqual(len(ls), 1)
diff --git a/binary_search_tool/test/cmd_script.py b/binary_search_tool/test/cmd_script.py
index 5be3f959..bfd56052 100755
--- a/binary_search_tool/test/cmd_script.py
+++ b/binary_search_tool/test/cmd_script.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 # Copyright 2020 The Chromium OS Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
@@ -16,7 +16,7 @@ from __future__ import print_function
 import os
 import sys
 
-import common
+from binary_search_tool.test import common
 
 
 def Main(argv):
diff --git a/binary_search_tool/test/common.py b/binary_search_tool/test/common.py
index ae2c843f..cf5300f5 100755
--- a/binary_search_tool/test/common.py
+++ b/binary_search_tool/test/common.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 # Copyright 2020 The Chromium OS Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
@@ -14,33 +14,29 @@ WORKING_SET_FILE = 'working_set.txt'
 
 def ReadWorkingSet():
   working_set = []
-  f = open(WORKING_SET_FILE, 'r')
-  for l in f:
-    working_set.append(int(l))
-  f.close()
+  with open(WORKING_SET_FILE, 'r', encoding='utf-8') as f:
+    for l in f:
+      working_set.append(int(l))
   return working_set
 
 
 def WriteWorkingSet(working_set):
-  f = open(WORKING_SET_FILE, 'w')
-  for o in working_set:
-    f.write('{0}\n'.format(o))
-  f.close()
+  with open(WORKING_SET_FILE, 'w', encoding='utf-8') as f:
+    for o in working_set:
+      f.write('{0}\n'.format(o))
 
 
 def ReadObjectsFile():
   objects_file = []
-  f = open(OBJECTS_FILE, 'r')
-  for l in f:
-    objects_file.append(int(l))
-  f.close()
+  with open(OBJECTS_FILE, 'r', encoding='utf-8') as f:
+    for l in f:
+      objects_file.append(int(l))
   return objects_file
 
 
 def ReadObjectIndex(filename):
   object_index = []
-  f = open(filename, 'r')
-  for o in f:
-    object_index.append(int(o))
-  f.close()
+  with open(filename, 'r', encoding='utf-8') as f:
+    for o in f:
+      object_index.append(int(o))
   return object_index
diff --git a/binary_search_tool/test/gen_init_list.py b/binary_search_tool/test/gen_init_list.py
index 5c83206a..bc5dd8fe 100755
--- a/binary_search_tool/test/gen_init_list.py
+++ b/binary_search_tool/test/gen_init_list.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 # Copyright 2020 The Chromium OS Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
@@ -11,7 +11,7 @@ from __future__ import print_function
 import sys
 
 from cros_utils import command_executer
-import common
+from binary_search_tool.test import common
 
 
 def Main():
diff --git a/binary_search_tool/test/gen_obj.py b/binary_search_tool/test/gen_obj.py
index dc7aeefa..4f65c71b 100755
--- a/binary_search_tool/test/gen_obj.py
+++ b/binary_search_tool/test/gen_obj.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 # Copyright 2020 The Chromium OS Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
@@ -17,7 +17,7 @@ import os
 import random
 import sys
 
-import common
+from binary_search_tool.test import common
 
 
 def Main(argv):
@@ -82,12 +82,11 @@ def Main(argv):
   if os.path.isfile(common.WORKING_SET_FILE):
     os.remove(common.WORKING_SET_FILE)
 
-  f = open(common.OBJECTS_FILE, 'w')
-  w = open(common.WORKING_SET_FILE, 'w')
-  for i in obj_list:
-    f.write('{0}\n'.format(i))
-    w.write('{0}\n'.format(i))
-  f.close()
+  with open(common.OBJECTS_FILE, 'w', encoding='utf-8') as f:
+    with open(common.WORKING_SET_FILE, 'w', encoding='utf-8') as w:
+      for i in obj_list:
+        f.write('{0}\n'.format(i))
+        w.write('{0}\n'.format(i))
 
   obj_num = len(obj_list)
   bad_obj_num = obj_list.count(1)
diff --git a/binary_search_tool/test/generate_cmd.py b/binary_search_tool/test/generate_cmd.py
index 51e352e0..51b36b0a 100755
--- a/binary_search_tool/test/generate_cmd.py
+++ b/binary_search_tool/test/generate_cmd.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 # Copyright 2020 The Chromium OS Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
@@ -20,7 +20,7 @@ def Main():
   if not os.path.exists('./is_setup'):
     return 1
   file_name = 'cmd_script.sh'
-  with open(file_name, 'w') as f:
+  with open(file_name, 'w', encoding='utf-8') as f:
     f.write('Generated by generate_cmd.py')
   return 0
 
diff --git a/binary_search_tool/test/is_good.py b/binary_search_tool/test/is_good.py
index 55298da8..662921e8 100755
--- a/binary_search_tool/test/is_good.py
+++ b/binary_search_tool/test/is_good.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 # Copyright 2020 The Chromium OS Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
@@ -11,7 +11,7 @@ from __future__ import print_function
 import os
 import sys
 
-import common
+from binary_search_tool.test import common
 
 
 def Main():
diff --git a/binary_search_tool/test/is_good_noinc_prune.py b/binary_search_tool/test/is_good_noinc_prune.py
index aa0c5884..c0e42bb1 100755
--- a/binary_search_tool/test/is_good_noinc_prune.py
+++ b/binary_search_tool/test/is_good_noinc_prune.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 # Copyright 2020 The Chromium OS Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
@@ -17,16 +17,16 @@ from __future__ import print_function
 import os
 import sys
 
-import common
+from binary_search_tool.test import common
 
 
 def Main():
   working_set = common.ReadWorkingSet()
 
-  with open('noinc_prune_good', 'r') as good_args:
+  with open('noinc_prune_good', 'r', encoding='utf-8') as good_args:
     num_good_args = len(good_args.readlines())
 
-  with open('noinc_prune_bad', 'r') as bad_args:
+  with open('noinc_prune_bad', 'r', encoding='utf-8') as bad_args:
     num_bad_args = len(bad_args.readlines())
 
   num_args = num_good_args + num_bad_args
diff --git a/binary_search_tool/test/switch_tmp.py b/binary_search_tool/test/switch_tmp.py
index 28b0ac2f..0f3c4234 100755
--- a/binary_search_tool/test/switch_tmp.py
+++ b/binary_search_tool/test/switch_tmp.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 # Copyright 2020 The Chromium OS Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
@@ -16,7 +16,7 @@ from __future__ import print_function
 
 import sys
 
-import common
+from binary_search_tool.test import common
 
 
 def Main(argv):
@@ -28,7 +28,7 @@ def Main(argv):
     working_set[int(oi)] = 42
 
   common.WriteWorkingSet(working_set)
-  with open('tmp_file', 'w') as f:
+  with open('tmp_file', 'w', encoding='utf-8') as f:
     f.write(argv[1])
 
   return 0
diff --git a/binary_search_tool/test/switch_to_bad.py b/binary_search_tool/test/switch_to_bad.py
index 91b7e7a8..e3553eb6 100755
--- a/binary_search_tool/test/switch_to_bad.py
+++ b/binary_search_tool/test/switch_to_bad.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 # Copyright 2020 The Chromium OS Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
@@ -10,7 +10,7 @@ from __future__ import print_function
 
 import sys
 
-import common
+from binary_search_tool.test import common
 
 
 def Main(argv):
diff --git a/binary_search_tool/test/switch_to_bad_noinc_prune.py b/binary_search_tool/test/switch_to_bad_noinc_prune.py
index 0deaf1ff..81b558e1 100755
--- a/binary_search_tool/test/switch_to_bad_noinc_prune.py
+++ b/binary_search_tool/test/switch_to_bad_noinc_prune.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 # Copyright 2020 The Chromium OS Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
@@ -23,7 +23,7 @@ from __future__ import print_function
 import shutil
 import sys
 
-import common
+from binary_search_tool.test import common
 
 
 def Main(argv):
diff --git a/binary_search_tool/test/switch_to_bad_set_file.py b/binary_search_tool/test/switch_to_bad_set_file.py
index 829e25c5..5b941c62 100755
--- a/binary_search_tool/test/switch_to_bad_set_file.py
+++ b/binary_search_tool/test/switch_to_bad_set_file.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 # Copyright 2020 The Chromium OS Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
@@ -15,7 +15,7 @@ from __future__ import print_function
 import os
 import sys
 
-import common
+from binary_search_tool.test import common
 
 
 def Main(_):
diff --git a/binary_search_tool/test/switch_to_good.py b/binary_search_tool/test/switch_to_good.py
index 43ec8714..97479329 100755
--- a/binary_search_tool/test/switch_to_good.py
+++ b/binary_search_tool/test/switch_to_good.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 # Copyright 2020 The Chromium OS Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
@@ -15,7 +15,7 @@ from __future__ import print_function
 
 import sys
 
-import common
+from binary_search_tool.test import common
 
 
 def Main(argv):
diff --git a/binary_search_tool/test/switch_to_good_noinc_prune.py b/binary_search_tool/test/switch_to_good_noinc_prune.py
index 6d054f8f..0b91a0d8 100755
--- a/binary_search_tool/test/switch_to_good_noinc_prune.py
+++ b/binary_search_tool/test/switch_to_good_noinc_prune.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 # Copyright 2020 The Chromium OS Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
@@ -23,7 +23,7 @@ from __future__ import print_function
 import shutil
 import sys
 
-import common
+from binary_search_tool.test import common
 
 
 def Main(argv):
diff --git a/binary_search_tool/test/switch_to_good_set_file.py b/binary_search_tool/test/switch_to_good_set_file.py
index b27cf0f6..1cb05e0c 100755
--- a/binary_search_tool/test/switch_to_good_set_file.py
+++ b/binary_search_tool/test/switch_to_good_set_file.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 # Copyright 2020 The Chromium OS Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
@@ -19,7 +19,7 @@ from __future__ import print_function
 import os
 import sys
 
-import common
+from binary_search_tool.test import common
 
 
 def Main(_):
diff --git a/binary_search_tool/test/test_setup.py b/binary_search_tool/test/test_setup.py
index da531429..ecc8eb97 100755
--- a/binary_search_tool/test/test_setup.py
+++ b/binary_search_tool/test/test_setup.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 # Copyright 2020 The Chromium OS Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
@@ -13,7 +13,7 @@ import sys
 
 def Main():
   # create ./is_setup
-  with open('./is_setup', 'w'):
+  with open('./is_setup', 'w', encoding='utf-8'):
     pass
 
   return 0
diff --git a/binary_search_tool/test/test_setup_bad.py b/binary_search_tool/test/test_setup_bad.py
index abd9a83c..cbca3c21 100755
--- a/binary_search_tool/test/test_setup_bad.py
+++ b/binary_search_tool/test/test_setup_bad.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 # Copyright 2020 The Chromium OS Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be