Age | Commit message (Collapse) | Author |
|
This patch added unit tests for pass and transformation level bisect to
the bisecting tool. Since no compiler is involved in unit test, it used
cmd_script.py to simulate the output of compiler (to stderr).
BUG=chromium:878954
TEST=Passed all unit tests.
Change-Id: I599c079497333ec24f08f37c3315a16f28c0f887
Reviewed-on: https://chromium-review.googlesource.com/1298384
Commit-Ready: Zhizhou Yang <zhizhouy@google.com>
Tested-by: Zhizhou Yang <zhizhouy@google.com>
Reviewed-by: Caroline Tice <cmtice@chromium.org>
|
|
This patch provides pass and transformation level bisection support for
existing bisection tool.
Usage: When `--pass_bisect=script_path` is set, pass/transformation
level bisection will be turned on. It only works when prune is set to
False. Orignial bisection will end with the first bad item it find. This
patch then will rebuild and bisect the bad item using bad compiler with
pass/transformation level bisect limit. It will end up with the bad pass
name/index and bad transformation index for that pass.
Notice that this patch still need support from LLVM with this review:
https://reviews.llvm.org/D50031, which allows debugcounter to print info
we need.
The patch eventally add a new binary search process, and two build
function to saparately build with pass and transformation level limit.
A new BinarySearcherForPass is also introduced for pass level bisect.
The workflow is:
1) get bad item and command to build it with existing bisection.
2) rebuild bad item with pass level limit to -1 to get total pass
count.
3) binary search pass index until we got bad pass.
4) rebuild bad item with pass limit we get and also set
transformation level limit to -1 to get total number of
transforamtion.
5) binary search transformation index until we got bad
transformation.
BUG=chromium:878954
TEST=Ran test successfully with Android compiler wrapper. Will add
unittests once new patch in LLVM applied.
Change-Id: I08f376f15d12eea232da5887981c44184ffb9568
Reviewed-on: https://chromium-review.googlesource.com/1208855
Commit-Ready: Zhizhou Yang <zhizhouy@google.com>
Tested-by: Zhizhou Yang <zhizhouy@google.com>
Reviewed-by: Caroline Tice <cmtice@chromium.org>
|
|
In addition to fixing the lint errors, this also fixes the Python
formatting issues (ran tc_pyformat on nearly all the files).
BUG=chromium:570450
TEST=Ran all crosperf & bisect tool unit tests.
Ran afe_lock_machine.py (check machine status)
Ran full crosperf test (octane, speedometer, BootPerf) on alex.
Change-Id: Ic86f9192801ac67769f3de30f1c5f0d203ce0831
Reviewed-on: https://chromium-review.googlesource.com/471886
Commit-Ready: Caroline Tice <cmtice@chromium.org>
Tested-by: Caroline Tice <cmtice@chromium.org>
Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
|
|
REFACTOR WARNING! Run tc_pyformat on all python scripts.
TEST=Run unit tests, Android bisection
Change-Id: I9624e7be2670bdb29bcdb59404bf9e8c2f7d81c5
Reviewed-on: https://chrome-internal-review.googlesource.com/274615
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
|
|
If resuming a bisection, we want to make sure all current good items are
switched to good and all current bad items are switched to bad. This is
so the bisection tool will resume in a known, stable state. Incremental
builds are essentially diffs from previous builds, and the base
environment may have changed since the last execution.
TEST=Add test code, run unit tests
Change-Id: I9ac855efbcfb48bc86f245f33ee29d0c38aa7af9
Reviewed-on: https://chrome-internal-review.googlesource.com/274175
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
|
|
Refactor warning! Rename install script to be test_setup script.
TEST=Run unit tests
Change-Id: Iab7e01136fdcf2dedbfc94ece6ddb4b4fa3c0f68
Reviewed-on: https://chrome-internal-review.googlesource.com/273484
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
|
|
Previously, the user could run multiple levels of verification. This
would essentially repeat verification multiple times. It's unsure why a
user would want to set verify_level to any value other than 0 or 1. Thus
verify_level is more appropriate as a boolean.
TEST=Run unit tests
Change-Id: Id8832c54e2733881bf23f48af2a3b4c294117d1c
Reviewed-on: https://chrome-internal-review.googlesource.com/272766
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
|
|
Revert change that introduced bug when bad item is very last item in
list. Add tests that stress limits of binary search tool and try to look
for these off-by-one errors. Specifically one test checks for the bad
item being in every index of the binary search. Another tests checks for
if every single item is bad.
TEST=Add two new stress tests, run all unit tests
Change-Id: I2d5a0bda035b2c2b4994b0378aa416da19db651d
Reviewed-on: https://chrome-internal-review.googlesource.com/271916
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
|
|
Fix edge case where not all items were sent to switch_to_good script
when noincremental and prune were both set. Pruning progressively narrows
down the total number of items being searched across because some items
can be assumed good. This fix ensures that these "known good" items are
also sent to the switch script if noincremental is set.
TEST=Add unit test for this case, run other tests
Change-Id: I1145f6eea224f06ceb2352ec77e17e849d489606
Reviewed-on: https://chrome-internal-review.googlesource.com/270718
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
|
|
Provide temp files to scripts ($BISECT_GOOD_SET, $BISECT_BAD_SET) that
provide a full listing of the items in the good set and items in the bad
set. This can be used for debugging purposes or so scripts can analyze
the state of the current binary search iteration.
TEST=Add unit test that utilizes these files
Change-Id: I32bc393644f1cb998c2d818a9026486ef83cc4b8
Reviewed-on: https://chrome-internal-review.googlesource.com/270730
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
|
|
This CL finishes switching the subdirectory from 'utils' to
'cros_utils'. It changes all the remaining import statements to
use 'cros_utils'; it removes the 'cros_utils' symlink, and it
renames the 'utils' subdirectory to 'cros_utils'.
BUG=chromium:568195
TEST=ran crosperf & binary search tool unittests.
Change-Id: I7427f8bfb2ddac3a4b6108e46782039059684382
Reviewed-on: https://chrome-internal-review.googlesource.com/270396
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
|
|
Previously when prune was not set found_items would not be updated with
the single bad item found. This commit fixed this edge case and adds a
unit test for it.
TEST=Run unit tests
Change-Id: Iac56c53fd4de97fa12b14fc9ea1e05d021cc6495
Reviewed-on: https://chrome-internal-review.googlesource.com/268725
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
|
|
Add tests for failed verification and iterations limit.
Change-Id: I22c26ed1c5f779ee5f81256c2315a6c523e1dfa8
Reviewed-on: https://chrome-internal-review.googlesource.com/268243
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
|
|
Add the ability to override the default arguments given to the package
and object bisecting tools. This allows users to specify their own
test/install/etc. script but keep the others the same.
Example 1 (do boot test instead of interactive test):
./bisect.py package daisy 172.17.211.182 --test_script=cros_pkg/boot_test.sh
Example 2 (do package bisector system test instead of interactive test):
./bisect.py package daisy 172.17.211.182 \
--test_script=cros_pkg/testing_test.sh --install_script=""
TEST=Run unit tests and ran above examples among others (such as
overriding verbose, prune, etc. options)
CQ-DEPEND=CL:*266996
Change-Id: I5ee24826d4d7e22e7328ce98c6b8cda9917df533
Reviewed-on: https://chrome-internal-review.googlesource.com/266305
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
|
|
Add new script for unifying package and object bisection script
interfaces. Currently only package bisection is implemented.
TEST=Add unit test, test cros_pkg with testing script and interactive
script
Change-Id: I7491e3fb73eae863e24de3869092cfbf9df56c8d
Reviewed-on: https://chrome-internal-review.googlesource.com/266096
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
|
|
Updated binary_search_tool_tester's output checking to be more robust,
and fully check the state of the reported good/bad sets.
Change-Id: Id7ab8c006133d67b75ed902a437ca817e62cd219
Reviewed-on: https://chrome-internal-review.googlesource.com/266029
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
|
|
Add Run method so that the binary search tool can be easily invoked from
other Python scripts (see update to binary_search_tool_tester.py). Now
Main only does very basic arg_parse logic and leaves rest up to Run.
Test=Unit tests run, cros_pkg tester passes
Change-Id: I90400f9223c7aad0a2701944ba8282fb763f1cf7
Reviewed-on: https://chrome-internal-review.googlesource.com/265628
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
|
|
Running test_bad_save_state would previously leave a save state with no
symlink pointing to it. This would cause multiple save states to gather
and clutter up the directory. We now simply cleanup the generated save
state in this test case.
TEST=Run unit test and see if any state files remain
Change-Id: If594c7266b2ff24a260e746a4d06bddea2c0bf78
Reviewed-on: https://chrome-internal-review.googlesource.com/265286
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
|
|
Move around logic so that the binary search tool can be properly resumed
and add the --resume option.
TEST=Added unit test for LoadState
Change-Id: I7059fab9ac6c37ecfd79e4cf2ef976bdebfef8fb
Reviewed-on: https://chrome-internal-review.googlesource.com/264778
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
|
|
Fix the binary search tool so that temp files are created properly. This
means that temp files made for the switch scripts should be automatically
deleted once the switch script completes.
TEST=New unit test for temp file deletion
Change-Id: I5d36bfa07e274cd79965f182d85d6a7ea3006c85
Reviewed-on: https://chrome-internal-review.googlesource.com/264735
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
|
|
Change from using module wide variable common.installed to using a file
called "./installed". Previously common.installed wouldn't actually be
set properly because there would be two separate instances of Python
running.
TEST=More robust install unit test made
Change-Id: Iee87a52e0f54fc63837b25918764e804747525db
Reviewed-on: https://chrome-internal-review.googlesource.com/264745
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
|
|
Fix the unit tests so that they run tail on the correct logs. Also fix
edge case when only one bad item is found.
Change-Id: I31f34755440af51a8a2310d9393aa87dc852c31d
Reviewed-on: https://chrome-internal-review.googlesource.com/264695
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
|
|
SaveState will now save the current state of the binary search tool to a
temp file then atomically switch the symlink to the new file.
TEST=Added new unit test for SaveState
Change-Id: Idf97c8ef346700a6e653206083ee7ab75832dbd5
Reviewed-on: https://chrome-internal-review.googlesource.com/264419
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
|
|
Add optional install_script argument that will run before test_script
use this install_script to perform building/flashing/other misc. setup.
If your build or flash succeeds then return 0, otherwise return 1 and
this build will be "skipped" (equivalent to the test script returning
a 2 exit code).
Test=Update unit tests to test for successful and failed install
Change-Id: Ic492f08230796f50b7bb93aebe078960419a3c99
Reviewed-on: https://chrome-internal-review.googlesource.com/262988
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
|
|
Fix basic style errors in BST tests and remove use of optparse across
scripts.
Change-Id: I4f484fb22ca6bb6849d79144bf7bdb7a38e49468
Reviewed-on: https://chrome-internal-review.googlesource.com/261739
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
|
|
BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.
Change-Id: I0640cab2c8d15380badbfbc0db39e3063cc85471
Reviewed-on: https://chrome-internal-review.googlesource.com/244867
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
|
|
BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.
Change-Id: Ifda134bf5f26db04551b915c189814b86e750df3
Reviewed-on: https://chrome-internal-review.googlesource.com/244866
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
|
|
BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.
Change-Id: I2e0cf0d903b0ca7662132528ee6860ea8eaee83b
Reviewed-on: https://chrome-internal-review.googlesource.com/243412
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
|
|
BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.
Change-Id: I0cb94c77c26941fafb97cca2b9e9bb12cf4c210e
Reviewed-on: https://chrome-internal-review.googlesource.com/243411
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
|
|
This CL fixes all missing-shebang and bad-shebang lint warnings.
BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.
Change-Id: I41bd3516496b246e4e4a8ccc9f97fb25007738a7
Reviewed-on: https://chrome-internal-review.googlesource.com/243136
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
|
|
This gets rid of a lot of lint issues.
Ran by doing this:
for f in *.py; do echo -n "$f " ; if [ -x $f ]; then pyformat -i
--remove_trailing_comma --yapf --force_quote_type=double $f ; else
pyformat -i --remove_shebang --remove_trailing_comma --yapf
--force_quote_type=double $f ; fi ; done
BUG=chromium:567921
TEST=Ran simple crosperf run.
Change-Id: I59778835fdaa5f706d2e1765924389f9e97433d1
Reviewed-on: https://chrome-internal-review.googlesource.com/242031
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
|
|
Cleaned up the interfaces for the RunCommand routines.
These were returning different types (int or tuple) depending on
the value of the return_ouput parameter.
Returning different unrelated types from a routine is bad practice.
Linter complains about this with several warnings like this:
"Attempting to unpack a non-sequence defined at line XY of
utils.command_executer"
BUG=chromium:566256
TEST=ran crosperf with a example experiment file
Ran run_tests.
Change-Id: Ibb83ab9322c87558077fc4937ef5c0686bbe5417
Reviewed-on: https://chrome-internal-review.googlesource.com/241459
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Han Shen <shenhan@google.com>
|
|
We now can test the binary search tool with -
./binary_search_tool_tester.py
This will generate a set of object files with some bad ones, and call
the binary search tool to find them out.
TEST=gpylint and passed my test suite
BUG=None
Change-Id: I56fbda7f75fe3bc239e456161c48b7611c3a315d
Reviewed-on: https://chrome-internal-review.googlesource.com/150255
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
|