aboutsummaryrefslogtreecommitdiff
path: root/binary_search_tool/test
AgeCommit message (Collapse)Author
2020-02-10toolchain-utils: port binary_search_tool to python3Zhizhou Yang
This patch migrates bisect tool in toolchain-utils to python 3. BUG=chromium:1011676 TEST=Passed all unittests and run_bisect_tests.py Change-Id: Ia6dd48d927eddcbb2118058f63b33be843d3eb7a Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2042219 Tested-by: Zhizhou Yang <zhizhouy@google.com> Auto-Submit: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: George Burgess <gbiv@chromium.org>
2020-02-06binary_search_tool: fix failures in presubmitZhizhou Yang
This patch is to make binary_search_tool in toolchain-utils pass all presbumit checks, as a prerequisite for python 3 migration. BUG=chromium:1011676 TEST=Passed all unittests. Change-Id: Ie7880982f1ca3ae7e6c3d3f342a7a7c076fa9de9 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2037142 Tested-by: Zhizhou Yang <zhizhouy@google.com> Auto-Submit: Zhizhou Yang <zhizhouy@google.com> Commit-Queue: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: George Burgess <gbiv@chromium.org>
2019-07-25toolchain-utils: remove all xrangesGeorge Burgess IV
This removes all mention of xrange from toolchain-utils (modulo ones being changed in other CLs that are in flight). It's now an apparent lint error to use xrange, and it hinders our move to python3. As commented on If90d26664c70ccb73750f17573b89933fdb048f4, xrange -> range in python2 is really only a space concern (or speed in pathological cases), so migrations of this nature are generally super straightforward. I glanced at each of these callsites, and none of them appear to be pathological, so my hope is that this should all be Just Fine :) (Also fun to note that this includes a .diff file that has python code embedded in it.) BUG=None TEST=Presubmit tests Change-Id: Ic9f3ac3a5044d7a07da8a249bc505278d98203de Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1717130 Commit-Queue: George Burgess <gbiv@chromium.org> Commit-Queue: Luis Lozano <llozano@chromium.org> Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2018-10-30bisect tool: Adding unit tests for pass/transform level bisectZhizhou Yang
This patch added unit tests for pass and transformation level bisect to the bisecting tool. Since no compiler is involved in unit test, it used cmd_script.py to simulate the output of compiler (to stderr). BUG=chromium:878954 TEST=Passed all unit tests. Change-Id: I599c079497333ec24f08f37c3315a16f28c0f887 Reviewed-on: https://chromium-review.googlesource.com/1298384 Commit-Ready: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Caroline Tice <cmtice@chromium.org>
2018-09-10Bisect tool: Pass/Transformation level bisection for bad itemZhizhou Yang
This patch provides pass and transformation level bisection support for existing bisection tool. Usage: When `--pass_bisect=script_path` is set, pass/transformation level bisection will be turned on. It only works when prune is set to False. Orignial bisection will end with the first bad item it find. This patch then will rebuild and bisect the bad item using bad compiler with pass/transformation level bisect limit. It will end up with the bad pass name/index and bad transformation index for that pass. Notice that this patch still need support from LLVM with this review: https://reviews.llvm.org/D50031, which allows debugcounter to print info we need. The patch eventally add a new binary search process, and two build function to saparately build with pass and transformation level limit. A new BinarySearcherForPass is also introduced for pass level bisect. The workflow is: 1) get bad item and command to build it with existing bisection. 2) rebuild bad item with pass level limit to -1 to get total pass count. 3) binary search pass index until we got bad pass. 4) rebuild bad item with pass limit we get and also set transformation level limit to -1 to get total number of transforamtion. 5) binary search transformation index until we got bad transformation. BUG=chromium:878954 TEST=Ran test successfully with Android compiler wrapper. Will add unittests once new patch in LLVM applied. Change-Id: I08f376f15d12eea232da5887981c44184ffb9568 Reviewed-on: https://chromium-review.googlesource.com/1208855 Commit-Ready: Zhizhou Yang <zhizhouy@google.com> Tested-by: Zhizhou Yang <zhizhouy@google.com> Reviewed-by: Caroline Tice <cmtice@chromium.org>
2017-04-08[toolchain-utils] Fix remaining lint errors in toolchain-utils.Caroline Tice
In addition to fixing the lint errors, this also fixes the Python formatting issues (ran tc_pyformat on nearly all the files). BUG=chromium:570450 TEST=Ran all crosperf & bisect tool unit tests. Ran afe_lock_machine.py (check machine status) Ran full crosperf test (octane, speedometer, BootPerf) on alex. Change-Id: Ic86f9192801ac67769f3de30f1c5f0d203ce0831 Reviewed-on: https://chromium-review.googlesource.com/471886 Commit-Ready: Caroline Tice <cmtice@chromium.org> Tested-by: Caroline Tice <cmtice@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2016-08-16binary search tool: Run tc_pyformat on all python scriptsCassidy Burden
REFACTOR WARNING! Run tc_pyformat on all python scripts. TEST=Run unit tests, Android bisection Change-Id: I9624e7be2670bdb29bcdb59404bf9e8c2f7d81c5 Reviewed-on: https://chrome-internal-review.googlesource.com/274615 Commit-Ready: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com> Reviewed-by: Caroline Tice <cmtice@google.com>
2016-08-11binary search tool: Make 1st resumed iteration non-incrementalCassidy Burden
If resuming a bisection, we want to make sure all current good items are switched to good and all current bad items are switched to bad. This is so the bisection tool will resume in a known, stable state. Incremental builds are essentially diffs from previous builds, and the base environment may have changed since the last execution. TEST=Add test code, run unit tests Change-Id: I9ac855efbcfb48bc86f245f33ee29d0c38aa7af9 Reviewed-on: https://chrome-internal-review.googlesource.com/274175 Commit-Ready: Cassidy Burden <cburden@google.com> Tested-by: Cassidy Burden <cburden@google.com> Reviewed-by: Caroline Tice <cmtice@google.com>
2016-08-09binary search tool: Rename install to test_setupCassidy Burden
Refactor warning! Rename install script to be test_setup script. TEST=Run unit tests Change-Id: Iab7e01136fdcf2dedbfc94ece6ddb4b4fa3c0f68 Reviewed-on: https://chrome-internal-review.googlesource.com/273484 Commit-Ready: Cassidy Burden <cburden@google.com> Tested-by: Cassidy Burden <cburden@google.com> Reviewed-by: Caroline Tice <cmtice@google.com>
2016-08-03binary search tool: Change verify_level to booleanCassidy Burden
Previously, the user could run multiple levels of verification. This would essentially repeat verification multiple times. It's unsure why a user would want to set verify_level to any value other than 0 or 1. Thus verify_level is more appropriate as a boolean. TEST=Run unit tests Change-Id: Id8832c54e2733881bf23f48af2a3b4c294117d1c Reviewed-on: https://chrome-internal-review.googlesource.com/272766 Commit-Ready: Cassidy Burden <cburden@google.com> Tested-by: Cassidy Burden <cburden@google.com> Reviewed-by: Caroline Tice <cmtice@google.com>
2016-08-01binary search tool: Fix off by one bug, add stress testsCassidy Burden
Revert change that introduced bug when bad item is very last item in list. Add tests that stress limits of binary search tool and try to look for these off-by-one errors. Specifically one test checks for the bad item being in every index of the binary search. Another tests checks for if every single item is bad. TEST=Add two new stress tests, run all unit tests Change-Id: I2d5a0bda035b2c2b4994b0378aa416da19db651d Reviewed-on: https://chrome-internal-review.googlesource.com/271916 Commit-Ready: Cassidy Burden <cburden@google.com> Tested-by: Cassidy Burden <cburden@google.com> Reviewed-by: Caroline Tice <cmtice@google.com>
2016-07-26binary search tool: Fix edge case with noincremental and pruneCassidy Burden
Fix edge case where not all items were sent to switch_to_good script when noincremental and prune were both set. Pruning progressively narrows down the total number of items being searched across because some items can be assumed good. This fix ensures that these "known good" items are also sent to the switch script if noincremental is set. TEST=Add unit test for this case, run other tests Change-Id: I1145f6eea224f06ceb2352ec77e17e849d489606 Reviewed-on: https://chrome-internal-review.googlesource.com/270718 Commit-Ready: Cassidy Burden <cburden@google.com> Tested-by: Cassidy Burden <cburden@google.com> Reviewed-by: Caroline Tice <cmtice@google.com>
2016-07-26binary search tool: Provide temp files that hold GOOD/BAD setsCassidy Burden
Provide temp files to scripts ($BISECT_GOOD_SET, $BISECT_BAD_SET) that provide a full listing of the items in the good set and items in the bad set. This can be used for debugging purposes or so scripts can analyze the state of the current binary search iteration. TEST=Add unit test that utilizes these files Change-Id: I32bc393644f1cb998c2d818a9026486ef83cc4b8 Reviewed-on: https://chrome-internal-review.googlesource.com/270730 Commit-Ready: Cassidy Burden <cburden@google.com> Tested-by: Cassidy Burden <cburden@google.com> Reviewed-by: Caroline Tice <cmtice@google.com> Reviewed-by: Han Shen <shenhan@google.com>
2016-07-25[toolchain-utils] Finish switching utils/ to cros_utils/.Caroline Tice
This CL finishes switching the subdirectory from 'utils' to 'cros_utils'. It changes all the remaining import statements to use 'cros_utils'; it removes the 'cros_utils' symlink, and it renames the 'utils' subdirectory to 'cros_utils'. BUG=chromium:568195 TEST=ran crosperf & binary search tool unittests. Change-Id: I7427f8bfb2ddac3a4b6108e46782039059684382 Reviewed-on: https://chrome-internal-review.googlesource.com/270396 Commit-Ready: Caroline Tice <cmtice@google.com> Tested-by: Caroline Tice <cmtice@google.com> Reviewed-by: Cassidy Burden <cburden@google.com> Reviewed-by: Luis Lozano <llozano@chromium.org>
2016-07-12binary search tool: Update found_items when prune is not setCassidy Burden
Previously when prune was not set found_items would not be updated with the single bad item found. This commit fixed this edge case and adds a unit test for it. TEST=Run unit tests Change-Id: Iac56c53fd4de97fa12b14fc9ea1e05d021cc6495 Reviewed-on: https://chrome-internal-review.googlesource.com/268725 Commit-Ready: Cassidy Burden <cburden@google.com> Tested-by: Cassidy Burden <cburden@google.com> Reviewed-by: Han Shen <shenhan@google.com> Reviewed-by: Luis Lozano <llozano@chromium.org>
2016-07-08binary search tool: Add more unit testsCassidy Burden
Add tests for failed verification and iterations limit. Change-Id: I22c26ed1c5f779ee5f81256c2315a6c523e1dfa8 Reviewed-on: https://chrome-internal-review.googlesource.com/268243 Commit-Ready: Cassidy Burden <cburden@google.com> Tested-by: Cassidy Burden <cburden@google.com> Reviewed-by: Han Shen <shenhan@google.com> Reviewed-by: Luis Lozano <llozano@chromium.org>
2016-06-29Add argument overriding to bisect.pyCassidy Burden
Add the ability to override the default arguments given to the package and object bisecting tools. This allows users to specify their own test/install/etc. script but keep the others the same. Example 1 (do boot test instead of interactive test): ./bisect.py package daisy 172.17.211.182 --test_script=cros_pkg/boot_test.sh Example 2 (do package bisector system test instead of interactive test): ./bisect.py package daisy 172.17.211.182 \ --test_script=cros_pkg/testing_test.sh --install_script="" TEST=Run unit tests and ran above examples among others (such as overriding verbose, prune, etc. options) CQ-DEPEND=CL:*266996 Change-Id: I5ee24826d4d7e22e7328ce98c6b8cda9917df533 Reviewed-on: https://chrome-internal-review.googlesource.com/266305 Commit-Ready: Cassidy Burden <cburden@google.com> Tested-by: Cassidy Burden <cburden@google.com> Reviewed-by: Caroline Tice <cmtice@google.com>
2016-06-22Add initial unified bisection scriptCassidy Burden
Add new script for unifying package and object bisection script interfaces. Currently only package bisection is implemented. TEST=Add unit test, test cros_pkg with testing script and interactive script Change-Id: I7491e3fb73eae863e24de3869092cfbf9df56c8d Reviewed-on: https://chrome-internal-review.googlesource.com/266096 Commit-Ready: Cassidy Burden <cburden@google.com> Tested-by: Cassidy Burden <cburden@google.com> Reviewed-by: Caroline Tice <cmtice@google.com>
2016-06-22binary search tool: Make BST tester check output more robustCassidy Burden
Updated binary_search_tool_tester's output checking to be more robust, and fully check the state of the reported good/bad sets. Change-Id: Id7ab8c006133d67b75ed902a437ca817e62cd219 Reviewed-on: https://chrome-internal-review.googlesource.com/266029 Commit-Ready: Cassidy Burden <cburden@google.com> Tested-by: Cassidy Burden <cburden@google.com> Reviewed-by: Caroline Tice <cmtice@google.com>
2016-06-21binary search tool: Add Run method and move Main logic thereCassidy Burden
Add Run method so that the binary search tool can be easily invoked from other Python scripts (see update to binary_search_tool_tester.py). Now Main only does very basic arg_parse logic and leaves rest up to Run. Test=Unit tests run, cros_pkg tester passes Change-Id: I90400f9223c7aad0a2701944ba8282fb763f1cf7 Reviewed-on: https://chrome-internal-review.googlesource.com/265628 Commit-Ready: Cassidy Burden <cburden@google.com> Tested-by: Cassidy Burden <cburden@google.com> Reviewed-by: Caroline Tice <cmtice@google.com> Reviewed-by: Han Shen <shenhan@google.com>
2016-06-20binary search tool: Cleanup state files after SaveState unit testCassidy Burden
Running test_bad_save_state would previously leave a save state with no symlink pointing to it. This would cause multiple save states to gather and clutter up the directory. We now simply cleanup the generated save state in this test case. TEST=Run unit test and see if any state files remain Change-Id: If594c7266b2ff24a260e746a4d06bddea2c0bf78 Reviewed-on: https://chrome-internal-review.googlesource.com/265286 Commit-Ready: Cassidy Burden <cburden@google.com> Tested-by: Cassidy Burden <cburden@google.com> Reviewed-by: Caroline Tice <cmtice@google.com>
2016-06-16Add resuming to binary search toolCassidy Burden
Move around logic so that the binary search tool can be properly resumed and add the --resume option. TEST=Added unit test for LoadState Change-Id: I7059fab9ac6c37ecfd79e4cf2ef976bdebfef8fb Reviewed-on: https://chrome-internal-review.googlesource.com/264778 Commit-Ready: Cassidy Burden <cburden@google.com> Tested-by: Cassidy Burden <cburden@google.com> Reviewed-by: Caroline Tice <cmtice@google.com> Reviewed-by: Han Shen <shenhan@google.com>
2016-06-15Fix binary search tool's temp file creationCassidy Burden
Fix the binary search tool so that temp files are created properly. This means that temp files made for the switch scripts should be automatically deleted once the switch script completes. TEST=New unit test for temp file deletion Change-Id: I5d36bfa07e274cd79965f182d85d6a7ea3006c85 Reviewed-on: https://chrome-internal-review.googlesource.com/264735 Commit-Ready: Cassidy Burden <cburden@google.com> Tested-by: Cassidy Burden <cburden@google.com> Reviewed-by: Caroline Tice <cmtice@google.com>
2016-06-15Fix binary search tool install unit testsCassidy Burden
Change from using module wide variable common.installed to using a file called "./installed". Previously common.installed wouldn't actually be set properly because there would be two separate instances of Python running. TEST=More robust install unit test made Change-Id: Iee87a52e0f54fc63837b25918764e804747525db Reviewed-on: https://chrome-internal-review.googlesource.com/264745 Commit-Ready: Cassidy Burden <cburden@google.com> Tested-by: Cassidy Burden <cburden@google.com> Reviewed-by: Caroline Tice <cmtice@google.com>
2016-06-14Fix binary search tool tester to process output correctlyCassidy Burden
Fix the unit tests so that they run tail on the correct logs. Also fix edge case when only one bad item is found. Change-Id: I31f34755440af51a8a2310d9393aa87dc852c31d Reviewed-on: https://chrome-internal-review.googlesource.com/264695 Commit-Ready: Cassidy Burden <cburden@google.com> Tested-by: Cassidy Burden <cburden@google.com> Reviewed-by: Luis Lozano <llozano@chromium.org>
2016-06-14Update BinarySearchState's SaveState to be atomicCassidy Burden
SaveState will now save the current state of the binary search tool to a temp file then atomically switch the symlink to the new file. TEST=Added new unit test for SaveState Change-Id: Idf97c8ef346700a6e653206083ee7ab75832dbd5 Reviewed-on: https://chrome-internal-review.googlesource.com/264419 Commit-Ready: Cassidy Burden <cburden@google.com> Tested-by: Cassidy Burden <cburden@google.com> Reviewed-by: Caroline Tice <cmtice@google.com>
2016-06-08Add install_script argument to binary search tool.Cassidy Burden
Add optional install_script argument that will run before test_script use this install_script to perform building/flashing/other misc. setup. If your build or flash succeeds then return 0, otherwise return 1 and this build will be "skipped" (equivalent to the test script returning a 2 exit code). Test=Update unit tests to test for successful and failed install Change-Id: Ic492f08230796f50b7bb93aebe078960419a3c99 Reviewed-on: https://chrome-internal-review.googlesource.com/262988 Commit-Ready: Cassidy Burden <cburden@google.com> Tested-by: Cassidy Burden <cburden@google.com> Reviewed-by: Caroline Tice <cmtice@google.com>
2016-06-01Fix linting errors with binary search tool.Cassidy Burden
Fix basic style errors in BST tests and remove use of optparse across scripts. Change-Id: I4f484fb22ca6bb6849d79144bf7bdb7a38e49468 Reviewed-on: https://chrome-internal-review.googlesource.com/261739 Commit-Ready: Cassidy Burden <cburden@google.com> Tested-by: Cassidy Burden <cburden@google.com> Reviewed-by: Caroline Tice <cmtice@google.com> Reviewed-by: Han Shen <shenhan@google.com>
2016-01-23toolchain-utils: fix class-missing-docstring lint warnings.Rahul Chaudhry
BUG=chromium:570458 TEST='crosperf/run_tests.sh' passes. Change-Id: I0640cab2c8d15380badbfbc0db39e3063cc85471 Reviewed-on: https://chrome-internal-review.googlesource.com/244867 Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com> Tested-by: Rahul Chaudhry <rahulchaudhry@google.com> Reviewed-by: Luis Lozano <llozano@chromium.org>
2016-01-23toolchain-utils: fix module-missing-docstring lint warnings.Rahul Chaudhry
BUG=chromium:570458 TEST='crosperf/run_tests.sh' passes. Change-Id: Ifda134bf5f26db04551b915c189814b86e750df3 Reviewed-on: https://chrome-internal-review.googlesource.com/244866 Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com> Tested-by: Rahul Chaudhry <rahulchaudhry@google.com> Reviewed-by: Luis Lozano <llozano@chromium.org>
2016-01-04toolchain-utils: fix missing-print-function lint warnings.Rahul Chaudhry
BUG=chromium:570458 TEST='crosperf/run_tests.sh' passes. Change-Id: I2e0cf0d903b0ca7662132528ee6860ea8eaee83b Reviewed-on: https://chrome-internal-review.googlesource.com/243412 Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com> Tested-by: Rahul Chaudhry <rahulchaudhry@google.com> Reviewed-by: Caroline Tice <cmtice@google.com>
2016-01-04toolchain-utils: apply pyformat to the source files.Rahul Chaudhry
BUG=chromium:570458 TEST='crosperf/run_tests.sh' passes. Change-Id: I0cb94c77c26941fafb97cca2b9e9bb12cf4c210e Reviewed-on: https://chrome-internal-review.googlesource.com/243411 Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com> Tested-by: Rahul Chaudhry <rahulchaudhry@google.com> Reviewed-by: Caroline Tice <cmtice@google.com>
2015-12-28toolchain-utils: fix shebang related lint warnings.Rahul Chaudhry
This CL fixes all missing-shebang and bad-shebang lint warnings. BUG=chromium:570458 TEST='crosperf/run_tests.sh' passes. Change-Id: I41bd3516496b246e4e4a8ccc9f97fb25007738a7 Reviewed-on: https://chrome-internal-review.googlesource.com/243136 Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com> Tested-by: Rahul Chaudhry <rahulchaudhry@google.com> Reviewed-by: Luis Lozano <llozano@chromium.org>
2015-12-16Run pyformat on all the toolchain-utils files.Luis Lozano
This gets rid of a lot of lint issues. Ran by doing this: for f in *.py; do echo -n "$f " ; if [ -x $f ]; then pyformat -i --remove_trailing_comma --yapf --force_quote_type=double $f ; else pyformat -i --remove_shebang --remove_trailing_comma --yapf --force_quote_type=double $f ; fi ; done BUG=chromium:567921 TEST=Ran simple crosperf run. Change-Id: I59778835fdaa5f706d2e1765924389f9e97433d1 Reviewed-on: https://chrome-internal-review.googlesource.com/242031 Reviewed-by: Luis Lozano <llozano@chromium.org> Commit-Queue: Luis Lozano <llozano@chromium.org> Tested-by: Luis Lozano <llozano@chromium.org> Reviewed-by: Yunlian Jiang <yunlian@google.com>
2015-12-15crosperf: RunCommand should return one type of object.Luis Lozano
Cleaned up the interfaces for the RunCommand routines. These were returning different types (int or tuple) depending on the value of the return_ouput parameter. Returning different unrelated types from a routine is bad practice. Linter complains about this with several warnings like this: "Attempting to unpack a non-sequence defined at line XY of utils.command_executer" BUG=chromium:566256 TEST=ran crosperf with a example experiment file Ran run_tests. Change-Id: Ibb83ab9322c87558077fc4937ef5c0686bbe5417 Reviewed-on: https://chrome-internal-review.googlesource.com/241459 Commit-Ready: Luis Lozano <llozano@chromium.org> Tested-by: Luis Lozano <llozano@chromium.org> Reviewed-by: Han Shen <shenhan@google.com>
2014-11-27Move binary search tool to toolchain-utils with a comprehensive test suite.Han Shen
We now can test the binary search tool with - ./binary_search_tool_tester.py This will generate a set of object files with some bad ones, and call the binary search tool to find them out. TEST=gpylint and passed my test suite BUG=None Change-Id: I56fbda7f75fe3bc239e456161c48b7611c3a315d Reviewed-on: https://chrome-internal-review.googlesource.com/150255 Reviewed-by: Luis Lozano <llozano@chromium.org> Commit-Queue: Han Shen <shenhan@google.com> Tested-by: Han Shen <shenhan@google.com>
generated by cgit v1.2.3 (git 2.25.1) at 2024-06-27 15:01:54 +0000