aboutsummaryrefslogtreecommitdiff
path: root/afdo_tools
AgeCommit message (Collapse)Author
2021-10-01update_kernel_afdo: Skip 95 branchDenis Nikitin
Add a list of skipped chromium branches and adjust canary branch number calculation. BUG=None TEST=`./update_kernel_afdo` shows 96 as canary. Change-Id: If96711f5a0d52c128d4829e338b9b950001c6e75 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3199488 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Tested-by: Denis Nikitin <denik@chromium.org> Commit-Queue: Denis Nikitin <denik@chromium.org>
2021-09-21update_kernel_afdo: Remove 3.18Denis Nikitin
Remove 3.18 from the kernel AFDO update. BUG=b:199787442 TEST=./update_kernel_afdo Change-Id: I343d2048fb6bbd9c3c9f37bdfe64526a91c68220 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3162669 Tested-by: Denis Nikitin <denik@chromium.org> Reviewed-by: Bob Haarman <inglorion@chromium.org> Commit-Queue: Denis Nikitin <denik@chromium.org>
2021-09-15Excise cros_logging moduleChris McDonald
The cros_logging module is deprecated and all usages are being removed. The lint referred to by the comment removed in this CL is no longer a thing, so remove this mention of the deprecated module from the tree. BUG=b:191490453 TEST=None Change-Id: Ifcd4cb18e87fdbd3a92747b0fdbac434d51fd055 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/3161649 Tested-by: Chris McDonald <cjmcdonald@chromium.org> Auto-Submit: Chris McDonald <cjmcdonald@chromium.org> Commit-Queue: George Burgess <gbiv@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org>
2021-05-25update_kernel_afdo: Check last week profilesDenis Nikitin
The script handles Monday differently which make it failing when profiles are not uploaded yet. Instead we need to check profiles updated within the last week so we can safely run the script any day given that profiles are updated weekly. BUG=None TEST=./afdo_tools/update_kernel_afdo Change-Id: I15489e8462c0fbceb9a73c3e3bec3027e26b18d4 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2915677 Reviewed-by: George Burgess <gbiv@chromium.org> Commit-Queue: Denis Nikitin <denik@chromium.org> Tested-by: Denis Nikitin <denik@chromium.org>
2021-05-15afdo_tools: Improve update_kernel_afdo scriptDenis Nikitin
By default the script now will create a commit and give a prompt on how to submit the change. Neither sync or clean-up needed to submit the change. The new workflow doesn't touch existing checkout and ignores (and keeps) any local changes. Without arguments the script tries to update metadata in all channels: canary (main), beta and stable. To update afdo metadata in specific channel run: ./update_kernel_afdo canary|beta|stable. BUG=None TEST=afdo_tools/update_kernel_afdo produces 3 commits in release-R92, release-R91 and release-R90 Change-Id: I61c8c743a4634d2ab4e4837f8a31a31a6cea2c2a Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2872860 Tested-by: Denis Nikitin <denik@chromium.org> Reviewed-by: Caroline Tice <cmtice@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org> Commit-Queue: Denis Nikitin <denik@chromium.org>
2021-03-19afdo: Add 5.4 into kernel afdo metadataDenis Nikitin
5.4 kernel afdo generate builder was recently added. Add 5.4 into the list for pupr updates. BUG=b:180725525 TEST=`./afdo_tools/update_kernel_afdo` creates 5.4 data. Change-Id: If43727dbaf0fd6409f043d5978795219b580fcb5 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2774636 Tested-by: Denis Nikitin <denik@chromium.org> Reviewed-by: Caroline Tice <cmtice@chromium.org>
2020-10-22update_kernel_afdo: use gsutil.py instead of gsutilBob Haarman
This changes update_kernel_afdo to call gsutil.py instead of gsutil (without the ".py"). Reason: the non-.py does not exist everywhere, whereas gsutil.py is part of depot_tools. BUG=None TEST=./afdo_tools/update_kernel_afdo Change-Id: I7c2192f5d4ef85852a4c5023edb29e9a092bbb8d Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2490652 Tested-by: Bob Haarman <inglorion@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
2020-09-28afdo_metadata: Update kernel AFDO profilesTiancong Wang
Update chromeos-kernel-3_18 to R87-13482.0-1601286522 Update chromeos-kernel-4_4 to R87-13482.0-1601285703 Update chromeos-kernel-4_14 to R87-13482.0-1601285982 Update chromeos-kernel-4_19 to R87-13482.0-1601286143 Also fixed the script a little bit to supress a warning. BUG=None TEST=None Change-Id: I50be925fd9f598fb834a5b3153f86f29679844c7 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2436058 Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Commit-Queue: Tiancong Wang <tcwang@google.com> Tested-by: Tiancong Wang <tcwang@google.com>
2020-08-25afdo_tools: Update update_kernel_afdo to check previous branchTiancong Wang
When we just branch, and no profiles for the current canary branch can be found, we should check the profiles on previous branch. Also fix non-inclusive language in this patch. BUG=None TEST=kernel_afdo.json changes made locally Change-Id: I0de00274001ee7a120c40163ab93cb35e143acdf Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2376170 Tested-by: Tiancong Wang <tcwang@google.com> Reviewed-by: Bob Haarman <inglorion@chromium.org>
2020-07-28afdo_tools: Update script to sort profiles by timestampTiancong Wang
Our update_kernel_afdo tries to get all the profiles with gsutil. However, the command gives results by sorting the profile name, so the last profile is not necessarily the newest. A counter example is: R86-13310.3-1594633089.gcov.xz appears after R86-13310.18-1595237847.gcov.xz So this patch sorts the results of gsutil by the timestamp and grep all the master profiles (otherwise we might get some beta/stable profiles). BUG=None TEST=Tool can be used to generate kernel_afdo.json CL Change-Id: I9c9f6ee2ab446f5e07ea577e11f0c3d562bff5e3 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2321122 Commit-Queue: Tiancong Wang <tcwang@google.com> Tested-by: Tiancong Wang <tcwang@google.com> Reviewed-by: George Burgess <gbiv@chromium.org>
2020-06-24afdo_tools: add update_kernel_afdo scriptBob Haarman
This adds a script, update_kernel_afdo, which automatically updates afdo_metadata/kernel_afdo.json. The changes are then shown with git diff and can be committed with git commit after review. BUG=None TEST=Ran the script, including with some variations to see the failure modes. Change-Id: Ibe473a80a874e32ddcb9c292e1c018fac72e7292 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2261732 Commit-Queue: Bob Haarman <inglorion@chromium.org> Tested-by: Bob Haarman <inglorion@chromium.org> Reviewed-by: Tiancong Wang <tcwang@google.com>
2020-02-13toolchain-utils: migrate all in-use projects to python 3Zhizhou Yang
This patch migrates all in-use projects left to python 3. BUG=chromium:1011676 TEST=Passed unittests and launched scripts manually. Change-Id: I7f2de4e1131c05bacfac80667f3064da8adaebfd Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2051397 Reviewed-by: George Burgess <gbiv@chromium.org> Tested-by: Zhizhou Yang <zhizhouy@google.com> Auto-Submit: Zhizhou Yang <zhizhouy@google.com>
2019-10-17afdo_bisection: make docs slightly more helpfulGeorge Burgess IV
BUG=None TEST=None Change-Id: I81c38a80d7b84ac4ae65a3e75b232a0131a3f5ce Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1846266 Reviewed-by: Caroline Tice <cmtice@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2019-10-17afdo_bisection: remove absl dependencyGeorge Burgess IV
absl isn't available outside of the chroot, which makes using this script pretty difficult in some cases. The dependency isn't necessary (we only care for flags), and it's probably better if we parameterize functions/objects on the things they depend on, rather than having them reach into FLAGS. BUG=None TEST=Ran the tests Change-Id: Ic535b92ce580071ab5a346cde76f201941a748ee Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1839618 Reviewed-by: Caroline Tice <cmtice@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2019-07-29AFDO-Bisect: Add user documentationEmma Vukelj
This CL adds a README containing some information about what the script does, what the arguments are, and how to invoke it, for user benefit. BUG=None TEST=None Change-Id: I909c75e35545c7049b030fde2d080f6466addabe Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1718885 Reviewed-by: Caroline Tice <cmtice@chromium.org> Tested-by: Emma Vukelj <emmavukelj@google.com>
2019-07-24AFDO-Bisect: Remove separate parsing moduleEmma Vukelj
The initial parsing was written as a separate module imported by the main analysis and the tests. This is pretty unnecessary since it is only one relatively small function, and so this CL consolidates that code into the main functionality (and updates the tests to reflect this). BUG=None TEST=All existing tests succeed. Change-Id: I1d6ac4cf0abdbcdf7b837492580887d08d007bad Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1716324 Reviewed-by: George Burgess <gbiv@chromium.org> Reviewed-by: Caroline Tice <cmtice@chromium.org> Tested-by: Emma Vukelj <emmavukelj@google.com>
2019-07-24AFDO-Bisect: Write test confirming assumptions re state savingEmma Vukelj
This CL adds a test which confirms that the state saving used correctly produces the same exact profiles as if the run went straight through. BUG=None TEST=All tests, new and old, pass. Change-Id: I250bbf051165deeb673dedc61b3c6e6904cf4875 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1713235 Reviewed-by: George Burgess <gbiv@chromium.org> Tested-by: Emma Vukelj <emmavukelj@google.com>
2019-07-22AFDO-Bisect: Exit analysis on problem_status exit codeEmma Vukelj
This CL makes it such that if the user-provided external script returns PROBLEM_STATUS indicating that something went wrong (e.g. user sent SIGINT, or some part of script crashed), then the analysis raises an error stating so and terminates analysis. Receipt of the problem status code is not persisted in the state file. BUG=None TEST=Existing tests and added test confirming error raising all pass. Change-Id: Ie2ca6c608ed01b3717d5f09495660f99c3a5e05a Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1710021 Reviewed-by: Caroline Tice <cmtice@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org> Tested-by: Emma Vukelj <emmavukelj@google.com>
2019-07-19AFDO-Bisect: Implement state-savingEmma Vukelj
This CL adds state-saving functionality to this AFDO profile analysis tool. It introduces two new flags: state_file, and no_resume. The former allows the user to specify a specific state file for initial state-loading, and state-writing on each iteration. If this flag is not provided, a default file in the current working directory is used. The no_resume flag allows the user to disable initial state loading, beginning a fresh run of the script. TEST=All added tests pass BUG=None Change-Id: Ib288180dc6ca15d47e2720f222905e2aa6b964a0 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1708215 Reviewed-by: Caroline Tice <cmtice@chromium.org> Tested-by: Emma Vukelj <emmavukelj@google.com>
2019-07-17AFDO-Bisect: Make range search bisect O(logn) instead of O(n)Emma Vukelj
This CL changes the formerly-titled 'non_bisecting_search' to run faster by using bisection, and has thus been renamed to 'range_search' because it searches for a range of problematic functions. This CL also adds some basic status logging for the user's benefit. Change-Id: Ib37dd4817665f8378bfcc1335a2e82367d73db4f Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1703125 Reviewed-by: George Burgess <gbiv@chromium.org> Tested-by: Emma Vukelj <emmavukelj@google.com>
2019-07-15AFDO-Bisect: Initial draft of analysis algorithmEmma Vukelj
This CL is an initial draft of the AFDO profile analyzer at the heart of this project. It employs three different sorts of analysis (bisecting search, non-bisecting search, and by function diffs). BUG=None TEST=All added tests succeed. Change-Id: If584f30ef5a208c3da71123fe9212d8c3eac89c0 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1687775 Reviewed-by: Caroline Tice <cmtice@chromium.org> Tested-by: Emma Vukelj <emmavukelj@google.com>
2019-06-28AFDO-Bisect: Write a (very) basic AFDO prof parserEmma Vukelj
This CL introduces a script which does very basic parsing of AFDO text-based profiles which associates each top-level function with its associated profile data. BUG=None TEST=The added test is successful. Change-Id: I6c8d3b3b1049b81a5890cc9699e2798dda641eb9 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/1679371 Reviewed-by: Caroline Tice <cmtice@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org> Tested-by: Emma Vukelj <emmavukelj@google.com>
2019-03-28Add AFDO generation scriptsGeorge Burgess IV
People want to generate AFDO profiles, which is a somewhat error-prone and complicated process for a few reasons. So, have two scripts. The first of these allows you to easily generate an AFDO perf profile on a trybot. The second gives you a convenient way to turn that into an AFDO profile. Both have accompanying documentation, so the hope is that they're usable. BUG=None TEST=None Change-Id: I4afb665ffc8550c718b3867ca3d5b98aa4cd0901 Reviewed-on: https://chromium-review.googlesource.com/1526721 Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com> Tested-by: George Burgess <gbiv@chromium.org> Reviewed-by: Manoj Gupta <manojgupta@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org>
generated by cgit v1.2.3 (git 2.25.1) at 2024-06-26 16:48:50 +0000