summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2012-07-17cros_mark_as_stable: always setup stabilization branch when committingMike Frysinger
We need the stabilization branch in order to commit our updated cache files, so always set that branch up even if we don't have any ebuilds to work on. If there ends up not being any work, and we don't update any cache files, it'll just get thrown away. BUG=None TEST=`./buildbot/run_tests` passes TEST=`cros-cbuildbot --local arm-tegra2-paladin` and look at local branches after it finishes Change-Id: I4e5bece8aae6d59ba894f181266ee81f45db8efb Reviewed-on: https://gerrit.chromium.org/gerrit/27530 Reviewed-by: David James <davidjames@chromium.org> Commit-Ready: Mike Frysinger <vapier@chromium.org> Tested-by: Mike Frysinger <vapier@chromium.org>
2012-07-17Don't re-raise StepFailure exceptions in cbuildbot main script.David James
StepFailure exceptions are already recorded in the report and in the stage that failed, so there is no need to print them again. BUG=chromium-os:32596 TEST=Run cbuildbot and verify tracebacks are still printed out, but not quite as many times. Change-Id: Id1c5c537de1151d3f26ff28f50306b3cbd7a45de Reviewed-on: https://gerrit.chromium.org/gerrit/27460 Commit-Ready: David James <davidjames@chromium.org> Reviewed-by: David James <davidjames@chromium.org> Tested-by: David James <davidjames@chromium.org>
2012-07-16Remove unused arm chrome tot informational.Peter Mayo
BUG=None TEST=unittest Change-Id: Id688b1220924d57a24e7ad8ec6c8fd432b2438c3 Reviewed-on: https://gerrit.chromium.org/gerrit/27477 Reviewed-by: Mike Frysinger <vapier@chromium.org> Reviewed-by: David James <davidjames@chromium.org> Commit-Ready: Peter Mayo <petermayo@chromium.org> Tested-by: Peter Mayo <petermayo@chromium.org>
2012-07-16Revert "cbuildbot: force host bzip2 for now"Mike Frysinger
This reverts commit fcf7527ae51fdb4e8c9457d8a8e231bad723f44d. Now that pbzip2 inside the chroot is statically linked, we can execute it directly from outside the chroot and be confident that it'll work. BUG=chromium-os:32519 TEST=`cbuildbot arm-generic-paladin` works Change-Id: I26e9b0d60606cb9fd74e19e1b57d32a89c982e38 Reviewed-on: https://gerrit.chromium.org/gerrit/27539 Reviewed-by: David James <davidjames@chromium.org> Commit-Ready: Mike Frysinger <vapier@chromium.org> Tested-by: Mike Frysinger <vapier@chromium.org>
2012-07-16Fix GetProjectsLocalRevision api to be less dangerous to use.Brian Harring
Specifically, all consumers of this have always had to protect themselves against the manifest being revision locked; there's no reason to make each consumer of this have to use the same code, thus just fix the api. BUG=None TEST=manual Change-Id: I21d55e069968653aebe5dfe1d59ae5bfb3a089af Reviewed-on: https://gerrit.chromium.org/gerrit/27220 Commit-Ready: Brian Harring <ferringb@chromium.org> Reviewed-by: Brian Harring <ferringb@chromium.org> Tested-by: Brian Harring <ferringb@chromium.org>
2012-07-16osutils: add testsMike Frysinger
BUG=None TEST=`./osutils_unittest.py` passes Change-Id: I09bd959faec81b7a0d75ae376dc1f1e03d54ba7d Reviewed-on: https://gerrit.chromium.org/gerrit/26242 Commit-Ready: Mike Frysinger <vapier@chromium.org> Reviewed-by: Mike Frysinger <vapier@chromium.org> Tested-by: Mike Frysinger <vapier@chromium.org> Reviewed-by: David James <davidjames@chromium.org>
2012-07-16Treat warnings the same as failures in builderstage.David James
Right now, tracebacks for StepFailures are suppressed when they are errors, but not when they are warnings. We should treat them consistently. BUG=chromium-os:32596 TEST=Run cbuildbot and verify tracebacks are still printed out, but not quite as many times. Change-Id: I38d2e2acda40270203665f188b5a197613109d38 Reviewed-on: https://gerrit.chromium.org/gerrit/27459 Reviewed-by: Brian Harring <ferringb@chromium.org> Commit-Ready: David James <davidjames@chromium.org> Tested-by: David James <davidjames@chromium.org>
2012-07-16Calculate what packages broke the build in the commit queue.David James
This change teaches the commit queue to calculate what packages broke the build, and to not kick out changes that we think are innocent. We use a fairly simplistic algorithm to calculate breakage: If you changed a package, and that package broke, you probably broke the build. If there were multiple changes to a broken package, we fail them all. Some safeguards are implemented to ensure that bad changes are kicked out: 1) Changes to overlays (e.g. ebuilds, eclasses, etc.) are always kicked out if the build fails. 2) If a package fails that nobody changed, we kick out all of the changes. 3) If any failures occur that we can't explain, we kick out all of the changes. It is certainly possible to trick this algorithm: If one developer submits a change to libchromeos that breaks the power_manager, and another developer submits a change to the power_manager at the same time, only the power_manager change will be kicked out. That said, in that situation, the libchromeos change will likely be kicked out on the next run, thanks to safeguard #2 above. BUG=chromium-os:32249 TEST=Run remote trybot runs with example changes to make sure the right changes are always kicked out. New unit tests. Pylint. Change-Id: Ie1b50f36d3164b5ab56b35403569132dabe4d841 Reviewed-on: https://gerrit.chromium.org/gerrit/27444 Commit-Ready: David James <davidjames@chromium.org> Reviewed-by: David James <davidjames@chromium.org> Tested-by: David James <davidjames@chromium.org> Reviewed-by: Brian Harring <ferringb@chromium.org>
2012-07-15Remove deprecated builders.David James
ironhide is no longer supported, and the board has been removed, so remove the builders. Also remove deprecated corei7 informational builder. BUG=none TEST=cbuildbot_config_unittest.py Change-Id: I21350a58a7b466917fc09b9b2e96d5daeac413b5 Reviewed-on: https://gerrit.chromium.org/gerrit/27456 Tested-by: David James <davidjames@chromium.org> Reviewed-by: Peter Mayo <petermayo@chromium.org> Reviewed-by: Mike Frysinger <vapier@chromium.org> Commit-Ready: Peter Mayo <petermayo@chromium.org>
2012-07-13Use cPickle to pass messages between slaves and the master.David James
Instead of using raw strings, we should use cPickle to pass messages betwen the slaves and the master. In future, this will allow us to attach additional information to the messages such as what packages failed. BUG=chromium-os:32249 TEST=Verify with remote trybot that it still reads the right message from itself when a build fails. Change-Id: I7736b2298d10a01dd6960e77dbe84f557da6b3d1 Reviewed-on: https://gerrit.chromium.org/gerrit/27422 Reviewed-by: Chris Sosa <sosa@chromium.org> Tested-by: David James <davidjames@chromium.org> Commit-Ready: David James <davidjames@chromium.org>
2012-07-13Report all failures when the CQ fails, not just the first one.David James
When multiple stages of a build fail, we currently only report the first traceback and not the subsequent ones. This patch updates cbuildbot to report all the tracebacks that occur rather than just the first one. BUG=chromium-os:32249 TEST=Run remote trybot jobs with example build breaks and verify exceptions are correct. Unit tests. Pylint. Change-Id: Icd6bb96fd6cb87e68f2564e0324a67f0180bd197 Reviewed-on: https://gerrit.chromium.org/gerrit/27405 Commit-Ready: David James <davidjames@chromium.org> Reviewed-by: David James <davidjames@chromium.org> Tested-by: David James <davidjames@chromium.org>
2012-07-13Move all sending of failure messages to the master.Chris Sosa
This change modifies the CommitQueueCompletion logic to send one large message with an aggregate of all failed builds for a given pool rather than have each sub-builder separately send messages on each change. This has two parts: 1) This change modifies the UpdateStatus protocol to include a message along with the failure symlink when a build fails for a particular builder. This message is included in a text file that is stored along with the symlink in the corresponding inflight/failed/passed subdirectory in manfiest versions. 2) This changes also modifies the CommitQueueCompletion.HandleValidationFailure logic to gather failed status messages and send one aggregate message to each CL in a bad pool. This required splitting HandleValidationFailure logic in to two components in validation_pool.py -- one to generate the message for a given builder, and one to send a message + -1 on the change. This change is not ready as it missing testing. I want some early feedback on the approach while I work on the testing -- BUG=chromium-os:32218, chromium-os:31561 TEST=Unittests + pylint on all changed modules + tryjob with mario-paladin and observated commit message. Change-Id: I50bdb1340c9656e75a8a7de876a48d927dd1d99e Reviewed-on: https://gerrit.chromium.org/gerrit/26653 Tested-by: Chris Sosa <sosa@chromium.org> Reviewed-by: Chris Sosa <sosa@chromium.org> Commit-Ready: David James <davidjames@chromium.org>
2012-07-13Add --gerrit-chrome option.Ryan Cui
Enables cbuildbot to build Chrome from GERRIT_SOURCE. Contains safety checks so this is not used on remote trybots, since chrome_set_ver can potentially put the repository in a bad state. BUG=chromium-os:32456 TEST=Ran local trybot Change-Id: I6521d5b66df3aa022fa0e51744a9bf5285cf1882 Reviewed-on: https://gerrit.chromium.org/gerrit/27083 Commit-Ready: Ryan Cui <rcui@chromium.org> Reviewed-by: Ryan Cui <rcui@chromium.org> Tested-by: Ryan Cui <rcui@chromium.org>
2012-07-13Move hidden options together.Ryan Cui
BUG=None TEST=Ran cbuildbot Change-Id: Ie88e8845a601a9541c0a281089e7056463d1bcb2 Reviewed-on: https://gerrit.chromium.org/gerrit/27001 Commit-Ready: Ryan Cui <rcui@chromium.org> Reviewed-by: Ryan Cui <rcui@chromium.org> Tested-by: Ryan Cui <rcui@chromium.org>
2012-07-13Move new arm configurations to standard naming convention.Chris Sosa
For all other type of builds we've removed the arm-* suffix. Doing this also for canaries. I'm leaving kaen alone as it's been like that for a long time. BUG=None TEST=unittests Change-Id: Ib95946932b2b32e2de5ea1173455c7cf7b023f5f Reviewed-on: https://gerrit.chromium.org/gerrit/27368 Tested-by: Chris Sosa <sosa@chromium.org> Reviewed-by: Chris Masone <cmasone@chromium.org>
2012-07-12Only enable chrome_tests on amd64-generic-tot-chrome-pfq-informationalDavid James
1. chrome_tests doesn't actually run on builders where vm_tests is off. Previously, these tests were "enabled" on many builders, but they only actually ran on amd64-generic-tot-chrome-pfq-informational. I've updated the config to make this situation more clear. 2. vm_tests was disabled on amd64-generic-chromium-pfq because that builder previously was on a VM and could not run VM tests. Now that it's on a real builder, I've re-enabled VM tests. 3. I've also added chrome_tests=False in the arm config so that if we end up turning on chrome_tests in general, they won't get automatically turned on in arm configs where vm_tests are off. BUG=chromium-os:32524 TEST=Added unit test, and ran the unit test to verify. Change-Id: I49995636088c1f9fc4852aef26dce826af96f77a Reviewed-on: https://gerrit.chromium.org/gerrit/27310 Tested-by: David James <davidjames@chromium.org> Reviewed-by: Chris Sosa <sosa@chromium.org> Commit-Ready: David James <davidjames@chromium.org>
2012-07-12cbuildbot: force host bzip2 for nowMike Frysinger
Executing a dynamically linked binary inside of the chroot is extremely fragile. It assumes all libraries inside the chroot are not newer than the libraries outside of the chroot. With the recent glibc 2.15 upgrade (inside the chroot), this broke a lot of bots: /b/cbuild/chroot/usr/bin/pbzip2: /lib/libc.so.6: version `GLIBC_2.14' not found (required by /b/cbuild/chroot/usr/bin/pbzip2) Once we update pbzip2 to be statically linked, and we get updated bots, we can revert this change. BUG=chromium-os:32519 TEST=`./buildbot/run_tests` passes Change-Id: I7999e47a7039016085b6605ca60e9e872cf431ea Reviewed-on: https://gerrit.chromium.org/gerrit/27280 Commit-Ready: Mike Frysinger <vapier@chromium.org> Reviewed-by: Mike Frysinger <vapier@chromium.org> Tested-by: Mike Frysinger <vapier@chromium.org>
2012-07-11This change broke the canaries: ↵Chris Sosa
http://chromegw/i/chromeos/builders/x86-zgb%20canary/builds/2086/steps/HWTest%20%5Bbvt%5D%20%5Bx86-zgb%5D/logs/stdio Revert "Upload full autotest tarball only when necessary" This reverts commit 42ded592291c2d893aa8ec4fdcf6815386d23efd Change-Id: I00b497ebacf9d5b2fba39c6516b7da565fa65cbf Reviewed-on: https://gerrit.chromium.org/gerrit/27239 Reviewed-by: Chris Sosa <sosa@chromium.org> Tested-by: Chris Sosa <sosa@chromium.org>
2012-07-11Make --debug a remote option.Ryan Cui
So that --pass-through='--debug' is not necessary for running debug --buildbot runs on remote trybot. BUG=None TEST=Ran trybot Change-Id: Ic1f88061e70ec874617e41ddc1f4a2d93522d2ad Reviewed-on: https://gerrit.chromium.org/gerrit/27206 Reviewed-by: David James <davidjames@chromium.org> Commit-Ready: Ryan Cui <rcui@chromium.org> Tested-by: Ryan Cui <rcui@chromium.org>
2012-07-11Add check to ensure that the patch object is in fact a patch object.Brian Harring
BUG=chromium-os:27415,chromium-os:29546,chromium-os:30042 TEST=unittests Change-Id: If3b21632f3e766b8fbe56c0476c98253c2b4f585 Reviewed-on: https://gerrit.chromium.org/gerrit/27210 Commit-Ready: Brian Harring <ferringb@chromium.org> Tested-by: Brian Harring <ferringb@chromium.org> Reviewed-by: David James <davidjames@chromium.org>
2012-07-11Upload full autotest tarball only when necessaryYu-Ju Hong
The full autotest tarball (autotest.tar.bz2) is only needed for chromeos_offical builds to build the hwqual image. This change generates a full tarball only for these builds and archives the tarball locally. This saves the build time for non chromoes_official builds and eliminats unnecessary download/upload of the full tarball. In addition, the autotest tarballs are no longer included in image.zip to reduce uncessary waste of space/time. Note that the parital autotest tarball (autotest.tar) required for hw testing is not affected in this change. BUG=chromium-os:32431 TEST=unittest & remote trybot + hw test Change-Id: I6a8c88c0ab7107f2bdfde560f2aa71b588cc919f Reviewed-on: https://gerrit.chromium.org/gerrit/27193 Tested-by: Yu-Ju Hong <yjhong@chromium.org> Reviewed-by: Chris Sosa <sosa@chromium.org> Commit-Ready: Yu-Ju Hong <yjhong@chromium.org>
2012-07-11Enhance patch/cq guts to partially support cross gerrit dependencies.Brian Harring
Specifically, this enables internal->external CQ-DEPEND; technically it enables both, but the external slave paladins don't know which internal patches are in that pool, thus an external->internal CQ-DEPEND would fail on external builders. That deficiency will be sorted in a later patch. While this was originally intended to be done post cherry-picking, cherry-picking requires cbuildbot/cq unification, and CQ's internal issues in re: to tracking internal/external are exasperated by non GerritPatch instances being handed in (prior to this change, effectively GerritPatch was the only instance w/ internal tracking). While that known CQ issue had tried to be sidestepped to delay this specific CL/work, it's not looking sanely viable thus this patch being moved forward as a dep of unification/cherry-picking. Via this change, patch instances switch their .id attribute over to using our internal/external change_id form; specifically internal changes are prefixed with '*'. This is threaded throughout the guts of GitRepoPatch instances- including GerritDependencies. Due to this switch, our internal ChangeId formatting/validation code (patch.FormatChangeId specifically) is beefed up to do much more validation. Via the previous refactoring (I2394e9970ce0de829f01959), CQ guts rely on the internal .id attribute rather than the raw change_id, so the machinery is already in place to support that side of it; CQ has to modified however to remove earlier restrictions controlling where it looks for changes, allowing it to look at both gerrit instances. Finally, tests were tightened and expanded as necessary for this. BUG=chromium-os:27415,chromium-os:29546,chromium-os:29330 TEST=unittests TEST=cbuildbot mario-paladin --buildbot --remote --pass-through=--debug TEST=cbuildbot x86-generic-paladin --buildbot --debug <args> Change-Id: I01e53f6c13291006e14b5d15a95f41a801a762a7 Reviewed-on: https://gerrit.chromium.org/gerrit/26999 Reviewed-by: Brian Harring <ferringb@chromium.org> Tested-by: Brian Harring <ferringb@chromium.org> Commit-Ready: Brian Harring <ferringb@chromium.org>
2012-07-11cros_mirror: add missing shift to -j flagMike Frysinger
BUG=None TEST=`cros_mirror -j 13` doesn't error Change-Id: Id0960566554d7da4827e617afb86653b1092113d Reviewed-on: https://gerrit.chromium.org/gerrit/27138 Reviewed-by: Hung-Te Lin <hungte@chromium.org> Commit-Ready: Mike Frysinger <vapier@chromium.org> Tested-by: Mike Frysinger <vapier@chromium.org>
2012-07-10do not use binpkgs when using a fresh sdkMike Frysinger
When using the freshly created sdk, we do not want to force binpkgs when trying to update chroot packages because the binpkgs don't yet exist -- they're created by this step. Otherwise, we get into the situation where the bootstrapped sdk installs a newer package which is then downgraded when we attempt to use it for the first time. BUG=chromium-os:32354 TEST=`./buildbot/run_tests` passed TEST=`cbuildbot chromiumos-sdk` worked Change-Id: I223c03379f9e34d59e8a48e1ee94a2b3964c57ac Reviewed-on: https://gerrit.chromium.org/gerrit/27028 Reviewed-by: Brian Harring <ferringb@chromium.org> Tested-by: Mike Frysinger <vapier@chromium.org> Reviewed-by: Zdenek Behan <zbehan@chromium.org> Commit-Ready: Mike Frysinger <vapier@chromium.org>
2012-07-10Catch and suppress git clone exit code 128 (serverside error).Brian Harring
This occurs via gerrit and manifest-versions; specifically cloning when a git gc --auto fires serverside. This is basically a gerrit bug, triggered via manifest-version design flaws. BUG=chromium-os:32377,chromium-os:32440 TEST=n/a, requires triggering a server exception for it to be testable. Change-Id: I50ac1070678f1dff6cdc24510ff1f9e30c723c2a Reviewed-on: https://gerrit.chromium.org/gerrit/27042 Tested-by: Brian Harring <ferringb@chromium.org> Reviewed-by: Chris Sosa <sosa@chromium.org> Commit-Ready: Brian Harring <ferringb@chromium.org>
2012-07-10Fix no control files in autotest.tarYu-Ju Hong
This change fixes the bug that caused no control files were found in autotest.tar. Instead of passing only a target directory name to FindfilesWithPattern(), this change also passes the absolute path to the target directory. FindfilesWithPattern() changes its working directory to the absolute path before searching. BUG=chromium-os:32312 TEST=trybot Change-Id: I00fd30c6fed0e8beae4908d0f2df714f696a08d9 Reviewed-on: https://gerrit.chromium.org/gerrit/27050 Reviewed-by: Chris Sosa <sosa@chromium.org> Commit-Ready: Yu-Ju Hong <yjhong@chromium.org> Tested-by: Yu-Ju Hong <yjhong@chromium.org>
2012-07-10Call cros_workon from src/scripts.David James
cros_list_modified_packages runs in early phases of the build, so it needs to be robust to cros-devutils not being installed. This is the case for two reasons: 1) If you're in early stages of the chromiumos-sdk builder, cros-devutils won't be installed. 2) If you're on a preflight builder, the uprev stage may uninstall cros-devutils. BUG=chromium-os:31952 TEST=Local run of script. Remote trybot run. Search all repositories for references to CROSUTILS_LIB_DIR (which is removed here). Change-Id: I8beea00530aec47397a5ba11c80b08620a93c9dd Reviewed-on: https://gerrit.chromium.org/gerrit/27044 Tested-by: David James <davidjames@chromium.org> Reviewed-by: Chris Sosa <sosa@chromium.org> Commit-Ready: David James <davidjames@chromium.org>
2012-07-10Delete unused functions and scripts from chromite.David James
The functions deleted by this CL are not currently used, and many of them do not actually work. The following functions are deleted: - cros_build_lib.GetSrcRoot - cros_build_lib.GetOutputImageDir - cros_build_lib.IsProjectInternal The following files are deleted: - {bin,scripts}/cros_build_packages - lib/cros_check_output.py - lib/cros_subprocess.py - lib/text_menu.py - tests/build_image_test.py I've grepped through the source to make sure these files are not referenced anywhere. BUG=chromium-os:32443 TEST=Run all unit tests. Remote trybot runs. Change-Id: Ibdea2aa17fdb9956f59cdb63c77f633429cc3db4 Reviewed-on: https://gerrit.chromium.org/gerrit/27049 Tested-by: David James <davidjames@chromium.org> Reviewed-by: Chris Sosa <sosa@chromium.org> Commit-Ready: David James <davidjames@chromium.org>
2012-07-10Remove deadline for --local flag requirement.Ryan Cui
BUG=chromium-os:32378 TEST=Ran locally Change-Id: Ideb6e07460749f26866f9c3245b1b180dc80dc4e Reviewed-on: https://gerrit.chromium.org/gerrit/26976 Reviewed-by: Brian Harring <ferringb@chromium.org> Commit-Ready: Ryan Cui <rcui@chromium.org> Tested-by: Ryan Cui <rcui@chromium.org>
2012-07-10Track what packages failed and include it in exceptions.David James
It is helpful to track what packages failed so that we can include this information in messages from the commit queue. This will help developers realize which change caused a particular failure. Related refactoring changes: - Rename NonBacktraceBuildException to StepFailure, and move it to cbuildbot_results.py - Rename TestException to TestFailure, and make it inherit from StepFailure, since these exceptions always fail the step, and we don't want to print detailed tracebacks for them. - Rename BackgroundException to BackgroundFailure for consistency with other exceptions in the StepFailure hierarchy. BUG=chromium-os:32249 TEST=Run remote trybot jobs with example build breaks and verify exceptions are correct. Unit tests. Pylint. Change-Id: Icb983797aaf9827288d978dac38118a2c78f7096 Reviewed-on: https://gerrit.chromium.org/gerrit/26744 Commit-Ready: David James <davidjames@chromium.org> Reviewed-by: David James <davidjames@chromium.org> Tested-by: David James <davidjames@chromium.org>
2012-07-09Revert "cbuildbot: Use CQ logic for patch resolution/applying."Brian Harring
This reverts commit 11272651bfb62a63320d6bc39e05f590c84bc86e and 5a0de704cd175886e1e98d099c3d016bd58a9f4d . 3 Bugs in total have surfaced; one spotted shortly after, 2 others via remote; 2/3 are fixed, but production isn't the place to sort it out. BUG=None TEST=None Change-Id: I372825097b536ebdf305be200c6a1057a1142a29 Reviewed-on: https://gerrit.chromium.org/gerrit/26995 Reviewed-by: Brian Harring <ferringb@chromium.org> Tested-by: Brian Harring <ferringb@chromium.org>
2012-07-09Fix bugs introduced via Icd646d198721a916fd81effd31Brian Harring
1) Make PatchSeries handle revlocked manifests correctly. 2) Handle dumb-ass AttributeError w/in conflicts handling. BUG=None TEST=manul validation Change-Id: I865114e1d58a874454b46ac1a900d8af9abcd85f Reviewed-on: https://gerrit.chromium.org/gerrit/26993 Reviewed-by: David James <davidjames@chromium.org> Reviewed-by: Brian Harring <ferringb@chromium.org> Tested-by: Brian Harring <ferringb@chromium.org>
2012-07-09Execute strip_package from /usr/binRyan Cui
BUG=chromium-os:32214 TEST=remote trybot CQ-DEPEND=I708b5667bd96fbf56ec705e743cfa8a7d26052ae, I5e3f5ec0e4d81294d4701c297bcd23a8cb766761 Change-Id: Iadb750ce98c9cca77c506a33e4f7b64f476d473a Reviewed-on: https://gerrit.chromium.org/gerrit/26479 Commit-Ready: Ryan Cui <rcui@chromium.org> Reviewed-by: Ryan Cui <rcui@chromium.org> Tested-by: Ryan Cui <rcui@chromium.org>
2012-07-09Revert "Use cherry-pick rather than rebase."Brian Harring
This reverts commit 520291fe6c3f80ba133368f362771d48790cc98b Pulling the change; internal CQ's all just went bang. Shouldn't be caused by this, but need to verify. Change-Id: I693d5d5b98a0fd4705e1c4b61dd8d78465b8a1f0 Reviewed-on: https://gerrit.chromium.org/gerrit/26977 Reviewed-by: Brian Harring <ferringb@chromium.org> Tested-by: Brian Harring <ferringb@chromium.org>
2012-07-09Use cherry-pick rather than rebase.Brian Harring
Cherry-pick supports --strategy, and is a heck of a lot simpler than the rebase attempt here, and is *far* *far* friendlier to users' patches; we should no longer have failures in perfectly valid sequences of changes that apply cleanly, but rebase uncleanly. Via using cherry-pick, we /know/ exactly what we're getting, and what we're testing (rather than accidentally pulling in other revs via relying on rebase). Since we now operate at a finer granularity for patches, this rev suppresses a fairly large amount of general git noise that was output during apply invocations, instead summarizing it on it's own; specifically, if it can, it'll state which files were conflicting, if the cause was due to trivial, if the patch is already merged (thus is the cause). Note that since we're supressing the underlying git noise, we include (if available) the subject line of the patch. BUG=chromium-os:27415, chromium-os:30039 BUG=chromium-os:30042, chromium-os:30043 TEST=cbuildbot -g conflicting cls target TEST=cbuildbot -g change-that-conflicts-with-ToT Change-Id: Icd646d198721a916fd81effd31df30a70718210f Reviewed-on: https://gerrit.chromium.org/gerrit/20506 Commit-Ready: Brian Harring <ferringb@chromium.org> Reviewed-by: Brian Harring <ferringb@chromium.org> Tested-by: Brian Harring <ferringb@chromium.org> Reviewed-by: Matt Tennant <mtennant@chromium.org>
2012-07-09cbuildbot: Use CQ logic for patch resolution/applying.Brian Harring
Beyond dropping redundancy, this makes testing CQ patch application behaviour much easier- the core chunk of it is directly accessible via cbuildbot interface. In the process this expands cbuildbot support to automatically pull in paladin deps, and gerrit deps. This has the same flaw for bootstrap testing where a CQ-DEPEND that pulls in a chromite change isn't bootstrapped; this is outside this changes purview to fix however. Finally... this is required before we can turn on cherry-picking mode; specifically we need require the gerrit dep resolution angle to fully pull in the necessary parents. Note that this again break the pickle support for ValidationPool against current ToT; this is known (flagged by tests), but is required since the build root needs to be threaded down through the invoking code. BUG=chromium-os:27415,chromium-os:29546 TEST=checkout this patch, and run: cbuildbot x86-generic-full -g I60e1112c849f149f5c5ae0cc1ca6bd752bab72fd TEST=checkout this patch, and run: cbuildbot x86-generic-full -g I4325b5cf8ba0d48b21da9ae9043f95ec917c751f TEST=unittests, pylint, etc. Change-Id: I60e1112c849f149f5c5ae0cc1ca6bd752bab72fd Reviewed-on: https://gerrit.chromium.org/gerrit/20505 Reviewed-by: Brian Harring <ferringb@chromium.org> Tested-by: Brian Harring <ferringb@chromium.org>
2012-07-09Fix broken unit tests in cbuildbot_stages_unittest.pyYu-Ju Hong
This change use both autotest.tar and autotest.tar.bz2 in the BuildTargetStage tests. BUG=chromium-os:32417 TEST=unittest Change-Id: Ibbd73bda99af253e421f05a11b2c6c8060fb88ef Reviewed-on: https://gerrit.chromium.org/gerrit/26935 Commit-Ready: Yu-Ju Hong <yjhong@chromium.org> Tested-by: Yu-Ju Hong <yjhong@chromium.org> Reviewed-by: David James <davidjames@chromium.org>
2012-07-05Refactor CQ logic around the concept of transactions.Brian Harring
Previous CQ behaviour basically was optimistic, but violated some of the constraints it aimed for- for example, CQ-DEPEND cycles weren't strictly enforced (two nodes, if one applied, one didn't, the applied one would be left in place). The code is now broken down into two phases; resolution of each desired transaction, and application of each transaction, preferring the transaction pulling in the most revs (working it's way down to the smaller transactions- the notion being that long chains are more of a pain to land, thus give them preference over one offs that can quickly be rebased). For the application phase, if a resolved transaction fails to apply, the repo is *fully* restored to it's original state. Effectively, all or none for the commits in that transaction. Since we generate overlapping transactions, this isn't an issue for CQ mode- it'll pick up the sub-transaction w/in the greater one, trying to apply that (or skipping the sub-transaction if the greater transaction landed). In this way, we can ensure that all constraints for a change landing are properly honored. W/in the resolution phases, we have two modes of operation; frozen, and non-frozen. Frozen is long term CQ behaviour (which is preserved); only work from the patch pool it knows of. For non frozen, this allows this code to arbitrary resolve gerrit/paladin dependencies, querying as necessary. This is required for I60e1112c849f149f5c5ae0cc1ca6bd752bab72fd (which is a required step to enable cherry-picking via Icd646d198721a916fd81effd31df30a70718210f). Via the transaction support, it becomes possible for a patch to be tried multiple times via separate transactions; this unfortunately required a full overhaul of the error reporting for CQ; now, instead of assigning messages onto the patch object itself (which could be involved in multiple transactions, differing ways of being blocked from being submitted), exceptions are thrown, chained together with the causes of each step accessible. That change unfortunately means that this patch cannot go through CQ itself- it implicitly changes the pickle format for the 'conflicting_changes' argument of ValidationPool, and cannot be done in a compatible fashion. As such testToTCompatibility is known to fail w/ this CL (it'll work after the CL is in ToT), and to actually land this CL, we'll have to chump it in. Is what it is frankly; there isn't a way around it in this case. Finally, tests were heavily refactored, resulting in some minor API shifts (renames for consistancy), and chunking the tests up into logical groups of tests making it easier for debugging- one starts with TestPatchSeries, then to TestCoreLogic, then to user visible tests. This was done since the ordering mentioned above is literal- if PatchSeries is broken, everything descending from it is going to misbehave. BUG=chromium-os:27415, chromium-os:27256, chromium-os:29546 TEST=cbuildbot --remote x86-generic-paladin run TEST=run_tests.sh; note that validation_pool_unittest.py fails testToTCompatibility; per message above, this is unavoidable. Change-Id: I4325b5cf8ba0d48b21da9ae9043f95ec917c751f Reviewed-on: https://gerrit.chromium.org/gerrit/20811 Tested-by: Brian Harring <ferringb@chromium.org> Reviewed-by: David James <davidjames@chromium.org>
2012-07-03Tar only the necessary files for autotestYu-Ju Hong
Tar only the control files and the packages in the autotest tarball. Keep both tarred-only (autotest.tar) and zipped (autotest.tar.bz2) for backward compatibiliy. The zipped tarball is to be removed after all other changes are done. BUG=chromium-os:32312 TEST=remote trybot + hw testing Change-Id: I1cd0e3c029cfd0a9fa27f981b68ba2958c94dec1 Reviewed-on: https://gerrit.chromium.org/gerrit/26612 Commit-Ready: Yu-Ju Hong <yjhong@chromium.org> Tested-by: Yu-Ju Hong <yjhong@chromium.org> Reviewed-by: Yu-Ju Hong <yjhong@chromium.org>
2012-07-03Fix GerritDependencies when invoked for the first commit in a repository.Brian Harring
BUG=None TEST=updated patch_unittest.py Change-Id: I95b821b5c4da142373a3c6539591001cf9d84bb3 Reviewed-on: https://gerrit.chromium.org/gerrit/26733 Commit-Ready: Brian Harring <ferringb@chromium.org> Tested-by: Brian Harring <ferringb@chromium.org> Reviewed-by: David James <davidjames@chromium.org>
2012-07-03Add mox awareness to ValidationPool to make test debugging simpler.Brian Harring
BUG=None TEST=Induced a TB, note the actual TB comes through rather than mox complaints. Change-Id: Id5251d4110150d134dfead6dc61e6305a4a5313e Reviewed-on: https://gerrit.chromium.org/gerrit/24355 Reviewed-by: David James <davidjames@chromium.org> Commit-Ready: Brian Harring <ferringb@chromium.org> Tested-by: Brian Harring <ferringb@chromium.org>
2012-07-03Fix pylint error, update docstring.Brian Harring
For push_url, this is obviously dead code- else it would've failed by now. Thus removing it in full. BUG=None TEST=unittest's, remote submission. Change-Id: I6dc5a79c8a5f125000628280b96288c18ee32161 Reviewed-on: https://gerrit.chromium.org/gerrit/26667 Commit-Ready: Brian Harring <ferringb@chromium.org> Tested-by: Brian Harring <ferringb@chromium.org> Reviewed-by: Matt Tennant <mtennant@chromium.org>
2012-07-03Update all existing projects alternates, regardless of the manifest.Brian Harring
In transitioning across manifest changes, if the manifest has been changed outside repos control, in conjunction with repo reference, it's possible for repo sync to fail hard via our alternate machinery. Specifically if the manifest changes (dropping a project), we rewrite the alternate configuration dropping the alternate configuration for that project- in certain cases this can result in the projects git repo to be in a truly unusable state (essentially an empty repo but with enough git bits in place that git can't work its way out of it)- further, a state that repo can't recover from either. Thus rework the alternates machinery to instead bypass the manifest in full, instead looking at the underlying git configuration and using that (instead of repo list). This fixes the bug described above, and ultimately is a more correct solution for N>2 repo's referencing each other- in A->B->C, it's possible that A and C have a manifest using a project that B used prior- the git objects are still there, thus we should use B, in addition to C. Finally, two additional changes: 1) fix pylint complaints 2) add local_manifest.xml awareness for the cache. BUG=chromium-os:32285 TEST=manual; external tree, cbuildbot x86-generic-paladin --buildbot \ -r /path/way --sourceroot /invoking/tree --debug # Will replicate # it; note the sourceroot is critical, else referncing won't # occur. Change-Id: Ie2f40733455fd75d6e88c11c5859f12422124bcd Reviewed-on: https://gerrit.chromium.org/gerrit/26562 Commit-Ready: Brian Harring <ferringb@chromium.org> Reviewed-by: Brian Harring <ferringb@chromium.org> Tested-by: Brian Harring <ferringb@chromium.org>
2012-07-03Revert "Add the HWTest step to Link canary target."Scott Zawalski
Reverting for now. There are some bugs that require manual intervention for this target at the moment. This reverts commit 7cff380c849c3075dccf15b7541d8a79774449a4 Change-Id: If62bff43b5ecf4ab6f32e6c7dbb5a21da0d84186 Reviewed-on: https://gerrit.chromium.org/gerrit/26699 Commit-Ready: Scott Zawalski <scottz@chromium.org> Reviewed-by: Scott Zawalski <scottz@chromium.org> Tested-by: Scott Zawalski <scottz@chromium.org>
2012-07-03Remove dependency map from failures.David James
Dependency maps aren't very useful anymore for debugging failures, so don't show them anymore. This makes the failure output more terse so that the output will focus on the package that failed. BUG=none TEST=Confirm dependency maps aren't printed anymore when parallel_emerge fails. Change-Id: Ic8f02c51380b9f15be0cf4a1e711f8c580b101ff Reviewed-on: https://gerrit.chromium.org/gerrit/26543 Reviewed-by: Mike Frysinger <vapier@chromium.org> Commit-Ready: David James <davidjames@chromium.org> Tested-by: David James <davidjames@chromium.org>
2012-07-03Fix chrome_set_ver.py w.r.t. removal of cros.DEPSBen Chan
All dependencies in cros.DEPS have been moved to Chromium's main DEPS. This CL updates chrome_set_ver accordingly. BUG=chromium-os:31867 TEST=cros_workon_make chromeos-chrome Change-Id: I4384e6ae9a85410070bdfe1b4a802830e6b1acd5 Reviewed-on: https://gerrit.chromium.org/gerrit/26614 Reviewed-by: John Sheu <sheu@chromium.org> Tested-by: John Sheu <sheu@chromium.org> Reviewed-by: David James <davidjames@chromium.org> Commit-Ready: Ben Chan <benchan@chromium.org> Tested-by: Ben Chan <benchan@chromium.org>
2012-07-02Add the HWTest step to Link canary target.Scott Zawalski
TEST=Ran bvt and also cbuildbot_unittests. BUG=chrome-os-partner:9209 Change-Id: I4a145829b613f296ab3c1e196cfce4da34fea3c8 Reviewed-on: https://gerrit.chromium.org/gerrit/26646 Reviewed-by: Chris Sosa <sosa@chromium.org> Commit-Ready: Scott Zawalski <scottz@chromium.org> Reviewed-by: Scott Zawalski <scottz@chromium.org> Tested-by: Scott Zawalski <scottz@chromium.org>
2012-07-02Add exception info to failed commit queue runs.David James
If a commit queue run failed, comment on the commits with a message explaining exactly which stage failed and why. BUG=chromium-os:32199 TEST=Run example commit queue runs on trybot waterfall. Verify the commands run by buildbot actually work. Verify all unit tests pass. Run pylint on all modified files. Change-Id: If32437f57e8a1d62b8c82bf6e350781fbc8c5623 Reviewed-on: https://gerrit.chromium.org/gerrit/26546 Tested-by: David James <davidjames@chromium.org> Reviewed-by: Chris Sosa <sosa@chromium.org> Reviewed-by: Brian Harring <ferringb@chromium.org> Commit-Ready: David James <davidjames@chromium.org>
2012-07-02Add hw test as an option for remote trybotYu-Ju Hong
User can specify "--hwtest" to turn on hw testing for remote trybot. The feature is off by default. This change passes 'trybot-target/version' (e.g. 'trybot-lumpy-paladin/R22-2515.0.0-a1-b127') as an unique label to the autotest server, as opposed to 'target/version' in the buildbot runs. Hence the artifacts are stored at GOOGLE_STORAGE_URL_PREFIX/chrome-image-archive/trybot-target/version/ BUG=chromium-os:32272 TEST=unittest & remote trybot run using --bootstrap-args Change-Id: I07554de1c36e55bac0168b69640082d746bced23 Reviewed-on: https://gerrit.chromium.org/gerrit/26074 Tested-by: Yu-Ju Hong <yjhong@chromium.org> Reviewed-by: Chris Sosa <sosa@chromium.org> Commit-Ready: Yu-Ju Hong <yjhong@chromium.org>
2012-07-02chromite: Remove exynos-kernel from blacklistKatie Roberts-Hoffman
Removing the blacklist for exynos-kernel. This was originally needed as the exynos-kernel ebuild was public but the sources were private. The ebuild has since been moved private. For reference, see I4bb8fb7155ecaf5ccb11dcc19487040e55b0bf80. BUG=None TEST=trybot for daisy Change-Id: Ic1d41d9148211f4dfc14bd00fb325c99738aca93 Reviewed-on: https://gerrit.chromium.org/gerrit/19790 Reviewed-by: Doug Anderson <dianders@chromium.org> Commit-Ready: Doug Anderson <dianders@chromium.org> Tested-by: Doug Anderson <dianders@chromium.org>
generated by cgit v1.2.3 (git 2.25.1) at 2024-07-16 01:28:29 +0000