Chromite changes to support AFDO.

- Changes to support AFDO in the lumpy PFQ and the master PFQ (x86-generic).
- Enabled the support in the PFQ builder to generate the AFDO profile.
- Added alert in case of failures during the AFDO profile generation.
- Add support for passing USE=afdo_use to the canary builders. This is
  NOT enabled yet.
- Changes to keep the support for a single builder that does profile
  collection and optimization in a single builder.

At this point only the profile collection is enable. Optimized builds
are not enabled yet.

BUG=None
TEST=Used a modified builder that does 2 builds:
     1) Does a regular build. Collects the 'perf' data using the
        AFDO_Record autotest and calls GenerateOrFindAFDOData.
        This is the same that will go into the PFQ builder for deployment.
     2) Does an optimized build using the AFDO file specified in the
        ebuild. This is the same that will go in to the canary builder
        for deployment.
     Having a single builder greatly simplifies testing.
     Also did "white box" testing by calling directly the routines
     that were added.
     Tested PFQ builder Master and slave (lumpy-chromium-pfq and
     x86-generic-chromium-pfq).
     Tested that these builders were unaffected: master-paladin,
     x86-generic-paladin, falco-chrome-pfq.
     And ran run_tests.

Change-Id: I095dfaa874f3ffbe898fcb23f0139a226d7d6a21
Reviewed-on: https://chromium-review.googlesource.com/199250
Reviewed-by: David James <davidjames@chromium.org>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>

1 files changed, 15 insertions, 11 deletions

diff --git a/cbuildbot/cbuildbot_config_unittest.py b/cbuildbot/cbuildbot_config_unittest.py
index d28948620..95465113a 100755
--- a/cbuildbot/cbuildbot_config_unittest.py
+++ b/cbuildbot/cbuildbot_config_unittest.py
@@ -288,15 +288,17 @@ class CBuildBotTest(cros_test_lib.MoxTestCase):
           self.assertFalse(config[flag],
               'Config %s set %s without build_tests.' % (build_name, flag))
 
-  def testPGOInBackground(self):
-    """Verify that we don't try to build or use PGO data in the background."""
+  def testAFDOInBackground(self):
+    """Verify that we don't try to build or use AFDO data in the background."""
     for build_name, config in cbuildbot_config.config.iteritems():
       if config.build_packages_in_background:
         # It is unsupported to use the build_packages_in_background flags with
-        # the pgo_generate or pgo_use config options.
-        msg = 'Config %s uses build_packages_in_background with pgo_%s'
-        self.assertFalse(config.pgo_generate, msg % (build_name, 'generate'))
-        self.assertFalse(config.pgo_use, msg % (build_name, 'use'))
+        # the afdo_generate or afdo_use config options.
+        msg = 'Config %s uses build_packages_in_background with afdo_%s'
+        self.assertFalse(config.afdo_generate, msg % (build_name, 'generate'))
+        self.assertFalse(config.afdo_generate_min, msg % (build_name,
+                                                          'generate'))
+        self.assertFalse(config.afdo_use, msg % (build_name, 'use'))
 
   def testReleaseGroupInBackground(self):
     """Verify build_packages_in_background settings for release groups.
@@ -400,10 +402,12 @@ class CBuildBotTest(cros_test_lib.MoxTestCase):
                                             board))
             prebuilt_slave_boards[board] = slave['name']
 
-  def testCantBeBothTypesOfPGO(self):
-    """Using pgo_generate and pgo_use together doesn't work."""
+  def testCantBeBothTypesOfAFDO(self):
+    """Using afdo_generate and afdo_use together doesn't work."""
     for config in cbuildbot_config.config.values():
-      self.assertFalse(config['pgo_use'] and config['pgo_generate'])
+      self.assertFalse(config['afdo_use'] and config['afdo_generate'])
+      self.assertFalse(config['afdo_use'] and config['afdo_generate_min'])
+      self.assertFalse(config['afdo_generate'] and config['afdo_generate_min'])
 
   def testValidPrebuilts(self):
     """Verify all builders have valid prebuilt values."""
@@ -508,8 +512,8 @@ class FindFullTest(cros_test_lib.TestCase):
     """Test prefer internal over external when both exist."""
     self._CheckCanonicalConfig('daisy', 'release')
 
-  def testPGOCanonicalResolution(self):
-    """Test prefer non-PGO over PGO builder."""
+  def testAFDOCanonicalResolution(self):
+    """Test prefer non-AFDO over AFDO builder."""
     self._CheckCanonicalConfig('lumpy', 'release')
 
   def testOneFullConfigPerBoard(self):