toolchain-utils: update email sending for nightly test

This patch provides follow up to http://crrev.com/c/2119231.

Crosperf will return non-zero when terminated or at least one test
failed. When tests partially succeed, reports will still be generated,
otherwise no reports.

We will try to access the report no matter Crosperf's return value,
send email and copy json report to archive dir; and then raise error
when Crosperf returns non-zero accordingly.

Also add date to the email title.

BUG=chromium:1063703
TEST=Simply tested the coverage with print.

Change-Id: I4b3f3667cf3a0fd9cef4906052de38432684862c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2137941
Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
Commit-Queue: Zhizhou Yang <zhizhouy@google.com>
Tested-by: Zhizhou Yang <zhizhouy@google.com>
Auto-Submit: Zhizhou Yang <zhizhouy@google.com>

1 files changed, 24 insertions, 12 deletions

diff --git a/buildbot_test_toolchains.py b/buildbot_test_toolchains.py
index 95557344..184a1ecc 100755
--- a/buildbot_test_toolchains.py
+++ b/buildbot_test_toolchains.py
@@ -85,8 +85,8 @@ class ToolchainComparator(object):
       self._weekday = time.strftime('%a')
     else:
       self._weekday = weekday
-    timestamp = datetime.datetime.strftime(datetime.datetime.now(),
-                                           '%Y-%m-%d_%H:%M:%S')
+    self._date = datetime.date.today().strftime('%Y/%m/%d')
+    timestamp = datetime.datetime.now().strftime('%Y-%m-%d_%H:%M:%S')
     self._reports_dir = os.path.join(
         NIGHTLY_TESTS_RESULTS,
         '%s.%s' % (timestamp, board),
@@ -248,13 +248,7 @@ class ToolchainComparator(object):
                    noschedv2_opts=noschedv2_opts,
                    exp_file=experiment_file)
 
-    ret = self._ce.RunCommand(command)
-    if ret != 0:
-      raise RuntimeError('Crosperf execution error!')
-
-    # Copy json report to pending archives directory.
-    command = 'cp %s/*.json %s/.' % (self._reports_dir, PENDING_ARCHIVES_DIR)
-    ret = self._ce.RunCommand(command)
+    return self._ce.RunCommand(command)
 
   def _SendEmail(self):
     """Find email message generated by crosperf and send it."""
@@ -264,10 +258,20 @@ class ToolchainComparator(object):
       email_title = 'buildbot llvm test results'
       if USE_LLVM_NEXT_PATCH in self._patches_string:
         email_title = 'buildbot llvm_next test results'
-      command = ('cat %s | %s -s "%s, %s" -team -html' %
-                 (filename, MAIL_PROGRAM, email_title, self._board))
+      command = ('cat %s | %s -s "%s, %s %s" -team -html' %
+                 (filename, MAIL_PROGRAM, email_title, self._board, self._date))
       self._ce.RunCommand(command)
 
+  def _CopyJson(self):
+    # Copy json report to pending archives directory.
+    command = 'cp %s/*.json %s/.' % (self._reports_dir, PENDING_ARCHIVES_DIR)
+    ret = self._ce.RunCommand(command)
+    # Failing to access json report means that crosperf terminated or all tests
+    # failed, raise an error.
+    if ret != 0:
+      raise RuntimeError(
+          'Crosperf failed to run tests, cannot copy json report!')
+
   def DoAll(self):
     """Main function inside ToolchainComparator class.
 
@@ -295,8 +299,16 @@ class ToolchainComparator(object):
     print('vanilla_image: %s' % vanilla_image)
     print('nonafdo_image: %s' % nonafdo_image)
 
-    self._TestImages(trybot_image, vanilla_image, nonafdo_image)
+    ret = self._TestImages(trybot_image, vanilla_image, nonafdo_image)
+    # Always try to send report email as crosperf will generate report when
+    # tests partially succeeded.
     self._SendEmail()
+    self._CopyJson()
+    # Non-zero ret here means crosperf tests partially failed, raise error here
+    # so that toolchain summary report can catch it.
+    if ret != 0:
+      raise RuntimeError('Crosperf tests partially failed!')
+
     return 0