Crosperf: Treat benchmarks with story- test_args as different benchmarks

The bug refers to https://crbug.com/902466.

This patch modified the behavior of crosperf on determining duplicate
benchmarks. When there are two blocks with same benchmark name in
experiment file, we will append them to benchmark list first. After
that, we will rename the benchmark if it has test_args: 'story-filter'
or 'story-tag-filter'. The new benchmark name will be passed to
experiment so that we will treat them as two different runs and get
separate tables on the report.

BUG=chromium:902466
TEST=Passed all up-to-date unittests, tested and verified with one
experiment file containing following benchmark settings:
    benchmark: webrtc {
        iterations: 3
        test_args: --story-filter=datachannel
        suite: telemetry_Crosperf
    }
    benchmark: webrtc {
        iterations: 3
        test_args: --story-tag-filter=smoothness
        suite: telemetry_Crosperf
    }

Change-Id: Id733273a5f9f43d149407055c9c0da3b761ddeef
Reviewed-on: https://chromium-review.googlesource.com/1321415
Commit-Ready: Zhizhou Yang <zhizhouy@google.com>
Tested-by: Zhizhou Yang <zhizhouy@google.com>
Reviewed-by: Caroline Tice <cmtice@chromium.org>

1 files changed, 10 insertions, 6 deletions

diff --git a/crosperf/experiment_file.py b/crosperf/experiment_file.py
index 57eb52dc..12f9d5df 100644
--- a/crosperf/experiment_file.py
+++ b/crosperf/experiment_file.py
@@ -97,7 +97,7 @@ class ExperimentFile(object):
         field = self._ParseField(reader)
         settings.SetField(field[0], field[1], field[2])
       elif ExperimentFile._CLOSE_SETTINGS_RE.match(line):
-        return settings
+        return settings, settings_type
 
     raise EOFError('Unexpected EOF while parsing settings block.')
 
@@ -112,11 +112,15 @@ class ExperimentFile(object):
         if not line:
           continue
         elif ExperimentFile._OPEN_SETTINGS_RE.match(line):
-          new_settings = self._ParseSettings(reader)
-          if new_settings.name in settings_names:
-            raise SyntaxError(
-                "Duplicate settings name: '%s'." % new_settings.name)
-          settings_names[new_settings.name] = True
+          new_settings, settings_type = self._ParseSettings(reader)
+          # We will allow benchmarks with duplicated settings name for now.
+          # Further decision will be made when parsing benchmark details in
+          # ExperimentFactory.GetExperiment().
+          if settings_type != 'benchmark':
+            if new_settings.name in settings_names:
+              raise SyntaxError(
+                  "Duplicate settings name: '%s'." % new_settings.name)
+            settings_names[new_settings.name] = True
           self.all_settings.append(new_settings)
         elif ExperimentFile._FIELD_VALUE_RE.match(line):
           field = self._ParseField(reader)