| Age | Commit message (Collapse) | Author |
|---|
|
4185249 snap-temp-L63000000082739046
Change-Id: I5ad3b981f9a07ed8534b401ef63bc54fc705a1a6
|
|
Change-Id: I238bb50ed1907def19b23b0610eec87234ef4d51
Signed-off-by: Joel Fernandes <joelaf@google.com>
|
|
4169335 snap-temp-L20600000081126177
Change-Id: I46adc52774cb43a516c89a30cbb07ccdd4a59afe
|
|
Change-Id: I3e8ff4d9d11220cddeac955e9a949fc3464ecc36
Signed-off-by: Joel Fernandes <joelaf@google.com>
|
|
4165363 snap-temp-L49300000080728237
Change-Id: I95bd2a425252c930a8eb344490be768b0c989f4c
|
|
Forward propogate secondary DF into primary DF and return the merged DF.
Implements: https://github.com/ARM-software/trappy/issues/250
Change-Id: I312d77302bbca8bb13bfa598785ebc0cc879fe34
Signed-off-by: Joel Fernandes <joelaf@google.com>
|
|
There's a bug in trappy caching where corruption or removal of single CSVs
causes all CSVs to get removed. Fix this and add a test-case to cover it.
Also test that only missing items are regenerated.
Change-Id: Iea41c8e0a005515790b580e7f78f06bdd60141a3
Signed-off-by: Joel Fernandes <joelaf@google.com>
|
|
4152006 snap-temp-L91700000079405440
Change-Id: I1beb78c74f7bc96c5031133d1ff22fe621e0e6ae
|
|
- json params loading represents keys as unicode, use dumps to fix it
- finalize_objects need to be run always for cases where some events not
cached, fix it by moving it up and also handle bjackman's concern that we run
it twice by skipping finalize for cached events.
- fix time normalization breakage as a result of the above changes
Change-Id: I2011de0ae8112e937ee61baee8d53a63d0bbe85a
Signed-off-by: Joel Fernandes <joelaf@google.com>
|
|
Change-Id: Ie79698d90e0406cc11c52d364144ec08c33dfac4
Signed-off-by: Joel Fernandes <joelaf@google.com>
|
|
|
|
This reverts commit 9493bfaba69108b16db1ee21c53c7f0f95eec5ba.
|
|
This reverts commit c9243e261fb37be1b1149ae6d111e18745b75959.
|
|
This reverts commit c0a49a16d85ef5b732fafe02faad73c67df7b665.
|
|
This reverts commit 2b9b25c74416328a0f68af5b60b73534e4ccbd02.
|
|
|
|
Lets move this out of the main parsing loop so that the code is easier to read.
Suggested-by: Patrick Bellasi <patrick.bellasi@arm.com>
Signed-off-by: Joel Fernandes <joelaf@google.com>
Reviewed-by: KP Singh <kpsingh@google.com>
|
|
generate_data_dict uses try/except for integer conversion. This is expensive
consider the frequency of the calls. Optimize it by using a slightly more
uglier but much fast version of the same. With this I get a speed of ~8.5% in
parsing.
Change-Id: I909ad170756fd284c7d924950945e755880ceb5f
Signed-off-by: Joel Fernandes <joelaf@google.com>
Reviewed-by: KP Singh <kpsingh@google.com>
|
|
I found that trappy spends a lot of time looking for the array pattern "{}".
Vast majority of trace lines don't have them. Instead of running the regex
every time, check for the ={} pattern using the 'in' operator which is much
faster than the regex for the common case, and then do the regex sub if needed.
The speed up is huge and in my test run, I saw parsing stage on a 100MB trace
speed up from 13s -> 10.5s !!
Change-Id: I7ae68efc99eaf8844624871be2a4d66a7820a9b0
Signed-off-by: Joel Fernandes <joelaf@google.com>
Reviewed-by: KP Singh <kpsingh@google.com>
|
|
Its not necessary to check for unique word twice. Reusing the 'class detection'
path for this purpose is sufficient. This improve performance by 3-6% of the
parsing stage.
Change-Id: Iff8ebd0086d2ccc10bec14e3d40a63f3c04aaf1a
Signed-off-by: Joel Fernandes <joelaf@google.com>
Reviewed-by: KP Singh <kpsingh@google.com>
|
|
|
|
Some trace analysis tasks are most easily expressed by iterating over
events in chronological order. Add an apply_callbacks() method to the
GenericFtrace class that will run user-defined functions on trace
events in the order they occurred.
Signed-off-by: Connor O'Brien <connoro@google.com>
Reviewed-by: KP Singh <kpsingh@google.com>
|
|
|
|
Add test for invalid cache
Add test for caching with extra events
Add test for normalize_time and window parameters
Signed-off-by: Brendan Jackman <brendan.jackman@arm.com>
Signed-off-by: Joel Fernandes <joelaf@google.com>
Reviewed-by: KP Singh <kpsingh@google.com>
|
|
Store a JSON file with the parameters used to parse a trace in the cache. If, on
retrieval, we find a mismatch then simply invalidate the cache.
A richer implementation might detect whether we can still use the cache (e.g. if
the requested window is a subset of the cached one), but for now I don't think
that's worth the effort/complexity.
Reviewed-by: KP Singh <kpsingh@google.com>
|
|
Pandas is extremely fast at parsing csv to data frames. Astonishingly it takes
< 1s to serialize/deserialize a 100MB work of traces with 430000 events to/from
csv. We leverage this and write out a data frames into a csv file when they are
created for the first time. Next time we read it out if it exists. To make
sure, the cache isn't stale, we take the md5sum of the trace file and also
ensure all CSVs exist before reading from the cache. I get a speed up of 16s to
1s when parsing a 100MB trace.
Co-developed-by: Brendan Jackman <brendan.jackman@arm.com>
Signed-off-by: Joel Fernandes <joelaf@google.com>
Reviewed-by: KP Singh <kpsingh@google.com>
|
|
4140700 snap-temp-L81200000078275553
Change-Id: Iedd9a0cb3806eb9a3c6ed463108e6249b2c8abf5
|
|
|
|
|
|
tests/test_ftrace.py:
Stop creating and checking for existence of <trace>.raw.txt files.
tests/test_sched.py:
Stop creating raw.txt file.
tests/raw_trace.raw.txt:
Remove the file as it is no longer needed. The raw-formatted events
are moved into raw_trace.txt
tests/raw_trace.txt:
Replace the formatted sched_switch events with raw formatted events,
like we get when parsing a trace file.
tests/trace_empty.txt:
Remove the default sched_ events we would expect to parse in raw
format - since we want to have events here which we are not looking
for.
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Reviewed-by: KP Singh <kpsingh@google.com>
|
|
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Reviewed-by: KP Singh <kpsingh@google.com>
|
|
Since we no longer generate these, we should let the user either remove
their text files manually or merge the files if they no longer have the
.dat file. The exception describes how to deal with this format change
and which events should be moved from the raw to the formatted text file
if the user no longer has the .dat file.
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Reviewed-by: KP Singh <kpsingh@google.com>
|
|
Now that we have events constructed before generating trace files, we can
scan them and determine which events need to be formatted and which should
be unformatted (raw). trace-cmd report lets us specify events we want to
print raw. Use this to produce a single trace.txt which has all the
necessary formatted & raw events instead of calling trace-cmd report
twice.
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Reviewed-by: KP Singh <kpsingh@google.com>
|
|
This patch set is to modify the parsing step so that we don't need to
build a raw and a formatted trace file. To do that, we need to know
which events should have raw output and which should use their default
formatting. The events indicate which are which, but currently we
generate the trace files before we populate the events.
Splitting the initialisation into two parts means that we can populate
the events so that a later patch can create the text trace with each
event either formatted or raw as required.
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Reviewed-by: KP Singh <kpsingh@google.com>
|
|
When the sched_switch event is added to the parser the constructor is
not called so we never set the parse_raw flag. Ensure there is a class
variable to hold the correct value and use it in the constructor.
We use the raw file to extract the events since we actually do construct
one of these objects during parsing.
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Reviewed-by: KP Singh <kpsingh@google.com>
|
|
generate_data_dict uses try/except for integer conversion. This is expensive
consider the frequency of the calls. Optimize it by using a slightly more
uglier but much fast version of the same. With this I get a speed of ~8.5% in
parsing.
Change-Id: I909ad170756fd284c7d924950945e755880ceb5f
Signed-off-by: Joel Fernandes <joelaf@google.com>
|
|
I found that trappy spends a lot of time looking for the array pattern "{}".
Vast majority of trace lines don't have them. Instead of running the regex
every time, check for the ={} pattern using the 'in' operator which is much
faster than the regex for the common case.
The speed up is huge and in my test run, I saw parsing stage on a 100MB trace
speed up from 13s -> 10.5s !!
Change-Id: I7ae68efc99eaf8844624871be2a4d66a7820a9b0
Signed-off-by: Joel Fernandes <joelaf@google.com>
|
|
Its not necessary to check for unique word twice. Reusing the 'class detection'
path for this purpose is sufficient. This improve performance by 3-6% of the
parsing stage.
Change-Id: Iff8ebd0086d2ccc10bec14e3d40a63f3c04aaf1a
Signed-off-by: Joel Fernandes <joelaf@google.com>
|
|
4133428 snap-temp-L95800000077479875
Change-Id: Ib491fcdb8abe19e83e5f629fbd44a33c04a4f552
|
|
|
|
4124666 snap-temp-L64200000076596327
Change-Id: I48a3939f7463a70ad5a1cc3f99cc5c504cb4a566
|
|
Some trace analysis tasks are most easily expressed by iterating over
events in chronological order. Add a run_event_callbacks() method to
the GenericFTrace class that will run user-defined functions on trace
events in the order they occurred.
Signed-off-by: Connor O'Brien <connoro@google.com>
|
|
|
|
Signed-off-by: Joel Fernandes <joelaf@google.com>
Reviewed-by: KP Singh <kpsingh@google.com>
|
|
Signed-off-by: Joel Fernandes <joelaf@google.com>
Reviewed-by: KP Singh <kpsingh@google.com>
|
|
The regex is broken for strings like this:
C|594|HW_VSYNC_ON_0|1
This is because the "1" (data) is optional and so the greedy operator globs everything causing
func = "HW_VSYNC_ON_0|1" and data = NaN
Change the data to be greedy until '|' is encountered so with that:
func = "HW_VSYNC_ON_0" and data = "1".
Change-Id: I3695a8da8ad10598192cea08ca92e27271565ccf
Signed-off-by: Joel Fernandes <joelaf@google.com>
Reviewed-by: KP Singh <kpsingh@google.com>
|
|
I promised @derkling I would write this so here you go.
Signed-off-by: Joel Fernandes <joelaf@google.com>
Reviewed-by: KP Singh <kpsingh@google.com>
|
|
|
|
FTrace can be configured to report events timestamp using different
clock sources which can be selected via the trace_clock sysfs attribute
as described in:
https://www.kernel.org/doc/Documentation/trace/ftrace.txt
The global clock source reports time in [s] with a [us] resolution.
Other sources instead, like for example the boot clock, uses [ns].
Thus, in these last cases we do not have decimals in the timestamp.
Let's update the special fields regexp to match both [s] and [ns]
formatted times and do the required pre-processing to ensure that
DataFrames are alwasy expressed using a [s].[decimals] format.
This also update the base test to add a set of trace events which are
expressed in [ns] resolution.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
Reviewed-by: KP Singh <kpsingh@google.com>
|
|
4102303 snap-temp-L92600000074342209
Change-Id: I6c3c863df1aecafe890dab6b366f1068b7970337
|
