%cl-entities; ]> Callgrind: a heavyweight profiler Overview Callgrind is a Valgrind tool for profiling programs with the ability to construct a call graph from the execution. By default, the collected data consists of the number of instructions executed, their attribution to source lines, and call relationship among functions together with number of actually executed calls. Optionally, a cache simulator (similar to cachegrind) can produce further information about the memory access behavior of the application. The profile data is written out to a file at program termination. For presentation of the data, and interactive control of the profiling, two command line tools are provided: callgrind_annotate This command reads in the profile data, and prints a sorted lists of functions, optionally with source annotation. For graphical visualization of the data, check out KCachegrind. callgrind_control This command enables you to interactively observe and control the status of currently running applications, without stopping the application. You can get statistics information as well as the current stack trace, and you can request zeroing of counters or dumping of profile data. To use Callgrind, you must specify --tool=callgrind on the Valgrind command line. Functionality Cachegrind provides a flat profile: event counts (reads, misses etc.) attributed to functions exactly represent events which happened while the function itself was running, which also is called self or exclusive cost. In addition, Callgrind further attributes call sites inside functions with event counts for events which happened while the call was active, ie. while code was executed which actually was called from the given call site. Adding these call costs to the self cost of a function gives the so called inclusive cost. As an example, inclusive cost of main() should be almost 100 percent (apart from any cost spent in startup before main, such as initialization of the run time linker or construction of global C++ objects). Together with the call graph, this allows you to see the call chains starting from main(), inside which most of the events were happening. This especially is useful for functions called from multiple call sites, and where any optimization makes sense only by changing code in the caller (e.g. by reducing the call count). Callgrind's cache simulation is based on the Cachegrind tool. Read Cachegrind's documentation first; this page describes the features supported in addition to Cachegrind's features. Callgrinds ability to trace function call varies with the ISA of the platform it is run on. Its usage was specially tailored for x86 and amd64, and unfortunately, it currently happens to show quite bad call/return detection in PPC32/64 code (this is because there are only jump/branch instructions in the PPC ISA, and Callgrind has to rely on heuristics). Basic Usage As with Cachegrind, you probably want to compile with debugging info (the -g flag), but with optimization turned on. To start a profile run for a program, execute: callgrind [callgrind options] your-program [program options] While the simulation is running, you can observe execution with callgrind_control -b This will print out the current backtrace. To annotate the backtrace with event counts, run callgrind_control -e -b After program termination, a profile data file named callgrind.out.pid is generated with pid being the process ID of the execution of this profile run. The data file contains information about the calls made in the program among the functions executed, together with events of type Instruction Read Accesses (Ir). To generate a function-by-function summary from the profile data file, use callgrind_annotate [options] callgrind.out.pid This summary is similar to the output you get from a Cachegrind run with cg_annotate: the list of functions is ordered by exclusive cost of functions, which also are the ones that are shown. Important for the additional features of Callgrind are the following two options: : Instead of using exclusive cost of functions as sorting order, use and show inclusive cost. : Interleaved into the ordered list of function, show the callers and the callees of each function. In these lines, which represents executed calls, the cost gives the number of events spent in the call. Indented, above each given function, there is the list of callers, and below, the list of callees. The sum of events in calls to a given function (caller lines), as well as the sum of events in calls from the function (callee lines) together with the self cost, gives the total inclusive cost of the function. Use to get annotated source code for all relevant functions for which the source can be found. In addition to source annotation as produced by cg_annotate, you will see the annotated call sites with call counts. For all other options, look up the manual for cg_annotate. For better call graph browsing experience, it is highly recommended to use KCachegrind. If your code happens to spent relevant fractions of cost in cycles (sets of functions calling each other in a recursive manner), you have to use KCachegrind, as callgrind_annotate currently does not do any cycle detection, which is important to get correct results in this case. If you are additionally interested in measuring the cache behavior of your program, use Callgrind with the option However, expect a further slow down approximately by a factor of 2. If the program section you want to profile is somewhere in the middle of the run, it is beneficial to fast forward to this section without any profiling at all, and switch profiling on later. This is achieved by using and interactively use callgrind_control -i on before the interesting code section is about to be executed. To exactly specify the code position where profiling should start, use the client request CALLGRIND_START_INSTRUMENTATION. If you want to be able to see assembler annotation, specify . This will produce profile data at instruction granularity. Note that the resulting profile data can only be viewed with KCachegrind. For assembler annotation, it also is interesting to see more details of the control flow inside of functions, ie. (conditional) jumps. This will be collected by further specifying . Advanced Usage Multiple profiling dumps from one program run Often, you are not interested in characteristics of a full program run, but only of a small part of it (e.g. execution of one algorithm). If there are multiple algorithms or one algorithm running with different input data, it's even useful to get different profile information for multiple parts of one program run. Profile data files have names of the form callgrind.out.pid.part-threadID where pid is the PID of the running program, part is a number incremented on each dump (".part" is skipped for the dump at program termination), and threadID is a thread identification ("-threadID" is only used if you request dumps of individual threads with ). There are different ways to generate multiple profile dumps while a program is running under Callgrind's supervision. Nevertheless, all methods trigger the same action, which is "dump all profile information since the last dump or program start, and zero cost counters afterwards". To allow for zeroing cost counters without dumping, there is a second action "zero all cost counters now". The different methods are: Dump on program termination. This method is the standard way and doesn't need any special action from your side. Spontaneous, interactive dumping. Use callgrind_control -d [hint [PID/Name]] to request the dumping of profile information of the supervised application with PID or Name. hint is an arbitrary string you can optionally specify to later be able to distinguish profile dumps. The control program will not terminate before the dump is completely written. Note that the application must be actively running for detection of the dump command. So, for a GUI application, resize the window or for a server send a request. If you are using KCachegrind for browsing of profile information, you can use the toolbar button Force dump. This will request a dump and trigger a reload after the dump is written. Periodic dumping after execution of a specified number of basic blocks. For this, use the command line option . Dumping at enter/leave of all functions whose name starts with funcprefix. Use the option and . To zero cost counters before entering a function, use . The prefix method for specifying function names was choosen to ease the use with C++: you don't have to specify full signatures. You can specify these options multiple times for different function prefixes. Program controlled dumping. Put ]]> into your source and add CALLGRIND_DUMP_STATS; when you want a dump to happen. Use CALLGRIND_ZERO_STATS; to only zero cost centers. In Valgrind terminology, this method is called "Client requests". The given macros generate a special instruction pattern with no effect at all (i.e. a NOP). When run under Valgrind, the CPU simulation engine detects the special instruction pattern and triggers special actions like the ones described above. If you are running a multi-threaded application and specify the command line option , every thread will be profiled on its own and will create its own profile dump. Thus, the last two methods will only generate one dump of the currently running thread. With the other methods, you will get multiple dumps (one for each thread) on a dump request. Limiting the range of collected events For aggregating events (function enter/leave, instruction execution, memory access) into event numbers, first, the events must be recognizable by Callgrind, and second, the collection state must be switched on. Event collection is only possible if instrumentation for program code is switched on. This is the default, but for faster execution (identical to valgrind --tool=none), it can be switched off until the program reaches a state in which you want to start collecting profiling data. Callgrind can start without instrumentation by specifying option . Instrumentation can be switched on interactively with callgrind_control -i on and off by specifying "off" instead of "on". Furthermore, instrumentation state can be programatically changed with the macros CALLGRIND_START_INSTRUMENTATION; and CALLGRIND_STOP_INSTRUMENTATION;. In addition to enabling instrumentation, you must also enable event collection for the parts of your program you are interested in. By default, event collection is enabled everywhere. You can limit collection to specific function(s) by using . This will toggle the collection state on entering and leaving the specified functions. When this option is in effect, the default collection state at program start is "off". Only events happening while running inside of functions starting with funcprefix will be collected. Recursive calls of functions with funcprefix do not trigger any action. It is important to note that with instrumentation switched off, the cache simulator cannot see any memory access events, and thus, any simulated cache state will be frozen and wrong without instrumentation. Therefore, to get useful cache events (hits/misses) after switching on instrumentation, the cache first must warm up, probably leading to many cold misses which would not have happened in reality. If you do not want to see these, start event collection a few million instructions after you have switched on instrumentation. Avoiding cycles Each group of functions with any two of them happening to have a call chain from one to the other, is called a cycle. For example, with A calling B, B calling C, and C calling A, the three functions A,B,C build up one cycle. If a call chain goes multiple times around inside of a cycle, with profiling, you can not distinguish event counts coming from the first round or the second. Thus, it makes no sense to attach any inclusive cost to a call among functions inside of one cycle. If "A > B" appears multiple times in a call chain, you have no way to partition the one big sum of all appearances of "A > B". Thus, for profile data presentation, all functions of a cycle are seen as one big virtual function. Unfortunately, if you have an application using some callback mechanism (like any GUI program), or even with normal polymorphism (as in OO languages like C++), it's quite possible to get large cycles. As it is often impossible to say anything about performance behaviour inside of cycles, it is useful to introduce some mechanisms to avoid cycles in call graphs. This is done by treating the same function in different ways, depending on the current execution context, either by giving them different names, or by ignoring calls to functions. There is an option to ignore calls to a function with . E.g., you usually do not want to see the trampoline functions in the PLT sections for calls to functions in shared libraries. You can see the difference if you profile with . If a call is ignored, cost events happening will be attached to the enclosing function. If you have a recursive function, you can distinguish the first 10 recursion levels by specifying . Or for all functions with , but this will give you much bigger profile data files. In the profile data, you will see the recursion levels of "func" as the different functions with names "func", "func'2", "func'3" and so on. If you have call chains "A > B > C" and "A > C > B" in your program, you usually get a "false" cycle "B <> C". Use , and functions "B" and "C" will be treated as different functions depending on the direct caller. Using the apostrophe for appending this "context" to the function name, you get "A > B'A > C'B" and "A > C'A > B'C", and there will be no cycle. Use to get a 2-caller dependency for all functions. Note that doing this will increase the size of profile data files. Command line option reference In the following, options are grouped into classes, in same order as the output as callgrind --help. Miscellaneous options Show summary of options. This is a short version of this manual section. Show version of callgrind. Dump creation options These options influence the name and format of the profile data files. Specify the base name for the dump file names. To distinguish different profile runs of the same application, .<pid> is appended to the base dump file name with <pid> being the process ID of the profile run (with multiple dumps happening, the file name is modified further; see below). This option is especially usefull if your application changes its working directory. Usually, the dump file is generated in the current working directory of the application at program termination. By giving an absolute path with the base specification, you can force a fixed directory for the dump files. This specifies that event counting should be performed at per-instruction granularity. This allows for assembler code annotation, but currently the results can only be shown with KCachegrind. This specifies that event counting should be performed at source line granularity. This allows source annotation for sources which are compiled with debug information ("-g"). This option influences the output format of the profile data. It specifies whether strings (file and function names) should be identified by numbers. This shrinks the file size, but makes it more difficult for humans to read (which is not recommand either way). However, this currently has to be switched off if the files are to be read by callgrind_annotate! This option influences the output format of the profile data. It specifies whether numerical positions are always specified as absolute values or are allowed to be relative to previous numbers. This shrinks the file size, However, this currently has to be switched off if the files are to be read by callgrind_annotate! When multiple profile data parts are to be generated, these parts are appended to the same output file if this option is set to "yes". Not recommand. Activity options These options specify when actions relating to event counts are to be executed. For interactive control use callgrind_control. Dump profile data every <count> basic blocks. Whether a dump is needed is only checked when Valgrinds internal scheduler is run. Therefore, the minimum setting useful is about 100000. The count is a 64-bit value to make long dump periods possible. Dump when entering a function starting with <prefix> Zero all costs when entering a function starting with <prefix> Dump when leaving a function starting with <prefix> Data collection options These options specify when events are to be aggregated into event counts. Also see . Specify if you want Callgrind to start simulation and profiling from the beginning of the program. When set to no, Callgrind will not be able to collect any information, including calls, but it will have at most a slowdown of around 4, which is the minimum Valgrind overhead. Instrumentation can be interactively switched on via callgrind_control -i on. Note that the resulting call graph will most probably not contain main, but will contain all the functions executed after instrumentation was switched on. Instrumentation can also programatically switched on/off. See the Callgrind include file <callgrind.h> for the macro you have to use in your source code. For cache simulation, results will be less accurate when switching on instrumentation later in the program run, as the simulator starts with an empty cache at that moment. Switch on event collection later to cope with this error. Specify whether event collection is switched on at beginning of the profile run. To only look at parts of your program, you have two possibilities: Zero event counters before entering the program part you want to profile, and dump the event counters to a file after leaving that program part. Switch on/off collection state as needed to only see event counters happening while inside of the program part you want to profile. The second option can be used if the program part you want to profile is called many times. Option 1, i.e. creating a lot of dumps is not practical here. Collection state can be toggled at entry and exit of a given function with the option . If you use this flag, collection state should be switched off at the beginning. Note that the specification of --toggle-collect implicitly sets --collect-state=no. Collection state can be toggled also by using a Valgrind Client Request in your application. For this, include valgrind/callgrind.h and specify the macro CALLGRIND_TOGGLE_COLLECT at the needed positions. This only will have any effect if run under supervision of the Callgrind tool. Toggle collection on entry/exit of a function whose name starts with <prefix>. This specifies whether information for (conditional) jumps should be collected. As above, callgrind_annotate currently is not able to show you the data. You have to use KCachegrind to get jump arrows in the annotated code. Cost entity separation options These options specify how event counts should be attributed to execution contexts. More specifically, they specify e.g. if the recursion level or the call chain leading to a function should be accounted for, and whether the thread ID should be remembered. Also see . This option specifies whether profile data should be generated separately for every thread. If yes, the file names get "-threadID" appended. Separate function recursions, maximal <level>. See . Separate contexts by maximal <callers> functions in the call chain. See . Ignore calls to/from PLT sections. Ignore calls to/from a given function. E.g. if you have a call chain A > B > C, and you specify function B to be ignored, you will only see A > C. This is very convenient to skip functions handling callback behaviour. E.g. for the SIGNAL/SLOT mechanism in QT, you only want to see the function emitting a signal to call the slots connected to that signal. First, determine the real call chain to see the functions needed to be skipped, then use this option. Put a function into a separate group. This influences the context name for cycle avoidance. All functions inside of such a group are treated as being the same for context name building, which resembles the call chain leading to a context. By specifying function groups with this option, you can shorten the context name, as functions in the same group will not appear in sequence in the name. Separate <number> recursions for <function>. See . Separate <number> callers for <function>. See . Cache simulation options Specify if you want to do full cache simulation. By default, only instruction read accesses will be profiled.