diff options
Diffstat (limited to 'Documentation/trace')
| -rw-r--r-- | Documentation/trace/boottime-trace.rst | 2 | ||||
| -rw-r--r-- | Documentation/trace/debugging.rst | 2 | ||||
| -rw-r--r-- | Documentation/trace/events.rst | 8 | ||||
| -rw-r--r-- | Documentation/trace/fprobe.rst | 2 | ||||
| -rw-r--r-- | Documentation/trace/ftrace-uses.rst | 2 | ||||
| -rw-r--r-- | Documentation/trace/ftrace.rst | 14 | ||||
| -rw-r--r-- | Documentation/trace/histogram-design.rst | 175 | ||||
| -rw-r--r-- | Documentation/trace/histogram.rst | 40 | ||||
| -rw-r--r-- | Documentation/trace/rv/monitor_synthesis.rst | 2 |
9 files changed, 125 insertions, 122 deletions
diff --git a/Documentation/trace/boottime-trace.rst b/Documentation/trace/boottime-trace.rst index 3efac10adb36..651f3a2c01de 100644 --- a/Documentation/trace/boottime-trace.rst +++ b/Documentation/trace/boottime-trace.rst @@ -19,7 +19,7 @@ this uses bootconfig file to describe tracing feature programming. Options in the Boot Config ========================== -Here is the list of available options list for boot time tracing in +Here is the list of available options for boot time tracing in boot config file [1]_. All options are under "ftrace." or "kernel." prefix. See kernel parameters for the options which starts with "kernel." prefix [2]_. diff --git a/Documentation/trace/debugging.rst b/Documentation/trace/debugging.rst index d54bc500af80..4d88c346fc38 100644 --- a/Documentation/trace/debugging.rst +++ b/Documentation/trace/debugging.rst @@ -59,7 +59,7 @@ There is various methods of acquiring the state of the system when a kernel crash occurs. This could be from the oops message in printk, or one could use kexec/kdump. But these just show what happened at the time of the crash. It can be very useful in knowing what happened up to the point of the crash. -The tracing ring buffer, by default, is a circular buffer than will +The tracing ring buffer, by default, is a circular buffer that will overwrite older events with newer ones. When a crash happens, the content of the ring buffer will be all the events that lead up to the crash. diff --git a/Documentation/trace/events.rst b/Documentation/trace/events.rst index 2d88a2acacc0..18d112963dec 100644 --- a/Documentation/trace/events.rst +++ b/Documentation/trace/events.rst @@ -629,8 +629,8 @@ following: - tracing synthetic events from in-kernel code - the low-level "dynevent_cmd" API -7.1 Dyamically creating synthetic event definitions ---------------------------------------------------- +7.1 Dynamically creating synthetic event definitions +---------------------------------------------------- There are a couple ways to create a new synthetic event from a kernel module or other kernel code. @@ -944,8 +944,8 @@ Note that synth_event_trace_end() must be called at the end regardless of whether any of the add calls failed (say due to a bad field name being passed in). -7.3 Dyamically creating kprobe and kretprobe event definitions --------------------------------------------------------------- +7.3 Dynamically creating kprobe and kretprobe event definitions +--------------------------------------------------------------- To create a kprobe or kretprobe trace event from kernel code, the kprobe_event_gen_cmd_start() or kretprobe_event_gen_cmd_start() diff --git a/Documentation/trace/fprobe.rst b/Documentation/trace/fprobe.rst index 71cd40472d36..06b0edad0179 100644 --- a/Documentation/trace/fprobe.rst +++ b/Documentation/trace/fprobe.rst @@ -81,7 +81,7 @@ Same as ftrace, the registered callbacks will start being called some time after the register_fprobe() is called and before it returns. See :file:`Documentation/trace/ftrace.rst`. -Also, the unregister_fprobe() will guarantee that the both enter and exit +Also, the unregister_fprobe() will guarantee that both enter and exit handlers are no longer being called by functions after unregister_fprobe() returns as same as unregister_ftrace_function(). diff --git a/Documentation/trace/ftrace-uses.rst b/Documentation/trace/ftrace-uses.rst index e198854ace79..e225cc46b71e 100644 --- a/Documentation/trace/ftrace-uses.rst +++ b/Documentation/trace/ftrace-uses.rst @@ -193,7 +193,7 @@ FTRACE_OPS_FL_RECURSION Note, if this flag is set, then the callback will always be called with preemption disabled. If it is not set, then it is possible (but not guaranteed) that the callback will be called in - preemptable context. + preemptible context. FTRACE_OPS_FL_IPMODIFY Requires FTRACE_OPS_FL_SAVE_REGS set. If the callback is to "hijack" diff --git a/Documentation/trace/ftrace.rst b/Documentation/trace/ftrace.rst index af66a05e18cc..aef674df3afd 100644 --- a/Documentation/trace/ftrace.rst +++ b/Documentation/trace/ftrace.rst @@ -30,7 +30,7 @@ disabled and enabled, as well as for preemption and from a time a task is woken to the task is actually scheduled in. One of the most common uses of ftrace is the event tracing. -Throughout the kernel is hundreds of static event points that +Throughout the kernel are hundreds of static event points that can be enabled via the tracefs file system to see what is going on in certain parts of the kernel. @@ -383,7 +383,7 @@ of ftrace. Here is a list of some of the key files: not be listed in this count. If the callback registered to be traced by a function with - the "save regs" attribute (thus even more overhead), a 'R' + the "save regs" attribute (thus even more overhead), an 'R' will be displayed on the same line as the function that is returning registers. @@ -392,7 +392,7 @@ of ftrace. Here is a list of some of the key files: an 'I' will be displayed on the same line as the function that can be overridden. - If a non ftrace trampoline is attached (BPF) a 'D' will be displayed. + If a non-ftrace trampoline is attached (BPF) a 'D' will be displayed. Note, normal ftrace trampolines can also be attached, but only one "direct" trampoline can be attached to a given function at a time. @@ -402,7 +402,7 @@ of ftrace. Here is a list of some of the key files: If a function had either the "ip modify" or a "direct" call attached to it in the past, a 'M' will be shown. This flag is never cleared. It is - used to know if a function was every modified by the ftrace infrastructure, + used to know if a function was ever modified by the ftrace infrastructure, and can be used for debugging. If the architecture supports it, it will also show what callback @@ -418,7 +418,7 @@ of ftrace. Here is a list of some of the key files: This file contains all the functions that ever had a function callback to it via the ftrace infrastructure. It has the same format as - enabled_functions but shows all functions that have every been + enabled_functions but shows all functions that have ever been traced. To see any function that has every been modified by "ip modify" or a @@ -517,7 +517,7 @@ of ftrace. Here is a list of some of the key files: Whenever an event is recorded into the ring buffer, a "timestamp" is added. This stamp comes from a specified clock. By default, ftrace uses the "local" clock. This - clock is very fast and strictly per cpu, but on some + clock is very fast and strictly per CPU, but on some systems it may not be monotonic with respect to other CPUs. In other words, the local clocks may not be in sync with local clocks on other CPUs. @@ -868,7 +868,7 @@ Here is the list of current tracers that may be configured. "mmiotrace" - A special tracer that is used to trace binary module. + A special tracer that is used to trace binary modules. It will trace all the calls that a module makes to the hardware. Everything it writes and reads from the I/O as well. diff --git a/Documentation/trace/histogram-design.rst b/Documentation/trace/histogram-design.rst index 5765eb3e9efa..ae71b5bf97c6 100644 --- a/Documentation/trace/histogram-design.rst +++ b/Documentation/trace/histogram-design.rst @@ -11,13 +11,14 @@ histograms work and how the individual pieces map to the data structures used to implement them in trace_events_hist.c and tracing_map.c. -Note: All the ftrace histogram command examples assume the working - directory is the ftrace /tracing directory. For example:: +.. note:: + All the ftrace histogram command examples assume the working + directory is the ftrace /tracing directory. For example:: # cd /sys/kernel/tracing -Also, the histogram output displayed for those commands will be -generally be truncated - only enough to make the point is displayed. + Also, the histogram output displayed for those commands will be + generally be truncated - only enough to make the point is displayed. 'hist_debug' trace event files ============================== @@ -142,30 +143,30 @@ elements for a couple hypothetical keys and values.:: +--------------+ | | n_keys = n_fields - n_vals | | -The hist_data n_vals and n_fields delineate the extent of the fields[] | | -array and separate keys from values for the rest of the code. | | - -Below is a run-time representation of the tracing_map part of the | | -histogram, with pointers from various parts of the fields[] array | | -to corresponding parts of the tracing_map. | | - -The tracing_map consists of an array of tracing_map_entrys and a set | | -of preallocated tracing_map_elts (abbreviated below as map_entry and | | -map_elt). The total number of map_entrys in the hist_data.map array = | | -map->max_elts (actually map->map_size but only max_elts of those are | | -used. This is a property required by the map_insert() algorithm). | | - -If a map_entry is unused, meaning no key has yet hashed into it, its | | -.key value is 0 and its .val pointer is NULL. Once a map_entry has | | -been claimed, the .key value contains the key's hash value and the | | -.val member points to a map_elt containing the full key and an entry | | -for each key or value in the map_elt.fields[] array. There is an | | -entry in the map_elt.fields[] array corresponding to each hist_field | | -in the histogram, and this is where the continually aggregated sums | | -corresponding to each histogram value are kept. | | - -The diagram attempts to show the relationship between the | | -hist_data.fields[] and the map_elt.fields[] with the links drawn | | +The hist_data n_vals and n_fields delineate the extent of the fields[] +array and separate keys from values for the rest of the code. + +Below is a run-time representation of the tracing_map part of the +histogram, with pointers from various parts of the fields[] array +to corresponding parts of the tracing_map. + +The tracing_map consists of an array of tracing_map_entrys and a set +of preallocated tracing_map_elts (abbreviated below as map_entry and +map_elt). The total number of map_entrys in the hist_data.map array = +map->max_elts (actually map->map_size but only max_elts of those are +used. This is a property required by the map_insert() algorithm). + +If a map_entry is unused, meaning no key has yet hashed into it, its +.key value is 0 and its .val pointer is NULL. Once a map_entry has +been claimed, the .key value contains the key's hash value and the +.val member points to a map_elt containing the full key and an entry +for each key or value in the map_elt.fields[] array. There is an +entry in the map_elt.fields[] array corresponding to each hist_field +in the histogram, and this is where the continually aggregated sums +corresponding to each histogram value are kept. + +The diagram attempts to show the relationship between the +hist_data.fields[] and the map_elt.fields[] with the links drawn between diagrams:: +-----------+ | | @@ -380,7 +381,9 @@ entry, ts0, corresponding to the ts0 variable in the sched_waking trigger above. sched_waking histogram -----------------------:: +---------------------- + +.. code-block:: +------------------+ | hist_data |<-------------------------------------------------------+ @@ -440,31 +443,31 @@ sched_waking histogram n_keys = n_fields - n_vals | | | | | | -This is very similar to the basic case. In the above diagram, we can | | | -see a new .flags member has been added to the struct hist_field | | | -struct, and a new entry added to hist_data.fields representing the ts0 | | | -variable. For a normal val hist_field, .flags is just 0 (modulo | | | -modifier flags), but if the value is defined as a variable, the .flags | | | -contains a set FL_VAR bit. | | | - -As you can see, the ts0 entry's .var.idx member contains the index | | | -into the tracing_map_elts' .vars[] array containing variable values. | | | -This idx is used whenever the value of the variable is set or read. | | | -The map_elt.vars idx assigned to the given variable is assigned and | | | -saved in .var.idx by create_tracing_map_fields() after it calls | | | -tracing_map_add_var(). | | | - -Below is a representation of the histogram at run-time, which | | | -populates the map, along with correspondence to the above hist_data and | | | -hist_field data structures. | | | - -The diagram attempts to show the relationship between the | | | -hist_data.fields[] and the map_elt.fields[] and map_elt.vars[] with | | | -the links drawn between diagrams. For each of the map_elts, you can | | | -see that the .fields[] members point to the .sum or .offset of a key | | | -or val and the .vars[] members point to the value of a variable. The | | | -arrows between the two diagrams show the linkages between those | | | -tracing_map members and the field definitions in the corresponding | | | +This is very similar to the basic case. In the above diagram, we can +see a new .flags member has been added to the struct hist_field +struct, and a new entry added to hist_data.fields representing the ts0 +variable. For a normal val hist_field, .flags is just 0 (modulo +modifier flags), but if the value is defined as a variable, the .flags +contains a set FL_VAR bit. + +As you can see, the ts0 entry's .var.idx member contains the index +into the tracing_map_elts' .vars[] array containing variable values. +This idx is used whenever the value of the variable is set or read. +The map_elt.vars idx assigned to the given variable is assigned and +saved in .var.idx by create_tracing_map_fields() after it calls +tracing_map_add_var(). + +Below is a representation of the histogram at run-time, which +populates the map, along with correspondence to the above hist_data and +hist_field data structures. + +The diagram attempts to show the relationship between the +hist_data.fields[] and the map_elt.fields[] and map_elt.vars[] with +the links drawn between diagrams. For each of the map_elts, you can +see that the .fields[] members point to the .sum or .offset of a key +or val and the .vars[] members point to the value of a variable. The +arrows between the two diagrams show the linkages between those +tracing_map members and the field definitions in the corresponding hist_data fields[] members.:: +-----------+ | | | @@ -565,40 +568,40 @@ hist_data fields[] members.:: | | | | +---------------+ | | -For each used map entry, there's a map_elt pointing to an array of | | -.vars containing the current value of the variables associated with | | -that histogram entry. So in the above, the timestamp associated with | | -pid 999 is 113345679876, and the timestamp variable in the same | | -.var.idx for pid 4444 is 213499240729. | | - -sched_switch histogram | | ----------------------- | | - -The sched_switch histogram paired with the above sched_waking | | -histogram is shown below. The most important aspect of the | | -sched_switch histogram is that it references a variable on the | | -sched_waking histogram above. | | - -The histogram diagram is very similar to the others so far displayed, | | -but it adds variable references. You can see the normal hitcount and | | -key fields along with a new wakeup_lat variable implemented in the | | -same way as the sched_waking ts0 variable, but in addition there's an | | -entry with the new FL_VAR_REF (short for HIST_FIELD_FL_VAR_REF) flag. | | - -Associated with the new var ref field are a couple of new hist_field | | -members, var.hist_data and var_ref_idx. For a variable reference, the | | -var.hist_data goes with the var.idx, which together uniquely identify | | -a particular variable on a particular histogram. The var_ref_idx is | | -just the index into the var_ref_vals[] array that caches the values of | | -each variable whenever a hist trigger is updated. Those resulting | | -values are then finally accessed by other code such as trace action | | -code that uses the var_ref_idx values to assign param values. | | - -The diagram below describes the situation for the sched_switch | | +For each used map entry, there's a map_elt pointing to an array of +.vars containing the current value of the variables associated with +that histogram entry. So in the above, the timestamp associated with +pid 999 is 113345679876, and the timestamp variable in the same +.var.idx for pid 4444 is 213499240729. + +sched_switch histogram +---------------------- + +The sched_switch histogram paired with the above sched_waking +histogram is shown below. The most important aspect of the +sched_switch histogram is that it references a variable on the +sched_waking histogram above. + +The histogram diagram is very similar to the others so far displayed, +but it adds variable references. You can see the normal hitcount and +key fields along with a new wakeup_lat variable implemented in the +same way as the sched_waking ts0 variable, but in addition there's an +entry with the new FL_VAR_REF (short for HIST_FIELD_FL_VAR_REF) flag. + +Associated with the new var ref field are a couple of new hist_field +members, var.hist_data and var_ref_idx. For a variable reference, the +var.hist_data goes with the var.idx, which together uniquely identify +a particular variable on a particular histogram. The var_ref_idx is +just the index into the var_ref_vals[] array that caches the values of +each variable whenever a hist trigger is updated. Those resulting +values are then finally accessed by other code such as trace action +code that uses the var_ref_idx values to assign param values. + +The diagram below describes the situation for the sched_switch histogram referred to before:: - # echo 'hist:keys=next_pid:wakeup_lat=common_timestamp.usecs-$ts0' >> | | - events/sched/sched_switch/trigger | | + # echo 'hist:keys=next_pid:wakeup_lat=common_timestamp.usecs-$ts0' >> + events/sched/sched_switch/trigger | | +------------------+ | | | hist_data | | | diff --git a/Documentation/trace/histogram.rst b/Documentation/trace/histogram.rst index 2b98c1720a54..340bcb5099e7 100644 --- a/Documentation/trace/histogram.rst +++ b/Documentation/trace/histogram.rst @@ -186,8 +186,8 @@ Documentation written by Tom Zanussi The examples below provide a more concrete illustration of the concepts and typical usage patterns discussed above. -'special' event fields ------------------------- +2.1. 'special' event fields +--------------------------- There are a number of 'special event fields' available for use as keys or values in a hist trigger. These look like and behave as if @@ -204,16 +204,16 @@ Documentation written by Tom Zanussi common_cpu int the cpu on which the event occurred. ====================== ==== ======================================= -Extended error information --------------------------- +2.2. Extended error information +------------------------------- For some error conditions encountered when invoking a hist trigger command, extended error information is available via the - tracing/error_log file. See Error Conditions in - :file:`Documentation/trace/ftrace.rst` for details. + tracing/error_log file. See "Error conditions" section in + Documentation/trace/ftrace.rst for details. -6.2 'hist' trigger examples ---------------------------- +2.3. 'hist' trigger examples +---------------------------- The first set of examples creates aggregations using the kmalloc event. The fields that can be used for the hist trigger are listed @@ -840,7 +840,7 @@ Extended error information The compound key examples used a key and a sum value (hitcount) to sort the output, but we can just as easily use two keys instead. - Here's an example where we use a compound key composed of the the + Here's an example where we use a compound key composed of the common_pid and size event fields. Sorting with pid as the primary key and 'size' as the secondary key allows us to display an ordered summary of the recvfrom sizes, with counts, received by @@ -1608,8 +1608,8 @@ Extended error information Entries: 7 Dropped: 0 -2.2 Inter-event hist triggers ------------------------------ +2.4. Inter-event hist triggers +------------------------------ Inter-event hist triggers are hist triggers that combine values from one or more other events and create a histogram using that data. Data @@ -1685,8 +1685,8 @@ pseudo-file. These features are described in more detail in the following sections. -2.2.1 Histogram Variables -------------------------- +2.5. Histogram Variables +------------------------ Variables are simply named locations used for saving and retrieving values between matching events. A 'matching' event is defined as an @@ -1789,8 +1789,8 @@ or assigned to a variable and referenced in a subsequent expression:: Variables can even hold stacktraces, which are useful with synthetic events. -2.2.2 Synthetic Events ----------------------- +2.6. Synthetic Events +--------------------- Synthetic events are user-defined events generated from hist trigger variables or fields associated with one or more other events. Their @@ -1846,7 +1846,7 @@ the command that defined it with a '!':: At this point, there isn't yet an actual 'wakeup_latency' event instantiated in the event subsystem - for this to happen, a 'hist trigger action' needs to be instantiated and bound to actual fields -and variables defined on other events (see Section 2.2.3 below on +and variables defined on other events (see Section 2.7. below on how that is done using hist trigger 'onmatch' action). Once that is done, the 'wakeup_latency' synthetic event instance is created. @@ -2094,8 +2094,8 @@ histogram:: Entries: 7 Dropped: 0 -2.2.3 Hist trigger 'handlers' and 'actions' -------------------------------------------- +2.7. Hist trigger 'handlers' and 'actions' +------------------------------------------ A hist trigger 'action' is a function that's executed (in most cases conditionally) whenever a histogram entry is added or updated. @@ -2526,8 +2526,8 @@ The following commonly-used handler.action pairs are available: kworker/3:2-135 [003] d..3 49.823123: sched_switch: prev_comm=kworker/3:2 prev_pid=135 prev_prio=120 prev_state=T ==> next_comm=swapper/3 next_pid=0 next_prio=120 <idle>-0 [004] ..s7 49.823798: tcp_probe: src=10.0.0.10:54326 dest=23.215.104.193:80 mark=0x0 length=32 snd_nxt=0xe3ae2ff5 snd_una=0xe3ae2ecd snd_cwnd=10 ssthresh=2147483647 snd_wnd=28960 srtt=19604 rcv_wnd=29312 -3. User space creating a trigger --------------------------------- +2.8. User space creating a trigger +---------------------------------- Writing into /sys/kernel/tracing/trace_marker writes into the ftrace ring buffer. This can also act like an event, by writing into the trigger diff --git a/Documentation/trace/rv/monitor_synthesis.rst b/Documentation/trace/rv/monitor_synthesis.rst index ac808a7554f5..3a7d7b2f6cb6 100644 --- a/Documentation/trace/rv/monitor_synthesis.rst +++ b/Documentation/trace/rv/monitor_synthesis.rst @@ -181,7 +181,7 @@ which is the list of atomic propositions present in the LTL specification functions interacting with the Buchi automaton. While generating code, `rvgen` cannot understand the meaning of the atomic -propositions. Thus, that task is left for manual work. The recommended pratice +propositions. Thus, that task is left for manual work. The recommended practice is adding tracepoints to places where the atomic propositions change; and in the tracepoints' handlers: the Buchi automaton is executed using:: |