From 1c27b644c0fdbc61e113b8faee14baeb8df32486 Mon Sep 17 00:00:00 2001 From: "Paul E. McKenney" Date: Thu, 18 Jan 2018 19:58:55 -0800 Subject: Automate memory-barriers.txt; provide Linux-kernel memory model There is some reason to believe that Documentation/memory-barriers.txt could use some help, and a major purpose of this patch is to provide that help in the form of a design-time tool that can produce all valid executions of a small fragment of concurrent Linux-kernel code, which is called a "litmus test". This tool's functionality is roughly similar to a full state-space search. Please note that this is a design-time tool, not useful for regression testing. However, we hope that the underlying Linux-kernel memory model will be incorporated into other tools capable of analyzing large bodies of code for regression-testing purposes. The main tool is herd7, together with the linux-kernel.bell, linux-kernel.cat, linux-kernel.cfg, linux-kernel.def, and lock.cat files added by this patch. The herd7 executable takes the other files as input, and all of these files collectively define the Linux-kernel memory memory model. A brief description of each of these other files is provided in the README file. Although this tool does have its limitations, which are documented in the README file, it does improve on the version reported on in the LWN series (https://lwn.net/Articles/718628/ and https://lwn.net/Articles/720550/) by supporting locking and arithmetic, including a much wider variety of read-modify-write atomic operations. Please note that herd7 is not part of this submission, but is freely available from http://diy.inria.fr/sources/index.html (and via "git" at https://github.com/herd/herdtools7). A second tool is klitmus7, which converts litmus tests to loadable kernel modules for direct testing. As with herd7, the klitmus7 code is freely available from http://diy.inria.fr/sources/index.html (and via "git" at https://github.com/herd/herdtools7). Of course, litmus tests are not always the best way to fully understand a memory model, so this patch also includes Documentation/explanation.txt, which describes the memory model in detail. In addition, Documentation/recipes.txt provides example known-good and known-bad use cases for those who prefer working by example. This patch also includes a few sample litmus tests, and a great many more litmus tests are available at https://github.com/paulmckrcu/litmus. This patch was the result of a most excellent collaboration founded by Jade Alglave and also including Alan Stern, Andrea Parri, and Luc Maranget. For more details on the history of this collaboration, please refer to the Linux-kernel memory model presentations at 2016 LinuxCon EU, 2016 Kernel Summit, 2016 Linux Plumbers Conference, 2017 linux.conf.au, or 2017 Linux Plumbers Conference microconference. However, one aspect of the history does bear repeating due to weak copyright tracking earlier in this project, which extends back to early 2015. This weakness came to light in late 2017 after an LKMM presentation by Paul in which an audience member noted the similarity of some LKMM code to code in early published papers. This prompted a copyright review. From Alan Stern: To say that the model was mine is not entirely accurate. Pieces of it (especially the Scpv and Atomic axioms) were taken directly from Jade's models. And of course the Happens-before and Propagation relations and axioms were heavily based on Jade and Luc's work, even though they weren't identical to the earlier versions. Only the RCU portion was completely original. . . . One can make a much better case that I wrote the bulk of lock.cat. However, it was inspired by Luc's earlier version (and still shares some elements in common), and of course it benefited from feedback and testing from all members of our group. The model prior to Alan's was Luc Maranget's. From Luc: I totally agree on Alan Stern's account of the linux kernel model genesis. I thank him for his acknowledgments of my participation to previous model drafts. I'd like to complete Alan Stern's statement: any bell cat code I have written has its roots in discussions with Jade Alglave and Paul McKenney. Moreover I have borrowed cat and bell code written by Jade Alglave freely. This copyright review therefore resulted in late adds to the copyright statements of several files. Discussion of v1 has raised several issues, which we do not believe should block acceptance given that this level of change will be ongoing, just as it has been with memory-barriers.txt: o Under what conditions should ordering provided by pure locking be seen by CPUs not holding the relevant lock(s)? In particular, should the message-passing pattern be forbidden? o Should examples involving C11 release sequences be forbidden? Note that this C11 is still a moving target for this issue: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0735r0.html o Some details of the handling of internal dependencies for atomic read-modify-write atomic operations are still subject to debate. o Changes recently accepted into mainline greatly reduce the need to handle DEC Alpha as a special case. These changes add an smp_read_barrier_depends() to READ_ONCE(), thus causing Alpha to respect ordering of dependent reads. If these changes stick, the memory model can be simplified accordingly. o Will changes be required to accommodate RISC-V? Differences from v1: (http://lkml.kernel.org/r/20171113184031.GA26302@linux.vnet.ibm.com) o Add SPDX notations to .bell and .cat files, replacing textual license statements. o Add reference to upcoming ASPLOS paper to .bell and .cat files. o Updated identifier names in .bell and .cat files to match those used in the ASPLOS paper. o Updates to READMEs and other documentation based on review feedback. o Added a memory-ordering cheatsheet. o Update sigs to new Co-Developed-by and add acks and reviewed-bys. o Simplify rules detecting nested RCU read-side critical sections. o Update copyright statements as noted above. Co-Developed-by: Alan Stern Co-Developed-by: Andrea Parri Co-Developed-by: Jade Alglave Co-Developed-by: Luc Maranget Co-Developed-by: "Paul E. McKenney" Signed-off-by: Alan Stern Signed-off-by: Andrea Parri Signed-off-by: Jade Alglave Signed-off-by: Luc Maranget Signed-off-by: "Paul E. McKenney" Reviewed-by: Boqun Feng Acked-by: Will Deacon Acked-by: Peter Zijlstra Acked-by: Nicholas Piggin Acked-by: David Howells Acked-by: "Reshetova, Elena" Acked-by: Michal Hocko Acked-by: Akira Yokosawa Cc: --- tools/memory-model/Documentation/cheatsheet.txt | 30 + tools/memory-model/Documentation/explanation.txt | 1840 ++++++++++++++++++++ tools/memory-model/Documentation/recipes.txt | 570 ++++++ tools/memory-model/Documentation/references.txt | 107 ++ tools/memory-model/MAINTAINERS | 15 + tools/memory-model/README | 220 +++ tools/memory-model/linux-kernel.bell | 53 + tools/memory-model/linux-kernel.cat | 124 ++ tools/memory-model/linux-kernel.cfg | 21 + tools/memory-model/linux-kernel.def | 108 ++ .../litmus-tests/CoRR+poonceonce+Once.litmus | 19 + .../litmus-tests/CoRW+poonceonce+Once.litmus | 18 + .../litmus-tests/CoWR+poonceonce+Once.litmus | 18 + .../litmus-tests/CoWW+poonceonce.litmus | 11 + .../litmus-tests/IRIW+mbonceonces+OnceOnce.litmus | 35 + .../litmus-tests/IRIW+poonceonces+OnceOnce.litmus | 33 + .../litmus-tests/ISA2+poonceonces.litmus | 28 + ...cerelease+poacquirerelease+poacquireonce.litmus | 28 + .../litmus-tests/LB+ctrlonceonce+mbonceonce.litmus | 23 + .../LB+poacquireonce+pooncerelease.litmus | 21 + .../litmus-tests/LB+poonceonces.litmus | 21 + .../litmus-tests/MP+onceassign+derefonce.litmus | 25 + tools/memory-model/litmus-tests/MP+polocks.litmus | 24 + .../litmus-tests/MP+poonceonces.litmus | 20 + .../MP+pooncerelease+poacquireonce.litmus | 20 + .../memory-model/litmus-tests/MP+porevlocks.litmus | 24 + .../litmus-tests/MP+wmbonceonce+rmbonceonce.litmus | 22 + .../memory-model/litmus-tests/R+mbonceonces.litmus | 21 + .../memory-model/litmus-tests/R+poonceonces.litmus | 19 + tools/memory-model/litmus-tests/README | 125 ++ .../memory-model/litmus-tests/S+poonceonces.litmus | 19 + .../S+wmbonceonce+poacquireonce.litmus | 20 + .../litmus-tests/SB+mbonceonces.litmus | 23 + .../litmus-tests/SB+poonceonces.litmus | 21 + .../litmus-tests/WRC+poonceonces+Once.litmus | 27 + .../WRC+pooncerelease+rmbonceonce+Once.litmus | 28 + .../Z6.0+pooncelock+poonceLock+pombonce.litmus | 33 + .../Z6.0+pooncelock+pooncelock+pombonce.litmus | 32 + ...ooncerelease+poacquirerelease+mbonceonce.litmus | 28 + tools/memory-model/lock.cat | 99 ++ 40 files changed, 3973 insertions(+) create mode 100644 tools/memory-model/Documentation/cheatsheet.txt create mode 100644 tools/memory-model/Documentation/explanation.txt create mode 100644 tools/memory-model/Documentation/recipes.txt create mode 100644 tools/memory-model/Documentation/references.txt create mode 100644 tools/memory-model/MAINTAINERS create mode 100644 tools/memory-model/README create mode 100644 tools/memory-model/linux-kernel.bell create mode 100644 tools/memory-model/linux-kernel.cat create mode 100644 tools/memory-model/linux-kernel.cfg create mode 100644 tools/memory-model/linux-kernel.def create mode 100644 tools/memory-model/litmus-tests/CoRR+poonceonce+Once.litmus create mode 100644 tools/memory-model/litmus-tests/CoRW+poonceonce+Once.litmus create mode 100644 tools/memory-model/litmus-tests/CoWR+poonceonce+Once.litmus create mode 100644 tools/memory-model/litmus-tests/CoWW+poonceonce.litmus create mode 100644 tools/memory-model/litmus-tests/IRIW+mbonceonces+OnceOnce.litmus create mode 100644 tools/memory-model/litmus-tests/IRIW+poonceonces+OnceOnce.litmus create mode 100644 tools/memory-model/litmus-tests/ISA2+poonceonces.litmus create mode 100644 tools/memory-model/litmus-tests/ISA2+pooncerelease+poacquirerelease+poacquireonce.litmus create mode 100644 tools/memory-model/litmus-tests/LB+ctrlonceonce+mbonceonce.litmus create mode 100644 tools/memory-model/litmus-tests/LB+poacquireonce+pooncerelease.litmus create mode 100644 tools/memory-model/litmus-tests/LB+poonceonces.litmus create mode 100644 tools/memory-model/litmus-tests/MP+onceassign+derefonce.litmus create mode 100644 tools/memory-model/litmus-tests/MP+polocks.litmus create mode 100644 tools/memory-model/litmus-tests/MP+poonceonces.litmus create mode 100644 tools/memory-model/litmus-tests/MP+pooncerelease+poacquireonce.litmus create mode 100644 tools/memory-model/litmus-tests/MP+porevlocks.litmus create mode 100644 tools/memory-model/litmus-tests/MP+wmbonceonce+rmbonceonce.litmus create mode 100644 tools/memory-model/litmus-tests/R+mbonceonces.litmus create mode 100644 tools/memory-model/litmus-tests/R+poonceonces.litmus create mode 100644 tools/memory-model/litmus-tests/README create mode 100644 tools/memory-model/litmus-tests/S+poonceonces.litmus create mode 100644 tools/memory-model/litmus-tests/S+wmbonceonce+poacquireonce.litmus create mode 100644 tools/memory-model/litmus-tests/SB+mbonceonces.litmus create mode 100644 tools/memory-model/litmus-tests/SB+poonceonces.litmus create mode 100644 tools/memory-model/litmus-tests/WRC+poonceonces+Once.litmus create mode 100644 tools/memory-model/litmus-tests/WRC+pooncerelease+rmbonceonce+Once.litmus create mode 100644 tools/memory-model/litmus-tests/Z6.0+pooncelock+poonceLock+pombonce.litmus create mode 100644 tools/memory-model/litmus-tests/Z6.0+pooncelock+pooncelock+pombonce.litmus create mode 100644 tools/memory-model/litmus-tests/Z6.0+pooncerelease+poacquirerelease+mbonceonce.litmus create mode 100644 tools/memory-model/lock.cat (limited to 'tools') diff --git a/tools/memory-model/Documentation/cheatsheet.txt b/tools/memory-model/Documentation/cheatsheet.txt new file mode 100644 index 000000000000..1917712bce99 --- /dev/null +++ b/tools/memory-model/Documentation/cheatsheet.txt @@ -0,0 +1,30 @@ + Prior Operation Subsequent Operation + --------------- --------------------------- + C Self R W RWM Self R W DR DW RMW SV + __ ---- - - --- ---- - - -- -- --- -- + +Store, e.g., WRITE_ONCE() Y Y +Load, e.g., READ_ONCE() Y Y Y +Unsuccessful RMW operation Y Y Y +smp_read_barrier_depends() Y Y Y +*_dereference() Y Y Y Y +Successful *_acquire() R Y Y Y Y Y Y +Successful *_release() C Y Y Y W Y +smp_rmb() Y R Y Y R +smp_wmb() Y W Y Y W +smp_mb() & synchronize_rcu() CP Y Y Y Y Y Y Y Y +Successful full non-void RMW CP Y Y Y Y Y Y Y Y Y Y Y +smp_mb__before_atomic() CP Y Y Y a a a a Y +smp_mb__after_atomic() CP a a Y Y Y Y Y + + +Key: C: Ordering is cumulative + P: Ordering propagates + R: Read, for example, READ_ONCE(), or read portion of RMW + W: Write, for example, WRITE_ONCE(), or write portion of RMW + Y: Provides ordering + a: Provides ordering given intervening RMW atomic operation + DR: Dependent read (address dependency) + DW: Dependent write (address, data, or control dependency) + RMW: Atomic read-modify-write operation + SV Same-variable access diff --git a/tools/memory-model/Documentation/explanation.txt b/tools/memory-model/Documentation/explanation.txt new file mode 100644 index 000000000000..867e0ea69b6d --- /dev/null +++ b/tools/memory-model/Documentation/explanation.txt @@ -0,0 +1,1840 @@ +Explanation of the Linux-Kernel Memory Model +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +:Author: Alan Stern +:Created: October 2017 + +.. Contents + + 1. INTRODUCTION + 2. BACKGROUND + 3. A SIMPLE EXAMPLE + 4. A SELECTION OF MEMORY MODELS + 5. ORDERING AND CYCLES + 6. EVENTS + 7. THE PROGRAM ORDER RELATION: po AND po-loc + 8. A WARNING + 9. DEPENDENCY RELATIONS: data, addr, and ctrl + 10. THE READS-FROM RELATION: rf, rfi, and rfe + 11. CACHE COHERENCE AND THE COHERENCE ORDER RELATION: co, coi, and coe + 12. THE FROM-READS RELATION: fr, fri, and fre + 13. AN OPERATIONAL MODEL + 14. PROPAGATION ORDER RELATION: cumul-fence + 15. DERIVATION OF THE LKMM FROM THE OPERATIONAL MODEL + 16. SEQUENTIAL CONSISTENCY PER VARIABLE + 17. ATOMIC UPDATES: rmw + 18. THE PRESERVED PROGRAM ORDER RELATION: ppo + 19. AND THEN THERE WAS ALPHA + 20. THE HAPPENS-BEFORE RELATION: hb + 21. THE PROPAGATES-BEFORE RELATION: pb + 22. RCU RELATIONS: link, gp-link, rscs-link, and rcu-path + 23. ODDS AND ENDS + + + +INTRODUCTION +------------ + +The Linux-kernel memory model (LKMM) is rather complex and obscure. +This is particularly evident if you read through the linux-kernel.bell +and linux-kernel.cat files that make up the formal version of the +memory model; they are extremely terse and their meanings are far from +clear. + +This document describes the ideas underlying the LKMM. It is meant +for people who want to understand how the memory model was designed. +It does not go into the details of the code in the .bell and .cat +files; rather, it explains in English what the code expresses +symbolically. + +Sections 2 (BACKGROUND) through 5 (ORDERING AND CYCLES) are aimed +toward beginners; they explain what memory models are and the basic +notions shared by all such models. People already familiar with these +concepts can skim or skip over them. Sections 6 (EVENTS) through 12 +(THE FROM_READS RELATION) describe the fundamental relations used in +many memory models. Starting in Section 13 (AN OPERATIONAL MODEL), +the workings of the LKMM itself are covered. + +Warning: The code examples in this document are not written in the +proper format for litmus tests. They don't include a header line, the +initializations are not enclosed in braces, the global variables are +not passed by pointers, and they don't have an "exists" clause at the +end. Converting them to the right format is left as an exercise for +the reader. + + +BACKGROUND +---------- + +A memory consistency model (or just memory model, for short) is +something which predicts, given a piece of computer code running on a +particular kind of system, what values may be obtained by the code's +load instructions. The LKMM makes these predictions for code running +as part of the Linux kernel. + +In practice, people tend to use memory models the other way around. +That is, given a piece of code and a collection of values specified +for the loads, the model will predict whether it is possible for the +code to run in such a way that the loads will indeed obtain the +specified values. Of course, this is just another way of expressing +the same idea. + +For code running on a uniprocessor system, the predictions are easy: +Each load instruction must obtain the value written by the most recent +store instruction accessing the same location (we ignore complicating +factors such as DMA and mixed-size accesses.) But on multiprocessor +systems, with multiple CPUs making concurrent accesses to shared +memory locations, things aren't so simple. + +Different architectures have differing memory models, and the Linux +kernel supports a variety of architectures. The LKMM has to be fairly +permissive, in the sense that any behavior allowed by one of these +architectures also has to be allowed by the LKMM. + + +A SIMPLE EXAMPLE +---------------- + +Here is a simple example to illustrate the basic concepts. Consider +some code running as part of a device driver for an input device. The +driver might contain an interrupt handler which collects data from the +device, stores it in a buffer, and sets a flag to indicate the buffer +is full. Running concurrently on a different CPU might be a part of +the driver code being executed by a process in the midst of a read(2) +system call. This code tests the flag to see whether the buffer is +ready, and if it is, copies the data back to userspace. The buffer +and the flag are memory locations shared between the two CPUs. + +We can abstract out the important pieces of the driver code as follows +(the reason for using WRITE_ONCE() and READ_ONCE() instead of simple +assignment statements is discussed later): + + int buf = 0, flag = 0; + + P0() + { + WRITE_ONCE(buf, 1); + WRITE_ONCE(flag, 1); + } + + P1() + { + int r1; + int r2 = 0; + + r1 = READ_ONCE(flag); + if (r1) + r2 = READ_ONCE(buf); + } + +Here the P0() function represents the interrupt handler running on one +CPU and P1() represents the read() routine running on another. The +value 1 stored in buf represents input data collected from the device. +Thus, P0 stores the data in buf and then sets flag. Meanwhile, P1 +reads flag into the private variable r1, and if it is set, reads the +data from buf into a second private variable r2 for copying to +userspace. (Presumably if flag is not set then the driver will wait a +while and try again.) + +This pattern of memory accesses, where one CPU stores values to two +shared memory locations and another CPU loads from those locations in +the opposite order, is widely known as the "Message Passing" or MP +pattern. It is typical of memory access patterns in the kernel. + +Please note that this example code is a simplified abstraction. Real +buffers are usually larger than a single integer, real device drivers +usually use sleep and wakeup mechanisms rather than polling for I/O +completion, and real code generally doesn't bother to copy values into +private variables before using them. All that is beside the point; +the idea here is simply to illustrate the overall pattern of memory +accesses by the CPUs. + +A memory model will predict what values P1 might obtain for its loads +from flag and buf, or equivalently, what values r1 and r2 might end up +with after the code has finished running. + +Some predictions are trivial. For instance, no sane memory model would +predict that r1 = 42 or r2 = -7, because neither of those values ever +gets stored in flag or buf. + +Some nontrivial predictions are nonetheless quite simple. For +instance, P1 might run entirely before P0 begins, in which case r1 and +r2 will both be 0 at the end. Or P0 might run entirely before P1 +begins, in which case r1 and r2 will both be 1. + +The interesting predictions concern what might happen when the two +routines run concurrently. One possibility is that P1 runs after P0's +store to buf but before the store to flag. In this case, r1 and r2 +will again both be 0. (If P1 had been designed to read buf +unconditionally then we would instead have r1 = 0 and r2 = 1.) + +However, the most interesting possibility is where r1 = 1 and r2 = 0. +If this were to occur it would mean the driver contains a bug, because +incorrect data would get sent to the user: 0 instead of 1. As it +happens, the LKMM does predict this outcome can occur, and the example +driver code shown above is indeed buggy. + + +A SELECTION OF MEMORY MODELS +---------------------------- + +The first widely cited memory model, and the simplest to understand, +is Sequential Consistency. According to this model, systems behave as +if each CPU executed its instructions in order but with unspecified +timing. In other words, the instructions from the various CPUs get +interleaved in a nondeterministic way, always according to some single +global order that agrees with the order of the instructions in the +program source for each CPU. The model says that the value obtained +by each load is simply the value written by the most recently executed +store to the same memory location, from any CPU. + +For the MP example code shown above, Sequential Consistency predicts +that the undesired result r1 = 1, r2 = 0 cannot occur. The reasoning +goes like this: + + Since r1 = 1, P0 must store 1 to flag before P1 loads 1 from + it, as loads can obtain values only from earlier stores. + + P1 loads from flag before loading from buf, since CPUs execute + their instructions in order. + + P1 must load 0 from buf before P0 stores 1 to it; otherwise r2 + would be 1 since a load obtains its value from the most recent + store to the same address. + + P0 stores 1 to buf before storing 1 to flag, since it executes + its instructions in order. + + Since an instruction (in this case, P1's store to flag) cannot + execute before itself, the specified outcome is impossible. + +However, real computer hardware almost never follows the Sequential +Consistency memory model; doing so would rule out too many valuable +performance optimizations. On ARM and PowerPC architectures, for +instance, the MP example code really does sometimes yield r1 = 1 and +r2 = 0. + +x86 and SPARC follow yet a different memory model: TSO (Total Store +Ordering). This model predicts that the undesired outcome for the MP +pattern cannot occur, but in other respects it differs from Sequential +Consistency. One example is the Store Buffer (SB) pattern, in which +each CPU stores to its own shared location and then loads from the +other CPU's location: + + int x = 0, y = 0; + + P0() + { + int r0; + + WRITE_ONCE(x, 1); + r0 = READ_ONCE(y); + } + + P1() + { + int r1; + + WRITE_ONCE(y, 1); + r1 = READ_ONCE(x); + } + +Sequential Consistency predicts that the outcome r0 = 0, r1 = 0 is +impossible. (Exercise: Figure out the reasoning.) But TSO allows +this outcome to occur, and in fact it does sometimes occur on x86 and +SPARC systems. + +The LKMM was inspired by the memory models followed by PowerPC, ARM, +x86, Alpha, and other architectures. However, it is different in +detail from each of them. + + +ORDERING AND CYCLES +------------------- + +Memory models are all about ordering. Often this is temporal ordering +(i.e., the order in which certain events occur) but it doesn't have to +be; consider for example the order of instructions in a program's +source code. We saw above that Sequential Consistency makes an +important assumption that CPUs execute instructions in the same order +as those instructions occur in the code, and there are many other +instances of ordering playing central roles in memory models. + +The counterpart to ordering is a cycle. Ordering rules out cycles: +It's not possible to have X ordered before Y, Y ordered before Z, and +Z ordered before X, because this would mean that X is ordered before +itself. The analysis of the MP example under Sequential Consistency +involved just such an impossible cycle: + + W: P0 stores 1 to flag executes before + X: P1 loads 1 from flag executes before + Y: P1 loads 0 from buf executes before + Z: P0 stores 1 to buf executes before + W: P0 stores 1 to flag. + +In short, if a memory model requires certain accesses to be ordered, +and a certain outcome for the loads in a piece of code can happen only +if those accesses would form a cycle, then the memory model predicts +that outcome cannot occur. + +The LKMM is defined largely in terms of cycles, as we will see. + + +EVENTS +------ + +The LKMM does not work directly with the C statements that make up +kernel source code. Instead it considers the effects of those +statements in a more abstract form, namely, events. The model +includes three types of events: + + Read events correspond to loads from shared memory, such as + calls to READ_ONCE(), smp_load_acquire(), or + rcu_dereference(). + + Write events correspond to stores to shared memory, such as + calls to WRITE_ONCE(), smp_store_release(), or atomic_set(). + + Fence events correspond to memory barriers (also known as + fences), such as calls to smp_rmb() or rcu_read_lock(). + +These categories are not exclusive; a read or write event can also be +a fence. This happens with functions like smp_load_acquire() or +spin_lock(). However, no single event can be both a read and a write. +Atomic read-modify-write accesses, such as atomic_inc() or xchg(), +correspond to a pair of events: a read followed by a write. (The +write event is omitted for executions where it doesn't occur, such as +a cmpxchg() where the comparison fails.) + +Other parts of the code, those which do not involve interaction with +shared memory, do not give rise to events. Thus, arithmetic and +logical computations, control-flow instructions, or accesses to +private memory or CPU registers are not of central interest to the +memory model. They only affect the model's predictions indirectly. +For example, an arithmetic computation might determine the value that +gets stored to a shared memory location (or in the case of an array +index, the address where the value gets stored), but the memory model +is concerned only with the store itself -- its value and its address +-- not the computation leading up to it. + +Events in the LKMM can be linked by various relations, which we will +describe in the following sections. The memory model requires certain +of these relations to be orderings, that is, it requires them not to +have any cycles. + + +THE PROGRAM ORDER RELATION: po AND po-loc +----------------------------------------- + +The most important relation between events is program order (po). You +can think of it as the order in which statements occur in the source +code after branches are taken into account and loops have been +unrolled. A better description might be the order in which +instructions are presented to a CPU's execution unit. Thus, we say +that X is po-before Y (written as "X ->po Y" in formulas) if X occurs +before Y in the instruction stream. + +This is inherently a single-CPU relation; two instructions executing +on different CPUs are never linked by po. Also, it is by definition +an ordering so it cannot have any cycles. + +po-loc is a sub-relation of po. It links two memory accesses when the +first comes before the second in program order and they access the +same memory location (the "-loc" suffix). + +Although this may seem straightforward, there is one subtle aspect to +program order we need to explain. The LKMM was inspired by low-level +architectural memory models which describe the behavior of machine +code, and it retains their outlook to a considerable extent. The +read, write, and fence events used by the model are close in spirit to +individual machine instructions. Nevertheless, the LKMM describes +kernel code written in C, and the mapping from C to machine code can +be extremely complex. + +Optimizing compilers have great freedom in the way they translate +source code to object code. They are allowed to apply transformations +that add memory accesses, eliminate accesses, combine them, split them +into pieces, or move them around. Faced with all these possibilities, +the LKMM basically gives up. It insists that the code it analyzes +must contain no ordinary accesses to shared memory; all accesses must +be performed using READ_ONCE(), WRITE_ONCE(), or one of the other +atomic or synchronization primitives. These primitives prevent a +large number of compiler optimizations. In particular, it is +guaranteed that the compiler will not remove such accesses from the +generated code (unless it can prove the accesses will never be +executed), it will not change the order in which they occur in the +code (within limits imposed by the C standard), and it will not +introduce extraneous accesses. + +This explains why the MP and SB examples above used READ_ONCE() and +WRITE_ONCE() rather than ordinary memory accesses. Thanks to this +usage, we can be certain that in the MP example, P0's write event to +buf really is po-before its write event to flag, and similarly for the +other shared memory accesses in the examples. + +Private variables are not subject to this restriction. Since they are +not shared between CPUs, they can be accessed normally without +READ_ONCE() or WRITE_ONCE(), and there will be no ill effects. In +fact, they need not even be stored in normal memory at all -- in +principle a private variable could be stored in a CPU register (hence +the convention that these variables have names starting with the +letter 'r'). + + +A WARNING +--------- + +The protections provided by READ_ONCE(), WRITE_ONCE(), and others are +not perfect; and under some circumstances it is possible for the +compiler to undermine the memory model. Here is an example. Suppose +both branches of an "if" statement store the same value to the same +location: + + r1 = READ_ONCE(x); + if (r1) { + WRITE_ONCE(y, 2); + ... /* do something */ + } else { + WRITE_ONCE(y, 2); + ... /* do something else */ + } + +For this code, the LKMM predicts that the load from x will always be +executed before either of the stores to y. However, a compiler could +lift the stores out of the conditional, transforming the code into +something resembling: + + r1 = READ_ONCE(x); + WRITE_ONCE(y, 2); + if (r1) { + ... /* do something */ + } else { + ... /* do something else */ + } + +Given this version of the code, the LKMM would predict that the load +from x could be executed after the store to y. Thus, the memory +model's original prediction could be invalidated by the compiler. + +Another issue arises from the fact that in C, arguments to many +operators and function calls can be evaluated in any order. For +example: + + r1 = f(5) + g(6); + +The object code might call f(5) either before or after g(6); the +memory model cannot assume there is a fixed program order relation +between them. (In fact, if the functions are inlined then the +compiler might even interleave their object code.) + + +DEPENDENCY RELATIONS: data, addr, and ctrl +------------------------------------------ + +We say that two events are linked by a dependency relation when the +execution of the second event depends in some way on a value obtained +from memory by the first. The first event must be a read, and the +value it obtains must somehow affect what the second event does. +There are three kinds of dependencies: data, address (addr), and +control (ctrl). + +A read and a write event are linked by a data dependency if the value +obtained by the read affects the value stored by the write. As a very +simple example: + + int x, y; + + r1 = READ_ONCE(x); + WRITE_ONCE(y, r1 + 5); + +The value stored by the WRITE_ONCE obviously depends on the value +loaded by the READ_ONCE. Such dependencies can wind through +arbitrarily complicated computations, and a write can depend on the +values of multiple reads. + +A read event and another memory access event are linked by an address +dependency if the value obtained by the read affects the location +accessed by the other event. The second event can be either a read or +a write. Here's another simple example: + + int a[20]; + int i; + + r1 = READ_ONCE(i); + r2 = READ_ONCE(a[r1]); + +Here the location accessed by the second READ_ONCE() depends on the +index value loaded by the first. Pointer indirection also gives rise +to address dependencies, since the address of a location accessed +through a pointer will depend on the value read earlier from that +pointer. + +Finally, a read event and another memory access event are linked by a +control dependency if the value obtained by the read affects whether +the second event is executed at all. Simple example: + + int x, y; + + r1 = READ_ONCE(x); + if (r1) + WRITE_ONCE(y, 1984); + +Execution of the WRITE_ONCE() is controlled by a conditional expression +which depends on the value obtained by the READ_ONCE(); hence there is +a control dependency from the load to the store. + +It should be pretty obvious that events can only depend on reads that +come earlier in program order. Symbolically, if we have R ->data X, +R ->addr X, or R ->ctrl X (where R is a read event), then we must also +have R ->po X. It wouldn't make sense for a computation to depend +somehow on a value that doesn't get loaded from shared memory until +later in the code! + + +THE READS-FROM RELATION: rf, rfi, and rfe +----------------------------------------- + +The reads-from relation (rf) links a write event to a read event when +the value loaded by the read is the value that was stored by the +write. In colloquial terms, the load "reads from" the store. We +write W ->rf R to indicate that the load R reads from the store W. We +further distinguish the cases where the load and the store occur on +the same CPU (internal reads-from, or rfi) and where they occur on +different CPUs (external reads-from, or rfe). + +For our purposes, a memory location's initial value is treated as +though it had been written there by an imaginary initial store that +executes on a separate CPU before the program runs. + +Usage of the rf relation implicitly assumes that loads will always +read from a single store. It doesn't apply properly in the presence +of load-tearing, where a load obtains some of its bits from one store +and some of them from another store. Fortunately, use of READ_ONCE() +and WRITE_ONCE() will prevent load-tearing; it's not possible to have: + + int x = 0; + + P0() + { + WRITE_ONCE(x, 0x1234); + } + + P1() + { + int r1; + + r1 = READ_ONCE(x); + } + +and end up with r1 = 0x1200 (partly from x's initial value and partly +from the value stored by P0). + +On the other hand, load-tearing is unavoidable when mixed-size +accesses are used. Consider this example: + + union { + u32 w; + u16 h[2]; + } x; + + P0() + { + WRITE_ONCE(x.h[0], 0x1234); + WRITE_ONCE(x.h[1], 0x5678); + } + + P1() + { + int r1; + + r1 = READ_ONCE(x.w); + } + +If r1 = 0x56781234 (little-endian!) at the end, then P1 must have read +from both of P0's stores. It is possible to handle mixed-size and +unaligned accesses in a memory model, but the LKMM currently does not +attempt to do so. It requires all accesses to be properly aligned and +of the location's actual size. + + +CACHE COHERENCE AND THE COHERENCE ORDER RELATION: co, coi, and coe +------------------------------------------------------------------ + +Cache coherence is a general principle requiring that in a +multi-processor system, the CPUs must share a consistent view of the +memory contents. Specifically, it requires that for each location in +shared memory, the stores to that location must form a single global +ordering which all the CPUs agree on (the coherence order), and this +ordering must be consistent with the program order for accesses to +that location. + +To put it another way, for any variable x, the coherence order (co) of +the stores to x is simply the order in which the stores overwrite one +another. The imaginary store which establishes x's initial value +comes first in the coherence order; the store which directly +overwrites the initial value comes second; the store which overwrites +that value comes third, and so on. + +You can think of the coherence order as being the order in which the +stores reach x's location in memory (or if you prefer a more +hardware-centric view, the order in which the stores get written to +x's cache line). We write W ->co W' if W comes before W' in the +coherence order, that is, if the value stored by W gets overwritten, +directly or indirectly, by the value stored by W'. + +Coherence order is required to be consistent with program order. This +requirement takes the form of four coherency rules: + + Write-write coherence: If W ->po-loc W' (i.e., W comes before + W' in program order and they access the same location), where W + and W' are two stores, then W ->co W'. + + Write-read coherence: If W ->po-loc R, where W is a store and R + is a load, then R must read from W or from some other store + which comes after W in the coherence order. + + Read-write coherence: If R ->po-loc W, where R is a load and W + is a store, then the store which R reads from must come before + W in the coherence order. + + Read-read coherence: If R ->po-loc R', where R and R' are two + loads, then either they read from the same store or else the + store read by R comes before the store read by R' in the + coherence order. + +This is sometimes referred to as sequential consistency per variable, +because it means that the accesses to any single memory location obey +the rules of the Sequential Consistency memory model. (According to +Wikipedia, sequential consistency per variable and cache coherence +mean the same thing except that cache coherence includes an extra +requirement that every store eventually becomes visible to every CPU.) + +Any reasonable memory model will include cache coherence. Indeed, our +expectation of cache coherence is so deeply ingrained that violations +of its requirements look more like hardware bugs than programming +errors: + + int x; + + P0() + { + WRITE_ONCE(x, 17); + WRITE_ONCE(x, 23); + } + +If the final value stored in x after this code ran was 17, you would +think your computer was broken. It would be a violation of the +write-write coherence rule: Since the store of 23 comes later in +program order, it must also come later in x's coherence order and +thus must overwrite the store of 17. + + int x = 0; + + P0() + { + int r1; + + r1 = READ_ONCE(x); + WRITE_ONCE(x, 666); + } + +If r1 = 666 at the end, this would violate the read-write coherence +rule: The READ_ONCE() load comes before the WRITE_ONCE() store in +program order, so it must not read from that store but rather from one +coming earlier in the coherence order (in this case, x's initial +value). + + int x = 0; + + P0() + { + WRITE_ONCE(x, 5); + } + + P1() + { + int r1, r2; + + r1 = READ_ONCE(x); + r2 = READ_ONCE(x); + } + +If r1 = 5 (reading from P0's store) and r2 = 0 (reading from the +imaginary store which establishes x's initial value) at the end, this +would violate the read-read coherence rule: The r1 load comes before +the r2 load in program order, so it must not read from a store that +comes later in the coherence order. + +(As a minor curiosity, if this code had used normal loads instead of +READ_ONCE() in P1, on Itanium it sometimes could end up with r1 = 5 +and r2 = 0! This results from parallel execution of the operations +encoded in Itanium's Very-Long-Instruction-Word format, and it is yet +another motivation for using READ_ONCE() when accessing shared memory +locations.) + +Just like the po relation, co is inherently an ordering -- it is not +possible for a store to directly or indirectly overwrite itself! And +just like with the rf relation, we distinguish between stores that +occur on the same CPU (internal coherence order, or coi) and stores +that occur on different CPUs (external coherence order, or coe). + +On the other hand, stores to different memory locations are never +related by co, just as instructions on different CPUs are never +related by po. Coherence order is strictly per-location, or if you +prefer, each location has its own independent coherence order. + + +THE FROM-READS RELATION: fr, fri, and fre +----------------------------------------- + +The from-reads relation (fr) can be a little difficult for people to +grok. It describes the situation where a load reads a value that gets +overwritten by a store. In other words, we have R ->fr W when the +value that R reads is overwritten (directly or indirectly) by W, or +equivalently, when R reads from a store which comes earlier than W in +the coherence order. + +For example: + + int x = 0; + + P0() + { + int r1; + + r1 = READ_ONCE(x); + WRITE_ONCE(x, 2); + } + +The value loaded from x will be 0 (assuming cache coherence!), and it +gets overwritten by the value 2. Thus there is an fr link from the +READ_ONCE() to the WRITE_ONCE(). If the code contained any later +stores to x, there would also be fr links from the READ_ONCE() to +them. + +As with rf, rfi, and rfe, we subdivide the fr relation into fri (when +the load and the store are on the same CPU) and fre (when they are on +different CPUs). + +Note that the fr relation is determined entirely by the rf and co +relations; it is not independent. Given a read event R and a write +event W for the same location, we will have R ->fr W if and only if +the write which R reads from is co-before W. In symbols, + + (R ->fr W) := (there exists W' with W' ->rf R and W' ->co W). + + +AN OPERATIONAL MODEL +-------------------- + +The LKMM is based on various operational memory models, meaning that +the models arise from an abstract view of how a computer system +operates. Here are the main ideas, as incorporated into the LKMM. + +The system as a whole is divided into the CPUs and a memory subsystem. +The CPUs are responsible for executing instructions (not necessarily +in program order), and they communicate with the memory subsystem. +For the most part, executing an instruction requires a CPU to perform +only internal operations. However, loads, stores, and fences involve +more. + +When CPU C executes a store instruction, it tells the memory subsystem +to store a certain value at a certain location. The memory subsystem +propagates the store to all the other CPUs as well as to RAM. (As a +special case, we say that the store propagates to its own CPU at the +time it is executed.) The memory subsystem also determines where the +store falls in the location's coherence order. In particular, it must +arrange for the store to be co-later than (i.e., to overwrite) any +other store to the same location which has already propagated to CPU C. + +When a CPU executes a load instruction R, it first checks to see +whether there are any as-yet unexecuted store instructions, for the +same location, that come before R in program order. If there are, it +uses the value of the po-latest such store as the value obtained by R, +and we say that the store's value is forwarded to R. Otherwise, the +CPU asks the memory subsystem for the value to load and we say that R +is satisfied from memory. The memory subsystem hands back the value +of the co-latest store to the location in question which has already +propagated to that CPU. + +(In fact, the picture needs to be a little more complicated than this. +CPUs have local caches, and propagating a store to a CPU really means +propagating it to the CPU's local cache. A local cache can take some +time to process the stores that it receives, and a store can't be used +to satisfy one of the CPU's loads until it has been processed. On +most architectures, the local caches process stores in +First-In-First-Out order, and consequently the processing delay +doesn't matter for the memory model. But on Alpha, the local caches +have a partitioned design that results in non-FIFO behavior. We will +discuss this in more detail later.) + +Note that load instructions may be executed speculatively and may be +restarted under certain circumstances. The memory model ignores these +premature executions; we simply say that the load executes at the +final time it is forwarded or satisfied. + +Executing a fence (or memory barrier) instruction doesn't require a +CPU to do anything special other than informing the memory subsystem +about the fence. However, fences do constrain the way CPUs and the +memory subsystem handle other instructions, in two respects. + +First, a fence forces the CPU to execute various instructions in +program order. Exactly which instructions are ordered depends on the +type of fence: + + Strong fences, including smp_mb() and synchronize_rcu(), force + the CPU to execute all po-earlier instructions before any + po-later instructions; + + smp_rmb() forces the CPU to execute all po-earlier loads + before any po-later loads; + + smp_wmb() forces the CPU to execute all po-earlier stores + before any po-later stores; + + Acquire fences, such as smp_load_acquire(), force the CPU to + execute the load associated with the fence (e.g., the load + part of an smp_load_acquire()) before any po-later + instructions; + + Release fences, such as smp_store_release(), force the CPU to + execute all po-earlier instructions before the store + associated with the fence (e.g., the store part of an + smp_store_release()). + +Second, some types of fence affect the way the memory subsystem +propagates stores. When a fence instruction is executed on CPU C: + + For each other CPU C', smb_wmb() forces all po-earlier stores + on C to propagate to C' before any po-later stores do. + + For each other CPU C', any store which propagates to C before + a release fence is executed (including all po-earlier + stores executed on C) is forced to propagate to C' before the + store associated with the release fence does. + + Any store which propagates to C before a strong fence is + executed (including all po-earlier stores on C) is forced to + propagate to all other CPUs before any instructions po-after + the strong fence are executed on C. + +The propagation ordering enforced by release fences and strong fences +affects stores from other CPUs that propagate to CPU C before the +fence is executed, as well as stores that are executed on C before the +fence. We describe this property by saying that release fences and +strong fences are A-cumulative. By contrast, smp_wmb() fences are not +A-cumulative; they only affect the propagation of stores that are +executed on C before the fence (i.e., those which precede the fence in +program order). + +smp_read_barrier_depends(), rcu_read_lock(), rcu_read_unlock(), and +synchronize_rcu() fences have other properties which we discuss later. + + +PROPAGATION ORDER RELATION: cumul-fence +--------------------------------------- + +The fences which affect propagation order (i.e., strong, release, and +smp_wmb() fences) are collectively referred to as cumul-fences, even +though smp_wmb() isn't A-cumulative. The cumul-fence relation is +defined to link memory access events E and F whenever: + + E and F are both stores on the same CPU and an smp_wmb() fence + event occurs between them in program order; or + + F is a release fence and some X comes before F in program order, + where either X = E or else E ->rf X; or + + A strong fence event occurs between some X and F in program + order, where either X = E or else E ->rf X. + +The operational model requires that whenever W and W' are both stores +and W ->cumul-fence W', then W must propagate to any given CPU +before W' does. However, for different CPUs C and C', it does not +require W to propagate to C before W' propagates to C'. + + +DERIVATION OF THE LKMM FROM THE OPERATIONAL MODEL +------------------------------------------------- + +The LKMM is derived from the restrictions imposed by the design +outlined above. These restrictions involve the necessity of +maintaining cache coherence and the fact that a CPU can't operate on a +value before it knows what that value is, among other things. + +The formal version of the LKMM is defined by five requirements, or +axioms: + + Sequential consistency per variable: This requires that the + system obey the four coherency rules. + + Atomicity: This requires that atomic read-modify-write + operations really are atomic, that is, no other stores can + sneak into the middle of such an update. + + Happens-before: This requires that certain instructions are + executed in a specific order. + + Propagation: This requires that certain stores propagate to + CPUs and to RAM in a specific order. + + Rcu: This requires that RCU read-side critical sections and + grace periods obey the rules of RCU, in particular, the + Grace-Period Guarantee. + +The first and second are quite common; they can be found in many +memory models (such as those for C11/C++11). The "happens-before" and +"propagation" axioms have analogs in other memory models as well. The +"rcu" axiom is specific to the LKMM. + +Each of these axioms is discussed below. + + +SEQUENTIAL CONSISTENCY PER VARIABLE +----------------------------------- + +According to the principle of cache coherence, the stores to any fixed +shared location in memory form a global ordering. We can imagine +inserting the loads from that location into this ordering, by placing +each load between the store that it reads from and the following +store. This leaves the relative positions of loads that read from the +same store unspecified; let's say they are inserted in program order, +first for CPU 0, then CPU 1, etc. + +You can check that the four coherency rules imply that the rf, co, fr, +and po-loc relations agree with this global ordering; in other words, +whenever we have X ->rf Y or X ->co Y or X ->fr Y or X ->po-loc Y, the +X event comes before the Y event in the global ordering. The LKMM's +"coherence" axiom expresses this by requiring the union of these +relations not to have any cycles. This means it must not be possible +to find events + + X0 -> X1 -> X2 -> ... -> Xn -> X0, + +where each of the links is either rf, co, fr, or po-loc. This has to +hold if the accesses to the fixed memory location can be ordered as +cache coherence demands. + +Although it is not obvious, it can be shown that the converse is also +true: This LKMM axiom implies that the four coherency rules are +obeyed. + + +ATOMIC UPDATES: rmw +------------------- + +What does it mean to say that a read-modify-write (rmw) update, such +as atomic_inc(&x), is atomic? It means that the memory location (x in +this case) does not get altered between the read and the write events +making up the atomic operation. In particular, if two CPUs perform +atomic_inc(&x) concurrently, it must be guaranteed that the final +value of x will be the initial value plus two. We should never have +the following sequence of events: + + CPU 0 loads x obtaining 13; + CPU 1 loads x obtaining 13; + CPU 0 stores 14 to x; + CPU 1 stores 14 to x; + +where the final value of x is wrong (14 rather than 15). + +In this example, CPU 0's increment effectively gets lost because it +occurs in between CPU 1's load and store. To put it another way, the +problem is that the position of CPU 0's store in x's coherence order +is between the store that CPU 1 reads from and the store that CPU 1 +performs. + +The same analysis applies to all atomic update operations. Therefore, +to enforce atomicity the LKMM requires that atomic updates follow this +rule: Whenever R and W are the read and write events composing an +atomic read-modify-write and W' is the write event which R reads from, +there must not be any stores coming between W' and W in the coherence +order. Equivalently, + + (R ->rmw W) implies (there is no X with R ->fr X and X ->co W), + +where the rmw relation links the read and write events making up each +atomic update. This is what the LKMM's "atomic" axiom says. + + +THE PRESERVED PROGRAM ORDER RELATION: ppo +----------------------------------------- + +There are many situations where a CPU is obligated to execute two +instructions in program order. We amalgamate them into the ppo (for +"preserved program order") relation, which links the po-earlier +instruction to the po-later instruction and is thus a sub-relation of +po. + +The operational model already includes a description of one such +situation: Fences are a source of ppo links. Suppose X and Y are +memory accesses with X ->po Y; then the CPU must execute X before Y if +any of the following hold: + + A strong (smp_mb() or synchronize_rcu()) fence occurs between + X and Y; + + X and Y are both stores and an smp_wmb() fence occurs between + them; + + X and Y are both loads and an smp_rmb() fence occurs between + them; + + X is also an acquire fence, such as smp_load_acquire(); + + Y is also a release fence, such as smp_store_release(). + +Another possibility, not mentioned earlier but discussed in the next +section, is: + + X and Y are both loads, X ->addr Y (i.e., there is an address + dependency from X to Y), and an smp_read_barrier_depends() + fence occurs between them. + +Dependencies can also cause instructions to be executed in program +order. This is uncontroversial when the second instruction is a +store; either a data, address, or control dependency from a load R to +a store W will force the CPU to execute R before W. This is very +simply because the CPU cannot tell the memory subsystem about W's +store before it knows what value should be stored (in the case of a +data dependency), what location it should be stored into (in the case +of an address dependency), or whether the store should actually take +place (in the case of a control dependency). + +Dependencies to load instructions are more problematic. To begin with, +there is no such thing as a data dependency to a load. Next, a CPU +has no reason to respect a control dependency to a load, because it +can always satisfy the second load speculatively before the first, and +then ignore the result if it turns out that the second load shouldn't +be executed after all. And lastly, the real difficulties begin when +we consider address dependencies to loads. + +To be fair about it, all Linux-supported architectures do execute +loads in program order if there is an address dependency between them. +After all, a CPU cannot ask the memory subsystem to load a value from +a particular location before it knows what that location is. However, +the split-cache design used by Alpha can cause it to behave in a way +that looks as if the loads were executed out of order (see the next +section for more details). For this reason, the LKMM does not include +address dependencies between read events in the ppo relation unless an +smp_read_barrier_depends() fence is present. + +On the other hand, dependencies can indirectly affect the ordering of +two loads. This happens when there is a dependency from a load to a +store and a second, po-later load reads from that store: + + R ->dep W ->rfi R', + +where the dep link can be either an address or a data dependency. In +this situation we know it is possible for the CPU to execute R' before +W, because it can forward the value that W will store to R'. But it +cannot execute R' before R, because it cannot forward the value before +it knows what that value is, or that W and R' do access the same +location. However, if there is merely a control dependency between R +and W then the CPU can speculatively forward W to R' before executing +R; if the speculation turns out to be wrong then the CPU merely has to +restart or abandon R'. + +(In theory, a CPU might forward a store to a load when it runs across +an address dependency like this: + + r1 = READ_ONCE(ptr); + WRITE_ONCE(*r1, 17); + r2 = READ_ONCE(*r1); + +because it could tell that the store and the second load access the +same location even before it knows what the location's address is. +However, none of the architectures supported by the Linux kernel do +this.) + +Two memory accesses of the same location must always be executed in +program order if the second access is a store. Thus, if we have + + R ->po-loc W + +(the po-loc link says that R comes before W in program order and they +access the same location), the CPU is obliged to execute W after R. +If it executed W first then the memory subsystem would respond to R's +read request with the value stored by W (or an even later store), in +violation of the read-write coherence rule. Similarly, if we had + + W ->po-loc W' + +and the CPU executed W' before W, then the memory subsystem would put +W' before W in the coherence order. It would effectively cause W to +overwrite W', in violation of the write-write coherence rule. +(Interestingly, an early ARMv8 memory model, now obsolete, proposed +allowing out-of-order writes like this to occur. The model avoided +violating the write-write coherence rule by requiring the CPU not to +send the W write to the memory subsystem at all!) + +There is one last example of preserved program order in the LKMM: when +a load-acquire reads from an earlier store-release. For example: + + smp_store_release(&x, 123); + r1 = smp_load_acquire(&x); + +If the smp_load_acquire() ends up obtaining the 123 value that was +stored by the smp_store_release(), the LKMM says that the load must be +executed after the store; the store cannot be forwarded to the load. +This requirement does not arise from the operational model, but it +yields correct predictions on all architectures supported by the Linux +kernel, although for differing reasons. + +On some architectures, including x86 and ARMv8, it is true that the +store cannot be forwarded to the load. On others, including PowerPC +and ARMv7, smp_store_release() generates object code that starts with +a fence and smp_load_acquire() generates object code that ends with a +fence. The upshot is that even though the store may be forwarded to +the load, it is still true that any instruction preceding the store +will be executed before the load or any following instructions, and +the store will be executed before any instruction following the load. + + +AND THEN THERE WAS ALPHA +------------------------ + +As mentioned above, the Alpha architecture is unique in that it does +not appear to respect address dependencies to loads. This means that +code such as the following: + + int x = 0; + int y = -1; + int *ptr = &y; + + P0() + { + WRITE_ONCE(x, 1); + smp_wmb(); + WRITE_ONCE(ptr, &x); + } + + P1() + { + int *r1; + int r2; + + r1 = READ_ONCE(ptr); + r2 = READ_ONCE(*r1); + } + +can malfunction on Alpha systems. It is quite possible that r1 = &x +and r2 = 0 at the end, in spite of the address dependency. + +At first glance this doesn't seem to make sense. We know that the +smp_wmb() forces P0's store to x to propagate to P1 before the store +to ptr does. And since P1 can't execute its second load +until it knows what location to load from, i.e., after executing its +first load, the value x = 1 must have propagated to P1 before the +second load executed. So why doesn't r2 end up equal to 1? + +The answer lies in the Alpha's split local caches. Although the two +stores do reach P1's local cache in the proper order, it can happen +that the first store is processed by a busy part of the cache while +the second store is processed by an idle part. As a result, the x = 1 +value may not become available for P1's CPU to read until after the +ptr = &x value does, leading to the undesirable result above. The +final effect is that even though the two loads really are executed in +program order, it appears that they aren't. + +This could not have happened if the local cache had processed the +incoming stores in FIFO order. In constrast, other architectures +maintain at least the appearance of FIFO order. + +In practice, this difficulty is solved by inserting an +smp_read_barrier_depends() fence between P1's two loads. The effect +of this fence is to cause the CPU not to execute any po-later +instructions until after the local cache has finished processing all +the stores it has already received. Thus, if the code was changed to: + + P1() + { + int *r1; + int r2; + + r1 = READ_ONCE(ptr); + smp_read_barrier_depends(); + r2 = READ_ONCE(*r1); + } + +then we would never get r1 = &x and r2 = 0. By the time P1 executed +its second load, the x = 1 store would already be fully processed by +the local cache and available for satisfying the read request. + +The LKMM requires that smp_rmb(), acquire fences, and strong fences +share this property with smp_read_barrier_depends(): They do not allow +the CPU to execute any po-later instructions (or po-later loads in the +case of smp_rmb()) until all outstanding stores have been processed by +the local cache. In the case of a strong fence, the CPU first has to +wait for all of its po-earlier stores to propagate to every other CPU +in the system; then it has to wait for the local cache to process all +the stores received as of that time -- not just the stores received +when the strong fence began. + +And of course, none of this matters for any architecture other than +Alpha. + + +THE HAPPENS-BEFORE RELATION: hb +------------------------------- + +The happens-before relation (hb) links memory accesses that have to +execute in a certain order. hb includes the ppo relation and two +others, one of which is rfe. + +W ->rfe R implies that W and R are on different CPUs. It also means +that W's store must have propagated to R's CPU before R executed; +otherwise R could not have read the value stored by W. Therefore W +must have executed before R, and so we have W ->hb R. + +The equivalent fact need not hold if W ->rfi R (i.e., W and R are on +the same CPU). As we have already seen, the operational model allows +W's value to be forwarded to R in such cases, meaning that R may well +execute before W does. + +It's important to understand that neither coe nor fre is included in +hb, despite their similarities to rfe. For example, suppose we have +W ->coe W'. This means that W and W' are stores to the same location, +they execute on different CPUs, and W comes before W' in the coherence +order (i.e., W' overwrites W). Nevertheless, it is possible for W' to +execute before W, because the decision as to which store overwrites +the other is made later by the memory subsystem. When the stores are +nearly simultaneous, either one can come out on top. Similarly, +R ->fre W means that W overwrites the value which R reads, but it +doesn't mean that W has to execute after R. All that's necessary is +for the memory subsystem not to propagate W to R's CPU until after R +has executed, which is possible if W executes shortly before R. + +The third relation included in hb is like ppo, in that it only links +events that are on the same CPU. However it is more difficult to +explain, because it arises only indirectly from the requirement of +cache coherence. The relation is called prop, and it links two events +on CPU C in situations where a store from some other CPU comes after +the first event in the coherence order and propagates to C before the +second event executes. + +This is best explained with some examples. The simplest case looks +like this: + + int x; + + P0() + { + int r1; + + WRITE_ONCE(x, 1); + r1 = READ_ONCE(x); + } + + P1() + { + WRITE_ONCE(x, 8); + } + +If r1 = 8 at the end then P0's accesses must have executed in program +order. We can deduce this from the operational model; if P0's load +had executed before its store then the value of the store would have +been forwarded to the load, so r1 would have ended up equal to 1, not +8. In this case there is a prop link from P0's write event to its read +event, because P1's store came after P0's store in x's coherence +order, and P1's store propagated to P0 before P0's load executed. + +An equally simple case involves two loads of the same location that +read from different stores: + + int x = 0; + + P0() + { + int r1, r2; + + r1 = READ_ONCE(x); + r2 = READ_ONCE(x); + } + + P1() + { + WRITE_ONCE(x, 9); + } + +If r1 = 0 and r2 = 9 at the end then P0's accesses must have executed +in program order. If the second load had executed before the first +then the x = 9 store must have been propagated to P0 before the first +load executed, and so r1 would have been 9 rather than 0. In this +case there is a prop link from P0's first read event to its second, +because P1's store overwrote the value read by P0's first load, and +P1's store propagated to P0 before P0's second load executed. + +Less trivial examples of prop all involve fences. Unlike the simple +examples above, they can require that some instructions are executed +out of program order. This next one should look familiar: + + int buf = 0, flag = 0; + + P0() + { + WRITE_ONCE(buf, 1); + smp_wmb(); + WRITE_ONCE(flag, 1); + } + + P1() + { + int r1; + int r2; + + r1 = READ_ONCE(flag); + r2 = READ_ONCE(buf); + } + +This is the MP pattern again, with an smp_wmb() fence between the two +stores. If r1 = 1 and r2 = 0 at the end then there is a prop link +from P1's second load to its first (backwards!). The reason is +similar to the previous examples: The value P1 loads from buf gets +overwritten by P0's store to buf, the fence guarantees that the store +to buf will propagate to P1 before the store to flag does, and the +store to flag propagates to P1 before P1 reads flag. + +The prop link says that in order to obtain the r1 = 1, r2 = 0 result, +P1 must execute its second load before the first. Indeed, if the load +from flag were executed first, then the buf = 1 store would already +have propagated to P1 by the time P1's load from buf executed, so r2 +would have been 1 at the end, not 0. (The reasoning holds even for +Alpha, although the details are more complicated and we will not go +into them.) + +But what if we put an smp_rmb() fence between P1's loads? The fence +would force the two loads to be executed in program order, and it +would generate a cycle in the hb relation: The fence would create a ppo +link (hence an hb link) from the first load to the second, and the +prop relation would give an hb link from the second load to the first. +Since an instruction can't execute before itself, we are forced to +conclude that if an smp_rmb() fence is added, the r1 = 1, r2 = 0 +outcome is impossible -- as it should be. + +The formal definition of the prop relation involves a coe or fre link, +followed by an arbitrary number of cumul-fence links, ending with an +rfe link. You can concoct more exotic examples, containing more than +one fence, although this quickly leads to diminishing returns in terms +of complexity. For instance, here's an example containing a coe link +followed by two fences and an rfe link, utilizing the fact that +release fences are A-cumulative: + + int x, y, z; + + P0() + { + int r0; + + WRITE_ONCE(x, 1); + r0 = READ_ONCE(z); + } + + P1() + { + WRITE_ONCE(x, 2); + smp_wmb(); + WRITE_ONCE(y, 1); + } + + P2() + { + int r2; + + r2 = READ_ONCE(y); + smp_store_release(&z, 1); + } + +If x = 2, r0 = 1, and r2 = 1 after this code runs then there is a prop +link from P0's store to its load. This is because P0's store gets +overwritten by P1's store since x = 2 at the end (a coe link), the +smp_wmb() ensures that P1's store to x propagates to P2 before the +store to y does (the first fence), the store to y propagates to P2 +before P2's load and store execute, P2's smp_store_release() +guarantees that the stores to x and y both propagate to P0 before the +store to z does (the second fence), and P0's load executes after the +store to z has propagated to P0 (an rfe link). + +In summary, the fact that the hb relation links memory access events +in the order they execute means that it must not have cycles. This +requirement is the content of the LKMM's "happens-before" axiom. + +The LKMM defines yet another relation connected to times of +instruction execution, but it is not included in hb. It relies on the +particular properties of strong fences, which we cover in the next +section. + + +THE PROPAGATES-BEFORE RELATION: pb +---------------------------------- + +The propagates-before (pb) relation capitalizes on the special +features of strong fences. It links two events E and F whenever some +store is coherence-later than E and propagates to every CPU and to RAM +before F executes. The formal definition requires that E be linked to +F via a coe or fre link, an arbitrary number of cumul-fences, an +optional rfe link, a strong fence, and an arbitrary number of hb +links. Let's see how this definition works out. + +Consider first the case where E is a store (implying that the sequence +of links begins with coe). Then there are events W, X, Y, and Z such +that: + + E ->coe W ->cumul-fence* X ->rfe? Y ->strong-fence Z ->hb* F, + +where the * suffix indicates an arbitrary number of links of the +specified type, and the ? suffix indicates the link is optional (Y may +be equal to X). Because of the cumul-fence links, we know that W will +propagate to Y's CPU before X does, hence before Y executes and hence +before the strong fence executes. Because this fence is strong, we +know that W will propagate to every CPU and to RAM before Z executes. +And because of the hb links, we know that Z will execute before F. +Thus W, which comes later than E in the coherence order, will +propagate to every CPU and to RAM before F executes. + +The case where E is a load is exactly the same, except that the first +link in the sequence is fre instead of coe. + +The existence of a pb link from E to F implies that E must execute +before F. To see why, suppose that F executed first. Then W would +have propagated to E's CPU before E executed. If E was a store, the +memory subsystem would then be forced to make E come after W in the +coherence order, contradicting the fact that E ->coe W. If E was a +load, the memory subsystem would then be forced to satisfy E's read +request with the value stored by W or an even later store, +contradicting the fact that E ->fre W. + +A good example illustrating how pb works is the SB pattern with strong +fences: + + int x = 0, y = 0; + + P0() + { + int r0; + + WRITE_ONCE(x, 1); + smp_mb(); + r0 = READ_ONCE(y); + } + + P1() + { + int r1; + + WRITE_ONCE(y, 1); + smp_mb(); + r1 = READ_ONCE(x); + } + +If r0 = 0 at the end then there is a pb link from P0's load to P1's +load: an fre link from P0's load to P1's store (which overwrites the +value read by P0), and a strong fence between P1's store and its load. +In this example, the sequences of cumul-fence and hb links are empty. +Note that this pb link is not included in hb as an instance of prop, +because it does not start and end on the same CPU. + +Similarly, if r1 = 0 at the end then there is a pb link from P1's load +to P0's. This means that if both r1 and r2 were 0 there would be a +cycle in pb, which is not possible since an instruction cannot execute +before itself. Thus, adding smp_mb() fences to the SB pattern +prevents the r0 = 0, r1 = 0 outcome. + +In summary, the fact that the pb relation links events in the order +they execute means that it cannot have cycles. This requirement is +the content of the LKMM's "propagation" axiom. + + +RCU RELATIONS: link, gp-link, rscs-link, and rcu-path +----------------------------------------------------- + +RCU (Read-Copy-Update) is a powerful synchronization mechanism. It +rests on two concepts: grace periods and read-side critical sections. + +A grace period is the span of time occupied by a call to +synchronize_rcu(). A read-side critical section (or just critical +section, for short) is a region of code delimited by rcu_read_lock() +at the start and rcu_read_unlock() at the end. Critical sections can +be nested, although we won't make use of this fact. + +As far as memory models are concerned, RCU's main feature is its +Grace-Period Guarantee, which states that a critical section can never +span a full grace period. In more detail, the Guarantee says: + + If a critical section starts before a grace period then it + must end before the grace period does. In addition, every + store that propagates to the critical section's CPU before the + end of the critical section must propagate to every CPU before + the end of the grace period. + + If a critical section ends after a grace period ends then it + must start after the grace period does. In addition, every + store that propagates to the grace period's CPU before the + start of the grace period must propagate to every CPU before + the start of the critical section. + +Here is a simple example of RCU in action: + + int x, y; + + P0() + { + rcu_read_lock(); + WRITE_ONCE(x, 1); + WRITE_ONCE(y, 1); + rcu_read_unlock(); + } + + P1() + { + int r1, r2; + + r1 = READ_ONCE(x); + synchronize_rcu(); + r2 = READ_ONCE(y); + } + +The Grace Period Guarantee tells us that when this code runs, it will +never end with r1 = 1 and r2 = 0. The reasoning is as follows. r1 = 1 +means that P0's store to x propagated to P1 before P1 called +synchronize_rcu(), so P0's critical section must have started before +P1's grace period. On the other hand, r2 = 0 means that P0's store to +y, which occurs before the end of the critical section, did not +propagate to P1 before the end of the grace period, violating the +Guarantee. + +In the kernel's implementations of RCU, the business about stores +propagating to every CPU is realized by placing strong fences at +suitable places in the RCU-related code. Thus, if a critical section +starts before a grace period does then the critical section's CPU will +execute an smp_mb() fence after the end of the critical section and +some time before the grace period's synchronize_rcu() call returns. +And if a critical section ends after a grace period does then the +synchronize_rcu() routine will execute an smp_mb() fence at its start +and some time before the critical section's opening rcu_read_lock() +executes. + +What exactly do we mean by saying that a critical section "starts +before" or "ends after" a grace period? Some aspects of the meaning +are pretty obvious, as in the example above, but the details aren't +entirely clear. The LKMM formalizes this notion by means of a +relation with the unfortunately generic name "link". It is a very +general relation; among other things, X ->link Z includes cases where +X happens-before or is equal to some event Y which is equal to or +comes before Z in the coherence order. Taking Y = Z, this says that +X ->rfe Z implies X ->link Z, and taking Y = X, it says that X ->fr Z +and X ->co Z each imply X ->link Z. + +The formal definition of the link relation is more than a little +obscure, and we won't give it here. It is closely related to the pb +relation, and the details don't matter unless you want to comb through +a somewhat lengthy formal proof. Pretty much all you need to know +about link is the information in the preceding paragraph. + +The LKMM goes on to define the gp-link and rscs-link relations. They +bring grace periods and read-side critical sections into the picture, +in the following way: + + E ->gp-link F means there is a synchronize_rcu() fence event S + and an event X such that E ->po S, either S ->po X or S = X, + and X ->link F. In other words, E and F are connected by a + grace period followed by an instance of link. + + E ->rscs-link F means there is a critical section delimited by + an rcu_read_lock() fence L and an rcu_read_unlock() fence U, + and an event X such that E ->po U, either L ->po X or L = X, + and X ->link F. Roughly speaking, this says that some event + in the same critical section as E is connected by link to F. + +If we think of the link relation as standing for an extended "before", +then E ->gp-link F says that E executes before a grace period which +ends before F executes. (In fact it says more than this, because it +includes cases where E executes before a grace period and some store +propagates to F's CPU before F executes and doesn't propagate to some +other CPU until after the grace period ends.) Similarly, +E ->rscs-link F says that E is part of (or before the start of) a +critical section which starts before F executes. + +Putting this all together, the LKMM expresses the Grace Period +Guarantee by requiring that there are no cycles consisting of gp-link +and rscs-link connections in which the number of gp-link instances is +>= the number of rscs-link instances. It does this by defining the +rcu-path relation to link events E and F whenever it is possible to +pass from E to F by a sequence of gp-link and rscs-link connections +with at least as many of the former as the latter. The LKMM's "rcu" +axiom then says that there are no events E such that E ->rcu-path E. + +Justifying this axiom takes some intellectual effort, but it is in +fact a valid formalization of the Grace Period Guarantee. We won't +attempt to go through the detailed argument, but the following +analysis gives a taste of what is involved. Suppose we have a +violation of the first part of the Guarantee: A critical section +starts before a grace period, and some store propagates to the +critical section's CPU before the end of the critical section but +doesn't propagate to some other CPU until after the end of the grace +period. + +Putting symbols to these ideas, let L and U be the rcu_read_lock() and +rcu_read_unlock() fence events delimiting the critical section in +question, and let S be the synchronize_rcu() fence event for the grace +period. Saying that the critical section starts before S means there +are events E and F where E is po-after L (which marks the start of the +critical section), E is "before" F in the sense of the link relation, +and F is po-before the grace period S: + + L ->po E ->link F ->po S. + +Let W be the store mentioned above, let Z come before the end of the +critical section and witness that W propagates to the critical +section's CPU by reading from W, and let Y on some arbitrary CPU be a +witness that W has not propagated to that CPU, where Y happens after +some event X which is po-after S. Symbolically, this amounts to: + + S ->po X ->hb* Y ->fr W ->rf Z ->po U. + +The fr link from Y to W indicates that W has not propagated to Y's CPU +at the time that Y executes. From this, it can be shown (see the +discussion of the link relation earlier) that X and Z are connected by +link, yielding: + + S ->po X ->link Z ->po U. + +These formulas say that S is po-between F and X, hence F ->gp-link Z +via X. They also say that Z comes before the end of the critical +section and E comes after its start, hence Z ->rscs-link F via E. But +now we have a forbidden cycle: F ->gp-link Z ->rscs-link F. Thus the +"rcu" axiom rules out this violation of the Grace Period Guarantee. + +For something a little more down-to-earth, let's see how the axiom +works out in practice. Consider the RCU code example from above, this +time with statement labels added to the memory access instructions: + + int x, y; + + P0() + { + rcu_read_lock(); + W: WRITE_ONCE(x, 1); + X: WRITE_ONCE(y, 1); + rcu_read_unlock(); + } + + P1() + { + int r1, r2; + + Y: r1 = READ_ONCE(x); + synchronize_rcu(); + Z: r2 = READ_ONCE(y); + } + + +If r2 = 0 at the end then P0's store at X overwrites the value +that P1's load at Z reads from, so we have Z ->fre X and thus +Z ->link X. In addition, there is a synchronize_rcu() between Y and +Z, so therefore we have Y ->gp-link X. + +If r1 = 1 at the end then P1's load at Y reads from P0's store at W, +so we have W ->link Y. In addition, W and X are in the same critical +section, so therefore we have X ->rscs-link Y. + +This gives us a cycle, Y ->gp-link X ->rscs-link Y, with one gp-link +and one rscs-link, violating the "rcu" axiom. Hence the outcome is +not allowed by the LKMM, as we would expect. + +For contrast, let's see what can happen in a more complicated example: + + int x, y, z; + + P0() + { + int r0; + + rcu_read_lock(); + W: r0 = READ_ONCE(x); + X: WRITE_ONCE(y, 1); + rcu_read_unlock(); + } + + P1() + { + int r1; + + Y: r1 = READ_ONCE(y); + synchronize_rcu(); + Z: WRITE_ONCE(z, 1); + } + + P2() + { + int r2; + + rcu_read_lock(); + U: r2 = READ_ONCE(z); + V: WRITE_ONCE(x, 1); + rcu_read_unlock(); + } + +If r0 = r1 = r2 = 1 at the end, then similar reasoning to before shows +that W ->rscs-link Y via X, Y ->gp-link U via Z, and U ->rscs-link W +via V. And just as before, this gives a cycle: + + W ->rscs-link Y ->gp-link U ->rscs-link W. + +However, this cycle has fewer gp-link instances than rscs-link +instances, and consequently the outcome is not forbidden by the LKMM. +The following instruction timing diagram shows how it might actually +occur: + +P0 P1 P2 +-------------------- -------------------- -------------------- +rcu_read_lock() +X: WRITE_ONCE(y, 1) + Y: r1 = READ_ONCE(y) + synchronize_rcu() starts + . rcu_read_lock() + . V: WRITE_ONCE(x, 1) +W: r0 = READ_ONCE(x) . +rcu_read_unlock() . + synchronize_rcu() ends + Z: WRITE_ONCE(z, 1) + U: r2 = READ_ONCE(z) + rcu_read_unlock() + +This requires P0 and P2 to execute their loads and stores out of +program order, but of course they are allowed to do so. And as you +can see, the Grace Period Guarantee is not violated: The critical +section in P0 both starts before P1's grace period does and ends +before it does, and the critical section in P2 both starts after P1's +grace period does and ends after it does. + + +ODDS AND ENDS +------------- + +This section covers material that didn't quite fit anywhere in the +earlier sections. + +The descriptions in this document don't always match the formal +version of the LKMM exactly. For example, the actual formal +definition of the prop relation makes the initial coe or fre part +optional, and it doesn't require the events linked by the relation to +be on the same CPU. These differences are very unimportant; indeed, +instances where the coe/fre part of prop is missing are of no interest +because all the other parts (fences and rfe) are already included in +hb anyway, and where the formal model adds prop into hb, it includes +an explicit requirement that the events being linked are on the same +CPU. + +Another minor difference has to do with events that are both memory +accesses and fences, such as those corresponding to smp_load_acquire() +calls. In the formal model, these events aren't actually both reads +and fences; rather, they are read events with an annotation marking +them as acquires. (Or write events annotated as releases, in the case +smp_store_release().) The final effect is the same. + +Although we didn't mention it above, the instruction execution +ordering provided by the smp_rmb() fence doesn't apply to read events +that are part of a non-value-returning atomic update. For instance, +given: + + atomic_inc(&x); + smp_rmb(); + r1 = READ_ONCE(y); + +it is not guaranteed that the load from y will execute after the +update to x. This is because the ARMv8 architecture allows +non-value-returning atomic operations effectively to be executed off +the CPU. Basically, the CPU tells the memory subsystem to increment +x, and then the increment is carried out by the memory hardware with +no further involvement from the CPU. Since the CPU doesn't ever read +the value of x, there is nothing for the smp_rmb() fence to act on. + +The LKMM defines a few extra synchronization operations in terms of +things we have already covered. In particular, rcu_dereference() and +lockless_dereference() are both treated as a READ_ONCE() followed by +smp_read_barrier_depends() -- which also happens to be how they are +defined in include/linux/rcupdate.h and include/linux/compiler.h, +respectively. + +There are a few oddball fences which need special treatment: +smp_mb__before_atomic(), smp_mb__after_atomic(), and +smp_mb__after_spinlock(). The LKMM uses fence events with special +annotations for them; they act as strong fences just like smp_mb() +except for the sets of events that they order. Instead of ordering +all po-earlier events against all po-later events, as smp_mb() does, +they behave as follows: + + smp_mb__before_atomic() orders all po-earlier events against + po-later atomic updates and the events following them; + + smp_mb__after_atomic() orders po-earlier atomic updates and + the events preceding them against all po-later events; + + smp_mb_after_spinlock() orders po-earlier lock acquisition + events and the events preceding them against all po-later + events. + +The LKMM includes locking. In fact, there is special code for locking +in the formal model, added in order to make tools run faster. +However, this special code is intended to be exactly equivalent to +concepts we have already covered. A spinlock_t variable is treated +the same as an int, and spin_lock(&s) is treated the same as: + + while (cmpxchg_acquire(&s, 0, 1) != 0) + cpu_relax(); + +which waits until s is equal to 0 and then atomically sets it to 1, +and where the read part of the atomic update is also an acquire fence. +An alternate way to express the same thing would be: + + r = xchg_acquire(&s, 1); + +along with a requirement that at the end, r = 0. spin_unlock(&s) is +treated the same as: + + smp_store_release(&s, 0); + +Interestingly, RCU and locking each introduce the possibility of +deadlock. When faced with code sequences such as: + + spin_lock(&s); + spin_lock(&s); + spin_unlock(&s); + spin_unlock(&s); + +or: + + rcu_read_lock(); + synchronize_rcu(); + rcu_read_unlock(); + +what does the LKMM have to say? Answer: It says there are no allowed +executions at all, which makes sense. But this can also lead to +misleading results, because if a piece of code has multiple possible +executions, some of which deadlock, the model will report only on the +non-deadlocking executions. For example: + + int x, y; + + P0() + { + int r0; + + WRITE_ONCE(x, 1); + r0 = READ_ONCE(y); + } + + P1() + { + rcu_read_lock(); + if (READ_ONCE(x) > 0) { + WRITE_ONCE(y, 36); + synchronize_rcu(); + } + rcu_read_unlock(); + } + +Is it possible to end up with r0 = 36 at the end? The LKMM will tell +you it is not, but the model won't mention that this is because P1 +will self-deadlock in the executions where it stores 36 in y. diff --git a/tools/memory-model/Documentation/recipes.txt b/tools/memory-model/Documentation/recipes.txt new file mode 100644 index 000000000000..ee4309a87fc4 --- /dev/null +++ b/tools/memory-model/Documentation/recipes.txt @@ -0,0 +1,570 @@ +This document provides "recipes", that is, litmus tests for commonly +occurring situations, as well as a few that illustrate subtly broken but +attractive nuisances. Many of these recipes include example code from +v4.13 of the Linux kernel. + +The first section covers simple special cases, the second section +takes off the training wheels to cover more involved examples, +and the third section provides a few rules of thumb. + + +Simple special cases +==================== + +This section presents two simple special cases, the first being where +there is only one CPU or only one memory location is accessed, and the +second being use of that old concurrency workhorse, locking. + + +Single CPU or single memory location +------------------------------------ + +If there is only one CPU on the one hand or only one variable +on the other, the code will execute in order. There are (as +usual) some things to be careful of: + +1. Some aspects of the C language are unordered. For example, + in the expression "f(x) + g(y)", the order in which f and g are + called is not defined; the object code is allowed to use either + order or even to interleave the computations. + +2. Compilers are permitted to use the "as-if" rule. That is, a + compiler can emit whatever code it likes for normal accesses, + as long as the results of a single-threaded execution appear + just as if the compiler had followed all the relevant rules. + To see this, compile with a high level of optimization and run + the debugger on the resulting binary. + +3. If there is only one variable but multiple CPUs, that variable + must be properly aligned and all accesses to that variable must + be full sized. Variables that straddle cachelines or pages void + your full-ordering warranty, as do undersized accesses that load + from or store to only part of the variable. + +4. If there are multiple CPUs, accesses to shared variables should + use READ_ONCE() and WRITE_ONCE() or stronger to prevent load/store + tearing, load/store fusing, and invented loads and stores. + There are exceptions to this rule, including: + + i. When there is no possibility of a given shared variable + being updated by some other CPU, for example, while + holding the update-side lock, reads from that variable + need not use READ_ONCE(). + + ii. When there is no possibility of a given shared variable + being either read or updated by other CPUs, for example, + when running during early boot, reads from that variable + need not use READ_ONCE() and writes to that variable + need not use WRITE_ONCE(). + + +Locking +------- + +Locking is well-known and straightforward, at least if you don't think +about it too hard. And the basic rule is indeed quite simple: Any CPU that +has acquired a given lock sees any changes previously seen or made by any +CPU before it released that same lock. Note that this statement is a bit +stronger than "Any CPU holding a given lock sees all changes made by any +CPU during the time that CPU was holding this same lock". For example, +consider the following pair of code fragments: + + /* See MP+polocks.litmus. */ + void CPU0(void) + { + WRITE_ONCE(x, 1); + spin_lock(&mylock); + WRITE_ONCE(y, 1); + spin_unlock(&mylock); + } + + void CPU1(void) + { + spin_lock(&mylock); + r0 = READ_ONCE(y); + spin_unlock(&mylock); + r1 = READ_ONCE(x); + } + +The basic rule guarantees that if CPU0() acquires mylock before CPU1(), +then both r0 and r1 must be set to the value 1. This also has the +consequence that if the final value of r0 is equal to 1, then the final +value of r1 must also be equal to 1. In contrast, the weaker rule would +say nothing about the final value of r1. + +The converse to the basic rule also holds, as illustrated by the +following litmus test: + + /* See MP+porevlocks.litmus. */ + void CPU0(void) + { + r0 = READ_ONCE(y); + spin_lock(&mylock); + r1 = READ_ONCE(x); + spin_unlock(&mylock); + } + + void CPU1(void) + { + spin_lock(&mylock); + WRITE_ONCE(x, 1); + spin_unlock(&mylock); + WRITE_ONCE(y, 1); + } + +This converse to the basic rule guarantees that if CPU0() acquires +mylock before CPU1(), then both r0 and r1 must be set to the value 0. +This also has the consequence that if the final value of r1 is equal +to 0, then the final value of r0 must also be equal to 0. In contrast, +the weaker rule would say nothing about the final value of r0. + +These examples show only a single pair of CPUs, but the effects of the +locking basic rule extend across multiple acquisitions of a given lock +across multiple CPUs. + +However, it is not necessarily the case that accesses ordered by +locking will be seen as ordered by CPUs not holding that lock. +Consider this example: + + /* See Z6.0+pooncelock+pooncelock+pombonce.litmus. */ + void CPU0(void) + { + spin_lock(&mylock); + WRITE_ONCE(x, 1); + WRITE_ONCE(y, 1); + spin_unlock(&mylock); + } + + void CPU1(void) + { + spin_lock(&mylock); + r0 = READ_ONCE(y); + WRITE_ONCE(z, 1); + spin_unlock(&mylock); + } + + void CPU2(void) + { + WRITE_ONCE(z, 2); + smp_mb(); + r1 = READ_ONCE(x); + } + +Counter-intuitive though it might be, it is quite possible to have +the final value of r0 be 1, the final value of z be 2, and the final +value of r1 be 0. The reason for this surprising outcome is that +CPU2() never acquired the lock, and thus did not benefit from the +lock's ordering properties. + +Ordering can be extended to CPUs not holding the lock by careful use +of smp_mb__after_spinlock(): + + /* See Z6.0+pooncelock+poonceLock+pombonce.litmus. */ + void CPU0(void) + { + spin_lock(&mylock); + WRITE_ONCE(x, 1); + WRITE_ONCE(y, 1); + spin_unlock(&mylock); + } + + void CPU1(void) + { + spin_lock(&mylock); + smp_mb__after_spinlock(); + r0 = READ_ONCE(y); + WRITE_ONCE(z, 1); + spin_unlock(&mylock); + } + + void CPU2(void) + { + WRITE_ONCE(z, 2); + smp_mb(); + r1 = READ_ONCE(x); + } + +This addition of smp_mb__after_spinlock() strengthens the lock acquisition +sufficiently to rule out the counter-intuitive outcome. + + +Taking off the training wheels +============================== + +This section looks at more complex examples, including message passing, +load buffering, release-acquire chains, store buffering. +Many classes of litmus tests have abbreviated names, which may be found +here: https://www.cl.cam.ac.uk/~pes20/ppc-supplemental/test6.pdf + + +Message passing (MP) +-------------------- + +The MP pattern has one CPU execute a pair of stores to a pair of variables +and another CPU execute a pair of loads from this same pair of variables, +but in the opposite order. The goal is to avoid the counter-intuitive +outcome in which the first load sees the value written by the second store +but the second load does not see the value written by the first store. +In the absence of any ordering, this goal may not be met, as can be seen +in the MP+poonceonces.litmus litmus test. This section therefore looks at +a number of ways of meeting this goal. + + +Release and acquire +~~~~~~~~~~~~~~~~~~~ + +Use of smp_store_release() and smp_load_acquire() is one way to force +the desired MP ordering. The general approach is shown below: + + /* See MP+pooncerelease+poacquireonce.litmus. */ + void CPU0(void) + { + WRITE_ONCE(x, 1); + smp_store_release(&y, 1); + } + + void CPU1(void) + { + r0 = smp_load_acquire(&y); + r1 = READ_ONCE(x); + } + +The smp_store_release() macro orders any prior accesses against the +store, while the smp_load_acquire macro orders the load against any +subsequent accesses. Therefore, if the final value of r0 is the value 1, +the final value of r1 must also be the value 1. + +The init_stack_slab() function in lib/stackdepot.c uses release-acquire +in this way to safely initialize of a slab of the stack. Working out +the mutual-exclusion design is left as an exercise for the reader. + + +Assign and dereference +~~~~~~~~~~~~~~~~~~~~~~ + +Use of rcu_assign_pointer() and rcu_dereference() is quite similar to the +use of smp_store_release() and smp_load_acquire(), except that both +rcu_assign_pointer() and rcu_dereference() operate on RCU-protected +pointers. The general approach is shown below: + + /* See MP+onceassign+derefonce.litmus. */ + int z; + int *y = &z; + int x; + + void CPU0(void) + { + WRITE_ONCE(x, 1); + rcu_assign_pointer(y, &x); + } + + void CPU1(void) + { + rcu_read_lock(); + r0 = rcu_dereference(y); + r1 = READ_ONCE(*r0); + rcu_read_unlock(); + } + +In this example, if the final value of r0 is &x then the final value of +r1 must be 1. + +The rcu_assign_pointer() macro has the same ordering properties as does +smp_store_release(), but the rcu_dereference() macro orders the load only +against later accesses that depend on the value loaded. A dependency +is present if the value loaded determines the address of a later access +(address dependency, as shown above), the value written by a later store +(data dependency), or whether or not a later store is executed in the +first place (control dependency). Note that the term "data dependency" +is sometimes casually used to cover both address and data dependencies. + +In lib/prime_numbers.c, the expand_to_next_prime() function invokes +rcu_assign_pointer(), and the next_prime_number() function invokes +rcu_dereference(). This combination mediates access to a bit vector +that is expanded as additional primes are needed. + + +Write and read memory barriers +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +It is usually better to use smp_store_release() instead of smp_wmb() +and to use smp_load_acquire() instead of smp_rmb(). However, the older +smp_wmb() and smp_rmb() APIs are still heavily used, so it is important +to understand their use cases. The general approach is shown below: + + /* See MP+wmbonceonce+rmbonceonce.litmus. */ + void CPU0(void) + { + WRITE_ONCE(x, 1); + smp_wmb(); + WRITE_ONCE(y, 1); + } + + void CPU1(void) + { + r0 = READ_ONCE(y); + smp_rmb(); + r1 = READ_ONCE(x); + } + +The smp_wmb() macro orders prior stores against later stores, and the +smp_rmb() macro orders prior loads against later loads. Therefore, if +the final value of r0 is 1, the final value of r1 must also be 1. + +The the xlog_state_switch_iclogs() function in fs/xfs/xfs_log.c contains +the following write-side code fragment: + + log->l_curr_block -= log->l_logBBsize; + ASSERT(log->l_curr_block >= 0); + smp_wmb(); + log->l_curr_cycle++; + +And the xlog_valid_lsn() function in fs/xfs/xfs_log_priv.h contains +the corresponding read-side code fragment: + + cur_cycle = ACCESS_ONCE(log->l_curr_cycle); + smp_rmb(); + cur_block = ACCESS_ONCE(log->l_curr_block); + +Alternatively, consider the following comment in function +perf_output_put_handle() in kernel/events/ring_buffer.c: + + * kernel user + * + * if (LOAD ->data_tail) { LOAD ->data_head + * (A) smp_rmb() (C) + * STORE $data LOAD $data + * smp_wmb() (B) smp_mb() (D) + * STORE ->data_head STORE ->data_tail + * } + +The B/C pairing is an example of the MP pattern using smp_wmb() on the +write side and smp_rmb() on the read side. + +Of course, given that smp_mb() is strictly stronger than either smp_wmb() +or smp_rmb(), any code fragment that would work with smp_rmb() and +smp_wmb() would also work with smp_mb() replacing either or both of the +weaker barriers. + + +Load buffering (LB) +------------------- + +The LB pattern has one CPU load from one variable and then store to a +second, while another CPU loads from the second variable and then stores +to the first. The goal is to avoid the counter-intuitive situation where +each load reads the value written by the other CPU's store. In the +absence of any ordering it is quite possible that this may happen, as +can be seen in the LB+poonceonces.litmus litmus test. + +One way of avoiding the counter-intuitive outcome is through the use of a +control dependency paired with a full memory barrier: + + /* See LB+ctrlonceonce+mbonceonce.litmus. */ + void CPU0(void) + { + r0 = READ_ONCE(x); + if (r0) + WRITE_ONCE(y, 1); + } + + void CPU1(void) + { + r1 = READ_ONCE(y); + smp_mb(); + WRITE_ONCE(x, 1); + } + +This pairing of a control dependency in CPU0() with a full memory +barrier in CPU1() prevents r0 and r1 from both ending up equal to 1. + +The A/D pairing from the ring-buffer use case shown earlier also +illustrates LB. Here is a repeat of the comment in +perf_output_put_handle() in kernel/events/ring_buffer.c, showing a +control dependency on the kernel side and a full memory barrier on +the user side: + + * kernel user + * + * if (LOAD ->data_tail) { LOAD ->data_head + * (A) smp_rmb() (C) + * STORE $data LOAD $data + * smp_wmb() (B) smp_mb() (D) + * STORE ->data_head STORE ->data_tail + * } + * + * Where A pairs with D, and B pairs with C. + +The kernel's control dependency between the load from ->data_tail +and the store to data combined with the user's full memory barrier +between the load from data and the store to ->data_tail prevents +the counter-intuitive outcome where the kernel overwrites the data +before the user gets done loading it. + + +Release-acquire chains +---------------------- + +Release-acquire chains are a low-overhead, flexible, and easy-to-use +method of maintaining order. However, they do have some limitations that +need to be fully understood. Here is an example that maintains order: + + /* See ISA2+pooncerelease+poacquirerelease+poacquireonce.litmus. */ + void CPU0(void) + { + WRITE_ONCE(x, 1); + smp_store_release(&y, 1); + } + + void CPU1(void) + { + r0 = smp_load_acquire(y); + smp_store_release(&z, 1); + } + + void CPU2(void) + { + r1 = smp_load_acquire(z); + r2 = READ_ONCE(x); + } + +In this case, if r0 and r1 both have final values of 1, then r2 must +also have a final value of 1. + +The ordering in this example is stronger than it needs to be. For +example, ordering would still be preserved if CPU1()'s smp_load_acquire() +invocation was replaced with READ_ONCE(). + +It is tempting to assume that CPU0()'s store to x is globally ordered +before CPU1()'s store to z, but this is not the case: + + /* See Z6.0+pooncerelease+poacquirerelease+mbonceonce.litmus. */ + void CPU0(void) + { + WRITE_ONCE(x, 1); + smp_store_release(&y, 1); + } + + void CPU1(void) + { + r0 = smp_load_acquire(y); + smp_store_release(&z, 1); + } + + void CPU2(void) + { + WRITE_ONCE(z, 2); + smp_mb(); + r1 = READ_ONCE(x); + } + +One might hope that if the final value of r0 is 1 and the final value +of z is 2, then the final value of r1 must also be 1, but it really is +possible for r1 to have the final value of 0. The reason, of course, +is that in this version, CPU2() is not part of the release-acquire chain. +This situation is accounted for in the rules of thumb below. + +Despite this limitation, release-acquire chains are low-overhead as +well as simple and powerful, at least as memory-ordering mechanisms go. + + +Store buffering +--------------- + +Store buffering can be thought of as upside-down load buffering, so +that one CPU first stores to one variable and then loads from a second, +while another CPU stores to the second variable and then loads from the +first. Preserving order requires nothing less than full barriers: + + /* See SB+mbonceonces.litmus. */ + void CPU0(void) + { + WRITE_ONCE(x, 1); + smp_mb(); + r0 = READ_ONCE(y); + } + + void CPU1(void) + { + WRITE_ONCE(y, 1); + smp_mb(); + r1 = READ_ONCE(x); + } + +Omitting either smp_mb() will allow both r0 and r1 to have final +values of 0, but providing both full barriers as shown above prevents +this counter-intuitive outcome. + +This pattern most famously appears as part of Dekker's locking +algorithm, but it has a much more practical use within the Linux kernel +of ordering wakeups. The following comment taken from waitqueue_active() +in include/linux/wait.h shows the canonical pattern: + + * CPU0 - waker CPU1 - waiter + * + * for (;;) { + * @cond = true; prepare_to_wait(&wq_head, &wait, state); + * smp_mb(); // smp_mb() from set_current_state() + * if (waitqueue_active(wq_head)) if (@cond) + * wake_up(wq_head); break; + * schedule(); + * } + * finish_wait(&wq_head, &wait); + +On CPU0, the store is to @cond and the load is in waitqueue_active(). +On CPU1, prepare_to_wait() contains both a store to wq_head and a call +to set_current_state(), which contains an smp_mb() barrier; the load is +"if (@cond)". The full barriers prevent the undesirable outcome where +CPU1 puts the waiting task to sleep and CPU0 fails to wake it up. + +Note that use of locking can greatly simplify this pattern. + + +Rules of thumb +============== + +There might seem to be no pattern governing what ordering primitives are +needed in which situations, but this is not the case. There is a pattern +based on the relation between the accesses linking successive CPUs in a +given litmus test. There are three types of linkage: + +1. Write-to-read, where the next CPU reads the value that the + previous CPU wrote. The LB litmus-test patterns contain only + this type of relation. In formal memory-modeling texts, this + relation is called "reads-from" and is usually abbreviated "rf". + +2. Read-to-write, where the next CPU overwrites the value that the + previous CPU read. The SB litmus test contains only this type + of relation. In formal memory-modeling texts, this relation is + often called "from-reads" and is sometimes abbreviated "fr". + +3. Write-to-write, where the next CPU overwrites the value written + by the previous CPU. The Z6.0 litmus test pattern contains a + write-to-write relation between the last access of CPU1() and + the first access of CPU2(). In formal memory-modeling texts, + this relation is often called "coherence order" and is sometimes + abbreviated "co". In the C++ standard, it is instead called + "modification order" and often abbreviated "mo". + +The strength of memory ordering required for a given litmus test to +avoid a counter-intuitive outcome depends on the types of relations +linking the memory accesses for the outcome in question: + +o If all links are write-to-read links, then the weakest + possible ordering within each CPU suffices. For example, in + the LB litmus test, a control dependency was enough to do the + job. + +o If all but one of the links are write-to-read links, then a + release-acquire chain suffices. Both the MP and the ISA2 + litmus tests illustrate this case. + +o If more than one of the links are something other than + write-to-read links, then a full memory barrier is required + between each successive pair of non-write-to-read links. This + case is illustrated by the Z6.0 litmus tests, both in the + locking and in the release-acquire sections. + +However, if you find yourself having to stretch these rules of thumb +to fit your situation, you should consider creating a litmus test and +running it on the model. diff --git a/tools/memory-model/Documentation/references.txt b/tools/memory-model/Documentation/references.txt new file mode 100644 index 000000000000..ba2e34c2ec3f --- /dev/null +++ b/tools/memory-model/Documentation/references.txt @@ -0,0 +1,107 @@ +This document provides background reading for memory models and related +tools. These documents are aimed at kernel hackers who are interested +in memory models. + + +Hardware manuals and models +=========================== + +o SPARC International Inc. (Ed.). 1994. "The SPARC Architecture + Reference Manual Version 9". SPARC International Inc. + +o Compaq Computer Corporation (Ed.). 2002. "Alpha Architecture + Reference Manual". Compaq Computer Corporation. + +o Intel Corporation (Ed.). 2002. "A Formal Specification of Intel + Itanium Processor Family Memory Ordering". Intel Corporation. + +o Intel Corporation (Ed.). 2002. "Intel 64 and IA-32 Architectures + Software Developer’s Manual". Intel Corporation. + +o Peter Sewell, Susmit Sarkar, Scott Owens, Francesco Zappa Nardelli, + and Magnus O. Myreen. 2010. "x86-TSO: A Rigorous and Usable + Programmer's Model for x86 Multiprocessors". Commun. ACM 53, 7 + (July, 2010), 89-97. http://doi.acm.org/10.1145/1785414.1785443 + +o IBM Corporation (Ed.). 2009. "Power ISA Version 2.06". IBM + Corporation. + +o ARM Ltd. (Ed.). 2009. "ARM Barrier Litmus Tests and Cookbook". + ARM Ltd. + +o Susmit Sarkar, Peter Sewell, Jade Alglave, Luc Maranget, and + Derek Williams. 2011. "Understanding POWER Multiprocessors". In + Proceedings of the 32Nd ACM SIGPLAN Conference on Programming + Language Design and Implementation (PLDI ’11). ACM, New York, + NY, USA, 175–186. + +o Susmit Sarkar, Kayvan Memarian, Scott Owens, Mark Batty, + Peter Sewell, Luc Maranget, Jade Alglave, and Derek Williams. + 2012. "Synchronising C/C++ and POWER". In Proceedings of the 33rd + ACM SIGPLAN Conference on Programming Language Design and + Implementation (PLDI '12). ACM, New York, NY, USA, 311-322. + +o ARM Ltd. (Ed.). 2014. "ARM Architecture Reference Manual (ARMv8, + for ARMv8-A architecture profile)". ARM Ltd. + +o Imagination Technologies, LTD. 2015. "MIPS(R) Architecture + For Programmers, Volume II-A: The MIPS64(R) Instruction, + Set Reference Manual". Imagination Technologies, + LTD. https://imgtec.com/?do-download=4302. + +o Shaked Flur, Kathryn E. Gray, Christopher Pulte, Susmit + Sarkar, Ali Sezgin, Luc Maranget, Will Deacon, and Peter + Sewell. 2016. "Modelling the ARMv8 Architecture, Operationally: + Concurrency and ISA". In Proceedings of the 43rd Annual ACM + SIGPLAN-SIGACT Symposium on Principles of Programming Languages + (POPL ’16). ACM, New York, NY, USA, 608–621. + +o Shaked Flur, Susmit Sarkar, Christopher Pulte, Kyndylan Nienhuis, + Luc Maranget, Kathryn E. Gray, Ali Sezgin, Mark Batty, and Peter + Sewell. 2017. "Mixed-size Concurrency: ARM, POWER, C/C++11, + and SC". In Proceedings of the 44th ACM SIGPLAN Symposium on + Principles of Programming Languages (POPL 2017). ACM, New York, + NY, USA, 429–442. + + +Linux-kernel memory model +========================= + +o Andrea Parri, Alan Stern, Luc Maranget, Paul E. McKenney, + and Jade Alglave. 2017. "A formal model of + Linux-kernel memory ordering - companion webpage". + http://moscova.inria.fr/∼maranget/cats7/linux/. (2017). [Online; + accessed 30-January-2017]. + +o Jade Alglave, Luc Maranget, Paul E. McKenney, Andrea Parri, and + Alan Stern. 2017. "A formal kernel memory-ordering model (part 1)" + Linux Weekly News. https://lwn.net/Articles/718628/ + +o Jade Alglave, Luc Maranget, Paul E. McKenney, Andrea Parri, and + Alan Stern. 2017. "A formal kernel memory-ordering model (part 2)" + Linux Weekly News. https://lwn.net/Articles/720550/ + + +Memory-model tooling +==================== + +o Daniel Jackson. 2002. "Alloy: A Lightweight Object Modelling + Notation". ACM Trans. Softw. Eng. Methodol. 11, 2 (April 2002), + 256–290. http://doi.acm.org/10.1145/505145.505149 + +o Jade Alglave, Luc Maranget, and Michael Tautschnig. 2014. "Herding + Cats: Modelling, Simulation, Testing, and Data Mining for Weak + Memory". ACM Trans. Program. Lang. Syst. 36, 2, Article 7 (July + 2014), 7:1–7:74 pages. + +o Jade Alglave, Patrick Cousot, and Luc Maranget. 2016. "Syntax and + semantics of the weak consistency model specification language + cat". CoRR abs/1608.07531 (2016). http://arxiv.org/abs/1608.07531 + + +Memory-model comparisons +======================== + +o Paul E. McKenney, Ulrich Weigand, Andrea Parri, and Boqun + Feng. 2016. "Linux-Kernel Memory Model". (6 June 2016). + http://open-std.org/JTC1/SC22/WG21/docs/papers/2016/p0124r2.html. diff --git a/tools/memory-model/MAINTAINERS b/tools/memory-model/MAINTAINERS new file mode 100644 index 000000000000..711cbe72d606 --- /dev/null +++ b/tools/memory-model/MAINTAINERS @@ -0,0 +1,15 @@ +LINUX KERNEL MEMORY MODEL +M: Alan Stern +M: Andrea Parri +M: Will Deacon +M: Peter Zijlstra +M: Boqun Feng +M: Nicholas Piggin +M: David Howells +M: Jade Alglave +M: Luc Maranget +M: "Paul E. McKenney" +L: linux-kernel@vger.kernel.org +S: Supported +T: git git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git +F: tools/memory-model/ diff --git a/tools/memory-model/README b/tools/memory-model/README new file mode 100644 index 000000000000..43ba49492111 --- /dev/null +++ b/tools/memory-model/README @@ -0,0 +1,220 @@ + ========================= + LINUX KERNEL MEMORY MODEL + ========================= + +============ +INTRODUCTION +============ + +This directory contains the memory model of the Linux kernel, written +in the "cat" language and executable by the (externally provided) +"herd7" simulator, which exhaustively explores the state space of +small litmus tests. + +In addition, the "klitmus7" tool (also externally provided) may be used +to convert a litmus test to a Linux kernel module, which in turn allows +that litmus test to be exercised within the Linux kernel. + + +============ +REQUIREMENTS +============ + +The "herd7" and "klitmus7" tools must be downloaded separately: + + https://github.com/herd/herdtools7 + +See "herdtools7/INSTALL.md" for installation instructions. + +Alternatively, Abhishek Bhardwaj has kindly provided a Docker image +of these tools at "abhishek40/memory-model". Abhishek suggests the +following commands to install and use this image: + + - Users should install Docker for their distribution. + - docker run -itd abhishek40/memory-model + - docker attach + +Gentoo users might wish to make use of Patrick McLean's package: + + https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-util/herdtools7 + +These packages may not be up-to-date with respect to the GitHub +repository. + + +================== +BASIC USAGE: HERD7 +================== + +The memory model is used, in conjunction with "herd7", to exhaustively +explore the state space of small litmus tests. + +For example, to run SB+mbonceonces.litmus against the memory model: + + $ herd7 -conf linux-kernel.cfg litmus-tests/SB+mbonceonces.litmus + +Here is the corresponding output: + + Test SB+mbonceonces Allowed + States 3 + 0:r0=0; 1:r0=1; + 0:r0=1; 1:r0=0; + 0:r0=1; 1:r0=1; + No + Witnesses + Positive: 0 Negative: 3 + Condition exists (0:r0=0 /\ 1:r0=0) + Observation SB+mbonceonces Never 0 3 + Time SB+mbonceonces 0.01 + Hash=d66d99523e2cac6b06e66f4c995ebb48 + +The "Positive: 0 Negative: 3" and the "Never 0 3" each indicate that +this litmus test's "exists" clause can not be satisfied. + +See "herd7 -help" or "herdtools7/doc/" for more information. + + +===================== +BASIC USAGE: KLITMUS7 +===================== + +The "klitmus7" tool converts a litmus test into a Linux kernel module, +which may then be loaded and run. + +For example, to run SB+mbonceonces.litmus against hardware: + + $ mkdir mymodules + $ klitmus7 -o mymodules litmus-tests/SB+mbonceonces.litmus + $ cd mymodules ; make + $ sudo sh run.sh + +The corresponding output includes: + + Test SB+mbonceonces Allowed + Histogram (3 states) + 644580 :>0:r0=1; 1:r0=0; + 644328 :>0:r0=0; 1:r0=1; + 711092 :>0:r0=1; 1:r0=1; + No + Witnesses + Positive: 0, Negative: 2000000 + Condition exists (0:r0=0 /\ 1:r0=0) is NOT validated + Hash=d66d99523e2cac6b06e66f4c995ebb48 + Observation SB+mbonceonces Never 0 2000000 + Time SB+mbonceonces 0.16 + +The "Positive: 0 Negative: 2000000" and the "Never 0 2000000" indicate +that during two million trials, the state specified in this litmus +test's "exists" clause was not reached. + +And, as with "herd7", please see "klitmus7 -help" or "herdtools7/doc/" +for more information. + + +==================== +DESCRIPTION OF FILES +==================== + +Documentation/cheatsheet.txt + Quick-reference guide to the Linux-kernel memory model. + +Documentation/explanation.txt + Describes the memory model in detail. + +Documentation/recipes.txt + Lists common memory-ordering patterns. + +Documentation/references.txt + Provides background reading. + +linux-kernel.bell + Categorizes the relevant instructions, including memory + references, memory barriers, atomic read-modify-write operations, + lock acquisition/release, and RCU operations. + + More formally, this file (1) lists the subtypes of the various + event types used by the memory model and (2) performs RCU + read-side critical section nesting analysis. + +linux-kernel.cat + Specifies what reorderings are forbidden by memory references, + memory barriers, atomic read-modify-write operations, and RCU. + + More formally, this file specifies what executions are forbidden + by the memory model. Allowed executions are those which + satisfy the model's "coherence", "atomic", "happens-before", + "propagation", and "rcu" axioms, which are defined in the file. + +linux-kernel.cfg + Convenience file that gathers the common-case herd7 command-line + arguments. + +linux-kernel.def + Maps from C-like syntax to herd7's internal litmus-test + instruction-set architecture. + +litmus-tests + Directory containing a few representative litmus tests, which + are listed in litmus-tests/README. A great deal more litmus + tests are available at https://github.com/paulmckrcu/litmus. + +lock.cat + Provides a front-end analysis of lock acquisition and release, + for example, associating a lock acquisition with the preceding + and following releases and checking for self-deadlock. + + More formally, this file defines a performance-enhanced scheme + for generation of the possible reads-from and coherence order + relations on the locking primitives. + +README + This file. + + +=========== +LIMITATIONS +=========== + +The Linux-kernel memory model has the following limitations: + +1. Compiler optimizations are not modeled. Of course, the use + of READ_ONCE() and WRITE_ONCE() limits the compiler's ability + to optimize, but there is Linux-kernel code that uses bare C + memory accesses. Handling this code is on the to-do list. + For more information, see Documentation/explanation.txt (in + particular, the "THE PROGRAM ORDER RELATION: po AND po-loc" + and "A WARNING" sections). + +2. Multiple access sizes for a single variable are not supported, + and neither are misaligned or partially overlapping accesses. + +3. Exceptions and interrupts are not modeled. In some cases, + this limitation can be overcome by modeling the interrupt or + exception with an additional process. + +4. I/O such as MMIO or DMA is not supported. + +5. Self-modifying code (such as that found in the kernel's + alternatives mechanism, function tracer, Berkeley Packet Filter + JIT compiler, and module loader) is not supported. + +6. Complete modeling of all variants of atomic read-modify-write + operations, locking primitives, and RCU is not provided. + For example, call_rcu() and rcu_barrier() are not supported. + However, a substantial amount of support is provided for these + operations, as shown in the linux-kernel.def file. + +The "herd7" tool has some additional limitations of its own, apart from +the memory model: + +1. Non-trivial data structures such as arrays or structures are + not supported. However, pointers are supported, allowing trivial + linked lists to be constructed. + +2. Dynamic memory allocation is not supported, although this can + be worked around in some cases by supplying multiple statically + allocated variables. + +Some of these limitations may be overcome in the future, but others are +more likely to be addressed by incorporating the Linux-kernel memory model +into other tools. diff --git a/tools/memory-model/linux-kernel.bell b/tools/memory-model/linux-kernel.bell new file mode 100644 index 000000000000..57112505f5e0 --- /dev/null +++ b/tools/memory-model/linux-kernel.bell @@ -0,0 +1,53 @@ +// SPDX-License-Identifier: GPL-2.0+ +(* + * Copyright (C) 2015 Jade Alglave , + * Copyright (C) 2016 Luc Maranget for Inria + * Copyright (C) 2017 Alan Stern , + * Andrea Parri + * + * An earlier version of this file appears in the companion webpage for + * "Frightening small children and disconcerting grown-ups: Concurrency + * in the Linux kernel" by Alglave, Maranget, McKenney, Parri, and Stern, + * which is to appear in ASPLOS 2018. + *) + +"Linux kernel memory model" + +enum Accesses = 'once (*READ_ONCE,WRITE_ONCE,ACCESS_ONCE*) || + 'release (*smp_store_release*) || + 'acquire (*smp_load_acquire*) || + 'noreturn (* R of non-return RMW *) +instructions R[{'once,'acquire,'noreturn}] +instructions W[{'once,'release}] +instructions RMW[{'once,'acquire,'release}] + +enum Barriers = 'wmb (*smp_wmb*) || + 'rmb (*smp_rmb*) || + 'mb (*smp_mb*) || + 'rb_dep (*smp_read_barrier_depends*) || + 'rcu-lock (*rcu_read_lock*) || + 'rcu-unlock (*rcu_read_unlock*) || + 'sync-rcu (*synchronize_rcu*) || + 'before_atomic (*smp_mb__before_atomic*) || + 'after_atomic (*smp_mb__after_atomic*) || + 'after_spinlock (*smp_mb__after_spinlock*) +instructions F[Barriers] + +(* Compute matching pairs of nested Rcu-lock and Rcu-unlock *) +let matched = let rec + unmatched-locks = Rcu-lock \ domain(matched) + and unmatched-unlocks = Rcu-unlock \ range(matched) + and unmatched = unmatched-locks | unmatched-unlocks + and unmatched-po = [unmatched] ; po ; [unmatched] + and unmatched-locks-to-unlocks = + [unmatched-locks] ; po ; [unmatched-unlocks] + and matched = matched | (unmatched-locks-to-unlocks \ + (unmatched-po ; unmatched-po)) + in matched + +(* Validate nesting *) +flag ~empty Rcu-lock \ domain(matched) as unbalanced-rcu-locking +flag ~empty Rcu-unlock \ range(matched) as unbalanced-rcu-locking + +(* Outermost level of nesting only *) +let crit = matched \ (po^-1 ; matched ; po^-1) diff --git a/tools/memory-model/linux-kernel.cat b/tools/memory-model/linux-kernel.cat new file mode 100644 index 000000000000..15b7a5dd8a9a --- /dev/null +++ b/tools/memory-model/linux-kernel.cat @@ -0,0 +1,124 @@ +// SPDX-License-Identifier: GPL-2.0+ +(* + * Copyright (C) 2015 Jade Alglave , + * Copyright (C) 2016 Luc Maranget for Inria + * Copyright (C) 2017 Alan Stern , + * Andrea Parri + * + * An earlier version of this file appears in the companion webpage for + * "Frightening small children and disconcerting grown-ups: Concurrency + * in the Linux kernel" by Alglave, Maranget, McKenney, Parri, and Stern, + * which is to appear in ASPLOS 2018. + *) + +"Linux kernel memory model" + +(* + * File "lock.cat" handles locks and is experimental. + * It can be replaced by include "cos.cat" for tests that do not use locks. + *) + +include "lock.cat" + +(*******************) +(* Basic relations *) +(*******************) + +(* Fences *) +let rb-dep = [R] ; fencerel(Rb_dep) ; [R] +let rmb = [R \ Noreturn] ; fencerel(Rmb) ; [R \ Noreturn] +let wmb = [W] ; fencerel(Wmb) ; [W] +let mb = ([M] ; fencerel(Mb) ; [M]) | + ([M] ; fencerel(Before_atomic) ; [RMW] ; po? ; [M]) | + ([M] ; po? ; [RMW] ; fencerel(After_atomic) ; [M]) | + ([M] ; po? ; [LKW] ; fencerel(After_spinlock) ; [M]) +let gp = po ; [Sync-rcu] ; po? + +let strong-fence = mb | gp + +(* Release Acquire *) +let acq-po = [Acquire] ; po ; [M] +let po-rel = [M] ; po ; [Release] +let rfi-rel-acq = [Release] ; rfi ; [Acquire] + +(**********************************) +(* Fundamental coherence ordering *) +(**********************************) + +(* Sequential Consistency Per Variable *) +let com = rf | co | fr +acyclic po-loc | com as coherence + +(* Atomic Read-Modify-Write *) +empty rmw & (fre ; coe) as atomic + +(**********************************) +(* Instruction execution ordering *) +(**********************************) + +(* Preserved Program Order *) +let dep = addr | data +let rwdep = (dep | ctrl) ; [W] +let overwrite = co | fr +let to-w = rwdep | (overwrite & int) +let rrdep = addr | (dep ; rfi) +let strong-rrdep = rrdep+ & rb-dep +let to-r = strong-rrdep | rfi-rel-acq +let fence = strong-fence | wmb | po-rel | rmb | acq-po +let ppo = rrdep* ; (to-r | to-w | fence) + +(* Propagation: Ordering from release operations and strong fences. *) +let A-cumul(r) = rfe? ; r +let cumul-fence = A-cumul(strong-fence | po-rel) | wmb +let prop = (overwrite & ext)? ; cumul-fence* ; rfe? + +(* + * Happens Before: Ordering from the passage of time. + * No fences needed here for prop because relation confined to one process. + *) +let hb = ppo | rfe | ((prop \ id) & int) +acyclic hb as happens-before + +(****************************************) +(* Write and fence propagation ordering *) +(****************************************) + +(* Propagation: Each non-rf link needs a strong fence. *) +let pb = prop ; strong-fence ; hb* +acyclic pb as propagation + +(*******) +(* RCU *) +(*******) + +(* + * Effect of read-side critical section proceeds from the rcu_read_lock() + * onward on the one hand and from the rcu_read_unlock() backwards on the + * other hand. + *) +let rscs = po ; crit^-1 ; po? + +(* + * The synchronize_rcu() strong fence is special in that it can order not + * one but two non-rf relations, but only in conjunction with an RCU + * read-side critical section. + *) +let link = hb* ; pb* ; prop + +(* Chains that affect the RCU grace-period guarantee *) +let gp-link = gp ; link +let rscs-link = rscs ; link + +(* + * A cycle containing at least as many grace periods as RCU read-side + * critical sections is forbidden. + *) +let rec rcu-path = + gp-link | + (gp-link ; rscs-link) | + (rscs-link ; gp-link) | + (rcu-path ; rcu-path) | + (gp-link ; rcu-path ; rscs-link) | + (rscs-link ; rcu-path ; gp-link) + +irreflexive rcu-path as rcu diff --git a/tools/memory-model/linux-kernel.cfg b/tools/memory-model/linux-kernel.cfg new file mode 100644 index 000000000000..3c8098e99f41 --- /dev/null +++ b/tools/memory-model/linux-kernel.cfg @@ -0,0 +1,21 @@ +macros linux-kernel.def +bell linux-kernel.bell +model linux-kernel.cat +graph columns +squished true +showevents noregs +movelabel true +fontsize 8 +xscale 2.0 +yscale 1.5 +arrowsize 0.8 +showinitrf false +showfinalrf false +showinitwrites false +splines spline +pad 0.1 +edgeattr hb,color,indigo +edgeattr co,color,blue +edgeattr mb,color,darkgreen +edgeattr wmb,color,darkgreen +edgeattr rmb,color,darkgreen diff --git a/tools/memory-model/linux-kernel.def b/tools/memory-model/linux-kernel.def new file mode 100644 index 000000000000..a397387f77cc --- /dev/null +++ b/tools/memory-model/linux-kernel.def @@ -0,0 +1,108 @@ +// SPDX-License-Identifier: GPL-2.0+ +// +// An earlier version of this file appears in the companion webpage for +// "Frightening small children and disconcerting grown-ups: Concurrency +// in the Linux kernel" by Alglave, Maranget, McKenney, Parri, and Stern, +// which is to appear in ASPLOS 2018. + +// ONCE +READ_ONCE(X) __load{once}(X) +WRITE_ONCE(X,V) { __store{once}(X,V); } + +// Release Acquire and friends +smp_store_release(X,V) { __store{release}(*X,V); } +smp_load_acquire(X) __load{acquire}(*X) +rcu_assign_pointer(X,V) { __store{release}(X,V); } +lockless_dereference(X) __load{lderef}(X) +rcu_dereference(X) __load{deref}(X) + +// Fences +smp_mb() { __fence{mb} ; } +smp_rmb() { __fence{rmb} ; } +smp_wmb() { __fence{wmb} ; } +smp_read_barrier_depends() { __fence{rb_dep}; } +smp_mb__before_atomic() { __fence{before_atomic} ; } +smp_mb__after_atomic() { __fence{after_atomic} ; } +smp_mb__after_spinlock() { __fence{after_spinlock} ; } + +// Exchange +xchg(X,V) __xchg{mb}(X,V) +xchg_relaxed(X,V) __xchg{once}(X,V) +xchg_release(X,V) __xchg{release}(X,V) +xchg_acquire(X,V) __xchg{acquire}(X,V) +cmpxchg(X,V,W) __cmpxchg{mb}(X,V,W) +cmpxchg_relaxed(X,V,W) __cmpxchg{once}(X,V,W) +cmpxchg_acquire(X,V,W) __cmpxchg{acquire}(X,V,W) +cmpxchg_release(X,V,W) __cmpxchg{release}(X,V,W) + +// Spinlocks +spin_lock(X) { __lock(X) ; } +spin_unlock(X) { __unlock(X) ; } +spin_trylock(X) __trylock(X) + +// RCU +rcu_read_lock() { __fence{rcu-lock}; } +rcu_read_unlock() { __fence{rcu-unlock};} +synchronize_rcu() { __fence{sync-rcu}; } +synchronize_rcu_expedited() { __fence{sync-rcu}; } + +// Atomic +atomic_read(X) READ_ONCE(*X) +atomic_set(X,V) { WRITE_ONCE(*X,V) ; } +atomic_read_acquire(X) smp_load_acquire(X) +atomic_set_release(X,V) { smp_store_release(X,V); } + +atomic_add(V,X) { __atomic_op(X,+,V) ; } +atomic_sub(V,X) { __atomic_op(X,-,V) ; } +atomic_inc(X) { __atomic_op(X,+,1) ; } +atomic_dec(X) { __atomic_op(X,-,1) ; } + +atomic_add_return(V,X) __atomic_op_return{mb}(X,+,V) +atomic_add_return_relaxed(V,X) __atomic_op_return{once}(X,+,V) +atomic_add_return_acquire(V,X) __atomic_op_return{acquire}(X,+,V) +atomic_add_return_release(V,X) __atomic_op_return{release}(X,+,V) +atomic_fetch_add(V,X) __atomic_fetch_op{mb}(X,+,V) +atomic_fetch_add_relaxed(V,X) __atomic_fetch_op{once}(X,+,V) +atomic_fetch_add_acquire(V,X) __atomic_fetch_op{acquire}(X,+,V) +atomic_fetch_add_release(V,X) __atomic_fetch_op{release}(X,+,V) + +atomic_inc_return(X) __atomic_op_return{mb}(X,+,1) +atomic_inc_return_relaxed(X) __atomic_op_return{once}(X,+,1) +atomic_inc_return_acquire(X) __atomic_op_return{acquire}(X,+,1) +atomic_inc_return_release(X) __atomic_op_return{release}(X,+,1) +atomic_fetch_inc(X) __atomic_fetch_op{mb}(X,+,1) +atomic_fetch_inc_relaxed(X) __atomic_fetch_op{once}(X,+,1) +atomic_fetch_inc_acquire(X) __atomic_fetch_op{acquire}(X,+,1) +atomic_fetch_inc_release(X) __atomic_fetch_op{release}(X,+,1) + +atomic_sub_return(V,X) __atomic_op_return{mb}(X,-,V) +atomic_sub_return_relaxed(V,X) __atomic_op_return{once}(X,-,V) +atomic_sub_return_acquire(V,X) __atomic_op_return{acquire}(X,-,V) +atomic_sub_return_release(V,X) __atomic_op_return{release}(X,-,V) +atomic_fetch_sub(V,X) __atomic_fetch_op{mb}(X,-,V) +atomic_fetch_sub_relaxed(V,X) __atomic_fetch_op{once}(X,-,V) +atomic_fetch_sub_acquire(V,X) __atomic_fetch_op{acquire}(X,-,V) +atomic_fetch_sub_release(V,X) __atomic_fetch_op{release}(X,-,V) + +atomic_dec_return(X) __atomic_op_return{mb}(X,-,1) +atomic_dec_return_relaxed(X) __atomic_op_return{once}(X,-,1) +atomic_dec_return_acquire(X) __atomic_op_return{acquire}(X,-,1) +atomic_dec_return_release(X) __atomic_op_return{release}(X,-,1) +atomic_fetch_dec(X) __atomic_fetch_op{mb}(X,-,1) +atomic_fetch_dec_relaxed(X) __atomic_fetch_op{once}(X,-,1) +atomic_fetch_dec_acquire(X) __atomic_fetch_op{acquire}(X,-,1) +atomic_fetch_dec_release(X) __atomic_fetch_op{release}(X,-,1) + +atomic_xchg(X,V) __xchg{mb}(X,V) +atomic_xchg_relaxed(X,V) __xchg{once}(X,V) +atomic_xchg_release(X,V) __xchg{release}(X,V) +atomic_xchg_acquire(X,V) __xchg{acquire}(X,V) +atomic_cmpxchg(X,V,W) __cmpxchg{mb}(X,V,W) +atomic_cmpxchg_relaxed(X,V,W) __cmpxchg{once}(X,V,W) +atomic_cmpxchg_acquire(X,V,W) __cmpxchg{acquire}(X,V,W) +atomic_cmpxchg_release(X,V,W) __cmpxchg{release}(X,V,W) + +atomic_sub_and_test(V,X) __atomic_op_return{mb}(X,-,V) == 0 +atomic_dec_and_test(X) __atomic_op_return{mb}(X,-,1) == 0 +atomic_inc_and_test(X) __atomic_op_return{mb}(X,+,1) == 0 +atomic_add_negative(V,X) __atomic_op_return{mb}(X,+,V) < 0 diff --git a/tools/memory-model/litmus-tests/CoRR+poonceonce+Once.litmus b/tools/memory-model/litmus-tests/CoRR+poonceonce+Once.litmus new file mode 100644 index 000000000000..5b83d57f6ac5 --- /dev/null +++ b/tools/memory-model/litmus-tests/CoRR+poonceonce+Once.litmus @@ -0,0 +1,19 @@ +C CoRR+poonceonce+Once + +{} + +P0(int *x) +{ + WRITE_ONCE(*x, 1); +} + +P1(int *x) +{ + int r0; + int r1; + + r0 = READ_ONCE(*x); + r1 = READ_ONCE(*x); +} + +exists (1:r0=1 /\ 1:r1=0) diff --git a/tools/memory-model/litmus-tests/CoRW+poonceonce+Once.litmus b/tools/memory-model/litmus-tests/CoRW+poonceonce+Once.litmus new file mode 100644 index 000000000000..fab91c13d52c --- /dev/null +++ b/tools/memory-model/litmus-tests/CoRW+poonceonce+Once.litmus @@ -0,0 +1,18 @@ +C CoRW+poonceonce+Once + +{} + +P0(int *x) +{ + int r0; + + r0 = READ_ONCE(*x); + WRITE_ONCE(*x, 1); +} + +P1(int *x) +{ + WRITE_ONCE(*x, 2); +} + +exists (x=2 /\ 0:r0=2) diff --git a/tools/memory-model/litmus-tests/CoWR+poonceonce+Once.litmus b/tools/memory-model/litmus-tests/CoWR+poonceonce+Once.litmus new file mode 100644 index 000000000000..6a35ec2042ea --- /dev/null +++ b/tools/memory-model/litmus-tests/CoWR+poonceonce+Once.litmus @@ -0,0 +1,18 @@ +C CoWR+poonceonce+Once + +{} + +P0(int *x) +{ + int r0; + + WRITE_ONCE(*x, 1); + r0 = READ_ONCE(*x); +} + +P1(int *x) +{ + WRITE_ONCE(*x, 2); +} + +exists (x=1 /\ 0:r0=2) diff --git a/tools/memory-model/litmus-tests/CoWW+poonceonce.litmus b/tools/memory-model/litmus-tests/CoWW+poonceonce.litmus new file mode 100644 index 000000000000..32a96b832021 --- /dev/null +++ b/tools/memory-model/litmus-tests/CoWW+poonceonce.litmus @@ -0,0 +1,11 @@ +C CoWW+poonceonce + +{} + +P0(int *x) +{ + WRITE_ONCE(*x, 1); + WRITE_ONCE(*x, 2); +} + +exists (x=1) diff --git a/tools/memory-model/litmus-tests/IRIW+mbonceonces+OnceOnce.litmus b/tools/memory-model/litmus-tests/IRIW+mbonceonces+OnceOnce.litmus new file mode 100644 index 000000000000..7eba2c68992b --- /dev/null +++ b/tools/memory-model/litmus-tests/IRIW+mbonceonces+OnceOnce.litmus @@ -0,0 +1,35 @@ +C IRIW+mbonceonces+OnceOnce + +{} + +P0(int *x) +{ + WRITE_ONCE(*x, 1); +} + +P1(int *x, int *y) +{ + int r0; + int r1; + + r0 = READ_ONCE(*x); + smp_mb(); + r1 = READ_ONCE(*y); +} + +P2(int *y) +{ + WRITE_ONCE(*y, 1); +} + +P3(int *x, int *y) +{ + int r0; + int r1; + + r0 = READ_ONCE(*y); + smp_mb(); + r1 = READ_ONCE(*x); +} + +exists (1:r0=1 /\ 1:r1=0 /\ 3:r0=1 /\ 3:r1=0) diff --git a/tools/memory-model/litmus-tests/IRIW+poonceonces+OnceOnce.litmus b/tools/memory-model/litmus-tests/IRIW+poonceonces+OnceOnce.litmus new file mode 100644 index 000000000000..b0556c6c75d4 --- /dev/null +++ b/tools/memory-model/litmus-tests/IRIW+poonceonces+OnceOnce.litmus @@ -0,0 +1,33 @@ +C IRIW+poonceonces+OnceOnce + +{} + +P0(int *x) +{ + WRITE_ONCE(*x, 1); +} + +P1(int *x, int *y) +{ + int r0; + int r1; + + r0 = READ_ONCE(*x); + r1 = READ_ONCE(*y); +} + +P2(int *y) +{ + WRITE_ONCE(*y, 1); +} + +P3(int *x, int *y) +{ + int r0; + int r1; + + r0 = READ_ONCE(*y); + r1 = READ_ONCE(*x); +} + +exists (1:r0=1 /\ 1:r1=0 /\ 3:r0=1 /\ 3:r1=0) diff --git a/tools/memory-model/litmus-tests/ISA2+poonceonces.litmus b/tools/memory-model/litmus-tests/ISA2+poonceonces.litmus new file mode 100644 index 000000000000..9a1a233d70c3 --- /dev/null +++ b/tools/memory-model/litmus-tests/ISA2+poonceonces.litmus @@ -0,0 +1,28 @@ +C ISA2+poonceonces + +{} + +P0(int *x, int *y) +{ + WRITE_ONCE(*x, 1); + WRITE_ONCE(*y, 1); +} + +P1(int *y, int *z) +{ + int r0; + + r0 = READ_ONCE(*y); + WRITE_ONCE(*z, 1); +} + +P2(int *x, int *z) +{ + int r0; + int r1; + + r0 = READ_ONCE(*z); + r1 = READ_ONCE(*x); +} + +exists (1:r0=1 /\ 2:r0=1 /\ 2:r1=0) diff --git a/tools/memory-model/litmus-tests/ISA2+pooncerelease+poacquirerelease+poacquireonce.litmus b/tools/memory-model/litmus-tests/ISA2+pooncerelease+poacquirerelease+poacquireonce.litmus new file mode 100644 index 000000000000..235195e87d4e --- /dev/null +++ b/tools/memory-model/litmus-tests/ISA2+pooncerelease+poacquirerelease+poacquireonce.litmus @@ -0,0 +1,28 @@ +C ISA2+pooncerelease+poacquirerelease+poacquireonce + +{} + +P0(int *x, int *y) +{ + WRITE_ONCE(*x, 1); + smp_store_release(y, 1); +} + +P1(int *y, int *z) +{ + int r0; + + r0 = smp_load_acquire(y); + smp_store_release(z, 1); +} + +P2(int *x, int *z) +{ + int r0; + int r1; + + r0 = smp_load_acquire(z); + r1 = READ_ONCE(*x); +} + +exists (1:r0=1 /\ 2:r0=1 /\ 2:r1=0) diff --git a/tools/memory-model/litmus-tests/LB+ctrlonceonce+mbonceonce.litmus b/tools/memory-model/litmus-tests/LB+ctrlonceonce+mbonceonce.litmus new file mode 100644 index 000000000000..dd5ac3a8974a --- /dev/null +++ b/tools/memory-model/litmus-tests/LB+ctrlonceonce+mbonceonce.litmus @@ -0,0 +1,23 @@ +C LB+ctrlonceonce+mbonceonce + +{} + +P0(int *x, int *y) +{ + int r0; + + r0 = READ_ONCE(*x); + if (r0) + WRITE_ONCE(*y, 1); +} + +P1(int *x, int *y) +{ + int r0; + + r0 = READ_ONCE(*y); + smp_mb(); + WRITE_ONCE(*x, 1); +} + +exists (0:r0=1 /\ 1:r0=1) diff --git a/tools/memory-model/litmus-tests/LB+poacquireonce+pooncerelease.litmus b/tools/memory-model/litmus-tests/LB+poacquireonce+pooncerelease.litmus new file mode 100644 index 000000000000..47bd61319d93 --- /dev/null +++ b/tools/memory-model/litmus-tests/LB+poacquireonce+pooncerelease.litmus @@ -0,0 +1,21 @@ +C LB+poacquireonce+pooncerelease + +{} + +P0(int *x, int *y) +{ + int r0; + + r0 = READ_ONCE(*x); + smp_store_release(y, 1); +} + +P1(int *x, int *y) +{ + int r0; + + r0 = smp_load_acquire(y); + WRITE_ONCE(*x, 1); +} + +exists (0:r0=1 /\ 1:r0=1) diff --git a/tools/memory-model/litmus-tests/LB+poonceonces.litmus b/tools/memory-model/litmus-tests/LB+poonceonces.litmus new file mode 100644 index 000000000000..a5cdf027e34b --- /dev/null +++ b/tools/memory-model/litmus-tests/LB+poonceonces.litmus @@ -0,0 +1,21 @@ +C LB+poonceonces + +{} + +P0(int *x, int *y) +{ + int r0; + + r0 = READ_ONCE(*x); + WRITE_ONCE(*y, 1); +} + +P1(int *x, int *y) +{ + int r0; + + r0 = READ_ONCE(*y); + WRITE_ONCE(*x, 1); +} + +exists (0:r0=1 /\ 1:r0=1) diff --git a/tools/memory-model/litmus-tests/MP+onceassign+derefonce.litmus b/tools/memory-model/litmus-tests/MP+onceassign+derefonce.litmus new file mode 100644 index 000000000000..1a2fe5830381 --- /dev/null +++ b/tools/memory-model/litmus-tests/MP+onceassign+derefonce.litmus @@ -0,0 +1,25 @@ +C MP+onceassign+derefonce.litmus + +{ +y=z; +z=0; +} + +P0(int *x, int **y) +{ + WRITE_ONCE(*x, 1); + rcu_assign_pointer(*y, x); +} + +P1(int *x, int **y) +{ + int *r0; + int r1; + + rcu_read_lock(); + r0 = rcu_dereference(*y); + r1 = READ_ONCE(*r0); + rcu_read_unlock(); +} + +exists (1:r0=x /\ 1:r1=0) diff --git a/tools/memory-model/litmus-tests/MP+polocks.litmus b/tools/memory-model/litmus-tests/MP+polocks.litmus new file mode 100644 index 000000000000..5fe6f1e3c452 --- /dev/null +++ b/tools/memory-model/litmus-tests/MP+polocks.litmus @@ -0,0 +1,24 @@ +C MP+polocks + +{} + +P0(int *x, int *y, spinlock_t *mylock) +{ + WRITE_ONCE(*x, 1); + spin_lock(mylock); + WRITE_ONCE(*y, 1); + spin_unlock(mylock); +} + +P1(int *x, int *y, spinlock_t *mylock) +{ + int r0; + int r1; + + spin_lock(mylock); + r0 = READ_ONCE(*y); + spin_unlock(mylock); + r1 = READ_ONCE(*x); +} + +exists (1:r0=1 /\ 1:r1=0) diff --git a/tools/memory-model/litmus-tests/MP+poonceonces.litmus b/tools/memory-model/litmus-tests/MP+poonceonces.litmus new file mode 100644 index 000000000000..46e1ac7ba126 --- /dev/null +++ b/tools/memory-model/litmus-tests/MP+poonceonces.litmus @@ -0,0 +1,20 @@ +C MP+poonceonces + +{} + +P0(int *x, int *y) +{ + WRITE_ONCE(*x, 1); + WRITE_ONCE(*y, 1); +} + +P1(int *x, int *y) +{ + int r0; + int r1; + + r0 = READ_ONCE(*y); + r1 = READ_ONCE(*x); +} + +exists (1:r0=1 /\ 1:r1=0) diff --git a/tools/memory-model/litmus-tests/MP+pooncerelease+poacquireonce.litmus b/tools/memory-model/litmus-tests/MP+pooncerelease+poacquireonce.litmus new file mode 100644 index 000000000000..0b00cc7293ba --- /dev/null +++ b/tools/memory-model/litmus-tests/MP+pooncerelease+poacquireonce.litmus @@ -0,0 +1,20 @@ +C MP+pooncerelease+poacquireonce + +{} + +P0(int *x, int *y) +{ + WRITE_ONCE(*x, 1); + smp_store_release(y, 1); +} + +P1(int *x, int *y) +{ + int r0; + int r1; + + r0 = smp_load_acquire(y); + r1 = READ_ONCE(*x); +} + +exists (1:r0=1 /\ 1:r1=0) diff --git a/tools/memory-model/litmus-tests/MP+porevlocks.litmus b/tools/memory-model/litmus-tests/MP+porevlocks.litmus new file mode 100644 index 000000000000..90d011c34f33 --- /dev/null +++ b/tools/memory-model/litmus-tests/MP+porevlocks.litmus @@ -0,0 +1,24 @@ +C MP+porevlocks + +{} + +P0(int *x, int *y, spinlock_t *mylock) +{ + int r0; + int r1; + + r0 = READ_ONCE(*y); + spin_lock(mylock); + r1 = READ_ONCE(*x); + spin_unlock(mylock); +} + +P1(int *x, int *y, spinlock_t *mylock) +{ + spin_lock(mylock); + WRITE_ONCE(*x, 1); + spin_unlock(mylock); + WRITE_ONCE(*y, 1); +} + +exists (0:r0=1 /\ 0:r1=0) diff --git a/tools/memory-model/litmus-tests/MP+wmbonceonce+rmbonceonce.litmus b/tools/memory-model/litmus-tests/MP+wmbonceonce+rmbonceonce.litmus new file mode 100644 index 000000000000..604ad41ea0c2 --- /dev/null +++ b/tools/memory-model/litmus-tests/MP+wmbonceonce+rmbonceonce.litmus @@ -0,0 +1,22 @@ +C MP+wmbonceonce+rmbonceonce + +{} + +P0(int *x, int *y) +{ + WRITE_ONCE(*x, 1); + smp_wmb(); + WRITE_ONCE(*y, 1); +} + +P1(int *x, int *y) +{ + int r0; + int r1; + + r0 = READ_ONCE(*y); + smp_rmb(); + r1 = READ_ONCE(*x); +} + +exists (1:r0=1 /\ 1:r1=0) diff --git a/tools/memory-model/litmus-tests/R+mbonceonces.litmus b/tools/memory-model/litmus-tests/R+mbonceonces.litmus new file mode 100644 index 000000000000..e69b9e3e9436 --- /dev/null +++ b/tools/memory-model/litmus-tests/R+mbonceonces.litmus @@ -0,0 +1,21 @@ +C R+mbonceonces + +{} + +P0(int *x, int *y) +{ + WRITE_ONCE(*x, 1); + smp_mb(); + WRITE_ONCE(*y, 1); +} + +P1(int *x, int *y) +{ + int r0; + + WRITE_ONCE(*y, 2); + smp_mb(); + r0 = READ_ONCE(*x); +} + +exists (y=2 /\ 1:r0=0) diff --git a/tools/memory-model/litmus-tests/R+poonceonces.litmus b/tools/memory-model/litmus-tests/R+poonceonces.litmus new file mode 100644 index 000000000000..f7a12e00f82d --- /dev/null +++ b/tools/memory-model/litmus-tests/R+poonceonces.litmus @@ -0,0 +1,19 @@ +C R+poonceonces + +{} + +P0(int *x, int *y) +{ + WRITE_ONCE(*x, 1); + WRITE_ONCE(*y, 1); +} + +P1(int *x, int *y) +{ + int r0; + + WRITE_ONCE(*y, 2); + r0 = READ_ONCE(*x); +} + +exists (y=2 /\ 1:r0=0) diff --git a/tools/memory-model/litmus-tests/README b/tools/memory-model/litmus-tests/README new file mode 100644 index 000000000000..9a3bb5949191 --- /dev/null +++ b/tools/memory-model/litmus-tests/README @@ -0,0 +1,125 @@ +This directory contains the following litmus tests: + +CoRR+poonceonce+Once.litmus + Test of read-read coherence, that is, whether or not two + successive reads from the same variable are ordered. + +CoRW+poonceonce+Once.litmus + Test of read-write coherence, that is, whether or not a read + from a given variable followed by a write to that same variable + are ordered. + +CoWR+poonceonce+Once.litmus + Test of write-read coherence, that is, whether or not a write + to a given variable followed by a read from that same variable + are ordered. + +CoWW+poonceonce.litmus + Test of write-write coherence, that is, whether or not two + successive writes to the same variable are ordered. + +IRIW+mbonceonces+OnceOnce.litmus + Test of independent reads from independent writes with smp_mb() + between each pairs of reads. In other words, is smp_mb() + sufficient to cause two different reading processes to agree on + the order of a pair of writes, where each write is to a different + variable by a different process. + +IRIW+poonceonces+OnceOnce.litmus + Test of independent reads from independent writes with nothing + between each pairs of reads. In other words, is anything at all + needed to cause two different reading processes to agree on the + order of a pair of writes, where each write is to a different + variable by a different process. + +ISA2+poonceonces.litmus + As below, but with store-release replaced with WRITE_ONCE() + and load-acquire replaced with READ_ONCE(). + +ISA2+pooncerelease+poacquirerelease+poacquireonce.litmus + Can a release-acquire chain order a prior store against + a later load? + +LB+ctrlonceonce+mbonceonce.litmus + Does a control dependency and an smp_mb() suffice for the + load-buffering litmus test, where each process reads from one + of two variables then writes to the other? + +LB+poacquireonce+pooncerelease.litmus + Does a release-acquire pair suffice for the load-buffering + litmus test, where each process reads from one of two variables then + writes to the other? + +LB+poonceonces.litmus + As above, but with store-release replaced with WRITE_ONCE() + and load-acquire replaced with READ_ONCE(). + +MP+onceassign+derefonce.litmus + As below, but with rcu_assign_pointer() and an rcu_dereference(). + +MP+polocks.litmus + As below, but with the second access of the writer process + and the first access of reader process protected by a lock. + +MP+poonceonces.litmus + As below, but without the smp_rmb() and smp_wmb(). + +MP+pooncerelease+poacquireonce.litmus + As below, but with a release-acquire chain. + +MP+porevlocks.litmus + As below, but with the first access of the writer process + and the second access of reader process protected by a lock. + +MP+wmbonceonce+rmbonceonce.litmus + Does a smp_wmb() (between the stores) and an smp_rmb() (between + the loads) suffice for the message-passing litmus test, where one + process writes data and then a flag, and the other process reads + the flag and then the data. (This is similar to the ISA2 tests, + but with two processes instead of three.) + +R+mbonceonces.litmus + This is the fully ordered (via smp_mb()) version of one of + the classic counterintuitive litmus tests that illustrates the + effects of store propagation delays. + +R+poonceonces.litmus + As above, but without the smp_mb() invocations. + +SB+mbonceonces.litmus + This is the fully ordered (again, via smp_mb() version of store + buffering, which forms the core of Dekker's mutual-exclusion + algorithm. + +SB+poonceonces.litmus + As above, but without the smp_mb() invocations. + +S+poonceonces.litmus + As below, but without the smp_wmb() and acquire load. + +S+wmbonceonce+poacquireonce.litmus + Can a smp_wmb(), instead of a release, and an acquire order + a prior store against a subsequent store? + +WRC+poonceonces+Once.litmus +WRC+pooncerelease+rmbonceonce+Once.litmus + These two are members of an extension of the MP litmus-test class + in which the first write is moved to a separate process. + +Z6.0+pooncelock+pooncelock+pombonce.litmus + Is the ordering provided by a spin_unlock() and a subsequent + spin_lock() sufficient to make ordering apparent to accesses + by a process not holding the lock? + +Z6.0+pooncelock+poonceLock+pombonce.litmus + As above, but with smp_mb__after_spinlock() immediately + following the spin_lock(). + +Z6.0+pooncerelease+poacquirerelease+mbonceonce.litmus + Is the ordering provided by a release-acquire chain sufficient + to make ordering apparent to accesses by a process that does + not participate in that release-acquire chain? + +A great many more litmus tests are available here: + + https://github.com/paulmckrcu/litmus diff --git a/tools/memory-model/litmus-tests/S+poonceonces.litmus b/tools/memory-model/litmus-tests/S+poonceonces.litmus new file mode 100644 index 000000000000..d0d541c8ec7d --- /dev/null +++ b/tools/memory-model/litmus-tests/S+poonceonces.litmus @@ -0,0 +1,19 @@ +C S+poonceonces + +{} + +P0(int *x, int *y) +{ + WRITE_ONCE(*x, 2); + WRITE_ONCE(*y, 1); +} + +P1(int *x, int *y) +{ + int r0; + + r0 = READ_ONCE(*y); + WRITE_ONCE(*x, 1); +} + +exists (x=2 /\ 1:r0=1) diff --git a/tools/memory-model/litmus-tests/S+wmbonceonce+poacquireonce.litmus b/tools/memory-model/litmus-tests/S+wmbonceonce+poacquireonce.litmus new file mode 100644 index 000000000000..1d292d0d6603 --- /dev/null +++ b/tools/memory-model/litmus-tests/S+wmbonceonce+poacquireonce.litmus @@ -0,0 +1,20 @@ +C S+wmbonceonce+poacquireonce + +{} + +P0(int *x, int *y) +{ + WRITE_ONCE(*x, 2); + smp_wmb(); + WRITE_ONCE(*y, 1); +} + +P1(int *x, int *y) +{ + int r0; + + r0 = smp_load_acquire(y); + WRITE_ONCE(*x, 1); +} + +exists (x=2 /\ 1:r0=1) diff --git a/tools/memory-model/litmus-tests/SB+mbonceonces.litmus b/tools/memory-model/litmus-tests/SB+mbonceonces.litmus new file mode 100644 index 000000000000..b76caa5af1af --- /dev/null +++ b/tools/memory-model/litmus-tests/SB+mbonceonces.litmus @@ -0,0 +1,23 @@ +C SB+mbonceonces + +{} + +P0(int *x, int *y) +{ + int r0; + + WRITE_ONCE(*x, 1); + smp_mb(); + r0 = READ_ONCE(*y); +} + +P1(int *x, int *y) +{ + int r0; + + WRITE_ONCE(*y, 1); + smp_mb(); + r0 = READ_ONCE(*x); +} + +exists (0:r0=0 /\ 1:r0=0) diff --git a/tools/memory-model/litmus-tests/SB+poonceonces.litmus b/tools/memory-model/litmus-tests/SB+poonceonces.litmus new file mode 100644 index 000000000000..c1797e03807e --- /dev/null +++ b/tools/memory-model/litmus-tests/SB+poonceonces.litmus @@ -0,0 +1,21 @@ +C SB+poonceonces + +{} + +P0(int *x, int *y) +{ + int r0; + + WRITE_ONCE(*x, 1); + r0 = READ_ONCE(*y); +} + +P1(int *x, int *y) +{ + int r0; + + WRITE_ONCE(*y, 1); + r0 = READ_ONCE(*x); +} + +exists (0:r0=0 /\ 1:r0=0) diff --git a/tools/memory-model/litmus-tests/WRC+poonceonces+Once.litmus b/tools/memory-model/litmus-tests/WRC+poonceonces+Once.litmus new file mode 100644 index 000000000000..f5e7c92f61cc --- /dev/null +++ b/tools/memory-model/litmus-tests/WRC+poonceonces+Once.litmus @@ -0,0 +1,27 @@ +C WRC+poonceonces+Once + +{} + +P0(int *x) +{ + WRITE_ONCE(*x, 1); +} + +P1(int *x, int *y) +{ + int r0; + + r0 = READ_ONCE(*x); + WRITE_ONCE(*y, 1); +} + +P2(int *x, int *y) +{ + int r0; + int r1; + + r0 = READ_ONCE(*y); + r1 = READ_ONCE(*x); +} + +exists (1:r0=1 /\ 2:r0=1 /\ 2:r1=0) diff --git a/tools/memory-model/litmus-tests/WRC+pooncerelease+rmbonceonce+Once.litmus b/tools/memory-model/litmus-tests/WRC+pooncerelease+rmbonceonce+Once.litmus new file mode 100644 index 000000000000..e3d0018025dd --- /dev/null +++ b/tools/memory-model/litmus-tests/WRC+pooncerelease+rmbonceonce+Once.litmus @@ -0,0 +1,28 @@ +C WRC+pooncerelease+rmbonceonce+Once + +{} + +P0(int *x) +{ + WRITE_ONCE(*x, 1); +} + +P1(int *x, int *y) +{ + int r0; + + r0 = READ_ONCE(*x); + smp_store_release(y, 1); +} + +P2(int *x, int *y) +{ + int r0; + int r1; + + r0 = READ_ONCE(*y); + smp_rmb(); + r1 = READ_ONCE(*x); +} + +exists (1:r0=1 /\ 2:r0=1 /\ 2:r1=0) diff --git a/tools/memory-model/litmus-tests/Z6.0+pooncelock+poonceLock+pombonce.litmus b/tools/memory-model/litmus-tests/Z6.0+pooncelock+poonceLock+pombonce.litmus new file mode 100644 index 000000000000..9c2cb53e6ef0 --- /dev/null +++ b/tools/memory-model/litmus-tests/Z6.0+pooncelock+poonceLock+pombonce.litmus @@ -0,0 +1,33 @@ +C Z6.0+pooncelock+poonceLock+pombonce + +{} + +P0(int *x, int *y, spinlock_t *mylock) +{ + spin_lock(mylock); + WRITE_ONCE(*x, 1); + WRITE_ONCE(*y, 1); + spin_unlock(mylock); +} + +P1(int *y, int *z, spinlock_t *mylock) +{ + int r0; + + spin_lock(mylock); + smp_mb__after_spinlock(); + r0 = READ_ONCE(*y); + WRITE_ONCE(*z, 1); + spin_unlock(mylock); +} + +P2(int *x, int *z) +{ + int r1; + + WRITE_ONCE(*z, 2); + smp_mb(); + r1 = READ_ONCE(*x); +} + +exists (1:r0=1 /\ z=2 /\ 2:r1=0) diff --git a/tools/memory-model/litmus-tests/Z6.0+pooncelock+pooncelock+pombonce.litmus b/tools/memory-model/litmus-tests/Z6.0+pooncelock+pooncelock+pombonce.litmus new file mode 100644 index 000000000000..c9a1f1a49ae1 --- /dev/null +++ b/tools/memory-model/litmus-tests/Z6.0+pooncelock+pooncelock+pombonce.litmus @@ -0,0 +1,32 @@ +C Z6.0+pooncelock+pooncelock+pombonce + +{} + +P0(int *x, int *y, spinlock_t *mylock) +{ + spin_lock(mylock); + WRITE_ONCE(*x, 1); + WRITE_ONCE(*y, 1); + spin_unlock(mylock); +} + +P1(int *y, int *z, spinlock_t *mylock) +{ + int r0; + + spin_lock(mylock); + r0 = READ_ONCE(*y); + WRITE_ONCE(*z, 1); + spin_unlock(mylock); +} + +P2(int *x, int *z) +{ + int r1; + + WRITE_ONCE(*z, 2); + smp_mb(); + r1 = READ_ONCE(*x); +} + +exists (1:r0=1 /\ z=2 /\ 2:r1=0) diff --git a/tools/memory-model/litmus-tests/Z6.0+pooncerelease+poacquirerelease+mbonceonce.litmus b/tools/memory-model/litmus-tests/Z6.0+pooncerelease+poacquirerelease+mbonceonce.litmus new file mode 100644 index 000000000000..25409a033514 --- /dev/null +++ b/tools/memory-model/litmus-tests/Z6.0+pooncerelease+poacquirerelease+mbonceonce.litmus @@ -0,0 +1,28 @@ +C Z6.0+pooncerelease+poacquirerelease+mbonceonce + +{} + +P0(int *x, int *y) +{ + WRITE_ONCE(*x, 1); + smp_store_release(y, 1); +} + +P1(int *y, int *z) +{ + int r0; + + r0 = smp_load_acquire(y); + smp_store_release(z, 1); +} + +P2(int *x, int *z) +{ + int r1; + + WRITE_ONCE(*z, 2); + smp_mb(); + r1 = READ_ONCE(*x); +} + +exists (1:r0=1 /\ z=2 /\ 2:r1=0) diff --git a/tools/memory-model/lock.cat b/tools/memory-model/lock.cat new file mode 100644 index 000000000000..ba4a4ec6d313 --- /dev/null +++ b/tools/memory-model/lock.cat @@ -0,0 +1,99 @@ +// SPDX-License-Identifier: GPL-2.0+ +(* + * Copyright (C) 2016 Luc Maranget for Inria + * Copyright (C) 2017 Alan Stern + *) + +(* Generate coherence orders and handle lock operations *) + +include "cross.cat" + +(* From lock reads to their partner lock writes *) +let lk-rmw = ([LKR] ; po-loc ; [LKW]) \ (po ; po) +let rmw = rmw | lk-rmw + +(* + * A paired LKR must always see an unlocked value; spin_lock() calls nested + * inside a critical section (for the same lock) always deadlock. + *) +empty ([LKW] ; po-loc ; [domain(lk-rmw)]) \ (po-loc ; [UL] ; po-loc) + as lock-nest + +(* The litmus test is invalid if an LKW event is not part of an RMW pair *) +flag ~empty LKW \ range(lk-rmw) as unpaired-LKW + +(* This will be allowed if we implement spin_is_locked() *) +flag ~empty LKR \ domain(lk-rmw) as unpaired-LKR + +(* There should be no R or W accesses to spinlocks *) +let ALL-LOCKS = LKR | LKW | UL | LF +flag ~empty [M \ IW] ; loc ; [ALL-LOCKS] as mixed-lock-accesses + +(* The final value of a spinlock should not be tested *) +flag ~empty [FW] ; loc ; [ALL-LOCKS] as lock-final + + +(* + * Put lock operations in their appropriate classes, but leave UL out of W + * until after the co relation has been generated. + *) +let R = R | LKR | LF +let W = W | LKW + +let Release = Release | UL +let Acquire = Acquire | LKR + + +(* Match LKW events to their corresponding UL events *) +let critical = ([LKW] ; po-loc ; [UL]) \ (po-loc ; [LKW | UL] ; po-loc) + +flag ~empty UL \ range(critical) as unmatched-unlock + +(* Allow up to one unmatched LKW per location; more must deadlock *) +let UNMATCHED-LKW = LKW \ domain(critical) +empty ([UNMATCHED-LKW] ; loc ; [UNMATCHED-LKW]) \ id as unmatched-locks + + +(* rfi for LF events: link each LKW to the LF events in its critical section *) +let rfi-lf = ([LKW] ; po-loc ; [LF]) \ ([LKW] ; po-loc ; [UL] ; po-loc) + +(* rfe for LF events *) +let all-possible-rfe-lf = + (* + * Given an LF event r, compute the possible rfe edges for that event + * (all those starting from LKW events in other threads), + * and then convert that relation to a set of single-edge relations. + *) + let possible-rfe-lf r = + let pair-to-relation p = p ++ 0 + in map pair-to-relation ((LKW * {r}) & loc & ext) + (* Do this for each LF event r that isn't in rfi-lf *) + in map possible-rfe-lf (LF \ range(rfi-lf)) + +(* Generate all rf relations for LF events *) +with rfe-lf from cross(all-possible-rfe-lf) +let rf = rf | rfi-lf | rfe-lf + + +(* Generate all co relations, including LKW events but not UL *) +let co0 = co0 | ([IW] ; loc ; [LKW]) | + (([LKW] ; loc ; [UNMATCHED-LKW]) \ [UNMATCHED-LKW]) +include "cos-opt.cat" +let W = W | UL +let M = R | W + +(* Merge UL events into co *) +let co = (co | critical | (critical^-1 ; co))+ +let coe = co & ext +let coi = co & int + +(* Merge LKR events into rf *) +let rf = rf | ([IW | UL] ; singlestep(co) ; lk-rmw^-1) +let rfe = rf & ext +let rfi = rf & int + +let fr = rf^-1 ; co +let fre = fr & ext +let fri = fr & int + +show co,rf,fr -- cgit v1.2.3 From 0d8dd67be013727ae57645ecd3ea2c36365d7da8 Mon Sep 17 00:00:00 2001 From: Song Liu Date: Wed, 6 Dec 2017 14:45:14 -0800 Subject: perf/headers: Sync new perf_event.h with the tools/include/uapi version perf_event.h is updated in previous patch, this patch applies the same changes to the tools/ version. This is part is put in a separate patch in case the two files are back ported separately. Signed-off-by: Song Liu Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Yonghong Song Reviewed-by: Josef Bacik Acked-by: Alexei Starovoitov Cc: Cc: Cc: Cc: Cc: Arnaldo Carvalho de Melo Cc: Jiri Olsa Cc: Linus Torvalds Cc: Namhyung Kim Cc: Peter Zijlstra Cc: Thomas Gleixner Link: http://lkml.kernel.org/r/20171206224518.3598254-5-songliubraving@fb.com Signed-off-by: Ingo Molnar --- tools/include/uapi/linux/perf_event.h | 4 ++++ 1 file changed, 4 insertions(+) (limited to 'tools') diff --git a/tools/include/uapi/linux/perf_event.h b/tools/include/uapi/linux/perf_event.h index c77c9a2ebbbb..5d49cfc509e7 100644 --- a/tools/include/uapi/linux/perf_event.h +++ b/tools/include/uapi/linux/perf_event.h @@ -380,10 +380,14 @@ struct perf_event_attr { __u32 bp_type; union { __u64 bp_addr; + __u64 kprobe_func; /* for perf_kprobe */ + __u64 uprobe_path; /* for perf_uprobe */ __u64 config1; /* extension of config */ }; union { __u64 bp_len; + __u64 kprobe_addr; /* when kprobe_func == NULL */ + __u64 probe_offset; /* for perf_[k,u]probe */ __u64 config2; /* extension of config1 */ }; __u64 branch_sample_type; /* enum perf_branch_sample_type */ -- cgit v1.2.3 From a7c8655b073d89303911c89d0fd9fc4be7631fbe Mon Sep 17 00:00:00 2001 From: "Paul E. McKenney" Date: Thu, 30 Nov 2017 15:36:35 -0800 Subject: sched/isolation: Eliminate NO_HZ_FULL_ALL Commit 6f1982fedd59 ("sched/isolation: Handle the nohz_full= parameter") broke CONFIG_NO_HZ_FULL_ALL=y kernels. This breakage is due to the code under CONFIG_NO_HZ_FULL_ALL failing to invoke the shiny new housekeeping functions. This means that rcutorture scenario TREE04 now emits RCU CPU stall warnings due to the RCU grace-period kthreads not being awakened at a time of their choosing, or perhaps even not at all: [ 27.731422] rcu_bh kthread starved for 21001 jiffies! g18446744073709551369 c18446744073709551368 f0x0 RCU_GP_WAIT_FQS(3) ->state=0x402 ->cpu=3 [ 27.731423] rcu_bh I14936 9 2 0x80080000 [ 27.731435] Call Trace: [ 27.731440] __schedule+0x31a/0x6d0 [ 27.731442] schedule+0x31/0x80 [ 27.731446] schedule_timeout+0x15a/0x320 [ 27.731453] ? call_timer_fn+0x130/0x130 [ 27.731457] rcu_gp_kthread+0x66c/0xea0 [ 27.731458] ? rcu_gp_kthread+0x66c/0xea0 Because no one has complained about CONFIG_NO_HZ_FULL_ALL=y being broken, I hypothesize that no one is in fact using it, other than rcutorture. This commit therefore eliminates CONFIG_NO_HZ_FULL_ALL and updates rcutorture's config files to instead use the nohz_full= kernel parameter to put the desired CPUs into nohz_full mode. Fixes: 6f1982fedd59 ("sched/isolation: Handle the nohz_full= parameter") Reported-by: kernel test robot Signed-off-by: Paul E. McKenney Cc: Frederic Weisbecker Cc: Thomas Gleixner Cc: Chris Metcalf Cc: Christoph Lameter Cc: Linus Torvalds Cc: Luiz Capitulino Cc: Mike Galbraith Cc: Peter Zijlstra Cc: Rik van Riel Cc: Wanpeng Li Cc: Ingo Molnar Cc: John Stultz Cc: Jonathan Corbet --- Documentation/timers/NO_HZ.txt | 7 ------- kernel/time/Kconfig | 10 ---------- kernel/time/tick-sched.c | 22 ++-------------------- .../selftests/rcutorture/configs/rcu/TASKS03 | 1 - .../selftests/rcutorture/configs/rcu/TASKS03.boot | 2 +- .../selftests/rcutorture/configs/rcu/TREE04 | 1 - .../selftests/rcutorture/configs/rcu/TREE04.boot | 2 +- .../selftests/rcutorture/configs/rcu/TREE07 | 1 - 8 files changed, 4 insertions(+), 42 deletions(-) (limited to 'tools') diff --git a/Documentation/timers/NO_HZ.txt b/Documentation/timers/NO_HZ.txt index 2dcaf9adb7a7..9591092da5e0 100644 --- a/Documentation/timers/NO_HZ.txt +++ b/Documentation/timers/NO_HZ.txt @@ -131,13 +131,6 @@ error message, and the boot CPU will be removed from the mask. Note that this means that your system must have at least two CPUs in order for CONFIG_NO_HZ_FULL=y to do anything for you. -Alternatively, the CONFIG_NO_HZ_FULL_ALL=y Kconfig parameter specifies -that all CPUs other than the boot CPU are adaptive-ticks CPUs. This -Kconfig parameter will be overridden by the "nohz_full=" boot parameter, -so that if both the CONFIG_NO_HZ_FULL_ALL=y Kconfig parameter and -the "nohz_full=1" boot parameter is specified, the boot parameter will -prevail so that only CPU 1 will be an adaptive-ticks CPU. - Finally, adaptive-ticks CPUs must have their RCU callbacks offloaded. This is covered in the "RCU IMPLICATIONS" section below. diff --git a/kernel/time/Kconfig b/kernel/time/Kconfig index f6b5f19223d6..78eabc41eaa6 100644 --- a/kernel/time/Kconfig +++ b/kernel/time/Kconfig @@ -113,16 +113,6 @@ config NO_HZ_FULL endchoice -config NO_HZ_FULL_ALL - bool "Full dynticks system on all CPUs by default (except CPU 0)" - depends on NO_HZ_FULL - help - If the user doesn't pass the nohz_full boot option to - define the range of full dynticks CPUs, consider that all - CPUs in the system are full dynticks by default. - Note the boot CPU will still be kept outside the range to - handle the timekeeping duty. - config NO_HZ bool "Old Idle dynticks config" depends on !ARCH_USES_GETTIMEOFFSET && GENERIC_CLOCKEVENTS diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index 29a5733eff83..ccd3782da0bf 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -405,30 +405,12 @@ static int tick_nohz_cpu_down(unsigned int cpu) return 0; } -static int tick_nohz_init_all(void) -{ - int err = -1; - -#ifdef CONFIG_NO_HZ_FULL_ALL - if (!alloc_cpumask_var(&tick_nohz_full_mask, GFP_KERNEL)) { - WARN(1, "NO_HZ: Can't allocate full dynticks cpumask\n"); - return err; - } - err = 0; - cpumask_setall(tick_nohz_full_mask); - tick_nohz_full_running = true; -#endif - return err; -} - void __init tick_nohz_init(void) { int cpu, ret; - if (!tick_nohz_full_running) { - if (tick_nohz_init_all() < 0) - return; - } + if (!tick_nohz_full_running) + return; /* * Full dynticks uses irq work to drive the tick rescheduling on safe diff --git a/tools/testing/selftests/rcutorture/configs/rcu/TASKS03 b/tools/testing/selftests/rcutorture/configs/rcu/TASKS03 index c70c51d5ded1..28568b72a31b 100644 --- a/tools/testing/selftests/rcutorture/configs/rcu/TASKS03 +++ b/tools/testing/selftests/rcutorture/configs/rcu/TASKS03 @@ -9,5 +9,4 @@ CONFIG_PREEMPT=y CONFIG_HZ_PERIODIC=n CONFIG_NO_HZ_IDLE=n CONFIG_NO_HZ_FULL=y -CONFIG_NO_HZ_FULL_ALL=y #CHECK#CONFIG_RCU_EXPERT=n diff --git a/tools/testing/selftests/rcutorture/configs/rcu/TASKS03.boot b/tools/testing/selftests/rcutorture/configs/rcu/TASKS03.boot index cd2a188eeb6d..838297c58318 100644 --- a/tools/testing/selftests/rcutorture/configs/rcu/TASKS03.boot +++ b/tools/testing/selftests/rcutorture/configs/rcu/TASKS03.boot @@ -1 +1 @@ -rcutorture.torture_type=tasks +rcutorture.torture_type=tasks nohz_full=1 diff --git a/tools/testing/selftests/rcutorture/configs/rcu/TREE04 b/tools/testing/selftests/rcutorture/configs/rcu/TREE04 index 27d22695d64c..24c9f6012e35 100644 --- a/tools/testing/selftests/rcutorture/configs/rcu/TREE04 +++ b/tools/testing/selftests/rcutorture/configs/rcu/TREE04 @@ -7,7 +7,6 @@ CONFIG_PREEMPT=n CONFIG_HZ_PERIODIC=n CONFIG_NO_HZ_IDLE=n CONFIG_NO_HZ_FULL=y -CONFIG_NO_HZ_FULL_ALL=y CONFIG_RCU_FAST_NO_HZ=y CONFIG_RCU_TRACE=y CONFIG_HOTPLUG_CPU=n diff --git a/tools/testing/selftests/rcutorture/configs/rcu/TREE04.boot b/tools/testing/selftests/rcutorture/configs/rcu/TREE04.boot index e34c33430447..e6071bb96c7d 100644 --- a/tools/testing/selftests/rcutorture/configs/rcu/TREE04.boot +++ b/tools/testing/selftests/rcutorture/configs/rcu/TREE04.boot @@ -1 +1 @@ -rcutorture.torture_type=rcu_bh rcutree.rcu_fanout_leaf=4 +rcutorture.torture_type=rcu_bh rcutree.rcu_fanout_leaf=4 nohz_full=1-7 diff --git a/tools/testing/selftests/rcutorture/configs/rcu/TREE07 b/tools/testing/selftests/rcutorture/configs/rcu/TREE07 index 0f4759f4232e..d7afb271a586 100644 --- a/tools/testing/selftests/rcutorture/configs/rcu/TREE07 +++ b/tools/testing/selftests/rcutorture/configs/rcu/TREE07 @@ -7,7 +7,6 @@ CONFIG_PREEMPT=n CONFIG_HZ_PERIODIC=n CONFIG_NO_HZ_IDLE=n CONFIG_NO_HZ_FULL=y -CONFIG_NO_HZ_FULL_ALL=n CONFIG_RCU_FAST_NO_HZ=n CONFIG_RCU_TRACE=y CONFIG_HOTPLUG_CPU=y -- cgit v1.2.3 From c3dec27b7f70a9ad5f777d943d51ecdfcd9824d0 Mon Sep 17 00:00:00 2001 From: Jiri Olsa Date: Tue, 6 Feb 2018 19:17:58 +0100 Subject: perf record: Put new line after target override warning There's no new-line after target-override warning, now: $ perf record -a --per-thread Warning: SYSTEM/CPU switch overriding PER-THREAD^C[ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.705 MB perf.data (2939 samples) ] with patch: $ perf record -a --per-thread Warning: SYSTEM/CPU switch overriding PER-THREAD ^C[ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.705 MB perf.data (2939 samples) ] Signed-off-by: Jiri Olsa Tested-by: Arnaldo Carvalho de Melo Cc: Alexander Shishkin Cc: David Ahern Cc: Namhyung Kim Cc: Peter Zijlstra Fixes: 16ad2ffb822c ("perf tools: Introduce perf_target__strerror()") Link: http://lkml.kernel.org/r/20180206181813.10943-3-jolsa@kernel.org Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/builtin-record.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'tools') diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c index bf4ca749d1ac..907267206973 100644 --- a/tools/perf/builtin-record.c +++ b/tools/perf/builtin-record.c @@ -1803,7 +1803,7 @@ int cmd_record(int argc, const char **argv) err = target__validate(&rec->opts.target); if (err) { target__strerror(&rec->opts.target, err, errbuf, BUFSIZ); - ui__warning("%s", errbuf); + ui__warning("%s\n", errbuf); } err = target__parse_uid(&rec->opts.target); -- cgit v1.2.3 From 3233b37a71c794e25a0a794185df8d6abd9f277e Mon Sep 17 00:00:00 2001 From: Jiri Olsa Date: Tue, 6 Feb 2018 19:17:59 +0100 Subject: perf script: Add --show-round-event to display PERF_RECORD_FINISHED_ROUND Adding --show-round-event to display PERF_RECORD_FINISHED_ROUND events like: # perf script --show-round-events 2>/dev/null yes 8591 [002] 124177.397597: 18 cpu/mem-stores/P: ff... yes 8591 [002] 124177.397615: 1 cpu/mem-loads,ldlat=30/P: ff... PERF_RECORD_FINISHED_ROUND perf 10380 [001] 124177.397622: 6 cpu/mem-loads,ldlat=30/P: ff... PERF_RECORD_FINISHED_ROUND swapper 0 [000] 124177.400518: 88 cpu/mem-stores/P: ff... swapper 0 [000] 124177.400521: 88 cpu/mem-stores/P: ff... Signed-off-by: Jiri Olsa Tested-by: Arnaldo Carvalho de Melo Cc: Alexander Shishkin Cc: David Ahern Cc: Namhyung Kim Cc: Peter Zijlstra Link: http://lkml.kernel.org/r/20180206181813.10943-4-jolsa@kernel.org Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/Documentation/perf-script.txt | 3 +++ tools/perf/builtin-script.c | 17 +++++++++++++++++ 2 files changed, 20 insertions(+) (limited to 'tools') diff --git a/tools/perf/Documentation/perf-script.txt b/tools/perf/Documentation/perf-script.txt index 7730c1d2b5d3..36ec0257f8d3 100644 --- a/tools/perf/Documentation/perf-script.txt +++ b/tools/perf/Documentation/perf-script.txt @@ -303,6 +303,9 @@ OPTIONS --show-lost-events Display lost events i.e. events of type PERF_RECORD_LOST. +--show-round-events + Display finished round events i.e. events of type PERF_RECORD_FINISHED_ROUND. + --demangle:: Demangle symbol names to human readable form. It's enabled by default, disable with --no-demangle. diff --git a/tools/perf/builtin-script.c b/tools/perf/builtin-script.c index ab19a6ee4093..cce926aeb0c0 100644 --- a/tools/perf/builtin-script.c +++ b/tools/perf/builtin-script.c @@ -1489,6 +1489,7 @@ struct perf_script { bool show_switch_events; bool show_namespace_events; bool show_lost_events; + bool show_round_events; bool allocated; bool per_event_dump; struct cpu_map *cpus; @@ -2104,6 +2105,16 @@ process_lost_event(struct perf_tool *tool, return 0; } +static int +process_finished_round_event(struct perf_tool *tool __maybe_unused, + union perf_event *event, + struct ordered_events *oe __maybe_unused) + +{ + perf_event__fprintf(event, stdout); + return 0; +} + static void sig_handler(int sig __maybe_unused) { session_done = 1; @@ -2200,6 +2211,10 @@ static int __cmd_script(struct perf_script *script) script->tool.namespaces = process_namespaces_event; if (script->show_lost_events) script->tool.lost = process_lost_event; + if (script->show_round_events) { + script->tool.ordered_events = false; + script->tool.finished_round = process_finished_round_event; + } if (perf_script__setup_per_event_dump(script)) { pr_err("Couldn't create the per event dump files\n"); @@ -3139,6 +3154,8 @@ int cmd_script(int argc, const char **argv) "Show namespace events (if recorded)"), OPT_BOOLEAN('\0', "show-lost-events", &script.show_lost_events, "Show lost events (if recorded)"), + OPT_BOOLEAN('\0', "show-round-events", &script.show_round_events, + "Show round events (if recorded)"), OPT_BOOLEAN('\0', "per-event-dump", &script.per_event_dump, "Dump trace output to files named by the monitored events"), OPT_BOOLEAN('f', "force", &symbol_conf.force, "don't complain, do it"), -- cgit v1.2.3 From 6baddfc6900eca7f6b360c91ff737890ab4f1d55 Mon Sep 17 00:00:00 2001 From: Jiri Olsa Date: Tue, 6 Feb 2018 19:18:00 +0100 Subject: tools lib api fs: Add filename__read_xll function Adding filename__read_xll function to be able to read files with hex numbers in, which do not have 0x prefix. Signed-off-by: Jiri Olsa Cc: Alexander Shishkin Cc: David Ahern Cc: Namhyung Kim Cc: Peter Zijlstra Link: http://lkml.kernel.org/r/20180206181813.10943-5-jolsa@kernel.org Signed-off-by: Arnaldo Carvalho de Melo --- tools/lib/api/fs/fs.c | 29 ++++++++++++++++++++++------- tools/lib/api/fs/fs.h | 1 + 2 files changed, 23 insertions(+), 7 deletions(-) (limited to 'tools') diff --git a/tools/lib/api/fs/fs.c b/tools/lib/api/fs/fs.c index b24afc0e6e81..8b0e4a4315bd 100644 --- a/tools/lib/api/fs/fs.c +++ b/tools/lib/api/fs/fs.c @@ -315,12 +315,8 @@ int filename__read_int(const char *filename, int *value) return err; } -/* - * Parses @value out of @filename with strtoull. - * By using 0 for base, the strtoull detects the - * base automatically (see man strtoull). - */ -int filename__read_ull(const char *filename, unsigned long long *value) +static int filename__read_ull_base(const char *filename, + unsigned long long *value, int base) { char line[64]; int fd = open(filename, O_RDONLY), err = -1; @@ -329,7 +325,7 @@ int filename__read_ull(const char *filename, unsigned long long *value) return -1; if (read(fd, line, sizeof(line)) > 0) { - *value = strtoull(line, NULL, 0); + *value = strtoull(line, NULL, base); if (*value != ULLONG_MAX) err = 0; } @@ -338,6 +334,25 @@ int filename__read_ull(const char *filename, unsigned long long *value) return err; } +/* + * Parses @value out of @filename with strtoull. + * By using 16 for base to treat the number as hex. + */ +int filename__read_xll(const char *filename, unsigned long long *value) +{ + return filename__read_ull_base(filename, value, 16); +} + +/* + * Parses @value out of @filename with strtoull. + * By using 0 for base, the strtoull detects the + * base automatically (see man strtoull). + */ +int filename__read_ull(const char *filename, unsigned long long *value) +{ + return filename__read_ull_base(filename, value, 0); +} + #define STRERR_BUFSIZE 128 /* For the buffer size of strerror_r */ int filename__read_str(const char *filename, char **buf, size_t *sizep) diff --git a/tools/lib/api/fs/fs.h b/tools/lib/api/fs/fs.h index dda49deefb52..8ebee35a6395 100644 --- a/tools/lib/api/fs/fs.h +++ b/tools/lib/api/fs/fs.h @@ -30,6 +30,7 @@ FS(bpf_fs) int filename__read_int(const char *filename, int *value); int filename__read_ull(const char *filename, unsigned long long *value); +int filename__read_xll(const char *filename, unsigned long long *value); int filename__read_str(const char *filename, char **buf, size_t *sizep); int filename__write_int(const char *filename, int value); -- cgit v1.2.3 From d9c5f32240f503481291a6d4e7246ee0a128d76d Mon Sep 17 00:00:00 2001 From: Jiri Olsa Date: Tue, 6 Feb 2018 19:18:01 +0100 Subject: tools lib api fs: Add sysfs__read_xll function Adding sysfs__read_xll function to be able to read sysfs files with hex numbers in, which do not have 0x prefix. Signed-off-by: Jiri Olsa Cc: Alexander Shishkin Cc: David Ahern Cc: Namhyung Kim Cc: Peter Zijlstra Link: http://lkml.kernel.org/r/20180206181813.10943-6-jolsa@kernel.org Signed-off-by: Arnaldo Carvalho de Melo --- tools/lib/api/fs/fs.c | 15 +++++++++++++-- tools/lib/api/fs/fs.h | 1 + 2 files changed, 14 insertions(+), 2 deletions(-) (limited to 'tools') diff --git a/tools/lib/api/fs/fs.c b/tools/lib/api/fs/fs.c index 8b0e4a4315bd..6a12bbf39f7b 100644 --- a/tools/lib/api/fs/fs.c +++ b/tools/lib/api/fs/fs.c @@ -432,7 +432,8 @@ int procfs__read_str(const char *entry, char **buf, size_t *sizep) return filename__read_str(path, buf, sizep); } -int sysfs__read_ull(const char *entry, unsigned long long *value) +static int sysfs__read_ull_base(const char *entry, + unsigned long long *value, int base) { char path[PATH_MAX]; const char *sysfs = sysfs__mountpoint(); @@ -442,7 +443,17 @@ int sysfs__read_ull(const char *entry, unsigned long long *value) snprintf(path, sizeof(path), "%s/%s", sysfs, entry); - return filename__read_ull(path, value); + return filename__read_ull_base(path, value, base); +} + +int sysfs__read_xll(const char *entry, unsigned long long *value) +{ + return sysfs__read_ull_base(entry, value, 16); +} + +int sysfs__read_ull(const char *entry, unsigned long long *value) +{ + return sysfs__read_ull_base(entry, value, 0); } int sysfs__read_int(const char *entry, int *value) diff --git a/tools/lib/api/fs/fs.h b/tools/lib/api/fs/fs.h index 8ebee35a6395..92d03b8396b1 100644 --- a/tools/lib/api/fs/fs.h +++ b/tools/lib/api/fs/fs.h @@ -40,6 +40,7 @@ int procfs__read_str(const char *entry, char **buf, size_t *sizep); int sysctl__read_int(const char *sysctl, int *value); int sysfs__read_int(const char *entry, int *value); int sysfs__read_ull(const char *entry, unsigned long long *value); +int sysfs__read_xll(const char *entry, unsigned long long *value); int sysfs__read_str(const char *entry, char **buf, size_t *sizep); int sysfs__read_bool(const char *entry, bool *value); -- cgit v1.2.3 From fdf7c49c200d1b9909e2204cec5bd68b48605c71 Mon Sep 17 00:00:00 2001 From: Jiri Olsa Date: Tue, 6 Feb 2018 19:18:12 +0100 Subject: perf tests: Fix dwarf unwind for stripped binaries When we strip the perf binary, dwarf unwind test stop to work. The reason is that strip will remove static function symbols, which we need to check for unwind. This change will keep this test working in cases where the global symbols are put into dynamic symbol table, which is the case on x86. It still won't work on powerpc. Making those 5 local functions global, and adding 'test_dwarf_unwind__' to their names. Committer testing: Before: # perf test dwarf 58: DWARF unwind : Ok # strip ~/bin/perf # perf test dwarf 58: DWARF unwind : FAILED! # perf test -v dwarf 58: DWARF unwind : --- start --- test child forked, pid 6590 unwind: thread map already set, dso=/home/acme/bin/perf unwind: access_mem addr 0x7ffce6c48098 val 48563f, offset 1144 unwind: test__dwarf_unwind:ip = 0x4a54e5 (0xa54e5) got: test__dwarf_unwind 0xa54e5, expecting test__dwarf_unwind unwind: '':ip = 0x4a50bb (0xa50bb) failed: got unresolved address 0xa50bb unwind failed test child finished with -1 ---- end ---- DWARF unwind: FAILED! # After: # perf test dwarf 58: DWARF unwind : Ok # strip ~/bin/perf # perf test dwarf 58: DWARF unwind : Ok # # perf test -v dwarf 58: DWARF unwind : --- start --- test child forked, pid 7219 unwind: thread map already set, dso=/home/acme/bin/perf unwind: access_mem addr 0x7fff007da2c8 val 48575f, offset 1144 unwind: test__arch_unwind_sample:ip = 0x589044 (0x189044) got: test__arch_unwind_sample 0x189044, expecting test__arch_unwind_sample unwind: test_dwarf_unwind__thread:ip = 0x4a52f7 (0xa52f7) got: test_dwarf_unwind__thread 0xa52f7, expecting test_dwarf_unwind__thread unwind: test_dwarf_unwind__compare:ip = 0x4a5468 (0xa5468) got: test_dwarf_unwind__compare 0xa5468, expecting test_dwarf_unwind__compare unwind: bsearch:ip = 0x7f6608ae94d8 (0x394d8) got: bsearch 0x394d8, expecting bsearch unwind: test_dwarf_unwind__krava_3:ip = 0x4a54d1 (0xa54d1) got: test_dwarf_unwind__krava_3 0xa54d1, expecting test_dwarf_unwind__krava_3 unwind: test_dwarf_unwind__krava_2:ip = 0x4a550b (0xa550b) got: test_dwarf_unwind__krava_2 0xa550b, expecting test_dwarf_unwind__krava_2 unwind: test_dwarf_unwind__krava_1:ip = 0x4a554b (0xa554b) got: test_dwarf_unwind__krava_1 0xa554b, expecting test_dwarf_unwind__krava_1 unwind: test__dwarf_unwind:ip = 0x4a5605 (0xa5605) got: test__dwarf_unwind 0xa5605, expecting test__dwarf_unwind test child finished with 0 ---- end ---- DWARF unwind: Ok # Signed-off-by: Jiri Olsa Tested-by: Arnaldo Carvalho de Melo Cc: Alexander Shishkin Cc: David Ahern Cc: Namhyung Kim Cc: Peter Zijlstra Link: http://lkml.kernel.org/r/20180206181813.10943-17-jolsa@kernel.org Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/tests/dwarf-unwind.c | 46 +++++++++++++++++++++++++++-------------- 1 file changed, 30 insertions(+), 16 deletions(-) (limited to 'tools') diff --git a/tools/perf/tests/dwarf-unwind.c b/tools/perf/tests/dwarf-unwind.c index 260418969120..2f008067d989 100644 --- a/tools/perf/tests/dwarf-unwind.c +++ b/tools/perf/tests/dwarf-unwind.c @@ -37,6 +37,19 @@ static int init_live_machine(struct machine *machine) mmap_handler, machine, true, 500); } +/* + * We need to keep these functions global, despite the + * fact that they are used only locally in this object, + * in order to keep them around even if the binary is + * stripped. If they are gone, the unwind check for + * symbol fails. + */ +int test_dwarf_unwind__thread(struct thread *thread); +int test_dwarf_unwind__compare(void *p1, void *p2); +int test_dwarf_unwind__krava_3(struct thread *thread); +int test_dwarf_unwind__krava_2(struct thread *thread); +int test_dwarf_unwind__krava_1(struct thread *thread); + #define MAX_STACK 8 static int unwind_entry(struct unwind_entry *entry, void *arg) @@ -45,12 +58,12 @@ static int unwind_entry(struct unwind_entry *entry, void *arg) char *symbol = entry->sym ? entry->sym->name : NULL; static const char *funcs[MAX_STACK] = { "test__arch_unwind_sample", - "unwind_thread", - "compare", + "test_dwarf_unwind__thread", + "test_dwarf_unwind__compare", "bsearch", - "krava_3", - "krava_2", - "krava_1", + "test_dwarf_unwind__krava_3", + "test_dwarf_unwind__krava_2", + "test_dwarf_unwind__krava_1", "test__dwarf_unwind" }; /* @@ -77,7 +90,7 @@ static int unwind_entry(struct unwind_entry *entry, void *arg) return strcmp((const char *) symbol, funcs[idx]); } -static noinline int unwind_thread(struct thread *thread) +noinline int test_dwarf_unwind__thread(struct thread *thread) { struct perf_sample sample; unsigned long cnt = 0; @@ -108,7 +121,7 @@ static noinline int unwind_thread(struct thread *thread) static int global_unwind_retval = -INT_MAX; -static noinline int compare(void *p1, void *p2) +noinline int test_dwarf_unwind__compare(void *p1, void *p2) { /* Any possible value should be 'thread' */ struct thread *thread = *(struct thread **)p1; @@ -117,17 +130,17 @@ static noinline int compare(void *p1, void *p2) /* Call unwinder twice for both callchain orders. */ callchain_param.order = ORDER_CALLER; - global_unwind_retval = unwind_thread(thread); + global_unwind_retval = test_dwarf_unwind__thread(thread); if (!global_unwind_retval) { callchain_param.order = ORDER_CALLEE; - global_unwind_retval = unwind_thread(thread); + global_unwind_retval = test_dwarf_unwind__thread(thread); } } return p1 - p2; } -static noinline int krava_3(struct thread *thread) +noinline int test_dwarf_unwind__krava_3(struct thread *thread) { struct thread *array[2] = {thread, thread}; void *fp = &bsearch; @@ -141,18 +154,19 @@ static noinline int krava_3(struct thread *thread) size_t, int (*)(void *, void *)); _bsearch = fp; - _bsearch(array, &thread, 2, sizeof(struct thread **), compare); + _bsearch(array, &thread, 2, sizeof(struct thread **), + test_dwarf_unwind__compare); return global_unwind_retval; } -static noinline int krava_2(struct thread *thread) +noinline int test_dwarf_unwind__krava_2(struct thread *thread) { - return krava_3(thread); + return test_dwarf_unwind__krava_3(thread); } -static noinline int krava_1(struct thread *thread) +noinline int test_dwarf_unwind__krava_1(struct thread *thread) { - return krava_2(thread); + return test_dwarf_unwind__krava_2(thread); } int test__dwarf_unwind(struct test *test __maybe_unused, int subtest __maybe_unused) @@ -189,7 +203,7 @@ int test__dwarf_unwind(struct test *test __maybe_unused, int subtest __maybe_unu goto out; } - err = krava_1(thread); + err = test_dwarf_unwind__krava_1(thread); thread__put(thread); out: -- cgit v1.2.3 From a7402c943bb4657cc0b44453177803fbead70990 Mon Sep 17 00:00:00 2001 From: Jiri Olsa Date: Tue, 6 Feb 2018 19:18:13 +0100 Subject: perf tools: Fix comment for sort__* compare functions In commit 2f15bd8c6c6e ("perf tools: Fix "Command" sort_entry's cmp and collapse function") we switched from pointer to string comparison. But failed to remove related comments. Removing them and adding another one to warn before pointer comparison in here. Signed-off-by: Jiri Olsa Cc: Alexander Shishkin Cc: David Ahern Cc: Namhyung Kim Cc: Peter Zijlstra Link: http://lkml.kernel.org/r/20180206181813.10943-18-jolsa@kernel.org Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/util/sort.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) (limited to 'tools') diff --git a/tools/perf/util/sort.c b/tools/perf/util/sort.c index 2da4d0456a03..e8514f651865 100644 --- a/tools/perf/util/sort.c +++ b/tools/perf/util/sort.c @@ -111,17 +111,20 @@ struct sort_entry sort_thread = { /* --sort comm */ +/* + * We can't use pointer comparison in functions below, + * because it gives different results based on pointer + * values, which could break some sorting assumptions. + */ static int64_t sort__comm_cmp(struct hist_entry *left, struct hist_entry *right) { - /* Compare the addr that should be unique among comm */ return strcmp(comm__str(right->comm), comm__str(left->comm)); } static int64_t sort__comm_collapse(struct hist_entry *left, struct hist_entry *right) { - /* Compare the addr that should be unique among comm */ return strcmp(comm__str(right->comm), comm__str(left->comm)); } -- cgit v1.2.3 From 8614ada0be7d7be84b85c006d526a9c8f76484fa Mon Sep 17 00:00:00 2001 From: Jiri Olsa Date: Tue, 6 Feb 2018 19:17:57 +0100 Subject: perf report: Ask for ordered events for --tasks option If we have the time in, keep the events in time order. Committer notes: Trying to be more verbose, what actual effect this will have in this particular case? Before and after this patch shows the artifacts: --- /tmp/before 2018-02-06 15:40:29.536411625 -0300 +++ /tmp/after 2018-02-06 15:40:51.963403599 -0300 @@ -5,34 +5,34 @@ 2540 2540 1818 | gnome-terminal- 3489 3489 2540 | bash 32433 32433 3489 | perf - 32434 32434 32433 | perf + 32434 32434 32433 | make 32441 32441 32434 | make 32514 32514 32441 | make 511 511 32514 | sh - 512 512 511 | sh + 512 512 511 | install We don't have 'perf' calling 'perf' calling 'make', etc, the second 'perf' actually is 'make', i.e. there was reordering of the relevant PERF_RECORD_COMM and PERF_RECORD_FORK records. Ditto for sh/install later on. Look for FORK and COMM meta events, for those tids: # perf report -D | egrep 'PERF_RECORD_(FORK|COMM)' | egrep '3243[34]' 0 14774650990679 0x1a3cd8 [0x38]: PERF_RECORD_FORK(32433:32433):(3489:3489) 1 14774652080381 0x1d6568 [0x30]: PERF_RECORD_COMM exec: perf:32433/32433 1 14774742473340 0x1dbb48 [0x38]: PERF_RECORD_FORK(32434:32434):(32433:32433) 0 14774752005779 0x1a4af8 [0x30]: PERF_RECORD_COMM exec: make:32434/32434 0 14774753997960 0x1a5578 [0x38]: PERF_RECORD_FORK(32435:32435):(32434:32434) 0 14774756070782 0x1a5618 [0x38]: PERF_RECORD_FORK(32438:32438):(32434:32434) 0 14774757772939 0x1a5680 [0x38]: PERF_RECORD_FORK(32440:32440):(32434:32434) 0 14774758230600 0x1a56e8 [0x38]: PERF_RECORD_FORK(32441:32441):(32434:32434) # First column is the cpu, second is the timestamp. So they are on different CPUs, thus ring buffers, and when we don't use the ordered_events class, we end up mixing that up, use it to take advantage of the PERF_RECORD_FINISHED_ROUND meta events to go on ordering the events using the PERF_SAMPLE_TIME present in the PERF_RECORD_{FORK,COMM,EXIT,SAMPLE,etc} records in the ring buffer. Signed-off-by: Jiri Olsa Tested-by: Arnaldo Carvalho de Melo Cc: Alexander Shishkin Cc: David Ahern Cc: Namhyung Kim Cc: Peter Zijlstra Link: http://lkml.kernel.org/r/20180206181813.10943-2-jolsa@kernel.org Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/builtin-report.c | 1 + 1 file changed, 1 insertion(+) (limited to 'tools') diff --git a/tools/perf/builtin-report.c b/tools/perf/builtin-report.c index 4ad5dc649716..8ef71669e7a0 100644 --- a/tools/perf/builtin-report.c +++ b/tools/perf/builtin-report.c @@ -614,6 +614,7 @@ static int stats_print(struct report *rep) static void tasks_setup(struct report *rep) { memset(&rep->tool, 0, sizeof(rep->tool)); + rep->tool.ordered_events = true; if (rep->mmaps_mode) { rep->tool.mmap = perf_event__process_mmap; rep->tool.mmap2 = perf_event__process_mmap2; -- cgit v1.2.3 From ad52b8cb4886f572b147b02f4c59a648bbf05f9c Mon Sep 17 00:00:00 2001 From: Jiri Olsa Date: Fri, 9 Feb 2018 10:27:34 +0100 Subject: perf report: Add support to display group output for non group events Add support to display group output for if non grouped events are detected and user forces --group option. Now for non-group events recorded like: $ perf record -e 'cycles,instructions' ls you can still get group output by using --group option in report: $ perf report --group --stdio ... # Overhead Command Shared Object Symbol # ................ ....... ................ ...................... # 17.67% 0.00% ls libc-2.25.so [.] _IO_do_write@@GLIB 15.59% 25.94% ls ls [.] calculate_columns 15.41% 31.35% ls libc-2.25.so [.] __strcoll_l ... Committer note: We should improve on this by making sure that the first line states that this is not a group, but since the user doesn't have to force group view when really using grouped events (e.g. '{cycles,instructions}'), the user better know what is being done... Requested-by: Stephane Eranian Signed-off-by: Jiri Olsa Tested-by: Arnaldo Carvalho de Melo Tested-by: Stephane Eranian Cc: Alexander Shishkin Cc: Andi Kleen Cc: David Ahern Cc: Namhyung Kim Cc: Peter Zijlstra Link: http://lkml.kernel.org/r/20180209092734.GB20449@krava Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/Documentation/perf-report.txt | 3 ++- tools/perf/builtin-report.c | 6 +++++- 2 files changed, 7 insertions(+), 2 deletions(-) (limited to 'tools') diff --git a/tools/perf/Documentation/perf-report.txt b/tools/perf/Documentation/perf-report.txt index 907e505b6309..a76b871f78a6 100644 --- a/tools/perf/Documentation/perf-report.txt +++ b/tools/perf/Documentation/perf-report.txt @@ -354,7 +354,8 @@ OPTIONS Path to objdump binary. --group:: - Show event group information together. + Show event group information together. It forces group output also + if there are no groups defined in data file. --demangle:: Demangle symbol names to human readable form. It's enabled by default, diff --git a/tools/perf/builtin-report.c b/tools/perf/builtin-report.c index 8ef71669e7a0..1eedb1815c4c 100644 --- a/tools/perf/builtin-report.c +++ b/tools/perf/builtin-report.c @@ -938,6 +938,7 @@ int cmd_report(int argc, const char **argv) "perf report []", NULL }; + bool group_set = false; struct report report = { .tool = { .sample = process_sample_event, @@ -1057,7 +1058,7 @@ int cmd_report(int argc, const char **argv) "Specify disassembler style (e.g. -M intel for intel syntax)"), OPT_BOOLEAN(0, "show-total-period", &symbol_conf.show_total_period, "Show a column with the sum of periods"), - OPT_BOOLEAN(0, "group", &symbol_conf.event_group, + OPT_BOOLEAN_SET(0, "group", &symbol_conf.event_group, &group_set, "Show event group information together"), OPT_CALLBACK_NOOPT('b', "branch-stack", &branch_mode, "", "use branch records for per branch histogram filling", @@ -1174,6 +1175,9 @@ repeat: has_br_stack = perf_header__has_feat(&session->header, HEADER_BRANCH_STACK); + if (group_set && !session->evlist->nr_groups) + perf_evlist__set_leader(session->evlist); + if (itrace_synth_opts.last_branch) has_br_stack = true; -- cgit v1.2.3 From db06a269ecbb1d71d534fc5713624eeeee0b8f92 Mon Sep 17 00:00:00 2001 From: yuzhoujian Date: Mon, 29 Jan 2018 10:25:22 +0100 Subject: perf stat: Add support to print counts for fixed times Introduce a new option to print counts for fixed number of times and update 'perf stat' documentation accordingly. Show below is the output of the new option for perf stat. $ perf stat -I 1000 --interval-count 2 -e cycles -a # time counts unit events 1.002827089 93,884,870 cycles 2.004231506 56,573,446 cycles We can just print the counts for several times with this newly introduced option. The usage of it is a little like 'vmstat', and it should be used together with "-I" option. $ vmstat -n 1 2 procs ---------memory-------------- --swap- ----io-- -system-- ------cpu--- r b swpd free buff cache si so bi bo in cs us sy id wa st 0 0 0 78270544 547484 51732076 0 0 0 20 1 1 1 0 99 0 0 0 0 0 78270512 547484 51732080 0 0 0 16 477 1555 0 0 100 0 0 Changes since v3: - merge interval_count check and times check to one line. - fix the wrong indent in stat.h - use stat_config.times instead of 'times' in cmd_stat function. Changes since v2: - none. Changes since v1: - change the name of the new option "times-print" to "interval-count". - keep the new option interval specifically. Signed-off-by: yuzhoujian Acked-by: Jiri Olsa Tested-by: Arnaldo Carvalho de Melo Cc: Adrian Hunter Cc: Alexander Shishkin Cc: David Ahern Cc: Kan Liang Cc: Milian Wolff Cc: Namhyung Kim Cc: Peter Zijlstra Cc: Wang Nan Link: http://lkml.kernel.org/r/1517217923-8302-2-git-send-email-ufo19890607@gmail.com Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/Documentation/perf-stat.txt | 5 +++++ tools/perf/builtin-stat.c | 20 +++++++++++++++++++- tools/perf/util/stat.h | 1 + 3 files changed, 25 insertions(+), 1 deletion(-) (limited to 'tools') diff --git a/tools/perf/Documentation/perf-stat.txt b/tools/perf/Documentation/perf-stat.txt index 823fce7674bb..47a21645f60c 100644 --- a/tools/perf/Documentation/perf-stat.txt +++ b/tools/perf/Documentation/perf-stat.txt @@ -146,6 +146,11 @@ Print count deltas every N milliseconds (minimum: 10ms) The overhead percentage could be high in some cases, for instance with small, sub 100ms intervals. Use with caution. example: 'perf stat -I 1000 -e cycles -a sleep 5' +--interval-count times:: +Print count deltas for fixed number of times. +This option should be used together with "-I" option. + example: 'perf stat -I 1000 --interval-count 2 -e cycles -a' + --metric-only:: Only print computed metrics. Print them in a single line. Don't show any raw values. Not supported with --per-thread. diff --git a/tools/perf/builtin-stat.c b/tools/perf/builtin-stat.c index 98bf9d32f222..7d1d7613bf56 100644 --- a/tools/perf/builtin-stat.c +++ b/tools/perf/builtin-stat.c @@ -168,6 +168,7 @@ static struct timespec ref_time; static struct cpu_map *aggr_map; static aggr_get_id_t aggr_get_id; static bool append_file; +static bool interval_count; static const char *output_name; static int output_fd; static int print_free_counters_hint; @@ -571,6 +572,7 @@ static struct perf_evsel *perf_evsel__reset_weak_group(struct perf_evsel *evsel) static int __run_perf_stat(int argc, const char **argv) { int interval = stat_config.interval; + int times = stat_config.times; char msg[BUFSIZ]; unsigned long long t0, t1; struct perf_evsel *counter; @@ -700,6 +702,8 @@ try_again: while (!waitpid(child_pid, &status, WNOHANG)) { nanosleep(&ts, NULL); process_interval(); + if (interval_count && !(--times)) + break; } } waitpid(child_pid, &status, 0); @@ -716,8 +720,11 @@ try_again: enable_counters(); while (!done) { nanosleep(&ts, NULL); - if (interval) + if (interval) { process_interval(); + if (interval_count && !(--times)) + break; + } } } @@ -1891,6 +1898,8 @@ static const struct option stat_options[] = { "command to run after to the measured command"), OPT_UINTEGER('I', "interval-print", &stat_config.interval, "print counts at regular interval in ms (>= 10)"), + OPT_INTEGER(0, "interval-count", &stat_config.times, + "print counts for fixed number of times"), OPT_SET_UINT(0, "per-socket", &stat_config.aggr_mode, "aggregate counts per processor socket", AGGR_SOCKET), OPT_SET_UINT(0, "per-core", &stat_config.aggr_mode, @@ -2870,6 +2879,15 @@ int cmd_stat(int argc, const char **argv) "The overhead percentage could be high in some cases. " "Please proceed with caution.\n"); } + if (stat_config.times && interval) + interval_count = true; + else if (stat_config.times && !interval) { + pr_err("interval-count option should be used together with " + "interval-print.\n"); + parse_options_usage(stat_usage, stat_options, "interval-count", 0); + parse_options_usage(stat_usage, stat_options, "I", 1); + goto out; + } if (perf_evlist__alloc_stats(evsel_list, interval)) goto out; diff --git a/tools/perf/util/stat.h b/tools/perf/util/stat.h index dbc6f7134f61..540fbb350e53 100644 --- a/tools/perf/util/stat.h +++ b/tools/perf/util/stat.h @@ -90,6 +90,7 @@ struct perf_stat_config { bool scale; FILE *output; unsigned int interval; + int times; struct runtime_stat *stats; int stats_num; }; -- cgit v1.2.3 From f1f8ad52f8bf1239282737a2a5c3bd450300cc78 Mon Sep 17 00:00:00 2001 From: yuzhoujian Date: Mon, 29 Jan 2018 10:25:23 +0100 Subject: perf stat: Add support to print counts after a period of time Introduce a new option to print counts after N milliseconds and update 'perf stat' documentation accordingly. Show below is the output of the new option for perf stat. $ perf stat --time 2000 -e cycles -a Performance counter stats for 'system wide': 157,260,423 cycles 2.003060766 seconds time elapsed We can print the count deltas after N milliseconds with this new introduced option. This option is not supported with "-I" option. In addition, according to Kangliang's patch(19afd10410957), the monitoring overhead for system-wide core event could be very high if the interval-print parameter was below 100ms, and the limitation value is 10ms. So the same warning will be displayed when the time is set between 10ms to 100ms, and the minimal time is limited to 10ms. Users can make a decision according to their spcific cases. Committer notes: This actually stops the workload after the specified time, then prints the counts. So I renamed the option to --timeout and updated the documentation to state that it will not just print the counts after the specified time, but will really stop the 'perf stat' session and print the counts. The rename from 'time' to 'timeout' also fixes the build in systems where 'time' is used by glibc and can't be used as a name of a variable, such as centos:5 and centos:6. Changes since v3: - none. Changes since v2: - modify the time check in __run_perf_stat func to keep some consistency with the workload case. - add the warning when the time is set between 10ms to 100ms. - add the pr_err when the time is set below 10ms. Changes since v1: - none. Signed-off-by: yuzhoujian Acked-by: Jiri Olsa Cc: Adrian Hunter Cc: Alexander Shishkin Cc: David Ahern Cc: Kan Liang Cc: Milian Wolff Cc: Namhyung Kim Cc: Peter Zijlstra Cc: Wang Nan Link: http://lkml.kernel.org/r/1517217923-8302-3-git-send-email-ufo19890607@gmail.com Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/Documentation/perf-stat.txt | 5 +++++ tools/perf/builtin-stat.c | 33 +++++++++++++++++++++++++++++++-- tools/perf/util/stat.h | 1 + 3 files changed, 37 insertions(+), 2 deletions(-) (limited to 'tools') diff --git a/tools/perf/Documentation/perf-stat.txt b/tools/perf/Documentation/perf-stat.txt index 47a21645f60c..2bbe79a50d3c 100644 --- a/tools/perf/Documentation/perf-stat.txt +++ b/tools/perf/Documentation/perf-stat.txt @@ -151,6 +151,11 @@ Print count deltas for fixed number of times. This option should be used together with "-I" option. example: 'perf stat -I 1000 --interval-count 2 -e cycles -a' +--timeout msecs:: +Stop the 'perf stat' session and print count deltas after N milliseconds (minimum: 10 ms). +This option is not supported with the "-I" option. + example: 'perf stat --time 2000 -e cycles -a' + --metric-only:: Only print computed metrics. Print them in a single line. Don't show any raw values. Not supported with --per-thread. diff --git a/tools/perf/builtin-stat.c b/tools/perf/builtin-stat.c index 7d1d7613bf56..2d49eccf98f2 100644 --- a/tools/perf/builtin-stat.c +++ b/tools/perf/builtin-stat.c @@ -573,6 +573,7 @@ static int __run_perf_stat(int argc, const char **argv) { int interval = stat_config.interval; int times = stat_config.times; + int timeout = stat_config.timeout; char msg[BUFSIZ]; unsigned long long t0, t1; struct perf_evsel *counter; @@ -586,6 +587,9 @@ static int __run_perf_stat(int argc, const char **argv) if (interval) { ts.tv_sec = interval / USEC_PER_MSEC; ts.tv_nsec = (interval % USEC_PER_MSEC) * NSEC_PER_MSEC; + } else if (timeout) { + ts.tv_sec = timeout / USEC_PER_MSEC; + ts.tv_nsec = (timeout % USEC_PER_MSEC) * NSEC_PER_MSEC; } else { ts.tv_sec = 1; ts.tv_nsec = 0; @@ -698,9 +702,11 @@ try_again: perf_evlist__start_workload(evsel_list); enable_counters(); - if (interval) { + if (interval || timeout) { while (!waitpid(child_pid, &status, WNOHANG)) { nanosleep(&ts, NULL); + if (timeout) + break; process_interval(); if (interval_count && !(--times)) break; @@ -720,6 +726,8 @@ try_again: enable_counters(); while (!done) { nanosleep(&ts, NULL); + if (timeout) + break; if (interval) { process_interval(); if (interval_count && !(--times)) @@ -1900,6 +1908,8 @@ static const struct option stat_options[] = { "print counts at regular interval in ms (>= 10)"), OPT_INTEGER(0, "interval-count", &stat_config.times, "print counts for fixed number of times"), + OPT_UINTEGER(0, "timeout", &stat_config.timeout, + "stop workload and print counts after a timeout period in ms (>= 10ms)"), OPT_SET_UINT(0, "per-socket", &stat_config.aggr_mode, "aggregate counts per processor socket", AGGR_SOCKET), OPT_SET_UINT(0, "per-core", &stat_config.aggr_mode, @@ -2697,7 +2707,7 @@ int cmd_stat(int argc, const char **argv) int status = -EINVAL, run_idx; const char *mode; FILE *output = stderr; - unsigned int interval; + unsigned int interval, timeout; const char * const stat_subcommands[] = { "record", "report" }; setlocale(LC_ALL, ""); @@ -2728,6 +2738,7 @@ int cmd_stat(int argc, const char **argv) return __cmd_report(argc, argv); interval = stat_config.interval; + timeout = stat_config.timeout; /* * For record command the -o is already taken care of. @@ -2879,6 +2890,7 @@ int cmd_stat(int argc, const char **argv) "The overhead percentage could be high in some cases. " "Please proceed with caution.\n"); } + if (stat_config.times && interval) interval_count = true; else if (stat_config.times && !interval) { @@ -2889,6 +2901,23 @@ int cmd_stat(int argc, const char **argv) goto out; } + if (timeout && timeout < 100) { + if (timeout < 10) { + pr_err("timeout must be >= 10ms.\n"); + parse_options_usage(stat_usage, stat_options, "timeout", 0); + goto out; + } else + pr_warning("timeout < 100ms. " + "The overhead percentage could be high in some cases. " + "Please proceed with caution.\n"); + } + if (timeout && interval) { + pr_err("timeout option is not supported with interval-print.\n"); + parse_options_usage(stat_usage, stat_options, "timeout", 0); + parse_options_usage(stat_usage, stat_options, "I", 1); + goto out; + } + if (perf_evlist__alloc_stats(evsel_list, interval)) goto out; diff --git a/tools/perf/util/stat.h b/tools/perf/util/stat.h index 540fbb350e53..2f44e386a0e8 100644 --- a/tools/perf/util/stat.h +++ b/tools/perf/util/stat.h @@ -90,6 +90,7 @@ struct perf_stat_config { bool scale; FILE *output; unsigned int interval; + unsigned int timeout; int times; struct runtime_stat *stats; int stats_num; -- cgit v1.2.3 From c53b4bb02b45ceec7a590e47820afbb5cef0bb81 Mon Sep 17 00:00:00 2001 From: Jiri Olsa Date: Thu, 15 Feb 2018 13:26:27 +0100 Subject: tools lib symbol: Skip non-address kallsyms line Adding check on failed attempt to parse the address and skip the line parsing early in that case. The address can be replaced with '(null)' string in case user don't have enough permissions, like: $ cat /proc/kallsyms (null) A irq_stack_union (null) A __per_cpu_start ... Signed-off-by: Jiri Olsa Cc: Alexander Shishkin Cc: David Ahern Cc: Namhyung Kim Cc: Peter Zijlstra Link: http://lkml.kernel.org/r/20180215122635.24029-2-jolsa@kernel.org Signed-off-by: Arnaldo Carvalho de Melo --- tools/lib/symbol/kallsyms.c | 4 ++++ 1 file changed, 4 insertions(+) (limited to 'tools') diff --git a/tools/lib/symbol/kallsyms.c b/tools/lib/symbol/kallsyms.c index 914cb8e3d40b..689b6a130dd7 100644 --- a/tools/lib/symbol/kallsyms.c +++ b/tools/lib/symbol/kallsyms.c @@ -38,6 +38,10 @@ int kallsyms__parse(const char *filename, void *arg, len = hex2u64(line, &start); + /* Skip the line if we failed to parse the address. */ + if (!len) + continue; + len++; if (len + 2 >= line_len) continue; -- cgit v1.2.3 From c39629614640a7a5331bf156b0d26effade0a67f Mon Sep 17 00:00:00 2001 From: Jiri Olsa Date: Thu, 15 Feb 2018 13:26:28 +0100 Subject: perf symbols: Check if we read regular file in dso__load() The current code in dso__load() calls is_regular_file(), but it checks its return value only after calling symsrc__init(). That can make symsrc__init() block in elf_* functions on reading the file if the file happens to be device and not regular one. Call symsrc__init() only for regular files. Also remove the symsrc__destroy() cleanup, which is not needed now, because we call symsrc__init() only for regular files. Signed-off-by: Jiri Olsa Cc: Alexander Shishkin Cc: David Ahern Cc: Namhyung Kim Cc: Peter Zijlstra Link: http://lkml.kernel.org/r/20180215122635.24029-3-jolsa@kernel.org Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/util/symbol.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) (limited to 'tools') diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c index cc065d4bfafc..e366e3060e6b 100644 --- a/tools/perf/util/symbol.c +++ b/tools/perf/util/symbol.c @@ -1582,7 +1582,7 @@ int dso__load(struct dso *dso, struct map *map) bool next_slot = false; bool is_reg; bool nsexit; - int sirc; + int sirc = -1; enum dso_binary_type symtab_type = binary_type_symtab[i]; @@ -1600,16 +1600,14 @@ int dso__load(struct dso *dso, struct map *map) nsinfo__mountns_exit(&nsc); is_reg = is_regular_file(name); - sirc = symsrc__init(ss, dso, name, symtab_type); + if (is_reg) + sirc = symsrc__init(ss, dso, name, symtab_type); if (nsexit) nsinfo__mountns_enter(dso->nsinfo, &nsc); - if (!is_reg || sirc < 0) { - if (sirc >= 0) - symsrc__destroy(ss); + if (!is_reg || sirc < 0) continue; - } if (!syms_ss && symsrc__has_symtab(ss)) { syms_ss = ss; -- cgit v1.2.3 From 81f981d7ec43ed93901c12b6521d39b06f1ed3d3 Mon Sep 17 00:00:00 2001 From: Jiri Olsa Date: Thu, 15 Feb 2018 13:26:29 +0100 Subject: perf machine: Free root_dir in machine__init() error path Free root_dir in machine__init() error path. Signed-off-by: Jiri Olsa Cc: Alexander Shishkin Cc: David Ahern Cc: Namhyung Kim Cc: Peter Zijlstra Link: http://lkml.kernel.org/r/20180215122635.24029-4-jolsa@kernel.org Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/util/machine.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) (limited to 'tools') diff --git a/tools/perf/util/machine.c b/tools/perf/util/machine.c index b05a67464c03..c976384f9022 100644 --- a/tools/perf/util/machine.c +++ b/tools/perf/util/machine.c @@ -50,6 +50,8 @@ static void machine__threads_init(struct machine *machine) int machine__init(struct machine *machine, const char *root_dir, pid_t pid) { + int err = -ENOMEM; + memset(machine, 0, sizeof(*machine)); map_groups__init(&machine->kmaps, machine); RB_CLEAR_NODE(&machine->rb_node); @@ -79,7 +81,7 @@ int machine__init(struct machine *machine, const char *root_dir, pid_t pid) char comm[64]; if (thread == NULL) - return -ENOMEM; + goto out; snprintf(comm, sizeof(comm), "[guest/%d]", pid); thread__set_comm(thread, comm, 0); @@ -87,7 +89,11 @@ int machine__init(struct machine *machine, const char *root_dir, pid_t pid) } machine->current_tid = NULL; + err = 0; +out: + if (err) + zfree(&machine->root_dir); return 0; } -- cgit v1.2.3 From 8c7f1bb37b29f140e08175132f3abb4d5ad229fc Mon Sep 17 00:00:00 2001 From: Jiri Olsa Date: Thu, 15 Feb 2018 13:26:30 +0100 Subject: perf machine: Move kernel mmap name into struct machine It simplifies and centralizes the code. The kernel mmap name is set for machine type, which we know from the beginning, so there's no reason to generate it every time we need it. Signed-off-by: Jiri Olsa Cc: Alexander Shishkin Cc: David Ahern Cc: Namhyung Kim Cc: Peter Zijlstra Link: http://lkml.kernel.org/r/20180215122635.24029-5-jolsa@kernel.org Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/util/build-id.c | 10 +++---- tools/perf/util/event.c | 5 +--- tools/perf/util/machine.c | 67 +++++++++++++++++++++++----------------------- tools/perf/util/machine.h | 3 +-- tools/perf/util/symbol.c | 3 +-- 5 files changed, 39 insertions(+), 49 deletions(-) (limited to 'tools') diff --git a/tools/perf/util/build-id.c b/tools/perf/util/build-id.c index 7f8553630c4d..537eadd81914 100644 --- a/tools/perf/util/build-id.c +++ b/tools/perf/util/build-id.c @@ -316,7 +316,6 @@ static int machine__write_buildid_table(struct machine *machine, struct feat_fd *fd) { int err = 0; - char nm[PATH_MAX]; struct dso *pos; u16 kmisc = PERF_RECORD_MISC_KERNEL, umisc = PERF_RECORD_MISC_USER; @@ -338,9 +337,8 @@ static int machine__write_buildid_table(struct machine *machine, name = pos->short_name; name_len = pos->short_name_len; } else if (dso__is_kcore(pos)) { - machine__mmap_name(machine, nm, sizeof(nm)); - name = nm; - name_len = strlen(nm); + name = machine->mmap_name; + name_len = strlen(name); } else { name = pos->long_name; name_len = pos->long_name_len; @@ -813,12 +811,10 @@ static int dso__cache_build_id(struct dso *dso, struct machine *machine) bool is_kallsyms = dso__is_kallsyms(dso); bool is_vdso = dso__is_vdso(dso); const char *name = dso->long_name; - char nm[PATH_MAX]; if (dso__is_kcore(dso)) { is_kallsyms = true; - machine__mmap_name(machine, nm, sizeof(nm)); - name = nm; + name = machine->mmap_name; } return build_id_cache__add_b(dso->build_id, sizeof(dso->build_id), name, dso->nsinfo, is_kallsyms, is_vdso); diff --git a/tools/perf/util/event.c b/tools/perf/util/event.c index 44e603c27944..4644e751a3e3 100644 --- a/tools/perf/util/event.c +++ b/tools/perf/util/event.c @@ -894,8 +894,6 @@ int perf_event__synthesize_kernel_mmap(struct perf_tool *tool, struct machine *machine) { size_t size; - const char *mmap_name; - char name_buff[PATH_MAX]; struct map *map = machine__kernel_map(machine); struct kmap *kmap; int err; @@ -918,7 +916,6 @@ int perf_event__synthesize_kernel_mmap(struct perf_tool *tool, return -1; } - mmap_name = machine__mmap_name(machine, name_buff, sizeof(name_buff)); if (machine__is_host(machine)) { /* * kernel uses PERF_RECORD_MISC_USER for user space maps, @@ -931,7 +928,7 @@ int perf_event__synthesize_kernel_mmap(struct perf_tool *tool, kmap = map__kmap(map); size = snprintf(event->mmap.filename, sizeof(event->mmap.filename), - "%s%s", mmap_name, kmap->ref_reloc_sym->name) + 1; + "%s%s", machine->mmap_name, kmap->ref_reloc_sym->name) + 1; size = PERF_ALIGN(size, sizeof(u64)); event->mmap.header.type = PERF_RECORD_MMAP; event->mmap.header.size = (sizeof(event->mmap) - diff --git a/tools/perf/util/machine.c b/tools/perf/util/machine.c index c976384f9022..b1f1961b13f4 100644 --- a/tools/perf/util/machine.c +++ b/tools/perf/util/machine.c @@ -48,6 +48,27 @@ static void machine__threads_init(struct machine *machine) } } +static int machine__set_mmap_name(struct machine *machine) +{ + if (machine__is_host(machine)) { + if (symbol_conf.vmlinux_name) + machine->mmap_name = strdup(symbol_conf.vmlinux_name); + else + machine->mmap_name = strdup("[kernel.kallsyms]"); + } else if (machine__is_default_guest(machine)) { + if (symbol_conf.default_guest_vmlinux_name) + machine->mmap_name = strdup(symbol_conf.default_guest_vmlinux_name); + else + machine->mmap_name = strdup("[guest.kernel.kallsyms]"); + } else { + if (asprintf(&machine->mmap_name, "[guest.kernel.kallsyms.%d]", + machine->pid) < 0) + machine->mmap_name = NULL; + } + + return machine->mmap_name ? 0 : -ENOMEM; +} + int machine__init(struct machine *machine, const char *root_dir, pid_t pid) { int err = -ENOMEM; @@ -75,6 +96,9 @@ int machine__init(struct machine *machine, const char *root_dir, pid_t pid) if (machine->root_dir == NULL) return -ENOMEM; + if (machine__set_mmap_name(machine)) + goto out; + if (pid != HOST_KERNEL_ID) { struct thread *thread = machine__findnew_thread(machine, -1, pid); @@ -92,8 +116,10 @@ int machine__init(struct machine *machine, const char *root_dir, pid_t pid) err = 0; out: - if (err) + if (err) { zfree(&machine->root_dir); + zfree(&machine->mmap_name); + } return 0; } @@ -186,6 +212,7 @@ void machine__exit(struct machine *machine) dsos__exit(&machine->dsos); machine__exit_vdso(machine); zfree(&machine->root_dir); + zfree(&machine->mmap_name); zfree(&machine->current_tid); for (i = 0; i < THREADS__TABLE_SIZE; i++) { @@ -328,20 +355,6 @@ void machines__process_guests(struct machines *machines, } } -char *machine__mmap_name(struct machine *machine, char *bf, size_t size) -{ - if (machine__is_host(machine)) - snprintf(bf, size, "[%s]", "kernel.kallsyms"); - else if (machine__is_default_guest(machine)) - snprintf(bf, size, "[%s]", "guest.kernel.kallsyms"); - else { - snprintf(bf, size, "[%s.%d]", "guest.kernel.kallsyms", - machine->pid); - } - - return bf; -} - void machines__set_id_hdr_size(struct machines *machines, u16 id_hdr_size) { struct rb_node *node; @@ -777,25 +790,13 @@ size_t machine__fprintf(struct machine *machine, FILE *fp) static struct dso *machine__get_kernel(struct machine *machine) { - const char *vmlinux_name = NULL; + const char *vmlinux_name = machine->mmap_name; struct dso *kernel; if (machine__is_host(machine)) { - vmlinux_name = symbol_conf.vmlinux_name; - if (!vmlinux_name) - vmlinux_name = DSO__NAME_KALLSYMS; - kernel = machine__findnew_kernel(machine, vmlinux_name, "[kernel]", DSO_TYPE_KERNEL); } else { - char bf[PATH_MAX]; - - if (machine__is_default_guest(machine)) - vmlinux_name = symbol_conf.default_guest_vmlinux_name; - if (!vmlinux_name) - vmlinux_name = machine__mmap_name(machine, bf, - sizeof(bf)); - kernel = machine__findnew_kernel(machine, vmlinux_name, "[guest.kernel]", DSO_TYPE_GUEST_KERNEL); @@ -1295,7 +1296,6 @@ static int machine__process_kernel_mmap_event(struct machine *machine, union perf_event *event) { struct map *map; - char kmmap_prefix[PATH_MAX]; enum dso_kernel_type kernel_type; bool is_kernel_mmap; @@ -1303,15 +1303,14 @@ static int machine__process_kernel_mmap_event(struct machine *machine, if (machine__uses_kcore(machine)) return 0; - machine__mmap_name(machine, kmmap_prefix, sizeof(kmmap_prefix)); if (machine__is_host(machine)) kernel_type = DSO_TYPE_KERNEL; else kernel_type = DSO_TYPE_GUEST_KERNEL; is_kernel_mmap = memcmp(event->mmap.filename, - kmmap_prefix, - strlen(kmmap_prefix) - 1) == 0; + machine->mmap_name, + strlen(machine->mmap_name) - 1) == 0; if (event->mmap.filename[0] == '/' || (!is_kernel_mmap && event->mmap.filename[0] == '[')) { map = machine__findnew_module_map(machine, event->mmap.start, @@ -1322,7 +1321,7 @@ static int machine__process_kernel_mmap_event(struct machine *machine, map->end = map->start + event->mmap.len; } else if (is_kernel_mmap) { const char *symbol_name = (event->mmap.filename + - strlen(kmmap_prefix)); + strlen(machine->mmap_name)); /* * Should be there already, from the build-id table in * the header. @@ -1363,7 +1362,7 @@ static int machine__process_kernel_mmap_event(struct machine *machine, up_read(&machine->dsos.lock); if (kernel == NULL) - kernel = machine__findnew_dso(machine, kmmap_prefix); + kernel = machine__findnew_dso(machine, machine->mmap_name); if (kernel == NULL) goto out_problem; diff --git a/tools/perf/util/machine.h b/tools/perf/util/machine.h index 5ce860b64c74..cb0a20f3a96b 100644 --- a/tools/perf/util/machine.h +++ b/tools/perf/util/machine.h @@ -43,6 +43,7 @@ struct machine { bool comm_exec; bool kptr_restrict_warned; char *root_dir; + char *mmap_name; struct threads threads[THREADS__TABLE_SIZE]; struct vdso_info *vdso_info; struct perf_env *env; @@ -142,8 +143,6 @@ struct machine *machines__find(struct machines *machines, pid_t pid); struct machine *machines__findnew(struct machines *machines, pid_t pid); void machines__set_id_hdr_size(struct machines *machines, u16 id_hdr_size); -char *machine__mmap_name(struct machine *machine, char *bf, size_t size); - void machines__set_comm_exec(struct machines *machines, bool comm_exec); struct machine *machine__new_host(void); diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c index e366e3060e6b..a1a312d99f30 100644 --- a/tools/perf/util/symbol.c +++ b/tools/perf/util/symbol.c @@ -1958,8 +1958,7 @@ static int dso__load_guest_kernel_sym(struct dso *dso, struct map *map) pr_debug("Using %s for symbols\n", kallsyms_filename); if (err > 0 && !dso__is_kcore(dso)) { dso->binary_type = DSO_BINARY_TYPE__GUEST_KALLSYMS; - machine__mmap_name(machine, path, sizeof(path)); - dso__set_long_name(dso, strdup(path), true); + dso__set_long_name(dso, machine->mmap_name, false); map__fixup_start(map); map__fixup_end(map); } -- cgit v1.2.3 From 05db6ff73d805ecc70947c9eee2ed9948d0be52b Mon Sep 17 00:00:00 2001 From: Jiri Olsa Date: Thu, 15 Feb 2018 13:26:31 +0100 Subject: perf machine: Generalize machine__set_kernel_mmap() So it could be called without event object, just with start and end values. It will be used in following patch. Signed-off-by: Jiri Olsa Cc: Alexander Shishkin Cc: David Ahern Cc: Namhyung Kim Cc: Peter Zijlstra Link: http://lkml.kernel.org/r/20180215122635.24029-6-jolsa@kernel.org Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/util/machine.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) (limited to 'tools') diff --git a/tools/perf/util/machine.c b/tools/perf/util/machine.c index b1f1961b13f4..292e70c774bd 100644 --- a/tools/perf/util/machine.c +++ b/tools/perf/util/machine.c @@ -1262,15 +1262,15 @@ int machine__create_kernel_maps(struct machine *machine) return 0; } -static void machine__set_kernel_mmap_len(struct machine *machine, - union perf_event *event) +static void machine__set_kernel_mmap(struct machine *machine, + u64 start, u64 end) { int i; for (i = 0; i < MAP__NR_TYPES; i++) { - machine->vmlinux_maps[i]->start = event->mmap.start; - machine->vmlinux_maps[i]->end = (event->mmap.start + - event->mmap.len); + machine->vmlinux_maps[i]->start = start; + machine->vmlinux_maps[i]->end = end; + /* * Be a bit paranoid here, some perf.data file came with * a zero sized synthesized MMAP event for the kernel. @@ -1375,7 +1375,8 @@ static int machine__process_kernel_mmap_event(struct machine *machine, if (strstr(kernel->long_name, "vmlinux")) dso__set_short_name(kernel, "[kernel.vmlinux]", false); - machine__set_kernel_mmap_len(machine, event); + machine__set_kernel_mmap(machine, event->mmap.start, + event->mmap.start + event->mmap.len); /* * Avoid using a zero address (kptr_restrict) for the ref reloc -- cgit v1.2.3 From 1fb87b8e9599932e1d8b11c3a1b03b4414aaf7ba Mon Sep 17 00:00:00 2001 From: Jiri Olsa Date: Thu, 15 Feb 2018 13:26:32 +0100 Subject: perf machine: Don't search for active kernel start in __machine__create_kernel_maps We should not search for the kernel start address in __machine__create_kernel_maps(), because it's being used in the 'report' code path, where we are interested in kernel MMAP data address (the one recorded via 'perf record', possibly on another machine, or an older or newer kernel on the same machine where analysis is being performed) instead of in current kernel address. The __machine__create_kernel_maps() function serves purely for creating the machines kernel maps and setting up the kmap group. The report code path then sets the address based on the data from kernel MMAP event in the machine__set_kernel_mmap() function. The kallsyms search address logic is used for test code, that calls machine__create_kernel_maps() to get current maps and calls machine__get_running_kernel_start() to get kernel starting address. Use machine__set_kernel_mmap() to set the kernel maps start address and moving map_groups__fixup_end to be call when all maps are in place. Also make __machine__create_kernel_maps static, because there's no external user. Signed-off-by: Jiri Olsa Cc: Alexander Shishkin Cc: David Ahern Cc: Namhyung Kim Cc: Peter Zijlstra Link: http://lkml.kernel.org/r/20180215122635.24029-7-jolsa@kernel.org Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/util/machine.c | 55 ++++++++++++++++++++++------------------------- tools/perf/util/machine.h | 1 - 2 files changed, 26 insertions(+), 30 deletions(-) (limited to 'tools') diff --git a/tools/perf/util/machine.c b/tools/perf/util/machine.c index 292e70c774bd..2db8d7dd0f80 100644 --- a/tools/perf/util/machine.c +++ b/tools/perf/util/machine.c @@ -856,13 +856,10 @@ static int machine__get_running_kernel_start(struct machine *machine, return 0; } -int __machine__create_kernel_maps(struct machine *machine, struct dso *kernel) +static int +__machine__create_kernel_maps(struct machine *machine, struct dso *kernel) { int type; - u64 start = 0; - - if (machine__get_running_kernel_start(machine, NULL, &start)) - return -1; /* In case of renewal the kernel map, destroy previous one */ machine__destroy_kernel_maps(machine); @@ -871,7 +868,7 @@ int __machine__create_kernel_maps(struct machine *machine, struct dso *kernel) struct kmap *kmap; struct map *map; - machine->vmlinux_maps[type] = map__new2(start, kernel, type); + machine->vmlinux_maps[type] = map__new2(0, kernel, type); if (machine->vmlinux_maps[type] == NULL) return -1; @@ -1222,6 +1219,24 @@ static int machine__create_modules(struct machine *machine) return 0; } +static void machine__set_kernel_mmap(struct machine *machine, + u64 start, u64 end) +{ + int i; + + for (i = 0; i < MAP__NR_TYPES; i++) { + machine->vmlinux_maps[i]->start = start; + machine->vmlinux_maps[i]->end = end; + + /* + * Be a bit paranoid here, some perf.data file came with + * a zero sized synthesized MMAP event for the kernel. + */ + if (machine->vmlinux_maps[i]->end == 0) + machine->vmlinux_maps[i]->end = ~0ULL; + } +} + int machine__create_kernel_maps(struct machine *machine) { struct dso *kernel = machine__get_kernel(machine); @@ -1246,40 +1261,22 @@ int machine__create_kernel_maps(struct machine *machine) "continuing anyway...\n", machine->pid); } - /* - * Now that we have all the maps created, just set the ->end of them: - */ - map_groups__fixup_end(&machine->kmaps); - if (!machine__get_running_kernel_start(machine, &name, &addr)) { if (name && maps__set_kallsyms_ref_reloc_sym(machine->vmlinux_maps, name, addr)) { machine__destroy_kernel_maps(machine); return -1; } + machine__set_kernel_mmap(machine, addr, 0); } + /* + * Now that we have all the maps created, just set the ->end of them: + */ + map_groups__fixup_end(&machine->kmaps); return 0; } -static void machine__set_kernel_mmap(struct machine *machine, - u64 start, u64 end) -{ - int i; - - for (i = 0; i < MAP__NR_TYPES; i++) { - machine->vmlinux_maps[i]->start = start; - machine->vmlinux_maps[i]->end = end; - - /* - * Be a bit paranoid here, some perf.data file came with - * a zero sized synthesized MMAP event for the kernel. - */ - if (machine->vmlinux_maps[i]->end == 0) - machine->vmlinux_maps[i]->end = ~0ULL; - } -} - static bool machine__uses_kcore(struct machine *machine) { struct dso *dso; diff --git a/tools/perf/util/machine.h b/tools/perf/util/machine.h index cb0a20f3a96b..50d587d34459 100644 --- a/tools/perf/util/machine.h +++ b/tools/perf/util/machine.h @@ -238,7 +238,6 @@ size_t machines__fprintf_dsos_buildid(struct machines *machines, FILE *fp, bool (skip)(struct dso *dso, int parm), int parm); void machine__destroy_kernel_maps(struct machine *machine); -int __machine__create_kernel_maps(struct machine *machine, struct dso *kernel); int machine__create_kernel_maps(struct machine *machine); int machines__create_kernel_maps(struct machines *machines, pid_t pid); -- cgit v1.2.3 From e8f3879f762ffe75a24fd354dd87f073214428fa Mon Sep 17 00:00:00 2001 From: Jiri Olsa Date: Thu, 15 Feb 2018 13:26:33 +0100 Subject: perf machine: Remove machine__load_kallsyms() The current machine__load_kallsyms() function has no caller, so replace it directly with __machine__load_kallsyms(). Also remove the no_kcore argument as it was always called with a 'true' value. Signed-off-by: Jiri Olsa Cc: Alexander Shishkin Cc: David Ahern Cc: Namhyung Kim Cc: Peter Zijlstra Link: http://lkml.kernel.org/r/20180215122635.24029-8-jolsa@kernel.org Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/tests/vmlinux-kallsyms.c | 2 +- tools/perf/util/machine.c | 14 ++++---------- tools/perf/util/machine.h | 2 -- 3 files changed, 5 insertions(+), 13 deletions(-) (limited to 'tools') diff --git a/tools/perf/tests/vmlinux-kallsyms.c b/tools/perf/tests/vmlinux-kallsyms.c index f6789fb029d6..58349297f9fb 100644 --- a/tools/perf/tests/vmlinux-kallsyms.c +++ b/tools/perf/tests/vmlinux-kallsyms.c @@ -56,7 +56,7 @@ int test__vmlinux_matches_kallsyms(struct test *test __maybe_unused, int subtest * be compacted against the list of modules found in the "vmlinux" * code and with the one got from /proc/modules from the "kallsyms" code. */ - if (__machine__load_kallsyms(&kallsyms, "/proc/kallsyms", type, true) <= 0) { + if (machine__load_kallsyms(&kallsyms, "/proc/kallsyms", type) <= 0) { pr_debug("dso__load_kallsyms "); goto out; } diff --git a/tools/perf/util/machine.c b/tools/perf/util/machine.c index 2db8d7dd0f80..fe27ef55cbb9 100644 --- a/tools/perf/util/machine.c +++ b/tools/perf/util/machine.c @@ -151,7 +151,7 @@ struct machine *machine__new_kallsyms(void) * ask for not using the kcore parsing code, once this one is fixed * to create a map per module. */ - if (machine && __machine__load_kallsyms(machine, "/proc/kallsyms", MAP__FUNCTION, true) <= 0) { + if (machine && machine__load_kallsyms(machine, "/proc/kallsyms", MAP__FUNCTION) <= 0) { machine__delete(machine); machine = NULL; } @@ -991,11 +991,11 @@ int machines__create_kernel_maps(struct machines *machines, pid_t pid) return machine__create_kernel_maps(machine); } -int __machine__load_kallsyms(struct machine *machine, const char *filename, - enum map_type type, bool no_kcore) +int machine__load_kallsyms(struct machine *machine, const char *filename, + enum map_type type) { struct map *map = machine__kernel_map(machine); - int ret = __dso__load_kallsyms(map->dso, filename, map, no_kcore); + int ret = __dso__load_kallsyms(map->dso, filename, map, true); if (ret > 0) { dso__set_loaded(map->dso, type); @@ -1010,12 +1010,6 @@ int __machine__load_kallsyms(struct machine *machine, const char *filename, return ret; } -int machine__load_kallsyms(struct machine *machine, const char *filename, - enum map_type type) -{ - return __machine__load_kallsyms(machine, filename, type, false); -} - int machine__load_vmlinux_path(struct machine *machine, enum map_type type) { struct map *map = machine__kernel_map(machine); diff --git a/tools/perf/util/machine.h b/tools/perf/util/machine.h index 50d587d34459..66cc200ef86f 100644 --- a/tools/perf/util/machine.h +++ b/tools/perf/util/machine.h @@ -225,8 +225,6 @@ struct map *machine__findnew_module_map(struct machine *machine, u64 start, const char *filename); int arch__fix_module_text_start(u64 *start, const char *name); -int __machine__load_kallsyms(struct machine *machine, const char *filename, - enum map_type type, bool no_kcore); int machine__load_kallsyms(struct machine *machine, const char *filename, enum map_type type); int machine__load_vmlinux_path(struct machine *machine, enum map_type type); -- cgit v1.2.3 From a73e24d240bc136619d382b1268f34d75c9d25ce Mon Sep 17 00:00:00 2001 From: Jiri Olsa Date: Thu, 15 Feb 2018 13:26:34 +0100 Subject: perf tools: Do not create kernel maps in sample__resolve() There's no need for kernel maps to be allocated at this point - sample processing. We search for kernel maps using the kernel map_groups in machine::kmaps which is static. If vmlinux maps for any reason still don't exist, the search correctly fails because they are not in the map group. Signed-off-by: Jiri Olsa Cc: Alexander Shishkin Cc: David Ahern Cc: Namhyung Kim Cc: Peter Zijlstra Link: http://lkml.kernel.org/r/20180215122635.24029-9-jolsa@kernel.org Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/util/event.c | 11 ----------- 1 file changed, 11 deletions(-) (limited to 'tools') diff --git a/tools/perf/util/event.c b/tools/perf/util/event.c index 4644e751a3e3..f0a6cbd033cc 100644 --- a/tools/perf/util/event.c +++ b/tools/perf/util/event.c @@ -1588,17 +1588,6 @@ int machine__resolve(struct machine *machine, struct addr_location *al, return -1; dump_printf(" ... thread: %s:%d\n", thread__comm_str(thread), thread->tid); - /* - * Have we already created the kernel maps for this machine? - * - * This should have happened earlier, when we processed the kernel MMAP - * events, but for older perf.data files there was no such thing, so do - * it now. - */ - if (sample->cpumode == PERF_RECORD_MISC_KERNEL && - machine__kernel_map(machine) == NULL) - machine__create_kernel_maps(machine); - thread__find_addr_map(thread, sample->cpumode, MAP__FUNCTION, sample->ip, al); dump_printf(" ...... dso: %s\n", al->map ? al->map->dso->long_name : -- cgit v1.2.3 From ab6e9a99345131cd8e54268d1d0dc04a33f7ed11 Mon Sep 17 00:00:00 2001 From: Jiri Olsa Date: Thu, 15 Feb 2018 13:26:35 +0100 Subject: perf tests: Use arch__compare_symbol_names to compare symbols The symbol search called by machine__find_kernel_symbol_by_name is using internally arch__compare_symbol_names function to compare 2 symbol names, because different archs have different ways of comparing symbols. Mostly for skipping '.' prefixes and similar. In test 1 when we try to find matching symbols in kallsyms and vmlinux, by address and by symbol name. When either is found we compare the pair symbol names by simple strcmp, which is not good enough for reasons explained in previous paragraph. On powerpc this can cause lockup, because even thought we found the pair, the compared names are different and don't match simple strcmp. Following code path is executed, that leads to lockup: - we find the pair in kallsyms by sym->start next_pair: - we compare the names and it fails - we find the pair by sym->name - the pair addresses match so we call goto next_pair because we assume the names match in this case Signed-off-by: Jiri Olsa Tested-by: Naveen N. Rao Acked-by: Naveen N. Rao Cc: Alexander Shishkin Cc: David Ahern Cc: Namhyung Kim Cc: Peter Zijlstra Fixes: 031b84c407c3 ("perf probe ppc: Enable matching against dot symbols automatically") Link: http://lkml.kernel.org/r/20180215122635.24029-10-jolsa@kernel.org Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/tests/vmlinux-kallsyms.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'tools') diff --git a/tools/perf/tests/vmlinux-kallsyms.c b/tools/perf/tests/vmlinux-kallsyms.c index 58349297f9fb..1e5adb65632a 100644 --- a/tools/perf/tests/vmlinux-kallsyms.c +++ b/tools/perf/tests/vmlinux-kallsyms.c @@ -125,7 +125,7 @@ int test__vmlinux_matches_kallsyms(struct test *test __maybe_unused, int subtest if (pair && UM(pair->start) == mem_start) { next_pair: - if (strcmp(sym->name, pair->name) == 0) { + if (arch__compare_symbol_names(sym->name, pair->name) == 0) { /* * kallsyms don't have the symbol end, so we * set that by using the next symbol start - 1, -- cgit v1.2.3 From 099c1130998fd2234a200b55d69713ec66d88325 Mon Sep 17 00:00:00 2001 From: Mathieu Poirier Date: Mon, 12 Feb 2018 13:32:35 -0700 Subject: perf cs-etm: Freeing allocated memory This patch frees all the memory allocated in function cs_etm__alloc_queue(). Signed-off-by: Mathieu Poirier Cc: Alexander Shishkin Cc: Jin Yao Cc: Namhyung Kim Cc: Peter Zijlstra Cc: linux-arm-kernel@lists.infradead.org Link: http://lkml.kernel.org/r/1518467557-18505-2-git-send-email-mathieu.poirier@linaro.org Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/util/cs-etm.c | 6 ++++++ 1 file changed, 6 insertions(+) (limited to 'tools') diff --git a/tools/perf/util/cs-etm.c b/tools/perf/util/cs-etm.c index b9f0a53dfa65..f2c98774e665 100644 --- a/tools/perf/util/cs-etm.c +++ b/tools/perf/util/cs-etm.c @@ -174,6 +174,12 @@ static void cs_etm__free_queue(void *priv) { struct cs_etm_queue *etmq = priv; + if (!etmq) + return; + + thread__zput(etmq->thread); + cs_etm_decoder__free(etmq->decoder); + zfree(&etmq->event_buf); free(etmq); } -- cgit v1.2.3 From 147c508f3004df6e2958f6c8867909531c2a15e2 Mon Sep 17 00:00:00 2001 From: Jin Yao Date: Mon, 12 Feb 2018 13:32:36 -0700 Subject: perf tools: Use target->per_thread and target->system_wide flags Mathieu Poirier reports issue in commit ("73c0ca1eee3d perf thread_map: Enumerate all threads from /proc") that it has negative impact on 'perf record --per-thread'. It has the effect of creating a kernel event for each thread in the system for 'perf record --per-thread'. Mathieu Poirier's patch ("perf util: Do not reuse target->per_thread flag") can fix this issue by creating a new target->all_threads flag. This patch is based on Mathieu Poirier's patch but it doesn't use a new target->all_threads flag. This patch just uses 'target->per_thread && target->system_wide' as a condition to check for all threads case. Signed-off-by: Jin Yao Cc: Alexander Shishkin Cc: Namhyung Kim Cc: Peter Zijlstra Cc: linux-arm-kernel@lists.infradead.org Fixes: 73c0ca1eee3d ("perf thread_map: Enumerate all threads from /proc") Link: http://lkml.kernel.org/r/1518467557-18505-3-git-send-email-mathieu.poirier@linaro.org Signed-off-by: Mathieu Poirier [Fixed checkpatch warning about line over 80 characters] Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/util/evlist.c | 21 ++++++++++++++++++++- tools/perf/util/thread_map.c | 4 ++-- tools/perf/util/thread_map.h | 2 +- 3 files changed, 23 insertions(+), 4 deletions(-) (limited to 'tools') diff --git a/tools/perf/util/evlist.c b/tools/perf/util/evlist.c index e5fc14e53c05..7b7d535396f7 100644 --- a/tools/perf/util/evlist.c +++ b/tools/perf/util/evlist.c @@ -1086,11 +1086,30 @@ int perf_evlist__mmap(struct perf_evlist *evlist, unsigned int pages) int perf_evlist__create_maps(struct perf_evlist *evlist, struct target *target) { + bool all_threads = (target->per_thread && target->system_wide); struct cpu_map *cpus; struct thread_map *threads; + /* + * If specify '-a' and '--per-thread' to perf record, perf record + * will override '--per-thread'. target->per_thread = false and + * target->system_wide = true. + * + * If specify '--per-thread' only to perf record, + * target->per_thread = true and target->system_wide = false. + * + * So target->per_thread && target->system_wide is false. + * For perf record, thread_map__new_str doesn't call + * thread_map__new_all_cpus. That will keep perf record's + * current behavior. + * + * For perf stat, it allows the case that target->per_thread and + * target->system_wide are all true. It means to collect system-wide + * per-thread data. thread_map__new_str will call + * thread_map__new_all_cpus to enumerate all threads. + */ threads = thread_map__new_str(target->pid, target->tid, target->uid, - target->per_thread); + all_threads); if (!threads) return -1; diff --git a/tools/perf/util/thread_map.c b/tools/perf/util/thread_map.c index 3e1038f6491c..729dad8f412d 100644 --- a/tools/perf/util/thread_map.c +++ b/tools/perf/util/thread_map.c @@ -323,7 +323,7 @@ out_free_threads: } struct thread_map *thread_map__new_str(const char *pid, const char *tid, - uid_t uid, bool per_thread) + uid_t uid, bool all_threads) { if (pid) return thread_map__new_by_pid_str(pid); @@ -331,7 +331,7 @@ struct thread_map *thread_map__new_str(const char *pid, const char *tid, if (!tid && uid != UINT_MAX) return thread_map__new_by_uid(uid); - if (per_thread) + if (all_threads) return thread_map__new_all_cpus(); return thread_map__new_by_tid_str(tid); diff --git a/tools/perf/util/thread_map.h b/tools/perf/util/thread_map.h index 0a806b99e73c..5ec91cfd1869 100644 --- a/tools/perf/util/thread_map.h +++ b/tools/perf/util/thread_map.h @@ -31,7 +31,7 @@ struct thread_map *thread_map__get(struct thread_map *map); void thread_map__put(struct thread_map *map); struct thread_map *thread_map__new_str(const char *pid, - const char *tid, uid_t uid, bool per_thread); + const char *tid, uid_t uid, bool all_threads); struct thread_map *thread_map__new_by_tid_str(const char *tid_str); -- cgit v1.2.3 From d2785de15f1bd42d613d56bbac5a007e7293b874 Mon Sep 17 00:00:00 2001 From: Mathieu Poirier Date: Mon, 12 Feb 2018 13:32:37 -0700 Subject: perf auxtrace arm: Fixing uninitialised variable When working natively on arm64 the compiler gets pesky and complains that variable 'i' is uninitialised, something that breaks the compilation. Here no further checks are needed since variable 'found_spe' can only be true if variable 'i' has been initialised as part of the for loop. Signed-off-by: Mathieu Poirier Cc: Alexander Shishkin Cc: Jin Yao Cc: Namhyung Kim Cc: Peter Zijlstra Cc: linux-arm-kernel@lists.infradead.org Link: http://lkml.kernel.org/r/1518467557-18505-4-git-send-email-mathieu.poirier@linaro.org Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/arch/arm/util/auxtrace.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'tools') diff --git a/tools/perf/arch/arm/util/auxtrace.c b/tools/perf/arch/arm/util/auxtrace.c index 2323581b157d..fa639e3e52ac 100644 --- a/tools/perf/arch/arm/util/auxtrace.c +++ b/tools/perf/arch/arm/util/auxtrace.c @@ -68,7 +68,7 @@ struct auxtrace_record bool found_spe = false; static struct perf_pmu **arm_spe_pmus = NULL; static int nr_spes = 0; - int i; + int i = 0; if (!evlist) return NULL; -- cgit v1.2.3 From 796bfadd831bdef0de06cd0253398c3fe011e459 Mon Sep 17 00:00:00 2001 From: Mathieu Poirier Date: Mon, 12 Feb 2018 16:38:57 -0700 Subject: perf cs-etm: Properly deal with cpu maps This patch allows the CoreSight AUX info section to fit topologies where only a subset of all available CPUs are present, avoiding at the same time accessing the ETM configuration areas of CPUs that have been offlined. Signed-off-by: Mathieu Poirier Cc: Alexander Shishkin Cc: Jiri Olsa Cc: Namhyung Kim Cc: Peter Zijlstra Cc: linux-arm-kernel@lists.infradead.org Link: http://lkml.kernel.org/r/1518478737-24649-1-git-send-email-mathieu.poirier@linaro.org Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/arch/arm/util/cs-etm.c | 51 +++++++++++++++++++++++++++------------ 1 file changed, 36 insertions(+), 15 deletions(-) (limited to 'tools') diff --git a/tools/perf/arch/arm/util/cs-etm.c b/tools/perf/arch/arm/util/cs-etm.c index fbfc055d3f4d..5c655ad4621e 100644 --- a/tools/perf/arch/arm/util/cs-etm.c +++ b/tools/perf/arch/arm/util/cs-etm.c @@ -298,12 +298,17 @@ cs_etm_info_priv_size(struct auxtrace_record *itr __maybe_unused, { int i; int etmv3 = 0, etmv4 = 0; - const struct cpu_map *cpus = evlist->cpus; + struct cpu_map *event_cpus = evlist->cpus; + struct cpu_map *online_cpus = cpu_map__new(NULL); /* cpu map is not empty, we have specific CPUs to work with */ - if (!cpu_map__empty(cpus)) { - for (i = 0; i < cpu_map__nr(cpus); i++) { - if (cs_etm_is_etmv4(itr, cpus->map[i])) + if (!cpu_map__empty(event_cpus)) { + for (i = 0; i < cpu__max_cpu(); i++) { + if (!cpu_map__has(event_cpus, i) || + !cpu_map__has(online_cpus, i)) + continue; + + if (cs_etm_is_etmv4(itr, i)) etmv4++; else etmv3++; @@ -311,6 +316,9 @@ cs_etm_info_priv_size(struct auxtrace_record *itr __maybe_unused, } else { /* get configuration for all CPUs in the system */ for (i = 0; i < cpu__max_cpu(); i++) { + if (!cpu_map__has(online_cpus, i)) + continue; + if (cs_etm_is_etmv4(itr, i)) etmv4++; else @@ -318,6 +326,8 @@ cs_etm_info_priv_size(struct auxtrace_record *itr __maybe_unused, } } + cpu_map__put(online_cpus); + return (CS_ETM_HEADER_SIZE + (etmv4 * CS_ETMV4_PRIV_SIZE) + (etmv3 * CS_ETMV3_PRIV_SIZE)); @@ -447,7 +457,9 @@ static int cs_etm_info_fill(struct auxtrace_record *itr, int i; u32 offset; u64 nr_cpu, type; - const struct cpu_map *cpus = session->evlist->cpus; + struct cpu_map *cpu_map; + struct cpu_map *event_cpus = session->evlist->cpus; + struct cpu_map *online_cpus = cpu_map__new(NULL); struct cs_etm_recording *ptr = container_of(itr, struct cs_etm_recording, itr); struct perf_pmu *cs_etm_pmu = ptr->cs_etm_pmu; @@ -458,8 +470,21 @@ static int cs_etm_info_fill(struct auxtrace_record *itr, if (!session->evlist->nr_mmaps) return -EINVAL; - /* If the cpu_map is empty all CPUs are involved */ - nr_cpu = cpu_map__empty(cpus) ? cpu__max_cpu() : cpu_map__nr(cpus); + /* If the cpu_map is empty all online CPUs are involved */ + if (cpu_map__empty(event_cpus)) { + cpu_map = online_cpus; + } else { + /* Make sure all specified CPUs are online */ + for (i = 0; i < cpu_map__nr(event_cpus); i++) { + if (cpu_map__has(event_cpus, i) && + !cpu_map__has(online_cpus, i)) + return -EINVAL; + } + + cpu_map = event_cpus; + } + + nr_cpu = cpu_map__nr(cpu_map); /* Get PMU type as dynamically assigned by the core */ type = cs_etm_pmu->type; @@ -472,15 +497,11 @@ static int cs_etm_info_fill(struct auxtrace_record *itr, offset = CS_ETM_SNAPSHOT + 1; - /* cpu map is not empty, we have specific CPUs to work with */ - if (!cpu_map__empty(cpus)) { - for (i = 0; i < cpu_map__nr(cpus) && offset < priv_size; i++) - cs_etm_get_metadata(cpus->map[i], &offset, itr, info); - } else { - /* get configuration for all CPUs in the system */ - for (i = 0; i < cpu__max_cpu(); i++) + for (i = 0; i < cpu__max_cpu() && offset < priv_size; i++) + if (cpu_map__has(cpu_map, i)) cs_etm_get_metadata(i, &offset, itr, info); - } + + cpu_map__put(online_cpus); return 0; } -- cgit v1.2.3 From ac2c30683803b08bdf35c08d712eafa1ca496164 Mon Sep 17 00:00:00 2001 From: Jaecheol Shin Date: Wed, 7 Feb 2018 18:52:05 +0900 Subject: perf annotate: Add missing arguments in Man page Some options must require an argument. But input, stdio-color, cpu have no them. So I added it. Signed-off-by: Jaecheol Shin Cc: Jiri Olsa Cc: Namhyung Kim Cc: Taeung Song Link: http://lkml.kernel.org/r/20180207095205.62715-1-jcgod413@gmail.com Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/Documentation/perf-annotate.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) (limited to 'tools') diff --git a/tools/perf/Documentation/perf-annotate.txt b/tools/perf/Documentation/perf-annotate.txt index c635eab6af54..292809c3c0ca 100644 --- a/tools/perf/Documentation/perf-annotate.txt +++ b/tools/perf/Documentation/perf-annotate.txt @@ -21,7 +21,7 @@ If there is no debug info in the object, then annotated assembly is displayed. OPTIONS ------- -i:: ---input=:: +--input=:: Input file name. (default: perf.data unless stdin is a fifo) -d:: @@ -69,7 +69,7 @@ OPTIONS --stdio:: Use the stdio interface. ---stdio-color:: +--stdio-color=:: 'always', 'never' or 'auto', allowing configuring color output via the command line, in addition to via "color.ui" .perfconfig. Use '--stdio-color always' to generate color even when redirecting @@ -84,7 +84,7 @@ OPTIONS --gtk:: Use the GTK interface. -C:: ---cpu:: Only report samples for the list of CPUs provided. Multiple CPUs can +--cpu=:: Only report samples for the list of CPUs provided. Multiple CPUs can be provided as a comma-separated list with no space: 0,1. Ranges of CPUs are specified with -: 0-2. Default is to report samples on all CPUs. -- cgit v1.2.3 From 577980a00063935815a55f461601579fd5e61f59 Mon Sep 17 00:00:00 2001 From: Sangwon Hong Date: Mon, 12 Feb 2018 05:38:36 +0900 Subject: perf kmem: Document a missing option & an argument First, 'perf kmem' has a '--force' option, but didn't document it on the man page. So add it. Second, the '--time' option has to get a value, but isn't documented on the man page. Describe it. Signed-off-by: Sangwon Hong Acked-by: Namhyung Kim Cc: Jiri Olsa Cc: Taeung Song Link: http://lkml.kernel.org/r/1518381517-30766-1-git-send-email-qpakzk@gmail.com [ Add blank like after --force block, as requested by Namhyung ] Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/Documentation/perf-kmem.txt | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) (limited to 'tools') diff --git a/tools/perf/Documentation/perf-kmem.txt b/tools/perf/Documentation/perf-kmem.txt index 479fc3261a50..85b8ac695c87 100644 --- a/tools/perf/Documentation/perf-kmem.txt +++ b/tools/perf/Documentation/perf-kmem.txt @@ -25,6 +25,10 @@ OPTIONS --input=:: Select the input file (default: perf.data unless stdin is a fifo) +-f:: +--force:: + Don't do ownership validation + -v:: --verbose:: Be more verbose. (show symbol address, etc) @@ -61,7 +65,7 @@ OPTIONS default, but this option shows live (currently allocated) pages instead. (This option works with --page option only) ---time:: +--time=,:: Only analyze samples within given time window: ,. Times have the format seconds.microseconds. If start is not given (i.e., time string is ',x.y') then analysis starts at the beginning of the file. If -- cgit v1.2.3 From 7e99b1972263c2f611d7f2fb67d09f3384006593 Mon Sep 17 00:00:00 2001 From: Sangwon Hong Date: Mon, 12 Feb 2018 05:38:37 +0900 Subject: perf mem: Document a missing option Add the missing --force option on the man page. Signed-off-by: Sangwon Hong Acked-by: Namhyung Kim Cc: Jiri Olsa Cc: Taeung Song Link: http://lkml.kernel.org/r/1518381517-30766-2-git-send-email-qpakzk@gmail.com Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/Documentation/perf-mem.txt | 4 ++++ 1 file changed, 4 insertions(+) (limited to 'tools') diff --git a/tools/perf/Documentation/perf-mem.txt b/tools/perf/Documentation/perf-mem.txt index 4be08a1e3f8d..b0211410969b 100644 --- a/tools/perf/Documentation/perf-mem.txt +++ b/tools/perf/Documentation/perf-mem.txt @@ -28,6 +28,10 @@ OPTIONS ...:: Any command you can specify in a shell. +-f:: +--force:: + Don't do ownership validation + -t:: --type=:: Select the memory operation type: load or store (default: load,store) -- cgit v1.2.3 From e573e978fb12e16094c0b39fad3dc4e6b4803c2c Mon Sep 17 00:00:00 2001 From: Robert Walker Date: Wed, 14 Feb 2018 11:24:39 +0000 Subject: perf cs-etm: Inject capabilitity for CoreSight traces Added user space perf functionality to translate CoreSight traces into instruction events with branch stack. To invoke the new functionality, use the perf inject tool with --itrace=il. For example, to translate the ETM trace from perf.data into last branch records in a new inj.data file: $ perf inject --itrace=i100000il128 -i perf.data -o perf.data.new The 'i' parameter to itrace generates periodic instruction events. The period between instruction events can be specified as a number of instructions suffixed by i (default 100000). The parameter to 'l' specifies the number of entries in the branch stack attached to instruction events. The 'b' parameter to itrace generates events on taken branches. This patch also fixes the contents of the branch events used in perf report - previously branch events were generated for each contiguous range of instructions executed. These are fixed to generate branch events between the last address of a range ending in an executed branch instruction and the start address of the next range. Based on patches by Sebastian Pop with additional fixes and support for specifying the instruction period. Originally-by: Sebastian Pop Signed-off-by: Robert Walker Acked-by: Mathieu Poirier Cc: coresight@lists.linaro.org Cc: linux-arm-kernel@lists.infradead.org Link: http://lkml.kernel.org/r/1518607481-4059-2-git-send-email-robert.walker@arm.com Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/util/cs-etm-decoder/cs-etm-decoder.c | 65 +++- tools/perf/util/cs-etm-decoder/cs-etm-decoder.h | 1 + tools/perf/util/cs-etm.c | 434 +++++++++++++++++++++--- 3 files changed, 436 insertions(+), 64 deletions(-) (limited to 'tools') diff --git a/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c b/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c index 1fb01849f1c7..8ff69dfd725a 100644 --- a/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c +++ b/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c @@ -78,6 +78,8 @@ int cs_etm_decoder__reset(struct cs_etm_decoder *decoder) { ocsd_datapath_resp_t dp_ret; + decoder->prev_return = OCSD_RESP_CONT; + dp_ret = ocsd_dt_process_data(decoder->dcd_tree, OCSD_OP_RESET, 0, 0, NULL, NULL); if (OCSD_DATA_RESP_IS_FATAL(dp_ret)) @@ -253,16 +255,16 @@ static void cs_etm_decoder__clear_buffer(struct cs_etm_decoder *decoder) decoder->packet_count = 0; for (i = 0; i < MAX_BUFFER; i++) { decoder->packet_buffer[i].start_addr = 0xdeadbeefdeadbeefUL; - decoder->packet_buffer[i].end_addr = 0xdeadbeefdeadbeefUL; - decoder->packet_buffer[i].exc = false; - decoder->packet_buffer[i].exc_ret = false; - decoder->packet_buffer[i].cpu = INT_MIN; + decoder->packet_buffer[i].end_addr = 0xdeadbeefdeadbeefUL; + decoder->packet_buffer[i].last_instr_taken_branch = false; + decoder->packet_buffer[i].exc = false; + decoder->packet_buffer[i].exc_ret = false; + decoder->packet_buffer[i].cpu = INT_MIN; } } static ocsd_datapath_resp_t cs_etm_decoder__buffer_packet(struct cs_etm_decoder *decoder, - const ocsd_generic_trace_elem *elem, const u8 trace_chan_id, enum cs_etm_sample_type sample_type) { @@ -278,18 +280,16 @@ cs_etm_decoder__buffer_packet(struct cs_etm_decoder *decoder, return OCSD_RESP_FATAL_SYS_ERR; et = decoder->tail; + et = (et + 1) & (MAX_BUFFER - 1); + decoder->tail = et; + decoder->packet_count++; + decoder->packet_buffer[et].sample_type = sample_type; - decoder->packet_buffer[et].start_addr = elem->st_addr; - decoder->packet_buffer[et].end_addr = elem->en_addr; decoder->packet_buffer[et].exc = false; decoder->packet_buffer[et].exc_ret = false; decoder->packet_buffer[et].cpu = *((int *)inode->priv); - - /* Wrap around if need be */ - et = (et + 1) & (MAX_BUFFER - 1); - - decoder->tail = et; - decoder->packet_count++; + decoder->packet_buffer[et].start_addr = 0xdeadbeefdeadbeefUL; + decoder->packet_buffer[et].end_addr = 0xdeadbeefdeadbeefUL; if (decoder->packet_count == MAX_BUFFER - 1) return OCSD_RESP_WAIT; @@ -297,6 +297,40 @@ cs_etm_decoder__buffer_packet(struct cs_etm_decoder *decoder, return OCSD_RESP_CONT; } +static ocsd_datapath_resp_t +cs_etm_decoder__buffer_range(struct cs_etm_decoder *decoder, + const ocsd_generic_trace_elem *elem, + const uint8_t trace_chan_id) +{ + int ret = 0; + struct cs_etm_packet *packet; + + ret = cs_etm_decoder__buffer_packet(decoder, trace_chan_id, + CS_ETM_RANGE); + if (ret != OCSD_RESP_CONT && ret != OCSD_RESP_WAIT) + return ret; + + packet = &decoder->packet_buffer[decoder->tail]; + + packet->start_addr = elem->st_addr; + packet->end_addr = elem->en_addr; + switch (elem->last_i_type) { + case OCSD_INSTR_BR: + case OCSD_INSTR_BR_INDIRECT: + packet->last_instr_taken_branch = elem->last_instr_exec; + break; + case OCSD_INSTR_ISB: + case OCSD_INSTR_DSB_DMB: + case OCSD_INSTR_OTHER: + default: + packet->last_instr_taken_branch = false; + break; + } + + return ret; + +} + static ocsd_datapath_resp_t cs_etm_decoder__gen_trace_elem_printer( const void *context, const ocsd_trc_index_t indx __maybe_unused, @@ -316,9 +350,8 @@ static ocsd_datapath_resp_t cs_etm_decoder__gen_trace_elem_printer( decoder->trace_on = true; break; case OCSD_GEN_TRC_ELEM_INSTR_RANGE: - resp = cs_etm_decoder__buffer_packet(decoder, elem, - trace_chan_id, - CS_ETM_RANGE); + resp = cs_etm_decoder__buffer_range(decoder, elem, + trace_chan_id); break; case OCSD_GEN_TRC_ELEM_EXCEPTION: decoder->packet_buffer[decoder->tail].exc = true; diff --git a/tools/perf/util/cs-etm-decoder/cs-etm-decoder.h b/tools/perf/util/cs-etm-decoder/cs-etm-decoder.h index 3d2e6205d186..a4fdd285b145 100644 --- a/tools/perf/util/cs-etm-decoder/cs-etm-decoder.h +++ b/tools/perf/util/cs-etm-decoder/cs-etm-decoder.h @@ -30,6 +30,7 @@ struct cs_etm_packet { enum cs_etm_sample_type sample_type; u64 start_addr; u64 end_addr; + u8 last_instr_taken_branch; u8 exc; u8 exc_ret; int cpu; diff --git a/tools/perf/util/cs-etm.c b/tools/perf/util/cs-etm.c index f2c98774e665..6e595d96c04d 100644 --- a/tools/perf/util/cs-etm.c +++ b/tools/perf/util/cs-etm.c @@ -32,6 +32,14 @@ #define MAX_TIMESTAMP (~0ULL) +/* + * A64 instructions are always 4 bytes + * + * Only A64 is supported, so can use this constant for converting between + * addresses and instruction counts, calculting offsets etc + */ +#define A64_INSTR_SIZE 4 + struct cs_etm_auxtrace { struct auxtrace auxtrace; struct auxtrace_queues queues; @@ -45,11 +53,15 @@ struct cs_etm_auxtrace { u8 snapshot_mode; u8 data_queued; u8 sample_branches; + u8 sample_instructions; int num_cpu; u32 auxtrace_type; u64 branches_sample_type; u64 branches_id; + u64 instructions_sample_type; + u64 instructions_sample_period; + u64 instructions_id; u64 **metadata; u64 kernel_start; unsigned int pmu_type; @@ -68,6 +80,12 @@ struct cs_etm_queue { u64 time; u64 timestamp; u64 offset; + u64 period_instructions; + struct branch_stack *last_branch; + struct branch_stack *last_branch_rb; + size_t last_branch_pos; + struct cs_etm_packet *prev_packet; + struct cs_etm_packet *packet; }; static int cs_etm__update_queues(struct cs_etm_auxtrace *etm); @@ -180,6 +198,10 @@ static void cs_etm__free_queue(void *priv) thread__zput(etmq->thread); cs_etm_decoder__free(etmq->decoder); zfree(&etmq->event_buf); + zfree(&etmq->last_branch); + zfree(&etmq->last_branch_rb); + zfree(&etmq->prev_packet); + zfree(&etmq->packet); free(etmq); } @@ -276,11 +298,35 @@ static struct cs_etm_queue *cs_etm__alloc_queue(struct cs_etm_auxtrace *etm, struct cs_etm_decoder_params d_params; struct cs_etm_trace_params *t_params; struct cs_etm_queue *etmq; + size_t szp = sizeof(struct cs_etm_packet); etmq = zalloc(sizeof(*etmq)); if (!etmq) return NULL; + etmq->packet = zalloc(szp); + if (!etmq->packet) + goto out_free; + + if (etm->synth_opts.last_branch || etm->sample_branches) { + etmq->prev_packet = zalloc(szp); + if (!etmq->prev_packet) + goto out_free; + } + + if (etm->synth_opts.last_branch) { + size_t sz = sizeof(struct branch_stack); + + sz += etm->synth_opts.last_branch_sz * + sizeof(struct branch_entry); + etmq->last_branch = zalloc(sz); + if (!etmq->last_branch) + goto out_free; + etmq->last_branch_rb = zalloc(sz); + if (!etmq->last_branch_rb) + goto out_free; + } + etmq->event_buf = malloc(PERF_SAMPLE_MAX_SIZE); if (!etmq->event_buf) goto out_free; @@ -335,6 +381,7 @@ static struct cs_etm_queue *cs_etm__alloc_queue(struct cs_etm_auxtrace *etm, goto out_free_decoder; etmq->offset = 0; + etmq->period_instructions = 0; return etmq; @@ -342,6 +389,10 @@ out_free_decoder: cs_etm_decoder__free(etmq->decoder); out_free: zfree(&etmq->event_buf); + zfree(&etmq->last_branch); + zfree(&etmq->last_branch_rb); + zfree(&etmq->prev_packet); + zfree(&etmq->packet); free(etmq); return NULL; @@ -395,6 +446,129 @@ static int cs_etm__update_queues(struct cs_etm_auxtrace *etm) return 0; } +static inline void cs_etm__copy_last_branch_rb(struct cs_etm_queue *etmq) +{ + struct branch_stack *bs_src = etmq->last_branch_rb; + struct branch_stack *bs_dst = etmq->last_branch; + size_t nr = 0; + + /* + * Set the number of records before early exit: ->nr is used to + * determine how many branches to copy from ->entries. + */ + bs_dst->nr = bs_src->nr; + + /* + * Early exit when there is nothing to copy. + */ + if (!bs_src->nr) + return; + + /* + * As bs_src->entries is a circular buffer, we need to copy from it in + * two steps. First, copy the branches from the most recently inserted + * branch ->last_branch_pos until the end of bs_src->entries buffer. + */ + nr = etmq->etm->synth_opts.last_branch_sz - etmq->last_branch_pos; + memcpy(&bs_dst->entries[0], + &bs_src->entries[etmq->last_branch_pos], + sizeof(struct branch_entry) * nr); + + /* + * If we wrapped around at least once, the branches from the beginning + * of the bs_src->entries buffer and until the ->last_branch_pos element + * are older valid branches: copy them over. The total number of + * branches copied over will be equal to the number of branches asked by + * the user in last_branch_sz. + */ + if (bs_src->nr >= etmq->etm->synth_opts.last_branch_sz) { + memcpy(&bs_dst->entries[nr], + &bs_src->entries[0], + sizeof(struct branch_entry) * etmq->last_branch_pos); + } +} + +static inline void cs_etm__reset_last_branch_rb(struct cs_etm_queue *etmq) +{ + etmq->last_branch_pos = 0; + etmq->last_branch_rb->nr = 0; +} + +static inline u64 cs_etm__last_executed_instr(struct cs_etm_packet *packet) +{ + /* + * The packet records the execution range with an exclusive end address + * + * A64 instructions are constant size, so the last executed + * instruction is A64_INSTR_SIZE before the end address + * Will need to do instruction level decode for T32 instructions as + * they can be variable size (not yet supported). + */ + return packet->end_addr - A64_INSTR_SIZE; +} + +static inline u64 cs_etm__instr_count(const struct cs_etm_packet *packet) +{ + /* + * Only A64 instructions are currently supported, so can get + * instruction count by dividing. + * Will need to do instruction level decode for T32 instructions as + * they can be variable size (not yet supported). + */ + return (packet->end_addr - packet->start_addr) / A64_INSTR_SIZE; +} + +static inline u64 cs_etm__instr_addr(const struct cs_etm_packet *packet, + u64 offset) +{ + /* + * Only A64 instructions are currently supported, so can get + * instruction address by muliplying. + * Will need to do instruction level decode for T32 instructions as + * they can be variable size (not yet supported). + */ + return packet->start_addr + offset * A64_INSTR_SIZE; +} + +static void cs_etm__update_last_branch_rb(struct cs_etm_queue *etmq) +{ + struct branch_stack *bs = etmq->last_branch_rb; + struct branch_entry *be; + + /* + * The branches are recorded in a circular buffer in reverse + * chronological order: we start recording from the last element of the + * buffer down. After writing the first element of the stack, move the + * insert position back to the end of the buffer. + */ + if (!etmq->last_branch_pos) + etmq->last_branch_pos = etmq->etm->synth_opts.last_branch_sz; + + etmq->last_branch_pos -= 1; + + be = &bs->entries[etmq->last_branch_pos]; + be->from = cs_etm__last_executed_instr(etmq->prev_packet); + be->to = etmq->packet->start_addr; + /* No support for mispredict */ + be->flags.mispred = 0; + be->flags.predicted = 1; + + /* + * Increment bs->nr until reaching the number of last branches asked by + * the user on the command line. + */ + if (bs->nr < etmq->etm->synth_opts.last_branch_sz) + bs->nr += 1; +} + +static int cs_etm__inject_event(union perf_event *event, + struct perf_sample *sample, u64 type) +{ + event->header.size = perf_event__sample_event_size(sample, type, 0); + return perf_event__synthesize_sample(event, type, 0, sample); +} + + static int cs_etm__get_trace(struct cs_etm_buffer *buff, struct cs_etm_queue *etmq) { @@ -459,35 +633,105 @@ static void cs_etm__set_pid_tid_cpu(struct cs_etm_auxtrace *etm, } } +static int cs_etm__synth_instruction_sample(struct cs_etm_queue *etmq, + u64 addr, u64 period) +{ + int ret = 0; + struct cs_etm_auxtrace *etm = etmq->etm; + union perf_event *event = etmq->event_buf; + struct perf_sample sample = {.ip = 0,}; + + event->sample.header.type = PERF_RECORD_SAMPLE; + event->sample.header.misc = PERF_RECORD_MISC_USER; + event->sample.header.size = sizeof(struct perf_event_header); + + sample.ip = addr; + sample.pid = etmq->pid; + sample.tid = etmq->tid; + sample.id = etmq->etm->instructions_id; + sample.stream_id = etmq->etm->instructions_id; + sample.period = period; + sample.cpu = etmq->packet->cpu; + sample.flags = 0; + sample.insn_len = 1; + sample.cpumode = event->header.misc; + + if (etm->synth_opts.last_branch) { + cs_etm__copy_last_branch_rb(etmq); + sample.branch_stack = etmq->last_branch; + } + + if (etm->synth_opts.inject) { + ret = cs_etm__inject_event(event, &sample, + etm->instructions_sample_type); + if (ret) + return ret; + } + + ret = perf_session__deliver_synth_event(etm->session, event, &sample); + + if (ret) + pr_err( + "CS ETM Trace: failed to deliver instruction event, error %d\n", + ret); + + if (etm->synth_opts.last_branch) + cs_etm__reset_last_branch_rb(etmq); + + return ret; +} + /* * The cs etm packet encodes an instruction range between a branch target * and the next taken branch. Generate sample accordingly. */ -static int cs_etm__synth_branch_sample(struct cs_etm_queue *etmq, - struct cs_etm_packet *packet) +static int cs_etm__synth_branch_sample(struct cs_etm_queue *etmq) { int ret = 0; struct cs_etm_auxtrace *etm = etmq->etm; struct perf_sample sample = {.ip = 0,}; union perf_event *event = etmq->event_buf; - u64 start_addr = packet->start_addr; - u64 end_addr = packet->end_addr; + struct dummy_branch_stack { + u64 nr; + struct branch_entry entries; + } dummy_bs; event->sample.header.type = PERF_RECORD_SAMPLE; event->sample.header.misc = PERF_RECORD_MISC_USER; event->sample.header.size = sizeof(struct perf_event_header); - sample.ip = start_addr; + sample.ip = cs_etm__last_executed_instr(etmq->prev_packet); sample.pid = etmq->pid; sample.tid = etmq->tid; - sample.addr = end_addr; + sample.addr = etmq->packet->start_addr; sample.id = etmq->etm->branches_id; sample.stream_id = etmq->etm->branches_id; sample.period = 1; - sample.cpu = packet->cpu; + sample.cpu = etmq->packet->cpu; sample.flags = 0; sample.cpumode = PERF_RECORD_MISC_USER; + /* + * perf report cannot handle events without a branch stack + */ + if (etm->synth_opts.last_branch) { + dummy_bs = (struct dummy_branch_stack){ + .nr = 1, + .entries = { + .from = sample.ip, + .to = sample.addr, + }, + }; + sample.branch_stack = (struct branch_stack *)&dummy_bs; + } + + if (etm->synth_opts.inject) { + ret = cs_etm__inject_event(event, &sample, + etm->branches_sample_type); + if (ret) + return ret; + } + ret = perf_session__deliver_synth_event(etm->session, event, &sample); if (ret) @@ -584,6 +828,24 @@ static int cs_etm__synth_events(struct cs_etm_auxtrace *etm, etm->sample_branches = true; etm->branches_sample_type = attr.sample_type; etm->branches_id = id; + id += 1; + attr.sample_type &= ~(u64)PERF_SAMPLE_ADDR; + } + + if (etm->synth_opts.last_branch) + attr.sample_type |= PERF_SAMPLE_BRANCH_STACK; + + if (etm->synth_opts.instructions) { + attr.config = PERF_COUNT_HW_INSTRUCTIONS; + attr.sample_period = etm->synth_opts.period; + etm->instructions_sample_period = attr.sample_period; + err = cs_etm__synth_event(session, &attr, id); + if (err) + return err; + etm->sample_instructions = true; + etm->instructions_sample_type = attr.sample_type; + etm->instructions_id = id; + id += 1; } return 0; @@ -591,20 +853,68 @@ static int cs_etm__synth_events(struct cs_etm_auxtrace *etm, static int cs_etm__sample(struct cs_etm_queue *etmq) { + struct cs_etm_auxtrace *etm = etmq->etm; + struct cs_etm_packet *tmp; int ret; - struct cs_etm_packet packet; + u64 instrs_executed; - while (1) { - ret = cs_etm_decoder__get_packet(etmq->decoder, &packet); - if (ret <= 0) + instrs_executed = cs_etm__instr_count(etmq->packet); + etmq->period_instructions += instrs_executed; + + /* + * Record a branch when the last instruction in + * PREV_PACKET is a branch. + */ + if (etm->synth_opts.last_branch && + etmq->prev_packet && + etmq->prev_packet->last_instr_taken_branch) + cs_etm__update_last_branch_rb(etmq); + + if (etm->sample_instructions && + etmq->period_instructions >= etm->instructions_sample_period) { + /* + * Emit instruction sample periodically + * TODO: allow period to be defined in cycles and clock time + */ + + /* Get number of instructions executed after the sample point */ + u64 instrs_over = etmq->period_instructions - + etm->instructions_sample_period; + + /* + * Calculate the address of the sampled instruction (-1 as + * sample is reported as though instruction has just been + * executed, but PC has not advanced to next instruction) + */ + u64 offset = (instrs_executed - instrs_over - 1); + u64 addr = cs_etm__instr_addr(etmq->packet, offset); + + ret = cs_etm__synth_instruction_sample( + etmq, addr, etm->instructions_sample_period); + if (ret) + return ret; + + /* Carry remaining instructions into next sample period */ + etmq->period_instructions = instrs_over; + } + + if (etm->sample_branches && + etmq->prev_packet && + etmq->prev_packet->sample_type == CS_ETM_RANGE && + etmq->prev_packet->last_instr_taken_branch) { + ret = cs_etm__synth_branch_sample(etmq); + if (ret) return ret; + } + if (etm->sample_branches || etm->synth_opts.last_branch) { /* - * If the packet contains an instruction range, generate an - * instruction sequence event. + * Swap PACKET with PREV_PACKET: PACKET becomes PREV_PACKET for + * the next incoming packet. */ - if (packet.sample_type & CS_ETM_RANGE) - cs_etm__synth_branch_sample(etmq, &packet); + tmp = etmq->packet; + etmq->packet = etmq->prev_packet; + etmq->prev_packet = tmp; } return 0; @@ -621,45 +931,73 @@ static int cs_etm__run_decoder(struct cs_etm_queue *etmq) etm->kernel_start = machine__kernel_start(etm->machine); /* Go through each buffer in the queue and decode them one by one */ -more: - buffer_used = 0; - memset(&buffer, 0, sizeof(buffer)); - err = cs_etm__get_trace(&buffer, etmq); - if (err <= 0) - return err; - /* - * We cannot assume consecutive blocks in the data file are contiguous, - * reset the decoder to force re-sync. - */ - err = cs_etm_decoder__reset(etmq->decoder); - if (err != 0) - return err; - - /* Run trace decoder until buffer consumed or end of trace */ - do { - processed = 0; - - err = cs_etm_decoder__process_data_block( - etmq->decoder, - etmq->offset, - &buffer.buf[buffer_used], - buffer.len - buffer_used, - &processed); - - if (err) + while (1) { + buffer_used = 0; + memset(&buffer, 0, sizeof(buffer)); + err = cs_etm__get_trace(&buffer, etmq); + if (err <= 0) + return err; + /* + * We cannot assume consecutive blocks in the data file are + * contiguous, reset the decoder to force re-sync. + */ + err = cs_etm_decoder__reset(etmq->decoder); + if (err != 0) return err; - etmq->offset += processed; - buffer_used += processed; + /* Run trace decoder until buffer consumed or end of trace */ + do { + processed = 0; + err = cs_etm_decoder__process_data_block( + etmq->decoder, + etmq->offset, + &buffer.buf[buffer_used], + buffer.len - buffer_used, + &processed); + if (err) + return err; + + etmq->offset += processed; + buffer_used += processed; + + /* Process each packet in this chunk */ + while (1) { + err = cs_etm_decoder__get_packet(etmq->decoder, + etmq->packet); + if (err <= 0) + /* + * Stop processing this chunk on + * end of data or error + */ + break; + + /* + * If the packet contains an instruction + * range, generate instruction sequence + * events. + */ + if (etmq->packet->sample_type & CS_ETM_RANGE) + err = cs_etm__sample(etmq); + } + } while (buffer.len > buffer_used); /* - * Nothing to do with an error condition, let's hope the next - * chunk will be better. + * Generate a last branch event for the branches left in + * the circular buffer at the end of the trace. */ - err = cs_etm__sample(etmq); - } while (buffer.len > buffer_used); + if (etm->sample_instructions && + etmq->etm->synth_opts.last_branch) { + struct branch_stack *bs = etmq->last_branch_rb; + struct branch_entry *be = + &bs->entries[etmq->last_branch_pos]; + + err = cs_etm__synth_instruction_sample( + etmq, be->to, etmq->period_instructions); + if (err) + return err; + } -goto more; + } return err; } -- cgit v1.2.3 From 256e751cac78739a4de2232450d3681b68b5845e Mon Sep 17 00:00:00 2001 From: Robert Walker Date: Wed, 14 Feb 2018 11:24:40 +0000 Subject: perf inject: Emit instruction records on ETM trace discontinuity There may be discontinuities in the ETM trace stream due to overflows or ETM configuration for selective trace. This patch emits an instruction sample with the pending branch stack when a TRACE ON packet occurs indicating a discontinuity in the trace data. A new packet type CS_ETM_TRACE_ON is added, which is emitted by the low level decoder when a TRACE ON occurs. The higher level decoder flushes the branch stack when this packet is emitted. Signed-off-by: Robert Walker Acked-by: Mathieu Poirier Cc: coresight@lists.linaro.org Cc: linux-arm-kernel@lists.infradead.org Link: http://lkml.kernel.org/r/1518607481-4059-3-git-send-email-robert.walker@arm.com Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/util/cs-etm-decoder/cs-etm-decoder.c | 9 +++ tools/perf/util/cs-etm-decoder/cs-etm-decoder.h | 1 + tools/perf/util/cs-etm.c | 80 ++++++++++++++++++------- 3 files changed, 67 insertions(+), 23 deletions(-) (limited to 'tools') diff --git a/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c b/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c index 8ff69dfd725a..640af88331b4 100644 --- a/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c +++ b/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c @@ -328,7 +328,14 @@ cs_etm_decoder__buffer_range(struct cs_etm_decoder *decoder, } return ret; +} +static ocsd_datapath_resp_t +cs_etm_decoder__buffer_trace_on(struct cs_etm_decoder *decoder, + const uint8_t trace_chan_id) +{ + return cs_etm_decoder__buffer_packet(decoder, trace_chan_id, + CS_ETM_TRACE_ON); } static ocsd_datapath_resp_t cs_etm_decoder__gen_trace_elem_printer( @@ -347,6 +354,8 @@ static ocsd_datapath_resp_t cs_etm_decoder__gen_trace_elem_printer( decoder->trace_on = false; break; case OCSD_GEN_TRC_ELEM_TRACE_ON: + resp = cs_etm_decoder__buffer_trace_on(decoder, + trace_chan_id); decoder->trace_on = true; break; case OCSD_GEN_TRC_ELEM_INSTR_RANGE: diff --git a/tools/perf/util/cs-etm-decoder/cs-etm-decoder.h b/tools/perf/util/cs-etm-decoder/cs-etm-decoder.h index a4fdd285b145..743f5f444304 100644 --- a/tools/perf/util/cs-etm-decoder/cs-etm-decoder.h +++ b/tools/perf/util/cs-etm-decoder/cs-etm-decoder.h @@ -24,6 +24,7 @@ struct cs_etm_buffer { enum cs_etm_sample_type { CS_ETM_RANGE = 1 << 0, + CS_ETM_TRACE_ON = 1 << 1, }; struct cs_etm_packet { diff --git a/tools/perf/util/cs-etm.c b/tools/perf/util/cs-etm.c index 6e595d96c04d..1b0d422373be 100644 --- a/tools/perf/util/cs-etm.c +++ b/tools/perf/util/cs-etm.c @@ -867,6 +867,7 @@ static int cs_etm__sample(struct cs_etm_queue *etmq) */ if (etm->synth_opts.last_branch && etmq->prev_packet && + etmq->prev_packet->sample_type == CS_ETM_RANGE && etmq->prev_packet->last_instr_taken_branch) cs_etm__update_last_branch_rb(etmq); @@ -920,6 +921,40 @@ static int cs_etm__sample(struct cs_etm_queue *etmq) return 0; } +static int cs_etm__flush(struct cs_etm_queue *etmq) +{ + int err = 0; + struct cs_etm_packet *tmp; + + if (etmq->etm->synth_opts.last_branch && + etmq->prev_packet && + etmq->prev_packet->sample_type == CS_ETM_RANGE) { + /* + * Generate a last branch event for the branches left in the + * circular buffer at the end of the trace. + * + * Use the address of the end of the last reported execution + * range + */ + u64 addr = cs_etm__last_executed_instr(etmq->prev_packet); + + err = cs_etm__synth_instruction_sample( + etmq, addr, + etmq->period_instructions); + etmq->period_instructions = 0; + + /* + * Swap PACKET with PREV_PACKET: PACKET becomes PREV_PACKET for + * the next incoming packet. + */ + tmp = etmq->packet; + etmq->packet = etmq->prev_packet; + etmq->prev_packet = tmp; + } + + return err; +} + static int cs_etm__run_decoder(struct cs_etm_queue *etmq) { struct cs_etm_auxtrace *etm = etmq->etm; @@ -971,32 +1006,31 @@ static int cs_etm__run_decoder(struct cs_etm_queue *etmq) */ break; - /* - * If the packet contains an instruction - * range, generate instruction sequence - * events. - */ - if (etmq->packet->sample_type & CS_ETM_RANGE) - err = cs_etm__sample(etmq); + switch (etmq->packet->sample_type) { + case CS_ETM_RANGE: + /* + * If the packet contains an instruction + * range, generate instruction sequence + * events. + */ + cs_etm__sample(etmq); + break; + case CS_ETM_TRACE_ON: + /* + * Discontinuity in trace, flush + * previous branch stack + */ + cs_etm__flush(etmq); + break; + default: + break; + } } } while (buffer.len > buffer_used); - /* - * Generate a last branch event for the branches left in - * the circular buffer at the end of the trace. - */ - if (etm->sample_instructions && - etmq->etm->synth_opts.last_branch) { - struct branch_stack *bs = etmq->last_branch_rb; - struct branch_entry *be = - &bs->entries[etmq->last_branch_pos]; - - err = cs_etm__synth_instruction_sample( - etmq, be->to, etmq->period_instructions); - if (err) - return err; - } - + if (err == 0) + /* Flush any remaining branch stack entries */ + err = cs_etm__flush(etmq); } return err; -- cgit v1.2.3 From fc2f52379b9d2232487d5e1dadb77883c10cdf47 Mon Sep 17 00:00:00 2001 From: Andi Kleen Date: Fri, 9 Feb 2018 08:39:09 -0800 Subject: perf report: Fix description for --mem-mode The "mem-loads" event only works when PEBS is enabled, so add the "/p" ("precise") suffix to the examples. Signed-off-by: Andi Kleen Cc: Jiri Olsa LPU-Reference: 20180209163909.9240-1-andi@firstfloor.org Link: https://lkml.kernel.org/n/tip-v0gcd4u9tktrvjjsp6y7ouv4@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/Documentation/perf-report.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'tools') diff --git a/tools/perf/Documentation/perf-report.txt b/tools/perf/Documentation/perf-report.txt index a76b871f78a6..cba16d8a970e 100644 --- a/tools/perf/Documentation/perf-report.txt +++ b/tools/perf/Documentation/perf-report.txt @@ -368,7 +368,7 @@ OPTIONS Use the data addresses of samples in addition to instruction addresses to build the histograms. To generate meaningful output, the perf.data file must have been obtained using perf record -d -W and using a - special event -e cpu/mem-loads/ or -e cpu/mem-stores/. See + special event -e cpu/mem-loads/p or -e cpu/mem-stores/p. See 'perf mem' for simpler access. --percent-limit:: -- cgit v1.2.3 From b40982e8468b46b8f7f5bba5a7e541ec04a29d7d Mon Sep 17 00:00:00 2001 From: Jin Yao Date: Mon, 29 Jan 2018 18:57:53 +0800 Subject: perf report: Fix wrong jump arrow MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit When we use perf report interactive annotate view, we can see the position of jump arrow is not correct. For example, 1. perf record -b ... 2. perf report 3. In interactive mode, select Annotate 'function' Percent│ IPC Cycle │ if (flag) 1.37 │0.4┌── 1 ↓ je 82 │ │ x += x / y + y / x; 0.00 │0.4│ 1310 movsd (%rsp),%xmm0 0.00 │0.4│ 565 movsd 0x8(%rsp),%xmm4 │0.4│ movsd 0x8(%rsp),%xmm1 │0.4│ movsd (%rsp),%xmm3 │0.4│ divsd %xmm4,%xmm0 0.00 │0.4│ 579 divsd %xmm3,%xmm1 │0.4│ movsd (%rsp),%xmm2 │0.4│ addsd %xmm1,%xmm0 │0.4│ addsd %xmm2,%xmm0 0.00 │0.4│ movsd %xmm0,(%rsp) │ │ volatile double x = 1212121212, y = 121212; │ │ │ │ s_randseed = time(0); │ │ srand(s_randseed); │ │ │ │ for (i = 0; i < 2000000000; i++) { 1.37 │0.4└─→ 82: sub $0x1,%ebx 28.21 │0.48 17 ↑ jne 38 The jump arrow in above example is not correct. It should add the width of IPC and Cycle. With this patch, the result is: Percent│ IPC Cycle │ if (flag) 1.37 │0.48 1 ┌──je 82 │ │ x += x / y + y / x; 0.00 │0.48 1310 │ movsd (%rsp),%xmm0 0.00 │0.48 565 │ movsd 0x8(%rsp),%xmm4 │0.48 │ movsd 0x8(%rsp),%xmm1 │0.48 │ movsd (%rsp),%xmm3 │0.48 │ divsd %xmm4,%xmm0 0.00 │0.48 579 │ divsd %xmm3,%xmm1 │0.48 │ movsd (%rsp),%xmm2 │0.48 │ addsd %xmm1,%xmm0 │0.48 │ addsd %xmm2,%xmm0 0.00 │0.48 │ movsd %xmm0,(%rsp) │ │ volatile double x = 1212121212, y = 121212; │ │ │ │ s_randseed = time(0); │ │ srand(s_randseed); │ │ │ │ for (i = 0; i < 2000000000; i++) { 1.37 │0.48 82:└─→sub $0x1,%ebx 28.21 │0.48 17 ↑ jne 38 Committer notes: Please note that only from LBRv5 (according to Jiri) onwards, i.e. >= Skylake is that we'll have the cycles counts in each branch record entry, so to see the Cycles and IPC columns, and be able to test this patch, one need a capable hardware. While applying this I first tested it on a Broadwell class machine and couldn't get those columns, will add code to the annotate browser to warn the user about that, i.e. you have branch records, but no cycles, use a more recent hardware to get the cycles and IPC columns. Signed-off-by: Jin Yao Cc: Alexander Shishkin Cc: Andi Kleen Cc: Jin Yao Cc: Jiri Olsa Cc: Kan Liang Cc: Peter Zijlstra Link: http://lkml.kernel.org/r/1517223473-14750-1-git-send-email-yao.jin@linux.intel.com Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/ui/browsers/annotate.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) (limited to 'tools') diff --git a/tools/perf/ui/browsers/annotate.c b/tools/perf/ui/browsers/annotate.c index 286427975112..e2f666391ac4 100644 --- a/tools/perf/ui/browsers/annotate.c +++ b/tools/perf/ui/browsers/annotate.c @@ -319,6 +319,7 @@ static void annotate_browser__draw_current_jump(struct ui_browser *browser) struct map_symbol *ms = ab->b.priv; struct symbol *sym = ms->sym; u8 pcnt_width = annotate_browser__pcnt_width(ab); + int width = 0; /* PLT symbols contain external offsets */ if (strstr(sym->name, "@plt")) @@ -340,13 +341,17 @@ static void annotate_browser__draw_current_jump(struct ui_browser *browser) to = (u64)btarget->idx; } + if (ab->have_cycles) + width = IPC_WIDTH + CYCLES_WIDTH; + ui_browser__set_color(browser, HE_COLORSET_JUMP_ARROWS); - __ui_browser__line_arrow(browser, pcnt_width + 2 + ab->addr_width, + __ui_browser__line_arrow(browser, + pcnt_width + 2 + ab->addr_width + width, from, to); if (is_fused(ab, cursor)) { ui_browser__mark_fused(browser, - pcnt_width + 3 + ab->addr_width, + pcnt_width + 3 + ab->addr_width + width, from - 1, to > from ? true : false); } -- cgit v1.2.3 From e3ebaa465136ecfedf9c6f4671df02bf625f8125 Mon Sep 17 00:00:00 2001 From: Jiri Olsa Date: Fri, 16 Feb 2018 13:36:19 +0100 Subject: perf report: Fix memory corruption in --branch-history mode --branch-history Jin Yao reported memory corrupton in perf report with branch info used for stack trace: > Following command lines will cause perf crash. > perf record -j call -g -a > perf report --branch-history > > *** Error in `perf': double free or corruption (!prev): 0x00000000104aa040 *** > ======= Backtrace: ========= > /lib/x86_64-linux-gnu/libc.so.6(+0x77725)[0x7f6b37254725] > /lib/x86_64-linux-gnu/libc.so.6(+0x7ff4a)[0x7f6b3725cf4a] > /lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7f6b37260abc] > perf[0x51b914] > perf(hist_entry_iter__add+0x1e5)[0x51f305] > perf[0x43cf01] > perf[0x4fa3bf] > perf[0x4fa923] > perf[0x4fd396] > perf[0x4f9614] > perf(perf_session__process_events+0x89e)[0x4fc38e] > perf(cmd_report+0x15d2)[0x43f202] > perf[0x4a059f] > perf(main+0x631)[0x427b71] > /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f6b371fd830] > perf(_start+0x29)[0x427d89] For the cumulative output, we allocate the he_cache array based on the --max-stack option value and populate it with data from 'callchain_cursor'. The --max-stack option value does not ensure now the limit for number of callchain_cursor nodes, so the cumulative iter code will allocate smaller array than it's actually needed and cause above corruption. I think the --max-stack limit does not apply here anyway, because we add callchain data as normal hist entries, while the --max-stack control the limit of single entry callchain depth. Using the callchain_cursor.nr as he_cache array count to fix this. Also removing struct hist_entry_iter::max_stack, because there's no longer any use for it. We need more fixes to ensure that the branch stack code follows properly the logic of --max-stack, which is not the case at the moment. Original-patch-by: Jin Yao Signed-off-by: Jiri Olsa Reported-by: Jin Yao Cc: Alexander Shishkin Cc: Andi Kleen Cc: Jiri Olsa Cc: Kan Liang Cc: Peter Zijlstra Link: http://lkml.kernel.org/r/20180216123619.GA9945@krava Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/util/hist.c | 4 +--- tools/perf/util/hist.h | 1 - 2 files changed, 1 insertion(+), 4 deletions(-) (limited to 'tools') diff --git a/tools/perf/util/hist.c b/tools/perf/util/hist.c index b6140950301e..44a8456cea10 100644 --- a/tools/perf/util/hist.c +++ b/tools/perf/util/hist.c @@ -879,7 +879,7 @@ iter_prepare_cumulative_entry(struct hist_entry_iter *iter, * cumulated only one time to prevent entries more than 100% * overhead. */ - he_cache = malloc(sizeof(*he_cache) * (iter->max_stack + 1)); + he_cache = malloc(sizeof(*he_cache) * (callchain_cursor.nr + 1)); if (he_cache == NULL) return -ENOMEM; @@ -1045,8 +1045,6 @@ int hist_entry_iter__add(struct hist_entry_iter *iter, struct addr_location *al, if (err) return err; - iter->max_stack = max_stack_depth; - err = iter->ops->prepare_entry(iter, al); if (err) goto out; diff --git a/tools/perf/util/hist.h b/tools/perf/util/hist.h index 02721b579746..e869cad4d89f 100644 --- a/tools/perf/util/hist.h +++ b/tools/perf/util/hist.h @@ -107,7 +107,6 @@ struct hist_entry_iter { int curr; bool hide_unresolved; - int max_stack; struct perf_evsel *evsel; struct perf_sample *sample; -- cgit v1.2.3 From 1350fb7d1b484afc1556579fb27400d036683453 Mon Sep 17 00:00:00 2001 From: Ravi Bangoria Date: Mon, 29 Jan 2018 14:04:15 +0530 Subject: tools include powerpc: Grab a copy of arch/powerpc/include/uapi/asm/unistd.h Will be used for generating the syscall id/string translation table. Committer notes: Update it already to catch with these csets applied since Ravi first submitted this patch: 3350eb2ea127 powerpc: sys_pkey_mprotect() system call 9499ec1b5e82 powerpc: sys_pkey_alloc() and sys_pkey_free() system calls So now 'perf trace' on ppc now knows about the pkey_ syscals. Signed-off-by: Ravi Bangoria Cc: Alexander Shishkin Cc: Hendrik Brueckner Cc: Jiri Olsa Cc: Michael Ellerman Cc: Namhyung Kim Cc: Thomas Richter Cc: linuxppc-dev@lists.ozlabs.org Link: http://lkml.kernel.org/r/20180129083417.31240-2-ravi.bangoria@linux.vnet.ibm.com Signed-off-by: Arnaldo Carvalho de Melo --- tools/arch/powerpc/include/uapi/asm/unistd.h | 402 +++++++++++++++++++++++++++ tools/perf/check-headers.sh | 1 + 2 files changed, 403 insertions(+) create mode 100644 tools/arch/powerpc/include/uapi/asm/unistd.h (limited to 'tools') diff --git a/tools/arch/powerpc/include/uapi/asm/unistd.h b/tools/arch/powerpc/include/uapi/asm/unistd.h new file mode 100644 index 000000000000..389c36fd8299 --- /dev/null +++ b/tools/arch/powerpc/include/uapi/asm/unistd.h @@ -0,0 +1,402 @@ +/* SPDX-License-Identifier: GPL-2.0+ WITH Linux-syscall-note */ +/* + * This file contains the system call numbers. + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License + * as published by the Free Software Foundation; either version + * 2 of the License, or (at your option) any later version. + */ +#ifndef _UAPI_ASM_POWERPC_UNISTD_H_ +#define _UAPI_ASM_POWERPC_UNISTD_H_ + + +#define __NR_restart_syscall 0 +#define __NR_exit 1 +#define __NR_fork 2 +#define __NR_read 3 +#define __NR_write 4 +#define __NR_open 5 +#define __NR_close 6 +#define __NR_waitpid 7 +#define __NR_creat 8 +#define __NR_link 9 +#define __NR_unlink 10 +#define __NR_execve 11 +#define __NR_chdir 12 +#define __NR_time 13 +#define __NR_mknod 14 +#define __NR_chmod 15 +#define __NR_lchown 16 +#define __NR_break 17 +#define __NR_oldstat 18 +#define __NR_lseek 19 +#define __NR_getpid 20 +#define __NR_mount 21 +#define __NR_umount 22 +#define __NR_setuid 23 +#define __NR_getuid 24 +#define __NR_stime 25 +#define __NR_ptrace 26 +#define __NR_alarm 27 +#define __NR_oldfstat 28 +#define __NR_pause 29 +#define __NR_utime 30 +#define __NR_stty 31 +#define __NR_gtty 32 +#define __NR_access 33 +#define __NR_nice 34 +#define __NR_ftime 35 +#define __NR_sync 36 +#define __NR_kill 37 +#define __NR_rename 38 +#define __NR_mkdir 39 +#define __NR_rmdir 40 +#define __NR_dup 41 +#define __NR_pipe 42 +#define __NR_times 43 +#define __NR_prof 44 +#define __NR_brk 45 +#define __NR_setgid 46 +#define __NR_getgid 47 +#define __NR_signal 48 +#define __NR_geteuid 49 +#define __NR_getegid 50 +#define __NR_acct 51 +#define __NR_umount2 52 +#define __NR_lock 53 +#define __NR_ioctl 54 +#define __NR_fcntl 55 +#define __NR_mpx 56 +#define __NR_setpgid 57 +#define __NR_ulimit 58 +#define __NR_oldolduname 59 +#define __NR_umask 60 +#define __NR_chroot 61 +#define __NR_ustat 62 +#define __NR_dup2 63 +#define __NR_getppid 64 +#define __NR_getpgrp 65 +#define __NR_setsid 66 +#define __NR_sigaction 67 +#define __NR_sgetmask 68 +#define __NR_ssetmask 69 +#define __NR_setreuid 70 +#define __NR_setregid 71 +#define __NR_sigsuspend 72 +#define __NR_sigpending 73 +#define __NR_sethostname 74 +#define __NR_setrlimit 75 +#define __NR_getrlimit 76 +#define __NR_getrusage 77 +#define __NR_gettimeofday 78 +#define __NR_settimeofday 79 +#define __NR_getgroups 80 +#define __NR_setgroups 81 +#define __NR_select 82 +#define __NR_symlink 83 +#define __NR_oldlstat 84 +#define __NR_readlink 85 +#define __NR_uselib 86 +#define __NR_swapon 87 +#define __NR_reboot 88 +#define __NR_readdir 89 +#define __NR_mmap 90 +#define __NR_munmap 91 +#define __NR_truncate 92 +#define __NR_ftruncate 93 +#define __NR_fchmod 94 +#define __NR_fchown 95 +#define __NR_getpriority 96 +#define __NR_setpriority 97 +#define __NR_profil 98 +#define __NR_statfs 99 +#define __NR_fstatfs 100 +#define __NR_ioperm 101 +#define __NR_socketcall 102 +#define __NR_syslog 103 +#define __NR_setitimer 104 +#define __NR_getitimer 105 +#define __NR_stat 106 +#define __NR_lstat 107 +#define __NR_fstat 108 +#define __NR_olduname 109 +#define __NR_iopl 110 +#define __NR_vhangup 111 +#define __NR_idle 112 +#define __NR_vm86 113 +#define __NR_wait4 114 +#define __NR_swapoff 115 +#define __NR_sysinfo 116 +#define __NR_ipc 117 +#define __NR_fsync 118 +#define __NR_sigreturn 119 +#define __NR_clone 120 +#define __NR_setdomainname 121 +#define __NR_uname 122 +#define __NR_modify_ldt 123 +#define __NR_adjtimex 124 +#define __NR_mprotect 125 +#define __NR_sigprocmask 126 +#define __NR_create_module 127 +#define __NR_init_module 128 +#define __NR_delete_module 129 +#define __NR_get_kernel_syms 130 +#define __NR_quotactl 131 +#define __NR_getpgid 132 +#define __NR_fchdir 133 +#define __NR_bdflush 134 +#define __NR_sysfs 135 +#define __NR_personality 136 +#define __NR_afs_syscall 137 /* Syscall for Andrew File System */ +#define __NR_setfsuid 138 +#define __NR_setfsgid 139 +#define __NR__llseek 140 +#define __NR_getdents 141 +#define __NR__newselect 142 +#define __NR_flock 143 +#define __NR_msync 144 +#define __NR_readv 145 +#define __NR_writev 146 +#define __NR_getsid 147 +#define __NR_fdatasync 148 +#define __NR__sysctl 149 +#define __NR_mlock 150 +#define __NR_munlock 151 +#define __NR_mlockall 152 +#define __NR_munlockall 153 +#define __NR_sched_setparam 154 +#define __NR_sched_getparam 155 +#define __NR_sched_setscheduler 156 +#define __NR_sched_getscheduler 157 +#define __NR_sched_yield 158 +#define __NR_sched_get_priority_max 159 +#define __NR_sched_get_priority_min 160 +#define __NR_sched_rr_get_interval 161 +#define __NR_nanosleep 162 +#define __NR_mremap 163 +#define __NR_setresuid 164 +#define __NR_getresuid 165 +#define __NR_query_module 166 +#define __NR_poll 167 +#define __NR_nfsservctl 168 +#define __NR_setresgid 169 +#define __NR_getresgid 170 +#define __NR_prctl 171 +#define __NR_rt_sigreturn 172 +#define __NR_rt_sigaction 173 +#define __NR_rt_sigprocmask 174 +#define __NR_rt_sigpending 175 +#define __NR_rt_sigtimedwait 176 +#define __NR_rt_sigqueueinfo 177 +#define __NR_rt_sigsuspend 178 +#define __NR_pread64 179 +#define __NR_pwrite64 180 +#define __NR_chown 181 +#define __NR_getcwd 182 +#define __NR_capget 183 +#define __NR_capset 184 +#define __NR_sigaltstack 185 +#define __NR_sendfile 186 +#define __NR_getpmsg 187 /* some people actually want streams */ +#define __NR_putpmsg 188 /* some people actually want streams */ +#define __NR_vfork 189 +#define __NR_ugetrlimit 190 /* SuS compliant getrlimit */ +#define __NR_readahead 191 +#ifndef __powerpc64__ /* these are 32-bit only */ +#define __NR_mmap2 192 +#define __NR_truncate64 193 +#define __NR_ftruncate64 194 +#define __NR_stat64 195 +#define __NR_lstat64 196 +#define __NR_fstat64 197 +#endif +#define __NR_pciconfig_read 198 +#define __NR_pciconfig_write 199 +#define __NR_pciconfig_iobase 200 +#define __NR_multiplexer 201 +#define __NR_getdents64 202 +#define __NR_pivot_root 203 +#ifndef __powerpc64__ +#define __NR_fcntl64 204 +#endif +#define __NR_madvise 205 +#define __NR_mincore 206 +#define __NR_gettid 207 +#define __NR_tkill 208 +#define __NR_setxattr 209 +#define __NR_lsetxattr 210 +#define __NR_fsetxattr 211 +#define __NR_getxattr 212 +#define __NR_lgetxattr 213 +#define __NR_fgetxattr 214 +#define __NR_listxattr 215 +#define __NR_llistxattr 216 +#define __NR_flistxattr 217 +#define __NR_removexattr 218 +#define __NR_lremovexattr 219 +#define __NR_fremovexattr 220 +#define __NR_futex 221 +#define __NR_sched_setaffinity 222 +#define __NR_sched_getaffinity 223 +/* 224 currently unused */ +#define __NR_tuxcall 225 +#ifndef __powerpc64__ +#define __NR_sendfile64 226 +#endif +#define __NR_io_setup 227 +#define __NR_io_destroy 228 +#define __NR_io_getevents 229 +#define __NR_io_submit 230 +#define __NR_io_cancel 231 +#define __NR_set_tid_address 232 +#define __NR_fadvise64 233 +#define __NR_exit_group 234 +#define __NR_lookup_dcookie 235 +#define __NR_epoll_create 236 +#define __NR_epoll_ctl 237 +#define __NR_epoll_wait 238 +#define __NR_remap_file_pages 239 +#define __NR_timer_create 240 +#define __NR_timer_settime 241 +#define __NR_timer_gettime 242 +#define __NR_timer_getoverrun 243 +#define __NR_timer_delete 244 +#define __NR_clock_settime 245 +#define __NR_clock_gettime 246 +#define __NR_clock_getres 247 +#define __NR_clock_nanosleep 248 +#define __NR_swapcontext 249 +#define __NR_tgkill 250 +#define __NR_utimes 251 +#define __NR_statfs64 252 +#define __NR_fstatfs64 253 +#ifndef __powerpc64__ +#define __NR_fadvise64_64 254 +#endif +#define __NR_rtas 255 +#define __NR_sys_debug_setcontext 256 +/* Number 257 is reserved for vserver */ +#define __NR_migrate_pages 258 +#define __NR_mbind 259 +#define __NR_get_mempolicy 260 +#define __NR_set_mempolicy 261 +#define __NR_mq_open 262 +#define __NR_mq_unlink 263 +#define __NR_mq_timedsend 264 +#define __NR_mq_timedreceive 265 +#define __NR_mq_notify 266 +#define __NR_mq_getsetattr 267 +#define __NR_kexec_load 268 +#define __NR_add_key 269 +#define __NR_request_key 270 +#define __NR_keyctl 271 +#define __NR_waitid 272 +#define __NR_ioprio_set 273 +#define __NR_ioprio_get 274 +#define __NR_inotify_init 275 +#define __NR_inotify_add_watch 276 +#define __NR_inotify_rm_watch 277 +#define __NR_spu_run 278 +#define __NR_spu_create 279 +#define __NR_pselect6 280 +#define __NR_ppoll 281 +#define __NR_unshare 282 +#define __NR_splice 283 +#define __NR_tee 284 +#define __NR_vmsplice 285 +#define __NR_openat 286 +#define __NR_mkdirat 287 +#define __NR_mknodat 288 +#define __NR_fchownat 289 +#define __NR_futimesat 290 +#ifdef __powerpc64__ +#define __NR_newfstatat 291 +#else +#define __NR_fstatat64 291 +#endif +#define __NR_unlinkat 292 +#define __NR_renameat 293 +#define __NR_linkat 294 +#define __NR_symlinkat 295 +#define __NR_readlinkat 296 +#define __NR_fchmodat 297 +#define __NR_faccessat 298 +#define __NR_get_robust_list 299 +#define __NR_set_robust_list 300 +#define __NR_move_pages 301 +#define __NR_getcpu 302 +#define __NR_epoll_pwait 303 +#define __NR_utimensat 304 +#define __NR_signalfd 305 +#define __NR_timerfd_create 306 +#define __NR_eventfd 307 +#define __NR_sync_file_range2 308 +#define __NR_fallocate 309 +#define __NR_subpage_prot 310 +#define __NR_timerfd_settime 311 +#define __NR_timerfd_gettime 312 +#define __NR_signalfd4 313 +#define __NR_eventfd2 314 +#define __NR_epoll_create1 315 +#define __NR_dup3 316 +#define __NR_pipe2 317 +#define __NR_inotify_init1 318 +#define __NR_perf_event_open 319 +#define __NR_preadv 320 +#define __NR_pwritev 321 +#define __NR_rt_tgsigqueueinfo 322 +#define __NR_fanotify_init 323 +#define __NR_fanotify_mark 324 +#define __NR_prlimit64 325 +#define __NR_socket 326 +#define __NR_bind 327 +#define __NR_connect 328 +#define __NR_listen 329 +#define __NR_accept 330 +#define __NR_getsockname 331 +#define __NR_getpeername 332 +#define __NR_socketpair 333 +#define __NR_send 334 +#define __NR_sendto 335 +#define __NR_recv 336 +#define __NR_recvfrom 337 +#define __NR_shutdown 338 +#define __NR_setsockopt 339 +#define __NR_getsockopt 340 +#define __NR_sendmsg 341 +#define __NR_recvmsg 342 +#define __NR_recvmmsg 343 +#define __NR_accept4 344 +#define __NR_name_to_handle_at 345 +#define __NR_open_by_handle_at 346 +#define __NR_clock_adjtime 347 +#define __NR_syncfs 348 +#define __NR_sendmmsg 349 +#define __NR_setns 350 +#define __NR_process_vm_readv 351 +#define __NR_process_vm_writev 352 +#define __NR_finit_module 353 +#define __NR_kcmp 354 +#define __NR_sched_setattr 355 +#define __NR_sched_getattr 356 +#define __NR_renameat2 357 +#define __NR_seccomp 358 +#define __NR_getrandom 359 +#define __NR_memfd_create 360 +#define __NR_bpf 361 +#define __NR_execveat 362 +#define __NR_switch_endian 363 +#define __NR_userfaultfd 364 +#define __NR_membarrier 365 +#define __NR_mlock2 378 +#define __NR_copy_file_range 379 +#define __NR_preadv2 380 +#define __NR_pwritev2 381 +#define __NR_kexec_file_load 382 +#define __NR_statx 383 +#define __NR_pkey_alloc 384 +#define __NR_pkey_free 385 +#define __NR_pkey_mprotect 386 + +#endif /* _UAPI_ASM_POWERPC_UNISTD_H_ */ diff --git a/tools/perf/check-headers.sh b/tools/perf/check-headers.sh index 790ec25919a0..bf206ffe5c45 100755 --- a/tools/perf/check-headers.sh +++ b/tools/perf/check-headers.sh @@ -42,6 +42,7 @@ arch/parisc/include/uapi/asm/errno.h arch/powerpc/include/uapi/asm/errno.h arch/sparc/include/uapi/asm/errno.h arch/x86/include/uapi/asm/errno.h +arch/powerpc/include/uapi/asm/unistd.h include/asm-generic/bitops/arch_hweight.h include/asm-generic/bitops/const_hweight.h include/asm-generic/bitops/__fls.h -- cgit v1.2.3 From 8e2ff72aa35e1a8a53894ae46751a83543338e14 Mon Sep 17 00:00:00 2001 From: Ravi Bangoria Date: Mon, 29 Jan 2018 14:04:16 +0530 Subject: perf powerpc: Generate system call table from asm/unistd.h This should speed up accessing new system calls introduced with the kernel rather than waiting for libaudit updates to include them. Signed-off-by: Ravi Bangoria Cc: Alexander Shishkin Cc: Hendrik Brueckner Cc: Jiri Olsa Cc: Michael Ellerman Cc: Namhyung Kim Cc: Thomas Richter Cc: linuxppc-dev@lists.ozlabs.org Link: http://lkml.kernel.org/r/20180129083417.31240-3-ravi.bangoria@linux.vnet.ibm.com [ Made it generate syscall_32.c as well to fix the build on 32-bit ppc ] Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/arch/powerpc/Makefile | 25 +++++++++++++++ .../perf/arch/powerpc/entry/syscalls/mksyscalltbl | 37 ++++++++++++++++++++++ 2 files changed, 62 insertions(+) create mode 100755 tools/perf/arch/powerpc/entry/syscalls/mksyscalltbl (limited to 'tools') diff --git a/tools/perf/arch/powerpc/Makefile b/tools/perf/arch/powerpc/Makefile index 42dab7c8f508..a111239df182 100644 --- a/tools/perf/arch/powerpc/Makefile +++ b/tools/perf/arch/powerpc/Makefile @@ -6,3 +6,28 @@ endif HAVE_KVM_STAT_SUPPORT := 1 PERF_HAVE_ARCH_REGS_QUERY_REGISTER_OFFSET := 1 PERF_HAVE_JITDUMP := 1 + +# +# Syscall table generation for perf +# + +out := $(OUTPUT)arch/powerpc/include/generated/asm +header32 := $(out)/syscalls_32.c +header64 := $(out)/syscalls_64.c +sysdef := $(srctree)/tools/arch/powerpc/include/uapi/asm/unistd.h +sysprf := $(srctree)/tools/perf/arch/powerpc/entry/syscalls/ +systbl := $(sysprf)/mksyscalltbl + +# Create output directory if not already present +_dummy := $(shell [ -d '$(out)' ] || mkdir -p '$(out)') + +$(header64): $(sysdef) $(systbl) + $(Q)$(SHELL) '$(systbl)' '64' '$(CC)' $(sysdef) > $@ + +$(header32): $(sysdef) $(systbl) + $(Q)$(SHELL) '$(systbl)' '32' '$(CC)' $(sysdef) > $@ + +clean:: + $(call QUIET_CLEAN, powerpc) $(RM) $(header32) $(header64) + +archheaders: $(header32) $(header64) diff --git a/tools/perf/arch/powerpc/entry/syscalls/mksyscalltbl b/tools/perf/arch/powerpc/entry/syscalls/mksyscalltbl new file mode 100755 index 000000000000..ef52e1dd694b --- /dev/null +++ b/tools/perf/arch/powerpc/entry/syscalls/mksyscalltbl @@ -0,0 +1,37 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# +# Generate system call table for perf. Derived from +# s390 script. +# +# Copyright IBM Corp. 2017 +# Author(s): Hendrik Brueckner +# Changed by: Ravi Bangoria + +wordsize=$1 +gcc=$2 +input=$3 + +if ! test -r $input; then + echo "Could not read input file" >&2 + exit 1 +fi + +create_table() +{ + local wordsize=$1 + local max_nr + + echo "static const char *syscalltbl_powerpc_${wordsize}[] = {" + while read sc nr; do + printf '\t[%d] = "%s",\n' $nr $sc + max_nr=$nr + done + echo '};' + echo "#define SYSCALLTBL_POWERPC_${wordsize}_MAX_ID $max_nr" +} + +$gcc -m${wordsize} -E -dM -x c $input \ + |sed -ne 's/^#define __NR_//p' \ + |sort -t' ' -k2 -nu \ + |create_table ${wordsize} -- cgit v1.2.3 From 4281da235e3de91bb8deae44bc6506336ceaa88a Mon Sep 17 00:00:00 2001 From: Ravi Bangoria Date: Mon, 29 Jan 2018 14:04:17 +0530 Subject: perf trace powerpc: Use generated syscall table This should speed up accessing new system calls introduced with the kernel rather than waiting for libaudit updates to include them. It also enables users to specify wildcards, for example, perf trace -e 'open*', just like was already possible on x86 and s390. Signed-off-by: Ravi Bangoria Cc: Alexander Shishkin Cc: Hendrik Brueckner Cc: Jiri Olsa Cc: Michael Ellerman Cc: Namhyung Kim Cc: Thomas Richter Cc: linuxppc-dev@lists.ozlabs.org Link: http://lkml.kernel.org/r/20180129083417.31240-4-ravi.bangoria@linux.vnet.ibm.com [ Do it for ppc32 as well ] Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/Makefile.config | 2 ++ tools/perf/util/syscalltbl.c | 8 ++++++++ 2 files changed, 10 insertions(+) (limited to 'tools') diff --git a/tools/perf/Makefile.config b/tools/perf/Makefile.config index 0dfdaa9fa81e..577a5d2988fe 100644 --- a/tools/perf/Makefile.config +++ b/tools/perf/Makefile.config @@ -27,6 +27,8 @@ NO_SYSCALL_TABLE := 1 # Additional ARCH settings for ppc ifeq ($(SRCARCH),powerpc) NO_PERF_REGS := 0 + NO_SYSCALL_TABLE := 0 + CFLAGS += -I$(OUTPUT)arch/powerpc/include/generated LIBUNWIND_LIBS := -lunwind -lunwind-ppc64 endif diff --git a/tools/perf/util/syscalltbl.c b/tools/perf/util/syscalltbl.c index 303bdb84ab5a..895122d638dd 100644 --- a/tools/perf/util/syscalltbl.c +++ b/tools/perf/util/syscalltbl.c @@ -30,6 +30,14 @@ static const char **syscalltbl_native = syscalltbl_x86_64; #include const int syscalltbl_native_max_id = SYSCALLTBL_S390_64_MAX_ID; static const char **syscalltbl_native = syscalltbl_s390_64; +#elif defined(__powerpc64__) +#include +const int syscalltbl_native_max_id = SYSCALLTBL_POWERPC_64_MAX_ID; +static const char **syscalltbl_native = syscalltbl_powerpc_64; +#elif defined(__powerpc__) +#include +const int syscalltbl_native_max_id = SYSCALLTBL_POWERPC_32_MAX_ID; +static const char **syscalltbl_native = syscalltbl_powerpc_32; #endif struct syscall { -- cgit v1.2.3 From eca0fa28cd0df7369701dbee0e30ddce19c039b8 Mon Sep 17 00:00:00 2001 From: Thomas Richter Date: Tue, 13 Feb 2018 16:14:16 +0100 Subject: perf record: Provide detailed information on s390 CPU When perf record ... is setup to record data, the s390 cpu information was a fixed string "IBM/S390". Replace this string with one containing more information about the machine. The information included in the cpuid is a comma separated list: manufacturer,type,model-capacity,model[,version,authorization] with - manufacturer: up to 16 byte name of the manufacturer (IBM). - type: a four digit number refering to the machine generation. - model-capacitiy: up to 16 characters describing number of cpus etc. - model: up to 16 characters describing model. - version: the CPU-MF counter facility version number, available on LPARs only, omitted on z/VM guests. - authorization: the CPU-MF counter facility authorization level, available on LPARs only, omitted on z/VM guests. Before: [root@s8360047 perf]# ./perf record -- sleep 1 [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.001 MB perf.data (4 samples) ] [root@s8360047 perf]# ./perf report --header | fgrep cpuid # cpuid : IBM/S390 [root@s8360047 perf]# After: [root@s35lp76 perf]# ./perf report --header|fgrep cpuid # cpuid : IBM,3906,704,M03,3.5,002f [root@s35lp76 perf]# Signed-off-by: Thomas Richter Reviewed-by: Hendrik Brueckner Cc: Heiko Carstens Cc: Martin Schwidefsky Link: http://lkml.kernel.org/r/20180213151419.80737-1-tmricht@linux.vnet.ibm.com [ Use scnprintf instead of strncat to fix build errors on gcc GNU C99 5.4.0 20160609 -march=zEC12 -m64 -mzarch -ggdb3 -O6 -std=gnu99 -fPIC -fno-omit-frame-pointer -funwind-tables -fstack-protector-all ] Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/arch/s390/util/header.c | 130 +++++++++++++++++++++++++++++++++++-- 1 file changed, 125 insertions(+), 5 deletions(-) (limited to 'tools') diff --git a/tools/perf/arch/s390/util/header.c b/tools/perf/arch/s390/util/header.c index 9fa6c3e5782c..a78064c25ced 100644 --- a/tools/perf/arch/s390/util/header.c +++ b/tools/perf/arch/s390/util/header.c @@ -1,8 +1,9 @@ /* * Implementation of get_cpuid(). * - * Copyright 2014 IBM Corp. + * Copyright IBM Corp. 2014, 2018 * Author(s): Alexander Yarygin + * Thomas Richter * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License (version 2 only) @@ -13,16 +14,135 @@ #include #include #include +#include #include "../../util/header.h" +#include "../../util/util.h" + +#define SYSINFO_MANU "Manufacturer:" +#define SYSINFO_TYPE "Type:" +#define SYSINFO_MODEL "Model:" +#define SRVLVL_CPUMF "CPU-MF:" +#define SRVLVL_VERSION "version=" +#define SRVLVL_AUTHORIZATION "authorization=" +#define SYSINFO "/proc/sysinfo" +#define SRVLVL "/proc/service_levels" int get_cpuid(char *buffer, size_t sz) { - const char *cpuid = "IBM/S390"; + char *cp, *line = NULL, *line2; + char type[8], model[33], version[8], manufacturer[32], authorization[8]; + int tpsize = 0, mdsize = 0, vssize = 0, mfsize = 0, atsize = 0; + int read; + unsigned long line_sz; + size_t nbytes; + FILE *sysinfo; + + /* + * Scan /proc/sysinfo line by line and read out values for + * Manufacturer:, Type: and Model:, for example: + * Manufacturer: IBM + * Type: 2964 + * Model: 702 N96 + * The first word is the Model Capacity and the second word is + * Model (can be omitted). Both words have a maximum size of 16 + * bytes. + */ + memset(manufacturer, 0, sizeof(manufacturer)); + memset(type, 0, sizeof(type)); + memset(model, 0, sizeof(model)); + memset(version, 0, sizeof(version)); + memset(authorization, 0, sizeof(authorization)); + + sysinfo = fopen(SYSINFO, "r"); + if (sysinfo == NULL) + return -1; + + while ((read = getline(&line, &line_sz, sysinfo)) != -1) { + if (!strncmp(line, SYSINFO_MANU, strlen(SYSINFO_MANU))) { + line2 = line + strlen(SYSINFO_MANU); + + while ((cp = strtok_r(line2, "\n ", &line2))) { + mfsize += scnprintf(manufacturer + mfsize, + sizeof(manufacturer) - mfsize, "%s", cp); + } + } + + if (!strncmp(line, SYSINFO_TYPE, strlen(SYSINFO_TYPE))) { + line2 = line + strlen(SYSINFO_TYPE); + + while ((cp = strtok_r(line2, "\n ", &line2))) { + tpsize += scnprintf(type + tpsize, + sizeof(type) - tpsize, "%s", cp); + } + } + + if (!strncmp(line, SYSINFO_MODEL, strlen(SYSINFO_MODEL))) { + line2 = line + strlen(SYSINFO_MODEL); + + while ((cp = strtok_r(line2, "\n ", &line2))) { + mdsize += scnprintf(model + mdsize, sizeof(type) - mdsize, + "%s%s", model[0] ? "," : "", cp); + } + break; + } + } + fclose(sysinfo); - if (strlen(cpuid) + 1 > sz) + /* Missing manufacturer, type or model information should not happen */ + if (!manufacturer[0] || !type[0] || !model[0]) return -1; - strcpy(buffer, cpuid); - return 0; + /* + * Scan /proc/service_levels and return the CPU-MF counter facility + * version number and authorization level. + * Optional, does not exist on z/VM guests. + */ + sysinfo = fopen(SRVLVL, "r"); + if (sysinfo == NULL) + goto skip_sysinfo; + while ((read = getline(&line, &line_sz, sysinfo)) != -1) { + if (strncmp(line, SRVLVL_CPUMF, strlen(SRVLVL_CPUMF))) + continue; + + line2 = line + strlen(SRVLVL_CPUMF); + while ((cp = strtok_r(line2, "\n ", &line2))) { + if (!strncmp(cp, SRVLVL_VERSION, + strlen(SRVLVL_VERSION))) { + char *sep = strchr(cp, '='); + + vssize += scnprintf(version + vssize, + sizeof(version) - vssize, "%s", sep + 1); + } + if (!strncmp(cp, SRVLVL_AUTHORIZATION, + strlen(SRVLVL_AUTHORIZATION))) { + char *sep = strchr(cp, '='); + + atsize += scnprintf(authorization + atsize, + sizeof(authorization) - atsize, "%s", sep + 1); + } + } + } + fclose(sysinfo); + +skip_sysinfo: + free(line); + + if (version[0] && authorization[0] ) + nbytes = snprintf(buffer, sz, "%s,%s,%s,%s,%s", + manufacturer, type, model, version, + authorization); + else + nbytes = snprintf(buffer, sz, "%s,%s,%s", manufacturer, type, + model); + return (nbytes >= sz) ? -1 : 0; +} + +char *get_cpuid_str(struct perf_pmu *pmu __maybe_unused) +{ + char *buf = malloc(128); + + if (buf && get_cpuid(buf, 128) < 0) + zfree(&buf); + return buf; } -- cgit v1.2.3 From c59124fa59757fadc80ad881056a21f98c71b146 Mon Sep 17 00:00:00 2001 From: Thomas Richter Date: Tue, 13 Feb 2018 16:14:17 +0100 Subject: perf annotate: Scan cpuid for s390 and save machine type Scan the cpuid string and extract the type number for later use. Signed-off-by: Thomas Richter Reviewed-by: Hendrik Brueckner Cc: Heiko Carstens Cc: Martin Schwidefsky Link: http://lkml.kernel.org/r/20180213151419.80737-2-tmricht@linux.vnet.ibm.com Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/arch/s390/annotate/instructions.c | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) (limited to 'tools') diff --git a/tools/perf/arch/s390/annotate/instructions.c b/tools/perf/arch/s390/annotate/instructions.c index 8c72b44444cb..01df9d8303e1 100644 --- a/tools/perf/arch/s390/annotate/instructions.c +++ b/tools/perf/arch/s390/annotate/instructions.c @@ -23,12 +23,37 @@ static struct ins_ops *s390__associate_ins_ops(struct arch *arch, const char *na return ops; } +static int s390__cpuid_parse(struct arch *arch, char *cpuid) +{ + unsigned int family; + char model[16], model_c[16], cpumf_v[16], cpumf_a[16]; + int ret; + + /* + * cpuid string format: + * "IBM,family,model-capacity,model[,cpum_cf-version,cpum_cf-authorization]" + */ + ret = sscanf(cpuid, "%*[^,],%u,%[^,],%[^,],%[^,],%s", &family, model_c, + model, cpumf_v, cpumf_a); + if (ret >= 2) { + arch->family = family; + arch->model = 0; + return 0; + } + + return -1; +} + static int s390__annotate_init(struct arch *arch, char *cpuid __maybe_unused) { + int err = 0; + if (!arch->initialized) { arch->initialized = true; arch->associate_instruction_ops = s390__associate_ins_ops; + if (cpuid) + err = s390__cpuid_parse(arch, cpuid); } - return 0; + return err; } -- cgit v1.2.3 From 4cb7d3ecfca90684ad00f893c34a2028fcc5f764 Mon Sep 17 00:00:00 2001 From: Thomas Richter Date: Tue, 13 Feb 2018 16:14:18 +0100 Subject: perf cpuid: Introduce a platform specific cpuid compare function The function get_cpuid_str() is called by perf_pmu__getcpuid() and on s390 returns a complete description of the CPU and its capabilities, which is a comma separated list. To map the CPU type with the value defined in the pmu-events/arch/s390/mapfile.csv, introduce an architecture specific cpuid compare function named strcmp_cpuid_str() The currently used regex algorithm is defined as the weak default and will be used if no platform specific one is defined. This matches the current behavior. Signed-off-by: Thomas Richter Reviewed-by: Hendrik Brueckner Cc: Heiko Carstens Cc: Martin Schwidefsky Link: http://lkml.kernel.org/r/20180213151419.80737-3-tmricht@linux.vnet.ibm.com Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/arch/s390/util/header.c | 18 +++++++++++++++ tools/perf/util/header.h | 1 + tools/perf/util/pmu.c | 47 +++++++++++++++++++++++--------------- 3 files changed, 48 insertions(+), 18 deletions(-) (limited to 'tools') diff --git a/tools/perf/arch/s390/util/header.c b/tools/perf/arch/s390/util/header.c index a78064c25ced..231294b80dc4 100644 --- a/tools/perf/arch/s390/util/header.c +++ b/tools/perf/arch/s390/util/header.c @@ -146,3 +146,21 @@ char *get_cpuid_str(struct perf_pmu *pmu __maybe_unused) zfree(&buf); return buf; } + +/* + * Compare the cpuid string returned by get_cpuid() function + * with the name generated by the jevents file read from + * pmu-events/arch/s390/mapfile.csv. + * + * Parameter mapcpuid is the cpuid as stored in the + * pmu-events/arch/s390/mapfile.csv. This is just the type number. + * Parameter cpuid is the cpuid returned by function get_cpuid(). + */ +int strcmp_cpuid_str(const char *mapcpuid, const char *cpuid) +{ + char *cp = strchr(cpuid, ','); + + if (cp == NULL) + return -1; + return strncmp(cp + 1, mapcpuid, strlen(mapcpuid)); +} diff --git a/tools/perf/util/header.h b/tools/perf/util/header.h index f28aaaa3a440..942bdec6d70d 100644 --- a/tools/perf/util/header.h +++ b/tools/perf/util/header.h @@ -174,4 +174,5 @@ int write_padded(struct feat_fd *fd, const void *bf, int get_cpuid(char *buffer, size_t sz); char *get_cpuid_str(struct perf_pmu *pmu __maybe_unused); +int strcmp_cpuid_str(const char *s1, const char *s2); #endif /* __PERF_HEADER_H */ diff --git a/tools/perf/util/pmu.c b/tools/perf/util/pmu.c index 57e38fdf0b34..1111d5bf15ca 100644 --- a/tools/perf/util/pmu.c +++ b/tools/perf/util/pmu.c @@ -576,6 +576,34 @@ char * __weak get_cpuid_str(struct perf_pmu *pmu __maybe_unused) return NULL; } +/* Return zero when the cpuid from the mapfile.csv matches the + * cpuid string generated on this platform. + * Otherwise return non-zero. + */ +int __weak strcmp_cpuid_str(const char *mapcpuid, const char *cpuid) +{ + regex_t re; + regmatch_t pmatch[1]; + int match; + + if (regcomp(&re, mapcpuid, REG_EXTENDED) != 0) { + /* Warn unable to generate match particular string. */ + pr_info("Invalid regular expression %s\n", mapcpuid); + return 1; + } + + match = !regexec(&re, cpuid, 1, pmatch, 0); + regfree(&re); + if (match) { + size_t match_len = (pmatch[0].rm_eo - pmatch[0].rm_so); + + /* Verify the entire string matched. */ + if (match_len == strlen(cpuid)) + return 0; + } + return 1; +} + static char *perf_pmu__getcpuid(struct perf_pmu *pmu) { char *cpuid; @@ -610,31 +638,14 @@ struct pmu_events_map *perf_pmu__find_map(struct perf_pmu *pmu) i = 0; for (;;) { - regex_t re; - regmatch_t pmatch[1]; - int match; - map = &pmu_events_map[i++]; if (!map->table) { map = NULL; break; } - if (regcomp(&re, map->cpuid, REG_EXTENDED) != 0) { - /* Warn unable to generate match particular string. */ - pr_info("Invalid regular expression %s\n", map->cpuid); + if (!strcmp_cpuid_str(map->cpuid, cpuid)) break; - } - - match = !regexec(&re, cpuid, 1, pmatch, 0); - regfree(&re); - if (match) { - size_t match_len = (pmatch[0].rm_eo - pmatch[0].rm_so); - - /* Verify the entire string matched. */ - if (match_len == strlen(cpuid)) - break; - } } free(cpuid); return map; -- cgit v1.2.3 From b3be39c51cc58eb60c698cf64987e9d465a9263a Mon Sep 17 00:00:00 2001 From: Thomas Richter Date: Tue, 13 Feb 2018 16:14:19 +0100 Subject: perf test: Fix test case 23 for s390 z/VM or KVM guests On s390 perf can be executed on a LPAR with support for hardware events (i. e. cycles) or on a z/VM or KVM guest where no hardware events are supported. In this environment use software event named cpu-clock for this test case. Use the cpuid infrastructure functions to determine the cpuid on s390 which contains an indication of the cpu counter facility availability. Signed-off-by: Thomas Richter Reviewed-by: Hendrik Brueckner Cc: Heiko Carstens Cc: Martin Schwidefsky Link: http://lkml.kernel.org/r/20180213151419.80737-4-tmricht@linux.vnet.ibm.com Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/tests/code-reading.c | 33 +++++++++++++++++++++++++++++---- 1 file changed, 29 insertions(+), 4 deletions(-) (limited to 'tools') diff --git a/tools/perf/tests/code-reading.c b/tools/perf/tests/code-reading.c index 3bf7b145b826..c7115d369511 100644 --- a/tools/perf/tests/code-reading.c +++ b/tools/perf/tests/code-reading.c @@ -482,6 +482,34 @@ static void fs_something(void) } } +static const char *do_determine_event(bool excl_kernel) +{ + const char *event = excl_kernel ? "cycles:u" : "cycles"; + +#ifdef __s390x__ + char cpuid[128], model[16], model_c[16], cpum_cf_v[16]; + unsigned int family; + int ret, cpum_cf_a; + + if (get_cpuid(cpuid, sizeof(cpuid))) + goto out_clocks; + ret = sscanf(cpuid, "%*[^,],%u,%[^,],%[^,],%[^,],%x", &family, model_c, + model, cpum_cf_v, &cpum_cf_a); + if (ret != 5) /* Not available */ + goto out_clocks; + if (excl_kernel && (cpum_cf_a & 4)) + return event; + if (!excl_kernel && (cpum_cf_a & 2)) + return event; + + /* Fall through: missing authorization */ +out_clocks: + event = excl_kernel ? "cpu-clock:u" : "cpu-clock"; + +#endif + return event; +} + static void do_something(void) { fs_something(); @@ -592,10 +620,7 @@ static int do_test_code_reading(bool try_kcore) perf_evlist__set_maps(evlist, cpus, threads); - if (excl_kernel) - str = "cycles:u"; - else - str = "cycles"; + str = do_determine_event(excl_kernel); pr_debug("Parsing event '%s'\n", str); ret = parse_events(evlist, str, NULL); if (ret < 0) { -- cgit v1.2.3 From 0f19a038afdc592176c9a302f0d08be6a68ad74a Mon Sep 17 00:00:00 2001 From: Thomas Richter Date: Wed, 14 Feb 2018 08:03:03 +0100 Subject: perf test: Fix test case inet_pton to accept inlines. Using Fedora 27 and latest Linux kernel the test case trace+probe_libc_inet_pton.sh fails again on s390. This time is the inlining of functions which does not match. After an update of the glibc (from 2.26-16 to 2.26-24) the output is different The expected output is: __inet_pton (/usr/lib64/libc-2.26.so) gaih_inet (inlined) .... The actual output is: 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.061/0.061/0.061/0.000 ms 0.000 probe_libc:inet_pton:(3ffb2140448)) __inet_pton (inlined) gaih_inet.constprop.7 (/usr/lib64/libc-2.26.so) ... Fix this by being less strict on 'inlined' verses library name and accept both Signed-off-by: Thomas Richter Cc: Heiko Carstens Cc: Hendrik Brueckner Cc: Martin Schwidefsky Link: http://lkml.kernel.org/r/20180214070303.55757-1-tmricht@linux.vnet.ibm.com Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/tests/shell/trace+probe_libc_inet_pton.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) (limited to 'tools') diff --git a/tools/perf/tests/shell/trace+probe_libc_inet_pton.sh b/tools/perf/tests/shell/trace+probe_libc_inet_pton.sh index c446c894b297..8c4ab0b390c0 100755 --- a/tools/perf/tests/shell/trace+probe_libc_inet_pton.sh +++ b/tools/perf/tests/shell/trace+probe_libc_inet_pton.sh @@ -21,12 +21,12 @@ trace_libc_inet_pton_backtrace() { expected[3]=".*packets transmitted.*" expected[4]="rtt min.*" expected[5]="[0-9]+\.[0-9]+[[:space:]]+probe_libc:inet_pton:\([[:xdigit:]]+\)" - expected[6]=".*inet_pton[[:space:]]\($libc\)$" + expected[6]=".*inet_pton[[:space:]]\($libc|inlined\)$" case "$(uname -m)" in s390x) eventattr='call-graph=dwarf' - expected[7]="gaih_inet[[:space:]]\(inlined\)$" - expected[8]="__GI_getaddrinfo[[:space:]]\(inlined\)$" + expected[7]="gaih_inet.*[[:space:]]\($libc|inlined\)$" + expected[8]="__GI_getaddrinfo[[:space:]]\($libc|inlined\)$" expected[9]="main[[:space:]]\(.*/bin/ping.*\)$" expected[10]="__libc_start_main[[:space:]]\($libc\)$" expected[11]="_start[[:space:]]\(.*/bin/ping.*\)$" -- cgit v1.2.3 From 21316ac6803d4a1aadd74b896db8d60a92cd1140 Mon Sep 17 00:00:00 2001 From: Arnaldo Carvalho de Melo Date: Fri, 16 Feb 2018 15:26:51 -0300 Subject: perf tests shell lib: Use a wildcard to remove the vfs_getname probe In some situations the vfs_getname is being added both as requested and with a _1 suffix (inlines?): probe:vfs_getname_1 (on getname_flags:63@acme/git/linux/fs/namei.c with pathname) This ends up making the cleanup to miss that one, as it removes just 'probe:vfs_getname', which makes the second test to use this probe point to fail, since it finds that leftover from the first test, use a wildcard to remove both. Before: # perf test 60 61 62 63 60: Use vfs_getname probe to get syscall args filenames : FAILED! 61: probe libc's inet_pton & backtrace it with ping : Ok 62: Check open filename arg using perf trace + vfs_getname: FAILED! 63: Add vfs_getname probe to get syscall args filenames : Ok After: # perf test 60 61 62 63 60: Use vfs_getname probe to get syscall args filenames : Ok 61: probe libc's inet_pton & backtrace it with ping : Ok 62: Check open filename arg using perf trace + vfs_getname: Ok 63: Add vfs_getname probe to get syscall args filenames : Ok # Cc: Adrian Hunter Cc: David Ahern Cc: Jiri Olsa Cc: Masami Hiramatsu Cc: Namhyung Kim Cc: Thomas Richter Cc: Wang Nan Link: https://lkml.kernel.org/n/tip-2k5kutwr4ds36adiakyb4yvy@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/tests/shell/lib/probe_vfs_getname.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'tools') diff --git a/tools/perf/tests/shell/lib/probe_vfs_getname.sh b/tools/perf/tests/shell/lib/probe_vfs_getname.sh index 30a950c9d407..1c16e56cd93e 100644 --- a/tools/perf/tests/shell/lib/probe_vfs_getname.sh +++ b/tools/perf/tests/shell/lib/probe_vfs_getname.sh @@ -5,7 +5,7 @@ had_vfs_getname=$? cleanup_probe_vfs_getname() { if [ $had_vfs_getname -eq 1 ] ; then - perf probe -q -d probe:vfs_getname + perf probe -q -d probe:vfs_getname* fi } -- cgit v1.2.3 From 47812e00910407a0f14906c0a3bf2d803a616c6f Mon Sep 17 00:00:00 2001 From: Thomas Richter Date: Mon, 19 Feb 2018 11:24:44 +0100 Subject: perf s390: Fix reading cpuid model information Commit eca0fa28cd0d (perf record: Provide detailed information on s390 CPU") fixed a build error on Ubuntu. However the fix uses the wrong size to print the model information. Signed-off-by: Thomas Richter Cc: Heiko Carstens Cc: Hendrik Brueckner Cc: Martin Schwidefsky Fixes: eca0fa28cd0d ("perf record: Provide detailed information on s390 CPU") Link: http://lkml.kernel.org/r/20180219102444.96900-1-tmricht@linux.vnet.ibm.com Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/arch/s390/util/header.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'tools') diff --git a/tools/perf/arch/s390/util/header.c b/tools/perf/arch/s390/util/header.c index 231294b80dc4..a4c30f1c70be 100644 --- a/tools/perf/arch/s390/util/header.c +++ b/tools/perf/arch/s390/util/header.c @@ -81,7 +81,7 @@ int get_cpuid(char *buffer, size_t sz) line2 = line + strlen(SYSINFO_MODEL); while ((cp = strtok_r(line2, "\n ", &line2))) { - mdsize += scnprintf(model + mdsize, sizeof(type) - mdsize, + mdsize += scnprintf(model + mdsize, sizeof(model) - mdsize, "%s%s", model[0] ? "," : "", cp); } break; -- cgit v1.2.3 From 1d12cec6ce99614297e10945d917fd8a62cd2b09 Mon Sep 17 00:00:00 2001 From: Namhyung Kim Date: Mon, 19 Feb 2018 19:00:46 +0900 Subject: perf machine: Fix paranoid check in machine__set_kernel_mmap() The machine__set_kernel_mmap() is to setup addresses of the kernel map using external info. But it has a check when the address is given from an incorrect input which should have the start and end address of 0 (i.e. machine__process_kernel_mmap_event). But we also use the end address of 0 for a valid input so change it to check both start and end addresses. Signed-off-by: Namhyung Kim Acked-by: Jiri Olsa Cc: Alexander Shishkin Cc: David Ahern Cc: Peter Zijlstra Cc: kernel-team@lge.com Link: http://lkml.kernel.org/r/20180219101936.GD1583@sejong Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/util/machine.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'tools') diff --git a/tools/perf/util/machine.c b/tools/perf/util/machine.c index fe27ef55cbb9..12b7427444a3 100644 --- a/tools/perf/util/machine.c +++ b/tools/perf/util/machine.c @@ -1226,7 +1226,7 @@ static void machine__set_kernel_mmap(struct machine *machine, * Be a bit paranoid here, some perf.data file came with * a zero sized synthesized MMAP event for the kernel. */ - if (machine->vmlinux_maps[i]->end == 0) + if (start == 0 && end == 0) machine->vmlinux_maps[i]->end = ~0ULL; } } -- cgit v1.2.3 From 63cd02d84be5f7b3bc4f8fbb93cc1f871f84ae1d Mon Sep 17 00:00:00 2001 From: Changbin Du Date: Mon, 19 Feb 2018 10:33:29 +0800 Subject: perf ftrace: Append an EOL when write tracing files Before this change, the '--graph-funcs', '--nograph-funcs' and '--trace-funcs' options didn't work as expected when the doesn't exist. Because the kernel side hid possible errors. $ sudo ./perf ftrace -a --graph-depth 1 --graph-funcs abcdefg 0) 0.140 us | rcu_all_qs(); 3) 0.304 us | mutex_unlock(); 0) 0.153 us | find_vma(); 3) 0.088 us | __fsnotify_parent(); 0) 6.145 us | handle_mm_fault(); 3) 0.089 us | fsnotify(); 3) 0.161 us | __sb_end_write(); 3) 0.710 us | SyS_close(); 3) 7.848 us | exit_to_usermode_loop(); On the example above, I specified the function filter 'abcdefg' but all functions are enabled. The expected result is for all functions to be filtered, since there is no such function ('abcdefg') The original fix is to make the kernel support '\0' as end of string: https://lkml.org/lkml/2018/1/16/116 But above fix cannot be compatible with old kernels. Then Namhyung Kim suggest adding a space after function name. This patch will append an '\n' when write tracing file. After this fix, the perf will report correct error state. Also let it print an error if reset_tracing_files() fails. Committer testing: Now it prints: # perf ftrace -a --graph-depth 1 --graph-funcs abcdefg failed to set tracing filters # And for an existing function: # perf ftrace -a --graph-depth 1 --graph-funcs SyS_open 3) | SyS_open() { 3) ! 494.899 us | } 0) + 23.910 us | SyS_open(); 1) + 17.115 us | SyS_open(); 1) + 13.900 us | SyS_open(); ------------------------------------------ 3) qemu-sy-2817 => pickup-1290 ------------------------------------------ 3) + 20.021 us | SyS_open(); # Signed-off-by: Changbin Du Tested-by: Arnaldo Carvalho de Melo Cc: Jiri Olsa Cc: Namhyung Kim Cc: Peter Zijlstra Link: http://lkml.kernel.org/r/1519007609-14551-1-git-send-email-changbin.du@intel.com Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/builtin-ftrace.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) (limited to 'tools') diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c index 25a42acabee1..f42f228e8899 100644 --- a/tools/perf/builtin-ftrace.c +++ b/tools/perf/builtin-ftrace.c @@ -72,6 +72,7 @@ static int __write_tracing_file(const char *name, const char *val, bool append) ssize_t size = strlen(val); int flags = O_WRONLY; char errbuf[512]; + char *val_copy; file = get_tracing_file(name); if (!file) { @@ -91,12 +92,23 @@ static int __write_tracing_file(const char *name, const char *val, bool append) goto out; } - if (write(fd, val, size) == size) + /* + * Copy the original value and append a '\n'. Without this, + * the kernel can hide possible errors. + */ + val_copy = strdup(val); + if (!val_copy) + goto out_close; + val_copy[size] = '\n'; + + if (write(fd, val_copy, size + 1) == size + 1) ret = 0; else pr_debug("write '%s' to tracing/%s failed: %s\n", val, name, str_error_r(errno, errbuf, sizeof(errbuf))); + free(val_copy); +out_close: close(fd); out: put_tracing_file(file); @@ -280,8 +292,10 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv) signal(SIGCHLD, sig_handler); signal(SIGPIPE, sig_handler); - if (reset_tracing_files(ftrace) < 0) + if (reset_tracing_files(ftrace) < 0) { + pr_err("failed to reset ftrace\n"); goto out; + } /* reset ftrace buffer */ if (write_tracing_file("trace", "0") < 0) -- cgit v1.2.3 From d2ed5d2bdc5cd30b44dc52c44c63f08c0a31b845 Mon Sep 17 00:00:00 2001 From: Arnaldo Carvalho de Melo Date: Mon, 19 Feb 2018 12:24:13 -0300 Subject: perf python: Make twatch.py work with both python2 and python3 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Will be used to test patches allowing to build perf with python3, so that we make sure that we can build with both versions. Cc: Adrian Hunter Cc: David Ahern Cc: Jaroslav Škarvada Cc: Jiri Olsa Cc: Namhyung Kim Cc: Wang Nan Link: https://lkml.kernel.org/n/tip-c2ynv0ozr3eifzsyit6qgh3h@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/python/twatch.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) (limited to 'tools') diff --git a/tools/perf/python/twatch.py b/tools/perf/python/twatch.py index c235c22b107a..0a29c5c3079f 100755 --- a/tools/perf/python/twatch.py +++ b/tools/perf/python/twatch.py @@ -42,10 +42,10 @@ def main(context_switch = 0, thread = -1): event = evlist.read_on_cpu(cpu) if not event: continue - print "cpu: %2d, pid: %4d, tid: %4d" % (event.sample_cpu, - event.sample_pid, - event.sample_tid), - print event + print("cpu: {0}, pid: {1}, tid: {2} {3}".format(event.sample_cpu, + event.sample_pid, + event.sample_tid, + event)) if __name__ == '__main__': """ -- cgit v1.2.3 From 66dfdff03d196e51322c6a85c0d8db8bb2bdd655 Mon Sep 17 00:00:00 2001 From: Jaroslav Škarvada Date: Fri, 19 Jan 2018 21:56:41 +0100 Subject: perf tools: Add Python 3 support MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Added Python 3 support while keeping Python 2.7 compatibility. Committer notes: This doesn't make it to auto detect python 3, one has to explicitely ask it to build with python 3 devel files, here are the instructions provided by Jaroslav: --- $ cp -a tools/perf tools/python3-perf $ make V=1 prefix=/usr -C tools/perf PYTHON=/usr/bin/python2 all $ make V=1 prefix=/usr -C tools/python3-perf PYTHON=/usr/bin/python3 all $ make V=1 prefix=/usr -C tools/python3-perf PYTHON=/usr/bin/python3 DESTDIR=%{buildroot} install-python_ext $ make V=1 prefix=/usr -C tools/perf PYTHON=/usr/bin/python2 DESTDIR=%{buildroot} install-python_ext --- We need to make this automatic, just like the existing tests for checking if the python2 devel files are in place, allowing the build with python3 if available, fallbacking to python2 and then just disabling it if none are available. So, using the PYTHON variable to build it using O= we get: Before this patch: $ rpm -q python3 python3-devel python3-3.6.4-7.fc27.x86_64 python3-devel-3.6.4-7.fc27.x86_64 $ rm -rf /tmp/build/perf/ ; mkdir -p /tmp/build/perf ; make O=/tmp/build/perf PYTHON=/usr/bin/python3 -C tools/perf install-bin make: Entering directory '/home/acme/git/linux/tools/perf' Makefile.config:670: Python 3 is not yet supported; please set Makefile.config:671: PYTHON and/or PYTHON_CONFIG appropriately. Makefile.config:672: If you also have Python 2 installed, then Makefile.config:673: try something like: Makefile.config:674: Makefile.config:675: make PYTHON=python2 Makefile.config:676: Makefile.config:677: Otherwise, disable Python support entirely: Makefile.config:678: Makefile.config:679: make NO_LIBPYTHON=1 Makefile.config:680: Makefile.config:681: *** . Stop. make[1]: *** [Makefile.perf:212: sub-make] Error 2 make: *** [Makefile:110: install-bin] Error 2 make: Leaving directory '/home/acme/git/linux/tools/perf' $ After: $ make O=/tmp/build/perf PYTHON=python3 -C tools/perf install-bin $ ldd ~/bin/perf | grep python libpython3.6m.so.1.0 => /lib64/libpython3.6m.so.1.0 (0x00007f58a31e8000) $ rpm -qf /lib64/libpython3.6m.so.1.0 python3-libs-3.6.4-7.fc27.x86_64 $ Now verify that when using the binding the right ELF file is loaded, using perf trace: $ perf trace -e open* perf test python 0.051 ( 0.016 ms): perf/3927 openat(dfd: CWD, filename: /etc/ld.so.cache, flags: CLOEXEC ) = 3 18: 'import perf' in python : 8.849 ( 0.013 ms): sh/3929 openat(dfd: CWD, filename: /etc/ld.so.cache, flags: CLOEXEC ) = 3 25.572 ( 0.008 ms): python3/3931 openat(dfd: CWD, filename: /tmp/build/perf/python/perf.cpython-36m-x86_64-linux-gnu.so, flags: CLOEXEC) = 3 Ok $ And using tools/perf/python/twatch.py, to show PERF_RECORD_ metaevents: $ python3 tools/perf/python/twatch.py cpu: 3, pid: 16060, tid: 16060 { type: fork, pid: 5207, ppid: 16060, tid: 5207, ptid: 16060, time: 10798513015459} cpu: 3, pid: 16060, tid: 16060 { type: fork, pid: 5208, ppid: 16060, tid: 5208, ptid: 16060, time: 10798513562503} cpu: 0, pid: 5208, tid: 5208 { type: comm, pid: 5208, tid: 5208, comm: grep } cpu: 2, pid: 5207, tid: 5207 { type: comm, pid: 5207, tid: 5207, comm: ps } cpu: 2, pid: 5207, tid: 5207 { type: exit, pid: 5207, ppid: 5207, tid: 5207, ptid: 5207, time: 10798551337484} cpu: 3, pid: 5208, tid: 5208 { type: exit, pid: 5208, ppid: 5208, tid: 5208, ptid: 5208, time: 10798551292153} cpu: 3, pid: 601, tid: 601 { type: fork, pid: 5209, ppid: 601, tid: 5209, ptid: 601, time: 10801779977324} ^CTraceback (most recent call last): File "tools/perf/python/twatch.py", line 68, in main() File "tools/perf/python/twatch.py", line 40, in main evlist.poll(timeout = -1) KeyboardInterrupt $ # ps ax|grep twatch 5197 pts/8 S+ 0:00 python3 tools/perf/python/twatch.py # ls -la /proc/5197/smaps -r--r--r--. 1 acme acme 0 Feb 19 13:14 /proc/5197/smaps # grep python /proc/5197/smaps 558111307000-558111309000 r-xp 00000000 fd:00 3151710 /usr/bin/python3.6 558111508000-558111509000 r--p 00001000 fd:00 3151710 /usr/bin/python3.6 558111509000-55811150a000 rw-p 00002000 fd:00 3151710 /usr/bin/python3.6 7ffad6fc1000-7ffad7008000 r-xp 00000000 00:2d 220196 /tmp/build/perf/python/perf.cpython-36m-x86_64-linux-gnu.so 7ffad7008000-7ffad7207000 ---p 00047000 00:2d 220196 /tmp/build/perf/python/perf.cpython-36m-x86_64-linux-gnu.so 7ffad7207000-7ffad7208000 r--p 00046000 00:2d 220196 /tmp/build/perf/python/perf.cpython-36m-x86_64-linux-gnu.so 7ffad7208000-7ffad7215000 rw-p 00047000 00:2d 220196 /tmp/build/perf/python/perf.cpython-36m-x86_64-linux-gnu.so 7ffadea77000-7ffaded3d000 r-xp 00000000 fd:00 3151795 /usr/lib64/libpython3.6m.so.1.0 7ffaded3d000-7ffadef3c000 ---p 002c6000 fd:00 3151795 /usr/lib64/libpython3.6m.so.1.0 7ffadef3c000-7ffadef42000 r--p 002c5000 fd:00 3151795 /usr/lib64/libpython3.6m.so.1.0 7ffadef42000-7ffadefa5000 rw-p 002cb000 fd:00 3151795 /usr/lib64/libpython3.6m.so.1.0 # And with this patch, but building normally, without specifying the PYTHON=python3 part, which will make it use python2 if its devel files are available, like in this test: $ make O=/tmp/build/perf -C tools/perf install-bin $ ldd ~/bin/perf | grep python libpython2.7.so.1.0 => /lib64/libpython2.7.so.1.0 (0x00007f6a44410000) $ ldd /tmp/build/perf/python_ext_build/lib/perf.so | grep python libpython2.7.so.1.0 => /lib64/libpython2.7.so.1.0 (0x00007fed28a2c000) $ [acme@jouet perf]$ tools/perf/python/twatch.py cpu: 0, pid: 2817, tid: 2817 { type: fork, pid: 2817, ppid: 2817, tid: 8910, ptid: 2817, time: 11126454335306} cpu: 0, pid: 2817, tid: 2817 { type: comm, pid: 2817, tid: 8910, comm: worker } $ ps ax | grep twatch.py 8909 pts/8 S+ 0:00 /usr/bin/python tools/perf/python/twatch.py $ grep python /proc/8909/smaps 5579de658000-5579de659000 r-xp 00000000 fd:00 3156044 /usr/bin/python2.7 5579de858000-5579de859000 r--p 00000000 fd:00 3156044 /usr/bin/python2.7 5579de859000-5579de85a000 rw-p 00001000 fd:00 3156044 /usr/bin/python2.7 7f0de01f7000-7f0de023e000 r-xp 00000000 00:2d 230695 /tmp/build/perf/python/perf.so 7f0de023e000-7f0de043d000 ---p 00047000 00:2d 230695 /tmp/build/perf/python/perf.so 7f0de043d000-7f0de043e000 r--p 00046000 00:2d 230695 /tmp/build/perf/python/perf.so 7f0de043e000-7f0de044b000 rw-p 00047000 00:2d 230695 /tmp/build/perf/python/perf.so 7f0de6f0f000-7f0de6f13000 r-xp 00000000 fd:00 134975 /usr/lib64/python2.7/lib-dynload/_localemodule.so 7f0de6f13000-7f0de7113000 ---p 00004000 fd:00 134975 /usr/lib64/python2.7/lib-dynload/_localemodule.so 7f0de7113000-7f0de7114000 r--p 00004000 fd:00 134975 /usr/lib64/python2.7/lib-dynload/_localemodule.so 7f0de7114000-7f0de7115000 rw-p 00005000 fd:00 134975 /usr/lib64/python2.7/lib-dynload/_localemodule.so 7f0de7e73000-7f0de8052000 r-xp 00000000 fd:00 3173292 /usr/lib64/libpython2.7.so.1.0 7f0de8052000-7f0de8251000 ---p 001df000 fd:00 3173292 /usr/lib64/libpython2.7.so.1.0 7f0de8251000-7f0de8255000 r--p 001de000 fd:00 3173292 /usr/lib64/libpython2.7.so.1.0 7f0de8255000-7f0de8291000 rw-p 001e2000 fd:00 3173292 /usr/lib64/libpython2.7.so.1.0 $ Signed-off-by: Jaroslav Škarvada Tested-by: Arnaldo Carvalho de Melo Cc: Adrian Hunter Cc: David Ahern Cc: Jiri Olsa Cc: Namhyung Kim Cc: Wang Nan LPU-Reference: 20180119205641.24242-1-jskarvad@redhat.com Link: https://lkml.kernel.org/n/tip-8d7dt9kqp83vsz25hagug8fu@git.kernel.org [ Removed explicit check for python version, allowing it to really build with python3 ] Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/Makefile.config | 23 +--- tools/perf/Makefile.perf | 4 +- .../perf/scripts/python/Perf-Trace-Util/Context.c | 34 ++++- tools/perf/util/python.c | 95 ++++++++++--- .../util/scripting-engines/trace-event-python.c | 147 +++++++++++++++------ tools/perf/util/setup.py | 6 +- 6 files changed, 221 insertions(+), 88 deletions(-) (limited to 'tools') diff --git a/tools/perf/Makefile.config b/tools/perf/Makefile.config index 577a5d2988fe..89cb2a36b8ff 100644 --- a/tools/perf/Makefile.config +++ b/tools/perf/Makefile.config @@ -668,25 +668,10 @@ else ifneq ($(feature-libpython), 1) $(call disable-python,No 'Python.h' (for Python 2.x support) was found: disables Python support - please install python-devel/python-dev) else - ifneq ($(feature-libpython-version), 1) - $(warning Python 3 is not yet supported; please set) - $(warning PYTHON and/or PYTHON_CONFIG appropriately.) - $(warning If you also have Python 2 installed, then) - $(warning try something like:) - $(warning $(and ,)) - $(warning $(and ,) make PYTHON=python2) - $(warning $(and ,)) - $(warning Otherwise, disable Python support entirely:) - $(warning $(and ,)) - $(warning $(and ,) make NO_LIBPYTHON=1) - $(warning $(and ,)) - $(error $(and ,)) - else - LDFLAGS += $(PYTHON_EMBED_LDFLAGS) - EXTLIBS += $(PYTHON_EMBED_LIBADD) - LANG_BINDINGS += $(obj-perf)python/perf.so - $(call detected,CONFIG_LIBPYTHON) - endif + LDFLAGS += $(PYTHON_EMBED_LDFLAGS) + EXTLIBS += $(PYTHON_EMBED_LIBADD) + LANG_BINDINGS += $(obj-perf)python/perf.so + $(call detected,CONFIG_LIBPYTHON) endif endif endif diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf index 9b0351d3ce34..126ef6e0475c 100644 --- a/tools/perf/Makefile.perf +++ b/tools/perf/Makefile.perf @@ -302,7 +302,7 @@ PYTHON_EXTBUILD_LIB := $(PYTHON_EXTBUILD)lib/ PYTHON_EXTBUILD_TMP := $(PYTHON_EXTBUILD)tmp/ export PYTHON_EXTBUILD_LIB PYTHON_EXTBUILD_TMP -python-clean := $(call QUIET_CLEAN, python) $(RM) -r $(PYTHON_EXTBUILD) $(OUTPUT)python/perf.so +python-clean := $(call QUIET_CLEAN, python) $(RM) -r $(PYTHON_EXTBUILD) $(OUTPUT)python/perf*.so PYTHON_EXT_SRCS := $(shell grep -v ^\# util/python-ext-sources) PYTHON_EXT_DEPS := util/python-ext-sources util/setup.py $(LIBTRACEEVENT) $(LIBAPI) @@ -479,7 +479,7 @@ $(OUTPUT)python/perf.so: $(PYTHON_EXT_SRCS) $(PYTHON_EXT_DEPS) $(LIBTRACEEVENT_D $(PYTHON_WORD) util/setup.py \ --quiet build_ext; \ mkdir -p $(OUTPUT)python && \ - cp $(PYTHON_EXTBUILD_LIB)perf.so $(OUTPUT)python/ + cp $(PYTHON_EXTBUILD_LIB)perf*.so $(OUTPUT)python/ please_set_SHELL_PATH_to_a_more_modern_shell: $(Q)$$(:) diff --git a/tools/perf/scripts/python/Perf-Trace-Util/Context.c b/tools/perf/scripts/python/Perf-Trace-Util/Context.c index fcd1dd667906..1a0d27757eec 100644 --- a/tools/perf/scripts/python/Perf-Trace-Util/Context.c +++ b/tools/perf/scripts/python/Perf-Trace-Util/Context.c @@ -23,7 +23,17 @@ #include "../../../perf.h" #include "../../../util/trace-event.h" +#if PY_MAJOR_VERSION < 3 +#define _PyCapsule_GetPointer(arg1, arg2) \ + PyCObject_AsVoidPtr(arg1) + PyMODINIT_FUNC initperf_trace_context(void); +#else +#define _PyCapsule_GetPointer(arg1, arg2) \ + PyCapsule_GetPointer((arg1), (arg2)) + +PyMODINIT_FUNC PyInit_perf_trace_context(void); +#endif static PyObject *perf_trace_context_common_pc(PyObject *obj, PyObject *args) { @@ -34,7 +44,7 @@ static PyObject *perf_trace_context_common_pc(PyObject *obj, PyObject *args) if (!PyArg_ParseTuple(args, "O", &context)) return NULL; - scripting_context = PyCObject_AsVoidPtr(context); + scripting_context = _PyCapsule_GetPointer(context, NULL); retval = common_pc(scripting_context); return Py_BuildValue("i", retval); @@ -50,7 +60,7 @@ static PyObject *perf_trace_context_common_flags(PyObject *obj, if (!PyArg_ParseTuple(args, "O", &context)) return NULL; - scripting_context = PyCObject_AsVoidPtr(context); + scripting_context = _PyCapsule_GetPointer(context, NULL); retval = common_flags(scripting_context); return Py_BuildValue("i", retval); @@ -66,7 +76,7 @@ static PyObject *perf_trace_context_common_lock_depth(PyObject *obj, if (!PyArg_ParseTuple(args, "O", &context)) return NULL; - scripting_context = PyCObject_AsVoidPtr(context); + scripting_context = _PyCapsule_GetPointer(context, NULL); retval = common_lock_depth(scripting_context); return Py_BuildValue("i", retval); @@ -82,7 +92,25 @@ static PyMethodDef ContextMethods[] = { { NULL, NULL, 0, NULL} }; +#if PY_MAJOR_VERSION < 3 PyMODINIT_FUNC initperf_trace_context(void) { (void) Py_InitModule("perf_trace_context", ContextMethods); } +#else +PyMODINIT_FUNC PyInit_perf_trace_context(void) +{ + static struct PyModuleDef moduledef = { + PyModuleDef_HEAD_INIT, + "perf_trace_context", /* m_name */ + "", /* m_doc */ + -1, /* m_size */ + ContextMethods, /* m_methods */ + NULL, /* m_reload */ + NULL, /* m_traverse */ + NULL, /* m_clear */ + NULL, /* m_free */ + }; + return PyModule_Create(&moduledef); +} +#endif diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c index b1e999bd21ef..2918cac7a142 100644 --- a/tools/perf/util/python.c +++ b/tools/perf/util/python.c @@ -12,6 +12,30 @@ #include "print_binary.h" #include "thread_map.h" +#if PY_MAJOR_VERSION < 3 +#define _PyUnicode_FromString(arg) \ + PyString_FromString(arg) +#define _PyUnicode_AsString(arg) \ + PyString_AsString(arg) +#define _PyUnicode_FromFormat(...) \ + PyString_FromFormat(__VA_ARGS__) +#define _PyLong_FromLong(arg) \ + PyInt_FromLong(arg) + +#else + +#define _PyUnicode_FromString(arg) \ + PyUnicode_FromString(arg) +#define _PyUnicode_FromFormat(...) \ + PyUnicode_FromFormat(__VA_ARGS__) +#define _PyLong_FromLong(arg) \ + PyLong_FromLong(arg) +#endif + +#ifndef Py_TYPE +#define Py_TYPE(ob) (((PyObject*)(ob))->ob_type) +#endif + /* * Provide these two so that we don't have to link against callchain.c and * start dragging hist.c, etc. @@ -49,7 +73,11 @@ int eprintf(int level, int var, const char *fmt, ...) # define PyVarObject_HEAD_INIT(type, size) PyObject_HEAD_INIT(type) size, #endif +#if PY_MAJOR_VERSION < 3 PyMODINIT_FUNC initperf(void); +#else +PyMODINIT_FUNC PyInit_perf(void); +#endif #define member_def(type, member, ptype, help) \ { #member, ptype, \ @@ -107,7 +135,7 @@ static PyObject *pyrf_mmap_event__repr(struct pyrf_event *pevent) pevent->event.mmap.pgoff, pevent->event.mmap.filename) < 0) { ret = PyErr_NoMemory(); } else { - ret = PyString_FromString(s); + ret = _PyUnicode_FromString(s); free(s); } return ret; @@ -138,7 +166,7 @@ static PyMemberDef pyrf_task_event__members[] = { static PyObject *pyrf_task_event__repr(struct pyrf_event *pevent) { - return PyString_FromFormat("{ type: %s, pid: %u, ppid: %u, tid: %u, " + return _PyUnicode_FromFormat("{ type: %s, pid: %u, ppid: %u, tid: %u, " "ptid: %u, time: %" PRIu64 "}", pevent->event.header.type == PERF_RECORD_FORK ? "fork" : "exit", pevent->event.fork.pid, @@ -171,7 +199,7 @@ static PyMemberDef pyrf_comm_event__members[] = { static PyObject *pyrf_comm_event__repr(struct pyrf_event *pevent) { - return PyString_FromFormat("{ type: comm, pid: %u, tid: %u, comm: %s }", + return _PyUnicode_FromFormat("{ type: comm, pid: %u, tid: %u, comm: %s }", pevent->event.comm.pid, pevent->event.comm.tid, pevent->event.comm.comm); @@ -202,7 +230,7 @@ static PyObject *pyrf_throttle_event__repr(struct pyrf_event *pevent) { struct throttle_event *te = (struct throttle_event *)(&pevent->event.header + 1); - return PyString_FromFormat("{ type: %sthrottle, time: %" PRIu64 ", id: %" PRIu64 + return _PyUnicode_FromFormat("{ type: %sthrottle, time: %" PRIu64 ", id: %" PRIu64 ", stream_id: %" PRIu64 " }", pevent->event.header.type == PERF_RECORD_THROTTLE ? "" : "un", te->time, te->id, te->stream_id); @@ -237,7 +265,7 @@ static PyObject *pyrf_lost_event__repr(struct pyrf_event *pevent) pevent->event.lost.id, pevent->event.lost.lost) < 0) { ret = PyErr_NoMemory(); } else { - ret = PyString_FromString(s); + ret = _PyUnicode_FromString(s); free(s); } return ret; @@ -264,7 +292,7 @@ static PyMemberDef pyrf_read_event__members[] = { static PyObject *pyrf_read_event__repr(struct pyrf_event *pevent) { - return PyString_FromFormat("{ type: read, pid: %u, tid: %u }", + return _PyUnicode_FromFormat("{ type: read, pid: %u, tid: %u }", pevent->event.read.pid, pevent->event.read.tid); /* @@ -299,7 +327,7 @@ static PyObject *pyrf_sample_event__repr(struct pyrf_event *pevent) if (asprintf(&s, "{ type: sample }") < 0) { ret = PyErr_NoMemory(); } else { - ret = PyString_FromString(s); + ret = _PyUnicode_FromString(s); free(s); } return ret; @@ -330,7 +358,7 @@ tracepoint_field(struct pyrf_event *pe, struct format_field *field) } if (field->flags & FIELD_IS_STRING && is_printable_array(data + offset, len)) { - ret = PyString_FromString((char *)data + offset); + ret = _PyUnicode_FromString((char *)data + offset); } else { ret = PyByteArray_FromStringAndSize((const char *) data + offset, len); field->flags &= ~FIELD_IS_STRING; @@ -352,7 +380,7 @@ tracepoint_field(struct pyrf_event *pe, struct format_field *field) static PyObject* get_tracepoint_field(struct pyrf_event *pevent, PyObject *attr_name) { - const char *str = PyString_AsString(PyObject_Str(attr_name)); + const char *str = _PyUnicode_AsString(PyObject_Str(attr_name)); struct perf_evsel *evsel = pevent->evsel; struct format_field *field; @@ -416,7 +444,7 @@ static PyObject *pyrf_context_switch_event__repr(struct pyrf_event *pevent) !!(pevent->event.header.misc & PERF_RECORD_MISC_SWITCH_OUT)) < 0) { ret = PyErr_NoMemory(); } else { - ret = PyString_FromString(s); + ret = _PyUnicode_FromString(s); free(s); } return ret; @@ -528,7 +556,7 @@ static int pyrf_cpu_map__init(struct pyrf_cpu_map *pcpus, static void pyrf_cpu_map__delete(struct pyrf_cpu_map *pcpus) { cpu_map__put(pcpus->cpus); - pcpus->ob_type->tp_free((PyObject*)pcpus); + Py_TYPE(pcpus)->tp_free((PyObject*)pcpus); } static Py_ssize_t pyrf_cpu_map__length(PyObject *obj) @@ -597,7 +625,7 @@ static int pyrf_thread_map__init(struct pyrf_thread_map *pthreads, static void pyrf_thread_map__delete(struct pyrf_thread_map *pthreads) { thread_map__put(pthreads->threads); - pthreads->ob_type->tp_free((PyObject*)pthreads); + Py_TYPE(pthreads)->tp_free((PyObject*)pthreads); } static Py_ssize_t pyrf_thread_map__length(PyObject *obj) @@ -759,7 +787,7 @@ static int pyrf_evsel__init(struct pyrf_evsel *pevsel, static void pyrf_evsel__delete(struct pyrf_evsel *pevsel) { perf_evsel__exit(&pevsel->evsel); - pevsel->ob_type->tp_free((PyObject*)pevsel); + Py_TYPE(pevsel)->tp_free((PyObject*)pevsel); } static PyObject *pyrf_evsel__open(struct pyrf_evsel *pevsel, @@ -850,7 +878,7 @@ static int pyrf_evlist__init(struct pyrf_evlist *pevlist, static void pyrf_evlist__delete(struct pyrf_evlist *pevlist) { perf_evlist__exit(&pevlist->evlist); - pevlist->ob_type->tp_free((PyObject*)pevlist); + Py_TYPE(pevlist)->tp_free((PyObject*)pevlist); } static PyObject *pyrf_evlist__mmap(struct pyrf_evlist *pevlist, @@ -902,12 +930,16 @@ static PyObject *pyrf_evlist__get_pollfd(struct pyrf_evlist *pevlist, for (i = 0; i < evlist->pollfd.nr; ++i) { PyObject *file; +#if PY_MAJOR_VERSION < 3 FILE *fp = fdopen(evlist->pollfd.entries[i].fd, "r"); if (fp == NULL) goto free_list; file = PyFile_FromFile(fp, "perf", "r", NULL); +#else + file = PyFile_FromFd(evlist->pollfd.entries[i].fd, "perf", "r", -1, NULL, NULL, NULL, 1); +#endif if (file == NULL) goto free_list; @@ -1194,9 +1226,9 @@ static PyObject *pyrf__tracepoint(struct pyrf_evsel *pevsel, tp_format = trace_event__tp_format(sys, name); if (IS_ERR(tp_format)) - return PyInt_FromLong(-1); + return _PyLong_FromLong(-1); - return PyInt_FromLong(tp_format->id); + return _PyLong_FromLong(tp_format->id); } static PyMethodDef perf__methods[] = { @@ -1209,11 +1241,31 @@ static PyMethodDef perf__methods[] = { { .ml_name = NULL, } }; +#if PY_MAJOR_VERSION < 3 PyMODINIT_FUNC initperf(void) +#else +PyMODINIT_FUNC PyInit_perf(void) +#endif { PyObject *obj; int i; - PyObject *dict, *module = Py_InitModule("perf", perf__methods); + PyObject *dict; +#if PY_MAJOR_VERSION < 3 + PyObject *module = Py_InitModule("perf", perf__methods); +#else + static struct PyModuleDef moduledef = { + PyModuleDef_HEAD_INIT, + "perf", /* m_name */ + "", /* m_doc */ + -1, /* m_size */ + perf__methods, /* m_methods */ + NULL, /* m_reload */ + NULL, /* m_traverse */ + NULL, /* m_clear */ + NULL, /* m_free */ + }; + PyObject *module = PyModule_Create(&moduledef); +#endif if (module == NULL || pyrf_event__setup_types() < 0 || @@ -1221,7 +1273,11 @@ PyMODINIT_FUNC initperf(void) pyrf_evsel__setup_types() < 0 || pyrf_thread_map__setup_types() < 0 || pyrf_cpu_map__setup_types() < 0) +#if PY_MAJOR_VERSION < 3 return; +#else + return module; +#endif /* The page_size is placed in util object. */ page_size = sysconf(_SC_PAGE_SIZE); @@ -1270,7 +1326,7 @@ PyMODINIT_FUNC initperf(void) goto error; for (i = 0; perf__constants[i].name != NULL; i++) { - obj = PyInt_FromLong(perf__constants[i].value); + obj = _PyLong_FromLong(perf__constants[i].value); if (obj == NULL) goto error; PyDict_SetItemString(dict, perf__constants[i].name, obj); @@ -1280,6 +1336,9 @@ PyMODINIT_FUNC initperf(void) error: if (PyErr_Occurred()) PyErr_SetString(PyExc_ImportError, "perf: Init failed!"); +#if PY_MAJOR_VERSION >= 3 + return module; +#endif } /* diff --git a/tools/perf/util/scripting-engines/trace-event-python.c b/tools/perf/util/scripting-engines/trace-event-python.c index ea070883c593..10dd5fce082b 100644 --- a/tools/perf/util/scripting-engines/trace-event-python.c +++ b/tools/perf/util/scripting-engines/trace-event-python.c @@ -49,7 +49,37 @@ #include "print_binary.h" #include "stat.h" +#if PY_MAJOR_VERSION < 3 +#define _PyUnicode_FromString(arg) \ + PyString_FromString(arg) +#define _PyUnicode_FromStringAndSize(arg1, arg2) \ + PyString_FromStringAndSize((arg1), (arg2)) +#define _PyBytes_FromStringAndSize(arg1, arg2) \ + PyString_FromStringAndSize((arg1), (arg2)) +#define _PyLong_FromLong(arg) \ + PyInt_FromLong(arg) +#define _PyLong_AsLong(arg) \ + PyInt_AsLong(arg) +#define _PyCapsule_New(arg1, arg2, arg3) \ + PyCObject_FromVoidPtr((arg1), (arg2)) + PyMODINIT_FUNC initperf_trace_context(void); +#else +#define _PyUnicode_FromString(arg) \ + PyUnicode_FromString(arg) +#define _PyUnicode_FromStringAndSize(arg1, arg2) \ + PyUnicode_FromStringAndSize((arg1), (arg2)) +#define _PyBytes_FromStringAndSize(arg1, arg2) \ + PyBytes_FromStringAndSize((arg1), (arg2)) +#define _PyLong_FromLong(arg) \ + PyLong_FromLong(arg) +#define _PyLong_AsLong(arg) \ + PyLong_AsLong(arg) +#define _PyCapsule_New(arg1, arg2, arg3) \ + PyCapsule_New((arg1), (arg2), (arg3)) + +PyMODINIT_FUNC PyInit_perf_trace_context(void); +#endif #define TRACE_EVENT_TYPE_MAX \ ((1 << (sizeof(unsigned short) * 8)) - 1) @@ -135,7 +165,7 @@ static int get_argument_count(PyObject *handler) PyObject *arg_count_obj = PyObject_GetAttrString(code_obj, "co_argcount"); if (arg_count_obj) { - arg_count = (int) PyInt_AsLong(arg_count_obj); + arg_count = (int) _PyLong_AsLong(arg_count_obj); Py_DECREF(arg_count_obj); } Py_DECREF(code_obj); @@ -182,10 +212,10 @@ static void define_value(enum print_arg_type field_type, value = eval_flag(field_value); - PyTuple_SetItem(t, n++, PyString_FromString(ev_name)); - PyTuple_SetItem(t, n++, PyString_FromString(field_name)); - PyTuple_SetItem(t, n++, PyInt_FromLong(value)); - PyTuple_SetItem(t, n++, PyString_FromString(field_str)); + PyTuple_SetItem(t, n++, _PyUnicode_FromString(ev_name)); + PyTuple_SetItem(t, n++, _PyUnicode_FromString(field_name)); + PyTuple_SetItem(t, n++, _PyLong_FromLong(value)); + PyTuple_SetItem(t, n++, _PyUnicode_FromString(field_str)); try_call_object(handler_name, t); @@ -223,10 +253,10 @@ static void define_field(enum print_arg_type field_type, if (!t) Py_FatalError("couldn't create Python tuple"); - PyTuple_SetItem(t, n++, PyString_FromString(ev_name)); - PyTuple_SetItem(t, n++, PyString_FromString(field_name)); + PyTuple_SetItem(t, n++, _PyUnicode_FromString(ev_name)); + PyTuple_SetItem(t, n++, _PyUnicode_FromString(field_name)); if (field_type == PRINT_FLAGS) - PyTuple_SetItem(t, n++, PyString_FromString(delim)); + PyTuple_SetItem(t, n++, _PyUnicode_FromString(delim)); try_call_object(handler_name, t); @@ -325,12 +355,12 @@ static PyObject *get_field_numeric_entry(struct event_format *event, if (field->flags & FIELD_IS_SIGNED) { if ((long long)val >= LONG_MIN && (long long)val <= LONG_MAX) - obj = PyInt_FromLong(val); + obj = _PyLong_FromLong(val); else obj = PyLong_FromLongLong(val); } else { if (val <= LONG_MAX) - obj = PyInt_FromLong(val); + obj = _PyLong_FromLong(val); else obj = PyLong_FromUnsignedLongLong(val); } @@ -389,9 +419,9 @@ static PyObject *python_process_callchain(struct perf_sample *sample, pydict_set_item_string_decref(pysym, "end", PyLong_FromUnsignedLongLong(node->sym->end)); pydict_set_item_string_decref(pysym, "binding", - PyInt_FromLong(node->sym->binding)); + _PyLong_FromLong(node->sym->binding)); pydict_set_item_string_decref(pysym, "name", - PyString_FromStringAndSize(node->sym->name, + _PyUnicode_FromStringAndSize(node->sym->name, node->sym->namelen)); pydict_set_item_string_decref(pyelem, "sym", pysym); } @@ -406,7 +436,7 @@ static PyObject *python_process_callchain(struct perf_sample *sample, dsoname = map->dso->name; } pydict_set_item_string_decref(pyelem, "dso", - PyString_FromString(dsoname)); + _PyUnicode_FromString(dsoname)); } callchain_cursor_advance(&callchain_cursor); @@ -483,16 +513,16 @@ static PyObject *get_perf_sample_dict(struct perf_sample *sample, if (!dict_sample) Py_FatalError("couldn't create Python dictionary"); - pydict_set_item_string_decref(dict, "ev_name", PyString_FromString(perf_evsel__name(evsel))); - pydict_set_item_string_decref(dict, "attr", PyString_FromStringAndSize( + pydict_set_item_string_decref(dict, "ev_name", _PyUnicode_FromString(perf_evsel__name(evsel))); + pydict_set_item_string_decref(dict, "attr", _PyUnicode_FromStringAndSize( (const char *)&evsel->attr, sizeof(evsel->attr))); pydict_set_item_string_decref(dict_sample, "pid", - PyInt_FromLong(sample->pid)); + _PyLong_FromLong(sample->pid)); pydict_set_item_string_decref(dict_sample, "tid", - PyInt_FromLong(sample->tid)); + _PyLong_FromLong(sample->tid)); pydict_set_item_string_decref(dict_sample, "cpu", - PyInt_FromLong(sample->cpu)); + _PyLong_FromLong(sample->cpu)); pydict_set_item_string_decref(dict_sample, "ip", PyLong_FromUnsignedLongLong(sample->ip)); pydict_set_item_string_decref(dict_sample, "time", @@ -504,17 +534,17 @@ static PyObject *get_perf_sample_dict(struct perf_sample *sample, set_sample_read_in_dict(dict_sample, sample, evsel); pydict_set_item_string_decref(dict, "sample", dict_sample); - pydict_set_item_string_decref(dict, "raw_buf", PyString_FromStringAndSize( + pydict_set_item_string_decref(dict, "raw_buf", _PyBytes_FromStringAndSize( (const char *)sample->raw_data, sample->raw_size)); pydict_set_item_string_decref(dict, "comm", - PyString_FromString(thread__comm_str(al->thread))); + _PyUnicode_FromString(thread__comm_str(al->thread))); if (al->map) { pydict_set_item_string_decref(dict, "dso", - PyString_FromString(al->map->dso->name)); + _PyUnicode_FromString(al->map->dso->name)); } if (al->sym) { pydict_set_item_string_decref(dict, "symbol", - PyString_FromString(al->sym->name)); + _PyUnicode_FromString(al->sym->name)); } pydict_set_item_string_decref(dict, "callchain", callchain); @@ -574,9 +604,9 @@ static void python_process_tracepoint(struct perf_sample *sample, scripting_context->event_data = data; scripting_context->pevent = evsel->tp_format->pevent; - context = PyCObject_FromVoidPtr(scripting_context, NULL); + context = _PyCapsule_New(scripting_context, NULL, NULL); - PyTuple_SetItem(t, n++, PyString_FromString(handler_name)); + PyTuple_SetItem(t, n++, _PyUnicode_FromString(handler_name)); PyTuple_SetItem(t, n++, context); /* ip unwinding */ @@ -585,18 +615,18 @@ static void python_process_tracepoint(struct perf_sample *sample, Py_INCREF(callchain); if (!dict) { - PyTuple_SetItem(t, n++, PyInt_FromLong(cpu)); - PyTuple_SetItem(t, n++, PyInt_FromLong(s)); - PyTuple_SetItem(t, n++, PyInt_FromLong(ns)); - PyTuple_SetItem(t, n++, PyInt_FromLong(pid)); - PyTuple_SetItem(t, n++, PyString_FromString(comm)); + PyTuple_SetItem(t, n++, _PyLong_FromLong(cpu)); + PyTuple_SetItem(t, n++, _PyLong_FromLong(s)); + PyTuple_SetItem(t, n++, _PyLong_FromLong(ns)); + PyTuple_SetItem(t, n++, _PyLong_FromLong(pid)); + PyTuple_SetItem(t, n++, _PyUnicode_FromString(comm)); PyTuple_SetItem(t, n++, callchain); } else { - pydict_set_item_string_decref(dict, "common_cpu", PyInt_FromLong(cpu)); - pydict_set_item_string_decref(dict, "common_s", PyInt_FromLong(s)); - pydict_set_item_string_decref(dict, "common_ns", PyInt_FromLong(ns)); - pydict_set_item_string_decref(dict, "common_pid", PyInt_FromLong(pid)); - pydict_set_item_string_decref(dict, "common_comm", PyString_FromString(comm)); + pydict_set_item_string_decref(dict, "common_cpu", _PyLong_FromLong(cpu)); + pydict_set_item_string_decref(dict, "common_s", _PyLong_FromLong(s)); + pydict_set_item_string_decref(dict, "common_ns", _PyLong_FromLong(ns)); + pydict_set_item_string_decref(dict, "common_pid", _PyLong_FromLong(pid)); + pydict_set_item_string_decref(dict, "common_comm", _PyUnicode_FromString(comm)); pydict_set_item_string_decref(dict, "common_callchain", callchain); } for (field = event->format.fields; field; field = field->next) { @@ -615,7 +645,7 @@ static void python_process_tracepoint(struct perf_sample *sample, } if (field->flags & FIELD_IS_STRING && is_printable_array(data + offset, len)) { - obj = PyString_FromString((char *) data + offset); + obj = _PyUnicode_FromString((char *) data + offset); } else { obj = PyByteArray_FromStringAndSize((const char *) data + offset, len); field->flags &= ~FIELD_IS_STRING; @@ -668,7 +698,7 @@ static PyObject *tuple_new(unsigned int sz) static int tuple_set_u64(PyObject *t, unsigned int pos, u64 val) { #if BITS_PER_LONG == 64 - return PyTuple_SetItem(t, pos, PyInt_FromLong(val)); + return PyTuple_SetItem(t, pos, _PyLong_FromLong(val)); #endif #if BITS_PER_LONG == 32 return PyTuple_SetItem(t, pos, PyLong_FromLongLong(val)); @@ -677,12 +707,12 @@ static int tuple_set_u64(PyObject *t, unsigned int pos, u64 val) static int tuple_set_s32(PyObject *t, unsigned int pos, s32 val) { - return PyTuple_SetItem(t, pos, PyInt_FromLong(val)); + return PyTuple_SetItem(t, pos, _PyLong_FromLong(val)); } static int tuple_set_string(PyObject *t, unsigned int pos, const char *s) { - return PyTuple_SetItem(t, pos, PyString_FromString(s)); + return PyTuple_SetItem(t, pos, _PyUnicode_FromString(s)); } static int python_export_evsel(struct db_export *dbe, struct perf_evsel *evsel) @@ -1029,8 +1059,8 @@ process_stat(struct perf_evsel *counter, int cpu, int thread, u64 tstamp, return; } - PyTuple_SetItem(t, n++, PyInt_FromLong(cpu)); - PyTuple_SetItem(t, n++, PyInt_FromLong(thread)); + PyTuple_SetItem(t, n++, _PyLong_FromLong(cpu)); + PyTuple_SetItem(t, n++, _PyLong_FromLong(thread)); tuple_set_u64(t, n++, tstamp); tuple_set_u64(t, n++, count->val); @@ -1212,27 +1242,58 @@ static void set_table_handlers(struct tables *tables) SET_TABLE_HANDLER(call_return); } +#if PY_MAJOR_VERSION < 3 +static void _free_command_line(const char **command_line, int num) +{ + free(command_line); +} +#else +static void _free_command_line(wchar_t **command_line, int num) +{ + int i; + for (i = 0; i < num; i++) + PyMem_RawFree(command_line[i]); + free(command_line); +} +#endif + + /* * Start trace script */ static int python_start_script(const char *script, int argc, const char **argv) { struct tables *tables = &tables_global; +#if PY_MAJOR_VERSION < 3 const char **command_line; +#else + wchar_t **command_line; +#endif char buf[PATH_MAX]; int i, err = 0; FILE *fp; +#if PY_MAJOR_VERSION < 3 command_line = malloc((argc + 1) * sizeof(const char *)); command_line[0] = script; for (i = 1; i < argc + 1; i++) command_line[i] = argv[i - 1]; +#else + command_line = malloc((argc + 1) * sizeof(wchar_t *)); + command_line[0] = Py_DecodeLocale(script, NULL); + for (i = 1; i < argc + 1; i++) + command_line[i] = Py_DecodeLocale(argv[i - 1], NULL); +#endif Py_Initialize(); +#if PY_MAJOR_VERSION < 3 initperf_trace_context(); - PySys_SetArgv(argc + 1, (char **)command_line); +#else + PyInit_perf_trace_context(); + PySys_SetArgv(argc + 1, command_line); +#endif fp = fopen(script, "r"); if (!fp) { @@ -1262,12 +1323,12 @@ static int python_start_script(const char *script, int argc, const char **argv) goto error; } - free(command_line); + _free_command_line(command_line, argc + 1); return err; error: Py_Finalize(); - free(command_line); + _free_command_line(command_line, argc + 1); return err; } diff --git a/tools/perf/util/setup.py b/tools/perf/util/setup.py index af415febbc46..6891635b50c3 100644 --- a/tools/perf/util/setup.py +++ b/tools/perf/util/setup.py @@ -1,4 +1,4 @@ -#!/usr/bin/python2 +#!/usr/bin/python from os import getenv @@ -35,11 +35,11 @@ build_tmp = getenv('PYTHON_EXTBUILD_TMP') libtraceevent = getenv('LIBTRACEEVENT') libapikfs = getenv('LIBAPI') -ext_sources = [f.strip() for f in file('util/python-ext-sources') +ext_sources = [f.strip() for f in open('util/python-ext-sources') if len(f.strip()) > 0 and f[0] != '#'] # use full paths with source files -ext_sources = map(lambda x: '%s/%s' % (src_perf, x) , ext_sources) +ext_sources = list(map(lambda x: '%s/%s' % (src_perf, x) , ext_sources)) perf = Extension('perf', sources = ext_sources, -- cgit v1.2.3 From f4f2cf8bd8e3f6a802b6123bbde795d948560342 Mon Sep 17 00:00:00 2001 From: Lihao Liang Date: Thu, 7 Dec 2017 18:52:22 +0800 Subject: doc: Fix typo in rcutorture documentation Signed-off-by: Lihao Liang Signed-off-by: Paul E. McKenney --- tools/testing/selftests/rcutorture/doc/rcu-test-image.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'tools') diff --git a/tools/testing/selftests/rcutorture/doc/rcu-test-image.txt b/tools/testing/selftests/rcutorture/doc/rcu-test-image.txt index 66efb59a1bd1..449cf579d6f9 100644 --- a/tools/testing/selftests/rcutorture/doc/rcu-test-image.txt +++ b/tools/testing/selftests/rcutorture/doc/rcu-test-image.txt @@ -1,4 +1,4 @@ -This document describes one way to created the rcu-test-image file +This document describes one way to create the rcu-test-image file that contains the filesystem used by the guest-OS kernel. There are probably much better ways of doing this, and this filesystem could no doubt be smaller. It is probably also possible to simply download -- cgit v1.2.3 From db92ca3ab8c9aba213cfb3b6aabc95a9afa5d833 Mon Sep 17 00:00:00 2001 From: "Paul E. McKenney" Date: Mon, 11 Dec 2017 08:20:25 -0800 Subject: rcutorture: Update kvm.sh header comment The kvm.sh header comment is a bit of a relic, so this commit brings it up to date. Reported-by: Lihao Liang Signed-off-by: Paul E. McKenney --- tools/testing/selftests/rcutorture/bin/kvm.sh | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) (limited to 'tools') diff --git a/tools/testing/selftests/rcutorture/bin/kvm.sh b/tools/testing/selftests/rcutorture/bin/kvm.sh index 7d1f607f0f76..1fa05132774d 100755 --- a/tools/testing/selftests/rcutorture/bin/kvm.sh +++ b/tools/testing/selftests/rcutorture/bin/kvm.sh @@ -1,10 +1,8 @@ #!/bin/bash # -# Run a series of 14 tests under KVM. These are not particularly -# well-selected or well-tuned, but are the current set. -# -# Edit the definitions below to set the locations of the various directories, -# as well as the test duration. +# Run a series of tests under KVM. By default, this series is specified +# by the relevant CFLIST file, but can be overridden by the --configs +# command-line argument. # # Usage: kvm.sh [ options ] # -- cgit v1.2.3 From 67ad71ddbcae09d891f44d4cd08a9354bc8399b0 Mon Sep 17 00:00:00 2001 From: Lihao Liang Date: Fri, 12 Jan 2018 18:11:32 +0800 Subject: rcutorture: Add basic ARM64 support to run scripts This commit adds support of the qemu command qemu-system-aarch64 to rcutorture. Signed-off-by: Lihao Liang Signed-off-by: Paul E. McKenney --- tools/testing/selftests/rcutorture/bin/functions.sh | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) (limited to 'tools') diff --git a/tools/testing/selftests/rcutorture/bin/functions.sh b/tools/testing/selftests/rcutorture/bin/functions.sh index 07a13779eece..65f6655026f0 100644 --- a/tools/testing/selftests/rcutorture/bin/functions.sh +++ b/tools/testing/selftests/rcutorture/bin/functions.sh @@ -136,6 +136,9 @@ identify_boot_image () { qemu-system-x86_64|qemu-system-i386) echo arch/x86/boot/bzImage ;; + qemu-system-aarch64) + echo arch/arm64/boot/Image + ;; *) echo vmlinux ;; @@ -158,6 +161,9 @@ identify_qemu () { elif echo $u | grep -q "Intel 80386" then echo qemu-system-i386 + elif echo $u | grep -q aarch64 + then + echo qemu-system-aarch64 elif uname -a | grep -q ppc64 then echo qemu-system-ppc64 @@ -176,16 +182,20 @@ identify_qemu () { # Output arguments for the qemu "-append" string based on CPU type # and the TORTURE_QEMU_INTERACTIVE environment variable. identify_qemu_append () { + local console=ttyS0 case "$1" in qemu-system-x86_64|qemu-system-i386) echo noapic selinux=0 initcall_debug debug ;; + qemu-system-aarch64) + console=ttyAMA0 + ;; esac if test -n "$TORTURE_QEMU_INTERACTIVE" then echo root=/dev/sda else - echo console=ttyS0 + echo console=$console fi } @@ -197,6 +207,9 @@ identify_qemu_args () { case "$1" in qemu-system-x86_64|qemu-system-i386) ;; + qemu-system-aarch64) + echo -machine virt,gic-version=host -cpu host + ;; qemu-system-ppc64) echo -enable-kvm -M pseries -nodefaults echo -device spapr-vscsi @@ -254,7 +267,7 @@ specify_qemu_cpus () { echo $2 else case "$1" in - qemu-system-x86_64|qemu-system-i386) + qemu-system-x86_64|qemu-system-i386|qemu-system-aarch64) echo $2 -smp $3 ;; qemu-system-ppc64) -- cgit v1.2.3 From 642146b1ad9bf73b5d1384610d1314d833a560dc Mon Sep 17 00:00:00 2001 From: "Paul E. McKenney" Date: Fri, 12 Jan 2018 14:19:05 -0800 Subject: torture: Specify qemu memory size with --memory argument The 512 megabyte memory size has served quite well, but more memory is required when using large trace buffers on large systems. This commit therefore adds a --memory argument to the kvm.sh script, which allows the memory size to be specified on the command line, for example, "--memory 768", --memory 800M", or "--memory 2G". Reported-by: Sebastian Siewior Signed-off-by: Paul E. McKenney --- tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh | 4 ++-- tools/testing/selftests/rcutorture/bin/kvm.sh | 8 ++++++++ 2 files changed, 10 insertions(+), 2 deletions(-) (limited to 'tools') diff --git a/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh index 1b78a12740e5..5f8fbb0d7c17 100755 --- a/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh +++ b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh @@ -177,8 +177,8 @@ then exit 0 fi echo "NOTE: $QEMU either did not run or was interactive" > $resdir/console.log -echo $QEMU $qemu_args -m 512 -kernel $KERNEL -append \"$qemu_append $boot_args\" > $resdir/qemu-cmd -( $QEMU $qemu_args -m 512 -kernel $KERNEL -append "$qemu_append $boot_args"& echo $! > $resdir/qemu_pid; wait `cat $resdir/qemu_pid`; echo $? > $resdir/qemu-retval ) & +echo $QEMU $qemu_args -m $TORTURE_QEMU_MEM -kernel $KERNEL -append \"$qemu_append $boot_args\" > $resdir/qemu-cmd +( $QEMU $qemu_args -m $TORTURE_QEMU_MEM -kernel $KERNEL -append "$qemu_append $boot_args"& echo $! > $resdir/qemu_pid; wait `cat $resdir/qemu_pid`; echo $? > $resdir/qemu-retval ) & commandcompleted=0 sleep 10 # Give qemu's pid a chance to reach the file if test -s "$resdir/qemu_pid" diff --git a/tools/testing/selftests/rcutorture/bin/kvm.sh b/tools/testing/selftests/rcutorture/bin/kvm.sh index 1fa05132774d..f3c3d33150a0 100755 --- a/tools/testing/selftests/rcutorture/bin/kvm.sh +++ b/tools/testing/selftests/rcutorture/bin/kvm.sh @@ -42,6 +42,7 @@ TORTURE_BOOT_IMAGE="" TORTURE_INITRD="$KVM/initrd"; export TORTURE_INITRD TORTURE_KCONFIG_ARG="" TORTURE_KMAKE_ARG="" +TORTURE_QEMU_MEM=512 TORTURE_SHUTDOWN_GRACE=180 TORTURE_SUITE=rcu resdir="" @@ -68,6 +69,7 @@ usage () { echo " --kconfig Kconfig-options" echo " --kmake-arg kernel-make-arguments" echo " --mac nn:nn:nn:nn:nn:nn" + echo " --memory megabytes | nnnG" echo " --no-initrd" echo " --qemu-args qemu-arguments" echo " --qemu-cmd qemu-system-..." @@ -145,6 +147,11 @@ do TORTURE_QEMU_MAC=$2 shift ;; + --memory) + checkarg --memory "(memory size)" $# "$2" '^[0-9]\+[MG]\?$' error + TORTURE_QEMU_MEM=$2 + shift + ;; --no-initrd) TORTURE_INITRD=""; export TORTURE_INITRD ;; @@ -286,6 +293,7 @@ TORTURE_KMAKE_ARG="$TORTURE_KMAKE_ARG"; export TORTURE_KMAKE_ARG TORTURE_QEMU_CMD="$TORTURE_QEMU_CMD"; export TORTURE_QEMU_CMD TORTURE_QEMU_INTERACTIVE="$TORTURE_QEMU_INTERACTIVE"; export TORTURE_QEMU_INTERACTIVE TORTURE_QEMU_MAC="$TORTURE_QEMU_MAC"; export TORTURE_QEMU_MAC +TORTURE_QEMU_MEM="$TORTURE_QEMU_MEM"; export TORTURE_QEMU_MEM TORTURE_SHUTDOWN_GRACE="$TORTURE_SHUTDOWN_GRACE"; export TORTURE_SHUTDOWN_GRACE TORTURE_SUITE="$TORTURE_SUITE"; export TORTURE_SUITE if ! test -e $resdir -- cgit v1.2.3 From adcfe76c61732a0dda5116206c3741733491f1e8 Mon Sep 17 00:00:00 2001 From: "Paul E. McKenney" Date: Tue, 30 Jan 2018 19:36:38 -0800 Subject: torture: Default jitter off when running rcuperf The purpose of jitter is to expose concurrency bugs due to invalid assumptions about forward progress. There is usually little point in jitter when measuring performance. This commit therefore defaults jitter off when running rcuperf. You can override this by specifying the kvm.sh "--jitter" argument -after- the "--torture rcuperf" argument. No idea why you would want this, but if you do, that is how you do it. One example of a conccurrency bug that this jitter might expose is one in which the developer assumed that a given short region of code would be guaranteed to execute within some short time limit. Such assumptions are invalid in virtualized environments because the hupervisor can preempt the guest OS at any point, even when the guest OS thinks that it has disabled interrupts. Signed-off-by: Paul E. McKenney --- tools/testing/selftests/rcutorture/bin/kvm.sh | 6 ++++++ 1 file changed, 6 insertions(+) (limited to 'tools') diff --git a/tools/testing/selftests/rcutorture/bin/kvm.sh b/tools/testing/selftests/rcutorture/bin/kvm.sh index f3c3d33150a0..56610dbbdf73 100755 --- a/tools/testing/selftests/rcutorture/bin/kvm.sh +++ b/tools/testing/selftests/rcutorture/bin/kvm.sh @@ -179,6 +179,12 @@ do checkarg --torture "(suite name)" "$#" "$2" '^\(lock\|rcu\|rcuperf\)$' '^--' TORTURE_SUITE=$2 shift + if test "$TORTURE_SUITE" = rcuperf + then + # If you really want jitter for rcuperf, specify + # it after specifying rcuperf. (But why?) + jitter=0 + fi ;; *) echo Unknown argument $1 -- cgit v1.2.3 From cc839ce55d5c2d08bf25f9ddec6d9ee8854b87f4 Mon Sep 17 00:00:00 2001 From: "Paul E. McKenney" Date: Tue, 30 Jan 2018 22:44:33 -0800 Subject: torture: Adjust rcuperf trace processing to allow for workqueues The rcuperf event-trace processing assumes that expedited grace periods start and end on the same task, an assumption that was violated by moving expedited grace-period processing to workqueues. This commit removes this now-fallacious assumption from rcuperf's event-trace processing. Signed-off-by: Paul E. McKenney --- .../selftests/rcutorture/bin/kvm-recheck-rcuperf-ftrace.sh | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) (limited to 'tools') diff --git a/tools/testing/selftests/rcutorture/bin/kvm-recheck-rcuperf-ftrace.sh b/tools/testing/selftests/rcutorture/bin/kvm-recheck-rcuperf-ftrace.sh index 963f71289d22..dffb553a7bcc 100755 --- a/tools/testing/selftests/rcutorture/bin/kvm-recheck-rcuperf-ftrace.sh +++ b/tools/testing/selftests/rcutorture/bin/kvm-recheck-rcuperf-ftrace.sh @@ -39,7 +39,7 @@ sed -e 's/us : / : /' | tr -d '\015' | awk ' $8 == "start" { - if (starttask != "") + if (startseq != "") nlost++; starttask = $1; starttime = $3; @@ -47,17 +47,17 @@ $8 == "start" { } $8 == "end" { - if (starttask == $1 && startseq == $7) { + if (startseq == $7) { curgpdur = $3 - starttime; gptimes[++n] = curgpdur; gptaskcnt[starttask]++; sum += curgpdur; if (curgpdur > 1000) print "Long GP " starttime "us to " $3 "us (" curgpdur "us)"; - starttask = ""; + startseq = ""; } else { # Lost a message or some such, reset. - starttask = ""; + startseq = ""; nlost++; } } -- cgit v1.2.3 From 0da8c08d71133ba0dd9f5b24ae0b6519e00275d6 Mon Sep 17 00:00:00 2001 From: "Paul E. McKenney" Date: Tue, 30 Jan 2018 23:12:27 -0800 Subject: torture: Grace periods do not piggyback off of themselves The rcuperf trace-event processing counted every "done" trace event as a piggyback, which is incorrect because the task that started the grace period didn't piggyback at all. This commit fixes this problem by recording the task that started a given grace period and ignoring that task's "done" record for that grace period. Signed-off-by: Paul E. McKenney --- tools/testing/selftests/rcutorture/bin/kvm-recheck-rcuperf-ftrace.sh | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) (limited to 'tools') diff --git a/tools/testing/selftests/rcutorture/bin/kvm-recheck-rcuperf-ftrace.sh b/tools/testing/selftests/rcutorture/bin/kvm-recheck-rcuperf-ftrace.sh index dffb553a7bcc..8948f7926b21 100755 --- a/tools/testing/selftests/rcutorture/bin/kvm-recheck-rcuperf-ftrace.sh +++ b/tools/testing/selftests/rcutorture/bin/kvm-recheck-rcuperf-ftrace.sh @@ -44,6 +44,7 @@ $8 == "start" { starttask = $1; starttime = $3; startseq = $7; + seqtask[startseq] = starttask; } $8 == "end" { @@ -62,7 +63,7 @@ $8 == "end" { } } -$8 == "done" { +$8 == "done" && seqtask[$7] != $1 { piggybackcnt[$1]++; } -- cgit v1.2.3 From 85ba6bfe8bb2a4d907f7380a8f37b31616ad694e Mon Sep 17 00:00:00 2001 From: "Paul E. McKenney" Date: Thu, 1 Feb 2018 19:19:04 -0800 Subject: torture: Provide more sensible nreader/nwriter defaults for rcuperf The default values for nreader and nwriter are apparently not all that user-friendly, resulting in people doing scalability tests that ran all runs at large scale. This commit therefore makes both the nreaders and nwriters module default to the number of CPUs, and adds a comment to rcuperf.c stating that the number of CPUs should be specified using the nr_cpus kernel boot parameter. This commit also eliminates the redundant rcuperf scripting specification of default values for these parameters. Signed-off-by: Paul E. McKenney --- kernel/rcu/rcuperf.c | 21 ++++++++++++++++++- .../rcutorture/configs/rcuperf/ver_functions.sh | 24 +--------------------- 2 files changed, 21 insertions(+), 24 deletions(-) (limited to 'tools') diff --git a/kernel/rcu/rcuperf.c b/kernel/rcu/rcuperf.c index d1ebdf9868bb..777e7a6a0292 100644 --- a/kernel/rcu/rcuperf.c +++ b/kernel/rcu/rcuperf.c @@ -61,11 +61,30 @@ MODULE_AUTHOR("Paul E. McKenney "); #define VERBOSE_PERFOUT_ERRSTRING(s) \ do { if (verbose) pr_alert("%s" PERF_FLAG "!!! %s\n", perf_type, s); } while (0) +/* + * The intended use cases for the nreaders and nwriters module parameters + * are as follows: + * + * 1. Specify only the nr_cpus kernel boot parameter. This will + * set both nreaders and nwriters to the value specified by + * nr_cpus for a mixed reader/writer test. + * + * 2. Specify the nr_cpus kernel boot parameter, but set + * rcuperf.nreaders to zero. This will set nwriters to the + * value specified by nr_cpus for an update-only test. + * + * 3. Specify the nr_cpus kernel boot parameter, but set + * rcuperf.nwriters to zero. This will set nreaders to the + * value specified by nr_cpus for a read-only test. + * + * Various other use cases may of course be specified. + */ + torture_param(bool, gp_async, false, "Use asynchronous GP wait primitives"); torture_param(int, gp_async_max, 1000, "Max # outstanding waits per reader"); torture_param(bool, gp_exp, false, "Use expedited GP wait primitives"); torture_param(int, holdoff, 10, "Holdoff time before test start (s)"); -torture_param(int, nreaders, 0, "Number of RCU reader threads"); +torture_param(int, nreaders, -1, "Number of RCU reader threads"); torture_param(int, nwriters, -1, "Number of RCU updater threads"); torture_param(bool, shutdown, !IS_ENABLED(MODULE), "Shutdown at end of performance tests."); diff --git a/tools/testing/selftests/rcutorture/configs/rcuperf/ver_functions.sh b/tools/testing/selftests/rcutorture/configs/rcuperf/ver_functions.sh index b9603115d7c7..d36b8fd6f0fc 100644 --- a/tools/testing/selftests/rcutorture/configs/rcuperf/ver_functions.sh +++ b/tools/testing/selftests/rcutorture/configs/rcuperf/ver_functions.sh @@ -20,32 +20,10 @@ # # Authors: Paul E. McKenney -# rcuperf_param_nreaders bootparam-string -# -# Adds nreaders rcuperf module parameter if not already specified. -rcuperf_param_nreaders () { - if ! echo "$1" | grep -q "rcuperf.nreaders" - then - echo rcuperf.nreaders=-1 - fi -} - -# rcuperf_param_nwriters bootparam-string -# -# Adds nwriters rcuperf module parameter if not already specified. -rcuperf_param_nwriters () { - if ! echo "$1" | grep -q "rcuperf.nwriters" - then - echo rcuperf.nwriters=-1 - fi -} - # per_version_boot_params bootparam-string config-file seconds # # Adds per-version torture-module parameters to kernels supporting them. per_version_boot_params () { - echo $1 `rcuperf_param_nreaders "$1"` \ - `rcuperf_param_nwriters "$1"` \ - rcuperf.shutdown=1 \ + echo $1 rcuperf.shutdown=1 \ rcuperf.verbose=1 } -- cgit v1.2.3 From 48d44d4e8a583c66d9f376e18c1a1fcc445f4b64 Mon Sep 17 00:00:00 2001 From: Andrea Parri Date: Tue, 20 Feb 2018 15:25:01 -0800 Subject: tools/memory-model: Clarify the origin/scope of the tool name Ingo pointed out that: "The "memory model" name is overly generic, ambiguous and somewhat misleading, as we usually mean the virtual memory layout/model when we say "memory model". GCC too uses it in that sense [...]" Make it clear that tools/memory-model/ uses the term "memory model" as shorthand for "memory consistency model" by calling out this convention in tools/memory-model/README. Stick to the original "memory model" term in sources' headers and for the subsystem name. Suggested-by: Ingo Molnar Signed-off-by: Andrea Parri Signed-off-by: Paul E. McKenney Acked-by: Peter Zijlstra Acked-by: Will Deacon Acked-by: Alan Stern Cc: Linus Torvalds Cc: Thomas Gleixner Cc: akiyks@gmail.com Cc: boqun.feng@gmail.com Cc: dhowells@redhat.com Cc: j.alglave@ucl.ac.uk Cc: linux-arch@vger.kernel.org Cc: luc.maranget@inria.fr Cc: nborisov@suse.com Cc: npiggin@gmail.com Link: http://lkml.kernel.org/r/1519169112-20593-1-git-send-email-paulmck@linux.vnet.ibm.com Signed-off-by: Ingo Molnar --- tools/memory-model/MAINTAINERS | 2 +- tools/memory-model/README | 14 +++++++------- tools/memory-model/linux-kernel.bell | 2 +- tools/memory-model/linux-kernel.cat | 2 +- 4 files changed, 10 insertions(+), 10 deletions(-) (limited to 'tools') diff --git a/tools/memory-model/MAINTAINERS b/tools/memory-model/MAINTAINERS index 711cbe72d606..db3bd3fc0435 100644 --- a/tools/memory-model/MAINTAINERS +++ b/tools/memory-model/MAINTAINERS @@ -1,4 +1,4 @@ -LINUX KERNEL MEMORY MODEL +LINUX KERNEL MEMORY CONSISTENCY MODEL M: Alan Stern M: Andrea Parri M: Will Deacon diff --git a/tools/memory-model/README b/tools/memory-model/README index 43ba49492111..91414a49fac5 100644 --- a/tools/memory-model/README +++ b/tools/memory-model/README @@ -1,15 +1,15 @@ - ========================= - LINUX KERNEL MEMORY MODEL - ========================= + ===================================== + LINUX KERNEL MEMORY CONSISTENCY MODEL + ===================================== ============ INTRODUCTION ============ -This directory contains the memory model of the Linux kernel, written -in the "cat" language and executable by the (externally provided) -"herd7" simulator, which exhaustively explores the state space of -small litmus tests. +This directory contains the memory consistency model (memory model, for +short) of the Linux kernel, written in the "cat" language and executable +by the externally provided "herd7" simulator, which exhaustively explores +the state space of small litmus tests. In addition, the "klitmus7" tool (also externally provided) may be used to convert a litmus test to a Linux kernel module, which in turn allows diff --git a/tools/memory-model/linux-kernel.bell b/tools/memory-model/linux-kernel.bell index 57112505f5e0..b984bbda01a5 100644 --- a/tools/memory-model/linux-kernel.bell +++ b/tools/memory-model/linux-kernel.bell @@ -11,7 +11,7 @@ * which is to appear in ASPLOS 2018. *) -"Linux kernel memory model" +"Linux-kernel memory consistency model" enum Accesses = 'once (*READ_ONCE,WRITE_ONCE,ACCESS_ONCE*) || 'release (*smp_store_release*) || diff --git a/tools/memory-model/linux-kernel.cat b/tools/memory-model/linux-kernel.cat index 15b7a5dd8a9a..babe2b3b0bb3 100644 --- a/tools/memory-model/linux-kernel.cat +++ b/tools/memory-model/linux-kernel.cat @@ -11,7 +11,7 @@ * which is to appear in ASPLOS 2018. *) -"Linux kernel memory model" +"Linux-kernel memory consistency model" (* * File "lock.cat" handles locks and is experimental. -- cgit v1.2.3 From e7d74c9f900a12ea0bd5cabb3be142441530e24e Mon Sep 17 00:00:00 2001 From: Andrea Parri Date: Tue, 20 Feb 2018 15:25:02 -0800 Subject: MAINTAINERS: Add the Memory Consistency Model subsystem Move the contents of tools/memory-model/MAINTAINERS into the main MAINTAINERS file, removing tools/memory-model/MAINTAINERS. This allows get_maintainer.pl to correctly identify the maintainers of tools/memory-model/. Suggested-by: Ingo Molnar Signed-off-by: Andrea Parri Signed-off-by: Paul E. McKenney Acked-by: Peter Zijlstra Acked-by: Will Deacon Acked-by: Alan Stern Cc: Linus Torvalds Cc: Thomas Gleixner Cc: akiyks@gmail.com Cc: boqun.feng@gmail.com Cc: dhowells@redhat.com Cc: j.alglave@ucl.ac.uk Cc: linux-arch@vger.kernel.org Cc: luc.maranget@inria.fr Cc: nborisov@suse.com Cc: npiggin@gmail.com Link: http://lkml.kernel.org/r/1519169112-20593-2-git-send-email-paulmck@linux.vnet.ibm.com Signed-off-by: Ingo Molnar --- MAINTAINERS | 16 ++++++++++++++++ tools/memory-model/MAINTAINERS | 15 --------------- 2 files changed, 16 insertions(+), 15 deletions(-) delete mode 100644 tools/memory-model/MAINTAINERS (limited to 'tools') diff --git a/MAINTAINERS b/MAINTAINERS index 9a7f76eadae9..654c6c6154a3 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8148,6 +8148,22 @@ M: Kees Cook S: Maintained F: drivers/misc/lkdtm* +LINUX KERNEL MEMORY CONSISTENCY MODEL (LKMM) +M: Alan Stern +M: Andrea Parri +M: Will Deacon +M: Peter Zijlstra +M: Boqun Feng +M: Nicholas Piggin +M: David Howells +M: Jade Alglave +M: Luc Maranget +M: "Paul E. McKenney" +L: linux-kernel@vger.kernel.org +S: Supported +T: git git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git +F: tools/memory-model/ + LINUX SECURITY MODULE (LSM) FRAMEWORK M: Chris Wright L: linux-security-module@vger.kernel.org diff --git a/tools/memory-model/MAINTAINERS b/tools/memory-model/MAINTAINERS deleted file mode 100644 index db3bd3fc0435..000000000000 --- a/tools/memory-model/MAINTAINERS +++ /dev/null @@ -1,15 +0,0 @@ -LINUX KERNEL MEMORY CONSISTENCY MODEL -M: Alan Stern -M: Andrea Parri -M: Will Deacon -M: Peter Zijlstra -M: Boqun Feng -M: Nicholas Piggin -M: David Howells -M: Jade Alglave -M: Luc Maranget -M: "Paul E. McKenney" -L: linux-kernel@vger.kernel.org -S: Supported -T: git git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git -F: tools/memory-model/ -- cgit v1.2.3 From 8f32543b61d7daeddb5b64c80b5ad5f05cc97722 Mon Sep 17 00:00:00 2001 From: "Paul E. McKenney" Date: Tue, 20 Feb 2018 15:25:04 -0800 Subject: EXP litmus_tests: Add comments explaining tests' purposes This commit adds comments to the litmus tests summarizing what these tests are intended to demonstrate. [ paulmck: Apply Andrea's and Alan's feedback. ] Suggested-by: Ingo Molnar Signed-off-by: Paul E. McKenney Acked-by: Peter Zijlstra Cc: Linus Torvalds Cc: Thomas Gleixner Cc: akiyks@gmail.com Cc: boqun.feng@gmail.com Cc: dhowells@redhat.com Cc: j.alglave@ucl.ac.uk Cc: linux-arch@vger.kernel.org Cc: luc.maranget@inria.fr Cc: nborisov@suse.com Cc: npiggin@gmail.com Cc: parri.andrea@gmail.com Cc: stern@rowland.harvard.edu Cc: will.deacon@arm.com Link: http://lkml.kernel.org/r/1519169112-20593-4-git-send-email-paulmck@linux.vnet.ibm.com Signed-off-by: Ingo Molnar --- .../memory-model/litmus-tests/CoRR+poonceonce+Once.litmus | 7 +++++++ .../memory-model/litmus-tests/CoRW+poonceonce+Once.litmus | 7 +++++++ .../memory-model/litmus-tests/CoWR+poonceonce+Once.litmus | 7 +++++++ tools/memory-model/litmus-tests/CoWW+poonceonce.litmus | 7 +++++++ .../litmus-tests/IRIW+mbonceonces+OnceOnce.litmus | 10 ++++++++++ .../litmus-tests/IRIW+poonceonces+OnceOnce.litmus | 10 ++++++++++ tools/memory-model/litmus-tests/ISA2+poonceonces.litmus | 9 +++++++++ ...SA2+pooncerelease+poacquirerelease+poacquireonce.litmus | 11 +++++++++++ .../litmus-tests/LB+ctrlonceonce+mbonceonce.litmus | 11 +++++++++++ .../litmus-tests/LB+poacquireonce+pooncerelease.litmus | 8 ++++++++ tools/memory-model/litmus-tests/LB+poonceonces.litmus | 7 +++++++ .../litmus-tests/MP+onceassign+derefonce.litmus | 11 ++++++++++- tools/memory-model/litmus-tests/MP+polocks.litmus | 11 +++++++++++ tools/memory-model/litmus-tests/MP+poonceonces.litmus | 7 +++++++ .../litmus-tests/MP+pooncerelease+poacquireonce.litmus | 8 ++++++++ tools/memory-model/litmus-tests/MP+porevlocks.litmus | 11 +++++++++++ .../litmus-tests/MP+wmbonceonce+rmbonceonce.litmus | 8 ++++++++ tools/memory-model/litmus-tests/R+mbonceonces.litmus | 9 +++++++++ tools/memory-model/litmus-tests/R+poonceonces.litmus | 8 ++++++++ tools/memory-model/litmus-tests/S+poonceonces.litmus | 9 +++++++++ .../litmus-tests/S+wmbonceonce+poacquireonce.litmus | 7 +++++++ tools/memory-model/litmus-tests/SB+mbonceonces.litmus | 9 +++++++++ tools/memory-model/litmus-tests/SB+poonceonces.litmus | 8 ++++++++ .../memory-model/litmus-tests/WRC+poonceonces+Once.litmus | 8 ++++++++ .../litmus-tests/WRC+pooncerelease+rmbonceonce+Once.litmus | 8 ++++++++ .../Z6.0+pooncelock+poonceLock+pombonce.litmus | 9 +++++++++ .../Z6.0+pooncelock+pooncelock+pombonce.litmus | 8 ++++++++ .../Z6.0+pooncerelease+poacquirerelease+mbonceonce.litmus | 14 ++++++++++++++ 28 files changed, 246 insertions(+), 1 deletion(-) (limited to 'tools') diff --git a/tools/memory-model/litmus-tests/CoRR+poonceonce+Once.litmus b/tools/memory-model/litmus-tests/CoRR+poonceonce+Once.litmus index 5b83d57f6ac5..967f9f2a6226 100644 --- a/tools/memory-model/litmus-tests/CoRR+poonceonce+Once.litmus +++ b/tools/memory-model/litmus-tests/CoRR+poonceonce+Once.litmus @@ -1,5 +1,12 @@ C CoRR+poonceonce+Once +(* + * Result: Never + * + * Test of read-read coherence, that is, whether or not two successive + * reads from the same variable are ordered. + *) + {} P0(int *x) diff --git a/tools/memory-model/litmus-tests/CoRW+poonceonce+Once.litmus b/tools/memory-model/litmus-tests/CoRW+poonceonce+Once.litmus index fab91c13d52c..4635739f3974 100644 --- a/tools/memory-model/litmus-tests/CoRW+poonceonce+Once.litmus +++ b/tools/memory-model/litmus-tests/CoRW+poonceonce+Once.litmus @@ -1,5 +1,12 @@ C CoRW+poonceonce+Once +(* + * Result: Never + * + * Test of read-write coherence, that is, whether or not a read from + * a given variable and a later write to that same variable are ordered. + *) + {} P0(int *x) diff --git a/tools/memory-model/litmus-tests/CoWR+poonceonce+Once.litmus b/tools/memory-model/litmus-tests/CoWR+poonceonce+Once.litmus index 6a35ec2042ea..bb068c92d8da 100644 --- a/tools/memory-model/litmus-tests/CoWR+poonceonce+Once.litmus +++ b/tools/memory-model/litmus-tests/CoWR+poonceonce+Once.litmus @@ -1,5 +1,12 @@ C CoWR+poonceonce+Once +(* + * Result: Never + * + * Test of write-read coherence, that is, whether or not a write to a + * given variable and a later read from that same variable are ordered. + *) + {} P0(int *x) diff --git a/tools/memory-model/litmus-tests/CoWW+poonceonce.litmus b/tools/memory-model/litmus-tests/CoWW+poonceonce.litmus index 32a96b832021..0d9f0a958799 100644 --- a/tools/memory-model/litmus-tests/CoWW+poonceonce.litmus +++ b/tools/memory-model/litmus-tests/CoWW+poonceonce.litmus @@ -1,5 +1,12 @@ C CoWW+poonceonce +(* + * Result: Never + * + * Test of write-write coherence, that is, whether or not two successive + * writes to the same variable are ordered. + *) + {} P0(int *x) diff --git a/tools/memory-model/litmus-tests/IRIW+mbonceonces+OnceOnce.litmus b/tools/memory-model/litmus-tests/IRIW+mbonceonces+OnceOnce.litmus index 7eba2c68992b..50d5db9ea983 100644 --- a/tools/memory-model/litmus-tests/IRIW+mbonceonces+OnceOnce.litmus +++ b/tools/memory-model/litmus-tests/IRIW+mbonceonces+OnceOnce.litmus @@ -1,5 +1,15 @@ C IRIW+mbonceonces+OnceOnce +(* + * Result: Never + * + * Test of independent reads from independent writes with smp_mb() + * between each pairs of reads. In other words, is smp_mb() sufficient to + * cause two different reading processes to agree on the order of a pair + * of writes, where each write is to a different variable by a different + * process? + *) + {} P0(int *x) diff --git a/tools/memory-model/litmus-tests/IRIW+poonceonces+OnceOnce.litmus b/tools/memory-model/litmus-tests/IRIW+poonceonces+OnceOnce.litmus index b0556c6c75d4..4b54dd6a6cd9 100644 --- a/tools/memory-model/litmus-tests/IRIW+poonceonces+OnceOnce.litmus +++ b/tools/memory-model/litmus-tests/IRIW+poonceonces+OnceOnce.litmus @@ -1,5 +1,15 @@ C IRIW+poonceonces+OnceOnce +(* + * Result: Sometimes + * + * Test of independent reads from independent writes with nothing + * between each pairs of reads. In other words, is anything at all + * needed to cause two different reading processes to agree on the order + * of a pair of writes, where each write is to a different variable by a + * different process? + *) + {} P0(int *x) diff --git a/tools/memory-model/litmus-tests/ISA2+poonceonces.litmus b/tools/memory-model/litmus-tests/ISA2+poonceonces.litmus index 9a1a233d70c3..b321aa6f4ea5 100644 --- a/tools/memory-model/litmus-tests/ISA2+poonceonces.litmus +++ b/tools/memory-model/litmus-tests/ISA2+poonceonces.litmus @@ -1,5 +1,14 @@ C ISA2+poonceonces +(* + * Result: Sometimes + * + * Given a release-acquire chain ordering the first process's store + * against the last process's load, is ordering preserved if all of the + * smp_store_release() invocations are replaced by WRITE_ONCE() and all + * of the smp_load_acquire() invocations are replaced by READ_ONCE()? + *) + {} P0(int *x, int *y) diff --git a/tools/memory-model/litmus-tests/ISA2+pooncerelease+poacquirerelease+poacquireonce.litmus b/tools/memory-model/litmus-tests/ISA2+pooncerelease+poacquirerelease+poacquireonce.litmus index 235195e87d4e..025b0462ec9b 100644 --- a/tools/memory-model/litmus-tests/ISA2+pooncerelease+poacquirerelease+poacquireonce.litmus +++ b/tools/memory-model/litmus-tests/ISA2+pooncerelease+poacquirerelease+poacquireonce.litmus @@ -1,5 +1,16 @@ C ISA2+pooncerelease+poacquirerelease+poacquireonce +(* + * Result: Never + * + * This litmus test demonstrates that a release-acquire chain suffices + * to order P0()'s initial write against P2()'s final read. The reason + * that the release-acquire chain suffices is because in all but one + * case (P2() to P0()), each process reads from the preceding process's + * write. In memory-model-speak, there is only one non-reads-from + * (AKA non-rf) link, so release-acquire is all that is needed. + *) + {} P0(int *x, int *y) diff --git a/tools/memory-model/litmus-tests/LB+ctrlonceonce+mbonceonce.litmus b/tools/memory-model/litmus-tests/LB+ctrlonceonce+mbonceonce.litmus index dd5ac3a8974a..de6708229dd1 100644 --- a/tools/memory-model/litmus-tests/LB+ctrlonceonce+mbonceonce.litmus +++ b/tools/memory-model/litmus-tests/LB+ctrlonceonce+mbonceonce.litmus @@ -1,5 +1,16 @@ C LB+ctrlonceonce+mbonceonce +(* + * Result: Never + * + * This litmus test demonstrates that lightweight ordering suffices for + * the load-buffering pattern, in other words, preventing all processes + * reading from the preceding process's write. In this example, the + * combination of a control dependency and a full memory barrier are enough + * to do the trick. (But the full memory barrier could be replaced with + * another control dependency and order would still be maintained.) + *) + {} P0(int *x, int *y) diff --git a/tools/memory-model/litmus-tests/LB+poacquireonce+pooncerelease.litmus b/tools/memory-model/litmus-tests/LB+poacquireonce+pooncerelease.litmus index 47bd61319d93..07b9904b0e49 100644 --- a/tools/memory-model/litmus-tests/LB+poacquireonce+pooncerelease.litmus +++ b/tools/memory-model/litmus-tests/LB+poacquireonce+pooncerelease.litmus @@ -1,5 +1,13 @@ C LB+poacquireonce+pooncerelease +(* + * Result: Never + * + * Does a release-acquire pair suffice for the load-buffering litmus + * test, where each process reads from one of two variables then writes + * to the other? + *) + {} P0(int *x, int *y) diff --git a/tools/memory-model/litmus-tests/LB+poonceonces.litmus b/tools/memory-model/litmus-tests/LB+poonceonces.litmus index a5cdf027e34b..74c49cb3c37b 100644 --- a/tools/memory-model/litmus-tests/LB+poonceonces.litmus +++ b/tools/memory-model/litmus-tests/LB+poonceonces.litmus @@ -1,5 +1,12 @@ C LB+poonceonces +(* + * Result: Sometimes + * + * Can the counter-intuitive outcome for the load-buffering pattern + * be prevented even with no explicit ordering? + *) + {} P0(int *x, int *y) diff --git a/tools/memory-model/litmus-tests/MP+onceassign+derefonce.litmus b/tools/memory-model/litmus-tests/MP+onceassign+derefonce.litmus index 1a2fe5830381..97731b4bbdd8 100644 --- a/tools/memory-model/litmus-tests/MP+onceassign+derefonce.litmus +++ b/tools/memory-model/litmus-tests/MP+onceassign+derefonce.litmus @@ -1,4 +1,13 @@ -C MP+onceassign+derefonce.litmus +C MP+onceassign+derefonce + +(* + * Result: Never + * + * This litmus test demonstrates that rcu_assign_pointer() and + * rcu_dereference() suffice to ensure that an RCU reader will not see + * pre-initialization garbage when it traverses an RCU-protected data + * structure containing a newly inserted element. + *) { y=z; diff --git a/tools/memory-model/litmus-tests/MP+polocks.litmus b/tools/memory-model/litmus-tests/MP+polocks.litmus index 5fe6f1e3c452..712a4fcdf6ce 100644 --- a/tools/memory-model/litmus-tests/MP+polocks.litmus +++ b/tools/memory-model/litmus-tests/MP+polocks.litmus @@ -1,5 +1,16 @@ C MP+polocks +(* + * Result: Never + * + * This litmus test demonstrates how lock acquisitions and releases can + * stand in for smp_load_acquire() and smp_store_release(), respectively. + * In other words, when holding a given lock (or indeed after releasing a + * given lock), a CPU is not only guaranteed to see the accesses that other + * CPUs made while previously holding that lock, it is also guaranteed + * to see all prior accesses by those other CPUs. + *) + {} P0(int *x, int *y, spinlock_t *mylock) diff --git a/tools/memory-model/litmus-tests/MP+poonceonces.litmus b/tools/memory-model/litmus-tests/MP+poonceonces.litmus index 46e1ac7ba126..b2b60b84fb9d 100644 --- a/tools/memory-model/litmus-tests/MP+poonceonces.litmus +++ b/tools/memory-model/litmus-tests/MP+poonceonces.litmus @@ -1,5 +1,12 @@ C MP+poonceonces +(* + * Result: Maybe + * + * Can the counter-intuitive message-passing outcome be prevented with + * no ordering at all? + *) + {} P0(int *x, int *y) diff --git a/tools/memory-model/litmus-tests/MP+pooncerelease+poacquireonce.litmus b/tools/memory-model/litmus-tests/MP+pooncerelease+poacquireonce.litmus index 0b00cc7293ba..d52c68429722 100644 --- a/tools/memory-model/litmus-tests/MP+pooncerelease+poacquireonce.litmus +++ b/tools/memory-model/litmus-tests/MP+pooncerelease+poacquireonce.litmus @@ -1,5 +1,13 @@ C MP+pooncerelease+poacquireonce +(* + * Result: Never + * + * This litmus test demonstrates that smp_store_release() and + * smp_load_acquire() provide sufficient ordering for the message-passing + * pattern. + *) + {} P0(int *x, int *y) diff --git a/tools/memory-model/litmus-tests/MP+porevlocks.litmus b/tools/memory-model/litmus-tests/MP+porevlocks.litmus index 90d011c34f33..72c9276b363e 100644 --- a/tools/memory-model/litmus-tests/MP+porevlocks.litmus +++ b/tools/memory-model/litmus-tests/MP+porevlocks.litmus @@ -1,5 +1,16 @@ C MP+porevlocks +(* + * Result: Never + * + * This litmus test demonstrates how lock acquisitions and releases can + * stand in for smp_load_acquire() and smp_store_release(), respectively. + * In other words, when holding a given lock (or indeed after releasing a + * given lock), a CPU is not only guaranteed to see the accesses that other + * CPUs made while previously holding that lock, it is also guaranteed to + * see all prior accesses by those other CPUs. + *) + {} P0(int *x, int *y, spinlock_t *mylock) diff --git a/tools/memory-model/litmus-tests/MP+wmbonceonce+rmbonceonce.litmus b/tools/memory-model/litmus-tests/MP+wmbonceonce+rmbonceonce.litmus index 604ad41ea0c2..c078f38ff27a 100644 --- a/tools/memory-model/litmus-tests/MP+wmbonceonce+rmbonceonce.litmus +++ b/tools/memory-model/litmus-tests/MP+wmbonceonce+rmbonceonce.litmus @@ -1,5 +1,13 @@ C MP+wmbonceonce+rmbonceonce +(* + * Result: Never + * + * This litmus test demonstrates that smp_wmb() and smp_rmb() provide + * sufficient ordering for the message-passing pattern. However, it + * is usually better to use smp_store_release() and smp_load_acquire(). + *) + {} P0(int *x, int *y) diff --git a/tools/memory-model/litmus-tests/R+mbonceonces.litmus b/tools/memory-model/litmus-tests/R+mbonceonces.litmus index e69b9e3e9436..a0e884ad2132 100644 --- a/tools/memory-model/litmus-tests/R+mbonceonces.litmus +++ b/tools/memory-model/litmus-tests/R+mbonceonces.litmus @@ -1,5 +1,14 @@ C R+mbonceonces +(* + * Result: Never + * + * This is the fully ordered (via smp_mb()) version of one of the classic + * counterintuitive litmus tests that illustrates the effects of store + * propagation delays. Note that weakening either of the barriers would + * cause the resulting test to be allowed. + *) + {} P0(int *x, int *y) diff --git a/tools/memory-model/litmus-tests/R+poonceonces.litmus b/tools/memory-model/litmus-tests/R+poonceonces.litmus index f7a12e00f82d..5386f128a131 100644 --- a/tools/memory-model/litmus-tests/R+poonceonces.litmus +++ b/tools/memory-model/litmus-tests/R+poonceonces.litmus @@ -1,5 +1,13 @@ C R+poonceonces +(* + * Result: Sometimes + * + * This is the unordered (thus lacking smp_mb()) version of one of the + * classic counterintuitive litmus tests that illustrates the effects of + * store propagation delays. + *) + {} P0(int *x, int *y) diff --git a/tools/memory-model/litmus-tests/S+poonceonces.litmus b/tools/memory-model/litmus-tests/S+poonceonces.litmus index d0d541c8ec7d..8c9c2f81a580 100644 --- a/tools/memory-model/litmus-tests/S+poonceonces.litmus +++ b/tools/memory-model/litmus-tests/S+poonceonces.litmus @@ -1,5 +1,14 @@ C S+poonceonces +(* + * Result: Sometimes + * + * Starting with a two-process release-acquire chain ordering P0()'s + * first store against P1()'s final load, if the smp_store_release() + * is replaced by WRITE_ONCE() and the smp_load_acquire() replaced by + * READ_ONCE(), is ordering preserved? + *) + {} P0(int *x, int *y) diff --git a/tools/memory-model/litmus-tests/S+wmbonceonce+poacquireonce.litmus b/tools/memory-model/litmus-tests/S+wmbonceonce+poacquireonce.litmus index 1d292d0d6603..c53350205d28 100644 --- a/tools/memory-model/litmus-tests/S+wmbonceonce+poacquireonce.litmus +++ b/tools/memory-model/litmus-tests/S+wmbonceonce+poacquireonce.litmus @@ -1,5 +1,12 @@ C S+wmbonceonce+poacquireonce +(* + * Result: Never + * + * Can a smp_wmb(), instead of a release, and an acquire order a prior + * store against a subsequent store? + *) + {} P0(int *x, int *y) diff --git a/tools/memory-model/litmus-tests/SB+mbonceonces.litmus b/tools/memory-model/litmus-tests/SB+mbonceonces.litmus index b76caa5af1af..74b874ffa8da 100644 --- a/tools/memory-model/litmus-tests/SB+mbonceonces.litmus +++ b/tools/memory-model/litmus-tests/SB+mbonceonces.litmus @@ -1,5 +1,14 @@ C SB+mbonceonces +(* + * Result: Never + * + * This litmus test demonstrates that full memory barriers suffice to + * order the store-buffering pattern, where each process writes to the + * variable that the preceding process reads. (Locking and RCU can also + * suffice, but not much else.) + *) + {} P0(int *x, int *y) diff --git a/tools/memory-model/litmus-tests/SB+poonceonces.litmus b/tools/memory-model/litmus-tests/SB+poonceonces.litmus index c1797e03807e..10d550730b25 100644 --- a/tools/memory-model/litmus-tests/SB+poonceonces.litmus +++ b/tools/memory-model/litmus-tests/SB+poonceonces.litmus @@ -1,5 +1,13 @@ C SB+poonceonces +(* + * Result: Sometimes + * + * This litmus test demonstrates that at least some ordering is required + * to order the store-buffering pattern, where each process writes to the + * variable that the preceding process reads. + *) + {} P0(int *x, int *y) diff --git a/tools/memory-model/litmus-tests/WRC+poonceonces+Once.litmus b/tools/memory-model/litmus-tests/WRC+poonceonces+Once.litmus index f5e7c92f61cc..6a2bc12a1af1 100644 --- a/tools/memory-model/litmus-tests/WRC+poonceonces+Once.litmus +++ b/tools/memory-model/litmus-tests/WRC+poonceonces+Once.litmus @@ -1,5 +1,13 @@ C WRC+poonceonces+Once +(* + * Result: Sometimes + * + * This litmus test is an extension of the message-passing pattern, + * where the first write is moved to a separate process. Note that this + * test has no ordering at all. + *) + {} P0(int *x) diff --git a/tools/memory-model/litmus-tests/WRC+pooncerelease+rmbonceonce+Once.litmus b/tools/memory-model/litmus-tests/WRC+pooncerelease+rmbonceonce+Once.litmus index e3d0018025dd..97fcbffde9a0 100644 --- a/tools/memory-model/litmus-tests/WRC+pooncerelease+rmbonceonce+Once.litmus +++ b/tools/memory-model/litmus-tests/WRC+pooncerelease+rmbonceonce+Once.litmus @@ -1,5 +1,13 @@ C WRC+pooncerelease+rmbonceonce+Once +(* + * Result: Never + * + * This litmus test is an extension of the message-passing pattern, where + * the first write is moved to a separate process. Because it features + * a release and a read memory barrier, it should be forbidden. + *) + {} P0(int *x) diff --git a/tools/memory-model/litmus-tests/Z6.0+pooncelock+poonceLock+pombonce.litmus b/tools/memory-model/litmus-tests/Z6.0+pooncelock+poonceLock+pombonce.litmus index 9c2cb53e6ef0..415248fb6699 100644 --- a/tools/memory-model/litmus-tests/Z6.0+pooncelock+poonceLock+pombonce.litmus +++ b/tools/memory-model/litmus-tests/Z6.0+pooncelock+poonceLock+pombonce.litmus @@ -1,5 +1,14 @@ C Z6.0+pooncelock+poonceLock+pombonce +(* + * Result: Never + * + * This litmus test demonstrates how smp_mb__after_spinlock() may be + * used to ensure that accesses in different critical sections for a + * given lock running on different CPUs are nevertheless seen in order + * by CPUs not holding that lock. + *) + {} P0(int *x, int *y, spinlock_t *mylock) diff --git a/tools/memory-model/litmus-tests/Z6.0+pooncelock+pooncelock+pombonce.litmus b/tools/memory-model/litmus-tests/Z6.0+pooncelock+pooncelock+pombonce.litmus index c9a1f1a49ae1..10a2aa04cd07 100644 --- a/tools/memory-model/litmus-tests/Z6.0+pooncelock+pooncelock+pombonce.litmus +++ b/tools/memory-model/litmus-tests/Z6.0+pooncelock+pooncelock+pombonce.litmus @@ -1,5 +1,13 @@ C Z6.0+pooncelock+pooncelock+pombonce +(* + * Result: Sometimes + * + * This example demonstrates that a pair of accesses made by different + * processes each while holding a given lock will not necessarily be + * seen as ordered by a third process not holding that lock. + *) + {} P0(int *x, int *y, spinlock_t *mylock) diff --git a/tools/memory-model/litmus-tests/Z6.0+pooncerelease+poacquirerelease+mbonceonce.litmus b/tools/memory-model/litmus-tests/Z6.0+pooncerelease+poacquirerelease+mbonceonce.litmus index 25409a033514..a20fc3fafb53 100644 --- a/tools/memory-model/litmus-tests/Z6.0+pooncerelease+poacquirerelease+mbonceonce.litmus +++ b/tools/memory-model/litmus-tests/Z6.0+pooncerelease+poacquirerelease+mbonceonce.litmus @@ -1,5 +1,19 @@ C Z6.0+pooncerelease+poacquirerelease+mbonceonce +(* + * Result: Sometimes + * + * This litmus test shows that a release-acquire chain, while sufficient + * when there is but one non-reads-from (AKA non-rf) link, does not suffice + * if there is more than one. Of the three processes, only P1() reads from + * P0's write, which means that there are two non-rf links: P1() to P2() + * is a write-to-write link (AKA a "coherence" or just "co" link) and P2() + * to P0() is a read-to-write link (AKA a "from-reads" or just "fr" link). + * When there are two or more non-rf links, you typically will need one + * full barrier for each non-rf link. (Exceptions include some cases + * involving locking.) + *) + {} P0(int *x, int *y) -- cgit v1.2.3 From 62155147048f6c811b82cbb53bee246aee083774 Mon Sep 17 00:00:00 2001 From: "Paul E. McKenney" Date: Tue, 20 Feb 2018 15:25:05 -0800 Subject: README: Fix a couple of punctuation errors Signed-off-by: Paul E. McKenney Acked-by: Peter Zijlstra Cc: Linus Torvalds Cc: Thomas Gleixner Cc: akiyks@gmail.com Cc: boqun.feng@gmail.com Cc: dhowells@redhat.com Cc: j.alglave@ucl.ac.uk Cc: linux-arch@vger.kernel.org Cc: luc.maranget@inria.fr Cc: nborisov@suse.com Cc: npiggin@gmail.com Cc: parri.andrea@gmail.com Cc: stern@rowland.harvard.edu Cc: will.deacon@arm.com Link: http://lkml.kernel.org/r/1519169112-20593-5-git-send-email-paulmck@linux.vnet.ibm.com Signed-off-by: Ingo Molnar --- tools/memory-model/litmus-tests/README | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'tools') diff --git a/tools/memory-model/litmus-tests/README b/tools/memory-model/litmus-tests/README index 9a3bb5949191..dca7d823ad57 100644 --- a/tools/memory-model/litmus-tests/README +++ b/tools/memory-model/litmus-tests/README @@ -23,14 +23,14 @@ IRIW+mbonceonces+OnceOnce.litmus between each pairs of reads. In other words, is smp_mb() sufficient to cause two different reading processes to agree on the order of a pair of writes, where each write is to a different - variable by a different process. + variable by a different process? IRIW+poonceonces+OnceOnce.litmus Test of independent reads from independent writes with nothing between each pairs of reads. In other words, is anything at all needed to cause two different reading processes to agree on the order of a pair of writes, where each write is to a different - variable by a different process. + variable by a different process? ISA2+poonceonces.litmus As below, but with store-release replaced with WRITE_ONCE() -- cgit v1.2.3 From 8f7f2fbd00898deaf01e05a00095411811befd64 Mon Sep 17 00:00:00 2001 From: "Paul E. McKenney" Date: Tue, 20 Feb 2018 15:25:09 -0800 Subject: tools/memory-model: Add required herd7 version to README file LKMM and the herd7 tool are co-evolving, and out-of-date herd7 tools produce inaccurate results, often with no obvious error messages. This commit therefore adds the required herd7 version to the LKMM README file. Longer term, it would be good if .cat files could specify the required version in a manner allowing herd7 to produce clear diagnostics. Suggested-by: Akira Yokosawa Signed-off-by: Paul E. McKenney Acked-by: Peter Zijlstra Cc: Linus Torvalds Cc: Thomas Gleixner Cc: boqun.feng@gmail.com Cc: dhowells@redhat.com Cc: j.alglave@ucl.ac.uk Cc: linux-arch@vger.kernel.org Cc: luc.maranget@inria.fr Cc: nborisov@suse.com Cc: npiggin@gmail.com Cc: parri.andrea@gmail.com Cc: stern@rowland.harvard.edu Cc: will.deacon@arm.com Link: http://lkml.kernel.org/r/1519169112-20593-9-git-send-email-paulmck@linux.vnet.ibm.com Signed-off-by: Ingo Molnar --- tools/memory-model/README | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) (limited to 'tools') diff --git a/tools/memory-model/README b/tools/memory-model/README index 91414a49fac5..ea950c566ffd 100644 --- a/tools/memory-model/README +++ b/tools/memory-model/README @@ -20,7 +20,8 @@ that litmus test to be exercised within the Linux kernel. REQUIREMENTS ============ -The "herd7" and "klitmus7" tools must be downloaded separately: +Version 7.48 of the "herd7" and "klitmus7" tools must be downloaded +separately: https://github.com/herd/herdtools7 -- cgit v1.2.3 From 556bb7d252ae42d4653557325670e665087c38ad Mon Sep 17 00:00:00 2001 From: Alan Stern Date: Tue, 20 Feb 2018 15:25:10 -0800 Subject: tools/memory-model: Add a S lock-based external-view litmus test This commit adds a litmus test in which P0() and P1() form a lock-based S litmus test, with the addition of P2(), which observes P0()'s and P1()'s accesses with a full memory barrier but without the lock. This litmus test asks whether writes carried out by two different processes under the same lock will be seen in order by a third process not holding that lock. The answer to this question is "yes" for all architectures supporting the Linux kernel, but is "no" according to the current version of LKMM. A patch to LKMM is under development. Signed-off-by: Alan Stern Signed-off-by: Paul E. McKenney Acked-by: Peter Zijlstra Cc: Linus Torvalds Cc: Thomas Gleixner Cc: akiyks@gmail.com Cc: boqun.feng@gmail.com Cc: dhowells@redhat.com Cc: j.alglave@ucl.ac.uk Cc: linux-arch@vger.kernel.org Cc: luc.maranget@inria.fr Cc: nborisov@suse.com Cc: npiggin@gmail.com Cc: parri.andrea@gmail.com Cc: will.deacon@arm.com Link: http://lkml.kernel.org/r/1519169112-20593-10-git-send-email-paulmck@linux.vnet.ibm.com Signed-off-by: Ingo Molnar --- .../ISA2+pooncelock+pooncelock+pombonce.litmus | 41 ++++++++++++++++++++++ 1 file changed, 41 insertions(+) create mode 100644 tools/memory-model/litmus-tests/ISA2+pooncelock+pooncelock+pombonce.litmus (limited to 'tools') diff --git a/tools/memory-model/litmus-tests/ISA2+pooncelock+pooncelock+pombonce.litmus b/tools/memory-model/litmus-tests/ISA2+pooncelock+pooncelock+pombonce.litmus new file mode 100644 index 000000000000..7a39a0aaa976 --- /dev/null +++ b/tools/memory-model/litmus-tests/ISA2+pooncelock+pooncelock+pombonce.litmus @@ -0,0 +1,41 @@ +C ISA2+pooncelock+pooncelock+pombonce.litmus + +(* + * Result: Sometimes + * + * This test shows that the ordering provided by a lock-protected S + * litmus test (P0() and P1()) are not visible to external process P2(). + * This is likely to change soon. + *) + +{} + +P0(int *x, int *y, spinlock_t *mylock) +{ + spin_lock(mylock); + WRITE_ONCE(*x, 1); + WRITE_ONCE(*y, 1); + spin_unlock(mylock); +} + +P1(int *y, int *z, spinlock_t *mylock) +{ + int r0; + + spin_lock(mylock); + r0 = READ_ONCE(*y); + WRITE_ONCE(*z, 1); + spin_unlock(mylock); +} + +P2(int *x, int *z) +{ + int r1; + int r2; + + r2 = READ_ONCE(*z); + smp_mb(); + r1 = READ_ONCE(*x); +} + +exists (1:r0=1 /\ 2:r2=1 /\ 2:r1=0) -- cgit v1.2.3 From cac79a39f200ef73ae7fc8a429ce2859ebb118d9 Mon Sep 17 00:00:00 2001 From: "Paul E. McKenney" Date: Tue, 20 Feb 2018 15:25:11 -0800 Subject: tools/memory-model: Convert underscores to hyphens Typical cat-language code uses hyphens for word separators in identifiers, but several LKMM identifiers use underscores instead. This commit therefore converts underscores to hyphens in the .bell- and .cat-file identifiers corresponding to smp_mb__before_atomic(), smp_mb__after_atomic(), and smp_mb__after_spinlock(). Signed-off-by: Paul E. McKenney Acked-by: Peter Zijlstra Cc: Linus Torvalds Cc: Thomas Gleixner Cc: akiyks@gmail.com Cc: boqun.feng@gmail.com Cc: dhowells@redhat.com Cc: j.alglave@ucl.ac.uk Cc: linux-arch@vger.kernel.org Cc: luc.maranget@inria.fr Cc: nborisov@suse.com Cc: npiggin@gmail.com Cc: parri.andrea@gmail.com Cc: stern@rowland.harvard.edu Cc: will.deacon@arm.com Link: http://lkml.kernel.org/r/1519169112-20593-11-git-send-email-paulmck@linux.vnet.ibm.com Signed-off-by: Ingo Molnar --- tools/memory-model/linux-kernel.bell | 6 +++--- tools/memory-model/linux-kernel.cat | 6 +++--- tools/memory-model/linux-kernel.def | 6 +++--- 3 files changed, 9 insertions(+), 9 deletions(-) (limited to 'tools') diff --git a/tools/memory-model/linux-kernel.bell b/tools/memory-model/linux-kernel.bell index b984bbda01a5..18885ad15de9 100644 --- a/tools/memory-model/linux-kernel.bell +++ b/tools/memory-model/linux-kernel.bell @@ -28,9 +28,9 @@ enum Barriers = 'wmb (*smp_wmb*) || 'rcu-lock (*rcu_read_lock*) || 'rcu-unlock (*rcu_read_unlock*) || 'sync-rcu (*synchronize_rcu*) || - 'before_atomic (*smp_mb__before_atomic*) || - 'after_atomic (*smp_mb__after_atomic*) || - 'after_spinlock (*smp_mb__after_spinlock*) + 'before-atomic (*smp_mb__before_atomic*) || + 'after-atomic (*smp_mb__after_atomic*) || + 'after-spinlock (*smp_mb__after_spinlock*) instructions F[Barriers] (* Compute matching pairs of nested Rcu-lock and Rcu-unlock *) diff --git a/tools/memory-model/linux-kernel.cat b/tools/memory-model/linux-kernel.cat index babe2b3b0bb3..f0d27f813ec2 100644 --- a/tools/memory-model/linux-kernel.cat +++ b/tools/memory-model/linux-kernel.cat @@ -29,9 +29,9 @@ let rb-dep = [R] ; fencerel(Rb_dep) ; [R] let rmb = [R \ Noreturn] ; fencerel(Rmb) ; [R \ Noreturn] let wmb = [W] ; fencerel(Wmb) ; [W] let mb = ([M] ; fencerel(Mb) ; [M]) | - ([M] ; fencerel(Before_atomic) ; [RMW] ; po? ; [M]) | - ([M] ; po? ; [RMW] ; fencerel(After_atomic) ; [M]) | - ([M] ; po? ; [LKW] ; fencerel(After_spinlock) ; [M]) + ([M] ; fencerel(Before-atomic) ; [RMW] ; po? ; [M]) | + ([M] ; po? ; [RMW] ; fencerel(After-atomic) ; [M]) | + ([M] ; po? ; [LKW] ; fencerel(After-spinlock) ; [M]) let gp = po ; [Sync-rcu] ; po? let strong-fence = mb | gp diff --git a/tools/memory-model/linux-kernel.def b/tools/memory-model/linux-kernel.def index a397387f77cc..f5a1eb04cb64 100644 --- a/tools/memory-model/linux-kernel.def +++ b/tools/memory-model/linux-kernel.def @@ -21,9 +21,9 @@ smp_mb() { __fence{mb} ; } smp_rmb() { __fence{rmb} ; } smp_wmb() { __fence{wmb} ; } smp_read_barrier_depends() { __fence{rb_dep}; } -smp_mb__before_atomic() { __fence{before_atomic} ; } -smp_mb__after_atomic() { __fence{after_atomic} ; } -smp_mb__after_spinlock() { __fence{after_spinlock} ; } +smp_mb__before_atomic() { __fence{before-atomic} ; } +smp_mb__after_atomic() { __fence{after-atomic} ; } +smp_mb__after_spinlock() { __fence{after-spinlock} ; } // Exchange xchg(X,V) __xchg{mb}(X,V) -- cgit v1.2.3 From bf28ae5627442355dbb8d99238da4fb95c2dd4ec Mon Sep 17 00:00:00 2001 From: Alan Stern Date: Tue, 20 Feb 2018 15:25:12 -0800 Subject: tools/memory-model: Remove rb-dep, smp_read_barrier_depends, and lockless_dereference Since commit 76ebbe78f739 ("locking/barriers: Add implicit smp_read_barrier_depends() to READ_ONCE()") was merged for the 4.15 kernel, it has not been necessary to use smp_read_barrier_depends(). Similarly, commit 59ecbbe7b31c ("locking/barriers: Kill lockless_dereference()") removed lockless_dereference() from the kernel. Since these primitives are no longer part of the kernel, they do not belong in the Linux Kernel Memory Consistency Model. This patch removes them, along with the internal rb-dep relation, and updates the revelant documentation. Signed-off-by: Alan Stern Signed-off-by: Paul E. McKenney Acked-by: Peter Zijlstra Cc: Linus Torvalds Cc: Thomas Gleixner Cc: akiyks@gmail.com Cc: boqun.feng@gmail.com Cc: dhowells@redhat.com Cc: j.alglave@ucl.ac.uk Cc: linux-arch@vger.kernel.org Cc: luc.maranget@inria.fr Cc: nborisov@suse.com Cc: npiggin@gmail.com Cc: parri.andrea@gmail.com Cc: will.deacon@arm.com Link: http://lkml.kernel.org/r/1519169112-20593-12-git-send-email-paulmck@linux.vnet.ibm.com Signed-off-by: Ingo Molnar --- tools/memory-model/Documentation/cheatsheet.txt | 3 +- tools/memory-model/Documentation/explanation.txt | 81 +++++++++++++----------- tools/memory-model/linux-kernel.bell | 1 - tools/memory-model/linux-kernel.cat | 7 +- tools/memory-model/linux-kernel.def | 2 - 5 files changed, 46 insertions(+), 48 deletions(-) (limited to 'tools') diff --git a/tools/memory-model/Documentation/cheatsheet.txt b/tools/memory-model/Documentation/cheatsheet.txt index 1917712bce99..04e458acd6d4 100644 --- a/tools/memory-model/Documentation/cheatsheet.txt +++ b/tools/memory-model/Documentation/cheatsheet.txt @@ -6,8 +6,7 @@ Store, e.g., WRITE_ONCE() Y Y Load, e.g., READ_ONCE() Y Y Y Unsuccessful RMW operation Y Y Y -smp_read_barrier_depends() Y Y Y -*_dereference() Y Y Y Y +rcu_dereference() Y Y Y Y Successful *_acquire() R Y Y Y Y Y Y Successful *_release() C Y Y Y W Y smp_rmb() Y R Y Y R diff --git a/tools/memory-model/Documentation/explanation.txt b/tools/memory-model/Documentation/explanation.txt index 867e0ea69b6d..dae8b8cb2ad3 100644 --- a/tools/memory-model/Documentation/explanation.txt +++ b/tools/memory-model/Documentation/explanation.txt @@ -1,5 +1,5 @@ -Explanation of the Linux-Kernel Memory Model -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Explanation of the Linux-Kernel Memory Consistency Model +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ :Author: Alan Stern :Created: October 2017 @@ -35,25 +35,24 @@ Explanation of the Linux-Kernel Memory Model INTRODUCTION ------------ -The Linux-kernel memory model (LKMM) is rather complex and obscure. -This is particularly evident if you read through the linux-kernel.bell -and linux-kernel.cat files that make up the formal version of the -memory model; they are extremely terse and their meanings are far from -clear. +The Linux-kernel memory consistency model (LKMM) is rather complex and +obscure. This is particularly evident if you read through the +linux-kernel.bell and linux-kernel.cat files that make up the formal +version of the model; they are extremely terse and their meanings are +far from clear. This document describes the ideas underlying the LKMM. It is meant -for people who want to understand how the memory model was designed. -It does not go into the details of the code in the .bell and .cat -files; rather, it explains in English what the code expresses -symbolically. +for people who want to understand how the model was designed. It does +not go into the details of the code in the .bell and .cat files; +rather, it explains in English what the code expresses symbolically. Sections 2 (BACKGROUND) through 5 (ORDERING AND CYCLES) are aimed -toward beginners; they explain what memory models are and the basic -notions shared by all such models. People already familiar with these -concepts can skim or skip over them. Sections 6 (EVENTS) through 12 -(THE FROM_READS RELATION) describe the fundamental relations used in -many memory models. Starting in Section 13 (AN OPERATIONAL MODEL), -the workings of the LKMM itself are covered. +toward beginners; they explain what memory consistency models are and +the basic notions shared by all such models. People already familiar +with these concepts can skim or skip over them. Sections 6 (EVENTS) +through 12 (THE FROM_READS RELATION) describe the fundamental +relations used in many models. Starting in Section 13 (AN OPERATIONAL +MODEL), the workings of the LKMM itself are covered. Warning: The code examples in this document are not written in the proper format for litmus tests. They don't include a header line, the @@ -827,8 +826,8 @@ A-cumulative; they only affect the propagation of stores that are executed on C before the fence (i.e., those which precede the fence in program order). -smp_read_barrier_depends(), rcu_read_lock(), rcu_read_unlock(), and -synchronize_rcu() fences have other properties which we discuss later. +read_lock(), rcu_read_unlock(), and synchronize_rcu() fences have +other properties which we discuss later. PROPAGATION ORDER RELATION: cumul-fence @@ -988,8 +987,8 @@ Another possibility, not mentioned earlier but discussed in the next section, is: X and Y are both loads, X ->addr Y (i.e., there is an address - dependency from X to Y), and an smp_read_barrier_depends() - fence occurs between them. + dependency from X to Y), and X is a READ_ONCE() or an atomic + access. Dependencies can also cause instructions to be executed in program order. This is uncontroversial when the second instruction is a @@ -1015,9 +1014,9 @@ After all, a CPU cannot ask the memory subsystem to load a value from a particular location before it knows what that location is. However, the split-cache design used by Alpha can cause it to behave in a way that looks as if the loads were executed out of order (see the next -section for more details). For this reason, the LKMM does not include -address dependencies between read events in the ppo relation unless an -smp_read_barrier_depends() fence is present. +section for more details). The kernel includes a workaround for this +problem when the loads come from READ_ONCE(), and therefore the LKMM +includes address dependencies to loads in the ppo relation. On the other hand, dependencies can indirectly affect the ordering of two loads. This happens when there is a dependency from a load to a @@ -1114,11 +1113,12 @@ code such as the following: int *r1; int r2; - r1 = READ_ONCE(ptr); + r1 = ptr; r2 = READ_ONCE(*r1); } -can malfunction on Alpha systems. It is quite possible that r1 = &x +can malfunction on Alpha systems (notice that P1 uses an ordinary load +to read ptr instead of READ_ONCE()). It is quite possible that r1 = &x and r2 = 0 at the end, in spite of the address dependency. At first glance this doesn't seem to make sense. We know that the @@ -1141,11 +1141,15 @@ This could not have happened if the local cache had processed the incoming stores in FIFO order. In constrast, other architectures maintain at least the appearance of FIFO order. -In practice, this difficulty is solved by inserting an -smp_read_barrier_depends() fence between P1's two loads. The effect -of this fence is to cause the CPU not to execute any po-later -instructions until after the local cache has finished processing all -the stores it has already received. Thus, if the code was changed to: +In practice, this difficulty is solved by inserting a special fence +between P1's two loads when the kernel is compiled for the Alpha +architecture. In fact, as of version 4.15, the kernel automatically +adds this fence (called smp_read_barrier_depends() and defined as +nothing at all on non-Alpha builds) after every READ_ONCE() and atomic +load. The effect of the fence is to cause the CPU not to execute any +po-later instructions until after the local cache has finished +processing all the stores it has already received. Thus, if the code +was changed to: P1() { @@ -1153,13 +1157,15 @@ the stores it has already received. Thus, if the code was changed to: int r2; r1 = READ_ONCE(ptr); - smp_read_barrier_depends(); r2 = READ_ONCE(*r1); } then we would never get r1 = &x and r2 = 0. By the time P1 executed its second load, the x = 1 store would already be fully processed by -the local cache and available for satisfying the read request. +the local cache and available for satisfying the read request. Thus +we have yet another reason why shared data should always be read with +READ_ONCE() or another synchronization primitive rather than accessed +directly. The LKMM requires that smp_rmb(), acquire fences, and strong fences share this property with smp_read_barrier_depends(): They do not allow @@ -1751,11 +1757,10 @@ no further involvement from the CPU. Since the CPU doesn't ever read the value of x, there is nothing for the smp_rmb() fence to act on. The LKMM defines a few extra synchronization operations in terms of -things we have already covered. In particular, rcu_dereference() and -lockless_dereference() are both treated as a READ_ONCE() followed by -smp_read_barrier_depends() -- which also happens to be how they are -defined in include/linux/rcupdate.h and include/linux/compiler.h, -respectively. +things we have already covered. In particular, rcu_dereference() is +treated as READ_ONCE() and rcu_assign_pointer() is treated as +smp_store_release() -- which is basically how the Linux kernel treats +them. There are a few oddball fences which need special treatment: smp_mb__before_atomic(), smp_mb__after_atomic(), and diff --git a/tools/memory-model/linux-kernel.bell b/tools/memory-model/linux-kernel.bell index 18885ad15de9..432c7cf71b23 100644 --- a/tools/memory-model/linux-kernel.bell +++ b/tools/memory-model/linux-kernel.bell @@ -24,7 +24,6 @@ instructions RMW[{'once,'acquire,'release}] enum Barriers = 'wmb (*smp_wmb*) || 'rmb (*smp_rmb*) || 'mb (*smp_mb*) || - 'rb_dep (*smp_read_barrier_depends*) || 'rcu-lock (*rcu_read_lock*) || 'rcu-unlock (*rcu_read_unlock*) || 'sync-rcu (*synchronize_rcu*) || diff --git a/tools/memory-model/linux-kernel.cat b/tools/memory-model/linux-kernel.cat index f0d27f813ec2..df97db03b6c2 100644 --- a/tools/memory-model/linux-kernel.cat +++ b/tools/memory-model/linux-kernel.cat @@ -25,7 +25,6 @@ include "lock.cat" (*******************) (* Fences *) -let rb-dep = [R] ; fencerel(Rb_dep) ; [R] let rmb = [R \ Noreturn] ; fencerel(Rmb) ; [R \ Noreturn] let wmb = [W] ; fencerel(Wmb) ; [W] let mb = ([M] ; fencerel(Mb) ; [M]) | @@ -61,11 +60,9 @@ let dep = addr | data let rwdep = (dep | ctrl) ; [W] let overwrite = co | fr let to-w = rwdep | (overwrite & int) -let rrdep = addr | (dep ; rfi) -let strong-rrdep = rrdep+ & rb-dep -let to-r = strong-rrdep | rfi-rel-acq +let to-r = addr | (dep ; rfi) | rfi-rel-acq let fence = strong-fence | wmb | po-rel | rmb | acq-po -let ppo = rrdep* ; (to-r | to-w | fence) +let ppo = to-r | to-w | fence (* Propagation: Ordering from release operations and strong fences. *) let A-cumul(r) = rfe? ; r diff --git a/tools/memory-model/linux-kernel.def b/tools/memory-model/linux-kernel.def index f5a1eb04cb64..5dfb9c7f3462 100644 --- a/tools/memory-model/linux-kernel.def +++ b/tools/memory-model/linux-kernel.def @@ -13,14 +13,12 @@ WRITE_ONCE(X,V) { __store{once}(X,V); } smp_store_release(X,V) { __store{release}(*X,V); } smp_load_acquire(X) __load{acquire}(*X) rcu_assign_pointer(X,V) { __store{release}(X,V); } -lockless_dereference(X) __load{lderef}(X) rcu_dereference(X) __load{deref}(X) // Fences smp_mb() { __fence{mb} ; } smp_rmb() { __fence{rmb} ; } smp_wmb() { __fence{wmb} ; } -smp_read_barrier_depends() { __fence{rb_dep}; } smp_mb__before_atomic() { __fence{before-atomic} ; } smp_mb__after_atomic() { __fence{after-atomic} ; } smp_mb__after_spinlock() { __fence{after-spinlock} ; } -- cgit v1.2.3 From de7112868829b3286def38297848d5d2592b4a70 Mon Sep 17 00:00:00 2001 From: Sangwon Hong Date: Mon, 12 Feb 2018 04:37:44 +0900 Subject: perf kallsyms: Fix the usage on the man page First, all man pages highlight only perf and subcommands except 'perf kallsyms', which includes the full usage. Fix it for commands to monopolize underlines. Second, options can be ommited when executing 'perf kallsyms', so add square brackets between