user/sven/linux.git/tools/perf/scripts/python/export-to-postgresql.py, branch v5.2.12

perf script python: Add printdate function to SQL exporters

2019-03-11T19:13:02Z

Introduce a printdate function to eliminate the repetitive use of datetime.datetime.today() in the SQL exporting scripts. Signed-off-by: Tony Jones Acked-by: Adrian Hunter Link: http://lkml.kernel.org/r/20190309000518.2438-5-tonyj@suse.de Signed-off-by: Arnaldo Carvalho de Melo

perf script python: Add Python3 support to export-to-postgresql.py

2019-03-11T19:12:57Z

Support both Python2 and Python3 in the export-to-postgresql.py script. The use of 'from __future__' implies the minimum supported Python2 version is now v2.6 Signed-off-by: Tony Jones Link: http://lkml.kernel.org/r/20190309000518.2438-3-tonyj@suse.de Signed-off-by: Adrian Hunter Signed-off-by: Seeteena Thoufeek Signed-off-by: Arnaldo Carvalho de Melo

perf scripts python: export-to-postgresql.py: Export calls parent_id

2019-03-01T17:53:33Z

Export to the 'calls' table the newly created 'parent_id' and create an index for it. Signed-off-by: Adrian Hunter Cc: Jiri Olsa Link: https://lkml.kernel.org/n/tip-eybd6fnk6j9r7g643lsideoo@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo

perf scripts python: export-to-postgresql.py: Fix invalid input syntax for integer error

2019-03-01T17:53:12Z

Fix SQL query error "invalid input syntax for integer": Traceback (most recent call last): File "tools/perf/scripts/python/export-to-postgresql.py", line 465, in do_query(query, 'CREATE VIEW calls_view AS ' File "tools/perf/scripts/python/export-to-postgresql.py", line 274, in do_query raise Exception("Query failed: " + q.lastError().text()) Exception: Query failed: ERROR: invalid input syntax for integer: "" LINE 1: ...ch_count,call_id,return_id,CASE WHEN flags=0 THEN '' WHEN fl... ^ (22P02) QPSQL: Unable to create query Error running python script tools/perf/scripts/python/export-to-postgresql.py Signed-off-by: Adrian Hunter Cc: Jiri Olsa Fixes: f08046cb3082 ("perf thread-stack: Represent jmps to the start of a different symbol") Link: https://lkml.kernel.org/n/tip-strfpdozrvg7bi1xzrivxzqt@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo

perf thread-stack: Represent jmps to the start of a different symbol

2019-02-06T13:00:40Z

The compiler might optimize a call/ret combination by making it a jmp. However the thread-stack does not presently cater for that, so that such control flow is not visible in the call graph. Make it visible by recording on the stack a branch to the start of a different symbol. Note, that means when a ret pops the stack, all jmps must be popped off first. Example: $ cat jmp-to-fn.c __attribute__((noinline)) int bar(void) { return -1; } __attribute__((noinline)) int foo(void) { return bar() + 1; } int main() { return foo(); } $ gcc -ggdb3 -Wall -Wextra -O2 -o jmp-to-fn jmp-to-fn.c $ objdump -d jmp-to-fn 0000000000001040

: 1040: 31 c0 xor %eax,%eax 1042: e9 09 01 00 00 jmpq 1150 0000000000001140 : 1140: b8 ff ff ff ff mov $0xffffffff,%eax 1145: c3 retq 0000000000001150 : 1150: 31 c0 xor %eax,%eax 1152: e8 e9 ff ff ff callq 1140 1157: 83 c0 01 add $0x1,%eax 115a: c3 retq $ perf record -o jmp-to-fn.perf.data -e intel_pt/cyc/u ./jmp-to-fn [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0,017 MB jmp-to-fn.perf.data ] $ perf script -i jmp-to-fn.perf.data --itrace=be -s ~/libexec/perf-core/scripts/python/export-to-sqlite.py jmp-to-fn.db branches calls 2019-01-08 13:24:58.783069 Creating database... 2019-01-08 13:24:58.794650 Writing records... 2019-01-08 13:24:59.008050 Adding indexes 2019-01-08 13:24:59.015802 Done $ ~/libexec/perf-core/scripts/python/exported-sql-viewer.py jmp-to-fn.db Before: main -> bar After: main -> foo -> bar Committer testing: Install the python2-pyside package, then select these menu options on the GUI: "Reports" "Context sensitive callgraphs" Then go on expanding the symbols, to get, full picture when doing this on a fedora:29 with gcc version 8.2.1 20181215 (Red Hat 8.2.1-6) (GCC): jmp-to-fn PID:TID _start (ld-2.28.so) __libc_start_main main foo bar To verify that indeed, this fixes the problem. Signed-off-by: Adrian Hunter Tested-by: Arnaldo Carvalho de Melo Acked-by: Jiri Olsa Link: http://lkml.kernel.org/r/20190109091835.5570-5-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo

perf scripts python: call-graph-from-sql.py: Rename to exported-sql-viewer.py

2018-10-23T17:26:44Z

Additional reports will be added to the script so rename to reflect the more general purpose. Signed-off-by: Adrian Hunter Cc: Andi Kleen Cc: Jiri Olsa Link: http://lkml.kernel.org/r/20181001062853.28285-13-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo

perf script python: Fix export-to-postgresql.py occasional failure

2018-09-25T14:33:06Z

Occasional export failures were found to be caused by truncating 64-bit pointers to 32-bits. Fix by explicitly setting types for all ctype arguments and results. Signed-off-by: Adrian Hunter Cc: Jiri Olsa Cc: stable@vger.kernel.org Link: http://lkml.kernel.org/r/20180911114504.28516-2-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo

perf script python: Rename call-graph-from-postgresql.py to call-graph-from-sql.py

2017-08-15T19:38:06Z

Rename call-graph-from-postgresql.py to call-graph-from-sql.py in preparation for adding support to it for SQLite 3. Signed-off-by: Adrian Hunter Link: http://lkml.kernel.org/r/1501749090-20357-5-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo

perf scripts python: Fix missing call_path_id in export-to-postgresql script

2017-08-15T19:05:36Z

The export does not work if only branches are exported because of a missing column in the samples table. Fix by adding the missing call_path_id. Fixes: 3521f3bc9dae ("perf script: Update export-to-postgresql to support callchain export") Signed-off-by: Adrian Hunter Link: http://lkml.kernel.org/r/1501749090-20357-2-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo

perf script: Update export-to-postgresql to support callchain export

2016-05-06T16:00:55Z

Update the export-to-postgresql.py to support the newly introduced callchain export. callchains are added into the existing call_paths table and can now be associated with samples when the "callpaths" commandline option is used with the script. Ex.: $ perf script -s export-to-postgresql.py example_db all callchains Includes the following changes to enable callchain export via the python export APIs: - Add the "callchains" commandline option, which is used to enable callchain export by setting the perf_db_export_callchains global - Add perf_db_export_callchains checks for call_path table creation and population. - Add call_path_id to samples_table to conform with the new API example usage and output using a small test app: test_app.c: volatile int x = 0; void inc_x_loop() { int i; for(i=0; i<100000000; i++) x++; } void a() { inc_x_loop(); } void b() { inc_x_loop(); } int main() { a(); b(); return 0; } example usage: $ gcc -g -O0 test_app.c $ perf record --call-graph=dwarf ./a.out [ perf record: Woken up 77 times to write data ] [ perf record: Captured and wrote 19.373 MB perf.data (2404 samples) ] $ perf script -s scripts/python/export-to-postgresql.py example_db all callchains $ psql example_db example_db=# SELECT (SELECT name FROM symbols WHERE id = cps.symbol_id) as symbol, (SELECT name FROM symbols WHERE id = (SELECT symbol_id from call_paths where id = cps.parent_id)) as parent_symbol, sum(period) as event_count FROM samples join call_paths as cps on call_path_id = cps.id GROUP BY cps.id,evsel_id ORDER BY event_count DESC LIMIT 5; symbol | parent_symbol | event_count ------------------+--------------------------+------------- inc_x_loop | a | 734250982 inc_x_loop | b | 731028057 unknown | unknown | 1335858 task_tick_fair | scheduler_tick | 1238842 update_wall_time | tick_do_update_jiffies64 | 650373 (5 rows) The above data shows total "self time" in cycles for each call path that was sampled. It is intended to demonstrate how it accounts separately for the two ways to reach the "inc_x_loop" function(via "a" and "b"). Recursive common table expressions can be used as well to get cumulative time spent in a function as well, but that is beyond the scope of this basic example. Signed-off-by: Chris Phlipot Acked-by: Adrian Hunter Cc: Jiri Olsa Cc: Peter Zijlstra Link: http://lkml.kernel.org/r/1461831551-12213-7-git-send-email-cphlipot0@gmail.com Signed-off-by: Arnaldo Carvalho de Melo