summaryrefslogtreecommitdiff
path: root/tools/lib/python
AgeCommit message (Collapse)Author
11 daysdocs: kdoc: Fix pdfdocs build for toolsMauro Carvalho Chehab
the "\1" inside a docstring requires proper scaping to not be considered a hex character and break the build. Reported-by: Akira Yokosawa <akiyks@gmail.com> Closes: https://lore.kernel.org/linux-doc/63e99049-cc72-4156-83af-414fdde34312@gmail.com/ Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <2fff8ef1d0d64e8b68f15f5c07613f302d773855.1769500383.git.mchehab+huawei@kernel.org>
2026-01-23Merge branch 'mauro' into docs-mwJonathan Corbet
Mauro's work to include documentation from our Python modules. His cover letter follows: This is an extended version of: https://lore.kernel.org/linux-doc/cover.1768488832.git.mchehab+huawei@kernel.org/ It basically adds everything we currently have inside libs/tool/python to "tools" book inside documentation. This version should be independent of the other series yet to be merged, (including the jobserver one). The vast amount of changes here are docstring cleanups and additions. They mainly consists on: - ensuring that every phrase will end with a period, making it uniform along all files; - cleaning ups to better uniform docstrings; - variable descriptions now use "#:" markup, as it allows autodoc to add them inside the documentation; - added some missing docstrings; - some new blank lines at comments to make ReST syntax parser happy; - add a couple of sphinx markups (mainly, code blocks). Most of those are minor changes, affecting only comments. It also has one patch per libarary type, adding them to docs. For kernel-doc, I did the cleanups first, as there is one code block inside tools/lib/python/kdoc/latex_fonts.py that would cause a Sphinx crash without such markups. The series actually starts with 3 fixes: - avoid "*" markups on indexes with deep> 3 to override text - a variable rename to stop abusing doctree name - don't rely on cwd to get Documentation/ location patch 4 adds support to document scripts either at: - tools/ - scripts/ patch 5 contains a CSS to better display autodoc html output. For those who want to play with documentation, documenting a python file is very simple. All it takes is to use: .. automodule:: lib.python.<dir+name> Usually, we add a couple of control members to it to adjust the desired documentation scope (add/remove members, showing class inheritance, showing members that currently don't have docstrings, etc). That's why we're using: .. automodule:: lib.python.kdoc.enrich_formatter :members: :show-inheritance: :undoc-members: (and similar) inside tools/kdoc*.rst. autodoc allows filtering in/out members, file docstrings, etc. It also allows documenting just some members or functions with directives like: ..autofunction: ..automember: Sphinx also has a helper script to generate .rst files with documentation: $ sphinx-apidoc -o foobar tools/lib/python/ which can be helpful to discover what should be documented, although changes are needed to use what it produces.
2026-01-23docs: parse_features: make documentation more consistentMauro Carvalho Chehab
Do some changes to: - add missing documentation strings to vars; - add a missing docstring; - ensure that phases will end with a period. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <3722f10361638561a5ced18cf4f409930c88270b.1768838938.git.mchehab+huawei@kernel.org>
2026-01-23docs: jobserver: do some documentation improvementsMauro Carvalho Chehab
Make Sphinx handle better jobserver class documentation Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <18a9c1406bdead680e3ee5768c97ae8b2138e8ea.1768838938.git.mchehab+huawei@kernel.org>
2026-01-23docs: kabi: helpers: add documentation for each "enum" valueMauro Carvalho Chehab
Ensure that kABI module documentation will describe each debug bit. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <3b118b157e52d757bf82fd74f03b0f4bd9e8b8f1.1768838938.git.mchehab+huawei@kernel.org>
2026-01-23docs: kabi: helpers: add helper for debug bits 7 and 8Mauro Carvalho Chehab
The kabi logic supports 8 debug bits, but only 6 are currently documented. Document the remaining ones. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <60e99b9060396eac8621954d6b8a73af45df90fb.1768838938.git.mchehab+huawei@kernel.org>
2026-01-23docs: kabi: system_symbols: end docstring phrases with a dotMauro Carvalho Chehab
Some docstring classes are not ending with a dot. Fix to make it more uniform. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <efd0e150d8e12d8ea2665f54a96b1997f32897b7.1768838938.git.mchehab+huawei@kernel.org>
2026-01-23docs: python: abi_regex: do some improvements at documentationMauro Carvalho Chehab
Add documentation for two consts and ensure that all sentenses will end with a dot. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <5419ad89a5042c1571198c2f055866674808579b.1768838938.git.mchehab+huawei@kernel.org>
2026-01-23docs: python: abi_parser: do some improvements at documentationMauro Carvalho Chehab
Add documentation for two consts and ensure that all sentenses will end with a dot. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <c5756d7fd70697890130b41b2856c59144d01844.1768838938.git.mchehab+huawei@kernel.org>
2026-01-23docs: kdoc: python_version: Improve docstrings and commentsMauro Carvalho Chehab
In preparation to document kernel-doc module, improve its documentation. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <2153afaeb496e1bb8d3cc318fff26c3f99d99486.1768838938.git.mchehab+huawei@kernel.org>
2026-01-23docs: kdoc: enrich_formatter: Improve docstrings and commentsMauro Carvalho Chehab
In preparation to document kernel-doc module, improve its documentation. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <55ec8b896fe00529d326859cd094230fb5a2cd30.1768838938.git.mchehab+huawei@kernel.org>
2026-01-23docs: kdoc: parse_data_structs: Improve docstrings and commentsMauro Carvalho Chehab
In preparation to document kernel-doc module, improve its documentation. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <76ead85b4c13a8038180a792e270c3691d26cd25.1768838938.git.mchehab+huawei@kernel.org>
2026-01-23docs: kdoc_re: Improve docstrings and commentsMauro Carvalho Chehab
In preparation to document kernel-doc module, improve its documentation. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <14a12a43144d52345bfd405d0401d246f0885acf.1768838938.git.mchehab+huawei@kernel.org>
2026-01-23docs: kdoc_output: Improve docstrings and commentsMauro Carvalho Chehab
In preparation to document kernel-doc module, improve its documentation. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <ac03bf776f0929bbe822cd8269f2a31e275b8d6b.1768838938.git.mchehab+huawei@kernel.org>
2026-01-23docs: kdoc_parser: Improve docstrings and commentsMauro Carvalho Chehab
In preparation to document kernel-doc module, improve its documentation. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <b6aabe25b45e9751885becd544a4db82dbe11ff2.1768838938.git.mchehab+huawei@kernel.org>
2026-01-23docs: kdoc_item: Improve docstrings and commentsMauro Carvalho Chehab
In preparation to document kernel-doc module, improve its documentation. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <65a7c6bb318e7a8cbf5c115903d507568099151a.1768838938.git.mchehab+huawei@kernel.org>
2026-01-23docs: kdoc_files: Improve docstrings and commentsMauro Carvalho Chehab
In preparation to document kernel-doc module, improve its documentation. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <75d58878ad6f83f24f1c0ce9e04301a000ecbaa3.1768838938.git.mchehab+huawei@kernel.org>
2026-01-23docs: kdoc: latex_fonts: Improve docstrings and commentsMauro Carvalho Chehab
In preparation to document kernel-doc module, improve its documentation. Among the changes, it had to place the xml template inside a code block, as otherwise doc build would break. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <6e0eb2e245eae9b4f39cf231dee32df00b9e8b7b.1768838938.git.mchehab+huawei@kernel.org>
2026-01-20Merge branch 'mauro' into docs-mwJonathan Corbet
A combination of Mauro's -Werror work and my long-belated kernel-doc move.
2026-01-20jobserver: Split up the big try: blockJonathan Corbet
The parsing of jobserver options is done in a massive try: block that hides problems and (perhaps) bugs. Split up that block and make the logic explicit by moving the initial parsing of MAKEFLAGS out of that block. Add warnings in the places things can go wrong. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2026-01-20docs: kdoc: fix logic to handle unissued warningsMauro Carvalho Chehab
Changeset 469c1c9eb6c9 ("kernel-doc: Issue warnings that were silently discarded") didn't properly addressed the missing messages behavior, as it was calling directly python logger low-level function, instead of using the expected method to emit warnings. Basically, there are two methods to log messages: - self.config.log.warning() - This is the raw level to emit a warning. It just writes the a message at stderr, via python logging, as it is initialized as: self.config.log = logging.getLogger("kernel-doc") - self.config.warning() - This is where we actually consider a message as a warning, properly incrementing error count. Due to that, several parsing error messages are internally considered as success, causing -Werror to not work on such messages. While here, ensure that the last ignored entry will also be handled by adding an extra check at the end of the parse handler. Fixes: 469c1c9eb6c9 ("kernel-doc: Issue warnings that were silently discarded") Closes: https://lore.kernel.org/linux-doc/20260112091053.00cee29a@foz.lan/ Cc: stable@vger.kernel.org Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <95109a6585171da4d6900049deaa2634b41ee743.1768823489.git.mchehab+huawei@kernel.org>
2026-01-12tools: jobserver: Prevent deadlock caused by incorrect jobserver ↵Changbin Du
configuration and enhance error reporting When using GNU Make's jobserver feature in kernel builds, a bug in MAKEFLAGS propagation caused "--jobserver-auth=r,w" to reference an unintended file descriptor. This led to infinite loops in jobserver-exec's os.read() calls due to empty token. My shell opened /etc/passwd for some reason without closing it, and as a result, all child processes inherited this fd 3. $ ls -l /proc/self/fd total 0 lrwx------ 1 changbin changbin 64 Dec 25 13:03 0 -> /dev/pts/1 lrwx------ 1 changbin changbin 64 Dec 25 13:03 1 -> /dev/pts/1 lrwx------ 1 changbin changbin 64 Dec 25 13:03 2 -> /dev/pts/1 lr-x------ 1 changbin changbin 64 Dec 25 13:03 3 -> /etc/passwd lr-x------ 1 changbin changbin 64 Dec 25 13:03 4 -> /proc/1421383/fd In this case, the `make` should open a new file descriptor for jobserver control, but clearly, it did not do so and instead still passed fd 3 as "--jobserver-auth=3,4" in MAKEFLAGS. (The version of my gnu make is 4.3) This update ensures robustness against invalid jobserver configurations, even when `make` incorrectly pass non-pipe file descriptors. * Rejecting empty reads to prevent infinite loops on EOF. * Clearing `self.jobs` to avoid writing to incorrect files if invalid tokens are detected. * Printing detailed error messages to stderr to inform the user. Cc: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Changbin Du <changbin.du@huawei.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <20260108113836.2976527-1-changbin.du@huawei.com>
2025-12-22kdoc: allow dots in inline @param namesSteven Price
Inline kernel-doc blocks failed to parse tags containing dots (e.g. creator.process_name in panfrost_gem.h) because the @name regex only matched word characters. Modify the single-line pattern to match doc_inline_sect so it includes \. and parses the same as a multi-line comment. Signed-off-by: Steven Price <steven.price@arm.com> Acked-by: Randy Dunlap <rdunlap@infradead.org> Tested-by: Randy Dunlap <rdunlap@infradead.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <20251211104851.45330-1-steven.price@arm.com>
2025-12-22docs: kernel-doc.rst: Parse DEFINE_ macros without prefixesMauro Carvalho Chehab
Currently, the logic for vars require a type DEFINE_foo(); where type is usually "static". Make the logic more generic. Reported-by: Randy Dunlap <rdunlap@infradead.org> Closes: https://lore.kernel.org/linux-doc/e1dad7e4-a0ca-4be6-a33c-97b75175c12f@infradead.org/ Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <be16e087cbc065fbd041fb6d6f8fa5cf0426cca5.1765894964.git.mchehab+huawei@kernel.org>
2025-12-22kernel-doc: add support to handle DEFINE_ variablesMauro Carvalho Chehab
Improve the parser and output plugin to work with macros, adding support for the common pattern of using DEFINE_* to create variables. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <757a45100cfc493984574ff780aa9d90506eecb4.1765894964.git.mchehab+huawei@kernel.org>
2025-12-22kernel-doc: add support for handling global variablesMauro Carvalho Chehab
Specially on kAPI, sometimes it is desirable to be able to describe global variables that are part of kAPI. Documenting vars with Sphinx is simple, as we don't need to parse a data struct. All we need is the variable declaration and use native C domain ::c:var: to format it for us. Add support for it. Link: https://lore.kernel.org/linux-doc/491c3022-cef8-4860-a945-c9c4a3b63c09@infradead.org/T/#m947c25d95cb1d96a394410ab1131dc8e9e5013f1 Suggested-by: Randy Dunlap <rdunlap@infradead.org> Tested-by: Randy Dunlap <rdunlap@infradead.org> Acked-by: Randy Dunlap <rdunlap@infradead.org> Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <fa7d1c61a8de9150f71b318382f1507d3b13848d.1765894964.git.mchehab+huawei@kernel.org>
2025-11-29docs: kdoc: various fixes for grammar, spelling, punctuationRandy Dunlap
Correct grammar, spelling, and punctuation in comments, strings, print messages, logs. Change two instances of two spaces between words to just one space. codespell was used to find misspelled words. Signed-off-by: Randy Dunlap <rdunlap@infradead.org> Cc: Jonathan Corbet <corbet@lwn.net> Cc: linux-doc@vger.kernel.org Cc: Mauro Carvalho Chehab <mchehab@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <20251124041011.3030571-1-rdunlap@infradead.org>
2025-11-29docs: kdoc_parser: use '@' for Excess enum valueRandy Dunlap
kdoc is looking for "@value" here, so use that kind of string in the warning message. The "%value" can be confusing. This changes: Warning: drivers/net/wireless/mediatek/mt76/testmode.h:92 Excess enum value '%MT76_TM_ATTR_TX_PENDING' description in 'mt76_testmode_attr' to this: Warning: drivers/net/wireless/mediatek/mt76/testmode.h:92 Excess enum value '@MT76_TM_ATTR_TX_PENDING' description in 'mt76_testmode_attr' Signed-off-by: Randy Dunlap <rdunlap@infradead.org> Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <20251126061752.3497106-1-rdunlap@infradead.org>
2025-11-29docs: kdoc_parser: add data/function attributes to ignoreRandy Dunlap
Recognize and ignore __rcu (in struct members), __private (in struct members), and __always_unused (in function parameters) to prevent kernel-doc warnings: Warning: include/linux/rethook.h:38 struct member 'void (__rcu *handler' not described in 'rethook' Warning: include/linux/hrtimer_types.h:47 Invalid param: enum hrtimer_restart (*__private function)(struct hrtimer *) Warning: security/ipe/hooks.c:81 function parameter '__always_unused' not described in 'ipe_mmap_file' Warning: security/ipe/hooks.c:109 function parameter '__always_unused' not described in 'ipe_file_mprotect' There are more of these (in compiler_types.h, compiler_attributes.h) that can be added as needed. Signed-off-by: Randy Dunlap <rdunlap@infradead.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <20251127063117.150384-1-rdunlap@infradead.org>
2025-11-21tools/docs/get_feat.py: convert get_feat.pl to PythonMauro Carvalho Chehab
As we want to call Python code directly at the Sphinx extension, convert get_feat.pl to Python. The code was made to be (almost) bug-compatible with the Perl version, with two exceptions: 1. Currently, Perl script outputs a wrong table if arch is set to a non-existing value; 2. the ReST table output when --feat is used without --arch has an invalid format, as the number of characters for the table delimiters are wrong. Those two bugs were fixed while testing the conversion. Additionally, another caveat was solved: the output when --feat is used without arch and the feature doesn't exist doesn't contain an empty table anymore. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <03c26cee1ec567804735a33047e625ef5ab7bfa8.1763492868.git.mchehab+huawei@kernel.org>
2025-11-18docs: bring some order to our Python module hierarchyJonathan Corbet
Now that we have tools/lib/python for our Python modules, turn them into proper packages with a single namespace so that everything can just use tools/lib/python in sys.path. No functional change. Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <20251110220430.726665-3-corbet@lwn.net>
2025-11-18docs: Move the python libraries to tools/lib/pythonJonathan Corbet
"scripts/lib" was always a bit of an awkward place for Python modules. We already have tools/lib; create a tools/lib/python, move the libraries there, and update the users accordingly. While at it, move the contents of tools/docs/lib. Rather than make another directory, just put these documentation-oriented modules under "kdoc". Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <20251110220430.726665-2-corbet@lwn.net>