<feed xmlns='http://www.w3.org/2005/Atom'>
<title>user/sven/linux.git/scripts/Makefile.build, branch v6.6.28</title>
<subtitle>Linux Kernel
</subtitle>
<id>https://git.stealer.net/cgit.cgi/user/sven/linux.git/atom?h=v6.6.28</id>
<link rel='self' href='https://git.stealer.net/cgit.cgi/user/sven/linux.git/atom?h=v6.6.28'/>
<link rel='alternate' type='text/html' href='https://git.stealer.net/cgit.cgi/user/sven/linux.git/'/>
<updated>2023-07-23T18:15:31Z</updated>
<entry>
<title>kbuild: rust: avoid creating temporary files</title>
<updated>2023-07-23T18:15:31Z</updated>
<author>
<name>Miguel Ojeda</name>
<email>ojeda@kernel.org</email>
</author>
<published>2023-07-23T14:21:28Z</published>
<link rel='alternate' type='text/html' href='https://git.stealer.net/cgit.cgi/user/sven/linux.git/commit/?id=df01b7cfcef08bf3fdcac2909d0e1910781d6bfd'/>
<id>urn:sha1:df01b7cfcef08bf3fdcac2909d0e1910781d6bfd</id>
<content type='text'>
`rustc` outputs by default the temporary files (i.e. the ones saved
by `-Csave-temps`, such as `*.rcgu*` files) in the current working
directory when `-o` and `--out-dir` are not given (even if
`--emit=x=path` is given, i.e. it does not use those for temporaries).

Since out-of-tree modules are compiled from the `linux` tree,
`rustc` then tries to create them there, which may not be accessible.

Thus pass `--out-dir` explicitly, even if it is just for the temporary
files.

Similarly, do so for Rust host programs too.

Reported-by: Raphael Nestler &lt;raphael.nestler@gmail.com&gt;
Closes: https://github.com/Rust-for-Linux/linux/issues/1015
Reported-by: Andrea Righi &lt;andrea.righi@canonical.com&gt;
Tested-by: Raphael Nestler &lt;raphael.nestler@gmail.com&gt; # non-hostprogs
Tested-by: Andrea Righi &lt;andrea.righi@canonical.com&gt; # non-hostprogs
Fixes: 295d8398c67e ("kbuild: specify output names separately for each emission type from rustc")
Cc: stable@vger.kernel.org
Signed-off-by: Miguel Ojeda &lt;ojeda@kernel.org&gt;
Tested-by: Martin Rodriguez Reboredo &lt;yakoyoku@gmail.com&gt;
Signed-off-by: Masahiro Yamada &lt;masahiroy@kernel.org&gt;
</content>
</entry>
<entry>
<title>Merge tag 'kbuild-v6.5' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild</title>
<updated>2023-07-01T16:24:31Z</updated>
<author>
<name>Linus Torvalds</name>
<email>torvalds@linux-foundation.org</email>
</author>
<published>2023-07-01T16:24:31Z</published>
<link rel='alternate' type='text/html' href='https://git.stealer.net/cgit.cgi/user/sven/linux.git/commit/?id=ad2885979ea6657fa8d3da51a301ec0e998ad8e7'/>
<id>urn:sha1:ad2885979ea6657fa8d3da51a301ec0e998ad8e7</id>
<content type='text'>
Pull Kbuild updates from Masahiro Yamada:

 - Remove the deprecated rule to build *.dtbo from *.dts

 - Refactor section mismatch detection in modpost

 - Fix bogus ARM section mismatch detections

 - Fix error of 'make gtags' with O= option

 - Add Clang's target triple to KBUILD_CPPFLAGS to fix a build error
   with the latest LLVM version

 - Rebuild the built-in initrd when KBUILD_BUILD_TIMESTAMP is changed

 - Ignore more compiler-generated symbols for kallsyms

 - Fix 'make local*config' to handle the ${CONFIG_FOO} form in Makefiles

 - Enable more kernel-doc warnings with W=2

 - Refactor &lt;linux/export.h&gt; by generating KSYMTAB data by modpost

 - Deprecate &lt;asm/export.h&gt; and &lt;asm-generic/export.h&gt;

 - Remove the EXPORT_DATA_SYMBOL macro

 - Move the check for static EXPORT_SYMBOL back to modpost, which makes
   the build faster

 - Re-implement CONFIG_TRIM_UNUSED_KSYMS with one-pass algorithm

 - Warn missing MODULE_DESCRIPTION when building modules with W=1

 - Make 'make clean' robust against too long argument error

 - Exclude more objects from GCOV to fix CFI failures with GCOV

 - Allow 'make modules_install' to install modules.builtin and
   modules.builtin.modinfo even when CONFIG_MODULES is disabled

 - Include modules.builtin and modules.builtin.modinfo in the
   linux-image Debian package even when CONFIG_MODULES is disabled

 - Revive "Entering directory" logging for the latest Make version

* tag 'kbuild-v6.5' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (72 commits)
  modpost: define more R_ARM_* for old distributions
  kbuild: revive "Entering directory" for Make &gt;= 4.4.1
  kbuild: set correct abs_srctree and abs_objtree for package builds
  scripts/mksysmap: Ignore prefixed KCFI symbols
  kbuild: deb-pkg: remove the CONFIG_MODULES check in buildeb
  kbuild: builddeb: always make modules_install, to install modules.builtin*
  modpost: continue even with unknown relocation type
  modpost: factor out Elf_Sym pointer calculation to section_rel()
  modpost: factor out inst location calculation to section_rel()
  kbuild: Disable GCOV for *.mod.o
  kbuild: Fix CFI failures with GCOV
  kbuild: make clean rule robust against too long argument error
  script: modpost: emit a warning when the description is missing
  kbuild: make modules_install copy modules.builtin(.modinfo)
  linux/export.h: rename 'sec' argument to 'license'
  modpost: show offset from symbol for section mismatch warnings
  modpost: merge two similar section mismatch warnings
  kbuild: implement CONFIG_TRIM_UNUSED_KSYMS without recursion
  modpost: use null string instead of NULL pointer for default namespace
  modpost: squash sym_update_namespace() into sym_add_exported()
  ...
</content>
</entry>
<entry>
<title>kbuild: implement CONFIG_TRIM_UNUSED_KSYMS without recursion</title>
<updated>2023-06-22T12:21:06Z</updated>
<author>
<name>Masahiro Yamada</name>
<email>masahiroy@kernel.org</email>
</author>
<published>2023-06-11T15:50:57Z</published>
<link rel='alternate' type='text/html' href='https://git.stealer.net/cgit.cgi/user/sven/linux.git/commit/?id=5e9e95cc9148b82074a5eae283e63bce3f1aacfe'/>
<id>urn:sha1:5e9e95cc9148b82074a5eae283e63bce3f1aacfe</id>
<content type='text'>
When CONFIG_TRIM_UNUSED_KSYMS is enabled, Kbuild recursively traverses
the directory tree to determine which EXPORT_SYMBOL to trim. If an
EXPORT_SYMBOL turns out to be unused by anyone, Kbuild begins the
second traverse, where some source files are recompiled with their
EXPORT_SYMBOL() tuned into a no-op.

Linus stated negative opinions about this slowness in commits:

 - 5cf0fd591f2e ("Kbuild: disable TRIM_UNUSED_KSYMS option")
 - a555bdd0c58c ("Kbuild: enable TRIM_UNUSED_KSYMS again, with some guarding")

We can do this better now. The final data structures of EXPORT_SYMBOL
are generated by the modpost stage, so modpost can selectively emit
KSYMTAB entries that are really used by modules.

Commit f73edc8951b2 ("kbuild: unify two modpost invocations") is another
ground-work to do this in a one-pass algorithm. With the list of modules,
modpost sets sym-&gt;used if it is used by a module. modpost emits KSYMTAB
only for symbols with sym-&gt;used==true.

BTW, Nicolas explained why the trimming was implemented with recursion:

  https://lore.kernel.org/all/2o2rpn97-79nq-p7s2-nq5-8p83391473r@syhkavp.arg/

Actually, we never achieved that level of optimization where the chain
reaction of trimming comes into play because:

 - CONFIG_LTO_CLANG cannot remove any unused symbols
 - CONFIG_LD_DEAD_CODE_DATA_ELIMINATION is enabled only for vmlinux,
   but not modules

If deeper trimming is required, we need to revisit this, but I guess
that is unlikely to happen.

Signed-off-by: Masahiro Yamada &lt;masahiroy@kernel.org&gt;
</content>
</entry>
<entry>
<title>modpost: check static EXPORT_SYMBOL* by modpost again</title>
<updated>2023-06-22T12:21:06Z</updated>
<author>
<name>Masahiro Yamada</name>
<email>masahiroy@kernel.org</email>
</author>
<published>2023-06-11T15:50:54Z</published>
<link rel='alternate' type='text/html' href='https://git.stealer.net/cgit.cgi/user/sven/linux.git/commit/?id=6d62b1c46b1e6e1686a0cf6617c96c80d4ab5cd5'/>
<id>urn:sha1:6d62b1c46b1e6e1686a0cf6617c96c80d4ab5cd5</id>
<content type='text'>
Commit 31cb50b5590f ("kbuild: check static EXPORT_SYMBOL* by script
instead of modpost") moved the static EXPORT_SYMBOL* check from the
mostpost to a shell script because I thought it must be checked per
compilation unit to avoid false negatives.

I came up with an idea to do this in modpost, against combined ELF
files. The relocation entries in ELF will find the correct exported
symbol even if there exist symbols with the same name in different
compilation units.

Again, the same sample code.

  Makefile:

    obj-y += foo1.o foo2.o

  foo1.c:

    #include &lt;linux/export.h&gt;
    static void foo(void) {}
    EXPORT_SYMBOL(foo);

  foo2.c:

    void foo(void) {}

Then, modpost can catch it correctly.

    MODPOST Module.symvers
  ERROR: modpost: vmlinux: local symbol 'foo' was exported

Signed-off-by: Masahiro Yamada &lt;masahiroy@kernel.org&gt;
Reviewed-by: Nick Desaulniers &lt;ndesaulniers@google.com&gt;
</content>
</entry>
<entry>
<title>kbuild: generate KSYMTAB entries by modpost</title>
<updated>2023-06-22T12:17:10Z</updated>
<author>
<name>Masahiro Yamada</name>
<email>masahiroy@kernel.org</email>
</author>
<published>2023-06-11T15:50:52Z</published>
<link rel='alternate' type='text/html' href='https://git.stealer.net/cgit.cgi/user/sven/linux.git/commit/?id=ddb5cdbafaaad6b99d7007ae1740403124502d03'/>
<id>urn:sha1:ddb5cdbafaaad6b99d7007ae1740403124502d03</id>
<content type='text'>
Commit 7b4537199a4a ("kbuild: link symbol CRCs at final link, removing
CONFIG_MODULE_REL_CRCS") made modpost output CRCs in the same way
whether the EXPORT_SYMBOL() is placed in *.c or *.S.

For further cleanups, this commit applies a similar approach to the
entire data structure of EXPORT_SYMBOL().

The EXPORT_SYMBOL() compilation is split into two stages.

When a source file is compiled, EXPORT_SYMBOL() will be converted into
a dummy symbol in the .export_symbol section.

For example,

    EXPORT_SYMBOL(foo);
    EXPORT_SYMBOL_NS_GPL(bar, BAR_NAMESPACE);

will be encoded into the following assembly code:

    .section ".export_symbol","a"
    __export_symbol_foo:
            .asciz ""                      /* license */
            .asciz ""                      /* name space */
            .balign 8
            .quad foo                      /* symbol reference */
    .previous

    .section ".export_symbol","a"
    __export_symbol_bar:
            .asciz "GPL"                   /* license */
            .asciz "BAR_NAMESPACE"         /* name space */
            .balign 8
            .quad bar                      /* symbol reference */
    .previous

They are mere markers to tell modpost the name, license, and namespace
of the symbols. They will be dropped from the final vmlinux and modules
because the *(.export_symbol) will go into /DISCARD/ in the linker script.

Then, modpost extracts all the information about EXPORT_SYMBOL() from the
.export_symbol section, and generates the final C code:

    KSYMTAB_FUNC(foo, "", "");
    KSYMTAB_FUNC(bar, "_gpl", "BAR_NAMESPACE");

KSYMTAB_FUNC() (or KSYMTAB_DATA() if it is data) is expanded to struct
kernel_symbol that will be linked to the vmlinux or a module.

With this change, EXPORT_SYMBOL() works in the same way for *.c and *.S
files, providing the following benefits.

[1] Deprecate EXPORT_DATA_SYMBOL()

In the old days, EXPORT_SYMBOL() was only available in C files. To export
a symbol in *.S, EXPORT_SYMBOL() was placed in a separate *.c file.
arch/arm/kernel/armksyms.c is one example written in the classic manner.

Commit 22823ab419d8 ("EXPORT_SYMBOL() for asm") removed this limitation.
Since then, EXPORT_SYMBOL() can be placed close to the symbol definition
in *.S files. It was a nice improvement.

However, as that commit mentioned, you need to use EXPORT_DATA_SYMBOL()
for data objects on some architectures.

In the new approach, modpost checks symbol's type (STT_FUNC or not),
and outputs KSYMTAB_FUNC() or KSYMTAB_DATA() accordingly.

There are only two users of EXPORT_DATA_SYMBOL:

  EXPORT_DATA_SYMBOL_GPL(empty_zero_page)    (arch/ia64/kernel/head.S)
  EXPORT_DATA_SYMBOL(ia64_ivt)               (arch/ia64/kernel/ivt.S)

They are transformed as follows and output into .vmlinux.export.c

  KSYMTAB_DATA(empty_zero_page, "_gpl", "");
  KSYMTAB_DATA(ia64_ivt, "", "");

The other EXPORT_SYMBOL users in ia64 assembly are output as
KSYMTAB_FUNC().

EXPORT_DATA_SYMBOL() is now deprecated.

[2] merge &lt;linux/export.h&gt; and &lt;asm-generic/export.h&gt;

There are two similar header implementations:

  include/linux/export.h        for .c files
  include/asm-generic/export.h  for .S files

Ideally, the functionality should be consistent between them, but they
tend to diverge.

Commit 8651ec01daed ("module: add support for symbol namespaces.") did
not support the namespace for *.S files.

This commit shifts the essential implementation part to C, which supports
EXPORT_SYMBOL_NS() for *.S files.

&lt;asm/export.h&gt; and &lt;asm-generic/export.h&gt; will remain as a wrapper of
&lt;linux/export.h&gt; for a while.

They will be removed after #include &lt;asm/export.h&gt; directives are all
replaced with #include &lt;linux/export.h&gt;.

[3] Implement CONFIG_TRIM_UNUSED_KSYMS in one-pass algorithm (by a later commit)

When CONFIG_TRIM_UNUSED_KSYMS is enabled, Kbuild recursively traverses
the directory tree to determine which EXPORT_SYMBOL to trim. If an
EXPORT_SYMBOL turns out to be unused by anyone, Kbuild begins the
second traverse, where some source files are recompiled with their
EXPORT_SYMBOL() tuned into a no-op.

We can do this better now; modpost can selectively emit KSYMTAB entries
that are really used by modules.

Signed-off-by: Masahiro Yamada &lt;masahiroy@kernel.org&gt;
Reviewed-by: Nick Desaulniers &lt;ndesaulniers@google.com&gt;
</content>
</entry>
<entry>
<title>kbuild: enable kernel-doc -Wall for W=2</title>
<updated>2023-06-10T07:39:27Z</updated>
<author>
<name>Johannes Berg</name>
<email>johannes.berg@intel.com</email>
</author>
<published>2023-06-09T08:46:42Z</published>
<link rel='alternate' type='text/html' href='https://git.stealer.net/cgit.cgi/user/sven/linux.git/commit/?id=dd203fefd9c9e28bc141d144e032e263804c90bb'/>
<id>urn:sha1:dd203fefd9c9e28bc141d144e032e263804c90bb</id>
<content type='text'>
For W=2, we can enable more kernel-doc warnings,
such as missing return value descriptions etc.

Signed-off-by: Johannes Berg &lt;johannes.berg@intel.com&gt;
Signed-off-by: Masahiro Yamada &lt;masahiroy@kernel.org&gt;
</content>
</entry>
<entry>
<title>kernel-doc: don't let V=1 change outcome</title>
<updated>2023-06-10T07:39:02Z</updated>
<author>
<name>Johannes Berg</name>
<email>johannes.berg@intel.com</email>
</author>
<published>2023-06-09T08:46:41Z</published>
<link rel='alternate' type='text/html' href='https://git.stealer.net/cgit.cgi/user/sven/linux.git/commit/?id=56b0f453db74207633019f83758b4c11c66b75d0'/>
<id>urn:sha1:56b0f453db74207633019f83758b4c11c66b75d0</id>
<content type='text'>
The kernel-doc script currently reports a number of issues
only in "verbose" mode, but that's initialized from V=1
(via KBUILD_VERBOSE), so if you use KDOC_WERROR=1 then
adding V=1 might actually break the build. This is rather
unexpected.

Change kernel-doc to not change its behaviour wrt. errors
(or warnings) when verbose mode is enabled, but rather add
separate warning flags (and -Wall) for it. Allow enabling
those flags via environment/make variables in the kernel's
build system for easier user use, but to not have to parse
them in the script itself.

Signed-off-by: Johannes Berg &lt;johannes.berg@intel.com&gt;
Acked-by: Jonathan Corbet &lt;corbet@lwn.net&gt;
Signed-off-by: Masahiro Yamada &lt;masahiroy@kernel.org&gt;
</content>
</entry>
<entry>
<title>rust: upgrade to Rust 1.68.2</title>
<updated>2023-05-31T15:35:03Z</updated>
<author>
<name>Miguel Ojeda</name>
<email>ojeda@kernel.org</email>
</author>
<published>2023-04-18T21:43:47Z</published>
<link rel='alternate' type='text/html' href='https://git.stealer.net/cgit.cgi/user/sven/linux.git/commit/?id=3ed03f4da06ede71ac53cf25b9441a372e9f2487'/>
<id>urn:sha1:3ed03f4da06ede71ac53cf25b9441a372e9f2487</id>
<content type='text'>
This is the first upgrade to the Rust toolchain since the initial Rust
merge, from 1.62.0 to 1.68.2 (i.e. the latest).

# Context

The kernel currently supports only a single Rust version [1] (rather
than a minimum) given our usage of some "unstable" Rust features [2]
which do not promise backwards compatibility.

The goal is to reach a point where we can declare a minimum version for
the toolchain. For instance, by waiting for some of the features to be
stabilized. Therefore, the first minimum Rust version that the kernel
will support is "in the future".

# Upgrade policy

Given we will eventually need to reach that minimum version, it would be
ideal to upgrade the compiler from time to time to be as close as
possible to that goal and find any issues sooner. In the extreme, we
could upgrade as soon as a new Rust release is out. Of course, upgrading
so often is in stark contrast to what one normally would need for GCC
and LLVM, especially given the release schedule: 6 weeks for Rust vs.
half a year for LLVM and a year for GCC.

Having said that, there is no particular advantage to updating slowly
either: kernel developers in "stable" distributions are unlikely to be
able to use their distribution-provided Rust toolchain for the kernel
anyway [3]. Instead, by routinely upgrading to the latest instead,
kernel developers using Linux distributions that track the latest Rust
release may be able to use those rather than Rust-provided ones,
especially if their package manager allows to pin / hold back /
downgrade the version for some days during windows where the version may
not match. For instance, Arch, Fedora, Gentoo and openSUSE all provide
and track the latest version of Rust as they get released every 6 weeks.

Then, when the minimum version is reached, we will stop upgrading and
decide how wide the window of support will be. For instance, a year of
Rust versions. We will probably want to start small, and then widen it
over time, just like the kernel did originally for LLVM, see commit
3519c4d6e08e ("Documentation: add minimum clang/llvm version").

# Unstable features stabilized

This upgrade allows us to remove the following unstable features since
they were stabilized:

  - `feature(explicit_generic_args_with_impl_trait)` (1.63).
  - `feature(core_ffi_c)` (1.64).
  - `feature(generic_associated_types)` (1.65).
  - `feature(const_ptr_offset_from)` (1.65, *).
  - `feature(bench_black_box)` (1.66, *).
  - `feature(pin_macro)` (1.68).

The ones marked with `*` apply only to our old `rust` branch, not
mainline yet, i.e. only for code that we may potentially upstream.

With this patch applied, the only unstable feature allowed to be used
outside the `kernel` crate is `new_uninit`, though other code to be
upstreamed may increase the list.

Please see [2] for details.

# Other required changes

Since 1.63, `rustdoc` triggers the `broken_intra_doc_links` lint for
links pointing to exported (`#[macro_export]`) `macro_rules`. An issue
was opened upstream [4], but it turns out it is intended behavior. For
the moment, just add an explicit reference for each link. Later we can
revisit this if `rustdoc` removes the compatibility measure.

Nevertheless, this was helpful to discover a link that was pointing to
the wrong place unintentionally. Since that one was actually wrong, it
is fixed in a previous commit independently.

Another change was the addition of `cfg(no_rc)` and `cfg(no_sync)` in
upstream [5], thus remove our original changes for that.

Similarly, upstream now tests that it compiles successfully with
`#[cfg(not(no_global_oom_handling))]` [6], which allow us to get rid
of some changes, such as an `#[allow(dead_code)]`.

In addition, remove another `#[allow(dead_code)]` due to new uses
within the standard library.

Finally, add `try_extend_trusted` and move the code in `spec_extend.rs`
since upstream moved it for the infallible version.

# `alloc` upgrade and reviewing

There are a large amount of changes, but the vast majority of them are
due to our `alloc` fork being upgraded at once.

There are two kinds of changes to be aware of: the ones coming from
upstream, which we should follow as closely as possible, and the updates
needed in our added fallible APIs to keep them matching the newer
infallible APIs coming from upstream.

Instead of taking a look at the diff of this patch, an alternative
approach is reviewing a diff of the changes between upstream `alloc` and
the kernel's. This allows to easily inspect the kernel additions only,
especially to check if the fallible methods we already have still match
the infallible ones in the new version coming from upstream.

Another approach is reviewing the changes introduced in the additions in
the kernel fork between the two versions. This is useful to spot
potentially unintended changes to our additions.

To apply these approaches, one may follow steps similar to the following
to generate a pair of patches that show the differences between upstream
Rust and the kernel (for the subset of `alloc` we use) before and after
applying this patch:

    # Get the difference with respect to the old version.
    git -C rust checkout $(linux/scripts/min-tool-version.sh rustc)
    git -C linux ls-tree -r --name-only HEAD -- rust/alloc |
        cut -d/ -f3- |
        grep -Fv README.md |
        xargs -IPATH cp rust/library/alloc/src/PATH linux/rust/alloc/PATH
    git -C linux diff --patch-with-stat --summary -R &gt; old.patch
    git -C linux restore rust/alloc

    # Apply this patch.
    git -C linux am rust-upgrade.patch

    # Get the difference with respect to the new version.
    git -C rust checkout $(linux/scripts/min-tool-version.sh rustc)
    git -C linux ls-tree -r --name-only HEAD -- rust/alloc |
        cut -d/ -f3- |
        grep -Fv README.md |
        xargs -IPATH cp rust/library/alloc/src/PATH linux/rust/alloc/PATH
    git -C linux diff --patch-with-stat --summary -R &gt; new.patch
    git -C linux restore rust/alloc

Now one may check the `new.patch` to take a look at the additions (first
approach) or at the difference between those two patches (second
approach). For the latter, a side-by-side tool is recommended.

Link: https://rust-for-linux.com/rust-version-policy [1]
Link: https://github.com/Rust-for-Linux/linux/issues/2 [2]
Link: https://lore.kernel.org/rust-for-linux/CANiq72mT3bVDKdHgaea-6WiZazd8Mvurqmqegbe5JZxVyLR8Yg@mail.gmail.com/ [3]
Link: https://github.com/rust-lang/rust/issues/106142 [4]
Link: https://github.com/rust-lang/rust/pull/89891 [5]
Link: https://github.com/rust-lang/rust/pull/98652 [6]
Reviewed-by: Björn Roy Baron &lt;bjorn3_gh@protonmail.com&gt;
Reviewed-by: Gary Guo &lt;gary@garyguo.net&gt;
Reviewed-By: Martin Rodriguez Reboredo &lt;yakoyoku@gmail.com&gt;
Tested-by: Ariel Miculas &lt;amiculas@cisco.com&gt;
Tested-by: David Gow &lt;davidgow@google.com&gt;
Tested-by: Boqun Feng &lt;boqun.feng@gmail.com&gt;
Link: https://lore.kernel.org/r/20230418214347.324156-4-ojeda@kernel.org
[ Removed `feature(core_ffi_c)` from `uapi` ]
Signed-off-by: Miguel Ojeda &lt;ojeda@kernel.org&gt;
</content>
</entry>
<entry>
<title>rust: add pin-init API core</title>
<updated>2023-04-12T16:41:05Z</updated>
<author>
<name>Benno Lossin</name>
<email>benno.lossin@proton.me</email>
</author>
<published>2023-04-08T12:25:45Z</published>
<link rel='alternate' type='text/html' href='https://git.stealer.net/cgit.cgi/user/sven/linux.git/commit/?id=90e53c5e70a69159ec255fec361f7dcf9cf36eae'/>
<id>urn:sha1:90e53c5e70a69159ec255fec361f7dcf9cf36eae</id>
<content type='text'>
This API is used to facilitate safe pinned initialization of structs. It
replaces cumbersome `unsafe` manual initialization with elegant safe macro
invocations.

Due to the size of this change it has been split into six commits:
1. This commit introducing the basic public interface: traits and
   functions to represent and create initializers.
2. Adds the `#[pin_data]`, `pin_init!`, `try_pin_init!`, `init!` and
   `try_init!` macros along with their internal types.
3. Adds the `InPlaceInit` trait that allows using an initializer to create
   an object inside of a `Box&lt;T&gt;` and other smart pointers.
4. Adds the `PinnedDrop` trait and adds macro support for it in
   the `#[pin_data]` macro.
5. Adds the `stack_pin_init!` macro allowing to pin-initialize a struct on
   the stack.
6. Adds the `Zeroable` trait and `init::zeroed` function to initialize
   types that have `0x00` in all bytes as a valid bit pattern.

--

In this section the problem that the new pin-init API solves is outlined.
This message describes the entirety of the API, not just the parts
introduced in this commit. For a more granular explanation and additional
information on pinning and this issue, view [1].

Pinning is Rust's way of enforcing the address stability of a value. When a
value gets pinned it will be impossible for safe code to move it to another
location. This is done by wrapping pointers to said object with `Pin&lt;P&gt;`.
This wrapper prevents safe code from creating mutable references to the
object, preventing mutable access, which is needed to move the value.
`Pin&lt;P&gt;` provides `unsafe` functions to circumvent this and allow
modifications regardless. It is then the programmer's responsibility to
uphold the pinning guarantee.

Many kernel data structures require a stable address, because there are
foreign pointers to them which would get invalidated by moving the
structure. Since these data structures are usually embedded in structs to
use them, this pinning property propagates to the container struct.
Resulting in most structs in both Rust and C code needing to be pinned.

So if we want to have a `mutex` field in a Rust struct, this struct also
needs to be pinned, because a `mutex` contains a `list_head`. Additionally
initializing a `list_head` requires already having the final memory
location available, because it is initialized by pointing it to itself. But
this presents another challenge in Rust: values have to be initialized at
all times. There is the `MaybeUninit&lt;T&gt;` wrapper type, which allows
handling uninitialized memory, but this requires using the `unsafe` raw
pointers and a casting the type to the initialized variant.

This problem gets exacerbated when considering encapsulation and the normal
safety requirements of Rust code. The fields of the Rust `Mutex&lt;T&gt;` should
not be accessible to normal driver code. After all if anyone can modify
the fields, there is no way to ensure the invariants of the `Mutex&lt;T&gt;` are
upheld. But if the fields are inaccessible, then initialization of a
`Mutex&lt;T&gt;` needs to be somehow achieved via a function or a macro. Because
the `Mutex&lt;T&gt;` must be pinned in memory, the function cannot return it by
value. It also cannot allocate a `Box` to put the `Mutex&lt;T&gt;` into, because
that is an unnecessary allocation and indirection which would hurt
performance.

The solution in the rust tree (e.g. this commit: [2]) that is replaced by
this API is to split this function into two parts:

1. A `new` function that returns a partially initialized `Mutex&lt;T&gt;`,
2. An `init` function that requires the `Mutex&lt;T&gt;` to be pinned and that
   fully initializes the `Mutex&lt;T&gt;`.

Both of these functions have to be marked `unsafe`, since a call to `new`
needs to be accompanied with a call to `init`, otherwise using the
`Mutex&lt;T&gt;` could result in UB. And because calling `init` twice also is not
safe. While `Mutex&lt;T&gt;` initialization cannot fail, other structs might
also have to allocate memory, which would result in conditional successful
initialization requiring even more manual accommodation work.

Combine this with the problem of pin-projections -- the way of accessing
fields of a pinned struct -- which also have an `unsafe` API, pinned
initialization is riddled with `unsafe` resulting in very poor ergonomics.
Not only that, but also having to call two functions possibly multiple
lines apart makes it very easy to forget it outright or during refactoring.

Here is an example of the current way of initializing a struct with two
synchronization primitives (see [3] for the full example):

    struct SharedState {
        state_changed: CondVar,
        inner: Mutex&lt;SharedStateInner&gt;,
    }

    impl SharedState {
        fn try_new() -&gt; Result&lt;Arc&lt;Self&gt;&gt; {
            let mut state = Pin::from(UniqueArc::try_new(Self {
                // SAFETY: `condvar_init!` is called below.
                state_changed: unsafe { CondVar::new() },
                // SAFETY: `mutex_init!` is called below.
                inner: unsafe {
                    Mutex::new(SharedStateInner { token_count: 0 })
                },
            })?);

            // SAFETY: `state_changed` is pinned when `state` is.
            let pinned = unsafe {
                state.as_mut().map_unchecked_mut(|s| &amp;mut s.state_changed)
            };
            kernel::condvar_init!(pinned, "SharedState::state_changed");

            // SAFETY: `inner` is pinned when `state` is.
            let pinned = unsafe {
                state.as_mut().map_unchecked_mut(|s| &amp;mut s.inner)
            };
            kernel::mutex_init!(pinned, "SharedState::inner");

            Ok(state.into())
        }
    }

The pin-init API of this patch solves this issue by providing a
comprehensive solution comprised of macros and traits. Here is the example
from above using the pin-init API:

    #[pin_data]
    struct SharedState {
        #[pin]
        state_changed: CondVar,
        #[pin]
        inner: Mutex&lt;SharedStateInner&gt;,
    }

    impl SharedState {
        fn new() -&gt; impl PinInit&lt;Self&gt; {
            pin_init!(Self {
                state_changed &lt;- new_condvar!("SharedState::state_changed"),
                inner &lt;- new_mutex!(
                    SharedStateInner { token_count: 0 },
                    "SharedState::inner",
                ),
            })
        }
    }

Notably the way the macro is used here requires no `unsafe` and thus comes
with the usual Rust promise of safe code not introducing any memory
violations. Additionally it is now up to the caller of `new()` to decide
the memory location of the `SharedState`. They can choose at the moment
`Arc&lt;T&gt;`, `Box&lt;T&gt;` or the stack.

--

The API has the following architecture:
1. Initializer traits `PinInit&lt;T, E&gt;` and `Init&lt;T, E&gt;` that act like
   closures.
2. Macros to create these initializer traits safely.
3. Functions to allow manually writing initializers.

The initializers (an `impl PinInit&lt;T, E&gt;`) receive a raw pointer pointing
to uninitialized memory and their job is to fully initialize a `T` at that
location. If initialization fails, they return an error (`E`) by value.

This way of initializing cannot be safely exposed to the user, since it
relies upon these properties outside of the control of the trait:
- the memory location (slot) needs to be valid memory,
- if initialization fails, the slot should not be read from,
- the value in the slot should be pinned, so it cannot move and the memory
  cannot be deallocated until the value is dropped.

This is why using an initializer is facilitated by another trait that
ensures these requirements.

These initializers can be created manually by just supplying a closure that
fulfills the same safety requirements as `PinInit&lt;T, E&gt;`. But this is an
`unsafe` operation. To allow safe initializer creation, the `pin_init!` is
provided along with three other variants: `try_pin_init!`, `try_init!` and
`init!`. These take a modified struct initializer as a parameter and
generate a closure that initializes the fields in sequence.
The macros take great care in upholding the safety requirements:
- A shadowed struct type is used as the return type of the closure instead
  of `()`. This is to prevent early returns, as these would prevent full
  initialization.
- To ensure every field is only initialized once, a normal struct
  initializer is placed in unreachable code. The type checker will emit
  errors if a field is missing or specified multiple times.
- When initializing a field fails, the whole initializer will fail and
  automatically drop fields that have been initialized earlier.
- Only the correct initializer type is allowed for unpinned fields. You
  cannot use a `impl PinInit&lt;T, E&gt;` to initialize a structurally not pinned
  field.

To ensure the last point, an additional macro `#[pin_data]` is needed. This
macro annotates the struct itself and the user specifies structurally
pinned and not pinned fields.

Because dropping a pinned struct is also not allowed to break the pinning
invariants, another macro attribute `#[pinned_drop]` is needed. This
macro is introduced in a following commit.

These two macros also have mechanisms to ensure the overall safety of the
API. Additionally, they utilize a combined proc-macro, declarative macro
design: first a proc-macro enables the outer attribute syntax `#[...]` and
does some important pre-parsing. Notably this prepares the generics such
that the declarative macro can handle them using token trees. Then the
actual parsing of the structure and the emission of code is handled by a
declarative macro.

For pin-projections the crates `pin-project` [4] and `pin-project-lite` [5]
had been considered, but were ultimately rejected:
- `pin-project` depends on `syn` [6] which is a very big dependency, around
  50k lines of code.
- `pin-project-lite` is a more reasonable 5k lines of code, but contains a
  very complex declarative macro to parse generics. On top of that it
  would require modification that would need to be maintained
  independently.

Link: https://rust-for-linux.com/the-safe-pinned-initialization-problem [1]
Link: https://github.com/Rust-for-Linux/linux/tree/0a04dc4ddd671efb87eef54dde0fb38e9074f4be [2]
Link: https://github.com/Rust-for-Linux/linux/blob/f509ede33fc10a07eba3da14aa00302bd4b5dddd/samples/rust/rust_miscdev.rs [3]
Link: https://crates.io/crates/pin-project [4]
Link: https://crates.io/crates/pin-project-lite [5]
Link: https://crates.io/crates/syn [6]
Co-developed-by: Gary Guo &lt;gary@garyguo.net&gt;
Signed-off-by: Gary Guo &lt;gary@garyguo.net&gt;
Signed-off-by: Benno Lossin &lt;benno.lossin@proton.me&gt;
Reviewed-by: Alice Ryhl &lt;aliceryhl@google.com&gt;
Reviewed-by: Wedson Almeida Filho &lt;wedsonaf@gmail.com&gt;
Reviewed-by: Andreas Hindborg &lt;a.hindborg@samsung.com&gt;
Link: https://lore.kernel.org/r/20230408122429.1103522-7-y86-dev@protonmail.com
Signed-off-by: Miguel Ojeda &lt;ojeda@kernel.org&gt;
</content>
</entry>
<entry>
<title>rust: enable the `pin_macro` feature</title>
<updated>2023-04-12T16:41:04Z</updated>
<author>
<name>Benno Lossin</name>
<email>benno.lossin@proton.me</email>
</author>
<published>2023-04-08T12:25:18Z</published>
<link rel='alternate' type='text/html' href='https://git.stealer.net/cgit.cgi/user/sven/linux.git/commit/?id=2d19d369c0c6dade11b8e3448c158655dbaa7b77'/>
<id>urn:sha1:2d19d369c0c6dade11b8e3448c158655dbaa7b77</id>
<content type='text'>
This feature enables the use of the `pin!` macro for the `stack_pin_init!`
macro. This feature is already stabilized in Rust version 1.68.

Signed-off-by: Benno Lossin &lt;benno.lossin@proton.me&gt;
Reviewed-by: Alice Ryhl &lt;aliceryhl@google.com&gt;
Reviewed-by: Gary Guo &lt;gary@garyguo.net&gt;
Reviewed-by: Andreas Hindborg &lt;a.hindborg@samsung.com&gt;
Acked-by: Boqun Feng &lt;boqun.feng@gmail.com&gt;
Link: https://lore.kernel.org/r/20230408122429.1103522-2-y86-dev@protonmail.com
Signed-off-by: Miguel Ojeda &lt;ojeda@kernel.org&gt;
</content>
</entry>
</feed>
