user/sven/linux.git/include/linux/export.h, branch v5.6.10

export.h: reduce __ksymtab_strings string duplication by using "MS" section flags

2019-12-16T09:35:33Z

Commit c3a6cf19e695 ("export: avoid code duplication in include/linux/export.h") refactors export.h quite nicely, but introduces a slight increase in memory usage due to using the empty string "" instead of NULL to indicate that an exported symbol has no namespace. As mentioned in that commit, this meant an increase of 1 byte per exported symbol without a namespace. For example, if a kernel configuration has about 10k exported symbols, this would mean that the size of __ksymtab_strings would increase by roughly 10kB. We can alleviate this situation by utilizing the SHF_MERGE and SHF_STRING section flags. SHF_MERGE|SHF_STRING indicate to the linker that the data in the section are null-terminated strings that can be merged to eliminate duplication. More specifically, from the binutils documentation - "for sections with both M and S, a string which is a suffix of a larger string is considered a duplicate. Thus "def" will be merged with "abcdef"; A reference to the first "def" will be changed to a reference to "abcdef"+3". Thus, all the empty strings would be merged as well as any strings that can be merged according to the cited method above. For example, "memset" and "__memset" would be merged to just "__memset" in __ksymtab_strings. As of v5.4-rc5, the following statistics were gathered with x86 defconfig with approximately 10.7k exported symbols. Size of __ksymtab_strings in vmlinux: ------------------------------------- v5.4-rc5: 213834 bytes v5.4-rc5 with commit c3a6cf19e695: 224455 bytes v5.4-rc5 with this patch: 205759 bytes So, we already see memory savings of ~8kB compared to vanilla -rc5 and savings of nearly 18.7kB compared to -rc5 with commit c3a6cf19e695 on top. Unfortunately, as of this writing, strings will not get deduplicated for kernel modules, as ld does not do the deduplication for SHF_MERGE|SHF_STRINGS sections for relocatable files (ld -r), which kernel modules are. A patch for ld is currently being worked on to hopefully allow for string deduplication in relocatable files in the future. Suggested-by: Rasmus Villemoes Reviewed-by: Masahiro Yamada Reviewed-by: Matthias Maennich Signed-off-by: Jessica Yu

Merge tag 'modules-for-v5.5' of git://git.kernel.org/pub/scm/linux/kernel/git/jeyu/linux

2019-12-05T20:27:16Z

Pull modules updates from Jessica Yu: "Summary of modules changes for the 5.5 merge window: - Refactor include/linux/export.h and remove code duplication between EXPORT_SYMBOL and EXPORT_SYMBOL_NS to make it more readable. The most notable change is that no namespace is represented by an empty string "" rather than NULL. - Fix a module load/unload race where waiter(s) trying to load the same module weren't being woken up when a module finally goes away" * tag 'modules-for-v5.5' of git://git.kernel.org/pub/scm/linux/kernel/git/jeyu/linux: kernel/module.c: wakeup processes in module_wq on module unload moduleparam: fix parameter description mismatch export: avoid code duplication in include/linux/export.h

export,module: add SPDX GPL-2.0 license identifier to headers with no license

2019-11-14T03:36:53Z

Commit b24413180f56 ("License cleanup: add SPDX GPL-2.0 license identifier to files with no license") took care of a lot of files without any license information. These headers were not processed by the tool perhaps because they contain "GPL" in the code. I do not see any license boilerplate in them, so they fall back to GPL version 2 only, which is the project default. Signed-off-by: Masahiro Yamada Link: https://lore.kernel.org/r/20191018045053.8424-1-yamada.masahiro@socionext.com Signed-off-by: Greg Kroah-Hartman

export: avoid code duplication in include/linux/export.h

2019-10-28T15:38:26Z

include/linux/export.h has lots of code duplication between EXPORT_SYMBOL and EXPORT_SYMBOL_NS. To improve the maintainability and readability, unify the implementation. When the symbol has no namespace, pass the empty string "" to the 'ns' parameter. The drawback of this change is, it grows the code size. When the symbol has no namespace, sym->namespace was previously NULL, but it is now an empty string "". So, it increases 1 byte for every no namespace EXPORT_SYMBOL. A typical kernel configuration has 10K exported symbols, so it increases 10KB in rough estimation. I did not come up with a good idea to refactor it without increasing the code size. I am not sure how big a deal it is, but at least include/linux/export.h looks nicer. Reviewed-by: Greg Kroah-Hartman Signed-off-by: Masahiro Yamada [maennich: rebase on top of 3 fixes for the namespace feature] Signed-off-by: Matthias Maennich Signed-off-by: Jessica Yu

symbol namespaces: revert to previous __ksymtab name scheme

2019-10-18T13:32:52Z

The introduction of Symbol Namespaces changed the naming schema of the __ksymtab entries from __kysmtab__symbol to __ksymtab_NAMESPACE.symbol. That caused some breakages in tools that depend on the name layout in either the binaries(vmlinux,*.ko) or in System.map. E.g. kmod's depmod would not be able to read System.map without a patch to support symbol namespaces. A warning reported by depmod for namespaced symbols would look like depmod: WARNING: [...]/uas.ko needs unknown symbol usb_stor_adjust_quirks In order to address this issue, revert to the original naming scheme and rather read the __kstrtabns_ entries and their corresponding values from __ksymtab_strings to update the namespace values for symbols. After having read all symbols and handled them in handle_modversions(), the symbols are created. In a second pass, read the __kstrtabns_ entries and update the namespaces accordingly. Fixes: 8651ec01daed ("module: add support for symbol namespaces.") Reported-by: Stefan Wahren Suggested-by: Masahiro Yamada Acked-by: Will Deacon Reviewed-by: Greg Kroah-Hartman Reviewed-by: Masahiro Yamada Signed-off-by: Matthias Maennich Signed-off-by: Jessica Yu

module: rename __kstrtab_ns_* to __kstrtabns_* to avoid symbol conflict

2019-10-07T16:25:06Z

The module namespace produces __strtab_ns_ symbols to store namespace strings, but it does not guarantee the name uniqueness. This is a potential problem because we have exported symbols starting with "ns_". For example, kernel/capability.c exports the following symbols: EXPORT_SYMBOL(ns_capable); EXPORT_SYMBOL(capable); Assume a situation where those are converted as follows: EXPORT_SYMBOL_NS(ns_capable, some_namespace); EXPORT_SYMBOL_NS(capable, some_namespace); The former expands to "__kstrtab_ns_capable" and "__kstrtab_ns_ns_capable", and the latter to "__kstrtab_capable" and "__kstrtab_ns_capable". Then, we have the duplicated "__kstrtab_ns_capable". To ensure the uniqueness, rename "__kstrtab_ns_*" to "__kstrtabns_*". Reviewed-by: Matthias Maennich Signed-off-by: Masahiro Yamada Signed-off-by: Jessica Yu

module: swap the order of symbol.namespace

2019-10-07T16:24:48Z

Currently, EXPORT_SYMBOL_NS(_GPL) constructs the kernel symbol as follows: __ksymtab_SYMBOL.NAMESPACE The sym_extract_namespace() in modpost allocates memory for the part SYMBOL.NAMESPACE when '.' is contained. One problem is that the pointer returned by strdup() is lost because the symbol name will be copied to malloc'ed memory by alloc_symbol(). No one will keep track of the pointer of strdup'ed memory. sym->namespace still points to the NAMESPACE part. So, you can free it with complicated code like this: free(sym->namespace - strlen(sym->name) - 1); It complicates memory free. To fix it elegantly, I swapped the order of the symbol and the namespace as follows: __ksymtab_NAMESPACE.SYMBOL then, simplified sym_extract_namespace() so that it allocates memory only for the NAMESPACE part. I prefer this order because it is intuitive and also matches to major languages. For example, NAMESPACE::NAME in C++, MODULE.NAME in Python. Reviewed-by: Matthias Maennich Signed-off-by: Masahiro Yamada Signed-off-by: Jessica Yu

Merge tag 'modules-for-v5.4' of git://git.kernel.org/pub/scm/linux/kernel/git/jeyu/linux

2019-09-22T17:34:46Z

Pull modules updates from Jessica Yu: "The main bulk of this pull request introduces a new exported symbol namespaces feature. The number of exported symbols is increasingly growing with each release (we're at about 31k exports as of 5.3-rc7) and we currently have no way of visualizing how these symbols are "clustered" or making sense of this huge export surface. Namespacing exported symbols allows kernel developers to more explicitly partition and categorize exported symbols, as well as more easily limiting the availability of namespaced symbols to other parts of the kernel. For starters, we have introduced the USB_STORAGE namespace to demonstrate the API's usage. I have briefly summarized the feature and its main motivations in the tag below. Summary: - Introduce exported symbol namespaces. This new feature allows subsystem maintainers to partition and categorize their exported symbols into explicit namespaces. Module authors are now required to import the namespaces they need. Some of the main motivations of this feature include: allowing kernel developers to better manage the export surface, allow subsystem maintainers to explicitly state that usage of some exported symbols should only be limited to certain users (think: inter-module or inter-driver symbols, debugging symbols, etc), as well as more easily limiting the availability of namespaced symbols to other parts of the kernel. With the module import requirement, it is also easier to spot the misuse of exported symbols during patch review. Two new macros are introduced: EXPORT_SYMBOL_NS() and EXPORT_SYMBOL_NS_GPL(). The API is thoroughly documented in Documentation/kbuild/namespaces.rst. - Some small code and kbuild cleanups here and there" * tag 'modules-for-v5.4' of git://git.kernel.org/pub/scm/linux/kernel/git/jeyu/linux: module: Remove leftover '#undef' from export header module: remove unneeded casts in cmp_name() module: move CONFIG_UNUSED_SYMBOLS to the sub-menu of MODULES module: remove redundant 'depends on MODULES' module: Fix link failure due to invalid relocation on namespace offset usb-storage: export symbols in USB_STORAGE namespace usb-storage: remove single-use define for debugging docs: Add documentation for Symbol Namespaces scripts: Coccinelle script for namespace dependencies. modpost: add support for generating namespace dependencies export: allow definition default namespaces in Makefiles or sources module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS modpost: add support for symbol namespaces module: add support for symbol namespaces. export: explicitly align struct kernel_symbol module: support reading multiple values per modinfo tag

export.h, genksyms: do not make genksyms calculate CRC of trimmed symbols

2019-09-14T02:40:13Z

Arnd Bergmann reported false-positive modpost warnings detected by his randconfig testing of linux-next. Actually, this happens under the combination of CONFIG_MODVERSIONS and CONFIG_TRIM_UNUSED_KSYMS since commit 15bfc2348d54 ("modpost: check for static EXPORT_SYMBOL* functions"). For example, arch/arm/config/multi_v7_defconfig + CONFIG_MODVERSIONS + CONFIG_TRIM_UNUSED_KSYMS produces the following false-positives: WARNING: "__lshrdi3" [vmlinux] is a static (unknown) WARNING: "__ashrdi3" [vmlinux] is a static (unknown) WARNING: "__aeabi_lasr" [vmlinux] is a static (unknown) WARNING: "__aeabi_llsr" [vmlinux] is a static (unknown) WARNING: "ftrace_set_clr_event" [vmlinux] is a static (unknown) WARNING: "__muldi3" [vmlinux] is a static (unknown) WARNING: "__aeabi_ulcmp" [vmlinux] is a static (unknown) WARNING: "__ucmpdi2" [vmlinux] is a static (unknown) WARNING: "__aeabi_lmul" [vmlinux] is a static (unknown) WARNING: "__bswapsi2" [vmlinux] is a static (unknown) WARNING: "__bswapdi2" [vmlinux] is a static (unknown) WARNING: "__ashldi3" [vmlinux] is a static (unknown) WARNING: "__aeabi_llsl" [vmlinux] is a static (unknown) The root cause of the problem is not in the modpost, but in the implementation of CONFIG_TRIM_UNUSED_KSYMS. If there is at least one untrimmed symbol in the file, genksyms is invoked to calculate CRC of *all* the exported symbols in that file even if some of them have been trimmed due to no caller existing. As a result, .tmp_*.ver files contain CRC of trimmed symbols, thus unneeded, orphan __crc* symbols are added to objects. It had been harmless until recently. With commit 15bfc2348d54 ("modpost: check for static EXPORT_SYMBOL* functions"), it is now harmful because the bogus __crc* symbols make modpost call sym_update_crc() to add the symbols to the hash table, but there is no one that clears the ->is_static member. I gave Fixes to the first commit that uncovered the issue, but the potential problem has long existed since commit f235541699bc ("export.h: allow for per-symbol configurable EXPORT_SYMBOL()"). Fixes: 15bfc2348d54 ("modpost: check for static EXPORT_SYMBOL* functions") Reported-by: Arnd Bergmann Signed-off-by: Masahiro Yamada Tested-by: Arnd Bergmann

module: Fix link failure due to invalid relocation on namespace offset

2019-09-11T16:53:30Z

Commit 8651ec01daed ("module: add support for symbol namespaces.") broke linking for arm64 defconfig: | lib/crypto/arc4.o: In function `__ksymtab_arc4_setkey': | arc4.c:(___ksymtab+arc4_setkey+0x8): undefined reference to `no symbol' | lib/crypto/arc4.o: In function `__ksymtab_arc4_crypt': | arc4.c:(___ksymtab+arc4_crypt+0x8): undefined reference to `no symbol' This is because the dummy initialisation of the 'namespace_offset' field in 'struct kernel_symbol' when using EXPORT_SYMBOL on architectures with support for PREL32 locations uses an offset from an absolute address (0) in an effort to trick 'offset_to_pointer' into behaving as a NOP, allowing non-namespaced symbols to be treated in the same way as those belonging to a namespace. Unfortunately, place-relative relocations require a symbol reference rather than an absolute value and, although x86 appears to get away with this due to placing the kernel text at the top of the address space, it almost certainly results in a runtime failure if the kernel is relocated dynamically as a result of KASLR. Rework 'namespace_offset' so that a value of 0, which cannot occur for a valid namespaced symbol, indicates that the corresponding symbol does not belong to a namespace. Cc: Matthias Maennich Cc: Jessica Yu Cc: Ard Biesheuvel Cc: Catalin Marinas Fixes: 8651ec01daed ("module: add support for symbol namespaces.") Reported-by: kbuild test robot Tested-by: Matthias Maennich Tested-by: Ard Biesheuvel Reviewed-by: Matthias Maennich Acked-by: Ard Biesheuvel Signed-off-by: Will Deacon Signed-off-by: Jessica Yu