user/sven/linux.git/scripts/Makefile.headersinst, branch v5.4.15

kbuild: fix 'No such file or directory' warning for headers_install

2019-06-23T18:43:03Z

Since commit d5470d14431e ("kbuild: re-implement Makefile.headersinst without recursion"), headers_install emits an ugly warning. $ make headers_install [ snip ] UPD include/generated/uapi/linux/version.h find: ‘./include/uapi/Kbuild’: No such file or directory HDRINST usr/include/video/uvesafb.h ... This happens for GNU Make <= 4.2.1 When I wrote that commit, I missed this warning because I was using the state-of-the-art Make version compiled from the git tree. $(wildcard $(src)/*/) is intended to match to only existing directories since it has a trailing slash, but actually matches to regular files too. (include/uapi/Kbuild in this case) This is a bug of GNU Make, and was fixed by: | commit b7acb10e86dc8f5fdf2a2bbd87e1059c315e31d6 | Author: spagoveanu@gmail.com | Date: Wed Jun 20 02:03:48 2018 +0300 | | * src/dir.c: Preserve glob d_type field We need to cater to old Make versions. Add '$(filter %/,...) to filter out the regular files. Fixes: d5470d14431e ("kbuild: re-implement Makefile.headersinst without recursion") Signed-off-by: Masahiro Yamada

kbuild: simplify scripts/headers_install.sh

2019-06-15T10:57:02Z

Now that headers_install.sh is invoked per file, remove the for-loop in the shell script. Signed-off-by: Masahiro Yamada

kbuild: re-implement Makefile.headersinst without recursion

2019-06-15T10:57:02Z

Since commit fcc8487d477a ("uapi: export all headers under uapi directories"), the headers in uapi directories are all exported by default although exceptional cases are still allowed by the syntax 'no-export-headers'. The traditional directory descending has been kept (in a somewhat hacky way), but it is actually unneeded. Get rid of it to simplify the code. Also, handle files one by one instead of the previous per-directory processing. This will emit much more log, but I like it. Signed-off-by: Masahiro Yamada

kbuild: add 'headers' target to build up uapi headers in usr/include

2019-06-15T10:57:02Z

In Linux build system, build targets and installation targets are separated. Examples are: - 'make vmlinux' -> 'make install' - 'make modules' -> 'make modules_install' - 'make dtbs' -> 'make dtbs_install' - 'make vdso' -> 'make vdso_install' The intention is to run the build targets under the normal privilege, then the installation targets under the root privilege since we need the write permission to the system directories. We have 'make headers_install' but the corresponding 'make headers' stage does not exist. The purpose of headers_install is to provide the kernel interface to C library. So, nobody would try to install headers to /usr/include directly. If 'sudo make INSTALL_HDR_PATH=/usr/include headers_install' were run, some build artifacts in the kernel tree would be owned by root because some of uapi headers are generated by 'uapi-asm-generic', 'archheaders' targets. Anyway, I believe it makes sense to split the header installation into two stages. [1] 'make headers' Process headers in uapi directories by scripts/headers_install.sh and copy them to usr/include [2] 'make headers_install' Copy '*.h' verbatim from usr/include to $(INSTALL_HDR_PATH)/include For the backward compatibility, 'headers_install' depends on 'headers'. Some samples expect uapi headers in usr/include. So, the 'headers' target is useful to build up them in the fixed location usr/include irrespective of INSTALL_HDR_PATH. Another benefit is to stop polluting the final destination with the time-stamp files '.install' and '.check'. Maybe you can see them in your toolchains. Lastly, my main motivation is to prepare for compile-testing uapi headers. To build something, we have to save an object and .*.cmd somewhere. The usr/include/ will be the work directory for that. Signed-off-by: Masahiro Yamada

kbuild: generate asm-generic wrappers if mandatory headers are missing

2019-01-06T00:46:51Z

Some time ago, Sam pointed out a certain degree of overwrap between generic-y and mandatory-y. (https://lkml.org/lkml/2017/7/10/121) I tweaked the meaning of mandatory-y a little bit; now it defines the minimum set of ASM headers that all architectures must have. If arch does not have specific implementation of a mandatory header, Kbuild will let it fallback to the asm-generic one by automatically generating a wrapper. This will allow to drop lots of redundant generic-y defines. Previously, "mandatory" was used in the context of UAPI, but I guess this can be extended to kernel space ASM headers. Suggested-by: Sam Ravnborg Signed-off-by: Masahiro Yamada Acked-by: Sam Ravnborg

kbuild: remove a special handling for *.agh in Makefile.headersinst

2018-12-08T01:52:20Z

scripts/Makefile.headersinst takes care of *.agh just for arch/cris/include/uapi/arch-v10/arch/sv_addr.agh because renaming exported headers is difficult (or impossible). This code is no longer necessary thanks to commit c690eddc2f3b ("CRIS: Drop support for the CRIS port"). Signed-off-by: Masahiro Yamada

Merge tag 'kbuild-v4.15' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild

2017-11-18T01:45:29Z

Pull Kbuild updates from Masahiro Yamada: "One of the most remarkable improvements in this cycle is, Kbuild is now able to cache the result of shell commands. Some variables are expensive to compute, for example, $(call cc-option,...) invokes the compiler. It is not efficient to redo this computation every time, even when we are not actually building anything. Kbuild creates a hidden file ".cache.mk" that contains invoked shell commands and their results. The speed-up should be noticeable. Summary: - Fix arch build issues (hexagon, sh) - Clean up various Makefiles and scripts - Fix wrong usage of {CFLAGS,LDFLAGS}_MODULE in arch Makefiles - Cache variables that are expensive to compute - Improve cc-ldopton and ld-option for Clang - Optimize output directory creation" * tag 'kbuild-v4.15' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (30 commits) kbuild: move coccicheck help from scripts/Makefile.help to top Makefile sh: decompressor: add shipped files to .gitignore frv: .gitignore: ignore vmlinux.lds selinux: remove unnecessary assignment to subdir- kbuild: specify FORCE in Makefile.headersinst as .PHONY target kbuild: remove redundant mkdir from ./Kbuild kbuild: optimize object directory creation for incremental build kbuild: create object directories simpler and faster kbuild: filter-out PHONY targets from "targets" kbuild: remove redundant $(wildcard ...) for cmd_files calculation kbuild: create directory for make cache only when necessary sh: select KBUILD_DEFCONFIG depending on ARCH kbuild: fix linker feature test macros when cross compiling with Clang kbuild: shrink .cache.mk when it exceeds 1000 lines kbuild: do not call cc-option before KBUILD_CFLAGS initialization kbuild: Cache a few more calls to the compiler kbuild: Add a cache for generated variables kbuild: add forward declaration of default target to Makefile.asm-generic kbuild: remove KBUILD_SUBDIR_ASFLAGS and KBUILD_SUBDIR_CCFLAGS hexagon/kbuild: replace CFLAGS_MODULE with KBUILD_CFLAGS_MODULE ...

kbuild: specify FORCE in Makefile.headersinst as .PHONY target

2017-11-16T00:11:18Z

Swap the order of ".PHONY: $(PHONY)" and "PHONY += FORCE" so that FORCE is correctly specified as a .PHONY target. Use a preferred way for specifying $(subdirs) as .PHONY targets. Signed-off-by: Masahiro Yamada

kbuild: remove redundant $(wildcard ...) for cmd_files calculation

2017-11-16T00:07:34Z

I do not see any reason why $(wildcard ...) needs to be called twice for computing cmd_files. Remove the first one. Signed-off-by: Masahiro Yamada

License cleanup: add SPDX GPL-2.0 license identifier to files with no license

2017-11-02T10:10:55Z

Many source files in the tree are missing licensing information, which makes it harder for compliance tools to determine the correct license. By default all files without license information are under the default license of the kernel, which is GPL version 2. Update the files which contain no license information with the 'GPL-2.0' SPDX license identifier. The SPDX identifier is a legally binding shorthand, which can be used instead of the full boiler plate text. This patch is based on work done by Thomas Gleixner and Kate Stewart and Philippe Ombredanne. How this work was done: Patches were generated and checked against linux-4.14-rc6 for a subset of the use cases: - file had no licensing information it it. - file was a */uapi/* one with no licensing information in it, - file was a */uapi/* one with existing licensing information, Further patches will be generated in subsequent months to fix up cases where non-standard license headers were used, and references to license had to be inferred by heuristics based on keywords. The analysis to determine which SPDX License Identifier to be applied to a file was done in a spreadsheet of side by side results from of the output of two independent scanners (ScanCode & Windriver) producing SPDX tag:value files created by Philippe Ombredanne. Philippe prepared the base worksheet, and did an initial spot review of a few 1000 files. The 4.13 kernel was the starting point of the analysis with 60,537 files assessed. Kate Stewart did a file by file comparison of the scanner results in the spreadsheet to determine which SPDX license identifier(s) to be applied to the file. She confirmed any determination that was not immediately clear with lawyers working with the Linux Foundation. Criteria used to select files for SPDX license identifier tagging was: - Files considered eligible had to be source code files. - Make and config files were included as candidates if they contained >5 lines of source - File already had some variant of a license header in it (even if <5 lines). All documentation files were explicitly excluded. The following heuristics were used to determine which SPDX license identifiers to apply. - when both scanners couldn't find any license traces, file was considered to have no license information in it, and the top level COPYING file license applied. For non */uapi/* files that summary was: SPDX license identifier # files ---------------------------------------------------|------- GPL-2.0 11139 and resulted in the first patch in this series. If that file was a */uapi/* path one, it was "GPL-2.0 WITH Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was: SPDX license identifier # files ---------------------------------------------------|------- GPL-2.0 WITH Linux-syscall-note 930 and resulted in the second patch in this series. - if a file had some form of licensing information in it, and was one of the */uapi/* ones, it was denoted with the Linux-syscall-note if any GPL family license was found in the file or had no licensing in it (per prior point). Results summary: SPDX license identifier # files ---------------------------------------------------|------ GPL-2.0 WITH Linux-syscall-note 270 GPL-2.0+ WITH Linux-syscall-note 169 ((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21 ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17 LGPL-2.1+ WITH Linux-syscall-note 15 GPL-1.0+ WITH Linux-syscall-note 14 ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5 LGPL-2.0+ WITH Linux-syscall-note 4 LGPL-2.1 WITH Linux-syscall-note 3 ((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3 ((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1 and that resulted in the third patch in this series. - when the two scanners agreed on the detected license(s), that became the concluded license(s). - when there was disagreement between the two scanners (one detected a license but the other didn't, or they both detected different licenses) a manual inspection of the file occurred. - In most cases a manual inspection of the information in the file resulted in a clear resolution of the license that should apply (and which scanner probably needed to revisit its heuristics). - When it was not immediately clear, the license identifier was confirmed with lawyers working with the Linux Foundation. - If there was any question as to the appropriate license identifier, the file was flagged for further research and to be revisited later in time. In total, over 70 hours of logged manual review was done on the spreadsheet to determine the SPDX license identifiers to apply to the source files by Kate, Philippe, Thomas and, in some cases, confirmation by lawyers working with the Linux Foundation. Kate also obtained a third independent scan of the 4.13 code base from FOSSology, and compared selected files where the other two scanners disagreed against that SPDX file, to see if there was new insights. The Windriver scanner is based on an older version of FOSSology in part, so they are related. Thomas did random spot checks in about 500 files from the spreadsheets for the uapi headers and agreed with SPDX license identifier in the files he inspected. For the non-uapi files Thomas did random spot checks in about 15000 files. In initial set of patches against 4.14-rc6, 3 files were found to have copy/paste license identifier errors, and have been fixed to reflect the correct identifier. Additionally Philippe spent 10 hours this week doing a detailed manual inspection and review of the 12,461 patched files from the initial patch version early this week with: - a full scancode scan run, collecting the matched texts, detected license ids and scores - reviewing anything where there was a license detected (about 500+ files) to ensure that the applied SPDX license was correct - reviewing anything where there was no detection but the patch license was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied SPDX license was correct This produced a worksheet with 20 files needing minor correction. This worksheet was then exported into 3 different .csv files for the different types of files to be modified. These .csv files were then reviewed by Greg. Thomas wrote a script to parse the csv files and add the proper SPDX tag to the file, in the format that the file expected. This script was further refined by Greg based on the output to detect more types of files automatically and to distinguish between header and source .c files (which need different comment types.) Finally Greg ran the script using the .csv files to generate the patches. Reviewed-by: Kate Stewart Reviewed-by: Philippe Ombredanne Reviewed-by: Thomas Gleixner Signed-off-by: Greg Kroah-Hartman