2012-05-05 01:18:41 -07:00
|
|
|
#!/bin/sh
|
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.
By default all files without license information are under the default
license of the kernel, which is GPL version 2.
Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier. The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.
This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.
How this work was done:
Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
- file had no licensing information it it.
- file was a */uapi/* one with no licensing information in it,
- file was a */uapi/* one with existing licensing information,
Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.
The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne. Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.
The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed. Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.
Criteria used to select files for SPDX license identifier tagging was:
- Files considered eligible had to be source code files.
- Make and config files were included as candidates if they contained >5
lines of source
- File already had some variant of a license header in it (even if <5
lines).
All documentation files were explicitly excluded.
The following heuristics were used to determine which SPDX license
identifiers to apply.
- when both scanners couldn't find any license traces, file was
considered to have no license information in it, and the top level
COPYING file license applied.
For non */uapi/* files that summary was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 11139
and resulted in the first patch in this series.
If that file was a */uapi/* path one, it was "GPL-2.0 WITH
Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 WITH Linux-syscall-note 930
and resulted in the second patch in this series.
- if a file had some form of licensing information in it, and was one
of the */uapi/* ones, it was denoted with the Linux-syscall-note if
any GPL family license was found in the file or had no licensing in
it (per prior point). Results summary:
SPDX license identifier # files
---------------------------------------------------|------
GPL-2.0 WITH Linux-syscall-note 270
GPL-2.0+ WITH Linux-syscall-note 169
((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21
((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17
LGPL-2.1+ WITH Linux-syscall-note 15
GPL-1.0+ WITH Linux-syscall-note 14
((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5
LGPL-2.0+ WITH Linux-syscall-note 4
LGPL-2.1 WITH Linux-syscall-note 3
((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3
((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1
and that resulted in the third patch in this series.
- when the two scanners agreed on the detected license(s), that became
the concluded license(s).
- when there was disagreement between the two scanners (one detected a
license but the other didn't, or they both detected different
licenses) a manual inspection of the file occurred.
- In most cases a manual inspection of the information in the file
resulted in a clear resolution of the license that should apply (and
which scanner probably needed to revisit its heuristics).
- When it was not immediately clear, the license identifier was
confirmed with lawyers working with the Linux Foundation.
- If there was any question as to the appropriate license identifier,
the file was flagged for further research and to be revisited later
in time.
In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases, confirmation
by lawyers working with the Linux Foundation.
Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights. The
Windriver scanner is based on an older version of FOSSology in part, so
they are related.
Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot checks
in about 15000 files.
In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect the
correct identifier.
Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial patch
version early this week with:
- a full scancode scan run, collecting the matched texts, detected
license ids and scores
- reviewing anything where there was a license detected (about 500+
files) to ensure that the applied SPDX license was correct
- reviewing anything where there was no detection but the patch license
was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
SPDX license was correct
This produced a worksheet with 20 files needing minor correction. This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.
These .csv files were then reviewed by Greg. Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected. This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.) Finally Greg ran the script using the .csv files to
generate the patches.
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2017-11-01 07:07:57 -07:00
|
|
|
# SPDX-License-Identifier: GPL-2.0
|
2012-05-05 01:18:41 -07:00
|
|
|
#
|
|
|
|
# link vmlinux
|
|
|
|
#
|
2022-09-24 11:19:14 -07:00
|
|
|
# vmlinux is linked from the objects in vmlinux.a and $(KBUILD_VMLINUX_LIBS).
|
|
|
|
# vmlinux.a contains objects that are linked unconditionally.
|
2019-01-16 17:10:04 -07:00
|
|
|
# $(KBUILD_VMLINUX_LIBS) are archives which are linked conditionally
|
|
|
|
# (not within --whole-archive), and do not require symbol indexes added.
|
2012-05-05 01:18:41 -07:00
|
|
|
#
|
|
|
|
# vmlinux
|
|
|
|
# ^
|
|
|
|
# |
|
2022-09-24 11:19:14 -07:00
|
|
|
# +--< vmlinux.a
|
2012-05-05 01:18:41 -07:00
|
|
|
# |
|
2017-06-19 08:52:05 -07:00
|
|
|
# +--< $(KBUILD_VMLINUX_LIBS)
|
|
|
|
# | +--< lib/lib.a + more
|
|
|
|
# |
|
2012-05-05 01:18:41 -07:00
|
|
|
# +-< ${kallsymso} (see description in KALLSYMS section)
|
|
|
|
#
|
|
|
|
# vmlinux version (uname -v) cannot be updated during normal
|
|
|
|
# descending-into-subdirs phase since we do not yet know if we need to
|
|
|
|
# update vmlinux.
|
|
|
|
# Therefore this step is delayed until just before final link of vmlinux.
|
|
|
|
#
|
|
|
|
# System.map is generated to document addresses of all kernel symbols
|
|
|
|
|
|
|
|
# Error out on error
|
|
|
|
set -e
|
|
|
|
|
kbuild: do not export LDFLAGS_vmlinux
When you clean the build tree for ARCH=arm, you may see the following
error message from 'nm' command:
$ make -j24 ARCH=arm clean
CLEAN arch/arm/crypto
CLEAN arch/arm/kernel
CLEAN arch/arm/mach-at91
CLEAN arch/arm/mach-omap2
CLEAN arch/arm/vdso
CLEAN certs
CLEAN lib
CLEAN usr
CLEAN net/wireless
CLEAN drivers/firmware/efi/libstub
nm: 'arch/arm/boot/compressed/../../../../vmlinux': No such file
/bin/sh: 1: arithmetic expression: expecting primary: " "
CLEAN arch/arm/boot/compressed
CLEAN drivers/scsi
CLEAN drivers/tty/vt
CLEAN arch/arm/boot
CLEAN vmlinux.symvers modules.builtin modules.builtin.modinfo
Even if you rerun the same command, the error message will not be
shown despite vmlinux is already gone.
To reproduce it, the parallel option -j is needed. Single thread
cleaning always executes 'archclean', 'vmlinuxclean' in this order,
so vmlinux still exists when arch/arm/boot/compressed/ is cleaned.
Looking at arch/arm/boot/compressed/Makefile does not help understand
the reason of the error message. Both KBSS_SZ and LDFLAGS_vmlinux are
assigned with '=' operator, hence, they are not expanded unless used.
Obviously, 'make clean' does not use them.
In fact, the root cause exists in the top Makefile:
export LDFLAGS_vmlinux
Since LDFLAGS_vmlinux is an exported variable, LDFLAGS_vmlinux in
arch/arm/boot/compressed/Makefile is expanded when scripts/Makefile.clean
has a command to execute. This is why the error message shows up only
when there exist build artifacts in arch/arm/boot/compressed/.
Adding 'unexport LDFLAGS_vmlinux' to arch/arm/boot/compressed/Makefile
will fix it as far as ARCH=arm is concerned, but I think the proper fix
is to get rid of 'export LDFLAGS_vmlinux' from the top Makefile.
LDFLAGS_vmlinux in the top Makefile contains linker flags for the top
vmlinux. LDFLAGS_vmlinux in arch/arm/boot/compressed/Makefile is for
arch/arm/boot/compressed/vmlinux. They just happen to have the same
variable name, but are used for different purposes. Stop shadowing
LDFLAGS_vmlinux.
This commit passes LDFLAGS_vmlinux to scripts/link-vmlinux.sh via a
command line parameter instead of via an environment variable. LD and
KBUILD_LDFLAGS are exported, but I did the same for consistency. Anyway,
they must be included in cmd_link-vmlinux to allow if_changed to detect
the changes in LD or KBUILD_LDFLAGS.
The following Makefiles are not affected:
arch/arm/boot/compressed/Makefile
arch/h8300/boot/compressed/Makefile
arch/nios2/boot/compressed/Makefile
arch/parisc/boot/compressed/Makefile
arch/s390/boot/compressed/Makefile
arch/sh/boot/compressed/Makefile
arch/sh/boot/romimage/Makefile
arch/x86/boot/compressed/Makefile
They use ':=' or '=' to clear the LDFLAGS_vmlinux inherited from the
top Makefile.
We need to take a closer look at the impact to unicore32 and xtensa.
arch/unicore32/boot/compressed/Makefile only uses '+=' operator for
LDFLAGS_vmlinux. So, the decompressor previously inherited the linker
flags from the top Makefile.
However, commit 70fac51feaf2 ("unicore32 additional architecture files:
boot process") was merged before commit 1f2bfbd00e46 ("kbuild: link of
vmlinux moved to a script"). So, I rather consider this is a bug fix of
1f2bfbd00e46.
arch/xtensa/boot/boot-elf/Makefile is also affected, but this is also
considered a fix for the same reason. It did not inherit LDFLAGS_vmlinux
when commit 4bedea945451 ("[PATCH] xtensa: Architecture support for
Tensilica Xtensa Part 2") was merged. I deleted $(LDFLAGS_vmlinux),
which is now empty.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Tested-by: Nick Desaulniers <ndesaulniers@google.com>
2020-07-01 12:29:36 -07:00
|
|
|
LD="$1"
|
|
|
|
KBUILD_LDFLAGS="$2"
|
|
|
|
LDFLAGS_vmlinux="$3"
|
|
|
|
|
2021-12-13 19:53:52 -07:00
|
|
|
is_enabled() {
|
|
|
|
grep -q "^$1=y" include/config/auto.conf
|
|
|
|
}
|
|
|
|
|
2012-05-05 01:18:41 -07:00
|
|
|
# Nice output in kbuild format
|
|
|
|
# Will be supressed by "make -s"
|
|
|
|
info()
|
|
|
|
{
|
2021-05-17 00:03:14 -07:00
|
|
|
printf " %-7s %s\n" "${1}" "${2}"
|
2012-05-05 01:18:41 -07:00
|
|
|
}
|
|
|
|
|
|
|
|
# Link of vmlinux
|
btf: expose BTF info through sysfs
Make .BTF section allocated and expose its contents through sysfs.
/sys/kernel/btf directory is created to contain all the BTFs present
inside kernel. Currently there is only kernel's main BTF, represented as
/sys/kernel/btf/kernel file. Once kernel modules' BTFs are supported,
each module will expose its BTF as /sys/kernel/btf/<module-name> file.
Current approach relies on a few pieces coming together:
1. pahole is used to take almost final vmlinux image (modulo .BTF and
kallsyms) and generate .BTF section by converting DWARF info into
BTF. This section is not allocated and not mapped to any segment,
though, so is not yet accessible from inside kernel at runtime.
2. objcopy dumps .BTF contents into binary file and subsequently
convert binary file into linkable object file with automatically
generated symbols _binary__btf_kernel_bin_start and
_binary__btf_kernel_bin_end, pointing to start and end, respectively,
of BTF raw data.
3. final vmlinux image is generated by linking this object file (and
kallsyms, if necessary). sysfs_btf.c then creates
/sys/kernel/btf/kernel file and exposes embedded BTF contents through
it. This allows, e.g., libbpf and bpftool access BTF info at
well-known location, without resorting to searching for vmlinux image
on disk (location of which is not standardized and vmlinux image
might not be even available in some scenarios, e.g., inside qemu
during testing).
Alternative approach using .incbin assembler directive to embed BTF
contents directly was attempted but didn't work, because sysfs_proc.o is
not re-compiled during link-vmlinux.sh stage. This is required, though,
to update embedded BTF data (initially empty data is embedded, then
pahole generates BTF info and we need to regenerate sysfs_btf.o with
updated contents, but it's too late at that point).
If BTF couldn't be generated due to missing or too old pahole,
sysfs_btf.c handles that gracefully by detecting that
_binary__btf_kernel_bin_start (weak symbol) is 0 and not creating
/sys/kernel/btf at all.
v2->v3:
- added Documentation/ABI/testing/sysfs-kernel-btf (Greg K-H);
- created proper kobject (btf_kobj) for btf directory (Greg K-H);
- undo v2 change of reusing vmlinux, as it causes extra kallsyms pass
due to initially missing __binary__btf_kernel_bin_{start/end} symbols;
v1->v2:
- allow kallsyms stage to re-use vmlinux generated by gen_btf();
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-08-12 11:39:47 -07:00
|
|
|
# ${1} - output file
|
2012-05-05 01:18:41 -07:00
|
|
|
vmlinux_link()
|
|
|
|
{
|
2019-09-05 10:59:38 -07:00
|
|
|
local output=${1}
|
2021-08-18 17:57:37 -07:00
|
|
|
local objs
|
|
|
|
local libs
|
2021-08-18 17:57:39 -07:00
|
|
|
local ld
|
|
|
|
local ldflags
|
|
|
|
local ldlibs
|
2012-05-05 01:18:41 -07:00
|
|
|
|
2019-09-20 08:36:47 -07:00
|
|
|
info LD ${output}
|
|
|
|
|
2019-09-05 10:59:38 -07:00
|
|
|
# skip output file argument
|
|
|
|
shift
|
|
|
|
|
2022-03-08 08:30:56 -07:00
|
|
|
if is_enabled CONFIG_LTO_CLANG || is_enabled CONFIG_X86_KERNEL_IBT; then
|
2021-08-18 17:57:37 -07:00
|
|
|
# Use vmlinux.o instead of performing the slow LTO link again.
|
|
|
|
objs=vmlinux.o
|
|
|
|
libs=
|
|
|
|
else
|
2022-09-24 11:19:14 -07:00
|
|
|
objs=vmlinux.a
|
2021-08-18 17:57:37 -07:00
|
|
|
libs="${KBUILD_VMLINUX_LIBS}"
|
|
|
|
fi
|
|
|
|
|
kbuild: link symbol CRCs at final link, removing CONFIG_MODULE_REL_CRCS
include/{linux,asm-generic}/export.h defines a weak symbol, __crc_*
as a placeholder.
Genksyms writes the version CRCs into the linker script, which will be
used for filling the __crc_* symbols. The linker script format depends
on CONFIG_MODULE_REL_CRCS. If it is enabled, __crc_* holds the offset
to the reference of CRC.
It is time to get rid of this complexity.
Now that modpost parses text files (.*.cmd) to collect all the CRCs,
it can generate C code that will be linked to the vmlinux or modules.
Generate a new C file, .vmlinux.export.c, which contains the CRCs of
symbols exported by vmlinux. It is compiled and linked to vmlinux in
scripts/link-vmlinux.sh.
Put the CRCs of symbols exported by modules into the existing *.mod.c
files. No additional build step is needed for modules. As before,
*.mod.c are compiled and linked to *.ko in scripts/Makefile.modfinal.
No linker magic is used here. The new C implementation works in the
same way, whether CONFIG_RELOCATABLE is enabled or not.
CONFIG_MODULE_REL_CRCS is no longer needed.
Previously, Kbuild invoked additional $(LD) to update the CRCs in
objects, but this step is unneeded too.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Tested-by: Nathan Chancellor <nathan@kernel.org>
Tested-by: Nicolas Schier <nicolas@fjasle.eu>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com> # LLVM-14 (x86-64)
2022-05-13 04:39:22 -07:00
|
|
|
if is_enabled CONFIG_MODULES; then
|
|
|
|
objs="${objs} .vmlinux.export.o"
|
|
|
|
fi
|
|
|
|
|
kbuild: build init/built-in.a just once
Kbuild builds init/built-in.a twice; first during the ordinary
directory descending, second from scripts/link-vmlinux.sh.
We do this because UTS_VERSION contains the build version and the
timestamp. We cannot update it during the normal directory traversal
since we do not yet know if we need to update vmlinux. UTS_VERSION is
temporarily calculated, but omitted from the update check. Otherwise,
vmlinux would be rebuilt every time.
When Kbuild results in running link-vmlinux.sh, it increments the
version number in the .version file and takes the timestamp at that
time to really fix UTS_VERSION.
However, updating the same file twice is a footgun. To avoid nasty
timestamp issues, all build artifacts that depend on init/built-in.a
are atomically generated in link-vmlinux.sh, where some of them do not
need rebuilding.
To fix this issue, this commit changes as follows:
[1] Split UTS_VERSION out to include/generated/utsversion.h from
include/generated/compile.h
include/generated/utsversion.h is generated just before the
vmlinux link. It is generated under include/generated/ because
some decompressors (s390, x86) use UTS_VERSION.
[2] Split init_uts_ns and linux_banner out to init/version-timestamp.c
from init/version.c
init_uts_ns and linux_banner contain UTS_VERSION. During the ordinary
directory descending, they are compiled with __weak and used to
determine if vmlinux needs relinking. Just before the vmlinux link,
they are compiled without __weak to embed the real version and
timestamp.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2022-08-27 19:39:53 -07:00
|
|
|
objs="${objs} init/version-timestamp.o"
|
|
|
|
|
2021-08-18 17:57:39 -07:00
|
|
|
if [ "${SRCARCH}" = "um" ]; then
|
|
|
|
wl=-Wl,
|
|
|
|
ld="${CC}"
|
|
|
|
ldflags="${CFLAGS_vmlinux}"
|
|
|
|
ldlibs="-lutil -lrt -lpthread"
|
|
|
|
else
|
|
|
|
wl=
|
|
|
|
ld="${LD}"
|
|
|
|
ldflags="${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}"
|
|
|
|
ldlibs=
|
|
|
|
fi
|
|
|
|
|
|
|
|
ldflags="${ldflags} ${wl}--script=${objtree}/${KBUILD_LDS}"
|
|
|
|
|
2020-03-03 19:18:34 -07:00
|
|
|
# The kallsyms linking does not need debug symbols included.
|
2024-06-10 04:25:16 -07:00
|
|
|
if [ -n "${strip_debug}" ] ; then
|
2021-08-18 17:57:39 -07:00
|
|
|
ldflags="${ldflags} ${wl}--strip-debug"
|
2020-03-03 19:18:34 -07:00
|
|
|
fi
|
|
|
|
|
2021-12-13 19:53:52 -07:00
|
|
|
if is_enabled CONFIG_VMLINUX_MAP; then
|
2021-08-18 17:57:39 -07:00
|
|
|
ldflags="${ldflags} ${wl}-Map=${output}.map"
|
2021-03-05 02:27:07 -07:00
|
|
|
fi
|
|
|
|
|
2021-08-18 17:57:39 -07:00
|
|
|
${ld} ${ldflags} -o ${output} \
|
|
|
|
${wl}--whole-archive ${objs} ${wl}--no-whole-archive \
|
|
|
|
${wl}--start-group ${libs} ${wl}--end-group \
|
2024-06-10 04:25:16 -07:00
|
|
|
${kallsymso} ${btf_vmlinux_bin_o} ${ldlibs}
|
2012-05-05 01:18:41 -07:00
|
|
|
}
|
|
|
|
|
kbuild: add ability to generate BTF type info for vmlinux
This patch adds new config option to trigger generation of BTF type
information from DWARF debuginfo for vmlinux and kernel modules through
pahole, which in turn relies on libbpf for btf_dedup() algorithm.
The intent is to record compact type information of all types used
inside kernel, including all the structs/unions/typedefs/etc. This
enables BPF's compile-once-run-everywhere ([0]) approach, in which
tracing programs that are inspecting kernel's internal data (e.g.,
struct task_struct) can be compiled on a system running some kernel
version, but would be possible to run on other kernel versions (and
configurations) without recompilation, even if the layout of structs
changed and/or some of the fields were added, removed, or renamed.
This is only possible if BPF loader can get kernel type info to adjust
all the offsets correctly. This patch is a first time in this direction,
making sure that BTF type info is part of Linux kernel image in
non-loadable ELF section.
BTF deduplication ([1]) algorithm typically provides 100x savings
compared to DWARF data, so resulting .BTF section is not big as is
typically about 2MB in size.
[0] http://vger.kernel.org/lpc-bpf2018.html#session-2
[1] https://facebookmicrosites.github.io/bpf/blog/2018/11/14/btf-enhancement.html
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Acked-by: David S. Miller <davem@davemloft.net>
Acked-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-04-02 09:49:50 -07:00
|
|
|
# generate .BTF typeinfo from DWARF debuginfo
|
btf: expose BTF info through sysfs
Make .BTF section allocated and expose its contents through sysfs.
/sys/kernel/btf directory is created to contain all the BTFs present
inside kernel. Currently there is only kernel's main BTF, represented as
/sys/kernel/btf/kernel file. Once kernel modules' BTFs are supported,
each module will expose its BTF as /sys/kernel/btf/<module-name> file.
Current approach relies on a few pieces coming together:
1. pahole is used to take almost final vmlinux image (modulo .BTF and
kallsyms) and generate .BTF section by converting DWARF info into
BTF. This section is not allocated and not mapped to any segment,
though, so is not yet accessible from inside kernel at runtime.
2. objcopy dumps .BTF contents into binary file and subsequently
convert binary file into linkable object file with automatically
generated symbols _binary__btf_kernel_bin_start and
_binary__btf_kernel_bin_end, pointing to start and end, respectively,
of BTF raw data.
3. final vmlinux image is generated by linking this object file (and
kallsyms, if necessary). sysfs_btf.c then creates
/sys/kernel/btf/kernel file and exposes embedded BTF contents through
it. This allows, e.g., libbpf and bpftool access BTF info at
well-known location, without resorting to searching for vmlinux image
on disk (location of which is not standardized and vmlinux image
might not be even available in some scenarios, e.g., inside qemu
during testing).
Alternative approach using .incbin assembler directive to embed BTF
contents directly was attempted but didn't work, because sysfs_proc.o is
not re-compiled during link-vmlinux.sh stage. This is required, though,
to update embedded BTF data (initially empty data is embedded, then
pahole generates BTF info and we need to regenerate sysfs_btf.o with
updated contents, but it's too late at that point).
If BTF couldn't be generated due to missing or too old pahole,
sysfs_btf.c handles that gracefully by detecting that
_binary__btf_kernel_bin_start (weak symbol) is 0 and not creating
/sys/kernel/btf at all.
v2->v3:
- added Documentation/ABI/testing/sysfs-kernel-btf (Greg K-H);
- created proper kobject (btf_kobj) for btf directory (Greg K-H);
- undo v2 change of reusing vmlinux, as it causes extra kallsyms pass
due to initially missing __binary__btf_kernel_bin_{start/end} symbols;
v1->v2:
- allow kallsyms stage to re-use vmlinux generated by gen_btf();
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-08-12 11:39:47 -07:00
|
|
|
# ${1} - vmlinux image
|
kbuild: add ability to generate BTF type info for vmlinux
This patch adds new config option to trigger generation of BTF type
information from DWARF debuginfo for vmlinux and kernel modules through
pahole, which in turn relies on libbpf for btf_dedup() algorithm.
The intent is to record compact type information of all types used
inside kernel, including all the structs/unions/typedefs/etc. This
enables BPF's compile-once-run-everywhere ([0]) approach, in which
tracing programs that are inspecting kernel's internal data (e.g.,
struct task_struct) can be compiled on a system running some kernel
version, but would be possible to run on other kernel versions (and
configurations) without recompilation, even if the layout of structs
changed and/or some of the fields were added, removed, or renamed.
This is only possible if BPF loader can get kernel type info to adjust
all the offsets correctly. This patch is a first time in this direction,
making sure that BTF type info is part of Linux kernel image in
non-loadable ELF section.
BTF deduplication ([1]) algorithm typically provides 100x savings
compared to DWARF data, so resulting .BTF section is not big as is
typically about 2MB in size.
[0] http://vger.kernel.org/lpc-bpf2018.html#session-2
[1] https://facebookmicrosites.github.io/bpf/blog/2018/11/14/btf-enhancement.html
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Acked-by: David S. Miller <davem@davemloft.net>
Acked-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-04-02 09:49:50 -07:00
|
|
|
gen_btf()
|
|
|
|
{
|
2024-06-10 04:25:18 -07:00
|
|
|
local btf_data=${1}.btf.o
|
kbuild: add ability to generate BTF type info for vmlinux
This patch adds new config option to trigger generation of BTF type
information from DWARF debuginfo for vmlinux and kernel modules through
pahole, which in turn relies on libbpf for btf_dedup() algorithm.
The intent is to record compact type information of all types used
inside kernel, including all the structs/unions/typedefs/etc. This
enables BPF's compile-once-run-everywhere ([0]) approach, in which
tracing programs that are inspecting kernel's internal data (e.g.,
struct task_struct) can be compiled on a system running some kernel
version, but would be possible to run on other kernel versions (and
configurations) without recompilation, even if the layout of structs
changed and/or some of the fields were added, removed, or renamed.
This is only possible if BPF loader can get kernel type info to adjust
all the offsets correctly. This patch is a first time in this direction,
making sure that BTF type info is part of Linux kernel image in
non-loadable ELF section.
BTF deduplication ([1]) algorithm typically provides 100x savings
compared to DWARF data, so resulting .BTF section is not big as is
typically about 2MB in size.
[0] http://vger.kernel.org/lpc-bpf2018.html#session-2
[1] https://facebookmicrosites.github.io/bpf/blog/2018/11/14/btf-enhancement.html
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Acked-by: David S. Miller <davem@davemloft.net>
Acked-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-04-02 09:49:50 -07:00
|
|
|
|
2024-06-10 04:25:16 -07:00
|
|
|
info BTF "${btf_data}"
|
2021-10-29 05:57:29 -07:00
|
|
|
LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} ${1}
|
btf: expose BTF info through sysfs
Make .BTF section allocated and expose its contents through sysfs.
/sys/kernel/btf directory is created to contain all the BTFs present
inside kernel. Currently there is only kernel's main BTF, represented as
/sys/kernel/btf/kernel file. Once kernel modules' BTFs are supported,
each module will expose its BTF as /sys/kernel/btf/<module-name> file.
Current approach relies on a few pieces coming together:
1. pahole is used to take almost final vmlinux image (modulo .BTF and
kallsyms) and generate .BTF section by converting DWARF info into
BTF. This section is not allocated and not mapped to any segment,
though, so is not yet accessible from inside kernel at runtime.
2. objcopy dumps .BTF contents into binary file and subsequently
convert binary file into linkable object file with automatically
generated symbols _binary__btf_kernel_bin_start and
_binary__btf_kernel_bin_end, pointing to start and end, respectively,
of BTF raw data.
3. final vmlinux image is generated by linking this object file (and
kallsyms, if necessary). sysfs_btf.c then creates
/sys/kernel/btf/kernel file and exposes embedded BTF contents through
it. This allows, e.g., libbpf and bpftool access BTF info at
well-known location, without resorting to searching for vmlinux image
on disk (location of which is not standardized and vmlinux image
might not be even available in some scenarios, e.g., inside qemu
during testing).
Alternative approach using .incbin assembler directive to embed BTF
contents directly was attempted but didn't work, because sysfs_proc.o is
not re-compiled during link-vmlinux.sh stage. This is required, though,
to update embedded BTF data (initially empty data is embedded, then
pahole generates BTF info and we need to regenerate sysfs_btf.o with
updated contents, but it's too late at that point).
If BTF couldn't be generated due to missing or too old pahole,
sysfs_btf.c handles that gracefully by detecting that
_binary__btf_kernel_bin_start (weak symbol) is 0 and not creating
/sys/kernel/btf at all.
v2->v3:
- added Documentation/ABI/testing/sysfs-kernel-btf (Greg K-H);
- created proper kobject (btf_kobj) for btf directory (Greg K-H);
- undo v2 change of reusing vmlinux, as it causes extra kallsyms pass
due to initially missing __binary__btf_kernel_bin_{start/end} symbols;
v1->v2:
- allow kallsyms stage to re-use vmlinux generated by gen_btf();
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-08-12 11:39:47 -07:00
|
|
|
|
2024-06-10 04:25:16 -07:00
|
|
|
# Create ${btf_data} which contains just .BTF section but no symbols. Add
|
2020-03-18 15:27:46 -07:00
|
|
|
# SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all
|
|
|
|
# deletes all symbols including __start_BTF and __stop_BTF, which will
|
|
|
|
# be redefined in the linker script. Add 2>/dev/null to suppress GNU
|
|
|
|
# objcopy warnings: "empty loadable segment detected at ..."
|
|
|
|
${OBJCOPY} --only-section=.BTF --set-section-flags .BTF=alloc,readonly \
|
2024-06-10 04:25:16 -07:00
|
|
|
--strip-all ${1} "${btf_data}" 2>/dev/null
|
2020-03-18 15:27:46 -07:00
|
|
|
# Change e_type to ET_REL so that it can be used to link final vmlinux.
|
2024-02-12 19:05:10 -07:00
|
|
|
# GNU ld 2.35+ and lld do not allow an ET_EXEC input.
|
|
|
|
if is_enabled CONFIG_CPU_BIG_ENDIAN; then
|
|
|
|
et_rel='\0\1'
|
|
|
|
else
|
|
|
|
et_rel='\1\0'
|
|
|
|
fi
|
2024-06-10 04:25:16 -07:00
|
|
|
printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none
|
|
|
|
|
|
|
|
btf_vmlinux_bin_o=${btf_data}
|
kbuild: add ability to generate BTF type info for vmlinux
This patch adds new config option to trigger generation of BTF type
information from DWARF debuginfo for vmlinux and kernel modules through
pahole, which in turn relies on libbpf for btf_dedup() algorithm.
The intent is to record compact type information of all types used
inside kernel, including all the structs/unions/typedefs/etc. This
enables BPF's compile-once-run-everywhere ([0]) approach, in which
tracing programs that are inspecting kernel's internal data (e.g.,
struct task_struct) can be compiled on a system running some kernel
version, but would be possible to run on other kernel versions (and
configurations) without recompilation, even if the layout of structs
changed and/or some of the fields were added, removed, or renamed.
This is only possible if BPF loader can get kernel type info to adjust
all the offsets correctly. This patch is a first time in this direction,
making sure that BTF type info is part of Linux kernel image in
non-loadable ELF section.
BTF deduplication ([1]) algorithm typically provides 100x savings
compared to DWARF data, so resulting .BTF section is not big as is
typically about 2MB in size.
[0] http://vger.kernel.org/lpc-bpf2018.html#session-2
[1] https://facebookmicrosites.github.io/bpf/blog/2018/11/14/btf-enhancement.html
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Acked-by: David S. Miller <davem@davemloft.net>
Acked-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-04-02 09:49:50 -07:00
|
|
|
}
|
2012-05-05 01:18:41 -07:00
|
|
|
|
2024-06-10 04:25:16 -07:00
|
|
|
# Create ${2}.o file with all symbols from the ${1} object file
|
2012-05-05 01:18:41 -07:00
|
|
|
kallsyms()
|
|
|
|
{
|
|
|
|
local kallsymopt;
|
|
|
|
|
2021-12-13 19:53:52 -07:00
|
|
|
if is_enabled CONFIG_KALLSYMS_ALL; then
|
2012-09-06 14:11:25 -07:00
|
|
|
kallsymopt="${kallsymopt} --all-symbols"
|
2012-05-05 01:18:41 -07:00
|
|
|
fi
|
|
|
|
|
2021-12-13 19:53:52 -07:00
|
|
|
if is_enabled CONFIG_KALLSYMS_ABSOLUTE_PERCPU; then
|
2014-03-16 20:35:46 -07:00
|
|
|
kallsymopt="${kallsymopt} --absolute-percpu"
|
|
|
|
fi
|
|
|
|
|
2024-06-10 04:25:16 -07:00
|
|
|
info KSYMS "${2}.S"
|
|
|
|
scripts/kallsyms ${kallsymopt} "${1}" > "${2}.S"
|
|
|
|
|
|
|
|
info AS "${2}.o"
|
|
|
|
${CC} ${NOSTDINC_FLAGS} ${LINUXINCLUDE} ${KBUILD_CPPFLAGS} \
|
|
|
|
${KBUILD_AFLAGS} ${KBUILD_AFLAGS_KERNEL} -c -o "${2}.o" "${2}.S"
|
|
|
|
|
|
|
|
kallsymso=${2}.o
|
2012-05-05 01:18:41 -07:00
|
|
|
}
|
|
|
|
|
2024-06-10 04:25:18 -07:00
|
|
|
# Perform kallsyms for the given temporary vmlinux.
|
|
|
|
sysmap_and_kallsyms()
|
2019-08-13 08:15:32 -07:00
|
|
|
{
|
2024-06-10 04:25:18 -07:00
|
|
|
mksysmap "${1}" "${1}.syms"
|
|
|
|
kallsyms "${1}.syms" "${1}.kallsyms"
|
2020-09-24 08:45:46 -07:00
|
|
|
|
2024-06-10 04:25:18 -07:00
|
|
|
kallsyms_sysmap=${1}.syms
|
2019-08-13 08:15:32 -07:00
|
|
|
}
|
|
|
|
|
2012-05-05 01:18:41 -07:00
|
|
|
# Create map file with all symbols from ${1}
|
|
|
|
# See mksymap for additional details
|
|
|
|
mksysmap()
|
|
|
|
{
|
2022-09-26 02:02:26 -07:00
|
|
|
info NM ${2}
|
2024-06-05 10:58:09 -07:00
|
|
|
${NM} -n "${1}" | sed -f "${srctree}/scripts/mksysmap" > "${2}"
|
2012-05-05 01:18:41 -07:00
|
|
|
}
|
|
|
|
|
2019-12-03 17:46:31 -07:00
|
|
|
sorttable()
|
2012-05-28 10:32:28 -07:00
|
|
|
{
|
2019-12-03 17:46:31 -07:00
|
|
|
${objtree}/scripts/sorttable ${1}
|
2012-05-28 10:32:28 -07:00
|
|
|
}
|
|
|
|
|
2012-05-05 01:18:41 -07:00
|
|
|
cleanup()
|
|
|
|
{
|
btf: expose BTF info through sysfs
Make .BTF section allocated and expose its contents through sysfs.
/sys/kernel/btf directory is created to contain all the BTFs present
inside kernel. Currently there is only kernel's main BTF, represented as
/sys/kernel/btf/kernel file. Once kernel modules' BTFs are supported,
each module will expose its BTF as /sys/kernel/btf/<module-name> file.
Current approach relies on a few pieces coming together:
1. pahole is used to take almost final vmlinux image (modulo .BTF and
kallsyms) and generate .BTF section by converting DWARF info into
BTF. This section is not allocated and not mapped to any segment,
though, so is not yet accessible from inside kernel at runtime.
2. objcopy dumps .BTF contents into binary file and subsequently
convert binary file into linkable object file with automatically
generated symbols _binary__btf_kernel_bin_start and
_binary__btf_kernel_bin_end, pointing to start and end, respectively,
of BTF raw data.
3. final vmlinux image is generated by linking this object file (and
kallsyms, if necessary). sysfs_btf.c then creates
/sys/kernel/btf/kernel file and exposes embedded BTF contents through
it. This allows, e.g., libbpf and bpftool access BTF info at
well-known location, without resorting to searching for vmlinux image
on disk (location of which is not standardized and vmlinux image
might not be even available in some scenarios, e.g., inside qemu
during testing).
Alternative approach using .incbin assembler directive to embed BTF
contents directly was attempted but didn't work, because sysfs_proc.o is
not re-compiled during link-vmlinux.sh stage. This is required, though,
to update embedded BTF data (initially empty data is embedded, then
pahole generates BTF info and we need to regenerate sysfs_btf.o with
updated contents, but it's too late at that point).
If BTF couldn't be generated due to missing or too old pahole,
sysfs_btf.c handles that gracefully by detecting that
_binary__btf_kernel_bin_start (weak symbol) is 0 and not creating
/sys/kernel/btf at all.
v2->v3:
- added Documentation/ABI/testing/sysfs-kernel-btf (Greg K-H);
- created proper kobject (btf_kobj) for btf directory (Greg K-H);
- undo v2 change of reusing vmlinux, as it causes extra kallsyms pass
due to initially missing __binary__btf_kernel_bin_{start/end} symbols;
v1->v2:
- allow kallsyms stage to re-use vmlinux generated by gen_btf();
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-08-12 11:39:47 -07:00
|
|
|
rm -f .btf.*
|
2012-05-05 01:18:41 -07:00
|
|
|
rm -f System.map
|
|
|
|
rm -f vmlinux
|
2021-03-05 02:27:07 -07:00
|
|
|
rm -f vmlinux.map
|
2012-05-05 01:18:41 -07:00
|
|
|
}
|
|
|
|
|
|
|
|
# Use "make V=1" to debug this script
|
|
|
|
case "${KBUILD_VERBOSE}" in
|
|
|
|
*1*)
|
|
|
|
set -x
|
|
|
|
;;
|
|
|
|
esac
|
|
|
|
|
|
|
|
if [ "$1" = "clean" ]; then
|
|
|
|
cleanup
|
|
|
|
exit 0
|
|
|
|
fi
|
|
|
|
|
kbuild: build init/built-in.a just once
Kbuild builds init/built-in.a twice; first during the ordinary
directory descending, second from scripts/link-vmlinux.sh.
We do this because UTS_VERSION contains the build version and the
timestamp. We cannot update it during the normal directory traversal
since we do not yet know if we need to update vmlinux. UTS_VERSION is
temporarily calculated, but omitted from the update check. Otherwise,
vmlinux would be rebuilt every time.
When Kbuild results in running link-vmlinux.sh, it increments the
version number in the .version file and takes the timestamp at that
time to really fix UTS_VERSION.
However, updating the same file twice is a footgun. To avoid nasty
timestamp issues, all build artifacts that depend on init/built-in.a
are atomically generated in link-vmlinux.sh, where some of them do not
need rebuilding.
To fix this issue, this commit changes as follows:
[1] Split UTS_VERSION out to include/generated/utsversion.h from
include/generated/compile.h
include/generated/utsversion.h is generated just before the
vmlinux link. It is generated under include/generated/ because
some decompressors (s390, x86) use UTS_VERSION.
[2] Split init_uts_ns and linux_banner out to init/version-timestamp.c
from init/version.c
init_uts_ns and linux_banner contain UTS_VERSION. During the ordinary
directory descending, they are compiled with __weak and used to
determine if vmlinux needs relinking. Just before the vmlinux link,
they are compiled without __weak to embed the real version and
timestamp.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2022-08-27 19:39:53 -07:00
|
|
|
${MAKE} -f "${srctree}/scripts/Makefile.build" obj=init init/version-timestamp.o
|
|
|
|
|
2024-06-10 04:25:16 -07:00
|
|
|
btf_vmlinux_bin_o=
|
|
|
|
kallsymso=
|
|
|
|
strip_debug=
|
|
|
|
|
2024-06-10 04:25:17 -07:00
|
|
|
if is_enabled CONFIG_KALLSYMS; then
|
2024-08-29 06:51:25 -07:00
|
|
|
true > .tmp_vmlinux.kallsyms0.syms
|
2024-08-07 11:03:00 -07:00
|
|
|
kallsyms .tmp_vmlinux.kallsyms0.syms .tmp_vmlinux0.kallsyms
|
2024-06-10 04:25:18 -07:00
|
|
|
fi
|
|
|
|
|
|
|
|
if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then
|
|
|
|
|
|
|
|
# The kallsyms linking does not need debug symbols, but the BTF does.
|
|
|
|
if ! is_enabled CONFIG_DEBUG_INFO_BTF; then
|
|
|
|
strip_debug=1
|
|
|
|
fi
|
|
|
|
|
|
|
|
vmlinux_link .tmp_vmlinux1
|
2024-06-10 04:25:17 -07:00
|
|
|
fi
|
|
|
|
|
2021-12-13 19:53:52 -07:00
|
|
|
if is_enabled CONFIG_DEBUG_INFO_BTF; then
|
2024-06-10 04:25:18 -07:00
|
|
|
if ! gen_btf .tmp_vmlinux1; then
|
2019-11-27 09:14:10 -07:00
|
|
|
echo >&2 "Failed to generate BTF for vmlinux"
|
|
|
|
echo >&2 "Try to disable CONFIG_DEBUG_INFO_BTF"
|
|
|
|
exit 1
|
btf: expose BTF info through sysfs
Make .BTF section allocated and expose its contents through sysfs.
/sys/kernel/btf directory is created to contain all the BTFs present
inside kernel. Currently there is only kernel's main BTF, represented as
/sys/kernel/btf/kernel file. Once kernel modules' BTFs are supported,
each module will expose its BTF as /sys/kernel/btf/<module-name> file.
Current approach relies on a few pieces coming together:
1. pahole is used to take almost final vmlinux image (modulo .BTF and
kallsyms) and generate .BTF section by converting DWARF info into
BTF. This section is not allocated and not mapped to any segment,
though, so is not yet accessible from inside kernel at runtime.
2. objcopy dumps .BTF contents into binary file and subsequently
convert binary file into linkable object file with automatically
generated symbols _binary__btf_kernel_bin_start and
_binary__btf_kernel_bin_end, pointing to start and end, respectively,
of BTF raw data.
3. final vmlinux image is generated by linking this object file (and
kallsyms, if necessary). sysfs_btf.c then creates
/sys/kernel/btf/kernel file and exposes embedded BTF contents through
it. This allows, e.g., libbpf and bpftool access BTF info at
well-known location, without resorting to searching for vmlinux image
on disk (location of which is not standardized and vmlinux image
might not be even available in some scenarios, e.g., inside qemu
during testing).
Alternative approach using .incbin assembler directive to embed BTF
contents directly was attempted but didn't work, because sysfs_proc.o is
not re-compiled during link-vmlinux.sh stage. This is required, though,
to update embedded BTF data (initially empty data is embedded, then
pahole generates BTF info and we need to regenerate sysfs_btf.o with
updated contents, but it's too late at that point).
If BTF couldn't be generated due to missing or too old pahole,
sysfs_btf.c handles that gracefully by detecting that
_binary__btf_kernel_bin_start (weak symbol) is 0 and not creating
/sys/kernel/btf at all.
v2->v3:
- added Documentation/ABI/testing/sysfs-kernel-btf (Greg K-H);
- created proper kobject (btf_kobj) for btf directory (Greg K-H);
- undo v2 change of reusing vmlinux, as it causes extra kallsyms pass
due to initially missing __binary__btf_kernel_bin_{start/end} symbols;
v1->v2:
- allow kallsyms stage to re-use vmlinux generated by gen_btf();
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-08-12 11:39:47 -07:00
|
|
|
fi
|
|
|
|
fi
|
|
|
|
|
2021-12-13 19:53:52 -07:00
|
|
|
if is_enabled CONFIG_KALLSYMS; then
|
2012-05-05 01:18:41 -07:00
|
|
|
|
|
|
|
# kallsyms support
|
|
|
|
# Generate section listing all symbols and add it into vmlinux
|
2024-06-10 04:25:17 -07:00
|
|
|
# It's a four step process:
|
|
|
|
# 0) Generate a dummy __kallsyms with empty symbol list.
|
2021-10-08 04:24:38 -07:00
|
|
|
# 1) Link .tmp_vmlinux.kallsyms1 so it has all symbols and sections,
|
2024-06-10 04:25:17 -07:00
|
|
|
# with a dummy __kallsyms.
|
2012-05-05 01:18:41 -07:00
|
|
|
# Running kallsyms on that gives us .tmp_kallsyms1.o with
|
|
|
|
# the right size
|
2021-10-08 04:24:38 -07:00
|
|
|
# 2) Link .tmp_vmlinux.kallsyms2 so it now has a __kallsyms section of
|
2012-05-05 01:18:41 -07:00
|
|
|
# the right size, but due to the added section, some
|
|
|
|
# addresses have shifted.
|
2021-10-08 04:24:38 -07:00
|
|
|
# From here, we generate a correct .tmp_vmlinux.kallsyms2.o
|
2016-11-23 09:41:37 -07:00
|
|
|
# 3) That link may have expanded the kernel image enough that
|
|
|
|
# more linker branch stubs / trampolines had to be added, which
|
|
|
|
# introduces new names, which further expands kallsyms. Do another
|
|
|
|
# pass if that is the case. In theory it's possible this results
|
|
|
|
# in even more stubs, but unlikely.
|
|
|
|
# KALLSYMS_EXTRA_PASS=1 may also used to debug or work around
|
|
|
|
# other bugs.
|
|
|
|
# 4) The correct ${kallsymso} is linked into the final vmlinux.
|
2012-05-05 01:18:41 -07:00
|
|
|
#
|
|
|
|
# a) Verify that the System.map from vmlinux matches the map from
|
|
|
|
# ${kallsymso}.
|
|
|
|
|
2024-06-10 04:25:16 -07:00
|
|
|
# The kallsyms linking does not need debug symbols included.
|
|
|
|
strip_debug=1
|
|
|
|
|
2024-06-10 04:25:18 -07:00
|
|
|
sysmap_and_kallsyms .tmp_vmlinux1
|
2024-06-10 04:25:16 -07:00
|
|
|
size1=$(${CONFIG_SHELL} "${srctree}/scripts/file-size.sh" ${kallsymso})
|
2012-05-05 01:18:41 -07:00
|
|
|
|
2024-06-10 04:25:18 -07:00
|
|
|
vmlinux_link .tmp_vmlinux2
|
|
|
|
sysmap_and_kallsyms .tmp_vmlinux2
|
2019-08-13 08:15:32 -07:00
|
|
|
size2=$(${CONFIG_SHELL} "${srctree}/scripts/file-size.sh" ${kallsymso})
|
2016-11-23 09:41:37 -07:00
|
|
|
|
|
|
|
if [ $size1 -ne $size2 ] || [ -n "${KALLSYMS_EXTRA_PASS}" ]; then
|
2024-06-10 04:25:18 -07:00
|
|
|
vmlinux_link .tmp_vmlinux3
|
|
|
|
sysmap_and_kallsyms .tmp_vmlinux3
|
2012-05-05 01:18:41 -07:00
|
|
|
fi
|
|
|
|
fi
|
|
|
|
|
2024-06-10 04:25:16 -07:00
|
|
|
strip_debug=
|
|
|
|
|
|
|
|
vmlinux_link vmlinux
|
kbuild: add ability to generate BTF type info for vmlinux
This patch adds new config option to trigger generation of BTF type
information from DWARF debuginfo for vmlinux and kernel modules through
pahole, which in turn relies on libbpf for btf_dedup() algorithm.
The intent is to record compact type information of all types used
inside kernel, including all the structs/unions/typedefs/etc. This
enables BPF's compile-once-run-everywhere ([0]) approach, in which
tracing programs that are inspecting kernel's internal data (e.g.,
struct task_struct) can be compiled on a system running some kernel
version, but would be possible to run on other kernel versions (and
configurations) without recompilation, even if the layout of structs
changed and/or some of the fields were added, removed, or renamed.
This is only possible if BPF loader can get kernel type info to adjust
all the offsets correctly. This patch is a first time in this direction,
making sure that BTF type info is part of Linux kernel image in
non-loadable ELF section.
BTF deduplication ([1]) algorithm typically provides 100x savings
compared to DWARF data, so resulting .BTF section is not big as is
typically about 2MB in size.
[0] http://vger.kernel.org/lpc-bpf2018.html#session-2
[1] https://facebookmicrosites.github.io/bpf/blog/2018/11/14/btf-enhancement.html
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Acked-by: David S. Miller <davem@davemloft.net>
Acked-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-04-02 09:49:50 -07:00
|
|
|
|
2020-07-11 14:53:24 -07:00
|
|
|
# fill in BTF IDs
|
2024-09-13 10:37:52 -07:00
|
|
|
if is_enabled CONFIG_DEBUG_INFO_BTF; then
|
2020-09-23 11:57:34 -07:00
|
|
|
info BTFIDS vmlinux
|
|
|
|
${RESOLVE_BTFIDS} vmlinux
|
2020-07-11 14:53:24 -07:00
|
|
|
fi
|
|
|
|
|
kbuild: avoid unneeded kallsyms step 3
Since commit 951bcae6c5a0 ("kallsyms: Avoid weak references for kallsyms
symbols"), the kallsyms step 3 always occurs.
You can compare the build logs.
[Before 951bcae6c5a0]
$ git checkout 951bcae6c5a0^
$ make defconfig all
[ snip ]
LD .tmp_vmlinux.kallsyms1
NM .tmp_vmlinux.kallsyms1.syms
KSYMS .tmp_vmlinux.kallsyms1.S
AS .tmp_vmlinux.kallsyms1.S
LD .tmp_vmlinux.kallsyms2
NM .tmp_vmlinux.kallsyms2.syms
KSYMS .tmp_vmlinux.kallsyms2.S
AS .tmp_vmlinux.kallsyms2.S
LD vmlinux
[After 951bcae6c5a0]
$ git checkout 951bcae6c5a0
$ make defconfig all
[ snip ]
LD .tmp_vmlinux.kallsyms1
NM .tmp_vmlinux.kallsyms1.syms
KSYMS .tmp_vmlinux.kallsyms1.S
AS .tmp_vmlinux.kallsyms1.S
LD .tmp_vmlinux.kallsyms2
NM .tmp_vmlinux.kallsyms2.syms
KSYMS .tmp_vmlinux.kallsyms2.S
AS .tmp_vmlinux.kallsyms2.S
LD .tmp_vmlinux.kallsyms3 # should not happen
NM .tmp_vmlinux.kallsyms3.syms # should not happen
KSYMS .tmp_vmlinux.kallsyms3.S # should not happen
AS .tmp_vmlinux.kallsyms3.S # should not happen
LD vmlinux
The resulting vmlinux is correct, but it always requires an additional
linking step.
The symbols produced by kallsyms are excluded from kallsyms itself
because they were previously missing in step 1. With those symbols
excluded, the symbol lists matched between step 1 and step 2,
eliminating the need for step 3. Now, this has a negative effect.
Since 951bcae6c5a0, the PROVIDE() directives provide the fallback
definitions, which are not trimmed from the sysbol list in step 1
because ${kallsymso_prev} is empty at this point.
In step 2, ${kallsymso_prev} is set, and the kallsyms_* symbols are
trimmed from the symbol list.
Due to the table size difference between step 1 and step 2 (the former
is larger due to the presence of kallsyms_*), step 3 is triggered.
Now that the kallsyms_* symbols are always linked, let's stop omitting
them from kallsyms. This avoids unnecessary step 3.
Fixes: 951bcae6c5a0 ("kallsyms: Avoid weak references for kallsyms symbols")
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2024-05-20 05:42:09 -07:00
|
|
|
mksysmap vmlinux System.map
|
2021-12-12 04:33:58 -07:00
|
|
|
|
2021-12-13 19:53:52 -07:00
|
|
|
if is_enabled CONFIG_BUILDTIME_TABLE_SORT; then
|
2019-12-03 17:46:31 -07:00
|
|
|
info SORTTAB vmlinux
|
2019-12-03 17:46:33 -07:00
|
|
|
if ! sorttable vmlinux; then
|
|
|
|
echo >&2 Failed to sort kernel tables
|
|
|
|
exit 1
|
|
|
|
fi
|
2012-05-28 10:32:28 -07:00
|
|
|
fi
|
|
|
|
|
2012-05-05 01:18:41 -07:00
|
|
|
# step a (see comment above)
|
2021-12-13 19:53:52 -07:00
|
|
|
if is_enabled CONFIG_KALLSYMS; then
|
2024-06-10 04:25:18 -07:00
|
|
|
if ! cmp -s System.map "${kallsyms_sysmap}"; then
|
2012-07-07 14:04:40 -07:00
|
|
|
echo >&2 Inconsistent kallsyms data
|
2023-03-06 14:32:53 -07:00
|
|
|
echo >&2 'Try "make KALLSYMS_EXTRA_PASS=1" as a workaround'
|
2012-05-05 01:18:41 -07:00
|
|
|
exit 1
|
|
|
|
fi
|
|
|
|
fi
|
2021-03-05 03:02:12 -07:00
|
|
|
|
|
|
|
# For fixdep
|
|
|
|
echo "vmlinux: $0" > .vmlinux.d
|