kevin/linux
1
Fork 0
Code

ftrace: make extra rcu_is_watching() validation check optional

Browse Source 
Introduce CONFIG_FTRACE_VALIDATE_RCU_IS_WATCHING config option to
control whether ftrace low-level code performs additional
rcu_is_watching()-based validation logic in an attempt to catch noinstr
violations.

This check is expected to never be true and is mostly useful for
low-level validation of ftrace subsystem invariants. For most users it
should probably be kept disabled to eliminate unnecessary runtime
overhead.

This improves BPF multi-kretprobe (relying on ftrace and rethook
infrastructure) runtime throughput by 2%, according to BPF benchmarks ([0]).

  [0] https://lore.kernel.org/bpf/CAEf4BzauQ2WKMjZdc9s0rBWa01BYbgwHN6aNDXQSHYia47pQ-w@mail.gmail.com/

Link: https://lore.kernel.org/all/20240418190909.704286-1-andrii@kernel.org/

Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Paul E. McKenney <paulmck@kernel.org>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
This commit is contained in:
Andrii Nakryiko 2024-04-18 12:09:08 -07:00 committed by Masami Hiramatsu (Google)
parent 0dc715295d
commit b0e28a4b5b
2 changed files with 14 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
include/linux/trace_recursion.h
View File

@ -135,7 +135,7 @@ extern void ftrace_record_recursion(unsigned long ip, unsigned long parent_ip);
# define do_ftrace_record_recursion(ip, pip) do { } while (0) # define do_ftrace_record_recursion(ip, pip) do { } while (0)
#endif #endif
#ifdef CONFIG_ARCH_WANTS_NO_INSTR #ifdef CONFIG_FTRACE_VALIDATE_RCU_IS_WATCHING
# define trace_warn_on_no_rcu(ip) \ # define trace_warn_on_no_rcu(ip) \
({ \ ({ \
bool __ret = !rcu_is_watching(); \ bool __ret = !rcu_is_watching(); \

13
kernel/trace/Kconfig
View File

@ -974,6 +974,19 @@ config FTRACE_RECORD_RECURSION_SIZE
This file can be reset, but the limit can not change in This file can be reset, but the limit can not change in
size at runtime. size at runtime.
config FTRACE_VALIDATE_RCU_IS_WATCHING
bool "Validate RCU is on during ftrace execution"
depends on FUNCTION_TRACER
depends on ARCH_WANTS_NO_INSTR
help
All callbacks that attach to the function tracing have some sort of
protection against recursion. This option is only to verify that
ftrace (and other users of ftrace_test_recursion_trylock()) are not
called outside of RCU, as if they are, it can cause a race. But it
also has a noticeable overhead when enabled.
If unsure, say N
config RING_BUFFER_RECORD_RECURSION config RING_BUFFER_RECORD_RECURSION
bool "Record functions that recurse in the ring buffer" bool "Record functions that recurse in the ring buffer"
depends on FTRACE_RECORD_RECURSION depends on FTRACE_RECORD_RECURSION