OSDN Git Service

bpf: Convert BPF_DISPATCHER to use static_call() (not ftrace)
authorPeter Zijlstra <peterz@infradead.org>
Thu, 3 Nov 2022 12:00:14 +0000 (13:00 +0100)
committerDaniel Borkmann <daniel@iogearbox.net>
Fri, 4 Nov 2022 22:13:57 +0000 (23:13 +0100)
commitc86df29d11dfba27c0a1f5039cd6fe387fbf4239
tree3e3bde30f24960b1459df5114c9214336af0f71a
parent18acb7fac22ff7b36c7ea5a76b12996e7b7dbaba
bpf: Convert BPF_DISPATCHER to use static_call() (not ftrace)

The dispatcher function is currently abusing the ftrace __fentry__
call location for its own purposes -- this obviously gives trouble
when the dispatcher and ftrace are both in use.

A previous solution tried using __attribute__((patchable_function_entry()))
which works, except it is GCC-8+ only, breaking the build on the
earlier still supported compilers. Instead use static_call() -- which
has its own annotations and does not conflict with ftrace -- to
rewrite the dispatch function.

By using: return static_call()(ctx, insni, bpf_func) you get a perfect
forwarding tail call as function body (iow a single jmp instruction).
By having the default static_call() target be bpf_dispatcher_nop_func()
it retains the default behaviour (an indirect call to the argument
function). Only once a dispatcher program is attached is the target
rewritten to directly call the JIT'ed image.

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Tested-by: Björn Töpel <bjorn@kernel.org>
Tested-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Björn Töpel <bjorn@kernel.org>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Link: https://lkml.kernel.org/r/Y1/oBlK0yFk5c/Im@hirez.programming.kicks-ass.net
Link: https://lore.kernel.org/bpf/20221103120647.796772565@infradead.org
include/linux/bpf.h
kernel/bpf/dispatcher.c