We are currently being somewhat inconsistent with the libbpf include paths,
which makes it difficult to move files from the kernel into an external
libbpf-using project without adjusting include paths.
Having the bpf/ subdir of $INCLUDEDIR in the include path has never been a
requirement for building against libbpf before, and indeed the libbpf pkg-config
file doesn't include it. So let's make all libbpf includes across the kernel
tree use the bpf/ prefix in their includes. Since bpftool skeleton generation
emits code with a libbpf include, this also ensures that those can be used in
existing external projects using the regular pkg-config include path.
This turns out to be a somewhat invasive change in the number of files touched;
however, the actual changes to files are fairly trivial (most of them are simply
made with 'sed'). The series is split to make the change for one tool subdir at
a time, while trying not to break the build along the way. It is structured like
this:
- Patch 1-3: Trivial fixes to Makefiles for issues I discovered while changing
the include paths.
- Patch 4-8: Change the include directives to use the bpf/ prefix, and updates
Makefiles to make sure tools/lib/ is part of the include path, but without
removing tools/lib/bpf
- Patch 9-11: Remove tools/lib/bpf from include paths to make sure we don't
inadvertently re-introduce includes without the bpf/ prefix.
Changelog:
v5:
- Combine the libbpf build rules in selftests Makefile (using Andrii's
suggestion for a make rule).
- Re-use self-tests libbpf build for runqslower (new patch 10)
- Formatting fixes
v4:
- Move runqslower error on missing BTF into make rule
- Make sure we don't always force a rebuild selftests
- Rebase on latest bpf-next (dropping patch 11)
v3:
- Don't add the kernel build dir to the runqslower Makefile, pass it in from
selftests instead.
- Use libbpf's 'make install_headers' in selftests instead of trying to
generate bpf_helper_defs.h in-place (to also work on read-only filesystems).
- Use a scratch builddir for both libbpf and bpftool when building in selftests.
- Revert bpf_helpers.h to quoted include instead of angled include with a bpf/
prefix.
- Fix a few style nits from Andrii
v2:
- Do a full cleanup of libbpf includes instead of just changing the
bpf_helper_defs.h include.
---
Toke Høiland-Jørgensen (11):
samples/bpf: Don't try to remove user's homedir on clean
tools/bpf/runqslower: Fix override option for VMLINUX_BTF
selftests: Pass VMLINUX_BTF to runqslower Makefile
tools/runqslower: Use consistent include paths for libbpf
selftests: Use consistent include paths for libbpf
bpftool: Use consistent include paths for libbpf
perf: Use consistent include paths for libbpf
samples/bpf: Use consistent include paths for libbpf
tools/runqslower: Remove tools/lib/bpf from include path
runsqslower: Support user-specified libbpf include and object paths
selftests: Refactor build to remove tools/lib/bpf from include path
samples/bpf/Makefile | 5 +-
samples/bpf/cpustat_kern.c | 2 -
samples/bpf/fds_example.c | 2 -
samples/bpf/hbm.c | 4 +
samples/bpf/hbm_kern.h | 4 +
samples/bpf/ibumad_kern.c | 2 -
samples/bpf/ibumad_user.c | 2 -
samples/bpf/lathist_kern.c | 2 -
samples/bpf/lwt_len_hist_kern.c | 2 -
samples/bpf/map_perf_test_kern.c | 4 +
samples/bpf/offwaketime_kern.c | 4 +
samples/bpf/offwaketime_user.c | 2 -
samples/bpf/parse_ldabs.c | 2 -
samples/bpf/parse_simple.c | 2 -
samples/bpf/parse_varlen.c | 2 -
samples/bpf/sampleip_kern.c | 4 +
samples/bpf/sampleip_user.c | 2 -
samples/bpf/sock_flags_kern.c | 2 -
samples/bpf/sockex1_kern.c | 2 -
samples/bpf/sockex1_user.c | 2 -
samples/bpf/sockex2_kern.c | 2 -
samples/bpf/sockex2_user.c | 2 -
samples/bpf/sockex3_kern.c | 2 -
samples/bpf/spintest_kern.c | 4 +
samples/bpf/spintest_user.c | 2 -
samples/bpf/syscall_tp_kern.c | 2 -
samples/bpf/task_fd_query_kern.c | 2 -
samples/bpf/task_fd_query_user.c | 2 -
samples/bpf/tc_l2_redirect_kern.c | 2 -
samples/bpf/tcbpf1_kern.c | 2 -
samples/bpf/tcp_basertt_kern.c | 4 +
samples/bpf/tcp_bufs_kern.c | 4 +
samples/bpf/tcp_clamp_kern.c | 4 +
samples/bpf/tcp_cong_kern.c | 4 +
samples/bpf/tcp_dumpstats_kern.c | 4 +
samples/bpf/tcp_iw_kern.c | 4 +
samples/bpf/tcp_rwnd_kern.c | 4 +
samples/bpf/tcp_synrto_kern.c | 4 +
samples/bpf/tcp_tos_reflect_kern.c | 4 +
samples/bpf/test_cgrp2_tc_kern.c | 2 -
samples/bpf/test_current_task_under_cgroup_kern.c | 2 -
samples/bpf/test_lwt_bpf.c | 2 -
samples/bpf/test_map_in_map_kern.c | 4 +
samples/bpf/test_overhead_kprobe_kern.c | 4 +
samples/bpf/test_overhead_raw_tp_kern.c | 2 -
samples/bpf/test_overhead_tp_kern.c | 2 -
samples/bpf/test_probe_write_user_kern.c | 4 +
samples/bpf/trace_event_kern.c | 4 +
samples/bpf/trace_event_user.c | 2 -
samples/bpf/trace_output_kern.c | 2 -
samples/bpf/trace_output_user.c | 2 -
samples/bpf/tracex1_kern.c | 4 +
samples/bpf/tracex2_kern.c | 4 +
samples/bpf/tracex3_kern.c | 4 +
samples/bpf/tracex4_kern.c | 4 +
samples/bpf/tracex5_kern.c | 4 +
samples/bpf/tracex6_kern.c | 2 -
samples/bpf/tracex7_kern.c | 2 -
samples/bpf/xdp1_kern.c | 2 -
samples/bpf/xdp1_user.c | 4 +
samples/bpf/xdp2_kern.c | 2 -
samples/bpf/xdp2skb_meta_kern.c | 2 -
samples/bpf/xdp_adjust_tail_kern.c | 2 -
samples/bpf/xdp_adjust_tail_user.c | 4 +
samples/bpf/xdp_fwd_kern.c | 2 -
samples/bpf/xdp_fwd_user.c | 2 -
samples/bpf/xdp_monitor_kern.c | 2 -
samples/bpf/xdp_redirect_cpu_kern.c | 2 -
samples/bpf/xdp_redirect_cpu_user.c | 2 -
samples/bpf/xdp_redirect_kern.c | 2 -
samples/bpf/xdp_redirect_map_kern.c | 2 -
samples/bpf/xdp_redirect_map_user.c | 2 -
samples/bpf/xdp_redirect_user.c | 2 -
samples/bpf/xdp_router_ipv4_kern.c | 2 -
samples/bpf/xdp_router_ipv4_user.c | 2 -
samples/bpf/xdp_rxq_info_kern.c | 2 -
samples/bpf/xdp_rxq_info_user.c | 4 +
samples/bpf/xdp_sample_pkts_kern.c | 2 -
samples/bpf/xdp_sample_pkts_user.c | 2 -
samples/bpf/xdp_tx_iptunnel_kern.c | 2 -
samples/bpf/xdp_tx_iptunnel_user.c | 2 -
samples/bpf/xdpsock_kern.c | 2 -
samples/bpf/xdpsock_user.c | 6 +-
tools/bpf/bpftool/Documentation/bpftool-gen.rst | 2 -
tools/bpf/bpftool/Makefile | 2 -
tools/bpf/bpftool/btf.c | 8 +-
tools/bpf/bpftool/btf_dumper.c | 2 -
tools/bpf/bpftool/cgroup.c | 2 -
tools/bpf/bpftool/common.c | 4 +
tools/bpf/bpftool/feature.c | 4 +
tools/bpf/bpftool/gen.c | 10 ++-
tools/bpf/bpftool/jit_disasm.c | 2 -
tools/bpf/bpftool/main.c | 4 +
tools/bpf/bpftool/map.c | 4 +
tools/bpf/bpftool/map_perf_ring.c | 4 +
tools/bpf/bpftool/net.c | 8 +-
tools/bpf/bpftool/netlink_dumper.c | 4 +
tools/bpf/bpftool/perf.c | 2 -
tools/bpf/bpftool/prog.c | 6 +-
tools/bpf/bpftool/xlated_dumper.c | 2 -
tools/bpf/runqslower/Makefile | 36 ++++++-----
tools/bpf/runqslower/runqslower.bpf.c | 2 -
tools/bpf/runqslower/runqslower.c | 4 +
tools/perf/examples/bpf/5sec.c | 2 -
tools/perf/examples/bpf/empty.c | 2 -
tools/perf/examples/bpf/sys_enter_openat.c | 2 -
tools/perf/include/bpf/pid_filter.h | 2 -
tools/perf/include/bpf/stdio.h | 2 -
tools/perf/include/bpf/unistd.h | 2 -
tools/testing/selftests/bpf/.gitignore | 4 -
tools/testing/selftests/bpf/Makefile | 65 ++++++++++----------
tools/testing/selftests/bpf/bpf_tcp_helpers.h | 4 +
tools/testing/selftests/bpf/bpf_trace_helpers.h | 2 -
tools/testing/selftests/bpf/bpf_util.h | 2 -
tools/testing/selftests/bpf/prog_tests/cpu_mask.c | 2 -
.../testing/selftests/bpf/prog_tests/perf_buffer.c | 2 -
tools/testing/selftests/bpf/progs/bpf_dctcp.c | 4 +
tools/testing/selftests/bpf/progs/bpf_flow.c | 4 +
tools/testing/selftests/bpf/progs/connect4_prog.c | 4 +
tools/testing/selftests/bpf/progs/connect6_prog.c | 4 +
tools/testing/selftests/bpf/progs/dev_cgroup.c | 2 -
tools/testing/selftests/bpf/progs/fentry_test.c | 2 -
tools/testing/selftests/bpf/progs/fexit_bpf2bpf.c | 2 -
.../selftests/bpf/progs/fexit_bpf2bpf_simple.c | 2 -
tools/testing/selftests/bpf/progs/fexit_test.c | 2 -
.../selftests/bpf/progs/get_cgroup_id_kern.c | 2 -
tools/testing/selftests/bpf/progs/kfree_skb.c | 4 +
tools/testing/selftests/bpf/progs/loop1.c | 4 +
tools/testing/selftests/bpf/progs/loop2.c | 4 +
tools/testing/selftests/bpf/progs/loop3.c | 4 +
tools/testing/selftests/bpf/progs/loop4.c | 2 -
tools/testing/selftests/bpf/progs/loop5.c | 2 -
tools/testing/selftests/bpf/progs/netcnt_prog.c | 2 -
tools/testing/selftests/bpf/progs/pyperf.h | 2 -
.../testing/selftests/bpf/progs/sample_map_ret0.c | 2 -
tools/testing/selftests/bpf/progs/sendmsg4_prog.c | 4 +
tools/testing/selftests/bpf/progs/sendmsg6_prog.c | 4 +
.../selftests/bpf/progs/socket_cookie_prog.c | 4 +
.../selftests/bpf/progs/sockmap_parse_prog.c | 4 +
.../selftests/bpf/progs/sockmap_tcp_msg_prog.c | 4 +
.../selftests/bpf/progs/sockmap_verdict_prog.c | 4 +
.../testing/selftests/bpf/progs/sockopt_inherit.c | 2 -
tools/testing/selftests/bpf/progs/sockopt_multi.c | 2 -
tools/testing/selftests/bpf/progs/sockopt_sk.c | 2 -
tools/testing/selftests/bpf/progs/strobemeta.h | 2 -
tools/testing/selftests/bpf/progs/tailcall1.c | 2 -
tools/testing/selftests/bpf/progs/tailcall2.c | 2 -
tools/testing/selftests/bpf/progs/tailcall3.c | 2 -
tools/testing/selftests/bpf/progs/tailcall4.c | 2 -
tools/testing/selftests/bpf/progs/tailcall5.c | 2 -
tools/testing/selftests/bpf/progs/tcp_rtt.c | 2 -
.../testing/selftests/bpf/progs/test_adjust_tail.c | 2 -
.../selftests/bpf/progs/test_attach_probe.c | 2 -
tools/testing/selftests/bpf/progs/test_btf_haskv.c | 2 -
tools/testing/selftests/bpf/progs/test_btf_newkv.c | 2 -
tools/testing/selftests/bpf/progs/test_btf_nokv.c | 2 -
.../testing/selftests/bpf/progs/test_core_extern.c | 2 -
.../selftests/bpf/progs/test_core_reloc_arrays.c | 4 +
.../bpf/progs/test_core_reloc_bitfields_direct.c | 4 +
.../bpf/progs/test_core_reloc_bitfields_probed.c | 4 +
.../bpf/progs/test_core_reloc_existence.c | 4 +
.../selftests/bpf/progs/test_core_reloc_flavors.c | 4 +
.../selftests/bpf/progs/test_core_reloc_ints.c | 4 +
.../selftests/bpf/progs/test_core_reloc_kernel.c | 4 +
.../selftests/bpf/progs/test_core_reloc_misc.c | 4 +
.../selftests/bpf/progs/test_core_reloc_mods.c | 4 +
.../selftests/bpf/progs/test_core_reloc_nesting.c | 4 +
.../bpf/progs/test_core_reloc_primitives.c | 4 +
.../bpf/progs/test_core_reloc_ptr_as_arr.c | 4 +
.../selftests/bpf/progs/test_core_reloc_size.c | 4 +
.../selftests/bpf/progs/test_get_stack_rawtp.c | 2 -
.../testing/selftests/bpf/progs/test_global_data.c | 2 -
.../selftests/bpf/progs/test_global_func1.c | 2 -
.../selftests/bpf/progs/test_global_func3.c | 2 -
.../selftests/bpf/progs/test_global_func5.c | 2 -
.../selftests/bpf/progs/test_global_func6.c | 2 -
.../selftests/bpf/progs/test_global_func7.c | 2 -
tools/testing/selftests/bpf/progs/test_l4lb.c | 4 +
.../selftests/bpf/progs/test_l4lb_noinline.c | 4 +
.../selftests/bpf/progs/test_lirc_mode2_kern.c | 2 -
.../selftests/bpf/progs/test_lwt_ip_encap.c | 4 +
.../selftests/bpf/progs/test_lwt_seg6local.c | 4 +
.../testing/selftests/bpf/progs/test_map_in_map.c | 2 -
tools/testing/selftests/bpf/progs/test_map_lock.c | 2 -
tools/testing/selftests/bpf/progs/test_mmap.c | 2 -
tools/testing/selftests/bpf/progs/test_obj_id.c | 2 -
tools/testing/selftests/bpf/progs/test_overhead.c | 4 +
.../testing/selftests/bpf/progs/test_perf_buffer.c | 2 -
tools/testing/selftests/bpf/progs/test_pinning.c | 2 -
.../selftests/bpf/progs/test_pinning_invalid.c | 2 -
.../testing/selftests/bpf/progs/test_pkt_access.c | 4 +
.../selftests/bpf/progs/test_pkt_md_access.c | 2 -
.../testing/selftests/bpf/progs/test_probe_user.c | 4 +
.../selftests/bpf/progs/test_queue_stack_map.h | 2 -
.../testing/selftests/bpf/progs/test_rdonly_maps.c | 2 -
tools/testing/selftests/bpf/progs/test_seg6_loop.c | 4 +
.../bpf/progs/test_select_reuseport_kern.c | 4 +
.../selftests/bpf/progs/test_send_signal_kern.c | 2 -
.../selftests/bpf/progs/test_sk_lookup_kern.c | 4 +
.../selftests/bpf/progs/test_skb_cgroup_id_kern.c | 2 -
tools/testing/selftests/bpf/progs/test_skb_ctx.c | 2 -
tools/testing/selftests/bpf/progs/test_skeleton.c | 2 -
.../selftests/bpf/progs/test_sock_fields_kern.c | 4 +
tools/testing/selftests/bpf/progs/test_spin_lock.c | 2 -
.../selftests/bpf/progs/test_stacktrace_build_id.c | 2 -
.../selftests/bpf/progs/test_stacktrace_map.c | 2 -
.../selftests/bpf/progs/test_sysctl_loop1.c | 2 -
.../selftests/bpf/progs/test_sysctl_loop2.c | 2 -
.../testing/selftests/bpf/progs/test_sysctl_prog.c | 2 -
tools/testing/selftests/bpf/progs/test_tc_edt.c | 4 +
tools/testing/selftests/bpf/progs/test_tc_tunnel.c | 4 +
.../bpf/progs/test_tcp_check_syncookie_kern.c | 4 +
.../testing/selftests/bpf/progs/test_tcp_estats.c | 2 -
.../testing/selftests/bpf/progs/test_tcpbpf_kern.c | 4 +
.../selftests/bpf/progs/test_tcpnotify_kern.c | 4 +
.../testing/selftests/bpf/progs/test_tracepoint.c | 2 -
.../testing/selftests/bpf/progs/test_tunnel_kern.c | 4 +
.../selftests/bpf/progs/test_verif_scale1.c | 2 -
.../selftests/bpf/progs/test_verif_scale2.c | 2 -
.../selftests/bpf/progs/test_verif_scale3.c | 2 -
tools/testing/selftests/bpf/progs/test_xdp.c | 4 +
.../testing/selftests/bpf/progs/test_xdp_bpf2bpf.c | 2 -
tools/testing/selftests/bpf/progs/test_xdp_loop.c | 4 +
tools/testing/selftests/bpf/progs/test_xdp_meta.c | 2 -
.../selftests/bpf/progs/test_xdp_noinline.c | 4 +
.../selftests/bpf/progs/test_xdp_redirect.c | 2 -
tools/testing/selftests/bpf/progs/test_xdp_vlan.c | 4 +
tools/testing/selftests/bpf/progs/xdp_dummy.c | 2 -
.../testing/selftests/bpf/progs/xdp_redirect_map.c | 2 -
tools/testing/selftests/bpf/progs/xdp_tx.c | 2 -
tools/testing/selftests/bpf/progs/xdping_kern.c | 4 +
tools/testing/selftests/bpf/test_cpp.cpp | 6 +-
tools/testing/selftests/bpf/test_hashmap.c | 2 -
tools/testing/selftests/bpf/test_progs.h | 2 -
tools/testing/selftests/bpf/test_sock.c | 2 -
tools/testing/selftests/bpf/test_sockmap_kern.h | 4 +
tools/testing/selftests/bpf/test_sysctl.c | 2 -
tools/testing/selftests/bpf/trace_helpers.h | 2 -
238 files changed, 392 insertions(+), 388 deletions(-)
From: Toke Høiland-Jørgensen <[email protected]>
Add a VMLINUX_BTF variable with the locally-built path when calling the
runqslower Makefile from selftests. This makes sure a simple 'make'
invocation in the selftests dir works even when there is no BTF information
for the running kernel. Do a wildcard expansion and include the same paths
for BTF for the running kernel as in the runqslower Makefile, to make it
possible to build selftests without having a vmlinux in the local tree.
Also fix the make invocation to use $(OUTPUT)/tools as the destination
directory instead of $(CURDIR)/tools.
Fixes: 3a0d3092a4ed ("selftests/bpf: Build runqslower from selftests")
Acked-by: Andrii Nakryiko <[email protected]>
Signed-off-by: Toke Høiland-Jørgensen <[email protected]>
---
tools/testing/selftests/bpf/Makefile | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index 246d09ffb296..8240282aef7f 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -124,10 +124,14 @@ $(OUTPUT)/test_stub.o: test_stub.c
$(call msg,CC,,$@)
$(CC) -c $(CFLAGS) -o $@ $<
+VMLINUX_BTF_PATHS := $(abspath ../../../../vmlinux) \
+ /sys/kernel/btf/vmlinux \
+ /boot/vmlinux-$(shell uname -r)
+VMLINUX_BTF:= $(firstword $(wildcard $(VMLINUX_BTF_PATHS)))
.PHONY: $(OUTPUT)/runqslower
$(OUTPUT)/runqslower: force
- $(Q)$(MAKE) $(submake_extras) -C $(TOOLSDIR)/bpf/runqslower \
- OUTPUT=$(CURDIR)/tools/
+ $(Q)$(MAKE) $(submake_extras) -C $(TOOLSDIR)/bpf/runqslower \
+ OUTPUT=$(OUTPUT)/tools/ VMLINUX_BTF=$(VMLINUX_BTF)
BPFOBJ := $(OUTPUT)/libbpf.a
From: Toke Høiland-Jørgensen <[email protected]>
The 'clean' rule in the samples/bpf Makefile tries to remove backup
files (ending in ~). However, if no such files exist, it will instead try
to remove the user's home directory. While the attempt is mostly harmless,
it does lead to a somewhat scary warning like this:
rm: cannot remove '~': Is a directory
Fix this by using find instead of shell expansion to locate any actual
backup files that need to be removed.
Fixes: b62a796c109c ("samples/bpf: allow make to be run from samples/bpf/ directory")
Acked-by: Jesper Dangaard Brouer <[email protected]>
Signed-off-by: Toke Høiland-Jørgensen <[email protected]>
---
samples/bpf/Makefile | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/samples/bpf/Makefile b/samples/bpf/Makefile
index 5b89c0370f33..f86d713a17a5 100644
--- a/samples/bpf/Makefile
+++ b/samples/bpf/Makefile
@@ -254,7 +254,7 @@ all:
clean:
$(MAKE) -C ../../ M=$(CURDIR) clean
- @rm -f *~
+ @find $(CURDIR) -type f -name '*~' -delete
$(LIBBPF): FORCE
# Fix up variables inherited from Kbuild that tools/ build system won't like
From: Toke Høiland-Jørgensen <[email protected]>
This adds support for specifying the libbpf include and object paths as
arguments to the runqslower Makefile, to support reusing the libbpf version
built as part of the selftests.
Signed-off-by: Toke Høiland-Jørgensen <[email protected]>
---
tools/bpf/runqslower/Makefile | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)
diff --git a/tools/bpf/runqslower/Makefile b/tools/bpf/runqslower/Makefile
index b90044caf270..faf5418609ea 100644
--- a/tools/bpf/runqslower/Makefile
+++ b/tools/bpf/runqslower/Makefile
@@ -6,7 +6,9 @@ LLVM_STRIP := llvm-strip
DEFAULT_BPFTOOL := $(OUTPUT)/sbin/bpftool
BPFTOOL ?= $(DEFAULT_BPFTOOL)
LIBBPF_SRC := $(abspath ../../lib/bpf)
-INCLUDES := -I$(OUTPUT) -I$(abspath ../../lib)
+BPFOBJ := $(OUTPUT)/libbpf.a
+BPF_INCLUDE := $(OUTPUT)
+INCLUDES := -I$(BPF_INCLUDE) -I$(OUTPUT) -I$(abspath ../../lib)
CFLAGS := -g -Wall
# Try to detect best kernel BTF source
@@ -37,7 +39,7 @@ clean:
$(call msg,CLEAN)
$(Q)rm -rf $(OUTPUT) runqslower
-$(OUTPUT)/runqslower: $(OUTPUT)/runqslower.o $(OUTPUT)/libbpf.a
+$(OUTPUT)/runqslower: $(OUTPUT)/runqslower.o $(BPFOBJ)
$(call msg,BINARY,$@)
$(Q)$(CC) $(CFLAGS) -lelf -lz $^ -o $@
@@ -50,7 +52,7 @@ $(OUTPUT)/%.skel.h: $(OUTPUT)/%.bpf.o | $(BPFTOOL)
$(call msg,GEN-SKEL,$@)
$(Q)$(BPFTOOL) gen skeleton $< > $@
-$(OUTPUT)/%.bpf.o: %.bpf.c $(OUTPUT)/libbpf.a | $(OUTPUT)
+$(OUTPUT)/%.bpf.o: %.bpf.c $(BPFOBJ) | $(OUTPUT)
$(call msg,BPF,$@)
$(Q)$(CLANG) -g -O2 -target bpf $(INCLUDES) \
-c $(filter %.c,$^) -o $@ && \
@@ -73,9 +75,9 @@ $(OUTPUT)/vmlinux.h: $(VMLINUX_BTF_PATH) | $(OUTPUT) $(BPFTOOL)
fi
$(Q)$(BPFTOOL) btf dump file $(VMLINUX_BTF_PATH) format c > $@
-$(OUTPUT)/libbpf.a: | $(OUTPUT)
+$(BPFOBJ): | $(OUTPUT)
$(Q)$(MAKE) $(submake_extras) -C $(LIBBPF_SRC) \
- OUTPUT=$(abs_out)/ $(abs_out)/libbpf.a
+ OUTPUT=$(abspath $(dir $@))/ $(abspath $@)
$(DEFAULT_BPFTOOL):
$(Q)$(MAKE) $(submake_extras) -C ../bpftool \
From: Toke Høiland-Jørgensen <[email protected]>
To make sure no new files are introduced that doesn't include the bpf/
prefix in its #include, remove tools/lib/bpf from the include path
entirely.
Instead, we introduce a new header files directory under the scratch tools/
dir, and add a rule to run the 'install_headers' rule from libbpf to have a
full set of consistent libbpf headers in $(OUTPUT)/tools/include/bpf, and
then use $(OUTPUT)/tools/include as the include path for selftests.
For consistency we also make sure we put all the scratch build files from
other bpftool and libbpf into tools/build/, so everything stays within
selftests/.
Signed-off-by: Toke Høiland-Jørgensen <[email protected]>
---
tools/testing/selftests/bpf/.gitignore | 4 +-
tools/testing/selftests/bpf/Makefile | 59 ++++++++++++++++----------------
2 files changed, 30 insertions(+), 33 deletions(-)
diff --git a/tools/testing/selftests/bpf/.gitignore b/tools/testing/selftests/bpf/.gitignore
index 1d14e3ab70be..ec464859c6b6 100644
--- a/tools/testing/selftests/bpf/.gitignore
+++ b/tools/testing/selftests/bpf/.gitignore
@@ -29,8 +29,6 @@ test_tcpnotify_user
test_libbpf
test_tcp_check_syncookie_user
test_sysctl
-libbpf.pc
-libbpf.so.*
test_hashmap
test_btf_dump
xdping
@@ -39,4 +37,4 @@ test_cpp
/no_alu32
/bpf_gcc
/tools
-bpf_helper_defs.h
+
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index 2cd91e0524cf..be8fe404a086 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -20,8 +20,8 @@ CLANG ?= clang
LLC ?= llc
LLVM_OBJCOPY ?= llvm-objcopy
BPF_GCC ?= $(shell command -v bpf-gcc;)
-CFLAGS += -g -Wall -O2 $(GENFLAGS) -I$(CURDIR) -I$(APIDIR) -I$(LIBDIR) \
- -I$(BPFDIR) -I$(GENDIR) -I$(TOOLSINCDIR) \
+CFLAGS += -g -Wall -O2 $(GENFLAGS) -I$(CURDIR) -I$(APIDIR) \
+ -I$(INCLUDE_DIR) -I$(GENDIR) -I$(LIBDIR) -I$(TOOLSINCDIR) \
-Dbpf_prog_load=bpf_prog_test_load \
-Dbpf_load_program=bpf_test_load_program
LDLIBS += -lcap -lelf -lz -lrt -lpthread
@@ -97,11 +97,15 @@ OVERRIDE_TARGETS := 1
override define CLEAN
$(call msg,CLEAN)
$(RM) -r $(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED) $(TEST_GEN_FILES) $(EXTRA_CLEAN)
- $(MAKE) -C $(BPFDIR) OUTPUT=$(OUTPUT)/ clean
endef
include ../lib.mk
+SCRATCH_DIR := $(OUTPUT)/tools
+BUILD_DIR := $(SCRATCH_DIR)/build
+INCLUDE_DIR := $(SCRATCH_DIR)/include
+BPFOBJ := $(BUILD_DIR)/libbpf/libbpf.a
+
# Define simple and short `make test_progs`, `make test_sysctl`, etc targets
# to build individual tests.
# NOTE: Semicolon at the end is critical to override lib.mk's default static
@@ -120,7 +124,7 @@ $(OUTPUT)/urandom_read: urandom_read.c
$(call msg,BINARY,,$@)
$(CC) -o $@ $< -Wl,--build-id
-$(OUTPUT)/test_stub.o: test_stub.c
+$(OUTPUT)/test_stub.o: test_stub.c $(BPFOBJ)
$(call msg,CC,,$@)
$(CC) -c $(CFLAGS) -o $@ $<
@@ -128,12 +132,10 @@ VMLINUX_BTF_PATHS := $(abspath ../../../../vmlinux) \
/sys/kernel/btf/vmlinux \
/boot/vmlinux-$(shell uname -r)
VMLINUX_BTF:= $(firstword $(wildcard $(VMLINUX_BTF_PATHS)))
-.PHONY: $(OUTPUT)/runqslower
-$(OUTPUT)/runqslower: force
+$(OUTPUT)/runqslower: $(BPFOBJ)
$(Q)$(MAKE) $(submake_extras) -C $(TOOLSDIR)/bpf/runqslower \
- OUTPUT=$(OUTPUT)/tools/ VMLINUX_BTF=$(VMLINUX_BTF)
-
-BPFOBJ := $(OUTPUT)/libbpf.a
+ OUTPUT=$(SCRATCH_DIR)/ VMLINUX_BTF=$(VMLINUX_BTF) \
+ BPFOBJ=$(BPFOBJ) BPF_INCLUDE=$(INCLUDE_DIR)
$(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED): $(OUTPUT)/test_stub.o $(BPFOBJ)
@@ -151,25 +153,22 @@ $(OUTPUT)/test_netcnt: cgroup_helpers.c
$(OUTPUT)/test_sock_fields: cgroup_helpers.c
$(OUTPUT)/test_sysctl: cgroup_helpers.c
-.PHONY: force
-
-# force a rebuild of BPFOBJ when its dependencies are updated
-force:
-
-DEFAULT_BPFTOOL := $(OUTPUT)/tools/sbin/bpftool
+DEFAULT_BPFTOOL := $(SCRATCH_DIR)/sbin/bpftool
BPFTOOL ?= $(DEFAULT_BPFTOOL)
+$(DEFAULT_BPFTOOL): $(BPFOBJ) | $(BUILD_DIR)/bpftool
+ $(Q)$(MAKE) $(submake_extras) -C $(BPFTOOLDIR) \
+ OUTPUT=$(BUILD_DIR)/bpftool/ \
+ prefix= DESTDIR=$(SCRATCH_DIR)/ install
-$(DEFAULT_BPFTOOL): force
- $(Q)$(MAKE) $(submake_extras) -C $(BPFTOOLDIR) \
- prefix= DESTDIR=$(OUTPUT)/tools/ install
-
-$(BPFOBJ): force
- $(Q)$(MAKE) $(submake_extras) -C $(BPFDIR) OUTPUT=$(OUTPUT)/
+$(BPFOBJ): $(wildcard $(BPFDIR)/*.c $(BPFDIR)/*.h $(BPFDIR)/Makefile) \
+ ../../../include/uapi/linux/bpf.h \
+ | $(INCLUDE_DIR) $(BUILD_DIR)/libbpf
+ $(Q)$(MAKE) $(submake_extras) -C $(BPFDIR) OUTPUT=$(BUILD_DIR)/libbpf/ \
+ DESTDIR=$(SCRATCH_DIR) prefix= all install_headers
-BPF_HELPERS := $(OUTPUT)/bpf_helper_defs.h $(wildcard $(BPFDIR)/bpf_*.h)
-$(OUTPUT)/bpf_helper_defs.h: $(BPFOBJ)
- $(Q)$(MAKE) $(submake_extras) -C $(BPFDIR) \
- OUTPUT=$(OUTPUT)/ $(OUTPUT)/bpf_helper_defs.h
+$(BUILD_DIR)/libbpf $(BUILD_DIR)/bpftool $(INCLUDE_DIR):
+ $(call msg,MKDIR,,$@)
+ mkdir -p $@
# Get Clang's default includes on this system, as opposed to those seen by
# '-target bpf'. This fixes "missing" files on some architectures/distros,
@@ -189,8 +188,8 @@ MENDIAN=$(if $(IS_LITTLE_ENDIAN),-mlittle-endian,-mbig-endian)
CLANG_SYS_INCLUDES = $(call get_sys_includes,$(CLANG))
BPF_CFLAGS = -g -D__TARGET_ARCH_$(SRCARCH) $(MENDIAN) \
- -I$(OUTPUT) -I$(CURDIR) -I$(CURDIR)/include/uapi \
- -I$(APIDIR) -I$(LIBDIR) -I$(BPFDIR) -I$(abspath $(OUTPUT)/../usr/include)
+ -I$(INCLUDE_DIR) -I$(CURDIR) -I$(CURDIR)/include/uapi \
+ -I$(APIDIR) -I$(abspath $(OUTPUT)/../usr/include)
CLANG_CFLAGS = $(CLANG_SYS_INCLUDES) \
-Wno-compare-distinct-pointer-types
@@ -279,7 +278,7 @@ $(TRUNNER_BPF_PROGS_DIR)$(if $2,-)$2-bpfobjs := y
$(TRUNNER_BPF_OBJS): $(TRUNNER_OUTPUT)/%.o: \
$(TRUNNER_BPF_PROGS_DIR)/%.c \
$(TRUNNER_BPF_PROGS_DIR)/*.h \
- $$(BPF_HELPERS) | $(TRUNNER_OUTPUT)
+ $$(BPFOBJ) | $(TRUNNER_OUTPUT)
$$(call $(TRUNNER_BPF_BUILD_RULE),$$<,$$@, \
$(TRUNNER_BPF_CFLAGS), \
$(TRUNNER_BPF_LDFLAGS))
@@ -392,7 +391,7 @@ $(OUTPUT)/test_cpp: test_cpp.cpp $(OUTPUT)/test_core_extern.skel.h $(BPFOBJ)
$(call msg,CXX,,$@)
$(CXX) $(CFLAGS) $^ $(LDLIBS) -o $@
-EXTRA_CLEAN := $(TEST_CUSTOM_PROGS) \
+EXTRA_CLEAN := $(TEST_CUSTOM_PROGS) $(SCRATCH_DIR) \
prog_tests/tests.h map_tests/tests.h verifier/tests.h \
feature \
- $(addprefix $(OUTPUT)/,*.o *.skel.h no_alu32 bpf_gcc tools)
+ $(addprefix $(OUTPUT)/,*.o *.skel.h no_alu32 bpf_gcc)
On Mon, Jan 20, 2020 at 5:08 AM Toke Høiland-Jørgensen <[email protected]> wrote:
>
> We are currently being somewhat inconsistent with the libbpf include paths,
> which makes it difficult to move files from the kernel into an external
> libbpf-using project without adjusting include paths.
>
> Having the bpf/ subdir of $INCLUDEDIR in the include path has never been a
> requirement for building against libbpf before, and indeed the libbpf pkg-config
> file doesn't include it. So let's make all libbpf includes across the kernel
> tree use the bpf/ prefix in their includes. Since bpftool skeleton generation
> emits code with a libbpf include, this also ensures that those can be used in
> existing external projects using the regular pkg-config include path.
>
> This turns out to be a somewhat invasive change in the number of files touched;
> however, the actual changes to files are fairly trivial (most of them are simply
> made with 'sed'). The series is split to make the change for one tool subdir at
> a time, while trying not to break the build along the way. It is structured like
> this:
>
> - Patch 1-3: Trivial fixes to Makefiles for issues I discovered while changing
> the include paths.
>
> - Patch 4-8: Change the include directives to use the bpf/ prefix, and updates
> Makefiles to make sure tools/lib/ is part of the include path, but without
> removing tools/lib/bpf
>
> - Patch 9-11: Remove tools/lib/bpf from include paths to make sure we don't
> inadvertently re-introduce includes without the bpf/ prefix.
>
> Changelog:
>
> v5:
> - Combine the libbpf build rules in selftests Makefile (using Andrii's
> suggestion for a make rule).
> - Re-use self-tests libbpf build for runqslower (new patch 10)
> - Formatting fixes
>
> v4:
> - Move runqslower error on missing BTF into make rule
> - Make sure we don't always force a rebuild selftests
> - Rebase on latest bpf-next (dropping patch 11)
>
> v3:
> - Don't add the kernel build dir to the runqslower Makefile, pass it in from
> selftests instead.
> - Use libbpf's 'make install_headers' in selftests instead of trying to
> generate bpf_helper_defs.h in-place (to also work on read-only filesystems).
> - Use a scratch builddir for both libbpf and bpftool when building in selftests.
> - Revert bpf_helpers.h to quoted include instead of angled include with a bpf/
> prefix.
> - Fix a few style nits from Andrii
>
> v2:
> - Do a full cleanup of libbpf includes instead of just changing the
> bpf_helper_defs.h include.
>
> ---
>
Looks good, it's a clear improvement on what we had before, thanks!
It doesn't re-build bpftool when bpftool sources changes, but I think
it was like that even before, so no need to block on that. Would be
nice to have a follow up fixing that, though. $(wildcard
$(BPFTOOL_DIR)/*.[ch] $(BPFTOOL_DIR)/Makefile) should do it, same as
for libbpf.
So, for the series:
Acked-by: Andrii Nakryiko <[email protected]>
Tested-by: Andrii Nakryiko <[email protected]>
> Toke Høiland-Jørgensen (11):
> samples/bpf: Don't try to remove user's homedir on clean
> tools/bpf/runqslower: Fix override option for VMLINUX_BTF
> selftests: Pass VMLINUX_BTF to runqslower Makefile
> tools/runqslower: Use consistent include paths for libbpf
> selftests: Use consistent include paths for libbpf
> bpftool: Use consistent include paths for libbpf
> perf: Use consistent include paths for libbpf
> samples/bpf: Use consistent include paths for libbpf
> tools/runqslower: Remove tools/lib/bpf from include path
> runsqslower: Support user-specified libbpf include and object paths
> selftests: Refactor build to remove tools/lib/bpf from include path
>
>
[...]
On Mon, Jan 20, 2020 at 2:21 PM Andrii Nakryiko
<[email protected]> wrote:
>
> On Mon, Jan 20, 2020 at 5:08 AM Toke Høiland-Jørgensen <[email protected]> wrote:
> >
> > We are currently being somewhat inconsistent with the libbpf include paths,
> > which makes it difficult to move files from the kernel into an external
> > libbpf-using project without adjusting include paths.
> >
> > Having the bpf/ subdir of $INCLUDEDIR in the include path has never been a
> > requirement for building against libbpf before, and indeed the libbpf pkg-config
> > file doesn't include it. So let's make all libbpf includes across the kernel
> > tree use the bpf/ prefix in their includes. Since bpftool skeleton generation
> > emits code with a libbpf include, this also ensures that those can be used in
> > existing external projects using the regular pkg-config include path.
> >
> > This turns out to be a somewhat invasive change in the number of files touched;
> > however, the actual changes to files are fairly trivial (most of them are simply
> > made with 'sed'). The series is split to make the change for one tool subdir at
> > a time, while trying not to break the build along the way. It is structured like
> > this:
> >
> > - Patch 1-3: Trivial fixes to Makefiles for issues I discovered while changing
> > the include paths.
> >
> > - Patch 4-8: Change the include directives to use the bpf/ prefix, and updates
> > Makefiles to make sure tools/lib/ is part of the include path, but without
> > removing tools/lib/bpf
> >
> > - Patch 9-11: Remove tools/lib/bpf from include paths to make sure we don't
> > inadvertently re-introduce includes without the bpf/ prefix.
> >
> > Changelog:
> >
> > v5:
> > - Combine the libbpf build rules in selftests Makefile (using Andrii's
> > suggestion for a make rule).
> > - Re-use self-tests libbpf build for runqslower (new patch 10)
> > - Formatting fixes
> >
> > v4:
> > - Move runqslower error on missing BTF into make rule
> > - Make sure we don't always force a rebuild selftests
> > - Rebase on latest bpf-next (dropping patch 11)
> >
> > v3:
> > - Don't add the kernel build dir to the runqslower Makefile, pass it in from
> > selftests instead.
> > - Use libbpf's 'make install_headers' in selftests instead of trying to
> > generate bpf_helper_defs.h in-place (to also work on read-only filesystems).
> > - Use a scratch builddir for both libbpf and bpftool when building in selftests.
> > - Revert bpf_helpers.h to quoted include instead of angled include with a bpf/
> > prefix.
> > - Fix a few style nits from Andrii
> >
> > v2:
> > - Do a full cleanup of libbpf includes instead of just changing the
> > bpf_helper_defs.h include.
> >
> > ---
> >
>
> Looks good, it's a clear improvement on what we had before, thanks!
>
> It doesn't re-build bpftool when bpftool sources changes, but I think
> it was like that even before, so no need to block on that. Would be
> nice to have a follow up fixing that, though. $(wildcard
> $(BPFTOOL_DIR)/*.[ch] $(BPFTOOL_DIR)/Makefile) should do it, same as
> for libbpf.
>
> So, for the series:
>
> Acked-by: Andrii Nakryiko <[email protected]>
> Tested-by: Andrii Nakryiko <[email protected]>
Applied. Thanks
Andrii Nakryiko <[email protected]> writes:
> On Mon, Jan 20, 2020 at 5:08 AM Toke Høiland-Jørgensen <[email protected]> wrote:
>>
>> We are currently being somewhat inconsistent with the libbpf include paths,
>> which makes it difficult to move files from the kernel into an external
>> libbpf-using project without adjusting include paths.
>>
>> Having the bpf/ subdir of $INCLUDEDIR in the include path has never been a
>> requirement for building against libbpf before, and indeed the libbpf pkg-config
>> file doesn't include it. So let's make all libbpf includes across the kernel
>> tree use the bpf/ prefix in their includes. Since bpftool skeleton generation
>> emits code with a libbpf include, this also ensures that those can be used in
>> existing external projects using the regular pkg-config include path.
>>
>> This turns out to be a somewhat invasive change in the number of files touched;
>> however, the actual changes to files are fairly trivial (most of them are simply
>> made with 'sed'). The series is split to make the change for one tool subdir at
>> a time, while trying not to break the build along the way. It is structured like
>> this:
>>
>> - Patch 1-3: Trivial fixes to Makefiles for issues I discovered while changing
>> the include paths.
>>
>> - Patch 4-8: Change the include directives to use the bpf/ prefix, and updates
>> Makefiles to make sure tools/lib/ is part of the include path, but without
>> removing tools/lib/bpf
>>
>> - Patch 9-11: Remove tools/lib/bpf from include paths to make sure we don't
>> inadvertently re-introduce includes without the bpf/ prefix.
>>
>> Changelog:
>>
>> v5:
>> - Combine the libbpf build rules in selftests Makefile (using Andrii's
>> suggestion for a make rule).
>> - Re-use self-tests libbpf build for runqslower (new patch 10)
>> - Formatting fixes
>>
>> v4:
>> - Move runqslower error on missing BTF into make rule
>> - Make sure we don't always force a rebuild selftests
>> - Rebase on latest bpf-next (dropping patch 11)
>>
>> v3:
>> - Don't add the kernel build dir to the runqslower Makefile, pass it in from
>> selftests instead.
>> - Use libbpf's 'make install_headers' in selftests instead of trying to
>> generate bpf_helper_defs.h in-place (to also work on read-only filesystems).
>> - Use a scratch builddir for both libbpf and bpftool when building in selftests.
>> - Revert bpf_helpers.h to quoted include instead of angled include with a bpf/
>> prefix.
>> - Fix a few style nits from Andrii
>>
>> v2:
>> - Do a full cleanup of libbpf includes instead of just changing the
>> bpf_helper_defs.h include.
>>
>> ---
>>
>
> Looks good, it's a clear improvement on what we had before, thanks!
>
> It doesn't re-build bpftool when bpftool sources changes, but I think
> it was like that even before, so no need to block on that. Would be
> nice to have a follow up fixing that, though. $(wildcard
> $(BPFTOOL_DIR)/*.[ch] $(BPFTOOL_DIR)/Makefile) should do it, same as
> for libbpf.
Yeah, I did realise there was some potential for improvement for bpftool
as well, but I got enough of Makefiles for now :)
I'll see if I can't circle back to this at some point...
> So, for the series:
>
> Acked-by: Andrii Nakryiko <[email protected]>
> Tested-by: Andrii Nakryiko <[email protected]>
Thanks!
-Toke