LinuxLists.cc - [PATCH] libbpf: Fix build on read-only filesystems

2019-12-20 03:27:32

by Namhyung Kim

[permalink] [raw]

Subject: [PATCH] libbpf: Fix build on read-only filesystems

2019-12-20 20:30:39

by Andrii Nakryiko

[permalink] [raw]

Subject: Re: [PATCH] libbpf: Fix build on read-only filesystems

2019-12-20 20:50:16

by Arnaldo Carvalho de Melo

[permalink] [raw]

Subject: Re: [PATCH] libbpf: Fix build on read-only filesystems

Em Fri, Dec 20, 2019 at 12:29:36PM -0800, Andrii Nakryiko escreveu:
> On Thu, Dec 19, 2019 at 7:26 PM Namhyung Kim <[email protected]> wrote:
> > I got the following error when I tried to build perf on a read-only
> > filesystem with O=dir option.
> >
> > $ cd /some/where/ro/linux/tools/perf
> > $ make O=$HOME/build/perf
> > ...
> > CC /home/namhyung/build/perf/lib.o
> > /bin/sh: bpf_helper_defs.h: Read-only file system
> > make[3]: *** [Makefile:184: bpf_helper_defs.h] Error 1
> > make[2]: *** [Makefile.perf:778: /home/namhyung/build/perf/libbpf.a] Error 2
> > make[2]: *** Waiting for unfinished jobs....
> > LD /home/namhyung/build/perf/libperf-in.o
> > AR /home/namhyung/build/perf/libperf.a
> > PERF_VERSION = 5.4.0
> > make[1]: *** [Makefile.perf:225: sub-make] Error 2
> > make: *** [Makefile:70: all] Error 2
> >
> > It was becaused bpf_helper_defs.h was generated in current directory.
> > Move it to OUTPUT directory.
> >
> > Signed-off-by: Namhyung Kim <[email protected]>
> > ---
>
> Overall nothing is obviously broken, except you need to fix up
> selftests/bpf's Makefile as well.
>
> BTW, this patch doesn't apply cleanly to latest bpf-next, so please rebase.
>
> Also subject prefix should look like [PATCH bpf-next] if it's meant to
> be applied against bpf-next.

Shouldn't this be applied to the current merge window since a behaviour
that people relied, i.e. using O= to generate the build in a separate
directory, since its not possible to use the source dir tree as it is
read-only is now broken, i.e. isn't this a regression?

- Arnaldo

> > tools/lib/bpf/Makefile | 14 +++++++-------
> > 1 file changed, 7 insertions(+), 7 deletions(-)
> >
>
> [...]

--

- Arnaldo

2019-12-20 21:47:02

by Andrii Nakryiko

[permalink] [raw]

Subject: Re: [PATCH] libbpf: Fix build on read-only filesystems

On Fri, Dec 20, 2019 at 12:47 PM Arnaldo Carvalho de Melo
<[email protected]> wrote:
>
> Em Fri, Dec 20, 2019 at 12:29:36PM -0800, Andrii Nakryiko escreveu:
> > On Thu, Dec 19, 2019 at 7:26 PM Namhyung Kim <[email protected]> wrote:
> > > I got the following error when I tried to build perf on a read-only
> > > filesystem with O=dir option.
> > >
> > > $ cd /some/where/ro/linux/tools/perf
> > > $ make O=$HOME/build/perf
> > > ...
> > > CC /home/namhyung/build/perf/lib.o
> > > /bin/sh: bpf_helper_defs.h: Read-only file system
> > > make[3]: *** [Makefile:184: bpf_helper_defs.h] Error 1
> > > make[2]: *** [Makefile.perf:778: /home/namhyung/build/perf/libbpf.a] Error 2
> > > make[2]: *** Waiting for unfinished jobs....
> > > LD /home/namhyung/build/perf/libperf-in.o
> > > AR /home/namhyung/build/perf/libperf.a
> > > PERF_VERSION = 5.4.0
> > > make[1]: *** [Makefile.perf:225: sub-make] Error 2
> > > make: *** [Makefile:70: all] Error 2
> > >
> > > It was becaused bpf_helper_defs.h was generated in current directory.
> > > Move it to OUTPUT directory.
> > >
> > > Signed-off-by: Namhyung Kim <[email protected]>
> > > ---
> >
> > Overall nothing is obviously broken, except you need to fix up
> > selftests/bpf's Makefile as well.
> >
> > BTW, this patch doesn't apply cleanly to latest bpf-next, so please rebase.
> >
> > Also subject prefix should look like [PATCH bpf-next] if it's meant to
> > be applied against bpf-next.
>
> Shouldn't this be applied to the current merge window since a behaviour
> that people relied, i.e. using O= to generate the build in a separate
> directory, since its not possible to use the source dir tree as it is
> read-only is now broken, i.e. isn't this a regression?

Sure, it can be applied against bpf as well, but selftests still need
to be fixed first.

>
> - Arnaldo
>
> > > tools/lib/bpf/Makefile | 14 +++++++-------
> > > 1 file changed, 7 insertions(+), 7 deletions(-)
> > >
> >
> > [...]
>
> --
>
> - Arnaldo

2019-12-20 21:54:31

by Arnaldo Carvalho de Melo

[permalink] [raw]

Subject: Re: [PATCH] libbpf: Fix build on read-only filesystems

Em Fri, Dec 20, 2019 at 01:45:52PM -0800, Andrii Nakryiko escreveu:
> On Fri, Dec 20, 2019 at 12:47 PM Arnaldo Carvalho de Melo <[email protected]> wrote:
> > Em Fri, Dec 20, 2019 at 12:29:36PM -0800, Andrii Nakryiko escreveu:
> > > On Thu, Dec 19, 2019 at 7:26 PM Namhyung Kim <[email protected]> wrote:
> > > > I got the following error when I tried to build perf on a read-only
> > > > filesystem with O=dir option.

> > > > $ cd /some/where/ro/linux/tools/perf
> > > > $ make O=$HOME/build/perf
> > > > ...
> > > > CC /home/namhyung/build/perf/lib.o
> > > > /bin/sh: bpf_helper_defs.h: Read-only file system
> > > > make[3]: *** [Makefile:184: bpf_helper_defs.h] Error 1
> > > > make[2]: *** [Makefile.perf:778: /home/namhyung/build/perf/libbpf.a] Error 2
> > > > make[2]: *** Waiting for unfinished jobs....
> > > > LD /home/namhyung/build/perf/libperf-in.o
> > > > AR /home/namhyung/build/perf/libperf.a
> > > > PERF_VERSION = 5.4.0
> > > > make[1]: *** [Makefile.perf:225: sub-make] Error 2
> > > > make: *** [Makefile:70: all] Error 2

> > > > It was becaused bpf_helper_defs.h was generated in current directory.
> > > > Move it to OUTPUT directory.

> > > > Signed-off-by: Namhyung Kim <[email protected]>
> > > > ---

> > > Overall nothing is obviously broken, except you need to fix up
> > > selftests/bpf's Makefile as well.

> > > BTW, this patch doesn't apply cleanly to latest bpf-next, so please rebase.

> > > Also subject prefix should look like [PATCH bpf-next] if it's meant to
> > > be applied against bpf-next.

> > Shouldn't this be applied to the current merge window since a behaviour
> > that people relied, i.e. using O= to generate the build in a separate
> > directory, since its not possible to use the source dir tree as it is
> > read-only is now broken, i.e. isn't this a regression?

> Sure, it can be applied against bpf as well, but selftests still need
> to be fixed first.

I guess this can be done on a separate patch? I.e. if the user doesn't
use selftests the only regression it will see is when trying to build
tools/perf using O=.

I think two patches is best, better granularity, do you see a strict
need for both to be in the same patch?

- Arnaldo

2019-12-20 22:01:50

by Andrii Nakryiko

[permalink] [raw]

Subject: Re: [PATCH] libbpf: Fix build on read-only filesystems

On Fri, Dec 20, 2019 at 1:53 PM Arnaldo Carvalho de Melo
<[email protected]> wrote:
>
> Em Fri, Dec 20, 2019 at 01:45:52PM -0800, Andrii Nakryiko escreveu:
> > On Fri, Dec 20, 2019 at 12:47 PM Arnaldo Carvalho de Melo <[email protected]> wrote:
> > > Em Fri, Dec 20, 2019 at 12:29:36PM -0800, Andrii Nakryiko escreveu:
> > > > On Thu, Dec 19, 2019 at 7:26 PM Namhyung Kim <[email protected]> wrote:
> > > > > I got the following error when I tried to build perf on a read-only
> > > > > filesystem with O=dir option.
>
> > > > > $ cd /some/where/ro/linux/tools/perf
> > > > > $ make O=$HOME/build/perf
> > > > > ...
> > > > > CC /home/namhyung/build/perf/lib.o
> > > > > /bin/sh: bpf_helper_defs.h: Read-only file system
> > > > > make[3]: *** [Makefile:184: bpf_helper_defs.h] Error 1
> > > > > make[2]: *** [Makefile.perf:778: /home/namhyung/build/perf/libbpf.a] Error 2
> > > > > make[2]: *** Waiting for unfinished jobs....
> > > > > LD /home/namhyung/build/perf/libperf-in.o
> > > > > AR /home/namhyung/build/perf/libperf.a
> > > > > PERF_VERSION = 5.4.0
> > > > > make[1]: *** [Makefile.perf:225: sub-make] Error 2
> > > > > make: *** [Makefile:70: all] Error 2
>
> > > > > It was becaused bpf_helper_defs.h was generated in current directory.
> > > > > Move it to OUTPUT directory.
>
> > > > > Signed-off-by: Namhyung Kim <[email protected]>
> > > > > ---
>
> > > > Overall nothing is obviously broken, except you need to fix up
> > > > selftests/bpf's Makefile as well.
>
> > > > BTW, this patch doesn't apply cleanly to latest bpf-next, so please rebase.
>
> > > > Also subject prefix should look like [PATCH bpf-next] if it's meant to
> > > > be applied against bpf-next.
>
> > > Shouldn't this be applied to the current merge window since a behaviour
> > > that people relied, i.e. using O= to generate the build in a separate
> > > directory, since its not possible to use the source dir tree as it is
> > > read-only is now broken, i.e. isn't this a regression?
>
> > Sure, it can be applied against bpf as well, but selftests still need
> > to be fixed first.
>
> I guess this can be done on a separate patch? I.e. if the user doesn't
> use selftests the only regression it will see is when trying to build
> tools/perf using O=.
>
> I think two patches is best, better granularity, do you see a strict
> need for both to be in the same patch?

Sure, it can be two separate patches, but they should go in together,
otherwise selftests will be broken.

>
> - Arnaldo

2019-12-20 22:03:01

by Andrii Nakryiko

[permalink] [raw]

Subject: Re: [PATCH] libbpf: Fix build on read-only filesystems

On Thu, Dec 19, 2019 at 7:26 PM Namhyung Kim <[email protected]> wrote:
>
> I got the following error when I tried to build perf on a read-only
> filesystem with O=dir option.
>
> $ cd /some/where/ro/linux/tools/perf
> $ make O=$HOME/build/perf
> ...
> CC /home/namhyung/build/perf/lib.o
> /bin/sh: bpf_helper_defs.h: Read-only file system
> make[3]: *** [Makefile:184: bpf_helper_defs.h] Error 1
> make[2]: *** [Makefile.perf:778: /home/namhyung/build/perf/libbpf.a] Error 2
> make[2]: *** Waiting for unfinished jobs....
> LD /home/namhyung/build/perf/libperf-in.o
> AR /home/namhyung/build/perf/libperf.a
> PERF_VERSION = 5.4.0
> make[1]: *** [Makefile.perf:225: sub-make] Error 2
> make: *** [Makefile:70: all] Error 2
>
> It was becaused bpf_helper_defs.h was generated in current directory.
> Move it to OUTPUT directory.
>
> Signed-off-by: Namhyung Kim <[email protected]>
> ---
> tools/lib/bpf/Makefile | 14 +++++++-------
> 1 file changed, 7 insertions(+), 7 deletions(-)
>
> diff --git a/tools/lib/bpf/Makefile b/tools/lib/bpf/Makefile
> index 99425d0be6ff..2f42a35f4634 100644
> --- a/tools/lib/bpf/Makefile
> +++ b/tools/lib/bpf/Makefile
> @@ -159,7 +159,7 @@ all: fixdep
>
> all_cmd: $(CMD_TARGETS) check
>
> -$(BPF_IN_SHARED): force elfdep bpfdep bpf_helper_defs.h
> +$(BPF_IN_SHARED): force elfdep bpfdep $(OUTPUT)bpf_helper_defs.h

btw, there is a lot of $(OUTPUT)bpf_helper_defs.h repetition, could
you please extract it into a variable, similar to BPF_IN_SHARED and
others? E.g., just BPF_HELPER_DEFS would work. Thanks!

> @(test -f ../../include/uapi/linux/bpf.h -a -f ../../../include/uapi/linux/bpf.h && ( \
> (diff -B ../../include/uapi/linux/bpf.h ../../../include/uapi/linux/bpf.h >/dev/null) || \
> echo "Warning: Kernel ABI header at 'tools/include/uapi/linux/bpf.h' differs from latest version at 'include/uapi/linux/bpf.h'" >&2 )) || true
> @@ -177,12 +177,12 @@ $(BPF_IN_SHARED): force elfdep bpfdep bpf_helper_defs.h
> echo "Warning: Kernel ABI header at 'tools/include/uapi/linux/if_xdp.h' differs from latest version at 'include/uapi/linux/if_xdp.h'" >&2 )) || true
> $(Q)$(MAKE) $(build)=libbpf OUTPUT=$(SHARED_OBJDIR) CFLAGS="$(CFLAGS) $(SHLIB_FLAGS)"

[...]

2019-12-20 22:06:11

by Arnaldo Carvalho de Melo

[permalink] [raw]

Subject: Re: [PATCH] libbpf: Fix build on read-only filesystems

Em Fri, Dec 20, 2019 at 02:00:48PM -0800, Andrii Nakryiko escreveu:
> On Fri, Dec 20, 2019 at 1:53 PM Arnaldo Carvalho de Melo <[email protected]> wrote:
> > Em Fri, Dec 20, 2019 at 01:45:52PM -0800, Andrii Nakryiko escreveu:
> > > On Fri, Dec 20, 2019 at 12:47 PM Arnaldo Carvalho de Melo <[email protected]> wrote:
> > > > Shouldn't this be applied to the current merge window since a behaviour
> > > > that people relied, i.e. using O= to generate the build in a separate
> > > > directory, since its not possible to use the source dir tree as it is
> > > > read-only is now broken, i.e. isn't this a regression?

> > > Sure, it can be applied against bpf as well, but selftests still need
> > > to be fixed first.

> > I guess this can be done on a separate patch? I.e. if the user doesn't
> > use selftests the only regression it will see is when trying to build
> > tools/perf using O=.

> > I think two patches is best, better granularity, do you see a strict
> > need for both to be in the same patch?

> Sure, it can be two separate patches, but they should go in together,
> otherwise selftests will be broken.

Sure, both have to be fixed :-)

- Arnaldo

2019-12-21 08:27:43

by Namhyung Kim

[permalink] [raw]

Subject: Re: [PATCH] libbpf: Fix build on read-only filesystems

Hello,

On Sat, Dec 21, 2019 at 5:29 AM Andrii Nakryiko
<[email protected]> wrote:
>
> On Thu, Dec 19, 2019 at 7:26 PM Namhyung Kim <[email protected]> wrote:
> >
> > I got the following error when I tried to build perf on a read-only
> > filesystem with O=dir option.
> >
> > $ cd /some/where/ro/linux/tools/perf
> > $ make O=$HOME/build/perf
> > ...
> > CC /home/namhyung/build/perf/lib.o
> > /bin/sh: bpf_helper_defs.h: Read-only file system
> > make[3]: *** [Makefile:184: bpf_helper_defs.h] Error 1
> > make[2]: *** [Makefile.perf:778: /home/namhyung/build/perf/libbpf.a] Error 2
> > make[2]: *** Waiting for unfinished jobs....
> > LD /home/namhyung/build/perf/libperf-in.o
> > AR /home/namhyung/build/perf/libperf.a
> > PERF_VERSION = 5.4.0
> > make[1]: *** [Makefile.perf:225: sub-make] Error 2
> > make: *** [Makefile:70: all] Error 2
> >
> > It was becaused bpf_helper_defs.h was generated in current directory.
> > Move it to OUTPUT directory.
> >
> > Signed-off-by: Namhyung Kim <[email protected]>
> > ---
>
> Overall nothing is obviously broken, except you need to fix up
> selftests/bpf's Makefile as well.

Thanks for pointing this out. It's because bpf selftest also needs the
bpf_helper_defs.h right? But I'm currently having a problem with LLVM
when building the selftests. Can you help me testing the patch below?
(It should be applied after this patch. Are you ok with it?)

>
> BTW, this patch doesn't apply cleanly to latest bpf-next, so please rebase.
>
> Also subject prefix should look like [PATCH bpf-next] if it's meant to
> be applied against bpf-next.

Will do.

Thanks
Namhyung

-----------8<-------------
diff --git a/tools/testing/selftests/bpf/Makefile
b/tools/testing/selftests/bpf/Makefile
index 866fc1cadd7c..897877f7849b 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -151,9 +151,9 @@ $(DEFAULT_BPFTOOL): force
$(BPFOBJ): force
$(MAKE) -C $(BPFDIR) OUTPUT=$(OUTPUT)/

-BPF_HELPERS := $(BPFDIR)/bpf_helper_defs.h $(wildcard $(BPFDIR)/bpf_*.h)
-$(BPFDIR)/bpf_helper_defs.h:
- $(MAKE) -C $(BPFDIR) OUTPUT=$(OUTPUT)/ bpf_helper_defs.h
+BPF_HELPERS := $(OUTPUT)/bpf_helper_defs.h $(wildcard $(BPFDIR)/bpf_*.h)
+$(OUTPUT)/bpf_helper_defs.h:
+ $(MAKE) -C $(BPFDIR) OUTPUT=$(OUTPUT)/ $(OUTPUT)/bpf_helper_defs.h

# Get Clang's default includes on this system, as opposed to those seen by
# '-target bpf'. This fixes "missing" files on some architectures/distros,

2019-12-21 16:24:05

by Alexei Starovoitov

[permalink] [raw]

Subject: Re: [PATCH] libbpf: Fix build on read-only filesystems

On Sat, Dec 21, 2019 at 05:25:51PM +0900, Namhyung Kim wrote:
> Hello,
>
> On Sat, Dec 21, 2019 at 5:29 AM Andrii Nakryiko
> <[email protected]> wrote:
> >
> > On Thu, Dec 19, 2019 at 7:26 PM Namhyung Kim <[email protected]> wrote:
> > >
> > > I got the following error when I tried to build perf on a read-only
> > > filesystem with O=dir option.
> > >
> > > $ cd /some/where/ro/linux/tools/perf
> > > $ make O=$HOME/build/perf
> > > ...
> > > CC /home/namhyung/build/perf/lib.o
> > > /bin/sh: bpf_helper_defs.h: Read-only file system
> > > make[3]: *** [Makefile:184: bpf_helper_defs.h] Error 1
> > > make[2]: *** [Makefile.perf:778: /home/namhyung/build/perf/libbpf.a] Error 2
> > > make[2]: *** Waiting for unfinished jobs....
> > > LD /home/namhyung/build/perf/libperf-in.o
> > > AR /home/namhyung/build/perf/libperf.a
> > > PERF_VERSION = 5.4.0
> > > make[1]: *** [Makefile.perf:225: sub-make] Error 2
> > > make: *** [Makefile:70: all] Error 2
> > >
> > > It was becaused bpf_helper_defs.h was generated in current directory.
> > > Move it to OUTPUT directory.
> > >
> > > Signed-off-by: Namhyung Kim <[email protected]>
> > > ---
> >
> > Overall nothing is obviously broken, except you need to fix up
> > selftests/bpf's Makefile as well.
>
> Thanks for pointing this out. It's because bpf selftest also needs the
> bpf_helper_defs.h right? But I'm currently having a problem with LLVM
> when building the selftests. Can you help me testing the patch below?
> (It should be applied after this patch. Are you ok with it?)
>
>
> >
> > BTW, this patch doesn't apply cleanly to latest bpf-next, so please rebase.
> >
> > Also subject prefix should look like [PATCH bpf-next] if it's meant to
> > be applied against bpf-next.
>
> Will do.
>
> Thanks
> Namhyung
>
> -----------8<-------------
> diff --git a/tools/testing/selftests/bpf/Makefile
> b/tools/testing/selftests/bpf/Makefile
> index 866fc1cadd7c..897877f7849b 100644
> --- a/tools/testing/selftests/bpf/Makefile
> +++ b/tools/testing/selftests/bpf/Makefile
> @@ -151,9 +151,9 @@ $(DEFAULT_BPFTOOL): force
> $(BPFOBJ): force
> $(MAKE) -C $(BPFDIR) OUTPUT=$(OUTPUT)/
>
> -BPF_HELPERS := $(BPFDIR)/bpf_helper_defs.h $(wildcard $(BPFDIR)/bpf_*.h)
> -$(BPFDIR)/bpf_helper_defs.h:
> - $(MAKE) -C $(BPFDIR) OUTPUT=$(OUTPUT)/ bpf_helper_defs.h
> +BPF_HELPERS := $(OUTPUT)/bpf_helper_defs.h $(wildcard $(BPFDIR)/bpf_*.h)
> +$(OUTPUT)/bpf_helper_defs.h:
> + $(MAKE) -C $(BPFDIR) OUTPUT=$(OUTPUT)/ $(OUTPUT)/bpf_helper_defs.h

The fix makes sense, but you cannot break it down into two patches.
The selftests/bpf are absolutely essential for everyone working on bpf.
For both developers and maintainers. You cannot break them in one patch
and then try to fix in another.
Please resubmit as one patch and tag the subject as [PATCH bpf].

2019-12-23 03:06:28

by Namhyung Kim

[permalink] [raw]

Subject: [PATCH bpf] libbpf: Fix build on read-only filesystems

I got the following error when I tried to build perf on a read-only
filesystem with O=dir option.

$ cd /some/where/ro/linux/tools/perf
$ make O=$HOME/build/perf
...
CC /home/namhyung/build/perf/lib.o
/bin/sh: bpf_helper_defs.h: Read-only file system
make[3]: *** [Makefile:184: bpf_helper_defs.h] Error 1
make[2]: *** [Makefile.perf:778: /home/namhyung/build/perf/libbpf.a] Error 2
make[2]: *** Waiting for unfinished jobs....
LD /home/namhyung/build/perf/libperf-in.o
AR /home/namhyung/build/perf/libperf.a
PERF_VERSION = 5.4.0
make[1]: *** [Makefile.perf:225: sub-make] Error 2
make: *** [Makefile:70: all] Error 2

It was becaused bpf_helper_defs.h was generated in current directory.
Move it to OUTPUT directory.

Signed-off-by: Namhyung Kim <[email protected]>
---
tools/lib/bpf/Makefile | 15 ++++++++-------
tools/testing/selftests/bpf/Makefile | 6 +++---
2 files changed, 11 insertions(+), 10 deletions(-)

diff --git a/tools/lib/bpf/Makefile b/tools/lib/bpf/Makefile
index defae23a0169..97830e46d1a0 100644
--- a/tools/lib/bpf/Makefile
+++ b/tools/lib/bpf/Makefile
@@ -138,6 +138,7 @@ STATIC_OBJDIR := $(OUTPUT)staticobjs/
BPF_IN_SHARED := $(SHARED_OBJDIR)libbpf-in.o
BPF_IN_STATIC := $(STATIC_OBJDIR)libbpf-in.o
VERSION_SCRIPT := libbpf.map
+BPF_HELPER_DEFS := $(OUTPUT)bpf_helper_defs.h

LIB_TARGET := $(addprefix $(OUTPUT),$(LIB_TARGET))
LIB_FILE := $(addprefix $(OUTPUT),$(LIB_FILE))
@@ -159,7 +160,7 @@ all: fixdep

all_cmd: $(CMD_TARGETS) check

-$(BPF_IN_SHARED): force elfdep bpfdep bpf_helper_defs.h
+$(BPF_IN_SHARED): force elfdep bpfdep $(BPF_HELPER_DEFS)
@(test -f ../../include/uapi/linux/bpf.h -a -f ../../../include/uapi/linux/bpf.h && ( \
(diff -B ../../include/uapi/linux/bpf.h ../../../include/uapi/linux/bpf.h >/dev/null) || \
echo "Warning: Kernel ABI header at 'tools/include/uapi/linux/bpf.h' differs from latest version at 'include/uapi/linux/bpf.h'" >&2 )) || true
@@ -177,12 +178,12 @@ $(BPF_IN_SHARED): force elfdep bpfdep bpf_helper_defs.h
echo "Warning: Kernel ABI header at 'tools/include/uapi/linux/if_xdp.h' differs from latest version at 'include/uapi/linux/if_xdp.h'" >&2 )) || true
$(Q)$(MAKE) $(build)=libbpf OUTPUT=$(SHARED_OBJDIR) CFLAGS="$(CFLAGS) $(SHLIB_FLAGS)"

-$(BPF_IN_STATIC): force elfdep bpfdep bpf_helper_defs.h
+$(BPF_IN_STATIC): force elfdep bpfdep $(BPF_HELPER_DEFS)
$(Q)$(MAKE) $(build)=libbpf OUTPUT=$(STATIC_OBJDIR)

-bpf_helper_defs.h: $(srctree)/tools/include/uapi/linux/bpf.h
+$(BPF_HELPER_DEFS): $(srctree)/tools/include/uapi/linux/bpf.h
$(Q)$(srctree)/scripts/bpf_helpers_doc.py --header \
- --file $(srctree)/tools/include/uapi/linux/bpf.h > bpf_helper_defs.h
+ --file $(srctree)/tools/include/uapi/linux/bpf.h > $(BPF_HELPER_DEFS)

$(OUTPUT)libbpf.so: $(OUTPUT)libbpf.so.$(LIBBPF_VERSION)

@@ -243,7 +244,7 @@ install_lib: all_cmd
$(call do_install_mkdir,$(libdir_SQ)); \
cp -fpR $(LIB_FILE) $(DESTDIR)$(libdir_SQ)

-install_headers: bpf_helper_defs.h
+install_headers: $(BPF_HELPER_DEFS)
$(call QUIET_INSTALL, headers) \
$(call do_install,bpf.h,$(prefix)/include/bpf,644); \
$(call do_install,libbpf.h,$(prefix)/include/bpf,644); \
@@ -251,7 +252,7 @@ install_headers: bpf_helper_defs.h
$(call do_install,libbpf_util.h,$(prefix)/include/bpf,644); \
$(call do_install,xsk.h,$(prefix)/include/bpf,644); \
$(call do_install,bpf_helpers.h,$(prefix)/include/bpf,644); \
- $(call do_install,bpf_helper_defs.h,$(prefix)/include/bpf,644); \
+ $(call do_install,$(BPF_HELPER_DEFS),$(prefix)/include/bpf,644); \
$(call do_install,bpf_tracing.h,$(prefix)/include/bpf,644); \
$(call do_install,bpf_endian.h,$(prefix)/include/bpf,644); \
$(call do_install,bpf_core_read.h,$(prefix)/include/bpf,644);
@@ -271,7 +272,7 @@ install: install_lib install_pkgconfig
clean:
$(call QUIET_CLEAN, libbpf) $(RM) -rf $(CMD_TARGETS) \
*.o *~ *.a *.so *.so.$(LIBBPF_MAJOR_VERSION) .*.d .*.cmd \
- *.pc LIBBPF-CFLAGS bpf_helper_defs.h \
+ *.pc LIBBPF-CFLAGS $(BPF_HELPER_DEFS) \
$(SHARED_OBJDIR) $(STATIC_OBJDIR)
$(call QUIET_CLEAN, core-gen) $(RM) $(OUTPUT)FEATURE-DUMP.libbpf

diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index e0fe01d9ec33..e2fd6f8d579c 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -120,9 +120,9 @@ $(OUTPUT)/test_cgroup_attach: cgroup_helpers.c
$(BPFOBJ): force
$(MAKE) -C $(BPFDIR) OUTPUT=$(OUTPUT)/

-BPF_HELPERS := $(BPFDIR)/bpf_helper_defs.h $(wildcard $(BPFDIR)/bpf_*.h)
-$(BPFDIR)/bpf_helper_defs.h:
- $(MAKE) -C $(BPFDIR) OUTPUT=$(OUTPUT)/ bpf_helper_defs.h
+BPF_HELPERS := $(OUTPUT)/bpf_helper_defs.h $(wildcard $(BPFDIR)/bpf_*.h)
+$(OUTPUT)/bpf_helper_defs.h:
+ $(MAKE) -C $(BPFDIR) OUTPUT=$(OUTPUT)/ $(OUTPUT)/bpf_helper_defs.h

# Get Clang's default includes on this system, as opposed to those seen by
# '-target bpf'. This fixes "missing" files on some architectures/distros,
--
2.24.1.735.g03f4e72817-goog

2019-12-23 05:46:31

by Andrii Nakryiko

[permalink] [raw]

Subject: Re: [PATCH bpf] libbpf: Fix build on read-only filesystems

On Sun, Dec 22, 2019 at 7:05 PM Namhyung Kim <[email protected]> wrote:
>
> I got the following error when I tried to build perf on a read-only
> filesystem with O=dir option.
>
> $ cd /some/where/ro/linux/tools/perf
> $ make O=$HOME/build/perf
> ...
> CC /home/namhyung/build/perf/lib.o
> /bin/sh: bpf_helper_defs.h: Read-only file system
> make[3]: *** [Makefile:184: bpf_helper_defs.h] Error 1
> make[2]: *** [Makefile.perf:778: /home/namhyung/build/perf/libbpf.a] Error 2
> make[2]: *** Waiting for unfinished jobs....
> LD /home/namhyung/build/perf/libperf-in.o
> AR /home/namhyung/build/perf/libperf.a
> PERF_VERSION = 5.4.0
> make[1]: *** [Makefile.perf:225: sub-make] Error 2
> make: *** [Makefile:70: all] Error 2
>
> It was becaused bpf_helper_defs.h was generated in current directory.
> Move it to OUTPUT directory.
>
> Signed-off-by: Namhyung Kim <[email protected]>
> ---

Looks good, thanks!

Tested-by: Andrii Nakryiko <[email protected]>
Acked-by: Andrii Nakryiko <[email protected]>

> tools/lib/bpf/Makefile | 15 ++++++++-------
> tools/testing/selftests/bpf/Makefile | 6 +++---
> 2 files changed, 11 insertions(+), 10 deletions(-)
>

[...]

2019-12-23 05:55:51

by Andrii Nakryiko

[permalink] [raw]

Subject: Re: [PATCH bpf] libbpf: Fix build on read-only filesystems

On Sun, Dec 22, 2019 at 9:45 PM Andrii Nakryiko
<[email protected]> wrote:
>
> On Sun, Dec 22, 2019 at 7:05 PM Namhyung Kim <[email protected]> wrote:
> >
> > I got the following error when I tried to build perf on a read-only
> > filesystem with O=dir option.
> >
> > $ cd /some/where/ro/linux/tools/perf
> > $ make O=$HOME/build/perf
> > ...
> > CC /home/namhyung/build/perf/lib.o
> > /bin/sh: bpf_helper_defs.h: Read-only file system
> > make[3]: *** [Makefile:184: bpf_helper_defs.h] Error 1
> > make[2]: *** [Makefile.perf:778: /home/namhyung/build/perf/libbpf.a] Error 2
> > make[2]: *** Waiting for unfinished jobs....
> > LD /home/namhyung/build/perf/libperf-in.o
> > AR /home/namhyung/build/perf/libperf.a
> > PERF_VERSION = 5.4.0
> > make[1]: *** [Makefile.perf:225: sub-make] Error 2
> > make: *** [Makefile:70: all] Error 2
> >
> > It was becaused bpf_helper_defs.h was generated in current directory.
> > Move it to OUTPUT directory.
> >
> > Signed-off-by: Namhyung Kim <[email protected]>
> > ---
>
> Looks good, thanks!

just one minor thing: bpf_helper_defs.h has to be added to .gitignore
under selftests/bpf now

>
> Tested-by: Andrii Nakryiko <[email protected]>
> Acked-by: Andrii Nakryiko <[email protected]>
>
> > tools/lib/bpf/Makefile | 15 ++++++++-------
> > tools/testing/selftests/bpf/Makefile | 6 +++---
> > 2 files changed, 11 insertions(+), 10 deletions(-)
> >
>
> [...]

2019-12-23 06:11:01

by Namhyung Kim

[permalink] [raw]

Subject: Re: [PATCH bpf] libbpf: Fix build on read-only filesystems

Hello,

On Mon, Dec 23, 2019 at 2:55 PM Andrii Nakryiko
<[email protected]> wrote:
>
> On Sun, Dec 22, 2019 at 9:45 PM Andrii Nakryiko
> <[email protected]> wrote:
> >
> > On Sun, Dec 22, 2019 at 7:05 PM Namhyung Kim <[email protected]> wrote:
> > >
> > > I got the following error when I tried to build perf on a read-only
> > > filesystem with O=dir option.
> > >
> > > $ cd /some/where/ro/linux/tools/perf
> > > $ make O=$HOME/build/perf
> > > ...
> > > CC /home/namhyung/build/perf/lib.o
> > > /bin/sh: bpf_helper_defs.h: Read-only file system
> > > make[3]: *** [Makefile:184: bpf_helper_defs.h] Error 1
> > > make[2]: *** [Makefile.perf:778: /home/namhyung/build/perf/libbpf.a] Error 2
> > > make[2]: *** Waiting for unfinished jobs....
> > > LD /home/namhyung/build/perf/libperf-in.o
> > > AR /home/namhyung/build/perf/libperf.a
> > > PERF_VERSION = 5.4.0
> > > make[1]: *** [Makefile.perf:225: sub-make] Error 2
> > > make: *** [Makefile:70: all] Error 2
> > >
> > > It was becaused bpf_helper_defs.h was generated in current directory.
> > > Move it to OUTPUT directory.
> > >
> > > Signed-off-by: Namhyung Kim <[email protected]>
> > > ---
> >
> > Looks good, thanks!
>
> just one minor thing: bpf_helper_defs.h has to be added to .gitignore
> under selftests/bpf now
>
> >
> > Tested-by: Andrii Nakryiko <[email protected]>
> > Acked-by: Andrii Nakryiko <[email protected]>

Thanks a lot for the review, I'll send v3 soon

Thanks
Namhyung

2019-12-23 06:14:34

by Namhyung Kim

[permalink] [raw]

Subject: [PATCH bpf v3] libbpf: Fix build on read-only filesystems

I got the following error when I tried to build perf on a read-only
filesystem with O=dir option.

$ cd /some/where/ro/linux/tools/perf
$ make O=$HOME/build/perf
...
CC /home/namhyung/build/perf/lib.o
/bin/sh: bpf_helper_defs.h: Read-only file system
make[3]: *** [Makefile:184: bpf_helper_defs.h] Error 1
make[2]: *** [Makefile.perf:778: /home/namhyung/build/perf/libbpf.a] Error 2
make[2]: *** Waiting for unfinished jobs....
LD /home/namhyung/build/perf/libperf-in.o
AR /home/namhyung/build/perf/libperf.a
PERF_VERSION = 5.4.0
make[1]: *** [Makefile.perf:225: sub-make] Error 2
make: *** [Makefile:70: all] Error 2

It was becaused bpf_helper_defs.h was generated in current directory.
Move it to OUTPUT directory.

Tested-by: Andrii Nakryiko <[email protected]>
Acked-by: Andrii Nakryiko <[email protected]>
Signed-off-by: Namhyung Kim <[email protected]>
---
tools/lib/bpf/Makefile | 15 ++++++++-------
tools/testing/selftests/bpf/.gitignore | 1 +
tools/testing/selftests/bpf/Makefile | 6 +++---
3 files changed, 12 insertions(+), 10 deletions(-)

diff --git a/tools/lib/bpf/Makefile b/tools/lib/bpf/Makefile
index defae23a0169..97830e46d1a0 100644
--- a/tools/lib/bpf/Makefile
+++ b/tools/lib/bpf/Makefile
@@ -138,6 +138,7 @@ STATIC_OBJDIR := $(OUTPUT)staticobjs/
BPF_IN_SHARED := $(SHARED_OBJDIR)libbpf-in.o
BPF_IN_STATIC := $(STATIC_OBJDIR)libbpf-in.o
VERSION_SCRIPT := libbpf.map
+BPF_HELPER_DEFS := $(OUTPUT)bpf_helper_defs.h

LIB_TARGET := $(addprefix $(OUTPUT),$(LIB_TARGET))
LIB_FILE := $(addprefix $(OUTPUT),$(LIB_FILE))
@@ -159,7 +160,7 @@ all: fixdep

all_cmd: $(CMD_TARGETS) check

-$(BPF_IN_SHARED): force elfdep bpfdep bpf_helper_defs.h
+$(BPF_IN_SHARED): force elfdep bpfdep $(BPF_HELPER_DEFS)
@(test -f ../../include/uapi/linux/bpf.h -a -f ../../../include/uapi/linux/bpf.h && ( \
(diff -B ../../include/uapi/linux/bpf.h ../../../include/uapi/linux/bpf.h >/dev/null) || \
echo "Warning: Kernel ABI header at 'tools/include/uapi/linux/bpf.h' differs from latest version at 'include/uapi/linux/bpf.h'" >&2 )) || true
@@ -177,12 +178,12 @@ $(BPF_IN_SHARED): force elfdep bpfdep bpf_helper_defs.h
echo "Warning: Kernel ABI header at 'tools/include/uapi/linux/if_xdp.h' differs from latest version at 'include/uapi/linux/if_xdp.h'" >&2 )) || true
$(Q)$(MAKE) $(build)=libbpf OUTPUT=$(SHARED_OBJDIR) CFLAGS="$(CFLAGS) $(SHLIB_FLAGS)"

-$(BPF_IN_STATIC): force elfdep bpfdep bpf_helper_defs.h
+$(BPF_IN_STATIC): force elfdep bpfdep $(BPF_HELPER_DEFS)
$(Q)$(MAKE) $(build)=libbpf OUTPUT=$(STATIC_OBJDIR)

-bpf_helper_defs.h: $(srctree)/tools/include/uapi/linux/bpf.h
+$(BPF_HELPER_DEFS): $(srctree)/tools/include/uapi/linux/bpf.h
$(Q)$(srctree)/scripts/bpf_helpers_doc.py --header \
- --file $(srctree)/tools/include/uapi/linux/bpf.h > bpf_helper_defs.h
+ --file $(srctree)/tools/include/uapi/linux/bpf.h > $(BPF_HELPER_DEFS)

$(OUTPUT)libbpf.so: $(OUTPUT)libbpf.so.$(LIBBPF_VERSION)

@@ -243,7 +244,7 @@ install_lib: all_cmd
$(call do_install_mkdir,$(libdir_SQ)); \
cp -fpR $(LIB_FILE) $(DESTDIR)$(libdir_SQ)

-install_headers: bpf_helper_defs.h
+install_headers: $(BPF_HELPER_DEFS)
$(call QUIET_INSTALL, headers) \
$(call do_install,bpf.h,$(prefix)/include/bpf,644); \
$(call do_install,libbpf.h,$(prefix)/include/bpf,644); \
@@ -251,7 +252,7 @@ install_headers: bpf_helper_defs.h
$(call do_install,libbpf_util.h,$(prefix)/include/bpf,644); \
$(call do_install,xsk.h,$(prefix)/include/bpf,644); \
$(call do_install,bpf_helpers.h,$(prefix)/include/bpf,644); \
- $(call do_install,bpf_helper_defs.h,$(prefix)/include/bpf,644); \
+ $(call do_install,$(BPF_HELPER_DEFS),$(prefix)/include/bpf,644); \
$(call do_install,bpf_tracing.h,$(prefix)/include/bpf,644); \
$(call do_install,bpf_endian.h,$(prefix)/include/bpf,644); \
$(call do_install,bpf_core_read.h,$(prefix)/include/bpf,644);
@@ -271,7 +272,7 @@ install: install_lib install_pkgconfig
clean:
$(call QUIET_CLEAN, libbpf) $(RM) -rf $(CMD_TARGETS) \
*.o *~ *.a *.so *.so.$(LIBBPF_MAJOR_VERSION) .*.d .*.cmd \
- *.pc LIBBPF-CFLAGS bpf_helper_defs.h \
+ *.pc LIBBPF-CFLAGS $(BPF_HELPER_DEFS) \
$(SHARED_OBJDIR) $(STATIC_OBJDIR)
$(call QUIET_CLEAN, core-gen) $(RM) $(OUTPUT)FEATURE-DUMP.libbpf

diff --git a/tools/testing/selftests/bpf/.gitignore b/tools/testing/selftests/bpf/.gitignore
index 419652458da4..1ff0a9f49c01 100644
--- a/tools/testing/selftests/bpf/.gitignore
+++ b/tools/testing/selftests/bpf/.gitignore
@@ -40,3 +40,4 @@ xdping
test_cpp
/no_alu32
/bpf_gcc
+bpf_helper_defs.h
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index e0fe01d9ec33..e2fd6f8d579c 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -120,9 +120,9 @@ $(OUTPUT)/test_cgroup_attach: cgroup_helpers.c
$(BPFOBJ): force
$(MAKE) -C $(BPFDIR) OUTPUT=$(OUTPUT)/

-BPF_HELPERS := $(BPFDIR)/bpf_helper_defs.h $(wildcard $(BPFDIR)/bpf_*.h)
-$(BPFDIR)/bpf_helper_defs.h:
- $(MAKE) -C $(BPFDIR) OUTPUT=$(OUTPUT)/ bpf_helper_defs.h
+BPF_HELPERS := $(OUTPUT)/bpf_helper_defs.h $(wildcard $(BPFDIR)/bpf_*.h)
+$(OUTPUT)/bpf_helper_defs.h:
+ $(MAKE) -C $(BPFDIR) OUTPUT=$(OUTPUT)/ $(OUTPUT)/bpf_helper_defs.h

# Get Clang's default includes on this system, as opposed to those seen by
# '-target bpf'. This fixes "missing" files on some architectures/distros,
--
2.24.1.735.g03f4e72817-goog

2019-12-23 06:36:54

by Andrii Nakryiko

[permalink] [raw]

Subject: Re: [PATCH bpf v3] libbpf: Fix build on read-only filesystems

On Sun, Dec 22, 2019 at 10:14 PM Namhyung Kim <[email protected]> wrote:
>
> I got the following error when I tried to build perf on a read-only
> filesystem with O=dir option.
>
> $ cd /some/where/ro/linux/tools/perf
> $ make O=$HOME/build/perf
> ...
> CC /home/namhyung/build/perf/lib.o
> /bin/sh: bpf_helper_defs.h: Read-only file system
> make[3]: *** [Makefile:184: bpf_helper_defs.h] Error 1
> make[2]: *** [Makefile.perf:778: /home/namhyung/build/perf/libbpf.a] Error 2
> make[2]: *** Waiting for unfinished jobs....
> LD /home/namhyung/build/perf/libperf-in.o
> AR /home/namhyung/build/perf/libperf.a
> PERF_VERSION = 5.4.0
> make[1]: *** [Makefile.perf:225: sub-make] Error 2
> make: *** [Makefile:70: all] Error 2
>
> It was becaused bpf_helper_defs.h was generated in current directory.
> Move it to OUTPUT directory.
>
> Tested-by: Andrii Nakryiko <[email protected]>
> Acked-by: Andrii Nakryiko <[email protected]>
> Signed-off-by: Namhyung Kim <[email protected]>
> ---
> tools/lib/bpf/Makefile | 15 ++++++++-------
> tools/testing/selftests/bpf/.gitignore | 1 +
> tools/testing/selftests/bpf/Makefile | 6 +++---
> 3 files changed, 12 insertions(+), 10 deletions(-)
>

[...]

> diff --git a/tools/testing/selftests/bpf/.gitignore b/tools/testing/selftests/bpf/.gitignore
> index 419652458da4..1ff0a9f49c01 100644
> --- a/tools/testing/selftests/bpf/.gitignore
> +++ b/tools/testing/selftests/bpf/.gitignore
> @@ -40,3 +40,4 @@ xdping
> test_cpp
> /no_alu32
> /bpf_gcc
> +bpf_helper_defs.h

looks good, thanks!

[...]

2019-12-23 14:40:40

by Daniel Borkmann

[permalink] [raw]

Subject: Re: [PATCH bpf v3] libbpf: Fix build on read-only filesystems

On 12/23/19 7:29 AM, Andrii Nakryiko wrote:
> On Sun, Dec 22, 2019 at 10:14 PM Namhyung Kim <[email protected]> wrote:
>>
>> I got the following error when I tried to build perf on a read-only
>> filesystem with O=dir option.
>>
>> $ cd /some/where/ro/linux/tools/perf
>> $ make O=$HOME/build/perf
>> ...
>> CC /home/namhyung/build/perf/lib.o
>> /bin/sh: bpf_helper_defs.h: Read-only file system
>> make[3]: *** [Makefile:184: bpf_helper_defs.h] Error 1
>> make[2]: *** [Makefile.perf:778: /home/namhyung/build/perf/libbpf.a] Error 2
>> make[2]: *** Waiting for unfinished jobs....
>> LD /home/namhyung/build/perf/libperf-in.o
>> AR /home/namhyung/build/perf/libperf.a
>> PERF_VERSION = 5.4.0
>> make[1]: *** [Makefile.perf:225: sub-make] Error 2
>> make: *** [Makefile:70: all] Error 2
>>
>> It was becaused bpf_helper_defs.h was generated in current directory.
>> Move it to OUTPUT directory.
>>
>> Tested-by: Andrii Nakryiko <[email protected]>
>> Acked-by: Andrii Nakryiko <[email protected]>
>> Signed-off-by: Namhyung Kim <[email protected]>
>> ---
>> tools/lib/bpf/Makefile | 15 ++++++++-------
>> tools/testing/selftests/bpf/.gitignore | 1 +
>> tools/testing/selftests/bpf/Makefile | 6 +++---
>> 3 files changed, 12 insertions(+), 10 deletions(-)
>>
>
> [...]
>
>> diff --git a/tools/testing/selftests/bpf/.gitignore b/tools/testing/selftests/bpf/.gitignore
>> index 419652458da4..1ff0a9f49c01 100644
>> --- a/tools/testing/selftests/bpf/.gitignore
>> +++ b/tools/testing/selftests/bpf/.gitignore
>> @@ -40,3 +40,4 @@ xdping
>> test_cpp
>> /no_alu32
>> /bpf_gcc
>> +bpf_helper_defs.h
>
> looks good, thanks!

Applied, thanks!