2018-12-05 11:30:39

by Masahiro Yamada

[permalink] [raw]
Subject: [PATCH 1/3] kbuild: refactor Makefile.asm-generic

- Use conventional $(MAKE) $(asm-generic)=<dir> style
for directory descending

- Remove unneeded FORCE since "all" is a phony target

- Remove unneeded "_dummy :=" assignment

- Skip $(shell mkdir ...) when headers exist in the directory

- Misc cleanups

Signed-off-by: Masahiro Yamada <[email protected]>
---

Makefile | 8 ++++----
scripts/Makefile.asm-generic | 37 +++++++++++++++++++------------------
2 files changed, 23 insertions(+), 22 deletions(-)

diff --git a/Makefile b/Makefile
index ab38eb2..3af0379 100644
--- a/Makefile
+++ b/Makefile
@@ -1122,13 +1122,13 @@ prepare0: archprepare
prepare: prepare0 prepare-objtool

# Support for using generic headers in asm-generic
+asm-generic := -f $(srctree)/scripts/Makefile.asm-generic obj
+
PHONY += asm-generic uapi-asm-generic
asm-generic: uapi-asm-generic
- $(Q)$(MAKE) -f $(srctree)/scripts/Makefile.asm-generic \
- src=asm obj=arch/$(SRCARCH)/include/generated/asm
+ $(Q)$(MAKE) $(asm-generic)=arch/$(SRCARCH)/include/generated/asm
uapi-asm-generic:
- $(Q)$(MAKE) -f $(srctree)/scripts/Makefile.asm-generic \
- src=uapi/asm obj=arch/$(SRCARCH)/include/generated/uapi/asm
+ $(Q)$(MAKE) $(asm-generic)=arch/$(SRCARCH)/include/generated/uapi/asm

PHONY += prepare-objtool
prepare-objtool: $(objtool_target)
diff --git a/scripts/Makefile.asm-generic b/scripts/Makefile.asm-generic
index 32ad8e9..760323e 100644
--- a/scripts/Makefile.asm-generic
+++ b/scripts/Makefile.asm-generic
@@ -2,41 +2,42 @@
# include/asm-generic contains a lot of files that are used
# verbatim by several architectures.
#
-# This Makefile reads the file arch/$(SRCARCH)/include/$(src)/Kbuild
+# This Makefile reads the file arch/$(SRCARCH)/include/(uapi/)/asm/Kbuild
# and for each file listed in this file with generic-y creates
-# a small wrapper file in $(obj) (arch/$(SRCARCH)/include/generated/$(src))
+# a small wrapper file in arch/$(SRCARCH)/include/generated/(uapi/)/asm.

PHONY := all
all:

-kbuild-file := $(srctree)/arch/$(SRCARCH)/include/$(src)/Kbuild
--include $(kbuild-file)
+src := $(subst /generated,,$(obj))
+-include $(src)/Kbuild

include scripts/Kbuild.include

-# Create output directory if not already present
-_dummy := $(shell [ -d $(obj) ] || mkdir -p $(obj))
+generic-y := $(addprefix $(obj)/, $(generic-y))
+generated-y := $(addprefix $(obj)/, $(generated-y))

-# Stale wrappers when the corresponding files are removed from generic-y
-# need removing.
-generated-y := $(generic-y) $(generated-y)
-all-files := $(patsubst %, $(obj)/%, $(generated-y))
-old-headers := $(wildcard $(obj)/*.h)
-unwanted := $(filter-out $(all-files),$(old-headers))
+# Remove stale wrappers when the corresponding files are removed from generic-y
+old-headers := $(wildcard $(obj)/*.h)
+unwanted := $(filter-out $(generic-y) $(generated-y),$(old-headers))

quiet_cmd_wrap = WRAP $@
-cmd_wrap = echo "\#include <asm-generic/$*.h>" >$@
+ cmd_wrap = echo "\#include <asm-generic/$*.h>" > $@

quiet_cmd_remove = REMOVE $(unwanted)
-cmd_remove = rm -f $(unwanted)
+ cmd_remove = rm -f $(unwanted)

-all: $(patsubst %, $(obj)/%, $(generic-y)) FORCE
- $(if $(unwanted),$(call cmd,remove),)
+all: $(generic-y)
+ $(if $(unwanted),$(call cmd,remove))
@:

$(obj)/%.h:
$(call cmd,wrap)

-PHONY += FORCE
+# Create output directory. Skip it if at least one old header exists
+# since we know the output directory already exists.
+ifeq ($(old-headers),)
+$(shell mkdir -p $(obj))
+endif
+
.PHONY: $(PHONY)
-FORCE: ;
--
2.7.4



2018-12-05 11:31:28

by Masahiro Yamada

[permalink] [raw]
Subject: [PATCH 2/3] kbuild: generate asm-generic wrappers if mandatory headers are missing

Some time ago, Sam pointed out a certain degree of overwrap between
generic-y and mandatory-y. (https://lkml.org/lkml/2017/7/10/121)

I a bit tweaked the meaning of mandatory-y; now it defines the minimum
set of ASM headers that all architectures must have.

If arch does not have specific implementation of a mandatory header,
Kbuild will let it fallback to the asm-generic one by automatically
generating a wrapper. This will allow to drop lots of redundant
generic-y defines.

Previously, "mandatory" was used in the context of UAPI, but I guess
this can be extended to kernel space ASM headers.

Suggested-by: Sam Ravnborg <[email protected]>
Signed-off-by: Masahiro Yamada <[email protected]>
---

Documentation/kbuild/makefiles.txt | 9 ++++++---
scripts/Makefile.asm-generic | 3 +++
scripts/Makefile.headersinst | 7 -------
3 files changed, 9 insertions(+), 10 deletions(-)

diff --git a/Documentation/kbuild/makefiles.txt b/Documentation/kbuild/makefiles.txt
index 8da26c6..bf28c47 100644
--- a/Documentation/kbuild/makefiles.txt
+++ b/Documentation/kbuild/makefiles.txt
@@ -1296,9 +1296,12 @@ See subsequent chapter for the syntax of the Kbuild file.

--- 7.4 mandatory-y

- mandatory-y is essentially used by include/uapi/asm-generic/Kbuild.asm
- to define the minimum set of headers that must be exported in
- include/asm.
+ mandatory-y is essentially used by include/(uapi/)asm-generic/Kbuild.asm
+ to define the minimum set of ASM headers that all architectures must have.
+
+ This works like optional generic-y. If a mandatory header is missing
+ in arch/$(ARCH)/include/(uapi/)/asm, Kbuild will automatically generate
+ a wrapper of the asm-generic one.

The convention is to list one subdir per line and
preferably in alphabetic order.
diff --git a/scripts/Makefile.asm-generic b/scripts/Makefile.asm-generic
index 760323e..c7cd23d 100644
--- a/scripts/Makefile.asm-generic
+++ b/scripts/Makefile.asm-generic
@@ -14,6 +14,9 @@ src := $(subst /generated,,$(obj))

include scripts/Kbuild.include

+# If arch does not implement mandatory headers, fallback to asm-generic ones.
+generic-y += $(foreach f, $(mandatory-y), $(if $(wildcard $(srctree)/$(src)/$(f)),,$(f)))
+
generic-y := $(addprefix $(obj)/, $(generic-y))
generated-y := $(addprefix $(obj)/, $(generated-y))

diff --git a/scripts/Makefile.headersinst b/scripts/Makefile.headersinst
index d5e1314..0d4a96d 100644
--- a/scripts/Makefile.headersinst
+++ b/scripts/Makefile.headersinst
@@ -57,13 +57,6 @@ check-file := $(installdir)/.check
all-files := $(header-files) $(genhdr-files)
output-files := $(addprefix $(installdir)/, $(all-files))

-ifneq ($(mandatory-y),)
-missing := $(filter-out $(all-files),$(mandatory-y))
-ifneq ($(missing),)
-$(error Some mandatory headers ($(missing)) are missing in $(obj))
-endif
-endif
-
# Work out what needs to be removed
oldheaders := $(patsubst $(installdir)/%,%,$(wildcard $(installdir)/*.h))
unwanted := $(filter-out $(all-files),$(oldheaders))
--
2.7.4


2018-12-06 15:06:04

by Christoph Hellwig

[permalink] [raw]
Subject: Re: [PATCH 2/3] kbuild: generate asm-generic wrappers if mandatory headers are missing

On Wed, Dec 05, 2018 at 08:28:05PM +0900, Masahiro Yamada wrote:
> Some time ago, Sam pointed out a certain degree of overwrap between
> generic-y and mandatory-y. (https://lkml.org/lkml/2017/7/10/121)
>
> I a bit tweaked the meaning of mandatory-y; now it defines the minimum
> set of ASM headers that all architectures must have.
>
> If arch does not have specific implementation of a mandatory header,
> Kbuild will let it fallback to the asm-generic one by automatically
> generating a wrapper. This will allow to drop lots of redundant
> generic-y defines.
>
> Previously, "mandatory" was used in the context of UAPI, but I guess
> this can be extended to kernel space ASM headers.

How useful is it to keep the generic-y behavior around at all vs making
everything useful mandatory?

2018-12-06 18:07:52

by Sam Ravnborg

[permalink] [raw]
Subject: Re: [PATCH 2/3] kbuild: generate asm-generic wrappers if mandatory headers are missing

On Wed, Dec 05, 2018 at 08:28:05PM +0900, Masahiro Yamada wrote:
> Some time ago, Sam pointed out a certain degree of overwrap between
> generic-y and mandatory-y. (https://lkml.org/lkml/2017/7/10/121)
>
> I a bit tweaked the meaning of mandatory-y; now it defines the minimum
> set of ASM headers that all architectures must have.
>
> If arch does not have specific implementation of a mandatory header,
> Kbuild will let it fallback to the asm-generic one by automatically
> generating a wrapper. This will allow to drop lots of redundant
> generic-y defines.
>
> Previously, "mandatory" was used in the context of UAPI, but I guess
> this can be extended to kernel space ASM headers.
>
> Suggested-by: Sam Ravnborg <[email protected]>
> Signed-off-by: Masahiro Yamada <[email protected]>

Nice work!

For the full series:
Acked-by: Sam Ravnborg <[email protected]>

Have you considered to warn if generic-y contains a header listed
in mandatory-y - to prevent that they sneak back in.
And to catch when we lift a header from available to mandatory.

Sam

2018-12-11 13:01:21

by Masahiro Yamada

[permalink] [raw]
Subject: Re: [PATCH 2/3] kbuild: generate asm-generic wrappers if mandatory headers are missing

Hi Christoph,


On Fri, Dec 7, 2018 at 12:04 AM Christoph Hellwig <[email protected]> wrote:
>
> On Wed, Dec 05, 2018 at 08:28:05PM +0900, Masahiro Yamada wrote:
> > Some time ago, Sam pointed out a certain degree of overwrap between
> > generic-y and mandatory-y. (https://lkml.org/lkml/2017/7/10/121)
> >
> > I a bit tweaked the meaning of mandatory-y; now it defines the minimum
> > set of ASM headers that all architectures must have.
> >
> > If arch does not have specific implementation of a mandatory header,
> > Kbuild will let it fallback to the asm-generic one by automatically
> > generating a wrapper. This will allow to drop lots of redundant
> > generic-y defines.
> >
> > Previously, "mandatory" was used in the context of UAPI, but I guess
> > this can be extended to kernel space ASM headers.
>
> How useful is it to keep the generic-y behavior around at all vs making
> everything useful mandatory?


What I can tell is not all architectures
support kvm_para.h, ucontext.h

I guess they will stay as arch-specific generic-y,
but I am not an expert in this area.


kvm_para.h is missing csky, nds32, riscv.

ucontext.h is missing in alpha, arm, m68k, parisc, sparc, xtensa



bpf_perf_event.h could be promoted to mandatory-y ?
All architectures have it.


--
Best Regards
Masahiro Yamada

2018-12-11 13:04:43

by Masahiro Yamada

[permalink] [raw]
Subject: Re: [PATCH 2/3] kbuild: generate asm-generic wrappers if mandatory headers are missing

Hi Sam,


On Fri, Dec 7, 2018 at 3:06 AM Sam Ravnborg <[email protected]> wrote:
>
> On Wed, Dec 05, 2018 at 08:28:05PM +0900, Masahiro Yamada wrote:
> > Some time ago, Sam pointed out a certain degree of overwrap between
> > generic-y and mandatory-y. (https://lkml.org/lkml/2017/7/10/121)
> >
> > I a bit tweaked the meaning of mandatory-y; now it defines the minimum
> > set of ASM headers that all architectures must have.
> >
> > If arch does not have specific implementation of a mandatory header,
> > Kbuild will let it fallback to the asm-generic one by automatically
> > generating a wrapper. This will allow to drop lots of redundant
> > generic-y defines.
> >
> > Previously, "mandatory" was used in the context of UAPI, but I guess
> > this can be extended to kernel space ASM headers.
> >
> > Suggested-by: Sam Ravnborg <[email protected]>
> > Signed-off-by: Masahiro Yamada <[email protected]>
>
> Nice work!
>
> For the full series:
> Acked-by: Sam Ravnborg <[email protected]>
>
> Have you considered to warn if generic-y contains a header listed
> in mandatory-y - to prevent that they sneak back in.
> And to catch when we lift a header from available to mandatory.


Yes, I also thought of this,
and probably we should do it.





--
Best Regards
Masahiro Yamada

2018-12-11 16:32:44

by Palmer Dabbelt

[permalink] [raw]
Subject: Re: [PATCH 2/3] kbuild: generate asm-generic wrappers if mandatory headers are missing

On Tue, 11 Dec 2018 04:59:26 PST (-0800), [email protected] wrote:
> Hi Christoph,
>
>
> On Fri, Dec 7, 2018 at 12:04 AM Christoph Hellwig <[email protected]> wrote:
>>
>> On Wed, Dec 05, 2018 at 08:28:05PM +0900, Masahiro Yamada wrote:
>> > Some time ago, Sam pointed out a certain degree of overwrap between
>> > generic-y and mandatory-y. (https://lkml.org/lkml/2017/7/10/121)
>> >
>> > I a bit tweaked the meaning of mandatory-y; now it defines the minimum
>> > set of ASM headers that all architectures must have.
>> >
>> > If arch does not have specific implementation of a mandatory header,
>> > Kbuild will let it fallback to the asm-generic one by automatically
>> > generating a wrapper. This will allow to drop lots of redundant
>> > generic-y defines.
>> >
>> > Previously, "mandatory" was used in the context of UAPI, but I guess
>> > this can be extended to kernel space ASM headers.
>>
>> How useful is it to keep the generic-y behavior around at all vs making
>> everything useful mandatory?
>
>
> What I can tell is not all architectures
> support kvm_para.h, ucontext.h
>
> I guess they will stay as arch-specific generic-y,
> but I am not an expert in this area.
>
>
> kvm_para.h is missing csky, nds32, riscv.

It looks like RISC-V missed it and everyone else copied us. I don't see any
reason why the generic version wouldn't work on RISC-V, as it just has failures
for all the calls.

> ucontext.h is missing in alpha, arm, m68k, parisc, sparc, xtensa
>
>
>
> bpf_perf_event.h could be promoted to mandatory-y ?
> All architectures have it.

2018-12-16 15:33:16

by Masahiro Yamada

[permalink] [raw]
Subject: Re: [PATCH 1/3] kbuild: refactor Makefile.asm-generic

On Wed, Dec 5, 2018 at 8:29 PM Masahiro Yamada
<[email protected]> wrote:
>
> - Use conventional $(MAKE) $(asm-generic)=<dir> style
> for directory descending
>
> - Remove unneeded FORCE since "all" is a phony target
>
> - Remove unneeded "_dummy :=" assignment
>
> - Skip $(shell mkdir ...) when headers exist in the directory
>
> - Misc cleanups
>
> Signed-off-by: Masahiro Yamada <[email protected]>

1/3 applied to linux-kbuild.

I will send v2 for the rest in the series later.



--
Best Regards
Masahiro Yamada