2022-11-13 11:31:35

by Masahiro Yamada

[permalink] [raw]
Subject: [PATCH 0/3] kbuild: detect objects shared among multiple modules


Commit 637a642f5ca5 ("zstd: Fixing mixed module-builtin objects")
only fixed zstd, but similar potential issues are remaining treewide.

My plan is to merge the first two in the next merge window
(to block new breakages), the last one after fixing warnings.



Masahiro Yamada (3):
kbuild: add kbuild-file macro
kbuild: warn objects shared among multiple modules
[pending] kbuild: make forbid sharing objects among multiple modules

scripts/Kbuild.include | 5 +++++
scripts/Makefile.asm-generic | 6 +++---
scripts/Makefile.build | 6 +-----
scripts/Makefile.clean | 5 +----
scripts/Makefile.dtbinst | 2 +-
scripts/Makefile.lib | 8 ++++----
scripts/Makefile.vmlinux_o | 5 +----
7 files changed, 16 insertions(+), 21 deletions(-)

--
2.34.1



2022-11-13 11:32:03

by Masahiro Yamada

[permalink] [raw]
Subject: [PATCH 1/3] kbuild: add kbuild-file macro

While building, installing, cleaning, Kbuild visits sub-directories
and includes 'Kbuild' or 'Makefile' that exists there.

Add 'kbuild-file' macro, and reuse it from scripts/Makefie.*

Signed-off-by: Masahiro Yamada <[email protected]>
---

scripts/Kbuild.include | 5 +++++
scripts/Makefile.asm-generic | 6 +++---
scripts/Makefile.build | 6 +-----
scripts/Makefile.clean | 5 +----
scripts/Makefile.dtbinst | 2 +-
5 files changed, 11 insertions(+), 13 deletions(-)

diff --git a/scripts/Kbuild.include b/scripts/Kbuild.include
index 2bc08ace38a3..cbe28744637b 100644
--- a/scripts/Kbuild.include
+++ b/scripts/Kbuild.include
@@ -40,6 +40,11 @@ escsq = $(subst $(squote),'\$(squote)',$1)
# Quote a string to pass it to C files. foo => '"foo"'
stringify = $(squote)$(quote)$1$(quote)$(squote)

+###
+# The path to Kbuild or Makefile. Kbuild has precedence over Makefile.
+kbuild-dir = $(if $(filter /%,$(src)),$(src),$(srctree)/$(src))
+kbuild-file = $(or $(wildcard $(kbuild-dir)/Kbuild),$(kbuild-dir)/Makefile)
+
###
# Easy method for doing a status message
kecho := :
diff --git a/scripts/Makefile.asm-generic b/scripts/Makefile.asm-generic
index 1d501c57f9ef..8d01b37b7677 100644
--- a/scripts/Makefile.asm-generic
+++ b/scripts/Makefile.asm-generic
@@ -10,15 +10,15 @@ PHONY := all
all:

src := $(subst /generated,,$(obj))
--include $(src)/Kbuild
+
+include $(srctree)/scripts/Kbuild.include
+-include $(kbuild-file)

# $(generic)/Kbuild lists mandatory-y. Exclude um since it is a special case.
ifneq ($(SRCARCH),um)
include $(srctree)/$(generic)/Kbuild
endif

-include $(srctree)/scripts/Kbuild.include
-
redundant := $(filter $(mandatory-y) $(generated-y), $(generic-y))
redundant += $(foreach f, $(generic-y), $(if $(wildcard $(srctree)/$(src)/$(f)),$(f)))
redundant := $(sort $(redundant))
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index 41f3602fc8de..37cf88d076e8 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -38,11 +38,7 @@ subdir-ccflags-y :=

include $(srctree)/scripts/Kbuild.include
include $(srctree)/scripts/Makefile.compiler
-
-# The filename Kbuild has precedence over Makefile
-kbuild-dir := $(if $(filter /%,$(src)),$(src),$(srctree)/$(src))
-include $(or $(wildcard $(kbuild-dir)/Kbuild),$(kbuild-dir)/Makefile)
-
+include $(kbuild-file)
include $(srctree)/scripts/Makefile.lib

# Do not include hostprogs rules unless needed.
diff --git a/scripts/Makefile.clean b/scripts/Makefile.clean
index 878cec648959..3649900696dd 100644
--- a/scripts/Makefile.clean
+++ b/scripts/Makefile.clean
@@ -9,10 +9,7 @@ PHONY := __clean
__clean:

include $(srctree)/scripts/Kbuild.include
-
-# The filename Kbuild has precedence over Makefile
-kbuild-dir := $(if $(filter /%,$(src)),$(src),$(srctree)/$(src))
-include $(or $(wildcard $(kbuild-dir)/Kbuild),$(kbuild-dir)/Makefile)
+include $(kbuild-file)

# Figure out what we need to build from the various variables
# ==========================================================================
diff --git a/scripts/Makefile.dtbinst b/scripts/Makefile.dtbinst
index 190d781e84f4..2ab936e4179d 100644
--- a/scripts/Makefile.dtbinst
+++ b/scripts/Makefile.dtbinst
@@ -15,7 +15,7 @@ __dtbs_install:

include include/config/auto.conf
include $(srctree)/scripts/Kbuild.include
-include $(src)/Makefile
+include $(kbuild-file)

dtbs := $(addprefix $(dst)/, $(dtb-y) $(if $(CONFIG_OF_ALL_DTBS),$(dtb-)))
subdirs := $(addprefix $(obj)/, $(subdir-y) $(subdir-m))
--
2.34.1


2022-11-13 11:35:35

by Masahiro Yamada

[permalink] [raw]
Subject: [PATCH 2/3] kbuild: warn objects shared among multiple modules

If an object is shared among multiple modules, some of them are
configured as 'm', but the others as 'y', the shared object is
linked to the modules and vmlinux. This is a potential issue because
the expected CFLAGS are different between modules and builtins.

Commit 637a642f5ca5 ("zstd: Fixing mixed module-builtin objects")
reported that this could be even more fatal in some cases such as
Clang LTO.

That commit fixed lib/zlib/zstd_{compress,decompress}, but there are
still more instances of breakage.

This commit adds a W=1 warning for shared objects, so that the kbuild
test robot, which provides build tests with W=1, will avoid a new
breakage slipping in.

Quick compile tests on v6.1-rc4 detected the following:

scripts/Makefile.build:252: ./drivers/crypto/marvell/octeontx2/Makefile: cn10k_cpt.o is added to multiple modules: rvu_cptpf rvu_cptvf
scripts/Makefile.build:252: ./drivers/crypto/marvell/octeontx2/Makefile: otx2_cptlf.o is added to multiple modules: rvu_cptpf rvu_cptvf
scripts/Makefile.build:252: ./drivers/crypto/marvell/octeontx2/Makefile: otx2_cpt_mbox_common.o is added to multiple modules: rvu_cptpf rvu_cptvf
scripts/Makefile.build:252: ./drivers/edac/Makefile: skx_common.o is added to multiple modules: i10nm_edac skx_edac
scripts/Makefile.build:252: ./drivers/gpu/drm/bridge/imx/Makefile: imx-ldb-helper.o is added to multiple modules: imx8qm-ldb imx8qxp-ldb
scripts/Makefile.build:252: ./drivers/mfd/Makefile: rsmu_core.o is added to multiple modules: rsmu-i2c rsmu-spi
scripts/Makefile.build:252: ./drivers/mtd/tests/Makefile: mtd_test.o is added to multiple modules: mtd_nandbiterrs mtd_oobtest mtd_pagetest mtd_readtest mtd_speedtest mtd_stresstest mtd_subpagetest mtd_torturetest
scripts/Makefile.build:252: ./drivers/net/dsa/ocelot/Makefile: felix.o is added to multiple modules: mscc_felix mscc_seville
scripts/Makefile.build:252: ./drivers/net/ethernet/cavium/liquidio/Makefile: cn23xx_pf_device.o is added to multiple modules: liquidio liquidio_vf
scripts/Makefile.build:252: ./drivers/net/ethernet/cavium/liquidio/Makefile: cn23xx_vf_device.o is added to multiple modules: liquidio liquidio_vf
scripts/Makefile.build:252: ./drivers/net/ethernet/cavium/liquidio/Makefile: cn66xx_device.o is added to multiple modules: liquidio liquidio_vf
scripts/Makefile.build:252: ./drivers/net/ethernet/cavium/liquidio/Makefile: cn68xx_device.o is added to multiple modules: liquidio liquidio_vf
scripts/Makefile.build:252: ./drivers/net/ethernet/cavium/liquidio/Makefile: lio_core.o is added to multiple modules: liquidio liquidio_vf
scripts/Makefile.build:252: ./drivers/net/ethernet/cavium/liquidio/Makefile: lio_ethtool.o is added to multiple modules: liquidio liquidio_vf
scripts/Makefile.build:252: ./drivers/net/ethernet/cavium/liquidio/Makefile: octeon_device.o is added to multiple modules: liquidio liquidio_vf
scripts/Makefile.build:252: ./drivers/net/ethernet/cavium/liquidio/Makefile: octeon_droq.o is added to multiple modules: liquidio liquidio_vf
scripts/Makefile.build:252: ./drivers/net/ethernet/cavium/liquidio/Makefile: octeon_mailbox.o is added to multiple modules: liquidio liquidio_vf
scripts/Makefile.build:252: ./drivers/net/ethernet/cavium/liquidio/Makefile: octeon_mem_ops.o is added to multiple modules: liquidio liquidio_vf
scripts/Makefile.build:252: ./drivers/net/ethernet/cavium/liquidio/Makefile: octeon_nic.o is added to multiple modules: liquidio liquidio_vf
scripts/Makefile.build:252: ./drivers/net/ethernet/cavium/liquidio/Makefile: request_manager.o is added to multiple modules: liquidio liquidio_vf
scripts/Makefile.build:252: ./drivers/net/ethernet/cavium/liquidio/Makefile: response_manager.o is added to multiple modules: liquidio liquidio_vf
scripts/Makefile.build:252: ./drivers/net/ethernet/freescale/dpaa2/Makefile: dpaa2-mac.o is added to multiple modules: fsl-dpaa2-eth fsl-dpaa2-switch
scripts/Makefile.build:252: ./drivers/net/ethernet/freescale/dpaa2/Makefile: dpmac.o is added to multiple modules: fsl-dpaa2-eth fsl-dpaa2-switch
scripts/Makefile.build:252: ./drivers/net/ethernet/freescale/enetc/Makefile: enetc_cbdr.o is added to multiple modules: fsl-enetc fsl-enetc-vf
scripts/Makefile.build:252: ./drivers/net/ethernet/freescale/enetc/Makefile: enetc_ethtool.o is added to multiple modules: fsl-enetc fsl-enetc-vf
scripts/Makefile.build:252: ./drivers/net/ethernet/freescale/enetc/Makefile: enetc.o is added to multiple modules: fsl-enetc fsl-enetc-vf
scripts/Makefile.build:252: ./drivers/net/ethernet/hisilicon/hns3/Makefile: hns3_common/hclge_comm_cmd.o is added to multiple modules: hclge hclgevf
scripts/Makefile.build:252: ./drivers/net/ethernet/hisilicon/hns3/Makefile: hns3_common/hclge_comm_rss.o is added to multiple modules: hclge hclgevf
scripts/Makefile.build:252: ./drivers/net/ethernet/hisilicon/hns3/Makefile: hns3_common/hclge_comm_tqp_stats.o is added to multiple modules: hclge hclgevf
scripts/Makefile.build:252: ./drivers/net/ethernet/marvell/octeontx2/nic/Makefile: otx2_dcbnl.o is added to multiple modules: rvu_nicpf rvu_nicvf
scripts/Makefile.build:252: ./drivers/net/ethernet/marvell/octeontx2/nic/Makefile: otx2_devlink.o is added to multiple modules: rvu_nicpf rvu_nicvf
scripts/Makefile.build:252: ./drivers/net/ethernet/ti/Makefile: cpsw_ale.o is added to multiple modules: keystone_netcp keystone_netcp_ethss ti_cpsw ti_cpsw_new
scripts/Makefile.build:252: ./drivers/net/ethernet/ti/Makefile: cpsw_ethtool.o is added to multiple modules: ti_cpsw ti_cpsw_new
scripts/Makefile.build:252: ./drivers/net/ethernet/ti/Makefile: cpsw_priv.o is added to multiple modules: ti_cpsw ti_cpsw_new
scripts/Makefile.build:252: ./drivers/net/ethernet/ti/Makefile: cpsw_sl.o is added to multiple modules: ti_cpsw ti_cpsw_new
scripts/Makefile.build:252: ./drivers/net/ethernet/ti/Makefile: davinci_cpdma.o is added to multiple modules: ti_cpsw ti_cpsw_new ti_davinci_emac
scripts/Makefile.build:252: ./drivers/platform/x86/intel/int3472/Makefile: common.o is added to multiple modules: intel_skl_int3472_discrete intel_skl_int3472_tps68470
scripts/Makefile.build:252: ./sound/soc/codecs/Makefile: wcd-clsh-v2.o is added to multiple modules: snd-soc-wcd9335 snd-soc-wcd934x snd-soc-wcd938x

Once all the warnings are fixed, it can become an error irrespective of
W= option.

Signed-off-by: Masahiro Yamada <[email protected]>
---

scripts/Makefile.build | 6 ++++++
1 file changed, 6 insertions(+)

diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index 37cf88d076e8..799df12b53f3 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -222,6 +222,10 @@ endif

cmd_check_local_export = $(srctree)/scripts/check-local-export $@

+ifneq ($(findstring 1, $(KBUILD_EXTRA_WARN)),)
+cmd_warn_shared_object = $(if $(word 2, $(modname-multi)),$(warning $(kbuild-file): $*.o is added to multiple modules: $(modname-multi)))
+endif
+
define rule_cc_o_c
$(call cmd_and_fixdep,cc_o_c)
$(call cmd,gen_ksymdeps)
@@ -231,6 +235,7 @@ define rule_cc_o_c
$(call cmd,gen_objtooldep)
$(call cmd,gen_symversions_c)
$(call cmd,record_mcount)
+ $(call cmd,warn_shared_object)
endef

define rule_as_o_S
@@ -239,6 +244,7 @@ define rule_as_o_S
$(call cmd,check_local_export)
$(call cmd,gen_objtooldep)
$(call cmd,gen_symversions_S)
+ $(call cmd,warn_shared_object)
endef

# Built-in and composite module parts
--
2.34.1


2022-11-13 12:21:48

by Masahiro Yamada

[permalink] [raw]
Subject: [PATCH 3/3] [pending] kbuild: make forbid sharing objects among multiple modules

*** This patch is applicable after all the warnings are fixed.
*** It may take a couple of development cycles.

Now that all the warnings are fixed, the warning can be promoted to
an error.

Simplify the modules.builtin rule because modname always consists of
a single word.

Signed-off-by: Masahiro Yamada <[email protected]>
---

scripts/Makefile.build | 6 ------
scripts/Makefile.lib | 8 ++++----
scripts/Makefile.vmlinux_o | 5 +----
3 files changed, 5 insertions(+), 14 deletions(-)

diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index 799df12b53f3..37cf88d076e8 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -222,10 +222,6 @@ endif

cmd_check_local_export = $(srctree)/scripts/check-local-export $@

-ifneq ($(findstring 1, $(KBUILD_EXTRA_WARN)),)
-cmd_warn_shared_object = $(if $(word 2, $(modname-multi)),$(warning $(kbuild-file): $*.o is added to multiple modules: $(modname-multi)))
-endif
-
define rule_cc_o_c
$(call cmd_and_fixdep,cc_o_c)
$(call cmd,gen_ksymdeps)
@@ -235,7 +231,6 @@ define rule_cc_o_c
$(call cmd,gen_objtooldep)
$(call cmd,gen_symversions_c)
$(call cmd,record_mcount)
- $(call cmd,warn_shared_object)
endef

define rule_as_o_S
@@ -244,7 +239,6 @@ define rule_as_o_S
$(call cmd,check_local_export)
$(call cmd,gen_objtooldep)
$(call cmd,gen_symversions_S)
- $(call cmd,warn_shared_object)
endef

# Built-in and composite module parts
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index 3aa384cec76b..d73bfe0fabda 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -106,13 +106,13 @@ endif

# Finds the multi-part object the current object will be linked into.
# If the object belongs to two or more multi-part objects, list them all.
-modname-multi = $(sort $(foreach m,$(multi-obj-ym),\
+__modname-multi = $(sort $(foreach m,$(multi-obj-ym),\
$(if $(filter $*.o, $(call suffix-search, $m, .o, -objs -y -m)),$(m:.o=))))

-__modname = $(or $(modname-multi),$(basetarget))
+modname-multi = $(if $(word 2, $(__modname-multi)), $(error $(kbuild-file): $*.o is added to multiple modules: $(__modname-multi)))$(__modname-multi)

-modname = $(subst $(space),:,$(__modname))
-modfile = $(addprefix $(obj)/,$(__modname))
+modname = $(or $(modname-multi),$(basetarget))
+modfile = $(addprefix $(obj)/,$(modname))

# target with $(obj)/ and its suffix stripped
target-stem = $(basename $(patsubst $(obj)/%,%,$@))
diff --git a/scripts/Makefile.vmlinux_o b/scripts/Makefile.vmlinux_o
index 0edfdb40364b..2faf4960aa7a 100644
--- a/scripts/Makefile.vmlinux_o
+++ b/scripts/Makefile.vmlinux_o
@@ -74,13 +74,10 @@ modules.builtin.modinfo: vmlinux.o FORCE
# module.builtin
# ---------------------------------------------------------------------------

-# The second line aids cases where multiple modules share the same object.
-
quiet_cmd_modules_builtin = GEN $@
cmd_modules_builtin = \
tr '\0' '\n' < $< | \
- sed -n 's/^[[:alnum:]:_]*\.file=//p' | \
- tr ' ' '\n' | uniq | sed -e 's:^:kernel/:' -e 's/$$/.ko/' > $@
+ sed -n 's@^[[:alnum:]:_]*\.file=\(.*\)@kernel/\1.ko@p' | uniq > $@

targets += modules.builtin
modules.builtin: modules.builtin.modinfo FORCE
--
2.34.1


2022-11-13 12:38:09

by Masahiro Yamada

[permalink] [raw]
Subject: Re: [PATCH 2/3] kbuild: warn objects shared among multiple modules

On Sun, Nov 13, 2022 at 8:15 PM Masahiro Yamada <[email protected]> wrote:
>
> If an object is shared among multiple modules, some of them are
> configured as 'm', but the others as 'y', the shared object is
> linked to the modules and vmlinux. This is a potential issue because
> the expected CFLAGS are different between modules and builtins.
>
> Commit 637a642f5ca5 ("zstd: Fixing mixed module-builtin objects")
> reported that this could be even more fatal in some cases such as
> Clang LTO.
>
> That commit fixed lib/zlib/zstd_{compress,decompress}, but there are
> still more instances of breakage.
>
> This commit adds a W=1 warning for shared objects, so that the kbuild
> test robot, which provides build tests with W=1, will avoid a new
> breakage slipping in.
>
> Quick compile tests on v6.1-rc4 detected the following:
>

Also

scripts/Makefile.build:252: ./drivers/block/rnbd/Makefile:
rnbd-common.o is added to multiple modules: rnbd-client rnbd-server



> scripts/Makefile.build:252: ./drivers/crypto/marvell/octeontx2/Makefile: cn10k_cpt.o is added to multiple modules: rvu_cptpf rvu_cptvf
> scripts/Makefile.build:252: ./drivers/crypto/marvell/octeontx2/Makefile: otx2_cptlf.o is added to multiple modules: rvu_cptpf rvu_cptvf
> scripts/Makefile.build:252: ./drivers/crypto/marvell/octeontx2/Makefile: otx2_cpt_mbox_common.o is added to multiple modules: rvu_cptpf rvu_cptvf
> scripts/Makefile.build:252: ./drivers/edac/Makefile: skx_common.o is added to multiple modules: i10nm_edac skx_edac
> scripts/Makefile.build:252: ./drivers/gpu/drm/bridge/imx/Makefile: imx-ldb-helper.o is added to multiple modules: imx8qm-ldb imx8qxp-ldb
> scripts/Makefile.build:252: ./drivers/mfd/Makefile: rsmu_core.o is added to multiple modules: rsmu-i2c rsmu-spi
> scripts/Makefile.build:252: ./drivers/mtd/tests/Makefile: mtd_test.o is added to multiple modules: mtd_nandbiterrs mtd_oobtest mtd_pagetest mtd_readtest mtd_speedtest mtd_stresstest mtd_subpagetest mtd_torturetest
> scripts/Makefile.build:252: ./drivers/net/dsa/ocelot/Makefile: felix.o is added to multiple modules: mscc_felix mscc_seville
> scripts/Makefile.build:252: ./drivers/net/ethernet/cavium/liquidio/Makefile: cn23xx_pf_device.o is added to multiple modules: liquidio liquidio_vf
> scripts/Makefile.build:252: ./drivers/net/ethernet/cavium/liquidio/Makefile: cn23xx_vf_device.o is added to multiple modules: liquidio liquidio_vf
> scripts/Makefile.build:252: ./drivers/net/ethernet/cavium/liquidio/Makefile: cn66xx_device.o is added to multiple modules: liquidio liquidio_vf
> scripts/Makefile.build:252: ./drivers/net/ethernet/cavium/liquidio/Makefile: cn68xx_device.o is added to multiple modules: liquidio liquidio_vf
> scripts/Makefile.build:252: ./drivers/net/ethernet/cavium/liquidio/Makefile: lio_core.o is added to multiple modules: liquidio liquidio_vf
> scripts/Makefile.build:252: ./drivers/net/ethernet/cavium/liquidio/Makefile: lio_ethtool.o is added to multiple modules: liquidio liquidio_vf
> scripts/Makefile.build:252: ./drivers/net/ethernet/cavium/liquidio/Makefile: octeon_device.o is added to multiple modules: liquidio liquidio_vf
> scripts/Makefile.build:252: ./drivers/net/ethernet/cavium/liquidio/Makefile: octeon_droq.o is added to multiple modules: liquidio liquidio_vf
> scripts/Makefile.build:252: ./drivers/net/ethernet/cavium/liquidio/Makefile: octeon_mailbox.o is added to multiple modules: liquidio liquidio_vf
> scripts/Makefile.build:252: ./drivers/net/ethernet/cavium/liquidio/Makefile: octeon_mem_ops.o is added to multiple modules: liquidio liquidio_vf
> scripts/Makefile.build:252: ./drivers/net/ethernet/cavium/liquidio/Makefile: octeon_nic.o is added to multiple modules: liquidio liquidio_vf
> scripts/Makefile.build:252: ./drivers/net/ethernet/cavium/liquidio/Makefile: request_manager.o is added to multiple modules: liquidio liquidio_vf
> scripts/Makefile.build:252: ./drivers/net/ethernet/cavium/liquidio/Makefile: response_manager.o is added to multiple modules: liquidio liquidio_vf
> scripts/Makefile.build:252: ./drivers/net/ethernet/freescale/dpaa2/Makefile: dpaa2-mac.o is added to multiple modules: fsl-dpaa2-eth fsl-dpaa2-switch
> scripts/Makefile.build:252: ./drivers/net/ethernet/freescale/dpaa2/Makefile: dpmac.o is added to multiple modules: fsl-dpaa2-eth fsl-dpaa2-switch
> scripts/Makefile.build:252: ./drivers/net/ethernet/freescale/enetc/Makefile: enetc_cbdr.o is added to multiple modules: fsl-enetc fsl-enetc-vf
> scripts/Makefile.build:252: ./drivers/net/ethernet/freescale/enetc/Makefile: enetc_ethtool.o is added to multiple modules: fsl-enetc fsl-enetc-vf
> scripts/Makefile.build:252: ./drivers/net/ethernet/freescale/enetc/Makefile: enetc.o is added to multiple modules: fsl-enetc fsl-enetc-vf
> scripts/Makefile.build:252: ./drivers/net/ethernet/hisilicon/hns3/Makefile: hns3_common/hclge_comm_cmd.o is added to multiple modules: hclge hclgevf
> scripts/Makefile.build:252: ./drivers/net/ethernet/hisilicon/hns3/Makefile: hns3_common/hclge_comm_rss.o is added to multiple modules: hclge hclgevf
> scripts/Makefile.build:252: ./drivers/net/ethernet/hisilicon/hns3/Makefile: hns3_common/hclge_comm_tqp_stats.o is added to multiple modules: hclge hclgevf
> scripts/Makefile.build:252: ./drivers/net/ethernet/marvell/octeontx2/nic/Makefile: otx2_dcbnl.o is added to multiple modules: rvu_nicpf rvu_nicvf
> scripts/Makefile.build:252: ./drivers/net/ethernet/marvell/octeontx2/nic/Makefile: otx2_devlink.o is added to multiple modules: rvu_nicpf rvu_nicvf
> scripts/Makefile.build:252: ./drivers/net/ethernet/ti/Makefile: cpsw_ale.o is added to multiple modules: keystone_netcp keystone_netcp_ethss ti_cpsw ti_cpsw_new
> scripts/Makefile.build:252: ./drivers/net/ethernet/ti/Makefile: cpsw_ethtool.o is added to multiple modules: ti_cpsw ti_cpsw_new
> scripts/Makefile.build:252: ./drivers/net/ethernet/ti/Makefile: cpsw_priv.o is added to multiple modules: ti_cpsw ti_cpsw_new
> scripts/Makefile.build:252: ./drivers/net/ethernet/ti/Makefile: cpsw_sl.o is added to multiple modules: ti_cpsw ti_cpsw_new
> scripts/Makefile.build:252: ./drivers/net/ethernet/ti/Makefile: davinci_cpdma.o is added to multiple modules: ti_cpsw ti_cpsw_new ti_davinci_emac
> scripts/Makefile.build:252: ./drivers/platform/x86/intel/int3472/Makefile: common.o is added to multiple modules: intel_skl_int3472_discrete intel_skl_int3472_tps68470
> scripts/Makefile.build:252: ./sound/soc/codecs/Makefile: wcd-clsh-v2.o is added to multiple modules: snd-soc-wcd9335 snd-soc-wcd934x snd-soc-wcd938x
>
> Once all the warnings are fixed, it can become an error irrespective of
> W= option.
>
> Signed-off-by: Masahiro Yamada <[email protected]>
> ---
>
> scripts/Makefile.build | 6 ++++++
> 1 file changed, 6 insertions(+)
>
> diff --git a/scripts/Makefile.build b/scripts/Makefile.build
> index 37cf88d076e8..799df12b53f3 100644
> --- a/scripts/Makefile.build
> +++ b/scripts/Makefile.build
> @@ -222,6 +222,10 @@ endif
>
> cmd_check_local_export = $(srctree)/scripts/check-local-export $@
>
> +ifneq ($(findstring 1, $(KBUILD_EXTRA_WARN)),)
> +cmd_warn_shared_object = $(if $(word 2, $(modname-multi)),$(warning $(kbuild-file): $*.o is added to multiple modules: $(modname-multi)))
> +endif
> +
> define rule_cc_o_c
> $(call cmd_and_fixdep,cc_o_c)
> $(call cmd,gen_ksymdeps)
> @@ -231,6 +235,7 @@ define rule_cc_o_c
> $(call cmd,gen_objtooldep)
> $(call cmd,gen_symversions_c)
> $(call cmd,record_mcount)
> + $(call cmd,warn_shared_object)
> endef
>
> define rule_as_o_S
> @@ -239,6 +244,7 @@ define rule_as_o_S
> $(call cmd,check_local_export)
> $(call cmd,gen_objtooldep)
> $(call cmd,gen_symversions_S)
> + $(call cmd,warn_shared_object)
> endef
>
> # Built-in and composite module parts
> --
> 2.34.1
>


--
Best Regards
Masahiro Yamada

2022-11-17 22:16:06

by Nicolas Schier

[permalink] [raw]
Subject: Re: [PATCH 1/3] kbuild: add kbuild-file macro

On Sun, 13 Nov 2022 20:15:23 +0900 Masahiro Yamada wrote:
> While building, installing, cleaning, Kbuild visits sub-directories
> and includes 'Kbuild' or 'Makefile' that exists there.
>
> Add 'kbuild-file' macro, and reuse it from scripts/Makefie.*
>
> Signed-off-by: Masahiro Yamada <[email protected]>
> ---
>
> scripts/Kbuild.include | 5 +++++
> scripts/Makefile.asm-generic | 6 +++---
> scripts/Makefile.build | 6 +-----
> scripts/Makefile.clean | 5 +----
> scripts/Makefile.dtbinst | 2 +-
> 5 files changed, 11 insertions(+), 13 deletions(-)
>
> diff --git a/scripts/Kbuild.include b/scripts/Kbuild.include
> index 2bc08ace38a3..cbe28744637b 100644
> --- a/scripts/Kbuild.include
> +++ b/scripts/Kbuild.include
> @@ -40,6 +40,11 @@ escsq = $(subst $(squote),'\$(squote)',$1)
> # Quote a string to pass it to C files. foo => '"foo"'
> stringify = $(squote)$(quote)$1$(quote)$(squote)
>
> +###
> +# The path to Kbuild or Makefile. Kbuild has precedence over Makefile.
> +kbuild-dir = $(if $(filter /%,$(src)),$(src),$(srctree)/$(src))
> +kbuild-file = $(or $(wildcard $(kbuild-dir)/Kbuild),$(kbuild-dir)/Makefile)
> +
> ###
> # Easy method for doing a status message
> kecho := :
> diff --git a/scripts/Makefile.asm-generic b/scripts/Makefile.asm-generic
> index 1d501c57f9ef..8d01b37b7677 100644
> --- a/scripts/Makefile.asm-generic
> +++ b/scripts/Makefile.asm-generic
> @@ -10,15 +10,15 @@ PHONY := all
> all:
>
> src := $(subst /generated,,$(obj))
> --include $(src)/Kbuild
> +
> +include $(srctree)/scripts/Kbuild.include
> +-include $(kbuild-file)
>
> # $(generic)/Kbuild lists mandatory-y. Exclude um since it is a special case.
> ifneq ($(SRCARCH),um)
> include $(srctree)/$(generic)/Kbuild
> endif
>
> -include $(srctree)/scripts/Kbuild.include
> -
> redundant := $(filter $(mandatory-y) $(generated-y), $(generic-y))
> redundant += $(foreach f, $(generic-y), $(if $(wildcard $(srctree)/$(src)/$(f)),$(f)))
> redundant := $(sort $(redundant))
> diff --git a/scripts/Makefile.build b/scripts/Makefile.build
> index 41f3602fc8de..37cf88d076e8 100644
> --- a/scripts/Makefile.build
> +++ b/scripts/Makefile.build
> @@ -38,11 +38,7 @@ subdir-ccflags-y :=
>
> include $(srctree)/scripts/Kbuild.include
> include $(srctree)/scripts/Makefile.compiler
> -
> -# The filename Kbuild has precedence over Makefile
> -kbuild-dir := $(if $(filter /%,$(src)),$(src),$(srctree)/$(src))
> -include $(or $(wildcard $(kbuild-dir)/Kbuild),$(kbuild-dir)/Makefile)
> -
> +include $(kbuild-file)
> include $(srctree)/scripts/Makefile.lib
>
> # Do not include hostprogs rules unless needed.
> diff --git a/scripts/Makefile.clean b/scripts/Makefile.clean
> index 878cec648959..3649900696dd 100644
> --- a/scripts/Makefile.clean
> +++ b/scripts/Makefile.clean
> @@ -9,10 +9,7 @@ PHONY := __clean
> __clean:
>
> include $(srctree)/scripts/Kbuild.include
> -
> -# The filename Kbuild has precedence over Makefile
> -kbuild-dir := $(if $(filter /%,$(src)),$(src),$(srctree)/$(src))
> -include $(or $(wildcard $(kbuild-dir)/Kbuild),$(kbuild-dir)/Makefile)
> +include $(kbuild-file)
>
> # Figure out what we need to build from the various variables
> # ==========================================================================
> diff --git a/scripts/Makefile.dtbinst b/scripts/Makefile.dtbinst
> index 190d781e84f4..2ab936e4179d 100644
> --- a/scripts/Makefile.dtbinst
> +++ b/scripts/Makefile.dtbinst
> @@ -15,7 +15,7 @@ __dtbs_install:
>
> include include/config/auto.conf
> include $(srctree)/scripts/Kbuild.include
> -include $(src)/Makefile
> +include $(kbuild-file)
>
> dtbs := $(addprefix $(dst)/, $(dtb-y) $(if $(CONFIG_OF_ALL_DTBS),$(dtb-)))
> subdirs := $(addprefix $(obj)/, $(subdir-y) $(subdir-m))
> --
> 2.34.1

Reviewed-by: Nicolas Schier <[email protected]>

What do you think about using $(kbuild-file) also in Makefile.modpost?


diff --git a/scripts/Makefile.modpost b/scripts/Makefile.modpost
--- a/scripts/Makefile.modpost
+++ b/scripts/Makefile.modpost
@@ -93,7 +93,7 @@ obj := $(KBUILD_EXTMOD)
src := $(obj)

# Include the module's Makefile to find KBUILD_EXTRA_SYMBOLS
-include $(or $(wildcard $(src)/Kbuild), $(src)/Makefile)
+include $(kbuild-file)

module.symvers-if-present := $(wildcard Module.symvers)
output-symdump := $(KBUILD_EXTMOD)/Module.symvers


Kind regards,
Nicolas

--
epost|xmpp: [email protected] irc://oftc.net/nsc
↳ gpg: 18ed 52db e34f 860e e9fb c82b 7d97 0932 55a0 ce7f
-- frykten for herren er opphav til kunnskap --


Attachments:
(No filename) (4.58 kB)
signature.asc (849.00 B)
Download all attachments

2022-11-18 01:31:39

by Masahiro Yamada

[permalink] [raw]
Subject: Re: [PATCH 1/3] kbuild: add kbuild-file macro

On Fri, Nov 18, 2022 at 5:47 AM Nicolas Schier <[email protected]> wrote:
>
> On Sun, 13 Nov 2022 20:15:23 +0900 Masahiro Yamada wrote:
> > While building, installing, cleaning, Kbuild visits sub-directories
> > and includes 'Kbuild' or 'Makefile' that exists there.
> >
> > Add 'kbuild-file' macro, and reuse it from scripts/Makefie.*
> >
> > Signed-off-by: Masahiro Yamada <[email protected]>
> > ---
> >
> > scripts/Kbuild.include | 5 +++++
> > scripts/Makefile.asm-generic | 6 +++---
> > scripts/Makefile.build | 6 +-----
> > scripts/Makefile.clean | 5 +----
> > scripts/Makefile.dtbinst | 2 +-
> > 5 files changed, 11 insertions(+), 13 deletions(-)
> >
> > diff --git a/scripts/Kbuild.include b/scripts/Kbuild.include
> > index 2bc08ace38a3..cbe28744637b 100644
> > --- a/scripts/Kbuild.include
> > +++ b/scripts/Kbuild.include
> > @@ -40,6 +40,11 @@ escsq = $(subst $(squote),'\$(squote)',$1)
> > # Quote a string to pass it to C files. foo => '"foo"'
> > stringify = $(squote)$(quote)$1$(quote)$(squote)
> >
> > +###
> > +# The path to Kbuild or Makefile. Kbuild has precedence over Makefile.
> > +kbuild-dir = $(if $(filter /%,$(src)),$(src),$(srctree)/$(src))
> > +kbuild-file = $(or $(wildcard $(kbuild-dir)/Kbuild),$(kbuild-dir)/Makefile)
> > +
> > ###
> > # Easy method for doing a status message
> > kecho := :
> > diff --git a/scripts/Makefile.asm-generic b/scripts/Makefile.asm-generic
> > index 1d501c57f9ef..8d01b37b7677 100644
> > --- a/scripts/Makefile.asm-generic
> > +++ b/scripts/Makefile.asm-generic
> > @@ -10,15 +10,15 @@ PHONY := all
> > all:
> >
> > src := $(subst /generated,,$(obj))
> > --include $(src)/Kbuild
> > +
> > +include $(srctree)/scripts/Kbuild.include
> > +-include $(kbuild-file)
> >
> > # $(generic)/Kbuild lists mandatory-y. Exclude um since it is a special case.
> > ifneq ($(SRCARCH),um)
> > include $(srctree)/$(generic)/Kbuild
> > endif
> >
> > -include $(srctree)/scripts/Kbuild.include
> > -
> > redundant := $(filter $(mandatory-y) $(generated-y), $(generic-y))
> > redundant += $(foreach f, $(generic-y), $(if $(wildcard $(srctree)/$(src)/$(f)),$(f)))
> > redundant := $(sort $(redundant))
> > diff --git a/scripts/Makefile.build b/scripts/Makefile.build
> > index 41f3602fc8de..37cf88d076e8 100644
> > --- a/scripts/Makefile.build
> > +++ b/scripts/Makefile.build
> > @@ -38,11 +38,7 @@ subdir-ccflags-y :=
> >
> > include $(srctree)/scripts/Kbuild.include
> > include $(srctree)/scripts/Makefile.compiler
> > -
> > -# The filename Kbuild has precedence over Makefile
> > -kbuild-dir := $(if $(filter /%,$(src)),$(src),$(srctree)/$(src))
> > -include $(or $(wildcard $(kbuild-dir)/Kbuild),$(kbuild-dir)/Makefile)
> > -
> > +include $(kbuild-file)
> > include $(srctree)/scripts/Makefile.lib
> >
> > # Do not include hostprogs rules unless needed.
> > diff --git a/scripts/Makefile.clean b/scripts/Makefile.clean
> > index 878cec648959..3649900696dd 100644
> > --- a/scripts/Makefile.clean
> > +++ b/scripts/Makefile.clean
> > @@ -9,10 +9,7 @@ PHONY := __clean
> > __clean:
> >
> > include $(srctree)/scripts/Kbuild.include
> > -
> > -# The filename Kbuild has precedence over Makefile
> > -kbuild-dir := $(if $(filter /%,$(src)),$(src),$(srctree)/$(src))
> > -include $(or $(wildcard $(kbuild-dir)/Kbuild),$(kbuild-dir)/Makefile)
> > +include $(kbuild-file)
> >
> > # Figure out what we need to build from the various variables
> > # ==========================================================================
> > diff --git a/scripts/Makefile.dtbinst b/scripts/Makefile.dtbinst
> > index 190d781e84f4..2ab936e4179d 100644
> > --- a/scripts/Makefile.dtbinst
> > +++ b/scripts/Makefile.dtbinst
> > @@ -15,7 +15,7 @@ __dtbs_install:
> >
> > include include/config/auto.conf
> > include $(srctree)/scripts/Kbuild.include
> > -include $(src)/Makefile
> > +include $(kbuild-file)
> >
> > dtbs := $(addprefix $(dst)/, $(dtb-y) $(if $(CONFIG_OF_ALL_DTBS),$(dtb-)))
> > subdirs := $(addprefix $(obj)/, $(subdir-y) $(subdir-m))
> > --
> > 2.34.1
>
> Reviewed-by: Nicolas Schier <[email protected]>
>
> What do you think about using $(kbuild-file) also in Makefile.modpost?


Yes, we can replace it as well.
I will fix it.

Thanks for the close review, as always!






>
> diff --git a/scripts/Makefile.modpost b/scripts/Makefile.modpost
> --- a/scripts/Makefile.modpost
> +++ b/scripts/Makefile.modpost
> @@ -93,7 +93,7 @@ obj := $(KBUILD_EXTMOD)
> src := $(obj)
>
> # Include the module's Makefile to find KBUILD_EXTRA_SYMBOLS
> -include $(or $(wildcard $(src)/Kbuild), $(src)/Makefile)
> +include $(kbuild-file)
>
> module.symvers-if-present := $(wildcard Module.symvers)
> output-symdump := $(KBUILD_EXTMOD)/Module.symvers
>
>
> Kind regards,
> Nicolas
>
> --
> epost|xmpp: [email protected] irc://oftc.net/nsc
> ↳ gpg: 18ed 52db e34f 860e e9fb c82b 7d97 0932 55a0 ce7f
> -- frykten for herren er opphav til kunnskap --



--
Best Regards
Masahiro Yamada

2022-11-22 07:07:48

by Nicolas Schier

[permalink] [raw]
Subject: Re: [PATCH 3/3] [pending] kbuild: make forbid sharing objects among multiple modules

On Sun 13 Nov 2022 20:15:25 GMT, Masahiro Yamada wrote:
>
> *** This patch is applicable after all the warnings are fixed.
> *** It may take a couple of development cycles.
>
> Now that all the warnings are fixed, the warning can be promoted to
> an error.
>
> Simplify the modules.builtin rule because modname always consists of
> a single word.
>
> Signed-off-by: Masahiro Yamada <[email protected]>
> ---

Reviewed-by: Nicolas Schier <[email protected]>

>
> scripts/Makefile.build | 6 ------
> scripts/Makefile.lib | 8 ++++----
> scripts/Makefile.vmlinux_o | 5 +----
> 3 files changed, 5 insertions(+), 14 deletions(-)
>
> diff --git a/scripts/Makefile.build b/scripts/Makefile.build
> index 799df12b53f3..37cf88d076e8 100644
> --- a/scripts/Makefile.build
> +++ b/scripts/Makefile.build
> @@ -222,10 +222,6 @@ endif
>
> cmd_check_local_export = $(srctree)/scripts/check-local-export $@
>
> -ifneq ($(findstring 1, $(KBUILD_EXTRA_WARN)),)
> -cmd_warn_shared_object = $(if $(word 2, $(modname-multi)),$(warning $(kbuild-file): $*.o is added to multiple modules: $(modname-multi)))
> -endif
> -
> define rule_cc_o_c
> $(call cmd_and_fixdep,cc_o_c)
> $(call cmd,gen_ksymdeps)
> @@ -235,7 +231,6 @@ define rule_cc_o_c
> $(call cmd,gen_objtooldep)
> $(call cmd,gen_symversions_c)
> $(call cmd,record_mcount)
> - $(call cmd,warn_shared_object)
> endef
>
> define rule_as_o_S
> @@ -244,7 +239,6 @@ define rule_as_o_S
> $(call cmd,check_local_export)
> $(call cmd,gen_objtooldep)
> $(call cmd,gen_symversions_S)
> - $(call cmd,warn_shared_object)
> endef
>
> # Built-in and composite module parts
> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> index 3aa384cec76b..d73bfe0fabda 100644
> --- a/scripts/Makefile.lib
> +++ b/scripts/Makefile.lib
> @@ -106,13 +106,13 @@ endif
>
> # Finds the multi-part object the current object will be linked into.
> # If the object belongs to two or more multi-part objects, list them all.
> -modname-multi = $(sort $(foreach m,$(multi-obj-ym),\
> +__modname-multi = $(sort $(foreach m,$(multi-obj-ym),\
> $(if $(filter $*.o, $(call suffix-search, $m, .o, -objs -y -m)),$(m:.o=))))
>
> -__modname = $(or $(modname-multi),$(basetarget))
> +modname-multi = $(if $(word 2, $(__modname-multi)), $(error $(kbuild-file): $*.o is added to multiple modules: $(__modname-multi)))$(__modname-multi)
>
> -modname = $(subst $(space),:,$(__modname))
> -modfile = $(addprefix $(obj)/,$(__modname))
> +modname = $(or $(modname-multi),$(basetarget))
> +modfile = $(addprefix $(obj)/,$(modname))
>
> # target with $(obj)/ and its suffix stripped
> target-stem = $(basename $(patsubst $(obj)/%,%,$@))
> diff --git a/scripts/Makefile.vmlinux_o b/scripts/Makefile.vmlinux_o
> index 0edfdb40364b..2faf4960aa7a 100644
> --- a/scripts/Makefile.vmlinux_o
> +++ b/scripts/Makefile.vmlinux_o
> @@ -74,13 +74,10 @@ modules.builtin.modinfo: vmlinux.o FORCE
> # module.builtin
> # ---------------------------------------------------------------------------
>
> -# The second line aids cases where multiple modules share the same object.
> -
> quiet_cmd_modules_builtin = GEN $@
> cmd_modules_builtin = \
> tr '\0' '\n' < $< | \
> - sed -n 's/^[[:alnum:]:_]*\.file=//p' | \
> - tr ' ' '\n' | uniq | sed -e 's:^:kernel/:' -e 's/$$/.ko/' > $@
> + sed -n 's@^[[:alnum:]:_]*\.file=\(.*\)@kernel/\1.ko@p' | uniq > $@
>
> targets += modules.builtin
> modules.builtin: modules.builtin.modinfo FORCE
> --
> 2.34.1

--
epost|xmpp: [email protected] irc://oftc.net/nsc
↳ gpg: 18ed 52db e34f 860e e9fb c82b 7d97 0932 55a0 ce7f
-- frykten for herren er opphav til kunnskap --


Attachments:
(No filename) (3.70 kB)
signature.asc (849.00 B)
Download all attachments