2019-11-18 04:55:05

by Masahiro Yamada

[permalink] [raw]
Subject: [PATCH] kbuild: make single target builds even faster

Commit 2dffd23f81a3 ("kbuild: make single target builds much faster")
made the situation much better.

To improve it even more, apply the similar idea to the top Makefile.
Trim unrelated directories from build-dirs.

The single build code must be moved above the 'descend' target.

Signed-off-by: Masahiro Yamada <[email protected]>
---

Makefile | 90 +++++++++++++++++++++++++++++---------------------------
1 file changed, 47 insertions(+), 43 deletions(-)

diff --git a/Makefile b/Makefile
index 99470c19c158..345b8f2e2a9e 100644
--- a/Makefile
+++ b/Makefile
@@ -1633,6 +1633,50 @@ help:
PHONY += prepare
endif # KBUILD_EXTMOD

+# Single targets
+# ---------------------------------------------------------------------------
+# To build individual files in subdirectories, you can do like this:
+#
+# make foo/bar/baz.s
+#
+# The supported suffixes for single-target are listed in 'single-targets'
+#
+# To build only under specific subdirectories, you can do like this:
+#
+# make foo/bar/baz/
+
+ifdef single-build
+
+# .ko is special because modpost is needed
+single-ko := $(sort $(filter %.ko, $(MAKECMDGOALS)))
+single-no-ko := $(sort $(patsubst %.ko,%.mod, $(MAKECMDGOALS)))
+
+$(single-ko): single_modpost
+ @:
+$(single-no-ko): descend
+ @:
+
+ifeq ($(KBUILD_EXTMOD),)
+# For the single build of in-tree modules, use a temporary file to avoid
+# the situation of modules_install installing an invalid modules.order.
+MODORDER := .modules.tmp
+endif
+
+PHONY += single_modpost
+single_modpost: $(single-no-ko)
+ $(Q){ $(foreach m, $(single-ko), echo $(extmod-prefix)$m;) } > $(MODORDER)
+ $(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modpost
+
+KBUILD_MODULES := 1
+
+export KBUILD_SINGLE_TARGETS := $(addprefix $(extmod-prefix), $(single-no-ko))
+
+# trim unrelated directories
+build-dirs := $(foreach d, $(build-dirs), \
+ $(if $(filter $(d)/%, $(KBUILD_SINGLE_TARGETS)), $(d)))
+
+endif
+
# Handle descending into subdirectories listed in $(build-dirs)
# Preset locale variables to speed up the build process. Limit locale
# tweaks to this spot to avoid wrong language settings when running
@@ -1641,7 +1685,9 @@ endif # KBUILD_EXTMOD
PHONY += descend $(build-dirs)
descend: $(build-dirs)
$(build-dirs): prepare
- $(Q)$(MAKE) $(build)=$@ single-build=$(single-build) need-builtin=1 need-modorder=1
+ $(Q)$(MAKE) $(build)=$@ \
+ single-build=$(if $(filter-out $@/, $(single-no-ko)),1) \
+ need-builtin=1 need-modorder=1

clean-dirs := $(addprefix _clean_, $(clean-dirs))
PHONY += $(clean-dirs) clean
@@ -1744,48 +1790,6 @@ tools/%: FORCE
$(Q)mkdir -p $(objtree)/tools
$(Q)$(MAKE) LDFLAGS= MAKEFLAGS="$(tools_silent) $(filter --j% -j,$(MAKEFLAGS))" O=$(abspath $(objtree)) subdir=tools -C $(srctree)/tools/ $*

-# Single targets
-# ---------------------------------------------------------------------------
-# To build individual files in subdirectories, you can do like this:
-#
-# make foo/bar/baz.s
-#
-# The supported suffixes for single-target are listed in 'single-targets'
-#
-# To build only under specific subdirectories, you can do like this:
-#
-# make foo/bar/baz/
-
-ifdef single-build
-
-# .ko is special because modpost is needed
-single-ko := $(sort $(filter %.ko, $(MAKECMDGOALS)))
-single-no-ko := $(sort $(patsubst %.ko,%.mod, $(MAKECMDGOALS)))
-
-$(single-ko): single_modpost
- @:
-$(single-no-ko): descend
- @:
-
-ifeq ($(KBUILD_EXTMOD),)
-# For the single build of in-tree modules, use a temporary file to avoid
-# the situation of modules_install installing an invalid modules.order.
-MODORDER := .modules.tmp
-endif
-
-PHONY += single_modpost
-single_modpost: $(single-no-ko)
- $(Q){ $(foreach m, $(single-ko), echo $(extmod-prefix)$m;) } > $(MODORDER)
- $(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modpost
-
-KBUILD_MODULES := 1
-
-export KBUILD_SINGLE_TARGETS := $(addprefix $(extmod-prefix), $(single-no-ko))
-
-single-build = $(if $(filter-out $@/, $(single-no-ko)),1)
-
-endif
-
# FIXME Should go into a make.lib or something
# ===========================================================================

--
2.17.1


2019-11-18 14:30:11

by Jens Axboe

[permalink] [raw]
Subject: Re: [PATCH] kbuild: make single target builds even faster

On 11/17/19 9:52 PM, Masahiro Yamada wrote:
> Commit 2dffd23f81a3 ("kbuild: make single target builds much faster")
> made the situation much better.
>
> To improve it even more, apply the similar idea to the top Makefile.
> Trim unrelated directories from build-dirs.
>
> The single build code must be moved above the 'descend' target.

I tested linux-next, which does improve things a bit, and this one on
top further improves it. We're now not THAT far off the situation
before these changes, that's a huge win. Thanks for working on this!

You can add:

Tested-by: Jens Axboe <[email protected]>

to the commit, if you wish.

--
Jens Axboe

2019-11-23 06:44:30

by Masahiro Yamada

[permalink] [raw]
Subject: Re: [PATCH] kbuild: make single target builds even faster

On Mon, Nov 18, 2019 at 11:26 PM Jens Axboe <[email protected]> wrote:
>
> On 11/17/19 9:52 PM, Masahiro Yamada wrote:
> > Commit 2dffd23f81a3 ("kbuild: make single target builds much faster")
> > made the situation much better.
> >
> > To improve it even more, apply the similar idea to the top Makefile.
> > Trim unrelated directories from build-dirs.
> >
> > The single build code must be moved above the 'descend' target.
>
> I tested linux-next, which does improve things a bit, and this one on
> top further improves it. We're now not THAT far off the situation
> before these changes, that's a huge win. Thanks for working on this!
>
> You can add:
>
> Tested-by: Jens Axboe <[email protected]>
>
> to the commit, if you wish.
>


Applied to linux-kbuild with Jens' Tested-by.


--
Best Regards
Masahiro Yamada