These two patches should speed up the make deb-pkg build process by utilizing
the make jobserver properly, and using 'find | xargs' to parallelize debug
module installation.
I've tested with the resulting packages to ensure the symbols work as intended
by crash dumping a machine and using the dbg package symbols.
Using an Ubuntu distro conf + make olddefconfig on a 56-core machine, I can see
the following improvements (make clean && make deb-pkg -j`nproc`):
* Before
real 40m56.240s
user 182m17.292s
sys 140m18.812s
* After
real 36m22.633s
user 182m28.028s
sys 148m17.724s
Original submission: https://lkml.org/lkml/2015/4/24/617
Chris J Arges (2):
package: Makefile: ensure $MAKE can use jobserver
builddeb: parallelize debug module installation
scripts/package/Makefile | 2 +-
scripts/package/builddeb | 19 ++++++++++++-------
2 files changed, 13 insertions(+), 8 deletions(-)
--
1.9.1
When using make deb-pkg, builddeb is called without proper MAKEFLAGS due to the
script being invoked without '+'. This results in the following message when
building:
warning: jobserver unavailable: using -j1. Add `+' to parent make rule
Add the '+' so the make operations can be parallelized.
Signed-off-by: Chris J Arges <[email protected]>
---
scripts/package/Makefile | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/package/Makefile b/scripts/package/Makefile
index 99ca6e7..0dbfae7 100644
--- a/scripts/package/Makefile
+++ b/scripts/package/Makefile
@@ -88,7 +88,7 @@ quiet_cmd_builddeb = BUILDDEB
deb-pkg: FORCE
$(MAKE) KBUILD_SRC=
- $(call cmd,builddeb)
+ +$(call cmd,builddeb)
clean-dirs += $(objtree)/debian/
--
1.9.1
When building the dbg package, we use a large 'for module in $(find' loop that
can be easily parallelized by using 'find | xargs'. This patch modifies this
loop to use the later paradigm.
In addition, check if the user has requested a parallel build with make. If so,
add the appropriate flags to xargs to set MAXPROCS (-P) equal to the number of
processing units on the system.
Signed-off-by: Chris J Arges <[email protected]>
---
scripts/package/builddeb | 19 ++++++++++++-------
1 file changed, 12 insertions(+), 7 deletions(-)
diff --git a/scripts/package/builddeb b/scripts/package/builddeb
index 88dbf23..5849f0c 100755
--- a/scripts/package/builddeb
+++ b/scripts/package/builddeb
@@ -152,16 +152,21 @@ if grep -q '^CONFIG_MODULES=y' $KCONFIG_CONFIG ; then
rmdir "$tmpdir/lib/modules/$version"
fi
if [ -n "$BUILD_DEBUG" ] ; then
- for module in $(find $tmpdir/lib/modules/ -name *.ko -printf '%P\n'); do
- module=lib/modules/$module
- mkdir -p $(dirname $dbg_dir/usr/lib/debug/$module)
+ # If we've invoked make with -j, then parallelize; otherwise
+ # just use a single process.
+ procs=1
+ test "${MAKEFLAGS#*-j}" != "${MAKEFLAGS}" && procs=`getconf _NPROCESSORS_ONLN`
+ find $tmpdir/lib/modules/ -name *.ko -printf '%P\n' | xargs -n1 -P${procs} -I {} sh -c '
+ mkdir -p $(dirname '"$dbg_dir"'/usr/lib/debug/lib/modules/$1);'"
# only keep debug symbols in the debug file
- $OBJCOPY --only-keep-debug $tmpdir/$module $dbg_dir/usr/lib/debug/$module
+ $OBJCOPY --only-keep-debug $tmpdir/lib/modules/{} \
+ $dbg_dir/usr/lib/debug/lib/modules/{};
# strip original module from debug symbols
- $OBJCOPY --strip-debug $tmpdir/$module
+ $OBJCOPY --strip-debug $tmpdir/lib/modules/{};
# then add a link to those
- $OBJCOPY --add-gnu-debuglink=$dbg_dir/usr/lib/debug/$module $tmpdir/$module
- done
+ $OBJCOPY --add-gnu-debuglink=$dbg_dir/usr/lib/debug/lib/modules/{} \
+ $tmpdir/lib/modules/{};
+ " -- {}
fi
fi
--
1.9.1