Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S965300AbbD1KFX (ORCPT ); Tue, 28 Apr 2015 06:05:23 -0400 Received: from cantor2.suse.de ([195.135.220.15]:57129 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932907AbbD1KFV (ORCPT ); Tue, 28 Apr 2015 06:05:21 -0400 Message-ID: <553F5B5F.5000709@suse.cz> Date: Tue, 28 Apr 2015 12:05:19 +0200 From: Michal Marek User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Riku Voipio , Chris J Arges CC: linux-kbuild , linux-kernel@vger.kernel.org Subject: Re: [PATCH v2] builddeb: parallelize debug module installation References: <553E4633.8080606@suse.cz> <1430153019-31596-1-git-send-email-chris.j.arges@canonical.com> In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 1992 Lines: 42 On 2015-04-28 10:57, Riku Voipio wrote: > On 27 April 2015 at 19:43, Chris J Arges wrote: >> When building the dbg package, we use a large 'for module in $(find' loop that >> can be easily parallelized by using 'find | xargs'. This patch modifies this >> loop to use the later paradigm. >> >> In addition, ensure we add '-n1 -P0' to xargs to run as many processes as >> possible. >> >> Signed-off-by: Chris J Arges >> --- >> scripts/package/builddeb | 15 ++++++++------- >> 1 file changed, 8 insertions(+), 7 deletions(-) >> >> diff --git a/scripts/package/builddeb b/scripts/package/builddeb >> index 88dbf23..538f829 100755 >> --- a/scripts/package/builddeb >> +++ b/scripts/package/builddeb >> @@ -152,16 +152,17 @@ if grep -q '^CONFIG_MODULES=y' $KCONFIG_CONFIG ; then >> rmdir "$tmpdir/lib/modules/$version" >> fi >> if [ -n "$BUILD_DEBUG" ] ; then >> - for module in $(find $tmpdir/lib/modules/ -name *.ko -printf '%P\n'); do >> - module=lib/modules/$module >> - mkdir -p $(dirname $dbg_dir/usr/lib/debug/$module) >> + find $tmpdir/lib/modules/ -name *.ko -printf '%P\n' | xargs -n1 -P0 -I {} sh -c ' > > I would go with -P`getconf _NPROCESSORS_ONLN`. There can be thousands > of modules (allmodconfig will make 4500). Yep. I was thinking about retrieving the value of the make -j argument somehow, but this is not possible and we certainly do not want to implement the jobserver protocol in shell :-). So using the number of processors is a sensible choice. What can be done is to detect whether the -j option is in MAKEFLAGS and only then use multiple instances. Michal -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/