Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932211AbbEHN0M (ORCPT ); Fri, 8 May 2015 09:26:12 -0400 Received: from mail-wg0-f48.google.com ([74.125.82.48]:36494 "EHLO mail-wg0-f48.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932183AbbEHN0K (ORCPT ); Fri, 8 May 2015 09:26:10 -0400 Date: Fri, 8 May 2015 16:26:06 +0300 From: Alexey Dobriyan To: akpm@linux.foundation.org, mmarek@suse.cz Cc: linux-kernel@vger.kernel.org Subject: [PATCH v3] tags: much faster, parallel "make tags" Message-ID: <20150508132606.GA16613@p183.telecom.by> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2666 Lines: 93 ctags is single-threaded program. Split list of files to be tagged into equal parts, 1 part for each CPU and then merge the results. Speedup on one 2-way box I have is ~143 s => ~99 s (-31%). On another 4-way box: ~120 s => ~65 s (-46%!). Resulting "tags" files aren't byte-for-byte identical because ctags program numbers anon struct and enum declarations with "__anonNNN" symbols. If those lines are removed, "tags" file becomes byte-for-byte identical with those generated with current code. Signed-off-by: Alexey Dobriyan --- scripts/tags.sh | 36 +++++++++++++++++++++++++++++++----- 1 file changed, 31 insertions(+), 5 deletions(-) --- a/scripts/tags.sh +++ b/scripts/tags.sh @@ -152,7 +152,19 @@ dogtags() exuberant() { - all_target_sources | xargs $1 -a \ + rm -f .make-tags.* + + all_target_sources >.make-tags.src + NR_CPUS=$(getconf _NPROCESSORS_ONLN 2>/dev/null || echo 1) + NR_LINES=$(wc -l <.make-tags.src) + NR_LINES=$((($NR_LINES + $NR_CPUS - 1) / $NR_CPUS)) + + split -a 6 -d -l $NR_LINES .make-tags.src .make-tags.src. + + for i in .make-tags.src.*; do + N=$(echo $i | sed -e 's/.*\.//') + # -u: don't sort now, sort later + xargs <$i $1 -a -f .make-tags.$N -u \ -I __initdata,__exitdata,__initconst, \ -I __cpuinitdata,__initdata_memblock \ -I __refdata,__attribute,__maybe_unused,__always_unused \ @@ -211,7 +223,21 @@ exuberant() --regex-c='/DEFINE_PCI_DEVICE_TABLE\((\w*)/\1/v/' \ --regex-c='/(^\s)OFFSET\((\w*)/\2/v/' \ --regex-c='/(^\s)DEFINE\((\w*)/\2/v/' \ - --regex-c='/DEFINE_HASHTABLE\((\w*)/\1/v/' + --regex-c='/DEFINE_HASHTABLE\((\w*)/\1/v/' \ + & + done + wait + rm -f .make-tags.src .make-tags.src.* + + # write header + $1 -f $2 /dev/null + # remove headers + for i in .make-tags.*; do + sed -i -e '/^!/d' $i & + done + wait + sort .make-tags.* >>$2 + rm -f .make-tags.* all_kconfigs | xargs $1 -a \ --langdef=kconfig --language-force=kconfig \ @@ -276,7 +302,7 @@ emacs() xtags() { if $1 --version 2>&1 | grep -iq exuberant; then - exuberant $1 + exuberant $1 $2 elif $1 --version 2>&1 | grep -iq emacs; then emacs $1 else @@ -322,13 +348,13 @@ case "$1" in "tags") rm -f tags - xtags ctags + xtags ctags tags remove_structs=y ;; "TAGS") rm -f TAGS - xtags etags + xtags etags TAGS remove_structs=y ;; esac -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/