2023-06-01 01:37:16

by Jialu Xu

Subject: [PATCH] scripts/tags.sh: improve compiled sources generation

Use grep instead of sed for all compiled sources generation, it is three
times more efficient.

Signed-off-by: Jialu Xu <[email protected]>
---
scripts/tags.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/tags.sh b/scripts/tags.sh
index ea31640b2671..938dba219534 100755
--- a/scripts/tags.sh
+++ b/scripts/tags.sh
@@ -98,7 +98,7 @@ all_compiled_sources()
{
echo include/generated/autoconf.h
find $ignore -name "*.cmd" -exec \
- sed -n -E 's/^source_.* (.*)/\1/p; s/^ (\S.*) \\/\1/p' {} \+ |
+ grep -Poh '(?<=^ )\S+|(?<== )\S+[^\\](?=$)' {} \+ |
awk '!a[$0]++'
} | xargs realpath -esq $([ -z "$KBUILD_ABS_SRCTREE" ] && echo --relative-to=.) |
sort -u
--
2.30.2

2023-06-01 17:41:05

by Carlos Llamas

Subject: Re: [PATCH] scripts/tags.sh: improve compiled sources generation

On Thu, Jun 01, 2023 at 09:04:02AM +0800, Jialu Xu wrote:
> Use grep instead of sed for all compiled sources generation, it is three
> times more efficient.

Although I'm not sure how you obtained a 3x efficiency it did make
things faster for me. I suppose there are several factors involved.
I used defconfig for arm64 and got the following results:
real 1m50.790s
real 1m27.907s (with patch)

Tested-by: Carlos Llamas <[email protected]>

>
> Signed-off-by: Jialu Xu <[email protected]>
> ---
> scripts/tags.sh | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/scripts/tags.sh b/scripts/tags.sh
> index ea31640b2671..938dba219534 100755
> --- a/scripts/tags.sh
> +++ b/scripts/tags.sh
> @@ -98,7 +98,7 @@ all_compiled_sources()
> {
> echo include/generated/autoconf.h
> find $ignore -name "*.cmd" -exec \
> - sed -n -E 's/^source_.* (.*)/\1/p; s/^ (\S.*) \\/\1/p' {} \+ |
> + grep -Poh '(?<=^ )\S+|(?<== )\S+[^\\](?=$)' {} \+ |
> awk '!a[$0]++'
> } | xargs realpath -esq $([ -z "$KBUILD_ABS_SRCTREE" ] && echo --relative-to=.) |
> sort -u
> --
> 2.30.2
>

2023-06-02 02:09:52

by Jialu Xu

Subject: Re: Re: [PATCH] scripts/tags.sh: improve compiled sources generation

Here is a test log of `time make COMPILED_SOURCE=1 cscope`

# git show
commit 71f226c00c8a87de4698eda7d2a25e9b602d08e2 (HEAD -> tags.sh)
Author: Jialu Xu <[email protected]>
Date: Sat May 6 16:52:13 2023 +0800

scripts/tags.sh: improve compiled sources generation

Use grep instead of sed for all compiled sources generation, it is three
times more efficient.

diff --git a/scripts/tags.sh b/scripts/tags.sh
index ea31640b26715..938dba219534a 100755
--- a/scripts/tags.sh
+++ b/scripts/tags.sh
@@ -98,7 +98,7 @@ all_compiled_sources()
{
echo include/generated/autoconf.h
find $ignore -name "*.cmd" -exec \
- sed -n -E 's/^source_.* (.*)/\1/p; s/^ (\S.*) \\/\1/p' {} \+ |
+ grep -Poh '(?<=^ )\S+|(?<== )\S+[^\\](?=$)' {} \+ |
awk '!a[$0]++'
} | xargs realpath -esq $([ -z "$KBUILD_ABS_SRCTREE" ] && echo --relative-to=.) |
sort -u
# time make COMPILED_SOURCE=1 cscope
GEN cscope

real 0m3.869s
user 0m4.254s
sys 0m0.389s
# time make COMPILED_SOURCE=1 cscope
GEN cscope

real 0m3.880s
user 0m4.202s
sys 0m0.428s
# git revert HEAD
[tags.sh 4dc58acca6240] Revert "scripts/tags.sh: improve compiled sources generation"
1 file changed, 1 insertion(+), 1 deletion(-)
# git show
commit 4dc58acca6240364a2303b9d9a4f55db159d60c4 (HEAD -> tags.sh)
Author: Jialu Xu <[email protected]>
Date: Fri Jun 2 08:06:09 2023 +0800

Revert "scripts/tags.sh: improve compiled sources generation"

This reverts commit 71f226c00c8a87de4698eda7d2a25e9b602d08e2.

diff --git a/scripts/tags.sh b/scripts/tags.sh
index 938dba219534a..ea31640b26715 100755
--- a/scripts/tags.sh
+++ b/scripts/tags.sh
@@ -98,7 +98,7 @@ all_compiled_sources()
{
echo include/generated/autoconf.h
find $ignore -name "*.cmd" -exec \
- grep -Poh '(?<=^ )\S+|(?<== )\S+[^\\](?=$)' {} \+ |
+ sed -n -E 's/^source_.* (.*)/\1/p; s/^ (\S.*) \\/\1/p' {} \+ |
awk '!a[$0]++'
} | xargs realpath -esq $([ -z "$KBUILD_ABS_SRCTREE" ] && echo --relative-to=.) |
sort -u
# time make COMPILED_SOURCE=1 cscope
GEN cscope

real 0m15.855s
user 0m16.308s
sys 0m0.536s
# time make COMPILED_SOURCE=1 cscope
GEN cscope

real 0m16.043s
user 0m16.489s
sys 0m0.493s