2020-08-12 18:30:01

by Sumera Priyadarsini

Subject: [PATCH] scripts: coccicheck: Change default value for parallelism

By default, coccicheck utilizes all available threads to implement
parallelisation. However, when hyperthreading is enabled, this leads
to all threads per core being occupied resulting in longer wall-clock
times and higher power consumption. Hence, to improve performance,
modify coccicheck to use only one thread per core atmost.

In the cases where the total number of threads is more than 8 and
hyperthreading is enabled, it was observed that optimum performance
is achieved around one-fourth of the total number of cores.
Modify the script further to accommodate this use case.

Signed-off-by: Sumera Priyadarsini <[email protected]>
---
scripts/coccicheck | 9 +++++++++
1 file changed, 9 insertions(+)

diff --git a/scripts/coccicheck b/scripts/coccicheck
index e04d328210ac..dd228dcc915e 100755
--- a/scripts/coccicheck
+++ b/scripts/coccicheck
@@ -75,8 +75,17 @@ else
OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE"
fi

+ # Use only one thread per core by default if hyperthreading is enabled
+ THREADS_PER_CORE=$(lscpu | grep "Thread(s) per core: " | tr -cd [:digit:])
if [ -z "$J" ]; then
NPROC=$(getconf _NPROCESSORS_ONLN)
+ if [ $THREADS_PER_CORE -gt 1 -a $NPROC -gt 2 ] ; then
+ if [ $NPROC -gt 8 ] ; then
+ NPROC=$((NPROC/4))
+ else
+ NPROC=$((NPROC/2))
+ fi
+ fi
else
NPROC="$J"
fi
--
2.17.1

2020-08-12 20:18:39

by Julia Lawall

Subject: Re: [Cocci] [PATCH] scripts: coccicheck: Change default value for parallelism

On Wed, 12 Aug 2020, Sumera Priyadarsini wrote:

> By default, coccicheck utilizes all available threads to implement
> parallelisation. However, when hyperthreading is enabled, this leads
> to all threads per core being occupied resulting in longer wall-clock
> times and higher power consumption.

I have the feeling that the above sentence is not quite optimal.
Actually, using all of the available hardware threads would not be a bad
thing, if it was giving a benefit. The point is that it doesn't. It
makes the performance worse instead.

> Hence, to improve performance,
> modify coccicheck to use only one thread per core atmost.

"atmost" is not a word. It would be clearer to say "to use at most one
thread per core".

> In the cases where the total number of threads is more than 8 and
> hyperthreading is enabled, it was observed that optimum performance
> is achieved around one-fourth of the total number of cores.
> Modify the script further to accommodate this use case.

It would be nice to give some performance numbers and some information
about the machine used.

thanks,
julia

>
> Signed-off-by: Sumera Priyadarsini <[email protected]>
> ---
> scripts/coccicheck | 9 +++++++++
> 1 file changed, 9 insertions(+)
>
> diff --git a/scripts/coccicheck b/scripts/coccicheck
> index e04d328210ac..dd228dcc915e 100755
> --- a/scripts/coccicheck
> +++ b/scripts/coccicheck
> @@ -75,8 +75,17 @@ else
> OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE"
> fi
>
> + # Use only one thread per core by default if hyperthreading is enabled
> + THREADS_PER_CORE=$(lscpu | grep "Thread(s) per core: " | tr -cd [:digit:])
> if [ -z "$J" ]; then
> NPROC=$(getconf _NPROCESSORS_ONLN)
> + if [ $THREADS_PER_CORE -gt 1 -a $NPROC -gt 2 ] ; then
> + if [ $NPROC -gt 8 ] ; then
> + NPROC=$((NPROC/4))
> + else
> + NPROC=$((NPROC/2))
> + fi
> + fi
> else
> NPROC="$J"
> fi
> --
> 2.17.1
>
> _______________________________________________
> Cocci mailing list
> [email protected]
> https://systeme.lip6.fr/mailman/listinfo/cocci
>