2013-06-17 23:08:29

by Kees Cook

Subject: [PATCH] coccicheck: span checks across CPUs

This adds parallelism by default to the "coccicheck" target using
spatch's "-max" and "-index" arguments.

Signed-off-by: Kees Cook <[email protected]>
---
Documentation/coccinelle.txt | 5 +++++
scripts/coccicheck | 14 ++++++++++++--
2 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/Documentation/coccinelle.txt b/Documentation/coccinelle.txt
index 18de785..408439d 100644
--- a/Documentation/coccinelle.txt
+++ b/Documentation/coccinelle.txt
@@ -91,6 +91,11 @@ To enable verbose messages set the V= variable, for example:

make coccicheck MODE=report V=1

+By default, coccicheck tries to run as parallel as possible. To change
+the parallelism, set the J= variable. For example, to run across 4 CPUs:
+
+ make coccicheck MODE=report J=4
+

Using Coccinelle with a single semantic patch
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
diff --git a/scripts/coccicheck b/scripts/coccicheck
index 06fcb33..077e5b2 100755
--- a/scripts/coccicheck
+++ b/scripts/coccicheck
@@ -11,6 +11,12 @@ else
VERBOSE=0
fi

+if [ -z "$J" ]; then
+ NPROC=$(getconf _NPROCESSORS_ONLN)
+else
+ NPROC="$J"
+fi
+
FLAGS="$SPFLAGS -very_quiet"

# spatch only allows include directories with the syntax "-I include"
@@ -61,10 +67,14 @@ if [ "$ONLINE" = "0" ] ; then
fi

run_cmd() {
+ local i
if [ $VERBOSE -ne 0 ] ; then
- echo "Running: $@"
+ echo "Running ($NPROC in parallel): $@"
fi
- eval $@
+ for i in $(seq 0 $(( NPROC - 1)) ); do
+ eval $@ -max $NPROC -index $i &
+ done
+ wait
}


--
1.7.9.5


--
Kees Cook
Chrome OS Security

2013-06-18 11:56:48

by Nicolas Palix (LIG)

Subject: Re: [PATCH] coccicheck: span checks across CPUs

Hi Kees,

This is indeed something that was on my TO-DO list! :)
Thank you a lot.

Could you consider merging with the following or
something equivalent for handling running spatch processes
when make is interrupted ?

For the resubmission, please add Michal Marek (in CC)
who will apply it to the misc branch of
git://git.kernel.org/pub/scm/linux/kernel/git/mmarek/kbuild.git

Regards,

diff --git a/scripts/coccicheck b/scripts/coccicheck
index a222e92..907f11e 100755
--- a/scripts/coccicheck
+++ b/scripts/coccicheck
@@ -2,6 +2,17 @@

SPATCH="`which ${SPATCH:=spatch}`"

+trap kill_running SIGTERM SIGINT
+declare -a SPATCH_RUN
+
+kill_running() {
+ for i in $(seq 0 $(( NPROC - 1)) ); do
+ if [ $V -eq 2 ] ; then echo "Killing ${SPATCH_RUN[$i]}" ; fi
+ kill ${SPATCH_RUN[$i]}
+ done
+ wait
+}
+
# The verbosity may be set by the environmental parameter V=
# as for example with 'make V=1 coccicheck'

@@ -80,7 +91,9 @@ run_cmd() {
echo "Running ($NPROC in parallel): $@"
fi
for i in $(seq 0 $(( NPROC - 1)) ); do
- eval $@ -max $NPROC -index $i &
+ eval "$@ -max $NPROC -index $i &"
+ SPATCH_RUN[$i]=$!
+ if [ $V -eq 2 ] ; then echo "${SPATCH_RUN[$i]} running" ; fi
done
wait
}

Le 6/18/13 1:06 AM, Kees Cook a ?crit :
> This adds parallelism by default to the "coccicheck" target using
> spatch's "-max" and "-index" arguments.
>
> Signed-off-by: Kees Cook <[email protected]>
> ---
> Documentation/coccinelle.txt | 5 +++++
> scripts/coccicheck | 14 ++++++++++++--
> 2 files changed, 17 insertions(+), 2 deletions(-)
>
> diff --git a/Documentation/coccinelle.txt b/Documentation/coccinelle.txt
> index 18de785..408439d 100644
> --- a/Documentation/coccinelle.txt
> +++ b/Documentation/coccinelle.txt
> @@ -91,6 +91,11 @@ To enable verbose messages set the V= variable, for example:
>
> make coccicheck MODE=report V=1
>
> +By default, coccicheck tries to run as parallel as possible. To change
> +the parallelism, set the J= variable. For example, to run across 4 CPUs:
> +
> + make coccicheck MODE=report J=4
> +
>
> Using Coccinelle with a single semantic patch
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> diff --git a/scripts/coccicheck b/scripts/coccicheck
> index 06fcb33..077e5b2 100755
> --- a/scripts/coccicheck
> +++ b/scripts/coccicheck
> @@ -11,6 +11,12 @@ else
> VERBOSE=0
> fi
>
> +if [ -z "$J" ]; then
> + NPROC=$(getconf _NPROCESSORS_ONLN)
> +else
> + NPROC="$J"
> +fi
> +
> FLAGS="$SPFLAGS -very_quiet"
>
> # spatch only allows include directories with the syntax "-I include"
> @@ -61,10 +67,14 @@ if [ "$ONLINE" = "0" ] ; then
> fi
>
> run_cmd() {
> + local i
> if [ $VERBOSE -ne 0 ] ; then
> - echo "Running: $@"
> + echo "Running ($NPROC in parallel): $@"
> fi
> - eval $@
> + for i in $(seq 0 $(( NPROC - 1)) ); do
> + eval $@ -max $NPROC -index $i &
> + done
> + wait
> }
>
>
>