Received-SPF: softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
Date:   Fri, 25 Feb 2022 12:45:25 -0800
From:   "Paul E. McKenney" <paulmck@kernel.org>
To:     Paul Menzel <pmenzel@molgen.mpg.de>
Cc:     Josh Triplett <josh@joshtriplett.org>,
        Steven Rostedt <rostedt@goodmis.org>,
        Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
        Lai Jiangshan <jiangshanlai@gmail.com>,
        Shuah Khan <shuah@kernel.org>,
        Zhouyi Zhou <zhouzhouyi@gmail.com>, rcu@vger.kernel.org,
        linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH 2/2] torture: Make thread detection more robust by using
 lspcu
Message-ID: <20220225204525.GK4285@paulmck-ThinkPad-P17-Gen-1>
Reply-To: paulmck@kernel.org
References: <20220222120718.17141-1-pmenzel@molgen.mpg.de>
 <20220222120718.17141-2-pmenzel@molgen.mpg.de>
 <20220222174328.GL4285@paulmck-ThinkPad-P17-Gen-1>
 <1084da08-f67d-b5d7-add4-580e9246b7b7@molgen.mpg.de>
 <20220224205617.GY4285@paulmck-ThinkPad-P17-Gen-1>
 <bfab82a0-bf21-7782-f6fb-3adaf827ad25@molgen.mpg.de>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <bfab82a0-bf21-7782-f6fb-3adaf827ad25@molgen.mpg.de>
Precedence: bulk

On Fri, Feb 25, 2022 at 06:49:32AM +0100, Paul Menzel wrote:
> Dear Paul,
> 
> 
> Am 24.02.22 um 21:56 schrieb Paul E. McKenney:
> > On Thu, Feb 24, 2022 at 09:24:11AM +0100, Paul Menzel wrote:
> 
> > > Am 22.02.22 um 18:43 schrieb Paul E. McKenney:
> > > > On Tue, Feb 22, 2022 at 01:07:17PM +0100, Paul Menzel wrote:
> > > > > For consecutive numbers *lscpu* collapses the output and just shows the
> > > > > range with start and end. The processors are numbered that way on POWER8.
> > > > > 
> > > > >       $ sudo ppc64_cpu --smt=8
> > > > >       $ lscpu | grep '^NUMA node'
> > > > >       NUMA node(s):                    2
> > > > >       NUMA node0 CPU(s):               0-79
> > > > >       NUMA node8 CPU(s):               80-159
> > > > > 
> > > > > This causes the heuristic to detect the number threads per core, looking
> > > > > for the number after the first comma, to fail, and QEMU aborts because of
> > > > > invalid arguments.
> > > > > 
> > > > >       $ lscpu | sed -n -e '/^NUMA node0/s/^[^,]*,\([0-9]*\),.*$/\1/p'
> > > > >       $
> > > > > 
> > > > > (Before the last patch, the whole line was returned.)
> > > > > 
> > > > >       $ lscpu | grep '^NUMA node0' | sed -e 's/^[^,-]*(,|\-)\([0-9]*\),.*$/\1/'
> > > > >       NUMA node0 CPU(s):               0-79
> > > > > 
> > > > > *lscpu* shows the number of threads per core, so use that value directly.
> > > > > 
> > > > >       $ sudo ppc64_cpu --smt=8
> > > > >       $ lscpu | grep 'Thread(s) per core'
> > > > >       Thread(s) per core:              8
> > > > >       $ sudo ppc64_cpu --smt=off
> > > > >       $ lscpu | grep 'Thread(s) per core'
> > > > >       Thread(s) per core:              1
> > > > > 
> > > > > Note, the replaced heuristic is also incorrect for that case, where the
> > > > > threads per core are disabled.
> > > > > 
> > > > >       $ lscpu | sed -n -e '/^NUMA node0/s/^[^,]*,\([0-9]*\),.*$/\1/p'
> > > > >       8
> > > > > 
> > > > > Signed-off-by: Paul Menzel <pmenzel@molgen.mpg.de>
> > > > 
> > > > Makes sense, and thank you for chasing this down and for the fix!
> > > > 
> > > > But should this patch and 1/2 be merged?  Or am I confused and they
> > > > are somehow affecting two different lines of scripting?
> > > 
> > > You are right. I guess with 1/2 I just wanted to document clearly, what I
> > > learned in #sed@irc.libera.chat, that means, how to avoid using grep, when
> > > sed is used.
> > 
> > Nothing wrong with that!
> > 
> > I have merged the two patches as shown below.  Does this work for you?
> > 
> > 							Thanx, Paul
> > 
> > ------------------------------------------------------------------------
> > 
> > commit 9f0daba62e958c31326c7a9eae33651e3a3cc6b4
> > Author: Paul Menzel <pmenzel@molgen.mpg.de>
> > Date:   Tue Feb 22 13:07:16 2022 +0100
> > 
> >      torture: Make thread detection more robust by using lspcu
> >      For consecutive numbers the lscpu command collapses the output and just
> >      shows the range with start and end. The processors are numbered that
> >      way on POWER8.
> >          $ sudo ppc64_cpu --smt=8
> >          $ lscpu | grep '^NUMA node'
> >          NUMA node(s):                    2
> >          NUMA node0 CPU(s):               0-79
> >          NUMA node8 CPU(s):               80-159
> >      This causes the heuristic to detect the number threads per core, looking
> >      for the number after the first comma, to fail, and QEMU aborts because of
> >      invalid arguments.
> >          $ lscpu | grep '^NUMA node0' | sed -e 's/^[^,-]*(,|\-)\([0-9]*\),.*$/\1/'
> >          NUMA node0 CPU(s):               0-79
> >      But the lscpu command shows the number of threads per core:
> >          $ sudo ppc64_cpu --smt=8
> >          $ lscpu | grep 'Thread(s) per core'
> >          Thread(s) per core:              8
> >          $ sudo ppc64_cpu --smt=off
> >          $ lscpu | grep 'Thread(s) per core'
> >          Thread(s) per core:              1
> >      This commit therefore directly uses that value.
> 
> Maybe extend: …, and replaces `grep` by using using sed’s switch `-n` and
> the command p.

Ah, good point -- I will update this on the next rebase.

> >      Signed-off-by: Paul Menzel <pmenzel@molgen.mpg.de>
> >      Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
> > 
> > diff --git a/tools/testing/selftests/rcutorture/bin/functions.sh b/tools/testing/selftests/rcutorture/bin/functions.sh
> > index c35ba24f994c3..66d0414d8e4bc 100644
> > --- a/tools/testing/selftests/rcutorture/bin/functions.sh
> > +++ b/tools/testing/selftests/rcutorture/bin/functions.sh
> > @@ -301,7 +301,7 @@ specify_qemu_cpus () {
> >   			echo $2 -smp $3
> >   			;;
> >   		qemu-system-ppc64)
> > -			nt="`lscpu | grep '^NUMA node0' | sed -e 's/^[^,]*,\([0-9]*\),.*$/\1/'`"
> > +			nt="`lscpu | sed -n 's/^Thread(s) per core:\s*//p'`"
> >   			echo $2 -smp cores=`expr \( $3 + $nt - 1 \) / $nt`,threads=$nt
> >   			;;
> >   		esac
> 
> Thank you for doing that, and sorry for the extra work.

No complaints!  You are after all testing on powerpc, and getting that
working better is a good thing.

							Thanx, Paul