Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7;
MIME-Version: 1.0
References: <20231003232903.7109-1-frederic@kernel.org> <f214737a-6856-455f-ac86-9f7ec605b902@paulmck-laptop>
 <ZR0vjdp+BNiFm46+@lothringen>
In-Reply-To: <ZR0vjdp+BNiFm46+@lothringen>
From:   zhuangel570 <zhuangel570@gmail.com>
Date:   Sat, 7 Oct 2023 18:24:53 +0800
Message-ID: <CANZk6aQG4Lqyk8JkT_my3dwub4jGpn31wYRoCwW4oSH1x=sJ1A@mail.gmail.com>
Subject: Re: [PATCH 0/5] srcu fixes
To:     Frederic Weisbecker <frederic@kernel.org>
Cc:     "Paul E. McKenney" <paulmck@kernel.org>,
        LKML <linux-kernel@vger.kernel.org>,
        Neeraj upadhyay <neeraj.iitr10@gmail.com>,
        Joel Fernandes <joel@joelfernandes.org>,
        Boqun Feng <boqun.feng@gmail.com>,
        Uladzislau Rezki <urezki@gmail.com>, RCU <rcu@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Wed, Oct 4, 2023 at 5:25=E2=80=AFPM Frederic Weisbecker <frederic@kernel=
.org> wrote:
>
> On Tue, Oct 03, 2023 at 05:35:31PM -0700, Paul E. McKenney wrote:
> > On Wed, Oct 04, 2023 at 01:28:58AM +0200, Frederic Weisbecker wrote:
> > > Hi,
> > >
> > > This contains a fix for "SRCU: kworker hung in synchronize_srcu":
> > >
> > >     http://lore.kernel.org/CANZk6aR+CqZaqmMWrC2eRRPY12qAZnDZLwLnHZbNi=
=3DxXMB401g@mail.gmail.com
> > >
> > > And a few cleanups.
> > >
> > > Passed 50 hours of SRCU-P and SRCU-N.
> > >
> > > git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks=
.git
> > >     srcu/fixes
> > >
> > > HEAD: 7ea5adc5673b42ef06e811dca75e43d558cc87e0
> > >
> > > Thanks,
> > >     Frederic
> >
> > Very good, and a big "Thank You!!!" to all of you!
> >
> > I queued this series for testing purposes, and have started a bunch of
> > SRCU-P and SRCU-N tests on one set of systems, and a single SRCU-P and
> > SRCU-N on another system, but with both scenarios resized to 40 CPU eac=
h.
> >
> > While that is in flight, a few questions:
> >
> > o     Please check the Co-developed-by rules.  Last I knew, it was
> >       necessary to have a Signed-off-by after each Co-developed-by.
>
> Indeed! I'll try to collect the three of them within a few days. If some
> are missing, I'll put a Reported-by instead.
>
> >
> > o     Is it possible to get a Tested-by from the original reporter?
> >       Or is this not reproducible?
>
> It seems that the issue would trigger rarely. But I hope we can get one.

There is currently no way to reproduce this problem in our environment.
The problem has appeared on 2 machines, and each time it occurred, the
test had been running for more than a month.

BTW, I will run tests with these patches in our environment.

>
> >
> > o     Is it possible to convince rcutorture to find this sort of
> >       bug?  Seems like it should be, but easy to say...
>
> So at least the part where advance/accelerate fail is observed from time
> to time. But then we must meet two more rare events:
>
> 1) The CPU failing to ACC/ADV must also fail to start the grace period be=
cause
>   another CPU was faster.
>
> 2) The callbacks invocation must not run until that grace period has ende=
d (even
>   though we had a previous one completed with callbacks ready).
>
>   Or it can run after all but at least the acceleration part of it has to
>   happen after the end of the new grace period.
>
> Perhaps all these conditions can me met more often if we overcommit the n=
umber
> of vCPU. For example run 10 SRCU-P instances within 3 real CPUs. This cou=
ld
> introduce random breaks within the torture writers...
>
> Just an idea...
>
> >
> > o     Frederic, would you like to include this in your upcoming
> >       pull request?  Or does it need more time?
>
> At least the first patch yes. It should be easily backported and
> it should be enough to solve the race. I'll just wait a bit to collect
> more tags.
>
> Thanks!
>
> >
> >                                               Thanx, Paul
> >
> > > ---
> > >
> > > Frederic Weisbecker (5):
> > >       srcu: Fix callbacks acceleration mishandling
> > >       srcu: Only accelerate on enqueue time
> > >       srcu: Remove superfluous callbacks advancing from srcu_start_gp=
()
> > >       srcu: No need to advance/accelerate if no callback enqueued
> > >       srcu: Explain why callbacks invocations can't run concurrently
> > >
> > >
> > >  kernel/rcu/srcutree.c | 55 ++++++++++++++++++++++++++++++++++++-----=
----------
> > >  1 file changed, 39 insertions(+), 16 deletions(-)