Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Tue, 6 Sep 2022 21:11:10 +0200
From:   Frederic Weisbecker <frederic@kernel.org>
To:     Joel Fernandes <joel@joelfernandes.org>
Cc:     rcu@vger.kernel.org, linux-kernel@vger.kernel.org,
        rushikesh.s.kadam@intel.com, urezki@gmail.com,
        neeraj.iitr10@gmail.com, paulmck@kernel.org, rostedt@goodmis.org,
        vineeth@bitbyteword.org, boqun.feng@gmail.com
Subject: Re: [PATCH v5 06/18] rcu: Introduce call_rcu_lazy() API
 implementation
Message-ID: <20220906191110.GA187950@lothringen>
References: <20220901221720.1105021-1-joel@joelfernandes.org>
 <20220901221720.1105021-7-joel@joelfernandes.org>
 <20220902152132.GA115525@lothringen>
 <Yxa5Ch574cRZxRdo@google.com>
 <20220906151757.GA183806@lothringen>
 <cde6586e-ae61-5e85-3c9a-1ce7dd2464ed@joelfernandes.org>
 <64f5770c-df37-8975-200d-7908de23fa73@joelfernandes.org>
 <da45d265-52f9-6314-7fcd-ea71e2bf4cec@joelfernandes.org>
 <4f6061f0-0de7-2916-dc6e-9f5af9b944c0@joelfernandes.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <4f6061f0-0de7-2916-dc6e-9f5af9b944c0@joelfernandes.org>
Precedence: bulk

On Tue, Sep 06, 2022 at 12:43:52PM -0400, Joel Fernandes wrote:
> On 9/6/2022 12:38 PM, Joel Fernandes wrote:
> Ah, now I know why I got confused. I *used* to flush the bypass list before when
> !lazy CBs showed up. Paul suggested this is overkill. In this old overkill
> method, I was missing a wake up which was likely causing the boot regression.
> Forcing a wake up fixed that. Now in v5 I make it such that I don't do the flush
> on a !lazy rate-limit.
> 
> I am sorry for the confusion. Either way, in my defense this is just an extra
> bit of code that I have to delete. This code is hard. I have mostly relied on a
> test-driven development. But now thanks to this review and I am learning the
> code more and more...

Yeah this code is hard.

Especially as it's possible to flush from both sides and queue the timer
from both sides. And both sides read the bypass/lazy counter locklessly.
But only call_rcu_*() can queue/increase the bypass size whereas only
nocb_gp_wait() can cancel the timer. Phew!

Among the many possible dances between rcu_nocb_try_bypass()
and nocb_gp_wait(), I haven't found a way yet for the timer to be
set to LAZY when it should be BYPASS (or other kind of accident such
as an ignored callback).

In the worst case we may arm an earlier timer than necessary
(RCU_NOCB_WAKE_BYPASS instead of RCU_NOCB_WAKE_LAZY for example).

Famous last words...


> Thanks,
> 
>  - Joel
> 
> 
>