Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Thu, 14 May 2020 15:47:35 -0700
From:   "Paul E. McKenney" <paulmck@kernel.org>
To:     Frederic Weisbecker <frederic@kernel.org>
Cc:     LKML <linux-kernel@vger.kernel.org>,
        Steven Rostedt <rostedt@goodmis.org>,
        Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
        Lai Jiangshan <jiangshanlai@gmail.com>,
        Joel Fernandes <joel@joelfernandes.org>,
        Josh Triplett <josh@joshtriplett.org>
Subject: Re: [PATCH 08/10] rcu: Allow to deactivate nocb on a CPU
Message-ID: <20200514224735.GA2869@paulmck-ThinkPad-P72>
Reply-To: paulmck@kernel.org
References: <20200513164714.22557-1-frederic@kernel.org>
 <20200513164714.22557-9-frederic@kernel.org>
 <20200513183831.GV2869@paulmck-ThinkPad-P72>
 <20200513224525.GA18303@lenoir>
 <20200514154707.GL2869@paulmck-ThinkPad-P72>
 <20200514223021.GA4071@lenoir>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20200514223021.GA4071@lenoir>
User-Agent: Mutt/1.9.4 (2018-02-28)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Fri, May 15, 2020 at 12:30:23AM +0200, Frederic Weisbecker wrote:
> On Thu, May 14, 2020 at 08:47:07AM -0700, Paul E. McKenney wrote:
> > On Thu, May 14, 2020 at 12:45:26AM +0200, Frederic Weisbecker wrote:
> > This last seems best to me.  The transition from CBLIST_NOT_OFFLOADED
> > to CBLIST_OFFLOADING of course needs to be on the CPU in question with
> > at least bh disabled.  Probably best to be holding rcu_nocb_lock(),
> > but that might just be me being overly paranoid.
> 
> So that's in the case of offloading, right? Well, I don't think we'd
> need to even disable bh nor lock nocb. We just need the current CPU
> to see the local update of cblist->offloaded = CBLIST_OFFLOADING
> before the kthread is unparked:
> 
>     cblist->offloaded = CBLIST_OFFLOADING;
>     /* Make sure subsequent softirq lock nocb */
>     barrier();
>     kthread_unpark(rdp->nocb_cb_thread);
> 
> Now, although that guarantees that nocb_cb will see CBLIST_OFFLOADING
> upon unparking, it's not guaranteed that the nocb_gp will see it on its
> next round. Ok so eventually you're right, I should indeed lock nocb...

I suspect that our future selves would hate us much less if we held
that lock.  ;-)

> > > > > +static long rcu_nocb_rdp_deoffload(void *arg)
> > > > > +{
> > > > > +	struct rcu_data *rdp = arg;
> > > > > +
> > > > > +	WARN_ON_ONCE(rdp->cpu != raw_smp_processor_id());
> > > > > +	__rcu_nocb_rdp_deoffload(rdp);
> > > > > +
> > > > > +	return 0;
> > > > > +}
> > > > 
> > > > For example, is the problem caused by invocations of this
> > > > rcu_nocb_rdp_deoffload() function?
> > > 
> > > How so?
> > 
> > It looked to me like it wasn't excluding either rcu_barrier() or CPU
> > hotplug.  It might also not have been pinning onto the CPU in question,
> > but that might just be me misremembering.  Then again, I didn't see a
> > call to it, so maybe its callers set things up appropriately.
> > 
> > OK, I will bite...  What is the purpose of rcu_nocb_rdp_deoffload()?  ;-)
> 
> Ah it's called using work_on_cpu() which launch a workqueue on the
> target and waits for completion. And that whole thing is protected
> inside the barrier mutex and hotplug.

Ah!  Yet again, color me blind.

							Thanx, Paul

> > Agreed!  And I do believe that concurrent callback execution will
> > prove better than a possibly indefinite gap in callback execution.
> 
> Mutual agreement! :-)
> 
> Thanks.