Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Sun, 23 Sep 2018 16:54:13 -0700
From:   "Paul E. McKenney" <paulmck@linux.ibm.com>
To:     Joel Fernandes <joel@joelfernandes.org>
Cc:     LKML <linux-kernel@vger.kernel.org>
Subject: Re: Question about ->head field of rcu_segcblist
Reply-To: paulmck@linux.ibm.com
References: <CAEXW_YQd8s9GP-TACJCbT67ZCy5guTJMW+Jue3VsEX1777UhBQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAEXW_YQd8s9GP-TACJCbT67ZCy5guTJMW+Jue3VsEX1777UhBQ@mail.gmail.com>
User-Agent: Mutt/1.5.21 (2010-09-15)
Message-Id: <20180923235413.GA4222@linux.ibm.com>
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Sun, Sep 23, 2018 at 07:30:30PM -0400, Joel Fernandes wrote:
> Hi Paul,
> 
> I was parsing the Data-Structures document and had a question about
> the following "Important note" text.
> 
> Could it be clarified in the below text better why "remaining
> callbacks are placed back on the RCU_DONE_TAIL segment", is a reason
> for not depending on ->head for determining if no callbacks are
> associated with the rcu_segcblist? If callbacks are added back to the
> DONE_TAIL segment, then I would think rcu_head should be != NULL.
> Infact the "rsclp->head = *rsclp->tails[RCU_DONE_TAIL];" in
> rcu_segcblist_extract_done_cbs should set the ->head to  NULL if I
> understand correctly.

The rcu_segcblist_extract_done_cbs() function will set rsclp->head
to NULL only if there were no non-done callbacks on the rsclp list.
Otherwise, if there are non-done callbacks, then rsclp->head will
be set to the first non-done callback.

Either way, the problem is that the done callbacks can be removed
and re-added, but the count is not adjusted until the re-add.  So
you have to look at the count to see if there are callbacks.

Testing rsclp->head fails because it can be temporarily NULL, even
though there are callbacks hanging off of a pointer in rcu_do_batch()'s
stack frame.

Or am I misunderstanding your question?

							Thanx, Paul

> Important note: It is the ->len field that determines whether or not
> there are callbacks associated with this rcu_segcblist structure, not
> the ->head pointer. The reason for this is that all the
> ready-to-invoke callbacks (that is, those in the RCU_DONE_TAIL
> segment) are extracted all at once at callback-invocation time. If
> callback invocation must be postponed, for example, because a
> high-priority process just woke up on this CPU, then the remaining
> callbacks are placed back on the RCU_DONE_TAIL segment. Either way,
> the ->len and ->len_lazy counts are adjusted after the corresponding
> callbacks have been invoked, and so again it is the ->lencount that
> accurately reflects whether or not there are callbacks associated with
> this rcu_segcblist structure. Of course, off-CPU sampling of the ->len
> count requires the use of appropriate synchronization, for example,
> memory barriers. This synchronization can be a bit subtle,
> particularly in the case of rcu_barrier().
> 
> Thanks!
> 
>  - Joel
>