Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Fri, 6 Sep 2019 12:57:51 -0400
From:   Joel Fernandes <joel@joelfernandes.org>
To:     "Paul E. McKenney" <paulmck@kernel.org>
Cc:     linux-kernel@vger.kernel.org, Andy Lutomirski <luto@kernel.org>,
        Bjorn Helgaas <bhelgaas@google.com>,
        Ingo Molnar <mingo@redhat.com>,
        Josh Triplett <josh@joshtriplett.org>,
        Lai Jiangshan <jiangshanlai@gmail.com>,
        Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
        Petr Mladek <pmladek@suse.com>,
        "Rafael J. Wysocki" <rafael.j.wysocki@intel.com>,
        rcu@vger.kernel.org, Steven Rostedt <rostedt@goodmis.org>,
        Yafang Shao <laoar.shao@gmail.com>
Subject: Re: [PATCH -rcu dev 1/2] Revert b8c17e6664c4 ("rcu: Maintain special
 bits at bottom of ->dynticks counter")
Message-ID: <20190906165751.GA40876@google.com>
References: <20190904045910.GC144846@google.com>
 <20190904101210.GM4125@linux.ibm.com>
 <20190904135420.GB240514@google.com>
 <20190904231308.GB4125@linux.ibm.com>
 <20190905153620.GG26466@google.com>
 <20190905164329.GT4125@linux.ibm.com>
 <20190906000137.GA224720@google.com>
 <20190906150806.GA11355@google.com>
 <20190906152144.GF4051@linux.ibm.com>
 <20190906152753.GA18734@linux.ibm.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20190906152753.GA18734@linux.ibm.com>
User-Agent: Mutt/1.10.1 (2018-07-13)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Fri, Sep 06, 2019 at 08:27:53AM -0700, Paul E. McKenney wrote:
> On Fri, Sep 06, 2019 at 08:21:44AM -0700, Paul E. McKenney wrote:
> > On Fri, Sep 06, 2019 at 11:08:06AM -0400, Joel Fernandes wrote:
> > > On Thu, Sep 05, 2019 at 08:01:37PM -0400, Joel Fernandes wrote:
> > > [snip] 
> > > > > > > @@ -3004,7 +3007,7 @@ static int rcu_pending(void)
> > > > > > >  		return 0;
> > > > > > >  
> > > > > > >  	/* Is the RCU core waiting for a quiescent state from this CPU? */
> > > > > > > -	if (rdp->core_needs_qs && !rdp->cpu_no_qs.b.norm)
> > > > > > > +	if (READ_ONCE(rdp->core_needs_qs) && !rdp->cpu_no_qs.b.norm)
> > > > > > >  		return 1;
> > > > > > >  
> > > > > > >  	/* Does this CPU have callbacks ready to invoke? */
> > > > > > > @@ -3244,7 +3247,6 @@ int rcutree_prepare_cpu(unsigned int cpu)
> > > > > > >  	rdp->gp_seq = rnp->gp_seq;
> > > > > > >  	rdp->gp_seq_needed = rnp->gp_seq;
> > > > > > >  	rdp->cpu_no_qs.b.norm = true;
> > > > > > > -	rdp->core_needs_qs = false;
> > > > > > 
> > > > > > How about calling the new hint-clearing function here as well? Just for
> > > > > > robustness and consistency purposes?
> > > > > 
> > > > > This and the next function are both called during a CPU-hotplug online
> > > > > operation, so there is little robustness or consistency to be had by
> > > > > doing it twice.
> > > > 
> > > > Ok, sorry I missed you are clearing it below in the next function. That's
> > > > fine with me.
> > > > 
> > > > This patch looks good to me and I am Ok with merging of these changes into
> > > > the original patch with my authorship as you mentioned. Or if you wanted to
> > > > be author, that's fine too :)
> > > 
> > > Paul, does it make sense to clear these urgency hints in rcu_qs() as well?
> > > After all, we are clearing atleast one urgency hint there: the
> > > rcu_read_unlock_special::need_qs bit.

Makes sense.

> > We certainly don't want to turn off the scheduling-clock interrupt until
> > after the quiescent state has been reported to the RCU core.  And it might
> > still be useful to have a heavy quiescent state because the grace-period
> > kthread can detect that.  Just in case the CPU that just called rcu_qs()
> > is slow about actually reporting that quiescent state to the RCU core.
> 
> Hmmm...  Should ->need_qs ever be cleared from FQS to begin with?

I did not see the FQS loop clearing ->need_qs in the rcu_read_unlock_special
union after looking for a few minutes. Could you clarify which path this?

Or do you mean ->core_needs_qs? If so, I feel the FQS loop should clear it as
I think your patch does, since the FQS loop is essentially doing heavy-weight
RCU core processing right?

Also, where does FQS loop clear rdp.cpu_no_qs? Shouldn't it clear that as
well for the dyntick-idle CPUs?

thanks,

 - Joel