Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Fri, 5 Jul 2019 08:09:32 -0700
From:   "Paul E. McKenney" <paulmck@linux.ibm.com>
To:     Joel Fernandes <joel@joelfernandes.org>
Cc:     Byungchul Park <byungchul.park@lge.com>,
        linux-kernel@vger.kernel.org, Davidlohr Bueso <dave@stgolabs.net>,
        Josh Triplett <josh@joshtriplett.org>,
        Lai Jiangshan <jiangshanlai@gmail.com>,
        Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
        rcu@vger.kernel.org, Steven Rostedt <rostedt@goodmis.org>,
        kernel-team@android.com
Subject: Re: [PATCH] rcuperf: Make rcuperf kernel test more robust for
 !expedited mode
Reply-To: paulmck@linux.ibm.com
References: <20190704043431.208689-1-joel@joelfernandes.org>
 <20190704174044.GK26519@linux.ibm.com>
 <20190705035231.GA31088@X58A-UD3R>
 <20190705122450.GA82532@google.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20190705122450.GA82532@google.com>
User-Agent: Mutt/1.5.21 (2010-09-15)
Message-Id: <20190705150932.GO26519@linux.ibm.com>
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Fri, Jul 05, 2019 at 08:24:50AM -0400, Joel Fernandes wrote:
> On Fri, Jul 05, 2019 at 12:52:31PM +0900, Byungchul Park wrote:
> > On Thu, Jul 04, 2019 at 10:40:44AM -0700, Paul E. McKenney wrote:
> > > On Thu, Jul 04, 2019 at 12:34:30AM -0400, Joel Fernandes (Google) wrote:
> > > > It is possible that the rcuperf kernel test runs concurrently with init
> > > > starting up.  During this time, the system is running all grace periods
> > > > as expedited.  However, rcuperf can also be run for normal GP tests.
> > > > Right now, it depends on a holdoff time before starting the test to
> > > > ensure grace periods start later. This works fine with the default
> > > > holdoff time however it is not robust in situations where init takes
> > > > greater than the holdoff time to finish running. Or, as in my case:
> > > > 
> > > > I modified the rcuperf test locally to also run a thread that did
> > > > preempt disable/enable in a loop. This had the effect of slowing down
> > > > init. The end result was that the "batches:" counter in rcuperf was 0
> > > > causing a division by 0 error in the results. This counter was 0 because
> > > > only expedited GPs seem to happen, not normal ones which led to the
> > > > rcu_state.gp_seq counter remaining constant across grace periods which
> > > > unexpectedly happen to be expedited. The system was running expedited
> > > > RCU all the time because rcu_unexpedited_gp() would not have run yet
> > > > from init.  In other words, the test would concurrently with init
> > > > booting in expedited GP mode.
> > > > 
> > > > To fix this properly, let us check if system_state if SYSTEM_RUNNING
> > > > is set before starting the test. The system_state approximately aligns
> > 
> > Just minor typo..
> > 
> > To fix this properly, let us check if system_state if SYSTEM_RUNNING
> > is set before starting the test. ...
> > 
> > Should be
> > 
> > To fix this properly, let us check if system_state is set to
> > SYSTEM_RUNNING before starting the test. ...
> 
> That's a fair point. I wonder if Paul already fixed it up in his tree,
> however I am happy to resend if he hasn't. Paul, how would you like to handle
> this commit log nit?
> 
> it is just 'if ..' to 'is SYSTEM_RUNNING'

It now reads as follows:

	To fix this properly, this commit waits until system_state is
	set to SYSTEM_RUNNING before starting the test.  This change is
	made just before kernel_init() invokes rcu_end_inkernel_boot(),
	and this latter is what turns off boot-time expediting of RCU
	grace periods.

I dropped the last paragraph about late_initcall().  And I suspect that
the last clause from rcu_gp_is_expedited() can be dropped:

bool rcu_gp_is_expedited(void)
{
	return rcu_expedited || atomic_read(&rcu_expedited_nesting) ||
	       rcu_scheduler_active == RCU_SCHEDULER_INIT;
}

This is because rcu_expedited_nesting is initialized to 1, and is
decremented in rcu_end_inkernel_boot(), which is called long after
rcu_scheduler_active has been set to RCU_SCHEDULER_RUNNING, which
happens at core_initcall() time.  So if the last clause says "true",
so does the second-to-last clause.

The similar check in rcu_gp_is_normal() is still need, however, to allow
the power-management subsystem to invoke synchronize_rcu() just after
the scheduler has been initialized, but before RCU is aware of this.

So, how about the commit shown below?

							Thanx, Paul

------------------------------------------------------------------------

commit 1f7e72efe3c761c2b34da7b59e01ad69c657db10
Author: Paul E. McKenney <paulmck@linux.ibm.com>
Date:   Fri Jul 5 08:05:10 2019 -0700

    rcu: Remove redundant "if" condition from rcu_gp_is_expedited()
    
    Because rcu_expedited_nesting is initialized to 1 and not decremented
    until just before init is spawned, rcu_expedited_nesting is guaranteed
    to be non-zero whenever rcu_scheduler_active == RCU_SCHEDULER_INIT.
    This commit therefore removes this redundant "if" equality test.
    
    Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>

diff --git a/kernel/rcu/update.c b/kernel/rcu/update.c
index 249517058b13..64e9cc8609e7 100644
--- a/kernel/rcu/update.c
+++ b/kernel/rcu/update.c
@@ -136,8 +136,7 @@ static atomic_t rcu_expedited_nesting = ATOMIC_INIT(1);
  */
 bool rcu_gp_is_expedited(void)
 {
-	return rcu_expedited || atomic_read(&rcu_expedited_nesting) ||
-	       rcu_scheduler_active == RCU_SCHEDULER_INIT;
+	return rcu_expedited || atomic_read(&rcu_expedited_nesting);
 }
 EXPORT_SYMBOL_GPL(rcu_gp_is_expedited);