Date: Thu, 1 Dec 2016 10:42:52 -0800
From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Michal Hocko <mhocko@kernel.org>, Donald Buczek <buczek@molgen.mpg.de>,
        Paul Menzel <pmenzel@molgen.mpg.de>, dvteam@molgen.mpg.de,
        linux-mm@kvack.org, linux-kernel@vger.kernel.org,
        Josh Triplett <josh@joshtriplett.org>
Subject: Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and
 `mem_cgroup_shrink_node`
Reply-To: paulmck@linux.vnet.ibm.com
References: <20161130142955.GS3924@linux.vnet.ibm.com>
 <20161130163820.GQ3092@twins.programming.kicks-ass.net>
 <20161130170557.GK18432@dhcp22.suse.cz>
 <20161130175015.GR3092@twins.programming.kicks-ass.net>
 <20161130194019.GF3924@linux.vnet.ibm.com>
 <20161201053035.GC3092@twins.programming.kicks-ass.net>
 <20161201124024.GB3924@linux.vnet.ibm.com>
 <20161201163614.GL3092@twins.programming.kicks-ass.net>
 <20161201165918.GG3924@linux.vnet.ibm.com>
 <20161201180953.GO3045@worktop.programming.kicks-ass.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20161201180953.GO3045@worktop.programming.kicks-ass.net>
User-Agent: Mutt/1.5.21 (2010-09-15)
Message-Id: <20161201184252.GP3924@linux.vnet.ibm.com>
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 4404
Lines: 108

On Thu, Dec 01, 2016 at 07:09:53PM +0100, Peter Zijlstra wrote:
> On Thu, Dec 01, 2016 at 08:59:18AM -0800, Paul E. McKenney wrote:
> > On Thu, Dec 01, 2016 at 05:36:14PM +0100, Peter Zijlstra wrote:
> > > Well, with the above change cond_resched() is already sufficient, no?
> > 
> > Maybe.  Right now, cond_resched_rcu_qs() gets a quiescent state to
> > the RCU core in less than one jiffy, with my other change, this becomes
> > a handful of jiffies depending on HZ and NR_CPUS.  I expect this
> > increase to a handful of jiffies to be a non-event.
> > 
> > After my upcoming patch, cond_resched() will get a quiescent state to
> > the RCU core in about ten seconds.  While I am am not all that nervous
> > about the increase from less than a jiffy to a handful of jiffies,
> > increasing to ten seconds via cond_resched() does make me quite nervous.
> > Past experience indicates that someone's kernel will likely be fatally
> > inconvenienced by this magnitude of change.
> > 
> > Or am I misunderstanding what you are proposing?
> 
> No, that is indeed what I was proposing. Hurm.. OK let me ponder that a
> bit. There might be a few games we can play with !PREEMPT to avoid IPIs.
> 
> Thing is, I'm slightly uncomfortable with de-coupling rcu-sched from
> actual schedule() calls.

OK, what is the source of your discomfort?

There are several intermediate levels of evasive action:

0.	If there is another runnable task and certain other conditions
	are met, cond_resched() will invoke schedule(), which will
	provide an RCU quiescent state.

1.	All cond_resched_rcu_qs() invocations increment the CPU's
	rcu_qs_ctr per-CPU variable, which is treated by later
	invocations of RCU core as a quiescent state.  (I have
	a patch queued that causes RCU to ignore changes to this
	counter until the grace period is a few jiffies old.)

	In this case, the rcu_node locks plus smp_mb__after_unlock_lock()
	provide the needed ordering.

2.	If any cond_resched_rcu_qs() sees that an expedited grace
	period is waiting on the current CPU, it invokes rcu_sched_qs()
	to force RCU to see the quiescent state.  (To your point,
	rcu_sched_qs() is normally called from schedule(), but also
	from the scheduling-clock interrupt when it interrupts
	usermode or idle.)

	Again, the rcu_node locks plus smp_mb__after_unlock_lock()
	provide the needed ordering.

3.	If the grace period extends for more than 50 milliseconds
	(by default, tunable), all subsequent cond_resched_rcu_qs()
	invocations on that CPU turn into momentary periods of
	idleness from RCU's viewpoint.  (Atomically add 2 to the
	dyntick-idle counter.)

	Here, the atomic increment is surrounded by smp_mb__*_atomic()
	to provide the needed ordering, which should be a good substitute
	for actually passing through schedule().

4.	If the grace period extends for more than 21 seconds (by default),
	we emit an RCU CPU stall warning and then do a resched_cpu().
	I am proposing also doing a resched_cpu() halfway to RCU CPU
	stall-warning time.

5.	An RCU-sched expedited grace period does a local resched_cpu()
	from its IPI handler to force the CPU through a quiescent
	state.  (Yes, I could just invoke resched_cpu() from the
	task orchestrating the expedited grace period, but this approach
	allows more common code between RCU-preempt and RCU-sched
	expedited grace periods.)

> > > In fact, by doing the IPI thing we get the entire cond_resched*()
> > > family, and we could add the should_resched() guard to
> > > cond_resched_rcu().
> > 
> > So that cond_resched_rcu_qs() looks something like this, in order
> > to avoid the function call in the case where the scheduler has nothing
> > to do?
> 
> I was actually thinking of this:

Oh!  I had forgotten about cond_resched_rcu(), and thought you did a typo.

Acked-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>

> diff --git a/include/linux/sched.h b/include/linux/sched.h
> index 2d0c82e1d348..2dc7d8056b2a 100644
> --- a/include/linux/sched.h
> +++ b/include/linux/sched.h
> @@ -3374,9 +3374,11 @@ static inline int signal_pending_state(long state, struct task_struct *p)
>  static inline void cond_resched_rcu(void)
>  {
>  #if defined(CONFIG_DEBUG_ATOMIC_SLEEP) || !defined(CONFIG_PREEMPT_RCU)
> -	rcu_read_unlock();
> -	cond_resched();
> -	rcu_read_lock();
> +	if (should_resched(1)) {
> +		rcu_read_unlock();
> +		cond_resched();
> +		rcu_read_lock();
> +	}
>  #endif
>  }
> 
>