Date: Thu, 1 Dec 2016 08:59:18 -0800
From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Michal Hocko <mhocko@kernel.org>, Donald Buczek <buczek@molgen.mpg.de>,
        Paul Menzel <pmenzel@molgen.mpg.de>, dvteam@molgen.mpg.de,
        linux-mm@kvack.org, linux-kernel@vger.kernel.org,
        Josh Triplett <josh@joshtriplett.org>
Subject: Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and
 `mem_cgroup_shrink_node`
Reply-To: paulmck@linux.vnet.ibm.com
References: <20161130115320.GO3924@linux.vnet.ibm.com>
 <20161130131910.GF18432@dhcp22.suse.cz>
 <20161130142955.GS3924@linux.vnet.ibm.com>
 <20161130163820.GQ3092@twins.programming.kicks-ass.net>
 <20161130170557.GK18432@dhcp22.suse.cz>
 <20161130175015.GR3092@twins.programming.kicks-ass.net>
 <20161130194019.GF3924@linux.vnet.ibm.com>
 <20161201053035.GC3092@twins.programming.kicks-ass.net>
 <20161201124024.GB3924@linux.vnet.ibm.com>
 <20161201163614.GL3092@twins.programming.kicks-ass.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20161201163614.GL3092@twins.programming.kicks-ass.net>
User-Agent: Mutt/1.5.21 (2010-09-15)
Message-Id: <20161201165918.GG3924@linux.vnet.ibm.com>
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2072
Lines: 51

On Thu, Dec 01, 2016 at 05:36:14PM +0100, Peter Zijlstra wrote:
> On Thu, Dec 01, 2016 at 04:40:24AM -0800, Paul E. McKenney wrote:
> > On Thu, Dec 01, 2016 at 06:30:35AM +0100, Peter Zijlstra wrote:
> 
> > > Sure, we all dislike IPIs, but I'm thinking this half-way point is
> > > sensible, no point in issuing user visible annoyance if indeed we can
> > > prod things back to life, no?
> > > 
> > > Only if we utterly fail to make it respond should we bug the user with
> > > our failure..
> > 
> > Sold!  ;-)
> > 
> > I will put together a patch later today.
> > 
> > My intent is to hold off on the "upgrade cond_resched()" patch, one
> > step at a time.  Longer term, I do very much like the idea of having
> > cond_resched() do both scheduling and RCU quiescent states, assuming
> > that this avoids performance pitfalls.
> 
> Well, with the above change cond_resched() is already sufficient, no?

Maybe.  Right now, cond_resched_rcu_qs() gets a quiescent state to
the RCU core in less than one jiffy, with my other change, this becomes
a handful of jiffies depending on HZ and NR_CPUS.  I expect this
increase to a handful of jiffies to be a non-event.

After my upcoming patch, cond_resched() will get a quiescent state to
the RCU core in about ten seconds.  While I am am not all that nervous
about the increase from less than a jiffy to a handful of jiffies,
increasing to ten seconds via cond_resched() does make me quite nervous.
Past experience indicates that someone's kernel will likely be fatally
inconvenienced by this magnitude of change.

Or am I misunderstanding what you are proposing?

> In fact, by doing the IPI thing we get the entire cond_resched*()
> family, and we could add the should_resched() guard to
> cond_resched_rcu().

So that cond_resched_rcu_qs() looks something like this, in order
to avoid the function call in the case where the scheduler has nothing
to do?

#define cond_resched_rcu_qs() \
do { \
	if (!should_resched(current) || !cond_resched()) \
		rcu_note_voluntary_context_switch(current); \
} while (0)

							Thanx, Paul