Date: Thu, 13 Apr 2017 09:55:16 -0700
From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: linux-kernel@vger.kernel.org, mingo@kernel.org, jiangshanlai@gmail.com,
        dipankar@in.ibm.com, akpm@linux-foundation.org,
        mathieu.desnoyers@efficios.com, josh@joshtriplett.org,
        tglx@linutronix.de, rostedt@goodmis.org, dhowells@redhat.com,
        edumazet@google.com, fweisbec@gmail.com, oleg@redhat.com,
        bobby.prani@gmail.com
Subject: Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text
 more explicit about skew_tick
Reply-To: paulmck@linux.vnet.ibm.com
References: <20170412165441.GA17149@linux.vnet.ibm.com>
 <1492016149-18834-4-git-send-email-paulmck@linux.vnet.ibm.com>
 <20170413091535.r6iw7s3pc2znvl6b@hirez.programming.kicks-ass.net>
 <20170413160332.GZ3956@linux.vnet.ibm.com>
 <20170413161948.ymvzlzhporgmldvn@hirez.programming.kicks-ass.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20170413161948.ymvzlzhporgmldvn@hirez.programming.kicks-ass.net>
User-Agent: Mutt/1.5.21 (2010-09-15)
Message-Id: <20170413165516.GI3956@linux.vnet.ibm.com>
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2364
Lines: 52

On Thu, Apr 13, 2017 at 06:19:48PM +0200, Peter Zijlstra wrote:
> On Thu, Apr 13, 2017 at 09:03:33AM -0700, Paul E. McKenney wrote:
> > On Thu, Apr 13, 2017 at 11:15:35AM +0200, Peter Zijlstra wrote:
> > > On Wed, Apr 12, 2017 at 09:55:40AM -0700, Paul E. McKenney wrote:
> > > > If you set RCU_FANOUT_LEAF too high, you can get lock contention
> > > > on the leaf rcu_node, and you should boot with the skew_tick kernel
> > > > parameter set in order to avoid this lock contention.  This commit
> > > > therefore upgrades the RCU_FANOUT_LEAF help text to explicitly state
> > > > this.
> > > > 
> > > > Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> > > > ---
> > > >  init/Kconfig | 10 ++++++++--
> > > >  1 file changed, 8 insertions(+), 2 deletions(-)
> > > > 
> > > > diff --git a/init/Kconfig b/init/Kconfig
> > > > index a92f27da4a27..946e561e67b7 100644
> > > > --- a/init/Kconfig
> > > > +++ b/init/Kconfig
> > > > @@ -612,11 +612,17 @@ config RCU_FANOUT_LEAF
> > > >  	  initialization.  These systems tend to run CPU-bound, and thus
> > > >  	  are not helped by synchronized interrupts, and thus tend to
> > > >  	  skew them, which reduces lock contention enough that large
> > > > -	  leaf-level fanouts work well.
> > > > +	  leaf-level fanouts work well.  That said, setting leaf-level
> > > > +	  fanout to a large number will likely cause problematic
> > > > +	  lock contention on the leaf-level rcu_node structures unless
> > > > +	  you boot with the skew_tick kernel parameter.
> > > 
> > > Why mention a way out of a problem you shouldn't have to begin with?
> > > 
> > > Just state its bad and result in lock contention and leave it at that.
> > 
> > To avoid people tuning huge machines having to wait for me to give
> > them an answer as to why they are suffering lock contention after
> > cranking up the value of RCU_FANOUT_LEAF.
> > 
> > Or am I missing your point?
> 
> Your answer should be: don't do that then. Not provide them a shady work
> around.
> 
> tick skew isn't pretty and has other problems (there's a reason its not
> on by default). You're then doing two things you shouldn't.

The tick skew problem that I know of is energy efficiency for light
workloads.  This doesn't normally apply to the large heavily loaded
systems on which people skew ticks.

So what are the other problems?

							Thanx, Paul