Date: Wed, 19 Apr 2017 07:50:53 -0700
From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: linux-kernel@vger.kernel.org, mingo@kernel.org, jiangshanlai@gmail.com,
        dipankar@in.ibm.com, akpm@linux-foundation.org,
        mathieu.desnoyers@efficios.com, josh@joshtriplett.org,
        tglx@linutronix.de, rostedt@goodmis.org, dhowells@redhat.com,
        edumazet@google.com, fweisbec@gmail.com, oleg@redhat.com,
        bobby.prani@gmail.com
Subject: Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text
 more explicit about skew_tick
Reply-To: paulmck@linux.vnet.ibm.com
References: <20170413160332.GZ3956@linux.vnet.ibm.com>
 <20170413161948.ymvzlzhporgmldvn@hirez.programming.kicks-ass.net>
 <20170413165516.GI3956@linux.vnet.ibm.com>
 <20170413170434.xk4zq3p75pu3ubxw@hirez.programming.kicks-ass.net>
 <20170413173100.GL3956@linux.vnet.ibm.com>
 <20170413174631.56ycg545gwbsb4q2@hirez.programming.kicks-ass.net>
 <20170413181926.GP3956@linux.vnet.ibm.com>
 <20170413182309.vmyivo3oqrtfhhxt@hirez.programming.kicks-ass.net>
 <20170413184232.GQ3956@linux.vnet.ibm.com>
 <20170419132226.yvo3jyweb3d2a632@hirez.programming.kicks-ass.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20170419132226.yvo3jyweb3d2a632@hirez.programming.kicks-ass.net>
User-Agent: Mutt/1.5.21 (2010-09-15)
Message-Id: <20170419145053.GK3956@linux.vnet.ibm.com>
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 1246
Lines: 30

On Wed, Apr 19, 2017 at 03:22:26PM +0200, Peter Zijlstra wrote:
> On Thu, Apr 13, 2017 at 11:42:32AM -0700, Paul E. McKenney wrote:
> 
> > I believe that you are missing the fact that RCU grace-period
> > initialization and cleanup walks through the rcu_node tree breadth
> > first, using rcu_for_each_node_breadth_first().
> 
> Indeed. That is the part I completely missed.
> 
> >                                                 This macro (shown below)
> > implements this breadth-first walk using a simple sequential traversal of
> > the ->node[] array that provides the structures making up the rcu_node
> > tree.  As you can see, this scan is completely independent of how CPU
> > numbers might be mapped to rcu_data slots in the leaf rcu_node structures.
> 
> So this code is clearly not a hotpath, but still its performance
> matters?
> 
> Seems like you cannot win here :/

Welcome to my world!!!  ;-)

But yes, running on 4096-CPU systems can put some serious stress on
some surprising areas.  Especially when those systems have cache-miss
latencies well in excess of a microsecond, and the users are nevertheless
expecting scheduling latencies well below 100 microseconds.

It was a fun challenge, I grant you that!

							Thanx, Paul