Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Subject: Re: [PATCH 09/10] sched/fair: disable stealing if too many NUMA nodes
From:   Steven Sistare <steven.sistare@oracle.com>
To:     Peter Zijlstra <peterz@infradead.org>
Cc:     mingo@redhat.com, subhra.mazumdar@oracle.com,
        dhaval.giani@oracle.com, rohit.k.jain@oracle.com,
        daniel.m.jordan@oracle.com, pavel.tatashin@microsoft.com,
        matt@codeblueprint.co.uk, umgwanakikbuti@gmail.com,
        riel@redhat.com, jbacik@fb.com, juri.lelli@redhat.com,
        linux-kernel@vger.kernel.org
References: <1540220381-424433-1-git-send-email-steven.sistare@oracle.com>
 <1540220381-424433-10-git-send-email-steven.sistare@oracle.com>
 <20181022170604.GG3117@worktop.programming.kicks-ass.net>
 <1356a3cc-a9f2-6593-4737-1b20fc5daa75@oracle.com>
Organization: Oracle Corporation
Message-ID: <274b79fd-b001-3d30-3773-7f01940ca8a6@oracle.com>
Date:   Mon, 22 Oct 2018 15:21:20 -0400
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101
 Thunderbird/52.9.1
MIME-Version: 1.0
In-Reply-To: <1356a3cc-a9f2-6593-4737-1b20fc5daa75@oracle.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On 10/22/2018 2:47 PM, Steven Sistare wrote:
> On 10/22/2018 1:06 PM, Peter Zijlstra wrote:
>> On Mon, Oct 22, 2018 at 07:59:40AM -0700, Steve Sistare wrote:
>>> The STEAL feature causes regressions on hackbench on larger NUMA systems,
>>> so disable it on systems with more than sched_steal_node_limit nodes
>>> (default 2).
>>
>> How come? From a quick read the stealing is per LLC, where do we steal
>> across nodes?
> 
> See the complete explanation in this patch.  It is deeper than can be gleaned
> from a quick read.

I should have said a bit more.  Your quick take on stealing is correct, we do
not steal across nodes.  However, stealing reduces average run queue length which 
influences wake_affine migrations.  Now see the complete explanation.

- Steve