Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <b3d2c694-e2a2-63ea-817f-113b5445dfda@bytedance.com>
Date:   Thu, 8 Sep 2022 21:49:21 +0800
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0)
 Gecko/20100101 Thunderbird/102.2.1
Subject: Re: [PATCH] sched/fair: Fix misuse of available_idle_cpu()
To:     Peter Zijlstra <peterz@infradead.org>, Mel Gorman <mgorman@suse.de>
Cc:     Vincent Guittot <vincent.guittot@linaro.org>,
        Rohit Jain <rohit.k.jain@oracle.com>,
        Ingo Molnar <mingo@kernel.org>, linux-kernel@vger.kernel.org
References: <20220908080702.58938-1-wuyun.abel@bytedance.com>
 <20220908103632.yzm5boxrr3nmvhpm@suse.de>
 <YxnrTdwA493gIHS8@hirez.programming.kicks-ass.net>
Content-Language: en-US
From:   Abel Wu <wuyun.abel@bytedance.com>
In-Reply-To: <YxnrTdwA493gIHS8@hirez.programming.kicks-ass.net>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 9/8/22 9:17 PM, Peter Zijlstra wrote:
> On Thu, Sep 08, 2022 at 11:36:32AM +0100, Mel Gorman wrote:
>> On Thu, Sep 08, 2022 at 04:07:02PM +0800, Abel Wu wrote:
>>> The function available_idle_cpu() was introduced to distinguish
>>> between the code paths that cares if the vCPU is preempted and
>>> the ones don't care. In general, available_idle_cpu() is used in
>>> selecting cpus for immediate use, e.g. ttwu. While idle_cpu() is
>>> used in the paths that only cares about the cpu is idle or not,
>>> and __update_idle_core() is one of them.
>>>
>>> Use idle_cpu() instead in the idle path to make has_idle_core
>>> a better hint.
>>>
>>> Fixes: 943d355d7fee (sched/core: Distinguish between idle_cpu() calls based on desired effect, introduce available_idle_cpu())
>>> Signed-off-by: Abel Wu <wuyun.abel@bytedance.com>
>>
>> Seems fair. As vCPU preemption is specific to virtualisation, it is very
>> unlikely that SMT is exposed to the guest so the impact of the patch is
> 
> Right; only pinned guests typically expose such topology information
> (anything else would be quite broken).
> 
>> minimal but I still think it's right so;
> 
> I'm not convinced; all of select_idle_sibling() seems to use
> available_idle_cpu(), and that's the only consumer of
> __update_idle_core(), so in that respect the current state makes sense.

Hi Peter, Mel, thanks for your reviewing!

My thought was that the preempted core can become active again before
select_idle_sibling() is called, so using available_idle_cpu() in
__update_idle_core() can potentially lose the opportunity to kick an
idle core running. While the downside of using idle_cpu() is that a
full scan can be triggered irrespective of non-preempted cores exist,
but even available_idle_cpu() can not make sure of that either.

BTW, I am also confused with select_idle_core() in which all the cpus
of a core need to be non-preempted before the core can be taken as an
idle core. IMHO, it might be enough that at least one cpu of an idle
core is non-preempted and allowed by task's taskset.

Thanks & BR,
Abel