Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <4107a52c-131e-d683-898d-cdd2c2448137@linux.alibaba.com>
Date:   Fri, 15 Jul 2022 13:08:23 +0800
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0)
 Gecko/20100101 Thunderbird/91.11.0
Subject: Re: [PATCH 1/3] sched/core: Fix the bug that task won't enqueue into
 core tree when update cookie
Content-Language: en-US
To:     Peter Zijlstra <peterz@infradead.org>
Cc:     mingo@redhat.com, juri.lelli@redhat.com,
        vincent.guittot@linaro.org, dietmar.eggemann@arm.com,
        rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de,
        bristot@redhat.com, linux-kernel@vger.kernel.org
References: <1656403045-100840-1-git-send-email-CruzZhao@linux.alibaba.com>
 <1656403045-100840-2-git-send-email-CruzZhao@linux.alibaba.com>
 <YsKqf3mnv/aemeuC@hirez.programming.kicks-ass.net>
From:   cruzzhao <cruzzhao@linux.alibaba.com>
In-Reply-To: <YsKqf3mnv/aemeuC@hirez.programming.kicks-ass.net>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Precedence: bulk


在 2022/7/4 下午4:53, Peter Zijlstra 写道:

> Subject: sched/core: Fix the bug that task won't enqueue into core tree when update cookie
> From: Cruz Zhao <CruzZhao@linux.alibaba.com>
> Date: Tue, 28 Jun 2022 15:57:23 +0800
> 
> From: Cruz Zhao <CruzZhao@linux.alibaba.com>
> 
> In function sched_core_update_cookie(), a task will enqueue into the
> core tree only when it enqueued before, that is, if an uncookied task
> is cookied, it will not enqueue into the core tree until it enqueue
> again, which will result in unnecessary force idle.
> 
> Here follows the scenario:
>   CPU x and CPU y are a pair of SMT siblings.
>   1. Start task a running on CPU x without sleeping, and task b and
>      task c running on CPU y without sleeping.
>   2. We create a cookie and share it to task a and task b, and then
>      we create another cookie and share it to task c.
>   3. Simpling core_forceidle_sum of task a and b from /proc/PID/sched
> 
> And we will find out that core_forceidle_sum of task a takes 30%
> time of the sampling period, which shouldn't happen as task a and b
> have the same cookie.
> 
> Then we migrate task a to CPU x', migrate task b and c to CPU y', where
> CPU x' and CPU y' are a pair of SMT siblings, and sampling again, we
> will found out that core_forceidle_sum of task a and b are almost zero.
> 
> To solve this problem, we enqueue the task into the core tree if it's
> on rq.
> 
> Fixes: 6e33cad0af49("sched: Trivial core scheduling cookie management")
> Signed-off-by: Cruz Zhao <CruzZhao@linux.alibaba.com>
> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
> Link: https://lkml.kernel.org/r/1656403045-100840-2-git-send-email-CruzZhao@linux.alibaba.com
> ---
>  kernel/sched/core_sched.c |    9 +++++----
>  1 file changed, 5 insertions(+), 4 deletions(-)
> 
> --- a/kernel/sched/core_sched.c
> +++ b/kernel/sched/core_sched.c
> @@ -56,7 +56,6 @@ static unsigned long sched_core_update_c
>  	unsigned long old_cookie;
>  	struct rq_flags rf;
>  	struct rq *rq;
> -	bool enqueued;
>  
>  	rq = task_rq_lock(p, &rf);
>  
> @@ -68,14 +67,16 @@ static unsigned long sched_core_update_c
>  	 */
>  	SCHED_WARN_ON((p->core_cookie || cookie) && !sched_core_enabled(rq));
>  
> -	enqueued = sched_core_enqueued(p);
> -	if (enqueued)
> +	if (sched_core_enqueued(p))
>  		sched_core_dequeue(rq, p, DEQUEUE_SAVE);
>  
>  	old_cookie = p->core_cookie;
>  	p->core_cookie = cookie;
>  
> -	if (enqueued)
> +	/*
> +	 * Consider the cases: !prev_cookie and !cookie.
> +	 */
> +	if (cookie && task_on_rq_queued(p))
>  		sched_core_enqueue(rq, p);
>  
>  	/*
LGTM.