Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org F0ED0C447A9
MIME-Version: 1.0
References: <20200223184001.14248-1-qais.yousef@arm.com> <20200223184001.14248-6-qais.yousef@arm.com>
 <20200224061004.GH28029@codeaurora.org> <20200224121139.cbz2dt5heiouknif@e107158-lin.cambridge.arm.com>
In-Reply-To: <20200224121139.cbz2dt5heiouknif@e107158-lin.cambridge.arm.com>
From:   Pavan Kondeti <pkondeti@codeaurora.org>
Date:   Mon, 24 Feb 2020 21:34:41 +0530
Message-ID: <CAEU1=PncyV=-vqjkDHSJ4hUhhTfYUgVN-HAe4zXMHtFx1oc5XA@mail.gmail.com>
Subject: Re: [PATCH v2 5/6] sched/rt: Better manage pushing unfit tasks on wakeup
To:     Qais Yousef <qais.yousef@arm.com>
Cc:     Pavan Kondeti <pkondeti@codeaurora.org>,
        Ingo Molnar <mingo@kernel.org>,
        Peter Zijlstra <peterz@infradead.org>,
        Steven Rostedt <rostedt@goodmis.org>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Juri Lelli <juri.lelli@redhat.com>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
        LKML <linux-kernel@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

Hi Qais,

On Mon, Feb 24, 2020 at 5:42 PM Qais Yousef <qais.yousef@arm.com> wrote:
[...]
> We could do, temporarily, to get these fixes into 5.6. But I do think
> select_task_rq_rt() doesn't do a good enough job into pushing unfit tasks to
> the right CPUs.
>
> I don't understand the reasons behind your objection. It seems you think that
> select_task_rq_rt() should be enough, but not AFAICS. Can you be a bit more
> detailed please?
>
> FWIW, here's a screenshot of what I see
>
>         https://imgur.com/a/peV27nE
>
> After the first activation, select_task_rq_rt() fails to find the right CPU
> (due to the same move all tasks to the cpumask_fist()) - but when the task
> wakes up on 4, the logic I put causes it to migrate to CPU2, which is the 2nd
> big core. CPU1 and CPU2 are the big cores on Juno.
>
> Now maybe we should fix select_task_rq_rt() to better balance tasks, but not
> sure how easy is that.
>

Thanks for the trace. Now things are clear to me. Two RT tasks woke up
simultaneously and the first task got its previous CPU i.e CPU#1. The next task
goes through find_lowest_rq() and got the same CPU#1. Since this task priority
is not more than the just queued task (already queued on CPU#1), it is sent
to its previous CPU i.e CPU#4 in your case.

From task_woken_rt() path, CPU#4 attempts push_rt_tasks(). CPU#4 is
not overloaded,
but we have rt_task_fits_capacity() check which forces the push. Since the CPU
is not overloaded, your has_unfit_tasks() comes to rescue and push the
task. Since
the task has not scheduled in yet, it is eligible for push. You added checks
to skip resched_curr() in push_rt_tasks() otherwise the push won't happen.

Finally, I understood your patch. Obviously this is not clear to me
before. I am not
sure if this patch is the right approach to solve this race. I will
think a bit more.

Thanks,
Pavan
-- 
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a
Linux Foundation Collaborative Project