Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <030aacb0c1304e43ab917924dcf4f138@AcuMS.aculab.com>
 <20220411233447.rcencjivkhyltyxm@airbuntu> <4ca5cd70904d47bea0df93f7c0979c66@AcuMS.aculab.com>
 <CAKfTPtBWUvvFMsSnfcKsOpHFdvxvSWpjfuLpjwwQrwhQc7+xuw@mail.gmail.com> <20220413235719.xs72pm2kgihia46g@airbuntu>
In-Reply-To: <20220413235719.xs72pm2kgihia46g@airbuntu>
From:   Vincent Guittot <vincent.guittot@linaro.org>
Date:   Thu, 14 Apr 2022 09:54:20 +0200
Message-ID: <CAKfTPtBtzmgJNM=2ekmE0-HT+r=qLqBJ6R1cggkGCNHbb3FPdQ@mail.gmail.com>
Subject: Re: Scheduling tasks on idle cpu
To:     Qais Yousef <qais.yousef@arm.com>
Cc:     David Laight <David.Laight@aculab.com>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        "mingo@redhat.com" <mingo@redhat.com>,
        "peterz@infradead.org" <peterz@infradead.org>,
        "juri.lelli@redhat.com" <juri.lelli@redhat.com>,
        "rostedt@goodmis.org" <rostedt@goodmis.org>,
        "bsegall@google.com" <bsegall@google.com>,
        "mgorman@suse.de" <mgorman@suse.de>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "parth@linux.ibm.com" <parth@linux.ibm.com>,
        "chris.hyser@oracle.com" <chris.hyser@oracle.com>,
        "pkondeti@codeaurora.org" <pkondeti@codeaurora.org>,
        "Valentin.Schneider@arm.com" <Valentin.Schneider@arm.com>,
        "patrick.bellasi@matbug.net" <patrick.bellasi@matbug.net>,
        "pjt@google.com" <pjt@google.com>, "pavel@ucw.cz" <pavel@ucw.cz>,
        "tj@kernel.org" <tj@kernel.org>,
        "qperret@google.com" <qperret@google.com>,
        "tim.c.chen@linux.intel.com" <tim.c.chen@linux.intel.com>,
        Wei Wang <wvw@google.com>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Thu, 14 Apr 2022 at 01:57, Qais Yousef <qais.yousef@arm.com> wrote:
>
> On 04/12/22 11:07, Vincent Guittot wrote:
> > On Tue, 12 Apr 2022 at 10:39, David Laight <David.Laight@aculab.com> wrote:
> > > Yes I want the CFS scheduler to pick an idle cpu in preference
> > > to an active RT one.
> >
> > When task 34512 wakes up, scheduler checks if prev or this cpu are
> > idle which is not the case for you. Then, it compares the load of prev
> > and this_cpu and seems to select this_cpu (cpu17).
> >
> > Once cpu17 selected, it will try to find an idle cpu which shares LLC
> > but it seems that the scheduler didn't find one and finally keeps task
> > 34512 on this_cpu.
> >
> > Note that during the next tick, a load balance will be trigger if
> > this_cpu still have both RT and task 34512,
>
> David said there are idle cpus
>
>         "  There are two physical cpu with 20 cores each (with hyperthreading).
>         16, 18, 34, 36 and 38 were idle. So both 16 and 18 should be on the
>         same NUMA node. All the others are running the same RT thread code.  "
>
> Except for the possibility of them becoming idle just after the task has woken
> up, shouldn't one of them have been picked?

we don't loop on all cpus in the LLC to find an idle one but compute a
reasonable number of iteration based on the avg_idle

David can rerun is use case after disabling sched_feat(SIS_PROP)

>
> Thanks
>
> --
> Qais Yousef