Received-SPF: pass (google.com: domain of linux-kernel+bounces-73308-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Date: Tue, 20 Feb 2024 16:07:37 +0000
From: Qais Yousef <qyousef@layalina.io>
To: Dietmar Eggemann <dietmar.eggemann@arm.com>
Cc: Vincent Guittot <vincent.guittot@linaro.org>,
	Ingo Molnar <mingo@kernel.org>,
	Peter Zijlstra <peterz@infradead.org>, linux-kernel@vger.kernel.org,
	Pierre Gondois <Pierre.Gondois@arm.com>
Subject: Re: [PATCH v4 1/2] sched/fair: Check a task has a fitting cpu when
 updating misfit
Message-ID: <20240220160737.jlpewyyq3hp5nwgs@airbuntu>
References: <20240105222014.1025040-1-qyousef@layalina.io>
 <20240105222014.1025040-2-qyousef@layalina.io>
 <CAKfTPtBYcVWxYOhWZjzcQUB1ebUBA-B30hvToqGBq6JnfRUZNg@mail.gmail.com>
 <20240124222959.ikwnbxkcjaxuiqp2@airbuntu>
 <CAKfTPtDxqcrf0kaBQG_zpFx-DEZTMKfyxBu_bzCuZ_UZhJwOnA@mail.gmail.com>
 <20240126014602.wdcro3ajffpna4fp@airbuntu>
 <5249d534-d7ac-4cbc-a696-f269cb61c7d1@arm.com>
 <20240206150653.34ouqhbt4yz2cmgo@airbuntu>
 <f58fb725-39b6-4b76-bc23-bc0526bddccb@arm.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <f58fb725-39b6-4b76-bc23-bc0526bddccb@arm.com>

On 02/06/24 18:17, Dietmar Eggemann wrote:
> On 06/02/2024 16:06, Qais Yousef wrote:
> > On 02/05/24 20:49, Dietmar Eggemann wrote:
> >> On 26/01/2024 02:46, Qais Yousef wrote:
> >>> On 01/25/24 18:40, Vincent Guittot wrote:
> >>>> On Wed, 24 Jan 2024 at 23:30, Qais Yousef <qyousef@layalina.io> wrote:
> >>>>>
> >>>>> On 01/23/24 09:26, Vincent Guittot wrote:
> >>>>>> On Fri, 5 Jan 2024 at 23:20, Qais Yousef <qyousef@layalina.io> wrote:
> >>>>>>>
> >>>>>>> From: Qais Yousef <qais.yousef@arm.com>
> 
> [...]
> 
> >>> It seems we flatten topologies but not sched domains. I see all cpus shown as
> >>> core_siblings. The DT for apple silicon sets clusters in the cpu-map - which
> >>> seems the flatten topology stuff detect LLC correctly but still keeps the
> >>> sched-domains not flattened. Is this a bug? I thought we will end up with one
> >>> sched domain still.
> >>
> >> IMHO, if you have a cpu_map entry with > 1 cluster in your dtb, you end
> >> up with MC and PKG (former DIE) Sched Domain (SD) level. And misfit load
> > 
> > Hmm, okay. I thought the detection of topology where we know the LLC is shared
> > will cause the sched domains to collapse too.
> > 
> >> balance takes potentially longer on PKG than to MC.
> > 
> > Why potentially longer? We iterate through the domains the CPU belong to. If
> > the first iteration (at MC) pulled something, then once we go to PKG then we're
> > less likely to pull again?
> 
> There are a couple of mechanisms in place to let load-balance on higher
> sd levels happen less frequently, eg:
> 
>   load_balance() -> should_we_balance() + continue_balancing
> 
>   interval = get_sd_balance_interval(sd, busy) in rebalance_domains()
> 
>   rq->avg_idle versus sd->max_newidle_lb_cost

Okay thanks. That last one I missed.

> 
> > Anyway. I think I am hitting a bug here. The behavior doesn't look right to me
> > given the delays I'm seeing and the fact we do the ilb but for some reason fail
> > to pull
> 
> [...]
>