Received-SPF: pass (google.com: domain of linux-kernel+bounces-211104-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Message-ID: <d539bd2c-89f9-4a04-900c-41d257123163@arm.com>
Date: Wed, 12 Jun 2024 08:53:17 +0100
Precedence: bulk
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 2/2] cpuidle: teo: Introduce util-awareness
To: Qais Yousef <qyousef@layalina.io>
Cc: Kajetan Puchalski <kajetan.puchalski@arm.com>, rafael@kernel.org,
 daniel.lezcano@linaro.org, Dietmar.Eggemann@arm.com, dsmythies@telus.net,
 yu.chen.surf@gmail.com, linux-pm@vger.kernel.org,
 linux-kernel@vger.kernel.org, Peter Zijlstra <peterz@infradead.org>,
 Ulf Hansson <ulf.hansson@linaro.org>,
 Vincent Guittot <vincent.guittot@linaro.org>, Todd Kjos <tkjos@google.com>,
 "wvw@google.com" <wvw@google.com>
References: <20230105145159.1089531-1-kajetan.puchalski@arm.com>
 <20230105145159.1089531-3-kajetan.puchalski@arm.com>
 <20230711175814.zfavcn7xn3ia5va4@airbuntu>
 <ZLZ/btJw5LNVxVy8@e126311.manchester.arm.com>
 <20230718132432.w5xoxbqm54jmu6n5@airbuntu>
 <20230917010516.54dgcmms44wyfrvx@airbuntu>
 <CAKfTPtA6ZzRR-zMN7sodOW+N_P+GqwNv4tGR+aMB5VXRT2b5bg@mail.gmail.com>
 <20240529101950.bjpmmdqfhjg3aol6@airbuntu>
Content-Language: en-US
From: Lukasz Luba <lukasz.luba@arm.com>
In-Reply-To: <20240529101950.bjpmmdqfhjg3aol6@airbuntu>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Qais,

+Todd and Wei on CC

On 5/29/24 11:19, Qais Yousef wrote:
> On 05/28/24 11:29, Vincent Guittot wrote:
>> Hi All,
>>
>> I'm quite late on this thread but this patchset creates a major
>> regression for psci cpuidle driver when using the OSI mode (OS
>> initiated mode).  In such a case, cpuidle driver takes care only of
>> CPUs power state and the deeper C-states ,which includes cluster and
>> other power domains, are handled with power domain framework. In such
>> configuration ,cpuidle has only 2 c-states : WFI and cpu off states
>> and others states that include the clusters, are managed by genpd and
>> its governor.
>>
>> This patch selects cpuidle c-state N-1 as soon as the utilization is
>> above CPU capacity / 64 which means at most a level of 16 on the big
>> core but can be as low as 4 on little cores. These levels are very low
>> and the main result is that as soon as there is very little activity
>> on a CPU, cpuidle always selects WFI states whatever the estimated
>> sleep duration and which prevents any deeper states. Another effect is
>> that it also keeps the tick firing every 1ms in my case.
> 
> Unfortunately I think we need to revert this. We've been seeing the power
> regressions for a long while now and it doesn't seem we'll see an improvement
> soon based on last discussion.

Could you be more precised when you say 'we'?
It's not Vincent, because he said he cannot measure power on his end.

Do you mean Google ACK? Or Google Pixel Team?
You send emails from your private account and people are confused when
you say 'we'.

> 
>>
>> IMO, we should at least increase the utilization level
> 
> This won't help. We tried different values, unfortunately the logic is flawed.
> Utilization value on its own says nothing about the idleness of the system.

This is not true. When you up-migrate a task to big CPU, then CPU idle
gov can instantly benefit from utilization information and won't make
mistake based on old local history and won't use deep idle state.
So migrating the utilization from one CPU to another CPU says a lot
about the idleness to that destination CPU.

When Christian removed the util he got -4.5% lower score in GB5, so
this util has impact [1].

> I think best to revert and rethink the logic. Which is something we're pursuing
> and we'll share outcome when we have something to share. As it stands, this
> doesn't help. And we should really strive to avoid magic thresholds and values.
> They don't scale.

Please share the power numbers. It's not helping when you just say
some power regression w/o numbers, but with assumption that you
are working for big company.

Regards,
Lukasz

[1] 
https://lore.kernel.org/lkml/20240611112413.1241352-1-christian.loehle@arm.com/