Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Thu, 15 Apr 2021 16:27:59 +0100
From:   Vincent Donnefort <vincent.donnefort@arm.com>
To:     Quentin Perret <qperret@google.com>
Cc:     peterz@infradead.org, rjw@rjwysocki.net, viresh.kumar@linaro.org,
        vincent.guittot@linaro.org, linux-kernel@vger.kernel.org,
        ionela.voinescu@arm.com, lukasz.luba@arm.com,
        dietmar.eggemann@arm.com
Subject: Re: [PATCH] PM / EM: Inefficient OPPs detection
Message-ID: <20210415152758.GD391924@e120877-lin.cambridge.arm.com>
References: <1617901829-381963-1-git-send-email-vincent.donnefort@arm.com>
 <1617901829-381963-2-git-send-email-vincent.donnefort@arm.com>
 <YHg7pfGKhzlMrXqC@google.com>
 <20210415141207.GA391924@e120877-lin.cambridge.arm.com>
 <YHhWAvpReXjGwbl/@google.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <YHhWAvpReXjGwbl/@google.com>
User-Agent: Mutt/1.5.24 (2015-08-30)
Precedence: bulk

On Thu, Apr 15, 2021 at 03:04:34PM +0000, Quentin Perret wrote:
> On Thursday 15 Apr 2021 at 15:12:08 (+0100), Vincent Donnefort wrote:
> > On Thu, Apr 15, 2021 at 01:12:05PM +0000, Quentin Perret wrote:
> > > Hi Vincent,
> > > 
> > > On Thursday 08 Apr 2021 at 18:10:29 (+0100), Vincent Donnefort wrote:
> > > > Some SoCs, such as the sd855 have OPPs within the same performance domain,
> > > > whose cost is higher than others with a higher frequency. Even though
> > > > those OPPs are interesting from a cooling perspective, it makes no sense
> > > > to use them when the device can run at full capacity. Those OPPs handicap
> > > > the performance domain, when choosing the most energy-efficient CPU and
> > > > are wasting energy. They are inefficient.
> > > > 
> > > > Hence, add support for such OPPs to the Energy Model, which creates for
> > > > each OPP a performance state. The Energy Model can now be read using the
> > > > regular table, which contains all performance states available, or using
> > > > an efficient table, where inefficient performance states (and by
> > > > extension, inefficient OPPs) have been removed.
> > > > 
> > > > Currently, the efficient table is used in two paths. Schedutil, and
> > > > find_energy_efficient_cpu(). We have to modify both paths in the same
> > > > patch so they stay synchronized. The thermal framework still relies on
> > > > the original table and hence, DevFreq devices won't create the efficient
> > > > table.
> > > > 
> > > > As used in the hot-path, the efficient table is a lookup table, generated
> > > > dynamically when the perf domain is created. The complexity of searching
> > > > a performance state is hence changed from O(n) to O(1). This also
> > > > speeds-up em_cpu_energy() even if no inefficient OPPs have been found.
> > > 
> > > Interesting. Do you have measurements showing the benefits on wake-up
> > > duration? I remember doing so by hacking the wake-up path to force tasks
> > > into feec()/compute_energy() even when overutilized, and then running
> > > hackbench. Maybe something like that would work for you?
> > 
> > I'll give a try and see if I get improved numbers.
> > 
> > > 
> > > Just want to make sure we actually need all that complexity -- while
> > > it's good to reduce the asymptotic complexity, we're looking at a rather
> > > small problem (max 30 OPPs or so I expect?), so other effects may be
> > > dominating. Simply skipping inefficient OPPs could be implemented in a
> > > much simpler way I think.
> > 
> > I could indeed just skip the perf state if marked as ineffective. But the idea
> > was to avoid bringing another for loop in this hot-path.
> 
> Right, though it would just extend a little bit the existing loop, so
> the overhead is unlikely to be noticeable.

In the case where we let cpufreq_table resolution, it's a whole new loop that we
would bring. In the case where we rely only on the EM resolution and bypass the
cpufreq_table though it would be even. But with the look-up table, we're winning
everywhere :-) Anyway I'll see if I can measure any improvement here.

-- 
Vincent

> 
> > Also, not covered by this patch but probably we could get rid of the EM
> > complexity limit as the table resolution is way faster with this change.
> 
> Probably yeah. I was considering removing it since eb92692b2544
> ("sched/fair: Speed-up energy-aware wake-ups") but ended up keeping it
> as it's entirely untested on large systems. But maybe we can reconsider.
> 
> Thanks,
> Quentin