Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34;
Date:   Wed, 25 Oct 2023 15:47:55 +0200
From:   Stephan Gerhold <stephan@gerhold.net>
To:     Ulf Hansson <ulf.hansson@linaro.org>,
        Viresh Kumar <viresh.kumar@linaro.org>
Cc:     Viresh Kumar <vireshk@kernel.org>, Nishanth Menon <nm@ti.com>,
        Stephen Boyd <sboyd@kernel.org>, linux-pm@vger.kernel.org,
        Vincent Guittot <vincent.guittot@linaro.org>,
        "Rafael J. Wysocki" <rafael@kernel.org>,
        Stephan Gerhold <stephan.gerhold@kernkonzept.com>,
        Konrad Dybcio <konrad.dybcio@linaro.org>,
        Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>,
        linux-kernel@vger.kernel.org
Subject: Re: [PATCH 1/2] OPP: Use _set_opp_level() for single genpd case
Message-ID: <ZTkciw5AwufxQYnB@gerhold.net>
References: <cover.1697710527.git.viresh.kumar@linaro.org>
 <f709e9e273004be43efe3a2854a7e7b51a777f99.1697710527.git.viresh.kumar@linaro.org>
 <CAPDyKFqbnsdT0nqKwQhai875CwwpW_vepr816fL+i8yLh=YQhw@mail.gmail.com>
 <20231025065458.z3klmhahrcqh6qyw@vireshk-i7>
 <CAPDyKFr4vdsKVYEx0aF5k_a1bTjp3NzMpNgaXDJOJrvujT7iRg@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAPDyKFr4vdsKVYEx0aF5k_a1bTjp3NzMpNgaXDJOJrvujT7iRg@mail.gmail.com>
Content-Transfer-Encoding: 7bit
Precedence: bulk

On Wed, Oct 25, 2023 at 12:40:26PM +0200, Ulf Hansson wrote:
> On Wed, 25 Oct 2023 at 08:55, Viresh Kumar <viresh.kumar@linaro.org> wrote:
> >
> > On 19-10-23, 13:16, Ulf Hansson wrote:
> > > On Thu, 19 Oct 2023 at 12:22, Viresh Kumar <viresh.kumar@linaro.org> wrote:
> > > > +static int _link_required_opps(struct dev_pm_opp *opp, struct opp_table *opp_table,
> > > >                                struct opp_table *required_table, int index)
> > > >  {
> > > >         struct device_node *np;
> > > > @@ -314,6 +314,25 @@ static int _link_required_opps(struct dev_pm_opp *opp,
> > > >                 return -ENODEV;
> > > >         }
> > > >
> > > > +       /*
> > > > +        * There are two genpd (as required-opp) cases that we need to handle,
> > > > +        * devices with a single genpd and ones with multiple genpds.
> > > > +        *
> > > > +        * The single genpd case requires special handling as we need to use the
> > > > +        * same `dev` structure (instead of a virtual one provided by genpd
> > > > +        * core) for setting the performance state. Lets treat this as a case
> > > > +        * where the OPP's level is directly available without required genpd
> > > > +        * link in the DT.
> > > > +        *
> > > > +        * Just update the `level` with the right value, which
> > > > +        * dev_pm_opp_set_opp() will take care of in the normal path itself.
> > > > +        */
> > > > +       if (required_table->is_genpd && opp_table->required_opp_count == 1 &&
> > > > +           !opp_table->genpd_virt_devs) {
> > > > +               if (!WARN_ON(opp->level))
> > >
> > > Hmm. Doesn't this introduce an unnecessary limitation?
> > >
> > > An opp node that has a required-opps phande, may have "opp-hz",
> > > "opp-microvolt", etc. Why would we not allow the "opp-level" to be
> > > used too?
> >
> > Coming back to this, why would we ever want a device to have "opp-level" and
> > "required-opp" (set to genpd's table) ? That would mean we will call:
> >
> > dev_pm_domain_set_performance_state() twice to set different level values.
> 
> Yes - and that would be weird, especially since the PM domain (genpd)
> is already managing the aggregation and propagation to parent domains.
> 

FWIW I'm hitting this WARNing when trying to set up the parent domain
setup for CPR->RPMPD(MX) on MSM8916 that I discussed with Uffe recently
[1]. I know, me and all my weird OPP setups. :'D

Basically, I have cpufreq voting for performance states of the CPR genpd
(via required-opps). CPR is supposed to have <&rpmpd MSM8916_VDDMX_AO>
as parent genpd and translates to the parent performance state using the
"required-opps" in the *CPR* OPP table:

	cpr: power-controller@b018000 {
		compatible = "qcom,msm8916-cpr", "qcom,cpr";
		reg = <0x0b018000 0x1000>;
		/* ... */
		#power-domain-cells = <0>;
		operating-points-v2 = <&cpr_opp_table>;
		/* Supposed to be parent domain, not consumer */
		power-domains = <&rpmpd MSM8916_VDDMX_AO>;

		cpr_opp_table: opp-table {
			compatible = "operating-points-v2-qcom-level";

			cpr_opp1: opp1 {
				opp-level = <1>;
				qcom,opp-fuse-level = <1>;
				required-opps = <&rpmpd_opp_svs_soc>;
			};
			cpr_opp2: opp2 {
				opp-level = <2>;
				qcom,opp-fuse-level = <2>;
				required-opps = <&rpmpd_opp_nom>;
			};
			cpr_opp3: opp3 {
				opp-level = <3>;
				qcom,opp-fuse-level = <3>;
				required-opps = <&rpmpd_opp_super_turbo>;
			};
		};
	};

There are two problems with this:

 1. (Unrelated to $subject patch)
    Since there is only a single entry in "power-domains", the genpd
    core code automatically attaches the CPR platform device as consumer
    of the VDDMX_AO power domain. I don't want this, I want it to become
    child of the VDDMX_AO genpd.

    I added some hacky code to workaround this. One option that works is
    to add a second dummy entry to "power-domains", which will prevent
    the genpd core from attaching the power domain:

    	power-domains = <&rpmpd MSM8916_VDDMX_AO>, <0>;

    The other option is detaching the power domain again in probe(),
    after setting it up as parent domain:

	struct of_phandle_args parent, child;

	child.np = dev->of_node;
	child.args_count = 0;

	of_parse_phandle_with_args(dev->of_node, "power-domains",
 			           "#power-domain-cells", 0, &parent));
	of_genpd_add_subdomain(&parent, &child);

	/* Detach power domain since it's managed via the subdomain */
	dev_pm_domain_detach(dev, false);

    Is there a cleaner way to handle this? Mainly a question for Uffe.

 2. The OPP WARNing triggers with both variants because it just checks
    if "required-opps" has a single entry. I guess we need extra checks
    to exclude the "parent genpd" case compared to the "OPP" case.

	[    1.116244] WARNING: CPU: 2 PID: 36 at drivers/opp/of.c:331 _link_required_opps+0x180/0x1cc
	[    1.125897] Hardware name: Qualcomm Technologies, Inc. APQ 8016 SBC (DT)
	[    1.146887] pc : _link_required_opps+0x180/0x1cc
	[    1.146902] lr : _link_required_opps+0xdc/0x1cc
	[    1.276408] Call trace:
	[    1.283519]  _link_required_opps+0x180/0x1cc
	[    1.285779]  _of_add_table_indexed+0x61c/0xd40
	[    1.290292]  dev_pm_opp_of_add_table+0x10/0x18
	[    1.294546]  of_genpd_add_provider_simple+0x80/0x160
	[    1.298974]  cpr_probe+0x6a0/0x97c
	[    1.304092]  platform_probe+0x64/0xbc

It does seem to work correctly, with and without this patch. So I guess
another option might be to simply silence this WARN_ON(). :')

Thanks,
Stephan

[1]: https://lore.kernel.org/linux-pm/CAPDyKFoH5EOvRRKy-Bgp_B9B3rf=PUKK5N45s5PNgfBi55PaOQ@mail.gmail.com/