2022-07-21 05:35:26

by Martin Kepplinger

[permalink] [raw]
Subject: [PATCH v5 2/3] power: domain: handle genpd correctly when needing interrupts

If for example the power-domains' power-supply node (regulator) needs
interrupts to work, the current setup with noirq callbacks cannot
work; for example a pmic regulator on i2c, when suspending, usually already
times out during suspend_noirq:

[ 41.024193] buck4: failed to disable: -ETIMEDOUT

So fix system suspend and resume for these power-domains by using the
"outer" suspend/resume callbacks instead. Tested on the imx8mq-librem5 board,
but by looking at the dts, this will fix imx8mq-evk and possibly many other
boards too.

This is designed so that genpd providers just say "this genpd needs
interrupts" (by setting the flag) - without implying an implementation.

Initially system suspend problems had been discussed at
https://lore.kernel.org/linux-arm-kernel/[email protected]/
which led to discussing the pmic that contains the regulators which
serve as power-domain power-supplies:
https://lore.kernel.org/linux-pm/[email protected]/T/

Signed-off-by: Martin Kepplinger <[email protected]>
---
drivers/base/power/domain.c | 13 +++++++++++++
include/linux/pm_domain.h | 5 +++++
2 files changed, 18 insertions(+)

diff --git a/drivers/base/power/domain.c b/drivers/base/power/domain.c
index 5a2e0232862e..ef77700e0def 100644
--- a/drivers/base/power/domain.c
+++ b/drivers/base/power/domain.c
@@ -130,6 +130,7 @@ static const struct genpd_lock_ops genpd_spin_ops = {
#define genpd_is_active_wakeup(genpd) (genpd->flags & GENPD_FLAG_ACTIVE_WAKEUP)
#define genpd_is_cpu_domain(genpd) (genpd->flags & GENPD_FLAG_CPU_DOMAIN)
#define genpd_is_rpm_always_on(genpd) (genpd->flags & GENPD_FLAG_RPM_ALWAYS_ON)
+#define genpd_irq_on(genpd) (genpd->flags & GENPD_FLAG_IRQ_ON)

static inline bool irq_safe_dev_in_sleep_domain(struct device *dev,
const struct generic_pm_domain *genpd)
@@ -2079,6 +2080,13 @@ int pm_genpd_init(struct generic_pm_domain *genpd,
genpd->dev_ops.start = pm_clk_resume;
}

+ if (genpd_irq_on(genpd)) {
+ genpd->domain.ops.suspend = genpd_suspend_noirq;
+ genpd->domain.ops.resume = genpd_resume_noirq;
+ genpd->domain.ops.suspend_noirq = NULL;
+ genpd->domain.ops.resume_noirq = NULL;
+ }
+
/* The always-on governor works better with the corresponding flag. */
if (gov == &pm_domain_always_on_gov)
genpd->flags |= GENPD_FLAG_RPM_ALWAYS_ON;
@@ -2769,6 +2777,11 @@ static int __genpd_dev_pm_attach(struct device *dev, struct device *base_dev,
goto err;
dev_gpd_data(dev)->default_pstate = pstate;
}
+
+ if (pd->domain.ops.suspend_noirq && (pd->flags & GENPD_FLAG_IRQ_ON))
+ dev_err(dev, "PM domain %s needs irqs but uses noirq suspend\n",
+ pd->name);
+
return 1;

err:
diff --git a/include/linux/pm_domain.h b/include/linux/pm_domain.h
index 76bc9e3ef5ff..03bb86e43550 100644
--- a/include/linux/pm_domain.h
+++ b/include/linux/pm_domain.h
@@ -61,6 +61,10 @@
* GENPD_FLAG_MIN_RESIDENCY: Enable the genpd governor to consider its
* components' next wakeup when determining the
* optimal idle state.
+ *
+ * GENPD_FLAG_IRQ_ON: genpd needs irqs to be able to manage power
+ * on/off. Use the outer suspend/resume callbacks
+ * instead of noirq for example.
*/
#define GENPD_FLAG_PM_CLK BIT(0)
#define GENPD_FLAG_IRQ_SAFE BIT(1)
@@ -69,6 +73,7 @@
#define GENPD_FLAG_CPU_DOMAIN BIT(4)
#define GENPD_FLAG_RPM_ALWAYS_ON BIT(5)
#define GENPD_FLAG_MIN_RESIDENCY BIT(6)
+#define GENPD_FLAG_IRQ_ON BIT(7)

enum gpd_status {
GENPD_STATE_ON = 0, /* PM domain is on */
--
2.30.2


2022-07-21 11:29:12

by Ulf Hansson

[permalink] [raw]
Subject: Re: [PATCH v5 2/3] power: domain: handle genpd correctly when needing interrupts

On Thu, 21 Jul 2022 at 06:37, Martin Kepplinger
<[email protected]> wrote:
>
> If for example the power-domains' power-supply node (regulator) needs
> interrupts to work, the current setup with noirq callbacks cannot
> work; for example a pmic regulator on i2c, when suspending, usually already
> times out during suspend_noirq:
>
> [ 41.024193] buck4: failed to disable: -ETIMEDOUT
>
> So fix system suspend and resume for these power-domains by using the
> "outer" suspend/resume callbacks instead. Tested on the imx8mq-librem5 board,
> but by looking at the dts, this will fix imx8mq-evk and possibly many other
> boards too.
>
> This is designed so that genpd providers just say "this genpd needs
> interrupts" (by setting the flag) - without implying an implementation.
>
> Initially system suspend problems had been discussed at
> https://lore.kernel.org/linux-arm-kernel/[email protected]/
> which led to discussing the pmic that contains the regulators which
> serve as power-domain power-supplies:
> https://lore.kernel.org/linux-pm/[email protected]/T/
>
> Signed-off-by: Martin Kepplinger <[email protected]>
> ---
> drivers/base/power/domain.c | 13 +++++++++++++
> include/linux/pm_domain.h | 5 +++++
> 2 files changed, 18 insertions(+)
>
> diff --git a/drivers/base/power/domain.c b/drivers/base/power/domain.c
> index 5a2e0232862e..ef77700e0def 100644
> --- a/drivers/base/power/domain.c
> +++ b/drivers/base/power/domain.c
> @@ -130,6 +130,7 @@ static const struct genpd_lock_ops genpd_spin_ops = {
> #define genpd_is_active_wakeup(genpd) (genpd->flags & GENPD_FLAG_ACTIVE_WAKEUP)
> #define genpd_is_cpu_domain(genpd) (genpd->flags & GENPD_FLAG_CPU_DOMAIN)
> #define genpd_is_rpm_always_on(genpd) (genpd->flags & GENPD_FLAG_RPM_ALWAYS_ON)
> +#define genpd_irq_on(genpd) (genpd->flags & GENPD_FLAG_IRQ_ON)
>
> static inline bool irq_safe_dev_in_sleep_domain(struct device *dev,
> const struct generic_pm_domain *genpd)
> @@ -2079,6 +2080,13 @@ int pm_genpd_init(struct generic_pm_domain *genpd,
> genpd->dev_ops.start = pm_clk_resume;
> }
>
> + if (genpd_irq_on(genpd)) {
> + genpd->domain.ops.suspend = genpd_suspend_noirq;
> + genpd->domain.ops.resume = genpd_resume_noirq;
> + genpd->domain.ops.suspend_noirq = NULL;
> + genpd->domain.ops.resume_noirq = NULL;

Please move this a few lines above, just before we assign the _*noirq
callbacks. In this way you don't need to reset thosepointers.

> + }
> +
> /* The always-on governor works better with the corresponding flag. */
> if (gov == &pm_domain_always_on_gov)
> genpd->flags |= GENPD_FLAG_RPM_ALWAYS_ON;
> @@ -2769,6 +2777,11 @@ static int __genpd_dev_pm_attach(struct device *dev, struct device *base_dev,
> goto err;
> dev_gpd_data(dev)->default_pstate = pstate;
> }
> +
> + if (pd->domain.ops.suspend_noirq && (pd->flags & GENPD_FLAG_IRQ_ON))
> + dev_err(dev, "PM domain %s needs irqs but uses noirq suspend\n",
> + pd->name);
> +

This doesn't make sense, as it can never happen according to what we
do in pm_genpd_init().

What Lucas suggested in the other thread was to log a warning if a
device's (that gets attached to genpd) bus/driver has _*noirq
callbacks. That would make sense.

Thinking more about this, perhaps we should move to use the
_*late/early callbacks instead for genpd. This would decrease the
window of potential problematic consumers users (drivers/buses).

> return 1;
>
> err:
> diff --git a/include/linux/pm_domain.h b/include/linux/pm_domain.h
> index 76bc9e3ef5ff..03bb86e43550 100644
> --- a/include/linux/pm_domain.h
> +++ b/include/linux/pm_domain.h
> @@ -61,6 +61,10 @@
> * GENPD_FLAG_MIN_RESIDENCY: Enable the genpd governor to consider its
> * components' next wakeup when determining the
> * optimal idle state.
> + *
> + * GENPD_FLAG_IRQ_ON: genpd needs irqs to be able to manage power
> + * on/off. Use the outer suspend/resume callbacks
> + * instead of noirq for example.
> */
> #define GENPD_FLAG_PM_CLK BIT(0)
> #define GENPD_FLAG_IRQ_SAFE BIT(1)
> @@ -69,6 +73,7 @@
> #define GENPD_FLAG_CPU_DOMAIN BIT(4)
> #define GENPD_FLAG_RPM_ALWAYS_ON BIT(5)
> #define GENPD_FLAG_MIN_RESIDENCY BIT(6)
> +#define GENPD_FLAG_IRQ_ON BIT(7)
>
> enum gpd_status {
> GENPD_STATE_ON = 0, /* PM domain is on */
> --
> 2.30.2
>

Kind regards
Uffe

2022-07-21 17:22:47

by Martin Kepplinger

[permalink] [raw]
Subject: Re: [PATCH v5 2/3] power: domain: handle genpd correctly when needing interrupts

Am Donnerstag, dem 21.07.2022 um 13:16 +0200 schrieb Ulf Hansson:
> On Thu, 21 Jul 2022 at 06:37, Martin Kepplinger
> <[email protected]> wrote:
> >
> > If for example the power-domains' power-supply node (regulator)
> > needs
> > interrupts to work, the current setup with noirq callbacks cannot
> > work; for example a pmic regulator on i2c, when suspending, usually
> > already
> > times out during suspend_noirq:
> >
> > [   41.024193] buck4: failed to disable: -ETIMEDOUT
> >
> > So fix system suspend and resume for these power-domains by using
> > the
> > "outer" suspend/resume callbacks instead. Tested on the imx8mq-
> > librem5 board,
> > but by looking at the dts, this will fix imx8mq-evk and possibly
> > many other
> > boards too.
> >
> > This is designed so that genpd providers just say "this genpd needs
> > interrupts" (by setting the flag) - without implying an
> > implementation.
> >
> > Initially system suspend problems had been discussed at
> >
> > https://lore.kernel.org/linux-arm-kernel/[email protected]/
> > which led to discussing the pmic that contains the regulators which
> > serve as power-domain power-supplies:
> >
> > https://lore.kernel.org/linux-pm/[email protected]/T/
> >
> > Signed-off-by: Martin Kepplinger <[email protected]>
> > ---
> >  drivers/base/power/domain.c | 13 +++++++++++++
> >  include/linux/pm_domain.h   |  5 +++++
> >  2 files changed, 18 insertions(+)
> >
> > diff --git a/drivers/base/power/domain.c
> > b/drivers/base/power/domain.c
> > index 5a2e0232862e..ef77700e0def 100644
> > --- a/drivers/base/power/domain.c
> > +++ b/drivers/base/power/domain.c
> > @@ -130,6 +130,7 @@ static const struct genpd_lock_ops
> > genpd_spin_ops = {
> >  #define genpd_is_active_wakeup(genpd)  (genpd->flags &
> > GENPD_FLAG_ACTIVE_WAKEUP)
> >  #define genpd_is_cpu_domain(genpd)     (genpd->flags &
> > GENPD_FLAG_CPU_DOMAIN)
> >  #define genpd_is_rpm_always_on(genpd)  (genpd->flags &
> > GENPD_FLAG_RPM_ALWAYS_ON)
> > +#define genpd_irq_on(genpd)            (genpd->flags &
> > GENPD_FLAG_IRQ_ON)
> >
> >  static inline bool irq_safe_dev_in_sleep_domain(struct device
> > *dev,
> >                 const struct generic_pm_domain *genpd)
> > @@ -2079,6 +2080,13 @@ int pm_genpd_init(struct generic_pm_domain
> > *genpd,
> >                 genpd->dev_ops.start = pm_clk_resume;
> >         }
> >
> > +       if (genpd_irq_on(genpd)) {
> > +               genpd->domain.ops.suspend = genpd_suspend_noirq;
> > +               genpd->domain.ops.resume = genpd_resume_noirq;
> > +               genpd->domain.ops.suspend_noirq = NULL;
> > +               genpd->domain.ops.resume_noirq = NULL;
>
> Please move this a few lines above, just before we assign the _*noirq
> callbacks. In this way you don't need to reset thosepointers.

good point, thanks.

>
> > +       }
> > +
> >         /* The always-on governor works better with the
> > corresponding flag. */
> >         if (gov == &pm_domain_always_on_gov)
> >                 genpd->flags |= GENPD_FLAG_RPM_ALWAYS_ON;
> > @@ -2769,6 +2777,11 @@ static int __genpd_dev_pm_attach(struct
> > device *dev, struct device *base_dev,
> >                         goto err;
> >                 dev_gpd_data(dev)->default_pstate = pstate;
> >         }
> > +
> > +       if (pd->domain.ops.suspend_noirq && (pd->flags &
> > GENPD_FLAG_IRQ_ON))
> > +               dev_err(dev, "PM domain %s needs irqs but uses
> > noirq suspend\n",
> > +                       pd->name);
> > +
>
> This doesn't make sense, as it can never happen according to what we
> do in pm_genpd_init().
>
> What Lucas suggested in the other thread was to log a warning if a
> device's (that gets attached to genpd) bus/driver has _*noirq
> callbacks. That would make sense.
>
> Thinking more about this, perhaps we should move to use the
> _*late/early callbacks instead for genpd. This would decrease the
> window of potential problematic consumers users (drivers/buses).
>

ok, I'll probably send another version without this. I'm leaving for
vacations for a few weeks soon, so I'd rather do the essentials only
here and take notes about what could be added later, if that's ok.

thank you very much!

martin


> >         return 1;
> >
> >  err:
> > diff --git a/include/linux/pm_domain.h b/include/linux/pm_domain.h
> > index 76bc9e3ef5ff..03bb86e43550 100644
> > --- a/include/linux/pm_domain.h
> > +++ b/include/linux/pm_domain.h
> > @@ -61,6 +61,10 @@
> >   * GENPD_FLAG_MIN_RESIDENCY:   Enable the genpd governor to
> > consider its
> >   *                             components' next wakeup when
> > determining the
> >   *                             optimal idle state.
> > + *
> > + * GENPD_FLAG_IRQ_ON:          genpd needs irqs to be able to
> > manage power
> > + *                             on/off. Use the outer
> > suspend/resume callbacks
> > + *                             instead of noirq for example.
> >   */
> >  #define GENPD_FLAG_PM_CLK              BIT(0)
> >  #define GENPD_FLAG_IRQ_SAFE            BIT(1)
> > @@ -69,6 +73,7 @@
> >  #define GENPD_FLAG_CPU_DOMAIN          BIT(4)
> >  #define GENPD_FLAG_RPM_ALWAYS_ON       BIT(5)
> >  #define GENPD_FLAG_MIN_RESIDENCY       BIT(6)
> > +#define GENPD_FLAG_IRQ_ON              BIT(7)
> >
> >  enum gpd_status {
> >         GENPD_STATE_ON = 0,     /* PM domain is on */
> > --
> > 2.30.2
> >
>
> Kind regards
> Uffe