MIME-Version: 1.0
References: <20230119184228.683892-1-krzysztof.kozlowski@linaro.org>
 <20230124153411.nwkbjgoqgkua2icm@bogus> <c247710b-369a-67d0-0637-b32192d25e4d@linaro.org>
In-Reply-To: <c247710b-369a-67d0-0637-b32192d25e4d@linaro.org>
From:   Ulf Hansson <ulf.hansson@linaro.org>
Date:   Wed, 25 Jan 2023 11:08:04 +0100
Message-ID: <CAPDyKFrS6OER2h3XmHMfbjBA09HT8LYN9A90TPjd0RPim3jMaw@mail.gmail.com>
Subject: Re: [PATCH v3] cpuidle: psci: Do not suspend topology CPUs on PREEMPT_RT
To:     Sudeep Holla <sudeep.holla@arm.com>,
        Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
Cc:     "Rafael J. Wysocki" <rafael@kernel.org>,
        Daniel Lezcano <daniel.lezcano@linaro.org>,
        Lorenzo Pieralisi <lpieralisi@kernel.org>,
        linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org,
        linux-arm-kernel@lists.infradead.org,
        Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
        Adrien Thierry <athierry@redhat.com>,
        Brian Masney <bmasney@redhat.com>,
        linux-rt-users@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Wed, 25 Jan 2023 at 08:43, Krzysztof Kozlowski
<krzysztof.kozlowski@linaro.org> wrote:
>
> On 24/01/2023 16:34, Sudeep Holla wrote:
> > On Thu, Jan 19, 2023 at 07:42:28PM +0100, Krzysztof Kozlowski wrote:
> >> The runtime Power Management of CPU topology is not compatible with
> >> PREEMPT_RT:
> >> 1. Core cpuidle path disables IRQs.
> >> 2. Core cpuidle calls cpuidle-psci.
> >> 3. cpuidle-psci in __psci_enter_domain_idle_state() calls
> >>    pm_runtime_put_sync_suspend() and pm_runtime_get_sync() which use
> >>    spinlocks (which are sleeping on PREEMPT_RT).
> >>
> >> Deep sleep modes are not a priority of Realtime kernels because the
> >> latencies might become unpredictable.  On the other hand the PSCI CPU
> >> idle power domain is a parent of other devices and power domain
> >> controllers, thus it cannot be simply skipped (e.g. on Qualcomm SM8250).
> >>
> >> Disable the runtime PM calls from cpuidle-psci, which effectively stops
> >> suspending the cpuidle PSCI domain.  This is a trade-off between making
> >> PREEMPT_RT working and still having a proper power domain hierarchy in
> >> the system.
> >>
> >> Cc: Adrien Thierry <athierry@redhat.com>
> >> Cc: Brian Masney <bmasney@redhat.com>
> >> Cc: linux-rt-users@vger.kernel.org
> >> Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
> >>
> >> ---
> >>
> >> Changes since v1:
> >> 1. Re-work commit msg.
> >> 2. Add note to Kconfig.
> >>
> >> Several other patches were dropped, as this is the only one actually
> >> needed.  It effectively stops PSCI cpuidle power domains from suspending
> >> thus solving all other issues I experienced.
> >> ---
> >>  drivers/cpuidle/Kconfig.arm    | 3 +++
> >>  drivers/cpuidle/cpuidle-psci.c | 4 ++--
> >>  2 files changed, 5 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/drivers/cpuidle/Kconfig.arm b/drivers/cpuidle/Kconfig.arm
> >> index 747aa537389b..24429b5bfd1c 100644
> >> --- a/drivers/cpuidle/Kconfig.arm
> >> +++ b/drivers/cpuidle/Kconfig.arm
> >> @@ -24,6 +24,9 @@ config ARM_PSCI_CPUIDLE
> >>        It provides an idle driver that is capable of detecting and
> >>        managing idle states through the PSCI firmware interface.
> >>
> >> +      The driver is not yet compatible with PREEMPT_RT: no idle states will
> >> +      be entered by CPUs on such kernel.
> >> +
> >
> > Any particular reason for even compiling this file in or allowing the
> > ARM_PSCI_CPUIDLE when PREEMPT_RT=y ? If we can't enter idle states, we
> > can as well compile this file out ?
>
> It's the power domain sued for other devices, so we need it. Otherwise
> other devices will keep waiting for this missing power domain provider.

Yes.

And we are still able to use those idle states that are solely per
CPU, which is probably nice to have. No?

Kind regards
Uffe