Message-ID: <5f741a4f-f37d-079b-d464-59045ebef1ce@marcan.st>
Date:   Thu, 9 Feb 2023 00:42:17 +0900
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.9.1
Content-Language: en-US
To:     Sudeep Holla <sudeep.holla@arm.com>, Kazuki <kazukih0205@gmail.com>
Cc:     linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org,
        linux-arm-kernel@lists.infradead.org,
        "Rafael J. Wysocki" <rafael@kernel.org>,
        Daniel Lezcano <daniel.lezcano@linaro.org>,
        Lorenzo Pieralisi <lpieralisi@kernel.org>,
        Sven Peter <sven@svenpeter.dev>,
        Len Brown <len.brown@intel.com>, Pavel Machek <pavel@ucw.cz>
References: <20230204152747.drte4uitljzngdt6@kazuki-mac>
 <20230206101239.dret3fv65cnzpken@bogus>
 <20230207194818.exskn3dhyzqwr32v@kazuki-mac>
 <20230208103511.w7jzxw6spy6humdn@bogus>
From:   Hector Martin <marcan@marcan.st>
Subject: Re: s2idle breaks on machines without cpuidle support
In-Reply-To: <20230208103511.w7jzxw6spy6humdn@bogus>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 08/02/2023 19.35, Sudeep Holla wrote:
> On Wed, Feb 08, 2023 at 04:48:18AM +0900, Kazuki wrote:
>> On Mon, Feb 06, 2023 at 10:12:39AM +0000, Sudeep Holla wrote:
>>>
>>> What do you mean by break ? More details on the observation would be helpful.
>> For example, CLOCK_MONOTONIC doesn't stop even after suspend since
>> these chain of commands don't get called.
>>
>> call_cpuidle_s2idle->cpuidle_enter_s2idle->enter_s2idle_proper->tick_freeze->sched_clock_suspend (Function that pauses CLOCK_MONOTONIC)
>>
>> Which in turn causes programs like systemd to crash since it doesn't
>> expect this.
> 
> Yes expected IIUC. The per-cpu timers and counters continue to tick in
> WFI and hence CLOCK_MONOTONIC can't stop.

The hardware counters would also keep ticking in "real" s2idle (with
hypothetical PSCI idle support) and often in full suspend. There is a
flag for this (CLOCK_SOURCE_SUSPEND_NONSTOP) that is set by default for
ARM. So this isn't why CLOCK_MONOTONIC isn't stopping; there is
machinery to make the kernel's view of time stop anyway, it's just not
being invoked here.

This is somewhat orthogonal to the issue of PSCI. These machines can't
physically support PSCI and KVM at the same time (they do not have EL3),
so PSCI is not an option. We will be starting a conversation about how
to provide something "like" PSCI over some other sort of transport to
solve this soon. So that will "fix" this issue once it's all implemented.

However, these machines aren't the only ones without PSCI (search for
"spin-table" in arch/arm64/boot/dts, this isn't new and these aren't the
first). It seems broken that Linux currently implements s2idle in such a
way that it violates the userspace clock behavior contract on systems
without a cpuidle driver (and does so silently, to make it worse). So
that should be fixed regardless of whether we end up coming up with a
PSCI alternative or not for these platforms. There's no fundamental
reason why s2idle can't work properly with plain WFI.

- Hector