Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Content-Type: text/plain;
        charset=utf-8
Mime-Version: 1.0 (1.0)
Subject: Re: [PATCH v4] x86/power: Fix 'nosmt' vs. hibernation triple fault during resume
From:   Andy Lutomirski <luto@amacapital.net>
In-Reply-To: <nycvar.YFH.7.76.1905311628330.1962@cbobk.fhfr.pm>
Date:   Fri, 31 May 2019 07:46:44 -0700
Cc:     Andy Lutomirski <luto@kernel.org>,
        "Rafael J. Wysocki" <rjw@rjwysocki.net>,
        Josh Poimboeuf <jpoimboe@redhat.com>,
        "Rafael J. Wysocki" <rafael@kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        the arch/x86 maintainers <x86@kernel.org>,
        Pavel Machek <pavel@ucw.cz>, Ingo Molnar <mingo@redhat.com>,
        Borislav Petkov <bp@alien8.de>,
        "H. Peter Anvin" <hpa@zytor.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Linux PM <linux-pm@vger.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <B7AC83ED-3F11-42B9-8506-C842A5937B50@amacapital.net>
References: <nycvar.YFH.7.76.1905282326360.1962@cbobk.fhfr.pm> <20190531051456.fzkvn62qlkf6wqra@treble> <nycvar.YFH.7.76.1905311045240.1962@cbobk.fhfr.pm> <5564116.e9OFvgDRbB@kreacher> <CALCETrUpseta+NrhVwzzVFTe-BkBHtDUJBO22ci3mAsVR+XOog@mail.gmail.com> <nycvar.YFH.7.76.1905311628330.1962@cbobk.fhfr.pm>
To:     Jiri Kosina <jikos@kernel.org>
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk


> On May 31, 2019, at 7:31 AM, Jiri Kosina <jikos@kernel.org> wrote:
>=20
>> On Fri, 31 May 2019, Andy Lutomirski wrote:
>>=20
>> 2. Put the CPU all the way to sleep by sending it an INIT IPI.
>>=20
>> Version 2 seems very simple and robust.  Is there a reason we can't do
>> it?  We obviously don't want to do it for normal offline because it
>> might be a high-power state, but a cpu in the wait-for-SIPI state is
>> not going to exit that state all by itself.
>>=20
>> The patch to implement #2 should be short and sweet as long as we are
>> careful to only put genuine APs to sleep like this.  The only downside
>> I can see is that an new kernel resuming and old kernel that was
>> booted with nosmt is going to waste power, but I don't think that's a
>> showstopper.
>=20
> Well, if *that* is not an issue, than the original 3-liner that just=20
> forces them to 'hlt' [1] would be good enough as well.
>=20
>=20

Seems okay to me as long as we=E2=80=99re confident we won=E2=80=99t get a s=
purious interrupt.

In general, I don=E2=80=99t think we=E2=80=99re ever suppose to rely on mwai=
t *staying* asleep.  As I understand it, mwait can wake up whenever it wants=
, and the only real guarantee we have is that the CPU makes some effort to s=
tay asleep until an interrupt is received or the monitor address is poked.

As a trivial example, if we are in a VM and we get scheduled out at any poin=
t between MONITOR and the eventual intentional wakeup, we=E2=80=99re toast. S=
ame if we get an SMI due to bad luck or due to a thermal event happening sho=
rtly after pushing the power button to resume from hibernate.

For that matter, what actually happens if we get an SMI while halted?  Does R=
SM go directly to sleep or does it re-fetch the HLT?

It seems to me that we should just avoid the scenario where we have IP point=
ed to a bogus address and we just cross our fingers and hope the CPU doesn=E2=
=80=99t do anything.

I think that, as a short term fix, we should use HLT and, as a long term fix=
, we should either keep the CPU state fully valid or we should hard-offline t=
he CPU using documented mechanisms, e.g. the WAIT-for-SIPI state.=