Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Reply-To: Sean Christopherson <seanjc@google.com>
Date:   Mon, 14 Nov 2022 23:34:38 +0000
Mime-Version: 1.0
Message-ID: <20221114233441.3895891-1-seanjc@google.com>
Subject: [PATCH v3 0/3] x86/crash: Fix double NMI shootdown bug
From:   Sean Christopherson <seanjc@google.com>
To:     Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        Dave Hansen <dave.hansen@linux.intel.com>, x86@kernel.org
Cc:     "H. Peter Anvin" <hpa@zytor.com>, linux-kernel@vger.kernel.org,
        "Guilherme G . Piccoli" <gpiccoli@igalia.com>,
        Vitaly Kuznetsov <vkuznets@redhat.com>,
        Paolo Bonzini <pbonzini@redhat.com>,
        Tom Lendacky <thomas.lendacky@amd.com>,
        Sean Christopherson <seanjc@google.com>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

Tom,

I Cc'd you this time around because the APM doesn't explicitly state that
GIF is set when EFER.SVME is disabled.  KVM's nSVM emulation does set GIF
in this case, but it's not clear whether or not KVM is making up behavior.
If clearing EFER.SVME doesn't set GIF, then patch 1 needs to be modified
to try STGI before clearing EFER.SVME, e.g. if a crash is initiated from
KVM between CLGI and STGI.  Responding CPUs are "safe" because GIF=0 also
blocks NMIs, but the initiating CPU might leave GIF=0 when jumping into
the new kernel.

Fix a double NMI shootdown bug found and debugged by Guilherme, who did all
the hard work.  NMI shootdown is a one-time thing; the handler leaves NMIs
blocked and enters halt.  At best, a second (or third...) shootdown is an
expensive nop, at worst it can hang the kernel and prevent kexec'ing into
a new kernel, e.g. prior to the hardening of register_nmi_handler(), a
double shootdown resulted in a double list_add(), which is fatal when running
with CONFIG_BUG_ON_DATA_CORRUPTION=y.

With the "right" kexec/kdump configuration, emergency_vmx_disable_all() can
be reached after kdump_nmi_shootdown_cpus() (currently the only two users
of nmi_shootdown_cpus()).

To fix, move the disabling of virtualization into crash_nmi_callback(),
remove emergency_vmx_disable_all()'s callback, and do a shootdown for
emergency_vmx_disable_all() if and only if a shootdown hasn't yet occurred.
The only thing emergency_vmx_disable_all() cares about is disabling VMX/SVM
(obviously), and since I can't envision a use case for an NMI shootdown that
doesn't want to disable virtualization, doing that in the core handler means
emergency_vmx_disable_all() only needs to ensure _a_ shootdown occurs, it
doesn't care when that shootdown happened or what callback may have run.

Patch 2 is a related bug fix found while exploring ideas for patch 1.
Patch 3 is a cleanup to try to prevent future "fixed VMX but not SVM"
style bugs.

v3:
  - Re-collect Guilherme's Tested-by.
  - Tweak comment in patch 1 to reference STGI instead of CLGI.
  - Celebrate this series' half-birthday.

v2:
  - Use a NULL handler and crash_ipi_issued instead of a magic nop
    handler. [tglx]
  - Add comments to call out that modifying the existing handler
    once the NMI is sent may cause explosions.
  - Add a patch to cleanup cpu_emergency_vmxoff().
  - https://lore.kernel.org/all/20220518001647.1291448-1-seanjc@google.com

v1: https://lore.kernel.org/all/20220511234332.3654455-1-seanjc@google.com

Sean Christopherson (3):
  x86/crash: Disable virt in core NMI crash handler to avoid double
    shootdown
  x86/reboot: Disable virtualization in an emergency if SVM is supported
  x86/virt: Fold __cpu_emergency_vmxoff() into its sole caller

 arch/x86/include/asm/reboot.h  |  1 +
 arch/x86/include/asm/virtext.h | 14 +-----
 arch/x86/kernel/crash.c        | 16 +-----
 arch/x86/kernel/reboot.c       | 89 +++++++++++++++++++++++++---------
 4 files changed, 69 insertions(+), 51 deletions(-)


base-commit: dacca1e5e75d7c1297f1334cdc10491dcdd1b2b8
-- 
2.38.1.431.g37b22c650d-goog