Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3;
Date:   Mon, 25 Sep 2023 12:16:32 -0700
In-Reply-To: <8c6a1fc8-2ac5-4767-8b02-9ef56434724e@maciej.szmigiero.name>
Mime-Version: 1.0
References: <0ffde769702c6cdf6b6c18e1dcb28b25309af7f7.1695659717.git.maciej.szmigiero@oracle.com>
 <ZRHRsgjhOmIrxo0W@google.com> <8c6a1fc8-2ac5-4767-8b02-9ef56434724e@maciej.szmigiero.name>
Message-ID: <ZRHckCMwOv3jfSs7@google.com>
Subject: Re: [PATCH] KVM: x86: Ignore MSR_AMD64_BU_CFG access
From:   Sean Christopherson <seanjc@google.com>
To:     "Maciej S. Szmigiero" <mail@maciej.szmigiero.name>
Cc:     Paolo Bonzini <pbonzini@redhat.com>,
        Borislav Petkov <bp@alien8.de>, kvm@vger.kernel.org,
        x86@kernel.org, linux-kernel@vger.kernel.org,
        Tom Lendacky <thomas.lendacky@amd.com>
Content-Type: text/plain; charset="us-ascii"
Precedence: bulk

+Tom

On Mon, Sep 25, 2023, Maciej S. Szmigiero wrote:
> On 25.09.2023 20:30, Sean Christopherson wrote:
> >>
> >> Hyper-V enabled Windows Server 2022 KVM VM cannot be started on Zen1 Ryzen
> >> since it crashes at boot with SYSTEM_THREAD_EXCEPTION_NOT_HANDLED +
> >> STATUS_PRIVILEGED_INSTRUCTION (in other words, because of an unexpected #GP
> >> in the guest kernel).
> >>
> >> This is because Windows tries to set bit 8 in MSR_AMD64_BU_CFG and can't
> >> handle receiving a #GP when doing so.
> >
> > Any idea why?
>
> I guess it is trying to set some chicken bit?
>
> By the way, I tested Windows Server 2019 now - it has the same problem.
>
> So likely Windows 11 and newer version of Windows 10 have it, too.

...

> > > diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
> > > index 1d111350197f..c80a5cea80c4 100644
> > > --- a/arch/x86/include/asm/msr-index.h
> > > +++ b/arch/x86/include/asm/msr-index.h
> > > @@ -553,6 +553,7 @@
> > >   #define MSR_AMD64_CPUID_FN_1		0xc0011004
> > >   #define MSR_AMD64_LS_CFG		0xc0011020
> > >   #define MSR_AMD64_DC_CFG		0xc0011022
> > > +#define MSR_AMD64_BU_CFG		0xc0011023
> > 
> > What document actually defines this MSR?  All of the PPRs I can find for Family 17h
> > list it as:
> > 
> >     MSRC001_1023 [Table Walker Configuration] (Core::X86::Msr::TW_CFG)
> 
> It's partially documented in various AMD BKDGs, however I couldn't find
> any definition for this particular bit (8) - other than that it is reserved.

I found it as MSR_AMD64_BU_CFG for Model 16h, but that's Jaguar/Puma, not Zen1.
My guess is that Windows is trying to write this thing:

  MSRC001_1023 [Table Walker Configuration] (Core::X86::Msr::TW_CFG)
  Read-write. Reset: 0000_0000_0000_0000h.
  _lthree0_core[3,1]; MSRC001_1023

  Bits   Description
  63:50  Reserved.
  49     TwCfgCombineCr0Cd: combine CR0_CD for both threads of a core. Read-write. Reset: 0. Init: BIOS,1.
         1=The host Cr0_Cd values from the two threads are OR'd together and used by both threads.
  48:0   Reserved.

Though that still doesn't explain bit 8...  Perhaps a chicken-bit related to yet
another speculation bug?

Boris or Tom, any idea what Windows is doing?  I doubt it changes our options in
terms of "fixing" this in KVM, but having a somewhat accurate/helpful changelog
would be nice.