Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33;
Message-ID: <ac402dd4-8bf3-87a8-7ade-50d62997ce97@amd.com>
Date:   Mon, 25 Sep 2023 17:25:38 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.15.1
Subject: Re: [PATCH] KVM: x86: Ignore MSR_AMD64_BU_CFG access
Content-Language: en-US
To:     Sean Christopherson <seanjc@google.com>,
        "Maciej S. Szmigiero" <mail@maciej.szmigiero.name>
Cc:     Paolo Bonzini <pbonzini@redhat.com>,
        Borislav Petkov <bp@alien8.de>, kvm@vger.kernel.org,
        x86@kernel.org, linux-kernel@vger.kernel.org
References: <0ffde769702c6cdf6b6c18e1dcb28b25309af7f7.1695659717.git.maciej.szmigiero@oracle.com>
 <ZRHRsgjhOmIrxo0W@google.com>
 <8c6a1fc8-2ac5-4767-8b02-9ef56434724e@maciej.szmigiero.name>
 <ZRHckCMwOv3jfSs7@google.com>
From:   Tom Lendacky <thomas.lendacky@amd.com>
In-Reply-To: <ZRHckCMwOv3jfSs7@google.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?bXdjcTVEbmtid3pEVmpUZGJzemMyQUFnRERNdkFrLzhzOExZTEFtSlNxcG8x?=
 =?utf-8?B?Q1czN1lnZk81bVhndVZsQjJ1OXJoQVlhcklNYWNwb2hhL3B4U2V2QjloVTdG?=
 =?utf-8?B?MmZaM0xYSHNEWGdObEZOR2NFV3ROSU9BUzgvUmtTOS9VcjJuT2NBcUVLU0kx?=
 =?utf-8?B?VUdidEVYakMxeGZ1dHpWNlY1bWdUNmRxSUVYV3AzL2dYZVpWa0w5aDcrRkU2?=
 =?utf-8?B?cER2QVJyUStnR0VpYzNjUnRFOUI3QldkWTlUR2lwS3NiZnJ6alNtZGM1L0hX?=
 =?utf-8?B?QjFNNTR1S2xKalcrZ1NrK2RINGd3cG1FdkE3ZW1CMFl1U0pHSWNDUDkwQ1lZ?=
 =?utf-8?B?Z0FSdDNobkdGYWhVQS9XSHNEbG91WGcxTUh3eCtOTzZSaWQ2VldEV2dXL1d0?=
 =?utf-8?B?YjBwcHJudmFqSUVHdWdhaXNKLytNOVR2emt6RVA5d1ZwRHZaRGFmQThsTmtU?=
 =?utf-8?B?bVliTVBNV3B2eVdJS1htQ2EyKzZkL3lXYzQ0RHlqRWp5SkVhZzRObTBxYWxi?=
 =?utf-8?B?UnMyeXhUbFlDZFUreXltRWpyNVhtbVFDVmlLYTlUM3pLcWF5TUR0UlpIZVNp?=
 =?utf-8?B?VWVpMG55KzF1eEVwaFZPUXpoaStLS3VSOWI3d1ZpcENFVFJTRHRFMTU3VU9O?=
 =?utf-8?B?Q1JUSm90dHdJb0NwdUdzRy9pN3JVRnF5TFJ5SHVqUnFIbmU0cTRGanpBRlFt?=
 =?utf-8?B?L2IrVFJlMHdxT3dnejNYd0h2ejFrMU5ORDFrd1lnMmZWNExkSGk5dmZPaVhp?=
 =?utf-8?B?WERrZkptd3lrWThzb2E1RVlxSm9KZGltS09Rbmg4QlBnNmthUmNzQ0JWbGhR?=
 =?utf-8?B?aDdsMktYbk9DRVhCMkF1Qndac2ZPUWZPVFptNHY5RnNtQ0x5QTZ5bXY5OEVC?=
 =?utf-8?B?NDRXaTd3TndSNDhTdjRxbGdsdTlydGNZS3NpWCtvY1M0NXRPSUFtUlV0TVVB?=
 =?utf-8?B?NFppM2JSb1JWVnJtUHhydDVHTzU0Tkt2Y1o3NWxIUnpMOGlqUVZqTDVJcUpp?=
 =?utf-8?B?VHlkdlgzQ2d0Q0JybVNtNUdZVGk4TkU4MHZyTTdFenZvSWxsbHI0dmd4LzJE?=
 =?utf-8?B?NVF3bEtSSDViUFNyU1pxT2dqdkpZZ2RaczUzTXVkQ2srK21KUCtCOFlsYWND?=
 =?utf-8?B?RzNRQnh5UTVFOEdtT3NjVkdCOVhyU1lZUCtUYlE4TFVYcDB2UnQrWE41NFNr?=
 =?utf-8?B?VWJMUHBBeWJ2R0lGZVUwNTduNzFlNVU2ZUdnRlVjbGZRSm5HUFB5eHZjeTc3?=
 =?utf-8?B?TWlJenZNN1Z3V1RyNFRzemF1UVlBV2lJcC9DMFRjaDRNL3JRVDVDRStSTmhJ?=
 =?utf-8?B?QUk4Zzd3U28yekhQcnlrZmlrd3ErNmRNTnhZS0xNTFROcmRKYlRWVkZFRGJD?=
 =?utf-8?B?dHQ4UUk1ajJJdXlPNU9FbC9NSW1CMUZVWnpndWxqTXJKbnRrZUY4cHZhWkps?=
 =?utf-8?B?Q0RpelZsd0ZrMSsxTHAyM1Y0SmtzWjN3czdqbkFBYmVDQTgyYlhpVXRDTU5q?=
 =?utf-8?B?ajFvdm5mcmJWbUd6emJPZVFuZlZhYTE0TXBuUUxrdHpkbk91VDBsZ1J4ZzNu?=
 =?utf-8?B?ZzRsckxPbFZxWUxaQlVDNVQzelJqRjlPWGtYNTRPL0F1K0xFaTBNdzUzQ0h6?=
 =?utf-8?B?NzNMOGhFWXNUdTB6bGtKcXN3M0RMZkRMZmZpaG9DMmN2ZkJBMkZDdlRMUW5k?=
 =?utf-8?B?N3Q5TFN4b0ROanBBSEFQWElXY0o2Y2gyWkNsNVBXTlRjbFV1bUJJNWF2SHNn?=
 =?utf-8?B?eWRUaUtYR1FUYStEUWJRaVlIRTdtQTNjR1pCNG5zUXpzUFVtYmo4YXdPQXZC?=
 =?utf-8?B?aWx4RTRnUGREbC9Gc3hnWUZtaW8xVDkzTHM4eWxlQ3p0VmVCYndyelM5VUNH?=
 =?utf-8?B?MEtIOGNHSitoVmZkMXJRSlNnbEhpMXJtTTRVMWV2NEdKWXN6WC9BYTJyMnBx?=
 =?utf-8?B?bGxwbFlxL2pXTllnOFh0SEx5UU5DQkI3QUk1MHlXbWs1bVJpZTBFWHBGdGdQ?=
 =?utf-8?B?UE12dEZVdXAxYXdjeUZsb1hVYjhBbllQZmdCOHVDVGxPSzJsWEYwdThPb1lp?=
 =?utf-8?B?c04yNDBvdEQ0WmIxREROUThXZXcxbU05SUJabnVpMzZ3T21JMFZ1UXkrM2ZO?=
 =?utf-8?Q?AJ3Yh5UDaYHsowbzSrdMKF/mv?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c35c71e9-1348-4a4f-d180-08dbbe1659b7
X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5221.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Sep 2023 22:25:40.9080
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: pFYFtz4kNwwQxSJhtg14GzMKpMqIlkO7U5FA43C2ZTeBfyCWqMtypUwBMtxXSJnRxrUvve4O9LZ6ENH5bczxTg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4341
X-Spam-Status: No, score=-2.3 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS
	autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Mon, 25 Sep 2023 15:25:57 -0700 (PDT)

On 9/25/23 14:16, Sean Christopherson wrote:
> +Tom
> 
> On Mon, Sep 25, 2023, Maciej S. Szmigiero wrote:
>> On 25.09.2023 20:30, Sean Christopherson wrote:
>>>>
>>>> Hyper-V enabled Windows Server 2022 KVM VM cannot be started on Zen1 Ryzen
>>>> since it crashes at boot with SYSTEM_THREAD_EXCEPTION_NOT_HANDLED +
>>>> STATUS_PRIVILEGED_INSTRUCTION (in other words, because of an unexpected #GP
>>>> in the guest kernel).
>>>>
>>>> This is because Windows tries to set bit 8 in MSR_AMD64_BU_CFG and can't
>>>> handle receiving a #GP when doing so.
>>>
>>> Any idea why?
>>
>> I guess it is trying to set some chicken bit?
>>
>> By the way, I tested Windows Server 2019 now - it has the same problem.
>>
>> So likely Windows 11 and newer version of Windows 10 have it, too.
> 
> ...
> 
>>>> diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
>>>> index 1d111350197f..c80a5cea80c4 100644
>>>> --- a/arch/x86/include/asm/msr-index.h
>>>> +++ b/arch/x86/include/asm/msr-index.h
>>>> @@ -553,6 +553,7 @@
>>>>    #define MSR_AMD64_CPUID_FN_1		0xc0011004
>>>>    #define MSR_AMD64_LS_CFG		0xc0011020
>>>>    #define MSR_AMD64_DC_CFG		0xc0011022
>>>> +#define MSR_AMD64_BU_CFG		0xc0011023
>>>
>>> What document actually defines this MSR?  All of the PPRs I can find for Family 17h
>>> list it as:
>>>
>>>      MSRC001_1023 [Table Walker Configuration] (Core::X86::Msr::TW_CFG)
>>
>> It's partially documented in various AMD BKDGs, however I couldn't find
>> any definition for this particular bit (8) - other than that it is reserved.
> 
> I found it as MSR_AMD64_BU_CFG for Model 16h, but that's Jaguar/Puma, not Zen1.
> My guess is that Windows is trying to write this thing:
> 
>    MSRC001_1023 [Table Walker Configuration] (Core::X86::Msr::TW_CFG)
>    Read-write. Reset: 0000_0000_0000_0000h.
>    _lthree0_core[3,1]; MSRC001_1023
> 
>    Bits   Description
>    63:50  Reserved.
>    49     TwCfgCombineCr0Cd: combine CR0_CD for both threads of a core. Read-write. Reset: 0. Init: BIOS,1.
>           1=The host Cr0_Cd values from the two threads are OR'd together and used by both threads.
>    48:0   Reserved.
> 
> Though that still doesn't explain bit 8...  Perhaps a chicken-bit related to yet
> another speculation bug?
> 
> Boris or Tom, any idea what Windows is doing?  I doubt it changes our options in
> terms of "fixing" this in KVM, but having a somewhat accurate/helpful changelog
> would be nice.

It's definitely not related to a speculation bug, but I'm unsure what was 
told to Microsoft that has them performing that WRMSR. The patch does the 
proper thing, though, as a guest shouldn't be updating that setting.

And TW_CFG is the proper name of that MSR for Zen.

Thanks,
Tom