Received-SPF: pass (google.com: domain of linux-kernel+bounces-138952-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Message-ID: <e94d208e-964f-cfc0-60a4-fe70db52bec1@amd.com>
Date: Wed, 10 Apr 2024 11:08:11 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.15.1
To: Kai Huang <kai.huang@intel.com>, linux-kernel@vger.kernel.org
Cc: x86@kernel.org, dave.hansen@intel.com, bp@alien8.de,
 kirill.shutemov@linux.intel.com, tglx@linutronix.de, mingo@redhat.com,
 hpa@zytor.com, luto@kernel.org, peterz@infradead.org,
 rick.p.edgecombe@intel.com, ashish.kalra@amd.com, chao.gao@intel.com,
 bhe@redhat.com, nik.borisov@suse.com, pbonzini@redhat.com, seanjc@google.com
References: <cover.1712493366.git.kai.huang@intel.com>
 <33b985a8f4346f4bcf0944eaf37193a906b11af3.1712493366.git.kai.huang@intel.com>
Content-Language: en-US
From: Tom Lendacky <thomas.lendacky@amd.com>
Subject: Re: [PATCH v3 1/5] x86/kexec: do unconditional WBINVD for bare-metal
 in stop_this_cpu()
In-Reply-To: <33b985a8f4346f4bcf0944eaf37193a906b11af3.1712493366.git.kai.huang@intel.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?eU00RVVDMVdqa01raXpuK0s1U3d3eS9jWEh2cVJBNURLVTRaY2lTYmtQY21v?=
 =?utf-8?B?S29Iak1TMnd1SUtrSU5OZEV3VmdsQytuNUlxeExhTXdrcDBGSFEyeGcyM1ZP?=
 =?utf-8?B?bDVaTkhmSzJqUEFrSnZKQjZlL3pCSE1pOGI4Y1JqMloyaEhudFI0ZDJaOUNS?=
 =?utf-8?B?L28waU5VY1NydFdiQUcrd3BuMjNzTDVULzZoRkZXUHVta0hkaGQ5SlRxNDdN?=
 =?utf-8?B?YjY0dThvUXpsT1A5MjhEMW5OcFJaZzBrRWtyTk1zVllIbnlYY2RDZXBHenA2?=
 =?utf-8?B?NnplanZmYVR3SVBUeGdobkpFUWdUMHRncWJZWnNmam9BN2pUVXBMSExqek9l?=
 =?utf-8?B?MXdYclBWS09Xa3ZmSGpzQUhWVDJQRm81aUhXRVg3YXlrVGxyYVhkb1Q5V0lM?=
 =?utf-8?B?L2hVQ0E0QUoxa0VFdDgwd2tuZGpjOElwV2FjN1k1SmlKeU9zcGlHdDNFcE5y?=
 =?utf-8?B?VmJQWmVudWxrMWR3WTUxaFkzOEc2SUlPTHgzc04zWnVyL3hqSWtTWUNyK2h3?=
 =?utf-8?B?N3JzWXdpTXIzWjZPUzlWa1NZVTBkcEtDUkNYRmhJd2EzS1hFa25UQUZGbmJk?=
 =?utf-8?B?MkVZakt0UTE3QmVNS3o0b0ZtUDMyRGFxdHR5aDBVTVNZWlhXenBaajNXVTlV?=
 =?utf-8?B?a2VUeGFBV21xL1JFYXk2NEJUVENrdDMrUGdrR29McjJYdDU4Njl3dzRiclo2?=
 =?utf-8?B?MG9vR3p4R2tzZmhEN01lbGdrK2I4VHRhTDlJTE9nQStkSXVxSTlkZ0Nnd1pC?=
 =?utf-8?B?YUNHcVhNOVVDK24xZDFpWU0wOWhlWTdTKysremtweUYzWkVPSjR3QVFhZGNU?=
 =?utf-8?B?blhxZGVMZzJHT3BPbEJidm0rU0tBMHRCeW03aEJHMzVLWUxCVDU2TXh1bGFj?=
 =?utf-8?B?N3Ywa3IrV09EakVuR29rQnRxKzQwcDdOYmRhaTJCbVNwZVdsb09aczRBa3dG?=
 =?utf-8?B?aS8vSEVNYk1FUE51Z0pySHZIQ2JaNzRkc0ZFSWdRcTkvTVo1d2tkMXNiaEZj?=
 =?utf-8?B?M0FhMzlrQStaZFpJKzNCbGhsQ2crYXdSUUtpblNJSTNzZlJZYjEvWWw0TFdN?=
 =?utf-8?B?YnEzNG9FTTU5Q0FYVGRIQ0dNc0RzY2RKbHlLeCtUOFpZRkdEbWZWbTNsV3hV?=
 =?utf-8?B?aWJhaGEzZVFxWmhPc0U5bWM4eXRmYTdtcW9IVklNek9oS0ZpUXptYUhIaVFq?=
 =?utf-8?B?YmN4aWpFN3hCbWxnNjdaTTdrVklkNFl6TzAyN0kvSXc3L1Vndi9KNWM0R3kw?=
 =?utf-8?B?YWEwcGpxTHRmcUV2RkF6Qm40VUw3aVRYUVd4RGU3UGhvZjdZYXo5ckFLcmNF?=
 =?utf-8?B?NVV2cm41SlM5N0s2MVE4a2EyaGUzcTRLc3hGSFA5b0dJUmZEUFJ4NmJsMTNE?=
 =?utf-8?B?S3ZqRHJmVSt0ei9nNXNVUk1Qd0Vta0tmcVVIOHNYTjdZWUJFL3BxeVlRWTNn?=
 =?utf-8?B?NklZLzc0cDQ5a240aVZzZFAyRVNvUGcwTlgwOXdZWXNNM0kwZjlUNGU3SkZh?=
 =?utf-8?B?R0JFbGhzN2paKzd5RW1PQ2NEVDVaNW5DaHk3aDhqT3BTQzlzZlhwQnhiS0ta?=
 =?utf-8?B?MHcyanYwbW9kbDd5U1M1QmZaL3M1ZVNZMUdBRWpaUk1nN3hTNXNLQXl6QTVH?=
 =?utf-8?B?d0kyaTk0ZHpkTWF6cTExQUtCb1BCanB4VGRDb0FsWjFuSlVqTjl1MU04Tmda?=
 =?utf-8?B?MnZxSk5tanQyU1FxTm1SUjFDM1NiU0FuWGhjTWRRS0lwc1N4ZU5QbGtXaTc4?=
 =?utf-8?B?eitLbS9PYWN2bCtObS9Ib0dySmJHWTJheDJnSzQzZWVnd1h3Y0ViOHZYN0Jy?=
 =?utf-8?B?ZVhNZ2dhc0VRZ3MxWTBJS2gvRmRsbFFOVHhHQTIrVWQ0cU92eUExSHAzam91?=
 =?utf-8?B?ZGRCTHkyYk5KL1B2V3pReXlYUklyTTFxTEZ4UHBuOXQrZWVZMy9MTVEyVURI?=
 =?utf-8?B?VGxMOGs1QjNkS0hYN2hwVU92dzVYbURqakJSYVhveFc2d2FlOGxHVENIaUk2?=
 =?utf-8?B?RGNWc0xsQnhpTFE5MWJJOFA4Q1NKUTI5ZDJQRE5ldjdyL0dRd3l3TEJXT1R2?=
 =?utf-8?B?OXdiSml0QTIrenZsMXFMWWZMSTQrbXpQdVpoeDdVVm1DVEVQQ04zNW8rVnJ2?=
 =?utf-8?Q?fggIBERpKQovL6cNJ7lf+Wqzh?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ae57e548-d644-4674-0c68-08dc59786d10
X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5732.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Apr 2024 16:08:14.2279
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: lJVzM2ycKrIrLgmlTKFtlvlYodMneEj3Ll7maTLzKfQXOnCPqbWaDPBHtwYb+q2WDZqelOnw0JwvhdNsErMgxA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB7913

On 4/7/24 07:44, Kai Huang wrote:

> diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c
> index b8441147eb5e..5ba8a9c1e47a 100644
> --- a/arch/x86/kernel/process.c
> +++ b/arch/x86/kernel/process.c
> @@ -813,18 +813,16 @@ void __noreturn stop_this_cpu(void *dummy)
>   	mcheck_cpu_clear(c);
>   
>   	/*
> -	 * Use wbinvd on processors that support SME. This provides support
> -	 * for performing a successful kexec when going from SME inactive
> -	 * to SME active (or vice-versa). The cache must be cleared so that
> -	 * if there are entries with the same physical address, both with and
> -	 * without the encryption bit, they don't race each other when flushed
> -	 * and potentially end up with the wrong entry being committed to
> -	 * memory.
> +	 * The kernel could leave caches in incoherent state on SME/TDX
> +	 * capable platforms.  Flush cache to avoid silent memory
> +	 * corruption for these platforms.
>   	 *
> -	 * Test the CPUID bit directly because the machine might've cleared
> -	 * X86_FEATURE_SME due to cmdline options.
> +	 * stop_this_cpu() is not a fast path, just do unconditional
> +	 * WBINVD for simplicity.  But only do WBINVD for bare-metal
> +	 * as TDX guests and SEV-ES/SEV-SNP guests will get unexpected
> +	 * (and unnecessary) #VE and may unable to handle.

In addition to Kirill's comment on #VE...

This last part of the comment reads a bit odd since you say 
unconditional and then say only do WBINVD for bare-metal. Maybe 
something like this makes it a bit clearer?:

For TDX and SEV-ES/SEV-SNP guests, a WBINVD may cause an exception (#VE 
or #VC). However, all exception handling has been torn down at this 
point, so this would cause the guest to crash. Since memory within these 
types of guests is coherent only issue the WBINVD on bare-metal.

And you can expand the comment block out to at least 80 characters to 
make it more compact.

Thanks,
Tom

>   	 */
> -	if (c->extended_cpuid_level >= 0x8000001f && (cpuid_eax(0x8000001f) & BIT(0)))
> +	if (!boot_cpu_has(X86_FEATURE_HYPERVISOR))
>   		native_wbinvd();
>   
>   	/*