Received-SPF: pass (google.com: domain of linux-kernel+bounces-108147-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Message-ID: <689bbd29-aaf0-452e-a97f-41b8e3aa6224@intel.com>
Date: Wed, 20 Mar 2024 10:20:50 +1300
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 2/5] x86/kexec: do unconditional WBINVD in
 relocate_kernel()
Content-Language: en-US
To: Tom Lendacky <thomas.lendacky@amd.com>, "Kirill A. Shutemov"
	<kirill.shutemov@linux.intel.com>
CC: <linux-kernel@vger.kernel.org>, <x86@kernel.org>, <dave.hansen@intel.com>,
	<bp@alien8.de>, <tglx@linutronix.de>, <mingo@redhat.com>, <hpa@zytor.com>,
	<luto@kernel.org>, <peterz@infradead.org>, <rick.p.edgecombe@intel.com>,
	<ashish.kalra@amd.com>, <chao.gao@intel.com>, <bhe@redhat.com>,
	<nik.borisov@suse.com>, <pbonzini@redhat.com>, <seanjc@google.com>
References: <cover.1710811610.git.kai.huang@intel.com>
 <e1d37efb8951eb1d38493687b10a21b23353e35a.1710811610.git.kai.huang@intel.com>
 <tvembdwwh4immxytlfzlhpvd42dlfsz7sddb7msk23kdduhu3t@ogpc66hklorv>
 <38fca2fa-11b2-4eb7-9e59-dc5d524d172e@amd.com>
From: "Huang, Kai" <kai.huang@intel.com>
In-Reply-To: <38fca2fa-11b2-4eb7-9e59-dc5d524d172e@amd.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?NG85cXZkS0dsREFWMC92eXVmdHVoUUpXelhDTUZqc1FLZGk3SGJDa0t2R1lw?=
 =?utf-8?B?ZWwzaXArdEVPMjNDUVJ4UmJkdFlvb3pQUlgwVFo2ZHViWkVGZHRobmViUXdD?=
 =?utf-8?B?clRLVVhzREZETmFweG4zdFUwL1IzbmZWMVEvdmZRbnJhNkp3aCtXdDJ5ME94?=
 =?utf-8?B?OGFyaXpwWEtzaWdCQ09sZzdIam5neEU0aVBCUTQ4OU5VajhTWDBwZG1WWGoy?=
 =?utf-8?B?VnhReldnMDF6VCtYK0VnSUJXNElsaGNQSFlucnZhR1RNUzdnM0NWcnJRdFVp?=
 =?utf-8?B?ZkNmUGc3NEhMa1RiNHo0RTVla3Vlbm53STE3SitHNUU4NWtVV3VZV1ozTGpt?=
 =?utf-8?B?cHNjUlN1eThaVGJyQ2hka0Y3U1FHN2dqcjVSYUVBSGdVRlAxSzUvZEU2blc5?=
 =?utf-8?B?V29URUJLWDI5WndZTU9wcVdvR2VoeDJpaHJEd1VaSGhHOEZob1VkOWdzNGcr?=
 =?utf-8?B?Skg1WUZNTTBQYzloS0NUdUdpNnFnekZQMGRCTm9LSGJ3NzdxenJhNDlEMW9B?=
 =?utf-8?B?encvSG9IUlk3aEo0Ti8zdjN5bERDOHNQSDFvVi9XUThFYllXa2ZUVTVPeXNT?=
 =?utf-8?B?MGNGWkdMR1FMeUpIUzBMeHczLzVPVzhYbS9rN3g4U3F6eVplWW9nSkV2OGh3?=
 =?utf-8?B?U2I5NXF1OEsyenQ5Ty9Md1J5KzYwYVNkZ3N6aHZQZW9XM1QwcDBKd2Rqc2ox?=
 =?utf-8?B?UVBHRkFJNW13VnNGdk95MzF5WGVsZ1dLdE90Y3dQREZNTGtIMktjNytkYkJ1?=
 =?utf-8?B?amFyN29QQkhMcTlFZGNWbDZIWkF1dldOOE1PUURqcWRZRENRWk1EdE9WT2dN?=
 =?utf-8?B?YnN3U0hSYkFhVUNwVncvTHNEUHQzR0s4ZHhNNnR2bngwNE94cFczMUUzNWt6?=
 =?utf-8?B?T3U4bm54cWhHVStoK3hNLzQvYklrZko2QktUMmVyS2ZrOW53c1RoZTU0Uloz?=
 =?utf-8?B?bzltQ0M4dktqOG1uenYxRXpoRXk0WGtLTnlIT0ViZkdQaEhEWHpEZXptM3FQ?=
 =?utf-8?B?WUlYTTZlWVhPTVlNUXpma09jd20wOGNKdDNyZkZTL3dYRVRpUXZpdEVaMnVF?=
 =?utf-8?B?b0dGS1Q0dXk2VzhTVWlqL0RBbXhFa01mTmc4Uk5VTHZUQlovblJWOUtHYjd3?=
 =?utf-8?B?QmRKdEZHMVdsK2RnanFqLzVGbWtzdEhSQ0dHZGgwckQyaW93MTNRNjlmNlpG?=
 =?utf-8?B?WGoyNitqMTJjWE95TUtHM0VCd242bmhIUmlxaDhuUzhwZEFpOHgrQ0lZb09a?=
 =?utf-8?B?NU5TaURvNkhkSGdsdTlyTFl5R1o5RzdOSG40TFAwWThTcnFJMzgwU2pTTVNJ?=
 =?utf-8?B?UWFpaVc5R1dSUE1yOUxWNVBxNVBqQytLdnorSnVrR3VWYUM0bmtad1JMZ2ZG?=
 =?utf-8?B?ZHpmRFlZUlNvOVpWbk43QUhzTzlySmlHNmMyMTE5clpDS29VNnJMaEJ1Nm54?=
 =?utf-8?B?S1FrNU5oNE1jczVBajE4UnpYNkNBWUhGQkhqdENONXJ4SC9jY0NQenlkczRw?=
 =?utf-8?B?QkUvWTQ3RzhLYnBWR3hwQ3IrZ2ZRSDUybUFub0d1eENiTzJ1Y1pNQmxYdEpR?=
 =?utf-8?B?SHhvb1h6SEZSOUxNTC9SbW96Y2JaQTFqY21wWTNUc1Jnd1hHN1BNWllCZVlq?=
 =?utf-8?B?VytoZFhOcjlwalFiU2JTN3VIRk9GNG9UMDgzQ2MvZExRTjREb1VxVVNQcDE1?=
 =?utf-8?B?NkVkaitFclZ6QUtweGFBaXJWNm5yMzNhdjY2NS9YR2ZjMUFmd3RuS0dHSGVt?=
 =?utf-8?B?eGhpc2FlREVjY0hDMmFZdm1qMFZIOTUyd3B5RVZjY0wrVHhRQzE5enZqckVn?=
 =?utf-8?B?bWxHM3A2T2lSaWtCbDVVZzg5bS9LVzR6L0RydkV6dEVNaFowMFoza29MZlMx?=
 =?utf-8?B?ZmNmL0oyRmlyejhNNENaNHVJU3MveUU3SlRpVGJVL283RXZYME0wY2VqdHoy?=
 =?utf-8?B?Ky85Uzc3TlZTSlh4bTllcUw1ZWdtR3A5bUovNFZnQTEwNlQ5Y2NQemt0NEl3?=
 =?utf-8?B?bDFhYWZlSnBnY0w1akMzeWhCeGltOFVWQmlLMnFGQWpHNGNKSVpmNVBtZTVC?=
 =?utf-8?B?TUI1d013R0ZVd1dadlJNZ0dwRHR4aFkwSWpTUGNrNWtSYnZPbnlDNndjY25R?=
 =?utf-8?Q?LMECNKTWP1aWIzkDrne+BsZLL?=
X-MS-Exchange-CrossTenant-Network-Message-Id: c052df86-268d-419f-c1fa-08dc485a7ade
X-MS-Exchange-CrossTenant-AuthSource: BL1PR11MB5978.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Mar 2024 21:21:02.7471
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: /4UPrq3sBnXi/pMvwoH8TwMBaGlzFBkp0o7Anq1RAEVBRaOV5F3R06LEaktP4r+/vgUBXpuB/l2lfKIYlY2A9g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR11MB8520
X-OriginatorOrg: intel.com


On 20/03/2024 3:38 am, Tom Lendacky wrote:
> On 3/19/24 06:13, Kirill A. Shutemov wrote:
>> On Tue, Mar 19, 2024 at 01:48:45AM +0000, Kai Huang wrote:
>>> Both SME and TDX can leave caches in incoherent state due to memory
>>> encryption.  During kexec, the caches must be flushed before jumping to
>>> the second kernel to avoid silent memory corruption to the second 
>>> kernel.
>>>
>>> During kexec, the WBINVD in stop_this_cpu() flushes caches for all
>>> remote cpus when they are being stopped.  For SME, the WBINVD in
>>> relocate_kernel() flushes the cache for the last running cpu (which is
>>> executing the kexec).
>>>
>>> Similarly, for TDX after stopping all remote cpus with cache flushed, to
>>> support kexec, the kernel needs to flush cache for the last running cpu.
>>>
>>> Make the WBINVD in the relocate_kernel() unconditional to cover both SME
>>> and TDX.
>>
>> Nope. It breaks TDX guest. WBINVD triggers #VE for TDX guests.
> 
> Ditto for SEV-ES/SEV-SNP, a #VC is generated and crashes the guest.
> 

Oh I forgot these.

Hi Kirill,

Then I think patch 1 will also break TDX guest after your series to 
enable multiple cpus for the second kernel after kexec()?

Hi Tom,

I am not aware of kexec() support status for SEV-ES/SEV-SNP guests. 
Does patch 1 break them?