Received-SPF: pass (google.com: domain of linux-kernel+bounces-67421-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Date: Thu, 15 Feb 2024 11:39:18 -0600
From: John Allen <john.allen@amd.com>
To: Sean Christopherson <seanjc@google.com>, mlevitsk@redhat.com
Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com,
	weijiang.yang@intel.com, rick.p.edgecombe@intel.com, x86@kernel.org,
	thomas.lendacky@amd.com, bp@alien8.de
Subject: Re: [PATCH 6/9] KVM: SVM: Add MSR_IA32_XSS to the GHCB for
 hypervisor kernel
Message-ID: <Zc5MRqmspThUoB+n@AUS-L1-JOHALLEN.amd.com>
References: <20231010200220.897953-1-john.allen@amd.com>
 <20231010200220.897953-7-john.allen@amd.com>
 <5e413e05de559971cdc2d1a9281a8a271590f62b.camel@redhat.com>
 <ZUQvNIE9iU5TqJfw@google.com>
 <c077e005c64aa82c7eaf4252f322c4ca29a2d0af.camel@redhat.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <c077e005c64aa82c7eaf4252f322c4ca29a2d0af.camel@redhat.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?ZV9pD9C9ZZi/0tXbI8rVlT8ciWrTP+3RyWBK9K57U0X6d5GS2KA2Z98LpWya?=
 =?us-ascii?Q?GcvJafnLWsnxjHD+wiLRqay4gFSp1igbvdgzP/alc74BEGSm68GvC+6wzmRY?=
 =?us-ascii?Q?NYF0iYCG5CBqL6SAkxl/jybYy51jiC9ee4YN0MLrGI/p4NXNEUq74PqtY5bu?=
 =?us-ascii?Q?nudHISTWei+NiYndz8uJ9OAEw1KAh99tYaEXbdFJCuzs6zdVa3ncGZhnDQZz?=
 =?us-ascii?Q?prMKBNKursLnOT9ejsBUEybaZ36vcKoF0RQuSvYyAQqtchTERY1+qy3DBdRd?=
 =?us-ascii?Q?0vo3AxtkZ1C5pdYT0I5R6uZuKsh3v8pPoGDP7TCVoo+YXhEi/m8WrTBfoomU?=
 =?us-ascii?Q?JpT1EwZJ4nvvgEZR2XlBlBqNXXu9jpvsLbQ+EJ090tVolqeoCVreAuT7B691?=
 =?us-ascii?Q?4+5nkaIjgYTHz+2E1VkpT2s39mcQ2KFwGvyfjJ+OyNhfa/4H+ODoiUx/A+i1?=
 =?us-ascii?Q?r463NCGKxP39UQaK+6tkBihieNNHMBLCR0DHH7OsjMhn37L4j9j7Apo02ORE?=
 =?us-ascii?Q?UA9TE3KPpW59I2ZHRJb3r+AT8fbxr4urvPtuwG6v4iisks7EhwwPSnq9qQ4G?=
 =?us-ascii?Q?7in+GsrBugqzKamWd3HbQ/wOz9xPs6XwcJ+JOuluE2lNSaa2Ayj9lpic3ZnZ?=
 =?us-ascii?Q?wJ5D5V4q/iWHCIHN8hanw10sx+6UeZ+SimOcHNSYv76oMJtWP0TPE0ucsB7S?=
 =?us-ascii?Q?zX7QhE/9ODM7bfTvH3qmjMYXWO55GQuWSu/bjqEvgGGZmE38gUFdjMXRakBE?=
 =?us-ascii?Q?SNb/8TPhxwFFQ/LYXrNLwDd+FBTAEi2C+2nzPhaVaOjjSTqBJX/nnHVrRBqA?=
 =?us-ascii?Q?6B2R+8FYX71sVXlPKUX7NtzF1TJZQEGg5CpMyoQsj/Dy+lTEohEM5OoUyVer?=
 =?us-ascii?Q?k1o2HrOOTep8Ahr1/2ykrbHo9y5Bs+dP3/PuWNqdptqlNqRnl4YorrX/UFjB?=
 =?us-ascii?Q?/zdGCggtXOcmrtCWPrO7GWjKCb0wMRy2vr7lDdAzMqC+UZTbw6LhcdbdbimZ?=
 =?us-ascii?Q?i/odmBAaKjdk3WQPOflx/IPFNmr8xVfAXnkCFQDwfPQmmK46ZUkQjepvigJC?=
 =?us-ascii?Q?wFxSGWu18KDPw6VIaRV721/yIpQqiIEGwun0wdM2TwD/25R6colvvJmSzzm5?=
 =?us-ascii?Q?ixPweXb7EYJ3rMnJm/s//Q0EFWu/3PirqGi0b3gBmXta/DQLyoPGvfel7CUu?=
 =?us-ascii?Q?QucVEhPg6Y3W6fyW7lOrfXavzeB1hymLpdx9dxpkQeDjT2bTLqDB7qmNCxd/?=
 =?us-ascii?Q?Mwxjz3UEXV5UHbDq8tFQ9E1zldQvGy+2Kh+UmA9a6+Ar19xwPMxtgtytCgHJ?=
 =?us-ascii?Q?ggIN9d97DpBPpbcS5HoALz9ogQMchrG0BYCf047wRKs2c75C2NuHsT0h9yEz?=
 =?us-ascii?Q?2kiCNqSaKiYuvI5fRQO+1x+4BTCmRdGEpcaqSvse4es2OVhC79JQPHPpDaju?=
 =?us-ascii?Q?XDViznjqo5gYX/mwGlBjeYIkzCFgspXLW+NidrNDBItKIs9YufdXkQn7BmKj?=
 =?us-ascii?Q?VtLZbRLmfTJ0ki7PhqApYRmOo8me/rCpHffOj7CzeT1eBs+Wy6Uhe+3S1rvs?=
 =?us-ascii?Q?YNeuOLdh1Nu2DKUodiMez7Uueh9o1eynm4MKiqsp?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 384b7ce5-aec6-4c32-1c4a-08dc2e4d0e80
X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5995.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Feb 2024 17:39:27.3207
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 1lTYm3b5K7ftp2LTwJZlFUVVFTDjOMjKqve8N2uGHbwgDCC44bN9Qq2b6KdgD5LMUvoxM2Md4m/Q8cN7Aq/cWw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4239

On Tue, Nov 07, 2023 at 08:20:52PM +0200, Maxim Levitsky wrote:
> On Thu, 2023-11-02 at 16:22 -0700, Sean Christopherson wrote:
> > On Thu, Nov 02, 2023, Maxim Levitsky wrote:
> > > On Tue, 2023-10-10 at 20:02 +0000, John Allen wrote:
> > > > @@ -3032,6 +3037,9 @@ static void sev_es_init_vmcb(struct vcpu_svm *svm)
> > > >  		if (guest_cpuid_has(&svm->vcpu, X86_FEATURE_RDTSCP))
> > > >  			svm_clr_intercept(svm, INTERCEPT_RDTSCP);
> > > >  	}
> > > > +
> > > > +	if (kvm_caps.supported_xss)
> > > > +		set_msr_interception(vcpu, svm->msrpm, MSR_IA32_XSS, 1, 1);
> > > 
> > > This is not just a virtualization hole. This allows the guest to set MSR_IA32_XSS
> > > to whatever value it wants, and thus it might allow XSAVES to access some host msrs
> > > that guest must not be able to access.
> > > 
> > > AMD might not yet have such msrs, but on Intel side I do see various components
> > > like 'HDC State', 'HWP state' and such.
> > 
> > The approach AMD has taken with SEV-ES+ is to have ucode context switch everything
> > that the guest can access.  So, in theory, if/when AMD adds more XCR0/XSS-based
> > features, that state will also be context switched.
> > 
> > Don't get me wrong, I hate this with a passion, but it's not *quite* fatally unsafe,
> > just horrific.
> > 
> > > I understand that this is needed so that #VC handler could read this msr, and
> > > trying to read it will cause another #VC which is probably not allowed (I
> > > don't know this detail of SEV-ES)
> > > 
> > > I guess #VC handler should instead use a kernel cached value of this msr
> > > instead, or at least KVM should only allow reads and not writes to it.
> > 
> > Nope, doesn't work.  In addition to automatically context switching state, SEV-ES
> > also encrypts the guest state, i.e. KVM *can't* correctly virtualize XSS (or XCR0)
> > for the guest, because KVM *can't* load the guest's desired value into hardware.
> > 
> > The guest can do #VMGEXIT (a.k.a. VMMCALL) all it wants to request a certain XSS
> > or XCR0, and there's not a damn thing KVM can do to service the request.
> > 
> 
> Ah, I understand now. Everything makes sense, and yes, this is really ugly.

Hi Maxim and Sean,

It looks as though there are some broad changes that will need to happen
over the long term WRT to SEV-ES/SEV-SNP. In the short term, how would
you suggest I proceed with the SVM shstk series? Can we omit the SEV-ES
changes for now with an additional patch that disallows guest shstk when
SEV-ES is enabled? Subsequently, when we have a proper solution for the
concerns discussed here, we could submit another series for SEV-ES
support.

Thanks,
John