Received-SPF: pass (google.com: domain of linux-kernel+bounces-110792-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Message-ID: <a0155c6f-918b-47cd-9979-693118f896fc@intel.com>
Date: Fri, 22 Mar 2024 12:36:40 +1300
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v19 037/130] KVM: TDX: Make KVM_CAP_MAX_VCPUS backend
 specific
References: <cover.1708933498.git.isaku.yamahata@intel.com>
 <9bd868a287599eb2a854f6983f13b4500f47d2ae.1708933498.git.isaku.yamahata@intel.com>
Content-Language: en-US
From: "Huang, Kai" <kai.huang@intel.com>
To: <isaku.yamahata@intel.com>, <kvm@vger.kernel.org>,
	<linux-kernel@vger.kernel.org>
CC: <isaku.yamahata@gmail.com>, Paolo Bonzini <pbonzini@redhat.com>,
	<erdemaktas@google.com>, Sean Christopherson <seanjc@google.com>, Sagi Shahar
	<sagis@google.com>, <chen.bo@intel.com>, <hang.yuan@intel.com>,
	<tina.zhang@intel.com>
In-Reply-To: <9bd868a287599eb2a854f6983f13b4500f47d2ae.1708933498.git.isaku.yamahata@intel.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?YTBPWVJubXpyc1R1Yk9FK0lTd2NXakc4QWw2bWJKV01GWDBjQ29WQ0FZVWVL?=
 =?utf-8?B?Y0xVbW5VR0VIREtETU96M0VBMjl0cm9SNXFSYXhDU2pkZHdrUDUrbEdHallp?=
 =?utf-8?B?bnh3NW0rcjlRMWJFRTVLSzZJRTBRYmxDTFRFQXR6a1NWQ3R3YlFaa09KWW40?=
 =?utf-8?B?N3pZQXlaaVhWUEVOaTBQNWRGNnV6S1hwUnpoSHR6b1Z1NVpUT3RjRFB0Zm5M?=
 =?utf-8?B?S0hsdFhLc2pVVVRtVWxrdnByWERoNjBydFdaYjR1NWFZblhCZHdtR1hHazdY?=
 =?utf-8?B?MXBWSks0Y1cyRkJMeUZkWXM3bVF6bmtQT2NSM2FaV2hodmlVWTloQzdaYU81?=
 =?utf-8?B?bHhBMUlqelg3Y1F3WDNQV3NCVDJJODJWbVNVOUd2c0hWaXFqUXYxZFRxZUJG?=
 =?utf-8?B?YWlFL1cxYkpIazYvT3B1ZEFZSVZQOXRkZ3EzWDJzdEdvM3JWRTFCVWhrQ25T?=
 =?utf-8?B?M0ZLOVNzR2I3dmVMV2lTL2lhOHB3TWRHd2tVWnd3NUU0MFBTbjRvMWdEZjAx?=
 =?utf-8?B?VWZyUXNQM3V0U1J4UWZTS2RQMmxLWEZFMklyUkRiQVNrUFJRSmdDUTZqbXFs?=
 =?utf-8?B?UnFHeTg5NEVPMWRKSHpQTHRFWHpoOXUrR2NlQ2lXcnBrM0xwaHg1ZmhMWk9i?=
 =?utf-8?B?dXM0QTB3UXRBcFVkY1VSRzFzL3VMRmJUd2I1NEdFMndBdDBjL0VDbzVYeHhi?=
 =?utf-8?B?VW1tUkJkYk1sZE91ek9UVFBZV3FUZnZRWVlSNGl5M2VLbExzYWFCWmxKWCtt?=
 =?utf-8?B?KzhKRlJGVGlGbEk3eFA4OVZFQTR6Nllia0pyTHJGSDY5anBIWTVjb0x5Q1oz?=
 =?utf-8?B?VVArenhuQUF6ejZaTm56RVY0bmpBb1FKR2NTVWU2RENWcEZZd2l5SHRLbUMr?=
 =?utf-8?B?bjdKUUhJaUIxVkZLMXF0WjEyOVJyckdxN3lycCtSSGNaR2NGSkNEZ1FsZk5C?=
 =?utf-8?B?UEJsTWNjVk5rOWxaKzhYdHYvdDNHY05GUHdWTW4zL0J1M04xT20rdCtBaVNy?=
 =?utf-8?B?czBxNFN2UlNhLzRyRDBDYTRzRkxnUUJsZ3F5eTVDay9WZzJqRXFpYWFYb0FK?=
 =?utf-8?B?dzVIKzNZc2RRU0tKMUtmN2tVaWhVT2FNRTlseEFLTEtGMllFQm9BZTBrY3hL?=
 =?utf-8?B?djNwbHduWDh2Q2xCa1FYRklxZmM1QjE1bjN4OUVxUnUvMWpqTHlEK29pbWJW?=
 =?utf-8?B?aW5PeFlYNmNHaUZUVlBDZE8rU0FSQWtDV1RZZDVGQW9UaXlobjVLd3BwVGFJ?=
 =?utf-8?B?cTYyVHJoWDVoNUhLTGhjOU5VaHZxOGEydXJKQXZhV1dxY2I5S0FNVUV2ajZk?=
 =?utf-8?B?aDBNMlFWYk03eTBuQTdEWlJIcEV3TEErMFd6YjNBUUNTUnZVR09taXFWd0po?=
 =?utf-8?B?K0Zrb3V3dTJwcEwvbEIvMm1XYjU3ZTBHZDh5M3IzM2tXQjdpd0E5SFV1S1Vm?=
 =?utf-8?B?WlRBaWF6d2JiSjQ3OXFaSGZ6M0poclFyTzhxbFQvaW9qTm5mUmFXWmFvclRN?=
 =?utf-8?B?UkVSQTQ3anRiUkZUUWg2VFhRMmtLL09qVzRHQjVoQXZkWnVCbzNjWWI4eS90?=
 =?utf-8?B?dm1weTRlTW8vRXBwN2ZiTEF2U3h6TE5PNC8xQnFsSC8rYVVRbmwrZ0VXUXRM?=
 =?utf-8?B?cjdOS2NERDc4QUJYaThRRmxBd091Qy90VU4rZlNxYmF3eXJQM3FiV1daVU4w?=
 =?utf-8?B?T0dCTWdvS0s5MG9MMXpQNlc2UlNvdjBmSEFGWXVGSW5VcVVEdEd6d0huR2tU?=
 =?utf-8?B?NTF1S2Z6WG9SV2hEOCsvZ2tCL0lPMHlaMktHdFdhVWZTZjA0NEN4V1hvNFdP?=
 =?utf-8?B?Uk9Ta01iL3A5ZHpXSWJnWUJuNWpzaE9KbHRJWHJuS0lMSE4vaWtzUzlMWlFP?=
 =?utf-8?B?cHhpUUFXS3ZObWZWV2cvQWhzS0RJU2k2S3JLYjU0RDQ2V3JvdHVxYkkxK3VT?=
 =?utf-8?B?enJvemFiZ2l4aFFiV2p4Z21wSW1VYm43SzZUbC9LaklGR2d1bkppbVR0c3lI?=
 =?utf-8?B?bVJqMzdUSDNlcEZyN3lEcFA0T3EwZXM4ei84cVd4U3lGbytFZGlnUTgySUVu?=
 =?utf-8?B?eW9vbmUxNnNSbGpLT2tMS09lS01yRmhWN0E0SWFyd3N4Yi9kcG00MmwvYVd2?=
 =?utf-8?Q?zlnPoPN7i53aFRFjAsJY6sp60?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 7e40ead8-7a2a-4e99-0d1d-08dc49ffc837
X-MS-Exchange-CrossTenant-AuthSource: BL1PR11MB5978.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Mar 2024 23:36:50.5580
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: +2f6vhmPx6QR7Dxr9F4If6NKpIYgkiBYsFok7JTKCenvx3YzIxNw8GI0hpyFFeOa1Y8MbbwTmCBQYVC1U7EQtA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR11MB6476
X-OriginatorOrg: intel.com


On 26/02/2024 9:25 pm, Yamahata, Isaku wrote:
> From: Isaku Yamahata <isaku.yamahata@intel.com>
> 
> TDX has its own limitation on the maximum number of vcpus that the guest
> can accommodate.  

"limitation" -> "control".

"the guest" -> "a guest".

Allow x86 kvm backend to implement its own KVM_ENABLE_CAP
> handler and implement TDX backend for KVM_CAP_MAX_VCPUS.  

I am not sure we normally say "x86 KVM backend".  Just say "Allow KVM 
x86 ...".

user space VMM,
> e.g. qemu, can specify its value instead of KVM_MAX_VCPUS.

Grammar check.

> 
> When creating TD (TDH.MNG.INIT), the maximum number of vcpu needs to be
> specified as struct td_params_struct.  

'struct td_params_struct'??

Anyway, I don't think you need to mention such details.

and the value is a part of
> measurement.  The user space has to specify the value somehow.  

"and" -> "And" (grammar check please).

And add an empty line to start below as a new paragraph.

There are
> two options for it.
> option 1. API (Set KVM_CAP_MAX_VCPU) to specify the value (this patch)
> option 2. Add max_vcpu as a parameter to initialize the guest.
>            (TDG.MNG.INIT)

First of all, it seems to me that the two are not conflicting.

Based on the uapi/kvm.h:

   #define KVM_CAP_MAX_VCPUS 66       /* returns max vcpus per vm */

Currently KVM x86 doesn't allow to configure MAX_VCPU on VM-basis, but 
always reports KVM_MAX_VCPUS for _ALL_ VMs.  I.e., it doesn't support 
userspace to explicitly enable KVM_CAP_MAX_VCPUS for a given VM.

Now, if we allow the userspace to configure the MAX_VCPU for TDX guest 
(this could be a separate discussion in fact) due to attestation 
whatever, we need to support allowing userspace to configure MAX_VCPUS 
on VM-basis.

Therefore, option 1 isn't really an option to me, but is the thing that 
we _SHOULD_ do to support TDX.

So this pach should really just add "per-VM max vcpus" support for TDX, 
starting from:

	struct kvm_tdx {	/* or 'struct kvm_arch' ?? */
		...
		int max_vcpus;
	}

And in TDH.MNG.INIT, we need to manually check the MAX_VCPU specified in 
TD_PARAMS structure to make sure it matches to the record that we 
specified via KVM_CAP_MAX_VCPUS.

So how about:

"
TDX has its own mechanism to control the maximum number of VCPUs that 
the TDX guest can use.  When creating a TDX guest, the maximum number of 
vcpus needs to be passed to the TDX module as part of the measurement of 
the guest.

Because the value is part of the measurement, thus part of attestation, 
it better to allow the userspace to be able to configure it.  E.g. the 
users may want to precisely control the maximum number of vcpus their 
precious VMs can use.

The actual control itself must be done via the TDH.MNG.INIT SEAMCALL 
itself, where the number of maximum cpus is an input to the TDX module, 
but KVM needs to support the "per-VM number of maximum vcpus" and 
reflect that in the KVM_CAP_MAX_VCPUS.

Currently, the KVM x86 always reports KVM_MAX_VCPUS for all VMs but 
doesn't allow to enable KVM_CAP_MAX_VCPUS to configure the number of 
maximum vcpus on VM-basis.

Add "per-VM maximum vcpus" to KVM x86/TDX to accommodate TDX's needs.

The userspace-configured value then can be verified when KVM is actually 
creating the TDX guest.
"