Received-SPF: pass (google.com: domain of linux-kernel+bounces-130684-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161;
Message-ID: <92a60bea-99f5-4656-b205-da468f6c2b63@intel.com>
Date: Thu, 4 Apr 2024 11:26:19 +1300
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v19 038/130] KVM: TDX: create/destroy VM structure
To: "Yamahata, Isaku" <isaku.yamahata@intel.com>
CC: "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"isaku.yamahata@gmail.com" <isaku.yamahata@gmail.com>, Paolo Bonzini
	<pbonzini@redhat.com>, "Aktas, Erdem" <erdemaktas@google.com>, "Sean
 Christopherson" <seanjc@google.com>, Sagi Shahar <sagis@google.com>, "Chen,
 Bo2" <chen.bo@intel.com>, "Yuan, Hang" <hang.yuan@intel.com>, "Zhang, Tina"
	<tina.zhang@intel.com>, Sean Christopherson
	<sean.j.christopherson@intel.com>, "isaku.yamahata@linux.intel.com"
	<isaku.yamahata@linux.intel.com>
References: <cover.1708933498.git.isaku.yamahata@intel.com>
 <7a508f88e8c8b5199da85b7a9959882ddf390796.1708933498.git.isaku.yamahata@intel.com>
 <9f5c6259-78e1-4470-a013-91392bf3cea5@intel.com>
 <20240403172417.GE2444378@ls.amr.corp.intel.com>
Content-Language: en-US
From: "Huang, Kai" <kai.huang@intel.com>
In-Reply-To: <20240403172417.GE2444378@ls.amr.corp.intel.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?dXY4SjdsZWxSbXdia0ZZRUtoeFF1ZVNiUGtrS0Y4aTJRcHpuN3ZIek1NMkU3?=
 =?utf-8?B?ak5zVWtjOTcwSkl3SHNablBINkhpQnY1UVZWL3c4Ym96ejlOZmZPVmI5dzdr?=
 =?utf-8?B?SmtSRG5DK0h3dFNMeEtjaFZWZW9KVk56bTY1TXRwaDYzTnRQVUlHQWovcDhs?=
 =?utf-8?B?aWlrNWVjWFV6dG9PMmU4aGhmYStPM0lqY1V4em5qVitYNXZSaFMzVTdwOUtW?=
 =?utf-8?B?U1B0N2lKYjVTR1c4bVR2VE1UUnUvU1F2dWQyWGdVMUQrRHVYK0hQbm5GSGFB?=
 =?utf-8?B?eDU3K3AxWVQ5VFpsbXNQL0dpRmoyZ3NIemtGYVcxSmRDVWhSUHNabTJSaDVj?=
 =?utf-8?B?MFNNRHcyaytPSGFoUWk2b0NzT3E1V3A0eTV6aGU1SGRXTW5Ba3VyM2IvZC8v?=
 =?utf-8?B?TWtEZUNNQVNHY0tUQ1YveXhBNWZ0Qm9RWVZpWGZVbjdObGplSGRyMVNGUnlI?=
 =?utf-8?B?aWhZVER0dS9RTE5aOHhTcDdORlV5MWVxQVFKNUN5eFJvMmxPOThiQzlOdlpx?=
 =?utf-8?B?eG1aS0V2T1BsL0d0emdlZVU1ME5leUJ4T3E0SWxTNzRCclhPWEYzOEZObE00?=
 =?utf-8?B?VmdoWWxTMkkyR3JGNzV6NjFFT1ZNQ0dpbkRNZFV0Skk5ajBQYmJNZVJYczZr?=
 =?utf-8?B?N1BoWnNvRnhTc1NlY0RZWkkyeHo2QVd2eEJHSmp5azd3VWZBekc2d0R5cjlW?=
 =?utf-8?B?SGhxbm9aLzlFaHR4YW9OSUVPRUZSbVZRYjVLdFU2bFhCdzhGRjhYd2RnRStW?=
 =?utf-8?B?UkNIRStTeVJkZ0hRaER6bW9JU0puU21DbzdncHRySEJNWGtpRDBiU043MitX?=
 =?utf-8?B?ZVovaTRnR1hzQ2U1SG9XMUpnUzNBQ0JmMFcrTm9scXlQZEtKNFY3YmVHYVZC?=
 =?utf-8?B?cGhXWXJnVlAvRDlEOEgzOE5BbUZ0M3JMOURxQ0RRd1Z4L3ZLOTFBTmFjdURr?=
 =?utf-8?B?bmcraExiNGsvU2VWWU0vY2ZJVW1RNjVhcFc5REx6Sng1NlNEei9ESTB2NjZj?=
 =?utf-8?B?clQ5K3h0eVBmM2puYXVlZFdFSkRvcHY0SlBBRnVMclhPYURDSXQ4SWQxcHNR?=
 =?utf-8?B?dEtraWNzOElwbDdGWVMrT0pEV24yUndoMDRFZ21hQkRscER5WVZnVituYWs5?=
 =?utf-8?B?TWNtNXYwY3FodFRwUnpPZVRINEFuUWQrWWVCL1BEaFJCVTVRR0FDNm5UWUVX?=
 =?utf-8?B?bDRMOXYyclFWc29MRXo3bFFkQnRWWFNDS2NTUk5Da1lmS005dW1oWG1BTTIx?=
 =?utf-8?B?Y1dNZ003QmNIZmozWnVzQW5Oc2xPMDVJdXI0WExBb2VsaXd0aXp5SGtRaWJk?=
 =?utf-8?B?VkFtb1BSbi9OSmVCblBQZ3d2ZDdkTEhDdUlobnJxdnFXbmhLM3BPdFJmSWYv?=
 =?utf-8?B?SzNkRFlBaENWQzRlZ0RFUUNjNGk1aE1Oanc1b3ZFQkVMU3c5TWc1dEdtYXE1?=
 =?utf-8?B?SEZYVkVFN0hHZzhMcVlHVlgvejFudU9ScFlzMWtXVTJlNVhqUmZNa25TYlJy?=
 =?utf-8?B?L1BiTFBzbVNkOGxOY0poK3B2NkUrK2dhOXg0WUl1andBQ1gvbnVMb3lwMUZC?=
 =?utf-8?B?bWtLbG9DMVVGeHhVanR5RnFWMkY0R2R0VVdaZGZHQnhpVjdSVE5seXdPMGNp?=
 =?utf-8?B?eEp3eGdZSVNmMEM0RkZMR3lUZ0VaaEQyUTNiN1BaOE9IUFdhdDQrSWRZaXhi?=
 =?utf-8?B?Q29zdnUwQkZVbDgxdW92bi9sSkQ0OHRvMThzQVFOVzVsU3BKRzhyLzUrd2wy?=
 =?utf-8?B?NG1IVHdmTW1TZnBzejBjeThCTFhoaGVFRzhIeDF6S09ibE1VVGZmR2U0bkg5?=
 =?utf-8?B?Zy9SdlhqSUtTOHpCK0x3bUFCMXNadG9qVTR4L0lZek9iZlZpSXk0U2lTRHBC?=
 =?utf-8?B?cWd3WDN6ZHFBTHFlY1dFTGpSVk9hMnQ3Y2dsUktqdDgwZVhPL25FQlZ3bnYv?=
 =?utf-8?B?RXIzWXNzMi8vTmxDWm5icjRCMFRVdFJnaDVieUpvM05IK0N4OXN0dnRtTGRn?=
 =?utf-8?B?bVJVRnZYVGZjUndCWGg3NGZSemZpTDVSQmtPOG0zcVhVVjVoek5hNTE2MHlO?=
 =?utf-8?B?YzJIMlQwc0tEQkNnV04vNzBGb2laQVRFU2Juei9JeTVVMDZCUWtzS09zY3d2?=
 =?utf-8?Q?bHty8MLgvyGGeyvtG0wL66cDX?=
X-MS-Exchange-CrossTenant-Network-Message-Id: b1d6e46f-a095-4358-11bb-08dc542d1bd1
X-MS-Exchange-CrossTenant-AuthSource: BL1PR11MB5978.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Apr 2024 22:26:29.8591
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: uoYKn158Da7w92pHEsHRJtrNxgn4g8YIOTidHhhCywVM796MZqDR+JOfCXxdlOzTZcYWxBj+Opm5uH764gUI6w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR11MB7013
X-OriginatorOrg: intel.com


On 4/04/2024 6:24 am, Yamahata, Isaku wrote:
> On Thu, Mar 28, 2024 at 12:33:35PM +1300,
> "Huang, Kai" <kai.huang@intel.com> wrote:
> 
>>> +	kvm_tdx->tdr_pa = tdr_pa;
>>> +
>>> +	for_each_online_cpu(i) {
>>> +		int pkg = topology_physical_package_id(i);
>>> +
>>> +		if (cpumask_test_and_set_cpu(pkg, packages))
>>> +			continue;
>>> +
>>> +		/*
>>> +		 * Program the memory controller in the package with an
>>> +		 * encryption key associated to a TDX private host key id
>>> +		 * assigned to this TDR.  Concurrent operations on same memory
>>> +		 * controller results in TDX_OPERAND_BUSY.  Avoid this race by
>>> +		 * mutex.
>>> +		 */
>>
>> IIUC the race can only happen when you are creating multiple TDX guests
>> simulatenously?  Please clarify this in the comment.
>>
>> And I even don't think you need all these TDX module details:
>>
>> 		/*
>> 		 * Concurrent run of TDH.MNG.KEY.CONFIG on the same
>> 		 * package resluts in TDX_OPERAND_BUSY.  When creating
>> 		 * multiple TDX guests simultaneously this can run
>> 		 * concurrently.  Take the per-package lock to
>> 		 * serialize.
>> 		 */
> 
> As pointed by Chao, those mutex will be dropped.
> https://lore.kernel.org/kvm/ZfpwIespKy8qxWWE@chao-email/
> Also we would simplify cpu masks to track which package is online/offline,
> which cpu to use for each package somehow.

Please see my reply there.  I might be missing something, though.

> 
> 
>>> +		mutex_lock(&tdx_mng_key_config_lock[pkg]);
>>> +		ret = smp_call_on_cpu(i, tdx_do_tdh_mng_key_config,
>>> +				      &kvm_tdx->tdr_pa, true);
>>> +		mutex_unlock(&tdx_mng_key_config_lock[pkg]);
>>> +		if (ret)
>>> +			break;
>>> +	}
>>> +	cpus_read_unlock();
>>> +	free_cpumask_var(packages);
>>> +	if (ret) {
>>> +		i = 0;
>>> +		goto teardown;
>>> +	}
>>> +
>>> +	kvm_tdx->tdcs_pa = tdcs_pa;
>>> +	for (i = 0; i < tdx_info->nr_tdcs_pages; i++) {
>>> +		err = tdh_mng_addcx(kvm_tdx->tdr_pa, tdcs_pa[i]);
>>> +		if (err == TDX_RND_NO_ENTROPY) {
>>> +			/* Here it's hard to allow userspace to retry. */
>>> +			ret = -EBUSY;
>>> +			goto teardown;
>>> +		}
>>> +		if (WARN_ON_ONCE(err)) {
>>> +			pr_tdx_error(TDH_MNG_ADDCX, err, NULL);
>>> +			ret = -EIO;
>>> +			goto teardown;
>>> +		}
>>> +	}
>>> +
>>> +	/*
>>> +	 * Note, TDH_MNG_INIT cannot be invoked here.  TDH_MNG_INIT requires a dedicated
>>> +	 * ioctl() to define the configure CPUID values for the TD.
>>> +	 */
>>
>> Then, how about renaming this function to __tdx_td_create()?
> 
> So do we want to rename also ioctl name for consistency?
> i.e. KVM_TDX_INIT_VM => KVM_TDX_CREATE_VM.

Hmm.. but this __tdx_td_create() (the __tdx_td_init() in this patch) is 
called via kvm_x86_ops->vm_init(), but not IOCTL()?

If I read correctly, only TDH.MNG.INIT is called via IOCTL(), in that 
sense it makes more sense to name the IOCTL() as KVM_TDX_INIT_VM.

> 
> I don't have strong opinion those names. Maybe
> KVM_TDX_{INIT, CREATE, or CONFIG}_VM?
> And we can rename the function name to match it.
>