Received-SPF: pass (google.com: domain of linux-kernel+bounces-186776-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Message-ID: <8b344a16-b28a-4f75-9c1a-a4edf2aa4a11@intel.com>
Date: Thu, 23 May 2024 10:27:53 +1200
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 3/6] KVM: Add a module param to allow enabling
 virtualization when KVM is loaded
To: Sean Christopherson <seanjc@google.com>, Paolo Bonzini
	<pbonzini@redhat.com>
CC: <kvm@vger.kernel.org>, <linux-kernel@vger.kernel.org>, Chao Gao
	<chao.gao@intel.com>
References: <20240522022827.1690416-1-seanjc@google.com>
 <20240522022827.1690416-4-seanjc@google.com>
Content-Language: en-US
From: "Huang, Kai" <kai.huang@intel.com>
In-Reply-To: <20240522022827.1690416-4-seanjc@google.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Y2NreThUYVc3NVJWMGM4TEpNZHp0UXVuelMwS2JTOWtkKzhpUVQ5b0RYQzZi?=
 =?utf-8?B?dXppeTg0RUI4clk5bGFLZE5TMkxqeGkzcHpVSFFLM1J0M2w5VWRrTW5oaXg3?=
 =?utf-8?B?NUh1cTI2RjhxTmNackJtOXUzdGhLRzZSb2ZsNFBnb0k1RUozWG11bjFlLys5?=
 =?utf-8?B?Qm9iQ09SYXZVTEFhUndoYVRoRFJidW05K28vd1lYV2ZaS2hGaHJZMm91b3RH?=
 =?utf-8?B?TXJEcWVqWlh2ZHV2aTRyTWtOeHh4blgrcmtUUlE3Z00vOHh2V201dzZSSzZv?=
 =?utf-8?B?eEMwOVBKdHV3WGFWdnRBVCtpM2U4OEJFd2ZZQVp0bVlaN2N3c05kWnpOK21r?=
 =?utf-8?B?ZTJjazZZS0VFSGlEeGFKUWF4M0FOTVNuOUcvVFYwWHlKRmhFMlNkMkIzbFVK?=
 =?utf-8?B?bjBSL3N0cjJzT1VsZlRqL05HcWRqSHlCcXVaWWF4VWlPb2xLNnNTZHlEc0Z3?=
 =?utf-8?B?SXZhOFpETjN4WVpwUUJiMHQ3cjBvUHJsTHdxeVVaaXlVMW4zNjNCY3JNbUsv?=
 =?utf-8?B?ZGRidUZsWHdQcWZKZjliYXc4ajczZGF3UHFIREJmOURDRDlLZzRSMEtCdnVY?=
 =?utf-8?B?NHlZV29FU2lGMkEybHNpT1JLMno5RTNNSWg2MU5GMVloZ0pRQVBpRnQ5Z2dl?=
 =?utf-8?B?eUZTbit1ZTgxOS84U1ZOK2tEbVJNaDZsQWUxclluT1E3WHdRYk9OL1NPdzlk?=
 =?utf-8?B?ZC9sWm4xN3pOYXY3UmtSZmR3K0d4TVg5SjUrOU01czg3UzZMRVJqQzhtbGxO?=
 =?utf-8?B?ZGUveXlGWEcyb2p6cnJ6Qk0zaUhmdlBNeHE0WEtKMUptM1dUV2p0WlVEVzVM?=
 =?utf-8?B?ZXlIWDZMNDlpQ0xnTkJEWXlEZ3pTMnp6M3ZOV3BIa3FiSFZkaHE3MnFlaUxM?=
 =?utf-8?B?ekVoNHpEbDFwOXNnRTZ4b0lEdTIwc1dvdWNUWUNjcjVwa2xoMVZNRlpqZXhv?=
 =?utf-8?B?R3YvZTJpV24zaFRPY3FNdUVOOWpiRmVGNG91R3pwZGp6R1NWSWs1aVdpQzla?=
 =?utf-8?B?d2syY1Zpd3JsWjlGTHNXNUNFK1E2RXlnWEtMeWVOTk9wNGJOVGgwdlpmZWor?=
 =?utf-8?B?L1pod3JNNE40ZlMyWTBUUHIwMEN1WUJDMHc0VkdISzhudG5DZjZGRDRLejBu?=
 =?utf-8?B?TjVKbTA1WWlnR3AxMWwzbnVLWEtmc0hyYUxwcFZvejZXa0lPUEM3NTJCbjV5?=
 =?utf-8?B?dGlDMU1KbmQwa1RXdlhaY3hwNjcwNWxRZDY0bmJHUXBCZmpMTks3elJFakp5?=
 =?utf-8?B?L2FEOE9nNHc5b2pFQ1BVOTJuemwrRTJSZzM5d2pXbXd5dGcvYlkvcE0wUGhq?=
 =?utf-8?B?UzBFMUtuUnVuaFRTNXJybGZjaUl4ZEtkdG5GemRWcjFhWXdYVm1EbllkdlJQ?=
 =?utf-8?B?aVRhdDBCTDZ4cmNiZFFrTmtuS1kxRWhBM0Z5T3graDBNaitlWVFPZjhaY2pM?=
 =?utf-8?B?QWFDdGg1MUVsczdldU1lc2F0dGhYYmJ6S0NDenBjSHFTV1VWMzVPVVBBaGFO?=
 =?utf-8?B?aisxdlR5ZTk1MisyUDRFMER6M2laakVDQXpNVWNlQUNRRGhQa2VtNTNMQ2VC?=
 =?utf-8?B?cHMvZHlKMG0zSDUwY2thcmhZbGJMZlpUVG5CS3B6V0szemU3c2RuQ2pBaWxC?=
 =?utf-8?B?dTdUeERQRHB3RENYUGVvRFN0Q1ZIK1dPUkMyNG9nSlgzYms1cEEzNEowaVd6?=
 =?utf-8?B?cWx0T3I3UlFMREhneGEzdXI3UXFyRXRnK09uVWticWtvQnBuL2h5TTlWdlp4?=
 =?utf-8?B?NVFCcHlZdXdlOVk4RGdxMGdwQURpN0U4NmhhVU42c0ZLdCtiWEEvN2QxU1du?=
 =?utf-8?B?dlI4a2YzK0htNEsxOWIxNXBQajZtQmwyell1WTFubUw2KzlLb0hPdVZ1QkVH?=
 =?utf-8?B?R2d3ckw5RTBYRVFMbUwwdXUwZWJ3U3QreUZkc3VsU1FSVkF6WnlOQllpbDg2?=
 =?utf-8?B?SmVjbnFSbXVYNndxeDZuS0FqNnN6RnlobmxmM29rdXhSZkdQSUNNVGRET1BV?=
 =?utf-8?B?SU1wbDk3V2Z5WHFWL2dhY0dFeGVKRnoxUzFnbTA5WnRmc21wZ1AxK1FTQVhY?=
 =?utf-8?B?UG5VK3VmQVFGLzdXbVpvMW1qdGc0bkdLVVIrS01lRTNwa0txRHIzY2swcXlk?=
 =?utf-8?Q?FeR89zcNJf0/BHtb/nBkWk9eB?=
X-MS-Exchange-CrossTenant-Network-Message-Id: e973f8e4-b771-40b1-9046-08dc7aae70b8
X-MS-Exchange-CrossTenant-AuthSource: BL1PR11MB5978.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 May 2024 22:28:01.5867
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: vs7uA3XcvORExQtNZbOeVa0psFntvhu3tuw9ohMm6trcrh9rrEuAFv7okc4S6YLe8HOWTnZR64SrsrwfpbYTYQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR11MB7122
X-OriginatorOrg: intel.com


On 22/05/2024 2:28 pm, Sean Christopherson wrote:
> Add an off-by-default module param, enable_virt_at_load, to let userspace
> force virtualization to be enabled in hardware when KVM is initialized,
> i.e. just before /dev/kvm is exposed to userspace.  Enabling virtualization
> during KVM initialization allows userspace to avoid the additional latency
> when creating/destroying the first/last VM.  Now that KVM uses the cpuhp
> framework to do per-CPU enabling, the latency could be non-trivial as the
> cpuhup bringup/teardown is serialized across CPUs, e.g. the latency could
> be problematic for use case that need to spin up VMs quickly.

How about we defer this until there's a real complain that this isn't 
acceptable?  To me it doesn't sound "latency of creating the first VM" 
matters a lot in the real CSP deployments.

The concern of adding a new module param is once we add it, we need to 
maintain it even it is no longer needed in the future for backward 
compatibility.  Especially this param is in kvm.ko, and for all ARCHs.

E.g., I think _IF_ the core cpuhp code is enhanced to call those 
callbacks in parallel in cpuhp_setup_state(), then this issue could be 
mitigated to an unnoticeable level.

Or we just still do:

	cpus_read_lock();
	on_each_cpu(hardware_enable_nolock, ...);
	cpuhp_setup_state_nocalls_cpuslocked(...);
	cpus_read_unlock();

I think the main benefit of series is to put all virtualization enabling 
related things into one single function.  Whether using 
cpuhp_setup_state() or using on_each_cpu() shouldn't be the main point.