Received: by 2002:ab2:2994:0:b0:1ef:ca3e:3cd5 with SMTP id n20csp550825lqb; Thu, 14 Mar 2024 21:45:06 -0700 (PDT) X-Forwarded-Encrypted: i=3; AJvYcCXIaPx7kJ7FkU8/+z/2UEPSR28asXGgUyiz/yQqxIYmv4+CnPnFHwGX2Ned8qm9jjTinR55/gm82a5wedAvJX3FZ1xJCFLhCqEaFYp8VQ== X-Google-Smtp-Source: AGHT+IHRjPOsyIlNQSkFU3VxvxcH7AUG5WV5OK6BTt4ksn+Wtt/2ppn2SDdynbwjUAiU7jAzFeSF X-Received: by 2002:a17:906:13d7:b0:a46:35cb:cc89 with SMTP id g23-20020a17090613d700b00a4635cbcc89mr2393558ejc.22.1710477906689; Thu, 14 Mar 2024 21:45:06 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1710477906; cv=pass; d=google.com; s=arc-20160816; b=Hen98N7Xx8aY7jw8FtBeQEy/K/aT6ToVXi+X8xYYzyyU6zHDe9wO6qt1YcGToT+t2M mD1j7AFRC2r2Jo+b2PBckmST4CQ9pTma5bgH+7V3m3XqCRelQya8O3ySlisn3dhanINL 6z+YFbuBWVEDrjXJoUew5nXg+rXlXi6KEtEegWLmrpk0uFBwskpBSXG1N7TosYp/GlBq d05rxTZCU+9fR6HxTICZ3bjrpEVUH2v4g8TyhWdN6+IfqMDVie0ytUk3gQP0EF3zHz9F KHpMIY4oeJSWleawTB3RQGKwp7CaeFMwMV0MZsIdlyUIVhEa0iWcqnPhdZaU2eJMUc+T SO+A== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:in-reply-to:from:references:cc:to:subject :user-agent:mime-version:list-unsubscribe:list-subscribe:list-id :precedence:date:message-id:dkim-signature; bh=2rBIqj6M7DKo9PHGTexqA75z06Up2F861+nc5H1l1go=; fh=imCHPi/o9ZJ9c1vfrWeuKSQlXaddJS9CjzK1HY7FV1M=; b=RAWftE/OvJ8EGk39l5S4yPSlwrICV472tO/4NS8Nuh+zqyOCoK0Yr4v/7xF5u31WgJ 9qgBCf/sbo5oT2Ys7dbpOdF+dLRO5938tSgavIR96wqlVFPW5PC2z5p4jO54exV9hjTL 2CjW+rBpyIYyTgG765vU33dFx+1gxppJVu7tn4l2JfM5CalcoAfZaK1o1qIJXdgLEJaB 8DE5ytNBUPPunq3XJjjYLG2TsSyzCIuhDamFx9wuDuOM01RWVoSQ1enpQ4eRskoATbKx xy5vqnssevBjVZMCTEL3ZkkojJvZtCOUwEb7CJfJXbkXbH0N/IhvGMLZ86+xmTopSCsB YgEg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=EHRj5j0g; arc=pass (i=1 dkim=pass dkdomain=intel.com dmarc=pass fromdomain=linux.intel.com); spf=pass (google.com: domain of linux-kernel+bounces-104056-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-104056-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id bw17-20020a170906c1d100b00a46651b0961si1314301ejb.282.2024.03.14.21.45.06 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Mar 2024 21:45:06 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel+bounces-104056-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=EHRj5j0g; arc=pass (i=1 dkim=pass dkdomain=intel.com dmarc=pass fromdomain=linux.intel.com); spf=pass (google.com: domain of linux-kernel+bounces-104056-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-104056-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 114F01F22610 for ; Fri, 15 Mar 2024 04:45:06 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 3D4F7F9DE; Fri, 15 Mar 2024 04:44:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="EHRj5j0g" Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4B5A4DF42; Fri, 15 Mar 2024 04:44:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.13 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710477895; cv=none; b=lmB7pLb/s0x9Mlkr8kAJonF/TETo3MhyqQiOWJ/cmPEunoIcMEAwlR9X6IqIhAwOIIqEBBAdU2ek66pHpu5OcxZeeGTuawdquaj8wkFkBfZDWFMT8SaghJiXKEDuKBEDHMRJFuoGCNrCKYM8RPpS7/3Ob5dhjzgHfvGbR83a5gw= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710477895; c=relaxed/simple; bh=oO1aHmIUCDy4p53JFpDBTnJXbF7ykVbMlBtjCxRL2mc=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=LS04yTJ51pBg50YFwe5RvwpUs3W/zk+uZVUeMjIYgCjIZAtgdQUNSNnFz2qYQW0odw12XlQPU1PyiAkqyybSwIyMC52B3+N0p1RcleRN2py6SWS39zUPBKuVMcGgWx/is+YZ0RYdsslQIZMJgZ2lKyDUJBxdz4UF/ATJL9+aqvI= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=EHRj5j0g; arc=none smtp.client-ip=198.175.65.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1710477893; x=1742013893; h=message-id:date:mime-version:subject:to:cc:references: from:in-reply-to:content-transfer-encoding; bh=oO1aHmIUCDy4p53JFpDBTnJXbF7ykVbMlBtjCxRL2mc=; b=EHRj5j0g/aC8vzJZswpfYWKJtAGwV8pqiPLrbCNns1l1r8mFob2Z6xk1 pXf7gPkF2CJGbR1kB4ptYkq21+9K+YORBgVsIJjYDpMg5ww0rY8hsBAJ0 CFgMGeET1CMTSYIwUSUl770IQXKVYdOyxWZ8864XBUDrhZk6oAj6yS2GC 5Cm9smUJI6PqxiUnP2EtyDyW2lNPfSsJpNbQYBAkYo41hna79ZFvEhWH2 77emH7tBCWHHJcxuNYT/S3A7MJeSoifmb7EKMRe4528hC6UKUzX/JE+YP HR4JlF+bLCr9nJb6iMuSJ+4qhPFB6vqj7JVdfatBjMvBJYkMxHRd15lDd Q==; X-IronPort-AV: E=McAfee;i="6600,9927,11013"; a="16483132" X-IronPort-AV: E=Sophos;i="6.07,127,1708416000"; d="scan'208";a="16483132" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Mar 2024 21:44:52 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,127,1708416000"; d="scan'208";a="17205443" Received: from binbinwu-mobl.ccr.corp.intel.com (HELO [10.124.242.47]) ([10.124.242.47]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Mar 2024 21:44:50 -0700 Message-ID: <5470a429-cbbd-4946-b11a-ab86380d9b68@linux.intel.com> Date: Fri, 15 Mar 2024 12:44:46 +0800 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v19 023/130] KVM: TDX: Initialize the TDX module when loading the KVM intel kernel module To: Isaku Yamahata Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, isaku.yamahata@linux.intel.com References: <20240314162712.GO935089@ls.amr.corp.intel.com> From: Binbin Wu In-Reply-To: <20240314162712.GO935089@ls.amr.corp.intel.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 3/15/2024 12:27 AM, Isaku Yamahata wrote: > On Thu, Mar 14, 2024 at 10:05:35AM +0800, > Binbin Wu wrote: > >>> diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c >>> index 18cecf12c7c8..18aef6e23aab 100644 >>> --- a/arch/x86/kvm/vmx/main.c >>> +++ b/arch/x86/kvm/vmx/main.c >>> @@ -6,6 +6,22 @@ >>> #include "nested.h" >>> #include "pmu.h" >>> +static bool enable_tdx __ro_after_init; >>> +module_param_named(tdx, enable_tdx, bool, 0444); >>> + >>> +static __init int vt_hardware_setup(void) >>> +{ >>> + int ret; >>> + >>> + ret = vmx_hardware_setup(); >>> + if (ret) >>> + return ret; >>> + >>> + enable_tdx = enable_tdx && !tdx_hardware_setup(&vt_x86_ops); >>> + >>> + return 0; >>> +} >>> + >>> #define VMX_REQUIRED_APICV_INHIBITS \ >>> (BIT(APICV_INHIBIT_REASON_DISABLE)| \ >>> BIT(APICV_INHIBIT_REASON_ABSENT) | \ >>> @@ -22,6 +38,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { >>> .hardware_unsetup = vmx_hardware_unsetup, >>> + /* TDX cpu enablement is done by tdx_hardware_setup(). */ >> How about if there are some LPs that are offline. >> In tdx_hardware_setup(), only online LPs are initialed for TDX, right? > Correct. > > >> Then when an offline LP becoming online, it doesn't have a chance to call >> tdx_cpu_enable()? > KVM registers kvm_online/offline_cpu() @ kvm_main.c as cpu hotplug callbacks. > Eventually x86 kvm hardware_enable() is called on online/offline event. Yes, hardware_enable() will be called when online, but  hardware_enable() now is vmx_hardware_enable() right? It doens't call tdx_cpu_enable() during the online path.