Received-SPF: pass (google.com: domain of linux-kernel+bounces-82169-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Date: Mon, 26 Feb 2024 10:56:32 -0800
From: Isaku Yamahata <isaku.yamahata@linux.intel.com>
To: Yuan Yao <yuan.yao@linux.intel.com>
Cc: isaku.yamahata@intel.com, kvm@vger.kernel.org,
	linux-kernel@vger.kernel.org, isaku.yamahata@gmail.com,
	Paolo Bonzini <pbonzini@redhat.com>, erdemaktas@google.com,
	Sean Christopherson <seanjc@google.com>,
	Sagi Shahar <sagis@google.com>, Kai Huang <kai.huang@intel.com>,
	chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com,
	Sean Christopherson <sean.j.christopherson@intel.com>,
	isaku.yamahata@linux.intel.com
Subject: Re: [PATCH v18 024/121] KVM: TDX: create/destroy VM structure
Message-ID: <20240226185632.GJ177224@ls.amr.corp.intel.com>
References: <cover.1705965634.git.isaku.yamahata@intel.com>
 <167b3797f5928c580526f388761dcfb342626ad2.1705965634.git.isaku.yamahata@intel.com>
 <20240201083227.vwyqlptrr3bdwr7m@yy-desk-7060>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20240201083227.vwyqlptrr3bdwr7m@yy-desk-7060>

On Thu, Feb 01, 2024 at 04:32:27PM +0800,
Yuan Yao <yuan.yao@linux.intel.com> wrote:

..
> > +static int __tdx_reclaim_page(hpa_t pa)
> > +{
> > +	struct tdx_module_args out;
> > +	u64 err;
> > +
> > +	do {
> > +		err = tdh_phymem_page_reclaim(pa, &out);
> > +		/*
> > +		 * TDH.PHYMEM.PAGE.RECLAIM is allowed only when TD is shutdown.
> > +		 * state.  i.e. destructing TD.
> > +		 * TDH.PHYMEM.PAGE.RECLAIM requires TDR and target page.
> > +		 * Because we're destructing TD, it's rare to contend with TDR.
> > +		 */
> > +	} while (unlikely(err == (TDX_OPERAND_BUSY | TDX_OPERAND_ID_RCX)));
> 
> v16 changed to tdx module 1.5, so here should be TDX_OPERAND_ID_TDR, value 128ULL.

We should handle both RCX(SEPT) and TDR. So I make it err == RCX || err == TDR.

..

> > +static int __tdx_td_init(struct kvm *kvm)
> > +{
> > +	struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
> > +	cpumask_var_t packages;
> > +	unsigned long *tdcs_pa = NULL;
> > +	unsigned long tdr_pa = 0;
> > +	unsigned long va;
> > +	int ret, i;
> > +	u64 err;
> > +
> > +	ret = tdx_guest_keyid_alloc();
> > +	if (ret < 0)
> > +		return ret;
> > +	kvm_tdx->hkid = ret;
> > +
> > +	va = __get_free_page(GFP_KERNEL_ACCOUNT);
> > +	if (!va)
> > +		goto free_hkid;
> > +	tdr_pa = __pa(va);
> > +
> > +	tdcs_pa = kcalloc(tdx_info->nr_tdcs_pages, sizeof(*kvm_tdx->tdcs_pa),
> > +			  GFP_KERNEL_ACCOUNT | __GFP_ZERO);
> > +	if (!tdcs_pa)
> > +		goto free_tdr;
> > +	for (i = 0; i < tdx_info->nr_tdcs_pages; i++) {
> > +		va = __get_free_page(GFP_KERNEL_ACCOUNT);
> > +		if (!va)
> > +			goto free_tdcs;
> > +		tdcs_pa[i] = __pa(va);
> > +	}
> > +
> > +	if (!zalloc_cpumask_var(&packages, GFP_KERNEL)) {
> > +		ret = -ENOMEM;
> > +		goto free_tdcs;
> > +	}
> > +	cpus_read_lock();
> > +	/*
> > +	 * Need at least one CPU of the package to be online in order to
> > +	 * program all packages for host key id.  Check it.
> > +	 */
> > +	for_each_present_cpu(i)
> > +		cpumask_set_cpu(topology_physical_package_id(i), packages);
> > +	for_each_online_cpu(i)
> > +		cpumask_clear_cpu(topology_physical_package_id(i), packages);
> > +	if (!cpumask_empty(packages)) {
> > +		ret = -EIO;
> > +		/*
> > +		 * Because it's hard for human operator to figure out the
> > +		 * reason, warn it.
> > +		 */
> > +#define MSG_ALLPKG	"All packages need to have online CPU to create TD. Online CPU and retry.\n"
> > +		pr_warn_ratelimited(MSG_ALLPKG);
> > +		goto free_packages;
> > +	}
> 
> Generate/release hkid both requests to have "cpumask of at least 1
> cpu per each node", how about add one helper for this ? The helper also
> checks the cpus_read_lock() is held and return the cpumask if at least
> 1 cpu is online per node, thus this init funciotn can be simplified and
> become more easy to review.

We don't need cpumask to release hkid. So only tdx_td_init() needs cpumask
allocation. So I didn't to bother create a helper function with v19.
-- 
Isaku Yamahata <isaku.yamahata@linux.intel.com>