Message-ID: <2d7141b1-1d76-4e67-60d2-471a524c372e@intel.com>
Date:   Mon, 13 Feb 2023 14:28:43 -0800
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.6.1
Subject: Re: [PATCH v9 07/18] x86/virt/tdx: Do TDX module per-cpu
 initialization
Content-Language: en-US
To:     "Huang, Kai" <kai.huang@intel.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>
Cc:     "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "peterz@infradead.org" <peterz@infradead.org>,
        "tglx@linutronix.de" <tglx@linutronix.de>,
        "Christopherson,, Sean" <seanjc@google.com>,
        "pbonzini@redhat.com" <pbonzini@redhat.com>,
        "Williams, Dan J" <dan.j.williams@intel.com>,
        "Wysocki, Rafael J" <rafael.j.wysocki@intel.com>,
        "kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
        "Huang, Ying" <ying.huang@intel.com>,
        "Chatre, Reinette" <reinette.chatre@intel.com>,
        "Brown, Len" <len.brown@intel.com>,
        "Luck, Tony" <tony.luck@intel.com>,
        "ak@linux.intel.com" <ak@linux.intel.com>,
        "Yamahata, Isaku" <isaku.yamahata@intel.com>,
        "Gao, Chao" <chao.gao@intel.com>,
        "sathyanarayanan.kuppuswamy@linux.intel.com" 
        <sathyanarayanan.kuppuswamy@linux.intel.com>,
        "david@redhat.com" <david@redhat.com>,
        "bagasdotme@gmail.com" <bagasdotme@gmail.com>,
        "Shahar, Sagi" <sagis@google.com>,
        "imammedo@redhat.com" <imammedo@redhat.com>
References: <cover.1676286526.git.kai.huang@intel.com>
 <557c526a1190903d11d67c4e2c76e01f67f6eb15.1676286526.git.kai.huang@intel.com>
 <2d9172c5-e1e7-bf94-c52b-0e9bc5b5b319@intel.com>
 <BL1PR11MB5978F562548EA22BFFD13970F7DD9@BL1PR11MB5978.namprd11.prod.outlook.com>
From:   Dave Hansen <dave.hansen@intel.com>
In-Reply-To: <BL1PR11MB5978F562548EA22BFFD13970F7DD9@BL1PR11MB5978.namprd11.prod.outlook.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 2/13/23 13:13, Huang, Kai wrote:
> Perhaps I didn't explain clearly in the comment.  Below is the updated one:
> 
>                 /*
>                  * The previous call of __tdx_enable() may only have
>                  * initialized part of present cpus during module
>                  * initialization, and new cpus may have become online
>                  * since then w/o doing per-cpu initialization.
>                  *
>                  * For example, a new CPU can become online when KVM is
>                  * unloaded, in which case tdx_cpu_enable() is not called since
>                  * KVM's CPU online callback has been removed.
>                  *
>                  * To make sure all online cpus are TDX-runnable, always
>                  * do per-cpu initialization for all online cpus here
>                  * even the module has been initialized.
>                  */

This is voodoo.

I want a TDX-specific hotplug CPU handler.  Period.  Please make that
happen.  Put that code in this patch.  That handler should:

	1. Run after the KVM handler (if present)
	2. See if VMX is on
	3. If VMX is on:
	 3a. Run smp_func_module_lp_init(), else
	 3b. Mark the CPU as needing smp_func_module_lp_init()

Then, in the 'case TDX_MODULE_INITIALIZED:', you call a function to
iterate over the cpumask that was generated in 3b.

That makes the handoff *EXPLICIT*.  You know exactly which CPUs need
what done to them.  A CPU hotplug either explicitly involves doing the
work to make TDX work on the CPU, or explicitly defers the work to a
specific later time in a specific later piece of code.