Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2;
Message-ID: <4a8b6084-bd4b-46cf-b0b1-396684e7a7b3@suse.com>
Date:   Wed, 18 Oct 2023 12:14:44 +0300
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v14 07/23] x86/virt/tdx: Add skeleton to enable TDX on
 demand
Content-Language: en-US
To:     Kai Huang <kai.huang@intel.com>, linux-kernel@vger.kernel.org,
        kvm@vger.kernel.org
Cc:     x86@kernel.org, dave.hansen@intel.com,
        kirill.shutemov@linux.intel.com, peterz@infradead.org,
        tony.luck@intel.com, tglx@linutronix.de, bp@alien8.de,
        mingo@redhat.com, hpa@zytor.com, seanjc@google.com,
        pbonzini@redhat.com, rafael@kernel.org, david@redhat.com,
        dan.j.williams@intel.com, len.brown@intel.com, ak@linux.intel.com,
        isaku.yamahata@intel.com, ying.huang@intel.com, chao.gao@intel.com,
        sathyanarayanan.kuppuswamy@linux.intel.com, bagasdotme@gmail.com,
        sagis@google.com, imammedo@redhat.com
References: <cover.1697532085.git.kai.huang@intel.com>
 <4fd10771907ae276548140cf7f8746e2eb38821c.1697532085.git.kai.huang@intel.com>
From:   Nikolay Borisov <nik.borisov@suse.com>
In-Reply-To: <4fd10771907ae276548140cf7f8746e2eb38821c.1697532085.git.kai.huang@intel.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?K3FIQTFLelZKanBKZWEwOGhEMFVoMVdKcVBHZ1RiR0RYWGpLYmlvZVgzQ05W?=
 =?utf-8?B?RWY2S3hrTTFwZ2FsemZJVjhrRlVoOEJoYXkxeElHN2hxaFhzWjZ5OTlLdnA3?=
 =?utf-8?B?cXJLNDVZLzZqdjE1QXZ1bkQ4L1pLaiswekJGZEs3K1ZNcjlsaU5LWVBwYnFJ?=
 =?utf-8?B?MDZYdTkvZk5aNlFMMWsrTGlwR3pyVFJlVHhFc3Q4Tm1LcVBvSVRYeEgvemxn?=
 =?utf-8?B?NEtyd2FLR1RGa2JnK0Vxa0RCRlgvMlRMKzBDWEpDMFB4QVZzS0syWTNDa1hP?=
 =?utf-8?B?RTVQcUUxRXJNTktlTVNKcmtpZWhGZ3FSbGxDMjhVSkU0SEtXOUx6dXAxSTM0?=
 =?utf-8?B?cjh4UTVheFRORzlRUEJFV2VPby8xSk5LZ0M5dzU3SFpXVFU1cGZvTzJoSzZl?=
 =?utf-8?B?MW5iaWFyVHJ5dXFnZ2JxOVZaQXBLZk9kbmZnQytGMFI2QkFHVHZpYlhZUlU0?=
 =?utf-8?B?RUI1Uk8wc25vZ0R1bExaRlJVdEhKOGNxVEpmeUQ3ejJqYlFvektTZ2NBWjZD?=
 =?utf-8?B?cFAxRGtuSlFzYjRsWkEvYnI4ODVkU1pTL01zaEgzKzIzRm5maklDT3dnT3ZP?=
 =?utf-8?B?ZDhaT0hmRUZhVWhkY1hDRFAyQ1FYd041Rm1oZG9PMVluR2hNOS9TZUgyR0Ro?=
 =?utf-8?B?cnZMZDlkWUgrRVRUUGFpdFNGT3o3YTdVYk5aYkJTZy9MSWVaQzJTTmdJMlVM?=
 =?utf-8?B?bHBaQ0o2d1pkVzEvTlBYZ2E4a01qOGh0QnBZQktYZlhiTDJKbVVpMGRmRWlq?=
 =?utf-8?B?SWttLzVKVjlVcnZWZVBrTktvOFBqL0xMQ0JBNXZBalhwU1J3UWZad1dHWmhP?=
 =?utf-8?B?WTBleStjYnZJckhVMVRxQUxCTXZGTVpwUHFoQ2tET0VHcHVJL0IzYmROWEEz?=
 =?utf-8?B?b0c1UUdaNkpuMzZjWVc2SENYM0ZWWHgyaEE0UWVlYkx6cXBBYlZ1YWpjOHpI?=
 =?utf-8?B?RXozaFpnOFhEbkhkb28rdG5ieEtLNTFyWkxSTjJuQ0ZzUmlxZ0FXUllUUUVa?=
 =?utf-8?B?RFJ6UHZZN0RWWWE3QXBXVzlrYm1OUjZMTTF4eGhtZ3M3dWVNMEx5NG03T1oy?=
 =?utf-8?B?dkcyTHMzVFBxS1VlUGE1SytKU3hHcFhLUzdBMEJxU2NnVlFZdjdMK0VqTE9L?=
 =?utf-8?B?YlNSUnFTVEtmd2U4d0xJTVhrcjNwRkZ3SHZ5Vm84UlJKS2x2Vm5lR2x6c0li?=
 =?utf-8?B?OUZrSlNUdHVRYkNsUVhUT0pIUkJrWmNubjUzNDNsK0kzVUVieElQU3g4Q3pD?=
 =?utf-8?B?WWphZ2tCUW1rSkNaTE5HUkp2c1o3Zm5CR1hUa1piU3JtWkNIdTYxM0JkcFV3?=
 =?utf-8?B?bDk0MTFoNU1UM2UvSU40VGRsRVZqOXJncS8vR1RBdENaR2VPMHFjZDB4Z1Bk?=
 =?utf-8?B?REFOeEtxTDRuUEpkbGtPSUU5MEMyN0cxSVBXWEtlbXZvWm9LVDFWSytla053?=
 =?utf-8?B?SDMvK0tTbHJWVzlhM0FWaHJFQ3Q4dm81TXM4OHdwaHFNeVhySlV5WktuVTMx?=
 =?utf-8?B?WkRyV2tTQ1lZeEowRXBJQWt1Z25UQndmLzU4NWFOMk5OTGJYT2xjcWFRZ3dj?=
 =?utf-8?B?SWp5S1VPSytCaU1qcVVGRzNJVW8wd0RVRkQ0SVA4bzQreE8wdDQvSGxQNVFv?=
 =?utf-8?B?a3NNcUQwQWV0R3BHbnFKS3BZUTdlV3lpZ0JIdlpYQ0JYZGJySmVJWDVTM1U5?=
 =?utf-8?B?cm8zWDlhYlYxYjlVOGM4WDJCbnJINkhtRXcvSGE1RXlrQTRxdlJDQmtYdC9W?=
 =?utf-8?B?QzllU3RuNTJZMlpLNGZDcDA4bzJsMlFxMTFNbmJMcUpDZDlQRjlkWW1vV2JX?=
 =?utf-8?B?RzVQVWdIRmIyald5aXJMRTdoT0xOeXIyMjhCeXJhekx4cTVNSmJKcVNRYVVw?=
 =?utf-8?B?N0JSVllUM0xFVVBYT3hCSEVPcjN6ajVtWTlWMGF1ZVk3dHhqNUkvUGM3aDJW?=
 =?utf-8?B?b0l3ZnhEc3ZDUVQ1Ni9Wc2l4V0RyMlRVeUxMZktHZWlkZkV4aGV0WW5MYmw1?=
 =?utf-8?B?eTNWQXJzTmg2VWg0Y1pPejhFaTJsRnkvQ2dKQ0NPeFp0SE40bjR5UTVNekxk?=
 =?utf-8?B?ZUZyZFpjSktPaGZrYzZIVkVSSk9rMWhDU2Y2TEwwZmFuSDRKY3JpQjAyTkJW?=
 =?utf-8?B?RytnbjJLS21IanM0K2J5Qm9Xa096QjZmeUFvSlI1dUk3N1piZkJ1NGpiYURR?=
 =?utf-8?Q?bw4IhLBLta0ViBcN8RRewlPPG74J2PfTI+bu9RAyvZXa?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 303afe10-3e37-43ca-697d-08dbcfbaadf6
X-MS-Exchange-CrossTenant-AuthSource: PA4PR04MB7790.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Oct 2023 09:14:49.6113
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: jzPDzA/zrDAkIByah87alSE3xe70r4RX31wlp1pY6ObaE1M3LRCBXWwXHTKjruGHKR7lPjIrJ5/cbNrTs1FXMQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR04MB6945
X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,
	DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,
	SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no
	version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Wed, 18 Oct 2023 02:15:37 -0700 (PDT)


On 17.10.23 г. 13:14 ч., Kai Huang wrote:
> To enable TDX the kernel needs to initialize TDX from two perspectives:
> 1) Do a set of SEAMCALLs to initialize the TDX module to make it ready
> to create and run TDX guests; 2) Do the per-cpu initialization SEAMCALL
> on one logical cpu before the kernel wants to make any other SEAMCALLs
> on that cpu (including those involved during module initialization and
> running TDX guests).
> 
> The TDX module can be initialized only once in its lifetime.  Instead
> of always initializing it at boot time, this implementation chooses an
> "on demand" approach to initialize TDX until there is a real need (e.g
> when requested by KVM).  This approach has below pros:
> 
> 1) It avoids consuming the memory that must be allocated by kernel and
> given to the TDX module as metadata (~1/256th of the TDX-usable memory),
> and also saves the CPU cycles of initializing the TDX module (and the
> metadata) when TDX is not used at all.
> 
> 2) The TDX module design allows it to be updated while the system is
> running.  The update procedure shares quite a few steps with this "on
> demand" initialization mechanism.  The hope is that much of "on demand"
> mechanism can be shared with a future "update" mechanism.  A boot-time
> TDX module implementation would not be able to share much code with the
> update mechanism.
> 
> 3) Making SEAMCALL requires VMX to be enabled.  Currently, only the KVM
> code mucks with VMX enabling.  If the TDX module were to be initialized
> separately from KVM (like at boot), the boot code would need to be
> taught how to muck with VMX enabling and KVM would need to be taught how
> to cope with that.  Making KVM itself responsible for TDX initialization
> lets the rest of the kernel stay blissfully unaware of VMX.
> 
> Similar to module initialization, also make the per-cpu initialization
> "on demand" as it also depends on VMX being enabled.
> 
> Add two functions, tdx_enable() and tdx_cpu_enable(), to enable the TDX
> module and enable TDX on local cpu respectively.  For now tdx_enable()
> is a placeholder.  The TODO list will be pared down as functionality is
> added.
> 
> Export both tdx_cpu_enable() and tdx_enable() for KVM use.
> 
> In tdx_enable() use a state machine protected by mutex to make sure the
> initialization will only be done once, as tdx_enable() can be called
> multiple times (i.e. KVM module can be reloaded) and may be called
> concurrently by other kernel components in the future.
> 
> The per-cpu initialization on each cpu can only be done once during the
> module's life time.  Use a per-cpu variable to track its status to make
> sure it is only done once in tdx_cpu_enable().
> 
> Also, a SEAMCALL to do TDX module global initialization must be done
> once on any logical cpu before any per-cpu initialization SEAMCALL.  Do
> it inside tdx_cpu_enable() too (if hasn't been done).
> 
> tdx_enable() can potentially invoke SEAMCALLs on any online cpus.  The
> per-cpu initialization must be done before those SEAMCALLs are invoked
> on some cpu.  To keep things simple, in tdx_cpu_enable(), always do the
> per-cpu initialization regardless of whether the TDX module has been
> initialized or not.  And in tdx_enable(), don't call tdx_cpu_enable()
> but assume the caller has disabled CPU hotplug, done VMXON and
> tdx_cpu_enable() on all online cpus before calling tdx_enable().
> 
> Signed-off-by: Kai Huang <kai.huang@intel.com>
> Reviewed-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>


With the latest explanation from Kai :

Reviewed-by: Nikolay Borisov <nik.borisov@suse.com>