Received-SPF: pass (google.com: domain of linux-kernel+bounces-76276-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Date: Thu, 22 Feb 2024 17:24:38 +0800
From: Yan Zhao <yan.y.zhao@intel.com>
To: Sagi Shahar <sagis@google.com>
CC: <linux-kselftest@vger.kernel.org>, Ackerley Tng <ackerleytng@google.com>,
	Ryan Afranji <afranji@google.com>, Erdem Aktas <erdemaktas@google.com>, Isaku
 Yamahata <isaku.yamahata@intel.com>, Sean Christopherson <seanjc@google.com>,
	Paolo Bonzini <pbonzini@redhat.com>, Shuah Khan <shuah@kernel.org>, Peter
 Gonda <pgonda@google.com>, Haibo Xu <haibo1.xu@intel.com>, Chao Peng
	<chao.p.peng@linux.intel.com>, "Vishal Annapurve" <vannapurve@google.com>,
	Roger Wang <runanwang@google.com>, "Vipin Sharma" <vipinsh@google.com>,
	<jmattson@google.com>, <dmatlack@google.com>, <linux-kernel@vger.kernel.org>,
	<kvm@vger.kernel.org>, <linux-mm@kvack.org>
Subject: Re: [RFC PATCH v5 05/29] KVM: selftests: Add helper functions to
 create TDX VMs
Message-ID: <ZdcS1itARPpIy4P+@yzhao56-desk.sh.intel.com>
Reply-To: Yan Zhao <yan.y.zhao@intel.com>
References: <20231212204647.2170650-1-sagis@google.com>
 <20231212204647.2170650-6-sagis@google.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20231212204647.2170650-6-sagis@google.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?sSRg8GBkJyOOgaOQm5norZy8Rz/e0KrBWgEPGUKvzdLzp7mqZ8Pd+FuomZag?=
 =?us-ascii?Q?cDg9rt5m6K7HtT+HLmFHfe6NGdaURaAz94v0OjkTj4iM7WhFjcKhJzh1OM19?=
 =?us-ascii?Q?MasfKWnKyGQSUCwpUiRzlFv53B0wyogSqY+56nkgOZ4v5UEYzkONF839zw4+?=
 =?us-ascii?Q?dJMyDG1gb/0zq+M/nbdJ5wIeWShQksu/JEUjGkJro4b8NTpjGVx20MGBQ9ks?=
 =?us-ascii?Q?by9Qoe82PF8siNKt/g/8WTQ6uQ8XE+bIDtqzQQBdAKpjri/J6Ve7Vvo1qHKl?=
 =?us-ascii?Q?ugdSSFdrU0NQKes9X0w0AJiW6NBiQbeFcYkglOa6IPYJvXFPagviUvVStlqY?=
 =?us-ascii?Q?2IfQ9Ro8Iuj+15kXbjYhH7CiKFd6eY/H2ka81M22lk7GQWdIBfbyQZ2tY4Gg?=
 =?us-ascii?Q?kPWwmhjkcfapFn8SBvm6JUoNtq6v32QjR95M5bwXqMosdxPBghfRbcDSAgtB?=
 =?us-ascii?Q?63eg6hYPVvfXC+181Lp1mMqqwEH/C6ZlJHwVFlyWe6qHL/tgqqoCGZzVfXC1?=
 =?us-ascii?Q?4QLPyoLYsffO/9f4igsMblOwrSq5Kz44tXtwhtpoet4RCXRSyz6KqH5iaQIz?=
 =?us-ascii?Q?v4Kms0CQbE3lqR7uXAYgwhIxy1cw6a2+YowazC5/GJZ0C+QW/AI06Y4dNXm5?=
 =?us-ascii?Q?CZD9lbrCfXukoijXMqAsJj2AePeaPqRAVRm/BgQiyME2+ck0sRetpLddHlv4?=
 =?us-ascii?Q?DgddOBXxi5QRNahg0AAjI89E5Kpjlrl8V80tgafRClWN20d1aFgC7wx2hOza?=
 =?us-ascii?Q?xVzdazlxgFwgQ0x6BsCxfhNaZjDD9TamxnUo14bpdhusS4/pjiJeOxeKT2ow?=
 =?us-ascii?Q?uEs54+ppZPIGWRfItYUHNdO5Ca+A4EAokZ5suQwZKa3ARjrkuiOJUR42DS4U?=
 =?us-ascii?Q?WQEXgQ1O4Sp09X57wlkzSKga/YOdHtxeySxhXBUOP5HrW5X7TJqem/81n2Ev?=
 =?us-ascii?Q?pXpXbEfAnNkb5YY62Os+uYO9IsWC4l+w+t7JvQQem8S9+SNn31C5oT7k51CH?=
 =?us-ascii?Q?O6vAzA2uNG85miAxRwXdcGjtmz/m2XHNH8qnnEQp/CUYT3toOt1nrVi69Hhr?=
 =?us-ascii?Q?iYLJW5Pv7fQWzwb30MCF7GvP+nefQref4vDLD3/msrcK8r4oCH9cd+zQB51D?=
 =?us-ascii?Q?dsQPejAMtsQj0ZZnRFbtHj5cFtb+pAHYQLlzSvpb88gtEGTTnQBrnaRX+oW1?=
 =?us-ascii?Q?bFkJBLHjxzjxroVj0oqBvccMcESWUhGypaDlw7SCIVHbB+ePiQ0IIim/vmdS?=
 =?us-ascii?Q?dJy1vuVVIssfwFBHzMH2eOVGH9svv1jWxGM+frDHHqA2EaMEsZRM8jk+DMt3?=
 =?us-ascii?Q?hIVY/GOk18Z2bE8R2HWBjx9qVP5odM/TEx2u/wMbD99YinLmxj8u9OxfEXcI?=
 =?us-ascii?Q?LAfQkhWC6fn16/e47S/gi1iO2Yku3hOIxyDrGwni6OniYZM3+rz54cWPjfAL?=
 =?us-ascii?Q?5vlwHFTkgst/wedp1GU+ICpoPli9W0i93YFaBEYQPw9H6gf60gQaQJgNOZ23?=
 =?us-ascii?Q?DFIJmAcnSURR2miORqEtG8BX6UXfllTJlivXiaklPdWzskQl9qaa7vfytY9v?=
 =?us-ascii?Q?YueWAD4cgRy21BQeHwNL+HtvhHPC9bV/dRF6BBYV?=
X-MS-Exchange-CrossTenant-Network-Message-Id: b5b9a5c2-43eb-4d57-f7a9-08dc338c3f9a
X-MS-Exchange-CrossTenant-AuthSource: DS7PR11MB5966.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Feb 2024 09:54:23.8564
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: sJ4l8SQjaOZqH5k5P+wlnrEbdtg3tClwEzYwYIBV6WuyqMViMEAfqDzQFXI9u5hUEIDcTU12zNYkJ+rhRzoWng==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA2PR11MB4891
X-OriginatorOrg: intel.com

On Tue, Dec 12, 2023 at 12:46:20PM -0800, Sagi Shahar wrote:
> From: Erdem Aktas <erdemaktas@google.com>
> +/**
> + * Adds a vCPU to a TD (Trusted Domain) with minimum defaults. It will not set
> + * up any general purpose registers as they will be initialized by the TDX. In
> + * TDX, vCPUs RIP is set to 0xFFFFFFF0. See Intel TDX EAS Section "Initial State
> + * of Guest GPRs" for more information on vCPUs initial register values when
> + * entering the TD first time.
> + *
> + * Input Args:
> + *   vm - Virtual Machine
> + *   vcpuid - The id of the VCPU to add to the VM.
> + */
> +struct kvm_vcpu *td_vcpu_add(struct kvm_vm *vm, uint32_t vcpu_id, void *guest_code)
> +{
> +	struct kvm_vcpu *vcpu;
> +
> +	/*
> +	 * TD setup will not use the value of rip set in vm_vcpu_add anyway, so
> +	 * NULL can be used for guest_code.
> +	 */
> +	vcpu = vm_vcpu_add(vm, vcpu_id, NULL);
Rather than to call vm_vcpu_add(), is is better to call __vm_vcpu_add(),
__vm_vaddr_alloc() for vcpu->initial_stack_addr and vcpu_mp_state_set() only?

> +	tdx_td_vcpu_init(vcpu);
> +
> +	load_td_boot_parameters(addr_gpa2hva(vm, TD_BOOT_PARAMETERS_GPA),
> +				vcpu, guest_code);
> +
> +	return vcpu;
> +}
> +
..

> +static void td_setup_boot_code(struct kvm_vm *vm, enum vm_mem_backing_src_type src_type)
> +{
> +	vm_vaddr_t addr;
> +	size_t boot_code_allocation = round_up(TD_BOOT_CODE_SIZE, PAGE_SIZE);
> +	vm_paddr_t boot_code_base_gpa = FOUR_GIGABYTES_GPA - boot_code_allocation;
> +	size_t npages = DIV_ROUND_UP(boot_code_allocation, PAGE_SIZE);
> +
> +	vm_userspace_mem_region_add(vm, src_type, boot_code_base_gpa, 1, npages,
> +				    KVM_MEM_PRIVATE);
> +	addr = vm_vaddr_alloc_1to1(vm, boot_code_allocation, boot_code_base_gpa, 1);
> +	TEST_ASSERT_EQ(addr, boot_code_base_gpa);
> +
> +	load_td_boot_code(vm);
> +}
> +
> +static size_t td_boot_parameters_size(void)
> +{
> +	int max_vcpus = kvm_check_cap(KVM_CAP_MAX_VCPUS);
> +	size_t total_per_vcpu_parameters_size =
> +		max_vcpus * sizeof(struct td_per_vcpu_parameters);
> +
> +	return sizeof(struct td_boot_parameters) + total_per_vcpu_parameters_size;
> +}
> +
> +static void td_setup_boot_parameters(struct kvm_vm *vm, enum vm_mem_backing_src_type src_type)
> +{
> +	vm_vaddr_t addr;
> +	size_t boot_params_size = td_boot_parameters_size();
> +	int npages = DIV_ROUND_UP(boot_params_size, PAGE_SIZE);
> +	size_t total_size = npages * PAGE_SIZE;
> +
> +	vm_userspace_mem_region_add(vm, src_type, TD_BOOT_PARAMETERS_GPA, 2,
> +				    npages, KVM_MEM_PRIVATE);
> +	addr = vm_vaddr_alloc_1to1(vm, total_size, TD_BOOT_PARAMETERS_GPA, 2);
> +	TEST_ASSERT_EQ(addr, TD_BOOT_PARAMETERS_GPA);
> +}
> +
> +void td_initialize(struct kvm_vm *vm, enum vm_mem_backing_src_type src_type,
> +		   uint64_t attributes)
> +{
> +	uint64_t nr_pages_required;
> +
> +	tdx_enable_capabilities(vm);
> +
> +	tdx_configure_memory_encryption(vm);
> +
> +	tdx_td_init(vm, attributes);
> +
> +	nr_pages_required = vm_nr_pages_required(VM_MODE_DEFAULT, 1, 0);
> +
> +	/*
> +	 * Add memory (add 0th memslot) for TD. This will be used to setup the
> +	 * CPU (provide stack space for the CPU) and to load the elf file.
> +	 */
> +	vm_userspace_mem_region_add(vm, src_type, 0, 0, nr_pages_required,
> +				    KVM_MEM_PRIVATE);
> +
> +	kvm_vm_elf_load(vm, program_invocation_name);
> +
> +	vm_init_descriptor_tables(vm);
> +
> +	td_setup_boot_code(vm, src_type);
> +	td_setup_boot_parameters(vm, src_type);
> +}
Could we define slot ID macros for slot 0, 1, 2?
e.g. BOOT_SLOT_ID_0, BOOT_SLOT_ID_1,BOOT_SLOT_ID_2.