Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: [RFC 07/16] KVM: selftests: add SEV boot tests
To:     Michael Roth <michael.roth@amd.com>,
        linux-kselftest@vger.kernel.org
Cc:     kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org,
        Nathan Tempelman <natet@google.com>,
        Marc Orr <marcorr@google.com>,
        Steve Rutherford <srutherford@google.com>,
        Sean Christopherson <seanjc@google.com>,
        Mingwei Zhang <mizhang@google.com>,
        Brijesh Singh <brijesh.singh@amd.com>,
        Tom Lendacky <thomas.lendacky@amd.com>,
        Varad Gautam <varad.gautam@suse.com>,
        Shuah Khan <shuah@kernel.org>,
        Vitaly Kuznetsov <vkuznets@redhat.com>,
        David Woodhouse <dwmw@amazon.co.uk>,
        Ricardo Koller <ricarkol@google.com>,
        Jim Mattson <jmattson@google.com>,
        Wanpeng Li <wanpengli@tencent.com>,
        Joerg Roedel <joro@8bytes.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        "H . Peter Anvin" <hpa@zytor.com>
References: <20211005234459.430873-1-michael.roth@amd.com>
 <20211006203726.13402-1-michael.roth@amd.com>
From:   Krish Sadhukhan <krish.sadhukhan@oracle.com>
Message-ID: <4e3d8f38-f01f-c1aa-335b-2c248f512638@oracle.com>
Date:   Fri, 15 Oct 2021 19:55:53 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.6.0
In-Reply-To: <20211006203726.13402-1-michael.roth@amd.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?bHBDZGlFNkhieVpqVVZtTzhpSVhrUHBINDhMUDAzSGNMa0x6cVZWUmc5MUl5?=
 =?utf-8?B?LzY1QlJJdTA0OHh0T1NXdzBvUEdXaEdEckgxMUNZZ0RPN3NRNFlMZEFUNmRY?=
 =?utf-8?B?Q2ExekRiSGJxREtHa1dTQWZpdTRWOExKMmgvYTg5QlNwOFpMQjBmSFJLZy8v?=
 =?utf-8?B?WEpTQyt2eUFkWFFCSldEWU1UMGd0K1pLQ0c3RmlvWlhTWXFrNUlLQXc3Y3NY?=
 =?utf-8?B?SW1ockdyaXp6Ly9NbktnTUFGdEZRT2FDWk1IVmhNa01NUXB2Wm9kWWIxbVlW?=
 =?utf-8?B?d2dzV2Q2YnpGOUM3eEVUcWQ3OGh0RGVpSDNiS2NBbmtHd3NYWGVGd2xXN21Y?=
 =?utf-8?B?MWVmRzB6U0U4WHlhNlpzTlcyYmNOWUxUNHdFd1hENGY1QlRkU2NSYlFDZ25k?=
 =?utf-8?B?a0tYWkpFQ2NPQXJMZHZCU3JJYkpYVDNyRTZuaGdCTHpTVUcvL29QR2E2eU5z?=
 =?utf-8?B?ckRzVHg3RzdhclJnQjJKbjBGZFQvdGtpeHVobytLcFlSdTg0eWl0YzJJdGVV?=
 =?utf-8?B?VzJrdk5OK1RVZHl0MWQyVjZtc2MrWVZnNFBVOUgwZCtaQTV2MzVUaXhFL0cy?=
 =?utf-8?B?SkxjV3VBYm9qdFBpcGtjaDhkYndkZjg3V2hRRUVmcEhEemJxNm8rRHorS0hy?=
 =?utf-8?B?Wks1Ty9CWFdUbEF4S2poWlJyTXNzd1o1eUtURVoxWGwyYURWY2dNT1VhM0ZC?=
 =?utf-8?B?L3FhamRoemNBWVRwMEs0dVNvSHh2YWE4UW5FS0tCR1dRQlhYWHFHMGhhQjh1?=
 =?utf-8?B?aktmOU8xeG9VNG5ZQmo4bEVMOExhTlBJZW9odkVjNDN6ZzhZNGpuZTV0YmUv?=
 =?utf-8?B?UTU4aXhsK1h4YkVYUkdhOFJiVjB0TWZXQi9sYmZGbklsU3V4bXNkVnhFS2xF?=
 =?utf-8?B?TzNVTlh0alJQNkdtWDAvT1VEZUcvSEVWRXNkUk5Iamg1eUhsQzVLL0EzMVRC?=
 =?utf-8?B?bnp2cnJPd3Q2SUVGeDRWN1c1dWpwd21uVDI0REJucGlUQ01xQmtjaTJXNzFT?=
 =?utf-8?B?WURQeE13VmlQb3lpcjV5UmovNDlaRnVDQW5nZ2RhSXlmVHFtNERwYkpYcEgw?=
 =?utf-8?B?S2pPQnpySG1EczNPYlhnc3FBTjlEeWJCeVVNN3k5N24rVVRIM2hUYTYxcGVL?=
 =?utf-8?B?Mlk4b0M0WnB2KzMwQUFRQ0lHWnVKYzY1OE1YMmM5YXc0S1FQOFV6ajZXWkQr?=
 =?utf-8?B?SE1sNVNxTW5aTkNtQ3RDVGMzMmNPU2VqSU0wWE9TVG01aHpGM09SemtkejFB?=
 =?utf-8?B?MFBFZ3YvRk1MNmJHZFlZSnpGelFwYWVPSU0zR0oxMHE0bEd3NkJTZ04vZ3FH?=
 =?utf-8?B?dWRVMm5vYXZUa0lYSnc1TkdjbExjbEJzUjZJNG82NTRFSW1WMEM4UVZ5QUtz?=
 =?utf-8?B?V2grVlJMajBYVHNJdkIrQzdVTkFWdHBRV25kWEZabUVKYjZDMXBQSmhyZ3dx?=
 =?utf-8?B?QkZhNmdqN0VyczU0Qi9Kblp0MXBCV3NPWUx4ekNjZy9FNzBWaWljZXdiSUFu?=
 =?utf-8?B?Qlk5NjZPUXBtK2lETjFtUy9wWVhxWFQ4TDJmbGVtcDlPV09teVo5NEdYL3JM?=
 =?utf-8?B?L2UyLzVLRlhaMy9GZVlJS3JSMC9mSllCVmpON0ppLzlNQld0MmdPMi8yZUM2?=
 =?utf-8?B?dUhhenI5NEV6bFJkZVVNZ2NMVWRIa0psSXZ1MldMN0RXdjNlWmxEblF3dll2?=
 =?utf-8?B?Z3Z1bVFzcWxCKzFxTjNzVDR2NmZ2YXBxT2JGNENkYXYrQzQvbzh4UTgyd1pT?=
 =?utf-8?B?SStMK0JnZmc3TksrUlJVMk9idFRjSDFEQS9nR0tvMThxQ09KVHJLdDRjYXFt?=
 =?utf-8?B?ZDlEd01kOVdlY2dkK1M0Zz09?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e1d2276c-77b7-4244-5c46-08d990507a1f
X-MS-Exchange-CrossTenant-AuthSource: SN6PR10MB3021.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Oct 2021 02:55:57.3542
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: FVb6ZmDSgjaEo/xgiqQ+SEKKTZRf2wbvMffSMovy0K1vbpiERCpD8f1DhHianAnKMKYZleYDqYyB63+UKUnMmybt6BcvItzqhfhxVYvNX5s=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA2PR10MB4665
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10138 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 adultscore=0 malwarescore=0
 phishscore=0 mlxlogscore=999 bulkscore=0 suspectscore=0 mlxscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001
 definitions=main-2110160016
X-Proofpoint-GUID: 2enyz3H1q7jz6tgAAxTjOY8teLItVl5m
X-Proofpoint-ORIG-GUID: 2enyz3H1q7jz6tgAAxTjOY8teLItVl5m
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 10/6/21 1:37 PM, Michael Roth wrote:
> A common aspect of booting SEV guests is checking related CPUID/MSR
> bits and accessing shared/private memory. Add a basic test to cover
> this.
>
> This test will be expanded to cover basic boot of SEV-ES and SEV-SNP in
> subsequent patches.
>
> Signed-off-by: Michael Roth <michael.roth@amd.com>
> ---
>   tools/testing/selftests/kvm/.gitignore        |   1 +
>   tools/testing/selftests/kvm/Makefile          |   1 +
>   .../selftests/kvm/x86_64/sev_all_boot_test.c  | 252 ++++++++++++++++++
>   3 files changed, 254 insertions(+)
>   create mode 100644 tools/testing/selftests/kvm/x86_64/sev_all_boot_test.c
>
> diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore
> index 0709af0144c8..824f100bec2a 100644
> --- a/tools/testing/selftests/kvm/.gitignore
> +++ b/tools/testing/selftests/kvm/.gitignore
> @@ -38,6 +38,7 @@
>   /x86_64/xen_vmcall_test
>   /x86_64/xss_msr_test
>   /x86_64/vmx_pmu_msrs_test
> +/x86_64/sev_all_boot_test
>   /access_tracking_perf_test
>   /demand_paging_test
>   /dirty_log_test
> diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
> index c7a5e1c69e0c..aa8901bdbd22 100644
> --- a/tools/testing/selftests/kvm/Makefile
> +++ b/tools/testing/selftests/kvm/Makefile
> @@ -72,6 +72,7 @@ TEST_GEN_PROGS_x86_64 += x86_64/tsc_msrs_test
>   TEST_GEN_PROGS_x86_64 += x86_64/vmx_pmu_msrs_test
>   TEST_GEN_PROGS_x86_64 += x86_64/xen_shinfo_test
>   TEST_GEN_PROGS_x86_64 += x86_64/xen_vmcall_test
> +TEST_GEN_PROGS_x86_64 += x86_64/sev_all_boot_test
>   TEST_GEN_PROGS_x86_64 += access_tracking_perf_test
>   TEST_GEN_PROGS_x86_64 += demand_paging_test
>   TEST_GEN_PROGS_x86_64 += dirty_log_test
> diff --git a/tools/testing/selftests/kvm/x86_64/sev_all_boot_test.c b/tools/testing/selftests/kvm/x86_64/sev_all_boot_test.c
> new file mode 100644
> index 000000000000..8df7143ac17d
> --- /dev/null
> +++ b/tools/testing/selftests/kvm/x86_64/sev_all_boot_test.c
> @@ -0,0 +1,252 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Basic SEV boot tests.
> + *
> + * Copyright (C) 2021 Advanced Micro Devices
> + */
> +#define _GNU_SOURCE /* for program_invocation_short_name */
> +#include <fcntl.h>
> +#include <stdio.h>
> +#include <stdlib.h>
> +#include <string.h>
> +#include <sys/ioctl.h>
> +
> +#include "test_util.h"
> +
> +#include "kvm_util.h"
> +#include "processor.h"
> +#include "svm_util.h"
> +#include "linux/psp-sev.h"
> +#include "sev.h"
> +
> +#define VCPU_ID			2
> +#define PAGE_SIZE		4096
> +#define PAGE_STRIDE		32
> +
> +#define SHARED_PAGES		8192
> +#define SHARED_VADDR_MIN	0x1000000
> +
> +#define PRIVATE_PAGES		2048
> +#define PRIVATE_VADDR_MIN	(SHARED_VADDR_MIN + SHARED_PAGES * PAGE_SIZE)
> +
> +#define TOTAL_PAGES		(512 + SHARED_PAGES + PRIVATE_PAGES)
> +
> +static void fill_buf(uint8_t *buf, size_t pages, size_t stride, uint8_t val)
> +{
> +	int i, j;
> +
> +	for (i = 0; i < pages; i++)
> +		for (j = 0; j < PAGE_SIZE; j += stride)
> +			buf[i * PAGE_SIZE + j] = val;
> +}
> +
> +static bool check_buf(uint8_t *buf, size_t pages, size_t stride, uint8_t val)
> +{
> +	int i, j;
> +
> +	for (i = 0; i < pages; i++)
> +		for (j = 0; j < PAGE_SIZE; j += stride)
> +			if (buf[i * PAGE_SIZE + j] != val)
> +				return false;
> +
> +	return true;
> +}
> +
> +static void guest_test_start(struct sev_sync_data *sync)
> +{
> +	/* Initial guest check-in. */
> +	sev_guest_sync(sync, 1, 0);
> +}
> +
> +static void check_test_start(struct kvm_vm *vm, struct sev_sync_data *sync)
> +{
> +	struct kvm_run *run;
> +
> +	run = vcpu_state(vm, VCPU_ID);
> +	vcpu_run(vm, VCPU_ID);
> +
> +	/* Initial guest check-in. */
> +	sev_check_guest_sync(run, sync, 1);
> +}
> +
> +static void
> +guest_test_common(struct sev_sync_data *sync, uint8_t *shared_buf, uint8_t *private_buf)
> +{
> +	bool success;
> +
> +	/* Initial check-in for common. */
> +	sev_guest_sync(sync, 100, 0);
> +
> +	/* Ensure initial shared pages are intact. */
> +	success = check_buf(shared_buf, SHARED_PAGES, PAGE_STRIDE, 0x41);
> +	SEV_GUEST_ASSERT(sync, 103, success);
> +
> +	/* Ensure initial private pages are intact/encrypted. */
> +	success = check_buf(private_buf, PRIVATE_PAGES, PAGE_STRIDE, 0x42);
> +	SEV_GUEST_ASSERT(sync, 104, success);
> +
> +	/* Ensure host userspace can't read newly-written encrypted data. */
> +	fill_buf(private_buf, PRIVATE_PAGES, PAGE_STRIDE, 0x43);
> +
> +	sev_guest_sync(sync, 200, 0);
> +
> +	/* Ensure guest can read newly-written shared data from host. */
> +	success = check_buf(shared_buf, SHARED_PAGES, PAGE_STRIDE, 0x44);
> +	SEV_GUEST_ASSERT(sync, 201, success);
> +
> +	/* Ensure host can read newly-written shared data from guest. */
> +	fill_buf(shared_buf, SHARED_PAGES, PAGE_STRIDE, 0x45);
> +
> +	sev_guest_sync(sync, 300, 0);
> +}
> +
> +static void
> +check_test_common(struct kvm_vm *vm, struct sev_sync_data *sync,
> +		  uint8_t *shared_buf, uint8_t *private_buf)
> +{
> +	struct kvm_run *run = vcpu_state(vm, VCPU_ID);
> +	bool success;
> +
> +	/* Initial guest check-in. */
> +	vcpu_run(vm, VCPU_ID);
> +	sev_check_guest_sync(run, sync, 100);
> +
> +	/* Ensure initial private pages are intact/encrypted. */
> +	success = check_buf(private_buf, PRIVATE_PAGES, PAGE_STRIDE, 0x42);
> +	TEST_ASSERT(!success, "Initial guest memory not encrypted!");
> +
> +	vcpu_run(vm, VCPU_ID);
> +	sev_check_guest_sync(run, sync, 200);
> +
> +	/* Ensure host userspace can't read newly-written encrypted data. */
> +	success = check_buf(private_buf, PRIVATE_PAGES, PAGE_STRIDE, 0x43);
> +	TEST_ASSERT(!success, "Modified guest memory not encrypted!");
> +
> +	/* Ensure guest can read newly-written shared data from host. */
> +	fill_buf(shared_buf, SHARED_PAGES, PAGE_STRIDE, 0x44);
> +
> +	vcpu_run(vm, VCPU_ID);
> +	sev_check_guest_sync(run, sync, 300);
> +
> +	/* Ensure host can read newly-written shared data from guest. */
> +	success = check_buf(shared_buf, SHARED_PAGES, PAGE_STRIDE, 0x45);
> +	TEST_ASSERT(success, "Host can't read shared guest memory!");
> +}
> +
> +static void
> +guest_test_done(struct sev_sync_data *sync)
> +{
> +	sev_guest_done(sync, 10000, 0);
> +}
> +
> +static void
> +check_test_done(struct kvm_vm *vm, struct sev_sync_data *sync)
> +{
> +	struct kvm_run *run = vcpu_state(vm, VCPU_ID);
> +
> +	vcpu_run(vm, VCPU_ID);
> +	sev_check_guest_done(run, sync, 10000);
> +}
> +
> +static void __attribute__((__flatten__))
> +guest_sev_code(struct sev_sync_data *sync, uint8_t *shared_buf, uint8_t *private_buf)
> +{
> +	uint32_t eax, ebx, ecx, edx;
> +	uint64_t sev_status;
> +
> +	guest_test_start(sync);
> +
> +	/* Check SEV CPUID bit. */
> +	eax = 0x8000001f;
> +	ecx = 0;
> +	cpuid(&eax, &ebx, &ecx, &edx);
> +	SEV_GUEST_ASSERT(sync, 2, eax & (1 << 1));
> +
> +	/* Check SEV MSR bit. */
> +	sev_status = rdmsr(MSR_AMD64_SEV);
> +	SEV_GUEST_ASSERT(sync, 3, (sev_status & 0x1) == 1);
> +
Is there any need to do the cpuid and MSR tests every time the guest 
code is executed ?
> +	guest_test_common(sync, shared_buf, private_buf);
> +
> +	guest_test_done(sync);
> +}
> +
> +static void
> +setup_test_common(struct sev_vm *sev, void *guest_code, vm_vaddr_t *sync_vaddr,
> +		  vm_vaddr_t *shared_vaddr, vm_vaddr_t *private_vaddr)
> +{
> +	struct kvm_vm *vm = sev_get_vm(sev);
> +	uint8_t *shared_buf, *private_buf;
> +
> +	/* Set up VCPU and initial guest kernel. */
> +	vm_vcpu_add_default(vm, VCPU_ID, guest_code);
> +	kvm_vm_elf_load(vm, program_invocation_name);
> +
> +	/* Set up shared sync buffer. */
> +	*sync_vaddr = vm_vaddr_alloc_shared(vm, PAGE_SIZE, 0);
> +
> +	/* Set up buffer for reserved shared memory. */
> +	*shared_vaddr = vm_vaddr_alloc_shared(vm, SHARED_PAGES * PAGE_SIZE,
> +					      SHARED_VADDR_MIN);
> +	shared_buf = addr_gva2hva(vm, *shared_vaddr);
> +	fill_buf(shared_buf, SHARED_PAGES, PAGE_STRIDE, 0x41);
> +
> +	/* Set up buffer for reserved private memory. */
> +	*private_vaddr = vm_vaddr_alloc(vm, PRIVATE_PAGES * PAGE_SIZE,
> +					PRIVATE_VADDR_MIN);
> +	private_buf = addr_gva2hva(vm, *private_vaddr);
> +	fill_buf(private_buf, PRIVATE_PAGES, PAGE_STRIDE, 0x42);
> +}
> +
> +static void test_sev(void *guest_code, uint64_t policy)
> +{
> +	vm_vaddr_t sync_vaddr, shared_vaddr, private_vaddr;
> +	uint8_t *shared_buf, *private_buf;
> +	struct sev_sync_data *sync;
> +	uint8_t measurement[512];
> +	struct sev_vm *sev;
> +	struct kvm_vm *vm;
> +	int i;
> +
> +	sev = sev_vm_create(policy, TOTAL_PAGES);
> +	if (!sev)
> +		return;
> +	vm = sev_get_vm(sev);
> +
> +	setup_test_common(sev, guest_code, &sync_vaddr, &shared_vaddr, &private_vaddr);
> +
> +	/* Set up guest params. */
> +	vcpu_args_set(vm, VCPU_ID, 4, sync_vaddr, shared_vaddr, private_vaddr);
> +
> +	sync = addr_gva2hva(vm, sync_vaddr);
> +	shared_buf = addr_gva2hva(vm, shared_vaddr);
> +	private_buf = addr_gva2hva(vm, private_vaddr);
> +
> +	/* Allocations/setup done. Encrypt initial guest payload. */
> +	sev_vm_launch(sev);
> +
> +	/* Dump the initial measurement. A test to actually verify it would be nice. */
> +	sev_vm_measure(sev, measurement);
> +	pr_info("guest measurement: ");
> +	for (i = 0; i < 32; ++i)
> +		pr_info("%02x", measurement[i]);
> +	pr_info("\n");
> +
> +	sev_vm_launch_finish(sev);
Since the above section of this function is actually setup code and is 
not running the test yet, it is best placed in a setup function. My 
suggestion is that you place the above section into a function called 
setup_test_common() and within that function you further separate out 
the SEV-related setup into a function called setup_test_sev() or 
something similar. Then call the top-level setup function from within 
main().
> +
> +	/* Guest is ready to run. Do the tests. */
> +	check_test_start(vm, sync);
> +	check_test_common(vm, sync, shared_buf, private_buf);
> +	check_test_done(vm, sync);

These function names can be better. These functions are not just 
checking the result, they are actually running the guest code. So, may 
be, calling them 'test_start, test_common and test_done are better. I 
would just collapse them and place their code in test_sev() only if you 
separate the out the setup code as I suggested above.

> +
> +	sev_vm_free(sev);
> +}
> +
> +int main(int argc, char *argv[])
> +{
> +	/* SEV tests */
> +	test_sev(guest_sev_code, SEV_POLICY_NO_DBG);
> +	test_sev(guest_sev_code, 0);
> +
> +	return 0;
> +}