Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) client-ip=2620:137:e000::3:6;
Message-ID: <32c0080f-1bd1-32c4-2bfa-910537a50ca8@amd.com>
Date:   Tue, 3 Oct 2023 12:21:04 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.15.1
Subject: Re: [PATCH kernel v3] x86/compressed/64: reduce #VC nesting for
 intercepted CPUID for SEV-SNP guest
Content-Language: en-US
To:     Alexey Kardashevskiy <aik@amd.com>, x86@kernel.org
Cc:     linux-kernel@vger.kernel.org, Ingo Molnar <mingo@redhat.com>,
        Borislav Petkov <bp@alien8.de>
References: <20231003073123.1763564-1-aik@amd.com>
From:   Tom Lendacky <thomas.lendacky@amd.com>
In-Reply-To: <20231003073123.1763564-1-aik@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?MGpWNkFITHB2bGE0Um9KVFVoSC9jQ1pjRDBpRmlxSVc4TnJrM0Z0KzUvVS9Z?=
 =?utf-8?B?dXByZExTR3lrUGk5VlhUSFJ0ak5nZ0sxellaMDhXS1djTkhDV3F0NWdTcyt5?=
 =?utf-8?B?R1RQNnlqNzJ5cUpoNmFBdHRNWlNaRVBMc0VEVW13dXlGbzNibTlhZlB5NmRF?=
 =?utf-8?B?V0NPZmZncy9oRy90SWhyLzBYMmNwaDJTTGlJejRvRFdaa1UvZENZcWJ5UGt0?=
 =?utf-8?B?OGJ5eThmZHdodkJQbk1mMFNSV3Z5UHdRMDRNU3Fpa3Bpano5NW04TXJ4d2ti?=
 =?utf-8?B?UlcwUWZMSFBYeFJtY0c2VS96b2l5V1hvZDZFc2hGdnpWRmhOWm9rQXFpS1FG?=
 =?utf-8?B?VFlaM2Y0VlZLWUYrblFucitydndXOElDVGxVNU5vaGFBNnh2SUlDaWcrUHBQ?=
 =?utf-8?B?RGlCWDRFVWxWYXovMHAzU21HQ1lqdkdIMnY4VDBuc3hRUUMxWEFFdzVIVllR?=
 =?utf-8?B?akYyeSt1RHJjY3RoaVNQQks5SU90YW5pd1JhTkRyUVowMk13RjZwaW1TdzA3?=
 =?utf-8?B?dUJycDk0SGtoV1oxNmV2eHFuRktDYU9wd1Q5NjJHQnZScnlHOTgyTXNFaHVt?=
 =?utf-8?B?UWF4bXRxZTAwMkpLTWMvR2pRME02am80SkdjL1Z2dHY5c1V3T2NQUW02OTFC?=
 =?utf-8?B?NFVHczh2SFloQ25ZcG1LYnhDUDAvaXVZdFRXZlBtVnpkRGkzWnRwVWhFS3JC?=
 =?utf-8?B?T1RLSmx3V2d5WWJnamdIV3JDOHptRUU1UExPZnBoN2dTYmdYeGszNXNrRmJJ?=
 =?utf-8?B?MGdpU3kwYnRaeUd6VnFPUFFnTDN0em81R2JkTTJwYmV0L0ZYL2tMTC9sbVhM?=
 =?utf-8?B?UVE1Zmt5aEsxSjRoRmxBUHVQTDI0ajBwQVV2VFd2RFM4c25NRytEUWlGdHBT?=
 =?utf-8?B?MmtFQW02RGxzVHYzdSsvUnRoTi92WjRzWERCc1RNbHBjenlSWWN5QUF0SnBx?=
 =?utf-8?B?MHNRZTRZTzBJZEs4WFk3ZnFuZ2FiRVpEYVVkVm9lMGJPdlJ6R1gwSGJxNFEz?=
 =?utf-8?B?UUZvQ3l0ZDYvdk9zbzk1MndqN1VXSmU1ZGt0b3BIYUpQSFV3aHVoYjlwL2J3?=
 =?utf-8?B?RGRuTHNsa3FhRUsweEdjTGFYVkZ1NWE4QlRjRXluM0xnQUVvVGxaM3ZLNWU2?=
 =?utf-8?B?aHpqd01Qek5HWVB1QjJHSE5oS0lRMGxKR1N4TzVZZklielA4UDJqTFpwbVVh?=
 =?utf-8?B?NHJQdElBbFNhTnV3Zk9xVXR6MkljZ1lDSjMzSFB3cU5NemtoVTFPWlMrY3dN?=
 =?utf-8?B?aGhEbXdhbEJUMklBa0pQbG1kdWhwNmFqeWpIb0NYdzNCaHBWTXBkTlcxeEZF?=
 =?utf-8?B?aWVHRngyZUUzand0c1NWTnRRRVdmbjQ2MmpjUjBPUk9iQkJ2am1KZFozblVa?=
 =?utf-8?B?MEd5QU90RVR1QTBDMDBhTkFDNEFNcGM1OHhJVlQ2UTc0WGFudnY1OGFvL01R?=
 =?utf-8?B?dURRWlpQY3BBTWtFMlI5ZVgyejd4Z1FTMDNGRnRIbE5ERWpyVkJPOTl0SUU1?=
 =?utf-8?B?WGd2R2FlTDl4d2pPR2JHcXk3ZWN3VnB2QkpxN05abWNicVNNYVg5ZE5SL0pU?=
 =?utf-8?B?ZHIzbU5lN21EUXJPRU5Gam9kMUdWdzExeS9UZnNJZW9JY1dkenh4OXhMaThQ?=
 =?utf-8?B?cGx3ZjV4M2wwbDlLL2l0bnVST2xERi8xemsrSk92blpQdFVGbWdsclZQWUtS?=
 =?utf-8?B?dHNSR0Jaei9mM3I3Z1JJeGRoZHZKSkhjYTF3dFVsdXFzMUNTby9xZzRtM2ts?=
 =?utf-8?B?RHltMUtESUszZnBNS0gwNk93bDViQ0VhN3ZyUC9taVlTbGkwRzBHNjdJZmhI?=
 =?utf-8?B?QVU2ZU54OGhnejBTZ3Vwb0h6d2ZUZVFmZEwxMWlvbEJXaDFlMDhDQ29nRERa?=
 =?utf-8?B?cGx0RWJSUEVBVFd5bkxleDZ3bUhlUS9vdkNFdTNXMFM3R2RQaElCVGNOM0tP?=
 =?utf-8?B?eks0UitVUW9xYmpBZklJVy90ZFM1NHNrd0NKSEJkMWV5TDc5Tkg4VnRUdDdQ?=
 =?utf-8?B?cG1SNU5UY1FIY0R2aTYzVmkzSGUvSkIvN3VTeGZYcWMycXlxRDFJSnoxdHZ5?=
 =?utf-8?B?d25LRnM4UXk3QTQyWTVuZE5KaDFBUWJLbTVpSTNTZGx2OHhUWlNjOXplS3RQ?=
 =?utf-8?Q?koJEZo1O481DuGtWXa9+Q5mTH?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 09586c05-8fcf-4ff2-0be4-08dbc4352136
X-MS-Exchange-CrossTenant-AuthSource: DM4PR12MB5229.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Oct 2023 17:21:07.4628
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: A8fMw/hKlQjYua3ygkUUluiISrotjLn+3nnluLLSslGqET3kUv2UHZ0BA5hW8x8r6XSmV0+ypVAcLUJQB2dE9g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB5814
X-Spam-Status: No, score=-1.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS
	autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Tue, 03 Oct 2023 10:21:33 -0700 (PDT)

On 10/3/23 02:31, Alexey Kardashevskiy wrote:
> For certain intercepts an SNP guest uses the GHCB protocol to talk to
> the hypervisor from the #VC handler. The protocol requires a shared page so
> there is one per vCPU. In case NMI arrives in a middle of #VC or the NMI
> handler triggers a #VC, there is another "backup" GHCB page which stores
> the content of the first one while SVM_VMGEXIT_NMI_COMPLETE is sent.
> The vc_raw_handle_exception() handler manages main and backup GHCB pages
> via __sev_get_ghcb/__sev_put_ghcb.
> 
> This works fine for #VC and occasional NMIs but not so fine when the #VC
> handler causes intercept + another #VC. If NMI arrives during
> the second #VC, there are no more pages for SVM_VMGEXIT_NMI_COMPLETE.
> The problem place is the #VC CPUID handler which reads an MSR which
> triggers another #VC and if "perf" was running, panic happens:
> 
> Kernel panic - not syncing: Unable to handle #VC exception! GHCB and Backup GHCB are already in use
> 
> Add a helper similar to native_read_msr_safe() for making a direct hypercall
> in the SEV-ES environment. Use the new helper instead of the raw "rdmsr" to
> avoid the extra #VC event.
> 
> Fixes: ee0bfa08a345 ("x86/compressed/64: Add support for SEV-SNP CPUID table in #VC handlers")
> Signed-off-by: Alexey Kardashevskiy <aik@amd.com>
> ---
> 
> Based on:
> https://git.kernel.org/pub/scm/linux/kernel/git/bp/bp.git/log/?h=tip-x86-urgent
> which top at the time was:
> 62d5e970d022 "x86/sev: Change npages to unsigned long in snp_accept_memory()"
> 
> ---
> Changes:
> v3:
> * made it a function, mimic native_read_msr_safe() which 1) returns value 2) returns an error
> * removed debug backtraces the commit log as these were added for debugging and never
> appear with actual kernels
> 
> 
> v2:
> * de-uglify by defining rdmsr_safe_GHCB()
> ---
>   arch/x86/kernel/sev-shared.c | 27 +++++++++++++++++---
>   1 file changed, 23 insertions(+), 4 deletions(-)
> 
> diff --git a/arch/x86/kernel/sev-shared.c b/arch/x86/kernel/sev-shared.c
> index dcf325b7b022..494d92a71986 100644
> --- a/arch/x86/kernel/sev-shared.c
> +++ b/arch/x86/kernel/sev-shared.c
> @@ -241,6 +241,25 @@ static enum es_result sev_es_ghcb_hv_call(struct ghcb *ghcb,
>   	return verify_exception_info(ghcb, ctxt);
>   }
>   
> +
> +/* Paravirt SEV-ES rdmsr which avoids extra #VC event */
> +static unsigned long long ghcb_prot_read_msr(unsigned int msr, struct ghcb *ghcb,
> +					     struct es_em_ctxt *ctxt, int *err)

Alternatively you could return enum es_result and take xss as a 
parameter... six of one, half dozen of another I guess.

> +{
> +	unsigned long long ret = 0;
> +
> +	ghcb_set_rcx(ghcb, msr);
> +
> +	*err = sev_es_ghcb_hv_call(ghcb, ctxt, SVM_EXIT_MSR, 0, 0);
> +	if (*err == ES_OK)
> +		ret = (ghcb->save.rdx << 32) | ghcb->save.rax;

You should check ghcb_rax_is_valid(ghcb) and ghcb_rdx_is_valid(ghcb) 
before using the values.

> +
> +	/* Invalidate qwords for likely another following GHCB call */
> +	vc_ghcb_invalidate(ghcb);

We should probably call this on entry to the function, too, right? Not 
sure it really matters though.

Thanks,
Tom

> +
> +	return ret;
> +}
> +
>   static int __sev_cpuid_hv(u32 fn, int reg_idx, u32 *reg)
>   {
>   	u64 val;
> @@ -477,11 +496,11 @@ static int snp_cpuid_postprocess(struct ghcb *ghcb, struct es_em_ctxt *ctxt,
>   		if (leaf->subfn == 1) {
>   			/* Get XSS value if XSAVES is enabled. */
>   			if (leaf->eax & BIT(3)) {
> -				unsigned long lo, hi;
> +				int err = 0;
>   
> -				asm volatile("rdmsr" : "=a" (lo), "=d" (hi)
> -						     : "c" (MSR_IA32_XSS));
> -				xss = (hi << 32) | lo;
> +				xss = ghcb_prot_read_msr(MSR_IA32_XSS, ghcb, ctxt, &err);
> +				if (err != ES_OK)
> +					return -EINVAL;
>   			}
>   
>   			/*