Message-ID: <e3c53388-f332-5b52-c724-a42d8ea624a7@amd.com>
Date:   Wed, 1 Mar 2023 12:11:57 +0100
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Thunderbird/102.6.0
Subject: Re: [RFC PATCH V3 13/16] x86/sev: Add Check of #HV event in path
Content-Language: en-US
To:     Tianyu Lan <ltykernel@gmail.com>, luto@kernel.org,
        tglx@linutronix.de, mingo@redhat.com, bp@alien8.de,
        dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com,
        seanjc@google.com, pbonzini@redhat.com, jgross@suse.com,
        tiala@microsoft.com, kirill@shutemov.name,
        jiangshan.ljs@antgroup.com, peterz@infradead.org,
        ashish.kalra@amd.com, srutherford@google.com,
        akpm@linux-foundation.org, anshuman.khandual@arm.com,
        pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com,
        daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com,
        sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com,
        michael.roth@amd.com, thomas.lendacky@amd.com,
        venu.busireddy@oracle.com, sterritt@google.com,
        tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com
Cc:     linux-kernel@vger.kernel.org, kvm@vger.kernel.org,
        linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org
References: <20230122024607.788454-1-ltykernel@gmail.com>
 <20230122024607.788454-14-ltykernel@gmail.com>
From:   "Gupta, Pankaj" <pankaj.gupta@amd.com>
In-Reply-To: <20230122024607.788454-14-ltykernel@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?WG1KaDhRc0VaNHBWc2FOUGxpV2I1dzVudU1KR2xQSDJ0Q3hkUGpCdHMvbWZB?=
 =?utf-8?B?SjZBMkRheUlXVVdXZWJ3WCtwTjFZdjZscTZsdVhsemJNTi9kUHNhSFFyZkpW?=
 =?utf-8?B?R2dqZGRrY3E0QiswMXFBQlhkUUlpTU5UMDFsVmRUc1R2TUlvRjl3TEc4R2Ja?=
 =?utf-8?B?aE9xWGdNNnpybzMzU2RkNWE2dndSVmgrWEFOVlZ6MFNhZTNNTVhjbG9yNm9a?=
 =?utf-8?B?YUltOUJtWVdOSW4vMFJUQzE1aWZjMzZYWFhPQlgwVmxjV2x1T2p0S0VNa21Z?=
 =?utf-8?B?TWlWYnl6dWtldy9zbityMEpZcnk3MDlwZEZGeng5aldSM3ZSdVB0Q0FYWTl5?=
 =?utf-8?B?bXMwaDlJSGpFWEFmc3l2U3hjcXFHVllOWllBRk9hcWkyNVVJb1hZUU41SHVD?=
 =?utf-8?B?M1dEblBYckFET3lLd2JhMlk4VkdzZHJKcFU1aFdiQzNjMW9udkpaOE5NOXV1?=
 =?utf-8?B?WUQ1SWR5VHlVcG5hRVNsRzBJVVRjT2hoLzVjM0FPQzBFR05PTXp1U3BRWWZv?=
 =?utf-8?B?WkRmY2VURFNRbldiRVhZTVBjaGdBaWh0V20zT01nTHAyWmVOelFpcmxFTjE5?=
 =?utf-8?B?T1RMQXJLNENOUUhubVhhOERtQ09YQmpBWWRBM1dQWEltSG00YVlqa0gvYlox?=
 =?utf-8?B?eUNnSHNtcFhEdFBBakM4R0RWVFNsWmxRSGVZSk8weGlCejZjOUNCSks0OXdI?=
 =?utf-8?B?RGFJUXVicTZBVm9mdUYrbUZlQ2p4b3Q2QWViNXNNSTQ2NWU0Q21vYmFIZVJ5?=
 =?utf-8?B?SVJUdmhnOE9xaVNXZ3JQTXNDUDVXbmQ0NG0vQ3htQ3hlcktESlFZZjZXQlZM?=
 =?utf-8?B?QXh4aTZnM2Ryd0h2RXRFaXcyNEtQWTdBZjU5TjlQcDYwTEY4ZS9ZOWVpTnBL?=
 =?utf-8?B?NlBqbGVFSTQ3Z1Vjc2JudzFYYUJBSlZob1JlaUxlU2MrNEYrRzhoNWJ1K29u?=
 =?utf-8?B?aitmdm9aNUI4Mk5YaEdpeVp2cnpDNWhoUHBlRm00bHpHaWthTVFoVDFsZXp6?=
 =?utf-8?B?U3NRcGdTdGNkek5PODM2TTRpVDdsNERidlg1eG00emYyK01qa0lXK2NzQUo0?=
 =?utf-8?B?dTdmUmVNdjNkVnVlU1M3R2o3MzBEVG82MUZKRkFXRXRyaFBDWmZ3ZXpYU1hR?=
 =?utf-8?B?S3BQOHQrWTZ1dTlpS0FOTWRtbkZYRi9UNXBYbTVNeVBGUEV3S092dGQrNHRC?=
 =?utf-8?B?WlltUmhmd3I1SUZiUWdiL3BIcHdQRVh0bTNtejI4bUZncjVhNnFFSkt0RWsz?=
 =?utf-8?B?aVVCWE1JTWxiVWFQb1R6aVpwMVArSURHL3NaMit4VVl3cTRvYUVRK09NSm1i?=
 =?utf-8?B?L2NSOFJjTlRlT2l5MXhtMm5UTUVsM0ZsbURlZDNNWUJUSHZiMkg5M3Rhc2lY?=
 =?utf-8?B?alRHc1hvSFAvMHhkL3k1ZmFDV0pqQ3ZCR3lnaUtQbTFnSjdNWC93bTBNd0Rl?=
 =?utf-8?B?MGsxanZqNnhrdDRpM3lrYlpuSVdWTXBzSitIV09BSGUvZUlFQnRoSUZ5emcz?=
 =?utf-8?B?STg4ZnlrRFcwMVc2UG1oREdBRjNmY1JmSlgySitXYjAvRmJ4ZkVRczhlOUow?=
 =?utf-8?B?OUVEcGc3NFlOb00wRDFpVjlsakxuMkJCU3U4TVZvbm1CWW5GQWxGM2xZM2VD?=
 =?utf-8?B?OVZ6R29UaUcwaTU0ZUgvL3NXQnpKR0tkTEZiVjBnU05oc3gzVzE5cGlPZlUv?=
 =?utf-8?B?YzhNRVdMQ2tQT3k3YWxTL0lpTTFnVkhidXpIWjRSRmZhT2tWTlA3QkJTc212?=
 =?utf-8?B?YWMxUG1aems3eDYzTEtBSTJyYkNqcmhyN09TcnRNRHNzUmZXdWRZSjFqTFEx?=
 =?utf-8?B?aGZYMkZYazVJR2VTdVMxa3FqWTZhSVdtSjF1c3FYL3lKa285NmRwd1o3NGdB?=
 =?utf-8?B?QTB5RjN4RkdZQTFBQ2lpQzZTNzJIWGtTcG5qSkFjNWVjN2ROeDRvY2ZBQWg5?=
 =?utf-8?B?NHdjSVl1VEs0bFBFMjRyeUVEYkh3L2NxM0M4eml4NGFvWW4wVG56SlRBdzZp?=
 =?utf-8?B?YWswRk82NGh0bld5ZHNiaml0YVhNQjhxZUYwakc3Si9aT2txMk1XWDMyVXJ2?=
 =?utf-8?B?N0F5TFkxSXRMYmh6UEE3RXJhSDJRTWE4eHA1Z2V3R3liTVpuSHN0ZjNMMDlZ?=
 =?utf-8?Q?qf8xKo0JYzh4ZYXlSNsjLCHfi?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: bd344748-59f6-4966-4923-08db1a45cdff
X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB2810.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Mar 2023 11:12:11.7307
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Szf03JdYYiSP5oXU1GAYY1lxpGHgibA7co6G6kTL4Znbw7aWS790nhwuCM5jizbi7U/ORZCt9y3P6n9LGedbEQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB6828
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 1/22/2023 3:46 AM, Tianyu Lan wrote:
> From: Tianyu Lan <tiala@microsoft.com>
> 
> Add check_hv_pending() and check_hv_pending_after_irq() to
> check queued #HV event when irq is disabled.
> 
> Signed-off-by: Tianyu Lan <tiala@microsoft.com>
> ---
>   arch/x86/entry/entry_64.S       | 18 +++++++++++++++
>   arch/x86/include/asm/irqflags.h | 10 +++++++++
>   arch/x86/kernel/sev.c           | 39 +++++++++++++++++++++++++++++++++
>   3 files changed, 67 insertions(+)
> 
> diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S
> index 6baec7653f19..aec8dc4443d1 100644
> --- a/arch/x86/entry/entry_64.S
> +++ b/arch/x86/entry/entry_64.S
> @@ -1064,6 +1064,15 @@ SYM_CODE_END(paranoid_entry)
>    * R15 - old SPEC_CTRL
>    */
>   SYM_CODE_START_LOCAL(paranoid_exit)
> +#ifdef CONFIG_AMD_MEM_ENCRYPT
> +	/*
> +	 * If a #HV was delivered during execution and interrupts were
> +	 * disabled, then check if it can be handled before the iret
> +	 * (which may re-enable interrupts).
> +	 */
> +	mov     %rsp, %rdi
> +	call    check_hv_pending
> +#endif
>   	UNWIND_HINT_REGS
>   
>   	/*
> @@ -1188,6 +1197,15 @@ SYM_CODE_START(error_entry)
>   SYM_CODE_END(error_entry)
>   
>   SYM_CODE_START_LOCAL(error_return)
> +#ifdef CONFIG_AMD_MEM_ENCRYPT
> +	/*
> +	 * If a #HV was delivered during execution and interrupts were
> +	 * disabled, then check if it can be handled before the iret
> +	 * (which may re-enable interrupts).
> +	 */
> +	mov     %rsp, %rdi
> +	call    check_hv_pending
> +#endif
>   	UNWIND_HINT_REGS
>   	DEBUG_ENTRY_ASSERT_IRQS_OFF
>   	testb	$3, CS(%rsp)
> diff --git a/arch/x86/include/asm/irqflags.h b/arch/x86/include/asm/irqflags.h
> index 7793e52d6237..fe46e59168dd 100644
> --- a/arch/x86/include/asm/irqflags.h
> +++ b/arch/x86/include/asm/irqflags.h
> @@ -14,6 +14,10 @@
>   /*
>    * Interrupt control:
>    */
> +#ifdef CONFIG_AMD_MEM_ENCRYPT
> +void check_hv_pending(struct pt_regs *regs);
> +void check_hv_pending_irq_enable(void);
> +#endif
>   
>   /* Declaration required for gcc < 4.9 to prevent -Werror=missing-prototypes */
>   extern inline unsigned long native_save_fl(void);
> @@ -43,12 +47,18 @@ static __always_inline void native_irq_disable(void)
>   static __always_inline void native_irq_enable(void)
>   {
>   	asm volatile("sti": : :"memory");
> +#ifdef CONFIG_AMD_MEM_ENCRYPT
> +	check_hv_pending_irq_enable();
> +#endif
>   }
>   
>   static inline __cpuidle void native_safe_halt(void)
>   {
>   	mds_idle_clear_cpu_buffers();
>   	asm volatile("sti; hlt": : :"memory");
> +#ifdef CONFIG_AMD_MEM_ENCRYPT
> +	check_hv_pending_irq_enable();
> +#endif
>   }
>   
>   static inline __cpuidle void native_halt(void)
> diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c
> index a8862a2eff67..fe5e5e41433d 100644
> --- a/arch/x86/kernel/sev.c
> +++ b/arch/x86/kernel/sev.c
> @@ -179,6 +179,45 @@ void noinstr __sev_es_ist_enter(struct pt_regs *regs)
>   	this_cpu_write(cpu_tss_rw.x86_tss.ist[IST_INDEX_VC], new_ist);
>   }
>   
> +static void do_exc_hv(struct pt_regs *regs)
> +{
> +	/* Handle #HV exception. */
> +}
> +
> +void check_hv_pending(struct pt_regs *regs)
> +{
> +	if (!cc_platform_has(CC_ATTR_GUEST_SEV_SNP))
> +		return;
> +
> +	if ((regs->flags & X86_EFLAGS_IF) == 0)
> +		return;

Will this return and prevent guest from executing NMI's
while irqs are disabled?

Thanks,
Pankaj

> +
> +	do_exc_hv(regs);
> +}
> +
> +void check_hv_pending_irq_enable(void)
> +{
> +	unsigned long flags;
> +	struct pt_regs regs;
> +
> +	if (!cc_platform_has(CC_ATTR_GUEST_SEV_SNP))
> +		return;
> +
> +	memset(&regs, 0, sizeof(struct pt_regs));
> +	asm volatile("movl %%cs, %%eax;" : "=a" (regs.cs));
> +	asm volatile("movl %%ss, %%eax;" : "=a" (regs.ss));
> +	regs.orig_ax = 0xffffffff;
> +	regs.flags = native_save_fl();
> +
> +	/*
> +	 * Disable irq when handle pending #HV events after
> +	 * re-enabling irq.
> +	 */
> +	asm volatile("cli" : : : "memory");
> +	do_exc_hv(&regs);
> +	asm volatile("sti" : : : "memory");
> +}
> +
>   void noinstr __sev_es_ist_exit(void)
>   {
>   	unsigned long ist;