Received: by 2002:a05:6a10:206:0:0:0:0 with SMTP id 6csp4448552pxj; Tue, 8 Jun 2021 14:45:51 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyEznfaiY3LNuJy8XqAcQ+Lxc6JOK4dpANtdRu3UbYQhzt176Aj71pnRxntg/bsD7XSyR1L X-Received: by 2002:a05:6402:18c:: with SMTP id r12mr12124519edv.10.1623188751165; Tue, 08 Jun 2021 14:45:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1623188751; cv=none; d=google.com; s=arc-20160816; b=qBljt3Q16qRgSMqEPvd/js1DLELB6GEJ2c0Ai5LEwFxvhCZyeTfMlUzF838vYn5nNG M4R4MToF+cAp8TJvO+V6uJOBs2GDk/Nqt7S3AnPvsFt8s+zFNQ2a2D+5iR2IDUPbkqQi lbI4gyV+c8G7+bfQVFGkTnhaWYnhxlGJypqVhcu9/qrks+KusA4oRxn4DlGNDRQESVhU cmN1mLJjIEW+Z/KFOCZ27ikQ9J0xbeiWhtdLIef628MCVxu54uoSwr1GLNtnuvpfHGAI MTHRb8/hMwKh8X5bVjdhFi+4NyZbcNoulLnqfXVC8A+BWfEAGXKfFs2nf/KHNvKJbEDL hwCw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:subject:message-id:date:from:in-reply-to :references:mime-version:dkim-signature; bh=6UTFwgGaZl+fn/RYJyiPsWbP182wWR0oIXNeHYzwJQA=; b=a5mtzgz8yT9GU/xoKTjorzIwBNa43r7Kp5DIQ27jkjMDEwBq/Cll+8QF5pa5kQQcWw 91S1GlvnexjXFhy8vHwpakQD74G+jk+7pYGCDESfUiWL7A+zqAOSO0nMJ1CtviVfiJhJ TZAES1cJ13Nr/L7FK2zbczJNYJwvn+VShLanjW/lXTSa2mQys77Q463NoH8/QJes3GpT 0EUXdR/X4KSyiH6nTNkZCFf9kf2k2Iz4mMzp2raSOi9wyrihRP9xn6eckC4i1hma1i5G pC8BC3MdNTI3PLdY2Er3IGBCd7vsrBaVmaX15IY9mBvEvOUBQgXitRmZzbcfWgNK5N4n u+gg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel-com.20150623.gappssmtp.com header.s=20150623 header.b=vjMLtn3O; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id d26si806196edr.567.2021.06.08.14.45.28; Tue, 08 Jun 2021 14:45:51 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@intel-com.20150623.gappssmtp.com header.s=20150623 header.b=vjMLtn3O; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234643AbhFHVnw (ORCPT + 99 others); Tue, 8 Jun 2021 17:43:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42680 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234344AbhFHVnv (ORCPT ); Tue, 8 Jun 2021 17:43:51 -0400 Received: from mail-pg1-x534.google.com (mail-pg1-x534.google.com [IPv6:2607:f8b0:4864:20::534]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 49431C061574 for ; Tue, 8 Jun 2021 14:41:58 -0700 (PDT) Received: by mail-pg1-x534.google.com with SMTP id o9so14811187pgd.2 for ; Tue, 08 Jun 2021 14:41:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=6UTFwgGaZl+fn/RYJyiPsWbP182wWR0oIXNeHYzwJQA=; b=vjMLtn3OfN3Eun12/4wGEsK4B3v7SKRflcwqzjuqPCqhj+YyNKYgfhxXFTdqZM1Cqd HlCv+z/ZOJrjWw6tv2kuBukESKyE1Y9tkYFX6Qcmtp5yOnX/rwFI9rOk8+M9rfnGuUeM hPutUZt2M7CVvVJqDQfm56pTN7s+FdAD/GnjhFKjjq8vk4/crY3lazNZDaVFvlzYQ6Oi 9cMaLGpHF/blQ1Hpu77FJbEFMwtIeuEqBHrda0h14qh+8ualtiKfK9CpUik8k7VCriWq BPZi6cWJozc0ZDkDzn1BrobRbN4KQZd4h6KVYjTfFnbtbKopcQSR/FTjMYWj+dc1I+6J NkoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=6UTFwgGaZl+fn/RYJyiPsWbP182wWR0oIXNeHYzwJQA=; b=P+6+xMommj3CGnnZgS9ny0/K7FiQwwdW/wnkui86TBk/rv/H0uL+1msNJIxMn/pg7v L2oayE84cq3vfXcf/3lAfJPsGwI/fa+cy7tucy7l7SDmxjmTeh6UBKV94EzHXRSdNvy2 E+RMaF9M6ckPH9Gtp45UlHw2BNLEWqHw8JPYftG+IxqqyLHVPpaLY3C1is94OJnKZBl+ 5SLLYV8lKF9dvo/kvjmcGqYAawt8xs4Q0b1GjQr/zIbEOfHasjifAnWesnr8oDlWZJox hsLFuimTyVRndYhMIJqouxh8U8sqx3mJWRGBYuIjbQL91VCf59/4xdeElRDEPDmsdu+I BMnw== X-Gm-Message-State: AOAM532U5IHTMuMM+pgd2YKICN4q0l8hOZ7FQu0iZWzZxpkv2vVKMngR Ga9KV9tD4eL+UkMbIocrh8X4+yx/Zf32VzOMAKmC2Q== X-Received: by 2002:a63:195b:: with SMTP id 27mr287943pgz.450.1623188517724; Tue, 08 Jun 2021 14:41:57 -0700 (PDT) MIME-Version: 1.0 References: <20210608213527.739474-1-sathyanarayanan.kuppuswamy@linux.intel.com> In-Reply-To: <20210608213527.739474-1-sathyanarayanan.kuppuswamy@linux.intel.com> From: Dan Williams Date: Tue, 8 Jun 2021 14:41:46 -0700 Message-ID: Subject: Re: [RFC v2-fix-v3 1/1] x86/tdx: Skip WBINVD instruction for TDX guest To: Kuppuswamy Sathyanarayanan Cc: Peter Zijlstra , Andy Lutomirski , Dave Hansen , Tony Luck , Andi Kleen , Kirill Shutemov , Kuppuswamy Sathyanarayanan , Raj Ashok , Sean Christopherson , Linux Kernel Mailing List , "Rafael J. Wysocki" , Linux ACPI Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org [ add Rafael and linux-acpi ] On Tue, Jun 8, 2021 at 2:35 PM Kuppuswamy Sathyanarayanan wrote: > > Current TDX spec does not have support to emulate the WBINVD > instruction. So, add support to skip WBINVD instruction in > drivers that are currently enabled in the TDX guest. > > Functionally only devices outside the CPU (such as DMA devices, > or persistent memory for flushing) can notice the external side > effects from WBINVD's cache flushing for write back mappings. > One exception here is MKTME, but that is not visible outside > the TDX module and not possible inside a TDX guest. > > Currently TDX does not support DMA, because DMA typically needs > uncached access for MMIO, and the current TDX module always > sets the IgnorePAT bit, which prevents that. > > Persistent memory is also currently not supported. Another code > path that uses WBINVD is the MTRR driver, but EPT/virtualization > always disables MTRRs so those are not needed. This all implies > WBINVD is not needed with current TDX. > > So, most drivers/code-paths that use wbinvd instructions are > already disabled for TDX guest platforms via config-option/BIOS. > Following are the list of drivers that use wbinvd instruction > and are still enabled for TDX guests. > > drivers/acpi/sleep.c > drivers/acpi/acpica/hwsleep.c > > Since cache is always coherent in TDX guests, making wbinvd as > noop should not cause any issues. This behavior is the same as > KVM guest. > > Also, hwsleep shouldn't happen for TDX guest because the TDX > BIOS won't enable it, but it's better to disable it anyways > > Signed-off-by: Kuppuswamy Sathyanarayanan > --- > > Changes since RFC v2-fix-v2: > * Instead of handling WBINVD #VE exception as nop, we skip its > usage in currently enabled drivers. > * Adapted commit log for above change. > > arch/x86/kernel/tdx.c | 1 + > drivers/acpi/acpica/hwsleep.c | 12 +++++++++--- > drivers/acpi/sleep.c | 26 +++++++++++++++++++++++--- > include/linux/protected_guest.h | 2 ++ > 4 files changed, 35 insertions(+), 6 deletions(-) > > diff --git a/arch/x86/kernel/tdx.c b/arch/x86/kernel/tdx.c > index 1caf9fa5bb30..e33928131e6a 100644 > --- a/arch/x86/kernel/tdx.c > +++ b/arch/x86/kernel/tdx.c > @@ -100,6 +100,7 @@ bool tdx_protected_guest_has(unsigned long flag) > case PR_GUEST_MEM_ENCRYPT_ACTIVE: > case PR_GUEST_UNROLL_STRING_IO: > case PR_GUEST_SHARED_MAPPING_INIT: > + case PR_GUEST_DISABLE_WBINVD: > return true; > } > > diff --git a/drivers/acpi/acpica/hwsleep.c b/drivers/acpi/acpica/hwsleep.c > index 14baa13bf848..9d40df1b8a74 100644 > --- a/drivers/acpi/acpica/hwsleep.c > +++ b/drivers/acpi/acpica/hwsleep.c > @@ -9,6 +9,7 @@ > *****************************************************************************/ > > #include > +#include > #include "accommon.h" > > #define _COMPONENT ACPI_HARDWARE > @@ -108,9 +109,14 @@ acpi_status acpi_hw_legacy_sleep(u8 sleep_state) > pm1a_control |= sleep_enable_reg_info->access_bit_mask; > pm1b_control |= sleep_enable_reg_info->access_bit_mask; > > - /* Flush caches, as per ACPI specification */ > - > - ACPI_FLUSH_CPU_CACHE(); > + /* > + * WBINVD instruction is not supported in TDX > + * guest. Since ACPI_FLUSH_CPU_CACHE() uses > + * WBINVD, skip cache flushes for TDX guests. > + */ > + if (prot_guest_has(PR_GUEST_DISABLE_WBINVD)) > + /* Flush caches, as per ACPI specification */ > + ACPI_FLUSH_CPU_CACHE(); > > status = acpi_os_enter_sleep(sleep_state, pm1a_control, pm1b_control); > if (status == AE_CTRL_TERMINATE) { > diff --git a/drivers/acpi/sleep.c b/drivers/acpi/sleep.c > index df386571da98..3d6c213481f0 100644 > --- a/drivers/acpi/sleep.c > +++ b/drivers/acpi/sleep.c > @@ -18,6 +18,7 @@ > #include > #include > #include > +#include > #include > #include > > @@ -71,7 +72,14 @@ static int acpi_sleep_prepare(u32 acpi_state) > acpi_set_waking_vector(acpi_wakeup_address); > > } > - ACPI_FLUSH_CPU_CACHE(); > + > + /* > + * WBINVD instruction is not supported in TDX > + * guest. Since ACPI_FLUSH_CPU_CACHE() uses > + * WBINVD, skip cache flushes for TDX guests. > + */ > + if (prot_guest_has(PR_GUEST_DISABLE_WBINVD)) > + ACPI_FLUSH_CPU_CACHE(); > #endif > printk(KERN_INFO PREFIX "Preparing to enter system sleep state S%d\n", > acpi_state); > @@ -566,7 +574,13 @@ static int acpi_suspend_enter(suspend_state_t pm_state) > u32 acpi_state = acpi_target_sleep_state; > int error; > > - ACPI_FLUSH_CPU_CACHE(); > + /* > + * WBINVD instruction is not supported in TDX > + * guest. Since ACPI_FLUSH_CPU_CACHE() uses > + * WBINVD, skip cache flushes for TDX guests. > + */ > + if (prot_guest_has(PR_GUEST_DISABLE_WBINVD)) > + ACPI_FLUSH_CPU_CACHE(); > > trace_suspend_resume(TPS("acpi_suspend"), acpi_state, true); > switch (acpi_state) { > @@ -899,7 +913,13 @@ static int acpi_hibernation_enter(void) > { > acpi_status status = AE_OK; > > - ACPI_FLUSH_CPU_CACHE(); > + /* > + * WBINVD instruction is not supported in TDX > + * guest. Since ACPI_FLUSH_CPU_CACHE() uses > + * WBINVD, skip cache flushes for TDX guests. > + */ > + if (prot_guest_has(PR_GUEST_DISABLE_WBINVD)) > + ACPI_FLUSH_CPU_CACHE(); > > /* This shouldn't return. If it returns, we have a problem */ > status = acpi_enter_sleep_state(ACPI_STATE_S4); > diff --git a/include/linux/protected_guest.h b/include/linux/protected_guest.h > index adfa62e2615e..0ec4dab86f67 100644 > --- a/include/linux/protected_guest.h > +++ b/include/linux/protected_guest.h > @@ -18,6 +18,8 @@ > #define PR_GUEST_HOST_MEM_ENCRYPT 0x103 > /* Support for shared mapping initialization (after early init) */ > #define PR_GUEST_SHARED_MAPPING_INIT 0x104 > +/* Support to disable WBINVD */ > +#define PR_GUEST_DISABLE_WBINVD 0x105 > > #if defined(CONFIG_INTEL_TDX_GUEST) || defined(CONFIG_AMD_MEM_ENCRYPT) > > -- > 2.25.1 >