Received: by 2002:a05:7412:251c:b0:e2:908c:2ebd with SMTP id w28csp4419rda; Fri, 20 Oct 2023 17:50:56 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHHwZUzafdSrkNlQkkcrQer8gF0m9YwglvoLaUwH9LmZVdVZeJP0Yk37mdTo0EZPKxJ+91l X-Received: by 2002:a05:6870:7907:b0:1d5:8fb8:98ef with SMTP id hg7-20020a056870790700b001d58fb898efmr4356254oab.31.1697849456283; Fri, 20 Oct 2023 17:50:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697849456; cv=none; d=google.com; s=arc-20160816; b=nnxbBBjZoQp8kRxOQzIbMSgN4PWSc0gLTZ8P2iPiYWm58CFYsybcx/L+3iQJRVU7qZ K+Uv1ERnvgzjt8jOqzE/xXGgbS8jcxtGj83dntktIvVS+7G8O2SMfIJpd/8rOiCLh+gG KjcbVgRntMKBN/+TtpRh6wPtc4SG6V02B1BUIoiijxZK2TmkYwfyXgKZcTMdfI/jp/pY ISPBP+5iDzS1XiCwuwRgndcOE7cU1wAJvuQ5FaZVFHlP1LVCCjOd+2hT3YC/zQsZj871 5nx518q1GnZk+QhJwHbHH7dz2KC8wAZNJAd3oMeKgaSqTq2QYxS0H8nGUiclprSoqXX5 ejSA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:in-reply-to:content-disposition:mime-version :references:message-id:subject:cc:to:from:date:dkim-signature; bh=1FbGNYpQ4/v5S8mPpqTdQh9GM/Xw22DBcFlx1ehKOBg=; fh=WKr3UfMxhTZB20e6PBjls5vGIl/XClUxQy7NcuDEQJM=; b=oOlonH0PAvX8OPwe14JtBhmPCFY19wZXgtYdO0W0N21iTW+Z65eiKpPy8gSNRTIKhg v5pql+hzYUnI9gSYPTfhSzHfIU9pWZmlEtItzmESpPXGLLDPYczC15kq9whgDRfy2cnO Q23p1i6ZS4gRgqmJvscaxBCj2rzOlurSS52ZEcDgpNVCLXY2yG2n4KzlKo1v1LOHkv91 212X5HomxCxiYXzVg8m/6nq6vZyDttI5d59uAusVI0y1RiO4xJTMTwAZ40La3nCtm8HM JZoJDessz8br1xKL4d1mmU54kRLYptDKO75tZZxV2tTNwxaWDSvHxpSb3uuOFkcaTIE/ 4LRQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=IbsS6ABK; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id s10-20020a17090a948a00b002773cc10d3csi5135201pjo.78.2023.10.20.17.50.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Oct 2023 17:50:56 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=IbsS6ABK; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 2E66C8313341; Fri, 20 Oct 2023 17:50:30 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230251AbjJUAqv (ORCPT + 99 others); Fri, 20 Oct 2023 20:46:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40146 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229518AbjJUAqu (ORCPT ); Fri, 20 Oct 2023 20:46:50 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 68B5DD7; Fri, 20 Oct 2023 17:46:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697849205; x=1729385205; h=date:from:to:cc:subject:message-id:references: mime-version:in-reply-to; bh=63X4BWHvqfvxHGbMrOuUwwimh5i4L/jY9/rWTqH/Zkg=; b=IbsS6ABK4iiWRKYhZiKTcWCZp82vj3HVkHOMRQcQpWe3UUxcu2C+tQ8l Nx0DRpc9BumpcldaAOrtkmXwbmJHypSqHK5sVU5Hi+XXuoMinHpS/9IrF iE3mpuQi9hAH0ZIsdhVXaGopt80J1QuDHi/nizP5++hOCwaapwEX5LvPT m5sqf+W5UAfk7hZ7lLkkj13cbved+WnFOtntVOdK5A62fow/ueG/V7Dir XmFieWEuvKG3VykoFqSD9Ds2sQORpcPh9khF33dNjke9VJzvZPZufwBOB UX0lUT4vYAysB5wcyJwHWChgIxlGUN8+XsB/Xg5vB7ejKhnzazmlYnRah g==; X-IronPort-AV: E=McAfee;i="6600,9927,10869"; a="371667259" X-IronPort-AV: E=Sophos;i="6.03,239,1694761200"; d="scan'208";a="371667259" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Oct 2023 17:46:44 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10869"; a="827886437" X-IronPort-AV: E=Sophos;i="6.03,239,1694761200"; d="scan'208";a="827886437" Received: from hkchanda-mobl.amr.corp.intel.com (HELO desk) ([10.209.90.113]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Oct 2023 17:46:43 -0700 Date: Fri, 20 Oct 2023 17:46:33 -0700 From: Pawan Gupta To: Sean Christopherson Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Peter Zijlstra , Josh Poimboeuf , Andy Lutomirski , Jonathan Corbet , Paolo Bonzini , tony.luck@intel.com, ak@linux.intel.com, tim.c.chen@linux.intel.com, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kvm@vger.kernel.org, Alyssa Milburn , Daniel Sneddon , antonio.gomez.iglesias@linux.intel.com Subject: Re: [PATCH 6/6] KVM: VMX: Move VERW closer to VMentry for MDS mitigation Message-ID: <20231021004633.ymughma7zijosku5@desk> References: <20231020-delay-verw-v1-0-cff54096326d@linux.intel.com> <20231020-delay-verw-v1-6-cff54096326d@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Fri, 20 Oct 2023 17:50:30 -0700 (PDT) On Fri, Oct 20, 2023 at 03:55:07PM -0700, Sean Christopherson wrote: > On Fri, Oct 20, 2023, Pawan Gupta wrote: > > During VMentry VERW is executed to mitigate MDS. After VERW, any memory > > access like register push onto stack may put host data in MDS affected > > CPU buffers. A guest can then use MDS to sample host data. > > > > Although likelihood of secrets surviving in registers at current VERW > > callsite is less, but it can't be ruled out. Harden the MDS mitigation > > by moving the VERW mitigation late in VMentry path. > > > > Note that VERW for MMIO Stale Data mitigation is unchanged because of > > the complexity of per-guest conditional VERW which is not easy to handle > > that late in asm with no GPRs available. If the CPU is also affected by > > MDS, VERW is unconditionally executed late in asm regardless of guest > > having MMIO access. > > > > Signed-off-by: Pawan Gupta > > --- > > arch/x86/kvm/vmx/vmenter.S | 9 +++++++++ > > arch/x86/kvm/vmx/vmx.c | 10 +++++++--- > > 2 files changed, 16 insertions(+), 3 deletions(-) > > > > diff --git a/arch/x86/kvm/vmx/vmenter.S b/arch/x86/kvm/vmx/vmenter.S > > index be275a0410a8..efa716cf4727 100644 > > --- a/arch/x86/kvm/vmx/vmenter.S > > +++ b/arch/x86/kvm/vmx/vmenter.S > > @@ -1,6 +1,7 @@ > > /* SPDX-License-Identifier: GPL-2.0 */ > > #include > > #include > > +#include > > #include > > #include > > #include > > @@ -31,6 +32,8 @@ > > #define VCPU_R15 __VCPU_REGS_R15 * WORD_SIZE > > #endif > > > > +#define GUEST_CLEAR_CPU_BUFFERS USER_CLEAR_CPU_BUFFERS > > + > > .macro VMX_DO_EVENT_IRQOFF call_insn call_target > > /* > > * Unconditionally create a stack frame, getting the correct RSP on the > > @@ -177,10 +180,16 @@ SYM_FUNC_START(__vmx_vcpu_run) > > * the 'vmx_vmexit' label below. > > */ > > .Lvmresume: > > + /* Mitigate CPU data sampling attacks .e.g. MDS */ > > + GUEST_CLEAR_CPU_BUFFERS > > I have a very hard time believing that it's worth duplicating the mitigation > for VMRESUME vs. VMLAUNCH just to land it after a Jcc. VERW modifies the flags, so it either needs to be after Jcc or we push/pop flags that adds 2 extra memory operations. Please let me know if there is a better option. > Also, would it'd be better to put the NOP first? Or even better, out of line? > It'd be quite hilarious if the CPU pulled a stupid and speculated on the operand > of the NOP, i.e. if the user/guest controlled RAX allowed for pulling in data > after the VERW. I did confirm with CPU architects that NOP operand won't be dereferenced, even speculatively. But yes, even if it did, moving NOP first does take care of it.