Received: by 2002:a05:6358:3188:b0:123:57c1:9b43 with SMTP id q8csp5959615rwd; Mon, 5 Jun 2023 10:49:24 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5SAyMafW3xy97YrHTB8pzD/eymn794K/PpVczbDKSyuKnT39xqO/fVT7qEG2nxRPJLOWKv X-Received: by 2002:a05:6214:5087:b0:5dd:5c8d:866f with SMTP id kk7-20020a056214508700b005dd5c8d866fmr8124014qvb.23.1685987364169; Mon, 05 Jun 2023 10:49:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685987364; cv=none; d=google.com; s=arc-20160816; b=r+bU3LTHKOOJijmyu/DzVfg+6ITx4ptdvXxRE7yNs3uMYAMwC4P8igrEL4xilAagrD 0OmUzd1rLYLlkemlK/+iIyBJPpTEGMAder2rz2PNaOp0ZaPUk2uOXHJrK+SqZZf8fwMC lG75VtcV0oW3WxwuYGdhuInO1P5JQ0B/RByofH4A/chBqb4mBTwoInRQMGeFMXLfYzp7 slopRIY3LjTHNrUiXqHqSKiLEnf8ypZLRHlBT6Z0tmZzvQQH7JPTni4G5ytLRsoZwJ9L 2pzTeyP+EPPHZZPgKUPfjTsz2pjkVTf8wSif9dW6EchWtj7msqL+qj5Z8Yq/BbGJCbp3 JeGQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:in-reply-to:content-transfer-encoding :content-disposition:mime-version:references:message-id:subject:cc :to:from:date:dkim-signature; bh=EjQGFEI+Axu+iu1XvUKCACd56nq1aF9xxKk95qDJPyw=; b=UxlFUqkl59QgqKrkPgWcbaHgYy48TDBV34dVyJ52nqOcIyIOFVOIUTQge3x7a4+GiS 6jiROFUmoL60NT7z2uhBocvnJviitaNGFNGweGEAC4RwjBgX+aZGQz8xGu2V5S2JR/Fx PbBnkEo/Nv0niNOMWv8ngSS3f69PN/jeBR2OhgcinRrslD/vbrfWnzZwQt737eAMKQJG ddTB07Y2UcSvENqoZeh34Syjqfas4Bm9JEcqFBo7KITGs90IsKwKoXapiWK77QOQsklZ R88Rx9aFIxTAjmX9yQUsMVP6Z3sncPgvzewXxNvKRwdHd6nlb24hkeQu0FfZDOQJ/GDz bNcw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Ql4Z7OL0; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 7-20020a05621420a700b00626287fd4e5si5174741qvd.273.2023.06.05.10.49.08; Mon, 05 Jun 2023 10:49:24 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Ql4Z7OL0; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235349AbjFERc5 (ORCPT + 99 others); Mon, 5 Jun 2023 13:32:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49926 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234583AbjFERcz (ORCPT ); Mon, 5 Jun 2023 13:32:55 -0400 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CED3BA7; Mon, 5 Jun 2023 10:32:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1685986374; x=1717522374; h=date:from:to:cc:subject:message-id:references: mime-version:content-transfer-encoding:in-reply-to; bh=dpMi+L6MA+Ixw4FIKF3v5NXFh2IOYnea+R3019EXnfs=; b=Ql4Z7OL0kDEsNn5BR296J54690Y3coYaVYhrvLh7BKi9vGz00/RIMRK3 upobAorSjRcgmNrMkaSikPTpB0EhN1Y4nlHUsHLxQTaEXe2sgCv8Fwa5g a7soIbgCCZE8CgWIT9Z1hwO3XR/8TJYs8gGLl7GZ6Ro6LOyR2ZnMNR2at gm+dmJG95VyJNnxJ7Vd1+CPJ1f9y96dPu20ZPfVOXYK2pQQiw9jn1hkKN rhWvbpBSoetOcVnJQctB3vVxUdAAxmwW7zNjqp+6OFhUjGkuXTDidcSR2 MBTRV9t2+jdRIqZ98kaMJPX0DZ8gYOD8fVpvA026inflSnDNrglWhGt6n Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10732"; a="336050482" X-IronPort-AV: E=Sophos;i="6.00,218,1681196400"; d="scan'208";a="336050482" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Jun 2023 10:31:08 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10732"; a="883003346" X-IronPort-AV: E=Sophos;i="6.00,218,1681196400"; d="scan'208";a="883003346" Received: from wstone-mobl1.amr.corp.intel.com (HELO desk) ([10.212.251.108]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Jun 2023 10:31:07 -0700 Date: Mon, 5 Jun 2023 10:31:01 -0700 From: Pawan Gupta To: Jon Kohler Cc: Josh Poimboeuf , Andrew Cooper , Sean Christopherson , Paolo Bonzini , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , X86 ML , "H. Peter Anvin" , "Peter Zijlstra (Intel)" , Daniel Sneddon , "kvm @ vger . kernel . org" , LKML Subject: Re: [PATCH] KVM: VMX: remove LFENCE in vmx_spec_ctrl_restore_host() Message-ID: <20230605173101.iflfly3bt6ydvvyk@desk> References: <20230531150112.76156-1-jon@nutanix.com> <20230531231820.trrs2uugc24gegj4@treble> <20230601004202.63yulqs73kuh3ep6@treble> <846dd0c5-d431-e20e-fdb3-a4a26b6a22ca@citrix.com> <20230601012323.36te7hfv366danpf@desk> <20230601042345.52s5337uz62p6aow@treble> <21D1D290-7DE9-4864-A05B-A36779D9DC26@nutanix.com> <20230605163552.hi5kvh5wijegmus6@treble> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_PASS, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Jun 05, 2023 at 04:39:02PM +0000, Jon Kohler wrote: > > > > On Jun 5, 2023, at 12:35 PM, Josh Poimboeuf wrote: > > > > On Mon, Jun 05, 2023 at 02:29:02PM +0000, Jon Kohler wrote: > >> > >> > >>> On Jun 1, 2023, at 12:23 AM, Josh Poimboeuf wrote: > >>> > >>> On Wed, May 31, 2023 at 06:24:29PM -0700, Pawan Gupta wrote: > >>> > >>> ## 2023-05-31 > >>>> On Thu, Jun 01, 2023 at 01:50:48AM +0100, Andrew Cooper wrote: > >>>>> On 01/06/2023 1:42 am, Josh Poimboeuf wrote: > >>>>>> So each LFENCE has a distinct purpose. That said, there are no indirect > >>>>>> branches or unbalanced RETs between them. > >>>>> > >>>>> How lucky are you feeling? > >>>>> > >>>>> You're in C at this point, which means the compiler could have emitted a > >>>>> call to mem{cpy,cmp}() in place of a simple assignment/comparison. > >>>> > >>>> Moving the second LFENCE to the else part of WRMSR should be possible? > >>>> So that the serialization can be achived either by WRMSR or LFENCE. This > >>>> saves an LFENCE when host and guest value of MSR_SPEC_CTRL differ. > >>> > >>> Yes. Though in practice it might not make much of a difference. With > >>> wrmsr+lfence, the lfence has nothing to do so it might be almost > >>> instantaneous anyway. > >>> > >>> -- > >>> Josh > >> > >> Coming back to this, what if we hoisted call vmx_spec_ctrl_restore_host above > >> FILL_RETURN_BUFFER, and dropped this LFENCE as I did here? > >> > >> That way, we wouldn’t have to mess with the internal LFENCE in nospec-branch.h, > >> and that would act as the “final line of defense” LFENCE. > >> > >> Would that be acceptable? Or does FILL_RETURN_BUFFER *need* to occur > >> before any sort of calls no matter what? > > > > If we go by Intel's statement that only unbalanced RETs are a concern, > > that *might* be ok as long as there's a nice comment above the > > FILL_RETURN_BUFFER usage site describing the two purposes for the > > LFENCE. We would then need FILL_RETURN_BUFFER to unconditionally execute LFENCE to account for wrmsr branch misprediction. Currently LFENCE is not executed for !X86_BUG_EIBRS_PBRSB. > > However, based on Andy's concerns, which I've discussed with him > > privately (but I'm not qualified to agree or disagree with), we may want > > to just convert vmx_spec_ctrl_restore_host() to asm. Better safe than > > sorry. My original implementation of that function was actually asm. I > > can try to dig up that code. Note: VMexit CALL RET RET <---- This is also a problem if the first call hasn't retired yet. LFENCE Converting vmx_spec_ctrl_restore_host() to ASM should be able to take care of this.