Received: by 2002:a05:6358:3188:b0:123:57c1:9b43 with SMTP id q8csp1117343rwd; Wed, 7 Jun 2023 11:12:16 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ73eSew+hHDyZpzzp1ytEMCWrzsA9KSFKcLkPlqk7al7V1Y8IqbPJd+Yjn7Crwt3hKDpW0m X-Received: by 2002:a17:902:7c92:b0:1a6:4a64:4d27 with SMTP id y18-20020a1709027c9200b001a64a644d27mr2356588pll.40.1686161536088; Wed, 07 Jun 2023 11:12:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686161536; cv=none; d=google.com; s=arc-20160816; b=zs0HM7YiUiuKc4VnAycbTMTrx0WqeVphwM3wzvYbnwE6ViR/7waWnC7DDVbgwrpOeL 8wUiZoxk3aEvUX1M4QXtPe74h88c6USA/w0Ipm6Qtbt3xp2ip+yqFCKsuFE3h4Atr9a2 cDM2R0bJFd9sJExQOOYN+MCnv/W9RJOdMYOpoOeReMjN9sMYrStIoZLbIpE8MYG+IUzO 1n4uIaLdlKHBn8CckMJr7chc6iOgepM/IgP7HN0CWN5n/jjMNyhVxkjZATcPEERWE+Q1 JahQh6GDILi0UViyCUReP4GO5tD4U0ovAHiBH3UW84M/G3akJCPKz+2sVPRu3vxCKj+c lHlg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=9umhDDx5RLSqs4PUU15GxLF/naDFQMNpdyZWZONRojE=; b=wrAX+v1qiVPZLNG2ye5G8W1f5cZWH8JlQffwpozEXNqNnraE5vknnak8bydRiIEG/H Km/kUfFftb9eRZA92ssdgMunlSEqwQ8hISAINPtQdGT6g4uxXF7xyLG+bSbCKdopBAWz M1Ikh+H20oZEGzaS57hUlxzD6K8AwAlA6TxD1oXLoT0++cer48lb4aMuu7VD83SBtaJq ZEewbjNDiAOBkUaGT02UcG5Xb0A+eCpcw+NAChHlnKaRwBMPiUf0ca1wCtvAmSjcB+ly LOgsN1TtHuiHeDUj9eCUCZBMbSumfLfDDNnRmC3jJUCyrqt+tZOU34D5ZfArarmgD8O3 vq2A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=jjQl0Qb2; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ik8-20020a170902ab0800b001a687bb33f5si9049560plb.434.2023.06.07.11.12.02; Wed, 07 Jun 2023 11:12:16 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=jjQl0Qb2; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232357AbjFGRyB (ORCPT + 99 others); Wed, 7 Jun 2023 13:54:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48692 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231839AbjFGRyA (ORCPT ); Wed, 7 Jun 2023 13:54:00 -0400 Received: from mail-pf1-x44a.google.com (mail-pf1-x44a.google.com [IPv6:2607:f8b0:4864:20::44a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 134671FD4 for ; Wed, 7 Jun 2023 10:53:59 -0700 (PDT) Received: by mail-pf1-x44a.google.com with SMTP id d2e1a72fcca58-66227292952so883230b3a.3 for ; Wed, 07 Jun 2023 10:53:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686160438; x=1688752438; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=9umhDDx5RLSqs4PUU15GxLF/naDFQMNpdyZWZONRojE=; b=jjQl0Qb2QEnMWQpUxbfjqGMep3WxQM8t9AzfNRcnhkyFp2Yjau4KyZEbUX2bhIi3h7 c/mCL4niMQ4hQ7AloWu+grt8efyESK6tt4tBGwIpApfSwhkZrpSf+LLmiAltPzXQykpC OfK/ax2q05MKz4Mv0uT6J8algv1uNwakVijZuk6dFOEpLvv/X6OJEKh+e7GeTCdk0uCO kIt/EygSxf00YgehgrGq6VXJYAFkvU73GIjQYGLptkmX8b6CyzvpTemfkypm0t4TTAtM kq6zBdzMakjwa45BW5I+4w4ZsDvmgiugqm4s5iNphrpynjD/DgRkHyruVeJeZHcNnnFk 2gSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686160438; x=1688752438; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=9umhDDx5RLSqs4PUU15GxLF/naDFQMNpdyZWZONRojE=; b=Zwzugbsyfehk4GHIVrf+XPD41gX5zBxVNz0Ae07UhQ430SH1VtMJIrcSsQEzgGGVol RX7xdTU0UEDVBeNmxSXNk6dHAKWg19BWl8U+MNCS7XE1KKUSLQWH0Ml8x6WggaMaOAYD 9TEZHlogiCcVTk1gNSPWtXE3/bpxK1seP4JPH9Tt6KF++LbXpnhtxo530i4YPkUhg0qd f4AvOzFGLypUsoYKOoM554sRaHz2x14biMyT4F5lu3zzhxdi4qgTOhp6qnlYZVUYWVxR sEc2Zjn+IkSqpe+owIxkaPRUEEXnnYGR50hvD45Ht8R5wg0DOYcQc6aG/BMJEobH2lAy pAvA== X-Gm-Message-State: AC+VfDxGC3/kzjbFcuUlgpd6iq9Sgwu7zzBfOxLFvwF2gnirUuXNBDTr coh28LwQVS7ENWe+2g8D4GhNZjQki+s= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6a00:2d88:b0:651:cea6:f785 with SMTP id fb8-20020a056a002d8800b00651cea6f785mr2762560pfb.0.1686160438575; Wed, 07 Jun 2023 10:53:58 -0700 (PDT) Date: Wed, 7 Jun 2023 10:53:57 -0700 In-Reply-To: <20230607172243.c2bkw43hcet4sfnb@linux.intel.com> Mime-Version: 1.0 References: <20230602011518.787006-1-seanjc@google.com> <20230602011518.787006-2-seanjc@google.com> <20230607073728.vggwcoylibj3cp6s@linux.intel.com> <20230607172243.c2bkw43hcet4sfnb@linux.intel.com> Message-ID: Subject: Re: [PATCH 1/3] KVM: VMX: Retry APIC-access page reload if invalidation is in-progress From: Sean Christopherson To: Yu Zhang Cc: Paolo Bonzini , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Jason Gunthorpe , Alistair Popple , Robin Murphy Content-Type: text/plain; charset="us-ascii" X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Jun 08, 2023, Yu Zhang wrote: > Thanks again! One more thing that bothers me when reading the mmu notifier, > is about the TLB flush request. After the APIC access page is reloaded, the > TLB will be flushed (a single-context EPT invalidation on not-so-outdated > CPUs) in vmx_set_apic_access_page_addr(). But the mmu notifier will send the > KVM_REQ_TLB_FLUSH as well, by kvm_mmu_notifier_invalidate_range_start() -> > __kvm_handle_hva_range(), therefore causing the vCPU to trigger another TLB > flush - normally a global EPT invalidation I guess. Yes. > But, is this necessary? Flushing when KVM zaps SPTEs is definitely necessary. But the flush in vmx_set_apic_access_page_addr() *should* be redundant. > Could we try to return false in kvm_unmap_gfn_range() to indicate no more > flush is needed, if the range to be unmapped falls within guest APIC base, > and leaving the TLB invalidation work to vmx_set_apic_access_page_addr()? No, because vmx_flush_tlb_current(), a.k.a. KVM_REQ_TLB_FLUSH_CURRENT, flushes only the current root, i.e. on the current EP4TA. kvm_unmap_gfn_range() isn't tied to a single vCPU and so needs to flush all roots. We could in theory more precisely track which roots needs to be flushed, but in practice it's highly unlikely to matter as there is typically only one "main" root when TDP (EPT) is in use. In other words, KVM could avoid unnecessarily flushing entries for other roots, but it would incur non-trivial complexity, and the probability of the precise flushing having a measurable impact on guest performance is quite low, at least outside of nested scenarios. But as above, flushing in vmx_set_apic_access_page_addr() shouldn't be necessary. If there were SPTEs, then KVM would already have zapped and flushed. If there weren't SPTEs, then it should have been impossible for the guest to have valid TLB entries. KVM needs to flush when VIRTUALIZE_APIC_ACCESSES is toggled on, as the CPU could have non-vAPIC TLB entries, but that's handled by vmx_set_virtual_apic_mode(). I'll send a follow-up patch to drop the flush from vmx_set_apic_access_page_addr(), I don't *think* I'm missing an edge case...