Received: by 2002:a05:6a10:9848:0:0:0:0 with SMTP id x8csp877221pxf; Thu, 1 Apr 2021 16:38:37 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyr5824vahvekSUGKjVALIKp3xiWhV/Hj9lK04qLYcgV2L7MRkmwaAeR8A77ml4JI9U8xas X-Received: by 2002:a05:6402:4309:: with SMTP id m9mr13047339edc.25.1617320317598; Thu, 01 Apr 2021 16:38:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1617320317; cv=none; d=google.com; s=arc-20160816; b=uwEO9frQsHv3m4Qjspvmr1ZgqIiYX0vXIe/nSuRV2BI53aVemc8clYurqbv6QYezrX hUR135c3f6lAlU/NWuhXONPJ17FLyCkEwJxjxtCAsWz11/yoSk8rNheRrmpOdW+/sRUN Zmd7ovWWGZ7KmhXZ2Iq3J2vpyNowSIQWECT1IYgZmtRZ5LTkZaiVTtOUV+SBCi57hGZ4 WKkR/JrusAKnraiHFdaLASMSOgva0vGJcpQcPhDNpMffcQFE+wd6aJg+IwI9K49e7OKQ ix4WU8cQz51Z9keaaS3JtX6itGTLZA7VMoUTZkIvuqYSb09JbreEJbRgtrI+/VrYd2Aj u7Nw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:mime-version:message-id:date :dkim-signature; bh=vJ782tPIwg5HrJEEBMGpQmwNZsk9pDTuVerO6dJHWsw=; b=0cXJ1BZXDRQDxRfgFpV4dU6C85L3F+kelDQEKmp7g7ozzJg/lm/wxzKCObVrK9oUVL QCprVZ5sYvWqNgYvsR0Am2bJqgFNsWgudpLRv0bbtHUhH7fgd5ElablXlG7sRo8GSH8D 5cI4/ERK0unUOZ1kjFup/CJRKbaLb7LidKuBBqbQS8k91IKU8iuL7lQibWi0oR9LZTma aj2VrpFZQlumVzIjf8AOfu49thl74TWNBwU0/1lF/D47MXvtAJecPDrC0B3wrshFjTrv mvnLJDmByU0j8e25x20dMSamqDdzpgUVWJYPr8onlhWqS0Pg/8Jxr/UhCfTux+yxL5V0 txnw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=vTJwbEAu; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id e8si5435686edq.351.2021.04.01.16.38.13; Thu, 01 Apr 2021 16:38:37 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=vTJwbEAu; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235930AbhDAXhp (ORCPT + 99 others); Thu, 1 Apr 2021 19:37:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51324 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233677AbhDAXho (ORCPT ); Thu, 1 Apr 2021 19:37:44 -0400 Received: from mail-qk1-x74a.google.com (mail-qk1-x74a.google.com [IPv6:2607:f8b0:4864:20::74a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E6021C061788 for ; Thu, 1 Apr 2021 16:37:42 -0700 (PDT) Received: by mail-qk1-x74a.google.com with SMTP id p133so4772223qka.17 for ; Thu, 01 Apr 2021 16:37:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:message-id:mime-version:subject:from:to:cc; bh=vJ782tPIwg5HrJEEBMGpQmwNZsk9pDTuVerO6dJHWsw=; b=vTJwbEAu/t/5wjVhuKeAyWFRl/XKhnrScZ8p3fG7OVHAXGKID0UIxVoi3bwPA1Sa3z my7FswdYJWVOzOXsMwWMytT299p+3GmTzjBLfeTdg/IOiEhlCljIrAnmilsr5kCXRkcQ C9IiLRGlN2QDwVhACUN3SrDAmIiAQ7vc2E/YKySqepqxLCMrUGZYG+DnufFUJDWcfneZ L1R6Ybn7nUc0YA13CaKBcfj1q3FLfdISK9pc+q6bZ4FEkT1t6oit+5jjUDUz1cU3z+Re LurQ/wYnapokxkghUBE1y7e2c0fM5oHYTpP4ZVUqgWg2eRakJXUtvvMxCcgRlRXw5yhO FUhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:mime-version:subject:from:to:cc; bh=vJ782tPIwg5HrJEEBMGpQmwNZsk9pDTuVerO6dJHWsw=; b=rhYsUvn1D9nl8GayLul8XFxMIdb8sIhEsnGQQf1EGucdwUr+Lrhf0S/BcV3vVQAXGO BcNxrgM/NzKRMks4eCNOKqG1irgIwnSHL+ddqk+94/Ugp36dL5Mthp5qE3he/22qO54i 5TUmhdfH5vaaai7YNHsDDOVrkcvpwQU1SlrBbH7PKa4i8XTFNW6LxKMLU5I6kKuDzoDS LIzEKH1SY0yA9Y2ugfHP/jBqqr7taJIptuSOQ4d/z9jvAEVBOz0Sl7HYugt380uhSqzs LYsAoq8J8HYuKZfhiF+i9jtEjn9FO4vK5Du52bmm8jW0t6d2lywmh6lz9s7jjt3ZLBD7 l+wg== X-Gm-Message-State: AOAM530uq2RWJr+3szZVDKANM+YwRzk6CxHLn5hPeDSNcUtBqA+3/u7b 3vgBXh9O0F18v5GwlQqBaMr1yvRBpN9BSmAxqG/vM08VDYJFujklln+WxhYGmPQNiq+lLtdCedg Mda99LuWrdxv4ayNPr1Rj1B1Gvp+4slGuXyrM5X7DhzLrvZsPnVDyKtD1sJ07AT4fkFioZLPE X-Received: from bgardon.sea.corp.google.com ([2620:15c:100:202:e088:88b8:ea4a:22b6]) (user=bgardon job=sendgmr) by 2002:a0c:fd62:: with SMTP id k2mr10563221qvs.51.1617320261994; Thu, 01 Apr 2021 16:37:41 -0700 (PDT) Date: Thu, 1 Apr 2021 16:37:23 -0700 Message-Id: <20210401233736.638171-1-bgardon@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.31.0.208.g409f899ff0-goog Subject: [PATCH v2 00/13] More parallel operations for the TDP MMU From: Ben Gardon To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Paolo Bonzini , Peter Xu , Sean Christopherson , Peter Shier , Peter Feiner , Junaid Shahid , Jim Mattson , Yulei Zhang , Wanpeng Li , Vitaly Kuznetsov , Xiao Guangrong , Ben Gardon Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Now that the TDP MMU is able to handle page faults in parallel, it's a relatively small change to expand to other operations. This series allows zapping a range of GFNs, reclaiming collapsible SPTEs (when disabling dirty logging), and enabling dirty logging to all happen under the MMU lock in read mode. This is partly a cleanup + rewrite of the last few patches of the parallel page faults series. I've incorporated feedback from Sean and Paolo, but the patches have changed so much that I'm sending this as a separate series. Ran kvm-unit-tests + selftests on an SMP kernel + Intel Skylake, with the TDP MMU enabled and disabled. This series introduces no new failures or warnings. I know this will conflict horribly with the patches from Sean's series which were just queued, and I'll send a v2 to fix those conflicts + address any feedback on this v1. Changelog v2: -- Rebased patches on top of kvm/queue to incorporate Sean's recent TLB flushing changes -- Dropped patch 5: "KVM: x86/mmu: comment for_each_tdp_mmu_root requires MMU write lock" as the following patch to protect the roots list with RCU adds lockdep which makes the comment somewhat redundant. Ben Gardon (13): KVM: x86/mmu: Re-add const qualifier in kvm_tdp_mmu_zap_collapsible_sptes KVM: x86/mmu: Move kvm_mmu_(get|put)_root to TDP MMU KVM: x86/mmu: use tdp_mmu_free_sp to free roots KVM: x86/mmu: Merge TDP MMU put and free root KVM: x86/mmu: Refactor yield safe root iterator KVM: x86/mmu: Make TDP MMU root refcount atomic KVM: x86/mmu: handle cmpxchg failure in kvm_tdp_mmu_get_root KVM: x86/mmu: Protect the tdp_mmu_roots list with RCU KVM: x86/mmu: Allow zap gfn range to operate under the mmu read lock KVM: x86/mmu: Allow zapping collapsible SPTEs to use MMU read lock KVM: x86/mmu: Allow enabling / disabling dirty logging under MMU read lock KVM: x86/mmu: Fast invalidation for TDP MMU KVM: x86/mmu: Tear down roots in fast invalidation thread arch/x86/include/asm/kvm_host.h | 21 +- arch/x86/kvm/mmu/mmu.c | 115 +++++++--- arch/x86/kvm/mmu/mmu_internal.h | 27 +-- arch/x86/kvm/mmu/tdp_mmu.c | 375 +++++++++++++++++++++++--------- arch/x86/kvm/mmu/tdp_mmu.h | 28 ++- include/linux/kvm_host.h | 2 +- 6 files changed, 407 insertions(+), 161 deletions(-) -- 2.31.0.208.g409f899ff0-goog