Received: by 2002:a05:6358:7058:b0:131:369:b2a3 with SMTP id 24csp3115684rwp; Fri, 14 Jul 2023 18:04:58 -0700 (PDT) X-Google-Smtp-Source: APBJJlE4ArCNHgqHkXr7NPt2cjvMALTAMh3u70QZxbWBok7FywWIorpEzIm1LHfRxEOsxPdPWaUR X-Received: by 2002:a05:6a20:5487:b0:12f:dce2:b381 with SMTP id i7-20020a056a20548700b0012fdce2b381mr9405235pzk.3.1689383098121; Fri, 14 Jul 2023 18:04:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1689383098; cv=none; d=google.com; s=arc-20160816; b=jqtJ6bbNAjtAdqnHztsmWzkj2vKeXFPj+NcpU2LGB0gQpAfRyiPvcs658MiplEnF29 8D4CdQELEH92jmvDeMTuT/Alyo9fT0h9uS9RZqMVlItpdZwfkSvgsCeE13tdq6/9tXL9 giM84R9kL8QMJLwjdC0cKbkVGP4ReKNFRNZObYWvMJxrmCZr6hF/AKbuzyIm5ZXNTCEB Xb9+YN6yCbr3P4FNN0sTkX15xiXwur6oeqlI9M8VgIBlB4vUtpTHTzWrKkC436EWeMFv ml0KlJp1gSR76/HVTUCY+it7s2SErLhbkQqjx88zpXYGrOW+D/Q8GbASDokTnUxdWBk8 36og== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=WL/8letflrdUyrlyVQF85H/ANybjC/FZDEF4uZbqq6U=; fh=0mgYt7awHARBqooV7+boW/ys14vuECt03vTY4pGVtiM=; b=lWJyRMWAdN4zQOwxm4c7QPijE8UsmwskCk4t2CpLmoNvGd8aDYQA/dWoYB69n352Qx yjVUkhzqqiLHY6tGwAwwdAH52LA6QNLAESQTRsovAKuld/4EMlW1HX76+XTUWGoXyJ7x d2xHX5kQyIRwJef450d4qyOV5zd4k4u4EX3c0/RW+elzoZGq+NdzXXSZgvmiwtY1Hof1 UO4suMPpGj8mlFSeal+pACvonk0C7rZpWNkzAgPVBL797I3lsfWyehcrcFJ2U3MOpL49 n9n+Ne0gPSylm+SsSuWfFC9iQPEei/jhgDv/7Tv8enYnG5A6HQQSy2WAk+qhjTPxA/Pz k28Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=YVDFqRDQ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q16-20020a056a00089000b0066ce96ce70csi3950418pfj.122.2023.07.14.18.04.46; Fri, 14 Jul 2023 18:04:58 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=YVDFqRDQ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230235AbjGOAyY (ORCPT + 99 others); Fri, 14 Jul 2023 20:54:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59424 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230207AbjGOAyQ (ORCPT ); Fri, 14 Jul 2023 20:54:16 -0400 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 62EDA3A9F for ; Fri, 14 Jul 2023 17:54:14 -0700 (PDT) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-57704af0e64so20370147b3.0 for ; Fri, 14 Jul 2023 17:54:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689382453; x=1689987253; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=WL/8letflrdUyrlyVQF85H/ANybjC/FZDEF4uZbqq6U=; b=YVDFqRDQFjjSpZgD+yDd0+SvcfnYZvIm68d0fipWbsts27Blri9IZUegY1gTHk9JLn jW4cOIkHI7T+9t0jnXLUxFTuZO17IqR8lJ6cZXaSpbBldzPtgjsDIh8gJKx+Q5X4q5WY ZPeJbtlH0vB2beRgSMZIMEJ9rm99I7jkv+szOwydpHuY8BG/kmiU0z++M/vX2ikqkedk oaLqutULiyyyTrPh+XolZ6Wg+wDDEoirUIkGO2UNG8Fwk+w94SJmA9G1jANboqPpmGQC SLk94TOF3nNcENDMs73y7DqVrLGQulUm5QIb9no511Uazy0Gh8YgK2tpMEaTmnUm1hja FFGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689382453; x=1689987253; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=WL/8letflrdUyrlyVQF85H/ANybjC/FZDEF4uZbqq6U=; b=LBgsehc3MdcEnmRtMaT9JQY72pWjnZCIZoYk8xcheBXIUk9TG/6T91hSUwXpXGYQYx X9hwzMjpk8RtfjR8IM8Bt+aaYY2efw3A4rLAYBFKbrHpdv1BPbPEi5+I2/zetjuPUKfa AYS25/0Jb+zmiak2f9KB6DCY5Z4wFBcTDuW0OBdenJang7+G14+2zohoY1ThqigbvAEf h+TCNAMxqZ3H23zHFdls3Cs3bDY4254b49kAeOjEuQkuvckVPF04HkHGClM4qW/Hixvg b0LqK01uw03QY6YKmETBWOrYkAOWx0+XBgcO+wt/8smkKCFoDDP7cyxXuAAxnQQe1viL /Dmw== X-Gm-Message-State: ABy/qLaEi/iyPal2V9NN/OROyeKJNukY2rB/yMsk5zEtq3ZQrHh9NKD+ lkB15VgGoVNxWoYyOxe0CQKaTwdol75t X-Received: from rananta-linux.c.googlers.com ([fda3:e722:ac3:cc00:2b:ff92:c0a8:22b5]) (user=rananta job=sendgmr) by 2002:a81:b302:0:b0:565:a42c:79fe with SMTP id r2-20020a81b302000000b00565a42c79femr46926ywh.1.1689382453561; Fri, 14 Jul 2023 17:54:13 -0700 (PDT) Date: Sat, 15 Jul 2023 00:53:57 +0000 In-Reply-To: <20230715005405.3689586-1-rananta@google.com> Mime-Version: 1.0 References: <20230715005405.3689586-1-rananta@google.com> X-Mailer: git-send-email 2.41.0.455.g037347b96a-goog Message-ID: <20230715005405.3689586-4-rananta@google.com> Subject: [PATCH v6 03/11] KVM: Allow range-based TLB invalidation from common code From: Raghavendra Rao Ananta To: Oliver Upton , Marc Zyngier , James Morse , Suzuki K Poulose Cc: Paolo Bonzini , Sean Christopherson , Huacai Chen , Zenghui Yu , Anup Patel , Atish Patra , Jing Zhang , Colton Lewis , Raghavendra Rao Anata , David Matlack , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-mips@vger.kernel.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Gavin Shan Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE, USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: David Matlack Make kvm_flush_remote_tlbs_range() visible in common code and create a default implementation that just invalidates the whole TLB. This paves the way for several future features/cleanups: - Introduction of range-based TLBI on ARM. - Eliminating kvm_arch_flush_remote_tlbs_memslot() - Moving the KVM/x86 TDP MMU to common code. No functional change intended. Signed-off-by: David Matlack Signed-off-by: Raghavendra Rao Ananta Reviewed-by: Gavin Shan --- arch/x86/include/asm/kvm_host.h | 3 +++ arch/x86/kvm/mmu/mmu.c | 9 ++++----- arch/x86/kvm/mmu/mmu_internal.h | 3 --- include/linux/kvm_host.h | 9 +++++++++ virt/kvm/kvm_main.c | 13 +++++++++++++ 5 files changed, 29 insertions(+), 8 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index a2d3cfc2eb75..08900afbf2ad 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1804,6 +1804,9 @@ static inline int kvm_arch_flush_remote_tlbs(struct kvm *kvm) return -ENOTSUPP; } +#define __KVM_HAVE_ARCH_FLUSH_REMOTE_TLBS_RANGE +int kvm_arch_flush_remote_tlbs_range(struct kvm *kvm, gfn_t start_gfn, u64 pages); + #define kvm_arch_pmi_in_guest(vcpu) \ ((vcpu) && (vcpu)->arch.handling_intr_from_guest) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index ec169f5c7dce..aaa5e336703a 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -278,16 +278,15 @@ static inline bool kvm_available_flush_remote_tlbs_range(void) return kvm_x86_ops.flush_remote_tlbs_range; } -void kvm_flush_remote_tlbs_range(struct kvm *kvm, gfn_t start_gfn, - gfn_t nr_pages) +int kvm_arch_flush_remote_tlbs_range(struct kvm *kvm, gfn_t start_gfn, u64 pages) { int ret = -EOPNOTSUPP; if (kvm_x86_ops.flush_remote_tlbs_range) ret = static_call(kvm_x86_flush_remote_tlbs_range)(kvm, start_gfn, - nr_pages); - if (ret) - kvm_flush_remote_tlbs(kvm); + pages); + + return ret; } static gfn_t kvm_mmu_page_get_gfn(struct kvm_mmu_page *sp, int index); diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_internal.h index d39af5639ce9..86cb83bb3480 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -170,9 +170,6 @@ bool kvm_mmu_slot_gfn_write_protect(struct kvm *kvm, struct kvm_memory_slot *slot, u64 gfn, int min_level); -void kvm_flush_remote_tlbs_range(struct kvm *kvm, gfn_t start_gfn, - gfn_t nr_pages); - /* Flush the given page (huge or not) of guest memory. */ static inline void kvm_flush_remote_tlbs_gfn(struct kvm *kvm, gfn_t gfn, int level) { diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index e3f968b38ae9..a731967b24ff 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1359,6 +1359,7 @@ int kvm_vcpu_yield_to(struct kvm_vcpu *target); void kvm_vcpu_on_spin(struct kvm_vcpu *vcpu, bool yield_to_kernel_mode); void kvm_flush_remote_tlbs(struct kvm *kvm); +void kvm_flush_remote_tlbs_range(struct kvm *kvm, gfn_t gfn, u64 pages); #ifdef KVM_ARCH_NR_OBJS_PER_MEMORY_CACHE int kvm_mmu_topup_memory_cache(struct kvm_mmu_memory_cache *mc, int min); @@ -1486,6 +1487,14 @@ static inline int kvm_arch_flush_remote_tlbs(struct kvm *kvm) } #endif +#ifndef __KVM_HAVE_ARCH_FLUSH_REMOTE_TLBS_RANGE +static inline int kvm_arch_flush_remote_tlbs_range(struct kvm *kvm, + gfn_t gfn, u64 pages) +{ + return -EOPNOTSUPP; +} +#endif + #ifdef __KVM_HAVE_ARCH_NONCOHERENT_DMA void kvm_arch_register_noncoherent_dma(struct kvm *kvm); void kvm_arch_unregister_noncoherent_dma(struct kvm *kvm); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index d6b050786155..804470fccac7 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -366,6 +366,19 @@ void kvm_flush_remote_tlbs(struct kvm *kvm) } EXPORT_SYMBOL_GPL(kvm_flush_remote_tlbs); +void kvm_flush_remote_tlbs_range(struct kvm *kvm, gfn_t gfn, u64 pages) +{ + if (!kvm_arch_flush_remote_tlbs_range(kvm, gfn, pages)) + return; + + /* + * Fall back to a flushing entire TLBs if the architecture range-based + * TLB invalidation is unsupported or can't be performed for whatever + * reason. + */ + kvm_flush_remote_tlbs(kvm); +} + static void kvm_flush_shadow_all(struct kvm *kvm) { kvm_arch_flush_shadow_all(kvm); -- 2.41.0.455.g037347b96a-goog