Received: by 2002:a05:6358:a55:b0:ec:fcf4:3ecf with SMTP id 21csp1645148rwb; Fri, 13 Jan 2023 15:35:34 -0800 (PST) X-Google-Smtp-Source: AMrXdXtWkRlV3xx2c4D+gQa8sm0Q2iDKV7Kjjdyin/EpSZwpjYMs0+ZX2v06Nc4Ek2MhP+IR+dMx X-Received: by 2002:a17:907:6e16:b0:7c1:b64:e290 with SMTP id sd22-20020a1709076e1600b007c10b64e290mr121993652ejc.45.1673652934061; Fri, 13 Jan 2023 15:35:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673652934; cv=none; d=google.com; s=arc-20160816; b=0RyHDuYn5VLWgBLZwsP+ZfqlNzwfHdJmp9jXYLDAEyrr8/ZINdWTysh39swefrwMRC xmL/+z1gtDCSgFY37g04sE8YUY9GT2HLyWVl3kRqcnMDJaMcoWddpUM8xfmiueuGvX5U bQIWJtZoSYfIq0ZvFofQAbnpjoK/rCS9zlgOTj3CD4maP1jLx3+RKvt+b3PELwaUE53p w1rm3pvITNEti5dH77fh2+mud7qAHI654LbKSIfUk3WAyYYEFAmu2H34FKxz33n8rnrw oyIxg93u4vSTyPMvb1ren1/cSN3n3korbu6W0z/gj9s9vt0aGqfSf3XqYUApF4QgYEd1 OLZg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:in-reply-to:content-disposition:mime-version :references:message-id:subject:cc:to:from:date:dkim-signature; bh=yQZCEBEaghsOR+zhl/QRIXPVX+uCKPvU91VQuy/V94s=; b=1AO9us0Qcyj9N8Mmuac5Jasg/XfqzJdmmTjPTjHVJXQIj+jvhCMkQFSN2ln4eSWZFS +XeVZZidh/+thHUdWEsiaP5Rr13hvekZRsOJxScyiScd+Pqf8U93O/5bAwDbagb3cDRO f5uFL8dBR7deVQfLyFSGQgBGPEKF+KO+wpj2aHGnLWdppQjXGV17/uA6EYgMRFaheiKD i0glHvdJLnyWKMb5YTgv49/PzLQn1xfgkdh52AUZahA5MOTFZEW4Y9U//3Rjotz+48PL LeGKxe3FIGNIEtDm2oVBX73TN73ORiuyJtkTLQwbq2gSrLpwLzkv13SMJXekNoFmCi9U C3MQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=X76rLWdZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id gb27-20020a170907961b00b0084c7e7eb6fdsi24862829ejc.131.2023.01.13.15.35.20; Fri, 13 Jan 2023 15:35:34 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=X76rLWdZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231350AbjAMWuc (ORCPT + 52 others); Fri, 13 Jan 2023 17:50:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46480 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230144AbjAMWu3 (ORCPT ); Fri, 13 Jan 2023 17:50:29 -0500 Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F1FAB7DE02 for ; Fri, 13 Jan 2023 14:50:27 -0800 (PST) Received: by mail-pj1-x102b.google.com with SMTP id bj3so20601602pjb.0 for ; Fri, 13 Jan 2023 14:50:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=yQZCEBEaghsOR+zhl/QRIXPVX+uCKPvU91VQuy/V94s=; b=X76rLWdZrm8cdm781bGE/NuVpMIrvPOi9lGhvncTfFs81VVxsQB/QGGTWKxHh1GL1W d/AoRzt/BgZEnefw0frAVABgAhcfb0NjbvQQ6PThCX1YXedTBAqOobICTThGoiP6om5q O3KMRfb6M32Bg7CVQSQvIy3h9U9bkySatLBmzLp/OFGTsGTVJq+w1gHMbDBjfBgq7rHh ADYr+PPp2TF3nGbuwOKFE5/4euBQA3ZQaQSM4gqqLZGzGNJeHa+tfxiLoUS2z+t4YmPi kKxrLU/4iehY+TcpZNW1WLdBUrSh7cY4mTcXmXq7RCEGkbTb53lk/EI3QzzxovpqZqg9 0htA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=yQZCEBEaghsOR+zhl/QRIXPVX+uCKPvU91VQuy/V94s=; b=A4VZ5y3tJlBrzeNP8ZAq3kA6CHu29Aynhs8WFWbSNABetsvK3ANyNjPbPTVkpmh4SV l+QWyclIrtdsCW2vgMBk/NNTonP8yLURO+3x6IaghOHtE+eerTx/UDESM0AshMvufgQ3 yFhZdV4tNtrFcBxH21uhNCW7MPbCFLpGmhegc3SzV215+Yk/Akm4axXygmmQ4VCSRdpz fY0cSloKO4j4acF9Ag/+BeG8re5amjSODwWP7U6EhN4B/NlGh8G7kM8rt43tXp45RjKG iKnuhnL5X6FHCF8Ny1EytbGKVLhfi0SytE1it1syuujptN6kFh02GP05xyI5dCobbEUO L3Jg== X-Gm-Message-State: AFqh2koIhyuvBb/Xao/MXX7/xkW424XdhfUniYzJ2FBP9OsakIEnRBkL cEq5KtWuc10F/sDDMQnCHpMNmQ== X-Received: by 2002:a17:902:b10e:b0:191:4367:7fde with SMTP id q14-20020a170902b10e00b0019143677fdemr1360463plr.0.1673650227268; Fri, 13 Jan 2023 14:50:27 -0800 (PST) Received: from google.com (7.104.168.34.bc.googleusercontent.com. [34.168.104.7]) by smtp.gmail.com with ESMTPSA id 13-20020a170902c24d00b0019460ac7c6asm3819704plg.283.2023.01.13.14.50.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Jan 2023 14:50:26 -0800 (PST) Date: Fri, 13 Jan 2023 22:50:22 +0000 From: Sean Christopherson To: Chao Peng Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, linux-doc@vger.kernel.org, qemu-devel@nongnu.org, Paolo Bonzini , Jonathan Corbet , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Arnd Bergmann , Naoya Horiguchi , Miaohe Lin , x86@kernel.org, "H . Peter Anvin" , Hugh Dickins , Jeff Layton , "J . Bruce Fields" , Andrew Morton , Shuah Khan , Mike Rapoport , Steven Price , "Maciej S . Szmigiero" , Vlastimil Babka , Vishal Annapurve , Yu Zhang , "Kirill A . Shutemov" , luto@kernel.org, jun.nakajima@intel.com, dave.hansen@intel.com, ak@linux.intel.com, david@redhat.com, aarcange@redhat.com, ddutile@redhat.com, dhildenb@redhat.com, Quentin Perret , tabba@google.com, Michael Roth , mhocko@suse.com, wei.w.wang@intel.com Subject: Re: [PATCH v10 6/9] KVM: Unmap existing mappings when change the memory attributes Message-ID: References: <20221202061347.1070246-1-chao.p.peng@linux.intel.com> <20221202061347.1070246-7-chao.p.peng@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20221202061347.1070246-7-chao.p.peng@linux.intel.com> X-Spam-Status: No, score=-17.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS, USER_IN_DEF_DKIM_WL,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Dec 02, 2022, Chao Peng wrote: > @@ -785,11 +786,12 @@ struct kvm { > > #if defined(CONFIG_MMU_NOTIFIER) && defined(KVM_ARCH_WANT_MMU_NOTIFIER) > struct mmu_notifier mmu_notifier; > +#endif > unsigned long mmu_invalidate_seq; > long mmu_invalidate_in_progress; > gfn_t mmu_invalidate_range_start; > gfn_t mmu_invalidate_range_end; > -#endif Blech. The existing code is a bit ugly, and trying to extend for this use case makes things even worse. Rather than use the base MMU_NOTIFIER Kconfig and an arbitrary define, I think we should first add a proper Kconfig, e.g. KVM_GENERIC_MMU_NOTIFIER, to replace the combination. E.g config KVM_GENERIC_MMU_NOTIFIER select MMU_NOTIFIER bool and then all architectures that currently #define KVM_ARCH_WANT_MMU_NOTIFIER can simply select the Kconfig, which is everything except s390. "GENERIC" again because s390 does select MMU_NOTIFER and actually registers its own notifier for s390's version of protected VMs (at least, I think that's what its "pv" stands for). And then later down the line in this series, when the attributes and private mem needs to tie into the notifiers, we can do: config KVM_GENERIC_MEMORY_ATTRIBUTES select KVM_GENERIC_MMU_NOTIFIER bool I.e. that way this patch doesn't need to partially expose KVM's notifier stuff and can instead just keep the soon-to-be-existing KVM_GENERIC_MMU_NOTIFIER. Taking a depending on KVM_GENERIC_MMU_NOTIFIER for KVM_GENERIC_MEMORY_ATTRIBUTES makes sense, because AFAICT, changing any type of attribute, e.g. RWX bits, is going to necessitate unmapping the affected gfn range. > struct list_head devices; > u64 manual_dirty_log_protect; > struct dentry *debugfs_dentry; > @@ -1480,6 +1482,7 @@ bool kvm_arch_dy_has_pending_interrupt(struct kvm_vcpu *vcpu); > int kvm_arch_post_init_vm(struct kvm *kvm); > void kvm_arch_pre_destroy_vm(struct kvm *kvm); > int kvm_arch_create_vm_debugfs(struct kvm *kvm); > +bool kvm_arch_has_private_mem(struct kvm *kvm); The reference to private memory belongs in a later patch. More below. > +static void kvm_unmap_mem_range(struct kvm *kvm, gfn_t start, gfn_t end) > +{ > + struct kvm_gfn_range gfn_range; > + struct kvm_memory_slot *slot; > + struct kvm_memslots *slots; > + struct kvm_memslot_iter iter; > + int i; > + int r = 0; The return from kvm_unmap_gfn_range() is a bool, this should be: bool flush = false; > + > + gfn_range.pte = __pte(0); > + gfn_range.may_block = true; > + > + for (i = 0; i < KVM_ADDRESS_SPACE_NUM; i++) { > + slots = __kvm_memslots(kvm, i); > + > + kvm_for_each_memslot_in_gfn_range(&iter, slots, start, end) { > + slot = iter.slot; > + gfn_range.start = max(start, slot->base_gfn); > + gfn_range.end = min(end, slot->base_gfn + slot->npages); > + if (gfn_range.start >= gfn_range.end) > + continue; > + gfn_range.slot = slot; > + > + r |= kvm_unmap_gfn_range(kvm, &gfn_range); > + } > + } > + > + if (r) > + kvm_flush_remote_tlbs(kvm); > +} > + > static int kvm_vm_ioctl_set_mem_attributes(struct kvm *kvm, > struct kvm_memory_attributes *attrs) > { > gfn_t start, end; > unsigned long i; > void *entry; > + int idx; > u64 supported_attrs = kvm_supported_mem_attributes(kvm); > > - /* flags is currently not used. */ > + /* 'flags' is currently not used. */ Kind of a spurious change. > if (attrs->flags) > return -EINVAL; > if (attrs->attributes & ~supported_attrs) > @@ -2372,6 +2409,13 @@ static int kvm_vm_ioctl_set_mem_attributes(struct kvm *kvm, > > entry = attrs->attributes ? xa_mk_value(attrs->attributes) : NULL; > > + if (kvm_arch_has_private_mem(kvm)) { I think we should assume that any future attributes will necessitate unmapping and invalidation, i.e. drop the private mem check. That allows introducing kvm_arch_has_private_mem() in a later patch that is more directly related to private memory. > + KVM_MMU_LOCK(kvm); > + kvm_mmu_invalidate_begin(kvm); > + kvm_mmu_invalidate_range_add(kvm, start, end); > + KVM_MMU_UNLOCK(kvm); > + } > + > mutex_lock(&kvm->lock); > for (i = start; i < end; i++) > if (xa_err(xa_store(&kvm->mem_attr_array, i, entry, > @@ -2379,6 +2423,16 @@ static int kvm_vm_ioctl_set_mem_attributes(struct kvm *kvm, > break; > mutex_unlock(&kvm->lock); > > + if (kvm_arch_has_private_mem(kvm)) { > + idx = srcu_read_lock(&kvm->srcu); Mostly for reference, this goes away if slots_lock is used instead of kvm->lock. > + KVM_MMU_LOCK(kvm); > + if (i > start) > + kvm_unmap_mem_range(kvm, start, i); > + kvm_mmu_invalidate_end(kvm); > + KVM_MMU_UNLOCK(kvm); > + srcu_read_unlock(&kvm->srcu, idx); > + } > + > attrs->address = i << PAGE_SHIFT; > attrs->size = (end - i) << PAGE_SHIFT; > > -- > 2.25.1 >