Received: by 2002:a05:6358:16cc:b0:ea:6187:17c9 with SMTP id r12csp387730rwl; Wed, 4 Jan 2023 21:37:03 -0800 (PST) X-Google-Smtp-Source: AMrXdXsExDwIWGOoLsNiTOU8r5lJAG4ljxMB+wW0hw/jkEvBLbCYynHcy4gIOpvebJldP1c9I2Y5 X-Received: by 2002:a05:6402:5515:b0:491:6ea2:e875 with SMTP id fi21-20020a056402551500b004916ea2e875mr1988621edb.35.1672897023013; Wed, 04 Jan 2023 21:37:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1672897022; cv=none; d=google.com; s=arc-20160816; b=dxr4WjtRwZ8gkSW1Hd9ZUxJOqolRpKMp3vQpUrrFb1ehg8hcLRdDvKlILgjubpS52w Fg/ytPEj7V9+F5rnRnfX0Xpsll4k/fyVBxGmw5AJUcqc98DB2WlbpH2vCSaxTNMubVSx o+HE+u6V2WGh9mI7PZiCSVaB0HPqa1zHrU8Fwoq+KfWFqXx59ApqLIOxSCea4RUIZALr 1f8MRkwEe5MgDYys1zNMCMooFgxNt4y4lSD9X/8StIRyKqcQdeBaIUU1cekor6GtpOA7 SlZ8rvGP7ptxoiCHk1CA2XkhOXNhYmcAQ2/Q4VqnwCC8+khxaiMnJ58DkPJitQ3Xdio6 e7Wg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:in-reply-to:content-disposition:mime-version :references:reply-to:message-id:subject:cc:to:from:date :dkim-signature; bh=peko7V3SwHb1JTqnjFGiMJp3YC495aXUWo+ZefkmlHo=; b=IDDng93Q43tbut5YkFJL1vezNflFLWKFzux8NLo8fRf2wxgyHHdxGdqHG+S9UTEndx YibQmsbH2lLG7JrVmFmP0r1jrIOk0d7w32H8N2qJp4jMlxZDvHGnIU5cWrVWRZoc6PKh NttlGAdg3qAPR7ISowxR2C2/h0be9VRrH9TZNGxn11J69zSiuYjBnXMKCQG3YC+JpDih QcTwRh3QtJ2a36UdS1vqf8gkpOux219gnWMa8vGcNl1/E1X1c3Vrwzessn6Un5CPAOEf u08e6GQylMw7V1mVoXEYxQb/OjbXCiuw2L6tndFUriUbTfxvL+7reNRUjFD6pErUWM3Z R1Qw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=aaTwBrl+; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id y1-20020a056402440100b0046baedff35bsi37114610eda.291.2023.01.04.21.36.50; Wed, 04 Jan 2023 21:37:02 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=aaTwBrl+; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229793AbjAEEn4 (ORCPT + 55 others); Wed, 4 Jan 2023 23:43:56 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49038 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229816AbjAEEnv (ORCPT ); Wed, 4 Jan 2023 23:43:51 -0500 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 467E2203B; Wed, 4 Jan 2023 20:43:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1672893830; x=1704429830; h=date:from:to:cc:subject:message-id:reply-to:references: mime-version:in-reply-to; bh=M2HzjY4mMzV2AyGn4ieao8qywB/GzVzqvY8BmkeNGAs=; b=aaTwBrl+UagqUDhK8DOZenV2UW6rNb4bXw6yrioMermuNtlVnDztKop0 BaKuOmuqnqStJSXIdHAP95ur8Yvv1AADHQF4Te7SPTW0SzwOXMNa5fgEA 1MTxShykT801PMXNP7bXILH7biVOnHNDgegKPwsPNfoKNh9RTva+9wo93 JFqnXr53SCk3xIMMvIr+Xbgb+3JxYstciSD6jt81iUq/DpAU2q3v8LMoH wh6SbJqW+w0U+J5O5n5C5wdpXYyTeix4f+hnhim1Vum16eRQHUtY42/iF 0MSx9f9zcKqOYIwSHdI6V2SWQlZyUBcINs849Lo78cjaFasbHOpCSSxRs Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10580"; a="319820021" X-IronPort-AV: E=Sophos;i="5.96,302,1665471600"; d="scan'208";a="319820021" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jan 2023 20:43:49 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10580"; a="687765563" X-IronPort-AV: E=Sophos;i="5.96,302,1665471600"; d="scan'208";a="687765563" Received: from chaop.bj.intel.com (HELO localhost) ([10.240.193.75]) by orsmga001.jf.intel.com with ESMTP; 04 Jan 2023 20:43:36 -0800 Date: Thu, 5 Jan 2023 12:39:23 +0800 From: Chao Peng To: Sean Christopherson Cc: "Wang, Wei W" , "Qiang, Chenyi" , "kvm@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "linux-mm@kvack.org" , "linux-fsdevel@vger.kernel.org" , "linux-arch@vger.kernel.org" , "linux-api@vger.kernel.org" , "linux-doc@vger.kernel.org" , "qemu-devel@nongnu.org" , Paolo Bonzini , Jonathan Corbet , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Arnd Bergmann , Naoya Horiguchi , Miaohe Lin , "x86@kernel.org" , "H . Peter Anvin" , Hugh Dickins , Jeff Layton , "J . Bruce Fields" , Andrew Morton , Shuah Khan , Mike Rapoport , Steven Price , "Maciej S . Szmigiero" , Vlastimil Babka , Vishal Annapurve , Yu Zhang , "Kirill A . Shutemov" , "Lutomirski, Andy" , "Nakajima, Jun" , "Hansen, Dave" , "ak@linux.intel.com" , "david@redhat.com" , "aarcange@redhat.com" , "ddutile@redhat.com" , "dhildenb@redhat.com" , Quentin Perret , "tabba@google.com" , Michael Roth , "Hocko, Michal" Subject: Re: [PATCH v10 2/9] KVM: Introduce per-page memory attributes Message-ID: <20230105043923.GC2251521@chaop.bj.intel.com> Reply-To: Chao Peng References: <20221202061347.1070246-1-chao.p.peng@linux.intel.com> <20221202061347.1070246-3-chao.p.peng@linux.intel.com> <1c9bbaa5-eea3-351e-d6a0-cfbc32115c82@intel.com> <20230103013948.GA2178318@chaop.bj.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Jan 03, 2023 at 11:06:37PM +0000, Sean Christopherson wrote: > On Tue, Jan 03, 2023, Wang, Wei W wrote: > > On Tuesday, January 3, 2023 9:40 AM, Chao Peng wrote: > > > > Because guest memory defaults to private, and now this patch stores > > > > the attributes with KVM_MEMORY_ATTRIBUTE_PRIVATE instead of > > > _SHARED, > > > > it would bring more KVM_EXIT_MEMORY_FAULT exits at the beginning of > > > > boot time. Maybe it can be optimized somehow in other places? e.g. set > > > > mem attr in advance. > > > > > > KVM defaults to 'shared' because this ioctl can also be potentially used by > > > normal VMs and 'shared' sounds a value meaningful for both normal VMs and > > > confidential VMs. > > > > Do you mean a normal VM could have pages marked private? What's the usage? > > (If all the pages are just marked shared for normal VMs, then why do we need it) > > No, there are potential use cases for per-page attribute/permissions, e.g. to > make select pages read-only, exec-only, no-exec, etc... Right, normal VMs are not likely use private/shared bit. Not sure pKVM, but perhaps not call it 'normal' VMs in this context. But since the ioctl can be used by normal VMs for other bits (read-only, exec-only, no-exec, etc), a default 'private' looks strange for them. That's why I default it to 'shared' and for confidential guest, we can issue another call to this ioctl to set all the memory to 'private' before guest booting, if default 'private' is needed for guest. Like Wei mentioned, it's also possible to make the default dependents on vm_type, but that looks awkward to me from the API definition as well as the implementation, also the vm_type has not been introduced at this time. > > > > As for more KVM_EXIT_MEMORY_FAULT exits during the > > > booting time, yes, setting all memory to 'private' for confidential VMs through > > > this ioctl in userspace before guest launch is an approach for KVM userspace to > > > 'override' the KVM default and reduce the number of implicit conversions. > > > > Most pages of a confidential VM are likely to be private pages. It seems more efficient > > (and not difficult to check vm_type) to have KVM defaults to "private" for confidential VMs > > and defaults to "shared" for normal VMs. > > If done right, the default shouldn't matter all that much for efficiency. KVM > needs to be able to effeciently track large ranges regardless of the default, > otherwise the memory overhead and the presumably cost of lookups will be painful. > E.g. converting a 1GiB chunk to shared should ideally require one entry, not 256k > entries. I agree, KVM should have the ability to track large ranges efficiently. > > Looks like that behavior was changed in v8 in response to feedback[*] that doing > xa_store_range() on a subset of an existing range (entry) would overwrite the > entire existing range (entry), not just the smaller subset. xa_store_range() does > appear to be too simplistic for this use case, but looking at __filemap_add_folio(), > splitting an existing entry isn't super complex. Yes, xa_store_range() looks a perfect match for us initially but the 'overwriting the entire entry' behavior makes it incorrect for us when storing a subset on an existing large entry. xarray lib has utilities for splitting, the hard part is merging existing entries, as you also said below. Thanks for pointing out the __filemap_add_folio() example, it does look not too complex for splitting. > > Using xa_store() for the very initial implementation is ok, and probably a good > idea since it's more obviously correct and will give us a bisection point. But > we definitely want a more performant implementation sooner than later. The hardest > part will likely be merging existing entries, but that can be done separately too, > and is probably lower priority. > > E.g. (1) use xa_store() and always track at 4KiB granularity, (2) support storing > metadata in multi-index entries, and finally (3) support merging adjacent entries > with identical values. This path looks good to me. Thanks, Chao > > [*] https://lore.kernel.org/all/CAGtprH9xyw6bt4=RBWF6-v2CSpabOCpKq5rPz+e-9co7EisoVQ@mail.gmail.com