Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   "Wang, Wei W" <wei.w.wang@intel.com>
To:     "Lutomirski, Andy" <luto@kernel.org>,
        "Christopherson,, Sean" <seanjc@google.com>,
        David Hildenbrand <david@redhat.com>
CC:     Chao Peng <chao.p.peng@linux.intel.com>,
        kvm list <kvm@vger.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>,
        Linux API <linux-api@vger.kernel.org>,
        "linux-doc@vger.kernel.org" <linux-doc@vger.kernel.org>,
        "qemu-devel@nongnu.org" <qemu-devel@nongnu.org>,
        Paolo Bonzini <pbonzini@redhat.com>,
        Jonathan Corbet <corbet@lwn.net>,
        Vitaly Kuznetsov <vkuznets@redhat.com>,
        Wanpeng Li <wanpengli@tencent.com>,
        Jim Mattson <jmattson@google.com>,
        Joerg Roedel <joro@8bytes.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        the arch/x86 maintainers <x86@kernel.org>,
        "H. Peter Anvin" <hpa@zytor.com>, Hugh Dickins <hughd@google.com>,
        Jeff Layton <jlayton@kernel.org>,
        "J . Bruce Fields" <bfields@fieldses.org>,
        "Andrew Morton" <akpm@linux-foundation.org>,
        Shuah Khan <shuah@kernel.org>,
        "Mike Rapoport" <rppt@kernel.org>,
        Steven Price <steven.price@arm.com>,
        "Maciej S . Szmigiero" <mail@maciej.szmigiero.name>,
        Vlastimil Babka <vbabka@suse.cz>,
        Vishal Annapurve <vannapurve@google.com>,
        Yu Zhang <yu.c.zhang@linux.intel.com>,
        "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>,
        "Nakajima, Jun" <jun.nakajima@intel.com>,
        "Hansen, Dave" <dave.hansen@intel.com>,
        Andi Kleen <ak@linux.intel.com>,
        "aarcange@redhat.com" <aarcange@redhat.com>,
        "ddutile@redhat.com" <ddutile@redhat.com>,
        "dhildenb@redhat.com" <dhildenb@redhat.com>,
        "Quentin Perret" <qperret@google.com>,
        Michael Roth <michael.roth@amd.com>,
        "Hocko, Michal" <mhocko@suse.com>,
        Muchun Song <songmuchun@bytedance.com>,
        "Will Deacon" <will@kernel.org>, Marc Zyngier <maz@kernel.org>,
        Fuad Tabba <tabba@google.com>
Subject: RE: [PATCH v8 1/8] mm/memfd: Introduce userspace inaccessible memfd
Thread-Topic: [PATCH v8 1/8] mm/memfd: Introduce userspace inaccessible memfd
Thread-Index: AQHYyRA+DGITW09wp06ZhgFK30MXVK3mfdYAgACm74CAA0ZcgIABDdiQ
Date:   Thu, 22 Sep 2022 13:23:42 +0000
Message-ID: <DS0PR11MB63734C0FC6A7FABCAF826620DC4E9@DS0PR11MB6373.namprd11.prod.outlook.com>
References: <20220915142913.2213336-1-chao.p.peng@linux.intel.com>
 <20220915142913.2213336-2-chao.p.peng@linux.intel.com>
 <d16284f5-3493-2892-38e6-f1fa5c10bdbb@redhat.com>
 <Yyi+l3+p9lbBAC4M@google.com>
 <84e81d21-c800-4fd5-ad7c-f20bcdd7508b@www.fastmail.com>
In-Reply-To: <84e81d21-c800-4fd5-ad7c-f20bcdd7508b@www.fastmail.com>
Accept-Language: en-US
Content-Language: en-US
dlp-product: dlpe-windows
dlp-reaction: no-action
dlp-version: 11.6.500.17
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?NpVmU2cw1KTvNZU93PDsRrZX5Z1sbA/sl7eQ+TV2CuxgOeITWk6moKVG/+nw?=
 =?us-ascii?Q?VVTmkns8ydZdFkuB7Cx6hhqtYskwkT2XrY5bUVLty08QnOnQq+m1QOrAPcQS?=
 =?us-ascii?Q?Q2s1fTcam+aElPDqAjnTyd8Ryop8pXK2tk8Mp0Pllam3gUdZr3KCSbsMQdQv?=
 =?us-ascii?Q?Z8mz0el1wpfoorcTPjaQlqjO0PNCb/UZKPYbJhAOdVO+GgognqeAkQy4HSYG?=
 =?us-ascii?Q?LDzz4zrSHfshyg54ke2pdK4SsSNcVUghoB1gvxzthPkA5hgBfF//aVGnx6Yh?=
 =?us-ascii?Q?Peht1DdDxpMqND8m6z5Hi+3YKNgw6G0D5E+9FgMKB/BwVaTM2bJjsputwjWL?=
 =?us-ascii?Q?zyJbueCyxVZRetMgwGmUeLsIh6mjZcWdrTd7GZxt5Nzp/XpCq1GYB3fup8HT?=
 =?us-ascii?Q?lYFig0LBPWngtQdXtb0wZZ5HCNlHMzLlF5Mkk0esDkc0ZKPVbRRv3ZDeCizr?=
 =?us-ascii?Q?3OgwVZ92gTUyZgCFKQPW7Rcv8tIrIqeG6YEzsy+uz20KSI+7N3PQZO3MtFgL?=
 =?us-ascii?Q?lzo8BkShCnGAQmLXB5L/Khssw3TTyilGX0p9tvtYlP7EJiZUwpVN9zcINPus?=
 =?us-ascii?Q?0FO5L5/XQoVNS1+I1CoZ68BW8yYl8aZNapH55Z9YB5ngSe/KXS6SWtB0PLay?=
 =?us-ascii?Q?cOKq0TxRgwAFKw8FgWXYS/vRk+u3w17o08YGZ4GV9f/gnR0X5MmvF/25z7k6?=
 =?us-ascii?Q?ikRuKgk1ti0pI4AZIug++BuQ5+Wlq/etBVyGXVqOzbCBT7yN2rqdAHjyBYZo?=
 =?us-ascii?Q?yaLcLTsxvmiWE2LmxVszgmP2RIoIWQgrtU+93uZeRQ+4Atm0d3SX8YRTObXk?=
 =?us-ascii?Q?McgMpciDnA5+x9t/9kWiShrEbEh/EashlmDlKaTqE8rJJ+rbfp1tXnrNxCrF?=
 =?us-ascii?Q?pnfBvKohz0msnaKJTYy7NhQl4Y+2xUyp4j9/sje+GsFRvcH+qROd+PYPFxF5?=
 =?us-ascii?Q?LE9NjxIVcsDo/GuV8MECrtDwLlp7EvXREwyuk+sOHiUEg+csGUmiql4rTij2?=
 =?us-ascii?Q?bR4J9PkhEZfbhVS60fBArQpMXjnqocyTmEjo58jIZyzxR2+hNpP+/HbFMISv?=
 =?us-ascii?Q?Ew3i4GRA8h2ERdmVwAeRdN0KepPwOpuQAjHT+X6GlVJbqq27oom3msgGsc6a?=
 =?us-ascii?Q?aA03ardfeJp8Hrd7Fmps242W7go/TMhef2j9PBkLwLQvK0tUvObCDxZF5TDr?=
 =?us-ascii?Q?j/cnFC6Rodn4mXb/h1gDzUR3lMn8EQ8x9/owHUkTp1+tt3CVj/oojhkD3ihO?=
 =?us-ascii?Q?9RzuvXsg2e0Xm+9SZ5Rt/NNFOwZ8YXQ8wakgaK6akvOVoMLs12Tyt/iOPbb+?=
 =?us-ascii?Q?IX1+prcgbLhIeepifUQGHUzmw9YrVCvj58aVcQEM7YOGA3DfxFAQwDBDJWch?=
 =?us-ascii?Q?LhK1RLOMFJElPMtGqjhYo7IkftYinIi1OnErKnNp8eRo2ZN0vRw1ol4ZGKMQ?=
 =?us-ascii?Q?kGoQ8ZjIdKO0ODF8ZKXrxQbJmEPVIc8p+4liXfN2cw+nBjM2O3AMZmLr5NDU?=
 =?us-ascii?Q?ZdGMAO/pEW8XRqfI8yw5MMkHe1vNeJjaclQXHfmbGJD48UphojcrQfx6+Ox3?=
 =?us-ascii?Q?QSEDsjep7sjkWboTEf0W5j/0uBKSUCY6Ui8RY3Kz?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB6373.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 963dcb76-ac18-47f3-a5d4-08da9c9dab95
X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Sep 2022 13:23:42.8543
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: r8JZVmNYKEWC4BcUOlRwBnLmmbyOyqbhtM2naGi/wFNTIU58XYyGWvLl6CTKqzMlh/xocKdyaqRKHQDrxKKDgg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB5520
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,
        SPF_NONE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Thursday, September 22, 2022 5:11 AM, Andy Lutomirski wrote:
> To: Christopherson,, Sean <seanjc@google.com>; David Hildenbrand
> <david@redhat.com>
> Cc: Chao Peng <chao.p.peng@linux.intel.com>; kvm list
> <kvm@vger.kernel.org>; Linux Kernel Mailing List
> <linux-kernel@vger.kernel.org>; linux-mm@kvack.org;
> linux-fsdevel@vger.kernel.org; Linux API <linux-api@vger.kernel.org>;
> linux-doc@vger.kernel.org; qemu-devel@nongnu.org; Paolo Bonzini
> <pbonzini@redhat.com>; Jonathan Corbet <corbet@lwn.net>; Vitaly
> Kuznetsov <vkuznets@redhat.com>; Wanpeng Li <wanpengli@tencent.com>;
> Jim Mattson <jmattson@google.com>; Joerg Roedel <joro@8bytes.org>;
> Thomas Gleixner <tglx@linutronix.de>; Ingo Molnar <mingo@redhat.com>;
> Borislav Petkov <bp@alien8.de>; the arch/x86 maintainers <x86@kernel.org>=
;
> H. Peter Anvin <hpa@zytor.com>; Hugh Dickins <hughd@google.com>; Jeff
> Layton <jlayton@kernel.org>; J . Bruce Fields <bfields@fieldses.org>; And=
rew
> Morton <akpm@linux-foundation.org>; Shuah Khan <shuah@kernel.org>;
> Mike Rapoport <rppt@kernel.org>; Steven Price <steven.price@arm.com>;
> Maciej S . Szmigiero <mail@maciej.szmigiero.name>; Vlastimil Babka
> <vbabka@suse.cz>; Vishal Annapurve <vannapurve@google.com>; Yu Zhang
> <yu.c.zhang@linux.intel.com>; Kirill A. Shutemov
> <kirill.shutemov@linux.intel.com>; Nakajima, Jun <jun.nakajima@intel.com>=
;
> Hansen, Dave <dave.hansen@intel.com>; Andi Kleen <ak@linux.intel.com>;
> aarcange@redhat.com; ddutile@redhat.com; dhildenb@redhat.com; Quentin
> Perret <qperret@google.com>; Michael Roth <michael.roth@amd.com>;
> Hocko, Michal <mhocko@suse.com>; Muchun Song
> <songmuchun@bytedance.com>; Wang, Wei W <wei.w.wang@intel.com>;
> Will Deacon <will@kernel.org>; Marc Zyngier <maz@kernel.org>; Fuad Tabba
> <tabba@google.com>
> Subject: Re: [PATCH v8 1/8] mm/memfd: Introduce userspace inaccessible
> memfd
>=20
> (please excuse any formatting disasters.  my internet went out as I was
> composing this, and i did my best to rescue it.)
>=20
> On Mon, Sep 19, 2022, at 12:10 PM, Sean Christopherson wrote:
> > +Will, Marc and Fuad (apologies if I missed other pKVM folks)
> >
> > On Mon, Sep 19, 2022, David Hildenbrand wrote:
> >> On 15.09.22 16:29, Chao Peng wrote:
> >> > From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
> >> >
> >> > KVM can use memfd-provided memory for guest memory. For normal
> >> > userspace accessible memory, KVM userspace (e.g. QEMU) mmaps the
> >> > memfd into its virtual address space and then tells KVM to use the
> >> > virtual address to setup the mapping in the secondary page table (e.=
g.
> EPT).
> >> >
> >> > With confidential computing technologies like Intel TDX, the
> >> > memfd-provided memory may be encrypted with special key for special
> >> > software domain (e.g. KVM guest) and is not expected to be directly
> >> > accessed by userspace. Precisely, userspace access to such
> >> > encrypted memory may lead to host crash so it should be prevented.
> >>
> >> Initially my thaught was that this whole inaccessible thing is TDX
> >> specific and there is no need to force that on other mechanisms.
> >> That's why I suggested to not expose this to user space but handle
> >> the notifier requirements internally.
> >>
> >> IIUC now, protected KVM has similar demands. Either access
> >> (read/write) of guest RAM would result in a fault and possibly crash
> >> the hypervisor (at least not the whole machine IIUC).
> >
> > Yep.  The missing piece for pKVM is the ability to convert from shared
> > to private while preserving the contents, e.g. to hand off a large
> > buffer (hundreds of MiB) for processing in the protected VM.  Thoughts
> > on this at the bottom.
> >
> >> > This patch introduces userspace inaccessible memfd (created with
> >> > MFD_INACCESSIBLE). Its memory is inaccessible from userspace
> >> > through ordinary MMU access (e.g. read/write/mmap) but can be
> >> > accessed via in-kernel interface so KVM can directly interact with
> >> > core-mm without the need to map the memory into KVM userspace.
> >>
> >> With secretmem we decided to not add such "concept switch" flags and
> >> instead use a dedicated syscall.
> >>
> >
> > I have no personal preference whatsoever between a flag and a
> > dedicated syscall, but a dedicated syscall does seem like it would
> > give the kernel a bit more flexibility.
>=20
> The third option is a device node, e.g. /dev/kvm_secretmem or
> /dev/kvm_tdxmem or similar.  But if we need flags or other details in the
> future, maybe this isn't ideal.
>=20
> >
> >> What about memfd_inaccessible()? Especially, sealing and hugetlb are
> >> not even supported and it might take a while to support either.
> >
> > Don't know about sealing, but hugetlb support for "inaccessible"
> > memory needs to come sooner than later.  "inaccessible" in quotes
> > because we might want to choose a less binary name, e.g.
> > "restricted"?.
> >
> > Regarding pKVM's use case, with the shim approach I believe this can
> > be done by allowing userspace mmap() the "hidden" memfd, but with a
> > ton of restrictions piled on top.
> >
> > My first thought was to make the uAPI a set of KVM ioctls so that KVM
> > could tightly tightly control usage without taking on too much
> > complexity in the kernel, but working through things, routing the
> > behavior through the shim itself might not be all that horrific.
> >
> > IIRC, we discarded the idea of allowing userspace to map the "private"
> > fd because
> > things got too complex, but with the shim it doesn't seem _that_ bad.
>=20
> What's the exact use case?  Is it just to pre-populate the memory?

Add one more use case here. For TDX live migration support, on the destinat=
ion side,
we map the private fd during migration to store the encrypted private memor=
y data sent
from source, and at the end of migration, we unmap it and make it inaccessi=
ble before
resuming the TD to run.