Received: by 2002:a05:6358:a55:b0:ec:fcf4:3ecf with SMTP id 21csp6270902rwb; Wed, 18 Jan 2023 03:21:02 -0800 (PST) X-Google-Smtp-Source: AMrXdXvbgf1HH0VnkuNEpLyhcOQzKXx2NT++ttPCjfsHmFesv+o53h9uIw1rqOcyObkBiFqXC6hI X-Received: by 2002:a05:6a20:6014:b0:b6:1e7e:de4d with SMTP id r20-20020a056a20601400b000b61e7ede4dmr26794126pza.11.1674040862338; Wed, 18 Jan 2023 03:21:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674040862; cv=none; d=google.com; s=arc-20160816; b=oTLyS9eEzRR0cWBdFfPyawkSlu2uMvTdiV6zddyABFJm5oYZjVOvU2IMhyJi60yaK7 xjclEyXm41ciqVz7VI4FKCWOruMPiMEzWYyULGqctgGYrbAJkYqyITjrG90Gjf3tozt4 yHXtdJPoaqxGF5bMYVl3TojaeXa7y3YMv/wWh2FJUVXxHbEhOq44o/Q2QgIEjRCuFxd+ TD8hKSghTBZaxlfR8qwoMr24jkLT2brB0KyWnWDUgJP442Wg2U279P1HoXGOZuxhW+0e YD3R46OQa3QBwdQI0YHsF+hwYYm4Oj9K/9VA4OOCgIn0h02XZheIh802RnmZ6YBV3NtZ T4mQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:in-reply-to:content-disposition:mime-version :references:message-id:subject:cc:to:from:date:dkim-signature; bh=TRTfZL3StJ9hOSfK7f1T1cO6UgiEkghPJ6emW6oB5WM=; b=GcD5NlM7c4YB5xSkVQmkq0cKU6+OJIYZ3tW6c0+pTnMgNBXDkjzzX06HPQUZCwEmgw S2QXANFa2j/0G9NPj84kw3ZrELGKZVAhtt2D0yqvuyFnBHKxxa26p9M6ZDgOA/ZKbAY9 8DevX39CrafscKkxdXm1SVIkb5xDiMpNkPdEGDa1h7s9zMn65Y0gDr853ir16s2vbOOJ kAX95TyPZa18p+X2upl+vh/6CWAtSgsMuC4gpnk5Ed1iK1dc/lMSIPGChzLJ7h/sXleP HtQXi0g0vfA6FzOi7xVlRcRVmuUbc06K//o3hyV77XIOqiCDiO8licYDToE+S+L3jZK/ cMzA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=VsbaErPg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id g6-20020a636b06000000b00477bfab5a83si35018099pgc.2.2023.01.18.03.20.54; Wed, 18 Jan 2023 03:21:02 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=VsbaErPg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230188AbjARLKv (ORCPT + 47 others); Wed, 18 Jan 2023 06:10:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57280 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229796AbjARLKK (ORCPT ); Wed, 18 Jan 2023 06:10:10 -0500 Received: from mail-pl1-x62f.google.com (mail-pl1-x62f.google.com [IPv6:2607:f8b0:4864:20::62f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 86C3B474CC; Wed, 18 Jan 2023 02:17:27 -0800 (PST) Received: by mail-pl1-x62f.google.com with SMTP id k13so3833521plg.0; Wed, 18 Jan 2023 02:17:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=TRTfZL3StJ9hOSfK7f1T1cO6UgiEkghPJ6emW6oB5WM=; b=VsbaErPgHAtxesL07pT4eodeuGg/45NGQD74kOyycyZvMN11AFd1R05/VQA6pOEOrm +zxwrYqU4AmnxiqH+8m27DsZxy6AK2lyLEoY2L28M4Bd8yBmCYgHh6qqWFb9tkmS0Upk DFoOiBxzL8mQ3cryHK7mfGOVnB1y4dNKYc8PsriiEHN6yiVjRMiohggmLADpAupAIHf3 Yi2HIfGfi3OQhsL8QSI6wGsIOnLtWXdGoWzgG1MLLtQqob9avNYEoLlwUaOlAi98M2Ib MHSmmsmJvAhaAEurhrEnNaSQnZ90EywrUjC0wuH5ucqXTadVa/72xCkzwOOrmjaqXObR zxmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=TRTfZL3StJ9hOSfK7f1T1cO6UgiEkghPJ6emW6oB5WM=; b=rGyiVB0O0BuWnRJrsdAC6a7JUfk5vFtysVPd7Dtfwj++bfiurCIvxKL0zvKnRR58Xn +wL0tAWajyLY0iQNQXP9x+5ZUqE+G6rIva4FIW4XHiUgOCnvlExebvraLr6MOkFR7Bz2 /IqWDTvXTbvui4U+iCPsB2VDRwTl7G5yaaq6Zndf9N7TSgnu5x5q1he8TxcA/69W2K8Y Ha10jHD1Nznly30e0cPR7EOHA8WvNufiDO0jEd4uIYNdia6uvjKRnCnMUVd6EadMtslf aTbqrmxxQDRRn3MAC99cMtzhRpROXb3su1D3bGxteWLL/6OOo3Ibpt5En0uWy/F+fnlh 07+Q== X-Gm-Message-State: AFqh2kol8eVWxWhbFdHHzc8c/0unvI2W+nA0WC0PsE59s7fedacQVjOV RgsaZpZ0m7MBqcpS6GuDd6o= X-Received: by 2002:a17:902:ce82:b0:194:84ef:5f9c with SMTP id f2-20020a170902ce8200b0019484ef5f9cmr22922601plg.29.1674037046738; Wed, 18 Jan 2023 02:17:26 -0800 (PST) Received: from localhost ([192.55.54.55]) by smtp.gmail.com with ESMTPSA id o10-20020a1709026b0a00b001898ee9f723sm22814641plk.2.2023.01.18.02.17.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Jan 2023 02:17:26 -0800 (PST) Date: Wed, 18 Jan 2023 02:17:23 -0800 From: Isaku Yamahata To: Chao Peng Cc: Sean Christopherson , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, linux-doc@vger.kernel.org, qemu-devel@nongnu.org, Paolo Bonzini , Jonathan Corbet , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Arnd Bergmann , Naoya Horiguchi , Miaohe Lin , x86@kernel.org, "H . Peter Anvin" , Hugh Dickins , Jeff Layton , "J . Bruce Fields" , Andrew Morton , Shuah Khan , Mike Rapoport , Steven Price , "Maciej S . Szmigiero" , Vlastimil Babka , Vishal Annapurve , Yu Zhang , "Kirill A . Shutemov" , luto@kernel.org, jun.nakajima@intel.com, dave.hansen@intel.com, ak@linux.intel.com, david@redhat.com, aarcange@redhat.com, ddutile@redhat.com, dhildenb@redhat.com, Quentin Perret , tabba@google.com, Michael Roth , mhocko@suse.com, wei.w.wang@intel.com, isaku.yamahata@gmail.com Subject: Re: [PATCH v10 1/9] mm: Introduce memfd_restricted system call to create restricted user memory Message-ID: <20230118101723.GA2976263@ls.amr.corp.intel.com> References: <20221202061347.1070246-1-chao.p.peng@linux.intel.com> <20221202061347.1070246-2-chao.p.peng@linux.intel.com> <20230117124107.GA273037@chaop.bj.intel.com> <20230118081641.GA303785@chaop.bj.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20230118081641.GA303785@chaop.bj.intel.com> X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Jan 18, 2023 at 04:16:41PM +0800, Chao Peng wrote: > On Tue, Jan 17, 2023 at 04:34:15PM +0000, Sean Christopherson wrote: > > On Tue, Jan 17, 2023, Chao Peng wrote: > > > On Fri, Jan 13, 2023 at 09:54:41PM +0000, Sean Christopherson wrote: > > > > > + list_for_each_entry(notifier, &data->notifiers, list) { > > > > > + notifier->ops->invalidate_start(notifier, start, end); > > > > > > > > Two major design issues that we overlooked long ago: > > > > > > > > 1. Blindly invoking notifiers will not scale. E.g. if userspace configures a > > > > VM with a large number of convertible memslots that are all backed by a > > > > single large restrictedmem instance, then converting a single page will > > > > result in a linear walk through all memslots. I don't expect anyone to > > > > actually do something silly like that, but I also never expected there to be > > > > a legitimate usecase for thousands of memslots. > > > > > > > > 2. This approach fails to provide the ability for KVM to ensure a guest has > > > > exclusive access to a page. As discussed in the past, the kernel can rely > > > > on hardware (and maybe ARM's pKVM implementation?) for those guarantees, but > > > > only for SNP and TDX VMs. For VMs where userspace is trusted to some extent, > > > > e.g. SEV, there is value in ensuring a 1:1 association. > > > > > > > > And probably more importantly, relying on hardware for SNP and TDX yields a > > > > poor ABI and complicates KVM's internals. If the kernel doesn't guarantee a > > > > page is exclusive to a guest, i.e. if userspace can hand out the same page > > > > from a restrictedmem instance to multiple VMs, then failure will occur only > > > > when KVM tries to assign the page to the second VM. That will happen deep > > > > in KVM, which means KVM needs to gracefully handle such errors, and it means > > > > that KVM's ABI effectively allows plumbing garbage into its memslots. > > > > > > It may not be a valid usage, but in my TDX environment I do meet below > > > issue. > > > > > > kvm_set_user_memory AddrSpace#0 Slot#0 flags=0x4 gpa=0x0 size=0x80000000 ua=0x7fe1ebfff000 ret=0 > > > kvm_set_user_memory AddrSpace#0 Slot#1 flags=0x4 gpa=0xffc00000 size=0x400000 ua=0x7fe271579000 ret=0 > > > kvm_set_user_memory AddrSpace#0 Slot#2 flags=0x4 gpa=0xfeda0000 size=0x20000 ua=0x7fe1ec09f000 ret=-22 > > > > > > Slot#2('SMRAM') is actually an alias into system memory(Slot#0) in QEMU > > > and slot#2 fails due to below exclusive check. > > > > > > Currently I changed QEMU code to mark these alias slots as shared > > > instead of private but I'm not 100% confident this is correct fix. > > > > That's a QEMU bug of sorts. SMM is mutually exclusive with TDX, QEMU shouldn't > > be configuring SMRAM (or any SMM memslots for that matter) for TDX guests. > > Thanks for the confirmation. As long as we only bind one notifier for > each address, using xarray does make things simple. In the past, I had patches for qemu to disable PAM and SMRAM, but they were dropped for simplicity because SMRAM/PAM are disabled as reset state with unused memslot registered. TDX guest bios(TDVF or EDK2) doesn't enable them. Now we can revive them. -- Isaku Yamahata