Received: by 2002:a5d:925a:0:0:0:0:0 with SMTP id e26csp933977iol; Thu, 9 Jun 2022 18:03:41 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxGTh0DuxW+3qIWOT12h0JsNh/0cwyuj72wkcFoLnoostMOR2eirpeNzmAAR6YlLpgPOn/8 X-Received: by 2002:a05:6402:5008:b0:42d:c421:48c8 with SMTP id p8-20020a056402500800b0042dc42148c8mr48666452eda.422.1654823020892; Thu, 09 Jun 2022 18:03:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1654823020; cv=none; d=google.com; s=arc-20160816; b=HUcwC6F68dCOVEeFJif6YJdXAi4op+u9+G89ojWowq/guXVRroMGnGRwD5mnFnuWsE YQxlGeCWkbqz4QHKwSPhbBJ8iPLnZbtbcjY9vmCvl4p+YeIPQkVcnXyRCMwyaBgragba m37YKTqcmgKRXY7hTHSUd5Vyuz5ZfDc5AGEt77GEV/7TTzTc72YuOk/8fQXbmYBK4G1m TYeJtjCugW6fJbTh7EO0FOnsF4dQ3osSFN71a3QWvZ9Irj4Jm5MBs2M3zB/N0qh7VYlN HLOAsJ94ntJ+7UBZHxuZzl0Oww4uuBtIl0f1CCbFCN+aReSsIg0ioVmDr6TSyHrZEjWD WJVQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:subject:message-id:date:from:in-reply-to :references:mime-version:dkim-signature; bh=lN+fOqwv9bBcDqIpukqHItG5mOqsK9h2lCdbXTf1cPM=; b=kIgeNekMZDrXUjluIgRx3IcCuN7jhHBxdV++5O6KSfL78cCFUcTOYtkHFd0d/Ifcig x+epKVsR5Oo7PbnWYvwpQliyE/nzd3jrQc59ino5J4zOmsBjlpSHvHj0zcap7zJxIpYF Wqii7pl/jwvQfrOAt7Fca6fz1zhzg1QLKW7ij6Ke/ClZZERASsfgRCFnX4y5DTB5wUF6 ojBQTTByRqOxRRnaMl7xH/MGCNWdTfJB/HqqGjlC3pxZv8Ei3GKy6Ib8v4KC+XJPvwJS zjbhuNy9OfcUykoRexz9hvBKzWvhdYBWXHkFD1h9pP9yX/ko/Oi5s/M79/xIw6wQGmXn HNTw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=DayT0SdB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id o7-20020a170906974700b006fee2de2112si749885ejy.668.2022.06.09.18.03.13; Thu, 09 Jun 2022 18:03:40 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=DayT0SdB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239023AbiFJALl (ORCPT + 99 others); Thu, 9 Jun 2022 20:11:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47328 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231631AbiFJALh (ORCPT ); Thu, 9 Jun 2022 20:11:37 -0400 Received: from mail-oa1-x29.google.com (mail-oa1-x29.google.com [IPv6:2001:4860:4864:20::29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2BBB34BBAC for ; Thu, 9 Jun 2022 17:11:35 -0700 (PDT) Received: by mail-oa1-x29.google.com with SMTP id 586e51a60fabf-fb1ae0cd9cso1369668fac.13 for ; Thu, 09 Jun 2022 17:11:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=lN+fOqwv9bBcDqIpukqHItG5mOqsK9h2lCdbXTf1cPM=; b=DayT0SdBt1ymTKyry+8RI8dWvFXfKXAgjmO8VhNnB1yoDZK63YuzcCUTHD3X1Z3+t5 KxmGEbccYszaa9LA1rH+uPvXkUot/k7y+eYJ5dVw1vcNDUZMV9ll/y/F+7N08dr0Wzcn baBzETzPA15Z2tn9QgoF5S7+WOA0j8RJa/fH3l+HKIsIdEIRN75ENG6RuJQTaJE4f7s+ tdQ6ePOMDHk+W48X2uZ7KmmqPYEcwu7pMK65fJXjpYRLWoV5X97j/IkcgrkGUpYpxwIB E5LIO66muSm/XASIShQU2K4+YUS/ejVNHYtSotPk8eFHXkkvvETkJhzMm8POKoBM9Wup sSZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=lN+fOqwv9bBcDqIpukqHItG5mOqsK9h2lCdbXTf1cPM=; b=vtxgtmUoZfVTSCeldghlXolDnJgNr8xAsr2aEJ6tETVxQ/EnwdP5VRZ/gqp9RhjO4K DTniRANfuVn4bUV0UXOuGh1l0J2FxFbU59M/fa1pLZYlYZEDbzjYXQPKvvdwQs4GXb0D hJU+0zkJBYNIz0/m/pOvuWWozKIsPmL4qGLo4P/e6jgTTBY7ipjCRxFp5skVW5uWgptP SMBwsWlD69alLTlKfd00+QABgJi+Pw1ae3cfJs1lNM3TDVwcgDwYHPivZ1E6V9Zr+Thn pfqbIBpCKRFC6PyVZ4eCDMfn1BJiGqAl1g02zdtbNm8IxizjbLtF1wiyfO478Md3B+YZ iXCQ== X-Gm-Message-State: AOAM531YWjPfqXGUA6jBLnpiZy4+X2Lzt0rF0Y4AXVAIIs4CErvTPdL3 JYNVTF9rJtoH8mlz3rgolEiQDpDTA9KGwRcjEUDqTA== X-Received: by 2002:a05:6870:b616:b0:e2:f8bb:5eb with SMTP id cm22-20020a056870b61600b000e2f8bb05ebmr3338490oab.218.1654819893261; Thu, 09 Jun 2022 17:11:33 -0700 (PDT) MIME-Version: 1.0 References: <20220519153713.819591-1-chao.p.peng@linux.intel.com> <20220607065749.GA1513445@chaop.bj.intel.com> <20220608021820.GA1548172@chaop.bj.intel.com> In-Reply-To: <20220608021820.GA1548172@chaop.bj.intel.com> From: Marc Orr Date: Thu, 9 Jun 2022 17:11:21 -0700 Message-ID: Subject: Re: [PATCH v6 0/8] KVM: mm: fd-based approach for supporting KVM guest private memory To: Chao Peng Cc: Vishal Annapurve , kvm list , LKML , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-api@vger.kernel.org, linux-doc@vger.kernel.org, qemu-devel@nongnu.org, Paolo Bonzini , Jonathan Corbet , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Thomas Gleixner , Ingo Molnar , Borislav Petkov , x86 , "H . Peter Anvin" , Hugh Dickins , Jeff Layton , "J . Bruce Fields" , Andrew Morton , Mike Rapoport , Steven Price , "Maciej S . Szmigiero" , Vlastimil Babka , Yu Zhang , "Kirill A . Shutemov" , Andy Lutomirski , Jun Nakajima , Dave Hansen , Andi Kleen , David Hildenbrand , aarcange@redhat.com, ddutile@redhat.com, dhildenb@redhat.com, Quentin Perret , Michael Roth , mhocko@suse.com Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-17.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL,USER_IN_DEF_SPF_WL autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Jun 7, 2022 at 7:22 PM Chao Peng wrote: > > On Tue, Jun 07, 2022 at 05:55:46PM -0700, Marc Orr wrote: > > On Tue, Jun 7, 2022 at 12:01 AM Chao Peng wrote: > > > > > > On Mon, Jun 06, 2022 at 01:09:50PM -0700, Vishal Annapurve wrote: > > > > > > > > > > Private memory map/unmap and conversion > > > > > --------------------------------------- > > > > > Userspace's map/unmap operations are done by fallocate() ioctl on the > > > > > backing store fd. > > > > > - map: default fallocate() with mode=0. > > > > > - unmap: fallocate() with FALLOC_FL_PUNCH_HOLE. > > > > > The map/unmap will trigger above memfile_notifier_ops to let KVM map/unmap > > > > > secondary MMU page tables. > > > > > > > > > .... > > > > > QEMU: https://github.com/chao-p/qemu/tree/privmem-v6 > > > > > > > > > > An example QEMU command line for TDX test: > > > > > -object tdx-guest,id=tdx \ > > > > > -object memory-backend-memfd-private,id=ram1,size=2G \ > > > > > -machine q35,kvm-type=tdx,pic=no,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 > > > > > > > > > > > > > There should be more discussion around double allocation scenarios > > > > when using the private fd approach. A malicious guest or buggy > > > > userspace VMM can cause physical memory getting allocated for both > > > > shared (memory accessible from host) and private fds backing the guest > > > > memory. > > > > Userspace VMM will need to unback the shared guest memory while > > > > handling the conversion from shared to private in order to prevent > > > > double allocation even with malicious guests or bugs in userspace VMM. > > > > > > I don't know how malicious guest can cause that. The initial design of > > > this serie is to put the private/shared memory into two different > > > address spaces and gives usersapce VMM the flexibility to convert > > > between the two. It can choose respect the guest conversion request or > > > not. > > > > For example, the guest could maliciously give a device driver a > > private page so that a host-side virtual device will blindly write the > > private page. > > With this patch series, it's actually even not possible for userspace VMM > to allocate private page by a direct write, it's basically unmapped from > there. If it really wants to, it should so something special, by intention, > that's basically the conversion, which we should allow. I think Vishal did a better job to explain this scenario in his last reply than I did. > > > It's possible for a usrspace VMM to cause double allocation if it fails > > > to call the unback operation during the conversion, this may be a bug > > > or not. Double allocation may not be a wrong thing, even in conception. > > > At least TDX allows you to use half shared half private in guest, means > > > both shared/private can be effective. Unbacking the memory is just the > > > current QEMU implementation choice. > > > > Right. But the idea is that this patch series should accommodate all > > of the CVM architectures. Or at least that's what I know was > > envisioned last time we discussed this topic for SNP [*]. > > AFAICS, this series should work for both TDX and SNP, and other CVM > architectures. I don't see where TDX can work but SNP cannot, or I > missed something here? Agreed. I was just responding to the "At least TDX..." bit. Sorry for any confusion. > > > > Regardless, it's important to ensure that the VM respects its memory > > budget. For example, within Google, we run VMs inside of containers. > > So if we double allocate we're going to OOM. This seems acceptable for > > an early version of CVMs. But ultimately, I think we need a more > > robust way to ensure that the VM operates within its memory container. > > Otherwise, the OOM is going to be hard to diagnose and distinguish > > from a real OOM. > > Thanks for bringing this up. But in my mind I still think userspace VMM > can do and it's its responsibility to guarantee that, if that is hard > required. By design, userspace VMM is the decision-maker for page > conversion and has all the necessary information to know which page is > shared/private. It also has the necessary knobs to allocate/free the > physical pages for guest memory. Definitely, we should make userspace > VMM more robust. Vishal and Sean did a better job to articulate the concern in their most recent replies.