Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <ZFWli2/H5M8MZRiY@google.com> <diqzr0pb2zws.fsf@ackerleytng-ctop.c.googlers.com>
 <ZLGiEfJZTyl7M8mS@google.com>
In-Reply-To: <ZLGiEfJZTyl7M8mS@google.com>
From:   Vishal Annapurve <vannapurve@google.com>
Date:   Fri, 14 Jul 2023 16:09:50 -0700
Message-ID: <CAGtprH-VCqUgqK8gk40KaQZD8trXbWYk8KmA612Og1ep1Dko=Q@mail.gmail.com>
Subject: Re: Rename restrictedmem => guardedmem? (was: Re: [PATCH v10 0/9]
 KVM: mm: fd-based approach for supporting KVM)
To:     Sean Christopherson <seanjc@google.com>
Cc:     Ackerley Tng <ackerleytng@google.com>, david@redhat.com,
        chao.p.peng@linux.intel.com, pbonzini@redhat.com,
        vkuznets@redhat.com, jmattson@google.com, joro@8bytes.org,
        mail@maciej.szmigiero.name, vbabka@suse.cz,
        yu.c.zhang@linux.intel.com, kirill.shutemov@linux.intel.com,
        dhildenb@redhat.com, qperret@google.com, tabba@google.com,
        michael.roth@amd.com, wei.w.wang@intel.com, rppt@kernel.org,
        liam.merwick@oracle.com, isaku.yamahata@gmail.com,
        jarkko@kernel.org, kvm@vger.kernel.org,
        linux-kernel@vger.kernel.org, hughd@google.com, brauner@kernel.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Fri, Jul 14, 2023 at 12:29=E2=80=AFPM Sean Christopherson <seanjc@google=
.com> wrote:
> ...
> And _if_ there is a VMM that instantiates memory before KVM_CREATE_VM, IM=
O making
> the ioctl() /dev/kvm scoped would have no meaningful impact on adapting u=
serspace
> to play nice with the required ordering.  If userspace can get at /dev/kv=
m, then
> it can do KVM_CREATE_VM, because the only input to KVM_CREATE_VM is the t=
ype, i.e.
> the only dependencies for KVM_CREATE_VM should be known/resolved long bef=
ore the
> VMM knows it wants to use gmem.

I am not sure about the benefits of tying gmem creation to any given
kvm instance. I think the most important requirement here is that a
given gmem range is always tied to a single VM - This can be enforced
when memslots are bound to the gmem files.

I believe "Required ordering" is that gmem files are created first and
then supplied while creating the memslots whose gpa ranges can
generate private memory accesses.
Is there any other ordering we want to enforce here?

> ...
> Practically, I think that gives us a clean, intuitive way to handle intra=
-host
> migration.  Rather than transfer ownership of the file, instantiate a new=
 file
> for the target VM, using the gmem inode from the source VM, i.e. create a=
 hard
> link.  That'd probably require new uAPI, but I don't think that will be h=
ugely
> problematic.  KVM would need to ensure the new VM's guest_memfd can't be =
mapped
> until KVM_CAP_VM_MOVE_ENC_CONTEXT_FROM (which would also need to verify t=
he
> memslots/bindings are identical), but that should be easy enough to enfor=
ce.
>
> That way, a VM, its memslots, and its SPTEs are tied to the file, while a=
llowing
> the memory and the *contents* of memory to outlive the VM, i.e. be effect=
ively
> transfered to the new target VM.  And we'll maintain the invariant that e=
ach
> guest_memfd is bound 1:1 with a single VM.
>
> As above, that should also help us draw the line between mapping memory i=
nto a
> VM (file), and freeing/reclaiming the memory (inode).
>
> There will be extra complexity/overhead as we'll have to play nice with t=
he
> possibility of multiple files per inode, e.g. to zap mappings across all =
files
> when punching a hole, but the extra complexity is quite small, e.g. we ca=
n use
> address_space.private_list to keep track of the guest_memfd instances ass=
ociated
> with the inode.

Are we talking about a different usecase of sharing gmem fd across VMs
other than intra-host migration?
If not, ideally only one of the files should be catering to the guest
memory mappings at any given time. i.e. any inode should be ideally
bound to (through the file) a single kvm instance, as we we are
planning to ensure that guest_memfd can't be mapped until
KVM_CAP_VM_MOVE_ENC_CONTEXT_FROM is invoked on the target side.