MIME-Version: 1.0
References: <20230131145310.2069-1-longpeng2@huawei.com> <db99245c-606a-2f24-52fe-836a6972437f@redhat.com>
 <35b94992-0c6b-a190-1fce-5dda9c8dcf4b@huawei.com>
In-Reply-To: <35b94992-0c6b-a190-1fce-5dda9c8dcf4b@huawei.com>
From:   Jason Wang <jasowang@redhat.com>
Date:   Wed, 15 Feb 2023 10:00:32 +0800
Message-ID: <CACGkMEt0Rgkcmt9k4dWsp-qqtPvrM40mtgmSERc0A7Ve1wzKHw@mail.gmail.com>
Subject: Re: [PATCH] vhost-vdpa: cleanup memory maps when closing vdpa fds
To:     "Longpeng (Mike, Cloud Infrastructure Service Product Dept.)" 
        <longpeng2@huawei.com>
Cc:     mst@redhat.com, arei.gonglei@huawei.com, yechuan@huawei.com,
        huangzhichao@huawei.com, virtualization@lists.linux-foundation.org,
        linux-kernel@vger.kernel.org, kvm@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Tue, Feb 14, 2023 at 2:28 PM Longpeng (Mike, Cloud Infrastructure
Service Product Dept.) <longpeng2@huawei.com> wrote:
>
>
>
> =E5=9C=A8 2023/2/14 14:16, Jason Wang =E5=86=99=E9=81=93:
> >
> > =E5=9C=A8 2023/1/31 22:53, Longpeng(Mike) =E5=86=99=E9=81=93:
> >> From: Longpeng <longpeng2@huawei.com>
> >>
> >> We must cleanup all memory maps when closing the vdpa fds, otherwise
> >> some critical resources (e.g. memory, iommu map) will leaked if the
> >> userspace exits unexpectedly (e.g. kill -9).
> >
> >
> > Sounds like a bug of the kernel, should we fix there?
> >
>
> For example, the iommu map is setup when QEMU calls VHOST_IOTLB_UPDATE
> ioctl and it'll be freed if QEMU calls VHOST_IOTLB_INVALIDATE ioctl.
>
> So maybe we release these resources in vdpa framework in kernel is a
> suitable choice?

I think I need understand what does "resources" mean here:

For iommu mapping, it should be freed by vhost_vdpa_free_domain() in
vhost_vdpa_release()?

static int vhost_vdpa_release(struct inode *inode, struct file *filep)
{
        struct vhost_vdpa *v =3D filep->private_data;
        struct vhost_dev *d =3D &v->vdev;

        mutex_lock(&d->mutex);
        filep->private_data =3D NULL;
        vhost_vdpa_clean_irq(v);
        vhost_vdpa_reset(v);
        vhost_dev_stop(&v->vdev);
        vhost_vdpa_free_domain(v);
        vhost_vdpa_config_put(v);
        vhost_vdpa_cleanup(v);
        mutex_unlock(&d->mutex);

        atomic_dec(&v->opened);
        complete(&v->completion);

        return 0;
}

>
> By the way, Jason, can you reproduce the problem in your machine?
>

Haven't got time in doing this but it should be the responsibility of
the author to validate this anyhow.

Thanks

> > Thanks
> >
> >
> >>
> >> Signed-off-by: Longpeng <longpeng2@huawei.com>
> >> ---
> >>   drivers/vhost/vdpa.c | 13 +++++++++++++
> >>   1 file changed, 13 insertions(+)
> >>
> >> diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
> >> index a527eeeac637..37477cffa5aa 100644
> >> --- a/drivers/vhost/vdpa.c
> >> +++ b/drivers/vhost/vdpa.c
> >> @@ -823,6 +823,18 @@ static void vhost_vdpa_unmap(struct vhost_vdpa *v=
,
> >>           vhost_vdpa_remove_as(v, asid);
> >>   }
> >> +static void vhost_vdpa_clean_map(struct vhost_vdpa *v)
> >> +{
> >> +    struct vhost_vdpa_as *as;
> >> +    u32 asid;
> >> +
> >> +    for (asid =3D 0; asid < v->vdpa->nas; asid++) {
> >> +        as =3D asid_to_as(v, asid);
> >> +        if (as)
> >> +            vhost_vdpa_unmap(v, &as->iotlb, 0ULL, 0ULL - 1);
> >> +    }
> >> +}
> >> +
> >>   static int vhost_vdpa_va_map(struct vhost_vdpa *v,
> >>                    struct vhost_iotlb *iotlb,
> >>                    u64 iova, u64 size, u64 uaddr, u32 perm)
> >> @@ -1247,6 +1259,7 @@ static int vhost_vdpa_release(struct inode
> >> *inode, struct file *filep)
> >>       vhost_vdpa_clean_irq(v);
> >>       vhost_vdpa_reset(v);
> >>       vhost_dev_stop(&v->vdev);
> >> +    vhost_vdpa_clean_map(v);
> >>       vhost_vdpa_free_domain(v);
> >>       vhost_vdpa_config_put(v);
> >>       vhost_vdpa_cleanup(v);
> >
> > .
>