Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7;
MIME-Version: 1.0
References: <20230912130132.561193-1-dtatulea@nvidia.com> <20230912130132.561193-10-dtatulea@nvidia.com>
 <CACGkMEsC_rgnKyG3stFbc-Mz2yiKGwNUYYqG64jQbNpZBtnVvg@mail.gmail.com>
 <c42db8942523afe8bbf54815f672acd9e47cfa67.camel@nvidia.com>
 <CACGkMEvZESDuOiX_oOvMUh0YqCWYqvmD3Ve9YEJW3+RXXyvGew@mail.gmail.com> <4f9759182636f7bb3cb15bc8b6ec6afbe0d6053d.camel@nvidia.com>
In-Reply-To: <4f9759182636f7bb3cb15bc8b6ec6afbe0d6053d.camel@nvidia.com>
From:   Jason Wang <jasowang@redhat.com>
Date:   Mon, 9 Oct 2023 14:39:36 +0800
Message-ID: <CACGkMEvt5B_3i1wOs2yt0KmEYPpSMd_DRJ2==xzp9eMCwww4oQ@mail.gmail.com>
Subject: Re: [PATCH 09/16] vdpa/mlx5: Allow creation/deletion of any given mr struct
To:     Dragos Tatulea <dtatulea@nvidia.com>
Cc:     "xuanzhuo@linux.alibaba.com" <xuanzhuo@linux.alibaba.com>,
        Parav Pandit <parav@nvidia.com>,
        "virtualization@lists.linux-foundation.org" 
        <virtualization@lists.linux-foundation.org>,
        "eperezma@redhat.com" <eperezma@redhat.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "si-wei.liu@oracle.com" <si-wei.liu@oracle.com>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "mst@redhat.com" <mst@redhat.com>,
        Saeed Mahameed <saeedm@nvidia.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Sun, Oct 8, 2023 at 8:05=E2=80=AFPM Dragos Tatulea <dtatulea@nvidia.com>=
 wrote:
>
> On Sun, 2023-10-08 at 12:25 +0800, Jason Wang wrote:
> > On Tue, Sep 26, 2023 at 3:21=E2=80=AFPM Dragos Tatulea <dtatulea@nvidia=
.com> wrote:
> > >
> > > On Tue, 2023-09-26 at 12:44 +0800, Jason Wang wrote:
> > > > On Tue, Sep 12, 2023 at 9:02=E2=80=AFPM Dragos Tatulea <dtatulea@nv=
idia.com>
> > > > wrote:
> > > > >
> > > > > This patch adapts the mr creation/deletion code to be able to wor=
k with
> > > > > any given mr struct pointer. All the APIs are adapted to take an =
extra
> > > > > parameter for the mr.
> > > > >
> > > > > mlx5_vdpa_create/delete_mr doesn't need a ASID parameter anymore.=
 The
> > > > > check is done in the caller instead (mlx5_set_map).
> > > > >
> > > > > This change is needed for a followup patch which will introduce a=
n
> > > > > additional mr for the vq descriptor data.
> > > > >
> > > > > Signed-off-by: Dragos Tatulea <dtatulea@nvidia.com>
> > > > > ---
> > > >
> > > > Thinking of this decoupling I think I have a question.
> > > >
> > > > We advertise 2 address spaces and 2 groups. So we actually don't kn=
ow
> > > > for example which address spaces will be used by dvq.
> > > >
> > > > And actually we allow the user space to do something like
> > > >
> > > > set_group_asid(dvq_group, 0)
> > > > set_map(0)
> > > > set_group_asid(dvq_group, 1)
> > > > set_map(1)
> > > >
> > > > I wonder if the decoupling like this patch can work and why.
> > > >
> > > This scenario could indeed work. Especially if you look at the 13'th =
patch
> > > [0]
> > > where hw support is added. Are you wondering if this should work at a=
ll or
> > > if it
> > > should be blocked?
> >
> > It would be great if it can work with the following patches. But at
> > least for this patch, it seems not:
> >
> > For example, what happens if we switch back to group 0 for dvq?
> >
> > set_group_asid(dvq_group, 0)
> > set_map(0)
> > set_group_asid(dvq_group, 1)
> > set_map(1)
> > // here we destroy the mr created for asid 0
> > set_group_asid(dvq_group, 0)
> >
> If by destroy you mean .reset,

It's not rest. During the second map, the mr is destroyed by
mlx5_vdpa_change_map().

 I think it works: During .reset the mapping in
> ASID 0 is reset back to the DMA/pysical map (mlx5_vdpa_create_dma_mr). Am=
 I
> missing something?
>
> > Btw, if this is a new issue, I haven't checked whether or not it
> > exists before this series (if yes, we can fix on top).
>
> > >
> > > > It looks to me the most easy way is to let each AS be backed by an =
MR.
> > > > Then we don't even need to care about the dvq, cvq.
> > > That's what this patch series dowes.
> >
> > Good to know this, I will review the series.
> >
> I was planning to spin a v3 with Eugenio's suggestions. Should I wait for=
 your
> feedback before doing that?

You can post v3 and we can move the discussion there if you wish.

Thanks

>
> Thanks,
> Dragos