2023-01-12 15:24:52

by Eugenio Perez Martin

[permalink] [raw]
Subject: [RFC 1/3] vdpa/mlx5: reset iotlb at dup_iotlb

Regular memory region changes already reset cvq iotlb at set_map.
However this is not true if CVQ and data VQs are in different ASID.

Clean the CVQ iotlb every time we hit dup_iotlb.

Fixes: 8fcd20c30704 ("vdpa/mlx5: Support different address spaces for control and data")
Signed-off-by: Eugenio Pérez <[email protected]>
---
drivers/vdpa/mlx5/core/mr.c | 2 ++
1 file changed, 2 insertions(+)

diff --git a/drivers/vdpa/mlx5/core/mr.c b/drivers/vdpa/mlx5/core/mr.c
index 0a1e0b0dc37e..ae34dcac9a3f 100644
--- a/drivers/vdpa/mlx5/core/mr.c
+++ b/drivers/vdpa/mlx5/core/mr.c
@@ -456,6 +456,8 @@ static int dup_iotlb(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *src)
u64 start = 0, last = ULLONG_MAX;
int err;

+ vhost_iotlb_reset(mvdev->cvq.iotlb);
+
if (!src) {
err = vhost_iotlb_add_range(mvdev->cvq.iotlb, start, last, start, VHOST_ACCESS_RW);
return err;
--
2.31.1


2023-01-16 06:53:00

by Eli Cohen

[permalink] [raw]
Subject: RE: [RFC 1/3] vdpa/mlx5: reset iotlb at dup_iotlb

> From: Eugenio Pérez <[email protected]>
> Sent: Thursday, 12 January 2023 16:22
> To: [email protected]; Eli Cohen <[email protected]>
> Cc: [email protected]; Parav Pandit <[email protected]>;
> [email protected]; [email protected]; [email protected]
> foundation.org; [email protected]; [email protected]
> Subject: [RFC 1/3] vdpa/mlx5: reset iotlb at dup_iotlb
>
> Regular memory region changes already reset cvq iotlb at set_map.
> However this is not true if CVQ and data VQs are in different ASID.
>
> Clean the CVQ iotlb every time we hit dup_iotlb.
>
> Fixes: 8fcd20c30704 ("vdpa/mlx5: Support different address spaces for
> control and data")
> Signed-off-by: Eugenio Pérez <[email protected]>
> ---
> drivers/vdpa/mlx5/core/mr.c | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/drivers/vdpa/mlx5/core/mr.c b/drivers/vdpa/mlx5/core/mr.c
> index 0a1e0b0dc37e..ae34dcac9a3f 100644
> --- a/drivers/vdpa/mlx5/core/mr.c
> +++ b/drivers/vdpa/mlx5/core/mr.c
> @@ -456,6 +456,8 @@ static int dup_iotlb(struct mlx5_vdpa_dev *mvdev,
> struct vhost_iotlb *src)
> u64 start = 0, last = ULLONG_MAX;
> int err;
>
> + vhost_iotlb_reset(mvdev->cvq.iotlb);
> +

As far as I can see, mlx5_vdpa_destroy_mr() is called independently of the asid
and it will calls prune_iotlb() which resets the cvq iotlb. Am I missing something?

> if (!src) {
> err = vhost_iotlb_add_range(mvdev->cvq.iotlb, start, last,
> start, VHOST_ACCESS_RW);
> return err;
> --
> 2.31.1

2023-01-16 19:08:41

by Eugenio Perez Martin

[permalink] [raw]
Subject: Re: [RFC 1/3] vdpa/mlx5: reset iotlb at dup_iotlb

On Mon, Jan 16, 2023 at 7:32 AM Eli Cohen <[email protected]> wrote:
>
> > From: Eugenio Pérez <[email protected]>
> > Sent: Thursday, 12 January 2023 16:22
> > To: [email protected]; Eli Cohen <[email protected]>
> > Cc: [email protected]; Parav Pandit <[email protected]>;
> > [email protected]; [email protected]; [email protected]
> > foundation.org; [email protected]; [email protected]
> > Subject: [RFC 1/3] vdpa/mlx5: reset iotlb at dup_iotlb
> >
> > Regular memory region changes already reset cvq iotlb at set_map.
> > However this is not true if CVQ and data VQs are in different ASID.
> >
> > Clean the CVQ iotlb every time we hit dup_iotlb.
> >
> > Fixes: 8fcd20c30704 ("vdpa/mlx5: Support different address spaces for
> > control and data")
> > Signed-off-by: Eugenio Pérez <[email protected]>
> > ---
> > drivers/vdpa/mlx5/core/mr.c | 2 ++
> > 1 file changed, 2 insertions(+)
> >
> > diff --git a/drivers/vdpa/mlx5/core/mr.c b/drivers/vdpa/mlx5/core/mr.c
> > index 0a1e0b0dc37e..ae34dcac9a3f 100644
> > --- a/drivers/vdpa/mlx5/core/mr.c
> > +++ b/drivers/vdpa/mlx5/core/mr.c
> > @@ -456,6 +456,8 @@ static int dup_iotlb(struct mlx5_vdpa_dev *mvdev,
> > struct vhost_iotlb *src)
> > u64 start = 0, last = ULLONG_MAX;
> > int err;
> >
> > + vhost_iotlb_reset(mvdev->cvq.iotlb);
> > +
>
> As far as I can see, mlx5_vdpa_destroy_mr() is called independently of the asid
> and it will calls prune_iotlb() which resets the cvq iotlb. Am I missing something?
>

Sorry, my fault. It is not obvious from the patch but this prepares the code for
the next patch in the series.

With the next patch applied, prune_iotlb is not called in all
contexts. This patch may be dropped in future series.

Thanks!


> > if (!src) {
> > err = vhost_iotlb_add_range(mvdev->cvq.iotlb, start, last,
> > start, VHOST_ACCESS_RW);
> > return err;
> > --
> > 2.31.1
>