2023-11-03 12:27:08

by Steven Sistare

Subject: [PATCH V3] vdpa/mlx5: preserve CVQ vringh index

mlx5_vdpa does not preserve userland's view of vring base for the control
queue in the following sequence:

ioctl VHOST_SET_VRING_BASE
ioctl VHOST_VDPA_SET_STATUS VIRTIO_CONFIG_S_DRIVER_OK
mlx5_vdpa_set_status()
setup_cvq_vring()
vringh_init_iotlb()
vringh_init_kern()
vrh->last_avail_idx = 0;
ioctl VHOST_GET_VRING_BASE

To fix, restore the value of cvq->vring.last_avail_idx after calling
vringh_init_iotlb.

Fixes: 5262912ef3cf ("vdpa/mlx5: Add support for control VQ and MAC setting")

Signed-off-by: Steve Sistare <[email protected]>
Acked-by: Eugenio Pérez <[email protected]>
Acked-by: Jason Wang <[email protected]>
---
drivers/vdpa/mlx5/net/mlx5_vnet.c | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c
index 946488b8989f..ca972af3c89a 100644
--- a/drivers/vdpa/mlx5/net/mlx5_vnet.c
+++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c
@@ -2795,13 +2795,18 @@ static int setup_cvq_vring(struct mlx5_vdpa_dev *mvdev)
struct mlx5_control_vq *cvq = &mvdev->cvq;
int err = 0;

- if (mvdev->actual_features & BIT_ULL(VIRTIO_NET_F_CTRL_VQ))
+ if (mvdev->actual_features & BIT_ULL(VIRTIO_NET_F_CTRL_VQ)) {
+ u16 idx = cvq->vring.last_avail_idx;
+
err = vringh_init_iotlb(&cvq->vring, mvdev->actual_features,
MLX5_CVQ_MAX_ENT, false,
(struct vring_desc *)(uintptr_t)cvq->desc_addr,
(struct vring_avail *)(uintptr_t)cvq->driver_addr,
(struct vring_used *)(uintptr_t)cvq->device_addr);

+ if (!err)
+ cvq->vring.last_avail_idx = cvq->vring.last_used_idx = idx;
+ }
return err;
}

--
2.39.3

2023-11-03 12:28:18

by Steven Sistare

Subject: Re: [PATCH V3] vdpa/mlx5: preserve CVQ vringh index

In V3 I added the fixes tag and the acks. No other changes.

- Steve

On 11/3/2023 8:26 AM, Steve Sistare wrote:
> mlx5_vdpa does not preserve userland's view of vring base for the control
> queue in the following sequence:
>
> ioctl VHOST_SET_VRING_BASE
> ioctl VHOST_VDPA_SET_STATUS VIRTIO_CONFIG_S_DRIVER_OK
> mlx5_vdpa_set_status()
> setup_cvq_vring()
> vringh_init_iotlb()
> vringh_init_kern()
> vrh->last_avail_idx = 0;
> ioctl VHOST_GET_VRING_BASE
>
> To fix, restore the value of cvq->vring.last_avail_idx after calling
> vringh_init_iotlb.
>
> Fixes: 5262912ef3cf ("vdpa/mlx5: Add support for control VQ and MAC setting")
>
> Signed-off-by: Steve Sistare <[email protected]>
> Acked-by: Eugenio Pérez <[email protected]>
> Acked-by: Jason Wang <[email protected]>
> ---
> drivers/vdpa/mlx5/net/mlx5_vnet.c | 7 ++++++-
> 1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c
> index 946488b8989f..ca972af3c89a 100644
> --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c
> +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c
> @@ -2795,13 +2795,18 @@ static int setup_cvq_vring(struct mlx5_vdpa_dev *mvdev)
> struct mlx5_control_vq *cvq = &mvdev->cvq;
> int err = 0;
>
> - if (mvdev->actual_features & BIT_ULL(VIRTIO_NET_F_CTRL_VQ))
> + if (mvdev->actual_features & BIT_ULL(VIRTIO_NET_F_CTRL_VQ)) {
> + u16 idx = cvq->vring.last_avail_idx;
> +
> err = vringh_init_iotlb(&cvq->vring, mvdev->actual_features,
> MLX5_CVQ_MAX_ENT, false,
> (struct vring_desc *)(uintptr_t)cvq->desc_addr,
> (struct vring_avail *)(uintptr_t)cvq->driver_addr,
> (struct vring_used *)(uintptr_t)cvq->device_addr);
>
> + if (!err)
> + cvq->vring.last_avail_idx = cvq->vring.last_used_idx = idx;
> + }
> return err;
> }
>

2023-12-04 13:34:12

by Steven Sistare

Subject: Re: [PATCH V3] vdpa/mlx5: preserve CVQ vringh index

What's the decision here, pull it, or do not fix it?

- Steve

On 11/3/2023 8:26 AM, Steve Sistare wrote:
> mlx5_vdpa does not preserve userland's view of vring base for the control
> queue in the following sequence:
>
> ioctl VHOST_SET_VRING_BASE
> ioctl VHOST_VDPA_SET_STATUS VIRTIO_CONFIG_S_DRIVER_OK
> mlx5_vdpa_set_status()
> setup_cvq_vring()
> vringh_init_iotlb()
> vringh_init_kern()
> vrh->last_avail_idx = 0;
> ioctl VHOST_GET_VRING_BASE
>
> To fix, restore the value of cvq->vring.last_avail_idx after calling
> vringh_init_iotlb.
>
> Fixes: 5262912ef3cf ("vdpa/mlx5: Add support for control VQ and MAC setting")
>
> Signed-off-by: Steve Sistare <[email protected]>
> Acked-by: Eugenio Pérez <[email protected]>
> Acked-by: Jason Wang <[email protected]>
> ---
> drivers/vdpa/mlx5/net/mlx5_vnet.c | 7 ++++++-
> 1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c
> index 946488b8989f..ca972af3c89a 100644
> --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c
> +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c
> @@ -2795,13 +2795,18 @@ static int setup_cvq_vring(struct mlx5_vdpa_dev *mvdev)
> struct mlx5_control_vq *cvq = &mvdev->cvq;
> int err = 0;
>
> - if (mvdev->actual_features & BIT_ULL(VIRTIO_NET_F_CTRL_VQ))
> + if (mvdev->actual_features & BIT_ULL(VIRTIO_NET_F_CTRL_VQ)) {
> + u16 idx = cvq->vring.last_avail_idx;
> +
> err = vringh_init_iotlb(&cvq->vring, mvdev->actual_features,
> MLX5_CVQ_MAX_ENT, false,
> (struct vring_desc *)(uintptr_t)cvq->desc_addr,
> (struct vring_avail *)(uintptr_t)cvq->driver_addr,
> (struct vring_used *)(uintptr_t)cvq->device_addr);
>
> + if (!err)
> + cvq->vring.last_avail_idx = cvq->vring.last_used_idx = idx;
> + }
> return err;
> }
>

2023-12-04 13:36:10

by Michael S. Tsirkin

Subject: Re: [PATCH V3] vdpa/mlx5: preserve CVQ vringh index

It's in my tee and I'm sending it to Linus.

On Mon, Dec 04, 2023 at 08:33:22AM -0500, Steven Sistare wrote:
> What's the decision here, pull it, or do not fix it?
>
> - Steve
>
> On 11/3/2023 8:26 AM, Steve Sistare wrote:
> > mlx5_vdpa does not preserve userland's view of vring base for the control
> > queue in the following sequence:
> >
> > ioctl VHOST_SET_VRING_BASE
> > ioctl VHOST_VDPA_SET_STATUS VIRTIO_CONFIG_S_DRIVER_OK
> > mlx5_vdpa_set_status()
> > setup_cvq_vring()
> > vringh_init_iotlb()
> > vringh_init_kern()
> > vrh->last_avail_idx = 0;
> > ioctl VHOST_GET_VRING_BASE
> >
> > To fix, restore the value of cvq->vring.last_avail_idx after calling
> > vringh_init_iotlb.
> >
> > Fixes: 5262912ef3cf ("vdpa/mlx5: Add support for control VQ and MAC setting")
> >
> > Signed-off-by: Steve Sistare <[email protected]>
> > Acked-by: Eugenio P?rez <[email protected]>
> > Acked-by: Jason Wang <[email protected]>
> > ---
> > drivers/vdpa/mlx5/net/mlx5_vnet.c | 7 ++++++-
> > 1 file changed, 6 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c
> > index 946488b8989f..ca972af3c89a 100644
> > --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c
> > +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c
> > @@ -2795,13 +2795,18 @@ static int setup_cvq_vring(struct mlx5_vdpa_dev *mvdev)
> > struct mlx5_control_vq *cvq = &mvdev->cvq;
> > int err = 0;
> >
> > - if (mvdev->actual_features & BIT_ULL(VIRTIO_NET_F_CTRL_VQ))
> > + if (mvdev->actual_features & BIT_ULL(VIRTIO_NET_F_CTRL_VQ)) {
> > + u16 idx = cvq->vring.last_avail_idx;
> > +
> > err = vringh_init_iotlb(&cvq->vring, mvdev->actual_features,
> > MLX5_CVQ_MAX_ENT, false,
> > (struct vring_desc *)(uintptr_t)cvq->desc_addr,
> > (struct vring_avail *)(uintptr_t)cvq->driver_addr,
> > (struct vring_used *)(uintptr_t)cvq->device_addr);
> >
> > + if (!err)
> > + cvq->vring.last_avail_idx = cvq->vring.last_used_idx = idx;
> > + }
> > return err;
> > }
> >

2023-12-04 13:53:23

by Steven Sistare

Subject: Re: [PATCH V3] vdpa/mlx5: preserve CVQ vringh index

Glad to hear it, thanks - Steve

On 12/4/2023 8:35 AM, Michael S. Tsirkin wrote:
> It's in my tee and I'm sending it to Linus.
>
> On Mon, Dec 04, 2023 at 08:33:22AM -0500, Steven Sistare wrote:
>> What's the decision here, pull it, or do not fix it?
>>
>> - Steve
>>
>> On 11/3/2023 8:26 AM, Steve Sistare wrote:
>>> mlx5_vdpa does not preserve userland's view of vring base for the control
>>> queue in the following sequence:
>>>
>>> ioctl VHOST_SET_VRING_BASE
>>> ioctl VHOST_VDPA_SET_STATUS VIRTIO_CONFIG_S_DRIVER_OK
>>> mlx5_vdpa_set_status()
>>> setup_cvq_vring()
>>> vringh_init_iotlb()
>>> vringh_init_kern()
>>> vrh->last_avail_idx = 0;
>>> ioctl VHOST_GET_VRING_BASE
>>>
>>> To fix, restore the value of cvq->vring.last_avail_idx after calling
>>> vringh_init_iotlb.
>>>
>>> Fixes: 5262912ef3cf ("vdpa/mlx5: Add support for control VQ and MAC setting")
>>>
>>> Signed-off-by: Steve Sistare <[email protected]>
>>> Acked-by: Eugenio Pérez <[email protected]>
>>> Acked-by: Jason Wang <[email protected]>
>>> ---
>>> drivers/vdpa/mlx5/net/mlx5_vnet.c | 7 ++++++-
>>> 1 file changed, 6 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c
>>> index 946488b8989f..ca972af3c89a 100644
>>> --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c
>>> +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c
>>> @@ -2795,13 +2795,18 @@ static int setup_cvq_vring(struct mlx5_vdpa_dev *mvdev)
>>> struct mlx5_control_vq *cvq = &mvdev->cvq;
>>> int err = 0;
>>>
>>> - if (mvdev->actual_features & BIT_ULL(VIRTIO_NET_F_CTRL_VQ))
>>> + if (mvdev->actual_features & BIT_ULL(VIRTIO_NET_F_CTRL_VQ)) {
>>> + u16 idx = cvq->vring.last_avail_idx;
>>> +
>>> err = vringh_init_iotlb(&cvq->vring, mvdev->actual_features,
>>> MLX5_CVQ_MAX_ENT, false,
>>> (struct vring_desc *)(uintptr_t)cvq->desc_addr,
>>> (struct vring_avail *)(uintptr_t)cvq->driver_addr,
>>> (struct vring_used *)(uintptr_t)cvq->device_addr);
>>>
>>> + if (!err)
>>> + cvq->vring.last_avail_idx = cvq->vring.last_used_idx = idx;
>>> + }
>>> return err;
>>> }
>>>
>