Starting from an used_idx different than 0 is needed in use cases like
virtual machine migration. Not doing so and letting the caller set an
avail idx different than 0 causes destination device to try to use old
buffers that source driver already recover and are not available
anymore.
Since vdpa_sim does not support receive inflight descriptors as a
destination of a migration, let's set both avail_idx and used_idx the
same at vq start. This is how vhost-user works in a
VHOST_SET_VRING_BASE call.
Although the simple fix is to set last_used_idx at vdpasim_set_vq_state,
it would be reset at vdpasim_queue_ready. The last_avail_idx case is
fixed with commit a09f493c ("vdpa_sim: not reset state in
vdpasim_queue_ready"). Since the only option is to make it equal to
last_avail_idx, adding the only change needed here.
This was discovered and tested live migrating the vdpa_sim_net device.
Fixes: 2c53d0f64c06 ("vdpasim: vDPA device simulator")
Signed-off-by: Eugenio Pérez <[email protected]>
---
Cherry-picked from patch 2/2 of the series [1]. Differences are:
* Set the value of used_idx at vdpasim_queue_ready instead of fetching
from the guest vring like vhost-kernel.
v2: Actually update last_used_idx only at vdpasim_queue_ready.
Note that commit id present in the patch text is not in master but in
git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git.
[1] https://lkml.org/lkml/2023/1/18/1041
---
drivers/vdpa/vdpa_sim/vdpa_sim.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c
index 6a0a65814626..79ac585e40b9 100644
--- a/drivers/vdpa/vdpa_sim/vdpa_sim.c
+++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c
@@ -68,6 +68,7 @@ static void vdpasim_queue_ready(struct vdpasim *vdpasim, unsigned int idx)
(uintptr_t)vq->device_addr);
vq->vring.last_avail_idx = last_avail_idx;
+ vq->vring.last_used_idx = last_avail_idx;
vq->vring.notify = vdpasim_vq_notify;
}
--
2.31.1
On Fri, Feb 03, 2023 at 03:25:01PM +0100, Eugenio P?rez wrote:
>Starting from an used_idx different than 0 is needed in use cases like
>virtual machine migration. Not doing so and letting the caller set an
>avail idx different than 0 causes destination device to try to use old
>buffers that source driver already recover and are not available
>anymore.
>
>Since vdpa_sim does not support receive inflight descriptors as a
>destination of a migration, let's set both avail_idx and used_idx the
>same at vq start. This is how vhost-user works in a
>VHOST_SET_VRING_BASE call.
>
>Although the simple fix is to set last_used_idx at vdpasim_set_vq_state,
>it would be reset at vdpasim_queue_ready. The last_avail_idx case is
>fixed with commit a09f493c ("vdpa_sim: not reset state in
>vdpasim_queue_ready"). Since the only option is to make it equal to
>last_avail_idx, adding the only change needed here.
>
>This was discovered and tested live migrating the vdpa_sim_net device.
>
>Fixes: 2c53d0f64c06 ("vdpasim: vDPA device simulator")
>Signed-off-by: Eugenio P?rez <[email protected]>
>---
>Cherry-picked from patch 2/2 of the series [1]. Differences are:
>* Set the value of used_idx at vdpasim_queue_ready instead of fetching
> from the guest vring like vhost-kernel.
>
>v2: Actually update last_used_idx only at vdpasim_queue_ready.
>
>Note that commit id present in the patch text is not in master but in
>git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git.
>
>[1] https://lkml.org/lkml/2023/1/18/1041
>---
> drivers/vdpa/vdpa_sim/vdpa_sim.c | 1 +
> 1 file changed, 1 insertion(+)
>
>diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c
>index 6a0a65814626..79ac585e40b9 100644
>--- a/drivers/vdpa/vdpa_sim/vdpa_sim.c
>+++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c
>@@ -68,6 +68,7 @@ static void vdpasim_queue_ready(struct vdpasim *vdpasim, unsigned int idx)
> (uintptr_t)vq->device_addr);
>
> vq->vring.last_avail_idx = last_avail_idx;
>+ vq->vring.last_used_idx = last_avail_idx;
> vq->vring.notify = vdpasim_vq_notify;
> }
>
>-- 2.31.1
>
If you need to resend, I'd add a comment in the code following the
commit description.
Anyway, the patch LGTM:
Reviewed-by: Stefano Garzarella <[email protected]>
Thanks,
Stefano
On Fri, Feb 03, 2023 at 03:25:01PM +0100, Eugenio P?rez wrote:
> Starting from an used_idx different than 0 is needed in use cases like
> virtual machine migration. Not doing so and letting the caller set an
> avail idx different than 0 causes destination device to try to use old
> buffers that source driver already recover and are not available
> anymore.
>
> Since vdpa_sim does not support receive inflight descriptors as a
> destination of a migration, let's set both avail_idx and used_idx the
> same at vq start. This is how vhost-user works in a
> VHOST_SET_VRING_BASE call.
>
> Although the simple fix is to set last_used_idx at vdpasim_set_vq_state,
> it would be reset at vdpasim_queue_ready. The last_avail_idx case is
> fixed with commit a09f493c ("vdpa_sim: not reset state in
> vdpasim_queue_ready"). Since the only option is to make it equal to
> last_avail_idx, adding the only change needed here.
>
> This was discovered and tested live migrating the vdpa_sim_net device.
>
> Fixes: 2c53d0f64c06 ("vdpasim: vDPA device simulator")
> Signed-off-by: Eugenio P?rez <[email protected]>
> ---
> Cherry-picked from patch 2/2 of the series [1]. Differences are:
> * Set the value of used_idx at vdpasim_queue_ready instead of fetching
> from the guest vring like vhost-kernel.
>
> v2: Actually update last_used_idx only at vdpasim_queue_ready.
>
> Note that commit id present in the patch text is not in master but in
> git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git.
>
> [1] https://lkml.org/lkml/2023/1/18/1041
Can you post with a fixed hash please?
> ---
> drivers/vdpa/vdpa_sim/vdpa_sim.c | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c
> index 6a0a65814626..79ac585e40b9 100644
> --- a/drivers/vdpa/vdpa_sim/vdpa_sim.c
> +++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c
> @@ -68,6 +68,7 @@ static void vdpasim_queue_ready(struct vdpasim *vdpasim, unsigned int idx)
> (uintptr_t)vq->device_addr);
>
> vq->vring.last_avail_idx = last_avail_idx;
> + vq->vring.last_used_idx = last_avail_idx;
> vq->vring.notify = vdpasim_vq_notify;
> }
>
> --
> 2.31.1
On Tue, Feb 28, 2023 at 11:56 AM Stefano Garzarella <[email protected]> wrote:
>
> On Fri, Feb 03, 2023 at 03:25:01PM +0100, Eugenio Pérez wrote:
> >Starting from an used_idx different than 0 is needed in use cases like
> >virtual machine migration. Not doing so and letting the caller set an
> >avail idx different than 0 causes destination device to try to use old
> >buffers that source driver already recover and are not available
> >anymore.
> >
> >Since vdpa_sim does not support receive inflight descriptors as a
> >destination of a migration, let's set both avail_idx and used_idx the
> >same at vq start. This is how vhost-user works in a
> >VHOST_SET_VRING_BASE call.
> >
> >Although the simple fix is to set last_used_idx at vdpasim_set_vq_state,
> >it would be reset at vdpasim_queue_ready. The last_avail_idx case is
> >fixed with commit a09f493c ("vdpa_sim: not reset state in
> >vdpasim_queue_ready"). Since the only option is to make it equal to
> >last_avail_idx, adding the only change needed here.
> >
> >This was discovered and tested live migrating the vdpa_sim_net device.
> >
> >Fixes: 2c53d0f64c06 ("vdpasim: vDPA device simulator")
> >Signed-off-by: Eugenio Pérez <[email protected]>
> >---
> >Cherry-picked from patch 2/2 of the series [1]. Differences are:
> >* Set the value of used_idx at vdpasim_queue_ready instead of fetching
> > from the guest vring like vhost-kernel.
> >
> >v2: Actually update last_used_idx only at vdpasim_queue_ready.
> >
> >Note that commit id present in the patch text is not in master but in
> >git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git.
> >
> >[1] https://lkml.org/lkml/2023/1/18/1041
> >---
> > drivers/vdpa/vdpa_sim/vdpa_sim.c | 1 +
> > 1 file changed, 1 insertion(+)
> >
> >diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c
> >index 6a0a65814626..79ac585e40b9 100644
> >--- a/drivers/vdpa/vdpa_sim/vdpa_sim.c
> >+++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c
> >@@ -68,6 +68,7 @@ static void vdpasim_queue_ready(struct vdpasim *vdpasim, unsigned int idx)
> > (uintptr_t)vq->device_addr);
> >
> > vq->vring.last_avail_idx = last_avail_idx;
> >+ vq->vring.last_used_idx = last_avail_idx;
> > vq->vring.notify = vdpasim_vq_notify;
> > }
> >
> >-- 2.31.1
> >
>
> If you need to resend, I'd add a comment in the code following the
> commit description.
>
> Anyway, the patch LGTM:
>
> Reviewed-by: Stefano Garzarella <[email protected]>
>
Added in v3, thanks for the review!
> Thanks,
> Stefano
>
On Tue, Feb 28, 2023 at 1:13 PM Michael S. Tsirkin <[email protected]> wrote:
>
> On Fri, Feb 03, 2023 at 03:25:01PM +0100, Eugenio Pérez wrote:
> > Starting from an used_idx different than 0 is needed in use cases like
> > virtual machine migration. Not doing so and letting the caller set an
> > avail idx different than 0 causes destination device to try to use old
> > buffers that source driver already recover and are not available
> > anymore.
> >
> > Since vdpa_sim does not support receive inflight descriptors as a
> > destination of a migration, let's set both avail_idx and used_idx the
> > same at vq start. This is how vhost-user works in a
> > VHOST_SET_VRING_BASE call.
> >
> > Although the simple fix is to set last_used_idx at vdpasim_set_vq_state,
> > it would be reset at vdpasim_queue_ready. The last_avail_idx case is
> > fixed with commit a09f493c ("vdpa_sim: not reset state in
> > vdpasim_queue_ready"). Since the only option is to make it equal to
> > last_avail_idx, adding the only change needed here.
> >
> > This was discovered and tested live migrating the vdpa_sim_net device.
> >
> > Fixes: 2c53d0f64c06 ("vdpasim: vDPA device simulator")
> > Signed-off-by: Eugenio Pérez <[email protected]>
> > ---
> > Cherry-picked from patch 2/2 of the series [1]. Differences are:
> > * Set the value of used_idx at vdpasim_queue_ready instead of fetching
> > from the guest vring like vhost-kernel.
> >
> > v2: Actually update last_used_idx only at vdpasim_queue_ready.
> >
> > Note that commit id present in the patch text is not in master but in
> > git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git.
> >
> > [1] https://lkml.org/lkml/2023/1/18/1041
>
>
> Can you post with a fixed hash please?
>
Sent v3, please let me know if I need to do further modifications.
Thanks!
> > ---
> > drivers/vdpa/vdpa_sim/vdpa_sim.c | 1 +
> > 1 file changed, 1 insertion(+)
> >
> > diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c
> > index 6a0a65814626..79ac585e40b9 100644
> > --- a/drivers/vdpa/vdpa_sim/vdpa_sim.c
> > +++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c
> > @@ -68,6 +68,7 @@ static void vdpasim_queue_ready(struct vdpasim *vdpasim, unsigned int idx)
> > (uintptr_t)vq->device_addr);
> >
> > vq->vring.last_avail_idx = last_avail_idx;
> > + vq->vring.last_used_idx = last_avail_idx;
> > vq->vring.notify = vdpasim_vq_notify;
> > }
> >
> > --
> > 2.31.1
>