Implement suspend operation for vdpa_sim devices, so vhost-vdpa will offer
that backend feature and userspace can effectively suspend the device.
This is a must before getting virtqueue indexes (base) for live migration,
since the device could modify them after userland gets them. There are
individual ways to perform that action for some devices
(VHOST_NET_SET_BACKEND, VHOST_VSOCK_SET_RUNNING, ...) but there was no
way to perform it for any vhost device (and, in particular, vhost-vdpa).
After a successful return of ioctl the device must not process more virtqueue
descriptors. The device can answer to read or writes of config fields as if it
were not suspended. In particular, writing to "queue_enable" with a value of 1
will not make the device start processing virtqueue buffers.
In the future, we will provide features similar to
VHOST_USER_GET_INFLIGHT_FD so the device can save pending operations.
Applied on top of vhost branch.
Comments are welcome.
v8:
* v7 but incremental from vhost instead of isolated.
v7:
* Remove ioctl leftover argument and update doc accordingly.
v6:
* Remove the resume operation, making the ioctl simpler. We can always add
another ioctl for VM_STOP/VM_RESUME operation later.
* s/stop/suspend/ to differentiate more from reset.
* Clarify scope of the suspend operation.
v5:
* s/not stop/resume/ in doc.
v4:
* Replace VHOST_STOP to VHOST_VDPA_STOP in vhost ioctl switch case too.
v3:
* s/VHOST_STOP/VHOST_VDPA_STOP/
* Add documentation and requirements of the ioctl above its definition.
v2:
* Replace raw _F_STOP with BIT_ULL(_F_STOP).
* Fix obtaining of stop ioctl arg (it was not obtained but written).
* Add stop to vdpa_sim_blk.
[1] git://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git
Eugenio Pérez (3):
vdpa: delete unreachable branch on vdpasim_suspend
vdpa: Remove wrong doc of VHOST_VDPA_SUSPEND ioctl
vhost: Remove invalid parameter of VHOST_VDPA_SUSPEND ioctl
drivers/vdpa/vdpa_sim/vdpa_sim.c | 7 -------
include/linux/vdpa.h | 2 +-
include/uapi/linux/vhost.h | 17 ++++++-----------
3 files changed, 7 insertions(+), 19 deletions(-)
--
2.31.1
On Thu, Aug 11, 2022 at 03:53:50PM +0200, Eugenio P?rez wrote:
> Implement suspend operation for vdpa_sim devices, so vhost-vdpa will offer
> that backend feature and userspace can effectively suspend the device.
>
> This is a must before getting virtqueue indexes (base) for live migration,
> since the device could modify them after userland gets them. There are
> individual ways to perform that action for some devices
> (VHOST_NET_SET_BACKEND, VHOST_VSOCK_SET_RUNNING, ...) but there was no
> way to perform it for any vhost device (and, in particular, vhost-vdpa).
>
> After a successful return of ioctl the device must not process more virtqueue
> descriptors. The device can answer to read or writes of config fields as if it
> were not suspended. In particular, writing to "queue_enable" with a value of 1
> will not make the device start processing virtqueue buffers.
>
> In the future, we will provide features similar to
> VHOST_USER_GET_INFLIGHT_FD so the device can save pending operations.
>
> Applied on top of vhost branch.
>
> Comments are welcome.
>
> v8:
> * v7 but incremental from vhost instead of isolated.
Now I'm lost. incremental to what? Does the vhost branch now
have the correct bits?
> v7:
> * Remove ioctl leftover argument and update doc accordingly.
>
> v6:
> * Remove the resume operation, making the ioctl simpler. We can always add
> another ioctl for VM_STOP/VM_RESUME operation later.
> * s/stop/suspend/ to differentiate more from reset.
> * Clarify scope of the suspend operation.
>
> v5:
> * s/not stop/resume/ in doc.
>
> v4:
> * Replace VHOST_STOP to VHOST_VDPA_STOP in vhost ioctl switch case too.
>
> v3:
> * s/VHOST_STOP/VHOST_VDPA_STOP/
> * Add documentation and requirements of the ioctl above its definition.
>
> v2:
> * Replace raw _F_STOP with BIT_ULL(_F_STOP).
> * Fix obtaining of stop ioctl arg (it was not obtained but written).
> * Add stop to vdpa_sim_blk.
>
> [1] git://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git
>
> Eugenio P?rez (3):
> vdpa: delete unreachable branch on vdpasim_suspend
> vdpa: Remove wrong doc of VHOST_VDPA_SUSPEND ioctl
> vhost: Remove invalid parameter of VHOST_VDPA_SUSPEND ioctl
>
> drivers/vdpa/vdpa_sim/vdpa_sim.c | 7 -------
> include/linux/vdpa.h | 2 +-
> include/uapi/linux/vhost.h | 17 ++++++-----------
> 3 files changed, 7 insertions(+), 19 deletions(-)
>
> --
> 2.31.1
>
It was a leftover from previous versions.
Signed-off-by: Eugenio Pérez <[email protected]>
---
drivers/vdpa/vdpa_sim/vdpa_sim.c | 7 -------
1 file changed, 7 deletions(-)
diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c
index 213883487f9b..79a50edf8998 100644
--- a/drivers/vdpa/vdpa_sim/vdpa_sim.c
+++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c
@@ -509,16 +509,9 @@ static int vdpasim_reset(struct vdpa_device *vdpa)
static int vdpasim_suspend(struct vdpa_device *vdpa)
{
struct vdpasim *vdpasim = vdpa_to_sim(vdpa);
- int i;
spin_lock(&vdpasim->lock);
vdpasim->running = false;
- if (vdpasim->running) {
- /* Check for missed buffers */
- for (i = 0; i < vdpasim->dev_attr.nvqs; ++i)
- vdpasim_kick_vq(vdpa, i);
-
- }
spin_unlock(&vdpasim->lock);
return 0;
--
2.31.1
On Thu, Aug 11, 2022 at 3:58 PM Michael S. Tsirkin <[email protected]> wrote:
>
> On Thu, Aug 11, 2022 at 03:53:50PM +0200, Eugenio Pérez wrote:
> > Implement suspend operation for vdpa_sim devices, so vhost-vdpa will offer
> > that backend feature and userspace can effectively suspend the device.
> >
> > This is a must before getting virtqueue indexes (base) for live migration,
> > since the device could modify them after userland gets them. There are
> > individual ways to perform that action for some devices
> > (VHOST_NET_SET_BACKEND, VHOST_VSOCK_SET_RUNNING, ...) but there was no
> > way to perform it for any vhost device (and, in particular, vhost-vdpa).
> >
> > After a successful return of ioctl the device must not process more virtqueue
> > descriptors. The device can answer to read or writes of config fields as if it
> > were not suspended. In particular, writing to "queue_enable" with a value of 1
> > will not make the device start processing virtqueue buffers.
> >
> > In the future, we will provide features similar to
> > VHOST_USER_GET_INFLIGHT_FD so the device can save pending operations.
> >
> > Applied on top of vhost branch.
> >
> > Comments are welcome.
> >
> > v8:
> > * v7 but incremental from vhost instead of isolated.
>
> Now I'm lost. incremental to what? Does the vhost branch now
> have the correct bits?
>
This patch is intended to be applied on top of the current vhost
branch. In particular, on the top of commit
6a9720576cd00d30722c5f755bd17d4cfa9df636.
It basically deletes the code, the doc, and the unused ioctl argument.
Did I misunderstand what you meant with "incremental" in previous mail?
> > v7:
> > * Remove ioctl leftover argument and update doc accordingly.
> >
> > v6:
> > * Remove the resume operation, making the ioctl simpler. We can always add
> > another ioctl for VM_STOP/VM_RESUME operation later.
> > * s/stop/suspend/ to differentiate more from reset.
> > * Clarify scope of the suspend operation.
> >
> > v5:
> > * s/not stop/resume/ in doc.
> >
> > v4:
> > * Replace VHOST_STOP to VHOST_VDPA_STOP in vhost ioctl switch case too.
> >
> > v3:
> > * s/VHOST_STOP/VHOST_VDPA_STOP/
> > * Add documentation and requirements of the ioctl above its definition.
> >
> > v2:
> > * Replace raw _F_STOP with BIT_ULL(_F_STOP).
> > * Fix obtaining of stop ioctl arg (it was not obtained but written).
> > * Add stop to vdpa_sim_blk.
> >
> > [1] git://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git
> >
> > Eugenio Pérez (3):
> > vdpa: delete unreachable branch on vdpasim_suspend
> > vdpa: Remove wrong doc of VHOST_VDPA_SUSPEND ioctl
> > vhost: Remove invalid parameter of VHOST_VDPA_SUSPEND ioctl
> >
> > drivers/vdpa/vdpa_sim/vdpa_sim.c | 7 -------
> > include/linux/vdpa.h | 2 +-
> > include/uapi/linux/vhost.h | 17 ++++++-----------
> > 3 files changed, 7 insertions(+), 19 deletions(-)
> >
> > --
> > 2.31.1
> >
>
On Thu, Aug 11, 2022 at 9:54 PM Eugenio Pérez <[email protected]> wrote:
>
> It was a leftover from previous versions.
>
> Signed-off-by: Eugenio Pérez <[email protected]>
Acked-by: Jason Wang <[email protected]>
> ---
> drivers/vdpa/vdpa_sim/vdpa_sim.c | 7 -------
> 1 file changed, 7 deletions(-)
>
> diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c
> index 213883487f9b..79a50edf8998 100644
> --- a/drivers/vdpa/vdpa_sim/vdpa_sim.c
> +++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c
> @@ -509,16 +509,9 @@ static int vdpasim_reset(struct vdpa_device *vdpa)
> static int vdpasim_suspend(struct vdpa_device *vdpa)
> {
> struct vdpasim *vdpasim = vdpa_to_sim(vdpa);
> - int i;
>
> spin_lock(&vdpasim->lock);
> vdpasim->running = false;
> - if (vdpasim->running) {
> - /* Check for missed buffers */
> - for (i = 0; i < vdpasim->dev_attr.nvqs; ++i)
> - vdpasim_kick_vq(vdpa, i);
> -
> - }
> spin_unlock(&vdpasim->lock);
>
> return 0;
> --
> 2.31.1
>