Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20220520172325.980884-1-eperezma@redhat.com> <20220520172325.980884-2-eperezma@redhat.com>
 <79089dc4-07c4-369b-826c-1c6e12edcaff@oracle.com>
In-Reply-To: <79089dc4-07c4-369b-826c-1c6e12edcaff@oracle.com>
From:   Eugenio Perez Martin <eperezma@redhat.com>
Date:   Mon, 23 May 2022 21:20:14 +0200
Message-ID: <CAJaqyWd3BqZfmJv+eBYOGRwNz3OhNKjvHPiFOafSjzAnRMA_tQ@mail.gmail.com>
Subject: Re: [PATCH 1/4] vdpa: Add stop operation
To:     Si-Wei Liu <si-wei.liu@oracle.com>
Cc:     virtualization <virtualization@lists.linux-foundation.org>,
        Jason Wang <jasowang@redhat.com>,
        kvm list <kvm@vger.kernel.org>,
        "Michael S. Tsirkin" <mst@redhat.com>, netdev@vger.kernel.org,
        linux-kernel@vger.kernel.org,
        Stefano Garzarella <sgarzare@redhat.com>,
        Longpeng <longpeng2@huawei.com>,
        Zhu Lingshan <lingshan.zhu@intel.com>,
        Martin Petrus Hubertus Habets <martinh@xilinx.com>,
        Harpreet Singh Anand <hanand@xilinx.com>, dinang@xilinx.com,
        Eli Cohen <elic@nvidia.com>,
        Laurent Vivier <lvivier@redhat.com>, pabloc@xilinx.com,
        "Dawar, Gautam" <gautam.dawar@amd.com>,
        Xie Yongji <xieyongji@bytedance.com>, habetsm.xilinx@gmail.com,
        Christophe JAILLET <christophe.jaillet@wanadoo.fr>,
        tanuj.kamde@amd.com, Wu Zongyong <wuzongyong@linux.alibaba.com>,
        martinpo@xilinx.com, Cindy Lu <lulu@redhat.com>,
        ecree.xilinx@gmail.com, Parav Pandit <parav@nvidia.com>,
        Dan Carpenter <dan.carpenter@oracle.com>,
        Zhang Min <zhang.min9@zte.com.cn>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Sat, May 21, 2022 at 12:13 PM Si-Wei Liu <si-wei.liu@oracle.com> wrote:
>
>
>
> On 5/20/2022 10:23 AM, Eugenio P=C3=A9rez wrote:
> > This operation is optional: It it's not implemented, backend feature bi=
t
> > will not be exposed.
> >
> > Signed-off-by: Eugenio P=C3=A9rez <eperezma@redhat.com>
> > ---
> >   include/linux/vdpa.h | 6 ++++++
> >   1 file changed, 6 insertions(+)
> >
> > diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h
> > index 15af802d41c4..ddfebc4e1e01 100644
> > --- a/include/linux/vdpa.h
> > +++ b/include/linux/vdpa.h
> > @@ -215,6 +215,11 @@ struct vdpa_map_file {
> >    * @reset:                  Reset device
> >    *                          @vdev: vdpa device
> >    *                          Returns integer: success (0) or error (< =
0)
> > + * @stop:                    Stop or resume the device (optional, but =
it must
> > + *                           be implemented if require device stop)
> > + *                           @vdev: vdpa device
> > + *                           @stop: stop (true), not stop (false)
> > + *                           Returns integer: success (0) or error (< =
0)
> Is this uAPI meant to address all use cases described in the full blown
> _F_STOP virtio spec proposal, such as:
>
> --------------%<--------------
>
> ...... the device MUST finish any in flight
> operations after the driver writes STOP.  Depending on the device, it
> can do it
> in many ways as long as the driver can recover its normal operation if it
> resumes the device without the need of resetting it:
>
> - Drain and wait for the completion of all pending requests until a
>    convenient avail descriptor. Ignore any other posterior descriptor.
> - Return a device-specific failure for these descriptors, so the driver
>    can choose to retry or to cancel them.
> - Mark them as done even if they are not, if the kind of device can
>    assume to lose them.
> --------------%<--------------
>

Right, this is totally underspecified in this series.

I'll expand on it in the next version, but that text proposed to
virtio-comment was complicated and misleading. I find better to get
the previous version description. Would the next description work?

```
After the return of ioctl, the device MUST finish any pending operations li=
ke
in flight requests. It must also preserve all the necessary state (the
virtqueue vring base plus the possible device specific states) that is requ=
ired
for restoring in the future.

In the future, we will provide features similar to VHOST_USER_GET_INFLIGHT_=
FD
so the device can save pending operations.
```

Thanks for pointing it out!


> E.g. do I assume correctly all in flight requests are flushed after
> return from this uAPI call? Or some of pending requests may be subject
> to loss or failure? How does the caller/user specify these various
> options (if there are) for device stop?
>
> BTW, it would be nice to add the corresponding support to vdpa_sim_blk
> as well to demo the stop handling. To just show it on vdpa-sim-net IMHO
> is perhaps not so convincing.
>
> -Siwei
>
> >    * @get_config_size:                Get the size of the configuration=
 space includes
> >    *                          fields that are conditional on feature bi=
ts.
> >    *                          @vdev: vdpa device
> > @@ -316,6 +321,7 @@ struct vdpa_config_ops {
> >       u8 (*get_status)(struct vdpa_device *vdev);
> >       void (*set_status)(struct vdpa_device *vdev, u8 status);
> >       int (*reset)(struct vdpa_device *vdev);
> > +     int (*stop)(struct vdpa_device *vdev, bool stop);
> >       size_t (*get_config_size)(struct vdpa_device *vdev);
> >       void (*get_config)(struct vdpa_device *vdev, unsigned int offset,
> >                          void *buf, unsigned int len);
>