On Wed, Dec 16, 2020 at 02:47:57PM +0800, Jason Wang wrote:
Hi Jason,
I saw the patchset and will start reviewing it starting Dec 27. I am out
of office next week.
> Hi All:
>
> This series tries to add the support for control virtqueue in vDPA.
>
> Control virtqueue is used by networking device for accepting various
> commands from the driver. It's a must to support multiqueue and other
> configurations.
>
> When used by vhost-vDPA bus driver for VM, the control virtqueue
> should be shadowed via userspace VMM (Qemu) instead of being assigned
> directly to Guest. This is because Qemu needs to know the device state
> in order to start and stop device correctly (e.g for Live Migration).
>
> This requies to isolate the memory mapping for control virtqueue
> presented by vhost-vDPA to prevent guest from accesing it directly.
>
> To achieve this, vDPA introduce two new abstractions:
>
> - address space: identified through address space id (ASID) and a set
> of memory mapping in maintained
> - virtqueue group: the minimal set of virtqueues that must share an
> address space
>
> Device needs to advertise the following attributes to vDPA:
>
> - the number of address spaces supported in the device
> - the number of virtqueue groups supported in the device
> - the mappings from a specific virtqueue to its virtqueue groups
>
> The mappings from virtqueue to virtqueue groups is fixed and defined
> by vDPA device driver. E.g:
>
> - For the device that has hardware ASID support, it can simply
> advertise a per virtqueue virtqueue group.
> - For the device that does not have hardware ASID support, it can
> simply advertise a single virtqueue group that contains all
> virtqueues. Or if it wants a software emulated control virtqueue, it
> can advertise two virtqueue groups, one is for cvq, another is for
> the rest virtqueues.
>
> vDPA also allow to change the association between virtqueue group and
> address space. So in the case of control virtqueue, userspace
> VMM(Qemu) may use a dedicated address space for the control virtqueue
> group to isolate the memory mapping.
>
> The vhost/vhost-vDPA is also extend for the userspace to:
>
> - query the number of virtqueue groups and address spaces supported by
> the device
> - query the virtqueue group for a specific virtqueue
> - assocaite a virtqueue group with an address space
> - send ASID based IOTLB commands
>
> This will help userspace VMM(Qemu) to detect whether the control vq
> could be supported and isolate memory mappings of control virtqueue
> from the others.
>
> To demonstrate the usage, vDPA simulator is extended to support
> setting MAC address via a emulated control virtqueue.
>
> Please review.
>
> Changes since RFC:
>
> - tweak vhost uAPI documentation
> - switch to use device specific IOTLB really in patch 4
> - tweak the commit log
> - fix that ASID in vhost is claimed to be 32 actually but 16bit
> actually
> - fix use after free when using ASID with IOTLB batching requests
> - switch to use Stefano's patch for having separated iov
> - remove unused "used_as" variable
> - fix the iotlb/asid checking in vhost_vdpa_unmap()
>
> Thanks
>
> Jason Wang (20):
> vhost: move the backend feature bits to vhost_types.h
> virtio-vdpa: don't set callback if virtio doesn't need it
> vhost-vdpa: passing iotlb to IOMMU mapping helpers
> vhost-vdpa: switch to use vhost-vdpa specific IOTLB
> vdpa: add the missing comment for nvqs in struct vdpa_device
> vdpa: introduce virtqueue groups
> vdpa: multiple address spaces support
> vdpa: introduce config operations for associating ASID to a virtqueue
> group
> vhost_iotlb: split out IOTLB initialization
> vhost: support ASID in IOTLB API
> vhost-vdpa: introduce asid based IOTLB
> vhost-vdpa: introduce uAPI to get the number of virtqueue groups
> vhost-vdpa: introduce uAPI to get the number of address spaces
> vhost-vdpa: uAPI to get virtqueue group id
> vhost-vdpa: introduce uAPI to set group ASID
> vhost-vdpa: support ASID based IOTLB API
> vdpa_sim: advertise VIRTIO_NET_F_MTU
> vdpa_sim: factor out buffer completion logic
> vdpa_sim: filter destination mac address
> vdpasim: control virtqueue support
>
> Stefano Garzarella (1):
> vdpa_sim: split vdpasim_virtqueue's iov field in out_iov and in_iov
>
> drivers/vdpa/ifcvf/ifcvf_main.c | 9 +-
> drivers/vdpa/mlx5/net/mlx5_vnet.c | 11 +-
> drivers/vdpa/vdpa.c | 8 +-
> drivers/vdpa/vdpa_sim/vdpa_sim.c | 292 ++++++++++++++++++++++++------
> drivers/vhost/iotlb.c | 23 ++-
> drivers/vhost/vdpa.c | 246 ++++++++++++++++++++-----
> drivers/vhost/vhost.c | 23 ++-
> drivers/vhost/vhost.h | 4 +-
> drivers/virtio/virtio_vdpa.c | 2 +-
> include/linux/vdpa.h | 42 ++++-
> include/linux/vhost_iotlb.h | 2 +
> include/uapi/linux/vhost.h | 25 ++-
> include/uapi/linux/vhost_types.h | 10 +-
> 13 files changed, 561 insertions(+), 136 deletions(-)
>
> --
> 2.25.1
>