Hi Dragos, Eugenio and Si-Wei
My name is Lei Yang, a software Quality Engineer from Red Hat. And
always paying attention to improving the live migration downtime
issues because there are others QE asked about this problem when I
share live migration status recently. Therefore I would like to test
it in my environment. Before the testing I want to know if there is an
expectation of downtime range based on this series of patches? In
addition, QE also can help do a regression test based on this series
of patches if there is a requirement.
Regards and thanks
Lei
On Tue, Sep 12, 2023 at 9:04 PM Dragos Tatulea <[email protected]> wrote:
>
> This patch series adds support for vq descriptor table mappings which
> are used to improve vdpa live migration downtime. The improvement comes
> from using smaller mappings which take less time to create and destroy
> in hw.
>
> The first part adds the vdpa core changes from Si-Wei [0].
>
> The second part adds support in mlx5_vdpa:
> - Refactor the mr code to be able to cleanly add descriptor mappings.
> - Add hardware descriptor mr support.
> - Properly update iotlb for cvq during ASID switch.
>
> [0] https://lore.kernel.org/virtualization/[email protected]
>
> Dragos Tatulea (13):
> vdpa/mlx5: Create helper function for dma mappings
> vdpa/mlx5: Decouple cvq iotlb handling from hw mapping code
> vdpa/mlx5: Take cvq iotlb lock during refresh
> vdpa/mlx5: Collapse "dvq" mr add/delete functions
> vdpa/mlx5: Rename mr destroy functions
> vdpa/mlx5: Allow creation/deletion of any given mr struct
> vdpa/mlx5: Move mr mutex out of mr struct
> vdpa/mlx5: Improve mr update flow
> vdpa/mlx5: Introduce mr for vq descriptor
> vdpa/mlx5: Enable hw support for vq descriptor mapping
> vdpa/mlx5: Make iotlb helper functions more generic
> vdpa/mlx5: Update cvq iotlb mapping on ASID change
> Cover letter: vdpa/mlx5: Add support for vq descriptor mappings
>
> Si-Wei Liu (3):
> vdpa: introduce dedicated descriptor group for virtqueue
> vhost-vdpa: introduce descriptor group backend feature
> vhost-vdpa: uAPI to get dedicated descriptor group id
>
> drivers/vdpa/mlx5/core/mlx5_vdpa.h | 31 +++--
> drivers/vdpa/mlx5/core/mr.c | 191 ++++++++++++++++-------------
> drivers/vdpa/mlx5/core/resources.c | 6 +-
> drivers/vdpa/mlx5/net/mlx5_vnet.c | 100 ++++++++++-----
> drivers/vhost/vdpa.c | 27 ++++
> include/linux/mlx5/mlx5_ifc.h | 8 +-
> include/linux/mlx5/mlx5_ifc_vdpa.h | 7 +-
> include/linux/vdpa.h | 11 ++
> include/uapi/linux/vhost.h | 8 ++
> include/uapi/linux/vhost_types.h | 5 +
> 10 files changed, 264 insertions(+), 130 deletions(-)
>
> --
> 2.41.0
>
On Wed, Sep 13, 2023 at 3:03 AM Lei Yang <[email protected]> wrote:
>
> Hi Dragos, Eugenio and Si-Wei
>
> My name is Lei Yang, a software Quality Engineer from Red Hat. And
> always paying attention to improving the live migration downtime
> issues because there are others QE asked about this problem when I
> share live migration status recently. Therefore I would like to test
> it in my environment. Before the testing I want to know if there is an
> expectation of downtime range based on this series of patches? In
> addition, QE also can help do a regression test based on this series
> of patches if there is a requirement.
>
Hi Lei,
Thanks for offering the testing bandwidth!
I think we can only do regression tests here, as the userland part is
still not sent to qemu.
> Regards and thanks
> Lei
>
>
> On Tue, Sep 12, 2023 at 9:04 PM Dragos Tatulea <[email protected]> wrote:
> >
> > This patch series adds support for vq descriptor table mappings which
> > are used to improve vdpa live migration downtime. The improvement comes
> > from using smaller mappings which take less time to create and destroy
> > in hw.
> >
> > The first part adds the vdpa core changes from Si-Wei [0].
> >
> > The second part adds support in mlx5_vdpa:
> > - Refactor the mr code to be able to cleanly add descriptor mappings.
> > - Add hardware descriptor mr support.
> > - Properly update iotlb for cvq during ASID switch.
> >
> > [0] https://lore.kernel.org/virtualization/[email protected]
> >
> > Dragos Tatulea (13):
> > vdpa/mlx5: Create helper function for dma mappings
> > vdpa/mlx5: Decouple cvq iotlb handling from hw mapping code
> > vdpa/mlx5: Take cvq iotlb lock during refresh
> > vdpa/mlx5: Collapse "dvq" mr add/delete functions
> > vdpa/mlx5: Rename mr destroy functions
> > vdpa/mlx5: Allow creation/deletion of any given mr struct
> > vdpa/mlx5: Move mr mutex out of mr struct
> > vdpa/mlx5: Improve mr update flow
> > vdpa/mlx5: Introduce mr for vq descriptor
> > vdpa/mlx5: Enable hw support for vq descriptor mapping
> > vdpa/mlx5: Make iotlb helper functions more generic
> > vdpa/mlx5: Update cvq iotlb mapping on ASID change
> > Cover letter: vdpa/mlx5: Add support for vq descriptor mappings
> >
> > Si-Wei Liu (3):
> > vdpa: introduce dedicated descriptor group for virtqueue
> > vhost-vdpa: introduce descriptor group backend feature
> > vhost-vdpa: uAPI to get dedicated descriptor group id
> >
> > drivers/vdpa/mlx5/core/mlx5_vdpa.h | 31 +++--
> > drivers/vdpa/mlx5/core/mr.c | 191 ++++++++++++++++-------------
> > drivers/vdpa/mlx5/core/resources.c | 6 +-
> > drivers/vdpa/mlx5/net/mlx5_vnet.c | 100 ++++++++++-----
> > drivers/vhost/vdpa.c | 27 ++++
> > include/linux/mlx5/mlx5_ifc.h | 8 +-
> > include/linux/mlx5/mlx5_ifc_vdpa.h | 7 +-
> > include/linux/vdpa.h | 11 ++
> > include/uapi/linux/vhost.h | 8 ++
> > include/uapi/linux/vhost_types.h | 5 +
> > 10 files changed, 264 insertions(+), 130 deletions(-)
> >
> > --
> > 2.41.0
> >
>
On 9/13/2023 9:08 AM, Eugenio Perez Martin wrote:
> On Wed, Sep 13, 2023 at 3:03 AM Lei Yang <[email protected]> wrote:
>> Hi Dragos, Eugenio and Si-Wei
>>
>> My name is Lei Yang, a software Quality Engineer from Red Hat. And
>> always paying attention to improving the live migration downtime
>> issues because there are others QE asked about this problem when I
>> share live migration status recently. Therefore I would like to test
>> it in my environment. Before the testing I want to know if there is an
>> expectation of downtime range based on this series of patches? In
>> addition, QE also can help do a regression test based on this series
>> of patches if there is a requirement.
>>
> Hi Lei,
>
> Thanks for offering the testing bandwidth!
>
> I think we can only do regression tests here, as the userland part is
> still not sent to qemu.
Right. Regression for now, even QEMU has it, to exercise the relevant
feature it would need a supporting firmware that is not yet available
for now. Just stay tuned.
thanks for your patience,
-Siwei
>
>> Regards and thanks
>> Lei
>>
>>
>> On Tue, Sep 12, 2023 at 9:04 PM Dragos Tatulea <[email protected]> wrote:
>>> This patch series adds support for vq descriptor table mappings which
>>> are used to improve vdpa live migration downtime. The improvement comes
>>> from using smaller mappings which take less time to create and destroy
>>> in hw.
>>>
>>> The first part adds the vdpa core changes from Si-Wei [0].
>>>
>>> The second part adds support in mlx5_vdpa:
>>> - Refactor the mr code to be able to cleanly add descriptor mappings.
>>> - Add hardware descriptor mr support.
>>> - Properly update iotlb for cvq during ASID switch.
>>>
>>> [0] https://lore.kernel.org/virtualization/[email protected]
>>>
>>> Dragos Tatulea (13):
>>> vdpa/mlx5: Create helper function for dma mappings
>>> vdpa/mlx5: Decouple cvq iotlb handling from hw mapping code
>>> vdpa/mlx5: Take cvq iotlb lock during refresh
>>> vdpa/mlx5: Collapse "dvq" mr add/delete functions
>>> vdpa/mlx5: Rename mr destroy functions
>>> vdpa/mlx5: Allow creation/deletion of any given mr struct
>>> vdpa/mlx5: Move mr mutex out of mr struct
>>> vdpa/mlx5: Improve mr update flow
>>> vdpa/mlx5: Introduce mr for vq descriptor
>>> vdpa/mlx5: Enable hw support for vq descriptor mapping
>>> vdpa/mlx5: Make iotlb helper functions more generic
>>> vdpa/mlx5: Update cvq iotlb mapping on ASID change
>>> Cover letter: vdpa/mlx5: Add support for vq descriptor mappings
>>>
>>> Si-Wei Liu (3):
>>> vdpa: introduce dedicated descriptor group for virtqueue
>>> vhost-vdpa: introduce descriptor group backend feature
>>> vhost-vdpa: uAPI to get dedicated descriptor group id
>>>
>>> drivers/vdpa/mlx5/core/mlx5_vdpa.h | 31 +++--
>>> drivers/vdpa/mlx5/core/mr.c | 191 ++++++++++++++++-------------
>>> drivers/vdpa/mlx5/core/resources.c | 6 +-
>>> drivers/vdpa/mlx5/net/mlx5_vnet.c | 100 ++++++++++-----
>>> drivers/vhost/vdpa.c | 27 ++++
>>> include/linux/mlx5/mlx5_ifc.h | 8 +-
>>> include/linux/mlx5/mlx5_ifc_vdpa.h | 7 +-
>>> include/linux/vdpa.h | 11 ++
>>> include/uapi/linux/vhost.h | 8 ++
>>> include/uapi/linux/vhost_types.h | 5 +
>>> 10 files changed, 264 insertions(+), 130 deletions(-)
>>>
>>> --
>>> 2.41.0
>>>
QE tested this series with regression testing on real nic, there are
no new issues.
Tested-by: Lei Yang <[email protected]>
On Fri, Sep 15, 2023 at 2:34 PM Si-Wei Liu <[email protected]> wrote:
>
>
>
> On 9/13/2023 9:08 AM, Eugenio Perez Martin wrote:
> > On Wed, Sep 13, 2023 at 3:03 AM Lei Yang <[email protected]> wrote:
> >> Hi Dragos, Eugenio and Si-Wei
> >>
> >> My name is Lei Yang, a software Quality Engineer from Red Hat. And
> >> always paying attention to improving the live migration downtime
> >> issues because there are others QE asked about this problem when I
> >> share live migration status recently. Therefore I would like to test
> >> it in my environment. Before the testing I want to know if there is an
> >> expectation of downtime range based on this series of patches? In
> >> addition, QE also can help do a regression test based on this series
> >> of patches if there is a requirement.
> >>
> > Hi Lei,
> >
> > Thanks for offering the testing bandwidth!
> >
> > I think we can only do regression tests here, as the userland part is
> > still not sent to qemu.
> Right. Regression for now, even QEMU has it, to exercise the relevant
> feature it would need a supporting firmware that is not yet available
> for now. Just stay tuned.
>
> thanks for your patience,
> -Siwei
> >
> >> Regards and thanks
> >> Lei
> >>
> >>
> >> On Tue, Sep 12, 2023 at 9:04 PM Dragos Tatulea <[email protected]> wrote:
> >>> This patch series adds support for vq descriptor table mappings which
> >>> are used to improve vdpa live migration downtime. The improvement comes
> >>> from using smaller mappings which take less time to create and destroy
> >>> in hw.
> >>>
> >>> The first part adds the vdpa core changes from Si-Wei [0].
> >>>
> >>> The second part adds support in mlx5_vdpa:
> >>> - Refactor the mr code to be able to cleanly add descriptor mappings.
> >>> - Add hardware descriptor mr support.
> >>> - Properly update iotlb for cvq during ASID switch.
> >>>
> >>> [0] https://lore.kernel.org/virtualization/[email protected]
> >>>
> >>> Dragos Tatulea (13):
> >>> vdpa/mlx5: Create helper function for dma mappings
> >>> vdpa/mlx5: Decouple cvq iotlb handling from hw mapping code
> >>> vdpa/mlx5: Take cvq iotlb lock during refresh
> >>> vdpa/mlx5: Collapse "dvq" mr add/delete functions
> >>> vdpa/mlx5: Rename mr destroy functions
> >>> vdpa/mlx5: Allow creation/deletion of any given mr struct
> >>> vdpa/mlx5: Move mr mutex out of mr struct
> >>> vdpa/mlx5: Improve mr update flow
> >>> vdpa/mlx5: Introduce mr for vq descriptor
> >>> vdpa/mlx5: Enable hw support for vq descriptor mapping
> >>> vdpa/mlx5: Make iotlb helper functions more generic
> >>> vdpa/mlx5: Update cvq iotlb mapping on ASID change
> >>> Cover letter: vdpa/mlx5: Add support for vq descriptor mappings
> >>>
> >>> Si-Wei Liu (3):
> >>> vdpa: introduce dedicated descriptor group for virtqueue
> >>> vhost-vdpa: introduce descriptor group backend feature
> >>> vhost-vdpa: uAPI to get dedicated descriptor group id
> >>>
> >>> drivers/vdpa/mlx5/core/mlx5_vdpa.h | 31 +++--
> >>> drivers/vdpa/mlx5/core/mr.c | 191 ++++++++++++++++-------------
> >>> drivers/vdpa/mlx5/core/resources.c | 6 +-
> >>> drivers/vdpa/mlx5/net/mlx5_vnet.c | 100 ++++++++++-----
> >>> drivers/vhost/vdpa.c | 27 ++++
> >>> include/linux/mlx5/mlx5_ifc.h | 8 +-
> >>> include/linux/mlx5/mlx5_ifc_vdpa.h | 7 +-
> >>> include/linux/vdpa.h | 11 ++
> >>> include/uapi/linux/vhost.h | 8 ++
> >>> include/uapi/linux/vhost_types.h | 5 +
> >>> 10 files changed, 264 insertions(+), 130 deletions(-)
> >>>
> >>> --
> >>> 2.41.0
> >>>
>