Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34;
Message-ID: <b223d828-2c08-841f-47fb-7cb072fa5ec9@oracle.com>
Date:   Thu, 14 Sep 2023 23:34:20 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Thunderbird/102.15.0
Subject: Re: [PATCH 00/16] vdpa: Add support for vq descriptor mappings
Content-Language: en-US
To:     Eugenio Perez Martin <eperezma@redhat.com>,
        Lei Yang <leiyang@redhat.com>
Cc:     Dragos Tatulea <dtatulea@nvidia.com>,
        Jason Wang <jasowang@redhat.com>,
        "Michael S . Tsirkin" <mst@redhat.com>,
        Saeed Mahameed <saeedm@nvidia.com>,
        virtualization@lists.linux-foundation.org,
        linux-kernel@vger.kernel.org, Parav Pandit <parav@nvidia.com>,
        Xuan Zhuo <xuanzhuo@linux.alibaba.com>, kvm@vger.kernel.org
References: <20230912130132.561193-1-dtatulea@nvidia.com>
 <CAPpAL=w6KeBG5Ur037GNQa=n_fdoUwrFo+ATsFtX9HbWPHZvsg@mail.gmail.com>
 <CAJaqyWeVjKTPmGWwZ26TgebuzCaN8Z2FmPontHvZauOTQj0brQ@mail.gmail.com>
From:   Si-Wei Liu <si-wei.liu@oracle.com>
Organization: Oracle Corporation
In-Reply-To: <CAJaqyWeVjKTPmGWwZ26TgebuzCaN8Z2FmPontHvZauOTQj0brQ@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?dlhLRW1xVUswRUhRSnVPcWhmdDBteEZDZGtFY253amtiMHJoNjd1QmRMbm5x?=
 =?utf-8?B?TmwwMFFiUTJMMzBnMkl5S2JQanA2OHNGclRqUW5DYkpybm1TeUR1M1I0ZUdp?=
 =?utf-8?B?VkNEM2VOejdUbmI3MUEyZk9WeW9wTEVxalJMZU5WNm56ZHdkWGVOWUlqcWMz?=
 =?utf-8?B?WXZ2VkNwZEwyMVJVN0pKWjVueFJETVNQUkM3TFpib1dmSkxJcGk0NWlEeGFs?=
 =?utf-8?B?b1FxTFpRV0Q4Z1hZZnRRRGgxc0VXa084YWFzOUVacjhQOERyU3F0Vlp2aVR0?=
 =?utf-8?B?Um1jc3R1bll5NDQyODJUcjhhVXVITkNSeWNLQWpaQlBUVlUzQituZW8wZFpw?=
 =?utf-8?B?SVNxVnVSR1BEZE81VzBnVWVBNjVBbnduZ0ZySHcxR1JFUzA1eE1WWDhyQy85?=
 =?utf-8?B?bGRKd3dheWZ3L0lsaEFBQmpIb09Ic1JPUkRjM1Y2WURFdmQ4SlVkTGh2b2xx?=
 =?utf-8?B?WHNiNENBZTNiOTROK3FUOTgyWnJEVVgvZ1FKNW9FclE3UkxsUkZOYThVWlB5?=
 =?utf-8?B?UHBJYkFIQ0FmUE5xeFcvSk5hWnYvKytUdFF3RlZSdVBsZHUvUTJIcGgvekNy?=
 =?utf-8?B?ZEI5MVZUWGQ0cTBqd2FFbk9vT2llbUtOejJEb0Fja2V5NmNHbU9md3ViSXE4?=
 =?utf-8?B?Vnd4anptWUVYV0RkWm1TYVlLR3ZjS0xENW5ZMU13NGtpL0NQYjRFYkhzZHUy?=
 =?utf-8?B?Z0xaNWV1SENyMUhZTmR5cmJIWXRlK3M3WEM4VXUxSnV1SnlvMS9sUDZseXRj?=
 =?utf-8?B?dDlsbmtVQlFtWW5mb1AxSlFKQnJHT0VpbXZJdStIaFljS3ZIMkZ1R1A4VVJk?=
 =?utf-8?B?eWRneGp0UlRZM0d4SU5ZWjExRXR4SUpXOVhCZ1ByWnlkVFU5MDFob2J6c0hh?=
 =?utf-8?B?azB0bDg2OTNVZ1NNWm5mV0M2b2diTit5SXJTMUJiWk0yRFRlSXZ5KytxdDlj?=
 =?utf-8?B?ZUZxZDhVaGtHdEkzSFlpRXNtV0k5d0F5U2laL2dHcVhkZ0laMUtOL2ZEdEdp?=
 =?utf-8?B?MndFakpCMzFiMkg1dTBGNEhKbXEwS3E1SU1vYlNqaVRGMHAwS01uMW1IT0RL?=
 =?utf-8?B?ekJjenBCVldoTXpIZzNkSFp0RkdUOTdoV1luNEtvTkZwVVpZWk5ueHRLMHpS?=
 =?utf-8?B?NzRrTWxXOGlpYlBrUmRqdzZPM010NXk0bUFoYzdEUG5aM3BwYjltREovZ0Fv?=
 =?utf-8?B?am9VNnFDSnZSc0x0WXBiNEN3WVpJdDJac2tPYklSaXl3ZVF1T211VnB3WUhM?=
 =?utf-8?B?cnpHaWtROCsxbXlpc0Z3eUNxejZ3TmpQb1QvWjdFdkNVMGozWVZ1M3JNSVpP?=
 =?utf-8?B?bXB0akJ2TW5iSTJuTFNoUkh2VEppQXh4S2FDb1QyUm1NOHI2V1ROeDBRTXJS?=
 =?utf-8?B?SCtKWmtaY1NWLy9xdlV1dm13UjYxQkVNY1BiTlV2c2JPK08vOWxyRExHRGho?=
 =?utf-8?B?UDR0b2xrWUhyQVo5UWRYckh0VVpLRHFTdVRnVTErOEUzUnRmVkpMUmszSDMx?=
 =?utf-8?B?ZE9MUXArQU1QU0xITGsxZ1E2QlJnaFR1ZXkwNmVQY1pscXVydWM2S2JFV09o?=
 =?utf-8?B?VCtpaTlnV09BTVloQnY5aVJRWEVoOE11dEozczFQS0pmTEZhZXpRc2c5NE1C?=
 =?utf-8?B?dnZab2htWFhPZGQzSitiZHRMRFVMcjNTcDRSZ25mT2xkNHJjQm9raC9heXVj?=
 =?utf-8?B?Zjg2elVBY29HMm04UGtncmZKdGRuaU1oNndpYnNQQ0N5RCtGeHJ2NjN5SzEz?=
 =?utf-8?B?Tml6Rm40T29sVjlUNXlUeExjTHF6UkplUVQ2UjdsZHZqVzl5d08wM2ptbWZB?=
 =?utf-8?B?bER0bSt6TCt3RjZoMTl5MG9sdnZpR1hDblFQUGdoN3hPNjV5MFg1TmlNNXRP?=
 =?utf-8?B?Skw0N2c1cjlxSjRaTkNmdmZsMG1EaStFTXJRTHFBRWFoNTh5TGhNenVJNDg5?=
 =?utf-8?B?d0p2VGg4VnVwVGlGc2NwUVM2TFVkUCt6TWZNSmNjUkI2RG1QZUNFYithdXlY?=
 =?utf-8?B?Y3VOK1FHazdrMjAzYU96M2d6SlV3Y3c4eWVUQlFpclUzOGJnb0VUSCs0Sy9h?=
 =?utf-8?B?NCt6bnFaMmpZUGtmQkdMTEFJVmE5TnFoYkR2dk1EczBmMS9RUGgvNlAyc25F?=
 =?utf-8?Q?6Zkfw1yfV/LpG8+9/G+PcfKcQ?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: =?utf-8?B?cGxjZXo3M1BSNmlqeDhOY204THNWSmI1Sy9hQ2NkaVUybEIwOFVJOUh2WCtB?=
 =?utf-8?B?Um9zMmRmbW9LcGIyRHk0REpRQXY4SnVtc0s1dS9iV2Q2YjNaalpQT2Z2d1pH?=
 =?utf-8?B?eTQreTBySyt5NmZzMi8zdXRlaW5QYk13NHBMU3dyNnlyMjdPYUljbi96NmY4?=
 =?utf-8?B?SHhKdUp3cWMyUFJHMVNjV0tLbC9OOHZFczZJWnQ3TXd1TnR3b2FMZXVic2Rq?=
 =?utf-8?B?dEZ4aGFMWDl3YlJDSWRNSThrdU9XWWIrSUdldWYxeUI2N2hGNVU0ODNXRHpm?=
 =?utf-8?B?WUZMcUhpbGJiUjRaRmIzOExrRndQUzBjajV3elV4WFBNb0Zmc0FYZTMweHR5?=
 =?utf-8?B?Tlhld1FLYkg1TUs2cnB2VWhnajBoQThCV1Fkck1tQXVLUzVEV3dZVkUwd0lH?=
 =?utf-8?B?ZVlZRHlhbGpTc0FjZ0lDdDIrRDdvd25TVERKa1pZbVViNUVlbTV6UXozM05m?=
 =?utf-8?B?V0lHQkRrYVdmM2hVWUhEdGIxbVdleGJlOFphVlFpbkdWMGR3M3o2a1NLQjc3?=
 =?utf-8?B?emtiZXMzS1ZMbm41a1l0ZW5PYnNZMUZGUGpybnd2d2QvZkNjVXpCSVRKN3VH?=
 =?utf-8?B?V3JmVFhPUnR5ZWJNZUhVS25jSnh5WnJvNVRsZkM1aCtqS251MUtGUUZuYjNE?=
 =?utf-8?B?cGZGQWNGZ2dsSkVOc2FUZWlIMVA1ZWowVGd4ZXBQOExUV0llMTJYem9wU2xX?=
 =?utf-8?B?NmtBQVdpRHppS2pkNnlIcEtScHk3WG9rYUE1aEtxamNHbzBHWjdUdGhOV256?=
 =?utf-8?B?NFdyNlBBNDhHdlNTT1FhaDYrakVweWJ4U2FZcXpRVmtEaXZQeW9QY1lxMnZB?=
 =?utf-8?B?UCt4cGFrUVFEdDZrdFBtQzZUMTBJbGFZUFBZUmczeno2ZnBTcEdhNGUyakhh?=
 =?utf-8?B?YU93UDl0ZUx0ZHEwNjNPU2pISmRxOGFWVEtVeGQ4ZlVHZTh5VmMxTk5mT2hE?=
 =?utf-8?B?eFlWeHkyN05EMXdHd28wRmluNi82TENicENPUVBoSzBFVDBySDdpQnFyZ0FD?=
 =?utf-8?B?SnhoLzQzSTE0aVhpVTkwT2VLRGwyVTRGQ0YrRFRJQWROQm5zcmZIZGJ0VFV1?=
 =?utf-8?B?b3M4WjBDT3dRb3NtVnZkc3BxazFwaERoTldEcmZLU1plMmxTbmI1TGQ4REky?=
 =?utf-8?B?V2RlTi94SCtleXNlU2d4aGtoQWpXS0Vudm1rSVdBSGgwTFlETllFcjJscmls?=
 =?utf-8?B?Ti9aWitEZDZRNjE2Z2VSNWtTK1plUjc5M0d0a1kwdUpwWCtOTVpOZ2xKN0NU?=
 =?utf-8?Q?NS7BqFi5dfTi0GZ?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: cc378059-9ad3-4037-a9e9-08dbb5b5cc7f
X-MS-Exchange-CrossTenant-AuthSource: MW4PR10MB6535.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2023 06:34:23.1574
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 7jIfhXl0qthACs2/jSd2R6g2xG/apatVIJXGMIXdO5F8UVtMpvO6fTmLdKo36eEeORiRaKm7Wayh/ZDJn4SNOQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR10MB7321
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.267,Aquarius:18.0.980,Hydra:6.0.601,FMLib:17.11.176.26
 definitions=2023-09-15_05,2023-09-14_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 mlxscore=0 bulkscore=0
 adultscore=0 phishscore=0 spamscore=0 mlxlogscore=999 suspectscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2308100000
 definitions=main-2309150057
X-Proofpoint-ORIG-GUID: ZKy0hcK-5oPC8WflXS1mfZe4dssctfO-
X-Proofpoint-GUID: ZKy0hcK-5oPC8WflXS1mfZe4dssctfO-
X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Thu, 14 Sep 2023 23:34:46 -0700 (PDT)


On 9/13/2023 9:08 AM, Eugenio Perez Martin wrote:
> On Wed, Sep 13, 2023 at 3:03 AM Lei Yang <leiyang@redhat.com> wrote:
>> Hi Dragos, Eugenio and Si-Wei
>>
>> My name is Lei Yang, a software Quality Engineer from Red Hat.  And
>> always paying attention to improving the live migration downtime
>> issues because there are others QE asked about this problem when I
>> share live migration status  recently. Therefore I would like to test
>> it in my environment. Before the testing I want to know if there is an
>> expectation of downtime range based on this series of patches? In
>> addition, QE also can help do a regression test based on this series
>> of patches if there is a requirement.
>>
> Hi Lei,
>
> Thanks for offering the testing bandwidth!
>
> I think we can only do regression tests here, as the userland part is
> still not sent to qemu.
Right. Regression for now, even QEMU has it, to exercise the relevant 
feature it would need a supporting firmware that is not yet available 
for now. Just stay tuned.

thanks for your patience,
-Siwei
>
>> Regards and thanks
>> Lei
>>
>>
>> On Tue, Sep 12, 2023 at 9:04 PM Dragos Tatulea <dtatulea@nvidia.com> wrote:
>>> This patch series adds support for vq descriptor table mappings which
>>> are used to improve vdpa live migration downtime. The improvement comes
>>> from using smaller mappings which take less time to create and destroy
>>> in hw.
>>>
>>> The first part adds the vdpa core changes from Si-Wei [0].
>>>
>>> The second part adds support in mlx5_vdpa:
>>> - Refactor the mr code to be able to cleanly add descriptor mappings.
>>> - Add hardware descriptor mr support.
>>> - Properly update iotlb for cvq during ASID switch.
>>>
>>> [0] https://lore.kernel.org/virtualization/1694248959-13369-1-git-send-email-si-wei.liu@oracle.com
>>>
>>> Dragos Tatulea (13):
>>>    vdpa/mlx5: Create helper function for dma mappings
>>>    vdpa/mlx5: Decouple cvq iotlb handling from hw mapping code
>>>    vdpa/mlx5: Take cvq iotlb lock during refresh
>>>    vdpa/mlx5: Collapse "dvq" mr add/delete functions
>>>    vdpa/mlx5: Rename mr destroy functions
>>>    vdpa/mlx5: Allow creation/deletion of any given mr struct
>>>    vdpa/mlx5: Move mr mutex out of mr struct
>>>    vdpa/mlx5: Improve mr update flow
>>>    vdpa/mlx5: Introduce mr for vq descriptor
>>>    vdpa/mlx5: Enable hw support for vq descriptor mapping
>>>    vdpa/mlx5: Make iotlb helper functions more generic
>>>    vdpa/mlx5: Update cvq iotlb mapping on ASID change
>>>    Cover letter: vdpa/mlx5: Add support for vq descriptor mappings
>>>
>>> Si-Wei Liu (3):
>>>    vdpa: introduce dedicated descriptor group for virtqueue
>>>    vhost-vdpa: introduce descriptor group backend feature
>>>    vhost-vdpa: uAPI to get dedicated descriptor group id
>>>
>>>   drivers/vdpa/mlx5/core/mlx5_vdpa.h |  31 +++--
>>>   drivers/vdpa/mlx5/core/mr.c        | 191 ++++++++++++++++-------------
>>>   drivers/vdpa/mlx5/core/resources.c |   6 +-
>>>   drivers/vdpa/mlx5/net/mlx5_vnet.c  | 100 ++++++++++-----
>>>   drivers/vhost/vdpa.c               |  27 ++++
>>>   include/linux/mlx5/mlx5_ifc.h      |   8 +-
>>>   include/linux/mlx5/mlx5_ifc_vdpa.h |   7 +-
>>>   include/linux/vdpa.h               |  11 ++
>>>   include/uapi/linux/vhost.h         |   8 ++
>>>   include/uapi/linux/vhost_types.h   |   5 +
>>>   10 files changed, 264 insertions(+), 130 deletions(-)
>>>
>>> --
>>> 2.41.0
>>>