Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) client-ip=23.128.96.35;
Message-ID: <171a1147-2099-46da-81f1-fc953e9410e2@oracle.com>
Date:   Wed, 18 Oct 2023 16:47:48 -0700
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC v2 PATCH] vdpa_sim: implement .reset_map support
Content-Language: en-US
To:     Stefano Garzarella <sgarzare@redhat.com>
Cc:     jasowang@redhat.com, mst@redhat.com, eperezma@redhat.com,
        virtualization@lists.linux-foundation.org,
        linux-kernel@vger.kernel.org
References: <1697605893-30313-1-git-send-email-si-wei.liu@oracle.com>
 <becui44lhdptriz4ds7r2b22kazofwo7i44rydacjssnwb7mrq@f6sa74zjhlfp>
From:   Si-Wei Liu <si-wei.liu@oracle.com>
Organization: Oracle Corporation
In-Reply-To: <becui44lhdptriz4ds7r2b22kazofwo7i44rydacjssnwb7mrq@f6sa74zjhlfp>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?RVc4dFdhelZvT2lKblR5L2FtaXlvNkdVeVFZYTdicXBLckNaWGxUTmhVN2Vt?=
 =?utf-8?B?TzhWZjBNSTR3ZldtMjVwRXoyL1Z1dWpnbExvdXc2YXFQZ2tpQU1WY1Q4Q3pT?=
 =?utf-8?B?TGRFL0Iva1pTbVpKZ3JaV1RsVHRTa2lTcUdKUjlsOXdtWER5Lzg1MVVVZHNs?=
 =?utf-8?B?WlZoZ2EzZDEzUE8xeUx6YkQwS29zZ1BrT3l6M0lVYkFzVGFvVHVvMXN2Zkxh?=
 =?utf-8?B?WWpUR3dFU3Y0ZWdReGtYQW9BbWJ4RmxPOUM2dmtab0s2a0w0ejlwdk1Taklj?=
 =?utf-8?B?Wm5hL3dhd2ZMQlBmVFpRa0tWeW1PR0s0amtVenBjb2dLbUhON29BL1B2Qldh?=
 =?utf-8?B?Y3Z1VUZnaE1rY1BvdXc4MC9weXkzWG1mcEZPZzRMODVvb3BrYW9JK0k1Rm95?=
 =?utf-8?B?blYxaTFDQVlocS9vcmowVkJES0ZMUURSUXVoTzFsV0ZkOENMVVQ3L2FBT29H?=
 =?utf-8?B?Ujh2Zk4vOFB3K2NBMkk2QjFKY2tMQUdIZ25PWXJ0NjFnUFZOT2NlcE1qMDNn?=
 =?utf-8?B?TlZuTENkWEtyY1FSSGRBYWI5bDlnZ1JFQXd6L2IrSzBWMFJSTGFqMnRncndt?=
 =?utf-8?B?c3VsN2V4dnV0RkNhdGdJNmJhV1NvMVNVMnUrT3p2bzZ0N3NmSkpWREdud1h5?=
 =?utf-8?B?ZWZ6MExMaHFzM3hjVG9rYWszMHYyZi92L2RzMU1lMktvbFZSZXdjcmhvVFdm?=
 =?utf-8?B?bnFtWEd6cWV1MU52ZnF1N0UvZHlrNStVeER2cVUzWmFyMUFGU2NnSjUyVkJp?=
 =?utf-8?B?S1RsZ2hmMjM1QWJnbVZiVU90bnBTK1RzeXRMSHQ4THNYSVBHbFdWNmFFTUtm?=
 =?utf-8?B?ZlllbktNeUU1Q2FQWDY4ZG9nYWpuMGNBcFRmNG9OOGIxVUJKNnE4MlprMmR0?=
 =?utf-8?B?VHErZzF3UmYyTEJkWDVwN3FNbGFvdU5rZG5KeGZ1RmRWNUZlQ3orKzM5S2k2?=
 =?utf-8?B?N09VWkFIbHM0cFZQS09JTXpXcGU3L2Q0aFhxdDdxeWlyL01sTERnQ2krcyto?=
 =?utf-8?B?Z3hETlhlbXJVNnJvMXJxMzlFWENpV3l3NFE2c3piNnhrRDdVa25ISFJoTUVQ?=
 =?utf-8?B?M01GaXVXRzNiNlFvbFRPeFB5d3Jmb3ZOdXdmVFBRcXdXRWJTSjV4alZpaEpo?=
 =?utf-8?B?eU51OVdBK29iYW1GOHAvTEdXejd6VzlvWUJKbG42VDM0M3B3bjJ3NVZqN2JW?=
 =?utf-8?B?eFh4c2ZhSzVJcWZDb0QvRE43a0MrY1Z1WmxmMVlMV3FkVGEvckl6T1o2eExE?=
 =?utf-8?B?eVBwZUlyalFmaDF1Nkx1SzZtUUM4dmIzbVY2OFhQNWlEQmVyOTBFWjR3UU9a?=
 =?utf-8?B?Y2lWVXBFWXZGdjNHZDZ2amFQNWlnVDdYdDloQys5eUM5LzlKRXZJM3dVa0oy?=
 =?utf-8?B?Yy9ZUlhEQTRNMjdROXluVGJQSDRHQzBVOHU1Ulc0ZG1wOTBHcFNwaXQ4cHZB?=
 =?utf-8?B?cXV1MHBmQTB5TzVVa0UwRkh1ZDRnYjd4WXdPbEVydWV6Nk51MnFrdUNHT1Vy?=
 =?utf-8?B?ZG5uMUt2cmdNV2wyejBQM0lKVUQzSXRSbkdPOHI4N2xEbGhOSXpDSTZHZFo1?=
 =?utf-8?B?cWJBUEpTcUhnMHU2Mng5Z3NKYndQWUhObjJYRzI1cTJXeFlRMllMcDhJc2dB?=
 =?utf-8?B?K2pnVFA2bTJyd2NSVGVEQjRlUUhSVkROV0JpZ2FKT2QrNjhSOHV4MnYzWjk4?=
 =?utf-8?B?dDV5QTBoUDZvc3N1MDIzSXhPbHZObm00RU5vUDdLaWtwSTI1cE5mUThDWjVw?=
 =?utf-8?B?Vzk4a2NiNEhUQ0ZrZHk1WkhpdzFFQk1EaWVxYll2TUt0alY1TXFlQlNBYjZI?=
 =?utf-8?B?UW9sYkJ6cVVJWHUyUzJvT01FZUxpWFpTSkd6Z1oxUDY4cXIrY01RWHFTeHlM?=
 =?utf-8?B?dGd2YTRRVkJYd3hpbGh5UzlGSEhZZFlLR1FGU016a1pjU0ZWcFc5V2Uyclh0?=
 =?utf-8?B?azBLdXJtaGp5eVFxeGhWWXRlMUNzNDBnM2I4aUNQQUswZ0NheUtueHNWMy85?=
 =?utf-8?B?OXFGSllWMVhQWGlPVXRsTWJPbUNvOCtUTytsYi9hb2FEU3VHb0tMMzI1cThI?=
 =?utf-8?B?RjZHT3lndzR6YzBpVDV6cEpNb2VOT0dWNTVNWTd1U2laUldWaXVIRzVWT2Rh?=
 =?utf-8?Q?DsYtELc96Tr3I3JwQ3FqUG9hC?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: NOFnaL88h7NeDs9NQklBK5O6GMOu7Fqf6pDn/F0U6xpZFqyIa0kVvrqSpFsiS0Efa5tuWQDboDvucz5iNzM+sJb9dw3s7rImImh56UnLYEc0kSVfDf+w+YkoXrSn98gJpMxC/ab57UowGiY35EEHLC6CGKFgFlsJvasLlEsW/m9FgYPFa/7yBi5vJ51plHgSVu48unkX6S8ZUfSVcGmhC/wSMmNLTymJYmtbsGSkpPWdkNyczFCI7E0pzn4FVarrjE8pVkO0lZp0jQpweBymNS7w9MI4pFvANXsHSZSDPJjOMxcofQtGm9Wt2EHIlJ+guCvvhAaiJXHSNRQ/ynuNa/uwJ/J4hHsGGvdeKfX89IHhBZVh8qx7dufbxC7Gu3hoVqEsPUd99Yrnn+TWrzE+z7gYH/6DPROx9X/CGEUHZU9suUjtY4Seuu1XenWzgwv4XuoilrxKI4a1oJ9DL2X8KZ/bVMkIjwI5HC14nEr9KkVgwPhnHgyUADv+32hhF/sOOdYjmIVE57uTtXW3ZeDEQq3Nyff5nsmoGRFCtpd4MAsN8KElmZ8rd5Rpmw2UQz2h7fI4decWMIQbrEgKoZ2kmz7SDjXiz34+slfqLYdR9p4HnqY/x3CBrv9HeTOb8ymlWXIqc5Yq/SzYjZhCLsqcme0wYnM5DPiQpjdUKFmJFe4/mTq1irFKwmPywu1oyRBizWTpTENGsy5VOPtZ4yy1+Ik6ULJVuX5QXWngeIF0UXIy/GuAyl40uZUAORMRyyLwcpGUcPOyEpZOTrP3JzcSalP6np5KckZe/wEQ9Y3u4wiWjj9cTShHkyQ8U+GuesLu5KYS6K2cvy7RrFD585ZVMcBmHBrVr/JJ96nPZoGkfOtUwbDM+k+46BgVNfXWXl9R
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 40f01ccf-0cfc-4c0a-11dc-08dbd034a527
X-MS-Exchange-CrossTenant-AuthSource: MW4PR10MB6535.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Oct 2023 23:47:53.7299
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: b9DjPsK6fpjghp+jZENoTYum2naPwzBrMrTFLdhq6fJ5yzKEI619VGzsevLkc6Gmt8xDoRo0Fdcvbrl4loA7PQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR10MB5321
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.980,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2023-10-18_18,2023-10-18_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0
 phishscore=0 spamscore=0 suspectscore=0 malwarescore=0 bulkscore=0
 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2309180000 definitions=main-2310180197
X-Proofpoint-GUID: MxiAhUpqLFcoK8qyRC2OyzllKxIvt1If
X-Proofpoint-ORIG-GUID: MxiAhUpqLFcoK8qyRC2OyzllKxIvt1If
X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,
	DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,
	SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no
	version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Wed, 18 Oct 2023 16:48:43 -0700 (PDT)


On 10/18/2023 1:05 AM, Stefano Garzarella wrote:
> On Tue, Oct 17, 2023 at 10:11:33PM -0700, Si-Wei Liu wrote:
>> RFC only. Not tested on vdpa-sim-blk with user virtual address.
>> Works fine with vdpa-sim-net which uses physical address to map.
>>
>> This patch is based on top of [1].
>>
>> [1] 
>> https://lore.kernel.org/virtualization/1696928580-7520-1-git-send-email-si-wei.liu@oracle.com/
>>
>> Signed-off-by: Si-Wei Liu <si-wei.liu@oracle.com>
>>
>> ---
>> RFC v2:
>>  - initialize iotlb to passthrough mode in device add
>
> I tested this version and I didn't see any issue ;-)
Great, thank you so much for your help on testing my patch, Stefano!
Just for my own interest/curiosity, currently there's no vhost-vdpa 
backend client implemented for vdpa-sim-blk or any vdpa block device in 
userspace as yet, correct? So there was no test specific to vhost-vdpa 
that needs to be exercised, right?

Thanks,
-Siwei


>
> Tested-by: Stefano Garzarella <sgarzare@redhat.com>
>
>> ---
>> drivers/vdpa/vdpa_sim/vdpa_sim.c | 34 ++++++++++++++++++++++++--------
>> 1 file changed, 26 insertions(+), 8 deletions(-)
>>
>> diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c 
>> b/drivers/vdpa/vdpa_sim/vdpa_sim.c
>> index 76d41058add9..2a0a6042d61d 100644
>> --- a/drivers/vdpa/vdpa_sim/vdpa_sim.c
>> +++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c
>> @@ -151,13 +151,6 @@ static void vdpasim_do_reset(struct vdpasim 
>> *vdpasim)
>>                  &vdpasim->iommu_lock);
>>     }
>>
>> -    for (i = 0; i < vdpasim->dev_attr.nas; i++) {
>> -        vhost_iotlb_reset(&vdpasim->iommu[i]);
>> -        vhost_iotlb_add_range(&vdpasim->iommu[i], 0, ULONG_MAX,
>> -                      0, VHOST_MAP_RW);
>> -        vdpasim->iommu_pt[i] = true;
>> -    }
>> -
>>     vdpasim->running = true;
>>     spin_unlock(&vdpasim->iommu_lock);
>>
>> @@ -259,8 +252,12 @@ struct vdpasim *vdpasim_create(struct 
>> vdpasim_dev_attr *dev_attr,
>>     if (!vdpasim->iommu_pt)
>>         goto err_iommu;
>>
>> -    for (i = 0; i < vdpasim->dev_attr.nas; i++)
>> +    for (i = 0; i < vdpasim->dev_attr.nas; i++) {
>>         vhost_iotlb_init(&vdpasim->iommu[i], max_iotlb_entries, 0);
>> +        vhost_iotlb_add_range(&vdpasim->iommu[i], 0, ULONG_MAX, 0,
>> +                      VHOST_MAP_RW);
>> +        vdpasim->iommu_pt[i] = true;
>> +    }
>>
>>     for (i = 0; i < dev_attr->nvqs; i++)
>>         vringh_set_iotlb(&vdpasim->vqs[i].vring, &vdpasim->iommu[0],
>> @@ -637,6 +634,25 @@ static int vdpasim_set_map(struct vdpa_device 
>> *vdpa, unsigned int asid,
>>     return ret;
>> }
>>
>> +static int vdpasim_reset_map(struct vdpa_device *vdpa, unsigned int 
>> asid)
>> +{
>> +    struct vdpasim *vdpasim = vdpa_to_sim(vdpa);
>> +
>> +    if (asid >= vdpasim->dev_attr.nas)
>> +        return -EINVAL;
>> +
>> +    spin_lock(&vdpasim->iommu_lock);
>> +    if (vdpasim->iommu_pt[asid])
>> +        goto out;
>> +    vhost_iotlb_reset(&vdpasim->iommu[asid]);
>> +    vhost_iotlb_add_range(&vdpasim->iommu[asid], 0, ULONG_MAX,
>> +                  0, VHOST_MAP_RW);
>> +    vdpasim->iommu_pt[asid] = true;
>> +out:
>> +    spin_unlock(&vdpasim->iommu_lock);
>> +    return 0;
>> +}
>> +
>> static int vdpasim_bind_mm(struct vdpa_device *vdpa, struct mm_struct 
>> *mm)
>> {
>>     struct vdpasim *vdpasim = vdpa_to_sim(vdpa);
>> @@ -759,6 +775,7 @@ static const struct vdpa_config_ops 
>> vdpasim_config_ops = {
>>     .set_group_asid         = vdpasim_set_group_asid,
>>     .dma_map                = vdpasim_dma_map,
>>     .dma_unmap              = vdpasim_dma_unmap,
>> +    .reset_map              = vdpasim_reset_map,
>>     .bind_mm        = vdpasim_bind_mm,
>>     .unbind_mm        = vdpasim_unbind_mm,
>>     .free                   = vdpasim_free,
>> @@ -796,6 +813,7 @@ static const struct vdpa_config_ops 
>> vdpasim_batch_config_ops = {
>>     .get_iova_range         = vdpasim_get_iova_range,
>>     .set_group_asid         = vdpasim_set_group_asid,
>>     .set_map                = vdpasim_set_map,
>> +    .reset_map              = vdpasim_reset_map,
>>     .bind_mm        = vdpasim_bind_mm,
>>     .unbind_mm        = vdpasim_unbind_mm,
>>     .free                   = vdpasim_free,
>> -- 
>> 2.39.3
>>
>