Received-SPF: pass (google.com: domain of linux-kernel+bounces-29425-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Message-ID: <3324a580-8436-424b-bff1-5d7df046b938@oracle.com>
Date: Wed, 17 Jan 2024 15:34:43 -0500
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC V1 01/13] vhost-vdpa: count pinned memory
Content-Language: en-US
To: "Michael S. Tsirkin" <mst@redhat.com>
Cc: virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org,
        Jason Wang <jasowang@redhat.com>, Si-Wei Liu <si-wei.liu@oracle.com>,
        Eugenio Perez Martin <eperezma@redhat.com>,
        Xuan Zhuo <xuanzhuo@linux.alibaba.com>,
        Dragos Tatulea
 <dtatulea@nvidia.com>, Eli Cohen <elic@nvidia.com>,
        Xie Yongji <xieyongji@bytedance.com>
References: <1704919215-91319-1-git-send-email-steven.sistare@oracle.com>
 <1704919215-91319-2-git-send-email-steven.sistare@oracle.com>
 <20240110172306-mutt-send-email-mst@kernel.org>
From: Steven Sistare <steven.sistare@oracle.com>
Organization: Oracle Corporation
In-Reply-To: <20240110172306-mutt-send-email-mst@kernel.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?UFhWUUxCYk9hNVVnR1JEdFc4RG9mWXV2ODJVQW5MNmtxd3dwY2tpckhCUlpm?=
 =?utf-8?B?NnBWRWhMTTM5UjJTanU4OTVEZUlQaVh4S0RkQzhEM1FLeVNucEIrTHByTDdB?=
 =?utf-8?B?bGwyMGcrazF4NmRrcklrMkhxcW5HVjFLZGhDcHUzT095dUZBNmo5d0JXeXRB?=
 =?utf-8?B?TlhtMmNhUnB2dVNLd0JIUjJ0YnBiK1drbTNXU0hzc3NlZFZqaG1XRWJmN0Nn?=
 =?utf-8?B?aFM4TUE0b1BYSWE0OWtSTnZXeUxVaTAyWlJmaXFJbGZMbTRtUW44NnVlOERJ?=
 =?utf-8?B?QnlpVStxeXRwNDZjOTVMd3pQUkI4bis4SkJkR2xHQXA2Z2IwUGJjYTF0V3gr?=
 =?utf-8?B?NUhwMXN0OHV1d29sa3NLbUhZRTB4T3NnWXRCWlFwaUdKNFZDQm1UOVlVMXpp?=
 =?utf-8?B?QWgxZEhaODdPOUUvOFBjaEFGaHFKMjN4TU4zaHFQelRwek9ueTVTaC9CK2dT?=
 =?utf-8?B?SlJ4SnkwTHZWREZwampja2ZSR3kwQ1FVZVJCVFZJa1FWNFhDTGJnRVQ5elBB?=
 =?utf-8?B?OURSeDdqblBDMnNYd1Y5SklCQWZRSWxUWHo0aDhteTZtYXFiWitodHdTVFVy?=
 =?utf-8?B?UWNHajdUN2dVNEgxMmxlak9DTGZHa2xwdHQ0dzV2Mm5FS2tJcTU0bko4ZU5B?=
 =?utf-8?B?Z0drdVN4bmlpQnZOVDhpTStPL2RCeGJJYTkveEMvN3ZrNWNrL3VZYjczV0sz?=
 =?utf-8?B?ZkJyZTdjSjFuYnAxWHVReVFnMVFoRUNKTWtrNnFvcFNZQlRvc20zVE9iWFNU?=
 =?utf-8?B?SDRualFSYTFFZ3B2MnE4Y1d6U1VhWHdoVk9NbSswSERpK0ZCazllK1VlZlVr?=
 =?utf-8?B?QllQQnAyZUhML0lldFNWWjZGTG9md1YwK2I3V1I5Nzd3ZmY2TG15eEFFRnBu?=
 =?utf-8?B?K0gyQy96Q0NWek1acVNOQzk0SFZDelhGRG5XNEtUdjc1ZHVHaFdJMGVuc1Ax?=
 =?utf-8?B?MWt3RGVwczAyNzZ4b2JLVDFmOEZXY1IwcTFuRW1HeVFWdWtnd1h5WGFVN3pl?=
 =?utf-8?B?bE9mbi8zU0dwQjdPVFk0NzBLQnZQK2N1bWxjRmV4VEtPdmpTenp6Q3pmR2hD?=
 =?utf-8?B?c2hwZkRGWkhkMFN4ekUwRXBxWkFSSjBhbWQwWFJsQnR4eEJ3U1ZUdEk0a1Nq?=
 =?utf-8?B?TFNmQVhwQ2xmTGNUUnJqcUkvK3BLdzA0VldDaTU5RldMb29iTmw5a21JWC9J?=
 =?utf-8?B?WWNDVlFRUDRVZWVmaHExN1IwNVU2aWkrNUd0QUMxYVduTCs0TDE0VkxTcUx6?=
 =?utf-8?B?UEg3Q1lZTTJTVkdFVUszVXNtYktNY1JvdmRQRnRFRHFjMmVoNjFIT3NqaVVp?=
 =?utf-8?B?dEhhWUxIMTEwUU1SSHh0dmREYlJTT2p6eSs0aW5lSzBVN1hNVkZ2VFhyOU9J?=
 =?utf-8?B?VVpaemdUQzJkdmY3Vnc5b29KMDFRaUg0ZGUvRENjWXZiT1NFYStibndWbVJi?=
 =?utf-8?B?Y2tCK1BmNm5iZSswYU8yL254b3d2Z2QrSWZSWmw0K2JJdVgxV0UwY3U4U0pP?=
 =?utf-8?B?cHdpSnVJa1BuRkpRci9JKzAyK0tFVVIwZkN4cncwRnNOUFlVclpOZWN0NnpV?=
 =?utf-8?B?QUpNbkhHMGlwc1d4R3MxY0t5VVJLZ2NRSXFZMTlLMm1ackEwYnVYalMzV1ZL?=
 =?utf-8?B?aE9nc3QvTURJVnR2R2FhWTM0Rm44dE15TmN5Z216eERZU09ZVUF0aWJZSWZG?=
 =?utf-8?B?UHF5Y3NuMkFaV0VaaGdKL1d2SUVGWFJWZm1IVGRMVEVoTFRiUERBMVhySGVO?=
 =?utf-8?B?R0lLVGJoYkViREtwWkNKeDNmQVE1d2xFVFBVNTZZSmhwOTNIemMvY0daU1U0?=
 =?utf-8?B?eVhzWWtOb2tUaE9PQk5ZOXpJRE5sWWRvU1VPdlhYVTRlYy9HZ0pTN2lhZGxw?=
 =?utf-8?B?VmZtVTlINm9qNGhoRGloQkZHOWVpeDYwdlA5djlab1lkTlpmQU82NkpFbnBj?=
 =?utf-8?B?RzlXYktMZVlLVElvaHE4ZUVWSWV1d05YcmVjdjFDZTNEZEF3YTcvUTdSN2JN?=
 =?utf-8?B?MGZycFNiTEcrRldXdzd0R3NMT1NEaUFqZTA0ZnVXQ25OZ01ybk4rMGNGTmF6?=
 =?utf-8?B?KzVqQmNxZEpEUzlndGJjK3dwTGt3Q0pub3lSWlk2UzExOWcrT3Z1aWJ2NVR1?=
 =?utf-8?B?UGJnUlNaZmpWa0l3Z0dIaE9VV3R3SURoTDRSOHJtcWJNeXlhQVdXM3pJZGRJ?=
 =?utf-8?B?UEE9PQ==?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	CtPIadeqvIxXmheylrjPxwUuQ3G8FSt0x8OMdg2qQVi+T3knwugS3MkLcevZBw5z44hv+J7D0HlpQHZeuxB3fwYzFxWfJKXIpd9Se9CfDw6o4K2g/JmpuCmgcNQiOU8s91w1+5WA1Jinn9vqr7+vRGRsxA90klsOb2rv1j5merTUoirbQL1WoE1vN/XMItX5kXrn9Xzcs/NwocuBVoj5Zxtx6SARMDV8Z3Fi2QgAfIRI1LbkcYYD0sXyTdhgG4HW+ALphcAPNn4FZ9Hop+fgiT8WoJdy+XUdwjHR34KcKIw81d6WyueUOHiAkFp9hRfzpgip4tzH8qQ7JhV0G9LQG9J7JfQBKR4DjmzW6GtnTEGo6td86YwEKRRaV3pkdeo45NXNP51/bFg7UsWNYnUrTZ8wGXuwoSUkCtSnDTJ8i5s+mD1mrc87cnxIHOMiftuJoQa8jzGI5bBDhGVaf0authu6Pa19qlxWnABhGPkQPbwZWAQSy8JnrMWe64STJ9AUjr3c12EI2Pzh80xgkHkDAFiOUDVjG1X83lb/sQ7qCOal8rkj1y3ty7Qz9ayeSm9TVTj/pNFJLzddisKkQGp2CJKZFXUMKJtX4Fr8PXRCUWY=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: dbd43cfb-6a4c-470e-cadc-08dc179bbe49
X-MS-Exchange-CrossTenant-AuthSource: SA2PR10MB4684.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Jan 2024 20:34:46.0902
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: LyUOEI7Ls8DJou2vcVI7J6EognB+AAFkeNG1xRL6BkVSjSnwEiBxFZyc22l+jJ7LOqyf+38HXSAIwb4cGnXvAathZKNDU3c1xfnZVdIMWVI=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR10MB4451
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2024-01-17_12,2024-01-17_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 adultscore=0 phishscore=0
 mlxlogscore=958 suspectscore=0 bulkscore=0 mlxscore=0 malwarescore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000
 definitions=main-2401170150
X-Proofpoint-GUID: _lNCiHDzVYJaL26jm_RFg9uQJWLYb0Dw
X-Proofpoint-ORIG-GUID: _lNCiHDzVYJaL26jm_RFg9uQJWLYb0Dw

On 1/10/2024 5:24 PM, Michael S. Tsirkin wrote:
> On Wed, Jan 10, 2024 at 12:40:03PM -0800, Steve Sistare wrote:
>> Remember the count of pinned memory for the device.
>>
>> Signed-off-by: Steve Sistare <steven.sistare@oracle.com>
> 
> Can we have iommufd support in vdpa so we do not keep extending these hacks?

I assume this is rhetorical and not aimed specifically at me, but live update
interfaces for iommufd are on my todo list.

- Steve

>> ---
>>  drivers/vhost/vdpa.c | 7 +++++--
>>  1 file changed, 5 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
>> index da7ec77cdaff..10fb95bcca1a 100644
>> --- a/drivers/vhost/vdpa.c
>> +++ b/drivers/vhost/vdpa.c
>> @@ -59,6 +59,7 @@ struct vhost_vdpa {
>>  	int in_batch;
>>  	struct vdpa_iova_range range;
>>  	u32 batch_asid;
>> +	long pinned_vm;
>>  };
>>  
>>  static DEFINE_IDA(vhost_vdpa_ida);
>> @@ -893,6 +894,7 @@ static void vhost_vdpa_pa_unmap(struct vhost_vdpa *v, struct vhost_iotlb *iotlb,
>>  			unpin_user_page(page);
>>  		}
>>  		atomic64_sub(PFN_DOWN(map->size), &dev->mm->pinned_vm);
>> +		v->pinned_vm -= PFN_DOWN(map->size);
>>  		vhost_vdpa_general_unmap(v, map, asid);
>>  		vhost_iotlb_map_free(iotlb, map);
>>  	}
>> @@ -975,9 +977,10 @@ static int vhost_vdpa_map(struct vhost_vdpa *v, struct vhost_iotlb *iotlb,
>>  		return r;
>>  	}
>>  
>> -	if (!vdpa->use_va)
>> +	if (!vdpa->use_va) {
>>  		atomic64_add(PFN_DOWN(size), &dev->mm->pinned_vm);
>> -
>> +		v->pinned_vm += PFN_DOWN(size);
>> +	}
>>  	return 0;
>>  }
>>  
>> -- 
>> 2.39.3
>