Message-ID: <8d2b6b6d-5b1d-1784-8b95-865be650a6ea@intel.com>
Date:   Sat, 11 Mar 2023 09:31:41 -0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Firefox/91.0 Thunderbird/91.11.0
Subject: Re: [PATCH v2 08/16] iommu: define and export
 iommu_access_remote_vm()
Content-Language: en-US
From:   Fenghua Yu <fenghua.yu@intel.com>
To:     Jean-Philippe Brucker <jean-philippe@linaro.org>,
        Jacob Pan <jacob.jun.pan@linux.intel.com>
CC:     Vinod Koul <vkoul@kernel.org>, Dave Jiang <dave.jiang@intel.com>,
        <dmaengine@vger.kernel.org>,
        linux-kernel <linux-kernel@vger.kernel.org>,
        Alistair Popple <apopple@nvidia.com>,
        Joerg Roedel <joro@8bytes.org>,
        "Will Deacon" <will@kernel.org>,
        Robin Murphy <robin.murphy@arm.com>,
        "Lorenzo Stoakes" <lstoakes@gmail.com>,
        Christoph Hellwig <hch@infradead.org>, <iommu@lists.linux.dev>
References: <20230306163138.587484-1-fenghua.yu@intel.com>
 <20230306163138.587484-9-fenghua.yu@intel.com>
 <20230307084058.GA1330745@myrica>
 <156f9e28-e7d2-b443-1aa5-1f5447cb9bbd@intel.com>
In-Reply-To: <156f9e28-e7d2-b443-1aa5-1f5447cb9bbd@intel.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?WWVUdmVLMTNaNlV1L0dKWm8xVkVjbGtENmVSV3pkVUNHQUxYZ1BtbFI1YVlY?=
 =?utf-8?B?Z09ld3g5bnA4K3NTaUFvLy8vaVNCaVBWOUZTNzlGcFFFYmJuYm40Kyt4dUt3?=
 =?utf-8?B?YVZoQW1lL2xPUGV5Vjh4aGRuTUtsV0c2SmdCTUxCaktIWU5leEd0ckxtbU4z?=
 =?utf-8?B?WFYyQjkzeXNKZW40dWRNWUt3eVlpRWRTSlBwMmVBSUp1MTBCQ1VPT2ZEdHFN?=
 =?utf-8?B?NWFmck9lUG0vaXVpalFFRVRVcTgxRVRvQmFBbHM4SFpSOUFOZ0k0OUR1dWxF?=
 =?utf-8?B?SXdqZFJEZFdUQVR2QjFnQlRYajBrcHIyekdJY0dFV25KNW1lcThVbFlvNXAw?=
 =?utf-8?B?cWhqTGRoR0FVbTJiNTZaMExqYThtWlJYUmhrZ2NieFg4SDZXTnFKdnBZckJv?=
 =?utf-8?B?RkZwWnRUbC8yUVFZL0VzbmxnNzhOSmRjSzluYUdZcXBSR2llYTh2NkpYUk8w?=
 =?utf-8?B?aHRQNzBMUGxhR2xVa25RcFZHMzR4SDRoUWpsbjBLc2JXOVhkZ0ZPNzk0UXdv?=
 =?utf-8?B?ajVBSXFoSnFLQ0V4UThzTzUza3ExajBtUzM4empiVzA1ZXdwU3BXQ2s0cFBV?=
 =?utf-8?B?MFdiUHRUTlZMbm9WTzM2VkZOTG1TQWVRUVVFTkVDM2hXKzZtcmlOYjI2b1lP?=
 =?utf-8?B?MzZQY1ZweUdiSkdqSVRiR05Kd3FyaHFlZ1lsSks4NjdIMWJkbDUrN0gwVy9h?=
 =?utf-8?B?ZVNnc1BDbDhvaEVYNFNyQ2Y5ejFRM2llWFBGUFFXb3drVU5tQzkvNlc1anM3?=
 =?utf-8?B?aGVoeEhpeFJVL2V0WVh4NTh5c24yVkJQVFNydy9OV3k3K0ZsR2o2Y1g0eFVy?=
 =?utf-8?B?OTZ1OUIxM2NiZ051cTZFRXJNcnZqN3pxclp6Uk9WV2pSZ3htczRBaVdQS085?=
 =?utf-8?B?aVovbERJN1U4QlVRcFlDMmhTVGl1MEx3TE9uK2M5OVJFL1FvMVdLTFNDWWxT?=
 =?utf-8?B?bmZZamtoZmpzVDJaTFJuY0l3ZFV2SU16K2tLWDFCY1JJbW1JYnIvejNIYjcy?=
 =?utf-8?B?bzV6TUwxVzc4Ny9laXhIdlBXK3NHR2xXcmR2dkwxc3JGL3ozY0ZlVjlJNTlz?=
 =?utf-8?B?bmJ6N05tQmM0SFAxK2M2bFUrejZ3bUk4RStrN3FoVU43OFdWRk51TndZckNR?=
 =?utf-8?B?NDM5UlFmNHhQZ0VCSXdWQmtpSVRlQ2puSWUvVkx4RXdMcU1BSnFPR0hLOTFF?=
 =?utf-8?B?dnBDL1ZqSklpdkc0cnk1MkUvQjNvU3Y5bHFRdERFUDBMZ3RMSmtiVXRrMGR5?=
 =?utf-8?B?NUpFVzZCd0ltaU1sQVRhRlAra0hVVjA3S1crU1crUWhCUEZTRHVsaDNJUjgr?=
 =?utf-8?B?MjdSdGdJWDQzWWJ4WU1ReHZuRVRxbkdaM2oyMHVaazlzNTNEam9QOXIrWW5U?=
 =?utf-8?B?WTFycytTUktHYk4zV1h6dVpZa24vWXJVVUc1aUM3TGEyMWtabUtSYTVDR0dl?=
 =?utf-8?B?QkFPeFpIWXF1TXcvR2ZWZlpkd2UwMkllM1VPOWZtVWN3bmRnRmZJTVB6MUlw?=
 =?utf-8?B?OFJ3STQzZk5ocmM0T0pKSit2bCt4UFhDRHMwRFptQjkyc2xmdkkrdExxRklw?=
 =?utf-8?B?YkFtQ1dPKzVtN3BhdFVwWDAyRVRCWGpRcEgxU1RlbE5LUzVBS0R2RStsdjVv?=
 =?utf-8?B?bGRiWElKbXViSFRGUUtET0s1dVNqVUdjUjk1YWY5N0JqMmY1Q2ZNbXV6WFpG?=
 =?utf-8?B?a3ZER0dJcHAzQ244cXR1OU9xa0U0K0djckIrU09hbC8zMnI3TUZjWUpkK0cy?=
 =?utf-8?B?Zkxidyt1OGtuY2tKcFRxcDY2US9NSWduZGJUaDRUK05FaU1oazVqUVlpT3Uz?=
 =?utf-8?B?U05xcjcyNTlSWHJqS0piYnhRendEM2xLdEo4dkduNStWeDdIN3pZczltdml2?=
 =?utf-8?B?a0dMRWJaUzl1UWpmMEdiRWs0bVI2eDZzQmRtMEJFUTd0SVZOdVZ3SmFKdUJT?=
 =?utf-8?B?VHJDYXJwS2NaZnRJVk9wYktnWFpxZW5nOVdCdXZ4UG4vRTZFWFZYb0J0RHg5?=
 =?utf-8?B?cmovNHVmcUV2dnFSaU9iQ2cwVDUzNzFGSGE2SnpMMUhIaXRnZ0RWUnladEVR?=
 =?utf-8?B?WEFqYm96UnZObVdjSVViaWlYNEtXRTVRN2NTdXh4WFFacFdETXhVdmF2aitY?=
 =?utf-8?Q?OGyWMkZgH3J/u0XTfIQFq7jIB?=
X-MS-Exchange-CrossTenant-Network-Message-Id: e48d26d2-d753-4318-6a1e-08db22566db8
X-MS-Exchange-CrossTenant-AuthSource: IA1PR11MB6097.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Mar 2023 17:31:20.8421
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: rYxFuLVLJr73Ryi2bEn5udrJRYuuyLxnCZCnx4ijbMdiYR+UiNAQp5o1c/f9JRupiX0OsFSLJ0nAR9YHQsPqnA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR11MB6976
X-OriginatorOrg: intel.com
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi, Jean and Jacob,

On 3/7/23 08:33, Fenghua Yu wrote:
> Hi, Jean,
> 
> On 3/7/23 00:40, Jean-Philippe Brucker wrote:
>> Hi Fenghua,
>>
>> On Mon, Mar 06, 2023 at 08:31:30AM -0800, Fenghua Yu wrote:
>>> Define and export iommu_access_remote_vm() to allow IOMMU related
>>> drivers to access user address space by PASID.
>>>
>>> The IDXD driver would like to use it to write the user's completion
>>> record that the hardware device is not able to write to due to user
>>> page fault.
>>>
>>> Without the API, it's complex for IDXD driver to copy completion record
>>> to a process' fault address for two reasons:
>>> 1. access_remote_vm() is not exported and shouldn't be exported for
>>>     drivers because drivers may easily cause mm reference issue.
>>> 2. user frees fault address pages to trigger fault by IDXD device.
>>>
>>> The driver has to call iommu_sva_find(), kthread_use_mm(), re-implement
>>> majority of access_remote_vm() etc to access remote vm.
>>>
>>> This IOMMU specific API hides these details and provides a clean 
>>> interface
>>> for idxd driver and potentially other IOMMU related drivers.
>>>
>>> Suggested-by: Alistair Popple <apopple@nvidia.com>
>>> Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
>>> Cc: Joerg Roedel <joro@8bytes.org>
>>> Cc: Will Deacon <will@kernel.org>
>>> Cc: Robin Murphy <robin.murphy@arm.com>
>>> Cc: Alistair Popple <apopple@nvidia.com>
>>> Cc: Lorenzo Stoakes <lstoakes@gmail.com>
>>> Cc: Christoph Hellwig <hch@infradead.org>
>>> Cc: iommu@lists.linux.dev
>>> ---
>>> v2:
>>> - Define and export iommu_access_remote_vm() for IDXD driver to write
>>>    completion record to user address space. This change removes
>>>    patch 8 and 9 in v1 (Alistair Popple)
>>>
>>>   drivers/iommu/iommu-sva.c | 35 +++++++++++++++++++++++++++++++++++
>>>   include/linux/iommu.h     |  9 +++++++++
>>>   2 files changed, 44 insertions(+)
>>>
>>> diff --git a/drivers/iommu/iommu-sva.c b/drivers/iommu/iommu-sva.c
>>> index 24bf9b2b58aa..1d7a0aee58f7 100644
>>> --- a/drivers/iommu/iommu-sva.c
>>> +++ b/drivers/iommu/iommu-sva.c
>>> @@ -71,6 +71,41 @@ struct mm_struct *iommu_sva_find(ioasid_t pasid)
>>>   }
>>>   EXPORT_SYMBOL_GPL(iommu_sva_find);
>>> +/**
>>> + * iommu_access_remote_vm - access another process' address space by 
>>> PASID
>>> + * @pasid:    Process Address Space ID assigned to the mm
>>> + * @addr:    start address to access
>>> + * @buf:    source or destination buffer
>>> + * @len:    number of bytes to transfer
>>> + * @gup_flags:    flags modifying lookup behaviour
>>> + *
>>> + * Another process' address space is found by PASID. A reference on @mm
>>> + * is taken and released inside the function.
>>> + *
>>> + * Return: number of bytes copied from source to destination.
>>> + */
>>> +int iommu_access_remote_vm(ioasid_t pasid, unsigned long addr, void 
>>> *buf,
>>> +               int len, unsigned int gup_flags)
>>> +{
>>> +    struct mm_struct *mm;
>>> +    int copied;
>>> +
>>> +    mm = iommu_sva_find(pasid);
>>
>> The ability to find a mm by PASID is being removed, see
>> https://lore.kernel.org/linux-iommu/20230301235646.2692846-4-jacob.jun.pan@linux.intel.com/ 
>>
>>
> 
> Thank you very much for pointing out this.
> 
> I talked to Jacob just now. He will keep iommu_sva_find() function
> in his next version because this patch is still using the function. He 
> agrees that I can still call iommu_sva_find() in this patch.

Further comment from Jason confirms that iommu_sva_find() will be 
removed (https://lore.kernel.org/lkml/ZAjSsm4%2FPDRqViwa@nvidia.com/).

So cannot call iommu_sva_find() any more. Will maintain mm and find mm 
from PASID inside IDXD driver. And will implement accessing the remote 
mm inside IDXD driver although the implementation will have duplicate 
code as access_remote_vm().

Next version will only change IDXD driver code. There won't be IOMMU 
code change.

Thanks.

-Fenghua