Received-SPF: pass (google.com: domain of linux-kernel+bounces-419-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Message-ID: <7c398efc-8a2f-479d-bcff-ded8cc1ef3d0@intel.com>
Date: Fri, 15 Dec 2023 12:01:19 +0800
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v7 1/3] iommufd: Add data structure for Intel VT-d stage-1
 cache invalidation
Content-Language: en-US
To: Nicolin Chen <nicolinc@nvidia.com>, "Tian, Kevin" <kevin.tian@intel.com>
CC: "joro@8bytes.org" <joro@8bytes.org>, "alex.williamson@redhat.com"
	<alex.williamson@redhat.com>, "jgg@nvidia.com" <jgg@nvidia.com>,
	"robin.murphy@arm.com" <robin.murphy@arm.com>, "baolu.lu@linux.intel.com"
	<baolu.lu@linux.intel.com>, "cohuck@redhat.com" <cohuck@redhat.com>,
	"eric.auger@redhat.com" <eric.auger@redhat.com>, "kvm@vger.kernel.org"
	<kvm@vger.kernel.org>, "mjrosato@linux.ibm.com" <mjrosato@linux.ibm.com>,
	"chao.p.peng@linux.intel.com" <chao.p.peng@linux.intel.com>,
	"yi.y.sun@linux.intel.com" <yi.y.sun@linux.intel.com>, "peterx@redhat.com"
	<peterx@redhat.com>, "jasowang@redhat.com" <jasowang@redhat.com>,
	"shameerali.kolothum.thodi@huawei.com"
	<shameerali.kolothum.thodi@huawei.com>, "lulu@redhat.com" <lulu@redhat.com>,
	"suravee.suthikulpanit@amd.com" <suravee.suthikulpanit@amd.com>,
	"iommu@lists.linux.dev" <iommu@lists.linux.dev>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-kselftest@vger.kernel.org" <linux-kselftest@vger.kernel.org>, "Duan,
 Zhenzhong" <zhenzhong.duan@intel.com>, "joao.m.martins@oracle.com"
	<joao.m.martins@oracle.com>, "Zeng, Xin" <xin.zeng@intel.com>, "Zhao, Yan Y"
	<yan.y.zhao@intel.com>
References: <20231117131816.24359-1-yi.l.liu@intel.com>
 <20231117131816.24359-2-yi.l.liu@intel.com>
 <c967e716-9112-4d1a-b6f7-9a005e28202d@intel.com>
 <BN9PR11MB5276D14D2A7FF60B41A6A7B48C93A@BN9PR11MB5276.namprd11.prod.outlook.com>
 <ZXu5whlIGfiq16wF@Asurada-Nvidia>
 <BN9PR11MB52766D7F774510E0181CC89B8C93A@BN9PR11MB5276.namprd11.prod.outlook.com>
 <ZXvI2IiXwwuHRE8V@Asurada-Nvidia>
From: Yi Liu <yi.l.liu@intel.com>
In-Reply-To: <ZXvI2IiXwwuHRE8V@Asurada-Nvidia>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?bTVNSjN5a21veEI5d2NkVVVIa0dmLzVKbHRuMGN3c3dwNCtEa2JrcTE2K3JX?=
 =?utf-8?B?VmRjdGtNTUwzK2xHcC9nenRySEVPQXF2S1ByNE82cXdmbFoyU2EyV0lxaHlI?=
 =?utf-8?B?WmhZMHVEUUFHNkR6WTRPQklQSk8zcHpUNkpOZ2ZlenZiV3NEOEZzRkJMdlBw?=
 =?utf-8?B?V05PbEsvcHZYd3NYQXQrbHJYbWI3S3AyR0tERTZWSlQ5Z2p1ejE0KzNTU2hP?=
 =?utf-8?B?dnowN2wrOHMvL1VXUDJ0L2cxR09YMStUK2pQbEdhUkdSd05YT21sZHBxT2Q4?=
 =?utf-8?B?SFJGVXBla0tIbE5vOXFxc0dmRjYyRzhkOGVHNUduanZlM0dmZmg3WWtZVWpW?=
 =?utf-8?B?TXV4VEo4Mmp5VmRCUFZvNUVCOVZ3NEFBNSsyWWNEbEpwMDhjU2FWU0MvSEdl?=
 =?utf-8?B?M1NiSkVXSStTQUk2ZG0zRE90NU9wTitwQ0pnQThZREtJV1hhdzJId20ybndl?=
 =?utf-8?B?TjZsUWp6RnYyRmRDaHZWV2JLT0FoNGhLMjViUW00emorb21kUmpwUGRGQnpL?=
 =?utf-8?B?ZXV4MDk5ZlArRGdCbjc2ZWUydmIybVorMlcxQWpwSkxWeFVKMURCRFgwek80?=
 =?utf-8?B?c1BrYUZ1T0Q5aXJTcCtqV3JwcldSQ0Q2Nm1NVUtoTlMvREllcUxwNmJKMUNL?=
 =?utf-8?B?eHBjUGdpZXh3MDNYd2o1djRWL3N4WnBEVy8rdENaSk9lUUlUVkdzZzJHd0FY?=
 =?utf-8?B?VVNiNTQxRE9QbHc3QzRvMzV6VDZ1bW9xbGdaSnpSTWJKU09JVGNoc2ZNcWRK?=
 =?utf-8?B?YWlTeHM5UnEvaW9weS9EQWJaK25WeEF6OG1ReEk3bERRYmcwMXBzWmZvU1FT?=
 =?utf-8?B?V1pNSkFMZFpCRWh6bFVvUGkwSEN0ZkE3VkpNWW5LVVk5Z25tcUdIWjZZMUxt?=
 =?utf-8?B?NVMyTHdZeFZ1dklSZ2JCb1RGYlpyNlVuUlBTQVhlVEVRM0dJeWFQTlRIK1Bo?=
 =?utf-8?B?aFpqOFZ5cXBKQjlxOHYxUGdqeGhCdTZrSEhrdVUvWjZja09ONEFML1BsNXho?=
 =?utf-8?B?eWJWaERPckp1MUNXVmJGdm03V0JhY1hZc01TbzM5eHFKWFdZTFpJTXV0RU04?=
 =?utf-8?B?VHZ2by9xN0YxZzhmQ0dudEY4ZHVBcmlhS2pVMlZqOS83c3VPVElsbGVFbnpS?=
 =?utf-8?B?TXM0blhBajVXNmlOOUN3SHlScVRudSt3UEdFbXU4TkVxd1lDL1hZUXM4VmRT?=
 =?utf-8?B?VTYrQkZxNkxYenEyNUE0TFBEQWVJY3llU0V3V3lCeHNaUVdYWGVFOVZ0ekJo?=
 =?utf-8?B?RzFDYnQ0aGlzdWVraElQcEhwREJBUHIyYzBoajFEK2dydmxsT1pJc002SmF3?=
 =?utf-8?B?amRXSmxERy9XWVJZTDF4ell6cWRJQzBER242bHJFNzFIRStxRDZyRlVVY1dX?=
 =?utf-8?B?aDBPK3g5OTNnVFBpcFVJWEFpUjk3QnZncWFMMW5CdGhxcDA0Q0E5dndiNE1o?=
 =?utf-8?B?cUNkUGxDQmtmSm40WnZxdWpwMWtUSXVSRGNCREV5ZUthcGRWZG13RllYdjdY?=
 =?utf-8?B?Skw1WGVoMmlWWVFGVnJ0ZzgwQ3hNZTU5Q01NamNmMlhVNXEwRnN1V24zZXNv?=
 =?utf-8?B?VU12SGxGOG9zRDN2RGtYL3Z2Lys3V1F2WkZ6REFxNGM4azFRNENpbmFtM09J?=
 =?utf-8?B?NEdVVnIyWXUyRzNmdU1heldYaTdBSDQ0Z3VLM3dIYlVsR3dtVjFpVmQxYld6?=
 =?utf-8?B?dU1QeXphU2k0VVJuamJ5ejVLYkdqZ01OaSsrOVpZQUhFRzlqTUIrUkpwMllm?=
 =?utf-8?B?c1JnMFJSa3BzemNsa01tSDRtRHBKb2xta2swOGNIK1RIdnh0elJmdFNOa09O?=
 =?utf-8?B?MmVxcXBVSEpRbE5GOENVak9SU2NaVzg1YW1UN3NwTU53a2NoMFZ3WkNiQ0Yz?=
 =?utf-8?B?YkR3MXprazJGNkNWTGM3VUp2cFg2VzFMT0ZUQ1VVVHU4cmR6aEFVMmEyYnFJ?=
 =?utf-8?B?S1gyOWpsLzQ5UUJORzZkdWtnR2V2L2NRdWoyVnA0T2JYMkpiMlRFUkthUE5h?=
 =?utf-8?B?VFRPTVkzOGV0UzhXMFh0RjRENDdCaUdxV25TY1BuLzBBYXptY0xQeStKd1RS?=
 =?utf-8?B?dXZqSjJqYkhudnRRUmdYWEZlUzlxWURab0h0Q1VhQmQ0M2w0WkNPaUVJZUdJ?=
 =?utf-8?Q?Tsjyv817d0WA3Hg0AZoxACwKf?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 4081d4c4-7a4c-485f-fd20-08dbfd221b49
X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB7529.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Dec 2023 03:58:33.7864
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: c7Tu+2x0wv7WQOAXMgvRN70SprRglQc2NXiPbUm3rXS2x3jN9nXGy1jTLIyhpKK37Hws4kuFxOS3oJrIzVJQBA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV3PR11MB8696
X-OriginatorOrg: intel.com

On 2023/12/15 11:32, Nicolin Chen wrote:
> On Fri, Dec 15, 2023 at 03:04:44AM +0000, Tian, Kevin wrote:
>>> From: Nicolin Chen <nicolinc@nvidia.com>
>>> Sent: Friday, December 15, 2023 10:28 AM
>>> On Fri, Dec 15, 2023 at 01:50:07AM +0000, Tian, Kevin wrote:
>>>>> From: Liu, Yi L <yi.l.liu@intel.com>
>>>>> Sent: Thursday, December 14, 2023 7:27 PM
>>>>>
>>>>> On 2023/11/17 21:18, Yi Liu wrote:> This adds the data structure for
>>>>> flushing iotlb for the nested domain
>>>>>
>>>>> +struct iommu_hwpt_vtd_s1_invalidate {
>>>>> +     __aligned_u64 addr;
>>>>> +     __aligned_u64 npages;
>>>>> +     __u32 flags;
>>>>> +     __u32 __reserved;
>>>>> +     __u32 error;
>>>>> +     __u32 dev_id;
>>>>> +};
>>>>>
>>>>> dev_id is used to report the failed device, userspace should be able to
>>> map
>>>>> it to a vRID, and inject it to VM as part of ITE/ICE error.
> 
>>>> and for this error reporting case what we actually require is the
>>>> reverse map i.e. pRID->vRID. Not sure whether we can leverage the
>>>> same RID mapping uAPI as for ARM/AMD but ignore viommu_id
>>>> and then store vRID under device_domain_info. a bit tricky on
>>>> life cycle management and also incompatible with SIOV...
>>>
>>> One thing that I am not very clear here: since both vRID and dev_id
>>> are given by the VMM, shouldn't it already know the mapping if the
>>> point is to translate (pRID->)dev_id->vRID?
>>>
>>
>> it's true for current Qemu.
>>
>> but there is plan to support Qemu accepting a fd passed by Libvirt.
>> In that case Qemu even doesn't see the sysfs path hence is not
>> aware of pRID. otherwise yes we could leave the translation to
>> VMM instead.
> 
> I think I misread Yi's narrative: dev_id is a working approach
> for VMM to convert to a vRID, while he is asking for a better
> alternative :)

In concept, dev_id works, but in reality we have problem to get a dev_id
for a given device in intel iommu driver, hence I'm asking for help here. :)

-- 
Regards,
Yi Liu