Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) client-ip=23.128.96.31;
Message-ID: <8c05763d-1668-4b99-af35-b43a34e966c2@intel.com>
Date:   Mon, 11 Dec 2023 15:53:40 +0800
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 2/6] iommufd: Add IOMMU_HWPT_INVALIDATE
Content-Language: en-US
To:     Jason Gunthorpe <jgg@nvidia.com>,
        "Tian, Kevin" <kevin.tian@intel.com>
CC:     "joro@8bytes.org" <joro@8bytes.org>,
        "alex.williamson@redhat.com" <alex.williamson@redhat.com>,
        "robin.murphy@arm.com" <robin.murphy@arm.com>,
        "baolu.lu@linux.intel.com" <baolu.lu@linux.intel.com>,
        "cohuck@redhat.com" <cohuck@redhat.com>,
        "eric.auger@redhat.com" <eric.auger@redhat.com>,
        "nicolinc@nvidia.com" <nicolinc@nvidia.com>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "mjrosato@linux.ibm.com" <mjrosato@linux.ibm.com>,
        "chao.p.peng@linux.intel.com" <chao.p.peng@linux.intel.com>,
        "yi.y.sun@linux.intel.com" <yi.y.sun@linux.intel.com>,
        "peterx@redhat.com" <peterx@redhat.com>,
        "jasowang@redhat.com" <jasowang@redhat.com>,
        "shameerali.kolothum.thodi@huawei.com" 
        <shameerali.kolothum.thodi@huawei.com>,
        "lulu@redhat.com" <lulu@redhat.com>,
        "suravee.suthikulpanit@amd.com" <suravee.suthikulpanit@amd.com>,
        "iommu@lists.linux.dev" <iommu@lists.linux.dev>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-kselftest@vger.kernel.org" <linux-kselftest@vger.kernel.org>,
        "Duan, Zhenzhong" <zhenzhong.duan@intel.com>,
        "joao.m.martins@oracle.com" <joao.m.martins@oracle.com>,
        "Zeng, Xin" <xin.zeng@intel.com>,
        "Zhao, Yan Y" <yan.y.zhao@intel.com>
References: <20231117130717.19875-1-yi.l.liu@intel.com>
 <20231117130717.19875-3-yi.l.liu@intel.com>
 <112383df-3ea3-475f-963f-5c53232a1bf4@intel.com>
 <BN9PR11MB5276283CAEAB7A24871B4B188C8BA@BN9PR11MB5276.namprd11.prod.outlook.com>
 <20231207144208.GJ2692119@nvidia.com>
From:   Yi Liu <yi.l.liu@intel.com>
In-Reply-To: <20231207144208.GJ2692119@nvidia.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ekNINW5uNmxZYUo3T0M2dE1RNnlaNjlHZCt0SzNlb0p1cC82amxVQzVGSzFT?=
 =?utf-8?B?Q1o1cDR4djZmbERXUTE3dVFDRVJ4QXhYSE13c3BaZTlTVFVmcjNmMi94SEg0?=
 =?utf-8?B?QWlTSWRVUVJpaTJBeFVKWE1rTVlOY1dmbm9xN24yRGdOS3RzZDVLajdsMFRL?=
 =?utf-8?B?bVJOcWhMamhWcDJwdjdRcEx1T3NIdmZyS04yWlpJSEpyVEFqKy9xNXlRZjVK?=
 =?utf-8?B?RE1NS1hXODNuVkVJL0NmZi9mTHI1V2xwanRGcjFQWkRlN2dxM1hZS2xOeHFk?=
 =?utf-8?B?eUZHNFR5TFozdnJYSElpLzhTTzA0TWY4SjBjRWkyRkFoSDlqUDFrU1VPUmxS?=
 =?utf-8?B?RUJvRGNUMCtncWphUXZyTWJoUkZralMyL1hzNW5JcGZXWTVDWVlXMkF6eEV4?=
 =?utf-8?B?RzAxQ1p6c1cxMHVvbCt4ck5Od25uUEpxcUUxTVdSL3pqMzV6K0lGdFhZV3RR?=
 =?utf-8?B?VFlHWmVXK05zS1l4TXg1OVFUUXZFSXE5bkZGZndnZTd1SGhwOGJhUEJ3SnRw?=
 =?utf-8?B?TDZqa2pDRVkxU2ZQSlE4eWVwejhuRUFTTDB2c05oVHVuK3pKaVFFS21oNlp6?=
 =?utf-8?B?TTNCRnJxVGJYcEJGMmVPYkduTVRVQ0RCc3hiNEFKWm8wNjh3VENoWml2Ynhu?=
 =?utf-8?B?WVlkSFp6MTVrdlhhQ3FPOW00eGJTV1hsdS85V2FHWXZrNlZ6c0Fnb212b09Y?=
 =?utf-8?B?VVc0YmZmM1dWbkFLbW9CZ0ZPS1ZuSFBtNVNpYVFUQkVWeWMySDZDSktiYU1N?=
 =?utf-8?B?VWxWSFlKYnJYMUFZeE00NW1IcDhzM0hLOUNaNnhSbnF0SlBvaFd2UDFTeHRk?=
 =?utf-8?B?eHpzWTJIbDYyN25peGYwU0MzQ3gycWMwa2l5ZVZleG9LbXdITXdWNk1aM1c4?=
 =?utf-8?B?N1V4WklqUnhWcTg2V285dCtKZUhPVk13akNseitXVGlSS1BQd3JqSUgxS1VY?=
 =?utf-8?B?Nzg2Tk9kM05mWnFvTEFBQzVObjNWZkJUajgrSTNUK3duSHdMYmdROTlCdmU1?=
 =?utf-8?B?MzRySHAwejMyYnZiQVBXdHk4V2xQSGZ1dmV3KzdQTWJzZTJxMXhtZnBwSy9u?=
 =?utf-8?B?UWI5OHpGZExjdzhEZFFEaE1NK3o3N3hGOFNiVDExMngvNW1JalQ2RC9XYm1L?=
 =?utf-8?B?THlVV21PR0o3SHNrOTMzdG5MdzFiNW5iempFb3B6Q0FONklJbk1zUlI5cUtO?=
 =?utf-8?B?OEUvZS9CZDZNc0FFRmx0cERoRHBidkYxdVdYY2E5U2huMEZEWDZvRmdjenJI?=
 =?utf-8?B?Qng4bHc4cDUvZEFEWDNRelpsTUxnZlFsTU45L0hPWFFGdGhxZzBZajBieDhY?=
 =?utf-8?B?c0ZaM2JoL3hDdzBuNVJpeG9xRk1MTkRNV2dwMklzWWVzNW8rS0VRc09IYldC?=
 =?utf-8?B?SE8reDJHR3NnZ1U0N2cvcjUybmVUK2FMWXFtSEEyVnFtbHpRbENaQkdTbkFJ?=
 =?utf-8?B?OHp4b3ViemNFMGN4MTdVVUVYeTZERkZobzZxUjdQVzViTi9Ka2hzRmxWazdN?=
 =?utf-8?B?d3pSWUFSRlZJc3hrdHprMXloWGtOZkJHeTZpaDU3RjVFYlZNWWRGbHdkcW4x?=
 =?utf-8?B?OElqWE1nNXRLN2hpaFl6UWlPN3FSb2V2TnRZeHB5bmNQYWtLSXFKRGJSSmJJ?=
 =?utf-8?B?OGpROUdaYmVZdzFkM3AzdWhYOWM5c2RVV0t4RVNDNzFjNjlzem1OcDlNbHZz?=
 =?utf-8?B?QkZwWjhnWHMyZ3RvdjNwUjJHTkR1MytLVFdpb1dZWTF5R2J6TFg5aGtFd3BM?=
 =?utf-8?B?MnZ2ZG9MYXg4RnRZR0RaWkFLbzc5eTNONWpTYmxrWmt1VGtKUGdDbXRGd2ZZ?=
 =?utf-8?B?NEcvTDM3akh5NFVDTWlVMitaWjhaMVZXcEdSNVZ1T3YxUFdPWk8wZE5EbkhB?=
 =?utf-8?B?OHFBSHNYb3FNYXI3MmxueENJODYzb0xLSGtIbW44Rk9xNzg0a2RlYm5NaHNn?=
 =?utf-8?B?MlJldHd4bEhISzNvdXhNTENQWGNhVFdkZXhjaEZLZ3AzYjgwTVJ5ek9YMit4?=
 =?utf-8?B?aitUTjlwT1JWMzE4YjBtU0dBWlBHMnREQ2R6RlhUNDBVdWxWcjFKNTNUUzhJ?=
 =?utf-8?B?V0RQYXJWNEtORUd4Vzc4ZmxnNFJQRmdMZnBmWjlkRkJ5N2F6QU5zNlovSjh0?=
 =?utf-8?Q?Cs6Y4lSJ3PV3e33bt2S9nRLZw?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 632767f8-25ae-461a-e147-08dbfa1deee2
X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB7529.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Dec 2023 07:51:07.5107
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: cKHGz70ET2wQIQJOyiebx/qKm9R7qK1LXDUDq32keiwKwbf547udFmOGks9rCC2j3r1HpFbk08vVyOnMQ//A7A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR11MB7199
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE
	autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Sun, 10 Dec 2023 23:51:34 -0800 (PST)

On 2023/12/7 22:42, Jason Gunthorpe wrote:
> On Thu, Dec 07, 2023 at 09:04:00AM +0000, Tian, Kevin wrote:
>>> From: Liu, Yi L <yi.l.liu@intel.com>
>>> Sent: Thursday, December 7, 2023 2:59 PM
>>>
>>> On 2023/11/17 21:07, Yi Liu wrote:
>>>> @@ -613,4 +614,38 @@ struct iommu_hwpt_get_dirty_bitmap {
>>>>    #define IOMMU_HWPT_GET_DIRTY_BITMAP _IO(IOMMUFD_TYPE, \
>>>>
>>> 	IOMMUFD_CMD_HWPT_GET_DIRTY_BITMAP)
>>>>
>>>> +/**
>>>> + * struct iommu_hwpt_invalidate - ioctl(IOMMU_HWPT_INVALIDATE)
>>>> + * @size: sizeof(struct iommu_hwpt_invalidate)
>>>> + * @hwpt_id: HWPT ID of a nested HWPT for cache invalidation
>>>> + * @reqs_uptr: User pointer to an array having @req_num of cache
>>> invalidation
>>>> + *             requests. The request entries in the array are of fixed width
>>>> + *             @req_len, and contain a user data structure for invalidation
>>>> + *             request specific to the given hardware page table.
>>>> + * @req_type: One of enum iommu_hwpt_data_type, defining the data
>>> type of all
>>>> + *            the entries in the invalidation request array. It should suit
>>>> + *            with the data_type passed per the allocation of the hwpt pointed
>>>> + *            by @hwpt_id.
>>>
>>> @Jason and Kevin,
>>>
>>> Here a check with you two. I had a conversation with Nic on the definition
>>> of req_type here. It was added to support potential multiple kinds of cache
>>> invalidation data types for a invalidating cache for a single hwpt type[1].
>>> But we defined it as reusing the hwpt_data_type. In this way, it is not
>>> able to support the potential case in[1]. is it? Shall we define a separate
>>> enum for invalidation data types? And how can we let user know the
>>> available invalidation data types for a hwpt type? Any idea?
>>>
>>> [1] https://lore.kernel.org/linux-
>>> iommu/20231018163720.GA3952@nvidia.com/
>>>
>>
>>  From that thread Jason mentioned to make the invalidation format
>> part of domain allocation. If that is the direction to go then there
>> won't be multiple invalidation formats per hwpt. The user should
>> create multiple hwpt's per invalidation format (though mixing
>> formats in one virtual platform is very unlikely)?
> 
> I think we could do either, but I have a vauge cleanness preference
> that the enums are just different? That would follow a pretty typical
> pattern for a structure tag to reflect the content of the structure.

Is this still following the direction to make the invalidation format
part of domain allocation?

-- 
Regards,
Yi Liu