Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) client-ip=2620:137:e000::3:5;
Message-ID: <1c3e0582-3177-4b05-b0e8-e292fa3f9ed9@intel.com>
Date:   Tue, 12 Dec 2023 11:44:22 +0800
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 3/3] vfio: Report PASID capability via VFIO_DEVICE_FEATURE
 ioctl
Content-Language: en-US
To:     "Tian, Kevin" <kevin.tian@intel.com>,
        Alex Williamson <alex.williamson@redhat.com>
CC:     "joro@8bytes.org" <joro@8bytes.org>,
        "jgg@nvidia.com" <jgg@nvidia.com>,
        "robin.murphy@arm.com" <robin.murphy@arm.com>,
        "baolu.lu@linux.intel.com" <baolu.lu@linux.intel.com>,
        "cohuck@redhat.com" <cohuck@redhat.com>,
        "eric.auger@redhat.com" <eric.auger@redhat.com>,
        "nicolinc@nvidia.com" <nicolinc@nvidia.com>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "mjrosato@linux.ibm.com" <mjrosato@linux.ibm.com>,
        "chao.p.peng@linux.intel.com" <chao.p.peng@linux.intel.com>,
        "yi.y.sun@linux.intel.com" <yi.y.sun@linux.intel.com>,
        "peterx@redhat.com" <peterx@redhat.com>,
        "jasowang@redhat.com" <jasowang@redhat.com>,
        "shameerali.kolothum.thodi@huawei.com" 
        <shameerali.kolothum.thodi@huawei.com>,
        "lulu@redhat.com" <lulu@redhat.com>,
        "suravee.suthikulpanit@amd.com" <suravee.suthikulpanit@amd.com>,
        "iommu@lists.linux.dev" <iommu@lists.linux.dev>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-kselftest@vger.kernel.org" <linux-kselftest@vger.kernel.org>,
        "Duan, Zhenzhong" <zhenzhong.duan@intel.com>,
        "joao.m.martins@oracle.com" <joao.m.martins@oracle.com>,
        "Zeng, Xin" <xin.zeng@intel.com>,
        "Zhao, Yan Y" <yan.y.zhao@intel.com>
References: <20231127063909.129153-1-yi.l.liu@intel.com>
 <20231127063909.129153-4-yi.l.liu@intel.com>
 <20231211110345.1b4526c6.alex.williamson@redhat.com>
 <BN9PR11MB5276DD7A14E938973DDEBEC18C8EA@BN9PR11MB5276.namprd11.prod.outlook.com>
From:   Yi Liu <yi.l.liu@intel.com>
In-Reply-To: <BN9PR11MB5276DD7A14E938973DDEBEC18C8EA@BN9PR11MB5276.namprd11.prod.outlook.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?bEphb3JGV1ZvM0tuK1ZKdDVLY2taeWRIKzBESFNwOXlTbHdsMXJjVkxUVTZx?=
 =?utf-8?B?QlZWMm1EVTlhQWFwaXFuSnByU3pTSXVySEdHcEVUZU9iN21CVjF4aEpUOUI2?=
 =?utf-8?B?bnlIQWllMWI2M00vd0J5eDNNUTcxRDVDNGorNGdQMGVmclgyYmpFMzJFdlZ4?=
 =?utf-8?B?eFV3R0Y3RDk3cGdyVGZsS0JYTDhHZ28wM1JhdDBYTVk1YjI4dkVWU2IyRE5m?=
 =?utf-8?B?QkdKYUdwYW5xVXVFNFNuVWwrTExxZHJ4dkpCZkE4TjJxRC96ZEZsaCtUbmpl?=
 =?utf-8?B?cG9wdW1GdzhLU1MxbTErNGZvRzRuNGF4ZGNvZ3dhUERndnhXa0szTkxyajVO?=
 =?utf-8?B?Y3R0a25nWFkxSlRjTlUxc2RPcDEwUU9PWXIzNU1jYVpLYmsrV1FyVXpWU2ow?=
 =?utf-8?B?aGkzYS9HL09jNkZtK2dvVTVmREZnRkRXdEhMaHFDaTFFelRvMG90UmFBVzht?=
 =?utf-8?B?U1pXQUI4eHFUY1B2ZkJFZVdyQUVBWEhMaEl5ZFhVdk1kWWlYeFhGNzZQeXk1?=
 =?utf-8?B?NVRwaUNZdVpnTzQ2c1FPMWFiQU9tZlZZaFQvcnMyU2IxMTdnSzI2Z3pFam9y?=
 =?utf-8?B?akxpVi9zOHRveUxtMDRBVG1NVVV6SEpkREIybEZsbktudDRnN2I0K1FWM0Ns?=
 =?utf-8?B?Ym5hN1NjcTU1WUo3VnZRbG9xQTYyRXYxS0gvcytSWXFmYjBUQjJrYm0rTzJH?=
 =?utf-8?B?QXpyalpldXByeU85UEhVejB2L0tHZnA0R1NYQmJ6YkU1TjRVWFQvRzAycTRj?=
 =?utf-8?B?NGxWNFRIYnBoQVpOSXF0K3pOSFFIR05IOG4raXdqbkwzMWVnb3Y1cFZNZjJV?=
 =?utf-8?B?V1l4VUthSXA1T3ZUaGNJU3k0TitZaDFIMERTMUtlZXI1Zmk4ZklpUEJxWnIr?=
 =?utf-8?B?bTBzTVFlR2c0dlc1OEZaaU11R2FJTHhiSklsQ2ZUakRua0MrQnQySWJqUklK?=
 =?utf-8?B?N20rN1NpZ1NuNWdseDlhRzE0dkVxajErUHRFS3ZVWHROWjk5SHRFVldYYjhn?=
 =?utf-8?B?bUNJemhpQ3BxN0E4eHNFOGhFZC9iQmpxd1g3aEF0RVQxcTZ4OFVRNnhTVHRH?=
 =?utf-8?B?ZEtWcHlQYmtSTDNVaURtMXZyaGlybHczT0Z3aGFFTFRyV1hub3B6S253NEVk?=
 =?utf-8?B?ck5xVHQ0R0E0R1dJM1dNaC9UTEZJTVoyOHJvYXBMbzlrR3YveEZtUlM1NFp6?=
 =?utf-8?B?ZFlzU2xoYjNseUwxV2U3b1lDT1hhV1FzWWRDanRXNlNjVTBTc0VMWVBCSmht?=
 =?utf-8?B?aXJKTHJSaVowV0wyK0lYaVhQNVRsdFNjRGVNWi9sTWtOVlpyK0ZkQzNVRms2?=
 =?utf-8?B?S0dRQkJGVzNzN1V6V1cwbHM4cnIrZFFPYVRzQU1hYkhWZW5NNU1kLzJwTyt6?=
 =?utf-8?B?aUpqVVd5dVFYejdsNFd4QmsvODBaaGtsWFRyUjVzamN2MVZsUmlMSTc5cXI3?=
 =?utf-8?B?SGN3T1Q3NTN5TzV3ZU13ay9NaWM2dHhkcWgzZDBXNE1wcHFhQlIzLzE1enZC?=
 =?utf-8?B?VnNTM2M4WHo2RkJhSmVqMVY4ZGtUUFZPSFlMVzJGejRIa3c0a3BhT3BCMUV2?=
 =?utf-8?B?QTF3Wm1KcXFiRGZIYmNkRndrMWtOUUNZSnNTdm5pZHVFK1A1SVV1bEpUbU9s?=
 =?utf-8?B?VzZJYWxEY0pHdjQzVnpuc2pscG45UGtBMTFhTzN4Z2pGdURmRzZpTVcyMFVM?=
 =?utf-8?B?ZG9QZW1pYm1JSVlDNDYvQWdjcXI2MEZZTWpkdGp4eHp6L0E2Z25ZL3FoQ1hp?=
 =?utf-8?B?ZEFqNXpBU0RsMC9HektWVE1LbmdXV1JFZWZwbHA0T1QrTzQ3OFAwSHM0YlBI?=
 =?utf-8?B?emlWQ3JoK3haelU5dGNOdUJNSTRQdStoNW52endkODkyUGw3R1VBbStPdHVw?=
 =?utf-8?B?bHd1SnN5USs1VDdodDV5WEE4ZFZzNVJiN3JSUFE4QURkc2VpbVZKY2w1M1NW?=
 =?utf-8?B?NVY5Yk5TekY0Q0dibTZQMC9kdnppOGdienpvTWhYMjZZQk9KZ3ZpU09oVTZn?=
 =?utf-8?B?UmJ6bkt0dEZWajlPTFBqcEI3c2NHcDBBdFFBbU9NL3YzWFNIQWFtc2ZYOGJp?=
 =?utf-8?B?enJxNnJtcGtjenkvUTNFdzNNUTcxbXRYcGo0a205Z2dqZGZpeTNleTVRQmNL?=
 =?utf-8?Q?0NXwrd4frk7+h2zKasMDp6OML?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 483eadb8-69c0-474c-d7e1-08dbfac445ce
X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB7529.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Dec 2023 03:41:49.8090
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: e4NvTbN/8uv8iZv5sZfUsRLEWwScCMjiNxQb0xeiH8W/IUngIxnefccI8SZGTKs5E9URVejrF6sS3Ha6pna3uA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR11MB4696
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE
	autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Mon, 11 Dec 2023 19:42:03 -0800 (PST)

On 2023/12/12 10:16, Tian, Kevin wrote:
>> From: Alex Williamson <alex.williamson@redhat.com>
>> Sent: Tuesday, December 12, 2023 2:04 AM
>>
>> On Sun, 26 Nov 2023 22:39:09 -0800
>> Yi Liu <yi.l.liu@intel.com> wrote:
>>
>>> This reports the PASID capability data to userspace via
>> VFIO_DEVICE_FEATURE,
>>> hence userspace could probe PASID capability by it. This is a bit different
>>> with other capabilities which are reported to userspace when the user
>> reads
>>> the device's PCI configuration space. There are two reasons for this.
>>>
>>>   - First, Qemu by default exposes all available PCI capabilities in vfio-pci
>>>     config space to the guest as read-only, so adding PASID capability in the
>>>     vfio-pci config space will make it exposed to the guest automatically while
>>>     an old Qemu doesn't really support it.
>>
>> Shouldn't we also be working on hiding the PASID capability in QEMU
>> ASAP?  This feature only allows QEMU to know PASID control is actually
>> available, not the guest.  Maybe we're hoping this is really only used
>> by VFs where there's no capability currently exposed to the guest?
> 
> We expect this to be used by both PF/VF. It doesn't make sense to have
> separate interfaces between them.
> 
> I'm not aware of that the PASID capability has been exported today. So
> yes we should fix QEMU asap. and also remove the line exposing it
> in vfio_pci_config.c.

Kernel side hides the PASID capability by setting its length as 0
in the below array. As a result, QEMU wont see it in the cap chain.
Do you mean we need to let QEMU always ignore it even if kernel side
does not hide it?

static const u16 pci_ext_cap_length[PCI_EXT_CAP_ID_MAX + 1] = {
	...
	[PCI_EXT_CAP_ID_PASID]  =       0,      /* not yet */
	...
};

So far, kernel is still hiding it.

> 
>>
>>>   - Second, PASID capability does not exit on VFs (instead shares the cap of
>>
>> s/exit/exist/
>>
>>>     the PF). Creating a virtual PASID capability in vfio-pci config space needs
>>>     to find a hole to place it, but doing so may require device specific
>>>     knowledge to avoid potential conflict with device specific registers like
>>>     hiden bits in VF config space. It's simpler by moving this burden to the
>>>     VMM instead of maintaining a quirk system in the kernel.
>>
>> This feels a bit like an incomplete solution though and we might
>> already posses device specific knowledge in the form of a variant
>> driver.  Should this feature structure include a flag + field that
>> could serve to generically indicate to the VMM a location for
>> implementing the PASID capability?  The default core implementation
>> might fill this only for PFs where clearly an emualted PASID capability
>> can overlap the physical capability.  Thanks,
>>
> 
> make sense

A location maybe not enough, may also need to know if any successive
cap, so that we can insert the capability into the cap chain.

-- 
Regards,
Yi Liu