Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) client-ip=2620:137:e000::3:5;
Date:   Tue, 12 Dec 2023 11:31:30 -0400
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     "Tian, Kevin" <kevin.tian@intel.com>
Cc:     "Liu, Yi L" <yi.l.liu@intel.com>,
        "joro@8bytes.org" <joro@8bytes.org>,
        "alex.williamson@redhat.com" <alex.williamson@redhat.com>,
        "robin.murphy@arm.com" <robin.murphy@arm.com>,
        "baolu.lu@linux.intel.com" <baolu.lu@linux.intel.com>,
        "cohuck@redhat.com" <cohuck@redhat.com>,
        "eric.auger@redhat.com" <eric.auger@redhat.com>,
        "nicolinc@nvidia.com" <nicolinc@nvidia.com>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "mjrosato@linux.ibm.com" <mjrosato@linux.ibm.com>,
        "chao.p.peng@linux.intel.com" <chao.p.peng@linux.intel.com>,
        "yi.y.sun@linux.intel.com" <yi.y.sun@linux.intel.com>,
        "peterx@redhat.com" <peterx@redhat.com>,
        "jasowang@redhat.com" <jasowang@redhat.com>,
        "shameerali.kolothum.thodi@huawei.com" 
        <shameerali.kolothum.thodi@huawei.com>,
        "lulu@redhat.com" <lulu@redhat.com>,
        "suravee.suthikulpanit@amd.com" <suravee.suthikulpanit@amd.com>,
        "iommu@lists.linux.dev" <iommu@lists.linux.dev>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-kselftest@vger.kernel.org" <linux-kselftest@vger.kernel.org>,
        "Duan, Zhenzhong" <zhenzhong.duan@intel.com>,
        "joao.m.martins@oracle.com" <joao.m.martins@oracle.com>,
        "Zeng, Xin" <xin.zeng@intel.com>,
        "Zhao, Yan Y" <yan.y.zhao@intel.com>
Subject: Re: [PATCH 3/3] vfio: Report PASID capability via
 VFIO_DEVICE_FEATURE ioctl
Message-ID: <20231212153130.GK3014157@nvidia.com>
References: <20231127063909.129153-1-yi.l.liu@intel.com>
 <20231127063909.129153-4-yi.l.liu@intel.com>
 <BN9PR11MB527639DBE4C433542F351F6D8C8BA@BN9PR11MB5276.namprd11.prod.outlook.com>
 <0bdae2ca-a200-4db1-a016-059730d1545e@intel.com>
 <BN9PR11MB52763C75E3D638B722CE63A78C8EA@BN9PR11MB5276.namprd11.prod.outlook.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <BN9PR11MB52763C75E3D638B722CE63A78C8EA@BN9PR11MB5276.namprd11.prod.outlook.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?p6fKD28jH8t1XDlmusErUHrezMT/tyXYMqivc36fjpeKwougPgjA5YnwZqJm?=
 =?us-ascii?Q?lYxCHoD2SczyL4uBYNHR/vlGLXGuSw64ZXeNE/Bsk3VwwORQMRG8oSXfZZ8J?=
 =?us-ascii?Q?e7/dhhUTSEXQqdtLXJdVA1SdZ1m9ABXRPqHg6dRPrKmPQTS4718jU0asZfg+?=
 =?us-ascii?Q?Mcqsbl86R9PKgrVmSFPW8lC0TKQ4+GdzoaZ6MrMBy0jhP0lGEiUHKXGW3SFu?=
 =?us-ascii?Q?UkHM0ilorlzaIl5/uJX7UpZ6VAmB38SHLV0pX0RmCAXX9uIuSZ7cyrf+85U/?=
 =?us-ascii?Q?36I6rv9Ev3dt/TKjUJytG3XjMs9QvTbNpcqIVyU9cXpsoO5i6dsDrTKFnUmV?=
 =?us-ascii?Q?yfeIXcr8MygojbZ5rgNJ9LZ7nUW8IroKBvMwhDYXRy4XKxKkeTyuLug0wnho?=
 =?us-ascii?Q?yqQ9uPoQsaQ4g40wp2JDehuY1g5v1oC8RDnp53xg3QYhiw/sbcfldtHCdDLh?=
 =?us-ascii?Q?iY2yUf33msxo9VthwGTRJkWDiIhEtKOqdlmDvYSyoE3DzvlT5zPrlGOqsEE5?=
 =?us-ascii?Q?yfh/eMLio47va9HZydD1mp7JY5hkc5+f5YPoKe6O8S0qITFtCnVcmHWb+6bC?=
 =?us-ascii?Q?W9HDfx/PrlXaxArS5EwLUvyJlBz6xa+TyMQ9k0jYjsKFBqpUj8wAq5Heux5c?=
 =?us-ascii?Q?Vw0Xq4WS2sLfURfK9xcaeh1uh5zY+n+C3ATCadSXUx0RysotI5aleIK8MMr4?=
 =?us-ascii?Q?vrOeiHmTCSUFfJMyMenpmGsqeyibaT2L8Xn/rG7KX0DZL5TuWSPjXQZhMT95?=
 =?us-ascii?Q?mPWhoXRcEPbQrKZsTU9QTjEdcIoX3ve5OdcPsKQKIbQadDkYZjKa73AkN27m?=
 =?us-ascii?Q?U+x5XOZlaFQBxjant2swbut46QmgSSdfsVqRZYscoXZk+8E1SRJVLbdDlOnM?=
 =?us-ascii?Q?3ucL+bOLpYmWuuRH7MBONlisd0f0lVRstSCmtoPLN5TlGAe5Uzh5X30phNgn?=
 =?us-ascii?Q?+In1/ir9g2ag4r1rZdENt5TD24RaGcv0+k7H/bWwXsCTp+btPB1rE748/u9j?=
 =?us-ascii?Q?KdhIctMCnJoDfroLAtn4RLQTLQRK+DdfyUv8QfpxPccq4lJAL1Z9jPuwW/m/?=
 =?us-ascii?Q?7Hh2799zKyVUgP9IuHdpJSnWtREWjd38tlAKt0j6xiPLb175Be8v1Ik0VLy3?=
 =?us-ascii?Q?uBBeJegthycRzvbmRd2GPYKGms4ziu2xfPoBo3c/8kS5jFhNtFekxCM7Y7Zr?=
 =?us-ascii?Q?/rRHwvIbMs/jhEkf9FVvWPbJbjbOlZnfcA/ULZ8sI1GX3EWMZ+M9eYr6dpIC?=
 =?us-ascii?Q?09v9HIpT9GOgrJ+80iUQB0AZL4DHSJA9sEvn/qNcFNkvcyYhitDJ9cr75ejv?=
 =?us-ascii?Q?a+wvlb5+SkB9RYavirY9SFNU78jrLjnRomF4FcFFy18bqKmtN/YDvYg1j+XH?=
 =?us-ascii?Q?9ZFM582suYC4EUO3tdD7nc9uIBw1+F6l2nJzn2V6BgvhWo6VsiUuUIs8DdWM?=
 =?us-ascii?Q?X/Q92rk34DZ7rQwCWVazY8vXWJQlR02QB0m1rU2ul6A9xA1R76Ld+tDyHFEt?=
 =?us-ascii?Q?goqvhpmn/DZHTua0ecYdJEF/BCzMxssA3zKxLZuRmOoB0tSpSk4aRVlIzjWM?=
 =?us-ascii?Q?xyRGdMtw8w2D/1lXyhDBQLzikgWSyPWoo9r7ekns?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1e9e61bf-394a-4479-6dfc-08dbfb276a9f
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Dec 2023 15:31:31.5682
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: R0F89YniDgzm33h0gJP1QBDL3Q3/TGlerhmR8D74Ex+wAmEWzYU6oT1NQEu5aLqf
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB8612
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE
	autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Tue, 12 Dec 2023 07:31:45 -0800 (PST)

On Tue, Dec 12, 2023 at 02:20:01AM +0000, Tian, Kevin wrote:
> > From: Liu, Yi L <yi.l.liu@intel.com>
> > Sent: Monday, December 11, 2023 4:08 PM
> > 
> > On 2023/12/7 16:47, Tian, Kevin wrote:
> > >> From: Liu, Yi L <yi.l.liu@intel.com>
> > >> Sent: Monday, November 27, 2023 2:39 PM
> > >>
> > >> +static int vfio_pci_core_feature_pasid(struct vfio_device *device, u32
> > flags,
> > >> +				       struct vfio_device_feature_pasid __user
> > >> *arg,
> > >> +				       size_t argsz)
> > >> +{
> > >> +	struct vfio_pci_core_device *vdev =
> > >> +		container_of(device, struct vfio_pci_core_device, vdev);
> > >> +	struct vfio_device_feature_pasid pasid = { 0 };
> > >> +	struct pci_dev *pdev = vdev->pdev;
> > >> +	u32 capabilities = 0;
> > >> +	int ret;
> > >> +
> > >> +	/* We do not support SET of the PASID capability */
> > >
> > > this line alone is meaningless. Please explain the reason e.g. due to
> > > no PASID capability per VF...
> > 
> > sure. I think the major reason is we don't allow userspace to change the
> > PASID configuration. is it?
> 
> if only PF it's still possible to develop a model allowing userspace to
> change.

More importantly the primary purpose of setting the PASID width is
because of the physical properties of the IOMMU HW.

IOMMU HW that supports virtualization should do so in a way that the
PASID with can be globally set to some value the hypervisor is aware
the HW can decode in all cases.

The VM should have no way to make the HW ignore (vs check for zero)
upper bits of the PASID that would require the physical PASID bits to
be reduced.

So we should never allow programming of this, VMM just fakes it and
ignores sets.

Similar argument for enable, IOMMU HW supporting virtualization should
always be able to decode PASID and reject PASID TLPs if the VM hasn't
configured the vIOMMU to decode them. The purpose of the disable bit
is to accommodate IOMMU HW that cannot decode the PASID TLP at all and
would become confused.

Jason