Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Thu, 22 Sep 2022 12:49:28 -0300
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Lu Baolu <baolu.lu@linux.intel.com>
Cc:     Joerg Roedel <joro@8bytes.org>,
        Christoph Hellwig <hch@infradead.org>,
        Bjorn Helgaas <bhelgaas@google.com>,
        Kevin Tian <kevin.tian@intel.com>,
        Ashok Raj <ashok.raj@intel.com>, Will Deacon <will@kernel.org>,
        Robin Murphy <robin.murphy@arm.com>,
        Jean-Philippe Brucker <jean-philippe@linaro.com>,
        Dave Jiang <dave.jiang@intel.com>,
        Fenghua Yu <fenghua.yu@intel.com>,
        Vinod Koul <vkoul@kernel.org>,
        Eric Auger <eric.auger@redhat.com>,
        Liu Yi L <yi.l.liu@intel.com>,
        Jacob jun Pan <jacob.jun.pan@intel.com>,
        Zhangfei Gao <zhangfei.gao@linaro.org>,
        Zhu Tony <tony.zhu@intel.com>, iommu@lists.linux.dev,
        linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v13 07/13] iommu/vt-d: Add SVA domain support
Message-ID: <YyyECCbmfsaDpDgJ@nvidia.com>
References: <20220906124458.46461-1-baolu.lu@linux.intel.com>
 <20220906124458.46461-8-baolu.lu@linux.intel.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20220906124458.46461-8-baolu.lu@linux.intel.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?8RNsruZQ/hwTrCC8YkuFVaL8KPwJHrz8loOxW3AvLvmZWg+roRyxeufY1ME7?=
 =?us-ascii?Q?wFUQrmlb1CLRzYDivY7mOhkGbbpIiu+2AP6QNasjzTOn5w8NpRQKOP6lrvbk?=
 =?us-ascii?Q?TQZeJVr2t9caKZsOPnBaJXoU9TAQj514iTHBPxH0pjzkpuWqJ4IGirYe9/07?=
 =?us-ascii?Q?LoklPvN6nRxg9fK0wKZDCGeTgbfOzIPzOleChMI0W61A7Yhk0VmhjT1JWEQs?=
 =?us-ascii?Q?/38X3iIjC3ix605CnHOtFqE9ifOfLbvArf4gcgtJreWuyM+Z72hcGRaD46FX?=
 =?us-ascii?Q?DKUIpzGTDdWfh7u2h8n49O4z4XEN21y/M4DY1ioX/aB+YB8pI5DHlFvqr3y+?=
 =?us-ascii?Q?fkG9/IK29id464PRiu8jP2fnAGw1l61td9wHfo85e+eKG6ElV/LHDswbxZfa?=
 =?us-ascii?Q?qCb0hgJWJ4vvekEuw2QbbtIfpPzNsb16lct9kcFMT55bVGyS3ib8vrIXF0I9?=
 =?us-ascii?Q?dQWuktaGInLLse61Hj1q9dLKTs5/50TW0zm4zuDPO75f0VPohXRYpZN39sWg?=
 =?us-ascii?Q?73mGjQxhDA/3VdN428m1QNzcis4yo2jltPi1fMfj7GRhT1BqYXu1bglElWce?=
 =?us-ascii?Q?ijFoC/IoaCBPgGYADGkIeN4mz5BT35afg/R1FSk4JR7b4BZCN5Zj9FXK7Yj+?=
 =?us-ascii?Q?D4qJtrfvSMT4yjkX2uzwyrhTqCGSW8kzFxnY0sxAXnKGH3gfISF0H91Vt7b9?=
 =?us-ascii?Q?umNymswiIg5bKVQHTH8r9XZKPl8QLC3tPg5zhIHa3/kPiCE6k/QaDct7qevc?=
 =?us-ascii?Q?ZLhhCfygmw66RTINLjiJu30IN/EmJD/qjeu4vSnrheQOTp+0sVPdib+2IcvI?=
 =?us-ascii?Q?dIuKoFiR0ZbnEvP+GzN149875juGAzlk1iejXrBPXSP++m9jjtpUavEsOYun?=
 =?us-ascii?Q?kqFkslV70LOI0Ac1IBinPYiJOwFtx1rcIvVKpb1mXLk+Ime5H5qCQxZYor/9?=
 =?us-ascii?Q?iDLn+G6mZ/cusKZ3DjtFdSNgn9HSk4AlUnWZjbq3WZp/St8YdO6vXyptIxNd?=
 =?us-ascii?Q?F7oFU+1CvzBWvApSR1HIsKmdyoB0LuUhFzo+z1eFt2K31dkhF4yrcYyAsn6V?=
 =?us-ascii?Q?lGdYYmIidth94aRsHQqmk/ZkLcLRhWg00rpb/ByY+yt4TVctKfnff+ObGZud?=
 =?us-ascii?Q?AntVJD46btABqBKJ8tGztTtKdy5S98cQi/2e+ytmw+UvZ4WwUm/xVFOPswrT?=
 =?us-ascii?Q?BRLfa9ET00YkGC/sXi/lBxYfOBHuoulFsljnLe2d/pEcsOex3/S90X2BhrvM?=
 =?us-ascii?Q?72/7prFRDF90pA75eyaj9pDZycO/2n7YvWlnEsCJsBZ0P+QkPX/lOgKkDPKa?=
 =?us-ascii?Q?odQ/fcB5r5sEAkE+LGMQHaaBXLqNFI9xqb/rmjM6XN9k596WEfYfcfcxY9oS?=
 =?us-ascii?Q?V60b9L4L9JOvGdgruAmDrjEsI7WryK9jxp4rpgwPE4zMJZQxL36jVZqn2E1Y?=
 =?us-ascii?Q?SxT3pvs54BaGfjNJ1cnzxcE9Ew+RFiDHGXBTAnAsTEB+RieJoQJoRDLB1Mno?=
 =?us-ascii?Q?xFiE/PWARPdEzbOr9fgDJU0hU8W73MYT4i0xHFQi9pxEyvFFKw9Oc/kesUoD?=
 =?us-ascii?Q?K2guJfKcWCW/TWN0U5CiqwHrxCdLsvtHt2KLn87r?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7515b2d1-5ff6-4a68-8199-08da9cb208dc
X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2022 15:49:29.4533
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: LqoiqVwHsm0m6xxPerh8diQcga2WoTVt3X5yizn6qf2rUrp2E0Ephzyv4tSyEJ1+
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR12MB6080
X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE
        autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Tue, Sep 06, 2022 at 08:44:52PM +0800, Lu Baolu wrote:
> +static void intel_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid)
> +{
> +	struct intel_iommu *iommu = device_to_iommu(dev, NULL, NULL);
> +	struct iommu_domain *domain;
> +
> +	/* Domain type specific cleanup: */
> +	domain = iommu_get_domain_for_dev_pasid(dev, pasid, 0);
> +	if (domain) {
> +		switch (domain->type) {
> +		case IOMMU_DOMAIN_SVA:
> +			intel_svm_remove_dev_pasid(dev, pasid);
> +			break;
> +		default:
> +			/* should never reach here */
> +			WARN_ON(1);
> +			break;

This is eventually going to need a lot more cleaning up to split out
the PASID from the SVM stuff.

SVA should *only* be a set of predefined handlers (in the core code!)
for the generic PRI mechanism, it shouldn't be entangled deeply into
PASID or the drivers like this.

When we get done with this, the flow should have the core code attach
a SVA domain to a PASID with PRI enabled and the core code should
supply a generic PRI implementation that does the mmu_notifier
stuff.

Also, stuff like this:

				/* We mandate that no page faults may be outstanding
				 * for the PASID when intel_svm_unbind_mm() is called.
				 * If that is not obeyed, subtle errors will happen.
				 * Let's make them less subtle... */

Are going to be problematic for VFIO as well. In a VFIO world the
entire RID and its entire PASID table has to be owned by VFIO and
never shared - so these sequencing issues should be solvable.

But this is all for further series..

Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>

Jason