Date:   Thu, 9 Feb 2023 16:39:37 -0400
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Nicolin Chen <nicolinc@nvidia.com>
Cc:     Yi Liu <yi.l.liu@intel.com>,
        Matthew Rosato <mjrosato@linux.ibm.com>, joro@8bytes.org,
        alex.williamson@redhat.com, kevin.tian@intel.com,
        robin.murphy@arm.com, cohuck@redhat.com, eric.auger@redhat.com,
        kvm@vger.kernel.org, chao.p.peng@linux.intel.com,
        yi.y.sun@linux.intel.com, peterx@redhat.com, jasowang@redhat.com,
        shameerali.kolothum.thodi@huawei.com, lulu@redhat.com,
        suravee.suthikulpanit@amd.com, iommu@lists.linux.dev,
        linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org,
        baolu.lu@linux.intel.com
Subject: Re: [PATCH 06/17] iommufd/hw_pagetable: Use domain_alloc_user op for
 domain allocation
Message-ID: <Y+VaCV7DRR1+mLYY@nvidia.com>
References: <20230209043153.14964-1-yi.l.liu@intel.com>
 <20230209043153.14964-7-yi.l.liu@intel.com>
 <25102c92-1831-be52-677d-60bbf2e11772@linux.ibm.com>
 <Y+U9QX4p5YX3/B3k@nvidia.com>
 <Y+VOw6dTnGapMm9L@Asurada-Nvidia>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <Y+VOw6dTnGapMm9L@Asurada-Nvidia>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?fgGm/6R/6jT9TbLryd1i5gyiCurpgBhrnAKZcJ8bISwBFHtqTH11mUgYSAht?=
 =?us-ascii?Q?jg4xbRv1xj3DzX/aKe3VkvveYFDHeVQ33XlL95+8fikGGME+2WI30+Y6G2+B?=
 =?us-ascii?Q?Utt39dN/kTOWbJLFyhg5h0bcyAwfVN2tcMrrAqenLJi6LMFYY2ZUVIpNH+4n?=
 =?us-ascii?Q?CZMcPDvVdrWlgy90HCxW/si40WyihPox4iP0sQ9bcwJ4oBrlciS6ptu/1Qdz?=
 =?us-ascii?Q?Tp1XEQuvtflbBj+xwhJxB1m1lT1ipZcyh8AKqxRop0ykaprX9GM09zTiWpOq?=
 =?us-ascii?Q?T9MiCkF7PPUv/Dz90E6JkU38BFGLiP+CAnIeMhTJnGmlu8BIPG+DRAWMSg+j?=
 =?us-ascii?Q?WajKCfM2wlg+NdNcMdp/MJzZOFY45AHmQCjJyeJWOU/7PQiaIHp6YxUjJqCX?=
 =?us-ascii?Q?m0VXjjnBYCAueI5+Q2FWu389iYXK3SGv8w1i+z4Nvpnujl2zWnfpW44J+UZJ?=
 =?us-ascii?Q?/0raaGh79x4Cd/DmWDDtHl3Xk9C0d9gslWdGzZl0FC1eXcb6hbNFfUVYzWEf?=
 =?us-ascii?Q?V6FAJ9HVA6nbWDY1VpBy6A2ML8IGe3xqr4aTqKTz4BblfieEOvvTZ4JGjttf?=
 =?us-ascii?Q?PL3JZxYhLLn5dz3jZAhIGi0jE4wtjPcIRbovvZMe+2eCPv0YmiKqPWQlvGEo?=
 =?us-ascii?Q?ezSvMIBNtIAffDCcUFlxxEI3xrbsZRQwk/MF1zmmMWSAFpCdr1IDQ2rhMBw9?=
 =?us-ascii?Q?i5JYxD4iyE/hWhS2oIGQqppp196rsDNocQ13ZYMidTLj1M3HxxDPxCwPGqC/?=
 =?us-ascii?Q?ONGtetO+6yUmPeUckzTOaEC+LFO4pyFS14QXr+ofn2XBIWh7J8BVs4oguAaS?=
 =?us-ascii?Q?ZWD6j3YhuLKIIqa9Vsd9sfuZLBwrMWoC8TX3Db5oZnHrXc8KZ5EifSzU6qJJ?=
 =?us-ascii?Q?NyM1C3UejJ3ZtEduzi/1Q465Wiv4C+v0n5o+AAOXuY+Zzdt7NeMp0EslKvV7?=
 =?us-ascii?Q?HfQcajTKBzZ+d9KL11rbI17hQ8GeOpDXY+NweatvnIh6hUfck7RLR+Ruj4id?=
 =?us-ascii?Q?uEnsx42kiI+4eWIgsUrHIfiz6DX/eBJJihuHHFb29JrQidgdgD7jyPf1xPxx?=
 =?us-ascii?Q?fysxOj9ybN5DVvCIgEVcW7vs5wbTBRL7LNSAKblf/8OdG8ghelQXGtzVd86Z?=
 =?us-ascii?Q?UrNiMTIu6qWDO/AqzjoooNnehGJV6GMJiIEiFOiDqLXrMoXh/EC86fTWaBt9?=
 =?us-ascii?Q?5QOyM3sI9qeQAcVlSUV7LUuJbclzk+wArRtjX7eTgYZLwV+QU9W9cuTb5ldL?=
 =?us-ascii?Q?h6w/7Dun7urtfkX9vMNuf/tjMoco8CuDFQf4aqUtFMkzl9Q20hAFp5iP8+NW?=
 =?us-ascii?Q?vZ0r+U1ZuOvguweYNBQp/YakHWfc7672ElpXrx5PVl8YniGP4N3xXJ0r3nfb?=
 =?us-ascii?Q?DoNcAl+ZSfsBSFDooXixMh4Qm396Mcniwv6deeVco+igrMrfoRJipeSATAtA?=
 =?us-ascii?Q?DrzWqzo4gQ87OBv46n5Il2Za6QDd4xG/ulfA4xFH9JMOuDdkzUPuAqhDDAfi?=
 =?us-ascii?Q?ecX3adex0ZIkHejxomXW1iMqzPjAe9oH+TxDuyKtMrJC5IOYLqr0WnbJ+fYI?=
 =?us-ascii?Q?wZG9y+Jqb0EQUHvNvxhpBXmF+GXbldzwMm0xMxRx?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 99ed5932-7b12-4138-739b-08db0addc37f
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Feb 2023 20:39:38.9060
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: VpIAJZWWqK1q7uqKM4PGHnhh/ReviyYExCPsFCXxmXJrAOuADYef/AwJv2KPR+pJ
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR12MB6004
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Thu, Feb 09, 2023 at 11:51:31AM -0800, Nicolin Chen wrote:
> On Thu, Feb 09, 2023 at 02:36:49PM -0400, Jason Gunthorpe wrote:
> > On Thu, Feb 09, 2023 at 12:59:58PM -0500, Matthew Rosato wrote:
> > > really should highlight that).  Otherwise, conditionally calling
> > > iommu_domain_alloc(dev->bus) when !ops->domain_alloc_user (instead
> > > of returning -EOPNOTSUPP) seems to restore the prior functionality
> > > for me.
> > 
> > Yes, that is right if the input user data is 0 length or full of 0s
> > then we should call the normal driver function
> 
> Maybe I am wrong, yet I recall that doing ops->domain_alloc_user
> without a fallback was intentional to rule out drivers that don't
> support IOMMUFD?

I think we moved away from that to the idea of using the dma_domain
patch I suggested..

> To be backward-compatible and concern about SMMU, we can opt in
> ops->domain_alloc_user upon availability and then add a fallback:
> 
> 	if ((!ops || !ops->domain_alloc_user) && user_data) {
> 		rc = -EOPNOTSUPP;
> 		goto out_abort;
> 	}
> 
> 	if (ops->domain_alloc_user)
> 		hwpt->domain = ops->domain_alloc_user(dev, NULL, NULL);
> 	else
> 		hwpt->domain = iommu_domain_alloc(dev->bus);
> 	if (!hwpt->domain) {
> 		rc = -ENOMEM;
> 		goto out_abort;
> 	}
> 
> Yet, even by doing so, this series having the PATCH 07/17 that
> moves iopt_table_add_domain() would temporally break IOMMUFD on
> ARM platforms, until we add the ops->domain_alloc_user to SMMU
> drivers.

Drop patch 7 and 8

Change patch 12 so it has a unique flow to allocate and IOAS map a
HWPT that does not try to share so much code with the existing flow.

The HWPT flow should always just call allocate and then map with no
effort to attach first. This will fail on ARM SMMU at this point, and
that is fine.

All the existing code should work exactly as it is now and not have
any need to be changed.

Where things when wrong was trying to share
"__iommufd_hw_pagetable_alloc", I think.

Don't try to consolidate at this point. Once all the drivers are
updated then we could try to consolidate things.

Jason