Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates
 216.228.117.160 as permitted sender) receiver=protection.outlook.com;
 client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C
Date:   Thu, 9 Feb 2023 14:22:26 -0800
From:   Nicolin Chen <nicolinc@nvidia.com>
To:     Jason Gunthorpe <jgg@nvidia.com>
CC:     Yi Liu <yi.l.liu@intel.com>,
        Matthew Rosato <mjrosato@linux.ibm.com>, <joro@8bytes.org>,
        <alex.williamson@redhat.com>, <kevin.tian@intel.com>,
        <robin.murphy@arm.com>, <cohuck@redhat.com>,
        <eric.auger@redhat.com>, <kvm@vger.kernel.org>,
        <chao.p.peng@linux.intel.com>, <yi.y.sun@linux.intel.com>,
        <peterx@redhat.com>, <jasowang@redhat.com>,
        <shameerali.kolothum.thodi@huawei.com>, <lulu@redhat.com>,
        <suravee.suthikulpanit@amd.com>, <iommu@lists.linux.dev>,
        <linux-kernel@vger.kernel.org>, <linux-kselftest@vger.kernel.org>,
        <baolu.lu@linux.intel.com>
Subject: Re: [PATCH 06/17] iommufd/hw_pagetable: Use domain_alloc_user op for
 domain allocation
Message-ID: <Y+VyIvKYmCdMG5i7@Asurada-Nvidia>
References: <20230209043153.14964-1-yi.l.liu@intel.com>
 <20230209043153.14964-7-yi.l.liu@intel.com>
 <25102c92-1831-be52-677d-60bbf2e11772@linux.ibm.com>
 <Y+U9QX4p5YX3/B3k@nvidia.com>
 <Y+VOw6dTnGapMm9L@Asurada-Nvidia>
 <Y+VaCV7DRR1+mLYY@nvidia.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <Y+VaCV7DRR1+mLYY@nvidia.com>
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Feb 2023 22:22:42.9383
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 2453d8aa-4a27-401d-6310-08db0aec29a5
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com]
X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT076.eop-nam11.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN9PR12MB5195
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Thu, Feb 09, 2023 at 04:39:37PM -0400, Jason Gunthorpe wrote:
> On Thu, Feb 09, 2023 at 11:51:31AM -0800, Nicolin Chen wrote:
> > On Thu, Feb 09, 2023 at 02:36:49PM -0400, Jason Gunthorpe wrote:
> > > On Thu, Feb 09, 2023 at 12:59:58PM -0500, Matthew Rosato wrote:
> > > > really should highlight that).  Otherwise, conditionally calling
> > > > iommu_domain_alloc(dev->bus) when !ops->domain_alloc_user (instead
> > > > of returning -EOPNOTSUPP) seems to restore the prior functionality
> > > > for me.
> > > 
> > > Yes, that is right if the input user data is 0 length or full of 0s
> > > then we should call the normal driver function
> > 
> > Maybe I am wrong, yet I recall that doing ops->domain_alloc_user
> > without a fallback was intentional to rule out drivers that don't
> > support IOMMUFD?
> 
> I think we moved away from that to the idea of using the dma_domain
> patch I suggested..
> 
> > To be backward-compatible and concern about SMMU, we can opt in
> > ops->domain_alloc_user upon availability and then add a fallback:
> > 
> > 	if ((!ops || !ops->domain_alloc_user) && user_data) {
> > 		rc = -EOPNOTSUPP;
> > 		goto out_abort;
> > 	}
> > 
> > 	if (ops->domain_alloc_user)
> > 		hwpt->domain = ops->domain_alloc_user(dev, NULL, NULL);
> > 	else
> > 		hwpt->domain = iommu_domain_alloc(dev->bus);
> > 	if (!hwpt->domain) {
> > 		rc = -ENOMEM;
> > 		goto out_abort;
> > 	}
> > 
> > Yet, even by doing so, this series having the PATCH 07/17 that
> > moves iopt_table_add_domain() would temporally break IOMMUFD on
> > ARM platforms, until we add the ops->domain_alloc_user to SMMU
> > drivers.
> 
> Drop patch 7 and 8
> 
> Change patch 12 so it has a unique flow to allocate and IOAS map a
> HWPT that does not try to share so much code with the existing flow.
> 
> The HWPT flow should always just call allocate and then map with no
> effort to attach first. This will fail on ARM SMMU at this point, and
> that is fine.
> 
> All the existing code should work exactly as it is now and not have
> any need to be changed.
> 
> Where things when wrong was trying to share
> "__iommufd_hw_pagetable_alloc", I think.
> 
> Don't try to consolidate at this point. Once all the drivers are
> updated then we could try to consolidate things.

Yea, I think that's the only way out for now. Though I am not
sure about other drivers yet, hopefully the SMMU driver(s) is
the last one that we need to update...

Thanks
Nic