Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Tue, 23 Aug 2022 21:02:59 -0300
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Robin Murphy <robin.murphy@arm.com>
Cc:     Takashi Iwai <tiwai@suse.de>, Lu Baolu <baolu.lu@linux.intel.com>,
        Joerg Roedel <jroedel@suse.de>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        Bjorn Helgaas <bhelgaas@google.com>,
        Eric Auger <eric.auger@redhat.com>,
        regressions@lists.linux.dev, linux-kernel@vger.kernel.org,
        Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>,
        Felix Kuehling <Felix.Kuehling@amd.com>,
        amd-gfx@lists.freedesktop.org,
        Alex Deucher <alexander.deucher@amd.com>,
        Christian =?utf-8?B?S8O2bmln?= <christian.koenig@amd.com>,
        Pan@nvidia.com, Xinhui <Xinhui.Pan@amd.com>
Subject: Re: [REGRESSION 5.19.x] AMD HD-audio devices missing on 5.19
Message-ID: <20220824000259.GA4090@nvidia.com>
References: <874jy4cqok.wl-tiwai@suse.de>
 <20220823010021.GA5967@nvidia.com>
 <87h723sdde.wl-tiwai@suse.de>
 <87ilmjqj1f.wl-tiwai@suse.de>
 <20220823202824.GA4516@nvidia.com>
 <478c4aba-897f-7e08-1df7-4e296538db9c@arm.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <478c4aba-897f-7e08-1df7-4e296538db9c@arm.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Cvmh0X/IKSdJY36lgvi8rABhHItlLlr5QQ6hfi8U0MmUXF4sDy6NOU288bzX?=
 =?us-ascii?Q?iWuLarKUH3iebsxG5oJBrafto5e2q9bAprysQhc7a+zZresf6blDKfslV9Q8?=
 =?us-ascii?Q?kk4rvB5ksOfwUM6SaKmqBlp+39iJ30Kr5z7rv48kxg068tsk8sjSTLsNKMDk?=
 =?us-ascii?Q?ygm0zcn6au9StPLWxx77/ffwODSs3vYoka0R0WQjuM4fnPkRz+5fXoFtAQcq?=
 =?us-ascii?Q?4cJ6FUUgfFX8vuUXdHtBYCpNMes6wnfyqYZpMCSV6xs0CmSf/Zfv4bIJHFVn?=
 =?us-ascii?Q?XiYY0mztQPAApuKapYVJeEKksoAj1VJberM3FF6JYfU4ofpTCb4T050z57B2?=
 =?us-ascii?Q?ASwK1k2NS646/foGX65UkX4ntYyaK8/cnWI9yXiy1psSqiGaEvPkOHNgYP2I?=
 =?us-ascii?Q?99CFm1UCCIUEWzLtbb695d5MACuO7cd3HkJsudDRKKW6fNLNWbIB1K/DAaoj?=
 =?us-ascii?Q?TRnRQSdgyl/kvhxCsstw+r4ahymImUrCyznIx4BViPcknhIrztPbjcy0ZFU4?=
 =?us-ascii?Q?UoyVxflZZSmKZBtrESkW9Y9Zi58EvBlqBdkD/cE7lOQqXbC9z3jGVaudEKPq?=
 =?us-ascii?Q?UzAfZD27V7cEpChuHcWeRwJOW1/1HJ1/aoFrTEFYurb4SeRuaa9GQ0T0nGFm?=
 =?us-ascii?Q?ja6aQFT+fR21BKz1Pq4W+EQrO08vdecxTxN5ARQI7lqCbhTYpVrokUhbiTxe?=
 =?us-ascii?Q?BrseLOugcMrDFdkQImFVkHpxHHFYQ1rOdwUqzuRuDhu/LJQe7YdfROjM0utw?=
 =?us-ascii?Q?U2G9Oervt+UUxnmTnLnguv4sgsFTmdsEDSbRQEsj76xNBV7/YSOwNQbTFqby?=
 =?us-ascii?Q?sW9uPEQT5MDEwQsz5kfvp/asZDbbZVPbbRzx/9LjBGIqmgEnt/gJ21wAPH4w?=
 =?us-ascii?Q?IkEoPMuFO53booM37hB00nDi6Y64bWJ5L9YHiXM+9CnRMWaAsgb9RwUOqJUu?=
 =?us-ascii?Q?36B90DW/49h9boJ+bGaMZwieeMc0eaRIbKkgOnUhmasKMvF+PaKT2+4pInq5?=
 =?us-ascii?Q?fmz8jYqgXOYmpsSZvPYn5Us912W1kR5p3+2vZcTFPF881scKRRhjK8hIFi28?=
 =?us-ascii?Q?Du2OJZXDSf99t1GIgkAH8XfOYAXOrzV4tzPtfG3dpsG4ogxkd5iS0FYpOrAs?=
 =?us-ascii?Q?xsJppzPUuVfdQrfujRtbQngsDCgexoqf1uyIwGTg17WuSGZCNeGBftkJgZ4d?=
 =?us-ascii?Q?5bukqwovFHlCAOzGKuoRkmXNYkzlS+qmLGSaVnUs+09a2fNJW6Zx+698xSFf?=
 =?us-ascii?Q?HlwfmxSz186f4w2QsX8AtI9UgV58GMjVkMAfiWTXq439tzz2qcMkXjhAq7DB?=
 =?us-ascii?Q?2V9+/uF6Bzw8RcLpE0fnGHX0hyjHjqd03O1DJhX8YQigT6SjogAr2wzxPxTd?=
 =?us-ascii?Q?8EgcAvYSRa54py7EZDEG154gy7rorDCDyXAX6EetBxTkwvi9taUq9nOqimP1?=
 =?us-ascii?Q?AoPmAixXYkH8YY/bGMX0iGx/pQeIwrQUmMvC1O+CLX5sm3YWFOfrdkwEKC37?=
 =?us-ascii?Q?gRw4Ju2CT2ZOPXxm1JtcLTFsYA2mNix2JJMu74VWb/EvUw5B1yZUrYLJZeqF?=
 =?us-ascii?Q?g86nxfc0iF+laa8qUjLp+AqpsajbFNUfwJGj6odi?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 93fb510f-d5a2-4f80-88c3-08da85640256
X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Aug 2022 00:03:01.0670
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: iFIXSnwtnG3kFdzKGnoY500bF3YQczLBksV/YeErNR6bdYAr6k+A8uRoJbvQO/bX
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB3880
X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,
        T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=no autolearn_force=no
        version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Tue, Aug 23, 2022 at 10:01:57PM +0100, Robin Murphy wrote:

> > diff --git a/drivers/iommu/amd/iommu_v2.c b/drivers/iommu/amd/iommu_v2.c
> > index 696d5555be5794..6a1f02c62dffcc 100644
> > --- a/drivers/iommu/amd/iommu_v2.c
> > +++ b/drivers/iommu/amd/iommu_v2.c
> > @@ -777,6 +777,8 @@ int amd_iommu_init_device(struct pci_dev *pdev, int pasids)
> >   	if (dev_state->domain == NULL)
> >   		goto out_free_states;
> > +	/* See iommu_is_default_domain() */
> > +	dev_state->domain->type = IOMMU_DOMAIN_IDENTITY;
> >   	amd_iommu_domain_direct_map(dev_state->domain);
> 
> Same question as 6 months ago, apparently: allocating an unmanaged domain
> with a pagetable then sucking out the pagetable is silly enough, but if
> we're going to then also call it a proper identity domain, we should really
> just allocate an identity domain directly; but then why not just enable_v2
> on the identity domain that we know is already there courtesy of
> def_domain_type?

Yeah, nobody who knows this code answered that question either..

Looking at it a bit, I think this comment will start to be a problem:

	/*
	 * Save us all sanity checks whether devices already in the
	 * domain support IOMMUv2. Just force that the domain has no
	 * devices attached when it is switched into IOMMUv2 mode.
	 */
	ret = -EBUSY;
	if (domain->dev_cnt > 0 || domain->flags & PD_IOMMUV2_MASK)
		goto out;

Beacuse we should have dev_cnt != 0 on the existing identity domain at
this point - worse if the probe order is backwards the sound driver
may even already be running when we reach this.

Plus the challenge of undoing it when the PASID user goes away.

Overall I can see how it is easier and more logical to transition
between two domains. We already have good infrastructure for doing
that.

From a core perspective I don't have a real problem with iommu drivers
using multiple iommu_domains to manage their internal operations, eg
for different operating modes. But you are right that it should be
cleaner and directly allocate the special domains it needs. This would
be much more self-descriptive if it called a function 'allocate v2
identity domain', for instance.

I think it would also make sense for the core to provide some API to
change the default domain (ie dma API domain) of a group, and that
would be a more logical, and self explanatory, API for iommu drivers
to use than attach/detach. ie:

   iommu_change_default_domain(group, amd_identity_domain_v2):
   iommu_change_default_domain(group, amd_identity_domain_v1):

At least for this effort I wanted something simple enough to backport
that maybe doesn't need to be an expert in the amd iommu to write..
[I checked some more and the hack to change the type looks like it is
OK on the free path, so maybe this even works]

My general hope is that we can convince AMD to work on this once the
generic PASID & PRI series lands, as this entire private path to the
GPU driver and non-standard PASID handling all needs to be aligned
with the upcoming core code. When doing that work it would make sense
to tidy and modernize this better. I added a bunch of AMD people to
this thread to that end. It sure would be good if AMD participated in
that series since they are going to have to use it too.

https://lore.kernel.org/linux-iommu/20220817012024.3251276-1-baolu.lu@linux.intel.com/

Regards,
Jason