Received-SPF: softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
Date:   Mon, 14 Feb 2022 10:02:36 -0400
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Joerg Roedel <joro@8bytes.org>
Cc:     Lu Baolu <baolu.lu@linux.intel.com>,
        Alex Williamson <alex.williamson@redhat.com>,
        Robin Murphy <robin.murphy@arm.com>,
        Christoph Hellwig <hch@infradead.org>,
        Kevin Tian <kevin.tian@intel.com>,
        Ashok Raj <ashok.raj@intel.com>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        Bjorn Helgaas <bhelgaas@google.com>,
        Will Deacon <will@kernel.org>,
        Dan Williams <dan.j.williams@intel.com>, rafael@kernel.org,
        Diana Craciun <diana.craciun@oss.nxp.com>,
        Cornelia Huck <cohuck@redhat.com>,
        Eric Auger <eric.auger@redhat.com>,
        Liu Yi L <yi.l.liu@intel.com>,
        Jacob jun Pan <jacob.jun.pan@intel.com>,
        Chaitanya Kulkarni <kch@nvidia.com>,
        Stuart Yoder <stuyoder@gmail.com>,
        Laurentiu Tudor <laurentiu.tudor@nxp.com>,
        Thierry Reding <thierry.reding@gmail.com>,
        David Airlie <airlied@linux.ie>,
        Daniel Vetter <daniel@ffwll.ch>,
        Jonathan Hunter <jonathanh@nvidia.com>,
        Li Yang <leoyang.li@nxp.com>,
        Dmitry Osipenko <digetx@gmail.com>,
        iommu@lists.linux-foundation.org, linux-pci@vger.kernel.org,
        kvm@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v1 5/8] iommu/amd: Use iommu_attach/detach_device()
Message-ID: <20220214140236.GC929467@nvidia.com>
References: <20220106022053.2406748-1-baolu.lu@linux.intel.com>
 <20220106022053.2406748-6-baolu.lu@linux.intel.com>
 <20220106143345.GC2328285@nvidia.com>
 <Ygo8iek2CwtPp2hj@8bytes.org>
 <20220214131544.GX4160@nvidia.com>
 <Ygpb6CxmTdUHiN50@8bytes.org>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <Ygpb6CxmTdUHiN50@8bytes.org>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?71+cAmYQtR9Ir+MR0PCUJTxBz6hQmgPSWjGNRsE2D1Bbh40RgF0uZr84P66A?=
 =?us-ascii?Q?1ZFbZyLznLwt3GhG480LdrJ9WTt2poz7AgGieBBB4Nt3vz7QB3PFOj3n1qAZ?=
 =?us-ascii?Q?n5h6LeQFirzSuZpq8ItN4Jnj5Ct3GCt+ff1Q4ZpUvue3QyKYPKaHChVTk0TE?=
 =?us-ascii?Q?W++QAqBTmUXssXzRRbsNdB4p9fYhx5O9z1FEQg9O1tg4vRe/aDOpra3lHMQh?=
 =?us-ascii?Q?6AFooNlElvAWCF/Acih2WP9tbzgY7hfoV+vU5zRfq4FvuvSD0ZE1nyVOlt5X?=
 =?us-ascii?Q?VreBlbOLmNfvZ1WGk8+wfNJwsJSdRcxV0bZyEGt0zM9un4iaCHHVKi0tpRsB?=
 =?us-ascii?Q?bItpcK6C+12swLP3V1RtSIUe4phokhIjMtit+MBy2OPjKEJu/cawUGW2W1fG?=
 =?us-ascii?Q?JCqdPc8KKmxKf6s4LumwPKRU2PGmIfLv8O7PsqkiECKfJvR3grFMDmHJhHQu?=
 =?us-ascii?Q?3jKOMDfgHpzCAk+V0+3V7pcZ2f1h3Z3pYK+bi+dJvJnjMoXMA1tsuGzhGeDY?=
 =?us-ascii?Q?ZezAJKocVBgljp9V2s8J5pu5Mu8GiPNGruVjrbp5PXIH8hKr1O6NXWJgeSdG?=
 =?us-ascii?Q?cJJYz9P6aWucRyZHZB40BFF0HpN9kMqOkYlhHtUoYnJ4NOp2nvu99qmyL30w?=
 =?us-ascii?Q?OCgOeF46STMBklDSYmbwRz2Trjy28jxPZZNwRhRukBc1UYfnneVJ67/LkDdf?=
 =?us-ascii?Q?hg/6UJG1GC5xTbYIWmnhkclVdn1oI7qGjSPEDFyCAR6yUnOaD5k+MrQZhIb2?=
 =?us-ascii?Q?+7vvdaBBv2Gnhaf+5/iMaA16I0QCtSkJr/mOSlykD494hfUj/QyZOQl33FNp?=
 =?us-ascii?Q?ej/97Csy3yyzI0S8QnXMhLuWUcUMCroBnksuUoCkaO6OPwhm1uadk3zEZRPM?=
 =?us-ascii?Q?ZeC2hGAOXn2CBp/Ax4HfQNxQkCJn5rPpleTlUDWZzIfyxme2lvfv18wPE04S?=
 =?us-ascii?Q?yjvjU3819Ya+63THLPjhK1Q7emq+74SmOn9HxYzI/FzENKDVV+3AnJb3rKR9?=
 =?us-ascii?Q?TsX1nPTndCTtRxtAHtkMo5Njqa0HwAJR2yf79DbevIaO3d+uExBPBXrMzVHc?=
 =?us-ascii?Q?e0IlegIRbbDgEpN6DBIo6mkDsdKAEXb6WrXDVPpAzgMrKF5eHKTrBZH+Astm?=
 =?us-ascii?Q?W9UoeaSiVpPIJibNsnnjDIx8X8WYd9ROxzYZIS2qtwM/gvc/dats8ElIBNxf?=
 =?us-ascii?Q?FFx7XicTJCMfatKnPQgcyYQ2FF+TyTunYwNl8IVdtq0bK7lWkwvFvSMWCIsT?=
 =?us-ascii?Q?MTfbx/tcZW/Pd0fMuVbXo+8obB4PWuA/M14sIE7ekfXgjGEtcytiHPxsUfSy?=
 =?us-ascii?Q?estJGR4AA+kIciN319k0Et782G9pn0b/Q1WPjsssvYWVjWZaFImB54SioIDd?=
 =?us-ascii?Q?Ns5DZHRTcCZ1yI6GbehFrX2LHiRoYq0BHXRlHeXpD4NmIbkkIuB1ngYNy3WB?=
 =?us-ascii?Q?qHiZ9hnAX568AVJjTVYEd7oRnXoMXm/mMI0mNKS7/oj12GTL+dh7P5jNr49b?=
 =?us-ascii?Q?CYYNwfyxKTA+vv1ePqSVIZLzip0FoBfKvkXBPTzXzzI3RAnfSKsr84vgwGQM?=
 =?us-ascii?Q?l7jmqnm9SFhRaIrKB2M=3D?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 44e4c1fa-5e6c-43a1-2c48-08d9efc2a7de
X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2022 14:02:37.0091
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 7sVy6m37oW1I3lDQSnKbjr9KyjF8WdqBAdqkOZRt7HrNTbGhU8QmVfaiivm/PH/i
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB3287
X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
	DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,RDNS_NONE,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE
	autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
	lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Mon, Feb 14, 2022 at 02:40:56PM +0100, Joerg Roedel wrote:
> On Mon, Feb 14, 2022 at 09:15:44AM -0400, Jason Gunthorpe wrote:
> > But how does the sound device know that this has been done to it?
> > 
> > eg how do we know the sound device hasn't been bound to VFIO or
> > something at this point?
> 
> The iommu_attach_group() call will fail when the group (which includes
> GPU and sound device) it not in its default-domain. So if VFIO attached
> the group to its own domain, there is a failure in this init function.

That works for VFIO, but it doesn't work for other in-kernel
drivers.. Is there something ensuring the group is only the GPU and
sound device? Is the GPU never an addin card?

I'd say the right way to code this after Lu's series to have both the
GPU and sound driver call iommu_attach_device() during their probe()'s
and specify the identity domain as the attaching domain.

That would be quite similar to how the Tegra drivers got arranged.

And then maybe someone could better guess what the "sound driver" is
since it would be marked with an iommu_attach_device() call..

Jason