Message-ID: <8209c172aaa415eace9dcd4b5675ef8506a34538.camel@linux.ibm.com>
Subject: Re: [PATCH v5 5/7] iommu/dma: Allow a single FQ in addition to
 per-CPU FQs
From:   Niklas Schnelle <schnelle@linux.ibm.com>
To:     Jason Gunthorpe <jgg@nvidia.com>,
        Robin Murphy <robin.murphy@arm.com>
Cc:     Joerg Roedel <joro@8bytes.org>, Will Deacon <will@kernel.org>,
        Wenjia Zhang <wenjia@linux.ibm.com>,
        Matthew Rosato <mjrosato@linux.ibm.com>,
        Gerd Bayer <gbayer@linux.ibm.com>,
        Pierre Morel <pmorel@linux.ibm.com>, iommu@lists.linux.dev,
        linux-s390@vger.kernel.org, borntraeger@linux.ibm.com,
        hca@linux.ibm.com, gor@linux.ibm.com,
        gerald.schaefer@linux.ibm.com, agordeev@linux.ibm.com,
        svens@linux.ibm.com, linux-kernel@vger.kernel.org,
        Julian Ruess <julianr@linux.ibm.com>
Date:   Mon, 30 Jan 2023 16:57:56 +0100
In-Reply-To: <Y9ficPosWtGqbDit@nvidia.com>
References: <20230124125037.3201345-1-schnelle@linux.ibm.com>
         <20230124125037.3201345-6-schnelle@linux.ibm.com>
         <1e016926-b965-3b71-07e1-1f63bc45f1a0@arm.com>
         <Y9ficPosWtGqbDit@nvidia.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.46.3 (3.46.3-1.fc37) 
MIME-Version: 1.0
Precedence: bulk

On Mon, 2023-01-30 at 11:29 -0400, Jason Gunthorpe wrote:
> On Mon, Jan 30, 2023 at 03:13:22PM +0000, Robin Murphy wrote:
>=20
> > Either way, the more I think about this the more I'm starting to agree =
that
> > adding more domain types for iommu-dma policy is a step in the wrong
> > direction. If I may, I'd like to fall back on the "or at least some def=
inite
> > internal flag" part of my original suggestion :)
>=20
> Yes please, lets try to remove IOMMU_DOMAIN_DMA, not add more :)
>=20
> At this point we should probably just sort of hackily add a ops flag
> to indicate single queue and when we fixup the policy logic we can
> make it a user selectable policy as well.
>=20
> Jason

Ok yeah I did like the policy idea you brought up a while back. If we
want to do the conversion to dma-iommu before that we do need an
interim solution though. So if I'm reading your ops flag idea right I
think it could even re-use much of the logic in this series and just
not introduce a new domain type.

I'm wondering if maybe instead of plain flag bits it makes more sense
to have a dma_iommu_options struct that contains the queue size and a
flags value which indicates whether a single or per-CPU queue is used.
Then I could add an iommu_ops callback that takes a device and a
pointer to the dma_iommu_options. That way we can still set the options
per device but don't need a whole extra domain type. This callback
would then just be called during initialization of the DMA-FQ domain
and not having the callback just leaves the defaults unchanged. Does
that go in the right direction?

Thanks,
Niklas