From: Liam Mark <[email protected]>
Some devices have a memory map which contains gaps or holes.
In order for the device to have as much IOVA space as possible,
allow its driver to inform the DMA-IOMMU layer that it should
not allocate addresses from these holes.
Change-Id: I15bd1d313d889c2572d0eb2adecf6bebde3267f7
Signed-off-by: Isaac J. Manjarres <[email protected]>
---
drivers/iommu/dma-iommu.c | 28 ++++++++++++++++++++++++++++
include/linux/dma-iommu.h | 9 +++++++++
2 files changed, 37 insertions(+)
diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index a2e96a5..3b83e1a 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -368,6 +368,34 @@ static int iommu_dma_deferred_attach(struct device *dev,
return 0;
}
+/*
+ * Should be called prior to using dma-apis
+ */
+int iommu_dma_reserve_iova(struct device *dev, dma_addr_t base,
+ u64 size)
+{
+ struct iommu_domain *domain;
+ struct iommu_dma_cookie *cookie;
+ struct iova_domain *iovad;
+ unsigned long pfn_lo, pfn_hi;
+
+ domain = iommu_get_domain_for_dev(dev);
+ if (!domain || !domain->iova_cookie)
+ return -EINVAL;
+
+ cookie = domain->iova_cookie;
+ iovad = &cookie->iovad;
+
+ /* iova will be freed automatically by put_iova_domain() */
+ pfn_lo = iova_pfn(iovad, base);
+ pfn_hi = iova_pfn(iovad, base + size - 1);
+ if (!reserve_iova(iovad, pfn_lo, pfn_hi))
+ return -EINVAL;
+
+ return 0;
+}
+EXPORT_SYMBOL(iommu_dma_reserve_iova);
+
/**
* dma_info_to_prot - Translate DMA API directions and attributes to IOMMU API
* page flags.
diff --git a/include/linux/dma-iommu.h b/include/linux/dma-iommu.h
index 2112f21..79eef7c 100644
--- a/include/linux/dma-iommu.h
+++ b/include/linux/dma-iommu.h
@@ -37,6 +37,9 @@ void iommu_dma_compose_msi_msg(struct msi_desc *desc,
void iommu_dma_get_resv_regions(struct device *dev, struct list_head *list);
+int iommu_dma_reserve_iova(struct device *dev, dma_addr_t base,
+ u64 size);
+
#else /* CONFIG_IOMMU_DMA */
struct iommu_domain;
@@ -78,5 +81,11 @@ static inline void iommu_dma_get_resv_regions(struct device *dev, struct list_he
{
}
+static inline int iommu_dma_reserve_iova(struct device *dev, dma_addr_t base,
+ u64 size)
+{
+ return -ENODEV;
+}
+
#endif /* CONFIG_IOMMU_DMA */
#endif /* __DMA_IOMMU_H */
--
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project
On Fri, Feb 14, 2020 at 02:58:16PM -0800, Isaac J. Manjarres wrote:
> From: Liam Mark <[email protected]>
>
> Some devices have a memory map which contains gaps or holes.
> In order for the device to have as much IOVA space as possible,
> allow its driver to inform the DMA-IOMMU layer that it should
> not allocate addresses from these holes.
Layering violation. dma-iommu is the translation layer between the
DMA API and the IOMMU API. And calls into it from drivers performing
DMA mappings need to go through the DMA API (and be documented there).
On 17/02/2020 8:01 am, Christoph Hellwig wrote:
> On Fri, Feb 14, 2020 at 02:58:16PM -0800, Isaac J. Manjarres wrote:
>> From: Liam Mark <[email protected]>
>>
>> Some devices have a memory map which contains gaps or holes.
>> In order for the device to have as much IOVA space as possible,
>> allow its driver to inform the DMA-IOMMU layer that it should
>> not allocate addresses from these holes.
>
> Layering violation. dma-iommu is the translation layer between the
> DMA API and the IOMMU API. And calls into it from drivers performing
> DMA mappings need to go through the DMA API (and be documented there).
+1
More than that, though, we already have "holes in the address space"
support for the sake of PCI host bridge windows - assuming this is the
same kind of thing (i.e. the holes are between memory regions and other
resources in PA space, so are only relevant once address translation
comes into the picture), then this is IOMMU API level stuff, so even a
DMA API level interface would be inappropriate.
Robin.
On 2020-02-17 07:50, Robin Murphy wrote:
> On 17/02/2020 8:01 am, Christoph Hellwig wrote:
>> On Fri, Feb 14, 2020 at 02:58:16PM -0800, Isaac J. Manjarres wrote:
>>> From: Liam Mark <[email protected]>
>>>
>>> Some devices have a memory map which contains gaps or holes.
>>> In order for the device to have as much IOVA space as possible,
>>> allow its driver to inform the DMA-IOMMU layer that it should
>>> not allocate addresses from these holes.
>>
>> Layering violation. dma-iommu is the translation layer between the
>> DMA API and the IOMMU API. And calls into it from drivers performing
>> DMA mappings need to go through the DMA API (and be documented there).
>
> +1
>
> More than that, though, we already have "holes in the address space"
> support for the sake of PCI host bridge windows - assuming this is the
> same kind of thing (i.e. the holes are between memory regions and
> other resources in PA space, so are only relevant once address
> translation comes into the picture), then this is IOMMU API level
To make sure that we're on the same page, this support alludes to the
handling in
dma-iommu.c that reserves portions of the IOVA space for the PCI host
bridge windows,
correct? If so, then yes, this is similar.
> stuff, so even a DMA API level interface would be inappropriate.
Does this mean that the driver should be managing the IOVA space and
mappings for this device using the IOMMU API? If so, is the rationale
for this because the device driver can have the information of what IOVA
ranges can and cannot be used? Shouldn't there be a generic way of
informing an IOMMU driver about these reserved ranges? Perhaps through a
device tree property, instead of deferring this type of management to
the driver?
>
> Robin.
On Fri, Feb 14, 2020 at 02:58:16PM -0800, Isaac J. Manjarres wrote:
> From: Liam Mark <[email protected]>
>
> Some devices have a memory map which contains gaps or holes.
> In order for the device to have as much IOVA space as possible,
> allow its driver to inform the DMA-IOMMU layer that it should
> not allocate addresses from these holes.
>
> Change-Id: I15bd1d313d889c2572d0eb2adecf6bebde3267f7
> Signed-off-by: Isaac J. Manjarres <[email protected]>
Ideally this is something put into the IOMMU firmware table by the
platform firmware. If its not there, a quirk is the best way to handle
this.
Regards,
Joerg
On Tue, Feb 18, 2020 at 05:57:18PM -0800, [email protected] wrote:
> On 2020-02-17 07:50, Robin Murphy wrote:
> > On 17/02/2020 8:01 am, Christoph Hellwig wrote:
> > > On Fri, Feb 14, 2020 at 02:58:16PM -0800, Isaac J. Manjarres wrote:
> > > > From: Liam Mark <[email protected]>
> > > >
> > > > Some devices have a memory map which contains gaps or holes.
> > > > In order for the device to have as much IOVA space as possible,
> > > > allow its driver to inform the DMA-IOMMU layer that it should
> > > > not allocate addresses from these holes.
> > >
> > > Layering violation. dma-iommu is the translation layer between the
> > > DMA API and the IOMMU API. And calls into it from drivers performing
> > > DMA mappings need to go through the DMA API (and be documented there).
> >
> > +1
> >
> > More than that, though, we already have "holes in the address space"
> > support for the sake of PCI host bridge windows - assuming this is the
> > same kind of thing (i.e. the holes are between memory regions and
> > other resources in PA space, so are only relevant once address
> > translation comes into the picture), then this is IOMMU API level
> To make sure that we're on the same page, this support alludes to the
> handling in
> dma-iommu.c that reserves portions of the IOVA space for the PCI host bridge
> windows,
> correct? If so, then yes, this is similar.
> > stuff, so even a DMA API level interface would be inappropriate.
> Does this mean that the driver should be managing the IOVA space and
> mappings for this device using the IOMMU API? If so, is the rationale for
> this because the device driver can have the information of what IOVA ranges
> can and cannot be used? Shouldn't there be a generic way of informing an
> IOMMU driver about these reserved ranges? Perhaps through a device tree
> property, instead of deferring this type of management to the driver?
Before we dive into designing that, can you please clarify whether the
reserved IOVA range applies to all DMA masters mastering through a
particular SMMU, or whether it's just about one specific master? I was
assuming the former, but wanted to be sure.
Thanks,
Will
On 2020-02-19 03:15, Will Deacon wrote:
> On Tue, Feb 18, 2020 at 05:57:18PM -0800, [email protected] wrote:
>> On 2020-02-17 07:50, Robin Murphy wrote:
>> > On 17/02/2020 8:01 am, Christoph Hellwig wrote:
>> > > On Fri, Feb 14, 2020 at 02:58:16PM -0800, Isaac J. Manjarres wrote:
>> > > > From: Liam Mark <[email protected]>
>> > > >
>> > > > Some devices have a memory map which contains gaps or holes.
>> > > > In order for the device to have as much IOVA space as possible,
>> > > > allow its driver to inform the DMA-IOMMU layer that it should
>> > > > not allocate addresses from these holes.
>> > >
>> > > Layering violation. dma-iommu is the translation layer between the
>> > > DMA API and the IOMMU API. And calls into it from drivers performing
>> > > DMA mappings need to go through the DMA API (and be documented there).
>> >
>> > +1
>> >
>> > More than that, though, we already have "holes in the address space"
>> > support for the sake of PCI host bridge windows - assuming this is the
>> > same kind of thing (i.e. the holes are between memory regions and
>> > other resources in PA space, so are only relevant once address
>> > translation comes into the picture), then this is IOMMU API level
>> To make sure that we're on the same page, this support alludes to the
>> handling in
>> dma-iommu.c that reserves portions of the IOVA space for the PCI host
>> bridge
>> windows,
>> correct? If so, then yes, this is similar.
>> > stuff, so even a DMA API level interface would be inappropriate.
>> Does this mean that the driver should be managing the IOVA space and
>> mappings for this device using the IOMMU API? If so, is the rationale
>> for
>> this because the device driver can have the information of what IOVA
>> ranges
>> can and cannot be used? Shouldn't there be a generic way of informing
>> an
>> IOMMU driver about these reserved ranges? Perhaps through a device
>> tree
>> property, instead of deferring this type of management to the driver?
>
> Before we dive into designing that, can you please clarify whether the
> reserved IOVA range applies to all DMA masters mastering through a
> particular SMMU, or whether it's just about one specific master? I was
> assuming the former, but wanted to be sure.
>
This situation currently applies to one master.
> Thanks,
>
> Will
Thanks,
Isaac
On Wed, Feb 19, 2020 at 12:06:28PM -0800, [email protected] wrote:
> On 2020-02-19 03:15, Will Deacon wrote:
> > On Tue, Feb 18, 2020 at 05:57:18PM -0800, [email protected] wrote:
> > > Does this mean that the driver should be managing the IOVA space and
> > > mappings for this device using the IOMMU API? If so, is the
> > > rationale for
> > > this because the device driver can have the information of what IOVA
> > > ranges
> > > can and cannot be used? Shouldn't there be a generic way of
> > > informing an
> > > IOMMU driver about these reserved ranges? Perhaps through a device
> > > tree
> > > property, instead of deferring this type of management to the driver?
> >
> > Before we dive into designing that, can you please clarify whether the
> > reserved IOVA range applies to all DMA masters mastering through a
> > particular SMMU, or whether it's just about one specific master? I was
> > assuming the former, but wanted to be sure.
> >
> This situation currently applies to one master.
Interesting. Is it problematic if the range is reserved for all masters
sharing that SMMU?
Will
On 02/20 08:45 am, Will Deacon wrote:
> On Wed, Feb 19, 2020 at 12:06:28PM -0800, [email protected] wrote:
> > On 2020-02-19 03:15, Will Deacon wrote:
> > > On Tue, Feb 18, 2020 at 05:57:18PM -0800, [email protected] wrote:
> > > > Does this mean that the driver should be managing the IOVA space and
> > > > mappings for this device using the IOMMU API? If so, is the
> > > > rationale for
> > > > this because the device driver can have the information of what IOVA
> > > > ranges
> > > > can and cannot be used? Shouldn't there be a generic way of
> > > > informing an
> > > > IOMMU driver about these reserved ranges? Perhaps through a device
> > > > tree
> > > > property, instead of deferring this type of management to the driver?
> > >
> > > Before we dive into designing that, can you please clarify whether the
> > > reserved IOVA range applies to all DMA masters mastering through a
> > > particular SMMU, or whether it's just about one specific master? I was
> > > assuming the former, but wanted to be sure.
> > >
> > This situation currently applies to one master.
>
> Interesting. Is it problematic if the range is reserved for all masters
> sharing that SMMU?
>
Yes, that would be an overkill. It certainly is useful and in
some cases quite helpful to not waste that range of IOVA space for
other masters on the same SMMU that can use it.
--
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project