by Christoph Hellwig

[permalink] [raw]

Subject: Re: [PATCH 17/18] dma-iommu: implement ->alloc_noncoherent

On Fri, Sep 25, 2020 at 06:46:22PM +0000, Tomasz Figa wrote:
> > +static void *iommu_dma_alloc_noncoherent(struct device *dev, size_t size,
> > + dma_addr_t *handle, enum dma_data_direction dir, gfp_t gfp)
> > +{
> > + if (!gfpflags_allow_blocking(gfp)) {
> > + struct page *page;
> > +
> > + page = dma_common_alloc_pages(dev, size, handle, dir, gfp);
> > + if (!page)
> > + return NULL;
> > + return page_address(page);
> > + }
> > +
> > + return iommu_dma_alloc_remap(dev, size, handle, gfp | __GFP_ZERO,
> > + PAGE_KERNEL, 0);
>
> iommu_dma_alloc_remap() makes use of the DMA_ATTR_ALLOC_SINGLE_PAGES attribute
> to optimize the allocations for devices which don't care about how contiguous
> the backing memory is. Do you think we could add an attrs argument to this
> function and pass it there?
>
> As ARM is being moved to the common iommu-dma layer as well, we'll probably
> make use of the argument to support the DMA_ATTR_NO_KERNEL_MAPPING attribute to
> conserve the vmalloc area.

We could probably at it. However I wonder why this is something the
drivers should care about. Isn't this really something that should
be a kernel-wide policy for a given system?

2020-09-26 15:38:26

by Tomasz Figa

[permalink] [raw]

Subject: Re: [PATCH 17/18] dma-iommu: implement ->alloc_noncoherent

On Sat, Sep 26, 2020 at 4:14 PM Christoph Hellwig <[email protected]> wrote:
>
> On Fri, Sep 25, 2020 at 06:46:22PM +0000, Tomasz Figa wrote:
> > > +static void *iommu_dma_alloc_noncoherent(struct device *dev, size_t size,
> > > + dma_addr_t *handle, enum dma_data_direction dir, gfp_t gfp)
> > > +{
> > > + if (!gfpflags_allow_blocking(gfp)) {
> > > + struct page *page;
> > > +
> > > + page = dma_common_alloc_pages(dev, size, handle, dir, gfp);
> > > + if (!page)
> > > + return NULL;
> > > + return page_address(page);
> > > + }
> > > +
> > > + return iommu_dma_alloc_remap(dev, size, handle, gfp | __GFP_ZERO,
> > > + PAGE_KERNEL, 0);
> >
> > iommu_dma_alloc_remap() makes use of the DMA_ATTR_ALLOC_SINGLE_PAGES attribute
> > to optimize the allocations for devices which don't care about how contiguous
> > the backing memory is. Do you think we could add an attrs argument to this
> > function and pass it there?
> >
> > As ARM is being moved to the common iommu-dma layer as well, we'll probably
> > make use of the argument to support the DMA_ATTR_NO_KERNEL_MAPPING attribute to
> > conserve the vmalloc area.
>
> We could probably at it. However I wonder why this is something the
> drivers should care about. Isn't this really something that should
> be a kernel-wide policy for a given system?

There are IOMMUs out there which support huge pages and those can
benefit *some* hardware depending on what kind of accesses they
perform, possibly on a per-buffer basis. At the same time, order > 0
allocations can be expensive, significantly affecting allocation
latency, so for devices which don't care about huge pages anyone would
prefer simple single-page allocations. Currently the drivers know the
best on whether the hardware they drive would care. There are some
decision factors listed in the documentation [1].

I can imagine cases where drivers could not be the best to decide
about this - for example, the workload could vary depending on the
userspace or a product decision regarding the performance vs
allocation latency, but we haven't seen such cases in practice yet.

[1] https://www.kernel.org/doc/html/latest/core-api/dma-attributes.html?highlight=dma_attr_alloc_single_pages#dma-attr-alloc-single-pages

Best regards,
Tomasz