2010-08-19 15:20:08

by Marin Mitov

[permalink] [raw]
Subject: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

Hi all,

struct device contains a member: struct dma_coherent_mem *dma_mem;
to hold information for a piece of memory declared dma-coherent.
Alternatively the same member could also be used to hold preallocated
dma-coherent memory for latter per-device use.

This tric is already used in drivers/staging/dt3155v4l.c
dt3155_alloc_coherent()/dt3155_free_coherent()

Here proposed for general use by popular demand from video4linux folks.
Helps for videobuf-dma-contig framework.

Signed-off-by: Marin Mitov <[email protected]>

======================================================================
--- a/drivers/base/dma-coherent.c 2010-08-19 15:50:42.000000000 +0300
+++ b/drivers/base/dma-coherent.c 2010-08-19 17:27:56.000000000 +0300
@@ -93,6 +93,83 @@ void *dma_mark_declared_memory_occupied(
EXPORT_SYMBOL(dma_mark_declared_memory_occupied);

/**
+ * dma_reserve_coherent_memory() - reserve coherent memory for per-device use
+ *
+ * @dev: device from which we allocate memory
+ * @size: size of requested memory area in bytes
+ * @flags: same as in dma_declare_coherent_memory()
+ *
+ * This function reserves coherent memory allocating it early (during probe())
+ * to support latter allocations from per-device coherent memory pools.
+ * For a given device one could use either dma_declare_coherent_memory() or
+ * dma_reserve_coherent_memory(), but not both, becase the result of these
+ * functions is stored in a single struct device member - dma_mem
+ *
+ * Returns DMA_MEMORY_MAP on success, or 0 if failed.
+ * (same as dma_declare_coherent_memory()
+ */
+int dma_reserve_coherent_memory(struct device *dev, size_t size, int flags)
+{
+ struct dma_coherent_mem *mem;
+ dma_addr_t dev_base;
+ int pages = size >> PAGE_SHIFT;
+ int bitmap_size = BITS_TO_LONGS(pages) * sizeof(long);
+
+ if ((flags & DMA_MEMORY_MAP) == 0)
+ goto out;
+ if (!size)
+ goto out;
+ if (dev->dma_mem)
+ goto out;
+
+ mem = kzalloc(sizeof(*mem), GFP_KERNEL);
+ if (!mem)
+ goto out;
+ mem->virt_base = dma_alloc_coherent(dev, size, &dev_base,
+ DT3155_COH_FLAGS);
+ if (!mem->virt_base)
+ goto err_alloc_coherent;
+ mem->bitmap = kzalloc(bitmap_size, GFP_KERNEL);
+ if (!mem->bitmap)
+ goto err_bitmap;
+
+ mem->device_base = dev_base;
+ mem->size = pages;
+ mem->flags = flags;
+ dev->dma_mem = mem;
+ return DMA_MEMORY_MAP;
+
+err_bitmap:
+ dma_free_coherent(dev, size, mem->virt_base, dev_base);
+err_alloc_coherent:
+ kfree(mem);
+out:
+ return 0;
+}
+EXPORT_SYMBOL(dma_reserve_coherent_memory);
+
+/**
+ * dma_free_reserved_memory() - free the reserved dma-coherent memoty
+ *
+ * @dev: device for which we free the dma-coherent memory
+ *
+ * same as dma_release_declared_memory()
+ */
+void dma_free_reserved_memory(struct device *dev)
+{
+ struct dma_coherent_mem *mem = dev->dma_mem;
+
+ if (!mem)
+ return;
+ dev->dma_mem = NULL;
+ dma_free_coherent(dev, mem->size << PAGE_SHIFT,
+ mem->virt_base, mem->device_base);
+ kfree(mem->bitmap);
+ kfree(mem);
+}
+EXPORT_SYMBOL(dma_free_reserved_memory);
+
+/**
* dma_alloc_from_coherent() - try to allocate memory from the per-device coherent area
*
* @dev: device from which we allocate memory


2010-08-20 07:18:09

by FUJITA Tomonori

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

On Thu, 19 Aug 2010 18:18:35 +0300
Marin Mitov <[email protected]> wrote:

> struct device contains a member: struct dma_coherent_mem *dma_mem;
> to hold information for a piece of memory declared dma-coherent.
> Alternatively the same member could also be used to hold preallocated
> dma-coherent memory for latter per-device use.

I think that drivers/base/dma-coherent.c is for architectures to
implement dma_alloc_coherent(). So using it for drivers doesn't look
correct.


> This tric is already used in drivers/staging/dt3155v4l.c
> dt3155_alloc_coherent()/dt3155_free_coherent()
>
> Here proposed for general use by popular demand from video4linux folks.
> Helps for videobuf-dma-contig framework.

What you guys exactly want to do? If you just want to pre-allocate
coherent memory for latter usage, why dma_pool API (mm/dmapool.c)
doesn't work?

2010-08-20 08:15:25

by Marin Mitov

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

On Friday, August 20, 2010 10:17:48 am FUJITA Tomonori wrote:
> On Thu, 19 Aug 2010 18:18:35 +0300
> Marin Mitov <[email protected]> wrote:
>
> > struct device contains a member: struct dma_coherent_mem *dma_mem;
> > to hold information for a piece of memory declared dma-coherent.
> > Alternatively the same member could also be used to hold preallocated
> > dma-coherent memory for latter per-device use.
>
> I think that drivers/base/dma-coherent.c is for architectures to
> implement dma_alloc_coherent(). So using it for drivers doesn't look
> correct.

It depends. Imagine your frame grabber has built-in RAM buffer on board
just as the frame buffer RAM on graphics cards, defined in BAR. You can use
dma_declare_coherent_memory()/dma_release_declared_memory() in
your driver and then use dma_alloc_coherent()/dma_free_coherent()
to allocate dma buffers from it and falling back transparently to system RAM
when this local resource is exhausted.

>
>
> > This tric is already used in drivers/staging/dt3155v4l.c
> > dt3155_alloc_coherent()/dt3155_free_coherent()
> >
> > Here proposed for general use by popular demand from video4linux folks.
> > Helps for videobuf-dma-contig framework.
>
> What you guys exactly want to do? If you just want to pre-allocate
> coherent memory for latter usage,

Yes, just to preallocate not coherent, but rather contiguous memory for latter usage.
We use coherent memory because it turns out to be contiguous.

> why dma_pool API (mm/dmapool.c) doesn't work?

I do not know why dma_pool API doesn't work for frame grabber buffers.
May be they are too big ~400KB. I have tried dma_pool APIs without success
some time ago, so I had to find some other way to solve my problem leading to
the proposed dma_reserve_coherent_memory()/dma_free_reserved_memory().

Thanks.

Marin Mitov

2010-08-20 08:35:26

by FUJITA Tomonori

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

On Fri, 20 Aug 2010 11:13:45 +0300
Marin Mitov <[email protected]> wrote:

> > > This tric is already used in drivers/staging/dt3155v4l.c
> > > dt3155_alloc_coherent()/dt3155_free_coherent()
> > >
> > > Here proposed for general use by popular demand from video4linux folks.
> > > Helps for videobuf-dma-contig framework.
> >
> > What you guys exactly want to do? If you just want to pre-allocate
> > coherent memory for latter usage,
>
> Yes, just to preallocate not coherent, but rather contiguous memory for latter usage.
> We use coherent memory because it turns out to be contiguous.

Hmm, you don't care about coherency? You just need contiguous memory?

Then, I prefer to invent the API to allocate contiguous
memory. Coherent memory is precious on some arches.


> > why dma_pool API (mm/dmapool.c) doesn't work?
>
> I do not know why dma_pool API doesn't work for frame grabber buffers.
> May be they are too big ~400KB. I have tried dma_pool APIs without success
> some time ago, so I had to find some other way to solve my problem leading to
> the proposed dma_reserve_coherent_memory()/dma_free_reserved_memory().

I think that dma_pool API is for small coherent memory (smaller than
PAGE_SIZE) so it might not work for you. However, the purpose of
dma_pool API is exactly for what you want to do, creating a pool for
coherent memory per device for drivers.

I don't see any reason why we can't extend the dma_pool API for your
case. And it looks better to me rather than inventing the new API.

2010-08-20 11:51:31

by Marin Mitov

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

On Friday, August 20, 2010 11:35:06 am FUJITA Tomonori wrote:
> On Fri, 20 Aug 2010 11:13:45 +0300
> Marin Mitov <[email protected]> wrote:
>
> > > > This tric is already used in drivers/staging/dt3155v4l.c
> > > > dt3155_alloc_coherent()/dt3155_free_coherent()
> > > >
> > > > Here proposed for general use by popular demand from video4linux folks.
> > > > Helps for videobuf-dma-contig framework.
> > >
> > > What you guys exactly want to do? If you just want to pre-allocate
> > > coherent memory for latter usage,
> >
> > Yes, just to preallocate not coherent, but rather contiguous memory for latter usage.
> > We use coherent memory because it turns out to be contiguous.
>
> Hmm, you don't care about coherency? You just need contiguous memory?

Yes. We just need contiguous memory. Coherency is important as far as when dma
transfer finishes user land is able to see the new data. Could be done by something like
dma_{,un}map_single()

>
> Then, I prefer to invent the API to allocate contiguous
> memory. Coherent memory is precious on some arches.

Sure, but in any case videobuf-dma-contig framework in drivers/media/video
is already built around dma-coherent (nevertheless it is precious), so the two new
functions are just a helpful extension to the existing use of dma-coherent memory.

In any case, as far as these two functions will be mainly used by media/video folks
they could be added not to the drivers/base/dma-coherent.c (where I see their place),
but to drivers/media/video/videobuf-dma-contig.c. In that case the disadvantage will be
that if someone out of the media tree will need this functionality he(she) will need to
compile media/videobuf-dma-contig.c
>
>
> > > why dma_pool API (mm/dmapool.c) doesn't work?
> >
> > I do not know why dma_pool API doesn't work for frame grabber buffers.
> > May be they are too big ~400KB. I have tried dma_pool APIs without success
> > some time ago, so I had to find some other way to solve my problem leading to
> > the proposed dma_reserve_coherent_memory()/dma_free_reserved_memory().
>
> I think that dma_pool API is for small coherent memory (smaller than
> PAGE_SIZE)

Yes.

> so it might not work for you. However, the purpose of
> dma_pool API is exactly for what you want to do, creating a pool for
> coherent memory per device for drivers.
>
> I don't see any reason why we can't extend the dma_pool API for your
> case. And it looks better to me rather than inventing the new API.

That will help. I will be happy if someone can do it. I am inpaciently waiting for
alloc_huhepages()/free_hugepages() API - (transparent hugepages patches, may be)
That also could be a solution for media/video folks with hardware that cannot do
scatter/gatter. Another solution will be an IOMMU that could present a scattered
user land buffer as contiguous dma address range (I have played in the past with
AGP-GART without great success).

Thanks.

Marin Mitov

2010-08-20 20:05:38

by Guennadi Liakhovetski

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

On Thu, 19 Aug 2010, Marin Mitov wrote:

> Hi all,
>
> struct device contains a member: struct dma_coherent_mem *dma_mem;
> to hold information for a piece of memory declared dma-coherent.
> Alternatively the same member could also be used to hold preallocated
> dma-coherent memory for latter per-device use.
>
> This tric is already used in drivers/staging/dt3155v4l.c
> dt3155_alloc_coherent()/dt3155_free_coherent()
>
> Here proposed for general use by popular demand from video4linux folks.
> Helps for videobuf-dma-contig framework.

Ok, so, we've got two solutions to this problem submitted on the same
day;) Following this thread:

http://marc.info/?t=128128236400002&r=1&w=2

on the ARM Linux kernel ML, I submitted a patch series

http://thread.gmane.org/gmane.linux.ports.sh.devel/8595

with a couple of fixes and improvements, the actual new API and a use
example. My approach is slightly different, in that instead of requiring
drivers to issue two calls - one to reserve RAM (usually
dma_alloc_coherent()) and one to assign it to a device, my patch follows
the suggestion from Russell King from the first thread and unites these
two operations. So, now we have a choice;) Unfortunately, these two patch
series went to orthogonal sets of recepients, I'm trying to fix this by
adding a couple of CC entries.

Thanks
Guennadi

>
> Signed-off-by: Marin Mitov <[email protected]>
>
> ======================================================================
> --- a/drivers/base/dma-coherent.c 2010-08-19 15:50:42.000000000 +0300
> +++ b/drivers/base/dma-coherent.c 2010-08-19 17:27:56.000000000 +0300
> @@ -93,6 +93,83 @@ void *dma_mark_declared_memory_occupied(
> EXPORT_SYMBOL(dma_mark_declared_memory_occupied);
>
> /**
> + * dma_reserve_coherent_memory() - reserve coherent memory for per-device use
> + *
> + * @dev: device from which we allocate memory
> + * @size: size of requested memory area in bytes
> + * @flags: same as in dma_declare_coherent_memory()
> + *
> + * This function reserves coherent memory allocating it early (during probe())
> + * to support latter allocations from per-device coherent memory pools.
> + * For a given device one could use either dma_declare_coherent_memory() or
> + * dma_reserve_coherent_memory(), but not both, becase the result of these
> + * functions is stored in a single struct device member - dma_mem
> + *
> + * Returns DMA_MEMORY_MAP on success, or 0 if failed.
> + * (same as dma_declare_coherent_memory()
> + */
> +int dma_reserve_coherent_memory(struct device *dev, size_t size, int flags)
> +{
> + struct dma_coherent_mem *mem;
> + dma_addr_t dev_base;
> + int pages = size >> PAGE_SHIFT;
> + int bitmap_size = BITS_TO_LONGS(pages) * sizeof(long);
> +
> + if ((flags & DMA_MEMORY_MAP) == 0)
> + goto out;
> + if (!size)
> + goto out;
> + if (dev->dma_mem)
> + goto out;
> +
> + mem = kzalloc(sizeof(*mem), GFP_KERNEL);
> + if (!mem)
> + goto out;
> + mem->virt_base = dma_alloc_coherent(dev, size, &dev_base,
> + DT3155_COH_FLAGS);
> + if (!mem->virt_base)
> + goto err_alloc_coherent;
> + mem->bitmap = kzalloc(bitmap_size, GFP_KERNEL);
> + if (!mem->bitmap)
> + goto err_bitmap;
> +
> + mem->device_base = dev_base;
> + mem->size = pages;
> + mem->flags = flags;
> + dev->dma_mem = mem;
> + return DMA_MEMORY_MAP;
> +
> +err_bitmap:
> + dma_free_coherent(dev, size, mem->virt_base, dev_base);
> +err_alloc_coherent:
> + kfree(mem);
> +out:
> + return 0;
> +}
> +EXPORT_SYMBOL(dma_reserve_coherent_memory);
> +
> +/**
> + * dma_free_reserved_memory() - free the reserved dma-coherent memoty
> + *
> + * @dev: device for which we free the dma-coherent memory
> + *
> + * same as dma_release_declared_memory()
> + */
> +void dma_free_reserved_memory(struct device *dev)
> +{
> + struct dma_coherent_mem *mem = dev->dma_mem;
> +
> + if (!mem)
> + return;
> + dev->dma_mem = NULL;
> + dma_free_coherent(dev, mem->size << PAGE_SHIFT,
> + mem->virt_base, mem->device_base);
> + kfree(mem->bitmap);
> + kfree(mem);
> +}
> +EXPORT_SYMBOL(dma_free_reserved_memory);
> +
> +/**
> * dma_alloc_from_coherent() - try to allocate memory from the per-device coherent area
> *
> * @dev: device from which we allocate memory
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>

---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

2010-08-26 05:41:12

by FUJITA Tomonori

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

On Fri, 20 Aug 2010 14:50:12 +0300
Marin Mitov <[email protected]> wrote:

> On Friday, August 20, 2010 11:35:06 am FUJITA Tomonori wrote:
> > On Fri, 20 Aug 2010 11:13:45 +0300
> > Marin Mitov <[email protected]> wrote:
> >
> > > > > This tric is already used in drivers/staging/dt3155v4l.c
> > > > > dt3155_alloc_coherent()/dt3155_free_coherent()
> > > > >
> > > > > Here proposed for general use by popular demand from video4linux folks.
> > > > > Helps for videobuf-dma-contig framework.
> > > >
> > > > What you guys exactly want to do? If you just want to pre-allocate
> > > > coherent memory for latter usage,
> > >
> > > Yes, just to preallocate not coherent, but rather contiguous memory for latter usage.
> > > We use coherent memory because it turns out to be contiguous.
> >
> > Hmm, you don't care about coherency? You just need contiguous memory?
>
> Yes. We just need contiguous memory. Coherency is important as far as when dma
> transfer finishes user land is able to see the new data. Could be done by something like
> dma_{,un}map_single()

Then, we should avoid using coherent memory as I exaplained before. In
addition, dma_alloc_coherent can't provide large enough contigous
memory for some drivers so this patch doesn't help much.

We need the proper API for contiguous memory. Seem that we could have
something:

http://lkml.org/lkml/2010/8/20/167

2010-08-26 06:06:11

by Marin Mitov

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

On Thursday, August 26, 2010 08:40:47 am FUJITA Tomonori wrote:
> On Fri, 20 Aug 2010 14:50:12 +0300
> Marin Mitov <[email protected]> wrote:
>
> > On Friday, August 20, 2010 11:35:06 am FUJITA Tomonori wrote:
> > > On Fri, 20 Aug 2010 11:13:45 +0300
> > > Marin Mitov <[email protected]> wrote:
> > >
> > > > > > This tric is already used in drivers/staging/dt3155v4l.c
> > > > > > dt3155_alloc_coherent()/dt3155_free_coherent()
> > > > > >
> > > > > > Here proposed for general use by popular demand from video4linux folks.
> > > > > > Helps for videobuf-dma-contig framework.
> > > > >
> > > > > What you guys exactly want to do? If you just want to pre-allocate
> > > > > coherent memory for latter usage,
> > > >
> > > > Yes, just to preallocate not coherent, but rather contiguous memory for latter usage.
> > > > We use coherent memory because it turns out to be contiguous.
> > >
> > > Hmm, you don't care about coherency? You just need contiguous memory?
> >
> > Yes. We just need contiguous memory. Coherency is important as far as when dma
> > transfer finishes user land is able to see the new data. Could be done by something like
> > dma_{,un}map_single()
>
> Then, we should avoid using coherent memory as I exaplained before. In
> addition, dma_alloc_coherent can't provide large enough contigous
> memory for some drivers so this patch doesn't help much.

Please, look at drivers/media/video/videobuf-dma-contig.c. Using coherent memory
is inavoidable for now, there is no alternative for it for now. The two new functions,
which I propose are just helpers for those of us who already use coherent memory
(via videobuf-dma-contig API). May be adding these two functions to
drivers/media/video/videobuf-dma-contig.c will be better solution?

Thanks.

Marin Mitov

>
> We need the proper API for contiguous memory. Seem that we could have
> something:
>
> http://lkml.org/lkml/2010/8/20/167
>

2010-08-26 06:24:42

by FUJITA Tomonori

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

On Thu, 26 Aug 2010 09:04:14 +0300
Marin Mitov <[email protected]> wrote:

> On Thursday, August 26, 2010 08:40:47 am FUJITA Tomonori wrote:
> > On Fri, 20 Aug 2010 14:50:12 +0300
> > Marin Mitov <[email protected]> wrote:
> >
> > > On Friday, August 20, 2010 11:35:06 am FUJITA Tomonori wrote:
> > > > On Fri, 20 Aug 2010 11:13:45 +0300
> > > > Marin Mitov <[email protected]> wrote:
> > > >
> > > > > > > This tric is already used in drivers/staging/dt3155v4l.c
> > > > > > > dt3155_alloc_coherent()/dt3155_free_coherent()
> > > > > > >
> > > > > > > Here proposed for general use by popular demand from video4linux folks.
> > > > > > > Helps for videobuf-dma-contig framework.
> > > > > >
> > > > > > What you guys exactly want to do? If you just want to pre-allocate
> > > > > > coherent memory for latter usage,
> > > > >
> > > > > Yes, just to preallocate not coherent, but rather contiguous memory for latter usage.
> > > > > We use coherent memory because it turns out to be contiguous.
> > > >
> > > > Hmm, you don't care about coherency? You just need contiguous memory?
> > >
> > > Yes. We just need contiguous memory. Coherency is important as far as when dma
> > > transfer finishes user land is able to see the new data. Could be done by something like
> > > dma_{,un}map_single()
> >
> > Then, we should avoid using coherent memory as I exaplained before. In
> > addition, dma_alloc_coherent can't provide large enough contigous
> > memory for some drivers so this patch doesn't help much.
>
> Please, look at drivers/media/video/videobuf-dma-contig.c. Using coherent memory
> is inavoidable for now, there is no alternative for it for now. The two new functions,
> which I propose are just helpers for those of us who already use coherent memory
> (via videobuf-dma-contig API). May be adding these two functions to
> drivers/media/video/videobuf-dma-contig.c will be better solution?

If you add something to the videobuf-dma-contig API, that's fine by me
because drivers/media/video/videobuf-dma-contig.c uses the own
structure and plays with dma_alloc_coherent. As long as a driver
doesn't touch device->dma_mem directly, it's fine, I think (that is,
dt3155v4l driver is broken). There are already some workarounds for
contigous memory in several drivers anyway.

We will have the proper API for contiguous memory. I don't think that
adding such workaround to the DMA API is a good idea.

2010-08-26 07:04:14

by Marin Mitov

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

On Thursday, August 26, 2010 09:24:19 am FUJITA Tomonori wrote:
> On Thu, 26 Aug 2010 09:04:14 +0300
> Marin Mitov <[email protected]> wrote:
>
> > On Thursday, August 26, 2010 08:40:47 am FUJITA Tomonori wrote:
> > > On Fri, 20 Aug 2010 14:50:12 +0300
> > > Marin Mitov <[email protected]> wrote:
> > >
> > > > On Friday, August 20, 2010 11:35:06 am FUJITA Tomonori wrote:
> > > > > On Fri, 20 Aug 2010 11:13:45 +0300
> > > > > Marin Mitov <[email protected]> wrote:
> > > > >
> > > > > > > > This tric is already used in drivers/staging/dt3155v4l.c
> > > > > > > > dt3155_alloc_coherent()/dt3155_free_coherent()
> > > > > > > >
> > > > > > > > Here proposed for general use by popular demand from video4linux folks.
> > > > > > > > Helps for videobuf-dma-contig framework.
> > > > > > >
> > > > > > > What you guys exactly want to do? If you just want to pre-allocate
> > > > > > > coherent memory for latter usage,
> > > > > >
> > > > > > Yes, just to preallocate not coherent, but rather contiguous memory for latter usage.
> > > > > > We use coherent memory because it turns out to be contiguous.
> > > > >
> > > > > Hmm, you don't care about coherency? You just need contiguous memory?
> > > >
> > > > Yes. We just need contiguous memory. Coherency is important as far as when dma
> > > > transfer finishes user land is able to see the new data. Could be done by something like
> > > > dma_{,un}map_single()
> > >
> > > Then, we should avoid using coherent memory as I exaplained before. In
> > > addition, dma_alloc_coherent can't provide large enough contigous
> > > memory for some drivers so this patch doesn't help much.
> >
> > Please, look at drivers/media/video/videobuf-dma-contig.c. Using coherent memory
> > is inavoidable for now, there is no alternative for it for now. The two new functions,
> > which I propose are just helpers for those of us who already use coherent memory
> > (via videobuf-dma-contig API). May be adding these two functions to
> > drivers/media/video/videobuf-dma-contig.c will be better solution?
>
> If you add something to the videobuf-dma-contig API, that's fine by me
> because drivers/media/video/videobuf-dma-contig.c uses the own
> structure and plays with dma_alloc_coherent. As long as a driver
> doesn't touch device->dma_mem directly, it's fine,

Why, my understanding is that device->dma_mem is designed exactly for keeping
some chunk of coherent memory for device's private use via dma_alloc_from_coherent()
(and that is what dt3155v4l driver is using it for).

> I think (that is, dt3155v4l driver is broken).

If you mean that allocating some coherent memory (4MB in case of dt3155v4l) during
pci probe() (during system booting) for device's latter use (that is dead for the rest
of the system) you are right. But this gives me at least 8 full size buffers warranted for
latter use. Without this hack the hardware will not work on strongly fragmented system.
With this hack even if the system is strongly fragmented, this chunk of 4MB is available
for use (though videobuf-dma-contig APIs and dma_alloc_from_coherent()) __transparently__
for users of videobuf-dma-contig (that is the gain - the transparency).

> There are already some workarounds for
> contigous memory in several drivers anyway.

Sure, can these workarounds be exposed as API for general use?

Thanks,

Marin Mitov

>
> We will have the proper API for contiguous memory. I don't think that
> adding such workaround to the DMA API is a good idea.
>

2010-08-26 09:06:16

by Guennadi Liakhovetski

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

On Thu, 26 Aug 2010, FUJITA Tomonori wrote:

> On Thu, 26 Aug 2010 09:04:14 +0300
> Marin Mitov <[email protected]> wrote:
>
> > On Thursday, August 26, 2010 08:40:47 am FUJITA Tomonori wrote:
> > > On Fri, 20 Aug 2010 14:50:12 +0300
> > > Marin Mitov <[email protected]> wrote:
> > >
> > > > On Friday, August 20, 2010 11:35:06 am FUJITA Tomonori wrote:
> > > > > On Fri, 20 Aug 2010 11:13:45 +0300
> > > > > Marin Mitov <[email protected]> wrote:
> > > > >
> > > > > > > > This tric is already used in drivers/staging/dt3155v4l.c
> > > > > > > > dt3155_alloc_coherent()/dt3155_free_coherent()
> > > > > > > >
> > > > > > > > Here proposed for general use by popular demand from video4linux folks.
> > > > > > > > Helps for videobuf-dma-contig framework.
> > > > > > >
> > > > > > > What you guys exactly want to do? If you just want to pre-allocate
> > > > > > > coherent memory for latter usage,
> > > > > >
> > > > > > Yes, just to preallocate not coherent, but rather contiguous memory for latter usage.
> > > > > > We use coherent memory because it turns out to be contiguous.
> > > > >
> > > > > Hmm, you don't care about coherency? You just need contiguous memory?
> > > >
> > > > Yes. We just need contiguous memory. Coherency is important as far as when dma
> > > > transfer finishes user land is able to see the new data. Could be done by something like
> > > > dma_{,un}map_single()
> > >
> > > Then, we should avoid using coherent memory as I exaplained before. In
> > > addition, dma_alloc_coherent can't provide large enough contigous
> > > memory for some drivers so this patch doesn't help much.
> >
> > Please, look at drivers/media/video/videobuf-dma-contig.c. Using coherent memory
> > is inavoidable for now, there is no alternative for it for now. The two new functions,
> > which I propose are just helpers for those of us who already use coherent memory
> > (via videobuf-dma-contig API). May be adding these two functions to
> > drivers/media/video/videobuf-dma-contig.c will be better solution?
>
> If you add something to the videobuf-dma-contig API, that's fine by me
> because drivers/media/video/videobuf-dma-contig.c uses the own
> structure and plays with dma_alloc_coherent. As long as a driver
> doesn't touch device->dma_mem directly, it's fine, I think (that is,
> dt3155v4l driver is broken). There are already some workarounds for
> contigous memory in several drivers anyway.

No, this will not work - this API has to be used from board code and
videobuf can be built modular.

> We will have the proper API for contiguous memory. I don't think that
> adding such workaround to the DMA API is a good idea.

We have currently a number of boards broken in the mainline. They must be
fixed for 2.6.36. I don't think the mentioned API will do this for us. So,
as I suggested earlier, we need either this or my patch series

http://thread.gmane.org/gmane.linux.ports.sh.devel/8595

for 2.6.36.

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

2010-08-26 09:17:50

by Uwe Kleine-König

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

Hello,

On Thu, Aug 26, 2010 at 11:06:20AM +0200, Guennadi Liakhovetski wrote:
> On Thu, 26 Aug 2010, FUJITA Tomonori wrote:
>
> > On Thu, 26 Aug 2010 09:04:14 +0300
> > Marin Mitov <[email protected]> wrote:
> >
> > > On Thursday, August 26, 2010 08:40:47 am FUJITA Tomonori wrote:
> > > > On Fri, 20 Aug 2010 14:50:12 +0300
> > > > Marin Mitov <[email protected]> wrote:
> > > >
> > > > > On Friday, August 20, 2010 11:35:06 am FUJITA Tomonori wrote:
> > > > > > On Fri, 20 Aug 2010 11:13:45 +0300
> > > > > > Marin Mitov <[email protected]> wrote:
> > > > > >
> > > > > > > > > This tric is already used in drivers/staging/dt3155v4l.c
> > > > > > > > > dt3155_alloc_coherent()/dt3155_free_coherent()
> > > > > > > > >
> > > > > > > > > Here proposed for general use by popular demand from video4linux folks.
> > > > > > > > > Helps for videobuf-dma-contig framework.
> > > > > > > >
> > > > > > > > What you guys exactly want to do? If you just want to pre-allocate
> > > > > > > > coherent memory for latter usage,
> > > > > > >
> > > > > > > Yes, just to preallocate not coherent, but rather contiguous memory for latter usage.
> > > > > > > We use coherent memory because it turns out to be contiguous.
> > > > > >
> > > > > > Hmm, you don't care about coherency? You just need contiguous memory?
> > > > >
> > > > > Yes. We just need contiguous memory. Coherency is important as far as when dma
> > > > > transfer finishes user land is able to see the new data. Could be done by something like
> > > > > dma_{,un}map_single()
> > > >
> > > > Then, we should avoid using coherent memory as I exaplained before. In
> > > > addition, dma_alloc_coherent can't provide large enough contigous
> > > > memory for some drivers so this patch doesn't help much.
> > >
> > > Please, look at drivers/media/video/videobuf-dma-contig.c. Using coherent memory
> > > is inavoidable for now, there is no alternative for it for now. The two new functions,
> > > which I propose are just helpers for those of us who already use coherent memory
> > > (via videobuf-dma-contig API). May be adding these two functions to
> > > drivers/media/video/videobuf-dma-contig.c will be better solution?
> >
> > If you add something to the videobuf-dma-contig API, that's fine by me
> > because drivers/media/video/videobuf-dma-contig.c uses the own
> > structure and plays with dma_alloc_coherent. As long as a driver
> > doesn't touch device->dma_mem directly, it's fine, I think (that is,
> > dt3155v4l driver is broken). There are already some workarounds for
> > contigous memory in several drivers anyway.
>
> No, this will not work - this API has to be used from board code and
> videobuf can be built modular.
>
> > We will have the proper API for contiguous memory. I don't think that
> > adding such workaround to the DMA API is a good idea.
>
> We have currently a number of boards broken in the mainline. They must be
> fixed for 2.6.36. I don't think the mentioned API will do this for us. So,
> as I suggested earlier, we need either this or my patch series
>
> http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
this seems to be more mature to me. The original patch in this thread
uses a symbol DT3155_COH_FLAGS which seems misplaced in generic code and
doesn't put the new functions in a header.

Best regards
Uwe

--
Pengutronix e.K. | Uwe Kleine-K?nig |
Industrial Linux Solutions | http://www.pengutronix.de/ |

2010-08-26 09:32:10

by FUJITA Tomonori

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

On Thu, 26 Aug 2010 11:06:20 +0200 (CEST)
Guennadi Liakhovetski <[email protected]> wrote:

> On Thu, 26 Aug 2010, FUJITA Tomonori wrote:
>
> > On Thu, 26 Aug 2010 09:04:14 +0300
> > Marin Mitov <[email protected]> wrote:
> >
> > > On Thursday, August 26, 2010 08:40:47 am FUJITA Tomonori wrote:
> > > > On Fri, 20 Aug 2010 14:50:12 +0300
> > > > Marin Mitov <[email protected]> wrote:
> > > >
> > > > > On Friday, August 20, 2010 11:35:06 am FUJITA Tomonori wrote:
> > > > > > On Fri, 20 Aug 2010 11:13:45 +0300
> > > > > > Marin Mitov <[email protected]> wrote:
> > > > > >
> > > > > > > > > This tric is already used in drivers/staging/dt3155v4l.c
> > > > > > > > > dt3155_alloc_coherent()/dt3155_free_coherent()
> > > > > > > > >
> > > > > > > > > Here proposed for general use by popular demand from video4linux folks.
> > > > > > > > > Helps for videobuf-dma-contig framework.
> > > > > > > >
> > > > > > > > What you guys exactly want to do? If you just want to pre-allocate
> > > > > > > > coherent memory for latter usage,
> > > > > > >
> > > > > > > Yes, just to preallocate not coherent, but rather contiguous memory for latter usage.
> > > > > > > We use coherent memory because it turns out to be contiguous.
> > > > > >
> > > > > > Hmm, you don't care about coherency? You just need contiguous memory?
> > > > >
> > > > > Yes. We just need contiguous memory. Coherency is important as far as when dma
> > > > > transfer finishes user land is able to see the new data. Could be done by something like
> > > > > dma_{,un}map_single()
> > > >
> > > > Then, we should avoid using coherent memory as I exaplained before. In
> > > > addition, dma_alloc_coherent can't provide large enough contigous
> > > > memory for some drivers so this patch doesn't help much.
> > >
> > > Please, look at drivers/media/video/videobuf-dma-contig.c. Using coherent memory
> > > is inavoidable for now, there is no alternative for it for now. The two new functions,
> > > which I propose are just helpers for those of us who already use coherent memory
> > > (via videobuf-dma-contig API). May be adding these two functions to
> > > drivers/media/video/videobuf-dma-contig.c will be better solution?
> >
> > If you add something to the videobuf-dma-contig API, that's fine by me
> > because drivers/media/video/videobuf-dma-contig.c uses the own
> > structure and plays with dma_alloc_coherent. As long as a driver
> > doesn't touch device->dma_mem directly, it's fine, I think (that is,
> > dt3155v4l driver is broken). There are already some workarounds for
> > contigous memory in several drivers anyway.
>
> No, this will not work - this API has to be used from board code and
> videobuf can be built modular.
>
> > We will have the proper API for contiguous memory. I don't think that
> > adding such workaround to the DMA API is a good idea.
>
> We have currently a number of boards broken in the mainline. They must be
> fixed for 2.6.36. I don't think the mentioned API will do this for us. So,
> as I suggested earlier, we need either this or my patch series
>
> http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
>
> for 2.6.36.

Why can't you revert a commit that causes the regression?

The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not
responsible for the regression. And the patchset even exnteds the
definition of the DMA API (dma_declare_coherent_memory). Such change
shouldn't applied after rc1. I think that DMA-API.txt says that
dma_declare_coherent_memory() handles coherent memory for a particular
device. It's not for the API that reserves coherent memory that can be
used for any device for a single device.

2010-08-26 09:43:43

by FUJITA Tomonori

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

On Thu, 26 Aug 2010 10:01:52 +0300
Marin Mitov <[email protected]> wrote:

> > If you add something to the videobuf-dma-contig API, that's fine by me
> > because drivers/media/video/videobuf-dma-contig.c uses the own
> > structure and plays with dma_alloc_coherent. As long as a driver
> > doesn't touch device->dma_mem directly, it's fine,
>
> Why, my understanding is that device->dma_mem is designed exactly for keeping
> some chunk of coherent memory for device's private use via dma_alloc_from_coherent()
> (and that is what dt3155v4l driver is using it for).

I don't think so. device->dma_mem can be accessed only via the
DMA-API. I think that the DMA-API says that
dma_declare_coherent_memory declares coherent memory that can be
access exclusively by a certain device. It's not for reserving
coherent memory that can be used for any device for a device.

Anway, you don't need coherent memory. So using the API for coherent
memory isn't a good idea.


> > There are already some workarounds for
> > contigous memory in several drivers anyway.
>
> Sure, can these workarounds be exposed as API for general use?

I don't think that's a good idea. Adding temporary workaround to the
generic API and removing it soon after that doesn't sound a good
developing maner.

2010-08-26 09:45:57

by Guennadi Liakhovetski

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

On Thu, 26 Aug 2010, FUJITA Tomonori wrote:

> Why can't you revert a commit that causes the regression?

See this reply, and the complete thread too.

http://marc.info/?l=linux-sh&m=128130485208262&w=2

> The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not
> responsible for the regression. And the patchset even exnteds the
> definition of the DMA API (dma_declare_coherent_memory). Such change
> shouldn't applied after rc1. I think that DMA-API.txt says that
> dma_declare_coherent_memory() handles coherent memory for a particular
> device. It's not for the API that reserves coherent memory that can be
> used for any device for a single device.

Anyway, we need a way to fix the regression.

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

2010-08-26 09:53:35

by Uwe Kleine-König

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

On Thu, Aug 26, 2010 at 06:30:02PM +0900, FUJITA Tomonori wrote:
> On Thu, 26 Aug 2010 11:06:20 +0200 (CEST)
> Guennadi Liakhovetski <[email protected]> wrote:
>
> > On Thu, 26 Aug 2010, FUJITA Tomonori wrote:
> >
> > > On Thu, 26 Aug 2010 09:04:14 +0300
> > > Marin Mitov <[email protected]> wrote:
> > >
> > > > On Thursday, August 26, 2010 08:40:47 am FUJITA Tomonori wrote:
> > > > > On Fri, 20 Aug 2010 14:50:12 +0300
> > > > > Marin Mitov <[email protected]> wrote:
> > > > >
> > > > > > On Friday, August 20, 2010 11:35:06 am FUJITA Tomonori wrote:
> > > > > > > On Fri, 20 Aug 2010 11:13:45 +0300
> > > > > > > Marin Mitov <[email protected]> wrote:
> > > > > > >
> > > > > > > > > > This tric is already used in drivers/staging/dt3155v4l.c
> > > > > > > > > > dt3155_alloc_coherent()/dt3155_free_coherent()
> > > > > > > > > >
> > > > > > > > > > Here proposed for general use by popular demand from video4linux folks.
> > > > > > > > > > Helps for videobuf-dma-contig framework.
> > > > > > > > >
> > > > > > > > > What you guys exactly want to do? If you just want to pre-allocate
> > > > > > > > > coherent memory for latter usage,
> > > > > > > >
> > > > > > > > Yes, just to preallocate not coherent, but rather contiguous memory for latter usage.
> > > > > > > > We use coherent memory because it turns out to be contiguous.
> > > > > > >
> > > > > > > Hmm, you don't care about coherency? You just need contiguous memory?
> > > > > >
> > > > > > Yes. We just need contiguous memory. Coherency is important as far as when dma
> > > > > > transfer finishes user land is able to see the new data. Could be done by something like
> > > > > > dma_{,un}map_single()
> > > > >
> > > > > Then, we should avoid using coherent memory as I exaplained before. In
> > > > > addition, dma_alloc_coherent can't provide large enough contigous
> > > > > memory for some drivers so this patch doesn't help much.
> > > >
> > > > Please, look at drivers/media/video/videobuf-dma-contig.c. Using coherent memory
> > > > is inavoidable for now, there is no alternative for it for now. The two new functions,
> > > > which I propose are just helpers for those of us who already use coherent memory
> > > > (via videobuf-dma-contig API). May be adding these two functions to
> > > > drivers/media/video/videobuf-dma-contig.c will be better solution?
> > >
> > > If you add something to the videobuf-dma-contig API, that's fine by me
> > > because drivers/media/video/videobuf-dma-contig.c uses the own
> > > structure and plays with dma_alloc_coherent. As long as a driver
> > > doesn't touch device->dma_mem directly, it's fine, I think (that is,
> > > dt3155v4l driver is broken). There are already some workarounds for
> > > contigous memory in several drivers anyway.
> >
> > No, this will not work - this API has to be used from board code and
> > videobuf can be built modular.
> >
> > > We will have the proper API for contiguous memory. I don't think that
> > > adding such workaround to the DMA API is a good idea.
> >
> > We have currently a number of boards broken in the mainline. They must be
> > fixed for 2.6.36. I don't think the mentioned API will do this for us. So,
> > as I suggested earlier, we need either this or my patch series
> >
> > http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
> >
> > for 2.6.36.
>
> Why can't you revert a commit that causes the regression?
>
> The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not
> responsible for the regression. And the patchset even exnteds the
> definition of the DMA API (dma_declare_coherent_memory). Such change
> shouldn't applied after rc1. I think that DMA-API.txt says that
> dma_declare_coherent_memory() handles coherent memory for a particular
> device. It's not for the API that reserves coherent memory that can be
> used for any device for a single device.
The patch that made the problem obvious for ARM is
309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12.
So this went in before v2.6.36-rc1. One of the "architectures which
similar restrictions" is x86 BTW.

And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it
addresses a hardware restriction.

Best regards
Uwe

--
Pengutronix e.K. | Uwe Kleine-K?nig |
Industrial Linux Solutions | http://www.pengutronix.de/ |

2010-08-26 09:53:44

by FUJITA Tomonori

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

On Thu, 26 Aug 2010 11:45:58 +0200 (CEST)
Guennadi Liakhovetski <[email protected]> wrote:

> On Thu, 26 Aug 2010, FUJITA Tomonori wrote:
>
> > Why can't you revert a commit that causes the regression?
>
> See this reply, and the complete thread too.
>
> http://marc.info/?l=linux-sh&m=128130485208262&w=2
>
> > The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not
> > responsible for the regression. And the patchset even exnteds the
> > definition of the DMA API (dma_declare_coherent_memory). Such change
> > shouldn't applied after rc1. I think that DMA-API.txt says that
> > dma_declare_coherent_memory() handles coherent memory for a particular
> > device. It's not for the API that reserves coherent memory that can be
> > used for any device for a single device.
>
> Anyway, we need a way to fix the regression.

Needs to find a different way.

2010-08-26 10:02:07

by FUJITA Tomonori

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

On Thu, 26 Aug 2010 11:53:11 +0200
Uwe Kleine-K$(D+S(Bnig <[email protected]> wrote:

> > > We have currently a number of boards broken in the mainline. They must be
> > > fixed for 2.6.36. I don't think the mentioned API will do this for us. So,
> > > as I suggested earlier, we need either this or my patch series
> > >
> > > http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
> > >
> > > for 2.6.36.
> >
> > Why can't you revert a commit that causes the regression?
> >
> > The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not
> > responsible for the regression. And the patchset even exnteds the
> > definition of the DMA API (dma_declare_coherent_memory). Such change
> > shouldn't applied after rc1. I think that DMA-API.txt says that
> > dma_declare_coherent_memory() handles coherent memory for a particular
> > device. It's not for the API that reserves coherent memory that can be
> > used for any device for a single device.
> The patch that made the problem obvious for ARM is
> 309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12.
> So this went in before v2.6.36-rc1. One of the "architectures which
> similar restrictions" is x86 BTW.
>
> And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it
> addresses a hardware restriction.

How these drivers were able to work without hitting the hardware restriction?

2010-08-26 10:16:53

by Marin Mitov

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

On Thursday, August 26, 2010 12:43:22 pm FUJITA Tomonori wrote:
> On Thu, 26 Aug 2010 10:01:52 +0300
> Marin Mitov <[email protected]> wrote:
>
> > > If you add something to the videobuf-dma-contig API, that's fine by me
> > > because drivers/media/video/videobuf-dma-contig.c uses the own
> > > structure and plays with dma_alloc_coherent. As long as a driver
> > > doesn't touch device->dma_mem directly, it's fine,
> >
> > Why, my understanding is that device->dma_mem is designed exactly for keeping
> > some chunk of coherent memory for device's private use via dma_alloc_from_coherent()
> > (and that is what dt3155v4l driver is using it for).
>
> I don't think so. device->dma_mem can be accessed only via the
> DMA-API. I think that the DMA-API says that
> dma_declare_coherent_memory declares coherent memory that can be
> access exclusively by a certain device.

Here I agree with you: "that can be access exclusively by a certain device"

> It's not for reserving
> coherent memory that can be used for any device for a device.

Here I disagree with you: "that can be used for any device for a device".
Reserved coherent memory can be only and exclusively used by
the __same__ device whose device->dma_mem is touched. No other devices
are influenced because their device->dma_mem are NULL. and
dma_alloc_from_coherent() is not invoked for them. That is why I think
this hack is not dangerous. If some device driver decide to reserve some
chunk of memory it is for its private use and no other device in the system
is influenced.

>
> Anway, you don't need coherent memory. So using the API for coherent
> memory isn't a good idea.

Here I agree with you, but for now we have no alternative in media/video
framework.

>
>
> > > There are already some workarounds for
> > > contigous memory in several drivers anyway.
> >
> > Sure, can these workarounds be exposed as API for general use?
>
> I don't think that's a good idea. Adding temporary workaround to the
> generic API and removing it soon after that doesn't sound a good
> developing maner.

Yes, it is just a temporary solution. Just enhancing an existing temporary solution.

Thanks,

Marin Mitov

>

2010-08-26 10:21:51

by Marin Mitov

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

On Thursday, August 26, 2010 12:17:25 pm Uwe Kleine-K?nig wrote:
> Hello,
>
> On Thu, Aug 26, 2010 at 11:06:20AM +0200, Guennadi Liakhovetski wrote:
> > On Thu, 26 Aug 2010, FUJITA Tomonori wrote:
> >
> > > On Thu, 26 Aug 2010 09:04:14 +0300
> > > Marin Mitov <[email protected]> wrote:
> > >
> > > > On Thursday, August 26, 2010 08:40:47 am FUJITA Tomonori wrote:
> > > > > On Fri, 20 Aug 2010 14:50:12 +0300
> > > > > Marin Mitov <[email protected]> wrote:
> > > > >
> > > > > > On Friday, August 20, 2010 11:35:06 am FUJITA Tomonori wrote:
> > > > > > > On Fri, 20 Aug 2010 11:13:45 +0300
> > > > > > > Marin Mitov <[email protected]> wrote:
> > > > > > >
> > > > > > > > > > This tric is already used in drivers/staging/dt3155v4l.c
> > > > > > > > > > dt3155_alloc_coherent()/dt3155_free_coherent()
> > > > > > > > > >
> > > > > > > > > > Here proposed for general use by popular demand from video4linux folks.
> > > > > > > > > > Helps for videobuf-dma-contig framework.
> > > > > > > > >
> > > > > > > > > What you guys exactly want to do? If you just want to pre-allocate
> > > > > > > > > coherent memory for latter usage,
> > > > > > > >
> > > > > > > > Yes, just to preallocate not coherent, but rather contiguous memory for latter usage.
> > > > > > > > We use coherent memory because it turns out to be contiguous.
> > > > > > >
> > > > > > > Hmm, you don't care about coherency? You just need contiguous memory?
> > > > > >
> > > > > > Yes. We just need contiguous memory. Coherency is important as far as when dma
> > > > > > transfer finishes user land is able to see the new data. Could be done by something like
> > > > > > dma_{,un}map_single()
> > > > >
> > > > > Then, we should avoid using coherent memory as I exaplained before. In
> > > > > addition, dma_alloc_coherent can't provide large enough contigous
> > > > > memory for some drivers so this patch doesn't help much.
> > > >
> > > > Please, look at drivers/media/video/videobuf-dma-contig.c. Using coherent memory
> > > > is inavoidable for now, there is no alternative for it for now. The two new functions,
> > > > which I propose are just helpers for those of us who already use coherent memory
> > > > (via videobuf-dma-contig API). May be adding these two functions to
> > > > drivers/media/video/videobuf-dma-contig.c will be better solution?
> > >
> > > If you add something to the videobuf-dma-contig API, that's fine by me
> > > because drivers/media/video/videobuf-dma-contig.c uses the own
> > > structure and plays with dma_alloc_coherent. As long as a driver
> > > doesn't touch device->dma_mem directly, it's fine, I think (that is,
> > > dt3155v4l driver is broken). There are already some workarounds for
> > > contigous memory in several drivers anyway.
> >
> > No, this will not work - this API has to be used from board code and
> > videobuf can be built modular.
> >
> > > We will have the proper API for contiguous memory. I don't think that
> > > adding such workaround to the DMA API is a good idea.
> >
> > We have currently a number of boards broken in the mainline. They must be
> > fixed for 2.6.36. I don't think the mentioned API will do this for us. So,
> > as I suggested earlier, we need either this or my patch series
> >
> > http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
> this seems to be more mature to me. The original patch in this thread
> uses a symbol DT3155_COH_FLAGS which seems misplaced in generic code and
> doesn't put the new functions in a header.

You are right. DT3155_COH_FLAGS should be defined, and a declaration should be
put in the headers.

But it is just RFC :-)

Marin Mitov

>
> Best regards
> Uwe
>
>

2010-08-26 17:49:45

by Russell King - ARM Linux

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

On Thu, Aug 26, 2010 at 06:51:48PM +0900, FUJITA Tomonori wrote:
> On Thu, 26 Aug 2010 11:45:58 +0200 (CEST)
> Guennadi Liakhovetski <[email protected]> wrote:
>
> > On Thu, 26 Aug 2010, FUJITA Tomonori wrote:
> >
> > > Why can't you revert a commit that causes the regression?
> >
> > See this reply, and the complete thread too.
> >
> > http://marc.info/?l=linux-sh&m=128130485208262&w=2
> >
> > > The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not
> > > responsible for the regression. And the patchset even exnteds the
> > > definition of the DMA API (dma_declare_coherent_memory). Such change
> > > shouldn't applied after rc1. I think that DMA-API.txt says that
> > > dma_declare_coherent_memory() handles coherent memory for a particular
> > > device. It's not for the API that reserves coherent memory that can be
> > > used for any device for a single device.
> >
> > Anyway, we need a way to fix the regression.
>
> Needs to find a different way.

No. ioremap on memory mapped by the kernel is just plain not permitted
with ARMv6 and ARMv7 architectures.

It's not something you can say "oh, need to find another way" because there
is _no_ software solution to having physical regions mapped multiple times
with different attributes. It's an architectural restriction.

We can't unmap the kernel's memory mapping either, as I've already explained
several times this month - and I'm getting frustrated at having to keep
on explaining that point.

Just accept the plain fact that multiple mappings of the same physical
regions have become illegal.

What we need is another alternative other than using ioremap on memory
already mapped by the kernel - eg, by reserving a certain chunk of
memory for this purpose at boot time which his _never_ mapped by the
kernel, except via ioremap.

2010-08-26 17:55:12

by Russell King - ARM Linux

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

On Thu, Aug 26, 2010 at 07:00:24PM +0900, FUJITA Tomonori wrote:
> On Thu, 26 Aug 2010 11:53:11 +0200
> Uwe Kleine-K?nig <[email protected]> wrote:
>
> > > > We have currently a number of boards broken in the mainline. They must be
> > > > fixed for 2.6.36. I don't think the mentioned API will do this for us. So,
> > > > as I suggested earlier, we need either this or my patch series
> > > >
> > > > http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
> > > >
> > > > for 2.6.36.
> > >
> > > Why can't you revert a commit that causes the regression?
> > >
> > > The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not
> > > responsible for the regression. And the patchset even exnteds the
> > > definition of the DMA API (dma_declare_coherent_memory). Such change
> > > shouldn't applied after rc1. I think that DMA-API.txt says that
> > > dma_declare_coherent_memory() handles coherent memory for a particular
> > > device. It's not for the API that reserves coherent memory that can be
> > > used for any device for a single device.
> > The patch that made the problem obvious for ARM is
> > 309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12.
> > So this went in before v2.6.36-rc1. One of the "architectures which
> > similar restrictions" is x86 BTW.
> >
> > And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it
> > addresses a hardware restriction.
>
> How these drivers were able to work without hitting the hardware restriction?

Well, OMAP processors have experienced lock-ups due to multiple mappings of
memory, so the restriction in the architecture manual is for real.

But more the issue is that the behaviour you get from a region is _totally_
unpredictable (as the arch manual says). With the VIPT caches, they can
be searched irrespective of whether the page tabkes indicate that it's
supposed to be cached or not - which means you can still hit cache lines
for an ioremap'd region.

And if you do, how do you know that the cached data is still valid - what
if it's some critical data that results in corruption - how do you know
whether that's happened or not? It might not even cause a kernel
exception.

We have to adhere to the restrictions placed upon us by the architecture
at hand, and if that means device drivers break, so be it - at least we
get to know what needs to be fixed for these restrictions.

2010-08-26 18:36:13

by Marin Mitov

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

On Thursday, August 26, 2010 08:49:09 pm Russell King - ARM Linux wrote:
> On Thu, Aug 26, 2010 at 06:51:48PM +0900, FUJITA Tomonori wrote:
> > On Thu, 26 Aug 2010 11:45:58 +0200 (CEST)
> > Guennadi Liakhovetski <[email protected]> wrote:
> >
> > > On Thu, 26 Aug 2010, FUJITA Tomonori wrote:
> > >
> > > > Why can't you revert a commit that causes the regression?
> > >
> > > See this reply, and the complete thread too.
> > >
> > > http://marc.info/?l=linux-sh&m=128130485208262&w=2
> > >
> > > > The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not
> > > > responsible for the regression. And the patchset even exnteds the
> > > > definition of the DMA API (dma_declare_coherent_memory). Such change
> > > > shouldn't applied after rc1. I think that DMA-API.txt says that
> > > > dma_declare_coherent_memory() handles coherent memory for a particular
> > > > device. It's not for the API that reserves coherent memory that can be
> > > > used for any device for a single device.
> > >
> > > Anyway, we need a way to fix the regression.
> >
> > Needs to find a different way.
>
> No. ioremap on memory mapped by the kernel is just plain not permitted
> with ARMv6 and ARMv7 architectures.

Hi Russell,

Just because ioremap on memory mapped by the kernel is just plain not permitted
I have proposed a new pair of functions: dma_reserve_coherent_memory()/dma_free_reserved_memory()

http://lkml.org/lkml/2010/8/19/200

but it is not quite well accepted from the community.
What is your opinion?

Thanks,

Marin Mitov

>
> It's not something you can say "oh, need to find another way" because there
> is _no_ software solution to having physical regions mapped multiple times
> with different attributes. It's an architectural restriction.
>
> We can't unmap the kernel's memory mapping either, as I've already explained
> several times this month - and I'm getting frustrated at having to keep
> on explaining that point.
>
> Just accept the plain fact that multiple mappings of the same physical
> regions have become illegal.
>
> What we need is another alternative other than using ioremap on memory
> already mapped by the kernel - eg, by reserving a certain chunk of
> memory for this purpose at boot time which his _never_ mapped by the
> kernel, except via ioremap.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>

2010-08-27 00:27:58

by FUJITA Tomonori

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

On Thu, 26 Aug 2010 18:54:40 +0100
Russell King - ARM Linux <[email protected]> wrote:

> On Thu, Aug 26, 2010 at 07:00:24PM +0900, FUJITA Tomonori wrote:
> > On Thu, 26 Aug 2010 11:53:11 +0200
> > Uwe Kleine-K$(D+S(Bnig <[email protected]> wrote:
> >
> > > > > We have currently a number of boards broken in the mainline. They must be
> > > > > fixed for 2.6.36. I don't think the mentioned API will do this for us. So,
> > > > > as I suggested earlier, we need either this or my patch series
> > > > >
> > > > > http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
> > > > >
> > > > > for 2.6.36.
> > > >
> > > > Why can't you revert a commit that causes the regression?
> > > >
> > > > The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not
> > > > responsible for the regression. And the patchset even exnteds the
> > > > definition of the DMA API (dma_declare_coherent_memory). Such change
> > > > shouldn't applied after rc1. I think that DMA-API.txt says that
> > > > dma_declare_coherent_memory() handles coherent memory for a particular
> > > > device. It's not for the API that reserves coherent memory that can be
> > > > used for any device for a single device.
> > > The patch that made the problem obvious for ARM is
> > > 309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12.
> > > So this went in before v2.6.36-rc1. One of the "architectures which
> > > similar restrictions" is x86 BTW.
> > >
> > > And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it
> > > addresses a hardware restriction.
> >
> > How these drivers were able to work without hitting the hardware restriction?
>
> Well, OMAP processors have experienced lock-ups due to multiple mappings of
> memory, so the restriction in the architecture manual is for real.
>
> But more the issue is that the behaviour you get from a region is _totally_
> unpredictable (as the arch manual says). With the VIPT caches, they can
> be searched irrespective of whether the page tabkes indicate that it's
> supposed to be cached or not - which means you can still hit cache lines
> for an ioremap'd region.
>
> And if you do, how do you know that the cached data is still valid - what
> if it's some critical data that results in corruption - how do you know
> whether that's happened or not? It might not even cause a kernel
> exception.
>
> We have to adhere to the restrictions placed upon us by the architecture
> at hand, and if that means device drivers break, so be it - at least we
> get to know what needs to be fixed for these restrictions.

I didn't say the commit is technically wrong. I simply meant that the
commit broke some of working systems (so some complain, I guess).

As I wrote, the related DMA API wasn't changed in 2.6.36-rc1. It's not
related with the regression at all. As long as nobody tries to extend
the API wrongly after rc2, I have no complaint.

btw, Marin Mitov said that these drivers don't need coherent memory,
they just want contiguous memory. Telling the page allocater to
reserve some memory at boot time is enough, I guess.

2010-08-27 04:42:11

by Uwe Kleine-König

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

Hello,

On Thu, Aug 26, 2010 at 07:00:24PM +0900, FUJITA Tomonori wrote:
> On Thu, 26 Aug 2010 11:53:11 +0200
> Uwe Kleine-K?nig <[email protected]> wrote:
>
> > > > We have currently a number of boards broken in the mainline. They must be
> > > > fixed for 2.6.36. I don't think the mentioned API will do this for us. So,
> > > > as I suggested earlier, we need either this or my patch series
> > > >
> > > > http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
> > > >
> > > > for 2.6.36.
> > >
> > > Why can't you revert a commit that causes the regression?
> > >
> > > The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not
> > > responsible for the regression. And the patchset even exnteds the
> > > definition of the DMA API (dma_declare_coherent_memory). Such change
> > > shouldn't applied after rc1. I think that DMA-API.txt says that
> > > dma_declare_coherent_memory() handles coherent memory for a particular
> > > device. It's not for the API that reserves coherent memory that can be
> > > used for any device for a single device.
> > The patch that made the problem obvious for ARM is
> > 309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12.
> > So this went in before v2.6.36-rc1. One of the "architectures which
> > similar restrictions" is x86 BTW.
> >
> > And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it
> > addresses a hardware restriction.
>
> How these drivers were able to work without hitting the hardware restriction?
In my case the machine in question is an ARMv5, the hardware restriction
is on ARMv6+ only. You could argue that so the breaking patch for arm
should only break ARMv6, but I don't think this is sensible from a
maintainers POV. We need an API that works independant of the machine
that runs the code. And it's good to let developers that don't have the full
range of machines supported by the kernel at hand notice when they
introduce an incompatibility.

Best regards
Uwe

--
Pengutronix e.K. | Uwe Kleine-K?nig |
Industrial Linux Solutions | http://www.pengutronix.de/ |

2010-08-27 05:01:37

by FUJITA Tomonori

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

On Fri, 27 Aug 2010 06:41:42 +0200
Uwe Kleine-K$(D+S(Bnig <[email protected]> wrote:

> Hello,
>
> On Thu, Aug 26, 2010 at 07:00:24PM +0900, FUJITA Tomonori wrote:
> > On Thu, 26 Aug 2010 11:53:11 +0200
> > Uwe Kleine-K$(D+S(Bnig <[email protected]> wrote:
> >
> > > > > We have currently a number of boards broken in the mainline. They must be
> > > > > fixed for 2.6.36. I don't think the mentioned API will do this for us. So,
> > > > > as I suggested earlier, we need either this or my patch series
> > > > >
> > > > > http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
> > > > >
> > > > > for 2.6.36.
> > > >
> > > > Why can't you revert a commit that causes the regression?
> > > >
> > > > The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not
> > > > responsible for the regression. And the patchset even exnteds the
> > > > definition of the DMA API (dma_declare_coherent_memory). Such change
> > > > shouldn't applied after rc1. I think that DMA-API.txt says that
> > > > dma_declare_coherent_memory() handles coherent memory for a particular
> > > > device. It's not for the API that reserves coherent memory that can be
> > > > used for any device for a single device.
> > > The patch that made the problem obvious for ARM is
> > > 309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12.
> > > So this went in before v2.6.36-rc1. One of the "architectures which
> > > similar restrictions" is x86 BTW.
> > >
> > > And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it
> > > addresses a hardware restriction.
> >
> > How these drivers were able to work without hitting the hardware restriction?
> In my case the machine in question is an ARMv5, the hardware restriction
> is on ARMv6+ only. You could argue that so the breaking patch for arm
> should only break ARMv6, but I don't think this is sensible from a
> maintainers POV. We need an API that works independant of the machine
> that runs the code.

Agreed. But insisting that the DMA API needs to be extended wrongly
after rc2 to fix the regression is not sensible too. The related DMA
API wasn't changed in 2.6.36-rc1. The API isn't responsible for the
regression at all.

2010-08-27 05:19:32

by Uwe Kleine-König

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

Hey,

On Fri, Aug 27, 2010 at 02:00:17PM +0900, FUJITA Tomonori wrote:
> On Fri, 27 Aug 2010 06:41:42 +0200
> Uwe Kleine-K?nig <[email protected]> wrote:
> > On Thu, Aug 26, 2010 at 07:00:24PM +0900, FUJITA Tomonori wrote:
> > > On Thu, 26 Aug 2010 11:53:11 +0200
> > > Uwe Kleine-K?nig <[email protected]> wrote:
> > >
> > > > > > We have currently a number of boards broken in the mainline. They must be
> > > > > > fixed for 2.6.36. I don't think the mentioned API will do this for us. So,
> > > > > > as I suggested earlier, we need either this or my patch series
> > > > > >
> > > > > > http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
> > > > > >
> > > > > > for 2.6.36.
> > > > >
> > > > > Why can't you revert a commit that causes the regression?
> > > > >
> > > > > The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not
> > > > > responsible for the regression. And the patchset even exnteds the
> > > > > definition of the DMA API (dma_declare_coherent_memory). Such change
> > > > > shouldn't applied after rc1. I think that DMA-API.txt says that
> > > > > dma_declare_coherent_memory() handles coherent memory for a particular
> > > > > device. It's not for the API that reserves coherent memory that can be
> > > > > used for any device for a single device.
> > > > The patch that made the problem obvious for ARM is
> > > > 309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12.
> > > > So this went in before v2.6.36-rc1. One of the "architectures which
> > > > similar restrictions" is x86 BTW.
> > > >
> > > > And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it
> > > > addresses a hardware restriction.
> > >
> > > How these drivers were able to work without hitting the hardware restriction?
> > In my case the machine in question is an ARMv5, the hardware restriction
> > is on ARMv6+ only. You could argue that so the breaking patch for arm
> > should only break ARMv6, but I don't think this is sensible from a
> > maintainers POV. We need an API that works independant of the machine
> > that runs the code.
>
> Agreed. But insisting that the DMA API needs to be extended wrongly
> after rc2 to fix the regression is not sensible too. The related DMA
> API wasn't changed in 2.6.36-rc1. The API isn't responsible for the
> regression at all.
I think this isn't about "responsiblity". Someone in arm-land found
that the way dma memory allocation worked for some time doesn't work
anymore on new generation chips. As pointing out this problem was
expected to find some matches it was merged in the merge window. One
such match is the current usage of the DMA API that doesn't currently
offer a way to do it right, so it needs a patch, no?

Best regards
Uwe

--
Pengutronix e.K. | Uwe Kleine-K?nig |
Industrial Linux Solutions | http://www.pengutronix.de/ |

2010-08-27 05:59:18

by FUJITA Tomonori

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

On Fri, 27 Aug 2010 07:19:07 +0200
Uwe Kleine-K$(D+S(Bnig <[email protected]> wrote:

> Hey,
>
> On Fri, Aug 27, 2010 at 02:00:17PM +0900, FUJITA Tomonori wrote:
> > On Fri, 27 Aug 2010 06:41:42 +0200
> > Uwe Kleine-K$(D+S(Bnig <[email protected]> wrote:
> > > On Thu, Aug 26, 2010 at 07:00:24PM +0900, FUJITA Tomonori wrote:
> > > > On Thu, 26 Aug 2010 11:53:11 +0200
> > > > Uwe Kleine-K$(D+S(Bnig <[email protected]> wrote:
> > > >
> > > > > > > We have currently a number of boards broken in the mainline. They must be
> > > > > > > fixed for 2.6.36. I don't think the mentioned API will do this for us. So,
> > > > > > > as I suggested earlier, we need either this or my patch series
> > > > > > >
> > > > > > > http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
> > > > > > >
> > > > > > > for 2.6.36.
> > > > > >
> > > > > > Why can't you revert a commit that causes the regression?
> > > > > >
> > > > > > The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not
> > > > > > responsible for the regression. And the patchset even exnteds the
> > > > > > definition of the DMA API (dma_declare_coherent_memory). Such change
> > > > > > shouldn't applied after rc1. I think that DMA-API.txt says that
> > > > > > dma_declare_coherent_memory() handles coherent memory for a particular
> > > > > > device. It's not for the API that reserves coherent memory that can be
> > > > > > used for any device for a single device.
> > > > > The patch that made the problem obvious for ARM is
> > > > > 309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12.
> > > > > So this went in before v2.6.36-rc1. One of the "architectures which
> > > > > similar restrictions" is x86 BTW.
> > > > >
> > > > > And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it
> > > > > addresses a hardware restriction.
> > > >
> > > > How these drivers were able to work without hitting the hardware restriction?
> > > In my case the machine in question is an ARMv5, the hardware restriction
> > > is on ARMv6+ only. You could argue that so the breaking patch for arm
> > > should only break ARMv6, but I don't think this is sensible from a
> > > maintainers POV. We need an API that works independant of the machine
> > > that runs the code.
> >
> > Agreed. But insisting that the DMA API needs to be extended wrongly
> > after rc2 to fix the regression is not sensible too. The related DMA
> > API wasn't changed in 2.6.36-rc1. The API isn't responsible for the
> > regression at all.
> I think this isn't about "responsiblity". Someone in arm-land found
> that the way dma memory allocation worked for some time doesn't work
> anymore on new generation chips. As pointing out this problem was
> expected to find some matches it was merged in the merge window. One
> such match is the current usage of the DMA API that doesn't currently
> offer a way to do it right, so it needs a patch, no?

No, I don't think so. We are talking about a regression, right?

On new generation chips, something often doesn't work (which have
worked on old chips for some time). It's not a regresiion. I don't
think that it's sensible to make large change (especially after rc1)
to fix such issue. If you say that the DMA API doesn't work on new
chips and proposes a patch for the next merge window, it's sensible, I
suppose.

Btw, the patch isn't a fix for the DMA API. It tries to extend the DMA
API (and IMO in the wrong way). In addition, the patch might break the
current code. I really don't think that applying such patch after rc1
is senseble.

2010-08-27 06:14:23

by Uwe Kleine-König

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

Hello,

On Fri, Aug 27, 2010 at 02:57:59PM +0900, FUJITA Tomonori wrote:
> On Fri, 27 Aug 2010 07:19:07 +0200
> Uwe Kleine-K?nig <[email protected]> wrote:
>
> > Hey,
> >
> > On Fri, Aug 27, 2010 at 02:00:17PM +0900, FUJITA Tomonori wrote:
> > > On Fri, 27 Aug 2010 06:41:42 +0200
> > > Uwe Kleine-K?nig <[email protected]> wrote:
> > > > On Thu, Aug 26, 2010 at 07:00:24PM +0900, FUJITA Tomonori wrote:
> > > > > On Thu, 26 Aug 2010 11:53:11 +0200
> > > > > Uwe Kleine-K?nig <[email protected]> wrote:
> > > > >
> > > > > > > > We have currently a number of boards broken in the mainline. They must be
> > > > > > > > fixed for 2.6.36. I don't think the mentioned API will do this for us. So,
> > > > > > > > as I suggested earlier, we need either this or my patch series
> > > > > > > >
> > > > > > > > http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
> > > > > > > >
> > > > > > > > for 2.6.36.
> > > > > > >
> > > > > > > Why can't you revert a commit that causes the regression?
> > > > > > >
> > > > > > > The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not
> > > > > > > responsible for the regression. And the patchset even exnteds the
> > > > > > > definition of the DMA API (dma_declare_coherent_memory). Such change
> > > > > > > shouldn't applied after rc1. I think that DMA-API.txt says that
> > > > > > > dma_declare_coherent_memory() handles coherent memory for a particular
> > > > > > > device. It's not for the API that reserves coherent memory that can be
> > > > > > > used for any device for a single device.
> > > > > > The patch that made the problem obvious for ARM is
> > > > > > 309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12.
> > > > > > So this went in before v2.6.36-rc1. One of the "architectures which
> > > > > > similar restrictions" is x86 BTW.
> > > > > >
> > > > > > And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it
> > > > > > addresses a hardware restriction.
> > > > >
> > > > > How these drivers were able to work without hitting the hardware restriction?
> > > > In my case the machine in question is an ARMv5, the hardware restriction
> > > > is on ARMv6+ only. You could argue that so the breaking patch for arm
> > > > should only break ARMv6, but I don't think this is sensible from a
> > > > maintainers POV. We need an API that works independant of the machine
> > > > that runs the code.
> > >
> > > Agreed. But insisting that the DMA API needs to be extended wrongly
> > > after rc2 to fix the regression is not sensible too. The related DMA
> > > API wasn't changed in 2.6.36-rc1. The API isn't responsible for the
> > > regression at all.
> > I think this isn't about "responsiblity". Someone in arm-land found
> > that the way dma memory allocation worked for some time doesn't work
> > anymore on new generation chips. As pointing out this problem was
> > expected to find some matches it was merged in the merge window. One
> > such match is the current usage of the DMA API that doesn't currently
> > offer a way to do it right, so it needs a patch, no?
>
> No, I don't think so. We are talking about a regression, right?
>
> On new generation chips, something often doesn't work (which have
> worked on old chips for some time). It's not a regresiion. I don't
> think that it's sensible to make large change (especially after rc1)
> to fix such issue. If you say that the DMA API doesn't work on new
> chips and proposes a patch for the next merge window, it's sensible, I
> suppose.
>
> Btw, the patch isn't a fix for the DMA API. It tries to extend the DMA
> API (and IMO in the wrong way). In addition, the patch might break the
> current code. I really don't think that applying such patch after rc1
> is senseble.
So you suggest to revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 or at
least restrict it to ARMv6+ and fix the problem during the next merge
window? Russell?

Best regards
Uwe

--
Pengutronix e.K. | Uwe Kleine-K?nig |
Industrial Linux Solutions | http://www.pengutronix.de/ |

2010-08-27 06:26:57

by Marin Mitov

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

On Friday, August 27, 2010 08:57:59 am FUJITA Tomonori wrote:
> On Fri, 27 Aug 2010 07:19:07 +0200
> Uwe Kleine-K$(D+S(Bnig <[email protected]> wrote:
>
> > Hey,
> >
> > On Fri, Aug 27, 2010 at 02:00:17PM +0900, FUJITA Tomonori wrote:
> > > On Fri, 27 Aug 2010 06:41:42 +0200
> > > Uwe Kleine-K$(D+S(Bnig <[email protected]> wrote:
> > > > On Thu, Aug 26, 2010 at 07:00:24PM +0900, FUJITA Tomonori wrote:
> > > > > On Thu, 26 Aug 2010 11:53:11 +0200
> > > > > Uwe Kleine-K$(D+S(Bnig <[email protected]> wrote:
> > > > >
> > > > > > > > We have currently a number of boards broken in the mainline. They must be
> > > > > > > > fixed for 2.6.36. I don't think the mentioned API will do this for us. So,
> > > > > > > > as I suggested earlier, we need either this or my patch series
> > > > > > > >
> > > > > > > > http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
> > > > > > > >
> > > > > > > > for 2.6.36.
> > > > > > >
> > > > > > > Why can't you revert a commit that causes the regression?
> > > > > > >
> > > > > > > The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not
> > > > > > > responsible for the regression. And the patchset even exnteds the
> > > > > > > definition of the DMA API (dma_declare_coherent_memory). Such change
> > > > > > > shouldn't applied after rc1. I think that DMA-API.txt says that
> > > > > > > dma_declare_coherent_memory() handles coherent memory for a particular
> > > > > > > device. It's not for the API that reserves coherent memory that can be
> > > > > > > used for any device for a single device.
> > > > > > The patch that made the problem obvious for ARM is
> > > > > > 309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12.
> > > > > > So this went in before v2.6.36-rc1. One of the "architectures which
> > > > > > similar restrictions" is x86 BTW.
> > > > > >
> > > > > > And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it
> > > > > > addresses a hardware restriction.
> > > > >
> > > > > How these drivers were able to work without hitting the hardware restriction?
> > > > In my case the machine in question is an ARMv5, the hardware restriction
> > > > is on ARMv6+ only. You could argue that so the breaking patch for arm
> > > > should only break ARMv6, but I don't think this is sensible from a
> > > > maintainers POV. We need an API that works independant of the machine
> > > > that runs the code.
> > >
> > > Agreed. But insisting that the DMA API needs to be extended wrongly
> > > after rc2 to fix the regression is not sensible too. The related DMA
> > > API wasn't changed in 2.6.36-rc1. The API isn't responsible for the
> > > regression at all.
> > I think this isn't about "responsiblity". Someone in arm-land found
> > that the way dma memory allocation worked for some time doesn't work
> > anymore on new generation chips. As pointing out this problem was
> > expected to find some matches it was merged in the merge window. One
> > such match is the current usage of the DMA API that doesn't currently
> > offer a way to do it right, so it needs a patch, no?
>
> No, I don't think so. We are talking about a regression, right?
>
> On new generation chips, something often doesn't work (which have
> worked on old chips for some time). It's not a regresiion. I don't
> think that it's sensible to make large change (especially after rc1)
> to fix such issue. If you say that the DMA API doesn't work on new
> chips and proposes a patch for the next merge window, it's sensible, I
> suppose.
>
> Btw, the patch isn't a fix for the DMA API. It tries to extend the DMA
> API (and IMO in the wrong way).
> In addition, the patch might break the
> current code.

To "break the current code" is simply not possible. Sorry to oppose. As you have written it
"extend the DMA API", so if you do not use the new API (and no current code is using it)
you cannot "break the current code".

Thanks,

Marin Mitov

> I really don't think that applying such patch after rc1
> is senseble.
>

2010-08-27 06:33:55

by FUJITA Tomonori

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

On Fri, 27 Aug 2010 09:23:21 +0300
Marin Mitov <[email protected]> wrote:

> On Friday, August 27, 2010 08:57:59 am FUJITA Tomonori wrote:
> > On Fri, 27 Aug 2010 07:19:07 +0200
> > Uwe Kleine-K$(D+S(Bnig <[email protected]> wrote:
> >
> > > Hey,
> > >
> > > On Fri, Aug 27, 2010 at 02:00:17PM +0900, FUJITA Tomonori wrote:
> > > > On Fri, 27 Aug 2010 06:41:42 +0200
> > > > Uwe Kleine-K$(D+S(Bnig <[email protected]> wrote:
> > > > > On Thu, Aug 26, 2010 at 07:00:24PM +0900, FUJITA Tomonori wrote:
> > > > > > On Thu, 26 Aug 2010 11:53:11 +0200
> > > > > > Uwe Kleine-K$(D+S(Bnig <[email protected]> wrote:
> > > > > >
> > > > > > > > > We have currently a number of boards broken in the mainline. They must be
> > > > > > > > > fixed for 2.6.36. I don't think the mentioned API will do this for us. So,
> > > > > > > > > as I suggested earlier, we need either this or my patch series
> > > > > > > > >
> > > > > > > > > http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
> > > > > > > > >
> > > > > > > > > for 2.6.36.
> > > > > > > >
> > > > > > > > Why can't you revert a commit that causes the regression?
> > > > > > > >
> > > > > > > > The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not
> > > > > > > > responsible for the regression. And the patchset even exnteds the
> > > > > > > > definition of the DMA API (dma_declare_coherent_memory). Such change
> > > > > > > > shouldn't applied after rc1. I think that DMA-API.txt says that
> > > > > > > > dma_declare_coherent_memory() handles coherent memory for a particular
> > > > > > > > device. It's not for the API that reserves coherent memory that can be
> > > > > > > > used for any device for a single device.
> > > > > > > The patch that made the problem obvious for ARM is
> > > > > > > 309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12.
> > > > > > > So this went in before v2.6.36-rc1. One of the "architectures which
> > > > > > > similar restrictions" is x86 BTW.
> > > > > > >
> > > > > > > And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it
> > > > > > > addresses a hardware restriction.
> > > > > >
> > > > > > How these drivers were able to work without hitting the hardware restriction?
> > > > > In my case the machine in question is an ARMv5, the hardware restriction
> > > > > is on ARMv6+ only. You could argue that so the breaking patch for arm
> > > > > should only break ARMv6, but I don't think this is sensible from a
> > > > > maintainers POV. We need an API that works independant of the machine
> > > > > that runs the code.
> > > >
> > > > Agreed. But insisting that the DMA API needs to be extended wrongly
> > > > after rc2 to fix the regression is not sensible too. The related DMA
> > > > API wasn't changed in 2.6.36-rc1. The API isn't responsible for the
> > > > regression at all.
> > > I think this isn't about "responsiblity". Someone in arm-land found
> > > that the way dma memory allocation worked for some time doesn't work
> > > anymore on new generation chips. As pointing out this problem was
> > > expected to find some matches it was merged in the merge window. One
> > > such match is the current usage of the DMA API that doesn't currently
> > > offer a way to do it right, so it needs a patch, no?
> >
> > No, I don't think so. We are talking about a regression, right?
> >
> > On new generation chips, something often doesn't work (which have
> > worked on old chips for some time). It's not a regresiion. I don't
> > think that it's sensible to make large change (especially after rc1)
> > to fix such issue. If you say that the DMA API doesn't work on new
> > chips and proposes a patch for the next merge window, it's sensible, I
> > suppose.
> >
> > Btw, the patch isn't a fix for the DMA API. It tries to extend the DMA
> > API (and IMO in the wrong way).
> > In addition, the patch might break the
> > current code.
>
> To "break the current code" is simply not possible. Sorry to oppose. As you have written it
> "extend the DMA API", so if you do not use the new API (and no current code is using it)
> you cannot "break the current code".

Looks like that the patch adds the new API that touches the exisitng
code. It means the existing code could break. So the exsising API
could break too.

http://thread.gmane.org/gmane.linux.ports.sh.devel/8595

2010-08-27 06:39:17

by Uwe Kleine-König

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

Hello,

On Fri, Aug 27, 2010 at 03:32:14PM +0900, FUJITA Tomonori wrote:
> On Fri, 27 Aug 2010 09:23:21 +0300
> Marin Mitov <[email protected]> wrote:
>
> > On Friday, August 27, 2010 08:57:59 am FUJITA Tomonori wrote:
> > > On Fri, 27 Aug 2010 07:19:07 +0200
> > > Uwe Kleine-K$(D+S(Bnig <[email protected]> wrote:
> > >
> > > > Hey,
> > > >
> > > > On Fri, Aug 27, 2010 at 02:00:17PM +0900, FUJITA Tomonori wrote:
> > > > > On Fri, 27 Aug 2010 06:41:42 +0200
> > > > > Uwe Kleine-K$(D+S(Bnig <[email protected]> wrote:
> > > > > > On Thu, Aug 26, 2010 at 07:00:24PM +0900, FUJITA Tomonori wrote:
> > > > > > > On Thu, 26 Aug 2010 11:53:11 +0200
> > > > > > > Uwe Kleine-K$(D+S(Bnig <[email protected]> wrote:
> > > > > > >
> > > > > > > > > > We have currently a number of boards broken in the mainline. They must be
> > > > > > > > > > fixed for 2.6.36. I don't think the mentioned API will do this for us. So,
> > > > > > > > > > as I suggested earlier, we need either this or my patch series
> > > > > > > > > >
> > > > > > > > > > http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
> > > > > > > > > >
> > > > > > > > > > for 2.6.36.
> > > > > > > > >
> > > > > > > > > Why can't you revert a commit that causes the regression?
> > > > > > > > >
> > > > > > > > > The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not
> > > > > > > > > responsible for the regression. And the patchset even exnteds the
> > > > > > > > > definition of the DMA API (dma_declare_coherent_memory). Such change
> > > > > > > > > shouldn't applied after rc1. I think that DMA-API.txt says that
> > > > > > > > > dma_declare_coherent_memory() handles coherent memory for a particular
> > > > > > > > > device. It's not for the API that reserves coherent memory that can be
> > > > > > > > > used for any device for a single device.
> > > > > > > > The patch that made the problem obvious for ARM is
> > > > > > > > 309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12.
> > > > > > > > So this went in before v2.6.36-rc1. One of the "architectures which
> > > > > > > > similar restrictions" is x86 BTW.
> > > > > > > >
> > > > > > > > And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it
> > > > > > > > addresses a hardware restriction.
> > > > > > >
> > > > > > > How these drivers were able to work without hitting the hardware restriction?
> > > > > > In my case the machine in question is an ARMv5, the hardware restriction
> > > > > > is on ARMv6+ only. You could argue that so the breaking patch for arm
> > > > > > should only break ARMv6, but I don't think this is sensible from a
> > > > > > maintainers POV. We need an API that works independant of the machine
> > > > > > that runs the code.
> > > > >
> > > > > Agreed. But insisting that the DMA API needs to be extended wrongly
> > > > > after rc2 to fix the regression is not sensible too. The related DMA
> > > > > API wasn't changed in 2.6.36-rc1. The API isn't responsible for the
> > > > > regression at all.
> > > > I think this isn't about "responsiblity". Someone in arm-land found
> > > > that the way dma memory allocation worked for some time doesn't work
> > > > anymore on new generation chips. As pointing out this problem was
> > > > expected to find some matches it was merged in the merge window. One
> > > > such match is the current usage of the DMA API that doesn't currently
> > > > offer a way to do it right, so it needs a patch, no?
> > >
> > > No, I don't think so. We are talking about a regression, right?
> > >
> > > On new generation chips, something often doesn't work (which have
> > > worked on old chips for some time). It's not a regresiion. I don't
> > > think that it's sensible to make large change (especially after rc1)
> > > to fix such issue. If you say that the DMA API doesn't work on new
> > > chips and proposes a patch for the next merge window, it's sensible, I
> > > suppose.
> > >
> > > Btw, the patch isn't a fix for the DMA API. It tries to extend the DMA
> > > API (and IMO in the wrong way).
> > > In addition, the patch might break the
> > > current code.
> >
> > To "break the current code" is simply not possible. Sorry to oppose. As you have written it
> > "extend the DMA API", so if you do not use the new API (and no current code is using it)
> > you cannot "break the current code".
>
> Looks like that the patch adds the new API that touches the exisitng
> code. It means the existing code could break. So the exsising API
> could break too.
>
> http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
I'm still trying to find out what you actually suggest we should do now.
Maybe this is a request for a minimal "fix" without the cleanups
Guennadi did? That is only patches 2(?), 4 and 5 of the series?

Best regards
Uwe

--
Pengutronix e.K. | Uwe Kleine-K?nig |
Industrial Linux Solutions | http://www.pengutronix.de/ |

2010-08-27 07:05:27

by Marin Mitov

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

On Friday, August 27, 2010 09:32:14 am FUJITA Tomonori wrote:
> On Fri, 27 Aug 2010 09:23:21 +0300
> Marin Mitov <[email protected]> wrote:
>
> > On Friday, August 27, 2010 08:57:59 am FUJITA Tomonori wrote:
> > > On Fri, 27 Aug 2010 07:19:07 +0200
> > > Uwe Kleine-K$(D+S(Bnig <[email protected]> wrote:
> > >
> > > > Hey,
> > > >
> > > > On Fri, Aug 27, 2010 at 02:00:17PM +0900, FUJITA Tomonori wrote:
> > > > > On Fri, 27 Aug 2010 06:41:42 +0200
> > > > > Uwe Kleine-K$(D+S(Bnig <[email protected]> wrote:
> > > > > > On Thu, Aug 26, 2010 at 07:00:24PM +0900, FUJITA Tomonori wrote:
> > > > > > > On Thu, 26 Aug 2010 11:53:11 +0200
> > > > > > > Uwe Kleine-K$(D+S(Bnig <[email protected]> wrote:
> > > > > > >
> > > > > > > > > > We have currently a number of boards broken in the mainline. They must be
> > > > > > > > > > fixed for 2.6.36. I don't think the mentioned API will do this for us. So,
> > > > > > > > > > as I suggested earlier, we need either this or my patch series
> > > > > > > > > >
> > > > > > > > > > http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
> > > > > > > > > >
> > > > > > > > > > for 2.6.36.
> > > > > > > > >
> > > > > > > > > Why can't you revert a commit that causes the regression?
> > > > > > > > >
> > > > > > > > > The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not
> > > > > > > > > responsible for the regression. And the patchset even exnteds the
> > > > > > > > > definition of the DMA API (dma_declare_coherent_memory). Such change
> > > > > > > > > shouldn't applied after rc1. I think that DMA-API.txt says that
> > > > > > > > > dma_declare_coherent_memory() handles coherent memory for a particular
> > > > > > > > > device. It's not for the API that reserves coherent memory that can be
> > > > > > > > > used for any device for a single device.
> > > > > > > > The patch that made the problem obvious for ARM is
> > > > > > > > 309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12.
> > > > > > > > So this went in before v2.6.36-rc1. One of the "architectures which
> > > > > > > > similar restrictions" is x86 BTW.
> > > > > > > >
> > > > > > > > And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it
> > > > > > > > addresses a hardware restriction.
> > > > > > >
> > > > > > > How these drivers were able to work without hitting the hardware restriction?
> > > > > > In my case the machine in question is an ARMv5, the hardware restriction
> > > > > > is on ARMv6+ only. You could argue that so the breaking patch for arm
> > > > > > should only break ARMv6, but I don't think this is sensible from a
> > > > > > maintainers POV. We need an API that works independant of the machine
> > > > > > that runs the code.
> > > > >
> > > > > Agreed. But insisting that the DMA API needs to be extended wrongly
> > > > > after rc2 to fix the regression is not sensible too. The related DMA
> > > > > API wasn't changed in 2.6.36-rc1. The API isn't responsible for the
> > > > > regression at all.
> > > > I think this isn't about "responsiblity". Someone in arm-land found
> > > > that the way dma memory allocation worked for some time doesn't work
> > > > anymore on new generation chips. As pointing out this problem was
> > > > expected to find some matches it was merged in the merge window. One
> > > > such match is the current usage of the DMA API that doesn't currently
> > > > offer a way to do it right, so it needs a patch, no?
> > >
> > > No, I don't think so. We are talking about a regression, right?
> > >
> > > On new generation chips, something often doesn't work (which have
> > > worked on old chips for some time). It's not a regresiion. I don't
> > > think that it's sensible to make large change (especially after rc1)
> > > to fix such issue. If you say that the DMA API doesn't work on new
> > > chips and proposes a patch for the next merge window, it's sensible, I
> > > suppose.
> > >
> > > Btw, the patch isn't a fix for the DMA API. It tries to extend the DMA
> > > API (and IMO in the wrong way).
> > > In addition, the patch might break the
> > > current code.
> >
> > To "break the current code" is simply not possible. Sorry to oppose. As you have written it
> > "extend the DMA API", so if you do not use the new API (and no current code is using it)
> > you cannot "break the current code".
>
> Looks like that the patch adds the new API that touches the exisitng
> code. It means the existing code could break. So the exsising API
> could break too.
>
> http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>

2010-08-28 06:17:56

by Marin Mitov

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

On Friday, August 27, 2010 09:32:14 am FUJITA Tomonori wrote:
> On Fri, 27 Aug 2010 09:23:21 +0300
> Marin Mitov <[email protected]> wrote:
>
> > On Friday, August 27, 2010 08:57:59 am FUJITA Tomonori wrote:
> > > On Fri, 27 Aug 2010 07:19:07 +0200
> > > Uwe Kleine-K$(D+S(Bnig <[email protected]> wrote:
> > >
> > > > Hey,
> > > >
> > > > On Fri, Aug 27, 2010 at 02:00:17PM +0900, FUJITA Tomonori wrote:
> > > > > On Fri, 27 Aug 2010 06:41:42 +0200
> > > > > Uwe Kleine-K$(D+S(Bnig <[email protected]> wrote:
> > > > > > On Thu, Aug 26, 2010 at 07:00:24PM +0900, FUJITA Tomonori wrote:
> > > > > > > On Thu, 26 Aug 2010 11:53:11 +0200
> > > > > > > Uwe Kleine-K$(D+S(Bnig <[email protected]> wrote:
> > > > > > >
> > > > > > > > > > We have currently a number of boards broken in the mainline. They must be
> > > > > > > > > > fixed for 2.6.36. I don't think the mentioned API will do this for us. So,
> > > > > > > > > > as I suggested earlier, we need either this or my patch series
> > > > > > > > > >
> > > > > > > > > > http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
> > > > > > > > > >
> > > > > > > > > > for 2.6.36.
> > > > > > > > >
> > > > > > > > > Why can't you revert a commit that causes the regression?
> > > > > > > > >
> > > > > > > > > The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not
> > > > > > > > > responsible for the regression. And the patchset even exnteds the
> > > > > > > > > definition of the DMA API (dma_declare_coherent_memory). Such change
> > > > > > > > > shouldn't applied after rc1. I think that DMA-API.txt says that
> > > > > > > > > dma_declare_coherent_memory() handles coherent memory for a particular
> > > > > > > > > device. It's not for the API that reserves coherent memory that can be
> > > > > > > > > used for any device for a single device.
> > > > > > > > The patch that made the problem obvious for ARM is
> > > > > > > > 309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12.
> > > > > > > > So this went in before v2.6.36-rc1. One of the "architectures which
> > > > > > > > similar restrictions" is x86 BTW.
> > > > > > > >
> > > > > > > > And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it
> > > > > > > > addresses a hardware restriction.
> > > > > > >
> > > > > > > How these drivers were able to work without hitting the hardware restriction?
> > > > > > In my case the machine in question is an ARMv5, the hardware restriction
> > > > > > is on ARMv6+ only. You could argue that so the breaking patch for arm
> > > > > > should only break ARMv6, but I don't think this is sensible from a
> > > > > > maintainers POV. We need an API that works independant of the machine
> > > > > > that runs the code.
> > > > >
> > > > > Agreed. But insisting that the DMA API needs to be extended wrongly
> > > > > after rc2 to fix the regression is not sensible too. The related DMA
> > > > > API wasn't changed in 2.6.36-rc1. The API isn't responsible for the
> > > > > regression at all.
> > > > I think this isn't about "responsiblity". Someone in arm-land found
> > > > that the way dma memory allocation worked for some time doesn't work
> > > > anymore on new generation chips. As pointing out this problem was
> > > > expected to find some matches it was merged in the merge window. One
> > > > such match is the current usage of the DMA API that doesn't currently
> > > > offer a way to do it right, so it needs a patch, no?
> > >
> > > No, I don't think so. We are talking about a regression, right?
> > >
> > > On new generation chips, something often doesn't work (which have
> > > worked on old chips for some time). It's not a regresiion. I don't
> > > think that it's sensible to make large change (especially after rc1)
> > > to fix such issue. If you say that the DMA API doesn't work on new
> > > chips and proposes a patch for the next merge window, it's sensible, I
> > > suppose.
> > >
> > > Btw, the patch isn't a fix for the DMA API. It tries to extend the DMA
> > > API (and IMO in the wrong way).
> > > In addition, the patch might break the
> > > current code.
> >
> > To "break the current code" is simply not possible. Sorry to oppose. As you have written it
> > "extend the DMA API", so if you do not use the new API (and no current code is using it)
> > you cannot "break the current code".
>
> Looks like that the patch adds the new API that touches the exisitng
> code. It means the existing code could break. So the exsising API
> could break too.
>
> http://thread.gmane.org/gmane.linux.ports.sh.devel/8595

The above reference is not my patch. I am speaking for my patch:

http://lkml.org/lkml/2010/8/19/200

The only point my patch touches the existing code is struct device's member dma_mem
and that is in condition you __use__ the new API, so you could decide yourself if it
could break the current code. As far as one does not use the new API - nothing is touched,
nothing can break. If one uses the new API, only the user can suffer if the new API have
bugs.

Thanks,

Marin Mitov

2010-08-28 07:12:12

by FUJITA Tomonori

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

On Sat, 28 Aug 2010 09:14:25 +0300
Marin Mitov <[email protected]> wrote:

> On Friday, August 27, 2010 09:32:14 am FUJITA Tomonori wrote:
> > On Fri, 27 Aug 2010 09:23:21 +0300
> > Marin Mitov <[email protected]> wrote:
> >
> > > On Friday, August 27, 2010 08:57:59 am FUJITA Tomonori wrote:
> > > > On Fri, 27 Aug 2010 07:19:07 +0200
> > > > Uwe Kleine-K$(D+S(Bnig <[email protected]> wrote:
> > > >
> > > > > Hey,
> > > > >
> > > > > On Fri, Aug 27, 2010 at 02:00:17PM +0900, FUJITA Tomonori wrote:
> > > > > > On Fri, 27 Aug 2010 06:41:42 +0200
> > > > > > Uwe Kleine-K$(D+S(Bnig <[email protected]> wrote:
> > > > > > > On Thu, Aug 26, 2010 at 07:00:24PM +0900, FUJITA Tomonori wrote:
> > > > > > > > On Thu, 26 Aug 2010 11:53:11 +0200
> > > > > > > > Uwe Kleine-K$(D+S(Bnig <[email protected]> wrote:
> > > > > > > >
> > > > > > > > > > > We have currently a number of boards broken in the mainline. They must be
> > > > > > > > > > > fixed for 2.6.36. I don't think the mentioned API will do this for us. So,
> > > > > > > > > > > as I suggested earlier, we need either this or my patch series
> > > > > > > > > > >
> > > > > > > > > > > http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
> > > > > > > > > > >
> > > > > > > > > > > for 2.6.36.
> > > > > > > > > >
> > > > > > > > > > Why can't you revert a commit that causes the regression?
> > > > > > > > > >
> > > > > > > > > > The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not
> > > > > > > > > > responsible for the regression. And the patchset even exnteds the
> > > > > > > > > > definition of the DMA API (dma_declare_coherent_memory). Such change
> > > > > > > > > > shouldn't applied after rc1. I think that DMA-API.txt says that
> > > > > > > > > > dma_declare_coherent_memory() handles coherent memory for a particular
> > > > > > > > > > device. It's not for the API that reserves coherent memory that can be
> > > > > > > > > > used for any device for a single device.
> > > > > > > > > The patch that made the problem obvious for ARM is
> > > > > > > > > 309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12.
> > > > > > > > > So this went in before v2.6.36-rc1. One of the "architectures which
> > > > > > > > > similar restrictions" is x86 BTW.
> > > > > > > > >
> > > > > > > > > And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it
> > > > > > > > > addresses a hardware restriction.
> > > > > > > >
> > > > > > > > How these drivers were able to work without hitting the hardware restriction?
> > > > > > > In my case the machine in question is an ARMv5, the hardware restriction
> > > > > > > is on ARMv6+ only. You could argue that so the breaking patch for arm
> > > > > > > should only break ARMv6, but I don't think this is sensible from a
> > > > > > > maintainers POV. We need an API that works independant of the machine
> > > > > > > that runs the code.
> > > > > >
> > > > > > Agreed. But insisting that the DMA API needs to be extended wrongly
> > > > > > after rc2 to fix the regression is not sensible too. The related DMA
> > > > > > API wasn't changed in 2.6.36-rc1. The API isn't responsible for the
> > > > > > regression at all.
> > > > > I think this isn't about "responsiblity". Someone in arm-land found
> > > > > that the way dma memory allocation worked for some time doesn't work
> > > > > anymore on new generation chips. As pointing out this problem was
> > > > > expected to find some matches it was merged in the merge window. One
> > > > > such match is the current usage of the DMA API that doesn't currently
> > > > > offer a way to do it right, so it needs a patch, no?
> > > >
> > > > No, I don't think so. We are talking about a regression, right?
> > > >
> > > > On new generation chips, something often doesn't work (which have
> > > > worked on old chips for some time). It's not a regresiion. I don't
> > > > think that it's sensible to make large change (especially after rc1)
> > > > to fix such issue. If you say that the DMA API doesn't work on new
> > > > chips and proposes a patch for the next merge window, it's sensible, I
> > > > suppose.
> > > >
> > > > Btw, the patch isn't a fix for the DMA API. It tries to extend the DMA
> > > > API (and IMO in the wrong way).
> > > > In addition, the patch might break the
> > > > current code.
> > >
> > > To "break the current code" is simply not possible. Sorry to oppose. As you have written it
> > > "extend the DMA API", so if you do not use the new API (and no current code is using it)
> > > you cannot "break the current code".
> >
> > Looks like that the patch adds the new API that touches the exisitng
> > code. It means the existing code could break. So the exsising API
> > could break too.
> >
> > http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
>
> The above reference is not my patch. I am speaking for my patch:
>
> http://lkml.org/lkml/2010/8/19/200

I think that I already NACK'ed the patch.

1) drivers/media/videobuf-dma-contig.c should not use
dma_alloc_coherent. We shouldn't support the proposed API.

2) I don't think that the DMA API (drivers/base/dma-mapping.c) is not
for creating "cache". Generally, the kernel uses "pool" concept for
something like that.

IMHO, reverting the commit 309caa9cc6ff39d261264ec4ff10e29489afc8f8
temporary (or temporary disabling it for systems that had worked) is
the most reasonable approach. I don't think that breaking systems that
had worked is a good idea even if the patch does the right thing. I
believe that we need to fix the broken solution
(videobuf-dma-contig.c) before the commit.

2010-08-28 07:22:49

by Marin Mitov

[permalink] [raw]
Subject: Re: [RFC][PATCH] add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

On Saturday, August 28, 2010 10:10:28 am FUJITA Tomonori wrote:
> On Sat, 28 Aug 2010 09:14:25 +0300
> Marin Mitov <[email protected]> wrote:
>
> > On Friday, August 27, 2010 09:32:14 am FUJITA Tomonori wrote:
> > > On Fri, 27 Aug 2010 09:23:21 +0300
> > > Marin Mitov <[email protected]> wrote:
> > >
> > > > On Friday, August 27, 2010 08:57:59 am FUJITA Tomonori wrote:
> > > > > On Fri, 27 Aug 2010 07:19:07 +0200
> > > > > Uwe Kleine-K$(D+S(Bnig <[email protected]> wrote:
> > > > >
> > > > > > Hey,
> > > > > >
> > > > > > On Fri, Aug 27, 2010 at 02:00:17PM +0900, FUJITA Tomonori wrote:
> > > > > > > On Fri, 27 Aug 2010 06:41:42 +0200
> > > > > > > Uwe Kleine-K$(D+S(Bnig <[email protected]> wrote:
> > > > > > > > On Thu, Aug 26, 2010 at 07:00:24PM +0900, FUJITA Tomonori wrote:
> > > > > > > > > On Thu, 26 Aug 2010 11:53:11 +0200
> > > > > > > > > Uwe Kleine-K$(D+S(Bnig <[email protected]> wrote:
> > > > > > > > >
> > > > > > > > > > > > We have currently a number of boards broken in the mainline. They must be
> > > > > > > > > > > > fixed for 2.6.36. I don't think the mentioned API will do this for us. So,
> > > > > > > > > > > > as I suggested earlier, we need either this or my patch series
> > > > > > > > > > > >
> > > > > > > > > > > > http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
> > > > > > > > > > > >
> > > > > > > > > > > > for 2.6.36.
> > > > > > > > > > >
> > > > > > > > > > > Why can't you revert a commit that causes the regression?
> > > > > > > > > > >
> > > > > > > > > > > The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not
> > > > > > > > > > > responsible for the regression. And the patchset even exnteds the
> > > > > > > > > > > definition of the DMA API (dma_declare_coherent_memory). Such change
> > > > > > > > > > > shouldn't applied after rc1. I think that DMA-API.txt says that
> > > > > > > > > > > dma_declare_coherent_memory() handles coherent memory for a particular
> > > > > > > > > > > device. It's not for the API that reserves coherent memory that can be
> > > > > > > > > > > used for any device for a single device.
> > > > > > > > > > The patch that made the problem obvious for ARM is
> > > > > > > > > > 309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12.
> > > > > > > > > > So this went in before v2.6.36-rc1. One of the "architectures which
> > > > > > > > > > similar restrictions" is x86 BTW.
> > > > > > > > > >
> > > > > > > > > > And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it
> > > > > > > > > > addresses a hardware restriction.
> > > > > > > > >
> > > > > > > > > How these drivers were able to work without hitting the hardware restriction?
> > > > > > > > In my case the machine in question is an ARMv5, the hardware restriction
> > > > > > > > is on ARMv6+ only. You could argue that so the breaking patch for arm
> > > > > > > > should only break ARMv6, but I don't think this is sensible from a
> > > > > > > > maintainers POV. We need an API that works independant of the machine
> > > > > > > > that runs the code.
> > > > > > >
> > > > > > > Agreed. But insisting that the DMA API needs to be extended wrongly
> > > > > > > after rc2 to fix the regression is not sensible too. The related DMA
> > > > > > > API wasn't changed in 2.6.36-rc1. The API isn't responsible for the
> > > > > > > regression at all.
> > > > > > I think this isn't about "responsiblity". Someone in arm-land found
> > > > > > that the way dma memory allocation worked for some time doesn't work
> > > > > > anymore on new generation chips. As pointing out this problem was
> > > > > > expected to find some matches it was merged in the merge window. One
> > > > > > such match is the current usage of the DMA API that doesn't currently
> > > > > > offer a way to do it right, so it needs a patch, no?
> > > > >
> > > > > No, I don't think so. We are talking about a regression, right?
> > > > >
> > > > > On new generation chips, something often doesn't work (which have
> > > > > worked on old chips for some time). It's not a regresiion. I don't
> > > > > think that it's sensible to make large change (especially after rc1)
> > > > > to fix such issue. If you say that the DMA API doesn't work on new
> > > > > chips and proposes a patch for the next merge window, it's sensible, I
> > > > > suppose.
> > > > >
> > > > > Btw, the patch isn't a fix for the DMA API. It tries to extend the DMA
> > > > > API (and IMO in the wrong way).
> > > > > In addition, the patch might break the
> > > > > current code.
> > > >
> > > > To "break the current code" is simply not possible. Sorry to oppose. As you have written it
> > > > "extend the DMA API", so if you do not use the new API (and no current code is using it)
> > > > you cannot "break the current code".
> > >
> > > Looks like that the patch adds the new API that touches the exisitng
> > > code. It means the existing code could break. So the exsising API
> > > could break too.
> > >
> > > http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
> >
> > The above reference is not my patch. I am speaking for my patch:
> >
> > http://lkml.org/lkml/2010/8/19/200
>
> I think that I already NACK'ed the patch.

OK.

Thanks,

Marin Mitov

>
> 1) drivers/media/videobuf-dma-contig.c should not use
> dma_alloc_coherent. We shouldn't support the proposed API.
>
> 2) I don't think that the DMA API (drivers/base/dma-mapping.c) is not
> for creating "cache". Generally, the kernel uses "pool" concept for
> something like that.
>
> IMHO, reverting the commit 309caa9cc6ff39d261264ec4ff10e29489afc8f8
> temporary (or temporary disabling it for systems that had worked) is
> the most reasonable approach. I don't think that breaking systems that
> had worked is a good idea even if the patch does the right thing. I
> believe that we need to fix the broken solution
> (videobuf-dma-contig.c) before the commit.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>