Hi. I'm using PCI Sound cards on MIPS platform which has noncoherent
DMA. There are some issues in ALSA for these platform.
(This topic comes from linux-mips ML.
http://www.linux-mips.org/cgi-bin/mesg.cgi?a=linux-mips&i=20060124030725.GA14063%40deprecation.cyrius.com)
1. virt_to_page vs. dma_alloc_coherent problem.
ALSA uses virt_to_page() to get 'struct page' for DMA area which was
allocated using dma_alloc_coherent(). On MIPS with
CONFIG_DMA_NONCOHERENT, typically physical address range
0x0-0x1fffffff are mapped to 0x8000000-0x9fffffff with cached and
mapped to 0xa0000000-0xbfffffff with uncached. If we got physical
address 0x01000000 for DMA, the virtual address is 0xa1000000. On the
other hand, virt_to_page() expects normal(cached) virtual address. So
we can use virt_to_page() with kmalloc() or dma_alloc_noncoherent(),
but not with dma_alloc_coherent().
Here is some fragments from kernel code. You can see what I mean exactly.
/* from include/asm-mips/mach-generic/spaces.h */
#define CAC_BASE 0x80000000
#define UNCAC_BASE 0xa0000000
#define PAGE_OFFSET 0x80000000UL
/* from include/asm-mips/page.h */
#define __pa(x) ((unsigned long) (x) - PAGE_OFFSET)
#define __va(x) ((void *)((unsigned long) (x) + PAGE_OFFSET))
#define pfn_to_page(pfn) (mem_map + (pfn))
#define virt_to_page(kaddr) pfn_to_page(__pa(kaddr) >> PAGE_SHIFT)
#define UNCAC_ADDR(addr) ((addr) - PAGE_OFFSET + UNCAC_BASE)
#define CAC_ADDR(addr) ((addr) - UNCAC_BASE + PAGE_OFFSET)
/* from arch/mips/mm/dma-noncoherent.c */
void *dma_alloc_coherent(struct device *dev, size_t size,
dma_addr_t * dma_handle, gfp_t gfp)
{
void *ret;
ret = dma_alloc_noncoherent(dev, size, dma_handle, gfp);
if (ret) {
dma_cache_wback_inv((unsigned long) ret, size);
ret = UNCAC_ADDR(ret);
}
return ret;
}
How do we fix this?
A. hack sound/core/memalloc.c, pcm_native.c, sgbuf.c
something like:
#if defined(__mips__) && defined(CONFIG_DMA_NONCOHERENT)
mark_pages(virt_to_page(CAC_ADDR(res)), pg); /* should be dma_to_page() */
#else
mark_pages(virt_to_page(res), pg); /* should be dma_to_page() */
#endif
It's ugly.
B. fix mips virt_to_page()
#define TO_PHYS_MASK 0x1fffffff
#define virt_to_page(kaddr) pfn_to_page(((kaddr) & TO_PHYS_MASK) >> PAGE_SHIFT)
a bit slower. MIPS need one instruction to create 0x80000000
constant, two instruction for 0x1fffffff. There are many usage of
virt_to_page() in mm/slab.c so its performance might be important.
C. introduce dma_to_page() which returns struct page * for dma_addr_t.
The comment in ALSA code (/* should be dma_to_page() */) mean this?
For MIPS, it would be:
#define dma_to_page(addr) pfn_to_page((addr) >> PAGE_SHIFT)
But I do not know for other platform.
Which is preferred, or is there other good way?
2. mmapping DMA area.
As described above, DMA area is uncached on those platform. So mmap
should ensure user programs do not cache these area.
snd_pcm_default_mmap() is used for mmapping the DMA area but its
vm_page_prot is not configured as uncached. On the other hand,
snd_pcm_lib_mmap_iomem() do it using pgprot_noncached().
snd_pcm_default_mmap() should do same thing for those MIPS platform.
static int snd_pcm_default_mmap(struct snd_pcm_substream *substream,
struct vm_area_struct *area)
{
#if defined(__mips__) && defined(CONFIG_DMA_NONCOHERENT)
area->vm_page_prot = pgprot_noncached(area->vm_page_prot);
#endif
area->vm_ops = &snd_pcm_vm_ops_data;
Is this a right fix? Are there any platform which have same problem?
---
Atsushi Nemoto
At Wed, 25 Jan 2006 23:50:07 +0900 (JST),
Atsushi Nemoto wrote:
>
> Hi. I'm using PCI Sound cards on MIPS platform which has noncoherent
> DMA. There are some issues in ALSA for these platform.
>
> (This topic comes from linux-mips ML.
> http://www.linux-mips.org/cgi-bin/mesg.cgi?a=linux-mips&i=20060124030725.GA14063%40deprecation.cyrius.com)
>
>
> 1. virt_to_page vs. dma_alloc_coherent problem.
>
> ALSA uses virt_to_page() to get 'struct page' for DMA area which was
> allocated using dma_alloc_coherent(). On MIPS with
> CONFIG_DMA_NONCOHERENT, typically physical address range
> 0x0-0x1fffffff are mapped to 0x8000000-0x9fffffff with cached and
> mapped to 0xa0000000-0xbfffffff with uncached. If we got physical
> address 0x01000000 for DMA, the virtual address is 0xa1000000. On the
> other hand, virt_to_page() expects normal(cached) virtual address. So
> we can use virt_to_page() with kmalloc() or dma_alloc_noncoherent(),
> but not with dma_alloc_coherent().
>
> Here is some fragments from kernel code. You can see what I mean exactly.
>
> /* from include/asm-mips/mach-generic/spaces.h */
> #define CAC_BASE 0x80000000
> #define UNCAC_BASE 0xa0000000
> #define PAGE_OFFSET 0x80000000UL
> /* from include/asm-mips/page.h */
> #define __pa(x) ((unsigned long) (x) - PAGE_OFFSET)
> #define __va(x) ((void *)((unsigned long) (x) + PAGE_OFFSET))
> #define pfn_to_page(pfn) (mem_map + (pfn))
> #define virt_to_page(kaddr) pfn_to_page(__pa(kaddr) >> PAGE_SHIFT)
> #define UNCAC_ADDR(addr) ((addr) - PAGE_OFFSET + UNCAC_BASE)
> #define CAC_ADDR(addr) ((addr) - UNCAC_BASE + PAGE_OFFSET)
> /* from arch/mips/mm/dma-noncoherent.c */
> void *dma_alloc_coherent(struct device *dev, size_t size,
> dma_addr_t * dma_handle, gfp_t gfp)
> {
> void *ret;
>
> ret = dma_alloc_noncoherent(dev, size, dma_handle, gfp);
> if (ret) {
> dma_cache_wback_inv((unsigned long) ret, size);
> ret = UNCAC_ADDR(ret);
> }
>
> return ret;
> }
>
> How do we fix this?
>
> A. hack sound/core/memalloc.c, pcm_native.c, sgbuf.c
>
> something like:
> #if defined(__mips__) && defined(CONFIG_DMA_NONCOHERENT)
> mark_pages(virt_to_page(CAC_ADDR(res)), pg); /* should be dma_to_page() */
> #else
> mark_pages(virt_to_page(res), pg); /* should be dma_to_page() */
> #endif
>
> It's ugly.
>
> B. fix mips virt_to_page()
>
> #define TO_PHYS_MASK 0x1fffffff
> #define virt_to_page(kaddr) pfn_to_page(((kaddr) & TO_PHYS_MASK) >> PAGE_SHIFT)
>
> a bit slower. MIPS need one instruction to create 0x80000000
> constant, two instruction for 0x1fffffff. There are many usage of
> virt_to_page() in mm/slab.c so its performance might be important.
>
> C. introduce dma_to_page() which returns struct page * for dma_addr_t.
>
> The comment in ALSA code (/* should be dma_to_page() */) mean this?
>
> For MIPS, it would be:
> #define dma_to_page(addr) pfn_to_page((addr) >> PAGE_SHIFT)
> But I do not know for other platform.
>
>
> Which is preferred, or is there other good way?
>
>
> 2. mmapping DMA area.
>
> As described above, DMA area is uncached on those platform. So mmap
> should ensure user programs do not cache these area.
> snd_pcm_default_mmap() is used for mmapping the DMA area but its
> vm_page_prot is not configured as uncached. On the other hand,
> snd_pcm_lib_mmap_iomem() do it using pgprot_noncached().
>
> snd_pcm_default_mmap() should do same thing for those MIPS platform.
>
> static int snd_pcm_default_mmap(struct snd_pcm_substream *substream,
> struct vm_area_struct *area)
> {
> #if defined(__mips__) && defined(CONFIG_DMA_NONCOHERENT)
> area->vm_page_prot = pgprot_noncached(area->vm_page_prot);
> #endif
> area->vm_ops = &snd_pcm_vm_ops_data;
>
> Is this a right fix? Are there any platform which have same problem?
The right fix is, IMO, to port dma_mmap_coherent() to all archs.
Currently, it's only on ARM.
Takashi
>>>>> On Wed, 25 Jan 2006 20:03:13 +0100, Takashi Iwai <[email protected]> said:
tiwai> The right fix is, IMO, to port dma_mmap_coherent() to all
tiwai> archs. Currently, it's only on ARM.
Thanks. Is this solve both issue #1 and #2 ?
The most part of issue #1 will vanish if NEED_RESERVE_PAGES was
defined, and currently only ARM define this. ARM defines
NEED_RESERVE_PAGES because it has dma_mmap_coherent(), right?
---
Atsushi Nemoto
At Fri, 27 Jan 2006 00:29:25 +0900 (JST),
Atsushi Nemoto wrote:
>
> >>>>> On Wed, 25 Jan 2006 20:03:13 +0100, Takashi Iwai <[email protected]> said:
>
> tiwai> The right fix is, IMO, to port dma_mmap_coherent() to all
> tiwai> archs. Currently, it's only on ARM.
>
> Thanks. Is this solve both issue #1 and #2 ?
Yes. In theory, dma_alloc_coherent() and dma_mmap_coherent() should
cover all the magic each arch requires.
> The most part of issue #1 will vanish if NEED_RESERVE_PAGES was
> defined, and currently only ARM define this. ARM defines
> NEED_RESERVE_PAGES because it has dma_mmap_coherent(), right?
Well, the whole page-reserve kludge should disappear anyway in near
future. Right now it's in the process.
The memory allocation stuff in ALSA is being largely rewritten on my
local tree, but it's not released yet. One reason is to wait for the
finish of page-reserve removals, and another reason is that I've had
little time to tidy them up :)
--
Takashi Iwai <[email protected]> ALSA Developer - http://www.alsa-project.org
On Thu, 26 Jan 2006, Takashi Iwai wrote:
> At Fri, 27 Jan 2006 00:29:25 +0900 (JST),
> Atsushi Nemoto wrote:
>
> > The most part of issue #1 will vanish if NEED_RESERVE_PAGES was
> > defined, and currently only ARM define this. ARM defines
> > NEED_RESERVE_PAGES because it has dma_mmap_coherent(), right?
>
> Well, the whole page-reserve kludge should disappear anyway in near
> future. Right now it's in the process.
Yes, mark_pages() and unmark_pages() can just be removed as soon as
you like.
I didn't reply to the original posting because I noticed they're not
all of the virt_to_page()s in sound/core, and sometimes a part-answer
distracts someone more competent from responding with the full answer.
And I'm in no hurry to remove these PageReserved traces myself, since
it's not a bad idea to hold on to the bad_page cross-checking for a
release or two.
Hugh
>>>>> On Thu, 26 Jan 2006 19:19:22 +0000 (GMT), Hugh Dickins <[email protected]> said:
>> Well, the whole page-reserve kludge should disappear anyway in near
>> future. Right now it's in the process.
hugh> Yes, mark_pages() and unmark_pages() can just be removed as soon
hugh> as you like.
When I tried undefining NEED_RESERVE_PAGES for MIPS on 2.6.13,
something did not work (I can not remember details...). But it seems
things have been changed in 2.6.15. I'll try again. Thanks.
hugh> I didn't reply to the original posting because I noticed they're
hugh> not all of the virt_to_page()s in sound/core, and sometimes a
hugh> part-answer distracts someone more competent from responding
hugh> with the full answer.
Yes, there is still virt_to_page() in snd_pcm_mmap_data_nopage() and
snd_malloc_sgbuf_pages(). If dma_mmap_coherent() was ported, former
might disappear but latter might not. So it seems virt_to_page()
issue still remains...
>>>>> On Thu, 26 Jan 2006 17:02:53 +0100, Takashi Iwai <[email protected]> said:
tiwai> The memory allocation stuff in ALSA is being largely rewritten
tiwai> on my local tree, but it's not released yet. One reason is to
tiwai> wait for the finish of page-reserve removals, and another
tiwai> reason is that I've had little time to tidy them up :)
OK, I'll wait. Thank you very much.
---
Atsushi Nemoto
> tiwai> The memory allocation stuff in ALSA is being largely rewritten
> tiwai> on my local tree, but it's not released yet. One reason is to
> tiwai> wait for the finish of page-reserve removals, and another
> tiwai> reason is that I've had little time to tidy them up :)
Do you have a rough idea when this may happen?
--
Martin Michlmayr
http://www.cyrius.com/
At Sat, 28 Jan 2006 00:45:40 +0900 (JST),
Atsushi Nemoto wrote:
>
> >>>>> On Thu, 26 Jan 2006 19:19:22 +0000 (GMT), Hugh Dickins <[email protected]> said:
>
> >> Well, the whole page-reserve kludge should disappear anyway in near
> >> future. Right now it's in the process.
>
> hugh> Yes, mark_pages() and unmark_pages() can just be removed as soon
> hugh> as you like.
>
> When I tried undefining NEED_RESERVE_PAGES for MIPS on 2.6.13,
> something did not work (I can not remember details...). But it seems
> things have been changed in 2.6.15. I'll try again. Thanks.
Yes, it was changed pretty much.
> hugh> I didn't reply to the original posting because I noticed they're
> hugh> not all of the virt_to_page()s in sound/core, and sometimes a
> hugh> part-answer distracts someone more competent from responding
> hugh> with the full answer.
>
> Yes, there is still virt_to_page() in snd_pcm_mmap_data_nopage() and
> snd_malloc_sgbuf_pages(). If dma_mmap_coherent() was ported, former
> might disappear but latter might not. So it seems virt_to_page()
> issue still remains...
In theory, you can call dma_mmap_coherent() on each page obtained via
dma_alloc_coherent() in the case of sg-buffer. It should work without
much overhead on most of archs, but ones like sparc32 may have
problems because each alloc_coherent call results in another
allocation a resource struct (if I understand the code correctly).
So, I rewrote the code to use the normal get_page() + map/sync.
Hopefully this works for archs with non-coherent mm...
Takashi
At Fri, 27 Jan 2006 15:49:28 +0000,
Martin Michlmayr wrote:
>
> > tiwai> The memory allocation stuff in ALSA is being largely rewritten
> > tiwai> on my local tree, but it's not released yet. One reason is to
> > tiwai> wait for the finish of page-reserve removals, and another
> > tiwai> reason is that I've had little time to tidy them up :)
>
> Do you have a rough idea when this may happen?
Hard to promise... Hopefully I can start working seriously again on
this issue in February.
Takashi
>>>>> On Fri, 27 Jan 2006 16:57:49 +0100, Takashi Iwai <[email protected]> said:
hugh> Yes, mark_pages() and unmark_pages() can just be removed as soon
hugh> as you like.
>> When I tried undefining NEED_RESERVE_PAGES for MIPS on 2.6.13,
>> something did not work (I can not remember details...). But it seems
>> things have been changed in 2.6.15. I'll try again. Thanks.
tiwai> Yes, it was changed pretty much.
I undefined NEED_RESERVE_PAGES on 2.6.15 and it seems OK on MIPS.
Thank you.
---
Atsushi Nemoto
At Mon, 30 Jan 2006 18:56:08 +0900 (JST),
Atsushi Nemoto wrote:
>
> >>>>> On Fri, 27 Jan 2006 16:57:49 +0100, Takashi Iwai <[email protected]> said:
> hugh> Yes, mark_pages() and unmark_pages() can just be removed as soon
> hugh> as you like.
>
> >> When I tried undefining NEED_RESERVE_PAGES for MIPS on 2.6.13,
> >> something did not work (I can not remember details...). But it seems
> >> things have been changed in 2.6.15. I'll try again. Thanks.
>
> tiwai> Yes, it was changed pretty much.
>
> I undefined NEED_RESERVE_PAGES on 2.6.15 and it seems OK on MIPS.
> Thank you.
Well, as Hugu pointed out, that page reservation plays no longer any
role. The patch below should work too on 2.6.15 or later.
Takashi
--- linux/sound/core/memalloc.c 23 Jan 2006 15:53:15 -0000 1.52
+++ linux/sound/core/memalloc.c 30 Jan 2006 10:16:31 -0000
@@ -141,10 +141,6 @@
#endif /* arch */
-#if ! defined(__arm__)
-#define NEED_RESERVE_PAGES
-#endif
-
/*
*
* Generic memory allocators
@@ -163,20 +159,6 @@
snd_allocated_pages -= 1 << order;
}
-static void mark_pages(struct page *page, int order)
-{
- struct page *last_page = page + (1 << order);
- while (page < last_page)
- SetPageReserved(page++);
-}
-
-static void unmark_pages(struct page *page, int order)
-{
- struct page *last_page = page + (1 << order);
- while (page < last_page)
- ClearPageReserved(page++);
-}
-
/**
* snd_malloc_pages - allocate pages with the given size
* @size: the size to allocate in bytes
@@ -195,10 +177,8 @@
snd_assert(gfp_flags != 0, return NULL);
gfp_flags |= __GFP_COMP; /* compound page lets parts be mapped */
pg = get_order(size);
- if ((res = (void *) __get_free_pages(gfp_flags, pg)) != NULL) {
- mark_pages(virt_to_page(res), pg);
+ if ((res = (void *) __get_free_pages(gfp_flags, pg)) != NULL)
inc_snd_pages(pg);
- }
return res;
}
@@ -217,7 +197,6 @@
return;
pg = get_order(size);
dec_snd_pages(pg);
- unmark_pages(virt_to_page(ptr), pg);
free_pages((unsigned long) ptr, pg);
}
@@ -242,12 +221,8 @@
| __GFP_NORETRY /* don't trigger OOM-killer */
| __GFP_NOWARN; /* no stack trace print - this call is non-critical */
res = dma_alloc_coherent(dev, PAGE_SIZE << pg, dma, gfp_flags);
- if (res != NULL) {
-#ifdef NEED_RESERVE_PAGES
- mark_pages(virt_to_page(res), pg); /* should be dma_to_page() */
-#endif
+ if (res != NULL)
inc_snd_pages(pg);
- }
return res;
}
@@ -262,9 +237,6 @@
return;
pg = get_order(size);
dec_snd_pages(pg);
-#ifdef NEED_RESERVE_PAGES
- unmark_pages(virt_to_page(ptr), pg); /* should be dma_to_page() */
-#endif
dma_free_coherent(dev, PAGE_SIZE << pg, ptr, dma);
}
>>>>> On Mon, 30 Jan 2006 11:18:54 +0100, Takashi Iwai <[email protected]> said:
tiwai> Well, as Hugu pointed out, that page reservation plays no longer any
tiwai> role. The patch below should work too on 2.6.15 or later.
The patch works for me. Thank you.
Acked-by: Atsushi Nemoto <[email protected]>
---
Atsushi Nemoto
On Mon, Jan 30, 2006 at 11:18:54AM +0100, Takashi Iwai wrote:
> > I undefined NEED_RESERVE_PAGES on 2.6.15 and it seems OK on MIPS.
> > Thank you.
>
> Well, as Hugu pointed out, that page reservation plays no longer any
> role. The patch below should work too on 2.6.15 or later.
That resolves the portability issues with the assumption on the return
value of dma_alloc_coherent(), thanks.
Acked-by: Ralf Baechle <[email protected]>
Ralf
* Takashi Iwai <[email protected]> [2006-01-30 11:18]:
> Well, as Hugu pointed out, that page reservation plays no longer any
> role. The patch below should work too on 2.6.15 or later.
It doesn't solve the problem I have, those "wait source ready timeout
0x1410 [0x8c8c8c8c]" messages on a Cobalt Qube2 with a 64-bit MIPS
kernel.
Detecting hardware: de4x5 via82cxxx es1371 usb_uhci
de4x5 disabled in configuration.
Skipping unavailable/built-in via82cxxx module.
Skipping unavailable/built-in es1371 module.
Loading uhci_hcd module.
usbcore: registered new driver usbfs
usbcore: registered new driver hub
USB Universal Host Controller Interface driver v2.3
uhci_hcd 0000:00:09.2: Found HC with no IRQ. Check BIOS/PCI 0000:00:09.2 setup!
uhci_hcd 0000:00:09.2: init 0000:00:09.2 fail, -19
Running 0dns-down to make sure resolv.conf is ok...done.
Setting up networking...done.
Starting hotplug subsystem:
pci
uhci-hcd: already loaded
wait source ready timeout 0x1410 [0x8c8c8c8c] <- repeated 180 times
AC'97 0 analog subsections not ready
wait source ready timeout 0x1410 [0x8c8c8c8c] <- repeated 453 times
snd-ens1371: loaded successfully
pci [success]
usb
usb [success]
isapnp
isapnp [success]
ide
ide [success]
input
input [success]
scsi
scsi [success]
done.
Setting up IP spoofing protection: rp_filter.
Configuring network interfaces...done.
Starting portmap daemon: portmap.
--
Martin Michlmayr
http://www.cyrius.com/
At Mon, 30 Jan 2006 15:46:17 +0000,
Martin Michlmayr wrote:
>
> * Takashi Iwai <[email protected]> [2006-01-30 11:18]:
> > Well, as Hugu pointed out, that page reservation plays no longer any
> > role. The patch below should work too on 2.6.15 or later.
>
> It doesn't solve the problem I have, those "wait source ready timeout
> 0x1410 [0x8c8c8c8c]" messages on a Cobalt Qube2 with a 64-bit MIPS
> kernel.
Of course not. Your problem is totally irrelevant with the page
allocation stuff we discussed.
> wait source ready timeout 0x1410 [0x8c8c8c8c] <- repeated 180 times
> AC'97 0 analog subsections not ready
> wait source ready timeout 0x1410 [0x8c8c8c8c] <- repeated 453 times
> snd-ens1371: loaded successfully
The messages above imply that the h/w access doesn't work properly,
e.g. the wrong PCI resource assignment or the insufficient
initialization.
Takashi