Hello everybody,
I develop video acquisition software using the video1394 interface.
The images grabbed by the camera and iee1394 bus are kept in kernel
memory and made available to the user program through a mmap call done
in the libdc1394 library :
dma_ring_buffer= mmap(0, vmmap.nb_buffers * vmmap.buf_size,
PROT_READ|PROT_WRITE,MAP_SHARED, craw->capture.dma_fd, 0);
Sometimes, my program crashes and produces a core file :) It seems to
me that the core file does not contain the mmap'ed memory and hence
I cannot replay my program with the same image for debugging purpose.
Is it possible to configure the kernel through /proc, or through the mmap
system call to have that mmapped segment in the core file, or do I need
to modify the kernel itself to obtain the behaviour I want ? If I
need to modify the kernel, can some kind soul provide me some pointers ?
Best regards
Philippe
On 7/1/08, Philippe De Muyter <[email protected]> wrote:
> Hello everybody,
>
> I develop video acquisition software using the video1394 interface.
> The images grabbed by the camera and iee1394 bus are kept in kernel
> memory and made available to the user program through a mmap call done
> in the libdc1394 library :
>
> dma_ring_buffer= mmap(0, vmmap.nb_buffers * vmmap.buf_size,
> PROT_READ|PROT_WRITE,MAP_SHARED, craw->capture.dma_fd, 0);
>
> Sometimes, my program crashes and produces a core file :) It seems to
> me that the core file does not contain the mmap'ed memory and hence
> I cannot replay my program with the same image for debugging purpose.
>
> Is it possible to configure the kernel through /proc, or through the mmap
> system call to have that mmapped segment in the core file, or do I need
> to modify the kernel itself to obtain the behaviour I want ? If I
> need to modify the kernel, can some kind soul provide me some pointers ?
Have a look at the section "Controlling which mappings are written to
the core dump" in a recent core.5 man page:
http://www.kernel.org/doc/man-pages/online/pages/man5/core.5.html
Cheers,
Michael
On Tue, Jul 01, 2008 at 08:16:11PM +0200, Michael Kerrisk wrote:
> On 7/1/08, Philippe De Muyter <[email protected]> wrote:
> > Hello everybody,
> >
> > I develop video acquisition software using the video1394 interface.
> > The images grabbed by the camera and iee1394 bus are kept in kernel
> > memory and made available to the user program through a mmap call done
> > in the libdc1394 library :
> >
> > dma_ring_buffer= mmap(0, vmmap.nb_buffers * vmmap.buf_size,
> > PROT_READ|PROT_WRITE,MAP_SHARED, craw->capture.dma_fd, 0);
> >
> > Sometimes, my program crashes and produces a core file :) It seems to
> > me that the core file does not contain the mmap'ed memory and hence
> > I cannot replay my program with the same image for debugging purpose.
> >
> > Is it possible to configure the kernel through /proc, or through the mmap
> > system call to have that mmapped segment in the core file, or do I need
> > to modify the kernel itself to obtain the behaviour I want ? If I
> > need to modify the kernel, can some kind soul provide me some pointers ?
>
>
> Have a look at the section "Controlling which mappings are written to
> the core dump" in a recent core.5 man page:
> http://www.kernel.org/doc/man-pages/online/pages/man5/core.5.html
Interesting (and somewhat off topic to your conversation here) - it
appears that when dumping mappings, the kernel ignores the maximum
core size set with "limit".
This is particularly interesting on a 64 bit kernel where a bug in
your code causes you to try to read something about 2Gb into your
alleged mmaped file (actual size ~500 bytes) and the segfault causes
a coredump.
Bron.
On Tue, Jul 1, 2008 at 11:44 PM, Bron Gondwana <[email protected]> wrote:
> On Tue, Jul 01, 2008 at 08:16:11PM +0200, Michael Kerrisk wrote:
>> On 7/1/08, Philippe De Muyter <[email protected]> wrote:
>> > Hello everybody,
>> >
>> > I develop video acquisition software using the video1394 interface.
>> > The images grabbed by the camera and iee1394 bus are kept in kernel
>> > memory and made available to the user program through a mmap call done
>> > in the libdc1394 library :
>> >
>> > dma_ring_buffer= mmap(0, vmmap.nb_buffers * vmmap.buf_size,
>> > PROT_READ|PROT_WRITE,MAP_SHARED, craw->capture.dma_fd, 0);
>> >
>> > Sometimes, my program crashes and produces a core file :) It seems to
>> > me that the core file does not contain the mmap'ed memory and hence
>> > I cannot replay my program with the same image for debugging purpose.
>> >
>> > Is it possible to configure the kernel through /proc, or through the mmap
>> > system call to have that mmapped segment in the core file, or do I need
>> > to modify the kernel itself to obtain the behaviour I want ? If I
>> > need to modify the kernel, can some kind soul provide me some pointers ?
>>
>>
>> Have a look at the section "Controlling which mappings are written to
>> the core dump" in a recent core.5 man page:
>> http://www.kernel.org/doc/man-pages/online/pages/man5/core.5.html
>
> Interesting (and somewhat off topic to your conversation here) - it
> appears that when dumping mappings, the kernel ignores the maximum
> core size set with "limit".
>
> This is particularly interesting on a 64 bit kernel where a bug in
> your code causes you to try to read something about 2Gb into your
> alleged mmaped file (actual size ~500 bytes) and the segfault causes
> a coredump.
Do you have a ssimple example program for this?
--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
man-pages online: http://www.kernel.org/doc/man-pages/online_pages.html
Found a bug? http://www.kernel.org/doc/man-pages/reporting_bugs.html
>> This is particularly interesting on a 64 bit kernel where a bug in
>> your code causes you to try to read something about 2Gb into your
>> alleged mmaped file (actual size ~500 bytes) and the segfault causes
>> a coredump.
>
> Do you have a ssimple example program for this?
Trying to reproduce the problem, I think it's actually related to sparse
files.
$ cat a.c
#include <stdlib.h>
int main() {
unsigned int i;
char * a = malloc(3000000000u);
*(int *)0 = 0;
}
$ gcc a.c
$ ulimit -c 10240 && ./a.out
$ ls -l
-rw------- 1 root root 3000082432 Jul 2 02:23 core.7761
$
It's clearly sparse, but slightly unintuitive that the ulimit doesn't
actually limit the filesize, just the size of the data written to the file.
If I change the code to include this line after the malloc():
for (i = 0; i < 3000000000u; i++) a[i] = i % 256;
I get:
-rw------- 1 root root 10485760 Jul 2 02:25 core.8992
More what you'd expect.
One interesting side effect of running a 64-bit kernel + 32-bit userland is
that previously bugs that might have previously caused malloc() to fail (eg
underflowing integer to a huge value), now succeed and allocate a huge chunk
of memory rather than failing and causing the program to bailout/crash on
dereference.
Rob
Hi Michael,
On Tue, Jul 01, 2008 at 08:16:11PM +0200, Michael Kerrisk wrote:
> On 7/1/08, Philippe De Muyter <[email protected]> wrote:
> > Hello everybody,
> >
> > I develop video acquisition software using the video1394 interface.
> > The images grabbed by the camera and iee1394 bus are kept in kernel
> > memory and made available to the user program through a mmap call done
> > in the libdc1394 library :
> >
> > dma_ring_buffer= mmap(0, vmmap.nb_buffers * vmmap.buf_size,
> > PROT_READ|PROT_WRITE,MAP_SHARED, craw->capture.dma_fd, 0);
> >
> > Sometimes, my program crashes and produces a core file :) It seems to
> > me that the core file does not contain the mmap'ed memory and hence
> > I cannot replay my program with the same image for debugging purpose.
> >
> > Is it possible to configure the kernel through /proc, or through the mmap
> > system call to have that mmapped segment in the core file, or do I need
> > to modify the kernel itself to obtain the behaviour I want ? If I
> > need to modify the kernel, can some kind soul provide me some pointers ?
>
>
> Have a look at the section "Controlling which mappings are written to
> the core dump" in a recent core.5 man page:
> http://www.kernel.org/doc/man-pages/online/pages/man5/core.5.html
thanks for the info. I didn't know about /proc/PID/coredump_filter.
that part was promising :
bit 2 Dump file-backed private mappings.
bit 3 Dump file-backed shared mappings.
The default value of coredump_filter is 0x3; this reflects traditional
Linux behavior and means that only anonymous memory segments are dumped.
Unfortunately, the part that applies to me (I have tested it) is the next one :
Memory-mapped I/O pages such as frame buffer are never dumped, [...],
regardless of the coredump_filter value.
Is that a design decision, or a mere finding of the way it is implemented
now ?
So, back to my original question :
Can some kind soul provide me some pointers to the way I should modify
the kernel to make the inclusion of the video1394 mmapped segment in
core files possible ?
best regards
Philippe
[CC+= [email protected]]
On Wed, Jul 2, 2008 at 12:50 PM, Philippe De Muyter <[email protected]> wrote:
> Hi Michael,
>
> On Tue, Jul 01, 2008 at 08:16:11PM +0200, Michael Kerrisk wrote:
>> On 7/1/08, Philippe De Muyter <[email protected]> wrote:
>> > Hello everybody,
>> >
>> > I develop video acquisition software using the video1394 interface.
>> > The images grabbed by the camera and iee1394 bus are kept in kernel
>> > memory and made available to the user program through a mmap call done
>> > in the libdc1394 library :
>> >
>> > dma_ring_buffer= mmap(0, vmmap.nb_buffers * vmmap.buf_size,
>> > PROT_READ|PROT_WRITE,MAP_SHARED, craw->capture.dma_fd, 0);
>> >
>> > Sometimes, my program crashes and produces a core file :) It seems to
>> > me that the core file does not contain the mmap'ed memory and hence
>> > I cannot replay my program with the same image for debugging purpose.
>> >
>> > Is it possible to configure the kernel through /proc, or through the mmap
>> > system call to have that mmapped segment in the core file, or do I need
>> > to modify the kernel itself to obtain the behaviour I want ? If I
>> > need to modify the kernel, can some kind soul provide me some pointers ?
>>
>>
>> Have a look at the section "Controlling which mappings are written to
>> the core dump" in a recent core.5 man page:
>> http://www.kernel.org/doc/man-pages/online/pages/man5/core.5.html
>
> thanks for the info. I didn't know about /proc/PID/coredump_filter.
>
> that part was promising :
>
> bit 2 Dump file-backed private mappings.
> bit 3 Dump file-backed shared mappings.
>
> The default value of coredump_filter is 0x3; this reflects traditional
> Linux behavior and means that only anonymous memory segments are dumped.
>
> Unfortunately, the part that applies to me (I have tested it) is the next one :
>
> Memory-mapped I/O pages such as frame buffer are never dumped, [...],
> regardless of the coredump_filter value.
>
> Is that a design decision, or a mere finding of the way it is implemented
> now ?
>
> So, back to my original question :
>
> Can some kind soul provide me some pointers to the way I should modify
> the kernel to make the inclusion of the video1394 mmapped segment in
> core files possible ?
Perhaps Hidehiro, who wrote the coredump_filter feature, can provide insight.
Cheers,
Michael
--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
man-pages online: http://www.kernel.org/doc/man-pages/online_pages.html
Found a bug? http://www.kernel.org/doc/man-pages/reporting_bugs.html
[ sorry, reposted to add cc [email protected] ]
Hi Michael,
On Tue, Jul 01, 2008 at 08:16:11PM +0200, Michael Kerrisk wrote:
> On 7/1/08, Philippe De Muyter <[email protected]> wrote:
> > Hello everybody,
> >
> > I develop video acquisition software using the video1394 interface.
> > The images grabbed by the camera and iee1394 bus are kept in kernel
> > memory and made available to the user program through a mmap call done
> > in the libdc1394 library :
> >
> > dma_ring_buffer= mmap(0, vmmap.nb_buffers * vmmap.buf_size,
> > PROT_READ|PROT_WRITE,MAP_SHARED, craw->capture.dma_fd, 0);
> >
> > Sometimes, my program crashes and produces a core file :) It seems to
> > me that the core file does not contain the mmap'ed memory and hence
> > I cannot replay my program with the same image for debugging purpose.
> >
> > Is it possible to configure the kernel through /proc, or through the mmap
> > system call to have that mmapped segment in the core file, or do I need
> > to modify the kernel itself to obtain the behaviour I want ? If I
> > need to modify the kernel, can some kind soul provide me some pointers ?
>
>
> Have a look at the section "Controlling which mappings are written to
> the core dump" in a recent core.5 man page:
> http://www.kernel.org/doc/man-pages/online/pages/man5/core.5.html
thanks for the info. I didn't know about /proc/PID/coredump_filter.
that part was promising :
bit 2 Dump file-backed private mappings.
bit 3 Dump file-backed shared mappings.
The default value of coredump_filter is 0x3; this reflects traditional
Linux behavior and means that only anonymous memory segments are dumped.
Unfortunately, the part that applies to me (I have tested it) is the next one :
Memory-mapped I/O pages such as frame buffer are never dumped, [...],
regardless of the coredump_filter value.
Is that a design decision, or a mere finding of the way it is implemented
now ?
So, back to my original question :
Can some kind soul provide me some pointers to the way I should modify
the kernel to make the inclusion of the video1394 mmapped segment in
core files possible ?
best regards
Philippe
[CC+= [email protected]]
On Wed, Jul 02, 2008 at 12:58:05PM +0200, Michael Kerrisk wrote:
> [CC+= [email protected]]
>
> On Wed, Jul 2, 2008 at 12:50 PM, Philippe De Muyter <[email protected]> wrote:
> > Hi Michael,
> >
> > On Tue, Jul 01, 2008 at 08:16:11PM +0200, Michael Kerrisk wrote:
> >> On 7/1/08, Philippe De Muyter <[email protected]> wrote:
> >> > Hello everybody,
> >> >
> >> > I develop video acquisition software using the video1394 interface.
> >> > The images grabbed by the camera and iee1394 bus are kept in kernel
> >> > memory and made available to the user program through a mmap call done
> >> > in the libdc1394 library :
> >> >
> >> > dma_ring_buffer= mmap(0, vmmap.nb_buffers * vmmap.buf_size,
> >> > PROT_READ|PROT_WRITE,MAP_SHARED, craw->capture.dma_fd, 0);
> >> >
> >> > Sometimes, my program crashes and produces a core file :) It seems to
> >> > me that the core file does not contain the mmap'ed memory and hence
> >> > I cannot replay my program with the same image for debugging purpose.
> >> >
> >> > Is it possible to configure the kernel through /proc, or through the mmap
> >> > system call to have that mmapped segment in the core file, or do I need
> >> > to modify the kernel itself to obtain the behaviour I want ? If I
> >> > need to modify the kernel, can some kind soul provide me some pointers ?
> >>
> >>
> >> Have a look at the section "Controlling which mappings are written to
> >> the core dump" in a recent core.5 man page:
> >> http://www.kernel.org/doc/man-pages/online/pages/man5/core.5.html
> >
> > thanks for the info. I didn't know about /proc/PID/coredump_filter.
> >
> > that part was promising :
> >
> > bit 2 Dump file-backed private mappings.
> > bit 3 Dump file-backed shared mappings.
> >
> > The default value of coredump_filter is 0x3; this reflects traditional
> > Linux behavior and means that only anonymous memory segments are dumped.
> >
> > Unfortunately, the part that applies to me (I have tested it) is the next one :
> >
> > Memory-mapped I/O pages such as frame buffer are never dumped, [...],
> > regardless of the coredump_filter value.
> >
> > Is that a design decision, or a mere finding of the way it is implemented
> > now ?
> >
> > So, back to my original question :
> >
> > Can some kind soul provide me some pointers to the way I should modify
> > the kernel to make the inclusion of the video1394 mmapped segment in
> > core files possible ?
>
> Perhaps Hidehiro, who wrote the coredump_filter feature, can provide insight.
>
> Cheers,
>
> Michael
>
> --
> Michael Kerrisk
> Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
> man-pages online: http://www.kernel.org/doc/man-pages/online_pages.html
> Found a bug? http://www.kernel.org/doc/man-pages/reporting_bugs.html
--
Philippe De Muyter phdm at macqel dot be Tel +32 27029044
Macq Electronique SA rue de l'Aeronef 2 B-1140 Bruxelles Fax +32 27029077
"Rob Mueller" <[email protected]> writes:
>
> It's clearly sparse, but slightly unintuitive that the ulimit doesn't
> actually limit the filesize, just the size of the data written to the
> file.
It's the only sane semantic. Imagine ulimit would limit the address
range as you seem to be asking for. This means if you set e.g. ulimit
-c 1G then the kernel would never dump any address (mmap or not) above
1GB. Never dumping the process stack for example. Clearly doesn't make
any sense. And mmap'ed files are not different from any other
mappings in this regard.
-Andi
On Wed, Jul 02, 2008 at 01:07:22PM +0200, Andi Kleen wrote:
> "Rob Mueller" <[email protected]> writes:
> >
> > It's clearly sparse, but slightly unintuitive that the ulimit doesn't
> > actually limit the filesize, just the size of the data written to the
> > file.
>
> It's the only sane semantic. Imagine ulimit would limit the address
> range as you seem to be asking for. This means if you set e.g. ulimit
> -c 1G then the kernel would never dump any address (mmap or not) above
> 1GB. Never dumping the process stack for example. Clearly doesn't make
> any sense. And mmap'ed files are not different from any other
> mappings in this regard.
Hmm.. the IO hit we got with those Cyrus crashes suggested that it was
more than just a small amount of IO being caused. I didn't check if the
files were sparse or not.
I guess we can probably create a test case that recreates it, even if it's
just running up a dodgy Cyrus instance on the machine that had the crashes
before and deliberately recreating a broken cache file to trigger it.
Bron.
>> On Wed, Jul 2, 2008 at 12:50 PM, Philippe De Muyter <[email protected]> wrote:
>>> Hi Michael,
>>>
>>> On Tue, Jul 01, 2008 at 08:16:11PM +0200, Michael Kerrisk wrote:
>>>> Have a look at the section "Controlling which mappings are written to
>>>> the core dump" in a recent core.5 man page:
>>>> http://www.kernel.org/doc/man-pages/online/pages/man5/core.5.html
>>> thanks for the info. I didn't know about /proc/PID/coredump_filter.
>>>
>>> that part was promising :
>>>
>>> bit 2 Dump file-backed private mappings.
>>> bit 3 Dump file-backed shared mappings.
>>>
>>> The default value of coredump_filter is 0x3; this reflects traditional
>>> Linux behavior and means that only anonymous memory segments are dumped.
>>>
>>> Unfortunately, the part that applies to me (I have tested it) is the next one :
>>>
>>> Memory-mapped I/O pages such as frame buffer are never dumped, [...],
>>> regardless of the coredump_filter value.
This shouldn't be a problem to you as far as I understand. I suppose
"memory mapped I/O pages" means registers of PCI devices, mapped into
the memory address space.
The DMA buffer which you would like to get included in the core file is
normal RAM (I suppose: allocated by the kernel in the kernel's virtual
address space, mapped into the application client's address space by
mmap(), and also mapped into the FireWire controller's local bus address
space for it to write received data into).
FireWire controllers also have memory-mapped I/O regions (per OHCI: at
least one region of at least 2048 bytes size). But your process never
accesses them, only the kernel drivers do (to start and stop DMA
programs and so on --- while the DMA programs and the DMA data buffers
live in normal RAM).
--
Stefan Richter
-=====-==--- -=== ---=-
http://arcgraph.de/sr/
On Wed, Jul 02, 2008 at 02:24:59PM +0200, Stefan Richter wrote:
>>> On Wed, Jul 2, 2008 at 12:50 PM, Philippe De Muyter <[email protected]>
>>> wrote:
>>>> Hi Michael,
>>>>
>>>> On Tue, Jul 01, 2008 at 08:16:11PM +0200, Michael Kerrisk wrote:
>>>>> Have a look at the section "Controlling which mappings are written to
>>>>> the core dump" in a recent core.5 man page:
>>>>> http://www.kernel.org/doc/man-pages/online/pages/man5/core.5.html
>>>> thanks for the info. I didn't know about /proc/PID/coredump_filter.
>>>>
>>>> that part was promising :
>>>>
>>>> bit 2 Dump file-backed private mappings.
>>>> bit 3 Dump file-backed shared mappings.
>>>>
>>>> The default value of coredump_filter is 0x3; this reflects
>>>> traditional
>>>> Linux behavior and means that only anonymous memory segments are
>>>> dumped.
>>>>
>>>> Unfortunately, the part that applies to me (I have tested it) is the
>>>> next one :
>>>>
>>>> Memory-mapped I/O pages such as frame buffer are never dumped, [...],
>>>> regardless of the coredump_filter value.
>
> This shouldn't be a problem to you as far as I understand. I suppose
> "memory mapped I/O pages" means registers of PCI devices, mapped into the
> memory address space.
>
> The DMA buffer which you would like to get included in the core file is
> normal RAM (I suppose: allocated by the kernel in the kernel's virtual
> address space, mapped into the application client's address space by
> mmap(), and also mapped into the FireWire controller's local bus address
> space for it to write received data into).
I agree with your analysis, but the sentence takes as exemple 'frame buffer'
which I don't think are registers. I have tested with a video1394 client,
and I do not get the mmapped video memory in the core file, even with
/proc/PID/coredump_filter set to 0xf. So, while I agree it seems technically
feasible, it does not seem to be currently implemented :(
Philippe
On Wed, Jul 02, 2008 at 12:50:27PM +0200, Philippe De Muyter wrote:
> Unfortunately, the part that applies to me (I have tested it) is the next one :
>
> Memory-mapped I/O pages such as frame buffer are never dumped, [...],
> regardless of the coredump_filter value.
>
> Is that a design decision, or a mere finding of the way it is implemented
> now ?
It's an implementation issue, and there are patches queue for 2.6.27
that allow gdb access to these regions, which should be easily
extendable to cover core dumps, too.
Hi,
Michael Kerrisk wrote:
> [CC+= [email protected]]
>
> On Wed, Jul 2, 2008 at 12:50 PM, Philippe De Muyter <[email protected]> wrote:
>
>>Hi Michael,
>>
>>On Tue, Jul 01, 2008 at 08:16:11PM +0200, Michael Kerrisk wrote:
>>
>>>On 7/1/08, Philippe De Muyter <[email protected]> wrote:
>>>
>>>>Hello everybody,
>>>>
>>>> I develop video acquisition software using the video1394 interface.
>>>> The images grabbed by the camera and iee1394 bus are kept in kernel
>>>> memory and made available to the user program through a mmap call done
>>>> in the libdc1394 library :
>>>>
>>>> dma_ring_buffer= mmap(0, vmmap.nb_buffers * vmmap.buf_size,
>>>> PROT_READ|PROT_WRITE,MAP_SHARED, craw->capture.dma_fd, 0);
>>>>
>>>> Sometimes, my program crashes and produces a core file :) It seems to
>>>> me that the core file does not contain the mmap'ed memory and hence
>>>> I cannot replay my program with the same image for debugging purpose.
>>>>
>>>> Is it possible to configure the kernel through /proc, or through the mmap
>>>> system call to have that mmapped segment in the core file, or do I need
>>>> to modify the kernel itself to obtain the behaviour I want ? If I
>>>> need to modify the kernel, can some kind soul provide me some pointers ?
>>>
>>>
>>>Have a look at the section "Controlling which mappings are written to
>>>the core dump" in a recent core.5 man page:
>>>http://www.kernel.org/doc/man-pages/online/pages/man5/core.5.html
>>
>>thanks for the info. I didn't know about /proc/PID/coredump_filter.
>>
>>that part was promising :
>>
>> bit 2 Dump file-backed private mappings.
>> bit 3 Dump file-backed shared mappings.
>>
>> The default value of coredump_filter is 0x3; this reflects traditional
>> Linux behavior and means that only anonymous memory segments are dumped.
>>
>>Unfortunately, the part that applies to me (I have tested it) is the next one :
>>
>> Memory-mapped I/O pages such as frame buffer are never dumped, [...],
>> regardless of the coredump_filter value.
>>
>>Is that a design decision, or a mere finding of the way it is implemented
>>now ?
MMIO pages have been not dumped since a long time ago, and I didn't target
them for the coredump_filter feature because I thought it was generally
danger to read MMIO pages. As for frame buffer we would be able to
safely access to it, but there is no way to tell it from other MMIO
mappings, AFAIK.
>>So, back to my original question :
>>
>>Can some kind soul provide me some pointers to the way I should modify
>>the kernel to make the inclusion of the video1394 mmapped segment in
>>core files possible ?
>
>
> Perhaps Hidehiro, who wrote the coredump_filter feature, can provide insight.
The following patch may help. To dump MMIO pages, set bit 5 of
coredump_filter.
$ echo 0x23 > /proc/<PID>/coredump_filter
Signed-off-by: Hidehiro Kawai <[email protected]>
---
This patch is not intended to be merged to the upstream kernel
because the safeness of reading VM_IO mappings has not been
proved.
Index: linux-2.6.26-rc5-mm3/fs/binfmt_elf.c
===================================================================
--- linux-2.6.26-rc5-mm3.orig/fs/binfmt_elf.c
+++ linux-2.6.26-rc5-mm3/fs/binfmt_elf.c
@@ -1141,11 +1141,18 @@ static unsigned long vma_dump_size(struc
if (vma->vm_flags & VM_ALWAYSDUMP)
goto whole;
- /* Do not dump I/O mapped devices or special mappings */
- if (vma->vm_flags & (VM_IO | VM_RESERVED))
+#define FILTER(type) (mm_flags & (1UL << MMF_DUMP_##type))
+
+ /* By default, do not dump memory mapped I/O mappings */
+ if (vma->vm_flags & VM_IO) {
+ if (FILTER(MMIO))
+ goto whole;
return 0;
+ }
-#define FILTER(type) (mm_flags & (1UL << MMF_DUMP_##type))
+ /* Do not dump special mappings */
+ if (vma->vm_flags & VM_RESERVED)
+ return 0;
/* By default, dump shared memory if mapped from an anonymous file. */
if (vma->vm_flags & VM_SHARED) {
Index: linux-2.6.26-rc5-mm3/include/linux/sched.h
===================================================================
--- linux-2.6.26-rc5-mm3.orig/include/linux/sched.h
+++ linux-2.6.26-rc5-mm3/include/linux/sched.h
@@ -403,8 +403,9 @@ extern int get_dumpable(struct mm_struct
#define MMF_DUMP_MAPPED_PRIVATE 4
#define MMF_DUMP_MAPPED_SHARED 5
#define MMF_DUMP_ELF_HEADERS 6
+#define MMF_DUMP_MMIO 7
#define MMF_DUMP_FILTER_SHIFT MMF_DUMPABLE_BITS
-#define MMF_DUMP_FILTER_BITS 5
+#define MMF_DUMP_FILTER_BITS 6
#define MMF_DUMP_FILTER_MASK \
(((1 << MMF_DUMP_FILTER_BITS) - 1) << MMF_DUMP_FILTER_SHIFT)
#define MMF_DUMP_FILTER_DEFAULT \
Regards,
--
Hidehiro Kawai
Hitachi, Systems Development Laboratory
Linux Technology Center
Hello Hidehiro,
On Thu, Jul 03, 2008 at 12:51:33PM +0900, Hidehiro Kawai wrote:
> Hi,
>
> Michael Kerrisk wrote:
>
> > [CC+= [email protected]]
> >
> > On Wed, Jul 2, 2008 at 12:50 PM, Philippe De Muyter <[email protected]> wrote:
> >
> >>Hi Michael,
> >>
> >>On Tue, Jul 01, 2008 at 08:16:11PM +0200, Michael Kerrisk wrote:
> >>
> >>>On 7/1/08, Philippe De Muyter <[email protected]> wrote:
> >>>
> >>>>Hello everybody,
> >>>>
> >>>> I develop video acquisition software using the video1394 interface.
> >>>> The images grabbed by the camera and iee1394 bus are kept in kernel
> >>>> memory and made available to the user program through a mmap call done
> >>>> in the libdc1394 library :
> >>>>
> >>>> dma_ring_buffer= mmap(0, vmmap.nb_buffers * vmmap.buf_size,
> >>>> PROT_READ|PROT_WRITE,MAP_SHARED, craw->capture.dma_fd, 0);
> >>>>
> >>>> Sometimes, my program crashes and produces a core file :) It seems to
> >>>> me that the core file does not contain the mmap'ed memory and hence
> >>>> I cannot replay my program with the same image for debugging purpose.
> >>>>
> >>>> Is it possible to configure the kernel through /proc, or through the mmap
> >>>> system call to have that mmapped segment in the core file, or do I need
> >>>> to modify the kernel itself to obtain the behaviour I want ? If I
> >>>> need to modify the kernel, can some kind soul provide me some pointers ?
> >>>
> >>>
> >>>Have a look at the section "Controlling which mappings are written to
> >>>the core dump" in a recent core.5 man page:
> >>>http://www.kernel.org/doc/man-pages/online/pages/man5/core.5.html
> >>
> >>thanks for the info. I didn't know about /proc/PID/coredump_filter.
> >>
> >>that part was promising :
> >>
> >> bit 2 Dump file-backed private mappings.
> >> bit 3 Dump file-backed shared mappings.
> >>
> >> The default value of coredump_filter is 0x3; this reflects traditional
> >> Linux behavior and means that only anonymous memory segments are dumped.
> >>
> >>Unfortunately, the part that applies to me (I have tested it) is the next one :
> >>
> >> Memory-mapped I/O pages such as frame buffer are never dumped, [...],
> >> regardless of the coredump_filter value.
> >>
> >>Is that a design decision, or a mere finding of the way it is implemented
> >>now ?
>
> MMIO pages have been not dumped since a long time ago, and I didn't target
> them for the coredump_filter feature because I thought it was generally
> danger to read MMIO pages. As for frame buffer we would be able to
> safely access to it, but there is no way to tell it from other MMIO
> mappings, AFAIK.
>
> >>So, back to my original question :
> >>
> >>Can some kind soul provide me some pointers to the way I should modify
> >>the kernel to make the inclusion of the video1394 mmapped segment in
> >>core files possible ?
> >
> >
> > Perhaps Hidehiro, who wrote the coredump_filter feature, can provide insight.
>
> The following patch may help. To dump MMIO pages, set bit 5 of
> coredump_filter.
>
> $ echo 0x23 > /proc/<PID>/coredump_filter
>
>
> Signed-off-by: Hidehiro Kawai <[email protected]>
> ---
> This patch is not intended to be merged to the upstream kernel
> because the safeness of reading VM_IO mappings has not been
> proved.
>
> Index: linux-2.6.26-rc5-mm3/fs/binfmt_elf.c
> ===================================================================
> --- linux-2.6.26-rc5-mm3.orig/fs/binfmt_elf.c
> +++ linux-2.6.26-rc5-mm3/fs/binfmt_elf.c
> @@ -1141,11 +1141,18 @@ static unsigned long vma_dump_size(struc
> if (vma->vm_flags & VM_ALWAYSDUMP)
> goto whole;
>
> - /* Do not dump I/O mapped devices or special mappings */
> - if (vma->vm_flags & (VM_IO | VM_RESERVED))
> +#define FILTER(type) (mm_flags & (1UL << MMF_DUMP_##type))
> +
> + /* By default, do not dump memory mapped I/O mappings */
> + if (vma->vm_flags & VM_IO) {
> + if (FILTER(MMIO))
> + goto whole;
> return 0;
> + }
>
> -#define FILTER(type) (mm_flags & (1UL << MMF_DUMP_##type))
> + /* Do not dump special mappings */
> + if (vma->vm_flags & VM_RESERVED)
> + return 0;
>
> /* By default, dump shared memory if mapped from an anonymous file. */
> if (vma->vm_flags & VM_SHARED) {
> Index: linux-2.6.26-rc5-mm3/include/linux/sched.h
> ===================================================================
> --- linux-2.6.26-rc5-mm3.orig/include/linux/sched.h
> +++ linux-2.6.26-rc5-mm3/include/linux/sched.h
> @@ -403,8 +403,9 @@ extern int get_dumpable(struct mm_struct
> #define MMF_DUMP_MAPPED_PRIVATE 4
> #define MMF_DUMP_MAPPED_SHARED 5
> #define MMF_DUMP_ELF_HEADERS 6
> +#define MMF_DUMP_MMIO 7
> #define MMF_DUMP_FILTER_SHIFT MMF_DUMPABLE_BITS
> -#define MMF_DUMP_FILTER_BITS 5
> +#define MMF_DUMP_FILTER_BITS 6
> #define MMF_DUMP_FILTER_MASK \
> (((1 << MMF_DUMP_FILTER_BITS) - 1) << MMF_DUMP_FILTER_SHIFT)
> #define MMF_DUMP_FILTER_DEFAULT \
>
>
>
> Regards,
Thanks for your patch, but it will not help here. Before applying it blindly
I asked myself if the mmapped region was tagged VM_IO, because it is actually
a simple ram zone, not an I/O zone, and the answer is it is not a VM_IO zone.
Details :
drivers/ieee1394/video1394.c:
static int video1394_mmap(struct file *file, struct vm_area_struct *vma)
{
[...]
return dma_region_mmap(&ctx->current_ctx->dma, file, vma);
}
drivers/ieee1394/dma.c:
int dma_region_mmap(struct dma_region *dma, struct file *file,
struct vm_area_struct *vma)
{
[...]
vma->vm_ops = &dma_region_vm_ops;
vma->vm_private_data = dma;
vma->vm_file = file;
vma->vm_flags |= VM_RESERVED;
return 0;
}
So, actually the zone I would like to get dumped in the core file is tagged
VM_RESERVED.
I see the following ways to solve my problem :
- do not tag the zone as VM_RESERVED in ieee1394::dma_region_mmap
- tag the zone as VM_ALWAYSDUMP in ieee1394::dma_region_mmap
- add a bit in coredump_filter to dump the VM_RESERVED zones.
As I don't know the real meaning of VM_RESERVED, I do not know which choice
is the best one for the official kernel tree, but locally I'll go for
adding VM_ALWAYSDUMP in ieee1394::dma_region_mmap.
Philippe
[Sorry, resent with CC += [email protected]]
Hello Hidehiro,
On Thu, Jul 03, 2008 at 12:51:33PM +0900, Hidehiro Kawai wrote:
> Hi,
>
> Michael Kerrisk wrote:
>
> > [CC+= [email protected]]
> >
> > On Wed, Jul 2, 2008 at 12:50 PM, Philippe De Muyter <[email protected]> wrote:
> >
> >>Hi Michael,
> >>
> >>On Tue, Jul 01, 2008 at 08:16:11PM +0200, Michael Kerrisk wrote:
> >>
> >>>On 7/1/08, Philippe De Muyter <[email protected]> wrote:
> >>>
> >>>>Hello everybody,
> >>>>
> >>>> I develop video acquisition software using the video1394 interface.
> >>>> The images grabbed by the camera and iee1394 bus are kept in kernel
> >>>> memory and made available to the user program through a mmap call done
> >>>> in the libdc1394 library :
> >>>>
> >>>> dma_ring_buffer= mmap(0, vmmap.nb_buffers * vmmap.buf_size,
> >>>> PROT_READ|PROT_WRITE,MAP_SHARED, craw->capture.dma_fd, 0);
> >>>>
> >>>> Sometimes, my program crashes and produces a core file :) It seems to
> >>>> me that the core file does not contain the mmap'ed memory and hence
> >>>> I cannot replay my program with the same image for debugging purpose.
> >>>>
> >>>> Is it possible to configure the kernel through /proc, or through the mmap
> >>>> system call to have that mmapped segment in the core file, or do I need
> >>>> to modify the kernel itself to obtain the behaviour I want ? If I
> >>>> need to modify the kernel, can some kind soul provide me some pointers ?
> >>>
> >>>
> >>>Have a look at the section "Controlling which mappings are written to
> >>>the core dump" in a recent core.5 man page:
> >>>http://www.kernel.org/doc/man-pages/online/pages/man5/core.5.html
> >>
> >>thanks for the info. I didn't know about /proc/PID/coredump_filter.
> >>
> >>that part was promising :
> >>
> >> bit 2 Dump file-backed private mappings.
> >> bit 3 Dump file-backed shared mappings.
> >>
> >> The default value of coredump_filter is 0x3; this reflects traditional
> >> Linux behavior and means that only anonymous memory segments are dumped.
> >>
> >>Unfortunately, the part that applies to me (I have tested it) is the next one :
> >>
> >> Memory-mapped I/O pages such as frame buffer are never dumped, [...],
> >> regardless of the coredump_filter value.
> >>
> >>Is that a design decision, or a mere finding of the way it is implemented
> >>now ?
>
> MMIO pages have been not dumped since a long time ago, and I didn't target
> them for the coredump_filter feature because I thought it was generally
> danger to read MMIO pages. As for frame buffer we would be able to
> safely access to it, but there is no way to tell it from other MMIO
> mappings, AFAIK.
>
> >>So, back to my original question :
> >>
> >>Can some kind soul provide me some pointers to the way I should modify
> >>the kernel to make the inclusion of the video1394 mmapped segment in
> >>core files possible ?
> >
> >
> > Perhaps Hidehiro, who wrote the coredump_filter feature, can provide insight.
>
> The following patch may help. To dump MMIO pages, set bit 5 of
> coredump_filter.
>
> $ echo 0x23 > /proc/<PID>/coredump_filter
>
>
> Signed-off-by: Hidehiro Kawai <[email protected]>
> ---
> This patch is not intended to be merged to the upstream kernel
> because the safeness of reading VM_IO mappings has not been
> proved.
>
> Index: linux-2.6.26-rc5-mm3/fs/binfmt_elf.c
> ===================================================================
> --- linux-2.6.26-rc5-mm3.orig/fs/binfmt_elf.c
> +++ linux-2.6.26-rc5-mm3/fs/binfmt_elf.c
> @@ -1141,11 +1141,18 @@ static unsigned long vma_dump_size(struc
> if (vma->vm_flags & VM_ALWAYSDUMP)
> goto whole;
>
> - /* Do not dump I/O mapped devices or special mappings */
> - if (vma->vm_flags & (VM_IO | VM_RESERVED))
> +#define FILTER(type) (mm_flags & (1UL << MMF_DUMP_##type))
> +
> + /* By default, do not dump memory mapped I/O mappings */
> + if (vma->vm_flags & VM_IO) {
> + if (FILTER(MMIO))
> + goto whole;
> return 0;
> + }
>
> -#define FILTER(type) (mm_flags & (1UL << MMF_DUMP_##type))
> + /* Do not dump special mappings */
> + if (vma->vm_flags & VM_RESERVED)
> + return 0;
>
> /* By default, dump shared memory if mapped from an anonymous file. */
> if (vma->vm_flags & VM_SHARED) {
> Index: linux-2.6.26-rc5-mm3/include/linux/sched.h
> ===================================================================
> --- linux-2.6.26-rc5-mm3.orig/include/linux/sched.h
> +++ linux-2.6.26-rc5-mm3/include/linux/sched.h
> @@ -403,8 +403,9 @@ extern int get_dumpable(struct mm_struct
> #define MMF_DUMP_MAPPED_PRIVATE 4
> #define MMF_DUMP_MAPPED_SHARED 5
> #define MMF_DUMP_ELF_HEADERS 6
> +#define MMF_DUMP_MMIO 7
> #define MMF_DUMP_FILTER_SHIFT MMF_DUMPABLE_BITS
> -#define MMF_DUMP_FILTER_BITS 5
> +#define MMF_DUMP_FILTER_BITS 6
> #define MMF_DUMP_FILTER_MASK \
> (((1 << MMF_DUMP_FILTER_BITS) - 1) << MMF_DUMP_FILTER_SHIFT)
> #define MMF_DUMP_FILTER_DEFAULT \
>
>
>
> Regards,
Thanks for your patch, but it will not help here. Before applying it blindly
I asked myself if the mmapped region was tagged VM_IO, because it is actually
a simple ram zone, not an I/O zone, and the answer is it is not a VM_IO zone.
Details :
drivers/ieee1394/video1394.c:
static int video1394_mmap(struct file *file, struct vm_area_struct *vma)
{
[...]
return dma_region_mmap(&ctx->current_ctx->dma, file, vma);
}
drivers/ieee1394/dma.c:
int dma_region_mmap(struct dma_region *dma, struct file *file,
struct vm_area_struct *vma)
{
[...]
vma->vm_ops = &dma_region_vm_ops;
vma->vm_private_data = dma;
vma->vm_file = file;
vma->vm_flags |= VM_RESERVED;
return 0;
}
So, actually the zone I would like to get dumped in the core file is tagged
VM_RESERVED.
I see the following ways to solve my problem :
- do not tag the zone as VM_RESERVED in ieee1394::dma_region_mmap
- tag the zone as VM_ALWAYSDUMP in ieee1394::dma_region_mmap
- add a bit in coredump_filter to dump the VM_RESERVED zones.
As I don't know the real meaning of VM_RESERVED, I do not know which choice
is the best one for the official kernel tree, but locally I'll go for
adding VM_ALWAYSDUMP in ieee1394::dma_region_mmap.
Philippe
Currently, core files do not contain the mmapped memory of the video1394
or dv1394 devices, which contain the actual video input, making it
impossible to analyse the cause of abnormal program termination for
image analysis or (de)compression software. Fix that.
Signed-off-by: Philippe De Muyter <[email protected]>
---
diff -r ced66ca0044f drivers/ieee1394/dma.c
--- a/drivers/ieee1394/dma.c Mon Jun 30 08:58:09 2008 -0700
+++ b/drivers/ieee1394/dma.c Thu Jul 3 18:40:24 2008 +0200
@@ -274,7 +274,7 @@ int dma_region_mmap(struct dma_region *d
vma->vm_ops = &dma_region_vm_ops;
vma->vm_private_data = dma;
vma->vm_file = file;
- vma->vm_flags |= VM_RESERVED;
+ vma->vm_flags |= VM_RESERVED | VM_ALWAYSDUMP;
return 0;
}
Hello,
Philippe De Muyter wrote:
> Hello Hidehiro,
>
> On Thu, Jul 03, 2008 at 12:51:33PM +0900, Hidehiro Kawai wrote:
>
>>Hi,
>>
>>Michael Kerrisk wrote:
>>
>>
>>>[CC+= [email protected]]
>>>
>>>On Wed, Jul 2, 2008 at 12:50 PM, Philippe De Muyter <[email protected]> wrote:
>>>
>>>
>>>>Hi Michael,
>>>>
>>>>On Tue, Jul 01, 2008 at 08:16:11PM +0200, Michael Kerrisk wrote:
>>>>
>>>>
>>>>>On 7/1/08, Philippe De Muyter <[email protected]> wrote:
>>>>>
>>>>>
>>>>>>Hello everybody,
>>>>>>
>>>>>>I develop video acquisition software using the video1394 interface.
>>>>>>The images grabbed by the camera and iee1394 bus are kept in kernel
>>>>>>memory and made available to the user program through a mmap call done
>>>>>>in the libdc1394 library :
>>>>>>
>>>>>>dma_ring_buffer= mmap(0, vmmap.nb_buffers * vmmap.buf_size,
>>>>>> PROT_READ|PROT_WRITE,MAP_SHARED, craw->capture.dma_fd, 0);
>>>>>>
>>>>>>Sometimes, my program crashes and produces a core file :) It seems to
>>>>>>me that the core file does not contain the mmap'ed memory and hence
>>>>>>I cannot replay my program with the same image for debugging purpose.
>>>>>>
>>>>>>Is it possible to configure the kernel through /proc, or through the mmap
>>>>>>system call to have that mmapped segment in the core file, or do I need
>>>>>>to modify the kernel itself to obtain the behaviour I want ? If I
>>>>>>need to modify the kernel, can some kind soul provide me some pointers ?
>>>>>
>>>>>
>>>>>Have a look at the section "Controlling which mappings are written to
>>>>>the core dump" in a recent core.5 man page:
>>>>>http://www.kernel.org/doc/man-pages/online/pages/man5/core.5.html
>>>>
>>>>thanks for the info. I didn't know about /proc/PID/coredump_filter.
>>>>
>>>>that part was promising :
>>>>
>>>> bit 2 Dump file-backed private mappings.
>>>> bit 3 Dump file-backed shared mappings.
>>>>
>>>> The default value of coredump_filter is 0x3; this reflects traditional
>>>> Linux behavior and means that only anonymous memory segments are dumped.
>>>>
>>>>Unfortunately, the part that applies to me (I have tested it) is the next one :
>>>>
>>>> Memory-mapped I/O pages such as frame buffer are never dumped, [...],
>>>> regardless of the coredump_filter value.
>>>>
>>>>Is that a design decision, or a mere finding of the way it is implemented
>>>>now ?
>>
>>MMIO pages have been not dumped since a long time ago, and I didn't target
>>them for the coredump_filter feature because I thought it was generally
>>danger to read MMIO pages. As for frame buffer we would be able to
>>safely access to it, but there is no way to tell it from other MMIO
>>mappings, AFAIK.
>>
>>
>>>>So, back to my original question :
>>>>
>>>>Can some kind soul provide me some pointers to the way I should modify
>>>>the kernel to make the inclusion of the video1394 mmapped segment in
>>>>core files possible ?
>>>
>>>
>>>Perhaps Hidehiro, who wrote the coredump_filter feature, can provide insight.
>>
>>The following patch may help. To dump MMIO pages, set bit 5 of
>>coredump_filter.
>>
>>$ echo 0x23 > /proc/<PID>/coredump_filter
>>
>>
>>Signed-off-by: Hidehiro Kawai <[email protected]>
>>---
>>This patch is not intended to be merged to the upstream kernel
>>because the safeness of reading VM_IO mappings has not been
>>proved.
>>
>>Index: linux-2.6.26-rc5-mm3/fs/binfmt_elf.c
>>===================================================================
>>--- linux-2.6.26-rc5-mm3.orig/fs/binfmt_elf.c
>>+++ linux-2.6.26-rc5-mm3/fs/binfmt_elf.c
>>@@ -1141,11 +1141,18 @@ static unsigned long vma_dump_size(struc
>> if (vma->vm_flags & VM_ALWAYSDUMP)
>> goto whole;
>>
>>- /* Do not dump I/O mapped devices or special mappings */
>>- if (vma->vm_flags & (VM_IO | VM_RESERVED))
>>+#define FILTER(type) (mm_flags & (1UL << MMF_DUMP_##type))
>>+
>>+ /* By default, do not dump memory mapped I/O mappings */
>>+ if (vma->vm_flags & VM_IO) {
>>+ if (FILTER(MMIO))
>>+ goto whole;
>> return 0;
>>+ }
>>
>>-#define FILTER(type) (mm_flags & (1UL << MMF_DUMP_##type))
>>+ /* Do not dump special mappings */
>>+ if (vma->vm_flags & VM_RESERVED)
>>+ return 0;
>>
>> /* By default, dump shared memory if mapped from an anonymous file. */
>> if (vma->vm_flags & VM_SHARED) {
>>Index: linux-2.6.26-rc5-mm3/include/linux/sched.h
>>===================================================================
>>--- linux-2.6.26-rc5-mm3.orig/include/linux/sched.h
>>+++ linux-2.6.26-rc5-mm3/include/linux/sched.h
>>@@ -403,8 +403,9 @@ extern int get_dumpable(struct mm_struct
>> #define MMF_DUMP_MAPPED_PRIVATE 4
>> #define MMF_DUMP_MAPPED_SHARED 5
>> #define MMF_DUMP_ELF_HEADERS 6
>>+#define MMF_DUMP_MMIO 7
>> #define MMF_DUMP_FILTER_SHIFT MMF_DUMPABLE_BITS
>>-#define MMF_DUMP_FILTER_BITS 5
>>+#define MMF_DUMP_FILTER_BITS 6
>> #define MMF_DUMP_FILTER_MASK \
>> (((1 << MMF_DUMP_FILTER_BITS) - 1) << MMF_DUMP_FILTER_SHIFT)
>> #define MMF_DUMP_FILTER_DEFAULT \
>>
>>
>>
>>Regards,
>
>
> Thanks for your patch, but it will not help here. Before applying it blindly
> I asked myself if the mmapped region was tagged VM_IO, because it is actually
> a simple ram zone, not an I/O zone, and the answer is it is not a VM_IO zone.
> Details :
>
> drivers/ieee1394/video1394.c:
> static int video1394_mmap(struct file *file, struct vm_area_struct *vma)
> {
> [...]
> return dma_region_mmap(&ctx->current_ctx->dma, file, vma);
> }
>
> drivers/ieee1394/dma.c:
> int dma_region_mmap(struct dma_region *dma, struct file *file,
> struct vm_area_struct *vma)
> {
> [...]
> vma->vm_ops = &dma_region_vm_ops;
> vma->vm_private_data = dma;
> vma->vm_file = file;
> vma->vm_flags |= VM_RESERVED;
>
> return 0;
> }
>
> So, actually the zone I would like to get dumped in the core file is tagged
> VM_RESERVED.
>
> I see the following ways to solve my problem :
> - do not tag the zone as VM_RESERVED in ieee1394::dma_region_mmap
> - tag the zone as VM_ALWAYSDUMP in ieee1394::dma_region_mmap
> - add a bit in coredump_filter to dump the VM_RESERVED zones.
>
> As I don't know the real meaning of VM_RESERVED, I do not know which choice
> is the best one for the official kernel tree, but locally I'll go for
> adding VM_ALWAYSDUMP in ieee1394::dma_region_mmap.
I'm afraid I don't know real usages of VM_RESERVED and VM_IO, either.
Allowing everyone to choose whether dump the dma region or not,
perhaps we need to introduce a new VM flag (e.g. VM_DUMPABLE) and
a coredump_filter bit which controls (VM_IO | VM_RESERVED) &&
VM_DUMPABLE area, for example.
I think it is also OK to just add VM_ALWAYSDUMP flag to the dma
region if the device driver knows the region is safely readable
and small enough.
Regards,
--
Hidehiro Kawai
Hitachi, Systems Development Laboratory
Linux Technology Center
Hidehiro Kawai wrote:
> Philippe De Muyter wrote:
>> drivers/ieee1394/dma.c:
>> int dma_region_mmap(struct dma_region *dma, struct file *file,
>> struct vm_area_struct *vma)
>> {
>> [...]
>> vma->vm_ops = &dma_region_vm_ops;
>> vma->vm_private_data = dma;
>> vma->vm_file = file;
>> vma->vm_flags |= VM_RESERVED;
>>
>> return 0;
>> }
>>
>> So, actually the zone I would like to get dumped in the core file is tagged
>> VM_RESERVED.
>>
>> I see the following ways to solve my problem :
>> - do not tag the zone as VM_RESERVED in ieee1394::dma_region_mmap
>> - tag the zone as VM_ALWAYSDUMP in ieee1394::dma_region_mmap
>> - add a bit in coredump_filter to dump the VM_RESERVED zones.
>>
>> As I don't know the real meaning of VM_RESERVED, I do not know which choice
>> is the best one for the official kernel tree,
I don't know these things either. But among else, VM_RESERVED prevents
a vma from being swapped out. Makes kind of sense, given that besides
the CPUs also the FireWire controller writes to this memory.
>> but locally I'll go for
>> adding VM_ALWAYSDUMP in ieee1394::dma_region_mmap.
>
> I'm afraid I don't know real usages of VM_RESERVED and VM_IO, either.
> Allowing everyone to choose whether dump the dma region or not,
> perhaps we need to introduce a new VM flag (e.g. VM_DUMPABLE) and
> a coredump_filter bit which controls (VM_IO | VM_RESERVED) &&
> VM_DUMPABLE area, for example.
>
> I think it is also OK to just add VM_ALWAYSDUMP flag to the dma
> region if the device driver knows the region is safely readable
> and small enough.
It is safely readable. I don't know if it is small enough. The size of
the DMA buffer is AFAIK chosen by userspace (by the application program
or maybe a library) which uses the character device file ABIs for
isochronous FireWire IO of raw1394, video1394, or dv1394.
--
Stefan Richter
-=====-==--- -=== --=--
http://arcgraph.de/sr/
Hi Hidehiro,
On Fri, Jul 04, 2008 at 02:50:25PM +0900, Hidehiro Kawai wrote:
[...]
> > I see the following ways to solve my problem :
> > - do not tag the zone as VM_RESERVED in ieee1394::dma_region_mmap
> > - tag the zone as VM_ALWAYSDUMP in ieee1394::dma_region_mmap
> > - add a bit in coredump_filter to dump the VM_RESERVED zones.
> >
> > As I don't know the real meaning of VM_RESERVED, I do not know which choice
> > is the best one for the official kernel tree, but locally I'll go for
> > adding VM_ALWAYSDUMP in ieee1394::dma_region_mmap.
>
> I'm afraid I don't know real usages of VM_RESERVED and VM_IO, either.
> Allowing everyone to choose whether dump the dma region or not,
> perhaps we need to introduce a new VM flag (e.g. VM_DUMPABLE) and
> a coredump_filter bit which controls (VM_IO | VM_RESERVED) &&
> VM_DUMPABLE area, for example.
>
> I think it is also OK to just add VM_ALWAYSDUMP flag to the dma
> region if the device driver knows the region is safely readable
> and small enough.
I have just submitted a patch doing that for ieee1394 dma regions:
http://marc.info/?l=linux-kernel&m=121510404729225&w=2
Thanks for your help
Philippe
On Fri, Jul 04, 2008 at 08:33:14AM +0200, Stefan Richter wrote:
> Hidehiro Kawai wrote:
>> Philippe De Muyter wrote:
>>> drivers/ieee1394/dma.c:
>>> int dma_region_mmap(struct dma_region *dma, struct file *file,
>>> struct vm_area_struct *vma)
>>> {
>>> [...]
>>> vma->vm_ops = &dma_region_vm_ops;
>>> vma->vm_private_data = dma;
>>> vma->vm_file = file;
>>> vma->vm_flags |= VM_RESERVED;
>>>
>>> return 0;
>>> }
>>>
>>> So, actually the zone I would like to get dumped in the core file is
>>> tagged
>>> VM_RESERVED.
>>>
>>> I see the following ways to solve my problem :
>>> - do not tag the zone as VM_RESERVED in ieee1394::dma_region_mmap
>>> - tag the zone as VM_ALWAYSDUMP in ieee1394::dma_region_mmap
>>> - add a bit in coredump_filter to dump the VM_RESERVED zones.
>>>
>>> As I don't know the real meaning of VM_RESERVED, I do not know which
>>> choice
>>> is the best one for the official kernel tree,
>
> I don't know these things either. But among else, VM_RESERVED prevents a
> vma from being swapped out. Makes kind of sense, given that besides the
> CPUs also the FireWire controller writes to this memory.
>
>>> but locally I'll go for
>>> adding VM_ALWAYSDUMP in ieee1394::dma_region_mmap.
>> I'm afraid I don't know real usages of VM_RESERVED and VM_IO, either.
>> Allowing everyone to choose whether dump the dma region or not,
>> perhaps we need to introduce a new VM flag (e.g. VM_DUMPABLE) and
>> a coredump_filter bit which controls (VM_IO | VM_RESERVED) &&
>> VM_DUMPABLE area, for example.
>> I think it is also OK to just add VM_ALWAYSDUMP flag to the dma
>> region if the device driver knows the region is safely readable
>> and small enough.
>
> It is safely readable. I don't know if it is small enough. The size of
> the DMA buffer is AFAIK chosen by userspace (by the application program or
> maybe a library) which uses the character device file ABIs for isochronous
> FireWire IO of raw1394, video1394, or dv1394.
For video1394, the size of the buffer is N times the size of a frame, with
N choosen by the application but small (let say < 32). The size of the
frame depends on the camera. I have already worked with 8bpp 1600x1200 cameras,but with a smaller N in that case.
1600x1200x32 gives 60M. If they were allocated by malloc instead of by mmap,
they would be dumped in the core files without any question about the size.
So I think we may consider the 'small enough' criteria is met.
Best regards
Philippe
On Fri, 4 Jul 2008, Stefan Richter wrote:
> Hidehiro Kawai wrote:
> > Philippe De Muyter wrote:
> > >
> > > As I don't know the real meaning of VM_RESERVED, I do not know which
> > > choice
> > > is the best one for the official kernel tree,
>
> I don't know these things either. But among else, VM_RESERVED prevents a vma
> from being swapped out. Makes kind of sense, given that besides the CPUs also
> the FireWire controller writes to this memory.
>
> > > but locally I'll go for
> > > adding VM_ALWAYSDUMP in ieee1394::dma_region_mmap.
> >
> > I'm afraid I don't know real usages of VM_RESERVED and VM_IO, either.
They're rather an embarrassment, nobody can say quite what VM_RESERVED
and VM_IO really mean, beyond not-your-ordinary-vma: VM_VOODOO, beware.
They both say "count me as reserved_vm", which means /proc/<pid>/status
leaves them out of VmSize.
Certainly VM_IO hints at special I/O memory, but in most cases there's
nothing very special about either: just memory preallocated by a driver.
In 2.4 VM_RESERVED indeed prevented swapout from looking at the pages
of that vma. We probably ought to have killed it as part of the 2.5
switch to rmap. In 2.6 the pages of a VM_RESERVED vma wouldn't be
considered for swapout, because they're not put on any LRU for that.
I think I promised (ah, that's a shameful phrase!) to get rid of it
a few years ago, but never quite got around to it.
VM_RESERVED and VM_IO are treated as equivalent in most tests, but
there's just a few places, e.g. get_user_pages, where we're scared
off by VM_IO but don't mind VM_RESERVED.
VM_ALWAYSDUMP: marvellous, a flag with an understandable name and
a clear purpose.
Hugh
Philippe De Muyter wrote:
> Currently, core files do not contain the mmapped memory of the video1394
> or dv1394 devices,
Ditto with /dev/raw1394 when the rawiso ioctl API is used.
> which contain the actual video input, making it
> impossible to analyse the cause of abnormal program termination for
> image analysis or (de)compression software. Fix that.
>
> Signed-off-by: Philippe De Muyter <[email protected]>
> ---
> diff -r ced66ca0044f drivers/ieee1394/dma.c
> --- a/drivers/ieee1394/dma.c Mon Jun 30 08:58:09 2008 -0700
> +++ b/drivers/ieee1394/dma.c Thu Jul 3 18:40:24 2008 +0200
> @@ -274,7 +274,7 @@ int dma_region_mmap(struct dma_region *d
> vma->vm_ops = &dma_region_vm_ops;
> vma->vm_private_data = dma;
> vma->vm_file = file;
> - vma->vm_flags |= VM_RESERVED;
> + vma->vm_flags |= VM_RESERVED | VM_ALWAYSDUMP;
>
> return 0;
> }
I'll commit it with the title changed to "dump mmapped iso buffers in
core files" if that's OK.
--
Stefan Richter
-=====-==--- -=== --=--
http://arcgraph.de/sr/
On Fri, Jul 04, 2008 at 08:33:17PM +0200, Stefan Richter wrote:
> Philippe De Muyter wrote:
>> Currently, core files do not contain the mmapped memory of the video1394
>> or dv1394 devices,
>
> Ditto with /dev/raw1394 when the rawiso ioctl API is used.
>
>> which contain the actual video input, making it
>> impossible to analyse the cause of abnormal program termination for
>> image analysis or (de)compression software. Fix that.
>> Signed-off-by: Philippe De Muyter <[email protected]>
>> ---
>> diff -r ced66ca0044f drivers/ieee1394/dma.c
>> --- a/drivers/ieee1394/dma.c Mon Jun 30 08:58:09 2008 -0700
>> +++ b/drivers/ieee1394/dma.c Thu Jul 3 18:40:24 2008 +0200
>> @@ -274,7 +274,7 @@ int dma_region_mmap(struct dma_region *d
>> vma->vm_ops = &dma_region_vm_ops;
>> vma->vm_private_data = dma;
>> vma->vm_file = file;
>> - vma->vm_flags |= VM_RESERVED;
>> + vma->vm_flags |= VM_RESERVED | VM_ALWAYSDUMP;
>> return 0;
>> }
>
> I'll commit it with the title changed to "dump mmapped iso buffers in core
> files" if that's OK.
OK for me
Philippe