Hello,
I'm not sure how to handle this,
while compiling firefox-3.6b1.source
I get this with the default compiling options,
as well as custom:
[ 532.942324] cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0
[ 532.942330] Pid: 16002, comm: cc1plus Tainted: P
2.6.32-rc5-00083-g04ea458 #2
[ 532.942333] Call Trace:
[ 532.942342] [<ffffffff810bce11>] T.417+0x7c/0x245
[ 532.942347] [<ffffffff810bd11c>] __out_of_memory+0x142/0x159
[ 532.942352] [<ffffffff810bd1a1>] out_of_memory+0x6e/0x9d
[ 532.942357] [<ffffffff810c0086>] __alloc_pages_nodemask+0x47e/0x5cc
[ 532.942363] [<ffffffff810d2115>] handle_mm_fault+0x25d/0x68e
[ 532.942369] [<ffffffff813dfc1d>] do_page_fault+0x2bb/0x2d3
[ 532.942373] [<ffffffff813ddb25>] page_fault+0x25/0x30
[ 532.942376] Mem-Info:
[ 532.942378] DMA per-cpu:
[ 532.942380] CPU 0: hi: 0, btch: 1 usd: 0
[ 532.942383] CPU 1: hi: 0, btch: 1 usd: 0
[ 532.942385] DMA32 per-cpu:
[ 532.942388] CPU 0: hi: 186, btch: 31 usd: 94
[ 532.942391] CPU 1: hi: 186, btch: 31 usd: 23
[ 532.942393] Normal per-cpu:
[ 532.942395] CPU 0: hi: 186, btch: 31 usd: 150
[ 532.942398] CPU 1: hi: 186, btch: 31 usd: 155
[ 532.942404] active_anon:707575 inactive_anon:264673 isolated_anon:0
[ 532.942406] active_file:58 inactive_file:33 isolated_file:0
[ 532.942407] unevictable:0 dirty:0 writeback:0 unstable:0 buffer:71
[ 532.942408] free:6998 slab_reclaimable:2697 slab_unreclaimable:16267
[ 532.942409] mapped:136 shmem:64 pagetables:2761 bounce:0
[ 532.942418] DMA free:15944kB min:28kB low:32kB high:40kB
active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15360kB
mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB
slab_reclaimable:0kB slab_unreclaimable:8kB kernel_stack:0kB
pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB
pages_scanned:0 all_unreclaimable? yes
[ 532.942425] lowmem_reserve[]: 0 2976 3986 3986
[ 532.942436] DMA32 free:10020kB min:6020kB low:7524kB high:9028kB
active_anon:2360492kB inactive_anon:590196kB active_file:84kB
inactive_file:68kB unevictable:0kB isolated(anon):0kB
isolated(file):0kB present:3047792kB mlocked:0kB dirty:0kB
writeback:0kB mapped:136kB shmem:80kB slab_reclaimable:0kB
slab_unreclaimable:88kB kernel_stack:0kB pagetables:6100kB
unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:448
all_unreclaimable? yes
[ 532.942443] lowmem_reserve[]: 0 0 1010 1010
[ 532.942454] Normal free:2028kB min:2040kB low:2548kB high:3060kB
active_anon:469808kB inactive_anon:468496kB active_file:148kB
inactive_file:64kB unevictable:0kB isolated(anon):0kB
isolated(file):0kB present:1034240kB mlocked:0kB dirty:0kB
writeback:0kB mapped:408kB shmem:176kB slab_reclaimable:10788kB
slab_unreclaimable:64972kB kernel_stack:800kB pagetables:4944kB
unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:413
all_unreclaimable? yes
[ 532.942461] lowmem_reserve[]: 0 0 0 0
[ 532.942465] DMA: 2*4kB 2*8kB 3*16kB 2*32kB 3*64kB 2*128kB 2*256kB
1*512kB 2*1024kB 2*2048kB 2*4096kB = 15944kB
[ 532.942478] DMA32: 7*4kB 7*8kB 5*16kB 4*32kB 2*64kB 1*128kB 1*256kB
4*512kB 3*1024kB 0*2048kB 1*4096kB = 10020kB
[ 532.942490] Normal: 507*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB
0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2028kB
[ 532.942501] 7974 total pagecache pages
[ 532.942503] 7778 pages in swap cache
[ 532.942506] Swap cache stats: add 112290, delete 104512, find 3595/4051
[ 532.942508] Free swap = 0kB
[ 532.942510] Total swap = 431632kB
[ 532.957941] 1048576 pages RAM
[ 532.957943] 40518 pages reserved
[ 532.957945] 312 pages shared
[ 532.957947] 1000176 pages non-shared
[ 532.957951] Out of memory: kill process 16001 (c++) score 543727 or a child
[ 532.957955] Killed process 16002 (cc1plus)
I just compiled the latest gcc snapshot a few days
ago.
--
Justin P. Mattock
On 11/02/2009 10:29 PM, Justin Mattock wrote:
> [ 532.942324] cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0
> [ 532.942330] Pid: 16002, comm: cc1plus Tainted: P
> 2.6.32-rc5-00083-g04ea458 #2
> [ 532.942333] Call Trace:
> [ 532.942342] [<ffffffff810bce11>] T.417+0x7c/0x245
> [ 532.942347] [<ffffffff810bd11c>] __out_of_memory+0x142/0x159
> [ 532.942352] [<ffffffff810bd1a1>] out_of_memory+0x6e/0x9d
> [ 532.942357] [<ffffffff810c0086>] __alloc_pages_nodemask+0x47e/0x5cc
> [ 532.942363] [<ffffffff810d2115>] handle_mm_fault+0x25d/0x68e
> [ 532.942369] [<ffffffff813dfc1d>] do_page_fault+0x2bb/0x2d3
> [ 532.942373] [<ffffffff813ddb25>] page_fault+0x25/0x30
...
> [ 532.957951] Out of memory: kill process 16001 (c++) score 543727 or a child
> [ 532.957955] Killed process 16002 (cc1plus)
>
> I just compiled the latest gcc snapshot a few days
> ago.
How many jobs did you run in parallel? Was there anything else memory
consuming running on that machine? Do you run the same jobs count every
time you compile such big c++ projects?
On Mon, Nov 2, 2009 at 1:49 PM, Jiri Slaby <[email protected]> wrote:
> On 11/02/2009 10:29 PM, Justin Mattock wrote:
>> [ ?532.942324] cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0
>> [ ?532.942330] Pid: 16002, comm: cc1plus Tainted: P
>> 2.6.32-rc5-00083-g04ea458 #2
>> [ ?532.942333] Call Trace:
>> [ ?532.942342] ?[<ffffffff810bce11>] T.417+0x7c/0x245
>> [ ?532.942347] ?[<ffffffff810bd11c>] __out_of_memory+0x142/0x159
>> [ ?532.942352] ?[<ffffffff810bd1a1>] out_of_memory+0x6e/0x9d
>> [ ?532.942357] ?[<ffffffff810c0086>] __alloc_pages_nodemask+0x47e/0x5cc
>> [ ?532.942363] ?[<ffffffff810d2115>] handle_mm_fault+0x25d/0x68e
>> [ ?532.942369] ?[<ffffffff813dfc1d>] do_page_fault+0x2bb/0x2d3
>> [ ?532.942373] ?[<ffffffff813ddb25>] page_fault+0x25/0x30
> ...
>> [ ?532.957951] Out of memory: kill process 16001 (c++) score 543727 or a child
>> [ ?532.957955] Killed process 16002 (cc1plus)
>>
>> I just compiled the latest gcc snapshot a few days
>> ago.
>
> How many jobs did you run in parallel? Was there anything else memory
> consuming running on that machine? Do you run the same jobs count every
> time you compile such big c++ projects?
>
This would be the only job running.
as for other types of jobs under ps aux there's only
udev,dbus,acpid,sshd,wicd running.
Now with this oom-killer I'm
hitting this on an imac9,1 with
gcc (GCC) 4.5.0 20091029 (experimental)
2.6.32-rc5-00083-g04ea458
On my other machine macbook pro ati chipset
(same system pure64)firefox seems to be going a lot farther
than on the imac(doing this in the past I think this takes a good
45min to compile).
the macbook has:
gcc (GCC) Cross-LFS 4.4.1.20090722) 4.4.1
2.6.32-rc4-00039-ga3f6f2e
--
Justin P. Mattock
On 11/02/2009 11:02 PM, Justin Mattock wrote:
> Now with this oom-killer I'm
> hitting this on an imac9,1 with
> gcc (GCC) 4.5.0 20091029 (experimental)
So there is probably a leak in the gcc chain. Does this happen with a
stable gcc version?
Jiri Slaby wrote:
> On 11/02/2009 11:02 PM, Justin Mattock wrote:
>
>> Now with this oom-killer I'm
>> hitting this on an imac9,1 with
>> gcc (GCC) 4.5.0 20091029 (experimental)
>>
>
> So there is probably a leak in the gcc chain. Does this happen with a
> stable gcc version?
>
>
well right now I'(on the macbook), seems to be going for a good ten
minutes before
I hit an error with a header file(nss)
I'll see if I can get this thing to compile all the way through.
(knock on wood) then go from there.
Justin P. Mattock
Jiri Slaby wrote:
> On 11/02/2009 11:02 PM, Justin Mattock wrote:
>
>> Now with this oom-killer I'm
>> hitting this on an imac9,1 with
>> gcc (GCC) 4.5.0 20091029 (experimental)
>>
>
> So there is probably a leak in the gcc chain. Does this happen with a
> stable gcc version?
>
>
o.k. I think it's something with the latest gcc(snapshot)
right now she's been compiling firefox for 45min
(then crapped out because I compiled nss without sqlite)
without no omm-killer.
Justin P. Mattock
> Hello,
> I'm not sure how to handle this,
> while compiling firefox-3.6b1.source
> I get this with the default compiling options,
> as well as custom:
>
>
> [ 532.942324] cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0
> [ 532.942330] Pid: 16002, comm: cc1plus Tainted: P
> 2.6.32-rc5-00083-g04ea458 #2
> [ 532.942333] Call Trace:
> [ 532.942342] [<ffffffff810bce11>] T.417+0x7c/0x245
> [ 532.942347] [<ffffffff810bd11c>] __out_of_memory+0x142/0x159
> [ 532.942352] [<ffffffff810bd1a1>] out_of_memory+0x6e/0x9d
> [ 532.942357] [<ffffffff810c0086>] __alloc_pages_nodemask+0x47e/0x5cc
> [ 532.942363] [<ffffffff810d2115>] handle_mm_fault+0x25d/0x68e
> [ 532.942369] [<ffffffff813dfc1d>] do_page_fault+0x2bb/0x2d3
> [ 532.942373] [<ffffffff813ddb25>] page_fault+0x25/0x30
> [ 532.942376] Mem-Info:
> [ 532.942378] DMA per-cpu:
> [ 532.942380] CPU 0: hi: 0, btch: 1 usd: 0
> [ 532.942383] CPU 1: hi: 0, btch: 1 usd: 0
> [ 532.942385] DMA32 per-cpu:
> [ 532.942388] CPU 0: hi: 186, btch: 31 usd: 94
> [ 532.942391] CPU 1: hi: 186, btch: 31 usd: 23
> [ 532.942393] Normal per-cpu:
> [ 532.942395] CPU 0: hi: 186, btch: 31 usd: 150
> [ 532.942398] CPU 1: hi: 186, btch: 31 usd: 155
> [ 532.942404] active_anon:707575 inactive_anon:264673 isolated_anon:0
> [ 532.942406] active_file:58 inactive_file:33 isolated_file:0
file cache (active_file + inactive_file) was very little. It indicate anyone waste too much memory.
I doubt you use buggy compiler.
> [ 532.942407] unevictable:0 dirty:0 writeback:0 unstable:0 buffer:71
> [ 532.942408] free:6998 slab_reclaimable:2697 slab_unreclaimable:16267
> [ 532.942409] mapped:136 shmem:64 pagetables:2761 bounce:0
> [ 532.942418] DMA free:15944kB min:28kB low:32kB high:40kB
> active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
> unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15360kB
> mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB
> slab_reclaimable:0kB slab_unreclaimable:8kB kernel_stack:0kB
> pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB
> pages_scanned:0 all_unreclaimable? yes
> [ 532.942425] lowmem_reserve[]: 0 2976 3986 3986
> [ 532.942436] DMA32 free:10020kB min:6020kB low:7524kB high:9028kB
> active_anon:2360492kB inactive_anon:590196kB active_file:84kB
> inactive_file:68kB unevictable:0kB isolated(anon):0kB
> isolated(file):0kB present:3047792kB mlocked:0kB dirty:0kB
> writeback:0kB mapped:136kB shmem:80kB slab_reclaimable:0kB
> slab_unreclaimable:88kB kernel_stack:0kB pagetables:6100kB
> unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:448
> all_unreclaimable? yes
> [ 532.942443] lowmem_reserve[]: 0 0 1010 1010
> [ 532.942454] Normal free:2028kB min:2040kB low:2548kB high:3060kB
> active_anon:469808kB inactive_anon:468496kB active_file:148kB
> inactive_file:64kB unevictable:0kB isolated(anon):0kB
> isolated(file):0kB present:1034240kB mlocked:0kB dirty:0kB
> writeback:0kB mapped:408kB shmem:176kB slab_reclaimable:10788kB
> slab_unreclaimable:64972kB kernel_stack:800kB pagetables:4944kB
> unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:413
> all_unreclaimable? yes
> [ 532.942461] lowmem_reserve[]: 0 0 0 0
> [ 532.942465] DMA: 2*4kB 2*8kB 3*16kB 2*32kB 3*64kB 2*128kB 2*256kB
> 1*512kB 2*1024kB 2*2048kB 2*4096kB = 15944kB
> [ 532.942478] DMA32: 7*4kB 7*8kB 5*16kB 4*32kB 2*64kB 1*128kB 1*256kB
> 4*512kB 3*1024kB 0*2048kB 1*4096kB = 10020kB
> [ 532.942490] Normal: 507*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB
> 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2028kB
> [ 532.942501] 7974 total pagecache pages
> [ 532.942503] 7778 pages in swap cache
> [ 532.942506] Swap cache stats: add 112290, delete 104512, find 3595/4051
> [ 532.942508] Free swap = 0kB
> [ 532.942510] Total swap = 431632kB
> [ 532.957941] 1048576 pages RAM
> [ 532.957943] 40518 pages reserved
> [ 532.957945] 312 pages shared
> [ 532.957947] 1000176 pages non-shared
> [ 532.957951] Out of memory: kill process 16001 (c++) score 543727 or a child
> [ 532.957955] Killed process 16002 (cc1plus)
>
> I just compiled the latest gcc snapshot a few days
> ago.
KOSAKI Motohiro wrote:
>> Hello,
>> I'm not sure how to handle this,
>> while compiling firefox-3.6b1.source
>> I get this with the default compiling options,
>> as well as custom:
>>
>>
>> [ 532.942324] cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0
>> [ 532.942330] Pid: 16002, comm: cc1plus Tainted: P
>> 2.6.32-rc5-00083-g04ea458 #2
>> [ 532.942333] Call Trace:
>> [ 532.942342] [<ffffffff810bce11>] T.417+0x7c/0x245
>> [ 532.942347] [<ffffffff810bd11c>] __out_of_memory+0x142/0x159
>> [ 532.942352] [<ffffffff810bd1a1>] out_of_memory+0x6e/0x9d
>> [ 532.942357] [<ffffffff810c0086>] __alloc_pages_nodemask+0x47e/0x5cc
>> [ 532.942363] [<ffffffff810d2115>] handle_mm_fault+0x25d/0x68e
>> [ 532.942369] [<ffffffff813dfc1d>] do_page_fault+0x2bb/0x2d3
>> [ 532.942373] [<ffffffff813ddb25>] page_fault+0x25/0x30
>> [ 532.942376] Mem-Info:
>> [ 532.942378] DMA per-cpu:
>> [ 532.942380] CPU 0: hi: 0, btch: 1 usd: 0
>> [ 532.942383] CPU 1: hi: 0, btch: 1 usd: 0
>> [ 532.942385] DMA32 per-cpu:
>> [ 532.942388] CPU 0: hi: 186, btch: 31 usd: 94
>> [ 532.942391] CPU 1: hi: 186, btch: 31 usd: 23
>> [ 532.942393] Normal per-cpu:
>> [ 532.942395] CPU 0: hi: 186, btch: 31 usd: 150
>> [ 532.942398] CPU 1: hi: 186, btch: 31 usd: 155
>> [ 532.942404] active_anon:707575 inactive_anon:264673 isolated_anon:0
>> [ 532.942406] active_file:58 inactive_file:33 isolated_file:0
>>
>
> file cache (active_file + inactive_file) was very little. It indicate anyone waste too much memory.
> I doubt you use buggy compiler.
>
>
>
>
hmm... then this is something with firefox then..
In that case I'll continue to build my system with
this compiler.
Although a bit concerned building everything
for the system with a compiler that shows some
memory issue,but if you say its not the compiler,
then I'll carry on with what I'm doing.
(and use an older compiler for firefox).
Justin P. Mattock
On Mon, 2 Nov 2009 13:29:29 -0800 Justin Mattock <[email protected]> wrote:
> Hello,
> I'm not sure how to handle this,
> while compiling firefox-3.6b1.source
> I get this with the default compiling options,
> as well as custom:
>
> ...
>
> active_anon:2360492kB inactive_anon:590196kB active_file:84kB
2.8GB of anonymous memory
> [ 532.942508] Free swap = 0kB
> [ 532.942510] Total swap = 431632kB
430MB of swap, all used up.
That's a genuine OOM. Something (presumably cc1plus) has consumed
waaaay too much memory, quite possibly leaked it.
It would help if the oom-killer were to print some information about
the oom-killed process's memory footprint.
Andrew Morton wrote:
> On Mon, 2 Nov 2009 13:29:29 -0800 Justin Mattock<[email protected]> wrote:
>
>
>> Hello,
>> I'm not sure how to handle this,
>> while compiling firefox-3.6b1.source
>> I get this with the default compiling options,
>> as well as custom:
>>
>> ...
>>
>> active_anon:2360492kB inactive_anon:590196kB active_file:84kB
>>
>
> 2.8GB of anonymous memory
>
>
figured it would be good enough
(I think I have 4gig's total)
>> [ 532.942508] Free swap = 0kB
>> [ 532.942510] Total swap = 431632kB
>>
>
> 430MB of swap, all used up.
>
>
yep, narrow down to the smallest amount.
> That's a genuine OOM. Something (presumably cc1plus) has consumed
> waaaay too much memory, quite possibly leaked it.
>
> It would help if the oom-killer were to print some information about
> the oom-killed process's memory footprint.
>
>
>
I still have everything setup(if you need me to add a debug patch
let me know)
as for compiling: libc compiled fine, kernel fine,
and every package on the clfs list up to boot up the fresh system.
(was figuring out how to compiling/install firefox before
I threw the old system away).
stable gcc(4.4*) on the macbook(same os/kernel) compiled fine
firefox, xulrunner, and in the process thunderbird...
Justin P. Mattock
On 11/04/2009 07:44 AM, Justin P. Mattock wrote:
> as for compiling: libc compiled fine, kernel fine,
> and every package on the clfs list up to boot up the fresh system.
It might be pretty c++ only, I think.
> On Mon, 2 Nov 2009 13:29:29 -0800 Justin Mattock <[email protected]> wrote:
>
> > Hello,
> > I'm not sure how to handle this,
> > while compiling firefox-3.6b1.source
> > I get this with the default compiling options,
> > as well as custom:
> >
> > ...
> >
> > active_anon:2360492kB inactive_anon:590196kB active_file:84kB
>
> 2.8GB of anonymous memory
>
> > [ 532.942508] Free swap = 0kB
> > [ 532.942510] Total swap = 431632kB
>
> 430MB of swap, all used up.
>
> That's a genuine OOM. Something (presumably cc1plus) has consumed
> waaaay too much memory, quite possibly leaked it.
>
> It would help if the oom-killer were to print some information about
> the oom-killed process's memory footprint.
>
How about this?
========
Subject: [PATCH] oom: show vsz and rss information of the killed process
In typical oom anylysis scenario, we frequently want to know the killed
process has memory leak or not at first step.
This patch add vsz and rss information to oom log for helping its
analysis. It save much times of debugging guys.
example:
===================================================================
rsyslogd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0
Pid: 1308, comm: rsyslogd Not tainted 2.6.32-rc6 #24
Call Trace:
[<ffffffff8132e35b>] ?_spin_unlock+0x2b/0x40
[<ffffffff810f186e>] oom_kill_process+0xbe/0x2b0
(snip)
492283 pages non-shared
Out of memory: kill process 2341 (memhog) score 527276 or a child
Killed process 2341 (memhog) vsz:1054552kB, anon-rss:970588kB, file-rss:4kB
===========================================================================
^
|
here
Signed-off-by: KOSAKI Motohiro <[email protected]>
---
mm/oom_kill.c | 15 ++++++++++++---
1 files changed, 12 insertions(+), 3 deletions(-)
diff --git a/mm/oom_kill.c b/mm/oom_kill.c
index ea2147d..498e6f6 100644
--- a/mm/oom_kill.c
+++ b/mm/oom_kill.c
@@ -337,6 +337,8 @@ static void dump_tasks(const struct mem_cgroup *mem)
} while_each_thread(g, p);
}
+#define K(x) ((x) << (PAGE_SHIFT-10))
+
/*
* Send SIGKILL to the selected process irrespective of CAP_SYS_RAW_IO
* flag though it's unlikely that we select a process with CAP_SYS_RAW_IO
@@ -356,9 +358,16 @@ static void __oom_kill_task(struct task_struct *p, int verbose)
return;
}
- if (verbose)
- printk(KERN_ERR "Killed process %d (%s)\n",
- task_pid_nr(p), p->comm);
+ if (verbose) {
+ task_lock(p);
+ printk(KERN_ERR "Killed process %d (%s) "
+ "vsz:%lukB, anon-rss:%lukB, file-rss:%lukB\n",
+ task_pid_nr(p), p->comm,
+ K(p->mm->total_vm),
+ K(get_mm_counter(p->mm, anon_rss)),
+ K(get_mm_counter(p->mm, file_rss)));
+ task_unlock(p);
+ }
/*
* We give our sacrificial lamb high priority and access to
--
1.6.2.5
Andrew Morton wrote:
> On Mon, 2 Nov 2009 13:29:29 -0800 Justin Mattock <[email protected]> wrote:
>
>> Hello,
>> I'm not sure how to handle this,
>> while compiling firefox-3.6b1.source
>> I get this with the default compiling options,
>> as well as custom:
>>
>> ...
>>
>> active_anon:2360492kB inactive_anon:590196kB active_file:84kB
>
> 2.8GB of anonymous memory
>
>> [ 532.942508] Free swap = 0kB
>> [ 532.942510] Total swap = 431632kB
>
> 430MB of swap, all used up.
>
> That's a genuine OOM. Something (presumably cc1plus) has consumed
> waaaay too much memory, quite possibly leaked it.
>
> It would help if the oom-killer were to print some information about
> the oom-killed process's memory footprint.
I would think that the quickest way to proceed would be to re-run the
failing compile command under gdb at the command-line and see what it's doing
when the oom killer signals it, wouldn't it? Or turn up the swap until it
doesn't get killed and see what info can be gleaned from the cc1(plus?)
-fmem-report output.
cheers,
DaveK
Dave Korn wrote:
> Andrew Morton wrote:
>
>> On Mon, 2 Nov 2009 13:29:29 -0800 Justin Mattock<[email protected]> wrote:
>>
>>
>>> Hello,
>>> I'm not sure how to handle this,
>>> while compiling firefox-3.6b1.source
>>> I get this with the default compiling options,
>>> as well as custom:
>>>
>>> ...
>>>
>>> active_anon:2360492kB inactive_anon:590196kB active_file:84kB
>>>
>> 2.8GB of anonymous memory
>>
>>
>>> [ 532.942508] Free swap = 0kB
>>> [ 532.942510] Total swap = 431632kB
>>>
>> 430MB of swap, all used up.
>>
>> That's a genuine OOM. Something (presumably cc1plus) has consumed
>> waaaay too much memory, quite possibly leaked it.
>>
>> It would help if the oom-killer were to print some information about
>> the oom-killed process's memory footprint.
>>
>
> I would think that the quickest way to proceed would be to re-run the
> failing compile command under gdb at the command-line and see what it's doing
> when the oom killer signals it, wouldn't it? Or turn up the swap until it
> doesn't get killed and see what info can be gleaned from the cc1(plus?)
> -fmem-report output.
>
> cheers,
> DaveK
>
>
>
I can try, only issue I have is I don't
use a distro, so building anything requires me
to hand compile it(hopefully not difficult for gdb).
So give me some time on this and I'll see if I can get this up
and running, and add that patch to kernel then go from there.
Justin P. Mattock
On Wed, 4 Nov 2009 18:32:16 +0900 (JST) KOSAKI Motohiro <[email protected]> wrote:
> > It would help if the oom-killer were to print some information about
> > the oom-killed process's memory footprint.
> >
>
>
> How about this?
looks good, thanks.
> ========
> Subject: [PATCH] oom: show vsz and rss information of the killed process
>
> In typical oom anylysis scenario, we frequently want to know the killed
> process has memory leak or not at first step.
> This patch add vsz and rss information to oom log for helping its
> analysis. It save much times of debugging guys.
>
> example:
> ===================================================================
> rsyslogd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0
> Pid: 1308, comm: rsyslogd Not tainted 2.6.32-rc6 #24
> Call Trace:
> [<ffffffff8132e35b>] ?_spin_unlock+0x2b/0x40
> [<ffffffff810f186e>] oom_kill_process+0xbe/0x2b0
>
> (snip)
>
> 492283 pages non-shared
> Out of memory: kill process 2341 (memhog) score 527276 or a child
> Killed process 2341 (memhog) vsz:1054552kB, anon-rss:970588kB, file-rss:4kB
> ===========================================================================
> ^
> |
> here
> ...
>
> + if (verbose) {
> + task_lock(p);
We need to be careful with which locks we take on the oom-killer path,
because it can be called by code which already holds locks. But I
expect task_lock() will be OK.
> + printk(KERN_ERR "Killed process %d (%s) "
> + "vsz:%lukB, anon-rss:%lukB, file-rss:%lukB\n",
> + task_pid_nr(p), p->comm,
> + K(p->mm->total_vm),
> + K(get_mm_counter(p->mm, anon_rss)),
> + K(get_mm_counter(p->mm, file_rss)));
> + task_unlock(p);
> + }
Justin P. Mattock wrote:
> I can try, only issue I have is I don't
> use a distro, so building anything requires me
> to hand compile it
Oh, ouch!
> (hopefully not difficult for gdb).
Indeed, hopefully not.
> So give me some time on this and I'll see if I can get this up
> and running, and add that patch to kernel then go from there.
The one thing you can still try straight away for minimal effort is the
-fmem-report option, but it's also the least informative...
cheers,
DaveK
On Wed, Nov 4, 2009 at 7:45 AM, Dave Korn
<[email protected]> wrote:
> Justin P. Mattock wrote:
>
>> I can try, only issue I have is I don't
>> use a distro, so building anything requires me
>> to hand compile it
>
> ?Oh, ouch!
>
I know.. I'm a horror for optimization
>> (hopefully not difficult for gdb).
>
> ?Indeed, hopefully not.
>
you never know, some packages big/small turn into brain surgery
just to get going.(I'll try after I do some morning exercises)
>> So give me some time on this and I'll see if I can get this up
>> and running, and add that patch to kernel then go from there.
>
> ?The one thing you can still try straight away for minimal effort is the
> -fmem-report option, but it's also the least informative...
>
> ? ?cheers,
> ? ? ?DaveK
>
>
O.k. here is the info from dmesg(with the patch added)
and what -fmem-report:
[ 205.931940] kjournald starting. Commit interval 5 seconds
[ 205.931957] EXT3-fs warning: maximal mount count reached, running
e2fsck is recommended
[ 205.935509] EXT3 FS on sdb1, internal journal
[ 205.935513] EXT3-fs: mounted filesystem with writeback data mode.
[ 205.956396] SELinux: initialized (dev sdb1, type ext3), uses xattr
[ 434.205304] __ratelimit: 75 callbacks suppressed
[ 434.205308] wicd-monitor invoked oom-killer: gfp_mask=0x201da,
order=0, oom_adj=0
[ 434.205313] Pid: 1563, comm: wicd-monitor Tainted: P
2.6.32-rc5-00081-g964fe08-dirty #36
[ 434.205316] Call Trace:
[ 434.205325] [<ffffffff810bc1af>] oom_kill_process+0x7c/0x243
[ 434.205330] [<ffffffff810bc6e0>] __out_of_memory+0x146/0x15d
[ 434.205335] [<ffffffff810bc909>] out_of_memory+0x6e/0x9d
[ 434.205339] [<ffffffff810bf7c0>] __alloc_pages_nodemask+0x498/0x5ce
[ 434.205345] [<ffffffff810c10e8>] __do_page_cache_readahead+0xa0/0x1a1
[ 434.205350] [<ffffffff810c1436>] ra_submit+0x1c/0x20
[ 434.205353] [<ffffffff810ba620>] filemap_fault+0x1a6/0x346
[ 434.205359] [<ffffffff810cf388>] __do_fault+0x4f/0x3d9
[ 434.205363] [<ffffffff810eec2e>] ? do_sync_read+0xe3/0x120
[ 434.205369] [<ffffffff811a2571>] ? file_has_perm+0x90/0x9e
[ 434.205373] [<ffffffff810d1cf7>] handle_mm_fault+0x3ab/0x6a7
[ 434.205379] [<ffffffff813d44a3>] do_page_fault+0x2bb/0x2d3
[ 434.205383] [<ffffffff813d23a5>] page_fault+0x25/0x30
[ 434.205386] Mem-Info:
[ 434.205388] DMA per-cpu:
[ 434.205391] CPU 0: hi: 0, btch: 1 usd: 0
[ 434.205394] CPU 1: hi: 0, btch: 1 usd: 0
[ 434.205396] DMA32 per-cpu:
[ 434.205399] CPU 0: hi: 186, btch: 31 usd: 125
[ 434.205401] CPU 1: hi: 186, btch: 31 usd: 105
[ 434.205404] Normal per-cpu:
[ 434.205406] CPU 0: hi: 186, btch: 31 usd: 172
[ 434.205409] CPU 1: hi: 186, btch: 31 usd: 154
[ 434.205416] active_anon:708764 inactive_anon:266208 isolated_anon:0
[ 434.205417] active_file:71 inactive_file:11 isolated_file:0
[ 434.205419] unevictable:0 dirty:0 writeback:0 unstable:0 buffer:74
[ 434.205420] free:6961 slab_reclaimable:2782 slab_unreclaimable:16224
[ 434.205421] mapped:65 shmem:35 pagetables:2861 bounce:0
[ 434.205430] DMA free:15944kB min:28kB low:32kB high:40kB
active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15360kB
mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB
slab_reclaimable:0kB slab_unreclaimable:8kB kernel_stack:0kB
pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB
pages_scanned:0 all_unreclaimable? yes
[ 434.205438] lowmem_reserve[]: 0 2976 3986 3986
[ 434.205449] DMA32 free:9976kB min:6020kB low:7524kB high:9028kB
active_anon:2360156kB inactive_anon:589924kB active_file:60kB
inactive_file:44kB unevictable:0kB isolated(anon):0kB
isolated(file):0kB present:3047792kB mlocked:0kB dirty:0kB
writeback:0kB mapped:88kB shmem:4kB slab_reclaimable:148kB
slab_unreclaimable:316kB kernel_stack:40kB pagetables:5952kB
unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:225
all_unreclaimable? yes
[ 434.205457] lowmem_reserve[]: 0 0 1010 1010
[ 434.205468] Normal free:1924kB min:2040kB low:2548kB high:3060kB
active_anon:474900kB inactive_anon:474908kB active_file:224kB
inactive_file:0kB unevictable:0kB isolated(anon):0kB
isolated(file):0kB present:1034240kB mlocked:0kB dirty:0kB
writeback:0kB mapped:172kB shmem:136kB slab_reclaimable:10980kB
slab_unreclaimable:64572kB kernel_stack:824kB pagetables:5492kB
unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:677
all_unreclaimable? yes
[ 434.205476] lowmem_reserve[]: 0 0 0 0
[ 434.205481] DMA: 2*4kB 2*8kB 3*16kB 2*32kB 3*64kB 2*128kB 2*256kB
1*512kB 2*1024kB 2*2048kB 2*4096kB = 15944kB
[ 434.205493] DMA32: 2*4kB 14*8kB 16*16kB 10*32kB 3*64kB 1*128kB
1*256kB 1*512kB 2*1024kB 1*2048kB 1*4096kB = 9976kB
[ 434.205505] Normal: 481*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB
0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1924kB
[ 434.205516] 8029 total pagecache pages
[ 434.205519] 7893 pages in swap cache
[ 434.205521] Swap cache stats: add 112490, delete 104597, find 5058/5479
[ 434.205524] Free swap = 0kB
[ 434.205526] Total swap = 431632kB
[ 434.220125] 1048576 pages RAM
[ 434.220127] 40493 pages reserved
[ 434.220129] 170 pages shared
[ 434.220131] 1000179 pages non-shared
[ 434.220135] Out of memory: kill process 7925 (c++) score 539395 or a child
[ 434.220141] Killed process 7926 (cc1plus) vsz:4280180kB,
anon-rss:3831924kB, file-rss:4kB
[ 434.259045] cc1plus: page allocation failure. order:0, mode:0x280da
[ 434.259051] Pid: 7926, comm: cc1plus Tainted: P
2.6.32-rc5-00081-g964fe08-dirty #36
[ 434.259054] Call Trace:
[ 434.259063] [<ffffffff810bf874>] __alloc_pages_nodemask+0x54c/0x5ce
[ 434.259070] [<ffffffff810d1bc3>] handle_mm_fault+0x277/0x6a7
[ 434.259076] [<ffffffff813d44a3>] do_page_fault+0x2bb/0x2d3
[ 434.259080] [<ffffffff813d23a5>] page_fault+0x25/0x30
[ 434.259083] Mem-Info:
[ 434.259085] DMA per-cpu:
[ 434.259088] CPU 0: hi: 0, btch: 1 usd: 0
[ 434.259090] CPU 1: hi: 0, btch: 1 usd: 0
[ 434.259092] DMA32 per-cpu:
[ 434.259095] CPU 0: hi: 186, btch: 31 usd: 125
[ 434.259098] CPU 1: hi: 186, btch: 31 usd: 105
[ 434.259100] Normal per-cpu:
[ 434.259103] CPU 0: hi: 186, btch: 31 usd: 172
[ 434.259106] CPU 1: hi: 186, btch: 31 usd: 154
[ 434.259113] active_anon:708764 inactive_anon:266208 isolated_anon:0
[ 434.259115] active_file:71 inactive_file:11 isolated_file:0
[ 434.259116] unevictable:0 dirty:0 writeback:0 unstable:0 buffer:74
[ 434.259117] free:6961 slab_reclaimable:2782 slab_unreclaimable:16224
[ 434.259119] mapped:65 shmem:35 pagetables:2861 bounce:0
[ 434.259128] DMA free:15944kB min:28kB low:32kB high:40kB
active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15360kB
mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB
slab_reclaimable:0kB slab_unreclaimable:8kB kernel_stack:0kB
pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB
pages_scanned:0 all_unreclaimable? yes
[ 434.259135] lowmem_reserve[]: 0 2976 3986 3986
[ 434.259147] DMA32 free:9976kB min:6020kB low:7524kB high:9028kB
active_anon:2360156kB inactive_anon:589924kB active_file:60kB
inactive_file:44kB unevictable:0kB isolated(anon):0kB
isolated(file):0kB present:3047792kB mlocked:0kB dirty:0kB
writeback:0kB mapped:88kB shmem:4kB slab_reclaimable:148kB
slab_unreclaimable:316kB kernel_stack:40kB pagetables:5952kB
unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:225
all_unreclaimable? yes
[ 434.259154] lowmem_reserve[]: 0 0 1010 1010
[ 434.259166] Normal free:1924kB min:2040kB low:2548kB high:3060kB
active_anon:474900kB inactive_anon:474908kB active_file:224kB
inactive_file:0kB unevictable:0kB isolated(anon):0kB
isolated(file):0kB present:1034240kB mlocked:0kB dirty:0kB
writeback:0kB mapped:172kB shmem:136kB slab_reclaimable:10980kB
slab_unreclaimable:64572kB kernel_stack:824kB pagetables:5492kB
unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:677
all_unreclaimable? yes
[ 434.259173] lowmem_reserve[]: 0 0 0 0
[ 434.259178] DMA: 2*4kB 2*8kB 3*16kB 2*32kB 3*64kB 2*128kB 2*256kB
1*512kB 2*1024kB 2*2048kB 2*4096kB = 15944kB
[ 434.259190] DMA32: 2*4kB 14*8kB 16*16kB 10*32kB 3*64kB 1*128kB
1*256kB 1*512kB 2*1024kB 1*2048kB 1*4096kB = 9976kB
[ 434.259202] Normal: 481*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB
0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1924kB
[ 434.259214] 8029 total pagecache pages
[ 434.259216] 7893 pages in swap cache
[ 434.259219] Swap cache stats: add 112490, delete 104597, find 5058/5480
[ 434.259221] Free swap = 0kB
[ 434.259223] Total swap = 431632kB
[ 434.273830] 1048576 pages RAM
[ 434.273832] 40493 pages reserved
[ 434.273834] 170 pages shared
[ 434.273836] 1000179 pages non-shared
/********** and -fmem-report **************/
Memory still allocated at the end of the compilation process
Size Allocated Used Overhead
8 56k 43k 1680
16 196k 81k 4312
32 188k 40k 3384
64 336k 232k 5376
128 132k 128k 1848
512 28k 13k 392
1024 20k 9216 280
2048 16k 8192 224
4096 348k 348k 4872
8192 56k 56k 392
16384 16k 16k 56
32768 64k 64k 112
24 256k 53k 4608
40 256k 199k 4096
48 800k 579k 12k
56 92k 6608 1472
72 72k 5040 1008
80 8192 1440 112
88 8192 704 112
96 940k 580k 12k
112 80k 39k 1120
120 16k 840 224
192 192k 157k 2688
136 600k 580k 8400
160 200k 172k 2800
176 976k 795k 13k
152 84k 33k 1176
104 120k 28k 1680
256 1040k 1019k 14k
144 4096 144 56
Total 7200k 5293k 104k
String pool
entries 9928
identifiers 6178 (62.23%)
slots 16384
deleted 3726
bytes 86k (17592186044415M overhead)
table size 128k
coll/search 0.3177
ins/search 0.1518
avg. entry 8.92 bytes (+/- 9.51)
longest entry 112
??? tree nodes created
(No per-node statistics)
Type hash: size 4093, 2550 elements, 1.102267 collisions
DECL_DEBUG_EXPR hash: size 1021, 0 elements, 0.015692 collisions
DECL_VALUE_EXPR hash: size 1021, 0 elements, 0.000000 collisions
no search statistics
No gimple statistics
Alias oracle query stats:
refs_may_alias_p: 11 disambiguations, 29 queries
ref_maybe_used_by_call_p: 0 disambiguations, 62 queries
call_may_clobber_ref_p: 0 disambiguations, 0 queries
PTA query stats:
pt_solution_includes: 7 disambiguations, 166 queries
pt_solutions_intersect: 0 disambiguations, 507 queries
jsxml.cpp
c++ -o jsxml.o -c -DOSTYPE=\"Linux2.6\" -DOSARCH=Linux
-DEXPORT_JS_API -DJS_USE_SAFE_ARENA
-I/home/name/LFS/firefox/mozilla-1.9.2/js/src -I.
-I./../../dist/include -I./../../dist/include/nsprpub
-I/usr/include/nspr
-I/home/name/LFS/firefox/mozilla-1.9.2/js/src -fPIC -fno-rtti
-fno-exceptions -Wall -Wpointer-arith -Woverloaded-virtual
-Wsynth -Wno-ctor-dtor-privacy -Wno-non-virtual-dtor -Wcast-align
-Wno-invalid-offsetof -Wno-variadic-macros -Wno-long-long
-pedantic -fno-strict-aliasing -pthread -pipe -DNDEBUG -DTRIMMED -m64
-mtune=core2 -march=core2 -O2 -pipe -fmem-report
-fomit-frame-pointer -DMOZILLA_CLIENT -include ./js-confdefs.h
-Wp,-MD,.deps/jsxml.pp
/home/name/LFS/firefox/mozilla-1.9.2/js/src/jsxml.cpp
{standard input}: Assembler messages:
{standard input}:271839: Warning: end of file not at end of a line;
newline inserted
{standard input}:271896: Error: suffix or operands invalid for `movq'
{standard input}:271896: Error: open CFI at the end of file; missing
.cfi_endproc directive
c++: Internal error: Killed (program cc1plus)
Please submit a full bug report.
See <http://gcc.gnu.org/bugs.html> for instructions.
make[4]: *** [jsxml.o] Error 1
make[4]: Leaving directory
`/home/name/LFS/firefox/mozilla-1.9.2/obj-x86_64-unknown-linux-gnu/js/src'
make[3]: *** [libs_tier_js] Error 2
make[3]: Leaving directory
`/home/name/LFS/firefox/mozilla-1.9.2/obj-x86_64-unknown-linux-gnu'
make[2]: *** [tier_js] Error 2
make[2]: Leaving directory
`/home/name/LFS/firefox/mozilla-1.9.2/obj-x86_64-unknown-linux-gnu'
make[1]: *** [default] Error 2
make[1]: Leaving directory
`/home/name/LFS/firefox/mozilla-1.9.2/obj-x86_64-unknown-linux-gnu'
make: *** [build] Error 2
--
Justin P. Mattock
Justin Mattock wrote:
> O.k. here is the info from dmesg(with the patch added)
> and what -fmem-report:
I don't know how to read the oom dmesg, but as to the -fmem-report:
> Memory still allocated at the end of the compilation process
> Size Allocated Used Overhead
> Total 7200k 5293k 104k
... what that's telling us is that there isn't a substantial leak in GCC, as
there's only 7 meg left unreclaimed by GC at the end. I think we'll have to
wait and see what the debugger tells us; either GCC really is using that much
memory in processing the file, or there's some kind of system or kernel bug
you're running into that is causing a leak in the VMM rather than the application.
> String pool
> bytes 86k (17592186044415M overhead)
0xFFFFFFFFFFF00000, lol, wut? It's possible that indicates some sort of
memory corruption going on. Maybe valgrind can help, do you have that?
cheers,
DaveK
Dave Korn wrote:
> Justin Mattock wrote:
>
>
>> O.k. here is the info from dmesg(with the patch added)
>> and what -fmem-report:
>>
>
> I don't know how to read the oom dmesg, but as to the -fmem-report:
>
>
>> Memory still allocated at the end of the compilation process
>> Size Allocated Used Overhead
>> Total 7200k 5293k 104k
>>
>
> ... what that's telling us is that there isn't a substantial leak in GCC, as
> there's only 7 meg left unreclaimed by GC at the end. I think we'll have to
> wait and see what the debugger tells us; either GCC really is using that much
> memory in processing the file, or there's some kind of system or kernel bug
> you're running into that is causing a leak in the VMM rather than the application.
>
>
just finished compiling and installing gdb/valgrind
>> String pool
>>
>
>
>> bytes 86k (17592186044415M overhead)
>>
>
> 0xFFFFFFFFFFF00000, lol, wut? It's possible that indicates some sort of
> memory corruption going on. Maybe valgrind can help, do you have that?
>
> cheers,
> DaveK
>
>
>
Not sure how to use these.(need to read)
Any quick commands I can do to get the info
to you?
Justin P. Mattock
here's what I did:
valgrind --tool=memcheck --leak-check=full -v make -f client.mk build
e2 -march=core2 -O2 -pipe -fomit-frame-pointer -DMOZILLA_CLIENT
-include ./js-confdefs.h -Wp,-MD,.deps/jsxml.pp
/home/justin/LFS/firefox/mozilla-1.9.2/js/src/jsxml.cpp
{standard input}: Assembler messages:
{standard input}:271839: Warning: end of file not at end of a line;
newline inserted
{standard input}:271896: Error: suffix or operands invalid for `movq'
{standard input}:271896: Error: open CFI at the end of file; missing
.cfi_endproc directive
c++: Internal error: Killed (program cc1plus)
Please submit a full bug report.
See <http://gcc.gnu.org/bugs.html> for instructions.
make[4]: *** [jsxml.o] Error 1
make[4]: Leaving directory
`/home/name/LFS/firefox/mozilla-1.9.2/obj-x86_64-unknown-linux-gnu/js/src'
make[3]: *** [libs_tier_js] Error 2
make[3]: Leaving directory
`/home/name/LFS/firefox/mozilla-1.9.2/obj-x86_64-unknown-linux-gnu'
make[2]: *** [tier_js] Error 2
make[2]: Leaving directory
`/home/name/LFS/firefox/mozilla-1.9.2/obj-x86_64-unknown-linux-gnu'
make[1]: *** [default] Error 2
make[1]: Leaving directory
`/home/name/LFS/firefox/mozilla-1.9.2/obj-x86_64-unknown-linux-gnu'
make: *** [build] Error 2
==4072==
==4072== HEAP SUMMARY:
==4072== in use at exit: 201,183 bytes in 4,237 blocks
==4072== total heap usage: 28,879 allocs, 24,642 frees, 2,947,434
bytes allocated
==4072==
==4072== Searching for pointers to 4,237 not-freed blocks
==4072== Checked 268,808 bytes
==4072==
==4072== 6 bytes in 1 blocks are possibly lost in loss record 41 of 295
==4072== at 0x4C2488A: malloc (vg_replace_malloc.c:195)
==4072== by 0x50ABEE1: strdup (in /lib/libc-2.10.90.so)
==4072== by 0x4118D8: xstrdup (in /usr/bin/make)
==4072== by 0x41BAA4: define_variable_in_set (in /usr/bin/make)
==4072== by 0x4160B4: eval (in /usr/bin/make)
==4072== by 0x416766: eval_makefile (in /usr/bin/make)
==4072== by 0x416A3F: read_all_makefiles (in /usr/bin/make)
==4072== by 0x410853: main (in /usr/bin/make)
==4072==
==4072== 14 bytes in 1 blocks are possibly lost in loss record 69 of 295
==4072== at 0x4C2488A: malloc (vg_replace_malloc.c:195)
==4072== by 0x411977: xmalloc (in /usr/bin/make)
==4072== by 0x411AA8: savestring (in /usr/bin/make)
==4072== by 0x41BB7A: define_variable_in_set (in /usr/bin/make)
==4072== by 0x410832: main (in /usr/bin/make)
==4072==
==4072== LEAK SUMMARY:
==4072== definitely lost: 0 bytes in 0 blocks
==4072== indirectly lost: 0 bytes in 0 blocks
==4072== possibly lost: 20 bytes in 2 blocks
==4072== still reachable: 201,163 bytes in 4,235 blocks
==4072== suppressed: 0 bytes in 0 blocks
==4072== Reachable blocks (those to which a pointer was found) are not shown.
==4072== To see them, rerun with: --leak-check=full --show-reachable=yes
==4072==
==4072== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 5 from 5)
--4072--
--4072-- used_suppression: 2 dl-hack3-cond-1
--4072-- used_suppression: 3 glibc-2.5.x-on-SUSE-10.2-(PPC)-2a
==4072==
==4072== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 5 from 5)
I'll try out gdb, and more of valgrind.
--
Justin P. Mattock
> > + if (verbose) {
> > + task_lock(p);
>
> We need to be careful with which locks we take on the oom-killer path,
> because it can be called by code which already holds locks. But I
> expect task_lock() will be OK.
Sure.
task_lock() is already used various oom path. I think this is ok.
Justin Mattock wrote:
> here's what I did:
> valgrind --tool=memcheck --leak-check=full -v make -f client.mk build
> ==4072== LEAK SUMMARY:
> I'll try out gdb, and more of valgrind.
Yep, that doesn't tell us a lot in its default modes. I'm not a valgrind
expert but it looks from the docs like you want to try the Massif tool: it
looks really thorough.
http://valgrind.org/docs/manual/ms-manual.html
cheers,
DaveK
On Wed, Nov 4, 2009 at 4:36 PM, Dave Korn
<[email protected]> wrote:
> Justin Mattock wrote:
>> here's what I did:
>> ?valgrind --tool=memcheck --leak-check=full -v make -f client.mk build
>
>> ==4072== LEAK SUMMARY:
>
>> I'll try out gdb, and more of valgrind.
>
> ?Yep, that doesn't tell us a lot in its default modes. ?I'm not a valgrind
> expert but it looks from the docs like you want to try the Massif tool: it
> looks really thorough.
>
> http://valgrind.org/docs/manual/ms-manual.html
>
> ? ?cheers,
> ? ? ?DaveK
>
>
>
o.k. round2 I can try gdb(now that I have somewhat of an idea of
how to use one of these tools).
in the meanwhile hopefully this is useful(if not just clip, and I'll try
to gather something useful)
using this for valgrind and the command(below in log)
valgrind -v --tool=memcheck --leak-check=yes --num-callers=10
--leak-check=full --show-reachable=yes --gen-suppressions=yes
(log of valgrind is a bit long, so here's just a few of what
suppresions printed out:)
==1830== Memcheck, a memory error detector
==1830== Copyright (C) 2002-2009, and GNU GPL'd, by Julian Seward et al.
==1830== Using Valgrind-3.5.0 and LibVEX; rerun with -h for copyright info
==1830== Command: c++ -o jsxml.o -c -DOSTYPE="Linux2.6" -DOSARCH=Linux
-DEXPORT_JS_API -DJS_USE_SAFE_ARENA
-I/home/justin/LFS/firefox/mozilla-1.9.2/js/src -I.
-I./../../dist/include -I./../../dist/include/nsprpub
-I/usr/include/nspr -I/home/justin/LFS/firefox/mozilla-1.9.2/js/src
-fPIC -fno-rtti -fno-exceptions -Wall -Wpointer-arith
-Woverloaded-virtual -Wsynth -Wno-ctor-dtor-privacy
-Wno-non-virtual-dtor -Wcast-align -Wno-invalid-offsetof
-Wno-variadic-macros -Wno-long-long -pedantic -fno-strict-aliasing
-pthread -pipe -DNDEBUG -DTRIMMED -m64 -mtune=core2 -march=core2 -O2
-pipe -fomit-frame-pointer -DMOZILLA_CLIENT -include ./js-confdefs.h
-Wp,-MD,.deps/jsxml.pp
/home/justin/LFS/firefox/mozilla-1.9.2/js/src/jsxml.cpp
==1830==
--1830-- Valgrind options:
--1830-- -v
--1830-- --tool=memcheck
--1830-- --leak-check=yes
--1830-- --num-callers=10
--1830-- --leak-check=full
--1830-- --show-reachable=yes
--1830-- --gen-suppressions=yes
--1830-- Contents of /proc/version:
--1830-- Linux version 2.6.32-rc5-00083-g04ea458 (justin@Linux-0)
(gcc version 4.4.1 (GCC for Cross-LFS 4.4.1.20090722) ) #2 SMP Sat Oct
24 21:38:54 PDT 2009
--1830-- Arch and hwcaps: AMD64, amd64-sse3-cx16
--1830-- Page sizes: currently 4096, max supported 4096
--1830-- Valgrind library directory: /usr/lib/valgrind
--1830-- Reading syms from /usr/bin/c++ (0x400000)
--1830-- Reading syms from /lib/ld-2.10.90.so (0x4000000)
--1830-- Reading syms from /usr/lib/valgrind/memcheck-amd64-linux (0x38000000)
--1830-- object doesn't have a dynamic symbol table
--1830-- Reading suppressions file: /usr/lib/valgrind/default.supp
--1830-- REDIR: 0x4016110 (strlen) redirected to 0x38040607
(vgPlain_amd64_linux_REDIR_FOR_strlen)
--1830-- Reading syms from
/usr/lib/valgrind/vgpreload_core-amd64-linux.so (0x4a20000)
--1830-- Reading syms from
/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so (0x4c21000)
==1830== WARNING: new redirection conflicts with existing -- ignoring it
--1830-- new: 0x04016110 (strlen ) R-> 0x04c24fd0 strlen
--1830-- REDIR: 0x4015f30 (index) redirected to 0x4c24d50 (index)
--1830-- REDIR: 0x40160e0 (strcmp) redirected to 0x4c252e0 (strcmp)
--1830-- Reading syms from /lib/libc-2.10.90.so (0x4e28000)
--1830-- REDIR: 0x4ea4c40 (rindex) redirected to 0x4c24bb0 (rindex)
--1830-- REDIR: 0x4e9e010 (malloc) redirected to 0x4c24808 (malloc)
--1830-- REDIR: 0x4ea3330 (strncmp) redirected to 0x4c25210 (strncmp)
--1830-- REDIR: 0x4ea7670 (memcpy) redirected to 0x4c253e0 (memcpy)
--1830-- REDIR: 0x4e9df30 (free) redirected to 0x4c23b44 (free)
--1830-- REDIR: 0x4ea17d0 (strcmp) redirected to 0x4c25280 (strcmp)
--1830-- REDIR: 0x4ea3170 (strlen) redirected to 0x4c24f90 (strlen)
--1830-- REDIR: 0x4ea1750 (index) redirected to 0x4c24c50 (index)
--1830-- REDIR: 0x4ea9da0 (strchrnul) redirected to 0x4c25ff0 (strchrnul)
--1830-- REDIR: 0x4ea6d30 (mempcpy) redirected to 0x4c260e0 (mempcpy)
--1830-- REDIR: 0x4ea5bc0 (memchr) redirected to 0x4c253a0 (memchr)
--1830-- REDIR: 0x4e9f060 (realloc) redirected to 0x4c248b9 (realloc)
--1830-- REDIR: 0x4ea31c0 (strnlen) redirected to 0x4c24f60 (strnlen)
--1830-- REDIR: 0x4ea7360 (stpcpy) redirected to 0x4c25c10 (stpcpy)
--1830-- REDIR: 0x4ea2c20 (strcpy) redirected to 0x4c24ff0 (strcpy)
--1830-- REDIR: 0x4ea9d50 (rawmemchr) redirected to 0x4c26020 (rawmemchr)
--1830-- REDIR: 0x4e9d5d0 (calloc) redirected to 0x4c2322c (calloc)
--1830-- REDIR: 0x4ea4b80 (strncpy) redirected to 0x4c250c0 (strncpy)
--1830-- REDIR: 0x4ea1590 (strcat) redirected to 0x4c24d90 (strcat)
--1830-- REDIR: 0x4e5bf40 (putenv) redirected to 0x4c263d0 (putenv)
{standard input}: Assembler messages:
{standard input}:271839: Warning: end of file not at end of a line;
newline inserted
{standard input}:271896: Error: suffix or operands invalid for `movq'
{standard input}:271896: Error: open CFI at the end of file; missing
.cfi_endproc directive
c++: Internal error: Killed (program cc1plus)
Please submit a full bug report.
See <http://gcc.gnu.org/bugs.html> for instructions.
==1830==
==1830== HEAP SUMMARY:
==1830== in use at exit: 25,444 bytes in 75 blocks
==1830== total heap usage: 212 allocs, 137 frees, 36,791 bytes allocated
==1830==
==1830== Searching for pointers to 75 not-freed blocks
==1830== Checked 74,760 bytes
==1830==
==1830== 1 bytes in 1 blocks are still reachable in loss record 1 of 59
==1830== at 0x4C232DA: calloc (vg_replace_malloc.c:418)
==1830== by 0x41BE49: xcalloc (xmalloc.c:162)
==1830== by 0x40F5D6: main (gcc.c:7270)
==1830==
==1830==
==1830== ---- Print suppression ? --- [Return/N/n/Y/y/C/c] ---- y
{
<insert_a_suppression_name_here>
Memcheck:Leak
fun:calloc
fun:xcalloc
fun:main
}
==1830== 4 bytes in 1 blocks are still reachable in loss record 2 of 59
==1830== at 0x4C2488A: malloc (vg_replace_malloc.c:195)
==1830== by 0x41BDF7: xmalloc (xmalloc.c:147)
==1830== by 0x402255: save_string (gcc.c:7660)
==1830== by 0x4034ED: add_preprocessor_option (gcc.c:3447)
==1830== by 0x408A06: process_command (gcc.c:3943)
==1830== by 0x40DE6F: main (gcc.c:6871)
==1830==
==1830==
==1830== ---- Print suppression ? --- [Return/N/n/Y/y/C/c] ---- y
{
<insert_a_suppression_name_here>
Memcheck:Leak
fun:malloc
fun:xmalloc
fun:save_string
fun:add_preprocessor_option
fun:process_command
fun:main
}
==1830== 6 bytes in 1 blocks are still reachable in loss record 3 of 59
==1830== at 0x4C2488A: malloc (vg_replace_malloc.c:195)
==1830== by 0x41BDF7: xmalloc (xmalloc.c:147)
==1830== by 0x41BF11: xstrdup (xstrdup.c:34)
==1830== by 0x411F75: update_path (prefix.c:273)
==1830== by 0x403631: add_prefix (gcc.c:2871)
==1830== by 0x40E1CF: main (gcc.c:7043)
==1830==
==1830==
==1830== ---- Print suppression ? --- [Return/N/n/Y/y/C/c] ---- y
{
<insert_a_suppression_name_here>
Memcheck:Leak
fun:malloc
fun:xmalloc
fun:xstrdup
fun:update_path
fun:add_prefix
fun:main
}
==1830== 7 bytes in 1 blocks are still reachable in loss record 4 of 59
==1830== at 0x4C2488A: malloc (vg_replace_malloc.c:195)
==1830== by 0x41BDF7: xmalloc (xmalloc.c:147)
==1830== by 0x40E64E: main (gcc.c:8236)
==1830==
==1830==
==1830== ---- Print suppression ? --- [Return/N/n/Y/y/C/c] ---- y
{
<insert_a_suppression_name_here>
Memcheck:Leak
fun:malloc
fun:xmalloc
fun:main
}
all I have to do is go into the directory
obj-x86_64-unknown-linux-gnu/js/src/
of mozillas source tree and use valgrind c++ ***
(I'll see what gdb does)
--
Justin P. Mattock
Justin Mattock wrote:
> ==1830== Command: c++ -o jsxml.o -c -DOSTYPE="Linux2.6" -DOSARCH=Linux
Ah, you're running it on the "c++" utility and it's reporting the stats for
that, but how it works is that "c++" (and "gcc", "g++", et al) is just a
driver, that parses the command line arguments and shells out to the actual
compiler ("cc1plus"), assembler and linker to get them to do all the work.
If you add "-v --save-temps" to the c++ invocation, it'll show you the
separate command lines it executes for the subprograms; the first invocation
will be of cc1plus, using the -E flag to generate the preprocessed source into
a .ii file, it's the second invocation you want, the one which uses the
"-fpreprocessed" flag and names the .ii file as input, which is the one that
actually then compiles the pre-processed source into assembly. For fuller
explanation, see the GCC wiki:
http://gcc.gnu.org/wiki/DebuggingGCC
cheers,
DaveK
Dave Korn wrote:
> Justin Mattock wrote:
>
>
>
>> ==1830== Command: c++ -o jsxml.o -c -DOSTYPE="Linux2.6" -DOSARCH=Linux
>>
>
> Ah, you're running it on the "c++" utility and it's reporting the stats for
> that, but how it works is that "c++" (and "gcc", "g++", et al) is just a
> driver, that parses the command line arguments and shells out to the actual
> compiler ("cc1plus"), assembler and linker to get them to do all the work.
>
> If you add "-v --save-temps" to the c++ invocation, it'll show you the
> separate command lines it executes for the subprograms; the first invocation
> will be of cc1plus, using the -E flag to generate the preprocessed source into
> a .ii file, it's the second invocation you want, the one which uses the
> "-fpreprocessed" flag and names the .ii file as input, which is the one that
> actually then compiles the pre-processed source into assembly. For fuller
> explanation, see the GCC wiki:
>
> http://gcc.gnu.org/wiki/DebuggingGCC
>
> cheers,
> DaveK
>
>
>
>
o.k. after some reading and some experimenting,
I finally got firefox to compile all the way without a
memory leak..
here's what I found:
using the default .mozconfig I see during compiling there
is a -O3 in the CFLAGS(mozilla default optimization)
When using my optimization I use -O2
by mistake I accidentally forgot to add the -O2 while
tweaking my .mozconfig for a right CFLAGS to use for
debugging, in doing so firefox compiled all the way.
So using a custom CFLAGS in mozconfig and leaving out
-O* works..
Any ideas what might be happening here?
Justin P. Mattock
Dave Korn wrote:
> Justin Mattock wrote:
>
>
>
>> ==1830== Command: c++ -o jsxml.o -c -DOSTYPE="Linux2.6" -DOSARCH=Linux
>>
>
> Ah, you're running it on the "c++" utility and it's reporting the stats for
> that, but how it works is that "c++" (and "gcc", "g++", et al) is just a
> driver, that parses the command line arguments and shells out to the actual
> compiler ("cc1plus"), assembler and linker to get them to do all the work.
>
> If you add "-v --save-temps" to the c++ invocation, it'll show you the
> separate command lines it executes for the subprograms; the first invocation
> will be of cc1plus, using the -E flag to generate the preprocessed source into
> a .ii file, it's the second invocation you want, the one which uses the
> "-fpreprocessed" flag and names the .ii file as input, which is the one that
> actually then compiles the pre-processed source into assembly. For fuller
> explanation, see the GCC wiki:
>
> http://gcc.gnu.org/wiki/DebuggingGCC
>
> cheers,
> DaveK
>
>
>
>
I didn't think at the time, but when compiling
4.5.*(snapshot) I added the pure64 patch to
gcc, everything seemed to go fine, but maybe
there needs to be more to it.
would this might cause an issue(memory leak or something)
like what I was receiving even though there's a symlink
to lib64?
BTW: I just cleared the deck and started fresh,
compiled gcc+pure64 patch(snapshot) into a single directory,
and then looked at ldd /usr/bin/gcc*
I see it pointing to /lib64(looking on my other system with
4.4.1 it points to /lib)
Justin P. Mattock