2010-01-21 11:10:16

by Luca Barbieri

Subject: perf record -g hangs the system

I'm experiencing a system lockup running the following command:
perf record -g true

The system hangs and does not respond to any sysrq.
No messages are printed to netconsole.

This is with 2.6.33-rc4 on an x86-32 Core 2 dual core machine.

Any tips on how to debug/fix this?

2010-01-21 11:18:58

by Frederic Weisbecker

Subject: Re: perf record -g hangs the system

On Thu, Jan 21, 2010 at 12:10:12PM +0100, Luca Barbieri wrote:
> I'm experiencing a system lockup running the following command:
> perf record -g true
>
> The system hangs and does not respond to any sysrq.
> No messages are printed to netconsole.
>
> This is with 2.6.33-rc4 on an x86-32 Core 2 dual core machine.
>
> Any tips on how to debug/fix this?



It should be fixed in latest linus's git tree with the
following patch:


commit c2c5d45d46c8c0fd34291dec958670ad4816796f
Author: Frederic Weisbecker <[email protected]>
Date: Thu Dec 31 03:52:25 2009 +0100

perf: Stop stack frame walking off kernel addresses boundaries

While processing kernel perf callchains, an bad entry can be
considered as a valid stack pointer but not as a kernel address.

In this case, we hang in an endless loop. This can happen in an
x86-32 kernel after processing the last entry in a kernel
stacktrace.

Just stop the stack frame walking after we encounter an invalid
kernel address.

This fixes a hard lockup in x86-32.

Signed-off-by: Frederic Weisbecker <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Arnaldo Carvalho de Melo <[email protected]>
Cc: Paul Mackerras <[email protected]>
LKML-Reference: <[email protected]>
Signed-off-by: Ingo Molnar <[email protected]>

diff --git a/arch/x86/kernel/dumpstack.c b/arch/x86/kernel/dumpstack.c
index c56bc28..6d81755 100644
--- a/arch/x86/kernel/dumpstack.c
+++ b/arch/x86/kernel/dumpstack.c
@@ -123,13 +123,15 @@ print_context_stack_bp(struct thread_info *tinfo,
while (valid_stack_ptr(tinfo, ret_addr, sizeof(*ret_addr), end)) {
unsigned long addr = *ret_addr;

- if (__kernel_text_address(addr)) {
- ops->address(data, addr, 1);
- frame = frame->next_frame;
- ret_addr = &frame->return_address;
- print_ftrace_graph_addr(addr, data, ops, tinfo, graph);
- }
+ if (!__kernel_text_address(addr))
+ break;
+
+ ops->address(data, addr, 1);
+ frame = frame->next_frame;
+ ret_addr = &frame->return_address;
+ print_ftrace_graph_addr(addr, data, ops, tinfo, graph);
}
+
return (unsigned long)frame;
}
EXPORT_SYMBOL_GPL(print_context_stack_bp);

2010-01-21 12:19:23

by Luca Barbieri

Subject: Re: perf record -g hangs the system

Thanks.
Switching to Linus master fixed that issue, and that commit was almost
surely responsible for that.