When tracing with tracepoints events the IP and CPSR are set to 0,
preventing the perf code to resolve the symbols:
./perf record -e kmem:kmalloc cal
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.007 MB perf.data (~321 samples) ]
./perf report
Overhead Command Shared Object Symbol
........ ....... ............. ...........
40.78% cal [unknown] [.]00000000
31.6% cal [unknown] [.]00000000
The examination of the gathered samples (perf report -D) shows the IP
is set to 0 and that the samples are considered as user space samples,
while the IP should be set from the registers and the samples should be
considered as kernel samples.
The fix is to implement perf_arch_fetch_caller_regs for ARM, which
fills the necessary registers used for the callchain unwinding and
to determine the user/kernel space property of the samples: ip, sp, fp
and cpsr.
Tested with perf record and tracepoints filtering (-e <tracepoint>), with
unwinding using fp (--call-graph fp) and dwarf info (--call-graph dwarf).
Reported by Sneha Priya on linaro-dev, cf.
http://lists.linaro.org/pipermail/linaro-dev/2014-May/017151.html
Signed-off-by: Jean Pihet <[email protected]>
Cc: Will Deacon <[email protected]>
Reported-by: Sneha Priya <[email protected]>
---
arch/arm/include/asm/perf_event.h | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/arch/arm/include/asm/perf_event.h b/arch/arm/include/asm/perf_event.h
index 7558775..5e31d46 100644
--- a/arch/arm/include/asm/perf_event.h
+++ b/arch/arm/include/asm/perf_event.h
@@ -26,6 +26,25 @@ struct pt_regs;
extern unsigned long perf_instruction_pointer(struct pt_regs *regs);
extern unsigned long perf_misc_flags(struct pt_regs *regs);
#define perf_misc_flags(regs) perf_misc_flags(regs)
+
+/*
+ * Take a snapshot of the regs.
+ * We only need a few of the regs:
+ * - ip for PERF_SAMPLE_IP
+ * - sp, fp for callchains
+ * - cpsr for user_mode() tests
+ */
+#define perf_arch_fetch_caller_regs(regs, __ip) { \
+ instruction_pointer(regs)= (__ip); \
+ __asm__ ( \
+ "mov %[_ARM_sp], sp \n\t" \
+ "mov %[_ARM_fp], fp \n\t" \
+ "mrs %[_ARM_cpsr], cpsr \n\t" \
+ : [_ARM_sp] "=r" (regs->ARM_sp), \
+ [_ARM_fp] "=r" (regs->ARM_fp), \
+ [_ARM_cpsr] "=r" (regs->ARM_cpsr) \
+ ); \
+}
#endif
#endif /* __ARM_PERF_EVENT_H__ */
--
1.8.1.2
Hi Jean,
On Tue, Jun 17, 2014 at 06:11:05PM +0100, Jean Pihet wrote:
> When tracing with tracepoints events the IP and CPSR are set to 0,
> preventing the perf code to resolve the symbols:
>
> ./perf record -e kmem:kmalloc cal
> [ perf record: Woken up 1 times to write data ]
> [ perf record: Captured and wrote 0.007 MB perf.data (~321 samples) ]
>
> ./perf report
> Overhead Command Shared Object Symbol
> ........ ....... ............. ...........
> 40.78% cal [unknown] [.]00000000
> 31.6% cal [unknown] [.]00000000
>
> The examination of the gathered samples (perf report -D) shows the IP
> is set to 0 and that the samples are considered as user space samples,
> while the IP should be set from the registers and the samples should be
> considered as kernel samples.
>
> The fix is to implement perf_arch_fetch_caller_regs for ARM, which
> fills the necessary registers used for the callchain unwinding and
> to determine the user/kernel space property of the samples: ip, sp, fp
> and cpsr.
Surely its only the CPSR that identifies whether it's user or kernel?
> Tested with perf record and tracepoints filtering (-e <tracepoint>), with
> unwinding using fp (--call-graph fp) and dwarf info (--call-graph dwarf).
Whilst the old ACPS unwinding only needs PC, FP and SP, is this definitely
true for exidx and DWARF-based unwinding? Given that libunwind ends up
running a state machine for the latter, can we guarantee that we won't hit
instructions that require access to other general purpose registers?
Will
Hi Will,
On 18 June 2014 14:53, Will Deacon <[email protected]> wrote:
> Hi Jean,
>
> On Tue, Jun 17, 2014 at 06:11:05PM +0100, Jean Pihet wrote:
>> When tracing with tracepoints events the IP and CPSR are set to 0,
>> preventing the perf code to resolve the symbols:
>>
>> ./perf record -e kmem:kmalloc cal
>> [ perf record: Woken up 1 times to write data ]
>> [ perf record: Captured and wrote 0.007 MB perf.data (~321 samples) ]
>>
>> ./perf report
>> Overhead Command Shared Object Symbol
>> ........ ....... ............. ...........
>> 40.78% cal [unknown] [.]00000000
>> 31.6% cal [unknown] [.]00000000
>>
>> The examination of the gathered samples (perf report -D) shows the IP
>> is set to 0 and that the samples are considered as user space samples,
>> while the IP should be set from the registers and the samples should be
>> considered as kernel samples.
>>
>> The fix is to implement perf_arch_fetch_caller_regs for ARM, which
>> fills the necessary registers used for the callchain unwinding and
>> to determine the user/kernel space property of the samples: ip, sp, fp
>> and cpsr.
>
> Surely its only the CPSR that identifies whether it's user or kernel?
Yes, user_mode() is used to determine the user/kernel property of the
samples. user_mode is defined as (((regs)->ARM_cpsr & 0xf) == 0) in
ptrace.h.
>
>> Tested with perf record and tracepoints filtering (-e <tracepoint>), with
>> unwinding using fp (--call-graph fp) and dwarf info (--call-graph dwarf).
>
> Whilst the old ACPS unwinding only needs PC, FP and SP, is this definitely
> true for exidx and DWARF-based unwinding? Given that libunwind ends up
> running a state machine for the latter, can we guarantee that we won't hit
> instructions that require access to other general purpose registers?
Yes. dwarf unwinding does not need anything extra. Once seeded all the
rest is extracted from the dwarf trace info.
I am currently stress testing the change, let me come back to you with
the results.
Thx,
Jean
>
> Will
On Fri, Jun 20, 2014 at 09:10:35AM +0100, Jean Pihet wrote:
> Hi Will,
Hi Jean,
> On 18 June 2014 14:53, Will Deacon <[email protected]> wrote:
> > On Tue, Jun 17, 2014 at 06:11:05PM +0100, Jean Pihet wrote:
> >> Tested with perf record and tracepoints filtering (-e <tracepoint>), with
> >> unwinding using fp (--call-graph fp) and dwarf info (--call-graph dwarf).
> >
> > Whilst the old ACPS unwinding only needs PC, FP and SP, is this definitely
> > true for exidx and DWARF-based unwinding? Given that libunwind ends up
> > running a state machine for the latter, can we guarantee that we won't hit
> > instructions that require access to other general purpose registers?
> Yes. dwarf unwinding does not need anything extra. Once seeded all the
> rest is extracted from the dwarf trace info.
Ok, but what if the LR isn't saved on the stack, for example? What if the
code you're trying to unwind is hand-written assembly annotated with CFI
directives?
Will
Hi Will,
On 25 June 2014 11:01, Will Deacon <[email protected]> wrote:
> On Fri, Jun 20, 2014 at 09:10:35AM +0100, Jean Pihet wrote:
>> Hi Will,
>
> Hi Jean,
>
>> On 18 June 2014 14:53, Will Deacon <[email protected]> wrote:
>> > On Tue, Jun 17, 2014 at 06:11:05PM +0100, Jean Pihet wrote:
>> >> Tested with perf record and tracepoints filtering (-e <tracepoint>), with
>> >> unwinding using fp (--call-graph fp) and dwarf info (--call-graph dwarf).
>> >
>> > Whilst the old ACPS unwinding only needs PC, FP and SP, is this definitely
>> > true for exidx and DWARF-based unwinding? Given that libunwind ends up
>> > running a state machine for the latter, can we guarantee that we won't hit
>> > instructions that require access to other general purpose registers?
>> Yes. dwarf unwinding does not need anything extra. Once seeded all the
>> rest is extracted from the dwarf trace info.
>
> Ok, but what if the LR isn't saved on the stack, for example? What if the
> code you're trying to unwind is hand-written assembly annotated with CFI
> directives?
Then in that case the unwinding is not possible unless the
hand-crafted asm is compatible with the requested unwinding method
(fp, dwarf etc.). Do you expect problems there, if so can you give
more details?
>
> Will
Jean
On Wed, Jun 25, 2014 at 03:54:14PM +0100, Jean Pihet wrote:
> Hi Will,
Hello,
> On 25 June 2014 11:01, Will Deacon <[email protected]> wrote:
> > On Fri, Jun 20, 2014 at 09:10:35AM +0100, Jean Pihet wrote:
> >> On 18 June 2014 14:53, Will Deacon <[email protected]> wrote:
> >> > On Tue, Jun 17, 2014 at 06:11:05PM +0100, Jean Pihet wrote:
> >> >> Tested with perf record and tracepoints filtering (-e <tracepoint>), with
> >> >> unwinding using fp (--call-graph fp) and dwarf info (--call-graph dwarf).
> >> >
> >> > Whilst the old ACPS unwinding only needs PC, FP and SP, is this definitely
> >> > true for exidx and DWARF-based unwinding? Given that libunwind ends up
> >> > running a state machine for the latter, can we guarantee that we won't hit
> >> > instructions that require access to other general purpose registers?
> >> Yes. dwarf unwinding does not need anything extra. Once seeded all the
> >> rest is extracted from the dwarf trace info.
> >
> > Ok, but what if the LR isn't saved on the stack, for example? What if the
> > code you're trying to unwind is hand-written assembly annotated with CFI
> > directives?
> Then in that case the unwinding is not possible unless the
> hand-crafted asm is compatible with the requested unwinding method
> (fp, dwarf etc.). Do you expect problems there, if so can you give
> more details?
To use a readily available AArch64 example, take a look at
__kernel_gettimeofday in arch/arm64/kernel/vdso/gettimeofday.S
It starts by moving the link register into x2, so that it can later call
__do_get_tspec without clobbering it. Furthermore, it doesn't make use of
the stack at all.
How can you unwind that using your current code?
Will
Hi Will,
On 26 June 2014 11:00, Will Deacon <[email protected]> wrote:
> On Wed, Jun 25, 2014 at 03:54:14PM +0100, Jean Pihet wrote:
>> Hi Will,
>
> Hello,
>
>> On 25 June 2014 11:01, Will Deacon <[email protected]> wrote:
>> > On Fri, Jun 20, 2014 at 09:10:35AM +0100, Jean Pihet wrote:
>> >> On 18 June 2014 14:53, Will Deacon <[email protected]> wrote:
>> >> > On Tue, Jun 17, 2014 at 06:11:05PM +0100, Jean Pihet wrote:
>> >> >> Tested with perf record and tracepoints filtering (-e <tracepoint>), with
>> >> >> unwinding using fp (--call-graph fp) and dwarf info (--call-graph dwarf).
>> >> >
>> >> > Whilst the old ACPS unwinding only needs PC, FP and SP, is this definitely
>> >> > true for exidx and DWARF-based unwinding? Given that libunwind ends up
>> >> > running a state machine for the latter, can we guarantee that we won't hit
>> >> > instructions that require access to other general purpose registers?
>> >> Yes. dwarf unwinding does not need anything extra. Once seeded all the
>> >> rest is extracted from the dwarf trace info.
>> >
>> > Ok, but what if the LR isn't saved on the stack, for example? What if the
>> > code you're trying to unwind is hand-written assembly annotated with CFI
>> > directives?
>> Then in that case the unwinding is not possible unless the
>> hand-crafted asm is compatible with the requested unwinding method
>> (fp, dwarf etc.). Do you expect problems there, if so can you give
>> more details?
>
> To use a readily available AArch64 example, take a look at
> __kernel_gettimeofday in arch/arm64/kernel/vdso/gettimeofday.S
>
> It starts by moving the link register into x2, so that it can later call
> __do_get_tspec without clobbering it. Furthermore, it doesn't make use of
> the stack at all.
>
> How can you unwind that using your current code?
That is interesting. In that case that particular function will not be
seen in the call chain since lr, fp are the ones from the caller. I
did not try on a real case, it would be nice to try it out, I can do
that as soon as I am back on ARM64.
Note: I was debugging a deadlock in perf doing call chain unwinding
and tracepoint triggering. A new patch set is on its way.
Thx & regards,
Jean
>
> Will