2013-06-26 11:09:22

by Andy Green

[permalink] [raw]
Subject: 3.10-rc6 "perf test" seems unhappy

Hi -

Running perf test on 3.10-rc6 gives some unexpected behaviours.

This is on a dual Cortex A9 board with perf userland cross-built from
same tree as the kernel.

root@aa9-eb:~# perf test
1: vmlinux symtab matches kallsyms : FAILED!
2: detect open syscall event : Ok
3: detect open syscall event on all cpus : Ok
4: read samples using the mmap interface : Ok
5: parse events tests : Warning:
function sizeof not defined
Warning: function sizeof not defined
Warning: function scsi_trace_parse_cdb not defined
Warning: function scsi_trace_parse_cdb not defined
Warning: function scsi_trace_parse_cdb not defined
Warning: function scsi_trace_parse_cdb not defined
Warning: function jiffies_to_msecs not defined
Warning: function jiffies_to_msecs not defined
Warning: bad op token {
Warning: bad op token {
Warning: bad op token {
Warning: Error: expected type 4 but read 0
Warning: bad op token {
Warning: Error: expected type 4 but read 0
Warning: bad op token {
Warning: bad op token {
Warning: bad op token {
Warning: bad op token {
Warning: bad op token {
Warning: unknown op '{'
Warning: unknown op '{'
Ok
6: Validate PERF_RECORD_* events & perf_sample fields : FAILED!
7: Test perf pmu format parsing : Ok
8: Test dso data interface : Ok
9: roundtrip evsel->name check : Ok
10: Check parsing of sched tracepoints fields : FAILED!
11: Generate and check syscalls:sys_enter_open event fields: Ok
12: struct perf_event_attr setup : (omitted) Ok
13: Test matching and linking mutliple hists : Ok
14: Try 'use perf' in python, checking link problems : FAILED!
15: Test breakpoint overflow signal handler : FAILED!
16: Test breakpoint overflow sampling : FAILED!
17: Test number of exit event of a simple workload : FAILED!
18: Test software clock events have valid period values : Ok


I started tracing the first failure... it stops because it can't find
"_stext" in /proc/kallsyms. But it is there

root@aa9-eb:~# cat /proc/kallsyms | grep _stext
800081c0 T _stext

test 5 (parse event tests) seems to be badly broken.

The others may be due to missing some config somewhere.

Any ideas what I am (or perf is) doing wrong?

-Andy


2013-06-26 13:39:34

by Prabhat Kumar Ravi

[permalink] [raw]
Subject: Re: 3.10-rc6 "perf test" seems unhappy

Hi,

On Wed, Jun 26, 2013 at 4:39 PM, Andy Green <[email protected]> wrote:
> Hi -
>
> Running perf test on 3.10-rc6 gives some unexpected behaviours.
>
> This is on a dual Cortex A9 board with perf userland cross-built from
> same tree as the kernel.
>
> root@aa9-eb:~# perf test
> 1: vmlinux symtab matches kallsyms : FAILED!

> I started tracing the first failure... it stops because it can't find
> "_stext" in /proc/kallsyms. But it is there
>
> root@aa9-eb:~# cat /proc/kallsyms | grep _stext
> 800081c0 T _stext
>

>
> The others may be due to missing some config somewhere.
>
> Any ideas what I am (or perf is) doing wrong?
>

I Think your Perf test is failing at dso__find_symbol_by_name
where in perf test we have something like:

kallsyms_map = machine__kernel_map(&kallsyms, type);

sym = map__find_symbol_by_name(kallsyms_map, ref_reloc_sym.name, NULL);
if (sym == NULL) {
pr_debug("dso__find_symbol_by_name ");
goto out;
}

Here sym is search for "_stext" which is NULL here so perf test fails here only,

Here _stext having same address as some other, like

c00081c0 T asm_do_IRQ
c00081c0 T _stext
c00081c0 T __exception_text_start

Function call is like this:
machine__load_kallsyms
--> dso__load_kallsyms
--> symbols__fixup_duplicate

So, it getting deleted, I think we should disable symbols__fixup_duplicate.

- Prabhat

2013-06-27 12:37:41

by Andy Green

[permalink] [raw]
Subject: Re: 3.10-rc6 "perf test" seems unhappy

On 26 June 2013 21:39, Prabhat Kumar Ravi <[email protected]> wrote:
> Hi,
>
> On Wed, Jun 26, 2013 at 4:39 PM, Andy Green <[email protected]> wrote:
>> Hi -
>>
>> Running perf test on 3.10-rc6 gives some unexpected behaviours.
>>
>> This is on a dual Cortex A9 board with perf userland cross-built from
>> same tree as the kernel.
>>
>> root@aa9-eb:~# perf test
>> 1: vmlinux symtab matches kallsyms : FAILED!
>
>> I started tracing the first failure... it stops because it can't find
>> "_stext" in /proc/kallsyms. But it is there
>>
>> root@aa9-eb:~# cat /proc/kallsyms | grep _stext
>> 800081c0 T _stext
>>
>
>>
>> The others may be due to missing some config somewhere.
>>
>> Any ideas what I am (or perf is) doing wrong?
>>
>
> I Think your Perf test is failing at dso__find_symbol_by_name
> where in perf test we have something like:
>
> kallsyms_map = machine__kernel_map(&kallsyms, type);
>
> sym = map__find_symbol_by_name(kallsyms_map, ref_reloc_sym.name, NULL);
> if (sym == NULL) {
> pr_debug("dso__find_symbol_by_name ");
> goto out;
> }
>
> Here sym is search for "_stext" which is NULL here so perf test fails here only,

Hi Prabhat -

> Here _stext having same address as some other, like
>
> c00081c0 T asm_do_IRQ
> c00081c0 T _stext
> c00081c0 T __exception_text_start
>
> Function call is like this:
> machine__load_kallsyms
> --> dso__load_kallsyms
> --> symbols__fixup_duplicate
>
> So, it getting deleted, I think we should disable symbols__fixup_duplicate.

That does indeed work around the first problem, thanks.

It succeeds to find _stext and then tries to load the vmlinux.

1: vmlinux symtab matches kallsyms :trying
/root/.debug/.build-id/47/93c412145cb348574991bf4075995805175367
failed in symsrc__init
trying vmlinux
failed in symsrc__init
trying /boot/vmlinux
failed in symsrc__init
trying /boot/vmlinux-3.10.0-linaro_fjslt-tracking-bsp2.0.0-rc1+
failed in symsrc__init
trying /lib/modules/3.10.0-linaro_fjslt-tracking-bsp2.0.0-rc1+/build/vmlinux
failed in symsrc__init
trying /usr/lib/debug/lib/modules/3.10.0-linaro_fjslt-tracking-bsp2.0.0-rc1+/vmlinux
do_load_sym result 0
Skip

Actually I put the correct vmlinux in the last place,
/usr/lib/debug/... it's able to find it but do_load_sym is returning
0, which is taken as fail, because it's unable to confirm the vintage
of the vmlinux is correct for the running kernel.

As a side note I also copied ./lib, ./tools and ./include to the
target and built perf there. The results are the same.

# perf test
1: vmlinux symtab matches kallsyms :trying
/root/.debug/.build-id/47/93c412145cb348574991bf4075995805175367
failed in symsrc__init
trying vmlinux
failed in symsrc__init
trying /boot/vmlinux
failed in symsrc__init
trying /boot/vmlinux-3.10.0-linaro_fjslt-tracking-bsp2.0.0-rc1+
failed in symsrc__init
trying /lib/modules/3.10.0-linaro_fjslt-tracking-bsp2.0.0-rc1+/build/vmlinux
failed in symsrc__init
trying /usr/lib/debug/lib/modules/3.10.0-linaro_fjslt-tracking-bsp2.0.0-rc1+/vmlinux
do_load_sym result 0
Skip
2: detect open syscall event : Ok
3: detect open syscall event on all cpus : Ok
4: read samples using the mmap interface : Ok
5: parse events tests : Warning:
function sizeof not defined
Warning: function sizeof not defined
Warning: function scsi_trace_parse_cdb not defined
Warning: function scsi_trace_parse_cdb not defined
Warning: function scsi_trace_parse_cdb not defined
Warning: function scsi_trace_parse_cdb not defined
Warning: function jiffies_to_msecs not defined
Warning: function jiffies_to_msecs not defined
Warning: bad op token {
Warning: bad op token {
Warning: bad op token {
Warning: Error: expected type 4 but read 0
Warning: bad op token {
Warning: Error: expected type 4 but read 0
Warning: bad op token {
Warning: bad op token {
Warning: bad op token {
Warning: bad op token {
Warning: bad op token {
Warning: unknown op '{'
Warning: unknown op '{'
Ok
6: Validate PERF_RECORD_* events & perf_sample fields : FAILED!
7: Test perf pmu format parsing : Ok
8: Test dso data interface : Ok
9: roundtrip evsel->name check : Ok
10: Check parsing of sched tracepoints fields : FAILED!
11: Generate and check syscalls:sys_enter_open event fields: Ok
12: struct perf_event_attr setup : (omitted) Ok
13: Test matching and linking mutliple hists : Ok
14: Try 'use perf' in python, checking link problems : FAILED!
15: Test breakpoint overflow signal handler : FAILED!
16: Test breakpoint overflow sampling : FAILED!
17: Test number of exit event of a simple workload : FAILED!
18: Test software clock events have valid period values : Ok

-Andy