2006-10-29 20:55:09

by Martin Lorenz

[permalink] [raw]
Subject: 2.6.19-rc3: more DWARFs and strange messages

With my recent kernel pulled from git on friday I see quite some DWARFs and
other strange messages.

lots of those:
[18038.721000] thinkpad_ec: thinkpad_ec_request_row: bad end STR3:
(0x11:0x00)->0x80

this was said to be triggered by the combination of hdaps and tp_smapi but I
do not load the tp_smapi module ...
hdaps is configured

and quite a few of those:

[18504.980000] BUG: warning at kernel/cpu.c:56/unlock_cpu_hotplug()
[18504.980000] [<c0103bdd>] dump_trace+0x69/0x1af
[18504.980000] [<c0103d3b>] show_trace_log_lvl+0x18/0x2c
[18504.980000] [<c01043da>] show_trace+0xf/0x11
[18504.980000] [<c01044dd>] dump_stack+0x15/0x17
[18504.980000] [<c0135e94>] unlock_cpu_hotplug+0x3d/0x66
[18504.980000] [<f92e67f3>] do_dbs_timer+0x1c2/0x229 [cpufreq_ondemand]
[18504.980000] [<c012ccb1>] run_workqueue+0x83/0xc5
[18504.980000] [<c012d5d5>] worker_thread+0xd9/0x10c
[18504.980000] [<c012fb36>] kthread+0xc2/0xf0
[18504.980000] [<c010398b>] kernel_thread_helper+0x7/0x10
[18504.980000] DWARF2 unwinder stuck at kernel_thread_helper+0x7/0x10
[18504.980000]
[18504.980000] Leftover inexact backtrace:
[18504.980000]
[18504.980000] =======================


full config and dmesg is in
http://www.lorenz.eu.org/~mlo/kernel/config.2.6.19-rc3-e1-ie-tp-43.3+1757-g18462d6b-dirty.gz
http://www.lorenz.eu.org/~mlo/kernel/dmesg.2.6.19-rc3-e1-ie-tp-43.3+1757-g18462d6b-dirty.boot
http://www.lorenz.eu.org/~mlo/kernel/dmesg.2.6.19-rc3-e1-ie-tp-43.3+1757-g18462d6b-dirty.run

and more
http://www.lorenz.eu.org/~mlo/kernel/?C=M;O=D

greets
mlo
--
Dipl.-Ing. Martin Lorenz

They that can give up essential liberty
to obtain a little temporary safety
deserve neither liberty nor safety.
Benjamin Franklin

please encrypt your mail to me
GnuPG key-ID: F1AAD37D
get it here:
http://blackhole.pca.dfn.de:11371/pks/lookup?op=get&search=0xF1AAD37D

ICQ UIN: 33588107


2006-10-31 14:31:47

by Andi Kleen

[permalink] [raw]
Subject: Re: 2.6.19-rc3: more DWARFs and strange messages

Martin Lorenz <[email protected]> writes:

> and quite a few of those:
>
> [18504.980000] BUG: warning at kernel/cpu.c:56/unlock_cpu_hotplug()
> [18504.980000] [<c0103bdd>] dump_trace+0x69/0x1af
> [18504.980000] [<c0103d3b>] show_trace_log_lvl+0x18/0x2c
> [18504.980000] [<c01043da>] show_trace+0xf/0x11
> [18504.980000] [<c01044dd>] dump_stack+0x15/0x17
> [18504.980000] [<c0135e94>] unlock_cpu_hotplug+0x3d/0x66
> [18504.980000] [<f92e67f3>] do_dbs_timer+0x1c2/0x229 [cpufreq_ondemand]
> [18504.980000] [<c012ccb1>] run_workqueue+0x83/0xc5
> [18504.980000] [<c012d5d5>] worker_thread+0xd9/0x10c
> [18504.980000] [<c012fb36>] kthread+0xc2/0xf0
> [18504.980000] [<c010398b>] kernel_thread_helper+0x7/0x10
> [18504.980000] DWARF2 unwinder stuck at kernel_thread_helper+0x7/0x10

What gcc / binutils version do you use?

> [18504.980000]
> [18504.980000] Leftover inexact backtrace:
> [18504.980000]
> [18504.980000] =======================
>

-Andi

2006-10-31 16:10:24

by Martin Lorenz

[permalink] [raw]
Subject: Re: 2.6.19-rc3: more DWARFs and strange messages

On Tue, Oct 31, 2006 at 03:31:41PM +0100, Andi Kleen wrote:
> Martin Lorenz <[email protected]> writes:
>
> > and quite a few of those:
> >
> > [18504.980000] BUG: warning at kernel/cpu.c:56/unlock_cpu_hotplug()
> > [18504.980000] [<c0103bdd>] dump_trace+0x69/0x1af
> > [18504.980000] [<c0103d3b>] show_trace_log_lvl+0x18/0x2c
> > [18504.980000] [<c01043da>] show_trace+0xf/0x11
> > [18504.980000] [<c01044dd>] dump_stack+0x15/0x17
> > [18504.980000] [<c0135e94>] unlock_cpu_hotplug+0x3d/0x66
> > [18504.980000] [<f92e67f3>] do_dbs_timer+0x1c2/0x229 [cpufreq_ondemand]
> > [18504.980000] [<c012ccb1>] run_workqueue+0x83/0xc5
> > [18504.980000] [<c012d5d5>] worker_thread+0xd9/0x10c
> > [18504.980000] [<c012fb36>] kthread+0xc2/0xf0
> > [18504.980000] [<c010398b>] kernel_thread_helper+0x7/0x10
> > [18504.980000] DWARF2 unwinder stuck at kernel_thread_helper+0x7/0x10
>
> What gcc / binutils version do you use?
$ gcc --version
gcc (GCC) 4.1.2 20061020 (prerelease) (Debian 4.1.1-17)

dpkg says:
ii gcc 4.1.1-13

$ ar --version
GNU ar 2.17 Debian GNU/Linux

dpkg says:
ii binutils 2.17-3


>
> > [18504.980000]
> > [18504.980000] Leftover inexact backtrace:
> > [18504.980000]
> > [18504.980000] =======================
> >
>
> -Andi
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/

gruss
mlo
--
Dipl.-Ing. Martin Lorenz

They that can give up essential liberty
to obtain a little temporary safety
deserve neither liberty nor safety.
Benjamin Franklin

please encrypt your mail to me
GnuPG key-ID: F1AAD37D
get it here:
http://blackhole.pca.dfn.de:11371/pks/lookup?op=get&search=0xF1AAD37D

ICQ UIN: 33588107

2006-10-31 16:26:27

by Jan Beulich

[permalink] [raw]
Subject: Re: 2.6.19-rc3: more DWARFs and strange messages

Can you perhaps get us arch/i386/kernel/{entry,process}.o,
.config, and (assuming you can reproduce the original problem)
the raw stack dump obtained with a sufficiently high kstack=
option?

Thanks, Jan

>>> Martin Lorenz <[email protected]> 31.10.06 17:08 >>>
On Tue, Oct 31, 2006 at 03:31:41PM +0100, Andi Kleen wrote:
> Martin Lorenz <[email protected]> writes:
>
> > and quite a few of those:
> >
> > [18504.980000] BUG: warning at kernel/cpu.c:56/unlock_cpu_hotplug()
> > [18504.980000] [<c0103bdd>] dump_trace+0x69/0x1af
> > [18504.980000] [<c0103d3b>] show_trace_log_lvl+0x18/0x2c
> > [18504.980000] [<c01043da>] show_trace+0xf/0x11
> > [18504.980000] [<c01044dd>] dump_stack+0x15/0x17
> > [18504.980000] [<c0135e94>] unlock_cpu_hotplug+0x3d/0x66
> > [18504.980000] [<f92e67f3>] do_dbs_timer+0x1c2/0x229 [cpufreq_ondemand]
> > [18504.980000] [<c012ccb1>] run_workqueue+0x83/0xc5
> > [18504.980000] [<c012d5d5>] worker_thread+0xd9/0x10c
> > [18504.980000] [<c012fb36>] kthread+0xc2/0xf0
> > [18504.980000] [<c010398b>] kernel_thread_helper+0x7/0x10
> > [18504.980000] DWARF2 unwinder stuck at kernel_thread_helper+0x7/0x10
>
> What gcc / binutils version do you use?
$ gcc --version
gcc (GCC) 4.1.2 20061020 (prerelease) (Debian 4.1.1-17)

dpkg says:
ii gcc 4.1.1-13

$ ar --version
GNU ar 2.17 Debian GNU/Linux

dpkg says:
ii binutils 2.17-3


>
> > [18504.980000]
> > [18504.980000] Leftover inexact backtrace:
> > [18504.980000]
> > [18504.980000] =======================
> >
>
> -Andi
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/

gruss
mlo
--
Dipl.-Ing. Martin Lorenz

They that can give up essential liberty
to obtain a little temporary safety
deserve neither liberty nor safety.
Benjamin Franklin

please encrypt your mail to me
GnuPG key-ID: F1AAD37D
get it here:
http://blackhole.pca.dfn.de:11371/pks/lookup?op=get&search=0xF1AAD37D

ICQ UIN: 33588107

2006-10-31 17:28:55

by Andi Kleen

[permalink] [raw]
Subject: Re: 2.6.19-rc3: more DWARFs and strange messages

On Tuesday 31 October 2006 17:28, Jan Beulich wrote:
> Can you perhaps get us arch/i386/kernel/{entry,process}.o,
> .config, and (assuming you can reproduce the original problem)
> the raw stack dump obtained with a sufficiently high kstack=
> option?

WARN_ON unfortunately doesn't dump the raw stack at all (maybe that
should be fixed)

-Andi

2006-11-01 15:31:04

by Martin Lorenz

[permalink] [raw]
Subject: Re: 2.6.19-rc3: more DWARFs and strange messages

On Tue, Oct 31, 2006 at 06:28:52PM +0100, Andi Kleen wrote:
> On Tuesday 31 October 2006 17:28, Jan Beulich wrote:
> > Can you perhaps get us arch/i386/kernel/{entry,process}.o,
> > .config, and (assuming you can reproduce the original problem)
> > the raw stack dump obtained with a sufficiently high kstack=
> > option?
>
> WARN_ON unfortunately doesn't dump the raw stack at all (maybe that
> should be fixed)

what is a reasonable kstack parameter to be informative for you?

>
> -Andi
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/

gruss
mlo
--
Dipl.-Ing. Martin Lorenz

They that can give up essential liberty
to obtain a little temporary safety
deserve neither liberty nor safety.
Benjamin Franklin

please encrypt your mail to me
GnuPG key-ID: F1AAD37D
get it here:
http://blackhole.pca.dfn.de:11371/pks/lookup?op=get&search=0xF1AAD37D

ICQ UIN: 33588107

2006-11-02 09:36:18

by Jan Beulich

[permalink] [raw]
Subject: Re: 2.6.19-rc3: more DWARFs and strange messages

>what is a reasonable kstack parameter to be informative for you?

This unfortunately depends on the depth of the stack that is in use
at the point the dump is taken. The only safe value would be to
dump the full stack size (kstack=1024 for 4k stack, kstack=2048
for 8k ones), but since it'll stop at a stack boundary perhaps that's
what you should go with.

As to Andi's remark regarding WARN_ON() - you'd have to address
that issue in a private patch first, or the addition of the kstack=
parameter will be useless. I presume it's likely you don't have the
time to do that...

Jan

2006-11-02 09:50:26

by Martin Lorenz

[permalink] [raw]
Subject: Re: 2.6.19-rc3: more DWARFs and strange messages

On Thu, Nov 02, 2006 at 09:37:58AM +0000, Jan Beulich wrote:
> >what is a reasonable kstack parameter to be informative for you?
>
> This unfortunately depends on the depth of the stack that is in use
> at the point the dump is taken. The only safe value would be to
> dump the full stack size (kstack=1024 for 4k stack, kstack=2048
> for 8k ones), but since it'll stop at a stack boundary perhaps that's
> what you should go with.
>
> As to Andi's remark regarding WARN_ON() - you'd have to address
> that issue in a private patch first, or the addition of the kstack=
> parameter will be useless. I presume it's likely you don't have the
> time to do that...

well...
to be honest, I need my computer to work during usual daily business
at the moment I can afford to do a compile and test a new kernel once a day,
but if it introduces instabilities that make my computer unusable for
productive work (which means programming, running heavy java applications
and and TeXing) I can't run it for more than a few minutes.

I started a bisection last weekend but had to give up after three
compile-reboot cycles.

so, please don't expect too much, but I will do my best to help

gruss
mlo
--
Dipl.-Ing. Martin Lorenz

They that can give up essential liberty
to obtain a little temporary safety
deserve neither liberty nor safety.
Benjamin Franklin

please encrypt your mail to me
GnuPG key-ID: F1AAD37D
get it here:
http://blackhole.pca.dfn.de:11371/pks/lookup?op=get&search=0xF1AAD37D

ICQ UIN: 33588107

2006-11-02 10:54:27

by Jan Beulich

[permalink] [raw]
Subject: Re: 2.6.19-rc3: more DWARFs and strange messages

Hmm, another thing you could do is a sysrq-t with the increased
kstack= - this should encounter several kernel_thread()-s, and all
of them should exhibit the problem you were seeing. That output
could then be of help.

Thanks, Jan


2006-11-06 10:54:42

by Adrian Bunk

[permalink] [raw]
Subject: Re: 2.6.19-rc3: more DWARFs and strange messages

Martin, did commit 4b96b1a10cb00c867103b21f0f2a6c91b705db11 that is now
in Linus' tree fix this issue?


On Sat, Oct 28, 2006 at 10:01:51PM +0200, Martin Lorenz wrote:
> With my recent kernel pulled from git on friday I see quite some DWARFs and
> other strange messages.
>
> lots of those:
> [18038.721000] thinkpad_ec: thinkpad_ec_request_row: bad end STR3:
> (0x11:0x00)->0x80
>
> this was said to be triggered by the combination of hdaps and tp_smapi but I
> do not load the tp_smapi module ...
> hdaps is configured
>
> and quite a few of those:
>
> [18504.980000] BUG: warning at kernel/cpu.c:56/unlock_cpu_hotplug()
> [18504.980000] [<c0103bdd>] dump_trace+0x69/0x1af
> [18504.980000] [<c0103d3b>] show_trace_log_lvl+0x18/0x2c
> [18504.980000] [<c01043da>] show_trace+0xf/0x11
> [18504.980000] [<c01044dd>] dump_stack+0x15/0x17
> [18504.980000] [<c0135e94>] unlock_cpu_hotplug+0x3d/0x66
> [18504.980000] [<f92e67f3>] do_dbs_timer+0x1c2/0x229 [cpufreq_ondemand]
> [18504.980000] [<c012ccb1>] run_workqueue+0x83/0xc5
> [18504.980000] [<c012d5d5>] worker_thread+0xd9/0x10c
> [18504.980000] [<c012fb36>] kthread+0xc2/0xf0
> [18504.980000] [<c010398b>] kernel_thread_helper+0x7/0x10
> [18504.980000] DWARF2 unwinder stuck at kernel_thread_helper+0x7/0x10
> [18504.980000]
> [18504.980000] Leftover inexact backtrace:
> [18504.980000]
> [18504.980000] =======================
>
>
> full config and dmesg is in
> http://www.lorenz.eu.org/~mlo/kernel/config.2.6.19-rc3-e1-ie-tp-43.3+1757-g18462d6b-dirty.gz
> http://www.lorenz.eu.org/~mlo/kernel/dmesg.2.6.19-rc3-e1-ie-tp-43.3+1757-g18462d6b-dirty.boot
> http://www.lorenz.eu.org/~mlo/kernel/dmesg.2.6.19-rc3-e1-ie-tp-43.3+1757-g18462d6b-dirty.run
>
> and more
> http://www.lorenz.eu.org/~mlo/kernel/?C=M;O=D
>
> greets
> mlo
> --
> Dipl.-Ing. Martin Lorenz
>
> They that can give up essential liberty
> to obtain a little temporary safety
> deserve neither liberty nor safety.
> Benjamin Franklin
>
> please encrypt your mail to me
> GnuPG key-ID: F1AAD37D
> get it here:
> http://blackhole.pca.dfn.de:11371/pks/lookup?op=get&search=0xF1AAD37D
>
> ICQ UIN: 33588107

2006-11-08 16:18:13

by Jan Beulich

[permalink] [raw]
Subject: Re: 2.6.19-rc3: more DWARFs and strange messages

>>> Martin Lorenz <[email protected]> 31.10.06 18:03 >>>
>On Tue, Oct 31, 2006 at 05:28:11PM +0100, Jan Beulich wrote:
>> Can you perhaps get us arch/i386/kernel/{entry,process}.o,
>> .config, and (assuming you can reproduce the original problem)
>> the raw stack dump obtained with a sufficiently high kstack=
>> option?
>
>config and the requested .o files are attached
>...hoping I diden't loose track of my kernel bilds
>stacktrace will follow ASAP

I'm unable to find anything pointing out the reason for this
misbehavior, and I continue to be unable to reproduce it on my
test systems.
Short of somebody else wanting to try to debug this, perhaps
the only way to get more insight is to add some more debug
printing to the unwinder and/or its callers, which I don't have
time to do immediately.

Jan