2008-06-06 18:58:25

by Arjan van de Ven

[permalink] [raw]
Subject: Top kernel oopses/warnings for the week of Friday June 6th, 2008

The http://www.kerneloops.org website collects kernel oops and
warning reports from various mailing lists and bugzillas as well as
with a client users can install to auto-submit oopses.
Below is a top 12 list of the traces collected in the last 7 days.
(Reports prior to 2.6.23 have been omitted in collecting the top 10)

This week, a total of 3425 oopses and warnings have been reported,
compared to 3522 reports in the previous week.

This weeks result look rather different from last weeks; mostly because I've spend
some time improving the "who is guilty" algorith to more accurately pinpoint the
blame on the really guilty party.


Per file statistics
751 external/madwifi/wrapper (P)
374 external/madwifi/binary (P)
295 drivers/parport/procfs.c
253 fs/sysfs/dir.c
197 fs/jbd/journal.c
180 security/selinux/hooks.c
95 kernel/time/tick-broadcast.c
39 external/rt/locking
39 net/mac80211/main.c
37 external/vmware/networking (P)
35 drivers/ata/libata-core.c
34 fs/ext3/super.c


Seen in untainted kernels
-------------------------
Rank 3: parport_device_proc_register (warning)
Reported 298 times (938 total reports)
Duplicate /proc registration in the parport driver
This warning was last seen in version 2.6.26-rc4-git2, and first seen in 2.6.24-rc5.
More info: http://www.kerneloops.org/searchweek.php?search=parport_device_proc_register

Rank 4: sysfs_add_one (warning)
Reported 242 times (992 total reports)
Duplicate sysfs registration in various components. We don't seem to be making progress on this
one unfortunately.
This warning was last seen in version 2.6.26-rc3, and first seen in 2.6.24-rc6.
More info: http://www.kerneloops.org/searchweek.php?search=sysfs_add_one

Rank 6: journal_update_superblock (warning)
Reported 171 times (709 total reports)
Likely caused by the user removing a USB stick while mounted
This warning was last seen in version 2.6.26, and first seen in 2.6.24-rc6-git1.
More info: http://www.kerneloops.org/searchweek.php?search=journal_update_superblock

Rank 8: tick_broadcast_oneshot_control (deadlock)
Reported 95 times (315 total reports)
The deadlock/timeout detector triggers in this codepath rather agressively. I'm hesitant
to pin this one on bad hardware, it's happening too repeastedly too much.
This deadlock was last seen in version 2.6.25.3, and first seen in 2.6.24-rc4.
More info: http://www.kerneloops.org/searchweek.php?search=tick_broadcast_oneshot_control

Rank 9: ieee80211_stop_tx_ba_session (warning)
Reported 39 times (200 total reports)
Bug in the iwl4965 driver
This warning was last seen in version 2.6.25.3, and first seen in 2.6.25-rc7-git6.
More info: http://www.kerneloops.org/searchweek.php?search=ieee80211_stop_tx_ba_session

Rank 11: ata_hsm_move (warning)
Reported 34 times (95 total reports)
Needs diagnosing
This warning was last seen in version 2.6.25.3, and first seen in 2.6.25-rc9-git1.
More info: http://www.kerneloops.org/searchweek.php?search=ata_hsm_move

Rank 12: set_irq_wake (warning)
Reported 32 times (94 total reports)
[fixed] Bug in serial_core.c where disable_irq_wake/enable_irq_wake were unbalanced
This warning was last seen in version 2.6.25.3, and first seen in 2.6.25-rc9-git1.
More info: http://www.kerneloops.org/searchweek.php?search=set_irq_wake

Rank 13: ext3_commit_super (warning)
Reported 32 times (174 total reports)
Likely caused by the user removing a USB stick while mounted
This warning was last seen in version 2.6.25.3, and first seen in 2.6.24.
More info: http://www.kerneloops.org/searchweek.php?search=ext3_commit_super


Caused by binary modules
------------------------
Rank 1: ath_dynamic_sysctl_register (warning)
Reported 646 times (2522 total reports)
[external] Bug in the proprietary madwifi driver
warning only shows up in tainted kernels
This warning was last seen in version 2.6.25.4, and first seen in 2.6.24.
More info: http://www.kerneloops.org/searchweek.php?search=ath_dynamic_sysctl_register

Rank 2: init_ath_hal (warning)
Reported 379 times (1485 total reports)
[external] Bug in the proprietary madwifi driver
warning only shows up in tainted kernels
This warning was last seen in version 2.6.25.4, and first seen in 2.6.24.
More info: http://www.kerneloops.org/searchweek.php?search=init_ath_hal

Rank 5: task_has_capability (warning)
Reported 178 times (369 total reports)
[external] Bug in the proprietary firegl driver
warning only shows up in tainted kernels
This warning was last seen in version 2.6.25.4, and first seen in 2.6.25.
More info: http://www.kerneloops.org/searchweek.php?search=task_has_capability

Rank 7: ath_sysctl_register (warning)
Reported 115 times (589 total reports)
[external] Bug in the proprietary madwifi driver
warning only shows up in tainted kernels
This warning was last seen in version 2.6.25.4, and first seen in 2.6.24-rc4-git4.
More info: http://www.kerneloops.org/searchweek.php?search=ath_sysctl_register

Rank 10: VNetBridgeDown (warning)
Reported 37 times (170 total reports)
[external] Bug in the proprietary VMWare drivers
warning only shows up in tainted kernels
This warning was last seen in version 2.6.25.4, and first seen in 2.6.24.
More info: http://www.kerneloops.org/searchweek.php?search=VNetBridgeDown


2008-06-06 20:43:20

by Thomas Gleixner

[permalink] [raw]
Subject: Re: Top kernel oopses/warnings for the week of Friday June 6th, 2008

On Fri, 6 Jun 2008, Arjan van de Ven wrote:
>
> Rank 8: tick_broadcast_oneshot_control (deadlock)
> Reported 95 times (315 total reports)
> The deadlock/timeout detector triggers in this codepath rather
> agressively. I'm hesitant
> to pin this one on bad hardware, it's happening too repeastedly too
> much.
> This deadlock was last seen in version 2.6.25.3, and first seen in
> 2.6.24-rc4.
> More info:
> http://www.kerneloops.org/searchweek.php?search=tick_broadcast_oneshot_control

Is there a way to tell on which kind of systems this shows up ?

I'm stll looking for someone having this problem, who is willing to go
through a meticulous debug session.

Thanks,

tglx

2008-06-06 20:55:58

by Arjan van de Ven

[permalink] [raw]
Subject: Re: Top kernel oopses/warnings for the week of Friday June 6th, 2008

On Fri, 6 Jun 2008 22:37:50 +0200 (CEST)
Thomas Gleixner <[email protected]> wrote:

> On Fri, 6 Jun 2008, Arjan van de Ven wrote:
> >
> > Rank 8: tick_broadcast_oneshot_control (deadlock)
> > Reported 95 times (315 total reports)
> > The deadlock/timeout detector triggers in this codepath
> > rather agressively. I'm hesitant
> > to pin this one on bad hardware, it's happening too
> > repeastedly too much.
> > This deadlock was last seen in version 2.6.25.3, and first
> > seen in 2.6.24-rc4.
> > More info:
> > http://www.kerneloops.org/searchweek.php?search=tick_broadcast_oneshot_control
>
> Is there a way to tell on which kind of systems this shows
> up ?

https://bugzilla.redhat.com/show_bug.cgi?id=444059
https://bugzilla.redhat.com/show_bug.cgi?id=442920

either one of those persons might be willing to help;)

--
If you want to reach me at my work email, use [email protected]
For development, discussion and tips for power savings,
visit http://www.lesswatts.org

2008-06-06 21:07:27

by Thomas Gleixner

[permalink] [raw]
Subject: Re: Top kernel oopses/warnings for the week of Friday June 6th, 2008

On Fri, 6 Jun 2008, Arjan van de Ven wrote:
> On Fri, 6 Jun 2008 22:37:50 +0200 (CEST)
> Thomas Gleixner <[email protected]> wrote:
>
> > On Fri, 6 Jun 2008, Arjan van de Ven wrote:
> > >
> > > Rank 8: tick_broadcast_oneshot_control (deadlock)
> > > Reported 95 times (315 total reports)
> > > The deadlock/timeout detector triggers in this codepath
> > > rather agressively. I'm hesitant
> > > to pin this one on bad hardware, it's happening too
> > > repeastedly too much.
> > > This deadlock was last seen in version 2.6.25.3, and first
> > > seen in 2.6.24-rc4.
> > > More info:
> > > http://www.kerneloops.org/searchweek.php?search=tick_broadcast_oneshot_control
> >
> > Is there a way to tell on which kind of systems this shows
> > up ?
>
> https://bugzilla.redhat.com/show_bug.cgi?id=444059
> https://bugzilla.redhat.com/show_bug.cgi?id=442920
>
> either one of those persons might be willing to help;)

Sigh, the magically resolved bug which takes ages to reproduce and
therefor makes bisection across thousands of commits a nightmare.

I was more looking for an "easy" reproducer :)

Thanks,

tglx

2008-06-08 04:38:20

by Greg KH

[permalink] [raw]
Subject: Re: Top kernel oopses/warnings for the week of Friday June 6th, 2008

On Fri, Jun 06, 2008 at 11:58:16AM -0700, Arjan van de Ven wrote:
>
> Rank 4: sysfs_add_one (warning)
> Reported 242 times (992 total reports)
> Duplicate sysfs registration in various components. We don't seem to be
> making progress on this
> one unfortunately.
> This warning was last seen in version 2.6.26-rc3, and first seen in
> 2.6.24-rc6.
> More info: http://www.kerneloops.org/searchweek.php?search=sysfs_add_one

I really think this is solved for the USB subsystem at the least. This
will be hit by other code under development at times, when the developer
gets something wrong.

If it does show up post 2.6.26-rc3 for the USB subsystem, please let me
know.

thanks,

greg k-h

2008-06-08 04:56:15

by Andrew Morton

[permalink] [raw]
Subject: Re: Top kernel oopses/warnings for the week of Friday June 6th, 2008

On Sat, 7 Jun 2008 21:34:05 -0700 Greg KH <[email protected]> wrote:

> On Fri, Jun 06, 2008 at 11:58:16AM -0700, Arjan van de Ven wrote:
> >
> > Rank 4: sysfs_add_one (warning)
> > Reported 242 times (992 total reports)
> > Duplicate sysfs registration in various components. We don't seem to be
> > making progress on this
> > one unfortunately.
> > This warning was last seen in version 2.6.26-rc3, and first seen in
> > 2.6.24-rc6.
> > More info: http://www.kerneloops.org/searchweek.php?search=sysfs_add_one
>
> I really think this is solved for the USB subsystem at the least. This
> will be hit by other code under development at times, when the developer
> gets something wrong.

For the sysfs vfs caches, yes. But for sysfs pagecache (which I expect
is only symlinks?) the pages will still be on the LRU?