2015-04-12 12:41:32

by Fengguang Wu

[permalink] [raw]
Subject: [serial] BUG: spinlock bad magic on CPU#0, swapper/0

Greetings,

0day kernel testing robot got the below dmesg and the first bad commit is

git://git.code.sf.net/p/openipmi/linux-ipmi master-ipmi-rebase

commit ccfb59efb7a78e606516ce878d8dde4690419a5f
Author: Corey Minyard <[email protected]>
AuthorDate: Mon Mar 19 16:00:55 2012 -0500
Commit: Corey Minyard <[email protected]>
CommitDate: Fri Apr 10 20:51:46 2015 -0500

serial: Add a polled interface to the 8250 driver

Remove the console code from the 8250 driver and add the poll code and
setup to use the console over the polled interface.

FIXME - The interactions between the 8250_early and 8250 code need to
be analyzed and tested.

Signed-off-by: Corey Minyard <[email protected]>

drivers/tty/serial/8250.c | 227 ++++++++++++++++++++++------------------------
1 file changed, 112 insertions(+), 115 deletions(-)

+------------------------------------------------------------------+------------+------------+------------+
| | 2c46c63ec3 | ccfb59efb7 | ccfb59efb7 |
+------------------------------------------------------------------+------------+------------+------------+
| boot_successes | 0 | 0 | 0 |
| boot_failures | 81 | 11 | 11 |
| Kernel_panic-not_syncing:Out_of_memory_and_no_killable_processes | 81 | | |
| backtrace:lock_torture_stats | 81 | | |
| BUG:spinlock_bad_magic_on_CPU | 0 | 11 | 11 |
| BUG:spinlock_lockup_suspected_on_CPU | 0 | 11 | 11 |
| backtrace:register_console | 0 | 11 | 11 |
| backtrace:serial8250_console_init | 0 | 11 | 11 |
| backtrace:console_init | 0 | 11 | 11 |
+------------------------------------------------------------------+------------+------------+------------+

[ 0.000000] Running RCU self tests
[ 0.000000] NR_IRQS:16 nr_irqs:16 16
[ 0.000000] CPU 0 irqstacks, hard=40090000 soft=40092000
[ 0.000000] BUG: spinlock bad magic on CPU#0, swapper/0
[ 0.000000] lock: serial8250_ports+0x160/0x640, .magic: 00000000, .owner: <none>/-1, .owner_cpu: 0
[ 0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 4.0.0-rc7-00023-gccfb59e #232
[ 0.000000] 41bcbe09 41bcbe09 41c61e7c 41944b76 41c61ea4 4107d544 41b77694 425ee8a0
[ 0.000000] 00000000 41bcbe09 ffffffff 00000000 425ee8a0 41b72abc 41c61eb4 4107d5b2
[ 0.000000] 425ee8a0 00000060 41c61ed0 4107d742 415c57a1 00000000 425ee8a0 00000060
[ 0.000000] Call Trace:
[ 0.000000] [<41944b76>] dump_stack+0x16/0x18
[ 0.000000] [<4107d544>] spin_dump+0x84/0xd0
[ 0.000000] [<4107d5b2>] spin_bug+0x22/0x30
[ 0.000000] [<4107d742>] do_raw_spin_lock+0x62/0x120
[ 0.000000] [<415c57a1>] ? serial8250_poll+0x91/0xf0
[ 0.000000] [<4194e25f>] _raw_spin_lock+0x3f/0x50
[ 0.000000] [<415c57a1>] ? serial8250_poll+0x91/0xf0
[ 0.000000] [<415c57a1>] serial8250_poll+0x91/0xf0
[ 0.000000] [<415c1559>] uartdrv_console_write+0x69/0x320
[ 0.000000] [<415c14f0>] ? uart_write_wakeup+0x40/0x40
[ 0.000000] [<41087fb8>] call_console_drivers+0x68/0x70
[ 0.000000] [<41088e34>] console_unlock+0x3c4/0x590
[ 0.000000] [<41089a7b>] register_console+0x17b/0x360
[ 0.000000] [<41d3ba71>] ? serial8250_isa_init_ports+0x127/0x15e
[ 0.000000] [<41d3baba>] serial8250_console_init+0x12/0x16
[ 0.000000] [<41d3b022>] console_init+0x18/0x20
[ 0.000000] [<41d15972>] start_kernel+0x24e/0x353
[ 0.000000] [<41d152b6>] i386_start_kernel+0x84/0x88
[ 0.000000] BUG: spinlock lockup suspected on CPU#0, swapper/0
[ 0.000000] lock: serial8250_ports+0x160/0x640, .magic: 00000000, .owner: <none>/-1, .owner_cpu: 0
[ 0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 4.0.0-rc7-00023-gccfb59e #232
[ 0.000000] 41bcbe09 41bcbe09 41c61e8c 41944b76 41c61eb4 4107d544 41b77694 425ee8a0
[ 0.000000] 00000000 41bcbe09 ffffffff 00000000 425ee8a0 000fa000 41c61ed0 4107d7ea
[ 0.000000] 000fa000 00000000 425ee8a0 00000060 425ee8a0 41c61eec 4194e25f 00000000
[ 0.000000] Call Trace:
[ 0.000000] [<41944b76>] dump_stack+0x16/0x18
[ 0.000000] [<4107d544>] spin_dump+0x84/0xd0
[ 0.000000] [<4107d7ea>] do_raw_spin_lock+0x10a/0x120
[ 0.000000] [<4194e25f>] _raw_spin_lock+0x3f/0x50
[ 0.000000] [<415c57a1>] ? serial8250_poll+0x91/0xf0
[ 0.000000] [<415c57a1>] serial8250_poll+0x91/0xf0
[ 0.000000] [<415c1559>] uartdrv_console_write+0x69/0x320
[ 0.000000] [<415c14f0>] ? uart_write_wakeup+0x40/0x40
[ 0.000000] [<41087fb8>] call_console_drivers+0x68/0x70
[ 0.000000] [<41088e34>] console_unlock+0x3c4/0x590
[ 0.000000] [<41089a7b>] register_console+0x17b/0x360
[ 0.000000] [<41d3ba71>] ? serial8250_isa_init_ports+0x127/0x15e
[ 0.000000] [<41d3baba>] serial8250_console_init+0x12/0x16
[ 0.000000] [<41d3b022>] console_init+0x18/0x20
[ 0.000000] [<41d15972>] start_kernel+0x24e/0x353
[ 0.000000] [<41d152b6>] i386_start_kernel+0x84/0x88
[ 0.000000] console [ttyS0] enabled
[ 0.000000] console [ttyS0] enabled
[ 0.000000] Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar

git bisect start ccfb59efb7a78e606516ce878d8dde4690419a5f 3259b12ae1c79c146809829412ddc0d11a1b8e26 --
git bisect good 25d7f9f0849e210aa5882fe6ee02cd37494e2e9b # 14:55 20+ 21 i2c: add a non-blocking interface
git bisect good d63196e85e63a2769423c3ce835c07545ff55cd1 # 15:29 20+ 21 i2c-i801: Remove redundant code and event-drive
git bisect good fd5a7939c7fa0998af2647d7d8bd12754817d352 # 16:07 20+ 20 serial: Add a function to abstract the tty code from the drivers
git bisect good 6f034345553ac00afcefc2a38599cef5871c05c2 # 16:40 20+ 20 serial: Add circular buffer helpers
git bisect good 2c46c63ec3d6206342771d9378f3b22c87ec21c6 # 17:28 20+ 21 serial: Add support for doing a console using the poll interface
# first bad commit: [ccfb59efb7a78e606516ce878d8dde4690419a5f] serial: Add a polled interface to the 8250 driver
git bisect good 2c46c63ec3d6206342771d9378f3b22c87ec21c6 # 18:05 60+ 81 serial: Add support for doing a console using the poll interface
# extra tests with DEBUG_INFO
git bisect good ccfb59efb7a78e606516ce878d8dde4690419a5f # 19:10 60+ 60 serial: Add a polled interface to the 8250 driver
# extra tests on HEAD of ipmi/master-ipmi-rebase
git bisect good bd19fd5205a9435a5edb0e819f4b15d52e855634 # 20:33 60+ 62 i2c-i801: Fix the logic for SMBUS I2C transfers
# extra tests on tree/branch ipmi/master-ipmi-rebase
git bisect good bd19fd5205a9435a5edb0e819f4b15d52e855634 # 21:23 60+ 122 i2c-i801: Fix the logic for SMBUS I2C transfers
# extra tests with first bad commit reverted
# extra tests on tree/branch linus/master
git bisect good 3259b12ae1c79c146809829412ddc0d11a1b8e26 # 23:07 60+ 82 Merge tag 'scsi-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi
# extra tests on tree/branch next/master
git bisect good ab738b3efaea11cf9f130c7fe9ebb7568e998fe6 # 00:45 60+ 60 Add linux-next specific files for 20150410


This script may reproduce the error.

----------------------------------------------------------------------------
#!/bin/bash

kernel=$1
initrd=quantal-core-i386.cgz

wget --no-clobber https://github.com/fengguang/reproduce-kernel-bug/raw/master/initrd/$initrd

kvm=(
qemu-system-x86_64
-enable-kvm
-cpu kvm64
-kernel $kernel
-initrd $initrd
-m 300
-smp 2
-device e1000,netdev=net0
-netdev user,id=net0
-boot order=nc
-no-reboot
-watchdog i6300esb
-rtc base=localtime
-serial stdio
-display none
-monitor null
)

append=(
hung_task_panic=1
earlyprintk=ttyS0,115200
rd.udev.log-priority=err
systemd.log_target=journal
systemd.log_level=warning
debug
apic=debug
sysrq_always_enabled
rcupdate.rcu_cpu_stall_timeout=100
panic=-1
softlockup_panic=1
nmi_watchdog=panic
oops=panic
load_ramdisk=2
prompt_ramdisk=0
console=ttyS0,115200
console=tty0
vga=normal
root=/dev/ram0
rw
drbd.minor_count=8
)

"${kvm[@]}" --append "${append[*]}"
----------------------------------------------------------------------------

Thanks,
Fengguang


Attachments:
(No filename) (8.43 kB)
dmesg-quantal-vp-4:20150411132945:i386-randconfig-r2-0411:4.0.0-rc7-00023-gccfb59e:232 (46.58 kB)
config-4.0.0-rc7-00023-gccfb59e (76.37 kB)
Download all attachments

2015-04-12 13:01:38

by Peter Hurley

[permalink] [raw]
Subject: Re: [serial] BUG: spinlock bad magic on CPU#0, swapper/0

Hi Fengguang,

On 04/12/2015 08:41 AM, Fengguang Wu wrote:
> Greetings,
>
> 0day kernel testing robot got the below dmesg and the first bad commit is
>
> git://git.code.sf.net/p/openipmi/linux-ipmi master-ipmi-rebase
>
> commit ccfb59efb7a78e606516ce878d8dde4690419a5f
> Author: Corey Minyard <[email protected]>
> AuthorDate: Mon Mar 19 16:00:55 2012 -0500
> Commit: Corey Minyard <[email protected]>
> CommitDate: Fri Apr 10 20:51:46 2015 -0500
>
> serial: Add a polled interface to the 8250 driver
>
> Remove the console code from the 8250 driver and add the poll code and
> setup to use the console over the polled interface.
>
> FIXME - The interactions between the 8250_early and 8250 code need to
> be analyzed and tested.
>
> Signed-off-by: Corey Minyard <[email protected]>
>
> drivers/tty/serial/8250.c | 227 ++++++++++++++++++++++------------------------
> 1 file changed, 112 insertions(+), 115 deletions(-)

This is not an upstream bug.

Regards,
Peter Hurley

2015-04-12 13:04:14

by Fengguang Wu

[permalink] [raw]
Subject: Re: [serial] BUG: spinlock bad magic on CPU#0, swapper/0

On Sun, Apr 12, 2015 at 09:01:31AM -0400, Peter Hurley wrote:
> Hi Fengguang,
>
> On 04/12/2015 08:41 AM, Fengguang Wu wrote:
> > Greetings,
> >
> > 0day kernel testing robot got the below dmesg and the first bad commit is
> >
> > git://git.code.sf.net/p/openipmi/linux-ipmi master-ipmi-rebase
> >
> > commit ccfb59efb7a78e606516ce878d8dde4690419a5f
> > Author: Corey Minyard <[email protected]>
> > AuthorDate: Mon Mar 19 16:00:55 2012 -0500
> > Commit: Corey Minyard <[email protected]>
> > CommitDate: Fri Apr 10 20:51:46 2015 -0500
> >
> > serial: Add a polled interface to the 8250 driver
> >
> > Remove the console code from the 8250 driver and add the poll code and
> > setup to use the console over the polled interface.
> >
> > FIXME - The interactions between the 8250_early and 8250 code need to
> > be analyzed and tested.
> >
> > Signed-off-by: Corey Minyard <[email protected]>
> >
> > drivers/tty/serial/8250.c | 227 ++++++++++++++++++++++------------------------
> > 1 file changed, 112 insertions(+), 115 deletions(-)
>
> This is not an upstream bug.

Ah yes, sorry for the noise!

Thanks,
Fengguang