(I am cc'ing linux-kernel)
刘生平 wrote:
> Hi Guys,
> I have done several experiments on Apple Xserver G5(using kernel config files: linux-2.6-torvalds/arch/powerpc/configs/g5_defconfig)
> and on moto_acta6101, found kmemtrace can cause kernel panic during system startup when enabling slab or slub except slob, I think it only happened on PPC64 platform, the kernel panic message as follows:
> Console: colour dummy device 80x25�console handover: boot [udbg0] -> real [ttyPZ0]
> Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes)
> Inode-cache hash table entries: 65536 (order: 7, 524288 bytes)
> Memory: 1007104k/1048576k available (5152k kernel code, 40816k reserved, 712k data, 208k bss, 236k init)
> Unable to handle kernel paging request for data at address 0x6000000000000000
> Faulting instruction address: 0xc0000000000cb5d0
> Oops: Kernel access of bad area, sig: 11 [#1]
> PREEMPT LTT NESTING LEVEL : 0
> PowerMac
> Modules linked in:
> NIP: c0000000000cb5d0 LR: c0000000000cb5c4 CTR: 0000000000000000
> REGS: c0000000005bf800 TRAP: 0300 Not tainted (2.6.27.11-WR3.0zz_standard)
> MSR: 9000000000009032 <EE,ME,IR,DR> CR: 84000084 XER: 00000000
> DAR: 6000000000000000, DSISR: 0000000008000000
> TASK = c00000000052dd50[0] 'swapper' THREAD: c0000000005bc000
> GPR00: c0000000000cb5c4 c0000000005bfa80 c0000000005bf8e0 6000000000000000
> GPR04: 00000000000004d0 d000000000000000 c000000001540068 0000000000000000
> GPR08: 0000000000000000 0000000000000000 0000000000000000 0000000800000000
> GPR12: 0000000084000084 c0000000005cd300 c00000000052da20 0000000000000391
> GPR16: d000000000005000 d000000000004fff 000000000023f3b8 c00000000045fbe8
> GPR20: c0000000005c9000 d000000000005000 c00000003d006e00 c00000000050dfe0
> GPR24: c0000000005bfd40 0000000000000391 c0000000000ade04 00000000000004d0
> GPR28: 0000000000000000 0000000000000001 c000000000562828 0000000000000005
> NIP [c0000000000cb5d0] .kmem_cache_alloc+0x50/0x180
> LR [c0000000000cb5c4] .kmem_cache_alloc+0x44/0x180
> Call Trace:
> [c0000000005bfa80] [c0000000000cb5c4] .kmem_cache_alloc+0x44/0x180 (unreliable)
> [c0000000005bfb30] [c0000000000ade04] .__pud_alloc+0x34/0xe0
> [c0000000005bfbc0] [c0000000000bc550] .map_vm_area+0x320/0x350
> [c0000000005bfcd0] [c0000000000bccf0] .__vmalloc_area_node+0x100/0x1d0
> [c0000000005bfd90] [c000000000092958] .relay_open+0x3a8/0x580
> [c0000000005bfe70] [c0000000000d114c] .kmemtrace_init+0x4c/0xe0
> [c0000000005bfef0] [c0000000004cd8a4] .start_kernel+0x2cc/0x410
> [c0000000005bff90] [c0000000000073a4] .start_here_common+0x3c/0x98
> Instruction dump:
> f8010010 ebc29180 f821ff51 7c7c1b78 7c9b2378 38000000 8bad01ca 980d01ca
> 481418e1 60000000 78631f24 7c7c182a <81230000> 2f890000 419e0108 3929ffff
> ---[ end trace 31fd0ba7d8756001 ]---
> Kernel panic - not syncing: Attempted to kill the idle task!
> Rebooting in 180 seconds..
> Apple RackMac3,1 5.1.7f1 BootROM built on 08/25/04 at 15:11:09
> Copyright 1994-2004 Apple Computer, Inc.
> All Rights Reserved.
>
> Welcome to Open Firmware, the system time and date is: 06:44:18 07/27/1970
>
> To continue booting, type "mac-boot" and press return.
> To shut down, type "shut-down" and press return.
>
>
> what's your suggestion?
Hmm, I can't really work out what exactly is going wrong there. I
suspect that the fact that ppc64 uses kmem_cache_alloc() for
__pud_alloc() -> pud_alloc_one() whereas x86 uses the page allocator is
the culprit here.
Eduard, thoughts?
Pekka
On Wed, 2009-01-21 at 14:01 +0200, Pekka Enberg wrote:
> (I am cc'ing linux-kernel)
>
> 刘生平 wrote:
> > Hi Guys,
> > I have done several experiments on Apple Xserver G5(using kernel config files: linux-2.6-torvalds/arch/powerpc/configs/g5_defconfig)
> > and on moto_acta6101, found kmemtrace can cause kernel panic during system startup when enabling slab or slub except slob, I think it only happened on PPC64 platform, the kernel panic message as follows:
> > Console: colour dummy device 80x25�console handover: boot [udbg0] -> real [ttyPZ0]
> > Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes)
> > Inode-cache hash table entries: 65536 (order: 7, 524288 bytes)
> > Memory: 1007104k/1048576k available (5152k kernel code, 40816k reserved, 712k data, 208k bss, 236k init)
> > Unable to handle kernel paging request for data at address 0x6000000000000000
> > Faulting instruction address: 0xc0000000000cb5d0
> > Oops: Kernel access of bad area, sig: 11 [#1]
> > PREEMPT LTT NESTING LEVEL : 0
> > PowerMac
> > Modules linked in:
> > NIP: c0000000000cb5d0 LR: c0000000000cb5c4 CTR: 0000000000000000
> > REGS: c0000000005bf800 TRAP: 0300 Not tainted (2.6.27.11-WR3.0zz_standard)
> > MSR: 9000000000009032 <EE,ME,IR,DR> CR: 84000084 XER: 00000000
> > DAR: 6000000000000000, DSISR: 0000000008000000
> > TASK = c00000000052dd50[0] 'swapper' THREAD: c0000000005bc000
> > GPR00: c0000000000cb5c4 c0000000005bfa80 c0000000005bf8e0 6000000000000000
> > GPR04: 00000000000004d0 d000000000000000 c000000001540068 0000000000000000
> > GPR08: 0000000000000000 0000000000000000 0000000000000000 0000000800000000
> > GPR12: 0000000084000084 c0000000005cd300 c00000000052da20 0000000000000391
> > GPR16: d000000000005000 d000000000004fff 000000000023f3b8 c00000000045fbe8
> > GPR20: c0000000005c9000 d000000000005000 c00000003d006e00 c00000000050dfe0
> > GPR24: c0000000005bfd40 0000000000000391 c0000000000ade04 00000000000004d0
> > GPR28: 0000000000000000 0000000000000001 c000000000562828 0000000000000005
> > NIP [c0000000000cb5d0] .kmem_cache_alloc+0x50/0x180
> > LR [c0000000000cb5c4] .kmem_cache_alloc+0x44/0x180
> > Call Trace:
> > [c0000000005bfa80] [c0000000000cb5c4] .kmem_cache_alloc+0x44/0x180 (unreliable)
> > [c0000000005bfb30] [c0000000000ade04] .__pud_alloc+0x34/0xe0
> > [c0000000005bfbc0] [c0000000000bc550] .map_vm_area+0x320/0x350
> > [c0000000005bfcd0] [c0000000000bccf0] .__vmalloc_area_node+0x100/0x1d0
> > [c0000000005bfd90] [c000000000092958] .relay_open+0x3a8/0x580
> > [c0000000005bfe70] [c0000000000d114c] .kmemtrace_init+0x4c/0xe0
> > [c0000000005bfef0] [c0000000004cd8a4] .start_kernel+0x2cc/0x410
> > [c0000000005bff90] [c0000000000073a4] .start_here_common+0x3c/0x98
> > Instruction dump:
> > f8010010 ebc29180 f821ff51 7c7c1b78 7c9b2378 38000000 8bad01ca 980d01ca
> > 481418e1 60000000 78631f24 7c7c182a <81230000> 2f890000 419e0108 3929ffff
> > ---[ end trace 31fd0ba7d8756001 ]---
> > Kernel panic - not syncing: Attempted to kill the idle task!
> > Rebooting in 180 seconds..
> > Apple RackMac3,1 5.1.7f1 BootROM built on 08/25/04 at 15:11:09
> > Copyright 1994-2004 Apple Computer, Inc.
> > All Rights Reserved.
> >
> > Welcome to Open Firmware, the system time and date is: 06:44:18 07/27/1970
> >
> > To continue booting, type "mac-boot" and press return.
> > To shut down, type "shut-down" and press return.
> >
> >
> > what's your suggestion?
>
> Hmm, I can't really work out what exactly is going wrong there. I
> suspect that the fact that ppc64 uses kmem_cache_alloc() for
> __pud_alloc() -> pud_alloc_one() whereas x86 uses the page allocator is
> the culprit here.
Looks to me as if kmemtrace is in the middle of being initialized at
this point, unexpectedly calls into slab, and slab is tripping over the
half-initialized tracing structure.
When you say 'except slob', does that mean slob worked or slob was
untested?
--
http://selenic.com : development and support for Mercurial and Linux
On Wed, Jan 21, 2009 at 02:01:55PM +0200, Pekka Enberg wrote:
> (I am cc'ing linux-kernel)
>
>
> Hmm, I can't really work out what exactly is going wrong there. I suspect
> that the fact that ppc64 uses kmem_cache_alloc() for __pud_alloc() ->
> pud_alloc_one() whereas x86 uses the page allocator is the culprit here.
>
> Eduard, thoughts?
>
> Pekka
Couldn't it be the same problem Ingo ran into? This is the relay
version, as far as I can tell. I'd suggest trying the ftrace version of
kmemtrace. In any case, it has been more recently updated.
http://git.kernel.org/?p=linux/kernel/git/tip/linux-2.6-tip.git
Eduard