Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751398AbVKIOs0 (ORCPT ); Wed, 9 Nov 2005 09:48:26 -0500 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1751402AbVKIOs0 (ORCPT ); Wed, 9 Nov 2005 09:48:26 -0500 Received: from sccrmhc11.comcast.net ([204.127.202.55]:29942 "EHLO sccrmhc11.comcast.net") by vger.kernel.org with ESMTP id S1751398AbVKIOsZ (ORCPT ); Wed, 9 Nov 2005 09:48:25 -0500 Date: Wed, 9 Nov 2005 08:48:03 -0600 From: Lee To: linuxtwidler@gmail.com Cc: Linux Kernel Subject: Re: kernel crash debugging Message-ID: <20051109084803.021b83f7@localhost> In-Reply-To: <20051109082007.07764fdf@localhost> References: <20051109082007.07764fdf@localhost> Reply-To: linuxtwidler@gmail.com X-Mailer: Sylpheed-Claws 1.0.5 (GTK+ 1.2.10; i686-pc-linux-gnu) Mime-Version: 1.0 Content-Type: multipart/mixed; boundary=Multipart_Wed__9_Nov_2005_08_48_03_-0600_vF8jTNtL_9PNWUgv Sender: linux-kernel-owner@vger.kernel.org X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 6671 Lines: 162 --Multipart_Wed__9_Nov_2005_08_48_03_-0600_vF8jTNtL_9PNWUgv Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Disposition: inline > Can anyone advise on the following: > - determining why the kernel is locking up > - locating where in the code is causing the lock up > - alternative methods of debugging > - increasing the console log level After booting my kernel with the options 'debug loglevel=9' I was able to get proper OOPS reports from the kernel on the serial console, which I have attached to this email. Please advise on how I can help debug / fix this problem. Thanks Lee -- Lee linuxtwidler@gmail.com 08:45:44 up 2 days, 13:56, 1 user, load average: 0.00, 0.00, 0.00 --Multipart_Wed__9_Nov_2005_08_48_03_-0600_vF8jTNtL_9PNWUgv Content-Type: text/plain; name=kernel-2.6.13-gentoo-r3-crash.txt Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=kernel-2.6.13-gentoo-r3-crash.txt Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: CPU: 1 EIP: 0060:[] Not tainted VLI EFLAGS: 00010082 (2.6.13-gentoo-r3-20051108-1407) EIP is at do_page_fault+0xcb/0x6bf eax: f7f08000 ebx: 0000000b ecx: 0000000d edx: 00000000 esi: 0000000e edi: c04689a2 ebp: f7f08110 esp: f7f08040 ds: 007b es: 007b ss: 0068 Process khubd (pid: 16, threadinfo=f7f07000 task=f7f06ac0) Stack: c05b904c 0000000d f7f08064 00000000 00000000 0000008c 00000000 00000000 f7f08118 f7f08118 c04689a2 00000000 0000000e 0000000b 00000000 00000000 00000000 00000000 00000000 00030001 00000000 00000000 00000000 00000000 Call Trace: ======================= Unable to handle kernel paging request at virtual address fffff030 printing eip: c0103e83 *pde = 0065e067 Oops: 0000 [#2] SMP DEBUG_PAGEALLOC Modules linked in: CPU: 1 EIP: 0060:[] Not tainted VLI EFLAGS: 00010083 (2.6.13-gentoo-r3-20051108-1407) EIP is at show_trace+0x83/0xc0 eax: 0000001c ebx: f7f07e94 ecx: fffffd32 edx: 00000001 esi: f7f07e94 edi: fffff000 ebp: f7f07e94 esp: f7f07e80 ds: 007b es: 007b ss: 0068 Process khubd (pid: 16, threadinfo=f7f07000 task=f7f06ac0) Stack: c0466ce5 f7f07e94 f7f080a0 00000018 00000000 f7f07eb0 c0103f3f 00000000 f7f08040 f7f07000 f7f0800c 00000000 f7f07ee8 c01040f0 00000000 f7f08040 00000010 f7f07000 f7f06ac0 00010082 c04cf322 00000068 00000001 f7f07000 Call Trace: [] show_stack+0x7f/0xa0 [] show_registers+0x160/0x1d0 [] die+0x100/0x180 [] do_page_fault+0x394/0x6bf [] error_code+0x4f/0x54 [] show_stack+0x7f/0xa0 [] show_registers+0x160/0x1d0 [] die+0x100/0x180 [] do_page_fault+0x394/0x6bf Code: 89 5c 24 04 e8 9f d1 01 00 89 5c 24 04 c7 04 24 7a fb 48 c0 e8 1f aa 03 00 c7 04 24 23 eb 46 c0 e8 83 d1 01 00 8b 36 eb bf 89 f3 <8b> 77 30 85 f6 74 0e c7 04 24 e5 6c 46 c0 e8 6a d1 01 00 eb 92 eip: c012b43b ------------[ cut here ]------------ kernel BUG at include/asm/spinlock.h:136! invalid operand: 0000 [#3] SMP DEBUG_PAGEALLOC Modules linked in: CPU: 1 EIP: 0060:[] Not tainted VLI EFLAGS: 00010096 (2.6.13-gentoo-r3-20051108-1407) EIP is at _spin_lock+0x3c/0x50 eax: 0000000e ebx: f7f08ff0 ecx: c04d1cec edx: 00000086 esi: f7f06ac0 edi: f7f08ff0 ebp: f7f07c80 esp: f7f07c74 ds: 007b es: 007b ss: 0068 Process khubd (pid: 16, threadinfo=f7f07000 task=f7f06ac0) Stack: c046955d c012b43b f7f09ea0 f7f07c9c c012b43b f7f07c9c c019b3d2 f7f06ac0 00000020 f7f06f94 f7f07cc0 c01224d9 f7f06ac0 00000001 f2b24f68 f7f07cd0 f7f07ce8 00000020 c1da8ac0 f7f07cfc c01234a7 f7f06ac0 00000011 c01440cc Call Trace: [] show_stack+0x7f/0xa0 [] show_registers+0x160/0x1d0 [] die+0x100/0x180 [] do_trap+0xb5/0xc0 [] do_invalid_op+0xbc/0xd0 [] error_code+0x4f/0x54 [] __exit_signal+0x3b/0x180 [] release_task+0x69/0x160 [] exit_notify+0x3c7/0x8d0 [] do_exit+0x1ff/0x3f0 [] do_trap+0x0/0xc0 [] do_page_fault+0x394/0x6bf [] error_code+0x4f/0x54 [] show_stack+0x7f/0xa0 [] show_registers+0x160/0x1d0 [] die+0x100/0x180 [] do_page_fault+0x394/0x6bf Code: de 75 15 f0 fe 0b 79 09 f3 90 80 3b 00 7e f9 eb f2 8b 5d fc 89 ec 5d c3 c7 04 24 5d 95 46 c0 8b 45 04 89 44 24 04 e8 24 1c cd ff <0f> 0b 88 00 b5 8d 46 c0 eb ce 8d 76 00 8d bc 27 00 00 00 00 81 <1>Fixing recursive fault but reboot is needed! Unable to handle kernel NULL pointer dereference at virtual address 00000000 printing eip: c0119cb6 *pde = 00000000 Oops: 0002 [#4] SMP DEBUG_PAGEALLOC Modules linked in: CPU: 1 EIP: 0060:[] Not tainted VLI EFLAGS: 00010086 (2.6.13-gentoo-r3-20051108-1407) EIP is at dequeue_task+0x16/0x50 eax: 00000000 ebx: f7f06ae0 ecx: f7f06ac0 edx: f7f06ac0 esi: 00000000 edi: f7f06ac0 ebp: f7f07a58 esp: f7f07a50 ds: 007b es: 007b ss: 0068 Process khubd (pid: 16, threadinfo=f7f07000 task=f7f06ac0) Stack: f7f06ac0 c1c13400 f7f07a6c c0119ff1 f7f06ac0 00000000 c1c13400 f7f07adc c044d5d7 f7f06ac0 c1c13400 00000286 f7f07a7c f7f06ac0 00000001 00000000 f7f07ab0 c012a4e9 f7f06ac0 00000046 c05b28cc 00000001 c1c13400 00000246 Call Trace: [] show_stack+0x7f/0xa0 [] show_registers+0x160/0x1d0 [] die+0x100/0x180 [] do_page_fault+0x394/0x6bf [] error_code+0x4f/0x54 [] deactivate_task+0x21/0x30 [] schedule+0x157/0xd90 [] do_exit+0x371/0x3f0 [] do_trap+0x0/0xc0 [] do_trap+0xb5/0xc0 [] do_invalid_op+0xbc/0xd0 [] error_code+0x4f/0x54 [] __exit_signal+0x3b/0x180 [] release_task+0x69/0x160 [] exit_notify+0x3c7/0x8d0 [] do_exit+0x1ff/0x3f0 [] do_trap+0x0/0xc0 [] do_page_fault+0x394/0x6bf [] error_code+0x4f/0x54 [] show_stack+0x7f/0xa0 [] show_registers+0x160/0x1d0 [] die+0x100/0x180 [] do_page_fault+0x394/0x6bf Code: 8b 7d fc 89 ec 5d c3 89 1c 24 e8 56 6a 03 00 eb e7 8d 74 26 00 55 89 e5 83 ec 08 89 74 24 04 8b 55 08 8b 75 0c 89 1c 24 8d 5a 20 0e 8b 42 20 8b 4b 04 89 01 89 48 04 c7 43 04 00 02 20 00 8b --Multipart_Wed__9_Nov_2005_08_48_03_-0600_vF8jTNtL_9PNWUgv-- - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/