Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756202AbYHVTri (ORCPT ); Fri, 22 Aug 2008 15:47:38 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1752205AbYHVTr0 (ORCPT ); Fri, 22 Aug 2008 15:47:26 -0400 Received: from g1t0029.austin.hp.com ([15.216.28.36]:24542 "EHLO g1t0029.austin.hp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751474AbYHVTrY (ORCPT ); Fri, 22 Aug 2008 15:47:24 -0400 Message-ID: <48AF17C4.4060606@hp.com> Date: Fri, 22 Aug 2008 15:47:16 -0400 From: "Alan D. Brunelle" User-Agent: Thunderbird 2.0.0.16 (X11/20080724) MIME-Version: 1.0 To: =?ISO-8859-1?Q?Bj=F6rn_Steinbrink?= CC: Andrew Morton , Petr Baudis , linux-kernel@vger.kernel.org, git@vger.kernel.org Subject: Re: Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected References: <48A36838.3050309@hp.com> <20080819124602.9e8e69f7.akpm@linux-foundation.org> <48AEDD3D.4060507@hp.com> <20080822092549.ddcb7e79.akpm@linux-foundation.org> <20080822171651.GP10544@machine.or.cz> <20080822105136.a8432875.akpm@linux-foundation.org> <20080822193730.GA1598@atjola.homenet> In-Reply-To: <20080822193730.GA1598@atjola.homenet> Content-Type: multipart/mixed; boundary="------------020308070009090704040704" X-Brightmail-Tracker: AAAAAQAAAAI= X-Whitelist: TRUE Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 19296 Lines: 329 This is a multi-part message in MIME format. --------------020308070009090704040704 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Bj?rn Steinbrink wrote: > On 2008.08.22 10:51:36 -0700, Andrew Morton wrote: >> On Fri, 22 Aug 2008 19:16:51 +0200 Petr Baudis wrote: >>> On Fri, Aug 22, 2008 at 09:25:49AM -0700, Andrew Morton wrote: >>>> One (probably wrong) approach is to run >>>> >>>> gitk 1c89ac55017f982355c7761e1c912c88c941483d >>>> >>>> then peer at the output, work out which real commits were in that >>>> merge. >>>> >>>> It looks like the merge ended with >>>> b1b135c8d619cb2c7045d6ee4e48375882518bb5 and started with >>>> 40c42076ebd362dc69210cccea101ac80b6d4bd4, so perhaps you can do >>>> >>>> git bisect bad b1b135c8d619cb2c7045d6ee4e48375882518bb5 >>>> git bisect good 40c42076ebd362dc69210cccea101ac80b6d4bd4 >>> ...I don't quite get this - according to the bisection log, >>> >>> # good: [b1b135c8d619cb2c7045d6ee4e48375882518bb5] fix spinlock recursion in hvc_console >>> >>> and now you want to mark it as bad? >> > > Alan provided his bisection log as an attachment to the original bug > report. > >> I assume that Alan's bisection search ended up saying that the merge >> commit (1c89ac55017f982355c7761e1c912c88c941483d) was the first bad >> commit. > > Yep, and that's totally correct as far as bisect is concerned. The > parents of that merge commit are: > 88fa08f67bee1a0c765237bdac106a32872f57d2 > b1b135c8d619cb2c7045d6ee4e48375882518bb5 > > And Alan marked both of them as good. > > So, unless Alan made a mistake during his bisection, each of the > branches is correct, but the merge did not lead to a correct result. So > while there were no textual conflicts, there were still incompatible > changes regarding the code semantics and compatibility was not restored > during the merge. It's important to note that even if I did make a mistake during the bisection process (and I certainly wouldn't discount that), recent kernels still fail: but when I take out that commit from a recent kernel, it fails. I put in Andrew's suggested patch (to help find things), and now I repeatedly get the problems in the attached log. Not being an x86 knowledgeable person, I'm a bit concerned about the RSP value?! (I enabled stack overflow checking, but that didn't stop things.) Alan --------------020308070009090704040704 Content-Type: text/plain; name="new_prob.txt" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="new_prob.txt" Loading, please wait... Begin: Loading essential drivers... ... [ 6.604300] fuse init (API version 7.9) [ 6.625529] ACPI: SSDT CFFD0D0A, 08C4 (r1 HPQOEM CPU_TM2 1 MSFT 100000E) [ 6.631954] BUG: unable to handle kernel NULL pointer dereference at 0000000000000858 [ 6.640583] IP: [] debug_mutex_add_waiter+0x32/0x80 [ 6.640583] PGD 21a84f067 PUD 21ad55067 PMD 0 [ 6.642421] Oops: 0002 [1] SMP [ 6.642421] CPU 2 [ 6.642421] Modules linked in: processor(+) fan thermal_sys fuse [ 6.642421] Pid: 1259, comm: modprobe Not tainted 2.6.27-rc3 #26 [ 6.642421] RIP: 0010:[] [] debug_mutex_add_waiter+0x32/0x80 [ 6.642421] RSP: 0018:ffff88021a1a5998 EFLAGS: 00010002 [ 6.642421] RAX: 0000000000000000 RBX: ffff88021a1a59d8 RCX: 0000000000000000 [ 6.642421] RDX: 0000000000000001 RSI: ffff88021a1a59d8 RDI: ffffffffa0091a60 [ 6.642421] RBP: ffff88021a1a59b8 R08: ffffffff811deea0 R09: ffff8800a6fed000 [ 6.642421] R10: ffffffffa008f524 R11: 0000000000000002 R12: ffffffffa0091a60 [ 6.642421] R13: ffff88021a1a4000 R14: ffff88021a34c0a0 R15: ffffffffa0091a98 [ 6.642421] FS: 00007f2c513606e0(0000) GS:ffff88022fc02e00(0000) knlGS:0000000000000000 [ 6.642421] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 6.642421] CR2: 0000000000000858 CR3: 000000021a8f7000 CR4: 00000000000006e0 [ 6.642421] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 6.642421] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 6.642421] Process modprobe (pid: 1259, threadinfo ffff88021a1a4000, task ffff88021a34c0a0) [ 6.642421] Stack: 0000000000000000 ffffffffa0091a60 0000000000000246 ffffffffa008f524 [ 6.642421] ffff88021a1a5a38 ffffffff8049f556 ffffffffa008f524 ffffffffa0091a18 [ 6.642421] ffff88021a1a59d8 ffff88021a1a59d8 1111111111111111 1111111111111111 [ 6.642421] Call Trace: [ 6.642421] [] ? get_idr+0x44/0xa0 [thermal_sys] [ 6.642421] [] mutex_lock_nested+0xa6/0x250 [ 6.642421] [] ? get_idr+0x44/0xa0 [thermal_sys] [ 6.642421] [] ? idr_pre_get+0x44/0x90 [ 6.642421] [] get_idr+0x44/0xa0 [thermal_sys] [ 6.642421] [] thermal_cooling_device_register+0x83/0x250 [thermal_sys] [ 6.642421] [] acpi_processor_start+0x64b/0x774 [processor] [ 6.642421] [] ? __sysfs_add_one+0x6b/0xa0 [ 6.642421] [] ? sysfs_do_create_link+0xbc/0x150 [ 6.642421] [] acpi_start_single_object+0x2d/0x52 [ 6.642421] [] acpi_device_probe+0x7e/0x92 [ 6.642421] [] driver_probe_device+0x9b/0x1a0 [ 6.642421] [] __driver_attach+0x86/0x90 [ 6.642421] [] ? __driver_attach+0x0/0x90 [ 6.642421] [] bus_for_each_dev+0x5d/0x90 [ 6.642421] [] driver_attach+0x1c/0x20 [ 6.642421] [] bus_add_driver+0x1e9/0x260 [ 6.642421] [] ? acpi_processor_init+0x0/0x107 [processor] [ 6.642421] [] driver_register+0x5f/0x140 [ 6.642421] [] ? acpi_processor_init+0x0/0x107 [processor] [ 6.642421] [] acpi_bus_register_driver+0x3e/0x40 [ 6.642421] [] acpi_processor_init+0x94/0x107 [processor] [ 6.642421] [] _stext+0x40/0x180 [ 6.642421] [] ? __vunmap+0xa1/0x110 [ 6.642421] [] sys_init_module+0x142/0x1dc0 [ 6.642421] [] ? __up_read+0x46/0xb0 [ 6.642421] [] ? cpu_down+0x0/0x70 [ 6.642421] [] system_call_fastpath+0x16/0x1b [ 6.642421] [ 6.642421] [ 6.642421] Code: 20 48 89 5d e8 4c 89 65 f0 48 89 f3 4c 89 6d f8 8b 47 08 49 89 d5 49 89 fc 89 c2 25 ff ff 00 00 c1 ea 10 39 c2 74 1d 49 8b 4 [ 6.652421] RIP [] debug_mutex_add_waiter+0x32/0x80 [ 6.652421] RSP [ 6.652421] CR2: 0000000000000858 [ 6.652421] ---[ end trace 53d26a9a6cd6aa38 ]--- [ 6.991546] ------------[ cut here ]------------ [ 6.996188] kernel BUG at kernel/sched.c:1155! [ 7.005292] invalid opcode: 0000 [2] SMP [ 7.005292] CPU 2 [ 7.005292] Modules linked in: processor(+) fan thermal_sys fuse [ 7.011541] Pid: 1259, comm: modprobe Tainted: G D 2.6.27-rc3 #26 [ 7.011541] RIP: 0010:[] [] resched_task+0x6b/0x70 [ 7.011541] RSP: 0018:ffff88022f0efe98 EFLAGS: 00010046 [ 7.011541] RAX: 00000000000006ee RBX: ffff880028092e00 RCX: ffffffff81046000 [ 7.011541] RDX: 00000000000006ee RSI: ffff88021a34c0a0 RDI: ffffffff805ee660 [ 7.011541] RBP: ffff88022f0efe98 R08: 0000000000000000 R09: 0000000000000001 [ 7.011541] R10: 0000000000000000 R11: 0000000000000000 R12: ffff88021a34c0a0 [ 7.011541] R13: 0000000000000001 R14: ffff880028092e98 R15: ffff88021a34c0d8 [ 7.011541] FS: 00007f2c513606e0(0000) GS:ffff88022fc02e00(0000) knlGS:0000000000000000 [ 7.011541] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 7.011541] CR2: 0000000000000858 CR3: 000000021a8f7000 CR4: 00000000000006e0 [ 7.011541] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 7.011541] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 7.011541] Process modprobe (pid: 1259, threadinfo ffff88021a1a4000, task ffff88021a34c0a0) [ 7.011541] Stack: ffff88022f0efed8 ffffffff8023774f ffff88022f0efed8 ffff8800280b6e00 [ 7.011541] ffffffff80238510 ffff8800280b2b20 ffff8800280b2aa0 7fffffffffffffff [ 7.011541] ffff88022f0efef8 ffffffff8023855d ffff8800280b2aa0 ffff8800280b7660 [ 7.011541] Call Trace: [ 7.011541] [] task_tick_fair+0xbf/0xd0 [ 7.011541] [] ? hrtick+0x0/0x90 [ 7.011541] [] hrtick+0x4d/0x90 [ 7.011541] [] __run_hrtimer+0x95/0xb0 [ 7.011541] [] hrtimer_interrupt+0x11b/0x190 [ 7.011541] [] smp_apic_timer_interrupt+0x83/0xc0 [ 7.011541] [] apic_timer_interrupt+0x66/0x70 [ 7.011541] [] ? _spin_unlock_irq+0x2b/0x30 [ 7.011541] [] ? __down_read+0xa5/0xb7 [ 7.011541] [] ? acct_collect+0x45/0x1d0 [ 7.011541] [] ? down_read+0x37/0x40 [ 7.011541] [] ? acct_collect+0x45/0x1d0 [ 7.011541] [] ? acct_collect+0x45/0x1d0 [ 7.011541] [] ? do_exit+0x18d/0x9b0 [ 7.011541] [] ? do_unblank_screen+0x19/0x130 [ 7.011541] [] ? oops_end+0x87/0x90 [ 7.011541] [] ? do_page_fault+0x663/0x800 [ 7.011541] [] ? error_exit+0x0/0x9a [ 7.011541] [] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.011541] [] ? debug_mutex_add_waiter+0x32/0x80 [ 7.011541] [] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.011541] [] ? mutex_lock_nested+0xa6/0x250 [ 7.011541] [] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.011541] [] ? idr_pre_get+0x44/0x90 [ 7.011541] [] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.011541] [] ? thermal_cooling_device_register+0x83/0x250 [thermal_sys] [ 7.011541] [] ? acpi_processor_start+0x64b/0x774 [processor] [ 7.011541] [] ? __sysfs_add_one+0x6b/0xa0 [ 7.011541] [] ? sysfs_do_create_link+0xbc/0x150 [ 7.011541] [] ? acpi_start_single_object+0x2d/0x52 [ 7.011541] [] ? acpi_device_probe+0x7e/0x92 [ 7.011541] [] ? driver_probe_device+0x9b/0x1a0 [ 7.011541] [] ? __driver_attach+0x86/0x90 [ 7.011541] [] ? __driver_attach+0x0/0x90 [ 7.011541] [] ? bus_for_each_dev+0x5d/0x90 [ 7.011541] [] ? driver_attach+0x1c/0x20 [ 7.011541] [] ? bus_add_driver+0x1e9/0x260 [ 7.011541] [] ? acpi_processor_init+0x0/0x107 [processor] [ 7.011541] [] ? driver_register+0x5f/0x140 [ 7.011541] [] ? acpi_processor_init+0x0/0x107 [processor] [ 7.011541] [] ? acpi_bus_register_driver+0x3e/0x40 [ 7.011541] [] ? acpi_processor_init+0x94/0x107 [processor] [ 7.011541] [] ? _stext+0x40/0x180 [ 7.011541] [] ? __vunmap+0xa1/0x110 [ 7.011541] [] ? sys_init_module+0x142/0x1dc0 [ 7.011541] [] ? __up_read+0x46/0xb0 [ 7.011541] [] ? cpu_down+0x0/0x70 [ 7.011541] [] ? system_call_fastpath+0x16/0x1b [ 7.011541] [ 7.011541] [ 7.011541] Code: 8b 47 08 8b 50 1c 65 8b 04 25 24 00 00 00 39 c2 74 0d 0f ae f0 48 8b 47 08 f6 40 18 04 74 02 c9 c3 89 d7 ff 15 1f 7b 3c 00 c [ 7.021541] RIP [] resched_task+0x6b/0x70 [ 7.021541] RSP [ 7.021541] ---[ end trace 53d26a9a6cd6aa38 ]--- [ 7.021541] Kernel panic - not syncing: Aiee, killing interrupt handler! [ 7.021541] ------------[ cut here ]------------ [ 7.021541] WARNING: at kernel/smp.c:328 smp_call_function_mask+0x25a/0x260() [ 7.021541] Modules linked in: processor(+) fan thermal_sys fuse [ 7.021541] Pid: 1259, comm: modprobe Tainted: G D 2.6.27-rc3 #26 [ 7.021541] [ 7.021541] Call Trace: [ 7.021541] [] warn_on_slowpath+0x5f/0x80 [ 7.021541] [] smp_call_function_mask+0x25a/0x260 [ 7.021541] [] ? string+0x3d/0xd0 [ 7.021541] [] ? vsnprintf+0x43b/0x720 [ 7.021541] [] ? string+0x3d/0xd0 [ 7.021541] [] ? string+0x3d/0xd0 [ 7.021541] [] ? vsnprintf+0x43b/0x720 [ 7.021541] [] ? number+0x2ae/0x2d0 [ 7.021541] [] ? number+0x2ae/0x2d0 [ 7.021541] [] ? kallsyms_lookup+0x5d/0xa0 [ 7.021541] [] ? number+0x2ae/0x2d0 [ 7.021541] [] ? vsnprintf+0x43b/0x720 [ 7.021541] [] ? sprintf+0x68/0x70 [ 7.021541] [] ? string+0x3d/0xd0 [ 7.021541] [] ? __atomic_notifier_call_chain+0x83/0xa0 [ 7.021541] [] ? __atomic_notifier_call_chain+0x0/0xa0 [ 7.021541] [] ? _spin_unlock+0x26/0x30 [ 7.021541] [] ? stop_this_cpu+0x0/0x30 [ 7.021541] [] smp_call_function+0x40/0x50 [ 7.021541] [] native_smp_send_stop+0x23/0x40 [ 7.021541] [] panic+0xaf/0x190 [ 7.021541] [] ? printk+0x67/0x70 [ 7.021541] [] ? mutex_unlock+0x9/0x10 [ 7.021541] [] ? blocking_notifier_call_chain+0x11/0x20 [ 7.021541] [] do_exit+0x909/0x9b0 [ 7.021541] [] ? do_unblank_screen+0x19/0x130 [ 7.021541] [] oops_end+0x87/0x90 [ 7.021541] [] die+0x5e/0x90 [ 7.021541] [] do_trap+0x130/0x150 [ 7.021541] [] do_invalid_op+0x92/0xb0 [ 7.021541] [] ? resched_task+0x6b/0x70 [ 7.021541] [] ? sched_clock_cpu+0x14e/0x190 [ 7.021541] [] error_exit+0x0/0x9a [ 7.021541] [] ? resched_task+0x6b/0x70 [ 7.021541] [] task_tick_fair+0xbf/0xd0 [ 7.021541] [] ? hrtick+0x0/0x90 [ 7.021541] [] hrtick+0x4d/0x90 [ 7.021541] [] __run_hrtimer+0x95/0xb0 [ 7.021541] [] hrtimer_interrupt+0x11b/0x190 [ 7.021541] [] smp_apic_timer_interrupt+0x83/0xc0 [ 7.021541] [] apic_timer_interrupt+0x66/0x70 [ 7.021541] [] ? _spin_unlock_irq+0x2b/0x30 [ 7.021541] [] ? __down_read+0xa5/0xb7 [ 7.021541] [] ? acct_collect+0x45/0x1d0 [ 7.021541] [] ? down_read+0x37/0x40 [ 7.021541] [] ? acct_collect+0x45/0x1d0 [ 7.021541] [] ? acct_collect+0x45/0x1d0 [ 7.021541] [] ? do_exit+0x18d/0x9b0 [ 7.021541] [] ? do_unblank_screen+0x19/0x130 [ 7.021541] [] ? oops_end+0x87/0x90 [ 7.021541] [] ? do_page_fault+0x663/0x800 [ 7.021541] [] ? error_exit+0x0/0x9a [ 7.021541] [] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.021541] [] ? debug_mutex_add_waiter+0x32/0x80 [ 7.021541] [] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.021541] [] ? mutex_lock_nested+0xa6/0x250 [ 7.021541] [] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.021541] [] ? idr_pre_get+0x44/0x90 [ 7.021541] [] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.021541] [] ? thermal_cooling_device_register+0x83/0x250 [thermal_sys] [ 7.021541] [] ? acpi_processor_start+0x64b/0x774 [processor] [ 7.021541] [] ? __sysfs_add_one+0x6b/0xa0 [ 7.021541] [] ? sysfs_do_create_link+0xbc/0x150 [ 7.021541] [] ? acpi_start_single_object+0x2d/0x52 [ 7.021541] [] ? acpi_device_probe+0x7e/0x92 [ 7.021541] [] ? driver_probe_device+0x9b/0x1a0 [ 7.021541] [] ? __driver_attach+0x86/0x90 [ 7.021541] [] ? __driver_attach+0x0/0x90 [ 7.021541] [] ? bus_for_each_dev+0x5d/0x90 [ 7.021541] [] ? driver_attach+0x1c/0x20 [ 7.021541] [] ? bus_add_driver+0x1e9/0x260 [ 7.021541] [] ? acpi_processor_init+0x0/0x107 [processor] [ 7.021541] [] ? driver_register+0x5f/0x140 [ 7.021541] [] ? acpi_processor_init+0x0/0x107 [processor] [ 7.021541] [] ? acpi_bus_register_driver+0x3e/0x40 [ 7.021541] [] ? acpi_processor_init+0x94/0x107 [processor] [ 7.021541] [] ? _stext+0x40/0x180 [ 7.021541] [] ? __vunmap+0xa1/0x110 [ 7.021541] [] ? sys_init_module+0x142/0x1dc0 [ 7.021541] [] ? __up_read+0x46/0xb0 [ 7.021541] [] ? cpu_down+0x0/0x70 [ 7.021541] [] ? system_call_fastpath+0x16/0x1b [ 7.021541] [ 7.021541] ---[ end trace 53d26a9a6cd6aa38 ]--- [ 7.990001] BUG: spinlock lockup on CPU#0, swapper/0, ffff8800280b6e00 [ 7.990001] Pid: 0, comm: swapper Tainted: G D W 2.6.27-rc3 #26 [ 7.990001] [ 7.990001] Call Trace: [ 7.990001] [] _raw_spin_lock+0x18e/0x1a0 [ 7.990001] [] _spin_lock+0x34/0x40 [ 7.990001] [] ? sched_rt_period_timer+0x126/0x1e0 [ 7.990001] [] sched_rt_period_timer+0x126/0x1e0 [ 7.990001] [] ? sched_rt_period_timer+0x0/0x1e0 [ 7.990001] [] __run_hrtimer+0x95/0xb0 [ 7.990001] [] hrtimer_interrupt+0x11b/0x190 [ 7.990001] [] smp_apic_timer_interrupt+0x83/0xc0 [ 7.990001] [] apic_timer_interrupt+0x66/0x70 [ 7.990001] [] ? mwait_idle+0x40/0x50 [ 7.990001] [] ? enter_idle+0x22/0x30 [ 7.990001] [] ? cpu_idle+0x4c/0xc0 [ 7.990001] [] ? rest_init+0x61/0x70 [ 7.990001] CTRL-A Z for help |115200 8N1 | NOR | Minicom 2.3-rc | VT102 | Online 00:03 --------------020308070009090704040704-- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/