Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751938AbXLCVBF (ORCPT ); Mon, 3 Dec 2007 16:01:05 -0500 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1750975AbXLCVAz (ORCPT ); Mon, 3 Dec 2007 16:00:55 -0500 Received: from baron.coile-labs.com ([65.14.39.133]:16421 "EHLO coraid.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1750709AbXLCVAy (ORCPT ); Mon, 3 Dec 2007 16:00:54 -0500 Date: Mon, 3 Dec 2007 16:00:05 -0500 From: "Ed L. Cashin" To: Andrew Morton Cc: jnelson-kernel-bugzilla@jamponi.net, bugme-daemon@bugzilla.kernel.org, nickpiggin@yahoo.com.au, clameter@sgi.com, linux-kernel@vger.kernel.org, rjw@sisk.pl Subject: Re: [Bugme-new] [Bug 9482] New: kernel GPF in 2.6.24 (g09f345da) Message-ID: <20071203210005.GB27094@coraid.com> References: <20071201122302.10dd9c44.akpm@linux-foundation.org> <20071203162137.GB25251@coraid.com> <20071203113459.d36c1a01.akpm@linux-foundation.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20071203113459.d36c1a01.akpm@linux-foundation.org> User-Agent: Mutt/1.5.16 (2007-06-11) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 5635 Lines: 131 On Mon, Dec 03, 2007 at 11:34:59AM -0800, Andrew Morton wrote: ... > Strange. It _looks_ like we've somehow caused smp_processor_id() to return > a not-possible CPU number. ... > Could you debug this a bit please? Find out which CPU number > __percpu_counter_add() is using, for a start? I'd do: ... > Alternatively, just do > > if (!cpu_possible(cpu)) > printk(...) > > in __percpu_counter_add(). Then you can proceed to work through the > various operations which smp_processor_id() does and find out where it went > wrong: print out %fs, mainly. > > If the cpu number is valid then perhaps something scribbled on the cpu's > per-cpu memory. I'll keep looking at this, but at a glance it looks like the cpu number is valid, because I don't trip a BUG_ON when I make the change below (the badval variable is noise, sorry). --- lx/lib/percpu_counter.c.20071130 2007-12-03 15:43:19.000000000 -0500 +++ lx/lib/percpu_counter.c 2007-12-03 15:47:38.000000000 -0500 @@ -33,7 +33,9 @@ void __percpu_counter_add(struct percpu_ s64 count; s32 *pcount; int cpu = get_cpu(); + u64 badval = 0xffffffffffffffffULL; + BUG_ON(!cpu_possible(cpu)); pcount = per_cpu_ptr(fbc->counters, cpu); count = *pcount + amount; if (count >= batch || count <= -batch) { The trace is, Unable to handle kernel paging request at ffffffffffffffff RIP: [] __percpu_counter_add+0x35/0x7f PGD 203067 PUD 204067 PMD 0 Oops: 0000 [1] SMP CPU 0 Modules linked in: aoe Pid: 2777, comm: bash Not tainted 2.6.24-rc3-47dbg #9 RIP: 0010:[] [] __percpu_counter_add+0x35/0x7f RSP: 0018:ffff810078d19aa8 EFLAGS: 00010086 RAX: ffffffffffffffff RBX: ffff81007fc71950 RCX: 0000000000000010 RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff810078d9a250 RBP: ffff810078d19ac8 R08: ffff81007cc077b0 R09: ffffffff802ae5ee R10: ffff810078d19aa8 R11: ffff810078cb59d8 R12: ffff81007c81c380 R13: ffff810078d9a250 R14: 0000000000000200 R15: ffff81007805f830 FS: 00002b341db94db0(0000) GS:ffffffff8078b000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: ffffffffffffffff CR3: 000000007b415000 CR4: 00000000000006e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process bash (pid: 2777, threadinfo ffff810078d18000, task ffff810078d160c0) Stack: ffff810078d19ac8 ffff81007fc71950 ffff81007c81c380 0000000000000000 ffff810078d19af8 ffffffff802ae682 0000100078d19ae8 ffff81007805f830 0000000000000200 ffff81007fc71950 ffff810078d19b18 ffffffff802ae75c Call Trace: [] __set_page_dirty+0xdc/0x121 [] mark_buffer_dirty+0x95/0x99 [] __block_commit_write+0x72/0xac [] block_write_end+0x4f/0x5b [] blkdev_write_end+0x1b/0x38 [] generic_file_buffered_write+0x1c0/0x648 [] current_fs_time+0x22/0x29 [] __generic_file_aio_write_nolock+0x358/0x3c2 [] filemap_fault+0x1c4/0x320 [] unlock_page+0x2d/0x31 [] generic_file_aio_write_nolock+0x3b/0x8d [] do_sync_write+0xe2/0x126 [] autoremove_wake_function+0x0/0x38 [] do_page_fault+0x3f8/0x7bb [] fd_install+0x5f/0x68 [] vfs_write+0xae/0x137 [] sys_write+0x47/0x70 [] system_call+0x7e/0x83 Code: 4c 8b 24 d0 49 63 04 24 48 8d 1c 30 48 63 c1 48 39 c3 7d 0a RIP [] __percpu_counter_add+0x35/0x7f RSP CR2: ffffffffffffffff BUG: sleeping function called from invalid context at kernel/rwsem.c:20 in_atomic():0, irqs_disabled():1 INFO: lockdep is turned off. Call Trace: [] debug_show_held_locks+0x1b/0x24 [] __might_sleep+0xc7/0xc9 [] down_read+0x1d/0x4a [] exit_mm+0x34/0xf7 [] do_exit+0x247/0x75b [] do_page_fault+0x6c7/0x7bb [] thread_return+0x42/0x86 [] :aoe:aoeblk_make_request+0x1c3/0x1d0 [] error_exit+0x0/0x9a [] __set_page_dirty+0x48/0x121 [] __percpu_counter_add+0x35/0x7f [] __set_page_dirty+0xdc/0x121 [] mark_buffer_dirty+0x95/0x99 [] __block_commit_write+0x72/0xac [] block_write_end+0x4f/0x5b [] blkdev_write_end+0x1b/0x38 [] generic_file_buffered_write+0x1c0/0x648 [] current_fs_time+0x22/0x29 [] __generic_file_aio_write_nolock+0x358/0x3c2 [] filemap_fault+0x1c4/0x320 [] unlock_page+0x2d/0x31 [] generic_file_aio_write_nolock+0x3b/0x8d [] do_sync_write+0xe2/0x126 [] autoremove_wake_function+0x0/0x38 [] do_page_fault+0x3f8/0x7bb [] fd_install+0x5f/0x68 [] vfs_write+0xae/0x137 [] sys_write+0x47/0x70 [] system_call+0x7e/0x83 -- Ed L Cashin -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/