Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753926Ab3JJBlW (ORCPT ); Wed, 9 Oct 2013 21:41:22 -0400 Received: from mga11.intel.com ([192.55.52.93]:43272 "EHLO mga11.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752827Ab3JJBlU (ORCPT ); Wed, 9 Oct 2013 21:41:20 -0400 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="4.90,1068,1371106800"; d="scan'208";a="408595818" Date: Thu, 10 Oct 2013 09:41:17 +0800 From: Fengguang Wu To: Dave Chinner Cc: Ben Myers , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, xfs@oss.sgi.com Subject: Re: [XFS on bad superblock] BUG: unable to handle kernel NULL pointer dereference at 00000003 Message-ID: <20131010014117.GA6017@localhost> References: <20131009073910.GA387@localhost> <20131010005900.GE2025@devil.localdomain> <20131010011640.GA5726@localhost> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20131010011640.GA5726@localhost> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 7186 Lines: 119 On Thu, Oct 10, 2013 at 09:16:40AM +0800, Fengguang Wu wrote: > On Thu, Oct 10, 2013 at 11:59:00AM +1100, Dave Chinner wrote: > > [add xfs@oss.sgi.com to cc] > > Thanks. > > To help debug the problem, I searched XFS in my tests' oops database > and find one kernel that failed 4 times (out of 12 total boots) with > basically the same error: > > 4 BUG: sleeping function called from invalid context at kernel/workqueue.c:2810 > 1 WARNING: CPU: 1 PID: 372 at lib/debugobjects.c:260 debug_print_object+0x94/0xa2() > 1 WARNING: CPU: 1 PID: 360 at lib/debugobjects.c:260 debug_print_object+0x94/0xa2() > 1 WARNING: CPU: 0 PID: 381 at lib/debugobjects.c:260 debug_print_object+0x94/0xa2() > 1 WARNING: CPU: 0 PID: 361 at lib/debugobjects.c:260 debug_print_object+0x94/0xa2() And some other messages in an older kernel: [ 39.004416] F2FS-fs (nbd2): unable to read second superblock [ 39.005088] XFS: Assertion failed: read && bp->b_ops, file: fs/xfs/xfs_buf.c, line: 1036 [ 39.005089] ------------[ cut here ]------------ [ 39.005096] WARNING: CPU: 1 PID: 20 at fs/xfs/xfs_message.c:100 asswarn+0x33/0x40() [ 39.005099] CPU: 1 PID: 20 Comm: kworker/1:0H Not tainted 3.11.0-rc1-00667-gf70eb07 #64 [ 39.005100] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 39.005106] Workqueue: xfslogd xfs_buf_iodone_work [ 39.005110] 0000000000000009 ffff88000ed91ce0 ffffffff824044a1 0000000000000000 [ 39.005113] ffff88000ed91d18 ffffffff8109a0a8 ffff880007d6cea8 ffff880007d6cd80 [ 39.005116] 0000000000000000 0000000000000000 0000000000000060 ffff88000ed91d28 [ 39.005116] Call Trace: [ 39.005122] [] dump_stack+0x4e/0x82 [ 39.005126] [] warn_slowpath_common+0x78/0xa0 [ 39.005129] [] warn_slowpath_null+0x1a/0x20 [ 39.005131] [] asswarn+0x33/0x40 [ 39.005134] [] xfs_buf_iodone_work+0x92/0x200 [ 39.005138] [] process_one_work+0x1fa/0x6f0 [ 39.005141] [] ? process_one_work+0x198/0x6f0 [ 39.005144] [] worker_thread+0x11d/0x3a0 [ 39.005146] [] ? process_one_work+0x6f0/0x6f0 [ 39.005149] [] kthread+0xed/0x100 [ 39.005153] [] ? trace_hardirqs_on_caller+0xfd/0x1c0 [ 39.005156] [] ? insert_kthread_work+0x80/0x80 [ 39.005159] [] ret_from_fork+0x7c/0xb0 [ 39.005162] [] ? insert_kthread_work+0x80/0x80 [ 39.005164] ---[ end trace be5b205ae29d07cd ]--- [ 39.028229] block nbd6: Attempted send on closed socket [ 39.028988] block nbd6: Attempted send on closed socket [ 39.029109] block nbd6: Attempted send on closed socket [ 41.550438] block nbd7: Attempted send on closed socket [ 41.550471] ------------[ cut here ]------------ [ 41.550476] WARNING: CPU: 1 PID: 878 at lib/list_debug.c:33 __list_add+0xac/0xc0() [ 41.550478] list_add corruption. prev->next should be next (ffff88000f3d7360), but was (null). (prev=ffff880008786a30). [ 41.550481] CPU: 1 PID: 878 Comm: mount Not tainted 3.11.0-rc1-00667-gf70eb07 #64 [ 41.550482] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 41.550485] 0000000000000009 ffff880007d6fb08 ffffffff824044a1 ffff880007d6fb50 [ 41.550488] ffff880007d6fb40 ffffffff8109a0a8 ffff880007c6b530 ffff88000f3d7360 [ 41.550491] ffff880008786a30 0000000000000007 0000000000000000 ffff880007d6fba0 [ 41.550491] Call Trace: [ 41.550499] [] dump_stack+0x4e/0x82 [ 41.550503] [] warn_slowpath_common+0x78/0xa0 [ 41.550505] [] warn_slowpath_fmt+0x4c/0x50 [ 41.550509] [] ? get_lock_stats+0x19/0x60 [ 41.550511] [] __list_add+0xac/0xc0 [ 41.550515] [] insert_work+0x43/0xa0 [ 41.550518] [] __queue_work+0x11b/0x510 [ 41.550520] [] queue_work_on+0x96/0xa0 [ 41.550526] [] ? _xfs_buf_ioend.constprop.15+0x26/0x30 [ 41.550529] [] xfs_buf_ioend+0x15c/0x260 [ 41.550531] [] ? xfsbdstrat+0x22/0x170 [ 41.550534] [] _xfs_buf_ioend.constprop.15+0x26/0x30 [ 41.550537] [] xfs_buf_iorequest+0x73/0x1a0 [ 41.550539] [] xfsbdstrat+0x22/0x170 [ 41.550542] [] xfs_buf_read_uncached+0x72/0xa0 [ 41.550546] [] xfs_readsb+0x176/0x250 [ 41.550550] [] xfs_fs_fill_super+0x24a/0x3a0 [ 41.550553] [] mount_bdev+0x1c0/0x200 [ 41.550556] [] ? xfs_parseargs+0xc10/0xc10 [ 41.550559] [] ? lock_release_holdtime.part.29+0xe6/0x160 [ 41.550561] [] xfs_fs_mount+0x15/0x20 [ 41.550563] [] mount_fs+0x39/0x1b0 [ 41.550567] [] ? __alloc_percpu+0x10/0x20 [ 41.550571] [] vfs_kern_mount+0x63/0xf0 [ 41.550574] [] do_mount+0x23e/0xa20 [ 41.550577] [] ? strndup_user+0x4b/0x60 [ 41.550579] [] SyS_mount+0x83/0xc0 [ 41.550583] [] tracesys+0xdd/0xe2 [ 41.550584] ---[ end trace 5193f938804cbcca ]--- [ 41.550820] block nbd15: Attempted send on closed socket [ 38.738751] block nbd9: Attempted send on closed socket [ 38.738775] F2FS-fs (nbd9): unable to read first superblock [ 38.738852] block nbd9: Attempted send on closed socket [ 38.738875] F2FS-fs (nbd9): unable to read second superblock ffff8800001cbde0 ffffffff810b9ddf ffff8800081dfe18 [ 38.740053] ffff88000f1d3638 ffff88000f1d3240 ffff88000f1d3298 ffff8800000394c8 [ 38.740053] Call Trace: [ 38.740053] [] ? mutex_unlock+0xe/0x10 [ 38.740053] [] ? manage_workers.isra.28+0x1ef/0x2b0 [ 38.740053] [] worker_thread+0x11d/0x3a0 [ 38.740053] [] ? process_one_work+0x6f0/0x6f0 [ 38.740053] [] kthread+0xed/0x100 [ 38.740053] [] ? trace_hardirqs_on_caller+0xfd/0x1c0 [ 38.740053] [] ? insert_kthread_work+0x80/0x80 [ 38.740053] [] ret_from_fork+0x7c/0xb0 [ 38.740053] [] ? insert_kthread_work+0x80/0x80 [ 38.740053] Code: 57 41 56 41 55 45 31 ed 41 54 49 89 f4 53 48 89 fb 48 83 ec 48 48 8b 06 48 89 c2 30 d2 a8 04 48 8b 47 48 4c 0f 45 ea 48 89 45 a0 <49> 8b 45 08 48 c7 45 b0 00 00 00 00 48 c7 45 b8 00 00 00 00 44 [ 38.740053] RIP [] process_one_work+0x35/0x6f0 [ 38.740053] RSP [ 38.740053] CR2: 0000000000000008 [ 38.740053] ---[ end trace b1f41925f36484b0 ]--- [ 38.740053] BUG: sleeping function called from invalid context at kernel/rwsem.c:20 Thanks, Fengguang -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/