Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S964984AbXH3TGB (ORCPT ); Thu, 30 Aug 2007 15:06:01 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S932865AbXH3TFx (ORCPT ); Thu, 30 Aug 2007 15:05:53 -0400 Received: from li6-103.members.linode.com ([66.160.141.103]:4405 "EHLO smtp.falooley.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932657AbXH3TFw (ORCPT ); Thu, 30 Aug 2007 15:05:52 -0400 X-Greylist: delayed 2497 seconds by postgrey-1.27 at vger.kernel.org; Thu, 30 Aug 2007 15:05:52 EDT Date: Thu, 30 Aug 2007 11:23:55 -0700 From: Jason Lunz To: jffs-dev@axis.com Cc: David Woodhouse , stable@kernel.org, lkml Subject: jffs2 deadlock introduced in linux 2.6.22.5 Message-ID: <20070830182354.GA25077@falooley.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.16 (2007-06-11) Sender: linux-kernel-owner@vger.kernel.org X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3726 Lines: 83 commit 1d8715b388c978b0f1b1bf4812fcee0e73b023d7 was added between 2.6.22.4 and 2.6.22.5 to cure a locking problem, but it seems to have introduced another (worse?) one. With a jffs2 filesystem (on block2mtd) on a 2.6.22.5 kernel, if I do anything that appends to a file with many small writes, I get what looks like a deadlock between the writer and the jffs2 gc thread. For example: # while true; do echo >> /some/file/on/jffs2; done will result in the bash hanging in D state, with these kernel stacks in dmesg after "echo t > /proc/sysrq-trigger": jffs2_gcd_mtd S DFD1EEA8 0 1086 2 (L-TLB) dfd1eebc 00000046 00000002 dfd1eea8 dfd1eea4 00000000 00000000 c0334a00 c0334a00 00000000 0000000a dfcb8550 2ee3df10 0000001a 00002280 dfcb8670 c1407a00 00000000 00000286 df9fa600 dfe20900 ffff414a c1407ec4 0000ffff Call Trace: [] __down_interruptible+0xb2/0x10b [] __sched_text_start+0x14b/0x8a4 [] default_wake_function+0x0/0xc [] __down_failed_interruptible+0x7/0xc [] jffs2_garbage_collect_pass+0x20/0x597 [jffs2] [] __dequeue_signal+0xd7/0x11c [] recalc_sigpending+0xb/0x1d [] dequeue_signal+0x9d/0x117 [] jffs2_garbage_collect_thread+0x11b/0x15a [jffs2] [] ret_from_fork+0x6/0x1c [] jffs2_garbage_collect_thread+0x0/0x15a [jffs2] [] jffs2_garbage_collect_thread+0x0/0x15a [jffs2] [] kernel_thread_helper+0x7/0x10 bash D CE2C85E0 0 2223 2219 (NOTLB) d834bb2c 00000086 00000000 ce2c85e0 ce2c85e0 ce8004c0 00000003 c0334a00 c0334a00 00000000 00000009 df93ca70 2ee3bc90 0000001a 0002ff74 df93cb90 c1407a00 00000000 00000000 00000000 dfe20900 00000000 c1407ec4 00000000 Call Trace: [] io_schedule+0x1e/0x28 [] sync_page+0x38/0x3b [] __wait_on_bit+0x33/0x58 [] sync_page+0x0/0x3b [] wait_on_page_bit+0x63/0x69 [] wake_bit_function+0x0/0x3c [] read_cache_page+0x28/0x3f [] jffs2_gc_fetch_page+0x26/0x3b [jffs2] [] jffs2_garbage_collect_live+0x992/0xf87 [jffs2] [] block2mtd_write+0x18f/0x1a6 [block2mtd] [] default_mtd_writev+0x0/0x9e [mtdcore] [] jffs2_flash_direct_writev+0x62/0xd0 [jffs2] [] jffs2_garbage_collect_pass+0x4ff/0x597 [jffs2] [] aufs_read_unlock+0x17/0x5f [aufs] [] ibend+0x39/0x3f [aufs] [] jffs2_reserve_space+0xb5/0x15b [jffs2] [] send_sig_info+0x55/0x65 [] jffs2_write_inode_range+0x5a/0x278 [jffs2] [] jffs2_commit_write+0xec/0x1be [jffs2] [] generic_file_buffered_write+0x3f1/0x5af [] dput+0x15/0xda [] __link_path_walk+0xb2d/0xc0e [] current_fs_time+0x41/0x46 [] __generic_file_aio_write_nolock+0x479/0x4c8 [] link_path_walk+0xa9/0xb3 [] generic_file_aio_write+0x61/0xc2 [] permission+0xc8/0xdb [] do_sync_write+0x0/0x10a [] do_sync_write+0xc7/0x10a [] open_namei+0x254/0x571 [] autoremove_wake_function+0x0/0x33 [] do_sync_write+0x0/0x10a [] vfs_write+0xa8/0x130 [] sys_write+0x41/0x67 [] syscall_call+0x7/0xb Given that I never saw any jffs2 deadlocks in other 2.6.22 kernels, maybe that commit should be reverted until a better solution is found? Jason - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/