Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756785AbZLNLBA (ORCPT ); Mon, 14 Dec 2009 06:01:00 -0500 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1756748AbZLNLA7 (ORCPT ); Mon, 14 Dec 2009 06:00:59 -0500 Received: from mail-fx0-f213.google.com ([209.85.220.213]:48707 "EHLO mail-fx0-f213.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756723AbZLNLA6 (ORCPT ); Mon, 14 Dec 2009 06:00:58 -0500 DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:cc:subject:date:message-id:x-mailer:in-reply-to:references; b=JJQ1wptFuszKso/FvkpmDj+h2ziyW0v7MALEEskcLxEtyxUztm574Qaf1Xxp8GEWYu 7ZDHT/NTckwBhBYWJr1WyKlQV3aXAILja8bH9fA7TxGJzH5VzYH4R8MAG9tPTaqVLsg4 dXC0oRsqnw8hLZBiS9uu8CvCkVMq243AhUo+o= From: Frederic Weisbecker To: Alexander Beregalov Cc: LKML , Frederic Weisbecker , Alexander Beregalov , Chris Mason , Thomas Gleixner , Ingo Molnar Subject: [PATCH 2/2] reiserfs: Fix reiserfs lock and journal lock inversion dependency Date: Mon, 14 Dec 2009 12:00:54 +0100 Message-Id: <1260788454-5683-1-git-send-regression-fweisbec@gmail.com> X-Mailer: git-send-email 1.6.2.3 In-Reply-To: References: Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 6059 Lines: 152 When we were using the bkl, we didn't care about dependencies against other locks, but the mutex conversion created new ones, which is why we have reiserfs_mutex_lock_safe(), which unlocks the reiserfs lock before acquiring another mutex. But this trick actually fails if we have acquired the reiserfs lock recursively, as we try to unlock it to acquire the new mutex without inverted dependency, but we eventually only decrease its depth. This happens in the case of a nested inode creation/deletion. Say we have no space left on the device, we create an inode and tak the lock but fail to create its entry, then we release the inode using iput(), which calls reiserfs_delete_inode() that takes the reiserfs lock recursively. The path eventually ends up in journal_begin() where we try to take the journal safely but we fail because of the reiserfs lock recursion: [ INFO: possible circular locking dependency detected ] 2.6.32-06486-g053fe57 #2 ------------------------------------------------------- vi/23454 is trying to acquire lock: (&journal->j_mutex){+.+...}, at: [] do_journal_begin_r+0x64/0x2f0 but task is already holding lock: (&REISERFS_SB(s)->lock){+.+.+.}, at: [] reiserfs_write_lock+0x28/0x40 which lock already depends on the new lock. the existing dependency chain (in reverse order) is: -> #1 (&REISERFS_SB(s)->lock){+.+.+.}: [] validate_chain+0xa23/0xf70 [] __lock_acquire+0x4e5/0xa70 [] lock_acquire+0x7a/0xa0 [] mutex_lock_nested+0x5f/0x2b0 [] reiserfs_write_lock+0x28/0x40 [] do_journal_begin_r+0x6b/0x2f0 [] journal_begin+0x7f/0x120 [] reiserfs_remount+0x212/0x4d0 [] do_remount_sb+0x67/0x140 [] do_mount+0x436/0x6b0 [] sys_mount+0x66/0xa0 [] sysenter_do_call+0x12/0x36 -> #0 (&journal->j_mutex){+.+...}: [] validate_chain+0xf68/0xf70 [] __lock_acquire+0x4e5/0xa70 [] lock_acquire+0x7a/0xa0 [] mutex_lock_nested+0x5f/0x2b0 [] do_journal_begin_r+0x64/0x2f0 [] journal_begin+0x7f/0x120 [] reiserfs_delete_inode+0x9f/0x140 [] generic_delete_inode+0x9c/0x150 [] generic_drop_inode+0x3d/0x60 [] iput+0x47/0x50 [] reiserfs_create+0x16c/0x1c0 [] vfs_create+0xc1/0x130 [] do_filp_open+0x81c/0x920 [] do_sys_open+0x4f/0x110 [] sys_open+0x29/0x40 [] sysenter_do_call+0x12/0x36 other info that might help us debug this: 2 locks held by vi/23454: #0: (&sb->s_type->i_mutex_key#5){+.+.+.}, at: [] do_filp_open+0x27e/0x920 #1: (&REISERFS_SB(s)->lock){+.+.+.}, at: [] reiserfs_write_lock+0x28/0x40 stack backtrace: Pid: 23454, comm: vi Not tainted 2.6.32-06486-g053fe57 #2 Call Trace: [] ? printk+0x18/0x1e [] print_circular_bug+0xc0/0xd0 [] validate_chain+0xf68/0xf70 [] ? trace_hardirqs_off+0xb/0x10 [] __lock_acquire+0x4e5/0xa70 [] lock_acquire+0x7a/0xa0 [] ? do_journal_begin_r+0x64/0x2f0 [] mutex_lock_nested+0x5f/0x2b0 [] ? do_journal_begin_r+0x64/0x2f0 [] ? do_journal_begin_r+0x64/0x2f0 [] ? delete_one_xattr+0x0/0x1c0 [] do_journal_begin_r+0x64/0x2f0 [] journal_begin+0x7f/0x120 [] ? reiserfs_delete_xattrs+0x15/0x50 [] reiserfs_delete_inode+0x9f/0x140 [] ? generic_delete_inode+0x5f/0x150 [] ? reiserfs_delete_inode+0x0/0x140 [] generic_delete_inode+0x9c/0x150 [] generic_drop_inode+0x3d/0x60 [] iput+0x47/0x50 [] reiserfs_create+0x16c/0x1c0 [] ? inode_permission+0x7d/0xa0 [] vfs_create+0xc1/0x130 [] ? reiserfs_create+0x0/0x1c0 [] do_filp_open+0x81c/0x920 [] ? trace_hardirqs_off+0xb/0x10 [] ? _spin_unlock+0x1d/0x20 [] ? alloc_fd+0xba/0xf0 [] do_sys_open+0x4f/0x110 [] sys_open+0x29/0x40 [] sysenter_do_call+0x12/0x36 To fix this, use reiserfs_lock_once() from reiserfs_delete_inode() which prevents from adding reiserfs lock recursion. Reported-by: Alexander Beregalov Signed-off-by: Frederic Weisbecker Cc: Chris Mason Cc: Ingo Molnar Cc: Thomas Gleixner --- fs/reiserfs/inode.c | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/fs/reiserfs/inode.c b/fs/reiserfs/inode.c index 3a28e77..bd615df 100644 --- a/fs/reiserfs/inode.c +++ b/fs/reiserfs/inode.c @@ -31,11 +31,12 @@ void reiserfs_delete_inode(struct inode *inode) JOURNAL_PER_BALANCE_CNT * 2 + 2 * REISERFS_QUOTA_INIT_BLOCKS(inode->i_sb); struct reiserfs_transaction_handle th; + int depth; int err; truncate_inode_pages(&inode->i_data, 0); - reiserfs_write_lock(inode->i_sb); + depth = reiserfs_write_lock_once(inode->i_sb); /* The = 0 happens when we abort creating a new inode for some reason like lack of space.. */ if (!(inode->i_state & I_NEW) && INODE_PKEY(inode)->k_objectid != 0) { /* also handles bad_inode case */ @@ -74,7 +75,7 @@ void reiserfs_delete_inode(struct inode *inode) out: clear_inode(inode); /* note this must go after the journal_end to prevent deadlock */ inode->i_blocks = 0; - reiserfs_write_unlock(inode->i_sb); + reiserfs_write_unlock_once(inode->i_sb, depth); } static void _make_cpu_key(struct cpu_key *key, int version, __u32 dirid, -- 1.6.2.3 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/