Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1162218AbbKFTyf (ORCPT ); Fri, 6 Nov 2015 14:54:35 -0500 Received: from mail.linuxfoundation.org ([140.211.169.12]:48098 "EHLO mail.linuxfoundation.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1757627AbbKFTmG (ORCPT ); Fri, 6 Nov 2015 14:42:06 -0500 From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Roman Gushchin , NeilBrown , Shaohua Li Subject: [PATCH 3.14 34/37] md/raid5: fix locking in handle_stripe_clean_event() Date: Fri, 6 Nov 2015 11:24:49 -0800 Message-Id: <20151106192412.947316376@linuxfoundation.org> X-Mailer: git-send-email 2.6.2 In-Reply-To: <20151106192410.681850286@linuxfoundation.org> References: <20151106192410.681850286@linuxfoundation.org> User-Agent: quilt/0.64 MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-15 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2618 Lines: 72 3.14-stable review patch. If anyone has any objections, please let me know. ------------------ From: Roman Gushchin commit b8a9d66d043ffac116100775a469f05f5158c16f upstream. After commit 566c09c53455 ("raid5: relieve lock contention in get_active_stripe()") __find_stripe() is called under conf->hash_locks + hash. But handle_stripe_clean_event() calls remove_hash() under conf->device_lock. Under some cirscumstances the hash chain can be circuited, and we get an infinite loop with disabled interrupts and locked hash lock in __find_stripe(). This leads to hard lockup on multiple CPUs and following system crash. I was able to reproduce this behavior on raid6 over 6 ssd disks. The devices_handle_discard_safely option should be set to enable trim support. The following script was used: for i in `seq 1 32`; do dd if=/dev/zero of=large$i bs=10M count=100 & done neilb: original was against a 3.x kernel. I forward-ported to 4.3-rc. This verison is suitable for any kernel since Commit: 59fc630b8b5f ("RAID5: batch adjacent full stripe write") (v4.1+). I'll post a version for earlier kernels to stable. Signed-off-by: Roman Gushchin Fixes: 566c09c53455 ("raid5: relieve lock contention in get_active_stripe()") Signed-off-by: NeilBrown Cc: Shaohua Li Cc: # 3.13 - 4.2 Signed-off-by: Greg Kroah-Hartman --- drivers/md/raid5.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -3029,6 +3029,8 @@ static void handle_stripe_clean_event(st } if (!discard_pending && test_bit(R5_Discard, &sh->dev[sh->pd_idx].flags)) { + int hash = sh->hash_lock_index; + clear_bit(R5_Discard, &sh->dev[sh->pd_idx].flags); clear_bit(R5_UPTODATE, &sh->dev[sh->pd_idx].flags); if (sh->qd_idx >= 0) { @@ -3042,9 +3044,9 @@ static void handle_stripe_clean_event(st * no updated data, so remove it from hash list and the stripe * will be reinitialized */ - spin_lock_irq(&conf->device_lock); + spin_lock_irq(conf->hash_locks + hash); remove_hash(sh); - spin_unlock_irq(&conf->device_lock); + spin_unlock_irq(conf->hash_locks + hash); if (test_bit(STRIPE_SYNC_REQUESTED, &sh->state)) set_bit(STRIPE_HANDLE, &sh->state); -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/