Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753654AbcCAXtt (ORCPT ); Tue, 1 Mar 2016 18:49:49 -0500 Received: from mail.linuxfoundation.org ([140.211.169.12]:45923 "EHLO mail.linuxfoundation.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753104AbcCAXrb (ORCPT ); Tue, 1 Mar 2016 18:47:31 -0500 From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Zheng Liu , Joshua Schmid , Eric Wheeler , Zhu Yanhai , Kent Overstreet , Jens Axboe Subject: [PATCH 3.14 006/130] bcache: fix a livelock when we cause a huge number of cache misses Date: Tue, 1 Mar 2016 15:44:17 -0800 Message-Id: <20160301234500.001841062@linuxfoundation.org> X-Mailer: git-send-email 2.7.2 In-Reply-To: <20160301234459.768886030@linuxfoundation.org> References: <20160301234459.768886030@linuxfoundation.org> User-Agent: quilt/0.64 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2553 Lines: 69 3.14-stable review patch. If anyone has any objections, please let me know. ------------------ From: Zheng Liu commit 2ef9ccbfcb90cf84bdba320a571b18b05c41101b upstream. Subject : [PATCH v2] bcache: fix a livelock in btree lock Date : Wed, 25 Feb 2015 20:32:09 +0800 (02/25/2015 04:32:09 AM) This commit tries to fix a livelock in bcache. This livelock might happen when we causes a huge number of cache misses simultaneously. When we get a cache miss, bcache will execute the following path. ->cached_dev_make_request() ->cached_dev_read() ->cached_lookup() ->bch->btree_map_keys() ->btree_root() <------------------------ ->bch_btree_map_keys_recurse() | ->cache_lookup_fn() | ->cached_dev_cache_miss() | ->bch_btree_insert_check_key() -| [If btree->seq is not equal to seq + 1, we should return EINTR and traverse btree again.] In bch_btree_insert_check_key() function we first need to check upgrade flag (op->lock == -1), and when this flag is true we need to release read btree->lock and try to take write btree->lock. During taking and releasing this write lock, btree->seq will be monotone increased in order to prevent other threads modify this in cache miss (see btree.h:74). But if there are some cache misses caused by some requested, we could meet a livelock because btree->seq is always changed by others. Thus no one can make progress. This commit will try to take write btree->lock if it encounters a race when we traverse btree. Although it sacrifice the scalability but we can ensure that only one can modify the btree. Signed-off-by: Zheng Liu Tested-by: Joshua Schmid Tested-by: Eric Wheeler Cc: Joshua Schmid Cc: Zhu Yanhai Cc: Kent Overstreet Signed-off-by: Jens Axboe Signed-off-by: Greg Kroah-Hartman --- drivers/md/bcache/btree.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) --- a/drivers/md/bcache/btree.c +++ b/drivers/md/bcache/btree.c @@ -2037,8 +2037,10 @@ int bch_btree_insert_check_key(struct bt rw_lock(true, b, b->level); if (b->key.ptr[0] != btree_ptr || - b->seq != seq + 1) + b->seq != seq + 1) { + op->lock = b->level; goto out; + } } SET_KEY_PTRS(check_key, 1);