Received: by 2002:a05:6a10:a0d1:0:0:0:0 with SMTP id j17csp2665014pxa; Mon, 17 Aug 2020 15:58:43 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzAonSuwBIulbl+KWa1Gq1OqfTLBCKzVgxpCC+L2yMoXsQMmmnuksaAMP/X/ZL3qF6X5D8m X-Received: by 2002:a50:cbc3:: with SMTP id l3mr17238113edi.58.1597705123265; Mon, 17 Aug 2020 15:58:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1597705123; cv=none; d=google.com; s=arc-20160816; b=vxvHRXgy/fB2Q+OzKEyY2aaqbZ3AtxxjtddS0c8TL9+7BlniWsDhpNugbdg1mXLh5o ETCFNtsz2zb7Tn9kxsKsXEqBUlNz4+6ujMpM86e5oVbtjTDSkEuku3AOSytIwwvzj9Bx Yg1h9aNJ2+EgrbYb/JrpBZyHnkHlBB7/1pdKW0o8tduppxe79Hup95LN2s1vE5TufTX4 yFKvP1CjafrqsmZV8IiLSnP850+3HEwFhrzD9juukQK8ssESMrX8TVYpW5yoky7+EIP6 kWN4ICao1NDseuT79rckM4pbnW/28z1Hk/aMlA5nGdWqGgKGgFK7pElhKrHn51q1hICb 0svA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=D5iwv+Af5WIRJV/lfNVehRrPVla5wFuyLnueSGJ1KHQ=; b=tQM7tgEly+w1i9tS+GJ7g2lBvWKR8pu8qkz0Vi2HniZ8Svp4xVBqYAFYxnZHYYDpbw 3065kAIRqH5d4nOScEuDyR9W1mv4qPrwtB+aA5Tk2XbdCR2Q/jnBuEDC5ZOoHcQqT4YP vwt5g97HqaduCDxwDbKInGENPM21Wp3l2ML8x7otUESJoVv1AgxezjHYNpqPkJfy6K54 UOlVCYrS0iGDAf7mW8YSzr/LoufHVAlYefjkI3eQrBpXeKOujj2t1ySxFCEDhHp24a2M DpoQL9k7JzS5xwSBaGzawtx4tI8HE8TaxjiLju8S+STINaWxqCFxhxoxa0th5Y612zYb XWKQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=lx7J+ALf; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id u16si11903572ejg.142.2020.08.17.15.58.20; Mon, 17 Aug 2020 15:58:43 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=lx7J+ALf; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2404403AbgHQTq2 (ORCPT + 99 others); Mon, 17 Aug 2020 15:46:28 -0400 Received: from mail.kernel.org ([198.145.29.99]:43750 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729353AbgHQPV1 (ORCPT ); Mon, 17 Aug 2020 11:21:27 -0400 Received: from localhost (83-86-89-107.cable.dynamic.v4.ziggo.nl [83.86.89.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id B6BFD20709; Mon, 17 Aug 2020 15:21:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1597677687; bh=n4p+AeW6JiuIsLx1WW7NJfTXfksep1+709oM9kBwEgI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lx7J+ALfUbQCjO1biPZ99KKu6gegwiNOdKXS6lXMg7QIRr/h2WJqqB/tTd9kwmX1e 9HON5CNEc+Dlx6Z9+/eEZhef3ld+aRxnZ4PknPqrvOXm4Y3mtIMK6Bu/W0jMos4bZd VOGqic4yfsYbsTwZw1Gxks2p9/e4v/Eu8SDMAaCw= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Sagi Grimberg , Christoph Hellwig , Sasha Levin Subject: [PATCH 5.8 071/464] nvme-tcp: fix controller reset hang during traffic Date: Mon, 17 Aug 2020 17:10:24 +0200 Message-Id: <20200817143837.169466159@linuxfoundation.org> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200817143833.737102804@linuxfoundation.org> References: <20200817143833.737102804@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Sagi Grimberg [ Upstream commit 2875b0aecabe2f081a8432e2bc85b85df0529490 ] commit fe35ec58f0d3 ("block: update hctx map when use multiple maps") exposed an issue where we may hang trying to wait for queue freeze during I/O. We call blk_mq_update_nr_hw_queues which in case of multiple queue maps (which we have now for default/read/poll) is attempting to freeze the queue. However we never started queue freeze when starting the reset, which means that we have inflight pending requests that entered the queue that we will not complete once the queue is quiesced. So start a freeze before we quiesce the queue, and unfreeze the queue after we successfully connected the I/O queues (and make sure to call blk_mq_update_nr_hw_queues only after we are sure that the queue was already frozen). This follows to how the pci driver handles resets. Fixes: fe35ec58f0d3 ("block: update hctx map when use multiple maps") Signed-off-by: Sagi Grimberg Signed-off-by: Christoph Hellwig Signed-off-by: Sasha Levin --- drivers/nvme/host/tcp.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c index f3a91818167b1..83bb329d4113a 100644 --- a/drivers/nvme/host/tcp.c +++ b/drivers/nvme/host/tcp.c @@ -1744,15 +1744,20 @@ static int nvme_tcp_configure_io_queues(struct nvme_ctrl *ctrl, bool new) ret = PTR_ERR(ctrl->connect_q); goto out_free_tag_set; } - } else { - blk_mq_update_nr_hw_queues(ctrl->tagset, - ctrl->queue_count - 1); } ret = nvme_tcp_start_io_queues(ctrl); if (ret) goto out_cleanup_connect_q; + if (!new) { + nvme_start_queues(ctrl); + nvme_wait_freeze(ctrl); + blk_mq_update_nr_hw_queues(ctrl->tagset, + ctrl->queue_count - 1); + nvme_unfreeze(ctrl); + } + return 0; out_cleanup_connect_q: @@ -1857,6 +1862,7 @@ static void nvme_tcp_teardown_io_queues(struct nvme_ctrl *ctrl, { if (ctrl->queue_count <= 1) return; + nvme_start_freeze(ctrl); nvme_stop_queues(ctrl); nvme_tcp_stop_io_queues(ctrl); if (ctrl->tagset) { -- 2.25.1