Received: by 2002:a05:6a10:c7c6:0:0:0:0 with SMTP id h6csp1467191pxy; Mon, 2 Aug 2021 02:15:27 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwS+szSHoxxY56CezH+3K5QRXjjriJio/IwSq0ATLurXi0jQK9NcCWzLi0k9l0f4x2avjLD X-Received: by 2002:a92:c849:: with SMTP id b9mr127035ilq.63.1627895727060; Mon, 02 Aug 2021 02:15:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1627895727; cv=none; d=google.com; s=arc-20160816; b=r4GqmqOl9dxfSsuYf/+qtxlIh1/vWAtLN5W8fh/8ulxFVko3tZ3BB6fE38YRzxPrPB hjC0ICSEu9uAKyqhxLPPnhbeg7uBWUxaCIWI8lsIyRCANz983kBlmMtNqm5IZbiIwzp9 ve/e8TUH96rARJcEXBDK9EgciyzC8fgou43TG0Ggl4h7bBwArjV/BOPc+HFTnWr6P/wt u8+1vAREZFgzIlLNnj7cSkaTM8vH8krbcqTw+Fth5sFHaEP/914+44+2RtRe4618J2Jh EGfVNyxE4oeYlPoPQ09angrnKYzIcgbKXPLggsFo/3Y3W1fYgEq76ZOK94zSYxKCXljw wsnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature:dkim-signature; bh=M0wV6lwFfROco+TITXP/8PqqejX24mSDpZVSzC6Ez5w=; b=vt7JuL5mYpr9wt5CRcY62JH+eR3+UYGHanJ2uFQUA/7h9cRxhSndOAC2lgQdRF6jwB DlS7qbAMxmr53ATtLayAk5WYHMiZzEws8M+Z+0N+N13GwT4G6e18PyvKSuN8KOMjNmU0 mYj7v1e/xX9gOvbZvf2qjhv3oW3ZTxPQm+vrPpYCvBwrP03K3H1P/7ZVjYe6PIzRklri m2sKMSTYHDGWDNyK5hWNZH9pKD7WYAgQ5Bid3O4eAbNNUCoEeoirRm/yVDAtDUcOlhn3 ztRalAGEWlzQQmOKAu8unFhHhAU3ewcpN1M8SMqgPKZdZzpFv3Y8iuT6Z3Xkvosf/Pvl Qq1g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=ejlpYeDl; dkim=neutral (no key) header.i=@suse.de header.s=susede2_ed25519 header.b=BLSWB3Qf; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id h13si12266648iop.97.2021.08.02.02.15.16; Mon, 02 Aug 2021 02:15:27 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=ejlpYeDl; dkim=neutral (no key) header.i=@suse.de header.s=susede2_ed25519 header.b=BLSWB3Qf; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232983AbhHBJOs (ORCPT + 99 others); Mon, 2 Aug 2021 05:14:48 -0400 Received: from smtp-out2.suse.de ([195.135.220.29]:58762 "EHLO smtp-out2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232871AbhHBJOc (ORCPT ); Mon, 2 Aug 2021 05:14:32 -0400 Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id D0BDD1FF54; Mon, 2 Aug 2021 09:14:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1627895661; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=M0wV6lwFfROco+TITXP/8PqqejX24mSDpZVSzC6Ez5w=; b=ejlpYeDlMy0PHYo/D0CWo36OsQsSj7ejGv9CX4w1f/zcfiDNlFht75Mkkli/+nLzZ32QMX OJqI/OwXvQGP9W/BFAugmCYIi4uagLiiMmKXp9DPkRmPzops0N7LLToehzmyPzIUmnb2aq tsbOhWb//ntnt70Vd9Ffb74wNBhwn3U= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1627895661; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=M0wV6lwFfROco+TITXP/8PqqejX24mSDpZVSzC6Ez5w=; b=BLSWB3QfiZmFyMbDlnhNktttJkxpeNHo95bUITJcF/ymEMeU3eXD5TJP4dnWPfJ9nOY1bR rmTKDlj/IXhQFPDg== Received: from adalid.arch.suse.de (adalid.arch.suse.de [10.161.8.13]) by relay2.suse.de (Postfix) with ESMTP id 06F04A3B83; Mon, 2 Aug 2021 09:14:21 +0000 (UTC) Received: by adalid.arch.suse.de (Postfix, from userid 17828) id ECB7C518C087; Mon, 2 Aug 2021 11:14:20 +0200 (CEST) From: Daniel Wagner To: linux-kernel@vger.kernel.org Cc: James Smart , Keith Busch , Ming Lei , Sagi Grimberg , Hannes Reinecke , Wen Xiong , Daniel Wagner Subject: [PATCH v4 0/8] Handle update hardware queues and queue freeze more carefully Date: Mon, 2 Aug 2021 11:14:11 +0200 Message-Id: <20210802091419.56425-1-dwagner@suse.de> X-Mailer: git-send-email 2.29.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi, This update version makes sure the unfreeze call is done when recreating the queues. I was able to reproduce hanging I/Os when we go into error recovery mode for FC and TCP [1]. Unfortunatly, I don't have access to a RDMA setup to verify but as the code is identically to the TCP, RDMA looks to like to suffer from the same problem. Thanks, Daniel [1] https://lore.kernel.org/linux-nvme/20210730094907.5vg7qebggttibogz@beryllium.lan/ https://lore.kernel.org/linux-nvme/20210730113415.wezsrvxv5cu4yz4x@beryllium.lan/ v1: - https://lore.kernel.org/linux-nvme/20210625101649.49296-1-dwagner@suse.de/ v2: - https://lore.kernel.org/linux-nvme/20210708092755.15660-1-dwagner@suse.de/ - reviewed tags collected - added 'update hardware queues' for all transport - added fix for fc hanger in nvme_wait_freeze_timeout v3: - https://lore.kernel.org/linux-nvme/20210720124353.127959-1-dwagner@suse.de/ - dropped 'nvme-fc: Freeze queues before destroying them' - added James' two patches v4: - added 'nvme-*: Unfreeze queues on reconnect' - added Hannes' reviewed tags Daniel Wagner (5): nvme-fc: Update hardware queues before using them nvme-rdma: Update number of hardware queues before using them nvme-fc: Wait with a timeout for queue to freeze nvme-tcp: Unfreeze queues on reconnect nvme-rdma: Unfreeze queues on reconnect Hannes Reinecke (1): nvme-tcp: Update number of hardware queues before using them James Smart (2): nvme-fc: avoid race between time out and tear down nvme-fc: fix controller reset hang during traffic drivers/nvme/host/fc.c | 28 +++++++++++++++++++--------- drivers/nvme/host/rdma.c | 15 ++++++++------- drivers/nvme/host/tcp.c | 18 +++++++++--------- 3 files changed, 36 insertions(+), 25 deletions(-) -- 2.29.2