Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Daniel Wagner <dwagner@suse.de>
To:     linux-nvme@lists.infradead.org
Cc:     linux-kernel@vger.kernel.org, Sagi Grimberg <sagi@grimberg.me>,
        James Smart <james.smart@broadcom.com>,
        Chaitanya Kulkarni <kch@nvidia.com>,
        Daniel Wagner <dwagner@suse.de>
Subject: [PATCH v2 0/4] nvmet-fcloop: unblock module removal
Date:   Tue, 11 Apr 2023 14:07:14 +0200
Message-Id: <20230411120718.14477-1-dwagner@suse.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Precedence: bulk

blktests is not able to unload the FC related modules. It is possible to unload
the modules but it still will not work correctly. The host and the controller
seem to be in a kind of live deadlock:

 loop: module loaded
 run blktests nvme/003 at 2023-04-11 13:55:57
 nvmet: adding nsid 1 to subsystem blktests-subsystem-1
 nvme nvme0: NVME-FC{0}: create association : host wwpn 0x20001100aa000002  rport wwpn 0x20001100aa000001: NQN "nqn.2014-08.org.nvmexpress.discovery"
 (NULL device *): {0:0} Association created
 [71] nvmet: ctrl 1 start keep-alive timer for 120 secs
 nvmet: creating discovery controller 1 for subsystem nqn.2014-08.org.nvmexpress.discovery for NQN nqn.2014-08.org.nvmexpress:uuid:242d4a24-2484-4a80-8234-d0169409c5e8.
 nvme nvme0: NVME-FC{0}: controller connect complete
 nvme nvme0: NVME-FC{0}: new ctrl: NQN "nqn.2014-08.org.nvmexpress.discovery"
 nvme nvme1: NVME-FC{1}: create association : host wwpn 0x20001100aa000002  rport wwpn 0x20001100aa000001: NQN "blktests-subsystem-1"
 (NULL device *): {0:1} Association created
 [453] nvmet: ctrl 2 start keep-alive timer for 5 secs
 nvmet: creating nvm controller 2 for subsystem blktests-subsystem-1 for NQN nqn.2014-08.org.nvmexpress:uuid:242d4a24-2484-4a80-8234-d0169409c5e8.
 [71] nvmet: adding queue 1 to ctrl 2.
 [45] nvmet: adding queue 2 to ctrl 2.
 [453] nvmet: adding queue 3 to ctrl 2.
 [105] nvmet: adding queue 4 to ctrl 2.
 nvme nvme1: NVME-FC{1}: controller connect complete
 nvme nvme1: NVME-FC{1}: new ctrl: NQN "blktests-subsystem-1"
 [453] nvmet: ctrl 2 reschedule traffic based keep-alive timer
 [105] nvmet: ctrl 2 update keep-alive timer for 5 secs
 [105] nvmet: ctrl 2 update keep-alive timer for 5 secs
 nvme nvme0: Removing ctrl: NQN "nqn.2014-08.org.nvmexpress.discovery"
 [45] nvmet: ctrl 1 stop keep-alive
 (NULL device *): {0:0} Association deleted
 (NULL device *): {0:0} Association freed
 (NULL device *): Disconnect LS failed: No Association
 nvme nvme1: rescanning namespaces.
 nvme nvme1: NVME-FC{1}: io failed due to lldd error 6
 nvme nvme1: NVME-FC{1}: transport association event: transport detected io error
 nvme nvme1: NVME-FC{1}: resetting controller
 [105] nvmet: ctrl 2 stop keep-alive
 nvme nvme0: NVME-FC{0}: create association : host wwpn 0x20001100aa000002  rport wwpn 0x20001100aa000001: NQN "nqn.2014-08.org.nvmexpress.discovery"
 (NULL device *): {0:1} Association deleted
 (NULL device *): {0:0} Association created
 (NULL device *): {0:1} Association freed
 nvmet: connect request for invalid subsystem nqn.2014-08.org.nvmexpress.discovery!
 nvme nvme0: Connect Invalid Data Parameter, subsysnqn "nqn.2014-08.org.nvmexpress.discovery"
 (NULL device *): Disconnect LS failed: No Association
 nvme nvme1: NVME-FC{1}: create association : host wwpn 0x20001100aa000002  rport wwpn 0x20001100aa000001: NQN "blktests-subsystem-1"
 nvme nvme0: NVME-FC{0}: reset: Reconnect attempt failed (16770)
 (NULL device *): {0:1} Association created
 nvme nvme0: NVME-FC{0}: reconnect failure
 nvmet: connect request for invalid subsystem blktests-subsystem-1!
 nvme nvme0: Removing ctrl: NQN "nqn.2014-08.org.nvmexpress.discovery"
 nvme nvme1: Connect Invalid Data Parameter, subsysnqn "blktests-subsystem-1"
 nvme nvme0: NVME-FC{0}: new ctrl: NQN "nqn.2014-08.org.nvmexpress.discovery"
 nvme nvme1: NVME-FC{1}: reset: Reconnect attempt failed (16770)
 nvme nvme1: NVME-FC{1}: reconnect failure
 nvme nvme1: Removing ctrl: NQN "blktests-subsystem-1"
 (NULL device *): {0:0} Association deleted
 (NULL device *): {0:0} Association freed
 (NULL device *): Disconnect LS failed: No Association
 (NULL device *): {0:1} Association deleted
 (NULL device *): {0:1} Association freed
 (NULL device *): Disconnect LS failed: No Association
 nvmet_fc: nvmet_fc_exit_module: targetport list not empty

I think these patches here are not very controversial and should propably go in
even if we still haven't fix for the above scenario.

v2:
  - added additional fixes

v1:
  - initial version
  - https://lore.kernel.org/linux-nvme/20230411092209.12719-1-dwagner@suse.de/

Daniel Wagner (4):
  nvmet-fcloop: Remove remote port from list when unlinking
  nvmet-fcloop: Do not wait on completion when unregister fails
  nvmet-fc: Do not wait in vain when unloading module
  nvmet-fc: Release reference on target port

 drivers/nvme/host/fc.c       | 20 +++++++++++++-------
 drivers/nvme/target/fc.c     |  1 +
 drivers/nvme/target/fcloop.c |  5 ++++-
 3 files changed, 18 insertions(+), 8 deletions(-)

-- 
2.40.0