2021-06-25 10:19:01

by Daniel Wagner

Subject: [PATCH 1/2] nvme-fc: Update hardware queues before using them

In case the number of hardware queues changes, do the update the
tagset and ctx to hctx first before using the mapping to recreate and
connnect the IO queues.

Signed-off-by: Daniel Wagner <[email protected]>
---
drivers/nvme/host/fc.c | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index 8a3c4814d21b..a9645cd89eca 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -2951,14 +2951,6 @@ nvme_fc_recreate_io_queues(struct nvme_fc_ctrl *ctrl)
if (ctrl->ctrl.queue_count == 1)
return 0;

- ret = nvme_fc_create_hw_io_queues(ctrl, ctrl->ctrl.sqsize + 1);
- if (ret)
- goto out_free_io_queues;
-
- ret = nvme_fc_connect_io_queues(ctrl, ctrl->ctrl.sqsize + 1);
- if (ret)
- goto out_delete_hw_queues;
-
if (prior_ioq_cnt != nr_io_queues) {
dev_info(ctrl->ctrl.device,
"reconnect: revising io queue count from %d to %d\n",
@@ -2968,6 +2960,14 @@ nvme_fc_recreate_io_queues(struct nvme_fc_ctrl *ctrl)
nvme_unfreeze(&ctrl->ctrl);
}

+ ret = nvme_fc_create_hw_io_queues(ctrl, ctrl->ctrl.sqsize + 1);
+ if (ret)
+ goto out_free_io_queues;
+
+ ret = nvme_fc_connect_io_queues(ctrl, ctrl->ctrl.sqsize + 1);
+ if (ret)
+ goto out_delete_hw_queues;
+
return 0;

out_delete_hw_queues:
--
2.29.2

2021-06-27 14:00:54

by James Smart

Subject: Re: [PATCH 1/2] nvme-fc: Update hardware queues before using them

On 6/25/2021 3:16 AM, Daniel Wagner wrote:
> In case the number of hardware queues changes, do the update the
> tagset and ctx to hctx first before using the mapping to recreate and
> connnect the IO queues.
>
> Signed-off-by: Daniel Wagner <[email protected]>
> ---
> drivers/nvme/host/fc.c | 16 ++++++++--------
> 1 file changed, 8 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
> index 8a3c4814d21b..a9645cd89eca 100644

Makes sense. Thanks. Although it does bring up that perhaps, if the
hw queue count changes, thus it no longer matches what was set on the
target, the new value should be set on the target to release resources
on the target.

Note: the same behavior exists in the other transports as we all started
from the same lineage. So those should be updated as well. Granted
you'll need to break out the queue count set and checking which was done
on fc but not on the other transports.

Reviewed-by: James Smart <[email protected]>

-- james

2021-06-29 01:33:44

by Ming Lei

Subject: Re: [PATCH 1/2] nvme-fc: Update hardware queues before using them

On Fri, Jun 25, 2021 at 12:16:48PM +0200, Daniel Wagner wrote:
> In case the number of hardware queues changes, do the update the
> tagset and ctx to hctx first before using the mapping to recreate and
> connnect the IO queues.
>
> Signed-off-by: Daniel Wagner <[email protected]>
> ---
> drivers/nvme/host/fc.c | 16 ++++++++--------
> 1 file changed, 8 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
> index 8a3c4814d21b..a9645cd89eca 100644
> --- a/drivers/nvme/host/fc.c
> +++ b/drivers/nvme/host/fc.c
> @@ -2951,14 +2951,6 @@ nvme_fc_recreate_io_queues(struct nvme_fc_ctrl *ctrl)
> if (ctrl->ctrl.queue_count == 1)
> return 0;
>
> - ret = nvme_fc_create_hw_io_queues(ctrl, ctrl->ctrl.sqsize + 1);
> - if (ret)
> - goto out_free_io_queues;
> -
> - ret = nvme_fc_connect_io_queues(ctrl, ctrl->ctrl.sqsize + 1);
> - if (ret)
> - goto out_delete_hw_queues;
> -
> if (prior_ioq_cnt != nr_io_queues) {
> dev_info(ctrl->ctrl.device,
> "reconnect: revising io queue count from %d to %d\n",
> @@ -2968,6 +2960,14 @@ nvme_fc_recreate_io_queues(struct nvme_fc_ctrl *ctrl)
> nvme_unfreeze(&ctrl->ctrl);
> }
>
> + ret = nvme_fc_create_hw_io_queues(ctrl, ctrl->ctrl.sqsize + 1);
> + if (ret)
> + goto out_free_io_queues;
> +
> + ret = nvme_fc_connect_io_queues(ctrl, ctrl->ctrl.sqsize + 1);
> + if (ret)
> + goto out_delete_hw_queues;
> +
> return 0;
>
> out_delete_hw_queues:
> --
> 2.29.2
>

This way may cause correct hctx_idx to be passed to blk_mq_alloc_request_hctx(), so:

Reviewed-by: Ming Lei <[email protected]>


Thanks,
Ming

2021-06-29 12:33:32

by Hannes Reinecke

Subject: Re: [PATCH 1/2] nvme-fc: Update hardware queues before using them

On 6/25/21 12:16 PM, Daniel Wagner wrote:
> In case the number of hardware queues changes, do the update the
> tagset and ctx to hctx first before using the mapping to recreate and
> connnect the IO queues.
>
> Signed-off-by: Daniel Wagner <[email protected]>
> ---
> drivers/nvme/host/fc.c | 16 ++++++++--------
> 1 file changed, 8 insertions(+), 8 deletions(-)
>
Reviewed-by: Hannes Reinecke <[email protected]>

Cheers,

Hannes
--
Dr. Hannes Reinecke Kernel Storage Architect
[email protected] +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer