Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: [PATCH v4 2/8] nvme-tcp: Update number of hardware queues before
 using them
To:     Keith Busch <kbusch@kernel.org>
Cc:     Daniel Wagner <dwagner@suse.de>, linux-nvme@lists.infradead.org,
        linux-kernel@vger.kernel.org,
        James Smart <james.smart@broadcom.com>,
        Ming Lei <ming.lei@redhat.com>, Hannes Reinecke <hare@suse.de>,
        Wen Xiong <wenxiong@us.ibm.com>
References: <20210802112658.75875-1-dwagner@suse.de>
 <20210802112658.75875-3-dwagner@suse.de>
 <8373c07f-f5df-1ec6-9fda-d0262fc1b377@grimberg.me>
 <20210809085250.xguvx5qiv2gxcoqk@carbon>
 <01d7878c-e396-1d6b-c383-8739ca0b3d11@grimberg.me>
 <20210811010718.GA3135947@dhcp-10-100-145-180.wdc.com>
From:   Sagi Grimberg <sagi@grimberg.me>
Message-ID: <079108ce-6ca0-800e-e3df-29d015a4530c@grimberg.me>
Date:   Tue, 10 Aug 2021 22:57:58 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <20210811010718.GA3135947@dhcp-10-100-145-180.wdc.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Precedence: bulk


>> On 8/9/21 1:52 AM, Daniel Wagner wrote:
>>> Hi Sagi,
>>>
>>> On Fri, Aug 06, 2021 at 12:57:17PM -0700, Sagi Grimberg wrote:
>>>>> -	ret = nvme_tcp_start_io_queues(ctrl);
>>>>> -	if (ret)
>>>>> -		goto out_cleanup_connect_q;
>>>>> -
>>>>> -	if (!new) {
>>>>> -		nvme_start_queues(ctrl);
>>>>> +	} else if (prior_q_cnt != ctrl->queue_count) {
>>>>
>>>> So if the queue count did not change we don't wait to make sure
>>>> the queue g_usage_counter ref made it to zero? What guarantees that it
>>>> did?
>>>
>>> Hmm, good point. we should always call nvme_wait_freeze_timeout()
>>> for !new queues. Is this what you are implying?
>>
>> I think we should always wait for the freeze to complete.
> 
> Don't the queues need to be started in order for the freeze to complete?
> Any enqueued requests on the quiesced queues will never complete this
> way, so the wait_freeze() will be stuck, right? If so, I think the
> nvme_start_queues() was in the correct place already.

Exactly what I was trying to point out (poorly though)