Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Subject: Re: [PATCH V11 0/4] blk-mq: refactor code of issue directly
To:     "jianchao.wang" <jianchao.w.wang@oracle.com>
Cc:     ming.lei@redhat.com, linux-block@vger.kernel.org,
        linux-kernel@vger.kernel.org
References: <1544152185-32667-1-git-send-email-jianchao.w.wang@oracle.com>
 <b2bcbb76-af6f-7660-2bc3-9796338cccff@kernel.dk>
 <0adf3419-bcce-93d8-51fb-aee7cbb5ae17@oracle.com>
 <16205e68-aa5e-c59d-364e-4164a0e51dc7@kernel.dk>
 <f052c016-e521-9f12-f4e5-d65f3814643a@kernel.dk>
 <a59a3a1d-94fb-938d-7ddf-969185cd5b48@oracle.com>
 <1e183b77-2c4d-71ff-b019-2b1070d2ed6b@kernel.dk>
 <38eb7c50-dfad-d9cb-f8ab-a8f5250b0ed7@oracle.com>
 <bd99f969-5eed-5365-6aa1-ce9b85dc84d6@kernel.dk>
 <6432264f-57e9-d405-079e-21c0aa17b08e@oracle.com>
From:   Jens Axboe <axboe@kernel.dk>
Message-ID: <7a827e9b-7f52-6073-94b8-105288ee289a@kernel.dk>
Date:   Sun, 9 Dec 2018 18:27:09 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.2.1
MIME-Version: 1.0
In-Reply-To: <6432264f-57e9-d405-079e-21c0aa17b08e@oracle.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On 12/9/18 6:18 PM, jianchao.wang wrote:
> 
> 
> On 12/7/18 11:47 AM, Jens Axboe wrote:
>> On 12/6/18 8:46 PM, jianchao.wang wrote:
>>>
>>>
>>> On 12/7/18 11:42 AM, Jens Axboe wrote:
>>>> On 12/6/18 8:41 PM, jianchao.wang wrote:
>>>>>
>>>>>
>>>>> On 12/7/18 11:34 AM, Jens Axboe wrote:
>>>>>> On 12/6/18 8:32 PM, Jens Axboe wrote:
>>>>>>> On 12/6/18 8:26 PM, jianchao.wang wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> On 12/7/18 11:16 AM, Jens Axboe wrote:
>>>>>>>>> On 12/6/18 8:09 PM, Jianchao Wang wrote:
>>>>>>>>>> Hi Jens
>>>>>>>>>>
>>>>>>>>>> Please consider this patchset for 4.21.
>>>>>>>>>>
>>>>>>>>>> It refactors the code of issue request directly to unify the interface
>>>>>>>>>> and make the code clearer and more readable.
>>>>>>>>>>
>>>>>>>>>> This patch set is rebased on the recent for-4.21/block and add the 1st
>>>>>>>>>> patch which inserts the non-read-write request to hctx dispatch
>>>>>>>>>> list to avoid to involve merge and io scheduler when bypass_insert
>>>>>>>>>> is true, otherwise, inserting is ignored, BLK_STS_RESOURCE is returned
>>>>>>>>>> and the caller will fail forever.
>>>>>>>>>>
>>>>>>>>>> The 2nd patch refactors the code of issue request directly to unify the
>>>>>>>>>> helper interface which could handle all the cases.
>>>>>>>>>>
>>>>>>>>>> The 3rd patch make blk_mq_sched_insert_requests issue requests directly
>>>>>>>>>> with 'bypass' false, then it needn't to handle the non-issued requests
>>>>>>>>>> any more.
>>>>>>>>>>
>>>>>>>>>> The 4th patch replace and kill the blk_mq_request_issue_directly.
>>>>>>>>>
>>>>>>>>> Sorry to keep iterating on this, but let's default to inserting to
>>>>>>>>> the dispatch list if we ever see busy from a direct dispatch. I'm fine
>>>>>>>>> with doing that for 4.21, as suggested by Ming, I just didn't want to
>>>>>>>>> fiddle with it for 4.20. This will prevent any merging on the request
>>>>>>>>> going forward, which I think is a much safer default.
>>>>>>>>>
>>>>>>>>> You do this already for some cases. Let's do it unconditionally for
>>>>>>>>> a request that was ever subjected to ->queue_rq() and we didn't either
>>>>>>>>> error or finish after the fact.
>>>>>>>>>
>>>>>>>> I have done it in this version if I get your point correctly.
>>>>>>>> Please refer to the following fragment in the 2nd patch.
>>>>>>>>
>>>>>>>> +	/*
>>>>>>>> +	 * If the request is issued unsuccessfully with
>>>>>>>> +	 * BLK_STS_DEV_RESOURCE or BLK_STS_RESOURCE, insert
>>>>>>>> +	 * the request to hctx dispatch list due to attached
>>>>>>>> +	 * lldd resource.
>>>>>>>> +	 */
>>>>>>>> +	force = true;
>>>>>>>> +	ret = __blk_mq_issue_directly(hctx, rq, cookie, last);
>>>>>>>> +out_unlock:
>>>>>>>> +	hctx_unlock(hctx, srcu_idx);
>>>>>>>> +out:
>>>>>>>> +	switch (ret) {
>>>>>>>> +	case BLK_STS_OK:
>>>>>>>> +		break;
>>>>>>>> +	case BLK_STS_DEV_RESOURCE:
>>>>>>>> +	case BLK_STS_RESOURCE:
>>>>>>>> +		if (force) {
>>>>>>>> +			blk_mq_request_bypass_insert(rq, run_queue);
>>>>>>>> +			ret = bypass ? BLK_STS_OK : ret;
>>>>>>>> +		} else if (!bypass) {
>>>>>>>> +			blk_mq_sched_insert_request(rq, false,
>>>>>>>> +						    run_queue, false);
>>>>>>>> +		}
>>>>>>>> +		break;
>>>>>>>> +	default:
>>>>>>>
>>>>>>> You are right, I missed that you set force = true before doing the
>>>>>>> issue. So this looks good to me!
>>>>>>
>>>>>> I applied your series. With this, we should be good to remove the
>>>>>> REQ_NOMERGE logic that was added for the corruption case, and the
>>>>>> blk_rq_can_direct_dispatch() as well?
>>>>>>
>>>>>
>>>>> Yes, it should be that.
>>>>> Every thing rejected by .queue_rq is ended or inserted into hctx dispatch
>>>>> list. And also direct-issue path is unified with normal path.
>>>>
>>>> Why are we doing that return value dance, depending on whether this
>>>> is a bypass insert or not? That seems confusing.
>>>>
>>>
>>> For the 'bypass == false' case, it need to know whether the request is issued
>>> successfully. This is for the 3rd patch.
>>> I used to use the returned cookie to identify the result, but you don't like it.
>>> So I have to use this return value.
>>
>> Makes sense, but could probably do with a comment. I'm going to let the
>> series float for a day or two to ensure others get a chance to review it,
>> then we can move forward.
>>
> 
> Do I need a respin about the comment ?

I pulled in the two fixes from this week, so it would probably need a
respin on top of that.

-- 
Jens Axboe