Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Subject: Re: [PATCH v2 2/2] io_uring: io_queue_link*() right after submit
To:     Bob Liu <bob.liu@oracle.com>, Jens Axboe <axboe@kernel.dk>,
        io-uring@vger.kernel.org, linux-block@vger.kernel.org,
        linux-kernel@vger.kernel.org
References: <cover.1572988512.git.asml.silence@gmail.com>
 <85a316b577e1b5204d27a96a7ce452ed6be3c2ae.1572988512.git.asml.silence@gmail.com>
 <8700c9a3-01aa-2af6-c275-1f17734c2cc5@oracle.com>
From:   Pavel Begunkov <asml.silence@gmail.com>
Message-ID: <81c2db6f-e262-328a-5917-71b30d9390a5@gmail.com>
Date:   Wed, 6 Nov 2019 12:06:09 +0300
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101
 Thunderbird/60.9.0
MIME-Version: 1.0
In-Reply-To: <8700c9a3-01aa-2af6-c275-1f17734c2cc5@oracle.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On 11/6/2019 11:36 AM, Bob Liu wrote:
> On 11/6/19 5:22 AM, Pavel Begunkov wrote:
>> After a call to io_submit_sqe(), it's already known whether it needs
>> to queue a link or not. Do it there, as it's simplier and doesn't keep
>> an extra variable across the loop.
>>
>> Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
>> ---
>>  fs/io_uring.c | 22 ++++++++++------------
>>  1 file changed, 10 insertions(+), 12 deletions(-)
>>
>> diff --git a/fs/io_uring.c b/fs/io_uring.c
>> index ebe2a4edd644..82c2da99cb5c 100644
>> --- a/fs/io_uring.c
>> +++ b/fs/io_uring.c
>> @@ -2687,7 +2687,6 @@ static int io_submit_sqes(struct io_ring_ctx *ctx, unsigned int nr,
>>  	struct io_submit_state state, *statep = NULL;
>>  	struct io_kiocb *link = NULL;
>>  	struct io_kiocb *shadow_req = NULL;
>> -	bool prev_was_link = false;
>>  	int i, submitted = 0;
>>  	bool mm_fault = false;
>>  
>> @@ -2710,17 +2709,6 @@ static int io_submit_sqes(struct io_ring_ctx *ctx, unsigned int nr,
>>  			}
>>  		}
>>  
>> -		/*
>> -		 * If previous wasn't linked and we have a linked command,
>> -		 * that's the end of the chain. Submit the previous link.
>> -		 */
>> -		if (!prev_was_link && link) {
>> -			io_queue_link_head(ctx, link, &link->submit, shadow_req);
>> -			link = NULL;
>> -			shadow_req = NULL;
>> -		}
>> -		prev_was_link = (s.sqe->flags & IOSQE_IO_LINK) != 0;
>> -
>>  		if (link && (s.sqe->flags & IOSQE_IO_DRAIN)) {
>>  			if (!shadow_req) {
>>  				shadow_req = io_get_req(ctx, NULL);
>> @@ -2741,6 +2729,16 @@ static int io_submit_sqes(struct io_ring_ctx *ctx, unsigned int nr,
>>  		trace_io_uring_submit_sqe(ctx, s.sqe->user_data, true, async);
>>  		io_submit_sqe(ctx, &s, statep, &link);
>>  		submitted++;
>> +
>> +		/*
>> +		 * If previous wasn't linked and we have a linked command,
>> +		 * that's the end of the chain. Submit the previous link.
>> +		 */
>> +		if (!(s.sqe->flags & IOSQE_IO_LINK) && link) 
> The behavior changed to 'current seq' instead of previous after dropping prev_was_link?
> 
The old behaviour was to remember @prev_was_link for current sqe, and
use at the beginning of the next iteration, where it becomes
"previous/last sqe". See, prev_was_link was set after io_queue_link_head.

If @i is iteration idx, then timeline was:
i:   sqe[i-1].is_link -> io_queue_link_head() # if (prev_was_link)
i:   sqe[i].is_link = prev_was_link = (sqe[i].flags & LINK)
i+1: sqe[i].is_link -> io_queue_link_head() # if (prev_was_link)
i+1: sqe[i+1].is_link = ...


After the change, it's done at the same loop iteration by swapping order
of checking @prev_was_link and io_queue_link_head().

i:   sqe[i].is_link = ...
i:   sqe[i].is_link -> io_queue_link_head()
i+1: sqe[i+1].is_link = ...
i+1: sqe[i+1].is_link -> io_queue_link_head()

Shouldn't change the behavior, if I'm not missing something.


>> +			io_queue_link_head(ctx, link, &link->submit, shadow_req);
>> +			link = NULL;
>> +			shadow_req = NULL;
>> +		}
>>  	}
>>  
>>  	if (link)
>>
>