Apart from unrelated first patch, this persues two goals:
1. start preparing io_uring to move resources handling into
opcode specific functions
2. make the first step towards long-standing optimisation ideas
Basically, it makes struct io_submit_state embedded into ctx, so
easily accessible and persistent, and then plays a bit around that.
v2: rebase
v3: drop the mm-related patch and rebase
Pavel Begunkov (6):
io_uring: always pass non-null io_submit_state
io_uring: place io_submit_state into ctx
io_uring: move ring_fd into io_submit_state
io_uring: move *link into io_submit_state
io_uring: persistent req bulk allocation cache
io_uring: optimise req bulk allocation cache
fs/io_uring.c | 166 +++++++++++++++++++++++++++-----------------------
1 file changed, 89 insertions(+), 77 deletions(-)
--
2.24.0
On 1/31/20 3:15 PM, Pavel Begunkov wrote:
> Apart from unrelated first patch, this persues two goals:
>
> 1. start preparing io_uring to move resources handling into
> opcode specific functions
>
> 2. make the first step towards long-standing optimisation ideas
>
> Basically, it makes struct io_submit_state embedded into ctx, so
> easily accessible and persistent, and then plays a bit around that.
Do you have any perf/latency numbers for this? Just curious if we
see any improvements on that front, cross submit persistence of
alloc caches should be a nice sync win, for example, or even
for peak iops by not having to replenish the pool for each batch.
I can try and run some here too.
--
Jens Axboe
On 01/02/2020 01:22, Jens Axboe wrote:
> On 1/31/20 3:15 PM, Pavel Begunkov wrote:
>> Apart from unrelated first patch, this persues two goals:
>>
>> 1. start preparing io_uring to move resources handling into
>> opcode specific functions
>>
>> 2. make the first step towards long-standing optimisation ideas
>>
>> Basically, it makes struct io_submit_state embedded into ctx, so
>> easily accessible and persistent, and then plays a bit around that.
>
> Do you have any perf/latency numbers for this? Just curious if we
> see any improvements on that front, cross submit persistence of
> alloc caches should be a nice sync win, for example, or even
> for peak iops by not having to replenish the pool for each batch.
>
> I can try and run some here too.
>
I tested the first version, but my drive is too slow, so it was only nops and
hence no offloading. Honestly, there waren't statistically significant results.
I'll rerun anyway.
I have a plan to reuse it for a tricky optimisation, but thinking twice, I can
just stash it until everything is done. That's not the first thing in TODO and
will take a while.
--
Pavel Begunkov
On 01/02/2020 01:32, Pavel Begunkov wrote:
> On 01/02/2020 01:22, Jens Axboe wrote:
>> On 1/31/20 3:15 PM, Pavel Begunkov wrote:
>>> Apart from unrelated first patch, this persues two goals:
>>>
>>> 1. start preparing io_uring to move resources handling into
>>> opcode specific functions
>>>
>>> 2. make the first step towards long-standing optimisation ideas
>>>
>>> Basically, it makes struct io_submit_state embedded into ctx, so
>>> easily accessible and persistent, and then plays a bit around that.
>>
>> Do you have any perf/latency numbers for this? Just curious if we
>> see any improvements on that front, cross submit persistence of
>> alloc caches should be a nice sync win, for example, or even
>> for peak iops by not having to replenish the pool for each batch.
>>
>> I can try and run some here too.
>>
>
> I tested the first version, but my drive is too slow, so it was only nops and
> hence no offloading. Honestly, there waren't statistically significant results.
> I'll rerun anyway.
>
> I have a plan to reuse it for a tricky optimisation, but thinking twice, I can
> just stash it until everything is done. That's not the first thing in TODO and
> will take a while.
>
I've got numbers, but there is nothing really interesting. Throughput is
insignificantly better with the patches, but I'd need much more experiments
across reboots to confirm that.
Let's postpone the patchset for later
--
Pavel Begunkov
On 1/31/20 5:31 PM, Pavel Begunkov wrote:
> On 01/02/2020 01:32, Pavel Begunkov wrote:
>> On 01/02/2020 01:22, Jens Axboe wrote:
>>> On 1/31/20 3:15 PM, Pavel Begunkov wrote:
>>>> Apart from unrelated first patch, this persues two goals:
>>>>
>>>> 1. start preparing io_uring to move resources handling into
>>>> opcode specific functions
>>>>
>>>> 2. make the first step towards long-standing optimisation ideas
>>>>
>>>> Basically, it makes struct io_submit_state embedded into ctx, so
>>>> easily accessible and persistent, and then plays a bit around that.
>>>
>>> Do you have any perf/latency numbers for this? Just curious if we
>>> see any improvements on that front, cross submit persistence of
>>> alloc caches should be a nice sync win, for example, or even
>>> for peak iops by not having to replenish the pool for each batch.
>>>
>>> I can try and run some here too.
>>>
>>
>> I tested the first version, but my drive is too slow, so it was only nops and
>> hence no offloading. Honestly, there waren't statistically significant results.
>> I'll rerun anyway.
>>
>> I have a plan to reuse it for a tricky optimisation, but thinking twice, I can
>> just stash it until everything is done. That's not the first thing in TODO and
>> will take a while.
>>
>
> I've got numbers, but there is nothing really interesting. Throughput is
> insignificantly better with the patches, but I'd need much more experiments
> across reboots to confirm that.
>
> Let's postpone the patchset for later
Sounds fine to me, no need to do it unless it's a nice cleanup, and/or
provides some nice improvements.
It would be great to see the splice stuff revamped, though :-)
--
Jens Axboe