Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: [PATCH] iosched: Add i10 I/O Scheduler
To:     Rachit Agarwal <rach4x0r@gmail.com>
Cc:     Jens Axboe <axboe@kernel.dk>, Christoph Hellwig <hch@lst.de>,
        linux-block@vger.kernel.org, linux-nvme@lists.infradead.org,
        linux-kernel@vger.kernel.org, Keith Busch <kbusch@kernel.org>,
        Ming Lei <ming.lei@redhat.com>,
        Jaehyun Hwang <jaehyun.hwang@cornell.edu>,
        Qizhe Cai <qc228@cornell.edu>,
        Midhul Vuppalapati <mvv25@cornell.edu>,
        Sagi Grimberg <sagi@lightbitslabs.com>,
        Rachit Agarwal <ragarwal@cornell.edu>
References: <20201112140752.1554-1-rach4x0r@gmail.com>
 <5a954c4e-aa84-834d-7d04-0ce3545d45c9@kernel.dk>
 <da0c7aea-d917-4f3a-5136-89c30d12ba1f@grimberg.me>
 <fd12993a-bcb7-7b45-5406-61da1979d49d@kernel.dk>
 <10993ce4-7048-a369-ea44-adf445acfca7@grimberg.me>
 <c4cb66f6-8a66-7973-dc03-0f4f61d0a1e4@kernel.dk>
 <cbe18a3d-8a6b-e775-81bb-3b3f11045183@grimberg.me>
 <26a1cd20-6b25-eaa6-7ab6-ba7f5afaf6dd@kernel.dk>
 <81cdcb58-9a23-8192-6213-7f2408a3b8ee@grimberg.me>
 <CAKeUqKKHg1wD19pnwJEd8whubnuGVic_ZhDjebaq3kKmY9TtsQ@mail.gmail.com>
From:   Sagi Grimberg <sagi@grimberg.me>
Message-ID: <0cce60db-5a41-2e1b-ba5d-7905966bec25@grimberg.me>
Date:   Mon, 30 Nov 2020 11:20:30 -0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <CAKeUqKKHg1wD19pnwJEd8whubnuGVic_ZhDjebaq3kKmY9TtsQ@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Precedence: bulk


> Dear all:
> 
> Thanks, again, for the very constructive decisions.
> 
> I am writing back with quite a few updates:
> 
> 1. We have now included a detailed comparison of i10 scheduler with 
> Kyber with NVMe-over-TCP 
> (https://github.com/i10-kernel/upstream-linux/blob/master/i10-evaluation.pdf). 
> In a nutshell, when operating with NVMe-over-TCP, i10 demonstrates the 
> core tradeoff: higher latency, but also higher throughput. This seems to 
> be the core tradeoff exposed by i10.
> 
> 2. We have now implemented an adaptive version of i10 I/O scheduler, 
> that uses the number of outstanding requests at the time of batch 
> dispatch (and whether the dispatch was triggered by timeout or not) to 
> adaptively set the batching size. The new results 
> (https://github.com/i10-kernel/upstream-linux/blob/master/i10-evaluation.pdf) 
> show that i10-adaptive further improves performance for low loads, while 
> keeping the performance for high loads. IMO, there is much to do on 
> designing improved adaptation algorithms.
> 
> 3. We have now updated the i10-evaluation document to include results 
> for local storage access. The core take-away here is that i10-adaptive 
> can achieve similar throughput and latency at low loads and at high 
> loads when compared to noop, but still requires more work for lower 
> loads. However, given that the tradeoff exposed by i10 scheduler is 
> particularly useful for remote storage devices (and as Jens suggested, 
> perhaps for virtualized local storage access), I do agree with Sagi -- I 
> think we should consider including it in the core, since this may be 
> useful for a broad range of new use cases.
> 
> We have also created a second version of the patch that includes these 
> updates: 
> https://github.com/i10-kernel/upstream-linux/blob/master/0002-iosched-Add-i10-I-O-Scheduler.patch
> 
> As always, thank you for the constructive discussion and I look forward 
> to working with you on this.

Thanks Rachit,

Would be good if you can send a formal patch for the adaptive queuing so
people can have a look.

One thing that was left on the table is weather this should be a full
blown scheduler or a core block infrastructure that would either be
set on-demand or by default.

I think that Jens and Ming expressed that this should be something that
we should place this in the block core, but I'd like to hear maybe
Ming can elaborate on his ideas how to do this.