Message-ID: <13cad320-db43-99db-ed0c-61b27a9a3447@gmail.com>
Date:   Fri, 14 Jan 2022 10:01:31 +0800
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0)
 Gecko/20100101 Thunderbird/91.4.1
Subject: Re: [PATCH 13/13] blk: introduce iostat per cgroup module
Content-Language: en-US
To:     Tejun Heo <tj@kernel.org>
Cc:     axboe@kernel.dk, jbacik@fb.com, bvanassche@acm.org,
        linux-block@vger.kernel.org, linux-kernel@vger.kernel.org
References: <20220110091046.17010-1-jianchao.wan9@gmail.com>
 <20220110091046.17010-14-jianchao.wan9@gmail.com>
 <Yd82dJMxdQkssu4k@slm.duckdns.org>
 <faafdf29-6d8b-82a8-c69d-4ab3e623b489@gmail.com>
 <YeBa68w4nQQvBM9Z@slm.duckdns.org>
From:   Wang Jianchao <jianchao.wan9@gmail.com>
In-Reply-To: <YeBa68w4nQQvBM9Z@slm.duckdns.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Precedence: bulk

Hi Tejun

Thanks so much your comment :)
I really appreciate it.

On 2022/1/14 1:01 上午, Tejun Heo wrote:
> Hello,
> 
> On Thu, Jan 13, 2022 at 10:40:27AM +0800, Wang Jianchao wrote:
>> bw/iops/lat of data or metadata of one cgroup is very basic statistics
> 
> bw and iops can already be derived from the cumulative counters in io.stat.
> Latencies without distribution are often what we tradtionally exposed not
> because they're great but because exposing distributions is so cumbersome
> from kernel. Also, latency numbers at cgroup level isn't *that* useful
> anyway. There isn't a lot you can deduce for the particular cgroup from that
> number.
I agree with this. But my customer would yell at me because they really want to
see that....
> 
>> which kernel could provide especially when cgroup is employed everywhere.
>> And we love to collect them all the time during the instance in cgroup is
>> running.
> 
> It's really not that difficult to collect these numbers from bpf with pretty
> low overhead. There's the problem of deployment for which there isn't "the"
> right and convenient way but it'd be more worthwhile to put efforts towards
> that.
> 
>>> What if I want to know what portions are random, or the
>>> distribution of IO sizes? 
>> This looks really detailed statistics :)
>>> Do I add another rq-qos policy or add another
>>> interface file with interface versioning?
>> This iostat module can not provide all the kinds of statistics we want
>> but just some very basic things. And maybe it can provide better hooks
>> to install the ebpf program to collect detailed statistics.
> 
> I mean, "really basic" means different things for different folks. Bytes /
> ios, we all seem to agree. Beyond that, who knows? There already are enough
> hooks to collect stats that you're trying to collect. The examples in py-bcc
> might be a good place to start.
Thanks for your suggestion. I will read it and try with bpf

Best Regards
Jianchao