Received-SPF: pass (google.com: domain of linux-kernel+bounces-179385-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161;
Message-ID: <3ed32279-904a-411d-91a4-a62f4ca2dde2@redhat.com>
Date: Tue, 14 May 2024 23:59:55 -0400
Precedence: bulk
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: cgroup io.stat propagation regression
To: Dan Schatzberg <schatzberg.dan@gmail.com>, Tejun Heo <tj@kernel.org>
Cc: linux-block@vger.kernel.org, cgroups@vger.kernel.org,
 linux-kernel@vger.kernel.org
References: <ZkO6l/ODzadSgdhC@dschatzberg-fedora-PF3DHTBV>
Content-Language: en-US
From: Waiman Long <longman@redhat.com>
In-Reply-To: <ZkO6l/ODzadSgdhC@dschatzberg-fedora-PF3DHTBV>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 5/14/24 15:25, Dan Schatzberg wrote:
> Hi Waiman,
>
> I've noticed that on recent kernels io.stat metrics don't propagate
> all the way up the hierarchy. Specifically, io.stat metrics of some
> leaf cgroup will be propagated to the parent, but not its grandparent.
>
> For a simple repro, run the following:
>
> systemd-run --slice test-test dd if=/dev/urandom of=/tmp/test bs=4096 count=1
>
> Then:
>
> cat /sys/fs/cgroup/test.slice/test-test.slice/io.stat
>
> Shows the parent cgroup stats and I see wbytes=4096 but the grandparent cgroup:
>
> cat /sys/fs/cgroup/test.slice/io.stat
>
> shows no writes.
>
> I believe this was caused by the change in "blk-cgroup: Optimize
> blkcg_rstat_flush()". When blkcg_rstat_flush is called on the parent
> cgroup, it exits early because the lockless list is empty since the
> parent cgroup never issued writes itself (e.g. in
> blk_cgroup_bio_start). However, in doing so it never propagated stats
> to its parent.
>
> Can you confirm if my understanding of the logic here is correct and
> advise on a fix?

Yes, I believe your analysis is correct. Thanks for spotting this iostat 
propagation problem.

I am working on a fix to address this problem and will post a patch once 
I have finished my testing.

Thanks,
Longman