Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <b4424767-dce7-08a9-3759-43cc9dfa4273@bytedance.com>
Date:   Wed, 19 Jul 2023 11:01:48 +0800
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0)
 Gecko/20100101 Thunderbird/102.12.0
Subject: Re: [External] Re: [PATCH] cgroup/rstat: record the cumulative
 per-cpu time of cgroup and its descendants
To:     Tejun Heo <tj@kernel.org>
Cc:     lizefan.x@bytedance.com, hannes@cmpxchg.org, mkoutny@suse.com,
        cgroups@vger.kernel.org, linux-kernel@vger.kernel.org
References: <20230717093612.40846-1-jiahao.os@bytedance.com>
 <ZLWb-LsBD041hMvr@slm.duckdns.org>
 <2655026d-6ae4-c14c-95b0-4177eefa434f@bytedance.com>
 <ZLcJ1nH8KzWzoQWj@slm.duckdns.org>
From:   Hao Jia <jiahao.os@bytedance.com>
In-Reply-To: <ZLcJ1nH8KzWzoQWj@slm.duckdns.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk


On 2023/7/19 Tejun Heo wrote:
> On Tue, Jul 18, 2023 at 06:08:50PM +0800, Hao Jia wrote:
>> https://github.com/jiaozhouxiaojia/cgv2-stat-percpu_test/tree/main
> 
> So, we run `stress -c 1` for 1 second in the asdf/test0 cgroup and
> asdf/cpu.stat correctly reports the cumulative usage. After removing
> asdf/test0 cgroup, asdf's usage_usec is still there. What's missing here?

Sorry, some of my expressions may have misled you.

Yes, cpu.stat will display the cumulative **global** cpu time of the 
cgroup and its descendants (the corresponding kernel variable is 
"cgrp->bstat"), and it will not be lost when the child cgroup is removed.

Similarly, we need a **per-cpu** variable to record the accumulated 
per-cpu time of cgroup and its descendants.
The existing kernel variable "cgroup_rstat_cpu(cgrp, cpu)->bstat" is not 
satisfied, it only records the per-cpu time of cgroup itself,
So I try to add "cgroup_rstat_cpu(cgrp, cpu)->cumul_bstat" to record 
per-cpu time of cgroup and its descendants.

In order to verify the correctness of my patch, I wrote a kernel module 
to compare the results of calculating the per-cpu time of cgroup and its 
descendants in two ways:
   Method 1. Traverse and add the per-cpu rstatc->bstat of cgroup and 
each of its descendants.
   Method 2. Directly read "cgroup_rstat_cpu(cgrp, cpu)->cumul_bstat" in 
the kernel.

When the child cgroup is not removed, the results calculated by the two 
methods should be equal.

> What are you adding?
I want to add a **per-cpu variable** to record the cumulative per-cpu 
time of cgroup and its descendants, which is similar to the variable 
"cgrp->bstat", but it is a per-cpu variable.
It is very useful and convenient for calculating the usage of cgroup on 
each cpu, and its behavior is similar to the "cpuacct.usage*" interface 
of cgroup v1.

Thanks,
Hao