Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Benjamin Segall <bsegall@google.com>
To:     Phil Auld <pauld@redhat.com>
Cc:     linux-kernel@vger.kernel.org, Juri Lelli <juri.lelli@redhat.com>,
        Ingo Molnar <mingo@redhat.com>,
        Daniel Bristot de Oliveira <bristot@redhat.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Valentin Schneider <vschneid@redhat.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Mel Gorman <mgorman@suse.de>,
        Frederic Weisbecker <frederic@kernel.org>,
        Tejun Heo <tj@kernel.org>
Subject: Re: [PATCH v2 1/2] sched, cgroup: Restore meaning to
 hierarchical_quota
References: <20230712133357.381137-1-pauld@redhat.com>
        <20230712133357.381137-2-pauld@redhat.com>
        <xm268rbkg4tg.fsf@google.com>
        <20230713132306.GA13342@lorien.usersys.redhat.com>
Date:   Thu, 13 Jul 2023 13:12:04 -0700
In-Reply-To: <20230713132306.GA13342@lorien.usersys.redhat.com> (Phil Auld's
        message of "Thu, 13 Jul 2023 09:23:06 -0400")
Message-ID: <xm26zg3zefl7.fsf@google.com>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
Precedence: bulk

Phil Auld <pauld@redhat.com> writes:

> On Wed, Jul 12, 2023 at 03:09:31PM -0700 Benjamin Segall wrote:
>> Phil Auld <pauld@redhat.com> writes:
>> 
>> > In cgroupv2 cfs_b->hierarchical_quota is set to -1 for all task
>> > groups due to the previous fix simply taking the min.  It should
>> > reflect a limit imposed at that level or by an ancestor. Even
>> > though cgroupv2 does not require child quota to be less than or
>> > equal to that of its ancestors the task group will still be
>> > constrained by such a quota so this should be shown here. Cgroupv1
>> > continues to set this correctly.
>> >
>> > In both cases, add initialization when a new task group is created
>> > based on the current parent's value (or RUNTIME_INF in the case of
>> > root_task_group). Otherwise, the field is wrong until a quota is
>> > changed after creation and __cfs_schedulable() is called.
>> >
>> > Fixes: c53593e5cb69 ("sched, cgroup: Don't reject lower cpu.max on ancestors")
>> > Signed-off-by: Phil Auld <pauld@redhat.com>
>> > Reviewed-by: Ben Segall <bsegall@google.com>
>> > Cc: Ingo Molnar <mingo@redhat.com>
>> > Cc: Peter Zijlstra <peterz@infradead.org>
>> > Cc: Vincent Guittot <vincent.guittot@linaro.org>
>> > Cc: Juri Lelli <juri.lelli@redhat.com>
>> > Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>
>> > Cc: Valentin Schneider <vschneid@redhat.com>
>> > Cc: Ben Segall <bsegall@google.com>
>> > Cc: Frederic Weisbecker <frederic@kernel.org>
>> > Cc: Tejun Heo <tj@kernel.org>
>> > ---
>> >
>> > v2: Improve comment about how setting hierarchical_quota correctly
>> >
>> > helps the scheduler. Remove extra parens.
>> >  kernel/sched/core.c  | 13 +++++++++----
>> >  kernel/sched/fair.c  |  7 ++++---
>> >  kernel/sched/sched.h |  2 +-
>> >  3 files changed, 14 insertions(+), 8 deletions(-)
>> >
>> > diff --git a/kernel/sched/core.c b/kernel/sched/core.c
>> > index a68d1276bab0..f80697a79baf 100644
>> > --- a/kernel/sched/core.c
>> > +++ b/kernel/sched/core.c
>> > @@ -9904,7 +9904,7 @@ void __init sched_init(void)
>> >  		ptr += nr_cpu_ids * sizeof(void **);
>> >  
>> >  		root_task_group.shares = ROOT_TASK_GROUP_LOAD;
>> > -		init_cfs_bandwidth(&root_task_group.cfs_bandwidth);
>> > +		init_cfs_bandwidth(&root_task_group.cfs_bandwidth, NULL);
>> >  #endif /* CONFIG_FAIR_GROUP_SCHED */
>> >  #ifdef CONFIG_RT_GROUP_SCHED
>> >  		root_task_group.rt_se = (struct sched_rt_entity **)ptr;
>> > @@ -11038,11 +11038,16 @@ static int tg_cfs_schedulable_down(struct task_group *tg, void *data)
>> >  
>> >  		/*
>> >  		 * Ensure max(child_quota) <= parent_quota.  On cgroup2,
>> > -		 * always take the min.  On cgroup1, only inherit when no
>> > -		 * limit is set:
>> > +		 * always take the non-RUNTIME_INF min.  On cgroup1, only
>> > +		 * inherit when no limit is set. In cgroup2 this is used
>> > +		 * by the scheduler to determine if a given CFS task has a
>> > +		 * bandwidth constraint at some higher level.
>> >  		 */
>> 
>> It's still used for determining this on cgroup1 (and the cgroup1 code
>> still works for that), right?
>>
>
> It would, except that the enforcement of child quota <= parent quota
> means that cfs_rq->runtime_enabled will be set and we'll hit that first
> on cgroup1.  So we don't really use it for this determination in
> cgroup1.

cgroup1 tg_cfs_schedulable_down only constricts child quota when it's
set. You can set quota=RUNTIME_INF on any cgroup, no matter what its
parent is. (The schedulable constraint is a little silly)