Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <20210608231132.32012-1-joshdon@google.com> <YMobzbLecaFYuLtq@slm.duckdns.org>
 <CABk29NtcRUwskBjrvLKkEKQ0hpNPSrdzrGAGZy+bHSfnznOUSg@mail.gmail.com> <YNb6CL6Q9CJnbB2R@mtj.duckdns.org>
In-Reply-To: <YNb6CL6Q9CJnbB2R@mtj.duckdns.org>
From:   Josh Don <joshdon@google.com>
Date:   Mon, 28 Jun 2021 21:57:42 -0700
Message-ID: <CABk29NtqZH8nJ1NGuBVisGDBJxAK0948vRau7RhYA4O_bvJ-ww@mail.gmail.com>
Subject: Re: [PATCH] sched: cgroup SCHED_IDLE support
To:     Tejun Heo <tj@kernel.org>
Cc:     Ingo Molnar <mingo@redhat.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Juri Lelli <juri.lelli@redhat.com>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
        Daniel Bristot de Oliveira <bristot@redhat.com>,
        Paul Turner <pjt@google.com>,
        David Rientjes <rientjes@google.com>,
        Oleg Rombakh <olegrom@google.com>,
        Viresh Kumar <viresh.kumar@linaro.org>,
        Steve Sistare <steven.sistare@oracle.com>,
        linux-kernel <linux-kernel@vger.kernel.org>,
        Rik van Riel <riel@surriel.com>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Sat, Jun 26, 2021 at 2:57 AM Tejun Heo <tj@kernel.org> wrote:
[snip]
>
> Would you care to share some concrete use cases?
>
> Thank you.
>
> --
> tejun

Sure thing, there are two use cases we've found compelling:

1. On a machine, different users are given their own top-level cgroup
(configured with an appropriate number of shares). Each user is free
to spawn any threads and create any additional cgroups within their
top-level group.
Some users would like to run high priority, latency-sensitive work
(for example, responding to an RPC) as well as some batch tasks (ie.
background work such as data manipulation, transcoding, etc.) within
their cgroup. The batch tasks should interfere minimally with the high
priority work. However, it is still desired that this batch work be
considered the same as the high priority work vs the jobs of some
other user on the machine.

To achieve this, the user sets up two sub-cgroups, one of which is
marked as idle. The idle cgroup will always be preempted on wakeup of
a task in the other sub-cgroup (but not a wakeup of another user's
task). This is not possible with the per-task SCHED_IDLE setting.
Cgroup shares/weight alone is also not as strong as SCHED_IDLE.

2. We can create a top-level idle cgroup in which we place users who
want to run some best-effort work (ie. some long running
computations). Since it is the top-level cgroup that is marked idle,
any other task on the machine will always preempt something running
within the top-level idle cgroup. We can also easily maintain the
relative weights between different users within the idle group.

This top-level idle group allows for soaking otherwise unused cycles,
and offers cheap machine quota for users who have latency-tolerant
jobs.