Date: Thu, 27 Oct 2016 23:34:35 +0100
From: Patrick Bellasi <patrick.bellasi@arm.com>
To: Tejun Heo <tj@kernel.org>
Cc: linux-kernel@vger.kernel.org, Ingo Molnar <mingo@kernel.org>,
        Peter Zijlstra <peterz@infradead.org>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Steve Muckle <steve.muckle@linaro.org>, Leo Yan <leo.yan@linaro.org>,
        Viresh Kumar <viresh.kumar@linaro.org>,
        "Rafael J . Wysocki" <rjw@rjwysocki.net>, Todd Kjos <tkjos@google.com>,
        Srinath Sridharan <srinathsr@google.com>,
        Andres Oportus <andresoportus@google.com>,
        Juri Lelli <juri.lelli@arm.com>,
        Morten Rasmussen <morten.rasmussen@arm.com>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Chris Redpath <chris.redpath@arm.com>,
        Robin Randhawa <robin.randhawa@arm.com>, Li Zefan <lizefan@huawei.com>,
        Johannes Weiner <hannes@cmpxchg.org>, Ingo Molnar <mingo@redhat.com>
Subject: Re: [RFC v2 5/8] sched/tune: add initial support for CGroups based
 boosting
Message-ID: <20161027223435.GC3668@derkdell>
References: <20161027174108.31139-1-patrick.bellasi@arm.com>
 <20161027174108.31139-6-patrick.bellasi@arm.com>
 <20161027183017.GA15876@htj.duckdns.org>
 <20161027201439.GA3668@derkdell>
 <20161027203947.GA14477@htj.duckdns.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20161027203947.GA14477@htj.duckdns.org>
User-Agent: Mutt/1.5.24 (2015-08-30)
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 5517
Lines: 132

On 27-Oct 16:39, Tejun Heo wrote:
> Hello, Patrick.
> 
> On Thu, Oct 27, 2016 at 09:14:39PM +0100, Patrick Bellasi wrote:
> > I'm wondering also how much confusing and complex it can be to
> > configure a system where you have not overlapping groups of tasks with
> > different bandwidth and boosting requirements.
> > 
> > For example, let assume we have three tasks: A, B, and C and we want:
> > 
> >    Bandwidth:  10% for A and B,  20% for C
> >    Boost:      10% for A,         0% for B and C
> > 
> > IMO, configuring such a set of constraints would be quite complex if
> > we expose the boost value through the cpu controller.
> 
> Going back to your use case point, when would we realistically need
> this?

If we really want to be generic, we cannot exclude there could be this
kind of scenarios. What this toy example aims to show is just that, in
general, how much we want to boost a task can be decoupled from the
bandwidth reservation it shares with others.

> > > Note that hierarchy
> > > support doesn't necessarily mean that boosting itself has to be
> > > hierarchical.
> > 
> > Initially I've actually considered such a design, however...
> > 
> > >It can be, for example, something along the line of
> > > "the descendants are allowed upto this level of boosting" so that the
> > > hierarchy just serves to assign the appropriate boosting values to the
> > > groups of tasks.
> > 
> > ... the current "single layer hierarchy" has been proposed instead for
> > two main reasons.
> > 
> > First, we was not able to think about realistic use-cases where we
> > need this "up to this level" semantic.
> > For boosting purposes, tasks are grouped based on their role and/or
> > importance in the system. This property is usually defined in
> > "absolute" terms instead of "relative" therms.
> > Does it make sense to say that task A can be boosted only up to how
> > much is task B? In our experience probably never.
> 
> There are basic semantics that people expect when they use cgroup for
> resource control and it enables things like layering and delegating
> configuration.

I see your point and I understand it, still I'm not completely
convinced that these concepts (i.e. layering and delegation) are
really required for the specific topic of "tasks classification" for
the purposes of energy-vs-performance tuning.

Perhaps this boils down to the fact that, for the specific needs of
tasks boosting, from the "Control Group" framework we are less
interested in the "Control" component than in the "Group" one.

> > The second reason is mainly related to the possibility to have an
> > efficient and low-overhead implementation. The currently defined
> > semantic for CPU boosting requires to perform certain operations at
> > each task enqueue and dequeue events. Some of these operations are
> > part of the hot path in the scheduler code. The flat hierarchy allows
> > to use per-cpu data structures and algorithms which aims at being
> > efficient in reducing the overheads incurred in doing the required
> > accounting.
> 
> Unless I'm misunderstanding, the actually applied attributes should be
> calculable during config changes or task migration, right?

Perhaps you are missing enqueue/dequeue operations, which is not
necessarily only due to tasks migrations.

For example, the semantic exposed by SchedTune is such that if we have
two tasks RUNNABLE on a CPU:
  T1 30% boosted
  T2 60% boosted
then the CPU will be boosted 60%, while T2 is running, and boosted
only 30% as soon as T2 goes to sleep and only T1 is still runnable.

> The
> hierarchy's function would be allowing layering and delegating
> configurations and shouldn't get in the way of actual enforcement.

Ok, I should better think about this distinction between
layering/delegation and control enforcement.
 
> > As a final remark, I would like to say that Google is currently using
> > SchedTune in Android to classify tasks by "importance" and feed this
> > information into the scheduler. Doing this exercise, so far we did not
> > spot limitations related to the usage of a flat hierarchy.
> > 
> > However, I like to have this discussion, which it's actually the main
> > goal of this RFC. My suggestion is just that we should think about
> > use-cases before and than introduce a more complex solution, but only
> > if we convince ourself that it can bring more benefits than burdens in
> > code maintainability.
> > 
> > Is your request for a "proper support for hierarchy" somehow related to
> > the requirements for the "unified hierarchy"? Or do you see also other
> > more functional/semantic aspects?
> 
> Not necessarily.  In general, all controllers, whether on v1 or v2,
> should be fully hierarchical for reasons mentioned above.  I get that
> flat was fine for android but flat hierarchy would be fine for most
> controllers for android.  That's not the only use case we should be
> considering, right?

So far we had experience mainly with Android and ChromeOS, which
exposes a valuable and quite interesting set of realistic use-cases,
especially if we consider the possibility to collect task's
information from "informed runtimes".

However, I absolutely agree with you, it's worth to consider all the
other use-cases we can think about.

> > > Thanks.
> > 
> > If you are going to attend LPC next week, I hope we can have a chat on
> > these topics.
> 
> Yeah, sure, I'll be around till Thursday.  Let's chat there.

Cool, thanks!

Cheers Patrick

-- 
#include <best/regards.h>

Patrick Bellasi