Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Subject: Re: Task group cleanups and optimizations (was: Re: [RFC 00/60]
 Coscheduling for Linux)
To:     Frederic Weisbecker <frederic@kernel.org>
Cc:     Peter Zijlstra <peterz@infradead.org>,
        Ingo Molnar <mingo@redhat.com>, linux-kernel@vger.kernel.org,
        Paul Turner <pjt@google.com>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Morten Rasmussen <morten.rasmussen@arm.com>,
        Tim Chen <tim.c.chen@linux.intel.com>,
        Rik van Riel <riel@surriel.com>
References: <20180907214047.26914-1-jschoenh@amazon.de>
 <20180914111251.GC24106@hirez.programming.kicks-ass.net>
 <1d86f497-9fef-0b19-50d6-d46ef1c0bffa@amazon.de>
 <282230fe-b8de-01f9-c19b-6070717ba5f8@amazon.de>
 <20180917094844.GR24124@hirez.programming.kicks-ass.net>
 <08b930d9-7ffe-7df3-ab35-e7b58073e489@amazon.de>
 <20181123165102.GB4855@lerouge>
From:   =?UTF-8?Q?Jan_H=2e_Sch=c3=b6nherr?= <jschoenh@amazon.de>
Openpgp: preference=signencrypt
Message-ID: <0f025121-d3b2-36c9-5c0c-7db1d118f317@amazon.de>
Date:   Tue, 4 Dec 2018 14:23:37 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.3.1
MIME-Version: 1.0
In-Reply-To: <20181123165102.GB4855@lerouge>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On 23/11/2018 17.51, Frederic Weisbecker wrote:
> On Tue, Sep 18, 2018 at 03:22:13PM +0200, Jan H. Schönherr wrote:
>> On 09/17/2018 11:48 AM, Peter Zijlstra wrote:
>>> Right, so the whole bandwidth thing becomes a pain; the simplest
>>> solution is to detect the throttle at task-pick time, dequeue and try
>>> again. But that is indeed quite horrible.
>>>
>>> I'm not quite sure how this will play out.
>>>
>>> Anyway, if we pull off this flattening feat, then you can no longer use
>>> the hierarchy for this co-scheduling stuff.
>>
>> Yeah. I might be a bit biased towards keeping or at least not fully throwing away
>> the nesting of CFS runqueues. ;)
> 
> One detail here, is that hierarchical task group a strong requirement for cosched
> or could you live with it flattened in the end?

Currently, it is a strong requirement.

As mentioned at the bottom of https://lkml.org/lkml/2018/10/19/859 it should be
possible to pull the hierarchical aspect out of CFS and implement it one level
higher. But that would be a major re-design of everything.

I use the hierarchical aspect to a) keep coscheduled groups in separate sets of runqeues,
so that it is easy to select/balance tasks within a particular group; and b) to implement
per-core, per-node, per-system runqueues that represent larger fractions of the system,
which then fan out into per-CPU runqueues (eventually).

Regards
Jan