Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Message-ID: <1516208339.31983.58.camel@gmx.de>
Subject: Re: [GIT PULL] isolation: 1Hz residual tick offloading v3
From:   Mike Galbraith <efault@gmx.de>
To:     Christopher Lameter <cl@linux.com>
Cc:     Frederic Weisbecker <frederic@kernel.org>,
        Luiz Capitulino <lcapitulino@redhat.com>,
        Ingo Molnar <mingo@kernel.org>,
        LKML <linux-kernel@vger.kernel.org>,
        Peter Zijlstra <peterz@infradead.org>,
        Chris Metcalf <cmetcalf@mellanox.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        "Paul E . McKenney" <paulmck@linux.vnet.ibm.com>,
        Wanpeng Li <kernellwp@gmail.com>,
        Rik van Riel <riel@redhat.com>
Date:   Wed, 17 Jan 2018 17:58:59 +0100
In-Reply-To: <alpine.DEB.2.20.1801171028460.21811@nuc-kabylake>
References: <1515039937-367-1-git-send-email-frederic@kernel.org>
         <20180112141813.32dcc84d@redhat.com> <20180116154055.GA27042@lerouge>
         <1516125498.6599.23.camel@gmx.de>
         <alpine.DEB.2.20.1801170847570.12151@nuc-kabylake>
         <1516204793.31983.39.camel@gmx.de>
         <alpine.DEB.2.20.1801171028460.21811@nuc-kabylake>
Content-Type: text/plain; charset="ISO-8859-15"
Mime-Version: 1.0
Content-Transfer-Encoding: 8BIT
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Wed, 2018-01-17 at 10:32 -0600, Christopher Lameter wrote:
> On Wed, 17 Jan 2018, Mike Galbraith wrote:
> 
> > Domain connectivity very much is a property of a set of CPUs, a rather
> > important one, and one managed by cpusets. ?NOHZ_FULL is a property of
> > a set of cpus, thus a most excellent fit. ?Other things are as well.
> 
> Not sure to what domain refers to in this context.

Scheduler domains, load balancing.

> > > We have sets of cpus associated with affinity masks in the form of bitmaps
> > > etc etc which is much more lightweight than having slug around the cgroup
> > > overhead everywhere.
> >
> > What does everywhere mean, set creation time?
> 
> You would need to create multiple cgroups to create what you want. Those
> will "inherit" characteristics from higher levels etc etc. It gets
> needlessly complicated and difficult to debug if something goes work.

It's only as complicated as you make it. ?What I create is dirt simple,
an exclusive system set and an exclusive realtime set, both directly
under root. ?It doesn't get any simpler than that.

> > > A simple bitmask is much better if you have to control detailed system
> > > behavior for each core and are planning each processes role because you
> > > need to make full use of the harware resources available.
> >
> > If you live in a static world, maybe.
> 
> Why would that be restricted to a static world?

Guess I misunderstood, unimportant.

> > I like the flexibility of being able to configure on the fly. ?One tiny
> > example: for a high performance aircraft manufacturer, having military
> > simulation background, I know that simulators frequently have to be
> > ready to go at the drop of a hat, so I twiddled cpusets to let them
> > flip their extra fancy video game (80 cores, real controls/avionics...
> > "game over, insert one gold bar to continue" kind of fancy) from low
> > power idle to full bore hard realtime with one poke to a cpuset file.
> >
> > Static may be fine for some, for others, dynamic is much better.
> 
> The problem is that I may be flipping a flag in a cpuset to enable
> something but some other cpuset somewhere in the complex hieracy does
> something different that causes a conflict.

That's what exclusive sets are for, zero set overlap. ?It would be very
difficult to both connect and disconnect scheduler domains :)

>  The directness to control is
> lost. Instead there is the fog of complexity created by the cgroups that
> have various plugins and whatnot.

You don't have to use any of the other controllers, I don't, just tell
systemthing to pretty please NOT co-mount controllers, and whatever to
ensure it keeps its tentacles off of your toys, and you're fine.

	-Mike