Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Message-ID: <1516204793.31983.39.camel@gmx.de>
Subject: Re: [GIT PULL] isolation: 1Hz residual tick offloading v3
From:   Mike Galbraith <efault@gmx.de>
To:     Christopher Lameter <cl@linux.com>
Cc:     Frederic Weisbecker <frederic@kernel.org>,
        Luiz Capitulino <lcapitulino@redhat.com>,
        Ingo Molnar <mingo@kernel.org>,
        LKML <linux-kernel@vger.kernel.org>,
        Peter Zijlstra <peterz@infradead.org>,
        Chris Metcalf <cmetcalf@mellanox.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        "Paul E . McKenney" <paulmck@linux.vnet.ibm.com>,
        Wanpeng Li <kernellwp@gmail.com>,
        Rik van Riel <riel@redhat.com>
Date:   Wed, 17 Jan 2018 16:59:53 +0100
In-Reply-To: <alpine.DEB.2.20.1801170847570.12151@nuc-kabylake>
References: <1515039937-367-1-git-send-email-frederic@kernel.org>
         <20180112141813.32dcc84d@redhat.com> <20180116154055.GA27042@lerouge>
         <1516125498.6599.23.camel@gmx.de>
         <alpine.DEB.2.20.1801170847570.12151@nuc-kabylake>
Content-Type: text/plain; charset="ISO-8859-15"
Mime-Version: 1.0
Content-Transfer-Encoding: 8BIT
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Wed, 2018-01-17 at 08:51 -0600, Christopher Lameter wrote:
> On Tue, 16 Jan 2018, Mike Galbraith wrote:
> 
> > > I tried to remove isolcpus or at least change the way it works so that its
> > > effects are reversible (ie: affine the init task instead of isolating domains)
> > > but that got nacked due to the behaviour's expectations for userspace.
> >
> > So we paint ourselves into a static corner forever more, despite every
> > bit of this being all about "properties of sets of cpus", ie precisely
> > what cpusets was born to do. ?That's sad, dynamic wasn't that far away.
> 
> cpusets was born in order to isolate applications to sets of processors.
> The properties of sets of cpus was not on the horizon when SGI started
> this.

Domain connectivity very much is a property of a set of CPUs, a rather
important one, and one managed by cpusets. ?NOHZ_FULL is a property of
a set of cpus, thus a most excellent fit. ?Other things are as well.

> We have sets of cpus associated with affinity masks in the form of bitmaps
> etc etc which is much more lightweight than having slug around the cgroup
> overhead everywhere.

What does everywhere mean, set creation time?

> A simple bitmask is much better if you have to control detailed system
> behavior for each core and are planning each processes role because you
> need to make full use of the harware resources available.

If you live in a static world, maybe.

I like the flexibility of being able to configure on the fly. ?One tiny
example: for a high performance aircraft manufacturer, having military
simulation background, I know that simulators frequently have to be
ready to go at the drop of a hat, so I twiddled cpusets to let them
flip their extra fancy video game (80 cores, real controls/avionics...
"game over, insert one gold bar to continue" kind of fancy) from low
power idle to full bore hard realtime with one poke to a cpuset file.

Static may be fine for some, for others, dynamic is much better.

	-Mike