Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751786AbdH1P1X (ORCPT ); Mon, 28 Aug 2017 11:27:23 -0400 Received: from mail-wm0-f67.google.com ([74.125.82.67]:33953 "EHLO mail-wm0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751453AbdH1P1V (ORCPT ); Mon, 28 Aug 2017 11:27:21 -0400 Date: Mon, 28 Aug 2017 17:27:15 +0200 From: Frederic Weisbecker To: Peter Zijlstra Cc: LKML , Chris Metcalf , Thomas Gleixner , Luiz Capitulino , Christoph Lameter , "Paul E . McKenney" , Ingo Molnar , Mike Galbraith , Rik van Riel , Wanpeng Li Subject: Re: [RFC PATCH 12/12] housekeeping: Reimplement isolcpus on housekeeping Message-ID: <20170828152714.GB32618@lerouge> References: <1503453071-952-1-git-send-email-fweisbec@gmail.com> <1503453071-952-13-git-send-email-fweisbec@gmail.com> <20170828100957.jcjhh77ylxvsyisy@hirez.programming.kicks-ass.net> <20170828132302.GA32618@lerouge> <20170828133116.zu3xujkkmb4cmks2@hirez.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170828133116.zu3xujkkmb4cmks2@hirez.programming.kicks-ass.net> User-Agent: Mutt/1.5.24 (2015-08-30) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2213 Lines: 45 On Mon, Aug 28, 2017 at 03:31:16PM +0200, Peter Zijlstra wrote: > On Mon, Aug 28, 2017 at 03:23:06PM +0200, Frederic Weisbecker wrote: > > On Mon, Aug 28, 2017 at 12:09:57PM +0200, Peter Zijlstra wrote: > > > On Wed, Aug 23, 2017 at 03:51:11AM +0200, Frederic Weisbecker wrote: > > > > We want to centralize the isolation features on the housekeeping > > > > subsystem and scheduler isolation is a significant part of it. > > > > > > > > While at it, this is a proposition for a reimplementation of isolcpus= > > > > that doesn't involve scheduler domain isolation. Therefore this > > > > brings a behaviour change: all user tasks inherit init/1 affinity which > > > > avoid the isolcpus= range. But if a task later overrides its affinity > > > > which turns out to intersect an isolated CPU, load balancing may occur > > > > on it. > > > > > > > > OTOH such a reimplementation that doesn't shortcut scheduler internals > > > > makes a better candidate for an interface extension to cpuset. > > > > > > Not sure we can do this. It'll break users that rely on the no > > > scheduling thing, that's a well documented part of isolcpus. > > > > That was my worry :-s That NULL domain was probably a design mistake and > > I fear we now have to maintain it. > > I'm fairly sure that was very intentional. If you want to isolate stuff > you don't want load-balancing. Yes I guess that was intentional. In fact having NULL domains is convenient as it also isolates from many things: tasks, workqueues, timers. Although for example I guess (IIUC) that if you create an unbound timer on a NULL domain, it will be stuck on it for ever as we can't walk any hierarchy from the current CPU domain. I'm not sure how much that can apply to unbound workqueues as well. But the thing is with NULL domains: things can not migrate in and neither can them migrate out, which is not exactly what CPU isolation wants. > You get the same NULL domain with cpusets if you disable balancing for a set of CPUs. Ok, I didn't know that. > > Now, I completely hate the isolcpus feature and wish is a speedy death, > but replacing it with something sensible is difficult because cgroups > :-( Ah, that would break cgroup somehow?