Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751744AbdH1QY1 (ORCPT ); Mon, 28 Aug 2017 12:24:27 -0400 Received: from bombadil.infradead.org ([65.50.211.133]:40301 "EHLO bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751684AbdH1QY0 (ORCPT ); Mon, 28 Aug 2017 12:24:26 -0400 Date: Mon, 28 Aug 2017 18:24:16 +0200 From: Peter Zijlstra To: Frederic Weisbecker Cc: LKML , Chris Metcalf , Thomas Gleixner , Luiz Capitulino , Christoph Lameter , "Paul E . McKenney" , Ingo Molnar , Mike Galbraith , Rik van Riel , Wanpeng Li Subject: Re: [RFC PATCH 12/12] housekeeping: Reimplement isolcpus on housekeeping Message-ID: <20170828162416.nmdfvutqaki4sahx@hirez.programming.kicks-ass.net> References: <1503453071-952-1-git-send-email-fweisbec@gmail.com> <1503453071-952-13-git-send-email-fweisbec@gmail.com> <20170828100957.jcjhh77ylxvsyisy@hirez.programming.kicks-ass.net> <20170828132302.GA32618@lerouge> <20170828133116.zu3xujkkmb4cmks2@hirez.programming.kicks-ass.net> <20170828152714.GB32618@lerouge> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170828152714.GB32618@lerouge> User-Agent: NeoMutt/20170609 (1.8.3) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2097 Lines: 52 On Mon, Aug 28, 2017 at 05:27:15PM +0200, Frederic Weisbecker wrote: > On Mon, Aug 28, 2017 at 03:31:16PM +0200, Peter Zijlstra wrote: > > I'm fairly sure that was very intentional. If you want to isolate stuff > > you don't want load-balancing. > > Yes I guess that was intentional. In fact having NULL domains is convenient > as it also isolates from many things: tasks, workqueues, timers. Huh, what? That's entirely unrelated to the NULL domain. The reason people like isolcpus= is that is ensures _nothing_ runs on those CPUs before you explicitly place something there. _That_ is what ensures there are no timers etc.. placed on those CPUs. Once you run something on that CPU, it stays there. It is also what I dislike about isolcpus, its a boot time feature, if you want to reconfigure your system you need a reboot. > Although for example I guess (IIUC) that if you create an unbound > timer on a NULL domain, it will be stuck on it for ever as we can't > walk any hierarchy from the current CPU domain. Not sure what you're on about. Timers have their own hierarchy. > I'm not sure how much that can apply to unbound workqueues > as well. Well, unbound workqueued will not immediately end up on those CPUs, since they'll have an affinity exlusive of those CPUs per construction. But IIRC there's an affinity setting for workqueues where you could force it on if you wanted to. > But the thing is with NULL domains: things can not migrate in and neither > can them migrate out, which is not exactly what CPU isolation wants. No, its exactly what they want. You get what you put in and nothing more. If you want something else, use cpusets. > > Now, I completely hate the isolcpus feature and wish is a speedy death, > > but replacing it with something sensible is difficult because cgroups > > :-( > > Ah, that would break cgroup somehow? Well, ideally something like this would start the system with all the 'crap' threads in !root cgroup. But that means cgroupfs needs to be populated with at least two directories on boot. And current cgroup cruft doesn't expect that.