Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756047AbbEUOq2 (ORCPT ); Thu, 21 May 2015 10:46:28 -0400 Received: from mail-wg0-f45.google.com ([74.125.82.45]:33363 "EHLO mail-wg0-f45.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753309AbbEUOqY (ORCPT ); Thu, 21 May 2015 10:46:24 -0400 Date: Thu, 21 May 2015 16:46:20 +0200 From: Frederic Weisbecker To: "Paul E. McKenney" Cc: Afzal Mohammed , Mike Galbraith , Sasha Levin , Ingo Molnar , LKML , Chris Metcalf , "Rafael J . Wysocki" , Peter Zijlstra , Dave Jones , Thomas Gleixner , Oleg Nesterov , Ingo Molnar , Rik van Riel , Martin Schwidefsky Subject: Re: [PATCH 4/4] nohz: Set isolcpus when nohz_full is set Message-ID: <20150521144619.GB14324@lerouge> References: <1430928266-24888-5-git-send-email-fweisbec@gmail.com> <55579CE0.5060801@gmail.com> <1431840650.3222.78.camel@gmail.com> <20150520203809.GA2940@afzalpc> <20150520210026.GC6776@linux.vnet.ibm.com> <20150521121246.GA4723@afzalpc> <20150521125759.GL6776@linux.vnet.ibm.com> <20150521130621.GA14324@lerouge> <20150521132905.GA8706@afzalpc> <20150521141409.GO6776@linux.vnet.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150521141409.GO6776@linux.vnet.ibm.com> User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2530 Lines: 51 On Thu, May 21, 2015 at 07:14:09AM -0700, Paul E. McKenney wrote: > On Thu, May 21, 2015 at 06:59:05PM +0530, Afzal Mohammed wrote: > > Hi, > > > > On Thu, May 21, 2015 at 03:06:23PM +0200, Frederic Weisbecker wrote: > > > On Thu, May 21, 2015 at 05:57:59AM -0700, Paul E. McKenney wrote: > > > > > > Indeed, NO_HZ_FULL is special purpose. You normally would select > > > > NO_HZ_FULL_ALL only on a system intended for heavy compute without > > > > normal-workload distractions or for some real-time systems. For mixed > > > > workloads, you would build with NO_HZ_FULL (but not NO_HZ_FULL_ALL) and > > > > use the boot parameters to select which CPUs are to be running the > > > > specialized portion of the workload. > > > > > > > > And you would of course need to lead enough CPUs running normally to > > > > handle the non-specialized portion of the workload. > > > > > > > > This sort of thing has traditionally required specialized kernels, > > > > so the cool thing here is that we can make Linux do it. Though, as > > > > you noticed, careful configuration is still required. > > > > > > > > Seem reasonable? > > > > Yes, thanks, some dots got connected :) > > > > > That said if he saw a big performance regression after applying these patches, > > > then there is likely a problem in the patchset. Well it could be due to that mode > > > which loops on full dynticks before resuming to userspace. Indeed when that is > > > enabled, I expect real throughput issues on workloads doing lots of kernel <-> > > > userspace roundtrips. We just need to make sure this thing only works when requested. > > > > With this change (& having NO_HZ_FULL_ALL), hackbench was being served > > only by the boot cpu, while w/o this change, all 8 (this is a quad > > core HT processor) was being used - observation based on 'top'. > > Good to know! And it will be interesting to see what Frederic decides > based on his review of the patchset. Ah wait I was confusing this patchset with the dataplane mode. No the performance loss seen by Afzal is actually expected. Now that we set all nohz full CPUs as isolcpus, when you start a process such as hackbench, it will be affine to CPU 0 unless you explicitly tweak the affinity of hackbench. So yeah this is a desired effect :-) -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/