Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752774AbbDDDnQ (ORCPT ); Fri, 3 Apr 2015 23:43:16 -0400 Received: from mail-wg0-f49.google.com ([74.125.82.49]:36448 "EHLO mail-wg0-f49.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752284AbbDDDnP (ORCPT ); Fri, 3 Apr 2015 23:43:15 -0400 Message-ID: <1428118989.3916.34.camel@gmail.com> Subject: Re: [PATCH 2/2] nohz: make nohz_full imply isolcpus From: Mike Galbraith To: Chris Metcalf Cc: "Peter Zijlstra (Intel)" , Frederic Weisbecker , "Paul E. McKenney" , "Rafael J. Wysocki" , Martin Schwidefsky , Ingo Molnar , linux-kernel@vger.kernel.org Date: Sat, 04 Apr 2015 05:43:09 +0200 In-Reply-To: <1428113039.3189.6.camel@gmail.com> References: <1428078248-5425-1-git-send-email-cmetcalf@ezchip.com> <1428078248-5425-2-git-send-email-cmetcalf@ezchip.com> <1428084499.3475.40.camel@gmail.com> <551EE81C.3050605@ezchip.com> <1428113039.3189.6.camel@gmail.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.16.0 Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3064 Lines: 80 On Sat, 2015-04-04 at 04:03 +0200, Mike Galbraith wrote: > On Fri, 2015-04-03 at 15:21 -0400, Chris Metcalf wrote: > > On 04/03/2015 02:08 PM, Mike Galbraith wrote: > > > On Fri, 2015-04-03 at 12:24 -0400, cmetcalf@ezchip.comwrote: > > > > From: Chris Metcalf > > > > > > > > It's not clear that nohz_full is useful without isolcpus also > > > > set, since otherwise the scheduler has to run periodically to > > > > try to determine whether to steal work from other cores. > > > > > > > > Signed-off-by: Chris Metcalf > > > Ack! nohz_full= as currently defined makes zero sense when the > > > cpu > > > set (which should be spelled cpuset) remains connected to the > > > scheduler. Perturbation of tasks to PREVENT cpu domination is > > > what > > > the scheduler does for a living. Sprinkling microsecond savers > > > all > > > over the kernel is pretty silly if you don't shut down the > > > mother > > > lode > > > of perturbation. > > > > Sounds like a thumbs up for this patch, then? :-) > > Yup. The other thumb turns in the up direction when folks start > spelling cpuset properly ;-) Static isolcpus was supposed to go > away. Speaking of microsecond savers, the (ick) deferment experiment below cut 60 core jitter in half. Shooting the clocksource watchdog fixes alternating ~15us/~5us tick on my desktop box. With workqueue twiddles and whatnot floating around, the thing is starting to look viable. --- kernel/sched/core.c | 5 +++-- kernel/time/clocksource.c | 5 +++++ 2 files changed, 8 insertions(+), 2 deletions(-) --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -2604,12 +2604,13 @@ u64 scheduler_tick_max_deferment(void) struct rq *rq = this_rq(); unsigned long next, now = ACCESS_ONCE(jiffies); - next = rq->last_sched_tick + HZ; + next = (rq->last_sched_tick + HZ) | (rq->clock & 0x3f); if (time_before_eq(next, now)) return 0; - return jiffies_to_nsecs(next - now); + /* Add noise to avoid CPUs colliding at tick boundaries */ + return jiffies_to_nsecs(next - now) | (rq->clock & 0xfffff); } #endif --- a/kernel/time/clocksource.c +++ b/kernel/time/clocksource.c @@ -267,8 +267,13 @@ static void clocksource_watchdog(unsigne * to each other. */ next_cpu = cpumask_next(raw_smp_processor_id(), cpu_online_mask); +skip_nohz_full: if (next_cpu >= nr_cpu_ids) next_cpu = cpumask_first(cpu_online_mask); + if (next_cpu && tick_nohz_full_cpu(next_cpu)) { + next_cpu = cpumask_next(next_cpu, cpu_online_mask); + goto skip_nohz_full; + } watchdog_timer.expires += WATCHDOG_INTERVAL; add_timer_on(&watchdog_timer, next_cpu); out: -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/