Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933935AbdGKQef (ORCPT ); Tue, 11 Jul 2017 12:34:35 -0400 Received: from merlin.infradead.org ([205.233.59.134]:54140 "EHLO merlin.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932526AbdGKQee (ORCPT ); Tue, 11 Jul 2017 12:34:34 -0400 Date: Tue, 11 Jul 2017 18:34:22 +0200 From: Peter Zijlstra To: Frederic Weisbecker Cc: Christoph Lameter , "Li, Aubrey" , Andi Kleen , Aubrey Li , tglx@linutronix.de, len.brown@intel.com, rjw@rjwysocki.net, tim.c.chen@linux.intel.com, arjan@linux.intel.com, paulmck@linux.vnet.ibm.com, yang.zhang.wz@gmail.com, x86@kernel.org, linux-kernel@vger.kernel.org Subject: Re: [RFC PATCH v1 00/11] Create fast idle path for short idle periods Message-ID: <20170711163422.etydkhhtgfthpfi5@hirez.programming.kicks-ass.net> References: <1499650721-5928-1-git-send-email-aubrey.li@intel.com> <20170710084647.zs6wkl3fumszd33g@hirez.programming.kicks-ass.net> <20170710144609.GD31832@tassilo.jf.intel.com> <20170710164206.5aon5kelbisxqyxq@hirez.programming.kicks-ass.net> <20170710172705.GA3441@tassilo.jf.intel.com> <20170711094157.5xcwkloxnjehieqv@hirez.programming.kicks-ass.net> <20170711160926.GA18805@lerouge> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170711160926.GA18805@lerouge> User-Agent: NeoMutt/20170609 (1.8.3) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2657 Lines: 75 On Tue, Jul 11, 2017 at 06:09:27PM +0200, Frederic Weisbecker wrote: > > > - tick_nohz_idle_enter costs 7058ns - 10726ns > > > - tick_nohz_idle_exit costs 8372ns - 20850ns > > > > Right, those are horrible expensive, but skipping them isn't 'hard', the > > only tricky bit is finding a condition that makes sense. > > Note you can statically disable it with nohz=0 boot parameter. Yeah, but that's bad for power usage, nobody wants that. > > See Mike's patch: https://patchwork.kernel.org/patch/2839221/ > > > > Combined with the above, and possibly a better condition, that should > > get rid of most of this. > > Such a patch could work well if the decision from the scheduler to not stop the tick > happens on idle entry. > > Now if sched_needs_cpu() first allows to stop the tick then refuses it later > in the end of an idle IRQ, this won't have the desired effect. As long as ts->tick_stopped=1, > it stays so until we really restart the tick. So the whole costly nohz machinery stays on. > > I guess it doesn't matter though, as we are talking about making fast idle entry so the > decision not to stop the tick is likely to be done once on idle entry, when ts->tick_stopped=0. > > One exception though: if the tick is already stopped when we enter idle (full nohz case). And > BTW stopping the tick outside idle shouldn't be concerned here. > > So I'd rather put that on can_stop_idle_tick(). Mike's patch much predates the existence of that function I think ;-) But sure.. > > > > > - totally from arch_cpu_idle_enter entry to arch_cpu_idle_exit return costs > > > 9122ns - 15318ns. > > > --In this period, rcu_idle_enter costs 1985ns - 2262ns, rcu_idle_exit costs > > > 1813ns - 3507ns > > > > Is that the POPF being painful? or something else? > > Probably that and the atomic_add_return(). I got properly lost in the RCU machinery. It wasn't at all clear to me if rcu_eqs_enter_common() was a slow-path function or not. Also, RCU_FAST_NO_HZ will make a fairly large difference here.. Paul what's the state of that thing, do we actually want that or not? But I think we can at the very least do this; it only gets called from kernel/sched/idle.c and both callsites have IRQs explicitly disabled by that point. diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 51d4c3acf32d..dccf2dc8155a 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -843,13 +843,8 @@ static void rcu_eqs_enter(bool user) */ void rcu_idle_enter(void) { - unsigned long flags; - - local_irq_save(flags); rcu_eqs_enter(false); - local_irq_restore(flags); } -EXPORT_SYMBOL_GPL(rcu_idle_enter); #ifdef CONFIG_NO_HZ_FULL /**