Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932987Ab3DPLdE (ORCPT ); Tue, 16 Apr 2013 07:33:04 -0400 Received: from mail-bk0-f54.google.com ([209.85.214.54]:45974 "EHLO mail-bk0-f54.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751820Ab3DPLdC (ORCPT ); Tue, 16 Apr 2013 07:33:02 -0400 Date: Tue, 16 Apr 2013 13:32:56 +0200 From: Ingo Molnar To: Robin Holt Cc: Ingo Molnar , Russ Anderson , Shawn Guo , Oleg Nesterov , Andrew Morton , "H. Peter Anvin" , Joe Perches , Lai Jiangshan , Linus Torvalds , Linux Kernel Mailing List , Michel Lespinasse , "Paul E. McKenney" , Paul Mackerras , Peter Zijlstra , "rusty@rustcorp.com.au" , Tejun Heo , the arch/x86 maintainers , Thomas Gleixner Subject: Re: [Patch -v4 1/4] Migrate shutdown/reboot to boot cpu. Message-ID: <20130416113256.GA10452@gmail.com> References: <20130416095827.GJ3672@sgi.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20130416095827.GJ3672@sgi.com> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3264 Lines: 94 * Robin Holt wrote: > We recently noticed that reboot of a 1024 cpu machine takes approx 16 > minutes of just stopping the cpus. The slowdown was tracked to commit > f96972f. > > The current implementation does all the work of hot removing the cpus > before halting the system. We are switching to just migrating to the > boot cpu and then continuing with shutdown/reboot. > > This also has the effect of not breaking x86's command line parameter for > specifying the reboot cpu. Note, this code was shamelessly copied from > arch/x86/kernel/reboot.c with bits removed pertaining to the reboot_cpu > command line parameter. > > Signed-off-by: Robin Holt > Tested-by: Shawn Guo > To: Ingo Molnar > To: Russ Anderson > To: Oleg Nesterov > Cc: Andrew Morton > Cc: "H. Peter Anvin" > Cc: Lai Jiangshan > Cc: Linus Torvalds > Cc: Linux Kernel Mailing List > Cc: Michel Lespinasse > Cc: Oleg Nesterov > Cc: "Paul E. McKenney" > Cc: Paul Mackerras > Cc: Peter Zijlstra > Cc: Robin Holt > Cc: "rusty@rustcorp.com.au" > Cc: Tejun Heo > Cc: the arch/x86 maintainers > Cc: Thomas Gleixner > Cc: > > --- > > Changes since -v1. > - Set PF_THREAD_BOUND before migrating to eliminate potential race. > - Modified kernel_power_off to also migrate instead of using > disable_nonboot_cpus(). > --- > kernel/sys.c | 22 +++++++++++++++++++--- > 1 file changed, 19 insertions(+), 3 deletions(-) > > diff --git a/kernel/sys.c b/kernel/sys.c > index 0da73cf..5ef7aa2 100644 > --- a/kernel/sys.c > +++ b/kernel/sys.c > @@ -357,6 +357,22 @@ int unregister_reboot_notifier(struct notifier_block *nb) > } > EXPORT_SYMBOL(unregister_reboot_notifier); > > +void migrate_to_reboot_cpu(void) It appears to be file-scope, so should be static I guess? > +{ > + /* The boot cpu is always logical cpu 0 */ > + int reboot_cpu_id = 0; > + > + /* Make certain the cpu I'm about to reboot on is online */ > + if (!cpu_online(reboot_cpu_id)) > + reboot_cpu_id = smp_processor_id(); Shouldn't we pick the first online CPU instead, to make it deterministic? Also, does this codepath prevent hotplug from going on in parallel? ( Plus, the smp_processor_id() is in a preemptible section AFAICS, so it will throw a warning with preempt debug on. ) > + > + /* Prevent races with other tasks migrating this task. */ (I guess the colon can be dropped here, like in the other comments.) > + current->flags |= PF_THREAD_BOUND; > + > + /* Make certain I only run on the appropriate processor */ > + set_cpus_allowed_ptr(current, cpumask_of(reboot_cpu_id)); > +} Thanks, Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/