Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752364Ab3DLKBd (ORCPT ); Fri, 12 Apr 2013 06:01:33 -0400 Received: from relay2.sgi.com ([192.48.179.30]:41647 "EHLO relay.sgi.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751386Ab3DLKBc (ORCPT ); Fri, 12 Apr 2013 06:01:32 -0400 Date: Fri, 12 Apr 2013 05:01:30 -0500 From: Robin Holt To: "Srivatsa S. Bhat" , Shawn Guo Cc: Ingo Molnar , Robin Holt , Paul Mackerras , Linus Torvalds , "H. Peter Anvin" , Andrew Morton , Linux Kernel Mailing List , Russ Anderson , Shawn Guo , Thomas Gleixner , Ingo Molnar , the arch/x86 maintainers , "Paul E. McKenney" , Tejun Heo , Oleg Nesterov , Lai Jiangshan , Michel Lespinasse , "rusty@rustcorp.com.au" , Peter Zijlstra Subject: Re: Bulk CPU Hotplug (Was Re: [PATCH] Do not force shutdown/reboot to boot cpu.) Message-ID: <20130412100130.GU3658@sgi.com> References: <5162EC1A.4050204@zytor.com> <20130408165916.GA3672@sgi.com> <20130410111620.GB29752@gmail.com> <20130411053106.GA9042@drongo> <5166B05E.8010904@linux.vnet.ibm.com> <20130411134837.GE3672@sgi.com> <20130412053718.GC3887@gmail.com> <5167A52F.6020503@linux.vnet.ibm.com> <20130412093149.GT3658@sgi.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20130412093149.GT3658@sgi.com> User-Agent: Mutt/1.5.20 (2009-06-14) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 4490 Lines: 120 Meant to send this to Shawn. Too early in the morning. Robin On Fri, Apr 12, 2013 at 04:31:49AM -0500, Robin Holt wrote: > On Fri, Apr 12, 2013 at 11:39:51AM +0530, Srivatsa S. Bhat wrote: > > On 04/12/2013 11:07 AM, Ingo Molnar wrote: > > > > > > * Robin Holt wrote: > > > > > >> For the v3.9 release, can we consider my awful patch? > > > > > > How about trying what I suggested, to make reboot affine to the boot CPU > > > explicitly, not by shutting down all the other CPUs, but by set_cpus_allowed() or > > > so? > > > > > > > I agree, that sounds like the right thing to do for 3.9. Of course, it would be > > nice if Shawn could verify that doing that doesn't break his platform due to > > some unknown corner case. > > > > > That should solve the regression, without the ugly special-casing - while giving > > > time to address the hot-unplug performance bottleneck. > > > > > > Once that is done disable_nonboot_cpus() can be used again for reboot. > > > > > > (But no strong feelings either way - both solutions are a workaround in a sense.) > > > >From 1767003c943325e52ac78cac6fdbaf2ab638888d Mon Sep 17 00:00:00 2001 > From: Robin Holt > Date: Wed, 3 Apr 2013 13:52:00 -0500 > Subject: [PATCH] Migrate shutdown/reboot to boot cpu. > > We recently noticed that reboot of a 1024 cpu machine takes approx 16 > minutes of just stopping the cpus. The slowdown was tracked to commit > f96972f. > > The current implementation does all the work of hot removing the cpus > before halting the system. We are switching to just migrating to the > boot cpu and then calling continuing with shutdown/reboot. > > This also has the effect of not breaking x86's command line parameter for > specifying the reboot cpu. Note, this code was shamelessly copied from > arch/x86/kernel/reboot.c with bits removed pertaining to the reboot_cpu > command line parameter. > > Signed-off-by: Robin Holt > To: Shawn Guo > To: Ingo Molnar > To: Russ Anderson > Cc: Andrew Morton > Cc: "H. Peter Anvin" > Cc: Lai Jiangshan > Cc: Linus Torvalds > Cc: Linux Kernel Mailing List > Cc: Michel Lespinasse > Cc: Oleg Nesterov > Cc: "Paul E. McKenney" > Cc: Paul Mackerras > Cc: Peter Zijlstra > Cc: Robin Holt > Cc: "rusty@rustcorp.com.au" > Cc: Tejun Heo > Cc: the arch/x86 maintainers > Cc: Thomas Gleixner > Cc: > --- > kernel/sys.c | 17 +++++++++++++++-- > 1 file changed, 15 insertions(+), 2 deletions(-) > > diff --git a/kernel/sys.c b/kernel/sys.c > index 0da73cf..4d1047d 100644 > --- a/kernel/sys.c > +++ b/kernel/sys.c > @@ -357,6 +357,19 @@ int unregister_reboot_notifier(struct notifier_block *nb) > } > EXPORT_SYMBOL(unregister_reboot_notifier); > > +void migrate_to_boot_cpu(void) > +{ > + /* The boot cpu is always logical cpu 0 */ > + int reboot_cpu_id = 0; > + > + /* Make certain the cpu I'm about to reboot on is online */ > + if (!cpu_online(reboot_cpu_id)) > + reboot_cpu_id = smp_processor_id(); > + > + /* Make certain I only run on the appropriate processor */ > + set_cpus_allowed_ptr(current, cpumask_of(reboot_cpu_id)); > +} > + > /** > * kernel_restart - reboot the system > * @cmd: pointer to buffer containing command to execute for restart > @@ -368,7 +381,7 @@ EXPORT_SYMBOL(unregister_reboot_notifier); > void kernel_restart(char *cmd) > { > kernel_restart_prepare(cmd); > - disable_nonboot_cpus(); > + migrate_to_boot_cpu(); > syscore_shutdown(); > if (!cmd) > printk(KERN_EMERG "Restarting system.\n"); > @@ -414,7 +427,7 @@ void kernel_power_off(void) > kernel_shutdown_prepare(SYSTEM_POWER_OFF); > if (pm_power_off_prepare) > pm_power_off_prepare(); > - disable_nonboot_cpus(); > + migrate_to_boot_cpu(); > syscore_shutdown(); > printk(KERN_EMERG "Power down.\n"); > kmsg_dump(KMSG_DUMP_POWEROFF); > -- > 1.8.1.2 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/