Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753048AbbELKMr (ORCPT ); Tue, 12 May 2015 06:12:47 -0400 Received: from mail.skyhub.de ([78.46.96.112]:38708 "EHLO mail.skyhub.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750865AbbELKMm (ORCPT ); Tue, 12 May 2015 06:12:42 -0400 Date: Tue, 12 May 2015 12:12:38 +0200 From: Borislav Petkov To: Ingo Molnar Cc: Len Brown , Jan =?utf-8?Q?H=2E_Sch=C3=B6nherr?= , Thomas Gleixner , X86 ML , "linux-kernel@vger.kernel.org" , Anthony Liguori , Ingo Molnar , "H. Peter Anvin" , Tim Deegan , Gang Wei , Linus Torvalds , Arjan van de Ven Subject: Re: [PATCH] x86: skip delays during SMP initialization similar to Xen Message-ID: <20150512101238.GA4287@pd.tnic> References: <1430732554-7294-1-git-send-email-jschoenh@amazon.de> <20150506082759.GA30019@gmail.com> <20150507102351.GA14347@gmail.com> <20150512094709.GB11477@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20150512094709.GB11477@gmail.com> User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 4165 Lines: 59 On Tue, May 12, 2015 at 11:47:09AM +0200, Ingo Molnar wrote: > So I was booting 120 CPUs with kvmtool (tools/kvm/ under -tip). > > Even with your patches applied it's 7 seconds (config attached): > > [ 0.152189] x86: Booting SMP configuration: > [ 0.152705] .... node #0, CPUs: #1 #2 #3 #4 #5 #6 #7 #8 #9 #10 #11 #12 #13 #14 #15 #16 #17 #18 #19 #20 #21 #22 #23 #24 #25 #26 #27 #28 #29 #30 #31 #32 #33 #34 #35 #36 #37 #38 #39 #40 #41 #42 #43 #44 #45 #46 #47 #48 #49 #50 #51 #52 #53 #54 #55 #56 #57 #58 #59 #60 #61 #62 #63 #64 #65 #66 #67 #68 #69 #70 #71 #72 #73 #74 #75 #76 #77 #78 #79 #80 #81 #82 #83 #84 #85 #86 #87 #88 #89 #90 #91 #92 #93 #94 #95 #96 #97 #98 #99 #100 #101 #102 #103 #104 #105 #106 #107 #108 #109 #110 #111 #112 #113 #114 #115 #116 #117 #118 #119 > [ 7.627192] x86: Booted up 1 node, 120 CPUs > [ 7.627795] smpboot: Total of 120 processors activated (672119.13 BogoMIPS) > [ 7.633325] devtmpfs: initialized > > so there's some other delay going on. It could very well be a kvmtool > related initialization delay? 10 seconds with qemu+kvm and latest tip/master: [ 0.176297] x86: Booting SMP configuration: [ 0.179640] .... node #0, CPUs: #1 #2 #3 #4 #5 #6 #7 #8 #9 #10 #11 #12 #13 #14 #15 #16 #17 #18 #19 #20 #21 #22 #23 #24 #25 #26 #27 #28 #29 #30 #31 #32 #33 #34 #35 #36 #37 #38 #39 #40 #41 #42 #43 #44 #45 #46 #47 #48 #49 #50 #51 #52 #53 #54 #55 #56 #57 #58 #59 #60 #61 #62 #63 #64 [ 0.036000] ------------[ cut here ]------------ [ 0.036000] WARNING: CPU: 64 PID: 0 at kernel/locking/lockdep.c:2755 lockdep_trace_alloc+0x10c/0x120() [ 0.036000] DEBUG_LOCKS_WARN_ON(irqs_disabled_flags(flags)) [ 0.036000] Modules linked in: [ 0.036000] CPU: 64 PID: 0 Comm: swapper/64 Not tainted 4.1.0-rc3+ #6 [ 0.036000] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20140531_083030-gandalf 04/01/2014 [ 0.036000] ffffffff818c0f15 ffff88006d357d08 ffffffff8168c89b 0000000000000000 [ 0.036000] ffff88006d357d58 ffff88006d357d48 ffffffff81057d55 ffff88006d357da8 [ 0.036000] 0000000000000080 0000000000000046 ffffffff81a4fb80 ffff88006d0af020 [ 0.036000] Call Trace: [ 0.036000] [] dump_stack+0x4f/0x7b [ 0.036000] [] warn_slowpath_common+0x95/0xe0 [ 0.036000] [] warn_slowpath_fmt+0x46/0x50 [ 0.036000] [] lockdep_trace_alloc+0x10c/0x120 [ 0.036000] [] __alloc_pages_nodemask+0xad/0xa20 [ 0.036000] [] ? mutex_lock_nested+0x2d0/0x3f0 [ 0.036000] [] ? __this_cpu_preempt_check+0x13/0x20 [ 0.036000] [] ? trace_hardirqs_off_caller+0xcb/0x170 [ 0.036000] [] ? mutex_lock_nested+0x2a4/0x3f0 [ 0.036000] [] ? init_espfix_ap.part.5+0xb6/0x270 [ 0.036000] [] ? update_max_interval+0x1a/0x40 [ 0.036000] [] __get_free_pages+0x1d/0x60 [ 0.036000] [] init_espfix_ap.part.5+0x164/0x270 [ 0.036000] [] init_espfix_ap+0x21/0x30 [ 0.036000] [] start_secondary+0xe8/0x180 [ 0.036000] ---[ end trace 2312a8d05943a0d2 ]--- [ 5.744137] #65 #66 #67 #68 #69 #70 #71 #72 #73 #74 #75 #76 #77 #78 #79 #80 #81 #82 #83 #84 #85 #86 #87 #88 #89 #90 #91 #92 #93 #94 #95 #96 #97 #98 #99 #100 #101 #102 #103 #104 #105 #106 #107 #108 #109 #110 #111 #112 #113 #114 #115 #116 #117 #118 #119 [ 10.416459] x86: Booted up 1 node, 120 CPUs [ 10.420012] smpboot: Total of 120 processors activated (963228.36 BogoMIPS) -- Regards/Gruss, Boris. ECO tip #101: Trim your mails when you reply. -- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/