Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751037AbbEBTIY (ORCPT ); Sat, 2 May 2015 15:08:24 -0400 Received: from mail.linuxfoundation.org ([140.211.169.12]:44273 "EHLO mail.linuxfoundation.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752283AbbEBTIQ (ORCPT ); Sat, 2 May 2015 15:08:16 -0400 From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Mike Galbraith , Borislav Petkov , "H. Peter Anvin" , Ian Malone , Josh Boyer , Len Brown , Len Brown , Linus Torvalds , Mike Galbraith , Peter Zijlstra , Thomas Gleixner , Ingo Molnar Subject: [PATCH 4.0 016/220] sched/idle/x86: Optimize unnecessary mwait_idle() resched IPIs Date: Sat, 2 May 2015 20:58:51 +0200 Message-Id: <20150502185855.081844017@linuxfoundation.org> X-Mailer: git-send-email 2.3.7 In-Reply-To: <20150502185854.333748961@linuxfoundation.org> References: <20150502185854.333748961@linuxfoundation.org> User-Agent: quilt/0.64 MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-15 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2540 Lines: 78 4.0-stable review patch. If anyone has any objections, please let me know. ------------------ From: Mike Galbraith commit f8e617f4582995f7c25ef25b4167213120ad122b upstream. To fully take advantage of MWAIT, apparently the CLFLUSH instruction needs another quirk on certain CPUs: proper barriers around it on certain machines. On a Q6600 SMP system, pipe-test scheduling performance, cross core, improves significantly: 3.8.13 487.2 KHz 1.000 3.13.0-master 415.5 KHz .852 3.13.0-master+ 415.2 KHz .852 + restore mwait_idle 3.13.0-master++ 488.5 KHz 1.002 + restore mwait_idle + IPI fix Since X86_BUG_CLFLUSH_MONITOR is already a quirk, don't create a separate quirk for the extra smp_mb()s. Signed-off-by: Mike Galbraith Cc: Borislav Petkov Cc: H. Peter Anvin Cc: Ian Malone Cc: Josh Boyer Cc: Len Brown Cc: Len Brown Cc: Linus Torvalds Cc: Mike Galbraith Cc: Peter Zijlstra Cc: Thomas Gleixner Link: http://lkml.kernel.org/r/1390061684.5566.4.camel@marge.simpson.net [ Ported to recent kernel, added comments about the quirk. ] Signed-off-by: Ingo Molnar Signed-off-by: Greg Kroah-Hartman --- arch/x86/kernel/process.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) --- a/arch/x86/kernel/process.c +++ b/arch/x86/kernel/process.c @@ -429,18 +429,22 @@ static int prefer_mwait_c1_over_halt(con static void mwait_idle(void) { - if (!need_resched()) { - if (this_cpu_has(X86_BUG_CLFLUSH_MONITOR)) + if (!current_set_polling_and_test()) { + if (this_cpu_has(X86_BUG_CLFLUSH_MONITOR)) { + smp_mb(); /* quirk */ clflush((void *)¤t_thread_info()->flags); + smp_mb(); /* quirk */ + } __monitor((void *)¤t_thread_info()->flags, 0, 0); - smp_mb(); if (!need_resched()) __sti_mwait(0, 0); else local_irq_enable(); - } else + } else { local_irq_enable(); + } + __current_clr_polling(); } void select_idle_routine(const struct cpuinfo_x86 *c) -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/