Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753265AbcKPMMj (ORCPT ); Wed, 16 Nov 2016 07:12:39 -0500 Received: from terminus.zytor.com ([198.137.202.10]:35306 "EHLO terminus.zytor.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752906AbcKPMMh (ORCPT ); Wed, 16 Nov 2016 07:12:37 -0500 Date: Wed, 16 Nov 2016 04:12:06 -0800 From: tip-bot for Martin Schwidefsky Message-ID: Cc: linux-kernel@vger.kernel.org, schwidefsky@de.ibm.com, hpa@zytor.com, tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, mingo@kernel.org Reply-To: tglx@linutronix.de, schwidefsky@de.ibm.com, hpa@zytor.com, linux-kernel@vger.kernel.org, mingo@kernel.org, torvalds@linux-foundation.org, peterz@infradead.org In-Reply-To: <1478523660-733-1-git-send-email-schwidefsky@de.ibm.com> References: <1478523660-733-1-git-send-email-schwidefsky@de.ibm.com> To: linux-tip-commits@vger.kernel.org Subject: [tip:sched/core] sched/x86: Do not clear PREEMPT_NEED_RESCHED on preempt count reset Git-Commit-ID: f285144f81e814f39342dbf5321d6ba939890b1b X-Mailer: tip-git-log-daemon Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8 Content-Disposition: inline Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 1986 Lines: 51 Commit-ID: f285144f81e814f39342dbf5321d6ba939890b1b Gitweb: http://git.kernel.org/tip/f285144f81e814f39342dbf5321d6ba939890b1b Author: Martin Schwidefsky AuthorDate: Mon, 7 Nov 2016 14:01:00 +0100 Committer: Ingo Molnar CommitDate: Wed, 16 Nov 2016 10:29:04 +0100 sched/x86: Do not clear PREEMPT_NEED_RESCHED on preempt count reset The per-cpu preempt count of x86 contains two values, the actual preempt count and the inverted PREEMPT_NEED_RESCHED bit. If a corrupted preempt count is detected the preempt_count_set() function is used to reset the preempt count. In case the inverted PREEMPT_NEED_RESCHED bit is zero at the time of the reset, the preemption indication is lost. Use raw_cpu_cmpxchg_4() to reset only the count part and leave the PREEMPT_NEED_RESCHED bit as it is. This improves the kernel's behavior when it runs into preempt count leaks and tries to fix them up. Signed-off-by: Martin Schwidefsky Signed-off-by: Peter Zijlstra (Intel) Cc: Linus Torvalds Cc: Peter Zijlstra Cc: Thomas Gleixner Link: http://lkml.kernel.org/r/1478523660-733-1-git-send-email-schwidefsky@de.ibm.com Signed-off-by: Ingo Molnar --- arch/x86/include/asm/preempt.h | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/preempt.h b/arch/x86/include/asm/preempt.h index 17f2186..ec1f3c6 100644 --- a/arch/x86/include/asm/preempt.h +++ b/arch/x86/include/asm/preempt.h @@ -24,7 +24,13 @@ static __always_inline int preempt_count(void) static __always_inline void preempt_count_set(int pc) { - raw_cpu_write_4(__preempt_count, pc); + int old, new; + + do { + old = raw_cpu_read_4(__preempt_count); + new = (old & PREEMPT_NEED_RESCHED) | + (pc & ~PREEMPT_NEED_RESCHED); + } while (raw_cpu_cmpxchg_4(__preempt_count, old, new) != old); } /*