Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
From:   KarimAllah Ahmed <karahmed@amazon.de>
To:     linux-kernel@vger.kernel.org
Cc:     KarimAllah Ahmed <karahmed@amazon.de>,
        Andi Kleen <ak@linux.intel.com>,
        Andrea Arcangeli <aarcange@redhat.com>,
        Andy Lutomirski <luto@kernel.org>,
        Arjan van de Ven <arjan@linux.intel.com>,
        Ashok Raj <ashok.raj@intel.com>,
        Asit Mallick <asit.k.mallick@intel.com>,
        Borislav Petkov <bp@suse.de>,
        Dan Williams <dan.j.williams@intel.com>,
        Dave Hansen <dave.hansen@intel.com>,
        David Woodhouse <dwmw@amazon.co.uk>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        "H . Peter Anvin" <hpa@zytor.com>, Ingo Molnar <mingo@redhat.com>,
        Janakarajan Natarajan <Janakarajan.Natarajan@amd.com>,
        Joerg Roedel <joro@8bytes.org>,
        Jun Nakajima <jun.nakajima@intel.com>,
        Laura Abbott <labbott@redhat.com>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        Masami Hiramatsu <mhiramat@kernel.org>,
        Paolo Bonzini <pbonzini@redhat.com>,
        Peter Zijlstra <peterz@infradead.org>,
        =?UTF-8?q?Radim=20Kr=C4=8Dm=C3=A1=C5=99?= <rkrcmar@redhat.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Tim Chen <tim.c.chen@linux.intel.com>,
        Tom Lendacky <thomas.lendacky@amd.com>, kvm@vger.kernel.org,
        x86@kernel.org
Subject: [RFC 08/10] x86/idle: Control Indirect Branch Speculation in idle
Date:   Sat, 20 Jan 2018 20:22:59 +0100
Message-Id: <1516476182-5153-9-git-send-email-karahmed@amazon.de>
In-Reply-To: <1516476182-5153-1-git-send-email-karahmed@amazon.de>
References: <1516476182-5153-1-git-send-email-karahmed@amazon.de>
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

From: Thomas Gleixner <tglx@linutronix.de>

Indirect Branch Speculation (IBS) is controlled per physical core. If one
thread disables it then it's disabled for the core. If a thread enters idle
it makes sense to reenable IBS so the sibling thread can run with full
speculation enabled in user space.

This makes only sense in mwait_idle_with_hints() because mwait_idle() can
serve an interrupt immediately before speculation can be stopped again. SKL
which requires IBRS should use mwait_idle_with_hints() so this is a non
issue and in the worst case a missed optimization.

Originally-by: Tim Chen <tim.c.chen@linux.intel.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/include/asm/mwait.h | 14 ++++++++++++++
 arch/x86/kernel/process.c    | 14 ++++++++++++++
 2 files changed, 28 insertions(+)

diff --git a/arch/x86/include/asm/mwait.h b/arch/x86/include/asm/mwait.h
index 39a2fb2..f173072 100644
--- a/arch/x86/include/asm/mwait.h
+++ b/arch/x86/include/asm/mwait.h
@@ -6,6 +6,7 @@
 #include <linux/sched/idle.h>
 
 #include <asm/cpufeature.h>
+#include <asm/nospec-branch.h>
 
 #define MWAIT_SUBSTATE_MASK		0xf
 #define MWAIT_CSTATE_MASK		0xf
@@ -106,7 +107,20 @@ static inline void mwait_idle_with_hints(unsigned long eax, unsigned long ecx)
 			mb();
 		}
 
+		/*
+		 * Indirect Branch Speculation (IBS) is controlled per
+		 * physical core. If one thread disables it, then it's
+		 * disabled on all threads of the core. The kernel disables
+		 * it on entry from user space. Reenable it on the thread
+		 * which goes idle so the other thread has a chance to run
+		 * with full speculation enabled in userspace.
+		 */
+		unrestrict_branch_speculation();
 		__monitor((void *)&current_thread_info()->flags, 0, 0);
+		/*
+		 * Restrict IBS again to protect kernel execution.
+		 */
+		restrict_branch_speculation();
 		if (!need_resched())
 			__mwait(eax, ecx);
 	}
diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c
index 3cb2486..f941c5d 100644
--- a/arch/x86/kernel/process.c
+++ b/arch/x86/kernel/process.c
@@ -461,6 +461,20 @@ static __cpuidle void mwait_idle(void)
 			mb(); /* quirk */
 		}
 
+		/*
+		 * Indirect Branch Speculation (IBS) is controlled per
+		 * physical core. If one thread disables it, then it's
+		 * disabled on all threads of the core. The kernel disables
+		 * it on entry from user space. For __sti_mwait() it's
+		 * wrong to reenable it because an interrupt can be served
+		 * before speculation can be stopped again.
+		 *
+		 * To plug that hole the interrupt entry code would need to
+		 * save current state and restore. Not worth the trouble as
+		 * SKL should not use mwait_idle(). It should use
+		 * mwait_idle_with_hints() which can do speculation control
+		 * safely.
+		 */
 		__monitor((void *)&current_thread_info()->flags, 0, 0);
 		if (!need_resched())
 			__sti_mwait(0, 0);
-- 
2.7.4