Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
From:   Tim Chen <tim.c.chen@linux.intel.com>
To:     linux-kernel@vger.kernel.org
Cc:     Tim Chen <tim.c.chen@linux.intel.com>,
        KarimAllah Ahmed <karahmed@amazon.de>,
        Andi Kleen <ak@linux.intel.com>,
        Andrea Arcangeli <aarcange@redhat.com>,
        Andy Lutomirski <luto@kernel.org>,
        Arjan van de Ven <arjan@linux.intel.com>,
        Ashok Raj <ashok.raj@intel.com>,
        Asit Mallick <asit.k.mallick@intel.com>,
        Borislav Petkov <bp@suse.de>,
        Dan Williams <dan.j.williams@intel.com>,
        Dave Hansen <dave.hansen@intel.com>,
        David Woodhouse <dwmw@amazon.co.uk>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        "H . Peter Anvin" <hpa@zytor.com>, Ingo Molnar <mingo@redhat.com>,
        Janakarajan Natarajan <Janakarajan.Natarajan@amd.com>,
        Joerg Roedel <joro@8bytes.org>,
        Jun Nakajima <jun.nakajima@intel.com>,
        Laura Abbott <labbott@redhat.com>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        Masami Hiramatsu <mhiramat@kernel.org>,
        Paolo Bonzini <pbonzini@redhat.com>,
        Peter Zijlstra <peterz@infradead.org>, rkrcmar@redhat.com,
        Thomas Gleixner <tglx@linutronix.de>,
        Tom Lendacky <thomas.lendacky@amd.com>, x86@kernel.org
Subject: [RFC PATCH 1/2] x86/ibpb: Skip IBPB when we switch back to same user process
Date:   Wed, 24 Jan 2018 16:36:41 -0800
Message-Id: <c5cc429013c2248ddebe0a8480b172ae70b29733.1516840211.git.tim.c.chen@linux.intel.com>
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

These two patches provide optimization to skip IBPB for this
commonly encountered scenario:
We could switch to a kernel idle thread and then back to the original
process such as:
process A -> idle -> process A

In such scenario, we do not have to do IBPB here even though the process
is non-dumpable, as we are switching back to the same process after
an hiatus.

The cost is to have an extra pointer to track the last mm we were using before
switching to the init_mm used by idle.  But avoiding the extra IBPB
is probably worth the extra memory for such a common scenario.

Signed-off-by: Tim Chen <tim.c.chen@linux.intel.com>
---
 arch/x86/include/asm/tlbflush.h |  2 ++
 arch/x86/mm/tlb.c               | 13 +++++++++----
 2 files changed, 11 insertions(+), 4 deletions(-)

diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h
index 3effd3c..c5e325f 100644
--- a/arch/x86/include/asm/tlbflush.h
+++ b/arch/x86/include/asm/tlbflush.h
@@ -174,6 +174,8 @@ struct tlb_state {
 	struct mm_struct *loaded_mm;
 	u16 loaded_mm_asid;
 	u16 next_asid;
+	/* last user mm */
+	struct mm_struct *last_usr_mm;
 
 	/*
 	 * We can be in one of several states:
diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c
index d5a35fe..86ed07f 100644
--- a/arch/x86/mm/tlb.c
+++ b/arch/x86/mm/tlb.c
@@ -220,6 +220,7 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next,
 	} else {
 		u16 new_asid;
 		bool need_flush;
+		struct mm_struct *last = this_cpu_read(cpu_tlbstate.last_usr_mm);
 
 		/*
 		 * Avoid user/user BTB poisoning by flushing the branch predictor
@@ -229,15 +230,17 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next,
                  * As an optimization flush indirect branches only when
                  * switching into processes that disable dumping.
                  *
-                 * This will not flush branches when switching into kernel
-                 * threads, but it would flush them when switching to the
-                 * idle thread and back.
+		 * This will not flush branches when switching into kernel
+		 * threads. It will also not flush if we switch to idle
+		 * thread and back to the same process. It will flush if we
+		 * switch to a different non-dumpable process.
                  *
                  * It might be useful to have a one-off cache here
                  * to also not flush the idle case, but we would need some
                  * kind of stable sequence number to remember the previous mm.
 		 */
-		if (tsk && tsk->mm && get_dumpable(tsk->mm) != SUID_DUMP_USER)
+		if (tsk && tsk->mm && (tsk->mm != last)
+			&& get_dumpable(tsk->mm) != SUID_DUMP_USER)
 			indirect_branch_prediction_barrier();
 
 		if (IS_ENABLED(CONFIG_VMAP_STACK)) {
@@ -288,6 +291,8 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next,
 			trace_tlb_flush_rcuidle(TLB_FLUSH_ON_TASK_SWITCH, 0);
 		}
 
+		if (next != &init_mm && next != last)
+			this_cpu_write(cpu_tlbstate.last_usr_mm, next);
 		this_cpu_write(cpu_tlbstate.loaded_mm, next);
 		this_cpu_write(cpu_tlbstate.loaded_mm_asid, new_asid);
 	}
-- 
2.9.4