Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
From:   Tim Chen <tim.c.chen@linux.intel.com>
To:     linux-kernel@vger.kernel.org
Cc:     Tim Chen <tim.c.chen@linux.intel.com>,
        KarimAllah Ahmed <karahmed@amazon.de>,
        Andi Kleen <ak@linux.intel.com>,
        Andrea Arcangeli <aarcange@redhat.com>,
        Andy Lutomirski <luto@kernel.org>,
        Arjan van de Ven <arjan@linux.intel.com>,
        Ashok Raj <ashok.raj@intel.com>,
        Asit Mallick <asit.k.mallick@intel.com>,
        Borislav Petkov <bp@suse.de>,
        Dan Williams <dan.j.williams@intel.com>,
        Dave Hansen <dave.hansen@intel.com>,
        David Woodhouse <dwmw@amazon.co.uk>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        "H . Peter Anvin" <hpa@zytor.com>, Ingo Molnar <mingo@redhat.com>,
        Janakarajan Natarajan <Janakarajan.Natarajan@amd.com>,
        Joerg Roedel <joro@8bytes.org>,
        Jun Nakajima <jun.nakajima@intel.com>,
        Laura Abbott <labbott@redhat.com>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        Masami Hiramatsu <mhiramat@kernel.org>,
        Paolo Bonzini <pbonzini@redhat.com>,
        Peter Zijlstra <peterz@infradead.org>, rkrcmar@redhat.com,
        Thomas Gleixner <tglx@linutronix.de>,
        Tom Lendacky <thomas.lendacky@amd.com>, x86@kernel.org
Subject: [RFC PATCH 2/2] x86/ibpb: Prevent missed IBPB flush
Date:   Wed, 24 Jan 2018 16:36:42 -0800
Message-Id: <332d3ac8a7018b98d8182ec86b5fc41239c667c6.1516840211.git.tim.c.chen@linux.intel.com>
In-Reply-To: <c5cc429013c2248ddebe0a8480b172ae70b29733.1516840211.git.tim.c.chen@linux.intel.com>
References: <c5cc429013c2248ddebe0a8480b172ae70b29733.1516840211.git.tim.c.chen@linux.intel.com>
In-Reply-To: <c5cc429013c2248ddebe0a8480b172ae70b29733.1516840211.git.tim.c.chen@linux.intel.com>
References: <c5cc429013c2248ddebe0a8480b172ae70b29733.1516840211.git.tim.c.chen@linux.intel.com>
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

It is possible that the last uesr mm that we recorded for a cpu was
released, and a new mm with identical address was allocated when we
check it again.  We could skip IBPB flush here for the process with
the new mm.

It is a difficult to exploit case as we have to exit() a process on a
cpu, free the mm, and fork() the victim to use the mm pointer on that
cpu. The exploiter needs the old mm to get recycled to the
newly forked process and no other processes run on the target cpu.

Nevertheless, the patch below is one way to close this hole by
adding a ref count to prevent the last user mm from being released.
It does add ref counting overhead, and extra memory cost of keeping an mm
(though not the VMAs and most of page tables) around longer than we will
otherwise need to. Any better solutions are welcomed.

Suggested-by: Dave Hansen <dave.hansen@intel.com>
Signed-off-by: Tim Chen <tim.c.chen@linux.intel.com>
---
 arch/x86/mm/tlb.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c
index 86ed07f..3bdaa10 100644
--- a/arch/x86/mm/tlb.c
+++ b/arch/x86/mm/tlb.c
@@ -7,6 +7,7 @@
 #include <linux/export.h>
 #include <linux/cpu.h>
 #include <linux/debugfs.h>
+#include <linux/sched/mm.h>
 
 #include <asm/tlbflush.h>
 #include <asm/mmu_context.h>
@@ -291,8 +292,17 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next,
 			trace_tlb_flush_rcuidle(TLB_FLUSH_ON_TASK_SWITCH, 0);
 		}
 
-		if (next != &init_mm && next != last)
+		if (next != &init_mm && next != last) {
+			if (last != NULL)
+				mmdrop(last);
+			/*
+			 * Keep 'next' allocated until we switch to another mm.
+			 * This keeps us from missing a flush of the branch predictors
+			 * if 'next' gets freed and reallocated.
+			 */
+			mmgrab(next);
 			this_cpu_write(cpu_tlbstate.last_usr_mm, next);
+		}
 		this_cpu_write(cpu_tlbstate.loaded_mm, next);
 		this_cpu_write(cpu_tlbstate.loaded_mm_asid, new_asid);
 	}
@@ -370,6 +380,7 @@ void initialize_tlbstate_and_flush(void)
 	write_cr3(build_cr3(mm->pgd, 0));
 
 	/* Reinitialize tlbstate. */
+	this_cpu_write(cpu_tlbstate.last_usr_mm, NULL);
 	this_cpu_write(cpu_tlbstate.loaded_mm_asid, 0);
 	this_cpu_write(cpu_tlbstate.next_asid, 1);
 	this_cpu_write(cpu_tlbstate.ctxs[0].ctx_id, mm->context.ctx_id);
-- 
2.9.4