Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751605AbdISKDf (ORCPT ); Tue, 19 Sep 2017 06:03:35 -0400 Received: from mail.kernel.org ([198.145.29.99]:53034 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751034AbdISKDe (ORCPT ); Tue, 19 Sep 2017 06:03:34 -0400 DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4982121BCE Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=mhiramat@kernel.org From: Masami Hiramatsu To: Ingo Molnar , mingo@redhat.com Cc: x86@kernel.org, Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, Peter Zijlstra , Ananth N Mavinakayanahalli , Thomas Gleixner , "H . Peter Anvin" , "Paul E . McKenney" , Alexei Starovoitov , Alexei Starovoitov Subject: [PATCH -tip v3 7/7] kprobes: Use synchronize_rcu_tasks() for optprobe with CONFIG_PREEMPT Date: Tue, 19 Sep 2017 19:03:00 +0900 Message-Id: <150581537997.32348.14125457458719053753.stgit@devbox> X-Mailer: git-send-email 2.13.5 In-Reply-To: <150581509713.32348.1905525476438163954.stgit@devbox> References: <150581509713.32348.1905525476438163954.stgit@devbox> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3110 Lines: 74 To enable jump optimized probe with CONFIG_PREEMPT, use synchronize_rcu_tasks() to wait for all tasks preempted on trampoline code back on track. Since the jump optimized kprobes can replace multiple instructions, there can be tasks which are preempted on the 2nd (or 3rd) instructions. If the kprobe replaces those instructions by a jump instruction, when those tasks back to the preempted place, it is a middle of the jump instruction and causes a kernel panic. To avoid such tragedies in advance, kprobe optimizer prepare a detour route using normal kprobe (e.g. int3 breakpoint on x86), and wait for the tasks which is interrrupted on such place by synchronize_sched() when CONFIG_PREEMPT=n. If CONFIG_PREEMPT=y, things be more complicated, because such interrupted thread can be preempted (other thread can be scheduled in interrupt handler.) So, kprobes optimizer has to wait for those tasks scheduled normally. In this case we can use synchronize_rcu_tasks() which ensures that all preempted tasks back on track and schedule it. Signed-off-by: Masami Hiramatsu Reviewed-by: Paul E. McKenney --- arch/Kconfig | 2 +- kernel/kprobes.c | 18 +++++++++++++----- 2 files changed, 14 insertions(+), 6 deletions(-) diff --git a/arch/Kconfig b/arch/Kconfig index 1aafb4efbb51..f75c8e8a229b 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -90,7 +90,7 @@ config STATIC_KEYS_SELFTEST config OPTPROBES def_bool y depends on KPROBES && HAVE_OPTPROBES - depends on !PREEMPT + select TASKS_RCU if PREEMPT config KPROBES_ON_FTRACE def_bool y diff --git a/kernel/kprobes.c b/kernel/kprobes.c index de73b843c623..21d42ed2aaa5 100644 --- a/kernel/kprobes.c +++ b/kernel/kprobes.c @@ -577,12 +577,20 @@ static void kprobe_optimizer(struct work_struct *work) /* * Step 2: Wait for quiesence period to ensure all running interrupts - * are done. Because optprobe may modify multiple instructions - * there is a chance that Nth instruction is interrupted. In that - * case, running interrupt can return to 2nd-Nth byte of jump - * instruction. This wait is for avoiding it. + * are done. Because optprobe may modify multiple instructions, + * there is a chance that the Nth instruction is interrupted. In that + * case, running interrupt can return to the Nth byte of jump + * instruction. This can be avoided by waiting for returning of + * such interrupts, since (until here) the first byte of the optimized + * probe is already replaced with normal kprobe (sw breakpoint) and + * all threads which reach to the probed address will hit it and + * bypass the copied instructions (instead of executing the original.) + * With CONFIG_PREEMPT, such interrupts can be preepmted. To wait + * for such thread, we will use synchronize_rcu_tasks() which ensures + * all preeempted tasks are scheduled normally (not preempted). + * So we can ensure there is no threads preempted at probed address. */ - synchronize_sched(); + synchronize_rcu_tasks(); /* Step 3: Optimize kprobes after quiesence period */ do_optimize_kprobes();