Received: by 2002:a25:86ce:0:0:0:0:0 with SMTP id y14csp2188962ybm; Thu, 23 May 2019 12:50:11 -0700 (PDT) X-Google-Smtp-Source: APXvYqyqIqunM1051BajJDut1ZynPdc34ACvvfJ3ogvsjfTBwj20ARfSTkEHQdO0TCPsUmUJ9kAh X-Received: by 2002:aa7:9289:: with SMTP id j9mr19505688pfa.251.1558641011008; Thu, 23 May 2019 12:50:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1558641011; cv=none; d=google.com; s=arc-20160816; b=wvqDHD75Esx3SxEd0KY3mDlCIq3+UlRAw/2ZDBUddyprXVCgKNmsV+1Jkz5JNGi6Cb L1izfaS7Uft8qcl44qHFjwtnitjYdCXpTxMqnWcr8C7C98LXaxijjMzfvB+4aon9Nq1E 8+rHpZEtaM7IN2Ztk6PVE5mGQumkGXGYsvk7imJypL+gw5uhw6CQfOkCIo0a77Z2et8S thdpYq6BJKXLY4jCxZC2epdE0D0x/Lj8MJRJQA0ziLG5EHelWynZpH3htLvnf9h8rjp1 /I9yOGkdUlX7FgWou0970xSdR8NEDpR5sckVnQmdACOESosn4VOBwswrLk76H4ac+n1+ YBBw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=jKMsWwhaPj/U6PgL8G49kb2ZuBTki4huExKOvAuQjB4=; b=w5jN5kVIpsnqwUots2LINZUaAiogK29TlnoVgJ2N6/mQM1Nhrz8BQkOpl7QJTuORIP loa2JUarl1gYjGOA6XA8qJPhTMjv+IWjDMUjp3BlRCKbbw+/wA295wkf7Ks2To8klAr/ KJl8vnk5DpCdywh71YXGD4MqmMeraDsOppP/Sx4l0YGRsv+hYpS1A7Z/Evf9BdAke+kq q5voiKzXfJzMOkCskFHQiQ6N+iXCQnDImG0IfCHcq6Dz6j4a4XRgpH0PzPuJwZLuOMLf VugqZ1sHcAjG+M3zn/7HdP2lfavMCJcAniEbGe/NCvpU3gQ/SL2luJmNQo2C7Af2ySNg b4LQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=UMi0w5+P; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id q20si766010pgj.216.2019.05.23.12.49.55; Thu, 23 May 2019 12:50:10 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=UMi0w5+P; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388273AbfEWTLs (ORCPT + 99 others); Thu, 23 May 2019 15:11:48 -0400 Received: from mail.kernel.org ([198.145.29.99]:45228 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388255AbfEWTLq (ORCPT ); Thu, 23 May 2019 15:11:46 -0400 Received: from localhost (83-86-89-107.cable.dynamic.v4.ziggo.nl [83.86.89.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 720FF2184B; Thu, 23 May 2019 19:11:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1558638704; bh=nWNaN2wnvTy5mQwGsxG4bSeirIXE/TpY6aau8ZKjlpU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UMi0w5+PWWAmRawBH2zYlTdWgDcjaPSY0U3VB1M0X8vH4PrnOS+g5ZU3n+Sxl97ws l/zQxDDM8GrXhiisk0Bl5kWT7xY+eRtEvdoICCT9HfO/SiR/TjykruIU5aHl/S/Pyl bkOrZRKkwKfncEl6jPoTy2i/wIWCxmXt2e3Ufq1w= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Andy Lutomirski , Nicolai Stange , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , the arch/x86 maintainers , Josh Poimboeuf , Jiri Kosina , Miroslav Benes , Petr Mladek , Joe Lawrence , Shuah Khan , Konrad Rzeszutek Wilk , Tim Chen , Sebastian Andrzej Siewior , Mimi Zohar , Juergen Gross , Nick Desaulniers , Nayna Jain , Masahiro Yamada , Joerg Roedel , "open list:KERNEL SELFTEST FRAMEWORK" , Masami Hiramatsu , "Peter Zijlstra (Intel)" , "Steven Rostedt (VMware)" Subject: [PATCH 4.14 34/77] ftrace/x86_64: Emulate call function while updating in breakpoint handler Date: Thu, 23 May 2019 21:05:52 +0200 Message-Id: <20190523181724.937933016@linuxfoundation.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190523181719.982121681@linuxfoundation.org> References: <20190523181719.982121681@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Peter Zijlstra commit 9e298e8604088a600d8100a111a532a9d342af09 upstream. Nicolai Stange discovered[1] that if live kernel patching is enabled, and the function tracer started tracing the same function that was patched, the conversion of the fentry call site during the translation of going from calling the live kernel patch trampoline to the iterator trampoline, would have as slight window where it didn't call anything. As live kernel patching depends on ftrace to always call its code (to prevent the function being traced from being called, as it will redirect it). This small window would allow the old buggy function to be called, and this can cause undesirable results. Nicolai submitted new patches[2] but these were controversial. As this is similar to the static call emulation issues that came up a while ago[3]. But after some debate[4][5] adding a gap in the stack when entering the breakpoint handler allows for pushing the return address onto the stack to easily emulate a call. [1] http://lkml.kernel.org/r/20180726104029.7736-1-nstange@suse.de [2] http://lkml.kernel.org/r/20190427100639.15074-1-nstange@suse.de [3] http://lkml.kernel.org/r/3cf04e113d71c9f8e4be95fb84a510f085aa4afa.1541711457.git.jpoimboe@redhat.com [4] http://lkml.kernel.org/r/CAHk-=wh5OpheSU8Em_Q3Hg8qw_JtoijxOdPtHru6d+5K8TWM=A@mail.gmail.com [5] http://lkml.kernel.org/r/CAHk-=wjvQxY4DvPrJ6haPgAa6b906h=MwZXO6G8OtiTGe=N7_w@mail.gmail.com [ Live kernel patching is not implemented on x86_32, thus the emulate calls are only for x86_64. ] Cc: Andy Lutomirski Cc: Nicolai Stange Cc: Thomas Gleixner Cc: Ingo Molnar Cc: Borislav Petkov Cc: "H. Peter Anvin" Cc: the arch/x86 maintainers Cc: Josh Poimboeuf Cc: Jiri Kosina Cc: Miroslav Benes Cc: Petr Mladek Cc: Joe Lawrence Cc: Shuah Khan Cc: Konrad Rzeszutek Wilk Cc: Tim Chen Cc: Sebastian Andrzej Siewior Cc: Mimi Zohar Cc: Juergen Gross Cc: Nick Desaulniers Cc: Nayna Jain Cc: Masahiro Yamada Cc: Joerg Roedel Cc: "open list:KERNEL SELFTEST FRAMEWORK" Cc: stable@vger.kernel.org Fixes: b700e7f03df5 ("livepatch: kernel: add support for live patching") Tested-by: Nicolai Stange Reviewed-by: Nicolai Stange Reviewed-by: Masami Hiramatsu Signed-off-by: Peter Zijlstra (Intel) [ Changed to only implement emulated calls for x86_64 ] Signed-off-by: Steven Rostedt (VMware) Signed-off-by: Greg Kroah-Hartman --- arch/x86/kernel/ftrace.c | 32 +++++++++++++++++++++++++++----- 1 file changed, 27 insertions(+), 5 deletions(-) --- a/arch/x86/kernel/ftrace.c +++ b/arch/x86/kernel/ftrace.c @@ -30,6 +30,7 @@ #include #include #include +#include #ifdef CONFIG_DYNAMIC_FTRACE @@ -229,6 +230,7 @@ int ftrace_modify_call(struct dyn_ftrace } static unsigned long ftrace_update_func; +static unsigned long ftrace_update_func_call; static int update_ftrace_func(unsigned long ip, void *new) { @@ -257,6 +259,8 @@ int ftrace_update_ftrace_func(ftrace_fun unsigned char *new; int ret; + ftrace_update_func_call = (unsigned long)func; + new = ftrace_call_replace(ip, (unsigned long)func); ret = update_ftrace_func(ip, new); @@ -292,13 +296,28 @@ int ftrace_int3_handler(struct pt_regs * if (WARN_ON_ONCE(!regs)) return 0; - ip = regs->ip - 1; - if (!ftrace_location(ip) && !is_ftrace_caller(ip)) - return 0; + ip = regs->ip - INT3_INSN_SIZE; - regs->ip += MCOUNT_INSN_SIZE - 1; +#ifdef CONFIG_X86_64 + if (ftrace_location(ip)) { + int3_emulate_call(regs, (unsigned long)ftrace_regs_caller); + return 1; + } else if (is_ftrace_caller(ip)) { + if (!ftrace_update_func_call) { + int3_emulate_jmp(regs, ip + CALL_INSN_SIZE); + return 1; + } + int3_emulate_call(regs, ftrace_update_func_call); + return 1; + } +#else + if (ftrace_location(ip) || is_ftrace_caller(ip)) { + int3_emulate_jmp(regs, ip + CALL_INSN_SIZE); + return 1; + } +#endif - return 1; + return 0; } static int ftrace_write(unsigned long ip, const char *val, int size) @@ -869,6 +888,8 @@ void arch_ftrace_update_trampoline(struc func = ftrace_ops_get_func(ops); + ftrace_update_func_call = (unsigned long)func; + /* Do a safe modify in case the trampoline is executing */ new = ftrace_call_replace(ip, (unsigned long)func); ret = update_ftrace_func(ip, new); @@ -965,6 +986,7 @@ static int ftrace_mod_jmp(unsigned long { unsigned char *new; + ftrace_update_func_call = 0UL; new = ftrace_jmp_replace(ip, (unsigned long)func); return update_ftrace_func(ip, new);