Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S964989AbbBCLw1 (ORCPT ); Tue, 3 Feb 2015 06:52:27 -0500 Received: from mail9.hitachi.co.jp ([133.145.228.44]:59966 "EHLO mail9.hitachi.co.jp" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932250AbbBCLw0 (ORCPT ); Tue, 3 Feb 2015 06:52:26 -0500 Message-ID: <54D0B673.5010407@hitachi.com> Date: Tue, 03 Feb 2015 20:52:19 +0900 From: Masami Hiramatsu Organization: Hitachi, Ltd., Japan User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:13.0) Gecko/20120614 Thunderbird/13.0.1 MIME-Version: 1.0 To: Petr Mladek Cc: Ingo Molnar , "David S. Miller" , Anil S Keshavamurthy , Ananth NMavinakayanahalli , Frederic Weisbecker , Steven Rostedt , Jiri Kosina , linux-kernel@vger.kernel.org, Ingo Molnar Subject: Re: Re: [PATCH v2] kprobes/x86: Use 5-byte NOP when the code might be modified by ftrace References: <1422899338-26399-1-git-send-email-pmladek@suse.cz> <54D07BB3.1020806@hitachi.com> <20150203113828.GD1576@dhcp128.suse.cz> In-Reply-To: <20150203113828.GD1576@dhcp128.suse.cz> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 6847 Lines: 182 (2015/02/03 20:38), Petr Mladek wrote: > On Tue 2015-02-03 16:41:39, Masami Hiramatsu wrote: >> (2015/02/03 2:48), Petr Mladek wrote: >>> can_probe() checks if the given address points to the beginning of >>> an instruction. It analyzes all the instructions from the beginning >>> of the function until the given address. The code might be modified >>> by another Kprobe. In this case, the current code is read into a buffer, >>> int3 breakpoint is replaced by the saved opcode in the buffer, and >>> can_probe() analyzes the buffer instead. >>> >>> There is a bug that __recover_probed_insn() tries to restore >>> the original code even for Kprobes using the ftrace framework. >>> But in this case, the opcode is not stored. See the difference >>> between arch_prepare_kprobe() and arch_prepare_kprobe_ftrace(). >>> The opcode is stored by arch_copy_kprobe() only from >>> arch_prepare_kprobe(). >>> >>> This patch makes Kprobe to use the ideal 5-byte NOP when the code >>> can be modified by ftrace. It is the original instruction, see >>> ftrace_make_nop() and ftrace_nop_replace(). >>> >>> Note that we always need to use the NOP for ftrace locations. Kprobes >>> do not block ftrace and the instruction might get modified at anytime. >>> It might even be in an inconsistent state because it is modified step >>> by step using the int3 breakpoint. >>> >>> The patch also fixes indentation of the touched comment. >>> >>> Note that I found this problem when playing with Kprobes. I did it >>> on x86_64 with gcc-4.8.3 that supported -mfentry. I modified >>> samples/kprobes/kprobe_example.c and added offset 5 to put >>> the probe right after the fentry area: >>> >>> --- cut --- >>> static struct kprobe kp = { >>> .symbol_name = "do_fork", >>> + .offset = 5, >>> }; >>> --- cut --- >>> >>> Then I was able to load kprobe_example before jprobe_example >>> but not the other way around: >>> >>> $> modprobe jprobe_example >>> $> modprobe kprobe_example >>> modprobe: ERROR: could not insert 'kprobe_example': Invalid or incomplete multibyte or wide character >>> >>> It did not make much sense and debugging pointed to the bug >>> described above. >>> >> >> This looks good to me :) >> >> Acked-by: Masami Hiramatsu >> >> Ingo, could you merge this as an urgent fix? > > Please, wait a bit, see below. > >> Thank you! >> >>> Signed-off-by: Petr Mladek >>> --- >>> arch/x86/kernel/kprobes/core.c | 42 ++++++++++++++++++++++++++++-------------- >>> 1 file changed, 28 insertions(+), 14 deletions(-) >>> >>> Changes against v1: >>> >>> + always use 5-byte NOP for ftrace location >>> + fix indentation of the touched comment >>> >>> diff --git a/arch/x86/kernel/kprobes/core.c b/arch/x86/kernel/kprobes/core.c >>> index 98f654d466e5..2f464b56766a 100644 >>> --- a/arch/x86/kernel/kprobes/core.c >>> +++ b/arch/x86/kernel/kprobes/core.c >>> @@ -223,27 +223,41 @@ static unsigned long >>> __recover_probed_insn(kprobe_opcode_t *buf, unsigned long addr) >>> { >>> struct kprobe *kp; >>> + unsigned long faddr; >>> >>> kp = get_kprobe((void *)addr); >>> - /* There is no probe, return original address */ >>> - if (!kp) >>> + faddr = ftrace_location(addr); > > I have just realized that ftrace_location() might return another > address if the given one points inside the ftrace_location. > This situation is not checked by this patch. I am going to work > on v3. Ah, I see. But when it happens, something goes wrong, since __recover_probed_insn() must be called with the address from where an instruction starts (e.g. function entry or the instruction boundary.) So, we just need "BUG_ON(faddr && faddr != addr)" for that case. Thank you, > > I knew that I should not have sent the patch just before leaving :-( > > Best Regards, > Petr > >>> + /* >>> + * Use the current code if it is not modified by Kprobe >>> + * and it cannot be modified by ftrace. >>> + */ >>> + if (!kp && !faddr) >>> return addr; >>> >>> /* >>> - * Basically, kp->ainsn.insn has an original instruction. >>> - * However, RIP-relative instruction can not do single-stepping >>> - * at different place, __copy_instruction() tweaks the displacement of >>> - * that instruction. In that case, we can't recover the instruction >>> - * from the kp->ainsn.insn. >>> + * Basically, kp->ainsn.insn has an original instruction. >>> + * However, RIP-relative instruction can not do single-stepping >>> + * at different place, __copy_instruction() tweaks the displacement of >>> + * that instruction. In that case, we can't recover the instruction >>> + * from the kp->ainsn.insn. >>> * >>> - * On the other hand, kp->opcode has a copy of the first byte of >>> - * the probed instruction, which is overwritten by int3. And >>> - * the instruction at kp->addr is not modified by kprobes except >>> - * for the first byte, we can recover the original instruction >>> - * from it and kp->opcode. >>> + * On the other hand, in case on normal Kprobe, kp->opcode has a copy >>> + * of the first byte of the probed instruction, which is overwritten >>> + * by int3. And the instruction at kp->addr is not modified by kprobes >>> + * except for the first byte, we can recover the original instruction >>> + * from it and kp->opcode. >>> + * >>> + * In case of Kprobes using ftrace, we do not have a copy of >>> + * the original instruction. In fact, the ftrace location might >>> + * be modified at anytime and even could be in an inconsistent state. >>> + * Fortunately, we know that the original code is the ideal 5-byte >>> + * long NOP. >>> */ >>> - memcpy(buf, kp->addr, MAX_INSN_SIZE * sizeof(kprobe_opcode_t)); >>> - buf[0] = kp->opcode; >>> + memcpy(buf, (void *)addr, MAX_INSN_SIZE * sizeof(kprobe_opcode_t)); >>> + if (faddr) >>> + memcpy(buf, ideal_nops[NOP_ATOMIC5], MCOUNT_INSN_SIZE); >>> + else >>> + buf[0] = kp->opcode; >>> return (unsigned long)buf; >>> } >>> >>> >> >> >> -- >> Masami HIRAMATSU >> Software Platform Research Dept. Linux Technology Research Center >> Hitachi, Ltd., Yokohama Research Laboratory >> E-mail: masami.hiramatsu.pt@hitachi.com >> >> > -- > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ > -- Masami HIRAMATSU Software Platform Research Dept. Linux Technology Research Center Hitachi, Ltd., Yokohama Research Laboratory E-mail: masami.hiramatsu.pt@hitachi.com -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/