Received: by 2002:a25:d7c1:0:0:0:0:0 with SMTP id o184csp4632437ybg; Tue, 29 Oct 2019 10:02:31 -0700 (PDT) X-Google-Smtp-Source: APXvYqy7/85O4vtSaJcTF7eaMtBchsh7FgUDN03DtgtCT7O6SPrDfu6urVObEm+sj8ygS92V9IEt X-Received: by 2002:adf:e544:: with SMTP id z4mr22225675wrm.6.1572368551706; Tue, 29 Oct 2019 10:02:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1572368551; cv=none; d=google.com; s=arc-20160816; b=LlzMWabJnU0iQkbexza7vZHNITt1r3Ixvoo/KljeVI1XKaKbIy7zhxbLhix69ftalu yObCMpUCw6dswqAYhHKE2qHeHE/nsZJMDlUh2PVxAMS3Q5Xru08rzlecTu7Gn5oip5/4 pLuCjaSzAddpCsf6RW6uLedalNf24WHp8JheUlA3d96yK0J03BE1p+BzBqmq6BUqUeX+ Q3O/Qn7F+wfk8wdWPOtxSjhOu73HqRnL6PV9t3jDI7oKhe6qzf1qf/ChqBUBu5zYoa/b SJpKHr/6w63glw1+XsyPrDNVJljHDp2h0wP6lM65JZ9Y5qmXb4xQolib1CoXWjjInSdr X6vg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=JxQJvqDj876F6t9mW1ZUsv8YZHwYQZuQOHHn1UUwH4A=; b=hcIEuTm+vFb6SueVcS7ZQb+X6JIW6Ver9PeZo86zxhVpj/s0JKbEwBz1RTuVMsY3X0 rw/12ittsUC4GDFy2nERcGvE+BpUntquaOt1fnGtLnmI9zB/yGZDLICxQ9nfsr2jVgKQ KfMYPKJ4IzBuK1jMgN7kFHHwUQEKZzRngH60/kWJcut6Ase2/5u69s7l37zT8G0K6dJc C0tRPx0p+oYUeMqMbVt0NyHsBRJq6qk/9UaEcK2Nsw6gKq639rB93VMQETdrc4514vRV oe202kYIEWnvfvKQR8RxR3tohvgyd1+ktsiGkSWCqckmy3aD6x8yPGgoKfPXxy4XdWRM TLBw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id mj11si5997004ejb.140.2019.10.29.10.02.05; Tue, 29 Oct 2019 10:02:31 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390715AbfJ2Q7I (ORCPT + 99 others); Tue, 29 Oct 2019 12:59:08 -0400 Received: from foss.arm.com ([217.140.110.172]:54646 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390643AbfJ2Q7I (ORCPT ); Tue, 29 Oct 2019 12:59:08 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7969A4F5; Tue, 29 Oct 2019 09:59:07 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 26C623F71F; Tue, 29 Oct 2019 09:59:05 -0700 (PDT) From: Mark Rutland To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, amit.kachhap@arm.com, catalin.marinas@arm.com, deller@gmx.de, duwe@suse.de, James.Bottomley@HansenPartnership.com, james.morse@arm.com, jeyu@kernel.org, jpoimboe@redhat.com, jthierry@redhat.com, linux-parisc@vger.kernel.org, mark.rutland@arm.com, mingo@redhat.com, peterz@infradead.org, rostedt@goodmis.org, svens@stackframe.org, takahiro.akashi@linaro.org, will@kernel.org Subject: [PATCHv2 4/8] arm64: module/ftrace: intialize PLT at load time Date: Tue, 29 Oct 2019 16:58:28 +0000 Message-Id: <20191029165832.33606-5-mark.rutland@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20191029165832.33606-1-mark.rutland@arm.com> References: <20191029165832.33606-1-mark.rutland@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Currently we lazily-initialize a module's ftrace PLT at runtime when we install the first ftrace call. To do so we have to apply a number of sanity checks, transiently mark the module text as RW, and perform an IPI as part of handling Neoverse-N1 erratum #1542419. We only expect the ftrace trampoline to point at ftrace_caller() (AKA FTRACE_ADDR), so let's simplify all of this by intializing the PLT at module load time, before the module loader marks the module RO and performs the intial I-cache maintenance for the module. Thus we can rely on the module having been correctly intialized, and can simplify the runtime work necessary to install an ftrace call in a module. This will also allow for the removal of module_disable_ro(). Tested by forcing ftrace_make_call() to use the module PLT, and then loading up a module after setting up ftrace with: | echo ":mod:" > set_ftrace_filter; | echo function > current_tracer; | modprobe Since FTRACE_ADDR is only defined when CONFIG_DYNAMIC_FTRACE is selected, we wrap its use along with most of module_init_ftrace_plt() with ifdeffery rather than using IS_ENABLED(). Signed-off-by: Mark Rutland Reviewed-by: Ard Biesheuvel Cc: Catalin Marinas Cc: James Morse Cc: Peter Zijlstra Cc: Will Deacon --- arch/arm64/kernel/ftrace.c | 55 ++++++++++++---------------------------------- arch/arm64/kernel/module.c | 32 +++++++++++++++++---------- 2 files changed, 35 insertions(+), 52 deletions(-) diff --git a/arch/arm64/kernel/ftrace.c b/arch/arm64/kernel/ftrace.c index 06e56b470315..822718eafdb4 100644 --- a/arch/arm64/kernel/ftrace.c +++ b/arch/arm64/kernel/ftrace.c @@ -73,10 +73,22 @@ int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr) if (offset < -SZ_128M || offset >= SZ_128M) { #ifdef CONFIG_ARM64_MODULE_PLTS - struct plt_entry trampoline, *dst; struct module *mod; /* + * There is only one ftrace trampoline per module. For now, + * this is not a problem since on arm64, all dynamic ftrace + * invocations are routed via ftrace_caller(). This will need + * to be revisited if support for multiple ftrace entry points + * is added in the future, but for now, the pr_err() below + * deals with a theoretical issue only. + */ + if (addr != FTRACE_ADDR) { + pr_err("ftrace: far branches to multiple entry points unsupported inside a single module\n"); + return -EINVAL; + } + + /* * On kernels that support module PLTs, the offset between the * branch instruction and its target may legally exceed the * range of an ordinary relative 'bl' opcode. In this case, we @@ -93,46 +105,7 @@ int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr) if (WARN_ON(!mod)) return -EINVAL; - /* - * There is only one ftrace trampoline per module. For now, - * this is not a problem since on arm64, all dynamic ftrace - * invocations are routed via ftrace_caller(). This will need - * to be revisited if support for multiple ftrace entry points - * is added in the future, but for now, the pr_err() below - * deals with a theoretical issue only. - * - * Note that PLTs are place relative, and plt_entries_equal() - * checks whether they point to the same target. Here, we need - * to check if the actual opcodes are in fact identical, - * regardless of the offset in memory so use memcmp() instead. - */ - dst = mod->arch.ftrace_trampoline; - trampoline = get_plt_entry(addr, dst); - if (memcmp(dst, &trampoline, sizeof(trampoline))) { - if (plt_entry_is_initialized(dst)) { - pr_err("ftrace: far branches to multiple entry points unsupported inside a single module\n"); - return -EINVAL; - } - - /* point the trampoline to our ftrace entry point */ - module_disable_ro(mod); - *dst = trampoline; - module_enable_ro(mod, true); - - /* - * Ensure updated trampoline is visible to instruction - * fetch before we patch in the branch. Although the - * architecture doesn't require an IPI in this case, - * Neoverse-N1 erratum #1542419 does require one - * if the TLB maintenance in module_enable_ro() is - * skipped due to rodata_enabled. It doesn't seem worth - * it to make it conditional given that this is - * certainly not a fast-path. - */ - flush_icache_range((unsigned long)&dst[0], - (unsigned long)&dst[1]); - } - addr = (unsigned long)dst; + addr = (unsigned long)mod->arch.ftrace_trampoline; #else /* CONFIG_ARM64_MODULE_PLTS */ return -EINVAL; #endif /* CONFIG_ARM64_MODULE_PLTS */ diff --git a/arch/arm64/kernel/module.c b/arch/arm64/kernel/module.c index 763a86d52fef..5f5bc3b94da7 100644 --- a/arch/arm64/kernel/module.c +++ b/arch/arm64/kernel/module.c @@ -9,6 +9,7 @@ #include #include +#include #include #include #include @@ -485,24 +486,33 @@ static const Elf_Shdr *find_section(const Elf_Ehdr *hdr, return NULL; } +int module_init_ftrace_plt(const Elf_Ehdr *hdr, + const Elf_Shdr *sechdrs, + struct module *mod) +{ +#if defined(CONFIG_ARM64_MODULE_PLTS) && defined(CONFIG_DYNAMIC_FTRACE) + const Elf_Shdr *s; + struct plt_entry *plt; + + s = find_section(hdr, sechdrs, ".text.ftrace_trampoline"); + if (!s) + return -ENOEXEC; + + plt = (void *)s->sh_addr; + *plt = get_plt_entry(FTRACE_ADDR, plt); + mod->arch.ftrace_trampoline = plt; +#endif + return 0; +} + int module_finalize(const Elf_Ehdr *hdr, const Elf_Shdr *sechdrs, struct module *me) { const Elf_Shdr *s; - s = find_section(hdr, sechdrs, ".altinstructions"); if (s) apply_alternatives_module((void *)s->sh_addr, s->sh_size); -#ifdef CONFIG_ARM64_MODULE_PLTS - if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE)) { - s = find_section(hdr, sechdrs, ".text.ftrace_trampoline"); - if (!s) - return -ENOEXEC; - me->arch.ftrace_trampoline = (void *)s->sh_addr; - } -#endif - - return 0; + return module_init_ftrace_plt(hdr, sechdrs, me); } -- 2.11.0