This series introduces a new ftrace feature CONFIG_FTRACE_FUNC_PROTOTYPE to
record function parameters and return value. It can be enabled/disabled at
runtime by a new trace option "record-funcproto".
To achieve this, we need the function prototype infomation and the location of
parameters. A new tool scripts/ftrace/funcprototype is provided to collect such
necessary infomation and put them into kernel. It walks through the DWARF debug
sections in each object file and output assembly code which defines the function
prototype data. Then the assembly code is built into the original object file.
Here is an example of memremap() function:
.section __funcprotostr, "a"
.P_memremap_0:
.string "offset"
.P_memremap_1:
.string "size"
.P_memremap_2:
.string "flags"
.section __funcproto, "a"
.quad memremap
.byte 0x8
.byte 0x3
.quad .P_memremap_0
.byte 0x8
.byte 0x55
.byte 0x0
.quad .P_memremap_1
.byte 0x8
.byte 0x54
.byte 0x0
.quad .P_memremap_2
.byte 0x8
.byte 0x51
.byte 0x0
Note that currently funcprototype only support global functions. Local functions
can also be supported by using the idea described in recordmcount.pl - temporary
change local functions to global.
The C ABI is arch specific. For arch which supports this feature must
implement a new arch-specific interface arch_fgraph_record_params() and
deliver the return value of function to ftrace core part.
This series only add support for x86_64 platform. Other platforms can be
supported in the future.
Here is an example of the graph trace of function pick_next_task_fair().
Note that we only record the parameter and return value of global
functions.
2) | pick_next_task_fair() {
2) | update_blocked_averages() {
2) 0.765 us | _raw_spin_lock_irqsave(lock=0xffff88807da2b100); /* ret=0x0000000000000082 */
2) 0.944 us | update_rq_clock(rq=0xffff88807da2b100);
2) 0.612 us | __update_load_avg_cfs_rq(now=0x000000251b8516ee, cfs_rq=0xffff8880754f7488); /* ret=0 */
2) 0.654 us | __update_load_avg_se(now=0x000000251b8516ee, cfs_rq=0xffff88807da2b180, se=0xffff88807be2e0d8); /* ret=0 */
2) 0.206 us | __update_load_avg_cfs_rq(now=0x000000251b8516ee, cfs_rq=0xffff88807da2b180); /* ret=0 */
2) | __update_load_avg_cfs_rq(now=0x000000251b8516ee, cfs_rq=0xffff888079b5fb18) {
2) 2.410 us | __accumulate_pelt_segments();
2) 3.103 us | } /* ret=1 */
2) 0.193 us | __update_load_avg_cfs_rq(now=0x000000251b8516ee, cfs_rq=0xffff88807da2b180); /* ret=0 */
2) | update_rt_rq_load_avg(now=0x000000251b8516ee, rq=0xffff88807da2b100, running=0) {
2) 0.258 us | __accumulate_pelt_segments();
2) 1.617 us | } /* ret=1 */
2) | update_dl_rq_load_avg(now=0x000000251b8516ee, rq=0xffff88807da2b100, running=0) {
2) 0.230 us | __accumulate_pelt_segments();
2) 1.511 us | } /* ret=1 */
2) 1.040 us | _raw_spin_unlock_irqrestore(lock=0xffff88807da2b100, flags=0x0000000000000082);
2) + 14.739 us | }
2) | load_balance() {
2) | find_busiest_group() {
2) 0.874 us | update_group_capacity(sd=0xffff88807c1d37d0, cpu=2);
2) 1.761 us | idle_cpu();
2) 0.262 us | idle_cpu();
2) 0.217 us | idle_cpu();
2) 6.338 us | }
2) 8.442 us | }
2) 1.823 us | __msecs_to_jiffies(m=0x00000006); /* ret=0x0000000000000002 */
2) | load_balance() {
2) | find_busiest_group() {
2) 0.434 us | idle_cpu();
2) 0.233 us | idle_cpu();
2) 0.210 us | idle_cpu();
2) 2.308 us | }
2) 2.821 us | }
2) 0.263 us | __msecs_to_jiffies(m=0x00000008); /* ret=0x0000000000000002 */
2) 0.977 us | _raw_spin_lock(lock=0xffff88807da2b100);
2) + 32.262 us | }
The printing rules of each value is:
o For signed value, it is always printed as decimal number.
o For unsigned value,
- For value has size great than 8, it is printed as '{..}'.
- For value has size of 1,2,4,8, it is printed as hexadecimal number.
- If failed to record a parameter, it is printed as '?'.
Changbin Du (11):
ftrace: move recordmcount tools to scripts/ftrace
ftrace: introduce new building tool funcprototype
asm-generic: add generic dwarf definition
ftrace/hash: add private data field
ftrace: create memcache for hash entries
ftrace: process function prototype data in vmlinux and modules
ftrace: prepare arch specific interfaces for function prototype
feature
ftrace: introduce core part of function prototype recording
x86_64: add function prototype recording support
ftrace: add doc for new option record-funcproto
MAINTAINERS: make scripts/ftrace/ maintained
Documentation/trace/ftrace.rst | 6 +
MAINTAINERS | 2 +
arch/arm/kernel/ftrace.c | 2 +-
arch/arm64/kernel/ftrace.c | 2 +-
arch/csky/kernel/ftrace.c | 2 +-
arch/microblaze/kernel/ftrace.c | 2 +-
arch/mips/kernel/ftrace.c | 2 +-
arch/nds32/kernel/ftrace.c | 5 +-
arch/parisc/kernel/ftrace.c | 2 +-
arch/powerpc/kernel/trace/ftrace.c | 2 +-
arch/riscv/kernel/ftrace.c | 2 +-
arch/s390/kernel/ftrace.c | 2 +-
arch/sh/kernel/ftrace.c | 2 +-
arch/sparc/kernel/ftrace.c | 2 +-
arch/x86/Kconfig | 1 +
arch/x86/kernel/ftrace.c | 84 +++-
arch/x86/kernel/ftrace_64.S | 4 +-
include/asm-generic/dwarf.h | 199 +++++++++
include/asm-generic/vmlinux.lds.h | 18 +
include/linux/ftrace.h | 55 ++-
include/linux/module.h | 4 +
kernel/module.c | 25 +-
kernel/trace/Kconfig | 19 +
kernel/trace/fgraph.c | 26 +-
kernel/trace/ftrace.c | 164 +++++++-
kernel/trace/trace.h | 20 +-
kernel/trace/trace_entries.h | 10 +
kernel/trace/trace_functions_graph.c | 108 ++++-
kernel/trace/trace_irqsoff.c | 3 +-
kernel/trace/trace_sched_wakeup.c | 3 +-
scripts/.gitignore | 1 -
scripts/Makefile | 2 +-
scripts/Makefile.build | 28 +-
scripts/ftrace/.gitignore | 6 +
scripts/ftrace/Makefile | 9 +
scripts/ftrace/funcprototype.c | 576 +++++++++++++++++++++++++++
scripts/{ => ftrace}/recordmcount.c | 0
scripts/{ => ftrace}/recordmcount.h | 0
scripts/{ => ftrace}/recordmcount.pl | 0
39 files changed, 1340 insertions(+), 60 deletions(-)
create mode 100644 include/asm-generic/dwarf.h
create mode 100644 scripts/ftrace/.gitignore
create mode 100644 scripts/ftrace/Makefile
create mode 100644 scripts/ftrace/funcprototype.c
rename scripts/{ => ftrace}/recordmcount.c (100%)
rename scripts/{ => ftrace}/recordmcount.h (100%)
rename scripts/{ => ftrace}/recordmcount.pl (100%)
mode change 100755 => 100644
--
2.20.1
Move ftrace tools to its own directory. We will add another tool later.
Cc: John F. Reiser <[email protected]>
Signed-off-by: Changbin Du <[email protected]>
---
scripts/.gitignore | 1 -
scripts/Makefile | 2 +-
scripts/Makefile.build | 10 +++++-----
scripts/ftrace/.gitignore | 4 ++++
scripts/ftrace/Makefile | 4 ++++
scripts/{ => ftrace}/recordmcount.c | 0
scripts/{ => ftrace}/recordmcount.h | 0
scripts/{ => ftrace}/recordmcount.pl | 0
8 files changed, 14 insertions(+), 7 deletions(-)
create mode 100644 scripts/ftrace/.gitignore
create mode 100644 scripts/ftrace/Makefile
rename scripts/{ => ftrace}/recordmcount.c (100%)
rename scripts/{ => ftrace}/recordmcount.h (100%)
rename scripts/{ => ftrace}/recordmcount.pl (100%)
mode change 100755 => 100644
diff --git a/scripts/.gitignore b/scripts/.gitignore
index 17f8cef88fa8..1b5b5d595d80 100644
--- a/scripts/.gitignore
+++ b/scripts/.gitignore
@@ -6,7 +6,6 @@ conmakehash
kallsyms
pnmtologo
unifdef
-recordmcount
sortextable
asn1_compiler
extract-cert
diff --git a/scripts/Makefile b/scripts/Makefile
index 16bcb8087899..d5992def49a8 100644
--- a/scripts/Makefile
+++ b/scripts/Makefile
@@ -14,7 +14,6 @@ hostprogs-$(CONFIG_BUILD_BIN2C) += bin2c
hostprogs-$(CONFIG_KALLSYMS) += kallsyms
hostprogs-$(CONFIG_LOGO) += pnmtologo
hostprogs-$(CONFIG_VT) += conmakehash
-hostprogs-$(BUILD_C_RECORDMCOUNT) += recordmcount
hostprogs-$(CONFIG_BUILDTIME_EXTABLE_SORT) += sortextable
hostprogs-$(CONFIG_ASN1) += asn1_compiler
hostprogs-$(CONFIG_MODULE_SIG) += sign-file
@@ -34,6 +33,7 @@ hostprogs-y += unifdef
subdir-$(CONFIG_GCC_PLUGINS) += gcc-plugins
subdir-$(CONFIG_MODVERSIONS) += genksyms
subdir-$(CONFIG_SECURITY_SELINUX) += selinux
+subdir-$(CONFIG_FTRACE) += ftrace
# Let clean descend into subdirs
subdir- += basic dtc gdb kconfig mod package
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index 2f66ed388d1c..67558983c518 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -188,18 +188,18 @@ endif
# files, including recordmcount.
sub_cmd_record_mcount = \
if [ $(@) != "scripts/mod/empty.o" ]; then \
- $(objtree)/scripts/recordmcount $(RECORDMCOUNT_FLAGS) "$(@)"; \
+ $(objtree)/scripts/ftrace/recordmcount $(RECORDMCOUNT_FLAGS) "$(@)"; \
fi;
-recordmcount_source := $(srctree)/scripts/recordmcount.c \
- $(srctree)/scripts/recordmcount.h
+recordmcount_source := $(srctree)/scripts/ftrace/recordmcount.c \
+ $(srctree)/scripts/ftrace/recordmcount.h
else
-sub_cmd_record_mcount = perl $(srctree)/scripts/recordmcount.pl "$(ARCH)" \
+sub_cmd_record_mcount = perl $(srctree)/scripts/ftrace/recordmcount.pl "$(ARCH)" \
"$(if $(CONFIG_CPU_BIG_ENDIAN),big,little)" \
"$(if $(CONFIG_64BIT),64,32)" \
"$(OBJDUMP)" "$(OBJCOPY)" "$(CC) $(KBUILD_CPPFLAGS) $(KBUILD_CFLAGS)" \
"$(LD) $(KBUILD_LDFLAGS)" "$(NM)" "$(RM)" "$(MV)" \
"$(if $(part-of-module),1,0)" "$(@)";
-recordmcount_source := $(srctree)/scripts/recordmcount.pl
+recordmcount_source := $(srctree)/scripts/ftrace/recordmcount.pl
endif # BUILD_C_RECORDMCOUNT
cmd_record_mcount = $(if $(findstring $(strip $(CC_FLAGS_FTRACE)),$(_c_flags)), \
$(sub_cmd_record_mcount))
diff --git a/scripts/ftrace/.gitignore b/scripts/ftrace/.gitignore
new file mode 100644
index 000000000000..54d582c8faad
--- /dev/null
+++ b/scripts/ftrace/.gitignore
@@ -0,0 +1,4 @@
+#
+# Generated files
+#
+recordmcount
diff --git a/scripts/ftrace/Makefile b/scripts/ftrace/Makefile
new file mode 100644
index 000000000000..6797e51473e5
--- /dev/null
+++ b/scripts/ftrace/Makefile
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: GPL-2.0
+
+hostprogs-$(BUILD_C_RECORDMCOUNT) += recordmcount
+always := $(hostprogs-y)
diff --git a/scripts/recordmcount.c b/scripts/ftrace/recordmcount.c
similarity index 100%
rename from scripts/recordmcount.c
rename to scripts/ftrace/recordmcount.c
diff --git a/scripts/recordmcount.h b/scripts/ftrace/recordmcount.h
similarity index 100%
rename from scripts/recordmcount.h
rename to scripts/ftrace/recordmcount.h
diff --git a/scripts/recordmcount.pl b/scripts/ftrace/recordmcount.pl
old mode 100755
new mode 100644
similarity index 100%
rename from scripts/recordmcount.pl
rename to scripts/ftrace/recordmcount.pl
--
2.20.1
This is a new ftrace tool to implement CONFIG_FTRACE_FUNC_PROTOTYPE
feature which allow ftrace record function parameters and return value
(see later patches).
Essentially funcprototype extracts only necessary information from
the DWARF debug sections in the ELF object file, including function
return type, parameter names and parameter data types. Then they will
be built into the original ELF object.
Here is an example for function memremap() in kernel/iomem.o. The
declaration is:
void *memremap(resource_size_t offset, size_t size, unsigned long flags)
The output of funcprototype tool is:
.section __funcprotostr, "a"
.P_memremap_0:
.string "offset"
.P_memremap_1:
.string "size"
.P_memremap_2:
.string "flags"
.section __funcproto, "a"
.quad memremap
.byte 0x8
.byte 0x3
.quad .P_memremap_0
.byte 0x8
.byte 0x55
.byte 0x0
.quad .P_memremap_1
.byte 0x8
.byte 0x54
.byte 0x0
.quad .P_memremap_2
.byte 0x8
.byte 0x51
.byte 0x0
The strings are placed in '__funcprotostr' section, and prototype
information is placed in '__funcproto' section. It equals to below
C struct:
struct func_param {
char *name;
uint8_t type;
uint8_t loc[2];
} __packed;
struct func_prototype {
unsigned long ip;
uint8_t ret_type;
uint8_t nr_param;
struct func_param params[0];
} __packed;
Signed-off-by: Changbin Du <[email protected]>
---
kernel/trace/Kconfig | 19 ++
scripts/Makefile.build | 18 +-
scripts/ftrace/.gitignore | 2 +
scripts/ftrace/Makefile | 7 +-
scripts/ftrace/funcprototype.c | 576 +++++++++++++++++++++++++++++++++
5 files changed, 620 insertions(+), 2 deletions(-)
create mode 100644 scripts/ftrace/funcprototype.c
diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig
index 98da8998c25c..20d1b0ae114d 100644
--- a/kernel/trace/Kconfig
+++ b/kernel/trace/Kconfig
@@ -38,6 +38,9 @@ config HAVE_FTRACE_MCOUNT_RECORD
help
See Documentation/trace/ftrace-design.rst
+config HAVE_FTRACE_FUNC_PROTOTYPE
+ bool
+
config HAVE_SYSCALL_TRACEPOINTS
bool
help
@@ -170,6 +173,22 @@ config FUNCTION_GRAPH_TRACER
the return value. This is done by setting the current return
address on the current task structure into a stack of calls.
+config FTRACE_FUNC_PROTOTYPE
+ bool "Support recording function parameters and return value"
+ default n
+ depends on DYNAMIC_FTRACE
+ depends on HAVE_FTRACE_FUNC_PROTOTYPE
+ depends on FUNCTION_GRAPH_TRACER
+ help
+ Enable the Function Graph Tracer to record function parameters and
+ return value. It can be dynamically enabled/disabled by the
+ 'record-funcproto' trace option.
+
+ By enabling this, function prototype information is built into
+ kernel. And the kernel size will increase by approximately 2%.
+
+ Say N if unsure.
+
config TRACE_PREEMPT_TOGGLE
bool
help
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index 67558983c518..d56850808d96 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -206,6 +206,21 @@ cmd_record_mcount = $(if $(findstring $(strip $(CC_FLAGS_FTRACE)),$(_c_flags)),
endif # CC_USING_RECORD_MCOUNT
endif # CONFIG_FTRACE_MCOUNT_RECORD
+ifdef CONFIG_FTRACE_FUNC_PROTOTYPE
+sub_cmd_funcprototype = \
+ $(srctree)/scripts/ftrace/funcprototype "$(@)" | \
+ $(CC) $(KBUILD_CPPFLAGS) $(KBUILD_CFLAGS) -c \
+ -o $(@D)/.tmp_$(@F:.o=.funcprototype) -x assembler -; \
+ $(LD) $(ld_flags) -r -o $(@D)/.tmp_$(@F) $@ $(@D)/.tmp_$(@F:.o=.funcprototype); \
+ mv -f $(@D)/.tmp_$(@F) $@; \
+ rm -f $(@D)/.tmp_$(@F:.o=.funcprototype);
+cmd_funcprototype = \
+ if $(OBJDUMP) -h $@ | grep -q __mcount_loc; then \
+ $(sub_cmd_funcprototype) \
+ fi
+funcprototype_source := $(srctree)/scripts/ftrace/funcprototype.c
+endif # CONFIG_FTRACE_FUNC_PROTOTYPE
+
ifdef CONFIG_STACK_VALIDATION
ifneq ($(SKIP_STACK_VALIDATION),1)
@@ -259,6 +274,7 @@ define rule_cc_o_c
$(call cmd,objtool)
$(call cmd,modversions_c)
$(call cmd,record_mcount)
+ $(call cmd,funcprototype)
endef
define rule_as_o_S
@@ -276,7 +292,7 @@ cmd_undef_syms = echo
endif
# Built-in and composite module parts
-$(obj)/%.o: $(src)/%.c $(recordmcount_source) $(objtool_dep) FORCE
+$(obj)/%.o: $(src)/%.c $(recordmcount_source) $(funcprototype_source) $(objtool_dep) FORCE
$(call cmd,force_checksrc)
$(call if_changed_rule,cc_o_c)
diff --git a/scripts/ftrace/.gitignore b/scripts/ftrace/.gitignore
index 54d582c8faad..92aa4f335656 100644
--- a/scripts/ftrace/.gitignore
+++ b/scripts/ftrace/.gitignore
@@ -2,3 +2,5 @@
# Generated files
#
recordmcount
+funcprototype
+
diff --git a/scripts/ftrace/Makefile b/scripts/ftrace/Makefile
index 6797e51473e5..c44d131b075c 100644
--- a/scripts/ftrace/Makefile
+++ b/scripts/ftrace/Makefile
@@ -1,4 +1,9 @@
# SPDX-License-Identifier: GPL-2.0
-hostprogs-$(BUILD_C_RECORDMCOUNT) += recordmcount
+hostprogs-$(BUILD_C_RECORDMCOUNT) := recordmcount
+
+hostprogs-$(CONFIG_FTRACE_FUNC_PROTOTYPE) += funcprototype
+HOSTLDLIBS_funcprototype += -lelf
+HOSTLDLIBS_funcprototype += -ldw
+
always := $(hostprogs-y)
diff --git a/scripts/ftrace/funcprototype.c b/scripts/ftrace/funcprototype.c
new file mode 100644
index 000000000000..064724047b19
--- /dev/null
+++ b/scripts/ftrace/funcprototype.c
@@ -0,0 +1,576 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * funcprototype.c: generate function prototypes of the locations of calls to
+ * 'mcount' so that ftrace can record function parameters and return value.
+ *
+ * Copyright 2019 Changbin Du <[email protected]>. All rights reserved.
+ *
+ * Usage: funcprototype [OPTION...] elf-file
+ *
+ * Here is an example for function memremap() in kernel/iomem.o. The
+ * declaration is:
+ * void *memremap(resource_size_t offset, size_t size, unsigned long flags)
+ *
+ * The output of funcprototype tool is:
+ * .section __funcprotostr, "a"
+ * .P_memremap_0:
+ * .string "offset"
+ * .P_memremap_1:
+ * .string "size"
+ * .P_memremap_2:
+ * .string "flags"
+ *
+ * .section __funcproto, "a"
+ * .quad memremap
+ * .byte 0x8
+ * .byte 0x3
+ * .quad .P_memremap_0
+ * .byte 0x8
+ * .byte 0x55
+ * .byte 0x0
+ * .quad .P_memremap_1
+ * .byte 0x8
+ * .byte 0x54
+ * .byte 0x0
+ * .quad .P_memremap_2
+ * .byte 0x8
+ * .byte 0x51
+ * .byte 0x0
+ */
+
+#include <assert.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <err.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <argp.h>
+#include <libelf.h>
+#include <gelf.h>
+#include <dwarf.h>
+#include <elfutils/libdw.h>
+#include <elfutils/libdwfl.h>
+
+struct func_param {
+ char *name;
+ uint8_t type;
+ u_int8_t loc[2]; /* Location expression, loc[0] is opcode */
+};
+
+struct func_prototype {
+ struct func_prototype *next;
+ bool skip;
+
+ char *name;
+ uint8_t ret_type;
+ uint8_t nr_param;
+ struct func_param *params;
+};
+
+#define MK_TYPE(sign, size) (((!!sign) << 7) | size)
+
+static bool is_64bit_obj;
+static struct func_prototype *func_prototype_list;
+
+
+static struct func_prototype *func_prototype_list_add_new(const char *name)
+{
+ struct func_prototype *proto;
+
+ proto = malloc(sizeof(*proto));
+ if (!proto)
+ errx(1, "no memory");
+ memset(proto, 0, sizeof(*proto));
+
+ proto->name = strdup(name);
+ if (!proto->name)
+ errx(1, "no memory");
+
+ if (!func_prototype_list) {
+ proto->next = NULL;
+ func_prototype_list = proto;
+ } else {
+ proto->next = func_prototype_list->next;
+ func_prototype_list->next = proto;
+ }
+
+ return proto;
+}
+
+static struct func_prototype *func_prototype_list_search(const char *name)
+{
+ struct func_prototype *proto;
+
+ for (proto = func_prototype_list; proto != NULL; proto = proto->next) {
+ if (!strcmp(proto->name, name))
+ return proto;
+ };
+ return NULL;
+}
+
+static void func_prototype_list_dumpnames(void)
+{
+ struct func_prototype *proto;
+
+ for (proto = func_prototype_list; proto != NULL; proto = proto->next)
+ printf("%s\n", proto->name);
+}
+
+static void func_prototype_list_destroy(void)
+{
+ struct func_prototype *proto;
+ int i;
+
+ while (func_prototype_list) {
+ proto = func_prototype_list;
+ func_prototype_list = func_prototype_list->next;
+
+ free(proto->name);
+ if (proto->params) {
+ for (i = 0; i < proto->nr_param; i++)
+ free(proto->params[i].name);
+ free(proto->params);
+ }
+ free(proto);
+ }
+}
+
+static bool is_mcount(char *name)
+{
+ return !strcmp(name, "__fentry__") ||
+ !strcmp(name, "_mcount") ||
+ !strcmp(name, "mcount");
+}
+
+static void check_elf(Elf *elf)
+{
+ GElf_Ehdr ehdr_mem;
+ GElf_Ehdr *ehdr;
+
+ ehdr = gelf_getehdr(elf, &ehdr_mem);
+ if (!ehdr)
+ errx(1, "cannot read ELF header");
+
+ is_64bit_obj = gelf_getclass(elf) == ELFCLASS64;
+
+ switch (ehdr->e_machine) {
+ case EM_386:
+ case EM_X86_64:
+ break;
+ default:
+ errx(1, "unsupported arch %d", ehdr->e_machine);
+ }
+}
+
+/**
+ * Search the symbole table to get the entry which matches @scn and @offset
+ * from relocation talbe.
+ */
+static char *search_mcount_caller(Elf *elf, GElf_Shdr *symshdr,
+ Elf_Data *symdata, int scn, int offset)
+{
+ int ndx;
+ char *caller;
+
+ for (ndx = 0; ndx < symshdr->sh_size / symshdr->sh_entsize; ++ndx) {
+ GElf_Sym sym;
+
+ gelf_getsym(symdata, ndx, &sym);
+
+ /* TODO: add local symobl support. */
+ if (GELF_ST_BIND(sym.st_info) == STB_GLOBAL &&
+ scn == sym.st_shndx && (offset >= sym.st_value) &&
+ (offset < sym.st_value + sym.st_size)) {
+ caller = elf_strptr(elf, symshdr->sh_link, sym.st_name);
+ return caller;
+ }
+ }
+
+ return NULL;
+}
+
+/* Get all functions that call to mcount. */
+static void get_mcount_callers(const char *elf_file)
+{
+ Elf *elf;
+ Elf_Scn *scn = NULL;
+ GElf_Shdr shdr;
+ int fd;
+ int ndx;
+
+ fd = open(elf_file, O_RDONLY);
+ if (fd < 0)
+ errx(1, "can not open %s", elf_file);
+
+ elf_version(EV_CURRENT);
+ elf = elf_begin(fd, ELF_C_READ, NULL);
+
+ check_elf(elf);
+
+ while ((scn = elf_nextscn(elf, scn)) != NULL) {
+ gelf_getshdr(scn, &shdr);
+
+ if (shdr.sh_type == SHT_REL || shdr.sh_type == SHT_RELA) {
+ Elf_Data *data = elf_getdata(scn, NULL);
+ Elf_Scn *symscn = elf_getscn(elf, shdr.sh_link);
+ Elf_Data *symdata = elf_getdata(symscn, NULL);
+ GElf_Shdr symshdr_mem;
+ GElf_Shdr *symshdr = gelf_getshdr(symscn, &symshdr_mem);
+
+ for (ndx = 0; ndx < shdr.sh_size / shdr.sh_entsize;
+ ++ndx) {
+ unsigned long sym_index;
+ unsigned long offset;
+ GElf_Sym sym;
+ char *symname;
+
+ if (shdr.sh_type == SHT_REL) {
+ GElf_Rel rel_mem;
+ GElf_Rel *rel = gelf_getrel(data, ndx,
+ &rel_mem);
+ sym_index = GELF_R_SYM(rel->r_info);
+ offset = rel->r_offset;
+ } else {
+ GElf_Rela rela_mem;
+ GElf_Rela *rela = gelf_getrela(
+ data, ndx, &rela_mem);
+ sym_index = GELF_R_SYM(rela->r_info);
+ offset = rela->r_offset;
+ }
+
+ gelf_getsym(symdata, sym_index, &sym);
+ symname = elf_strptr(elf, symshdr->sh_link,
+ sym.st_name);
+
+ if (is_mcount(symname)) {
+ const char *caller;
+
+ caller = search_mcount_caller(
+ elf, symshdr, symdata,
+ shdr.sh_info, offset);
+ if (caller)
+ func_prototype_list_add_new(caller);
+ }
+ }
+ }
+ }
+
+ elf_end(elf);
+ close(fd);
+}
+
+/*
+ * Get a variable size and sign info.
+ * TODO: Determine the expected display format. (e.g. size_t for "%lu").
+ */
+static void die_type_sign_bytes(Dwarf_Die *die, bool *is_signed, int *bytes)
+{
+ Dwarf_Attribute attr;
+ Dwarf_Die type;
+ int ret;
+
+ *bytes = 0;
+ *is_signed = false;
+
+ ret = dwarf_peel_type(dwarf_formref_die(
+ dwarf_attr_integrate(die, DW_AT_type, &attr), &type),
+ &type);
+ if (ret == 0) {
+ Dwarf_Word val;
+
+ ret = dwarf_formudata(dwarf_attr(&type, DW_AT_encoding,
+ &attr), &val);
+ if (ret == 0)
+ *is_signed = (val == DW_ATE_signed) ||
+ (val == DW_ATE_signed_char);
+
+ if (dwarf_aggregate_size(&type, &val) == 0)
+ *bytes = val;
+ }
+}
+
+static int get_func_nr_params(Dwarf_Die *funcdie)
+{
+ Dwarf_Die child;
+ int count = 0;
+
+ if (dwarf_child(funcdie, &child) == 0) {
+ do {
+ if (dwarf_tag(&child) == DW_TAG_formal_parameter)
+ count++;
+ } while (dwarf_siblingof(&child, &child) == 0);
+ }
+
+ return count;
+}
+
+static int get_loc_expr(const char *fname, Dwarf_Op *loc, uint8_t expr[2])
+{
+ int ret = 0;
+ int off;
+
+ switch (loc->atom) {
+ case DW_OP_reg0 ... DW_OP_reg31:
+ expr[0] = loc->atom;
+ expr[1] = 0;
+ break;
+ case DW_OP_fbreg:
+ off = (int32_t)loc->number;
+
+ /*
+ * Very few functions have number that exceeds
+ * (SCHAR_MIN, SCHAR_MAX). We skip these
+ * functions to keep protrotype data as small
+ * as possilbe.
+ */
+ if (off > SCHAR_MAX || off < SCHAR_MIN) {
+ warnx("%s: loc fbreg offset %d too large", fname, off);
+ ret = -1;
+ } else {
+ expr[0] = loc->atom;
+ expr[1] = off; /* The operand is signed */
+ }
+ break;
+ case DW_OP_breg0 ... DW_OP_breg31:
+ off = (int32_t)loc->number;
+
+ if (off > SCHAR_MAX || off < SCHAR_MIN) {
+ warnx("%s: loc bregx offset %d too large", fname, off);
+ ret = -1;
+ } else {
+ expr[0] = loc->atom;
+ expr[1] = off;
+ }
+ break;
+ default:
+ warnx("%s: unsupported loc operation 0x%x",
+ fname, loc->atom);
+ ret = -1;
+ };
+
+ return ret;
+}
+
+static int handle_function(Dwarf_Die *funcdie, void *arg)
+{
+ const char *name = dwarf_diename(funcdie);
+ Dwarf_Addr func_addr;
+ Dwarf_Die child;
+ struct func_prototype *proto;
+ int nr_params;
+ int sz, n = 0;
+
+ if (!dwarf_hasattr(funcdie, DW_AT_low_pc))
+ return 0;
+
+ /* Such symbol is a local function generated by GCC ipa-fnsplit. */
+ if (!dwarf_hasattr(funcdie, DW_AT_name))
+ return 0;
+
+ /* check whether it is a mcount caller. */
+ proto = func_prototype_list_search(name);
+ if (!proto)
+ return 0;
+
+ nr_params = get_func_nr_params(funcdie);
+ sz = sizeof(proto->params[0]) * nr_params;
+ proto->params = malloc(sz);
+ if (!proto->params)
+ errx(1, "no memory");
+
+ memset(proto->params, 0, sz);
+
+ dwarf_lowpc(funcdie, &func_addr);
+
+ /* get function return type */
+ if (dwarf_hasattr(funcdie, DW_AT_type)) {
+ bool is_signed;
+ int bytes;
+
+ die_type_sign_bytes(funcdie, &is_signed, &bytes);
+ proto->ret_type = MK_TYPE(is_signed, bytes);
+ } else
+ proto->ret_type = 0;
+
+ /* process function parameters. */
+ if (dwarf_child(funcdie, &child) == 0) {
+ do {
+ if (dwarf_tag(&child) == DW_TAG_formal_parameter) {
+ Dwarf_Attribute locattr;
+ Dwarf_Op *loc;
+ size_t nloc = 0;
+ bool is_signed;
+ int bytes;
+
+ die_type_sign_bytes(&child, &is_signed, &bytes);
+ proto->params[n].name = strdup(dwarf_diename(&child));
+ proto->params[n].type = MK_TYPE(is_signed, bytes);
+
+ if (!dwarf_hasattr(&child, DW_AT_location))
+ errx(1, "%s: no location attr", name);
+
+ dwarf_attr(&child, DW_AT_location, &locattr);
+ if (dwarf_getlocation(&locattr, &loc, &nloc) < 0) {
+ Dwarf_Addr base, begin, end;
+
+ if (dwarf_getlocations(
+ &locattr, 0, &base,
+ &begin, &end, &loc,
+ &nloc) <= 0)
+ errx(1, "%s: no param loc info",
+ name);
+ }
+ if (get_loc_expr(name, loc, proto->params[n].loc)) {
+ /* skip this function. */
+ proto->skip = true;
+ return 0;
+ }
+
+ n++;
+ };
+ } while (dwarf_siblingof(&child, &child) == 0);
+ }
+
+ proto->nr_param = n;
+ return 0;
+}
+
+static const Dwfl_Callbacks offline_callbacks = {
+ .find_debuginfo = dwfl_standard_find_debuginfo,
+ .section_address = dwfl_offline_section_address,
+};
+
+/* Iterate each DW_TAG_subprogram DIE to get their prototype info. */
+static void dwarf_get_prototypes(const char *elf_file)
+{
+ Dwfl *dwfl = NULL;
+ Dwarf_Die *cu = NULL;
+ Dwarf_Addr dwbias;
+ int ret;
+
+ dwfl = dwfl_begin(&offline_callbacks);
+ if (dwfl == NULL)
+ errx(1, "dwfl fail");
+
+ if (dwfl_report_offline(dwfl, "", elf_file, -1) == NULL)
+ errx(1, "dwfl report fail");
+
+ ret = dwfl_report_end(dwfl, NULL, NULL);
+ assert(ret == 0);
+
+ while ((cu = dwfl_nextcu(dwfl, cu, &dwbias)) != NULL)
+ dwarf_getfuncs(cu, &handle_function, NULL, 0);
+}
+
+static void print_prototypes_assembly(void)
+{
+ struct func_prototype *proto;
+ int i;
+
+ if (!func_prototype_list)
+ return;
+
+ printf(" .section __funcprotostr, \"a\"\n");
+ for (proto = func_prototype_list; proto != NULL; proto = proto->next) {
+ if (proto->skip)
+ continue;
+ for (i = 0; i < proto->nr_param; i++) {
+ printf(".P_%s_%d:\n", proto->name, i);
+ printf(" .string \"%s\"\n", proto->params[i].name);
+ }
+ };
+
+ printf("\n .section __funcproto, \"a\"\n");
+ for (proto = func_prototype_list; proto != NULL; proto = proto->next) {
+ if (proto->skip)
+ continue;
+ if (is_64bit_obj)
+ printf(" .quad %s\n", proto->name);
+ else
+ printf(" .long %s\n", proto->name);
+ printf(" .byte 0x%x\n", proto->ret_type);
+ printf(" .byte 0x%x\n", proto->nr_param);
+ for (i = 0; i < proto->nr_param; i++) {
+ if (is_64bit_obj)
+ printf(" .quad .P_%s_%d\n", proto->name, i);
+ else
+ printf(" .long .P_%s_%d\n", proto->name, i);
+ printf(" .byte 0x%x\n", proto->params[i].type);
+ printf(" .byte 0x%x\n", proto->params[i].loc[0]);
+ printf(" .byte 0x%x\n", proto->params[i].loc[1]);
+ }
+ printf("\n");
+ };
+}
+
+/* Program documentation. */
+static char doc[] =
+ "funcprototype -- a program to generate mcount caller prototypes";
+
+/* A description of the arguments we accept. */
+static const char args_doc[] = "elf-file";
+
+/* The options we understand. */
+static struct argp_option options[] = { { "mcount-callers", 'm', 0, 0,
+ "show mcount callers only" },
+ { 0 } };
+
+struct arguments {
+ char *elf_file;
+ int show_callers_only;
+};
+
+/* Parse options. */
+static error_t parse_opt(int key, char *arg, struct argp_state *state)
+{
+ struct arguments *arguments = state->input;
+
+ switch (key) {
+ case 'm':
+ arguments->show_callers_only = 1;
+ break;
+ case ARGP_KEY_ARG:
+ if (state->arg_num > 2) {
+ /* Too many arguments. */
+ argp_usage(state);
+ }
+ arguments->elf_file = arg;
+ break;
+ case ARGP_KEY_END:
+ if (state->arg_num < 1)
+ /* Not enough arguments. */
+ argp_usage(state);
+ break;
+ default:
+ return ARGP_ERR_UNKNOWN;
+ }
+ return 0;
+}
+
+/* Our argp parser. */
+static struct argp argp = { options, parse_opt, args_doc, doc };
+
+int main(int argc, char *argv[])
+{
+ struct arguments arguments;
+
+ arguments.show_callers_only = 0;
+ argp_parse(&argp, argc, argv, 0, 0, &arguments);
+
+ get_mcount_callers(arguments.elf_file);
+
+ if (arguments.show_callers_only) {
+ func_prototype_list_dumpnames();
+ goto free;
+ }
+
+ dwarf_get_prototypes(arguments.elf_file);
+ print_prototypes_assembly();
+
+free:
+ func_prototype_list_destroy();
+ return 0;
+}
--
2.20.1
Add generic DWARF constant definitions. We will use it later.
Signed-off-by: Changbin Du <[email protected]>
---
include/asm-generic/dwarf.h | 199 ++++++++++++++++++++++++++++++++++++
1 file changed, 199 insertions(+)
create mode 100644 include/asm-generic/dwarf.h
diff --git a/include/asm-generic/dwarf.h b/include/asm-generic/dwarf.h
new file mode 100644
index 000000000000..c705633c2a8f
--- /dev/null
+++ b/include/asm-generic/dwarf.h
@@ -0,0 +1,199 @@
+/* SPDX-License-Identifier: GPL-2.0
+ *
+ * Architecture independent definitions of DWARF.
+ *
+ * Copyright (C) 2019 Changbin Du <[email protected]>
+ */
+#ifndef __ASM_GENERIC_DWARF_H
+#define __ASM_GENERIC_DWARF_H
+
+/*
+ * DWARF expression operations
+ */
+#define DW_OP_addr 0x03
+#define DW_OP_deref 0x06
+#define DW_OP_const1u 0x08
+#define DW_OP_const1s 0x09
+#define DW_OP_const2u 0x0a
+#define DW_OP_const2s 0x0b
+#define DW_OP_const4u 0x0c
+#define DW_OP_const4s 0x0d
+#define DW_OP_const8u 0x0e
+#define DW_OP_const8s 0x0f
+#define DW_OP_constu 0x10
+#define DW_OP_consts 0x11
+#define DW_OP_dup 0x12
+#define DW_OP_drop 0x13
+#define DW_OP_over 0x14
+#define DW_OP_pick 0x15
+#define DW_OP_swap 0x16
+#define DW_OP_rot 0x17
+#define DW_OP_xderef 0x18
+#define DW_OP_abs 0x19
+#define DW_OP_and 0x1a
+#define DW_OP_div 0x1b
+#define DW_OP_minus 0x1c
+#define DW_OP_mod 0x1d
+#define DW_OP_mul 0x1e
+#define DW_OP_neg 0x1f
+#define DW_OP_not 0x20
+#define DW_OP_or 0x21
+#define DW_OP_plus 0x22
+#define DW_OP_plus_uconst 0x23
+#define DW_OP_shl 0x24
+#define DW_OP_shr 0x25
+#define DW_OP_shra 0x26
+#define DW_OP_xor 0x27
+#define DW_OP_skip 0x2f
+#define DW_OP_bra 0x28
+#define DW_OP_eq 0x29
+#define DW_OP_ge 0x2a
+#define DW_OP_gt 0x2b
+#define DW_OP_le 0x2c
+#define DW_OP_lt 0x2d
+#define DW_OP_ne 0x2e
+#define DW_OP_lit0 0x30
+#define DW_OP_lit1 0x31
+#define DW_OP_lit2 0x32
+#define DW_OP_lit3 0x33
+#define DW_OP_lit4 0x34
+#define DW_OP_lit5 0x35
+#define DW_OP_lit6 0x36
+#define DW_OP_lit7 0x37
+#define DW_OP_lit8 0x38
+#define DW_OP_lit9 0x39
+#define DW_OP_lit10 0x3a
+#define DW_OP_lit11 0x3b
+#define DW_OP_lit12 0x3c
+#define DW_OP_lit13 0x3d
+#define DW_OP_lit14 0x3e
+#define DW_OP_lit15 0x3f
+#define DW_OP_lit16 0x40
+#define DW_OP_lit17 0x41
+#define DW_OP_lit18 0x42
+#define DW_OP_lit19 0x43
+#define DW_OP_lit20 0x44
+#define DW_OP_lit21 0x45
+#define DW_OP_lit22 0x46
+#define DW_OP_lit23 0x47
+#define DW_OP_lit24 0x48
+#define DW_OP_lit25 0x49
+#define DW_OP_lit26 0x4a
+#define DW_OP_lit27 0x4b
+#define DW_OP_lit28 0x4c
+#define DW_OP_lit29 0x4d
+#define DW_OP_lit30 0x4e
+#define DW_OP_lit31 0x4f
+#define DW_OP_reg0 0x50
+#define DW_OP_reg1 0x51
+#define DW_OP_reg2 0x52
+#define DW_OP_reg3 0x53
+#define DW_OP_reg4 0x54
+#define DW_OP_reg5 0x55
+#define DW_OP_reg6 0x56
+#define DW_OP_reg7 0x57
+#define DW_OP_reg8 0x58
+#define DW_OP_reg9 0x59
+#define DW_OP_reg10 0x5a
+#define DW_OP_reg11 0x5b
+#define DW_OP_reg12 0x5c
+#define DW_OP_reg13 0x5d
+#define DW_OP_reg14 0x5e
+#define DW_OP_reg15 0x5f
+#define DW_OP_reg16 0x60
+#define DW_OP_reg17 0x61
+#define DW_OP_reg18 0x62
+#define DW_OP_reg19 0x63
+#define DW_OP_reg20 0x64
+#define DW_OP_reg21 0x65
+#define DW_OP_reg22 0x66
+#define DW_OP_reg23 0x67
+#define DW_OP_reg24 0x68
+#define DW_OP_reg25 0x69
+#define DW_OP_reg26 0x6a
+#define DW_OP_reg27 0x6b
+#define DW_OP_reg28 0x6c
+#define DW_OP_reg29 0x6d
+#define DW_OP_reg30 0x6e
+#define DW_OP_reg31 0x6f
+#define DW_OP_breg0 0x70
+#define DW_OP_breg1 0x71
+#define DW_OP_breg2 0x72
+#define DW_OP_breg3 0x73
+#define DW_OP_breg4 0x74
+#define DW_OP_breg5 0x75
+#define DW_OP_breg6 0x76
+#define DW_OP_breg7 0x77
+#define DW_OP_breg8 0x78
+#define DW_OP_breg9 0x79
+#define DW_OP_breg10 0x7a
+#define DW_OP_breg11 0x7b
+#define DW_OP_breg12 0x7c
+#define DW_OP_breg13 0x7d
+#define DW_OP_breg14 0x7e
+#define DW_OP_breg15 0x7f
+#define DW_OP_breg16 0x80
+#define DW_OP_breg17 0x81
+#define DW_OP_breg18 0x82
+#define DW_OP_breg19 0x83
+#define DW_OP_breg20 0x84
+#define DW_OP_breg21 0x85
+#define DW_OP_breg22 0x86
+#define DW_OP_breg23 0x87
+#define DW_OP_breg24 0x88
+#define DW_OP_breg25 0x89
+#define DW_OP_breg26 0x8a
+#define DW_OP_breg27 0x8b
+#define DW_OP_breg28 0x8c
+#define DW_OP_breg29 0x8d
+#define DW_OP_breg30 0x8e
+#define DW_OP_breg31 0x8f
+#define DW_OP_regx 0x90
+#define DW_OP_fbreg 0x91
+#define DW_OP_bregx 0x92
+#define DW_OP_piece 0x93
+#define DW_OP_deref_size 0x94
+#define DW_OP_xderef_size 0x95
+#define DW_OP_nop 0x96
+#define DW_OP_push_object_address 0x97
+#define DW_OP_call2 0x98
+#define DW_OP_call4 0x99
+#define DW_OP_call_ref 0x9a
+#define DW_OP_form_tls_address 0x9b
+#define DW_OP_call_frame_cfa 0x9c
+#define DW_OP_bit_piece 0x9d
+#define DW_OP_implicit_value 0x9e
+#define DW_OP_stack_value 0x9f
+#define DW_OP_implicit_pointer 0xa0
+#define DW_OP_addrx 0xa1
+#define DW_OP_constx 0xa2
+#define DW_OP_entry_value 0xa3
+#define DW_OP_const_type 0xa4
+#define DW_OP_regval_type 0xa5
+#define DW_OP_deref_type 0xa6
+#define DW_OP_xderef_type 0xa7
+#define DW_OP_convert 0xa8
+#define DW_OP_reinterpret 0xa9
+
+/* GNU extensions. */
+#define DW_OP_GNU_push_tls_address 0xe0
+#define DW_OP_GNU_uninit 0xf0
+#define DW_OP_GNU_encoded_addr 0xf1
+#define DW_OP_GNU_implicit_pointer 0xf2
+#define DW_OP_GNU_entry_value 0xf3
+#define DW_OP_GNU_const_type 0xf4
+#define DW_OP_GNU_regval_type 0xf5
+#define DW_OP_GNU_deref_type 0xf6
+#define DW_OP_GNU_convert 0xf7
+#define DW_OP_GNU_reinterpret 0xf9
+#define DW_OP_GNU_parameter_ref 0xfa
+
+/* GNU Debug Fission extensions. */
+#define DW_OP_GNU_addr_index 0xfb,
+#define DW_OP_GNU_const_index 0xfc
+#define DW_OP_GNU_variable_value 0xfd
+
+#define DW_OP_lo_user 0xe0
+#define DW_OP_hi_user 0xff
+
+#endif /* __ASM_GENERIC_DWARF_H */
--
2.20.1
We will reuse ftrace_hash to lookup function prototype information. So
we need an additional field to bind ftrace_func_entry to prototype
information.
Signed-off-by: Changbin Du <[email protected]>
---
kernel/trace/ftrace.c | 17 +++++++----------
kernel/trace/trace.h | 6 ++++++
2 files changed, 13 insertions(+), 10 deletions(-)
diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
index eca34503f178..a314f0768b2c 100644
--- a/kernel/trace/ftrace.c
+++ b/kernel/trace/ftrace.c
@@ -1017,11 +1017,6 @@ static bool update_all_ops;
# error Dynamic ftrace depends on MCOUNT_RECORD
#endif
-struct ftrace_func_entry {
- struct hlist_node hlist;
- unsigned long ip;
-};
-
struct ftrace_func_probe {
struct ftrace_probe_ops *probe_ops;
struct ftrace_ops ops;
@@ -1169,7 +1164,8 @@ static void __add_hash_entry(struct ftrace_hash *hash,
hash->count++;
}
-static int add_hash_entry(struct ftrace_hash *hash, unsigned long ip)
+static int add_hash_entry(struct ftrace_hash *hash, unsigned long ip,
+ void *priv)
{
struct ftrace_func_entry *entry;
@@ -1178,6 +1174,7 @@ static int add_hash_entry(struct ftrace_hash *hash, unsigned long ip)
return -ENOMEM;
entry->ip = ip;
+ entry->priv = priv;
__add_hash_entry(hash, entry);
return 0;
@@ -1346,7 +1343,7 @@ alloc_and_copy_ftrace_hash(int size_bits, struct ftrace_hash *hash)
size = 1 << hash->size_bits;
for (i = 0; i < size; i++) {
hlist_for_each_entry(entry, &hash->buckets[i], hlist) {
- ret = add_hash_entry(new_hash, entry->ip);
+ ret = add_hash_entry(new_hash, entry->ip, NULL);
if (ret < 0)
goto free_hash;
}
@@ -3694,7 +3691,7 @@ enter_record(struct ftrace_hash *hash, struct dyn_ftrace *rec, int clear_filter)
if (entry)
return 0;
- ret = add_hash_entry(hash, rec->ip);
+ ret = add_hash_entry(hash, rec->ip, NULL);
}
return ret;
}
@@ -4700,7 +4697,7 @@ ftrace_match_addr(struct ftrace_hash *hash, unsigned long ip, int remove)
return 0;
}
- return add_hash_entry(hash, ip);
+ return add_hash_entry(hash, ip, NULL);
}
static int
@@ -5380,7 +5377,7 @@ ftrace_graph_set_hash(struct ftrace_hash *hash, char *buffer)
if (entry)
continue;
- if (add_hash_entry(hash, rec->ip) < 0)
+ if (add_hash_entry(hash, rec->ip, NULL) < 0)
goto out;
} else {
if (entry) {
diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h
index 005f08629b8b..ad619c73a505 100644
--- a/kernel/trace/trace.h
+++ b/kernel/trace/trace.h
@@ -882,6 +882,12 @@ struct ftrace_hash {
struct rcu_head rcu;
};
+struct ftrace_func_entry {
+ struct hlist_node hlist;
+ unsigned long ip;
+ void *priv;
+};
+
struct ftrace_func_entry *
ftrace_lookup_ip(struct ftrace_hash *hash, unsigned long ip);
--
2.20.1
When CONFIG_FTRACE_FUNC_PROTOTYPE is enabled, thousands of
ftrace_func_entry instances are created. So create a dedicated
memcache to enhance performance.
Signed-off-by: Changbin Du <[email protected]>
---
kernel/trace/ftrace.c | 17 ++++++++++++++++-
1 file changed, 16 insertions(+), 1 deletion(-)
diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
index a314f0768b2c..cfcb8dad93ea 100644
--- a/kernel/trace/ftrace.c
+++ b/kernel/trace/ftrace.c
@@ -94,6 +94,8 @@ struct ftrace_ops *function_trace_op __read_mostly = &ftrace_list_end;
/* What to set function_trace_op to */
static struct ftrace_ops *set_function_trace_op;
+struct kmem_cache *hash_entry_cache;
+
static bool ftrace_pids_enabled(struct ftrace_ops *ops)
{
struct trace_array *tr;
@@ -1169,7 +1171,7 @@ static int add_hash_entry(struct ftrace_hash *hash, unsigned long ip,
{
struct ftrace_func_entry *entry;
- entry = kmalloc(sizeof(*entry), GFP_KERNEL);
+ entry = kmem_cache_alloc(hash_entry_cache, GFP_KERNEL);
if (!entry)
return -ENOMEM;
@@ -6153,6 +6155,15 @@ void __init ftrace_init(void)
if (ret)
goto failed;
+ hash_entry_cache = kmem_cache_create("ftrace-hash",
+ sizeof(struct ftrace_func_entry),
+ sizeof(struct ftrace_func_entry),
+ 0, NULL);
+ if (!hash_entry_cache) {
+ pr_err("failed to create ftrace hash entry cache\n");
+ goto failed;
+ }
+
count = __stop_mcount_loc - __start_mcount_loc;
if (!count) {
pr_info("ftrace: No functions to be traced?\n");
@@ -6172,6 +6183,10 @@ void __init ftrace_init(void)
return;
failed:
+ if (hash_entry_cache) {
+ kmem_cache_destroy(hash_entry_cache);
+ hash_entry_cache = NULL;
+ }
ftrace_disabled = 1;
}
--
2.20.1
Walk through the '__funcproto' section in vmlinux and kernel modules.
For each item we add it to a new ftrace hash table ftrace_prototype_hash.
When unloading a module, its items are removed from hash table.
Signed-off-by: Changbin Du <[email protected]>
---
include/asm-generic/vmlinux.lds.h | 18 ++++++++
include/linux/ftrace.h | 18 ++++++++
include/linux/module.h | 4 ++
kernel/module.c | 25 ++++++++--
kernel/trace/ftrace.c | 76 ++++++++++++++++++++++++++++++-
kernel/trace/trace.h | 4 ++
6 files changed, 140 insertions(+), 5 deletions(-)
diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
index cd28f63bfbc7..3b0a10cbf0ca 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -125,6 +125,23 @@
#define MCOUNT_REC()
#endif
+#ifdef CONFIG_FTRACE_FUNC_PROTOTYPE
+#define FUNC_PROTOTYPE \
+ . = ALIGN(8); \
+ __funcprotostr : AT(ADDR(__funcprotostr) - LOAD_OFFSET) { \
+ KEEP(*(__funcprotostr)) \
+ } \
+ \
+ . = ALIGN(8); \
+ __funcproto : AT(ADDR(__funcproto) - LOAD_OFFSET) { \
+ __start_funcproto = .; \
+ KEEP(*(__funcproto)) \
+ __stop_funcproto = .; \
+ }
+#else
+#define FUNC_PROTOTYPE
+#endif
+
#ifdef CONFIG_TRACE_BRANCH_PROFILING
#define LIKELY_PROFILE() __start_annotated_branch_profile = .; \
KEEP(*(_ftrace_annotated_branch)) \
@@ -396,6 +413,7 @@
} \
\
TRACEDATA \
+ FUNC_PROTOTYPE \
\
/* Kernel symbol table: Normal symbols */ \
__ksymtab : AT(ADDR(__ksymtab) - LOAD_OFFSET) { \
diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h
index 8a8cb3c401b2..f5aab37a8c34 100644
--- a/include/linux/ftrace.h
+++ b/include/linux/ftrace.h
@@ -361,6 +361,24 @@ struct dyn_ftrace {
struct dyn_arch_ftrace arch;
};
+#ifdef CONFIG_FTRACE_FUNC_PROTOTYPE
+struct func_param {
+ char *name;
+ uint8_t type;
+ uint8_t loc[2];
+} __packed;
+
+struct func_prototype {
+ unsigned long ip;
+ uint8_t ret_type;
+ uint8_t nr_param;
+ struct func_param params[0];
+} __packed;
+
+#define FTRACE_PROTOTYPE_SIGNED(t) (t & BIT(7))
+#define FTRACE_PROTOTYPE_SIZE(t) (t & GENMASK(6, 0))
+#endif
+
int ftrace_force_update(void);
int ftrace_set_filter_ip(struct ftrace_ops *ops, unsigned long ip,
int remove, int reset);
diff --git a/include/linux/module.h b/include/linux/module.h
index 1455812dd325..516062dfe567 100644
--- a/include/linux/module.h
+++ b/include/linux/module.h
@@ -477,6 +477,10 @@ struct module {
unsigned int num_ftrace_callsites;
unsigned long *ftrace_callsites;
#endif
+#ifdef CONFIG_FTRACE_FUNC_PROTOTYPE
+ struct func_prototype *funcproto_start;
+ size_t funcproto_sec_size;
+#endif
#ifdef CONFIG_LIVEPATCH
bool klp; /* Is this a livepatch module? */
diff --git a/kernel/module.c b/kernel/module.c
index 9ee93421269c..1c5eea7b6a28 100644
--- a/kernel/module.c
+++ b/kernel/module.c
@@ -360,17 +360,30 @@ static void *section_addr(const struct load_info *info, const char *name)
return (void *)info->sechdrs[find_sec(info, name)].sh_addr;
}
+/* Get info of a module section. */
+static void *section_info(const struct load_info *info,
+ const char *name,
+ size_t *size)
+{
+ unsigned int sec = find_sec(info, name);
+
+ /* Section 0 has sh_addr 0 and sh_size 0. */
+ *size = info->sechdrs[sec].sh_size;
+ return (void *)info->sechdrs[sec].sh_addr;
+}
+
/* Find a module section, or NULL. Fill in number of "objects" in section. */
static void *section_objs(const struct load_info *info,
const char *name,
size_t object_size,
unsigned int *num)
{
- unsigned int sec = find_sec(info, name);
+ void *addr;
+ size_t sz;
- /* Section 0 has sh_addr 0 and sh_size 0. */
- *num = info->sechdrs[sec].sh_size / object_size;
- return (void *)info->sechdrs[sec].sh_addr;
+ addr = section_info(info, name, &sz);
+ *num = sz / object_size;
+ return addr;
}
/* Provided by the linker */
@@ -3140,6 +3153,10 @@ static int find_module_sections(struct module *mod, struct load_info *info)
sizeof(*mod->ftrace_callsites),
&mod->num_ftrace_callsites);
#endif
+#ifdef CONFIG_FTRACE_FUNC_PROTOTYPE
+ mod->funcproto_start = section_info(info, "__funcproto",
+ &mod->funcproto_sec_size);
+#endif
#ifdef CONFIG_FUNCTION_ERROR_INJECTION
mod->ei_funcs = section_objs(info, "_error_injection_whitelist",
sizeof(*mod->ei_funcs),
diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
index cfcb8dad93ea..438b8b47198f 100644
--- a/kernel/trace/ftrace.c
+++ b/kernel/trace/ftrace.c
@@ -5060,6 +5060,9 @@ static DEFINE_MUTEX(graph_lock);
struct ftrace_hash *ftrace_graph_hash = EMPTY_HASH;
struct ftrace_hash *ftrace_graph_notrace_hash = EMPTY_HASH;
+#ifdef CONFIG_FTRACE_FUNC_PROTOTYPE
+struct ftrace_hash *ftrace_prototype_hash = EMPTY_HASH;
+#endif
enum graph_filter_type {
GRAPH_FILTER_NOTRACE = 0,
@@ -5615,6 +5618,46 @@ static int ftrace_process_locs(struct module *mod,
return ret;
}
+#ifdef CONFIG_FTRACE_FUNC_PROTOTYPE
+static int ftrace_process_funcproto(struct module *mod,
+ struct func_prototype *start,
+ struct func_prototype *end,
+ bool remove)
+{
+ struct ftrace_func_entry *ent;
+ struct func_prototype *proto;
+ int ret = 0;
+
+ mutex_lock(&ftrace_lock);
+
+restart:
+ proto = start;
+ while (proto < end) {
+ if (remove) {
+ ent = ftrace_lookup_ip(ftrace_prototype_hash,
+ proto->ip);
+ if (ent)
+ free_hash_entry(ftrace_prototype_hash, ent);
+ } else {
+ ret = add_hash_entry(ftrace_prototype_hash,
+ proto->ip, proto);
+ if (ret < 0) {
+ end = proto;
+ remove = 1;
+ goto restart;
+ }
+ }
+ proto = (struct func_prototype *)((char *)proto +
+ sizeof(*proto) +
+ sizeof(proto->params[0]) * proto->nr_param);
+ }
+
+ mutex_unlock(&ftrace_lock);
+
+ return ret;
+}
+#endif
+
struct ftrace_mod_func {
struct list_head list;
char *name;
@@ -5707,7 +5750,7 @@ static void ftrace_free_mod_map(struct rcu_head *rcu)
kfree(mod_map);
}
-void ftrace_release_mod(struct module *mod)
+void ftrace_release_dyn(struct module *mod)
{
struct ftrace_mod_map *mod_map;
struct ftrace_mod_map *n;
@@ -5773,6 +5816,17 @@ void ftrace_release_mod(struct module *mod)
}
}
+void ftrace_release_mod(struct module *mod)
+{
+ ftrace_release_dyn(mod);
+
+#ifdef CONFIG_FTRACE_FUNC_PROTOTYPE
+ ftrace_process_funcproto(mod, mod->funcproto_start,
+ (void *)mod->funcproto_start + mod->funcproto_sec_size,
+ true);
+#endif
+}
+
void ftrace_module_enable(struct module *mod)
{
struct dyn_ftrace *rec;
@@ -5852,6 +5906,11 @@ void ftrace_module_init(struct module *mod)
ftrace_process_locs(mod, mod->ftrace_callsites,
mod->ftrace_callsites + mod->num_ftrace_callsites);
+#ifdef CONFIG_FTRACE_FUNC_PROTOTYPE
+ ftrace_process_funcproto(mod, mod->funcproto_start,
+ (void *)mod->funcproto_start + mod->funcproto_sec_size,
+ false);
+#endif
}
static void save_ftrace_mod_rec(struct ftrace_mod_map *mod_map,
@@ -6146,6 +6205,10 @@ void __init ftrace_init(void)
{
extern unsigned long __start_mcount_loc[];
extern unsigned long __stop_mcount_loc[];
+#ifdef CONFIG_FTRACE_FUNC_PROTOTYPE
+ extern struct func_prototype __start_funcproto[];
+ extern struct func_prototype __stop_funcproto[];
+#endif
unsigned long count, flags;
int ret;
@@ -6179,6 +6242,17 @@ void __init ftrace_init(void)
__start_mcount_loc,
__stop_mcount_loc);
+#ifdef CONFIG_FTRACE_FUNC_PROTOTYPE
+ ftrace_prototype_hash = alloc_ftrace_hash(FTRACE_HASH_DEFAULT_BITS);
+ if (WARN_ON(!ftrace_prototype_hash))
+ goto failed;
+
+ ftrace_process_funcproto(NULL,
+ __start_funcproto,
+ __stop_funcproto,
+ false);
+#endif
+
set_ftrace_early_filters();
return;
diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h
index ad619c73a505..22433a15e340 100644
--- a/kernel/trace/trace.h
+++ b/kernel/trace/trace.h
@@ -940,6 +940,10 @@ extern void __trace_graph_return(struct trace_array *tr,
extern struct ftrace_hash *ftrace_graph_hash;
extern struct ftrace_hash *ftrace_graph_notrace_hash;
+#ifdef CONFIG_FTRACE_FUNC_PROTOTYPE
+extern struct ftrace_hash *ftrace_prototype_hash;
+#endif
+
static inline int ftrace_graph_addr(struct ftrace_graph_ent *trace)
{
unsigned long addr = trace->func;
--
2.20.1
To record function parameter and return value, we need the arch specific
code to pass the saved register context. It is only valid if the
CONFIG_FTRACE_FUNC_PROTOTYPE feature is enabled. This patch only changes
the interfaces, real implementation will be added later.
Signed-off-by: Changbin Du <[email protected]>
---
arch/arm/kernel/ftrace.c | 2 +-
arch/arm64/kernel/ftrace.c | 2 +-
arch/csky/kernel/ftrace.c | 2 +-
arch/microblaze/kernel/ftrace.c | 2 +-
arch/mips/kernel/ftrace.c | 2 +-
arch/nds32/kernel/ftrace.c | 5 +++--
arch/parisc/kernel/ftrace.c | 2 +-
arch/powerpc/kernel/trace/ftrace.c | 2 +-
arch/riscv/kernel/ftrace.c | 2 +-
arch/s390/kernel/ftrace.c | 2 +-
arch/sh/kernel/ftrace.c | 2 +-
arch/sparc/kernel/ftrace.c | 2 +-
arch/x86/kernel/ftrace.c | 2 +-
include/linux/ftrace.h | 10 +++++++---
kernel/trace/fgraph.c | 21 +++++++++++++++------
kernel/trace/ftrace.c | 4 +++-
kernel/trace/trace.h | 2 +-
kernel/trace/trace_functions_graph.c | 2 +-
kernel/trace/trace_irqsoff.c | 3 ++-
kernel/trace/trace_sched_wakeup.c | 3 ++-
20 files changed, 46 insertions(+), 28 deletions(-)
diff --git a/arch/arm/kernel/ftrace.c b/arch/arm/kernel/ftrace.c
index bda949fd84e8..fd01c08b2dcb 100644
--- a/arch/arm/kernel/ftrace.c
+++ b/arch/arm/kernel/ftrace.c
@@ -191,7 +191,7 @@ void prepare_ftrace_return(unsigned long *parent, unsigned long self_addr,
old = *parent;
*parent = return_hooker;
- if (function_graph_enter(old, self_addr, frame_pointer, NULL))
+ if (function_graph_enter(old, self_addr, frame_pointer, NULL, NULL))
*parent = old;
}
diff --git a/arch/arm64/kernel/ftrace.c b/arch/arm64/kernel/ftrace.c
index 171773257974..dc8cc516c00a 100644
--- a/arch/arm64/kernel/ftrace.c
+++ b/arch/arm64/kernel/ftrace.c
@@ -233,7 +233,7 @@ void prepare_ftrace_return(unsigned long self_addr, unsigned long *parent,
*/
old = *parent;
- if (!function_graph_enter(old, self_addr, frame_pointer, NULL))
+ if (!function_graph_enter(old, self_addr, frame_pointer, NULL, NULL))
*parent = return_hooker;
}
diff --git a/arch/csky/kernel/ftrace.c b/arch/csky/kernel/ftrace.c
index 44f4880179b7..5bc67f447e78 100644
--- a/arch/csky/kernel/ftrace.c
+++ b/arch/csky/kernel/ftrace.c
@@ -148,7 +148,7 @@ void prepare_ftrace_return(unsigned long *parent, unsigned long self_addr,
old = *parent;
if (!function_graph_enter(old, self_addr,
- *(unsigned long *)frame_pointer, parent)) {
+ *(unsigned long *)frame_pointer, parent, NULL)) {
/*
* For csky-gcc function has sub-call:
* subi sp, sp, 8
diff --git a/arch/microblaze/kernel/ftrace.c b/arch/microblaze/kernel/ftrace.c
index 224eea40e1ee..9722e98cd01d 100644
--- a/arch/microblaze/kernel/ftrace.c
+++ b/arch/microblaze/kernel/ftrace.c
@@ -62,7 +62,7 @@ void prepare_ftrace_return(unsigned long *parent, unsigned long self_addr)
return;
}
- if (function_graph_enter(old, self_addr, 0, NULL))
+ if (function_graph_enter(old, self_addr, 0, NULL, NULL))
*parent = old;
}
#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
diff --git a/arch/mips/kernel/ftrace.c b/arch/mips/kernel/ftrace.c
index 2625232bfe52..24668bf079d2 100644
--- a/arch/mips/kernel/ftrace.c
+++ b/arch/mips/kernel/ftrace.c
@@ -378,7 +378,7 @@ void prepare_ftrace_return(unsigned long *parent_ra_addr, unsigned long self_ra,
insns = core_kernel_text(self_ra) ? 2 : MCOUNT_OFFSET_INSNS + 1;
self_ra -= (MCOUNT_INSN_SIZE * insns);
- if (function_graph_enter(old_parent_ra, self_ra, fp, NULL))
+ if (function_graph_enter(old_parent_ra, self_ra, fp, NULL, NULL))
*parent_ra_addr = old_parent_ra;
return;
out:
diff --git a/arch/nds32/kernel/ftrace.c b/arch/nds32/kernel/ftrace.c
index fd2a54b8cd57..3dbf0017dfdf 100644
--- a/arch/nds32/kernel/ftrace.c
+++ b/arch/nds32/kernel/ftrace.c
@@ -217,7 +217,7 @@ void prepare_ftrace_return(unsigned long *parent, unsigned long self_addr,
old = *parent;
- if (!function_graph_enter(old, self_addr, frame_pointer, NULL))
+ if (!function_graph_enter(old, self_addr, frame_pointer, NULL, NULL))
*parent = return_hooker;
}
@@ -235,7 +235,8 @@ noinline void ftrace_graph_caller(void)
prepare_ftrace_return(parent_ip, selfpc, frame_pointer);
}
-extern unsigned long ftrace_return_to_handler(unsigned long frame_pointer);
+extern unsigned long ftrace_return_to_handler(unsigned long frame_pointer,
+ unsigned long retval);
void __naked return_to_handler(void)
{
__asm__ __volatile__ (
diff --git a/arch/parisc/kernel/ftrace.c b/arch/parisc/kernel/ftrace.c
index b6fb30f2e4bf..ea02f36e4f84 100644
--- a/arch/parisc/kernel/ftrace.c
+++ b/arch/parisc/kernel/ftrace.c
@@ -40,7 +40,7 @@ static void __hot prepare_ftrace_return(unsigned long *parent,
old = *parent;
- if (!function_graph_enter(old, self_addr, 0, NULL))
+ if (!function_graph_enter(old, self_addr, 0, NULL, NULL))
/* activate parisc_return_to_handler() as return point */
*parent = (unsigned long) &parisc_return_to_handler;
}
diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c
index be1ca98fce5c..78174bbb257e 100644
--- a/arch/powerpc/kernel/trace/ftrace.c
+++ b/arch/powerpc/kernel/trace/ftrace.c
@@ -956,7 +956,7 @@ unsigned long prepare_ftrace_return(unsigned long parent, unsigned long ip)
return_hooker = ppc_function_entry(return_to_handler);
- if (!function_graph_enter(parent, ip, 0, NULL))
+ if (!function_graph_enter(parent, ip, 0, NULL, NULL))
parent = return_hooker;
out:
return parent;
diff --git a/arch/riscv/kernel/ftrace.c b/arch/riscv/kernel/ftrace.c
index b94d8db5ddcc..18f836727950 100644
--- a/arch/riscv/kernel/ftrace.c
+++ b/arch/riscv/kernel/ftrace.c
@@ -142,7 +142,7 @@ void prepare_ftrace_return(unsigned long *parent, unsigned long self_addr,
*/
old = *parent;
- if (function_graph_enter(old, self_addr, frame_pointer, parent))
+ if (function_graph_enter(old, self_addr, frame_pointer, parent, NULL))
*parent = return_hooker;
}
diff --git a/arch/s390/kernel/ftrace.c b/arch/s390/kernel/ftrace.c
index 1bb85f60c0dd..5021a23c5089 100644
--- a/arch/s390/kernel/ftrace.c
+++ b/arch/s390/kernel/ftrace.c
@@ -209,7 +209,7 @@ unsigned long prepare_ftrace_return(unsigned long ra, unsigned long sp,
if (unlikely(atomic_read(¤t->tracing_graph_pause)))
goto out;
ip -= MCOUNT_INSN_SIZE;
- if (!function_graph_enter(ra, ip, 0, (void *) sp))
+ if (!function_graph_enter(ra, ip, 0, (void *) sp), NULL)
ra = (unsigned long) return_to_handler;
out:
return ra;
diff --git a/arch/sh/kernel/ftrace.c b/arch/sh/kernel/ftrace.c
index 1b04270e5460..3a8271993e9c 100644
--- a/arch/sh/kernel/ftrace.c
+++ b/arch/sh/kernel/ftrace.c
@@ -364,7 +364,7 @@ void prepare_ftrace_return(unsigned long *parent, unsigned long self_addr)
return;
}
- if (function_graph_enter(old, self_addr, 0, NULL))
+ if (function_graph_enter(old, self_addr, 0, NULL, NULL))
__raw_writel(old, parent);
}
#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
diff --git a/arch/sparc/kernel/ftrace.c b/arch/sparc/kernel/ftrace.c
index 684b84ce397f..2783185719ba 100644
--- a/arch/sparc/kernel/ftrace.c
+++ b/arch/sparc/kernel/ftrace.c
@@ -130,7 +130,7 @@ unsigned long prepare_ftrace_return(unsigned long parent,
if (unlikely(atomic_read(¤t->tracing_graph_pause)))
return parent + 8UL;
- if (function_graph_enter(parent, self_addr, frame_pointer, NULL))
+ if (function_graph_enter(parent, self_addr, frame_pointer, NULL, NULL))
return parent + 8UL;
return return_hooker;
diff --git a/arch/x86/kernel/ftrace.c b/arch/x86/kernel/ftrace.c
index 024c3053dbba..a044734167af 100644
--- a/arch/x86/kernel/ftrace.c
+++ b/arch/x86/kernel/ftrace.c
@@ -1072,7 +1072,7 @@ void prepare_ftrace_return(unsigned long self_addr, unsigned long *parent,
return;
}
- if (function_graph_enter(old, self_addr, frame_pointer, parent))
+ if (function_graph_enter(old, self_addr, frame_pointer, parent, NULL))
*parent = old;
}
#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h
index f5aab37a8c34..e615b5e639aa 100644
--- a/include/linux/ftrace.h
+++ b/include/linux/ftrace.h
@@ -757,9 +757,12 @@ struct ftrace_graph_ret {
/* Type of the callback handlers for tracing function graph*/
typedef void (*trace_func_graph_ret_t)(struct ftrace_graph_ret *); /* return */
-typedef int (*trace_func_graph_ent_t)(struct ftrace_graph_ent *); /* entry */
+/* @pt_regs is only available for CONFIG_FTRACE_FUNC_PROTOTYPE. */
+typedef int (*trace_func_graph_ent_t)(struct ftrace_graph_ent *,
+ struct pt_regs *); /* entry */
-extern int ftrace_graph_entry_stub(struct ftrace_graph_ent *trace);
+int ftrace_graph_entry_stub(struct ftrace_graph_ent *trace,
+ struct pt_regs *pt_regs);
#ifdef CONFIG_FUNCTION_GRAPH_TRACER
@@ -797,7 +800,8 @@ extern void return_to_handler(void);
extern int
function_graph_enter(unsigned long ret, unsigned long func,
- unsigned long frame_pointer, unsigned long *retp);
+ unsigned long frame_pointer, unsigned long *retp,
+ struct pt_regs *pt_regs);
struct ftrace_ret_stack *
ftrace_graph_get_ret_stack(struct task_struct *task, int idx);
diff --git a/kernel/trace/fgraph.c b/kernel/trace/fgraph.c
index 8dfd5021b933..7451dba84fee 100644
--- a/kernel/trace/fgraph.c
+++ b/kernel/trace/fgraph.c
@@ -96,8 +96,13 @@ ftrace_push_return_trace(unsigned long ret, unsigned long func,
return 0;
}
+/*
+ * Called from arch specific code. @pt_regs is only available for
+ * CONFIG_FTRACE_FUNC_PROTOTYPE.
+ */
int function_graph_enter(unsigned long ret, unsigned long func,
- unsigned long frame_pointer, unsigned long *retp)
+ unsigned long frame_pointer, unsigned long *retp,
+ struct pt_regs *pt_regs)
{
struct ftrace_graph_ent trace;
@@ -108,7 +113,7 @@ int function_graph_enter(unsigned long ret, unsigned long func,
goto out;
/* Only trace if the calling function expects to */
- if (!ftrace_graph_entry(&trace))
+ if (!ftrace_graph_entry(&trace, pt_regs))
goto out_ret;
return 0;
@@ -204,9 +209,11 @@ static struct notifier_block ftrace_suspend_notifier = {
/*
* Send the trace to the ring-buffer.
+ * @retval is only available for CONFIG_FTRACE_FUNC_PROTOTYPE.
* @return the original return address.
*/
-unsigned long ftrace_return_to_handler(unsigned long frame_pointer)
+unsigned long ftrace_return_to_handler(unsigned long frame_pointer,
+ unsigned long retval)
{
struct ftrace_graph_ret trace;
unsigned long ret;
@@ -327,7 +334,8 @@ void ftrace_graph_sleep_time_control(bool enable)
fgraph_sleep_time = enable;
}
-int ftrace_graph_entry_stub(struct ftrace_graph_ent *trace)
+int ftrace_graph_entry_stub(struct ftrace_graph_ent *trace,
+ struct pt_regs *pt_regs)
{
return 0;
}
@@ -417,11 +425,12 @@ ftrace_graph_probe_sched_switch(void *ignore, bool preempt,
next->ret_stack[index].calltime += timestamp;
}
-static int ftrace_graph_entry_test(struct ftrace_graph_ent *trace)
+static int ftrace_graph_entry_test(struct ftrace_graph_ent *trace,
+ struct pt_regs *pt_regs)
{
if (!ftrace_ops_test(&global_ops, trace->func, NULL))
return 0;
- return __ftrace_graph_entry(trace);
+ return __ftrace_graph_entry(trace, pt_regs);
}
/*
diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
index 438b8b47198f..a1683cc55838 100644
--- a/kernel/trace/ftrace.c
+++ b/kernel/trace/ftrace.c
@@ -794,7 +794,9 @@ void ftrace_graph_graph_time_control(bool enable)
fgraph_graph_time = enable;
}
-static int profile_graph_entry(struct ftrace_graph_ent *trace)
+/* @pt_regs is only available for CONFIG_FTRACE_FUNC_PROTOTYPE. */
+static int profile_graph_entry(struct ftrace_graph_ent *trace,
+ struct pt_regs *pt_regs)
{
struct ftrace_ret_stack *ret_stack;
diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h
index 22433a15e340..4b31176d443e 100644
--- a/kernel/trace/trace.h
+++ b/kernel/trace/trace.h
@@ -737,7 +737,7 @@ void print_trace_header(struct seq_file *m, struct trace_iterator *iter);
int trace_empty(struct trace_iterator *iter);
void trace_graph_return(struct ftrace_graph_ret *trace);
-int trace_graph_entry(struct ftrace_graph_ent *trace);
+int trace_graph_entry(struct ftrace_graph_ent *trace, struct pt_regs *pt_regs);
void set_graph_array(struct trace_array *tr);
void tracing_start_cmdline_record(void);
diff --git a/kernel/trace/trace_functions_graph.c b/kernel/trace/trace_functions_graph.c
index 78af97163147..f331a9ba946d 100644
--- a/kernel/trace/trace_functions_graph.c
+++ b/kernel/trace/trace_functions_graph.c
@@ -124,7 +124,7 @@ static inline int ftrace_graph_ignore_irqs(void)
return in_irq();
}
-int trace_graph_entry(struct ftrace_graph_ent *trace)
+int trace_graph_entry(struct ftrace_graph_ent *trace, struct pt_regs *pt_regs)
{
struct trace_array *tr = graph_array;
struct trace_array_cpu *data;
diff --git a/kernel/trace/trace_irqsoff.c b/kernel/trace/trace_irqsoff.c
index a745b0cee5d3..513e3544a45a 100644
--- a/kernel/trace/trace_irqsoff.c
+++ b/kernel/trace/trace_irqsoff.c
@@ -172,7 +172,8 @@ static int irqsoff_display_graph(struct trace_array *tr, int set)
return start_irqsoff_tracer(irqsoff_trace, set);
}
-static int irqsoff_graph_entry(struct ftrace_graph_ent *trace)
+static int irqsoff_graph_entry(struct ftrace_graph_ent *trace,
+ struct pt_regs *pt_regs)
{
struct trace_array *tr = irqsoff_trace;
struct trace_array_cpu *data;
diff --git a/kernel/trace/trace_sched_wakeup.c b/kernel/trace/trace_sched_wakeup.c
index 743b2b520d34..ce18f679930c 100644
--- a/kernel/trace/trace_sched_wakeup.c
+++ b/kernel/trace/trace_sched_wakeup.c
@@ -112,7 +112,8 @@ static int wakeup_display_graph(struct trace_array *tr, int set)
return start_func_tracer(tr, set);
}
-static int wakeup_graph_entry(struct ftrace_graph_ent *trace)
+static int wakeup_graph_entry(struct ftrace_graph_ent *trace,
+ struct pt_regs *pt_regs)
{
struct trace_array *tr = wakeup_trace;
struct trace_array_cpu *data;
--
2.20.1
This patch introduces the core part of our new CONFIG_FTRACE_FUNC_PROTOTYPE
feature. For arch which supports this feature must implement a new
arch-specific interface arch_fgraph_record_params().
In this patch, we add a new trace option "record-funcproto", and by now
only function graph tracer is supported. The major work is to handle
the printing stuff.
Here is an example of the graph trace of function pick_next_task_fair().
Note that we only record the parameter and return value of global
functions.
2) | pick_next_task_fair() {
2) | update_blocked_averages() {
2) 0.765 us | _raw_spin_lock_irqsave(lock=0xffff88807da2b100); /* ret=0x0000000000000082 */
2) 0.944 us | update_rq_clock(rq=0xffff88807da2b100);
2) 0.612 us | __update_load_avg_cfs_rq(now=0x000000251b8516ee, cfs_rq=0xffff8880754f7488); /* ret=0 */
2) 0.654 us | __update_load_avg_se(now=0x000000251b8516ee, cfs_rq=0xffff88807da2b180, se=0xffff88807be2e0d8); /* ret=0 */
2) 0.206 us | __update_load_avg_cfs_rq(now=0x000000251b8516ee, cfs_rq=0xffff88807da2b180); /* ret=0 */
2) | __update_load_avg_cfs_rq(now=0x000000251b8516ee, cfs_rq=0xffff888079b5fb18) {
2) 2.410 us | __accumulate_pelt_segments();
2) 3.103 us | } /* ret=1 */
2) 0.193 us | __update_load_avg_cfs_rq(now=0x000000251b8516ee, cfs_rq=0xffff88807da2b180); /* ret=0 */
2) | update_rt_rq_load_avg(now=0x000000251b8516ee, rq=0xffff88807da2b100, running=0) {
2) 0.258 us | __accumulate_pelt_segments();
2) 1.617 us | } /* ret=1 */
2) | update_dl_rq_load_avg(now=0x000000251b8516ee, rq=0xffff88807da2b100, running=0) {
2) 0.230 us | __accumulate_pelt_segments();
2) 1.511 us | } /* ret=1 */
2) 1.040 us | _raw_spin_unlock_irqrestore(lock=0xffff88807da2b100, flags=0x0000000000000082);
2) + 14.739 us | }
2) | load_balance() {
2) | find_busiest_group() {
2) 0.874 us | update_group_capacity(sd=0xffff88807c1d37d0, cpu=2);
2) 1.761 us | idle_cpu();
2) 0.262 us | idle_cpu();
2) 0.217 us | idle_cpu();
2) 6.338 us | }
2) 8.442 us | }
2) 1.823 us | __msecs_to_jiffies(m=0x00000006); /* ret=0x0000000000000002 */
2) | load_balance() {
2) | find_busiest_group() {
2) 0.434 us | idle_cpu();
2) 0.233 us | idle_cpu();
2) 0.210 us | idle_cpu();
2) 2.308 us | }
2) 2.821 us | }
2) 0.263 us | __msecs_to_jiffies(m=0x00000008); /* ret=0x0000000000000002 */
2) 0.977 us | _raw_spin_lock(lock=0xffff88807da2b100);
2) + 32.262 us | }
The printing rules of each value is:
o For signed value, it is always printed as decimal number.
o For unsigned value,
- For value has size great than 8, it is printed as '{..}'.
- For value has size of 1,2,4,8, it is printed as hexadecimal number.
- If failed to record a parameter, it is printed as '?'.
Signed-off-by: Changbin Du <[email protected]>
---
include/linux/ftrace.h | 27 +++++++
kernel/trace/fgraph.c | 5 ++
kernel/trace/ftrace.c | 50 +++++++++++++
kernel/trace/trace.h | 8 ++
kernel/trace/trace_entries.h | 10 +++
kernel/trace/trace_functions_graph.c | 106 +++++++++++++++++++++++++--
6 files changed, 201 insertions(+), 5 deletions(-)
diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h
index e615b5e639aa..82b92d355431 100644
--- a/include/linux/ftrace.h
+++ b/include/linux/ftrace.h
@@ -17,6 +17,7 @@
#include <linux/types.h>
#include <linux/init.h>
#include <linux/fs.h>
+#include <linux/trace_seq.h>
#include <asm/ftrace.h>
@@ -377,6 +378,9 @@ struct func_prototype {
#define FTRACE_PROTOTYPE_SIGNED(t) (t & BIT(7))
#define FTRACE_PROTOTYPE_SIZE(t) (t & GENMASK(6, 0))
+
+void ftrace_print_typed_val(struct trace_seq *s, uint8_t type,
+ unsigned long val);
#endif
int ftrace_force_update(void);
@@ -731,6 +735,13 @@ extern void ftrace_init(void);
static inline void ftrace_init(void) { }
#endif
+#ifdef CONFIG_FTRACE_FUNC_PROTOTYPE
+# define FTRACE_MAX_FUNC_PARAMS 10
+
+# define FTRACE_PROTOTYPE_SIGNED(t) (t & BIT(7))
+# define FTRACE_PROTOTYPE_SIZE(t) (t & GENMASK(6, 0))
+#endif
+
/*
* Structure that defines an entry function trace.
* It's already packed but the attribute "packed" is needed
@@ -739,6 +750,12 @@ static inline void ftrace_init(void) { }
struct ftrace_graph_ent {
unsigned long func; /* Current function */
int depth;
+#ifdef CONFIG_FTRACE_FUNC_PROTOTYPE
+ uint8_t nr_param;
+ char *param_names[FTRACE_MAX_FUNC_PARAMS];
+ uint8_t param_types[FTRACE_MAX_FUNC_PARAMS];
+ unsigned long param_values[FTRACE_MAX_FUNC_PARAMS];
+#endif
} __packed;
/*
@@ -753,8 +770,13 @@ struct ftrace_graph_ret {
unsigned long long calltime;
unsigned long long rettime;
int depth;
+#ifdef CONFIG_FTRACE_FUNC_PROTOTYPE
+ uint8_t ret_type;
+ unsigned long retval;
+#endif
} __packed;
+
/* Type of the callback handlers for tracing function graph*/
typedef void (*trace_func_graph_ret_t)(struct ftrace_graph_ret *); /* return */
/* @pt_regs is only available for CONFIG_FTRACE_FUNC_PROTOTYPE. */
@@ -842,6 +864,11 @@ static inline void unpause_graph_tracing(void)
{
atomic_dec(¤t->tracing_graph_pause);
}
+
+void arch_fgraph_record_params(struct ftrace_graph_ent *trace,
+ struct func_prototype *proto,
+ struct pt_regs *pt_regs);
+
#else /* !CONFIG_FUNCTION_GRAPH_TRACER */
#define __notrace_funcgraph
diff --git a/kernel/trace/fgraph.c b/kernel/trace/fgraph.c
index 7451dba84fee..26e452418249 100644
--- a/kernel/trace/fgraph.c
+++ b/kernel/trace/fgraph.c
@@ -220,6 +220,11 @@ unsigned long ftrace_return_to_handler(unsigned long frame_pointer,
ftrace_pop_return_trace(&trace, &ret, frame_pointer);
trace.rettime = trace_clock_local();
+
+#ifdef CONFIG_FTRACE_FUNC_PROTOTYPE
+ trace.retval = retval;
+#endif
+
ftrace_graph_return(&trace);
/*
* The ftrace_graph_return() may still access the current
diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
index a1683cc55838..1e6a96f1986b 100644
--- a/kernel/trace/ftrace.c
+++ b/kernel/trace/ftrace.c
@@ -5658,6 +5658,56 @@ static int ftrace_process_funcproto(struct module *mod,
return ret;
}
+
+void ftrace_print_typed_val(struct trace_seq *s, uint8_t type,
+ unsigned long val)
+{
+ unsigned int sz = FTRACE_PROTOTYPE_SIZE(type);
+ bool is_signed = FTRACE_PROTOTYPE_SIGNED(type);
+
+ /* Don't show complex types */
+ if (sz > sizeof(long)) {
+ trace_seq_printf(s, "{..}");
+ return;
+ }
+
+ switch (sz) {
+ case 0:
+ /* The value is not valid. */
+ trace_seq_printf(s, "?");
+ break;
+ case 1:
+ val &= GENMASK_ULL(7, 0);
+ if (is_signed)
+ trace_seq_printf(s, "%d", (char)val);
+ else
+ trace_seq_printf(s, "0x%02lx", val);
+ break;
+ case 2:
+ val &= GENMASK_ULL(15, 0);
+ if (is_signed)
+ trace_seq_printf(s, "%d", (short)val);
+ else
+ trace_seq_printf(s, "0x%04lx", val);
+ break;
+ case 4:
+ val &= GENMASK_ULL(31, 0);
+ if (is_signed)
+ trace_seq_printf(s, "%d", (int)val);
+ else
+ trace_seq_printf(s, "0x%08lx", val);
+ break;
+ case 8:
+ val &= GENMASK_ULL(63, 0);
+ if (is_signed)
+ trace_seq_printf(s, "%lld", (long long)val);
+ else
+ trace_seq_printf(s, "0x%016lx", val);
+ break;
+ default:
+ trace_seq_printf(s, "{badsize%d}", sz);
+ }
+}
#endif
struct ftrace_mod_func {
diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h
index 4b31176d443e..f10acad0140f 100644
--- a/kernel/trace/trace.h
+++ b/kernel/trace/trace.h
@@ -1231,6 +1231,13 @@ extern int trace_get_user(struct trace_parser *parser, const char __user *ubuf,
# define STACK_FLAGS
#endif
+#ifdef CONFIG_FTRACE_FUNC_PROTOTYPE
+# define FUNCPROTO_FLAGS \
+ C(RECORD_FUNCPROTO, "record-funcproto"),
+#else
+# define FUNCPROTO_FLAGS
+#endif
+
/*
* trace_iterator_flags is an enumeration that defines bit
* positions into trace_flags that controls the output.
@@ -1256,6 +1263,7 @@ extern int trace_get_user(struct trace_parser *parser, const char __user *ubuf,
C(LATENCY_FMT, "latency-format"), \
C(RECORD_CMD, "record-cmd"), \
C(RECORD_TGID, "record-tgid"), \
+ FUNCPROTO_FLAGS \
C(OVERWRITE, "overwrite"), \
C(STOP_ON_FREE, "disable_on_free"), \
C(IRQ_INFO, "irq-info"), \
diff --git a/kernel/trace/trace_entries.h b/kernel/trace/trace_entries.h
index fc8e97328e54..68b044ea8440 100644
--- a/kernel/trace/trace_entries.h
+++ b/kernel/trace/trace_entries.h
@@ -82,6 +82,12 @@ FTRACE_ENTRY_PACKED(funcgraph_entry, ftrace_graph_ent_entry,
__field_struct( struct ftrace_graph_ent, graph_ent )
__field_desc( unsigned long, graph_ent, func )
__field_desc( int, graph_ent, depth )
+#ifdef CONFIG_FTRACE_FUNC_PROTOTYPE
+ __field_desc( unsigned char, graph_ent, nr_param )
+ __array_desc( char *, graph_ent, param_names, FTRACE_MAX_FUNC_PARAMS)
+ __array_desc( uint8_t, graph_ent, param_types, FTRACE_MAX_FUNC_PARAMS)
+ __array_desc( unsigned long, graph_ent, param_values, FTRACE_MAX_FUNC_PARAMS)
+#endif
),
F_printk("--> %ps (%d)", (void *)__entry->func, __entry->depth),
@@ -101,6 +107,10 @@ FTRACE_ENTRY_PACKED(funcgraph_exit, ftrace_graph_ret_entry,
__field_desc( unsigned long long, ret, rettime )
__field_desc( unsigned long, ret, overrun )
__field_desc( int, ret, depth )
+#ifdef CONFIG_FTRACE_FUNC_PROTOTYPE
+ __field_desc( unsigned char, ret, ret_type)
+ __field_desc( unsigned long, ret, retval )
+#endif
),
F_printk("<-- %ps (%d) (start: %llx end: %llx) over: %d",
diff --git a/kernel/trace/trace_functions_graph.c b/kernel/trace/trace_functions_graph.c
index f331a9ba946d..ba4eb71646e9 100644
--- a/kernel/trace/trace_functions_graph.c
+++ b/kernel/trace/trace_functions_graph.c
@@ -169,6 +169,17 @@ int trace_graph_entry(struct ftrace_graph_ent *trace, struct pt_regs *pt_regs)
if (tracing_thresh)
return 1;
+#ifdef CONFIG_FTRACE_FUNC_PROTOTYPE
+ trace->nr_param = 0;
+ if (tr->trace_flags & TRACE_ITER_RECORD_FUNCPROTO) {
+ struct ftrace_func_entry *ent;
+
+ ent = ftrace_lookup_ip(ftrace_prototype_hash, trace->func);
+ if (ent)
+ arch_fgraph_record_params(trace, ent->priv, pt_regs);
+ }
+#endif
+
local_irq_save(flags);
cpu = raw_smp_processor_id();
data = per_cpu_ptr(tr->trace_buffer.data, cpu);
@@ -250,6 +261,21 @@ void trace_graph_return(struct ftrace_graph_ret *trace)
return;
}
+#ifdef CONFIG_FTRACE_FUNC_PROTOTYPE
+ if (tr->trace_flags & TRACE_ITER_RECORD_FUNCPROTO) {
+ struct ftrace_func_entry *ent;
+
+ ent = ftrace_lookup_ip(ftrace_prototype_hash, trace->func);
+ if (ent) {
+ /* The retval has been saved by trace_graph_return(). */
+ trace->ret_type =
+ ((struct func_prototype *)ent->priv)->ret_type;
+ } else
+ trace->ret_type = 0;
+ } else
+ trace->ret_type = 0;
+#endif
+
local_irq_save(flags);
cpu = raw_smp_processor_id();
data = per_cpu_ptr(tr->trace_buffer.data, cpu);
@@ -380,6 +406,71 @@ static void print_graph_lat_fmt(struct trace_seq *s, struct trace_entry *entry)
trace_seq_puts(s, " | ");
}
+#ifdef CONFIG_FTRACE_FUNC_PROTOTYPE
+static void print_graph_params(struct trace_seq *s,
+ struct ftrace_graph_ent *call,
+ struct ftrace_graph_ret *graph_ret)
+{
+ int i;
+
+ BUG_ON(call->nr_param > FTRACE_MAX_FUNC_PARAMS);
+
+ trace_seq_printf(s, "%ps(", (void *)call->func);
+ for (i = 0; i < call->nr_param; i++) {
+ if (i > 0)
+ trace_seq_printf(s, ", ");
+ trace_seq_printf(s, "%s=", call->param_names[i]);
+ ftrace_print_typed_val(s, call->param_types[i],
+ call->param_values[i]);
+ }
+
+ if (graph_ret) {
+ /* leaf */
+ if (graph_ret->ret_type) {
+ trace_seq_printf(s, "); /* ret=");
+ ftrace_print_typed_val(s, graph_ret->ret_type,
+ graph_ret->retval);
+ trace_seq_puts(s, " */\n");
+ } else
+ trace_seq_puts(s, ");\n");
+ } else
+ trace_seq_printf(s, ") {\n");
+}
+
+static void print_graph_retval(struct trace_seq *s,
+ struct ftrace_graph_ret *trace,
+ bool tail)
+{
+ if (trace->ret_type) {
+ if (tail)
+ trace_seq_puts(s, ", ");
+ else
+ trace_seq_puts(s, " /* ");
+
+ trace_seq_printf(s, "ret=");
+ ftrace_print_typed_val(s, trace->ret_type, trace->retval);
+
+ trace_seq_printf(s, " */");
+ }
+}
+#else
+static void print_graph_params(struct trace_seq *s,
+ struct ftrace_graph_ent *call,
+ struct ftrace_graph_ret *graph_ret)
+{
+ if (graph_ret)
+ trace_seq_printf(s, "%ps();\n", (void *)call->func);
+ else
+ trace_seq_printf(s, "%ps() {\n", (void *)call->func);
+}
+
+static void print_graph_retval(struct trace_seq *s,
+ struct ftrace_graph_ret *trace,
+ bool tail)
+{
+}
+#endif
+
/* If the pid changed since the last trace, output this event */
static void
verif_pid(struct trace_seq *s, pid_t pid, int cpu, struct fgraph_data *data)
@@ -665,7 +756,7 @@ print_graph_entry_leaf(struct trace_iterator *iter,
for (i = 0; i < call->depth * TRACE_GRAPH_INDENT; i++)
trace_seq_putc(s, ' ');
- trace_seq_printf(s, "%ps();\n", (void *)call->func);
+ print_graph_params(s, call, graph_ret);
print_graph_irq(iter, graph_ret->func, TRACE_GRAPH_RET,
cpu, iter->ent->pid, flags);
@@ -703,7 +794,7 @@ print_graph_entry_nested(struct trace_iterator *iter,
for (i = 0; i < call->depth * TRACE_GRAPH_INDENT; i++)
trace_seq_putc(s, ' ');
- trace_seq_printf(s, "%ps() {\n", (void *)call->func);
+ print_graph_params(s, call, NULL);
if (trace_seq_has_overflowed(s))
return TRACE_TYPE_PARTIAL_LINE;
@@ -950,10 +1041,15 @@ print_graph_return(struct ftrace_graph_ret *trace, struct trace_seq *s,
* belongs to, write out the function name. Always do
* that if the funcgraph-tail option is enabled.
*/
- if (func_match && !(flags & TRACE_GRAPH_PRINT_TAIL))
- trace_seq_puts(s, "}\n");
- else
+ if (func_match && !(flags & TRACE_GRAPH_PRINT_TAIL)) {
+ trace_seq_puts(s, "}");
+ print_graph_retval(s, trace, false);
+ trace_seq_puts(s, "\n");
+ } else {
trace_seq_printf(s, "} /* %ps */\n", (void *)trace->func);
+ print_graph_retval(s, trace, true);
+ trace_seq_puts(s, "\n");
+ }
/* Overrun */
if (flags & TRACE_GRAPH_PRINT_OVERRUN)
--
2.20.1
This patch implements the arch_fgraph_record_params() function for x86_64
platform and deliver the return value of function to ftrace core part.
Signed-off-by: Changbin Du <[email protected]>
---
arch/x86/Kconfig | 1 +
arch/x86/kernel/ftrace.c | 84 +++++++++++++++++++++++++++++++++++--
arch/x86/kernel/ftrace_64.S | 4 +-
3 files changed, 85 insertions(+), 4 deletions(-)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 222855cc0158..34e583bfdab8 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -31,6 +31,7 @@ config X86_64
select NEED_DMA_MAP_STATE
select SWIOTLB
select ARCH_HAS_SYSCALL_WRAPPER
+ select HAVE_FTRACE_FUNC_PROTOTYPE
config FORCE_DYNAMIC_FTRACE
def_bool y
diff --git a/arch/x86/kernel/ftrace.c b/arch/x86/kernel/ftrace.c
index a044734167af..fc0a062ce762 100644
--- a/arch/x86/kernel/ftrace.c
+++ b/arch/x86/kernel/ftrace.c
@@ -31,6 +31,7 @@
#include <asm/ftrace.h>
#include <asm/nops.h>
#include <asm/text-patching.h>
+#include <asm-generic/dwarf.h>
#ifdef CONFIG_DYNAMIC_FTRACE
@@ -918,7 +919,8 @@ static void *addr_from_call(void *ptr)
}
void prepare_ftrace_return(unsigned long self_addr, unsigned long *parent,
- unsigned long frame_pointer);
+ unsigned long frame_pointer,
+ struct pt_regs *pt_regs);
/*
* If the ops->trampoline was not allocated, then it probably
@@ -973,6 +975,82 @@ void arch_ftrace_trampoline_free(struct ftrace_ops *ops)
ops->trampoline = 0;
}
+#ifdef CONFIG_FTRACE_FUNC_PROTOTYPE
+void arch_fgraph_record_params(struct ftrace_graph_ent *trace,
+ struct func_prototype *proto,
+ struct pt_regs *pt_regs)
+{
+ int i;
+
+ trace->nr_param = min(proto->nr_param, (uint8_t)FTRACE_MAX_FUNC_PARAMS);
+
+ for (i = 0; i < trace->nr_param; i++) {
+ struct func_param *param = &proto->params[i];
+ unsigned int sz = FTRACE_PROTOTYPE_SIZE(param->type);
+ long off = (char)param->loc[1];
+ unsigned long value = 0;
+ bool good = true;
+
+ if (sz > sizeof(value)) {
+ /* Don't record value of complex type. */
+ trace->param_types[i] = param->type;
+ trace->param_values[i] = 0;
+ continue;
+ }
+
+ switch (param->loc[0]) {
+ case DW_OP_reg1:
+ value = pt_regs->dx;
+ break;
+ case DW_OP_reg2:
+ value = pt_regs->cx;
+ break;
+ case DW_OP_reg3:
+ value = pt_regs->bx;
+ break;
+ case DW_OP_reg4:
+ value = pt_regs->si;
+ break;
+ case DW_OP_reg5:
+ value = pt_regs->di;
+ break;
+ case DW_OP_reg6:
+ value = pt_regs->bp;
+ break;
+ case DW_OP_reg8:
+ value = pt_regs->r8;
+ break;
+ case DW_OP_reg9:
+ value = pt_regs->r9;
+ break;
+ case DW_OP_fbreg:
+ if (probe_kernel_read(&value,
+ (void *)pt_regs->bp + off,
+ sz))
+ good = false;
+ break;
+ case DW_OP_breg7:
+ if (probe_kernel_read(&value,
+ (void *)pt_regs->sp + off,
+ sz))
+ good = false;
+ break;
+ default:
+ /* unexpected loc expression */
+ good = false;
+ }
+
+ trace->param_names[i] = param->name;
+ if (good) {
+ trace->param_types[i] = param->type;
+ trace->param_values[i] = value;
+ } else {
+ /* set the type to 0 so we skip it when printing. */
+ trace->param_types[i] = 0;
+ }
+ }
+}
+#endif /* CONFIG_FTRACE_FUNC_PROTOTYPE */
#endif /* CONFIG_X86_64 */
#endif /* CONFIG_DYNAMIC_FTRACE */
@@ -1017,7 +1095,7 @@ int ftrace_disable_ftrace_graph_caller(void)
* in current thread info.
*/
void prepare_ftrace_return(unsigned long self_addr, unsigned long *parent,
- unsigned long frame_pointer)
+ unsigned long frame_pointer, struct pt_regs *pt_regs)
{
unsigned long old;
int faulted;
@@ -1072,7 +1150,7 @@ void prepare_ftrace_return(unsigned long self_addr, unsigned long *parent,
return;
}
- if (function_graph_enter(old, self_addr, frame_pointer, parent, NULL))
+ if (function_graph_enter(old, self_addr, frame_pointer, parent, pt_regs))
*parent = old;
}
#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
diff --git a/arch/x86/kernel/ftrace_64.S b/arch/x86/kernel/ftrace_64.S
index 809d54397dba..e01d6358e859 100644
--- a/arch/x86/kernel/ftrace_64.S
+++ b/arch/x86/kernel/ftrace_64.S
@@ -289,7 +289,8 @@ ENTRY(ftrace_graph_caller)
leaq MCOUNT_REG_SIZE+8(%rsp), %rsi
movq $0, %rdx /* No framepointers needed */
- call prepare_ftrace_return
+ movq %rsp, %rcx /* the fourth parameter */
+ call prepare_ftrace_return
restore_mcount_regs
@@ -304,6 +305,7 @@ ENTRY(return_to_handler)
movq %rax, (%rsp)
movq %rdx, 8(%rsp)
movq %rbp, %rdi
+ movq %rax, %rsi
call ftrace_return_to_handler
--
2.20.1
Just add the doc for our new feature.
Signed-off-by: Changbin Du <[email protected]>
---
Documentation/trace/ftrace.rst | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/Documentation/trace/ftrace.rst b/Documentation/trace/ftrace.rst
index f60079259669..c68fbbedb8bd 100644
--- a/Documentation/trace/ftrace.rst
+++ b/Documentation/trace/ftrace.rst
@@ -988,6 +988,7 @@ To see what is available, simply cat the file::
nolatency-format
record-cmd
norecord-tgid
+ norecord-funcproto
overwrite
nodisable_on_free
irq-info
@@ -1131,6 +1132,11 @@ Here are the available options:
mapped Thread Group IDs (TGID) mapping to pids. See
"saved_tgids".
+ record-funcproto
+ Record function parameters and return value. This option
+ is only supported by function_graph tracer on x86_64
+ platform by now.
+
overwrite
This controls what happens when the trace buffer is
full. If "1" (default), the oldest events are
--
2.20.1
Make scripts/ftrace/ maintained and I would like to help with reviewing
related patches.
Signed-off-by: Changbin Du <[email protected]>
---
MAINTAINERS | 2 ++
1 file changed, 2 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index 9cbcf167bdd0..ca012ea260d7 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -16293,6 +16293,7 @@ F: drivers/char/tpm/
TRACING
M: Steven Rostedt <[email protected]>
M: Ingo Molnar <[email protected]>
+R: Changbin Du <[email protected]>
T: git git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git perf/core
S: Maintained
F: Documentation/trace/ftrace.rst
@@ -16303,6 +16304,7 @@ F: include/linux/trace*.h
F: include/trace/
F: kernel/trace/
F: tools/testing/selftests/ftrace/
+F: scripts/ftrace/
TRACING MMIO ACCESSES (MMIOTRACE)
M: Steven Rostedt <[email protected]>
--
2.20.1
On Sun, Aug 25, 2019 at 09:23:22PM +0800, Changbin Du wrote:
> Add generic DWARF constant definitions. We will use it later.
>
> Signed-off-by: Changbin Du <[email protected]>
> ---
> include/asm-generic/dwarf.h | 199 ++++++++++++++++++++++++++++++++++++
> 1 file changed, 199 insertions(+)
> create mode 100644 include/asm-generic/dwarf.h
>
> diff --git a/include/asm-generic/dwarf.h b/include/asm-generic/dwarf.h
> new file mode 100644
> index 000000000000..c705633c2a8f
> --- /dev/null
> +++ b/include/asm-generic/dwarf.h
> @@ -0,0 +1,199 @@
> +/* SPDX-License-Identifier: GPL-2.0
> + *
> + * Architecture independent definitions of DWARF.
> + *
> + * Copyright (C) 2019 Changbin Du <[email protected]>
You're claiming copyright on dwarf definitions? ;-)
I'm thinking only Oracle was daft enough to think stuff like that was
copyrightable.
Also; I think it would be very good to not use/depend on DWARF for this.
You really don't need all of DWARF; I'm thikning you only need a few
types; for location we already have regs_get_kernel_argument() which
has all the logic to find the n-th argument.
On Sun, Aug 25, 2019 at 09:23:24PM +0800, Changbin Du wrote:
> When CONFIG_FTRACE_FUNC_PROTOTYPE is enabled, thousands of
> ftrace_func_entry instances are created. So create a dedicated
> memcache to enhance performance.
>
> Signed-off-by: Changbin Du <[email protected]>
> ---
> kernel/trace/ftrace.c | 17 ++++++++++++++++-
> 1 file changed, 16 insertions(+), 1 deletion(-)
>
> diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
> index a314f0768b2c..cfcb8dad93ea 100644
> --- a/kernel/trace/ftrace.c
> +++ b/kernel/trace/ftrace.c
> @@ -94,6 +94,8 @@ struct ftrace_ops *function_trace_op __read_mostly = &ftrace_list_end;
> /* What to set function_trace_op to */
> static struct ftrace_ops *set_function_trace_op;
>
> +struct kmem_cache *hash_entry_cache;
> +
> static bool ftrace_pids_enabled(struct ftrace_ops *ops)
> {
> struct trace_array *tr;
> @@ -1169,7 +1171,7 @@ static int add_hash_entry(struct ftrace_hash *hash, unsigned long ip,
> {
> struct ftrace_func_entry *entry;
>
> - entry = kmalloc(sizeof(*entry), GFP_KERNEL);
> + entry = kmem_cache_alloc(hash_entry_cache, GFP_KERNEL);
> if (!entry)
> return -ENOMEM;
>
> @@ -6153,6 +6155,15 @@ void __init ftrace_init(void)
> if (ret)
> goto failed;
>
> + hash_entry_cache = kmem_cache_create("ftrace-hash",
> + sizeof(struct ftrace_func_entry),
> + sizeof(struct ftrace_func_entry),
> + 0, NULL);
> + if (!hash_entry_cache) {
> + pr_err("failed to create ftrace hash entry cache\n");
> + goto failed;
> + }
Wait what; you already have then in the binary image, now you're
allocating extra memory for each of them?
Did you look at what ORC does? Is the binary search really not fast
enough?
Hi, Peter,
On Mon, Aug 26, 2019 at 09:42:15AM +0200, Peter Zijlstra wrote:
> On Sun, Aug 25, 2019 at 09:23:22PM +0800, Changbin Du wrote:
> > Add generic DWARF constant definitions. We will use it later.
> >
> > Signed-off-by: Changbin Du <[email protected]>
> > ---
> > include/asm-generic/dwarf.h | 199 ++++++++++++++++++++++++++++++++++++
> > 1 file changed, 199 insertions(+)
> > create mode 100644 include/asm-generic/dwarf.h
> >
> > diff --git a/include/asm-generic/dwarf.h b/include/asm-generic/dwarf.h
> > new file mode 100644
> > index 000000000000..c705633c2a8f
> > --- /dev/null
> > +++ b/include/asm-generic/dwarf.h
> > @@ -0,0 +1,199 @@
> > +/* SPDX-License-Identifier: GPL-2.0
> > + *
> > + * Architecture independent definitions of DWARF.
> > + *
> > + * Copyright (C) 2019 Changbin Du <[email protected]>
>
> You're claiming copyright on dwarf definitions? ;-)
>
> I'm thinking only Oracle was daft enough to think stuff like that was
> copyrightable.
>
ok, let me remove copyright line. I think SPDX claim is okay, right?
> Also; I think it would be very good to not use/depend on DWARF for this.
>
It only includes the DWARF expersion opcodes, not all of dwarf stuffs.
> You really don't need all of DWARF; I'm thikning you only need a few
> types; for location we already have regs_get_kernel_argument() which
> has all the logic to find the n-th argument.
>
regs_get_kernel_argument() can handle most cases, but if the size of one paramater
exceeds 64bit (it is rare in kernel), we must recalculate the locations. So I think
dwarf location descriptor is the most accurate one.
--
Cheers,
Changbin Du
On Mon, Aug 26, 2019 at 09:44:37AM +0200, Peter Zijlstra wrote:
> On Sun, Aug 25, 2019 at 09:23:24PM +0800, Changbin Du wrote:
> > When CONFIG_FTRACE_FUNC_PROTOTYPE is enabled, thousands of
> > ftrace_func_entry instances are created. So create a dedicated
> > memcache to enhance performance.
> >
> > Signed-off-by: Changbin Du <[email protected]>
> > ---
> > kernel/trace/ftrace.c | 17 ++++++++++++++++-
> > 1 file changed, 16 insertions(+), 1 deletion(-)
> >
> > diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
> > index a314f0768b2c..cfcb8dad93ea 100644
> > --- a/kernel/trace/ftrace.c
> > +++ b/kernel/trace/ftrace.c
> > @@ -94,6 +94,8 @@ struct ftrace_ops *function_trace_op __read_mostly = &ftrace_list_end;
> > /* What to set function_trace_op to */
> > static struct ftrace_ops *set_function_trace_op;
> >
> > +struct kmem_cache *hash_entry_cache;
> > +
> > static bool ftrace_pids_enabled(struct ftrace_ops *ops)
> > {
> > struct trace_array *tr;
> > @@ -1169,7 +1171,7 @@ static int add_hash_entry(struct ftrace_hash *hash, unsigned long ip,
> > {
> > struct ftrace_func_entry *entry;
> >
> > - entry = kmalloc(sizeof(*entry), GFP_KERNEL);
> > + entry = kmem_cache_alloc(hash_entry_cache, GFP_KERNEL);
> > if (!entry)
> > return -ENOMEM;
> >
> > @@ -6153,6 +6155,15 @@ void __init ftrace_init(void)
> > if (ret)
> > goto failed;
> >
> > + hash_entry_cache = kmem_cache_create("ftrace-hash",
> > + sizeof(struct ftrace_func_entry),
> > + sizeof(struct ftrace_func_entry),
> > + 0, NULL);
> > + if (!hash_entry_cache) {
> > + pr_err("failed to create ftrace hash entry cache\n");
> > + goto failed;
> > + }
>
> Wait what; you already have then in the binary image, now you're
> allocating extra memory for each of them?
>
No, here we only allocate ftrace hash entries. The prototype data is not copied.
The entry->priv points to prototype data in binary.
> Did you look at what ORC does? Is the binary search really not fast
> enough?
For ftrace, binary search is not enough. Just like the hash tables
(ftrace_graph_notrace_hash, ftrace_graph_hash) we already have which is used to
filter traced functions.
--
Cheers,
Changbin Du
On Sun, 25 Aug 2019 21:23:20 +0800
Changbin Du <[email protected]> wrote:
> Move ftrace tools to its own directory. We will add another tool later.
>
> Cc: John F. Reiser <[email protected]>
> Signed-off-by: Changbin Du <[email protected]>
> ---
> scripts/.gitignore | 1 -
> scripts/Makefile | 2 +-
> scripts/Makefile.build | 10 +++++-----
> scripts/ftrace/.gitignore | 4 ++++
> scripts/ftrace/Makefile | 4 ++++
> scripts/{ => ftrace}/recordmcount.c | 0
> scripts/{ => ftrace}/recordmcount.h | 0
> scripts/{ => ftrace}/recordmcount.pl | 0
> 8 files changed, 14 insertions(+), 7 deletions(-)
> create mode 100644 scripts/ftrace/.gitignore
> create mode 100644 scripts/ftrace/Makefile
> rename scripts/{ => ftrace}/recordmcount.c (100%)
> rename scripts/{ => ftrace}/recordmcount.h (100%)
> rename scripts/{ => ftrace}/recordmcount.pl (100%)
> mode change 100755 => 100644
Note, we are in the process of merging recordmcount with objtool. It
would be better to continue from that work.
http://lkml.kernel.org/r/2767f55f4a5fbf30ba0635aed7a9c5ee92ac07dd.1563992889.git.mhelsley@vmware.com
-- Steve
>
> diff --git a/scripts/.gitignore b/scripts/.gitignore
> index 17f8cef88fa8..1b5b5d595d80 100644
> --- a/scripts/.gitignore
> +++ b/scripts/.gitignore
> @@ -6,7 +6,6 @@ conmakehash
> kallsyms
> pnmtologo
> unifdef
> -recordmcount
> sortextable
> asn1_compiler
> extract-cert
> diff --git a/scripts/Makefile b/scripts/Makefile
> index 16bcb8087899..d5992def49a8 100644
> --- a/scripts/Makefile
> +++ b/scripts/Makefile
> @@ -14,7 +14,6 @@ hostprogs-$(CONFIG_BUILD_BIN2C) += bin2c
> hostprogs-$(CONFIG_KALLSYMS) += kallsyms
> hostprogs-$(CONFIG_LOGO) += pnmtologo
> hostprogs-$(CONFIG_VT) += conmakehash
> -hostprogs-$(BUILD_C_RECORDMCOUNT) += recordmcount
> hostprogs-$(CONFIG_BUILDTIME_EXTABLE_SORT) += sortextable
> hostprogs-$(CONFIG_ASN1) += asn1_compiler
> hostprogs-$(CONFIG_MODULE_SIG) += sign-file
> @@ -34,6 +33,7 @@ hostprogs-y += unifdef
> subdir-$(CONFIG_GCC_PLUGINS) += gcc-plugins
> subdir-$(CONFIG_MODVERSIONS) += genksyms
> subdir-$(CONFIG_SECURITY_SELINUX) += selinux
> +subdir-$(CONFIG_FTRACE) += ftrace
>
> # Let clean descend into subdirs
> subdir- += basic dtc gdb kconfig mod package
> diff --git a/scripts/Makefile.build b/scripts/Makefile.build
> index 2f66ed388d1c..67558983c518 100644
> --- a/scripts/Makefile.build
> +++ b/scripts/Makefile.build
> @@ -188,18 +188,18 @@ endif
> # files, including recordmcount.
> sub_cmd_record_mcount = \
> if [ $(@) != "scripts/mod/empty.o" ]; then \
> - $(objtree)/scripts/recordmcount $(RECORDMCOUNT_FLAGS) "$(@)"; \
> + $(objtree)/scripts/ftrace/recordmcount $(RECORDMCOUNT_FLAGS) "$(@)"; \
> fi;
> -recordmcount_source := $(srctree)/scripts/recordmcount.c \
> - $(srctree)/scripts/recordmcount.h
> +recordmcount_source := $(srctree)/scripts/ftrace/recordmcount.c \
> + $(srctree)/scripts/ftrace/recordmcount.h
> else
> -sub_cmd_record_mcount = perl $(srctree)/scripts/recordmcount.pl "$(ARCH)" \
> +sub_cmd_record_mcount = perl $(srctree)/scripts/ftrace/recordmcount.pl "$(ARCH)" \
> "$(if $(CONFIG_CPU_BIG_ENDIAN),big,little)" \
> "$(if $(CONFIG_64BIT),64,32)" \
> "$(OBJDUMP)" "$(OBJCOPY)" "$(CC) $(KBUILD_CPPFLAGS) $(KBUILD_CFLAGS)" \
> "$(LD) $(KBUILD_LDFLAGS)" "$(NM)" "$(RM)" "$(MV)" \
> "$(if $(part-of-module),1,0)" "$(@)";
> -recordmcount_source := $(srctree)/scripts/recordmcount.pl
> +recordmcount_source := $(srctree)/scripts/ftrace/recordmcount.pl
> endif # BUILD_C_RECORDMCOUNT
> cmd_record_mcount = $(if $(findstring $(strip $(CC_FLAGS_FTRACE)),$(_c_flags)), \
> $(sub_cmd_record_mcount))
> diff --git a/scripts/ftrace/.gitignore b/scripts/ftrace/.gitignore
> new file mode 100644
> index 000000000000..54d582c8faad
> --- /dev/null
> +++ b/scripts/ftrace/.gitignore
> @@ -0,0 +1,4 @@
> +#
> +# Generated files
> +#
> +recordmcount
> diff --git a/scripts/ftrace/Makefile b/scripts/ftrace/Makefile
> new file mode 100644
> index 000000000000..6797e51473e5
> --- /dev/null
> +++ b/scripts/ftrace/Makefile
> @@ -0,0 +1,4 @@
> +# SPDX-License-Identifier: GPL-2.0
> +
> +hostprogs-$(BUILD_C_RECORDMCOUNT) += recordmcount
> +always := $(hostprogs-y)
> diff --git a/scripts/recordmcount.c b/scripts/ftrace/recordmcount.c
> similarity index 100%
> rename from scripts/recordmcount.c
> rename to scripts/ftrace/recordmcount.c
> diff --git a/scripts/recordmcount.h b/scripts/ftrace/recordmcount.h
> similarity index 100%
> rename from scripts/recordmcount.h
> rename to scripts/ftrace/recordmcount.h
> diff --git a/scripts/recordmcount.pl b/scripts/ftrace/recordmcount.pl
> old mode 100755
> new mode 100644
> similarity index 100%
> rename from scripts/recordmcount.pl
> rename to scripts/ftrace/recordmcount.pl
On Mon, Aug 26, 2019 at 06:44:44PM -0400, Steven Rostedt wrote:
> On Sun, 25 Aug 2019 21:23:20 +0800
> Changbin Du <[email protected]> wrote:
>
> > Move ftrace tools to its own directory. We will add another tool later.
> >
> > Cc: John F. Reiser <[email protected]>
> > Signed-off-by: Changbin Du <[email protected]>
> > ---
> > scripts/.gitignore | 1 -
> > scripts/Makefile | 2 +-
> > scripts/Makefile.build | 10 +++++-----
> > scripts/ftrace/.gitignore | 4 ++++
> > scripts/ftrace/Makefile | 4 ++++
> > scripts/{ => ftrace}/recordmcount.c | 0
> > scripts/{ => ftrace}/recordmcount.h | 0
> > scripts/{ => ftrace}/recordmcount.pl | 0
> > 8 files changed, 14 insertions(+), 7 deletions(-)
> > create mode 100644 scripts/ftrace/.gitignore
> > create mode 100644 scripts/ftrace/Makefile
> > rename scripts/{ => ftrace}/recordmcount.c (100%)
> > rename scripts/{ => ftrace}/recordmcount.h (100%)
> > rename scripts/{ => ftrace}/recordmcount.pl (100%)
> > mode change 100755 => 100644
>
> Note, we are in the process of merging recordmcount with objtool. It
> would be better to continue from that work.
>
> http://lkml.kernel.org/r/2767f55f4a5fbf30ba0635aed7a9c5ee92ac07dd.1563992889.git.mhelsley@vmware.com
>
> -- Steve
Thanks for reminding. Let me check if prototype tool can merge into
objtool easily after above work.
--
Cheers,
Changbin Du