Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S965948AbcKOEIS (ORCPT ); Mon, 14 Nov 2016 23:08:18 -0500 Received: from szxga02-in.huawei.com ([119.145.14.65]:54709 "EHLO szxga02-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755250AbcKOEIJ (ORCPT ); Mon, 14 Nov 2016 23:08:09 -0500 From: Wang Nan To: , CC: , , , , Wang Nan , Jiri Olsa Subject: [PATCH 00/34] perf clang: Builtin clang and perfhook support Date: Tue, 15 Nov 2016 04:05:43 +0000 Message-ID: <20161115040617.69788-1-wangnan0@huawei.com> X-Mailer: git-send-email 2.10.1 MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [10.107.193.248] X-CFilter-Loop: Reflected Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 9452 Lines: 245 This is version 2 of perf builtin clang patch series. Compare to v1, add an exciting feature: jit compiling perf hook functions. This features allows script writer report result through BPF map in a customized way. At the end of this cover letter lists an example shows how to capture and report a syscall histogram. This patchset is based on current perf/core. In this patchset: Patch 1 - 4 are bugfixes left in my local tree. Patch 5 - 7 are preparation in libbpf. Patch 8 - 9 introduce perf hook to perf. Patch 10 - 21 are patches support builtin clang. Some of then are already collected by Arnaldo in tmp.perf/builtin-clang. They are slightly adjusted in this v2 series. Patch 22 - 29 add JIT compiling to builting clang and add them to perf hook. Patch 30 - 34 are easy of use improvements. 1) builtin clang defines macros and helpers by default using builtin include headers. Default headers can be turned off by -UBUILTIN_CLANG_DEFAULT_INCLUDE. 2) allow JITted perf hooks access more functions in libc. Example: build syscall histogram, exclude syscalls issued by perf itself. Please note following improvements: 1. Don't need define many bpf helpers by hand. See bpf_map_lookup_elem and bpf_get_current_pid_tgid. 2. See how pid of git is passed to BPF script attached to raw_syscalls:sys_enter. $ cat ./count_syscalls.c typedef unsigned long u64; #define BPF_MAP_TYPE_HASH 1 #define BPF_MAP_TYPE_ARRAY 2 enum GVAL { G_perf_pid, NR_GVALS }; struct bpf_map_def SEC("maps") GVALS = { .type = BPF_MAP_TYPE_ARRAY, .key_size = sizeof(int), .value_size = sizeof(u64), .max_entries = NR_GVALS, }; struct bpf_map_def SEC("maps") syscall_counter = { .type = BPF_MAP_TYPE_HASH, .key_size = sizeof(u64), .value_size = sizeof(u64), .max_entries = 512, }; SEC("raw_syscalls:sys_enter") int func(void *ctx) { int key = G_perf_pid; u64 id = *((u64 *)(ctx + 8)); int self_pid = bpf_get_current_pid_tgid() & 0xffffffff; int *perf_pid = bpf_map_lookup_elem(&GVALS, &key); u64 *counter; if (!perf_pid) return 0; if (*perf_pid == self_pid) return 0; counter = bpf_map_lookup_elem(&syscall_counter, &id); if (!counter) { u64 value = 1; bpf_map_update_elem(&syscall_counter, &id, &value, 0); return 0; } __sync_fetch_and_add(counter, 1); return 0; } SEC("perfhook:record_start") void record_start(void *ctx) { int perf_pid = getpid(), key = G_perf_pid; printf("Start count, perfpid=%d\n", perf_pid); jit_helper__map_update_elem(ctx, &GVALS, &key, &perf_pid, 0); } SEC("perfhook:record_end") void record_end(void *ctx) { u64 key = -1, value; while (!jit_helper__map_get_next_key(ctx, &syscall_counter, &key, &key)) { jit_helper__map_lookup_elem(ctx, &syscall_counter, &key, &value); printf("syscall %ld\tcount: %ld\n", (long)key, (long)value); } } char _license[] SEC("license") = "GPL"; int _version SEC("version") = LINUX_VERSION_CODE; $ sudo -s # ulimit -l unlimited # perf record -e ./count_syscalls.c echo "Haha" Start count, perfpid=25209 Haha [ perf record: Woken up 1 times to write data ] syscall 8 count: 6 syscall 11 count: 1 syscall 4 count: 6 syscall 21 count: 1 syscall 5 count: 3 syscall 231 count: 1 syscall 45 count: 3 syscall 0 count: 24 syscall 257 count: 1 syscall 59 count: 4 syscall 23 count: 9 syscall 78 count: 2 syscall 41 count: 4 syscall 72 count: 8 syscall 10 count: 3 syscall 321 count: 1 syscall 298 count: 7 syscall 16 count: 21 syscall 9 count: 16 syscall 1 count: 114 syscall 12 count: 3 syscall 14 count: 35 syscall 158 count: 1 syscall 2 count: 15 syscall 7 count: 18 syscall 3 count: 11 [ perf record: Captured and wrote 0.011 MB perf.data ] Eric Leblond (1): tools lib bpf: fix maps resolution Wang Nan (33): perf tools: Fix kernel version error in ubuntu perf record: Fix segfault when running with suid and kptr_restrict is 1 tools perf: Add missing struct defeinition in probe_event.h tools lib bpf: Add missing bpf map functions tools lib bpf: Add private field for bpf_object tools lib bpf: Retrive bpf_map through offset of bpf_map_def perf tools: Introduce perf hooks perf tools: Pass context to perf hook functions perf llvm: Extract helpers in llvm-utils.c tools build: Add feature detection for LLVM tools build: Add feature detection for clang perf build: Add clang and llvm compile and linking support perf clang: Add builtin clang support ant test case perf clang: Use real file system for #include perf clang: Allow passing CFLAGS to builtin clang perf clang: Update test case to use real BPF script perf clang: Support compile IR to BPF object and add testcase perf clang: Compile BPF script use builtin clang support perf clang: Pass full path to builtin clang perf clang: Pass CFLAGS to builtin clang perf clang jit: Wrap llvm::Module using PerfModule perf clang jit: Insignt BPF and JIT functions in a Module perf clang jit: add PerfModule::doJIT to JIT perfhook functions. perf clang jit: Export functions for jitted code perf clang jit: Actually JIT and hook in bpf loader perf clang jit: Collect the lowest address in maps section as map_base perf clang jit: Access BPF map perf clang jit: Allow jitted perf hook access BPF maps perf clang: Link BPF functions declaration into perf perf clang: Declare BPF functions for BPF scripts automatically perf clang: Include helpers to BPF scripts perf clang builtin: Define hook helpers by default perf clang git: Export getpid() to perf hook tools/build/feature/Makefile | 18 + tools/build/feature/test-clang.cpp | 21 ++ tools/build/feature/test-llvm.cpp | 8 + tools/lib/bpf/bpf.c | 56 +++ tools/lib/bpf/bpf.h | 7 + tools/lib/bpf/libbpf.c | 177 ++++++--- tools/lib/bpf/libbpf.h | 13 + tools/perf/Makefile.config | 62 +++- tools/perf/Makefile.perf | 23 +- tools/perf/builtin-record.c | 11 + tools/perf/tests/Build | 4 +- tools/perf/tests/bpf-script-example.c | 30 +- tools/perf/tests/bpf-script-test-kbuild.c | 2 + tools/perf/tests/bpf-script-test-prologue.c | 6 +- tools/perf/tests/bpf-script-test-relocation.c | 17 +- tools/perf/tests/builtin-test.c | 13 + tools/perf/tests/clang.c | 50 +++ tools/perf/tests/llvm.h | 7 + tools/perf/tests/make | 2 + tools/perf/tests/perf-hooks.c | 48 +++ tools/perf/tests/tests.h | 4 + tools/perf/util/Build | 5 + tools/perf/util/bpf-loader.c | 102 +++++- tools/perf/util/bpf-loader.h | 19 + tools/perf/util/c++/Build | 4 + tools/perf/util/c++/bpf-funcs-str.c | 228 ++++++++++++ tools/perf/util/c++/bpf-helper-str.c | 24 ++ tools/perf/util/c++/clang-bpf-includes.h | 13 + tools/perf/util/c++/clang-c.h | 63 ++++ tools/perf/util/c++/clang-test.cpp | 99 +++++ tools/perf/util/c++/clang.cpp | 498 ++++++++++++++++++++++++++ tools/perf/util/c++/clang.h | 61 ++++ tools/perf/util/jit-helpers.c | 60 ++++ tools/perf/util/jit-helpers.h | 30 ++ tools/perf/util/llvm-utils.c | 76 +++- tools/perf/util/llvm-utils.h | 15 +- tools/perf/util/perf-hooks-list.h | 3 + tools/perf/util/perf-hooks.c | 88 +++++ tools/perf/util/perf-hooks.h | 39 ++ tools/perf/util/probe-event.h | 2 + tools/perf/util/symbol.c | 2 +- tools/perf/util/util-cxx.h | 26 ++ tools/perf/util/util.c | 55 ++- 43 files changed, 1991 insertions(+), 100 deletions(-) create mode 100644 tools/build/feature/test-clang.cpp create mode 100644 tools/build/feature/test-llvm.cpp create mode 100644 tools/perf/tests/clang.c create mode 100644 tools/perf/tests/perf-hooks.c create mode 100644 tools/perf/util/c++/Build create mode 100644 tools/perf/util/c++/bpf-funcs-str.c create mode 100644 tools/perf/util/c++/bpf-helper-str.c create mode 100644 tools/perf/util/c++/clang-bpf-includes.h create mode 100644 tools/perf/util/c++/clang-c.h create mode 100644 tools/perf/util/c++/clang-test.cpp create mode 100644 tools/perf/util/c++/clang.cpp create mode 100644 tools/perf/util/c++/clang.h create mode 100644 tools/perf/util/jit-helpers.c create mode 100644 tools/perf/util/jit-helpers.h create mode 100644 tools/perf/util/perf-hooks-list.h create mode 100644 tools/perf/util/perf-hooks.c create mode 100644 tools/perf/util/perf-hooks.h create mode 100644 tools/perf/util/util-cxx.h Signed-off-by: Wang Nan Cc: Arnaldo Carvalho de Melo Cc: Alexei Starovoitov Cc: He Kuang Cc: Jiri Olsa Cc: Zefan Li Cc: pi3orama@163.com -- 2.10.1