Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752220AbbEJDC5 (ORCPT ); Sat, 9 May 2015 23:02:57 -0400 Received: from mail9.hitachi.co.jp ([133.145.228.44]:34133 "EHLO mail9.hitachi.co.jp" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752135AbbEJDCz (ORCPT ); Sat, 9 May 2015 23:02:55 -0400 Message-ID: <554ECA58.8030308@hitachi.com> Date: Sun, 10 May 2015 12:02:48 +0900 From: Masami Hiramatsu Organization: Hitachi, Ltd., Japan User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: Arnaldo Carvalho de Melo CC: ananth@in.ibm.com, Peter Zijlstra , hemant@linux.vnet.ibm.com, Linux Kernel Mailing List , David Ahern , namhyung@kernel.org, Jiri Olsa , Ingo Molnar Subject: Re: Re: [PATCH perf/core v2 0/4] perf-probe code cleanup and support wildcard for probe points References: <20150508010326.24812.56013.stgit@localhost.localdomain> <20150508193839.GQ7862@kernel.org> In-Reply-To: <20150508193839.GQ7862@kernel.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 11959 Lines: 182 On 2015/05/09 4:38, Arnaldo Carvalho de Melo wrote: > Em Fri, May 08, 2015 at 10:03:26AM +0900, Masami Hiramatsu escreveu: >> Hi, >> >> Here is a patches for wild card support. I've added two patches to >> reduce API parameters by introducing probe_conf. This simplifies >> --no-inlines option. (Thanks Arnaldo!) >> >> Changes from v1: >> - Update for the latest perf/core >> - Use perf_probe_event.target instead of passing it as an independent >> parameter [1/4] >> - Introduce probe_conf for configuring parameters of probe-finder and >> probe-event [2/4] >> >> Thank you, >> >> --- >> >> Masami Hiramatsu (4): >> perf probe: Use perf_probe_event.target instead of passing as an argument >> perf probe: Introduce probe_conf global configs >> perf-probe: Add --no-inlines option to avoid searching inline functions >> perf-probe: Support glob wildcards for function name > > Excellent stuff! Thanks, all tested. Thank you for this good example! :) Using this with perf-trace is certainly nice usage. Thank you! > > And here is the example I used to test it plus other goodies: > > System wide syscall tracing + kmalloc* kernel functions with its parameters, > filtering the pid of the sshd session to the test machine, to avoid a feedback > loop: > > [root@ssdandy ~]# perf probe --no-inlines --add 'kmalloc* $params' > Added new events: > probe:kmalloc_order (on kmalloc* with $params) > probe:kmalloc_order_trace (on kmalloc* with $params) > probe:kmalloc_slab (on kmalloc* with $params) > probe:kmalloc_large_node (on kmalloc* with $params) > > You can now use it in all perf tools, such as: > > perf record -e probe:kmalloc_large_node -aR sleep 1 > > [root@ssdandy ~]# perf trace --ev probe:kmalloc* --filter-pids 27206 > 0.000 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 0.010 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) > 0.343 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x98 flags=0x10) > 73.254 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 73.259 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) > 89.510 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 192.295 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 205.520 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 205.524 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) > 260.034 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 260.039 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x300 flags=0x10220) > 307.779 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 307.784 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) > 362.450 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 362.455 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x300 flags=0x10220) > 380.507 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 380.511 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) > 409.663 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 409.668 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) > 464.868 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 464.874 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x300 flags=0x10220) > 499.303 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 512.083 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 512.088 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) > 585.349 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 585.354 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) > 614.513 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 614.518 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) > 669.706 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 669.710 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x300 flags=0x10220) > 716.927 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 716.932 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) > 790.181 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 790.185 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) > 818.857 ( 0.000 ms): tuned/837 ... [continued]: select()) = 0 Timeout > 819.568 ( 0.651 ms): tuned/837 select(tvp: 0x7fb3bb1f9e50 ) ... > 819.568 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 819.573 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) > 879.453 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 879.458 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x300 flags=0x10220) > 922.236 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 924.408 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 924.412 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) > 1024.172 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 1024.177 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) > 1079.381 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 1079.386 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x300 flags=0x10220) > 1097.435 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 1097.439 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) > 1115.571 ( 0.000 ms): avahi-daemon/745 ... [continued]: poll()) = 0 Timeout > 1115.592 ( 0.006 ms): avahi-daemon/745 write(fd: 7, buf: 0x7ffc066e35bf, count: 1) = 1 > 1115.596 ( 0.002 ms): avahi-daemon/745 write(fd: 7, buf: 0x7ffc066e35bf, count: 1) = 1 > 1115.602 ( 0.003 ms): avahi-daemon/745 read(fd: 6, buf: 0x7ffc066e35e0, count: 10) = 2 > 1115.607 ( 0.003 ms): avahi-daemon/745 poll(ufds: 0xaffa40, nfds: 8 ) = 0 Timeout > 1115.612 ( 0.002 ms): avahi-daemon/745 write(fd: 7, buf: 0x7ffc066e35bf, count: 1) = 1 > 1115.616 ( 0.002 ms): avahi-daemon/745 write(fd: 7, buf: 0x7ffc066e35bf, count: 1) = 1 > 1115.620 ( 0.002 ms): avahi-daemon/745 read(fd: 6, buf: 0x7ffc066e35e0, count: 10) = 2 > 1126.606 (10.985 ms): avahi-daemon/745 poll(ufds: 0xaffa40, nfds: 8, timeout_msecs: 647 ) ... > 1126.606 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 1126.612 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) > 1199.853 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 1199.858 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) > 1216.042 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 1229.103 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 1229.107 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) > 1302.275 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 1302.279 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) > 1318.900 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 1331.434 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 1331.438 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) > 1360.755 ( 0.000 ms): lsmd/746 ... [continued]: select()) = 0 Timeout > 1360.764 ( 0.003 ms): lsmd/746 waitid(infop: 0x7ffeee2cdc90, options: 5 ) = -10 > 1404.691 (43.920 ms): lsmd/746 select(n: 6, inp: 0x7ffeee2cdd50, tvp: 0x7ffeee2cdd40 ) ... > 1404.691 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 1404.696 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) > 1434.623 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 1434.628 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) > 1523.298 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 1536.268 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 1536.273 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) > 1609.524 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 1609.528 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) > 1638.684 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 1638.688 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) > 1711.943 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 1711.948 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) > 1741.098 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 1741.102 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) > 1763.390 (647.769 ms): avahi-daemon/745 ... [continued]: poll()) = 0 Timeout > 1763.401 ( 0.003 ms): avahi-daemon/745 write(fd: 7, buf: 0x7ffc066e35bf, count: 1) = 1 > 1763.404 ( 0.002 ms): avahi-daemon/745 write(fd: 7, buf: 0x7ffc066e35bf, count: 1) = 1 > 1763.408 ( 0.002 ms): avahi-daemon/745 read(fd: 6, buf: 0x7ffc066e35e0, count: 10) = 2 > 1814.362 (50.952 ms): avahi-daemon/745 poll(ufds: 0xaffa40, nfds: 8, timeout_msecs: 204419 ) ... > 1814.362 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 1814.367 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) > 1820.100 (1001.183 ms): tuned/837 ... [continued]: select()) = 0 Timeout > 1843.529 (23.407 ms): tuned/837 select(tvp: 0x7fb3bb1f9e50 ) ... > 1843.529 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > 1843.533 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) > 1945.941 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) > ^C > [root@ssdandy ~]# perf probe -d '*' > Removed event: probe:kmalloc_large_node > Removed event: probe:kmalloc_order > Removed event: probe:kmalloc_order_trace > Removed event: probe:kmalloc_slab > [root@ssdandy ~]# > > > :-) > > - Arnaldo > -- > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ > -- Masami HIRAMATSU Linux Technology Research Center, System Productivity Research Dept. Center for Technology Innovation - Systems Engineering Hitachi, Ltd., Research & Development Group E-mail: masami.hiramatsu.pt@hitachi.com -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/