Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753038AbbEHTir (ORCPT ); Fri, 8 May 2015 15:38:47 -0400 Received: from mail.kernel.org ([198.145.29.136]:53110 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751435AbbEHTio (ORCPT ); Fri, 8 May 2015 15:38:44 -0400 Date: Fri, 8 May 2015 16:38:39 -0300 From: Arnaldo Carvalho de Melo To: Masami Hiramatsu Cc: ananth@in.ibm.com, Peter Zijlstra , hemant@linux.vnet.ibm.com, Linux Kernel Mailing List , David Ahern , namhyung@kernel.org, Jiri Olsa , Ingo Molnar Subject: Re: [PATCH perf/core v2 0/4] perf-probe code cleanup and support wildcard for probe points Message-ID: <20150508193839.GQ7862@kernel.org> References: <20150508010326.24812.56013.stgit@localhost.localdomain> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150508010326.24812.56013.stgit@localhost.localdomain> X-Url: http://acmel.wordpress.com User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 11031 Lines: 161 Em Fri, May 08, 2015 at 10:03:26AM +0900, Masami Hiramatsu escreveu: > Hi, > > Here is a patches for wild card support. I've added two patches to > reduce API parameters by introducing probe_conf. This simplifies > --no-inlines option. (Thanks Arnaldo!) > > Changes from v1: > - Update for the latest perf/core > - Use perf_probe_event.target instead of passing it as an independent > parameter [1/4] > - Introduce probe_conf for configuring parameters of probe-finder and > probe-event [2/4] > > Thank you, > > --- > > Masami Hiramatsu (4): > perf probe: Use perf_probe_event.target instead of passing as an argument > perf probe: Introduce probe_conf global configs > perf-probe: Add --no-inlines option to avoid searching inline functions > perf-probe: Support glob wildcards for function name Excellent stuff! Thanks, all tested. And here is the example I used to test it plus other goodies: System wide syscall tracing + kmalloc* kernel functions with its parameters, filtering the pid of the sshd session to the test machine, to avoid a feedback loop: [root@ssdandy ~]# perf probe --no-inlines --add 'kmalloc* $params' Added new events: probe:kmalloc_order (on kmalloc* with $params) probe:kmalloc_order_trace (on kmalloc* with $params) probe:kmalloc_slab (on kmalloc* with $params) probe:kmalloc_large_node (on kmalloc* with $params) You can now use it in all perf tools, such as: perf record -e probe:kmalloc_large_node -aR sleep 1 [root@ssdandy ~]# perf trace --ev probe:kmalloc* --filter-pids 27206 0.000 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 0.010 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) 0.343 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x98 flags=0x10) 73.254 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 73.259 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) 89.510 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 192.295 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 205.520 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 205.524 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) 260.034 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 260.039 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x300 flags=0x10220) 307.779 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 307.784 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) 362.450 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 362.455 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x300 flags=0x10220) 380.507 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 380.511 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) 409.663 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 409.668 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) 464.868 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 464.874 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x300 flags=0x10220) 499.303 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 512.083 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 512.088 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) 585.349 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 585.354 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) 614.513 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 614.518 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) 669.706 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 669.710 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x300 flags=0x10220) 716.927 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 716.932 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) 790.181 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 790.185 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) 818.857 ( 0.000 ms): tuned/837 ... [continued]: select()) = 0 Timeout 819.568 ( 0.651 ms): tuned/837 select(tvp: 0x7fb3bb1f9e50 ) ... 819.568 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 819.573 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) 879.453 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 879.458 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x300 flags=0x10220) 922.236 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 924.408 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 924.412 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) 1024.172 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 1024.177 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) 1079.381 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 1079.386 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x300 flags=0x10220) 1097.435 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 1097.439 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) 1115.571 ( 0.000 ms): avahi-daemon/745 ... [continued]: poll()) = 0 Timeout 1115.592 ( 0.006 ms): avahi-daemon/745 write(fd: 7, buf: 0x7ffc066e35bf, count: 1) = 1 1115.596 ( 0.002 ms): avahi-daemon/745 write(fd: 7, buf: 0x7ffc066e35bf, count: 1) = 1 1115.602 ( 0.003 ms): avahi-daemon/745 read(fd: 6, buf: 0x7ffc066e35e0, count: 10) = 2 1115.607 ( 0.003 ms): avahi-daemon/745 poll(ufds: 0xaffa40, nfds: 8 ) = 0 Timeout 1115.612 ( 0.002 ms): avahi-daemon/745 write(fd: 7, buf: 0x7ffc066e35bf, count: 1) = 1 1115.616 ( 0.002 ms): avahi-daemon/745 write(fd: 7, buf: 0x7ffc066e35bf, count: 1) = 1 1115.620 ( 0.002 ms): avahi-daemon/745 read(fd: 6, buf: 0x7ffc066e35e0, count: 10) = 2 1126.606 (10.985 ms): avahi-daemon/745 poll(ufds: 0xaffa40, nfds: 8, timeout_msecs: 647 ) ... 1126.606 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 1126.612 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) 1199.853 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 1199.858 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) 1216.042 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 1229.103 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 1229.107 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) 1302.275 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 1302.279 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) 1318.900 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 1331.434 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 1331.438 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) 1360.755 ( 0.000 ms): lsmd/746 ... [continued]: select()) = 0 Timeout 1360.764 ( 0.003 ms): lsmd/746 waitid(infop: 0x7ffeee2cdc90, options: 5 ) = -10 1404.691 (43.920 ms): lsmd/746 select(n: 6, inp: 0x7ffeee2cdd50, tvp: 0x7ffeee2cdd40 ) ... 1404.691 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 1404.696 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) 1434.623 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 1434.628 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) 1523.298 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 1536.268 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 1536.273 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) 1609.524 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 1609.528 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) 1638.684 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 1638.688 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) 1711.943 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 1711.948 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) 1741.098 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 1741.102 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) 1763.390 (647.769 ms): avahi-daemon/745 ... [continued]: poll()) = 0 Timeout 1763.401 ( 0.003 ms): avahi-daemon/745 write(fd: 7, buf: 0x7ffc066e35bf, count: 1) = 1 1763.404 ( 0.002 ms): avahi-daemon/745 write(fd: 7, buf: 0x7ffc066e35bf, count: 1) = 1 1763.408 ( 0.002 ms): avahi-daemon/745 read(fd: 6, buf: 0x7ffc066e35e0, count: 10) = 2 1814.362 (50.952 ms): avahi-daemon/745 poll(ufds: 0xaffa40, nfds: 8, timeout_msecs: 204419 ) ... 1814.362 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 1814.367 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) 1820.100 (1001.183 ms): tuned/837 ... [continued]: select()) = 0 Timeout 1843.529 (23.407 ms): tuned/837 select(tvp: 0x7fb3bb1f9e50 ) ... 1843.529 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) 1843.533 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x2c0 flags=0x10220) 1945.941 ( ): probe:kmalloc_slab:(ffffffff81198ab0) size=0x1c0 flags=0x10220) ^C [root@ssdandy ~]# perf probe -d '*' Removed event: probe:kmalloc_large_node Removed event: probe:kmalloc_order Removed event: probe:kmalloc_order_trace Removed event: probe:kmalloc_slab [root@ssdandy ~]# :-) - Arnaldo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/