Received: by 2002:a05:6a10:f347:0:0:0:0 with SMTP id d7csp422087pxu; Tue, 5 Jan 2021 15:12:40 -0800 (PST) X-Google-Smtp-Source: ABdhPJz7S4DDezHc9jdyZ5av5YCvK+C9nDjx5POGd2igr3/DflX4clqOur754ML7R3FajJzw/1fA X-Received: by 2002:a17:906:af79:: with SMTP id os25mr1105175ejb.275.1609888360227; Tue, 05 Jan 2021 15:12:40 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1609888360; cv=none; d=google.com; s=arc-20160816; b=HMoSay6U5eZFpRwQATNvqgYkI+pMdEspnrAfREeZhzJU/hM9PmX28sQndibsbmCxs2 Trd01QZkIuXSMYFJFGm0pVYWTcQ3BtIizs0XV0KDytKBWqM/pg51oXmt7DVknp2j1bnd sb2VcO16FCZgVE2A419dJF23s/nxltiVu+mp1G1yazsb94qtLi6FKdJ8FIukDub6uK9g KUc5D+qfOSL/IJar0GohrrSqZvJjveDPe6YfGAHNJIVvVvECE4Bs1HiB1As8+a+ateU5 xfCPIFyW59eoJHYUOsENSuzhiYRw9HSz0MG7xPohC/0XajN9CUZaygdFxTndouoi2nWq D74g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:message-id:in-reply-to :subject:cc:to:from:date:dkim-signature; bh=EVuLtaEoqyVF8xI9Fk+aPgVK+nQKGqBw67gRTMEOd70=; b=gSkm8a5TTrGzc306uzOMxrZdnSRMv2m03px17Jja7vszunjxUjfKR2uxxnqbaSSKXv EEHK8PEOt1AfsA35QflGLpzRPlYdgSCZmut1DnoWPr7dg+RPsepEGffhTaZTWsPj2ZiC d6LkjChy7Uwr4OmlpiXGeVMecKe82lCkzkEOLRC92ls2Kpt9jACGF2vlNVNFuIVsax79 DHMpUEQVRYNGRyumt4ty0wVYiVsiNfaZINgWYydFAFPbd8twhfU6OSDmJW/xtMTpfd05 otpDjmXDDXH/K7ER5WGgspbSLD4ZoXIsGhg/YEQJMJQlrliZR7+ZYdXl78sQN1eFk+yJ 1CbQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2020-01-29 header.b=0dlbbXl6; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id z5si267107ejj.425.2021.01.05.15.12.17; Tue, 05 Jan 2021 15:12:40 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2020-01-29 header.b=0dlbbXl6; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729214AbhAEWpl (ORCPT + 99 others); Tue, 5 Jan 2021 17:45:41 -0500 Received: from aserp2120.oracle.com ([141.146.126.78]:47320 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728058AbhAEWpk (ORCPT ); Tue, 5 Jan 2021 17:45:40 -0500 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 105MOr0A022106; Tue, 5 Jan 2021 22:44:41 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=date : from : to : cc : subject : in-reply-to : message-id : references : mime-version : content-type; s=corp-2020-01-29; bh=EVuLtaEoqyVF8xI9Fk+aPgVK+nQKGqBw67gRTMEOd70=; b=0dlbbXl6u7/DpS6PcTRxA+F2SUcyX7kazCvwbWmflGJLlP4XOrhp/pkCNu579jTxJ3DF FbWthhLCqItmTvZQvOMYsKfPwlVnF///LgLEHHQaGYDzuFMaCdTuQRTsHm2y1PL702Yl cJTEsx2ynmpC9v4lWp4VUu5W4O+xffmwWPkT64PPUCN1/zR1aeOSz61jwBJ+y03TKoEa UBwYMQIdLuNGe4F1muPmQrQMGQcH+f+12HjN+zg3MNP9KE1vXWHrWLbkMqAmxtizYQ89 Y/fHK1fbjVZsjurlCdgY4My5jmcLVBdXxMo6znt186TXFEwR5pVTcZwlIoc5pkhM/wxa ew== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by aserp2120.oracle.com with ESMTP id 35tgsku31g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Tue, 05 Jan 2021 22:44:41 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 105MPLL7186292; Tue, 5 Jan 2021 22:44:40 GMT Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by userp3030.oracle.com with ESMTP id 35vct6funb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 05 Jan 2021 22:44:40 +0000 Received: from abhmp0013.oracle.com (abhmp0013.oracle.com [141.146.116.19]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id 105Micqj022028; Tue, 5 Jan 2021 22:44:39 GMT Received: from dhcp-10-175-187-168.vpn.oracle.com (/10.175.187.168) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 05 Jan 2021 22:44:38 +0000 Date: Tue, 5 Jan 2021 22:44:29 +0000 (GMT) From: Alan Maguire X-X-Sender: alan@localhost To: Cong Wang cc: Alan Maguire , Alexei Starovoitov , Daniel Borkmann , andrii@kernel.org, Martin KaFai Lau , Song Liu , yhs@fb.com, John Fastabend , kpsingh@kernel.org, Nathan Chancellor , Nick Desaulniers , =?ISO-8859-15?Q?Toke_H=F8iland-J=F8rgensen?= , jean-philippe@linaro.org, bpf , Linux Kernel Network Developers , LKML Subject: Re: [RFC PATCH bpf-next] ksnoop: kernel argument/return value tracing/display using BTF In-Reply-To: Message-ID: References: <1609773991-10509-1-git-send-email-alan.maguire@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9855 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 malwarescore=0 phishscore=0 bulkscore=0 mlxlogscore=999 suspectscore=0 mlxscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101050129 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9855 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 spamscore=0 malwarescore=0 phishscore=0 impostorscore=0 bulkscore=0 clxscore=1011 priorityscore=1501 lowpriorityscore=0 adultscore=0 suspectscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101050129 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 5 Jan 2021, Cong Wang wrote: > On Mon, Jan 4, 2021 at 7:29 AM Alan Maguire wrote: > > > > BPF Type Format (BTF) provides a description of kernel data structures > > and of the types kernel functions utilize as arguments and return values. > > > > A helper was recently added - bpf_snprintf_btf() - that uses that > > description to create a string representation of the data provided, > > using the BTF id of its type. For example to create a string > > representation of a "struct sk_buff", the pointer to the skb > > is provided along with the type id of "struct sk_buff". > > > > Here that functionality is utilized to support tracing kernel > > function entry and return using k[ret]probes. The "struct pt_regs" > > context can be used to derive arguments and return values, and > > when the user supplies a function name we > > > > - look it up in /proc/kallsyms to find its address/module > > - look it up in the BTF kernel data to get types of arguments > > and return value > > - store a map representation of the trace information, keyed by > > instruction pointer > > - on function entry/return we look up the map to retrieve the BTF > > ids of the arguments/return values and can call bpf_snprintf_btf() > > with these argument/return values along with the type ids to store > > a string representation in the map. > > - this is then sent via perf event to userspace where it can be > > displayed. > > > > ksnoop can be used to show function signatures; for example: > > This is definitely quite useful! > > Is it possible to integrate this with bpftrace? That would save people > from learning yet another tool. ;) > I'd imagine (and hope!) other tracing tools will do this, but right now the aim is to make the task of tracing kernel data structures simpler, so having a tool dedicated to just that can hopefully help those discussions. There's a bit more work to be done to simplify that task, for example implementing Alexei's suggestion to support pretty-printing of data structures using BTF in libbpf. My hope is that we can evolve this tool - or something like it - to the point where we can solve that one problem easily, and that other more general tracers can then make use of that solution. I probably should have made all of this clearer in the patch submission, sorry about that. Alan