Received: by 2002:a05:6512:2355:0:0:0:0 with SMTP id p21csp212498lfu; Wed, 30 Mar 2022 21:13:09 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy0dewRd0vK7Bg9JfjOrY+h1vKCM6tlXsKXkQJhTa1HQgNh7mQJ5JjT/aY96FM1uSaknHQO X-Received: by 2002:a05:6a00:2290:b0:4fa:a99e:2e21 with SMTP id f16-20020a056a00229000b004faa99e2e21mr3265956pfe.20.1648699989629; Wed, 30 Mar 2022 21:13:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1648699989; cv=none; d=google.com; s=arc-20160816; b=tBhPbZKciP7/MQLFQyTYbODoyyDh5DgSfz1ka1fFf0U3cbDBiTcJQNPq6T4gs+FPTj 4QjjyLMy1anC0g6J3bUofUAkg+wPsEoKFr0p9uBATjJe6TMTazAVpduqr7DIm0lBEwco O4amB3vYeY8enLe5sdZP2D9K/jFmzPOKxLVlXj5NTITFpDTSi3sarbBtLcJja7ly1i/k 0AGwp+Pg/xhM9iJzMaDgeiTF/EUz78QsdoNQyVAEtXA2QJHwpOGopRKhbj4I37spXlvv tn0R/KwV5B+a4yq7kWKoHohYwXdHOqT5KWZSUSwg8mtRl/MfeWpeljM1TRiGedukFWyn UZAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:subject:message-id:date:from:in-reply-to :references:mime-version:dkim-signature; bh=+CAeQ6WbO75b05CAGgEyrOcN/e8ROoiFTSHaPW0jBEU=; b=Zb9JCayMCdvfoAtO4BGWpmAsaEumGZ0O61YBwHAjMjJkSh0p6Iw4DylWyeUVV0cYuR rSPolJXVIMA2/YocqkPW1Aw32bNZrhMYkH82T2ABmOzvJoKbSdbRfGL8SFkMC5UjkAnb moNBv+/eSrRaAMfeOERv1iclEoco4TjGS5rCEey82n32G3A8pMgbwSgDRcfxeDR3Ghjn Ut0sCe46ngla19X1OrfcVbEztImQx79Jiy/ZaFf48oq9l+cxgskMcQEaMPL7qZq0BZq5 1WG/laniWRZBg5ZKdNgwUD1WtAwKGCm+ObT9VsyE336Vw4XiU6mu3TGY/ts6xssSgoAU 7LGA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=W9mnOdTJ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net. [2620:137:e000::1:18]) by mx.google.com with ESMTPS id u2-20020a056a00098200b004fdada2b1d5si1790051pfg.327.2022.03.30.21.13.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 30 Mar 2022 21:13:09 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) client-ip=2620:137:e000::1:18; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=W9mnOdTJ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id C1F9910857F; Wed, 30 Mar 2022 20:19:52 -0700 (PDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350363AbiC3Sij (ORCPT + 99 others); Wed, 30 Mar 2022 14:38:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44026 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350056AbiC3SZP (ORCPT ); Wed, 30 Mar 2022 14:25:15 -0400 Received: from mail-pj1-x102f.google.com (mail-pj1-x102f.google.com [IPv6:2607:f8b0:4864:20::102f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C649CF14; Wed, 30 Mar 2022 11:22:43 -0700 (PDT) Received: by mail-pj1-x102f.google.com with SMTP id h23-20020a17090a051700b001c9c1dd3acbso906841pjh.3; Wed, 30 Mar 2022 11:22:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=+CAeQ6WbO75b05CAGgEyrOcN/e8ROoiFTSHaPW0jBEU=; b=W9mnOdTJXi67DDGPDvM/CI7eE6yWDHJKjY77YMX5aYlBYvnffHWK17C8Cb1lUi3X1A /e4/iaifYgfXWjdsta3C3K8yoO090sUdApnAaAyf8mEADC6v2dOia2eMoQvZ0Ht2e30s BcCn00DtCu5QXB9vb2NCfra9khR8kcoZvht/Nawa9CwDcaHjSXxPuxnd8CUROC5c+pXC ImusgsO9px5vn0X+ZswQgxp9rdFYInYG5F5DpwBAnP7Py++PEjxDFRy9JeaYOSbxYw2s UGLeAPZkDrc6LVYWcbzWq7Ad0OtZdYSZXxScLaP2LrAIEoitWcnYweUky1uT+YHJ4V2E 5jYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=+CAeQ6WbO75b05CAGgEyrOcN/e8ROoiFTSHaPW0jBEU=; b=hGoPIQIl4zRvNbNwadMIwgckx62Umad3L7S/2n0jfeevt4oum4ugXioipAZEwFfSJG o5dsPwlJTGwrsBhO3arlJn1XSn7jZacc3+uCWGsLvR/4yIhri68+hc88gooSSz2oVulJ vH63xUieGbV9HqKf9Q/sovVNP7QoWIzz+EdabWACuKFiat38mWzMciSNyToeSEpergpT Ob8C//hm5+Oa//7voT4zx2SrX203iE2KcU3I/vdptgnJ6fMvKrOcMv/gy2gU1kegpPDi NJjJzjZtiS8VrWIMH1h3h3ZBMUH/8JVnYqr7mq60gbneshylYurNVVVMjCGPusefs+nP 8dOw== X-Gm-Message-State: AOAM531o3THBaGslMLS8XTDMrDLe5ReQXwKN2mGi5YFmbXXpIL9yKcm8 xyJH/zef0xtAS2IkE1hjJvnTEbeeCw+yetucsWjm5Ptw X-Received: by 2002:a17:902:ba83:b0:154:727e:5fc5 with SMTP id k3-20020a170902ba8300b00154727e5fc5mr633290pls.55.1648664563315; Wed, 30 Mar 2022 11:22:43 -0700 (PDT) MIME-Version: 1.0 References: <20220329181935.2183-1-beaub@linux.microsoft.com> <20220329201057.GA2549@kbox> <20220329231137.GA3357@kbox> <20220330163411.GA1812@kbox> In-Reply-To: <20220330163411.GA1812@kbox> From: Alexei Starovoitov Date: Wed, 30 Mar 2022 11:22:32 -0700 Message-ID: Subject: Re: [PATCH] tracing/user_events: Add eBPF interface for user_event created events To: Beau Belgrave Cc: Song Liu , Steven Rostedt , Masami Hiramatsu , linux-trace-devel , LKML , bpf , Network Development , linux-arch , Mathieu Desnoyers Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RDNS_NONE, SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Mar 30, 2022 at 9:34 AM Beau Belgrave wrote: > > > > > > But you are fine with uprobe costs? uprobes appear to be much more costly > > > than a syscall approach on the hardware I've run on. Care to share the numbers? uprobe over USDT is a single trap. Not much slower compared to syscall with kpti. > > > > Can we achieve the same/similar performance with sys_bpf(BPF_PROG_RUN)? > > > > I think so, the tough part is how do you let the user-space know which > program is attached to run? In the current code this is done by the BPF > program attaching to the event via perf and we run the one there if > any when data is emitted out via write calls. > > I would want to make sure that operators can decide where the user-space > data goes (perf/ftrace/eBPF) after the code has been written. With the > current code this is done via the tracepoint callbacks that perf/ftrace > hook up when operators enable recording via perf, tracefs, libbpf, etc. > > We have managed code (C#/Java) where we cannot utilize stubs or traps > easily due to code movement. So we are limited in how we can approach > this problem. Having the interface be mmap/write has enabled this > for us, since it's easy to interact with in most languages and gives us > lifetime management of the trace objects between user-space and the > kernel. Then you should probably invest into making USDT work inside java applications instead of reinventing the wheel. As an alternative you can do a dummy write or any other syscall and attach bpf on the kernel side. No kernel changes are necessary.