Received: by 2002:a05:6358:16cc:b0:ea:6187:17c9 with SMTP id r12csp12245212rwl; Tue, 3 Jan 2023 11:09:43 -0800 (PST) X-Google-Smtp-Source: AMrXdXvl0vW+rWMVBwnUEsxAyVVaJ1yJoIxmXumSfzwhb75CEq8kjfB/MNgzvcK6X0BA9pQXsLTq X-Received: by 2002:a17:906:7b8e:b0:7c1:3f04:efa2 with SMTP id s14-20020a1709067b8e00b007c13f04efa2mr42804711ejo.29.1672772983431; Tue, 03 Jan 2023 11:09:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1672772983; cv=none; d=google.com; s=arc-20160816; b=0kwVVbGGJAKnuBHUYuaLh3CTirW2MFTySiuDSfoYpuRlKkJgHgnu1lWYmOkJut+gpS XF+1XmMoXAgoAXW0OF8x+rPoCvnXgo2kB/P5O6dKY6egyN4HY4h5cmXlm0/R04DEJDPf ToXhrg+LSzZrtVr1aaHv5Cua/GUy/6/f96Q7W2tU+EJo7APzNWAyFlB2zASoLGWrvwfL plYxqKkXC/xHyJLcAK26qRBOoxlv2IMuTPG17opvlWMkSSbEeU055Vy0AhZNY5qWUBkv mdyzN2q4yZ02TnCfeqgozBcZfsD7vjrQ81wxE3Coz+OnCpWFfyHIkngiQpzHe39T4WPp 4hTA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:mime-version:date :dkim-signature; bh=88y0UfhMzpIzcF+i0aeHj+160apjA13IPkO6jcFxsno=; b=w9W5mklRfIPOwv1irB7cJ5gFw5Sb+DFyQ9QmhLgnwajUEHWQdFqd+c6Tb10mDPernJ mxqPZ58MIDvQv2arMhdTTwQ33yD/GUMs2pJNZ5tkliw2/MUNvECr0Oo4OwjnLBgscsQN 8J5nRKWOYrNprvpo6NicF6QzP79Ps+hLE60Uu5kHqOhGpfJR9dGdC4mjxF8JL6CvbdmW Q+2gMmytf4faULFvD/of/Jhppc5FmyK0j/U4iapMNI2gm/+kbPPR1XaxGO/3hAuKI4tx OU0iIUggKnqAY8RJeVmDQp3VQlO20pU3Tbf5IAp749CYFDl4ZHZ4KpEzbdxEQTb7Br9Z g0hQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=pb4wfNxD; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id gb10-20020a170907960a00b00782ece08669si30915582ejc.13.2023.01.03.11.09.29; Tue, 03 Jan 2023 11:09:43 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=pb4wfNxD; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233549AbjACSyQ (ORCPT + 59 others); Tue, 3 Jan 2023 13:54:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35532 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230515AbjACSyO (ORCPT ); Tue, 3 Jan 2023 13:54:14 -0500 Received: from mail-pl1-x649.google.com (mail-pl1-x649.google.com [IPv6:2607:f8b0:4864:20::649]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 57251F7B for ; Tue, 3 Jan 2023 10:54:13 -0800 (PST) Received: by mail-pl1-x649.google.com with SMTP id p12-20020a170902e74c00b0019254f41f6aso21749672plf.3 for ; Tue, 03 Jan 2023 10:54:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=88y0UfhMzpIzcF+i0aeHj+160apjA13IPkO6jcFxsno=; b=pb4wfNxDUm8ufcVaIopM/P/7BA4nwETAThO5+GdYRFGByDsGP2OY1BG1OREWfUvfJP ioSBhW2guArKhAJWhPcMJp1n6UbR+F5/XlFdkvJMa0peVKimtw6aXNhEBCu0hvfCOLsf dJRI+jyoxciIuI85qNw3pZ0IGC6lbMwoCbT7LWuRVK83EEjofpiYL8p3qthyVMdllOpz 5wsvVXuK4J1Veze/4qYvKbIhTa1rBrAzpvuGbigsSElTN4KApnqobUTt5XVBwD6xnvHK F/8ueXFQuJA2KpAh4/rl6IEo7os7yCh85IY9WzWYsOOqQF+PG3xyp/BjkgZJbuUv3DiL sQjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=88y0UfhMzpIzcF+i0aeHj+160apjA13IPkO6jcFxsno=; b=fWjL/PbA5aLqF5Zn/yjNnzDC4x4GEtt8bUqx3sD4/xCaNqy1A0qovsmxQ+T6HX3ueW tgA2zPTz4Ck81Gywg/tQwBCFaApdl/TeFCAPaAgCr3WGthxKSqKAYOR4Bxu+pbAEs36M NekBUXX+YPQrWVFCPOORJuchYCUXzLSqB8bxZM9KEFYaQ4ksYmc2oz6CdPPimeWWsUWD eEB/P1zxS2V0G8CJXuHZv3jtKH+ei4IQo5kgQV/81LKD/cfFoWwxTFLmKnja7F+ZEQDx yueL6DYJnOZcgbKYmCqebdLIyhpoPoZR36OzRwBZm/aMcyqnr7hH/bqN8dwYIDb2o20C pqmQ== X-Gm-Message-State: AFqh2krwq7Qpa8fw3DRoHSIOKjp63KC8H8K9nUgQa5vQsa4XbBcCq6se HY1JBxRIazyKcTGa+HVbsSsAn/yda58WBRl67Qhah+HYG4/3teL5FDTx3k3itSzT3jTKTPvd+Wt /yYfhWiVh7tPyM2365LXakjp0b1pG2kB/KN54IMQW0UZuuelzHv6bZ9oEgu0pZZKDxBSXnhg= X-Received: from jstultz-noogler2.c.googlers.com ([fda3:e722:ac3:cc00:24:72f4:c0a8:600]) (user=jstultz job=sendgmr) by 2002:a17:902:e993:b0:192:b9ae:6def with SMTP id f19-20020a170902e99300b00192b9ae6defmr704540plb.83.1672772052716; Tue, 03 Jan 2023 10:54:12 -0800 (PST) Date: Tue, 3 Jan 2023 18:54:08 +0000 Mime-Version: 1.0 X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog Message-ID: <20230103185408.2874345-1-jstultz@google.com> Subject: [PATCH] trace: Add trace points for tasklet entry/exit From: John Stultz To: LKML Cc: Lingutla Chandrasekhar , Steven Rostedt , Masami Hiramatsu , Thomas Gleixner , "Paul E. McKenney" , "Connor O'Brien" , kernel-team@android.com, "J . Avila" , John Stultz Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Lingutla Chandrasekhar Tasklets are supposed to finish their work quickly and should not block the current running process, but it is not guaranteed that. Currently softirq_entry/exit can be used to know total tasklets execution time, but not helpful to track individual tasklet's execution time. With that we can't find any culprit tasklet function, which is taking more time. Add tasklet_entry/exit trace point support to track individual tasklet execution. This patch has been carried in the Android tree for awhile so I wanted to submit it for review upstream. Feedback would be appreciated! Cc: Steven Rostedt Cc: Masami Hiramatsu Cc: Thomas Gleixner Cc: "Paul E. McKenney" Cc: Connor O'Brien Cc: kernel-team@android.com Reviewed-by: Steven Rostedt (Google) Signed-off-by: Lingutla Chandrasekhar [elavila: Port to android-mainline] Signed-off-by: J. Avila [jstultz: Rebased to upstream, cut unused trace points, added comments for the tracepoints, reworded commit] Signed-off-by: John Stultz --- include/trace/events/irq.h | 43 ++++++++++++++++++++++++++++++++++++++ kernel/softirq.c | 9 ++++++-- 2 files changed, 50 insertions(+), 2 deletions(-) diff --git a/include/trace/events/irq.h b/include/trace/events/irq.h index eeceafaaea4c..da85851d4ec1 100644 --- a/include/trace/events/irq.h +++ b/include/trace/events/irq.h @@ -160,6 +160,49 @@ DEFINE_EVENT(softirq, softirq_raise, TP_ARGS(vec_nr) ); +DECLARE_EVENT_CLASS(tasklet, + + TP_PROTO(void *func), + + TP_ARGS(func), + + TP_STRUCT__entry( + __field( void *, func) + ), + + TP_fast_assign( + __entry->func = func; + ), + + TP_printk("function=%ps", __entry->func) +); + +/** + * tasklet_entry - called immediately before the tasklet is run + * @func: tasklet callback or function being run + * + * Used to find individual tasklet execution time + */ +DEFINE_EVENT(tasklet, tasklet_entry, + + TP_PROTO(void *func), + + TP_ARGS(func) +); + +/** + * tasklet_exit - called immediately after the tasklet is run + * @func: tasklet callback or function being run + * + * Used to find individual tasklet execution time + */ +DEFINE_EVENT(tasklet, tasklet_exit, + + TP_PROTO(void *func), + + TP_ARGS(func) +); + #endif /* _TRACE_IRQ_H */ /* This part must be outside protection */ diff --git a/kernel/softirq.c b/kernel/softirq.c index c8a6913c067d..dbd322524171 100644 --- a/kernel/softirq.c +++ b/kernel/softirq.c @@ -793,10 +793,15 @@ static void tasklet_action_common(struct softirq_action *a, if (tasklet_trylock(t)) { if (!atomic_read(&t->count)) { if (tasklet_clear_sched(t)) { - if (t->use_callback) + if (t->use_callback) { + trace_tasklet_entry(t->callback); t->callback(t); - else + trace_tasklet_exit(t->callback); + } else { + trace_tasklet_entry(t->func); t->func(t->data); + trace_tasklet_exit(t->func); + } } tasklet_unlock(t); continue; -- 2.39.0.314.g84b9a713c41-goog