Received: by 2002:a05:6358:d09b:b0:dc:cd0c:909e with SMTP id jc27csp11312348rwb; Fri, 25 Nov 2022 14:01:22 -0800 (PST) X-Google-Smtp-Source: AA0mqf5FroofjgRdDu9sI2R9mm1Plrn4t6wYNSwJ8P8BS98ZgPycpopSXCXxNcSxmvVyGauGCmTl X-Received: by 2002:a05:6402:1c04:b0:462:6ace:c522 with SMTP id ck4-20020a0564021c0400b004626acec522mr36330000edb.91.1669413682165; Fri, 25 Nov 2022 14:01:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669413682; cv=none; d=google.com; s=arc-20160816; b=rSefTAl2dF/CvcBuZKQIoD1y4yPQE1LmfgfNMcPAgrvjJSC2rxjlCRpYJ02EMA/PIS c0hIVsfYPpQ9fWhWp6YaVyJ3cAewre+DsKisEz+gf/pjc/bgvH2uIxiFKW9FWnpWZC9I B9EYNr37Edch10wufdCo5wKbHLdQkmHAOoJtlqUXUV443keh7ImVg+GLoPm+aJtTFETL lBrlxtdt79ANp9N0TlZea9Oo2bPUED+KTLrZubzqUWQbZkEQEzl1knXN0NqUDx5ryPw+ mAk6HZOv4sO0HfhcGFDelSpbV5Jff+7fT0jRWlimHhUDz0RuBvRFsQd3blKtxQv7wL9W qJUw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=aWg9iCmVtcgU19BjwQr7qHc7qmpPSCGL2/gExDLvHZM=; b=crlAs6OvQSkodB/dbYTusKw1j540MaViXsN7HwlqaRvW7RRjo+RexrKJI7HBLftNqw FgEYGmzHHmZh9QL11/kcPasTvKU+8frmlgZPvR1tjiaG7xh8rVXIJyzDLTi1SvAUFCzl bA48lDXg1BqgOUpvfp6I92guy9TyS1a+VS+/rXx9lh/2F6ygMOD4BU7lnxaNwwJlti0o 07ojavr1N9qVJwPBycqrnW2f+yEsJvcCJkAJmwdL7OvFpHX02yn4ndmolv63jM1OtiRn 16DOym+nX89Ns+vePOrj66NQvV0qDy8wKsyfipsfm4ntkU6cVJUdAVXq2Ywa4u848mza zcOA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="VuAbw/1n"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id rh16-20020a17090720f000b007819684b56fsi3077354ejb.225.2022.11.25.14.01.00; Fri, 25 Nov 2022 14:01:22 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="VuAbw/1n"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230030AbiKYVUk (ORCPT + 85 others); Fri, 25 Nov 2022 16:20:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57044 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229495AbiKYVUi (ORCPT ); Fri, 25 Nov 2022 16:20:38 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3AB6F2F3A5; Fri, 25 Nov 2022 13:20:37 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id DC609B82C23; Fri, 25 Nov 2022 21:20:35 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7CB5EC433D7; Fri, 25 Nov 2022 21:20:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1669411234; bh=UofQ2Om5TKKPZw19SG+8wKH8TntN+ojXWrOiFhpkAis=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VuAbw/1nYj19+PgmBb5Y3Tlxzw3F5c8blP52lPX36dfvLIrNBuvnd6VvVcJnWuEjz eFFoRQPvsI4BTMUeG89I6iBWGnL32xONihx/6HnJM8GLhbYi6Flt0t0DL/NbprIc+q 1AvJtEaBq+hwwJElK64wfyqyTJRik+TSted6iGwgv2y9htPBpp6vMLJs2+IKf+ZL0Z uUs8o3E3BN+0ELn+eBBxe2xV6w9iaYDS4udWHqLC2sqeGVy7ERe+I8vMwMK7h9qJOg Jd/OMx8G89ciDSgpTpmeJg3JJmzRz0QcJe9KZutZtDw3k4greXC5SkGAWGxUy7K3xn u5lfMBvzHhApw== From: Daniel Bristot de Oliveira To: Daniel Bristot de Oliveira , Steven Rostedt Cc: Masami Hiramatsu , Jonathan Corbet , Juri Lelli , Clark Williams , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V3 1/3] tracing/osnoise: Add PANIC_ON_STOP option Date: Fri, 25 Nov 2022 22:20:22 +0100 Message-Id: <31f67f8f5ff4bc4e1549b34411f918423abcaa78.1669409262.git.bristot@kernel.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Often the latency observed in a CPU is not caused by the work being done in the CPU itself, but by work done on another CPU that causes the hardware to stall all CPUs. In this case, it is interesting to know what is happening on ALL CPUs, and the best way to do this is via crash dump analysis. Add the PANIC_ON_STOP option to osnoise/timerlat tracers. The default behavior is having this option off. When enabled by the user, the system will panic after hitting a stop tracing condition. This option was motivated by a real scenario that Juri Lelli and I were debugging. Cc: Juri Lelli Cc: Daniel Bristot de Oliveira Cc: Steven Rostedt Cc: Masami Hiramatsu Cc: Jonathan Corbet Signed-off-by: Daniel Bristot de Oliveira --- kernel/trace/trace_osnoise.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/kernel/trace/trace_osnoise.c b/kernel/trace/trace_osnoise.c index 3f10dd1f2f1c..801eba0b5cf8 100644 --- a/kernel/trace/trace_osnoise.c +++ b/kernel/trace/trace_osnoise.c @@ -54,10 +54,11 @@ enum osnoise_options_index { OSN_DEFAULTS = 0, OSN_WORKLOAD, + OSN_PANIC_ON_STOP, OSN_MAX }; -static const char * const osnoise_options_str[OSN_MAX] = { "DEFAULTS", "OSNOISE_WORKLOAD" }; +static const char * const osnoise_options_str[OSN_MAX] = { "DEFAULTS", "OSNOISE_WORKLOAD", "PANIC_ON_STOP" }; #define OSN_DEFAULT_OPTIONS 0x2 unsigned long osnoise_options = OSN_DEFAULT_OPTIONS; @@ -1270,6 +1271,9 @@ static __always_inline void osnoise_stop_tracing(void) trace_array_printk_buf(tr->array_buffer.buffer, _THIS_IP_, "stop tracing hit on cpu %d\n", smp_processor_id()); + if (test_bit(OSN_PANIC_ON_STOP, &osnoise_options)) + panic("tracer hit stop condition on CPU %d\n", smp_processor_id()); + tracer_tracing_off(tr); } rcu_read_unlock(); -- 2.32.0