Received: by 2002:a05:6358:d09b:b0:dc:cd0c:909e with SMTP id jc27csp2817996rwb; Wed, 30 Nov 2022 11:15:06 -0800 (PST) X-Google-Smtp-Source: AA0mqf7RrNko+AKjn/Ck9WSrd80gnumdF2LqSprDeK59gMXJVxoUyjgNCOIz2SMIkPJ6wEDvboXy X-Received: by 2002:a17:906:844:b0:78c:2c03:804c with SMTP id f4-20020a170906084400b0078c2c03804cmr40677695ejd.107.1669835706432; Wed, 30 Nov 2022 11:15:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669835706; cv=none; d=google.com; s=arc-20160816; b=HA9S/r48cCzd2sHQlnPdZPMmp7mJLqO/OFWfIsn/gkwGqTiZDgnks8bfZCxmCL3aDU xU761cQ6J/oDVlOy4c74U2RBpU8zXXkuM1J8MIAInQGIDl1tuki+sibUsC4hPAQ6g09E cVvJm224BbpNKx2SzJ2WoBvWVbvS9gXdlu97dZN/iQ3sYGHcLUamMHehGOPRw79hDi3b iVhCCAiZLCswdqVzBe1h4qQYdRbYkVpM/oN4y94MvYF3lg5cOFxEh+zptkhi2O/E22hR wfto+KyfsC6WW4saxKLNJU90ymSZ+5KFOcZgshyeOFtlNXgwaG+Sc9l//me9wIspxfe0 P8hA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=aWg9iCmVtcgU19BjwQr7qHc7qmpPSCGL2/gExDLvHZM=; b=tEmsCmFS8SVUK9rY4+52bP59XEhBi8DEYmX+jmooFcGniYnTCLirzd0Fj6hU2GlU5h LNxGGxoz5A08FGX9tWp2bib/YkfRmIEgqxSL7pKLLtTKpAbJNfD2o5EUS7AjckatxW81 ZbY/s/vso8K//vyfSXXI35TOVf8Zq+WL6UHwFp6rKRlz/1q6LnYOpkjaofb3KWW+KBwU JNXb5zePSXGsYvpBFT6AcoQXe+BIZBDDscE+xeYeCGuAKOEFIb4W+RiBNRGjm3RYN+jn ffmsgPJ5YnmYrcabXhWCNA55nHLqwO/NUI01k2eMiDCKWaD4ULYv5YN+Io9UmVexbtRQ Y76w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=XMMfsAce; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id sh12-20020a1709076e8c00b007bb92113ca1si2044672ejc.67.2022.11.30.11.14.46; Wed, 30 Nov 2022 11:15:06 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=XMMfsAce; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230053AbiK3SV2 (ORCPT + 83 others); Wed, 30 Nov 2022 13:21:28 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48652 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230314AbiK3SUt (ORCPT ); Wed, 30 Nov 2022 13:20:49 -0500 Received: from sin.source.kernel.org (sin.source.kernel.org [IPv6:2604:1380:40e1:4800::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4F0638BD06; Wed, 30 Nov 2022 10:19:23 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sin.source.kernel.org (Postfix) with ESMTPS id 83F78CE1ACD; Wed, 30 Nov 2022 18:19:21 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B8EEAC43470; Wed, 30 Nov 2022 18:19:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1669832359; bh=UofQ2Om5TKKPZw19SG+8wKH8TntN+ojXWrOiFhpkAis=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XMMfsAceNIPA7SVzZd9xgHyYxzJcdYfpp9vfAK/FLUt8535HDUNnn3noVhN0M1G7D fOMtP0q8tDdeTome2UAGMdiwTGteRyuhJ/O2bVmzXbM0PSLpkxQi6dcyFtfbdSnAF5 T/fSXL441rkM7QD9tGFQ0Pv/NuvtEADFGpvnH5mAXk1e/tXMvzo59uqxCdeBZiffRR 1clWQTueMTNBqloHGE0Ccpkeb2q/I0fUXWfwE1lUrzokBPy7xzu2zPxL8i/fTaNPPl xDVxNl8T4GNj0smUjuAmWKwPP4nyd1S5NYxABS3Yol1YqMkbnl2MsTxbTV5b4fjkDo PomzbCOX1HSDQ== From: Daniel Bristot de Oliveira To: Daniel Bristot de Oliveira , Steven Rostedt Cc: Masami Hiramatsu , Jonathan Corbet , Juri Lelli , Clark Williams , Bagas Sanjaya , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V4 1/3] tracing/osnoise: Add PANIC_ON_STOP option Date: Wed, 30 Nov 2022 19:19:09 +0100 Message-Id: <249ce4287c6725543e6db845a6e0df621dc67db5.1669832184.git.bristot@kernel.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Often the latency observed in a CPU is not caused by the work being done in the CPU itself, but by work done on another CPU that causes the hardware to stall all CPUs. In this case, it is interesting to know what is happening on ALL CPUs, and the best way to do this is via crash dump analysis. Add the PANIC_ON_STOP option to osnoise/timerlat tracers. The default behavior is having this option off. When enabled by the user, the system will panic after hitting a stop tracing condition. This option was motivated by a real scenario that Juri Lelli and I were debugging. Cc: Juri Lelli Cc: Daniel Bristot de Oliveira Cc: Steven Rostedt Cc: Masami Hiramatsu Cc: Jonathan Corbet Signed-off-by: Daniel Bristot de Oliveira --- kernel/trace/trace_osnoise.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/kernel/trace/trace_osnoise.c b/kernel/trace/trace_osnoise.c index 3f10dd1f2f1c..801eba0b5cf8 100644 --- a/kernel/trace/trace_osnoise.c +++ b/kernel/trace/trace_osnoise.c @@ -54,10 +54,11 @@ enum osnoise_options_index { OSN_DEFAULTS = 0, OSN_WORKLOAD, + OSN_PANIC_ON_STOP, OSN_MAX }; -static const char * const osnoise_options_str[OSN_MAX] = { "DEFAULTS", "OSNOISE_WORKLOAD" }; +static const char * const osnoise_options_str[OSN_MAX] = { "DEFAULTS", "OSNOISE_WORKLOAD", "PANIC_ON_STOP" }; #define OSN_DEFAULT_OPTIONS 0x2 unsigned long osnoise_options = OSN_DEFAULT_OPTIONS; @@ -1270,6 +1271,9 @@ static __always_inline void osnoise_stop_tracing(void) trace_array_printk_buf(tr->array_buffer.buffer, _THIS_IP_, "stop tracing hit on cpu %d\n", smp_processor_id()); + if (test_bit(OSN_PANIC_ON_STOP, &osnoise_options)) + panic("tracer hit stop condition on CPU %d\n", smp_processor_id()); + tracer_tracing_off(tr); } rcu_read_unlock(); -- 2.32.0