Received: by 2002:a05:6a10:5bc5:0:0:0:0 with SMTP id os5csp373024pxb; Wed, 27 Oct 2021 04:42:29 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx9ATpSOIU7AnE6c1c468np9yGW06YEsg5zah0DX5MOmTGuYLewKnywDe7uAEOiUv0ko7ED X-Received: by 2002:aa7:8704:0:b0:47b:e938:f31 with SMTP id b4-20020aa78704000000b0047be9380f31mr21777838pfo.67.1635334948950; Wed, 27 Oct 2021 04:42:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1635334948; cv=none; d=google.com; s=arc-20160816; b=IcsxqhNrumfH/tRmEivXSKPh48iTnrzgLJcJTusELj8wC9I9KYtZ6sUIrw8JP0R0sH fD4FwNUdJ9GlBx8KvqO+lmaTu1BW6LBCJulqiChCHJXFYOyopXMb8K63vKl0g152aUC4 unx8tJAYh0RhO9DryeW4zVJnED2q4KkDHPtqCZvU/XNR0RtE47tpQ4NlScbv28HC+/ES TRbEPLxTO1bARM9K3GT+SleEV8WTpfqCHUX98TWM99CIWPeELsCta4ILPkubtcwzbpwu 3MtYIlj+iIF+/wR/ClOIrvF3MqajLDlAL5NDhd8u0aSKsU6oImk0q98UJW/ZWAZ1a1tB /b+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=RNZ88F8wnjPTIfEoYwKPjPkZADduzYPV8guTf+bvHo8=; b=FMlT+3ZGGFXtplTLULYCThjlB8YrZtJsYOqUQacmSxKA4vyCTdFPClPJms9Qzud7wJ FWcl5+l4kcwkF/JMfO6eJZDFo4xEU7ZEZqRMRXveyjG3TL4oDb8zJ8J4Rj0S8DB6j/i9 E7nRcTI0cXnRiUGL5GoXKcxTbJgyKx3cxy7A24rHaU+FicQv3oDHHyvNDrDj+74OJPqG jVLEhTOrV/ETmsyXxV6x60v3LbFGqYn21q52mU5td+QYiG8hZxGEpCG5o2Tm9/QgFrq2 qVxU1I6vzj0a0qPcWmcydw+cC/x4swyOdoSHowXJTzUUeQo0iIjqrxvBxX+qbJ+ekPMo tnzw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id o2si28412882pll.454.2021.10.27.04.42.14; Wed, 27 Oct 2021 04:42:28 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239811AbhJZWKo convert rfc822-to-8bit (ORCPT + 99 others); Tue, 26 Oct 2021 18:10:44 -0400 Received: from us-smtp-delivery-44.mimecast.com ([207.211.30.44]:26091 "EHLO us-smtp-delivery-44.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239805AbhJZWKl (ORCPT ); Tue, 26 Oct 2021 18:10:41 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-127-qk73EZyfMI-eBn9zjQAL5w-1; Tue, 26 Oct 2021 18:08:11 -0400 X-MC-Unique: qk73EZyfMI-eBn9zjQAL5w-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7475A18D6A2C; Tue, 26 Oct 2021 22:08:09 +0000 (UTC) Received: from x1.com (unknown [10.22.32.3]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5495D60BF1; Tue, 26 Oct 2021 22:08:05 +0000 (UTC) From: Daniel Bristot de Oliveira To: Steven Rostedt Cc: Tao Zhou , Daniel Bristot de Oliveira , Ingo Molnar , Tom Zanussi , Masami Hiramatsu , Juri Lelli , Clark Williams , John Kacur , Peter Zijlstra , Thomas Gleixner , Sebastian Andrzej Siewior , linux-rt-users@vger.kernel.org, linux-trace-devel@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V6 06/20] trace/osnoise: Allow multiple instances of the same tracer Date: Wed, 27 Oct 2021 00:06:17 +0200 Message-Id: <69cbbd98cce2515c84127c8827d733dc87b04823.1635284863.git.bristot@kernel.org> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=bristot@kernel.org X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: kernel.org Content-Transfer-Encoding: 8BIT Content-Type: text/plain; charset=WINDOWS-1252 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Currently, the user can start only one instance of timerlat/osnoise tracers and the tracers cannot run in parallel. As starting point to add more flexibility, let's allow the same tracer to run on different trace instances. The workload will start when the first trace_array (instance) is registered and stop when the last instance is unregistered. So, while this patch allows the same tracer to run in multiple instances (e.g., two instances running osnoise), it still does not allow instances of timerlat and osnoise in parallel (e.g., one timerlat and osnoise). That is because the osnoise: events have different behavior depending on which tracer is enabled (osnoise or timerlat). Enabling the parallel usage of these two tracers is in my TODO list. Cc: Steven Rostedt Cc: Ingo Molnar Cc: Tom Zanussi Cc: Masami Hiramatsu Cc: Juri Lelli Cc: Clark Williams Cc: John Kacur Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Sebastian Andrzej Siewior Cc: Daniel Bristot de Oliveira Cc: linux-rt-users@vger.kernel.org Cc: linux-trace-devel@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Daniel Bristot de Oliveira --- kernel/trace/trace_osnoise.c | 101 +++++++++++++++++++++++++++-------- 1 file changed, 78 insertions(+), 23 deletions(-) diff --git a/kernel/trace/trace_osnoise.c b/kernel/trace/trace_osnoise.c index 3db506f49a90..8681ffc3817b 100644 --- a/kernel/trace/trace_osnoise.c +++ b/kernel/trace/trace_osnoise.c @@ -64,6 +64,24 @@ static bool osnoise_has_registered_instances(void) list); } +/* + * osnoise_instance_registered - check if a tr is already registered + */ +static int osnoise_instance_registered(struct trace_array *tr) +{ + struct osnoise_instance *inst; + int found = 0; + + rcu_read_lock(); + list_for_each_entry_rcu(inst, &osnoise_instances, list) { + if (inst->tr == tr) + found = 1; + } + rcu_read_unlock(); + + return found; +} + /* * osnoise_register_instance - register a new trace instance * @@ -2048,6 +2066,16 @@ static int osnoise_workload_start(void) { int retval; + /* + * Instances need to be registered after calling workload + * start. Hence, if there is already an instance, the + * workload was already registered. Otherwise, this + * code is on the way to register the first instance, + * and the workload will start. + */ + if (osnoise_has_registered_instances()) + return 0; + osn_var_reset_all(); retval = osnoise_hook_events(); @@ -2075,6 +2103,13 @@ static int osnoise_workload_start(void) */ static void osnoise_workload_stop(void) { + /* + * Instances need to be unregistered before calling + * stop. Hence, if there is a registered instance, more + * than one instance is running, and the workload will not + * yet stop. Otherwise, this code is on the way to disable + * the last instance, and the workload can stop. + */ if (osnoise_has_registered_instances()) return; @@ -2096,7 +2131,11 @@ static void osnoise_tracer_start(struct trace_array *tr) { int retval; - if (osnoise_has_registered_instances()) + /* + * If the instance is already registered, there is no need to + * register it again. + */ + if (osnoise_instance_registered(tr)) return; retval = osnoise_workload_start(); @@ -2108,18 +2147,17 @@ static void osnoise_tracer_start(struct trace_array *tr) static void osnoise_tracer_stop(struct trace_array *tr) { - if (!osnoise_has_registered_instances()) - return; - osnoise_unregister_instance(tr); osnoise_workload_stop(); } static int osnoise_tracer_init(struct trace_array *tr) { - - /* Only allow one instance to enable this */ - if (osnoise_has_registered_instances()) + /* + * Only allow osnoise tracer if timerlat tracer is not running + * already. + */ + if (osnoise_data.timerlat_tracer) return -EBUSY; tr->max_latency = 0; @@ -2148,45 +2186,55 @@ static void timerlat_tracer_start(struct trace_array *tr) { int retval; - if (osnoise_has_registered_instances()) + /* + * If the instance is already registered, there is no need to + * register it again. + */ + if (osnoise_instance_registered(tr)) return; - osnoise_data.timerlat_tracer = 1; - retval = osnoise_workload_start(); if (retval) - goto out_err; + pr_err(BANNER "Error starting timerlat tracer\n"); osnoise_register_instance(tr); return; -out_err: - pr_err(BANNER "Error starting timerlat tracer\n"); } static void timerlat_tracer_stop(struct trace_array *tr) { int cpu; - if (!osnoise_has_registered_instances()) - return; - - for_each_online_cpu(cpu) - per_cpu(per_cpu_osnoise_var, cpu).sampling = 0; + osnoise_unregister_instance(tr); - osnoise_tracer_stop(tr); + /* + * Instruct the threads to stop only if this is the last instance. + */ + if (!osnoise_has_registered_instances()) { + for_each_online_cpu(cpu) + per_cpu(per_cpu_osnoise_var, cpu).sampling = 0; + } - osnoise_data.timerlat_tracer = 0; + osnoise_workload_stop(); } static int timerlat_tracer_init(struct trace_array *tr) { - /* Only allow one instance to enable this */ - if (osnoise_has_registered_instances()) + /* + * Only allow timerlat tracer if osnoise tracer is not running already. + */ + if (osnoise_has_registered_instances() && !osnoise_data.timerlat_tracer) return -EBUSY; - tr->max_latency = 0; + /* + * If this is the first instance, set timerlat_tracer to block + * osnoise tracer start. + */ + if (!osnoise_has_registered_instances()) + osnoise_data.timerlat_tracer = 1; + tr->max_latency = 0; timerlat_tracer_start(tr); return 0; @@ -2195,6 +2243,13 @@ static int timerlat_tracer_init(struct trace_array *tr) static void timerlat_tracer_reset(struct trace_array *tr) { timerlat_tracer_stop(tr); + + /* + * If this is the last instance, reset timerlat_tracer allowing + * osnoise to be started. + */ + if (!osnoise_has_registered_instances()) + osnoise_data.timerlat_tracer = 0; } static struct tracer timerlat_tracer __read_mostly = { -- 2.31.1