Received-SPF: pass (google.com: domain of linux-kernel+bounces-212269-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Date: Wed, 12 Jun 2024 17:17:48 -0400
From: Steven Rostedt <rostedt@goodmis.org>
To: Naresh Kamboju <naresh.kamboju@linaro.org>
Cc: open list <linux-kernel@vger.kernel.org>, lkft-triage@lists.linaro.org,
 Linux Regressions <regressions@lists.linux.dev>, Dan Carpenter
 <dan.carpenter@linaro.org>, Arnd Bergmann <arnd@arndb.de>, Masami Hiramatsu
 <mhiramat@kernel.org>, Mark Rutland <mark.rutland@arm.com>
Subject: Re: LTP tracing crashed on arm64 rk3399-rock-pi-4 - pc :
 ftrace_ops_test
Message-ID: <20240612171748.0bc6d9cb@rorschach.local.home>
In-Reply-To: <20240612125130.2c1d6d2d@rorschach.local.home>
References: <CA+G9fYsSVJQZH=nM=1cjTc94PgSnMF9y65BnOv6XSoCG_b6wmw@mail.gmail.com>
	<20240612125130.2c1d6d2d@rorschach.local.home>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

On Wed, 12 Jun 2024 12:51:30 -0400
Steven Rostedt <rostedt@goodmis.org> wrote:

> > [  100.600222] Hardware name: Radxa ROCK Pi 4B (DT)
> > [  100.600229] pstate: 800003c5 (Nzcv DAIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> > [  100.600239] pc : ftrace_ops_test+0x34/0x138  
> 
> Hmm, could you show the exact line of the above code? Specifically we have:
> 
> 	rcu_assign_pointer(hash.filter_hash, ops->func_hash->filter_hash);
> 	rcu_assign_pointer(hash.notrace_hash, ops->func_hash->notrace_hash);
> 
> Hmm, it's a NULL pointer dereference at 0x8, so ops is likely not NULL,
> as func_hash is much farther down. But if func_hash is NULL,
> filter_hash is at the 0x8 offset.
> 
> So now the question is, how did func_hash become NULL. It should always
> be pointing at something. May have to do with the subops. Will investigate.
> 

> 
> 
> > [  100.600258] lr : function_graph_enter+0x144/0x208

I wonder if we need the following patch:

diff --git a/kernel/trace/fgraph.c b/kernel/trace/fgraph.c
index 8317d1a7f43a..fc205ad167a9 100644
--- a/kernel/trace/fgraph.c
+++ b/kernel/trace/fgraph.c
@@ -641,7 +641,7 @@ int function_graph_enter(unsigned long ret, unsigned long func,
 	{
 		for_each_set_bit(i, &fgraph_array_bitmask,
 					 sizeof(fgraph_array_bitmask) * BITS_PER_BYTE) {
-			struct fgraph_ops *gops = fgraph_array[i];
+			struct fgraph_ops *gops = READ_ONCE(fgraph_array[i]);
 			int save_curr_ret_stack;
 
 			if (gops == &fgraph_stub)


Because if the compiler decides to re-read gops from fgraph_array[i] after the
above check for the following line that does:

			save_curr_ret_stack = current->curr_ret_stack;
			if (ftrace_ops_test(&gops->ops, func, NULL) &&
			    gops->entryfunc(&trace, gops))
				bitmap |= BIT(i);


and gops now points to fgraph_stub, it will trigger this bug.

Can you apply the above change and see if the bug goes away?

Thanks,

-- Steve