Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Fri, 18 Nov 2022 09:40:09 -0300
From:   Rafael Mendonca <rafaelmendsr@gmail.com>
To:     Steven Rostedt <rostedt@goodmis.org>
Cc:     Masami Hiramatsu <mhiramat@kernel.org>,
        "Tzvetomir Stoyanov (VMware)" <tz.stoyanov@gmail.com>,
        linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org,
        Tom Zanussi <zanussi@kernel.org>
Subject: Re: [PATCH] tracing/eprobe: Update cond flag before enabling trigger
Message-ID: <Y3d9KcpcwrEUUYKT@macondo>
References: <20221116192552.1066630-1-rafaelmendsr@gmail.com>
 <20221117211726.4bbbb96a@gandalf.local.home>
 <20221117213109.6119750e@gandalf.local.home>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20221117213109.6119750e@gandalf.local.home>
Precedence: bulk

On Thu, Nov 17, 2022 at 09:31:09PM -0500, Steven Rostedt wrote:
> On Thu, 17 Nov 2022 21:17:26 -0500
> Steven Rostedt <rostedt@goodmis.org> wrote:
> 
> > On Wed, 16 Nov 2022 16:25:51 -0300
> > Rafael Mendonca <rafaelmendsr@gmail.com> wrote:
> > 
> > > That happens because enable_eprobe() will eventually trigger the
> > > kmem/mm_page_alloc trace event:
> > > 
> > > - enable_eprobe [trace_eprobe.c]
> > >   - trace_event_trigger_enable_disable [trace_events_trigger.c]
> > >     - trace_event_enable_disable [trace_events.c]
> > >       - __ftrace_event_enable_disable [trace_events.c]
> > >         - trace_buffered_event_enable [trace.c]
> > >           - alloc_pages_node [gfp.h]
> > > 	   ...
> > >             - __alloc_pages [page_alloc.c]
> > >               - trace_mm_page_alloc // eprobe event file without TRIGGER_COND bit set
> > > 
> > > By the time kmem/mm_page_alloc trace event is hit, the eprobe event file
> > > does not have the TRIGGER_COND flag set yet, which causes the eprobe's
> > > trigger to be invoked (through the trace_trigger_soft_disabled() path)
> > > without a trace record, causing a NULL pointer dereference when fetching
> > > the event fields.
> > > 
> > > Fix this by setting the cond flag beforehand when enabling the eprobe's
> > > trigger.
> > > 
> > > Fixes: 7491e2c44278 ("tracing: Add a probe that attaches to trace events")
> > > Signed-off-by: Rafael Mendonca <rafaelmendsr@gmail.com>
> > > ---  
> > 
> > Thanks for the report, but I'm worried that this isn't enough because of
> > how memory ordering can happen on different architectures. That is, just
> > because you switch the order of updates, doesn't mean that the architecture
> > will honor it.
> > 
> > I don't want to add memory barriers in the fast path, but instead we can
> > simply check if rec is NULL in the handler.
> > 
> > So basically:
> > 
> > 
> > static void eprobe_trigger_func(struct event_trigger_data *data,
> > 				struct trace_buffer *buffer, void *rec,
> > 				struct ring_buffer_event *rbe)
> > {
> > 	struct eprobe_data *edata = data->private_data;
> > 
> > 	if (!rec)
> > 		return;
> > 
> > 	__eprobe_trace_func(edata, rec);
> > }
> > 
> > And this should be documented.
> > 
> 
> In fact, does this fix it for you?

It does. I found this while enabling eprobe for all events in my setup.
Doing the same test wih the proposed patch it did not trigger any issue.
Thanks.

> 
> I'm going to take this patch and reference you as a reported-by, as I have
> a lot of urgent code that needs to got upstream, and I need to start
> testing it.
> 
> Thanks!
> 
> -- Steve
> 
> diff --git a/kernel/trace/trace_eprobe.c b/kernel/trace/trace_eprobe.c
> index 5dd0617e5df6..6b31b74954d9 100644
> --- a/kernel/trace/trace_eprobe.c
> +++ b/kernel/trace/trace_eprobe.c
> @@ -563,6 +563,9 @@ static void eprobe_trigger_func(struct event_trigger_data *data,
>  {
>  	struct eprobe_data *edata = data->private_data;
>  
> +	if (!rec)
> +		return;
> +
>  	__eprobe_trace_func(edata, rec);
>  }
>  
>