Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Tue, 29 Sep 2020 09:25:29 +0200
From:   Peter Zijlstra <peterz@infradead.org>
To:     Kim Phillips <kim.phillips@amd.com>
Cc:     linux-kernel@vger.kernel.org, linux-tip-commits@vger.kernel.org,
        Thomas Gleixner <tglx@linutronix.de>,
        Alexandre Chartre <alexandre.chartre@oracle.com>,
        "Paul E. McKenney" <paulmck@kernel.org>, x86 <x86@kernel.org>
Subject: Re: [tip: core/rcu] rcu/tree: Mark the idle relevant functions
 noinstr
Message-ID: <20200929072529.GL2628@hirez.programming.kicks-ass.net>
References: <20200505134100.575356107@linutronix.de>
 <158991795300.17951.11897222265664137612.tip-bot2@tip-bot2>
 <b94de56b-1b37-07a0-f0de-12471ee5fc3d@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <b94de56b-1b37-07a0-f0de-12471ee5fc3d@amd.com>
Precedence: bulk

On Mon, Sep 28, 2020 at 05:22:33PM -0500, Kim Phillips wrote:
> On 5/19/20 2:52 PM, tip-bot2 for Thomas Gleixner wrote:
> > The following commit has been merged into the core/rcu branch of tip:
> > 
> > Commit-ID:     ff5c4f5cad33061b07c3fb9187506783c0f3cb66
> > Gitweb:        https://git.kernel.org/tip/ff5c4f5cad33061b07c3fb9187506783c0f3cb66
> > Author:        Thomas Gleixner <tglx@linutronix.de>
> > AuthorDate:    Fri, 13 Mar 2020 17:32:17 +01:00
> > Committer:     Thomas Gleixner <tglx@linutronix.de>
> > CommitterDate: Tue, 19 May 2020 15:51:20 +02:00
> > 
> > rcu/tree: Mark the idle relevant functions noinstr
> > 
> > These functions are invoked from context tracking and other places in the
> > low level entry code. Move them into the .noinstr.text section to exclude
> > them from instrumentation.
> > 
> > Mark the places which are safe to invoke traceable functions with
> > instrumentation_begin/end() so objtool won't complain.
> > 
> > Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> > Reviewed-by: Alexandre Chartre <alexandre.chartre@oracle.com>
> > Acked-by: Peter Zijlstra <peterz@infradead.org>
> > Acked-by: Paul E. McKenney <paulmck@kernel.org>
> > Link: https://lkml.kernel.org/r/20200505134100.575356107@linutronix.de
> > 
> > 
> > ---
> 
> I bisected a system hang condition down to this commit.

That's odd, mostly its the lack of noinstr that causes hangs, I've never
yet seen the presence of it cause problems.

> To reproduce the hang, compile the below code and execute it as root
> on an x86_64 server (AMD or Intel).  The code is opening a 
> PERF_TYPE_TRACEPOINT event with a non-zero pe.config.

In my experience, it is very relevant which exact tracepoint you end up
using.

PERF_COUNT_HW_INSTRUCTIONS is a very long and tedious way of writing 1
in this case, on my randonly selected test box this morning, trace event
1 is:

$ for file in /debug/tracing/events/*/*/id ;  do echo $file -- $(cat $file); done | grep " 1$"
/debug/tracing/events/ftrace/function/id -- 1

> If I revert the commit from Linus' ToT, the system stays up.

>    memset(&pe, 0, sizeof(struct perf_event_attr));
>    pe.type = PERF_TYPE_TRACEPOINT;
>    pe.size = sizeof(struct perf_event_attr);
>    pe.config = PERF_COUNT_HW_INSTRUCTIONS;
>    pe.disabled = 1;

Doubly funny for not actually enabling the event...

>    pe.exclude_kernel = 1;
>    pe.exclude_hv = 1;

Still, it seems to make my machine unhappy.. Let's see if I can get
anything useful out of it.