Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Mon, 8 Feb 2021 10:02:06 -0500
From:   Steven Rostedt <rostedt@goodmis.org>
To:     Josh Poimboeuf <jpoimboe@redhat.com>
Cc:     Linus Torvalds <torvalds@linux-foundation.org>,
        Borislav Petkov <bp@suse.de>,
        Dave Hansen <dave.hansen@intel.com>, x86-ml <x86@kernel.org>,
        lkml <linux-kernel@vger.kernel.org>,
        Alexei Starovoitov <ast@kernel.org>
Subject: Re: [GIT PULL] x86/urgent for v5.11-rc7
Message-ID: <20210208100206.3b74891e@gandalf.local.home>
In-Reply-To: <20210207224540.ercf5657pftibyaw@treble>
References: <20210207104022.GA32127@zn.tnic>
        <CAHk-=widXSyJ8W3vRrqO-zNP12A+odxg2J2_-oOUskz33wtfqA@mail.gmail.com>
        <20210207175814.GF32127@zn.tnic>
        <CAHk-=wi5z9S7x94SKYNj6qSHBqz+OD76GW=MDzo-KN2Fzm-V4Q@mail.gmail.com>
        <20210207224540.ercf5657pftibyaw@treble>
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Precedence: bulk

On Sun, 7 Feb 2021 16:45:40 -0600
Josh Poimboeuf <jpoimboe@redhat.com> wrote:

> > I do suspect involved people should start thinking about how they want
> > to deal with functions starting with
> > 
> >         endbr64
> >         call __fentry__
> > 
> > instead of the call being at the very top of the function.  
> 
> FWIW, objtool's already fine with it (otherwise we would have discovered
> the need to disable fcf-protection much sooner).

And this doesn't really affect tracing (note, another user that might be
affected is live kernel patching). The way this change was noticed, was
that there was a report of someone that was be able to connect a bpf
program to a function for one machine but not for another machine. The
other machine had this CET thingy.

The difference is, when you attach a probe to the start of a function,
kprobes will check if the probe address (start of function) is located at a
ftrace location (nop / __fentry__) and if it is, it would use the ftrace
infrastructure instead of attaching an int3 breakpoint. Because of the
enbr64 being at the start of the function, the check returned false (it was
not a ftrace location) and it attached an int3 breakpoint instead.

This uncovered another "bug". Peter Zijlstra made int3 handlers look like
NMIs (in_nmi() would return true in an int3 handler). The BPF programs would
not run in NMI context. But nobody noticed, because people usually attach
BPF programs to the start of a function using kprobes, and since kprobes
would use ftrace handlers (that don't set in_nmi() to true), everything
worked. But when the "endbr64" was added at the start of the program,
kprobes fell back to int3, and suddenly the BPF programs stopped working.

-- Steve