Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Wed, 26 Aug 2020 19:00:41 +0900
From:   Masami Hiramatsu <mhiramat@kernel.org>
To:     Masami Hiramatsu <mhiramat@kernel.org>
Cc:     "Eddy_Wu@trendmicro.com" <Eddy_Wu@trendmicro.com>,
        "peterz@infradead.org" <peterz@infradead.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "x86@kernel.org" <x86@kernel.org>
Subject: Re: x86/kprobes: kretprobe fails to triggered if kprobe at function
 entry is not optimized (trigger by int3 breakpoint)
Message-Id: <20200826190041.2d0ff0fbe154ba62163b0c00@kernel.org>
In-Reply-To: <20200826180645.9b609fc05df2a149564df1b8@kernel.org>
References: <8816bdbbc55c4d2397e0b02aad2825d3@trendmicro.com>
        <20200825005426.f592075d13be740cb3c9aa77@kernel.org>
        <7396e7b2079644a6aafd9670a111232b@trendmicro.com>
        <20200825151538.f856d701a34f4e0561a64932@kernel.org>
        <20200825120911.GX1362448@hirez.programming.kicks-ass.net>
        <d3027772a6834f89a1ddc07c0fefaa0a@trendmicro.com>
        <20200826172239.ff175be78ee5f3a4380579c3@kernel.org>
        <20200826180645.9b609fc05df2a149564df1b8@kernel.org>
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Wed, 26 Aug 2020 18:06:45 +0900
Masami Hiramatsu <mhiramat@kernel.org> wrote:

> On Wed, 26 Aug 2020 17:22:39 +0900
> Masami Hiramatsu <mhiramat@kernel.org> wrote:
> 
> > On Wed, 26 Aug 2020 07:07:09 +0000
> > "Eddy_Wu@trendmicro.com" <Eddy_Wu@trendmicro.com> wrote:
> > 
> > > 
> > > > -----Original Message-----
> > > > From: peterz@infradead.org <peterz@infradead.org>
> > > > Sent: Tuesday, August 25, 2020 8:09 PM
> > > > To: Masami Hiramatsu <mhiramat@kernel.org>
> > > > Cc: Eddy Wu (RD-TW) <Eddy_Wu@trendmicro.com>; linux-kernel@vger.kernel.org; x86@kernel.org; David S. Miller
> > > > <davem@davemloft.net>
> > > > Subject: Re: x86/kprobes: kretprobe fails to triggered if kprobe at function entry is not optimized (trigger by int3 breakpoint)
> > > >
> > > > Surely we can do a lockless list for this. We have llist_add() and
> > > > llist_del_first() to make a lockless LIFO/stack.
> > > >
> > > 
> > > llist operations require atomic cmpxchg, for some arch doesn't have CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG, in_nmi() check might still needed.
> > > (HAVE_KRETPROBES && !CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG): arc, arm, csky, mips
> > 
> > Good catch. In those cases, we can add in_nmi() check at arch dependent code.
> 
> Oops, in_nmi() check is needed in pre_kretprobe_handler() which has no
> arch dependent code. Hmm, so we still need an weak function to check it...

Oops, again. Sorry I found a big misunderstand. I found the in_nmi() check is
completely unnecessary with Jiri's commit 9b38cc704e84 ("kretprobe: Prevent 
triggering kretprobe from within kprobe_flush_task").

This commit introduced the kprobe_busy_begin/end() to the kretproeb trampoline
handler, which set a dummy kprobe to the per-cpu current kprobe pointer.
This current-kprobe is checked at the kprobe pre handler to prevent kprobes
(including kretprobe) recursion.

This means, if an NMI interrupts a kretprobe operation (both pre-handler and
trampoline-handler) and it hits the same kretprobe, this nested kretprobe
handlers never be called, because there is a current kprobe is already set.
Thus, we are totally safe from double-lock issue in the kretprobe handlers.

So we can just remove the in_nmi() check from pre_kretprobe_handler() if
we introduced a generic trampoline handler, since kprobe_busy_begin/end()
must called from the trampoline handlers. Currently it is used on x86 only.

Of course, this doesn't solve the llist_del_first() contention in the
pre_kretprobe_handler(). So anyway we need a lock for per-probe llist
(if I understand llist.h comment correctly.)

Thank you,

-- 
Masami Hiramatsu <mhiramat@kernel.org>