Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   "Eddy_Wu@trendmicro.com" <Eddy_Wu@trendmicro.com>
To:     Masami Hiramatsu <mhiramat@kernel.org>,
        Peter Zijlstra <peterz@infradead.org>
CC:     "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "x86@kernel.org" <x86@kernel.org>,
        "David S. Miller" <davem@davemloft.net>
Subject: x86/kprobes: kretprobe fails to triggered if kprobe at function entry
 is not optimized (trigger by int3 breakpoint)
Thread-Topic: x86/kprobes: kretprobe fails to triggered if kprobe at function
 entry is not optimized (trigger by int3 breakpoint)
Thread-Index: AdZ5/y9ucVi7Kf0NSwa8vnc6Q/ol0w==
Date:   Mon, 24 Aug 2020 12:02:58 +0000
Message-ID: <8816bdbbc55c4d2397e0b02aad2825d3@trendmicro.com>
Accept-Language: zh-TW, en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

Greetings!

Starting from kernel 5.8 (x86_64), kretprobe handler will always missed if =
corresponding kprobe on function entry is not optimized (using break point =
instead).
Step to reproduce this:
1) Build the kretprobe example module (CONFIG_SAMPLE_KRETPROBES=3Dm)
2) Disable jump optimization (`sysctl debug.kprobes-optimization=3D0` or re=
gister any kprobe.post_handler at same location)
3) Insert the kretprobe_example module
4) Launch some process to trigger _do_fork
5) Remove kretprobe_example module
6) dmesg shows that all probing instances are missed

Example output:
# sysctl debug.kprobes-optimization=3D0
debug.kprobes-optimization =3D 0
# insmod samples/kprobes/kretprobe_example.ko
# ls > /dev/null
# rmmod kretprobe_example
# dmesg
[48555.067295] Planted return probe at _do_fork: 0000000038ae0211
[48560.229459] kretprobe at 0000000038ae0211 unregistered
[48560.229460] Missed probing 3 instances of _do_fork

After bisecting, I found this behavior seems to introduce by this commit: (=
5.8-rc1)
0d00449c7a28a1514595630735df383dec606812 x86: Replace ist_enter() with nmi_=
enter()
This make kprobe_int3_handler() effectively running as NMI context, which p=
re_handler_kretprobe() explicitly checked to prevent recursion.

(in_nmi() check appears from v3.17)
f96f56780ca584930bb3a2769d73fd9a101bcbbe kprobes: Skip kretprobe hit in NMI=
 context to avoid deadlock

To make kretprobe work again with int3 breakpoint, I think we can replace t=
he in_nmi() check with in_nmi() =3D=3D (1 << NMI_SHIFT) at kprobe_int3_hand=
ler() and skip kretprobe if nested NMI.
Did a quick test on 5.9-rc2 and it seems to be working.
I'm not sure if it is the best way to do since it may also require change t=
o other architecture as well, any thought?


TREND MICRO EMAIL NOTICE

The information contained in this email and any attachments is confidential=
 and may be subject to copyright or other intellectual property protection.=
 If you are not the intended recipient, you are not authorized to use or di=
sclose this information, and we request that you notify us by reply mail or=
 telephone and delete the original message from your mail system.

For details about what personal information we collect and why, please see =
our Privacy Notice on our website at: Read privacy policy<http://www.trendm=
icro.com/privacy>