From: Jisheng Zhang <[email protected]>
Current riscv's kprobe handlers are run with both preemption and
interrupt enabled, this violates kprobe requirements. Fix this issue
by keeping interrupts disabled for BREAKPOINT exception.
Fixes: c22b0bcb1dd0 ("riscv: Add kprobes supported")
Signed-off-by: Jisheng Zhang <[email protected]>
---
arch/riscv/kernel/entry.S | 2 ++
1 file changed, 2 insertions(+)
diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
index 744f3209c48d..4114b65698ec 100644
--- a/arch/riscv/kernel/entry.S
+++ b/arch/riscv/kernel/entry.S
@@ -130,6 +130,8 @@ skip_context_tracking:
*/
andi t0, s1, SR_PIE
beqz t0, 1f
+ li t0, EXC_BREAKPOINT
+ beq s4, t0, 1f
#ifdef CONFIG_TRACE_IRQFLAGS
call trace_hardirqs_on
#endif
--
2.31.0
Hi Jisheng,
On Tue, 30 Mar 2021 02:16:24 +0800
Jisheng Zhang <[email protected]> wrote:
> From: Jisheng Zhang <[email protected]>
>
> Current riscv's kprobe handlers are run with both preemption and
> interrupt enabled, this violates kprobe requirements. Fix this issue
> by keeping interrupts disabled for BREAKPOINT exception.
Not only while the breakpoint exception but also until the end of
the single step (maybe you are using __BUG_INSN_32 ??) need to be
disable interrupts. Can this do that?
Thank you,
>
> Fixes: c22b0bcb1dd0 ("riscv: Add kprobes supported")
> Signed-off-by: Jisheng Zhang <[email protected]>
> ---
> arch/riscv/kernel/entry.S | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
> index 744f3209c48d..4114b65698ec 100644
> --- a/arch/riscv/kernel/entry.S
> +++ b/arch/riscv/kernel/entry.S
> @@ -130,6 +130,8 @@ skip_context_tracking:
> */
> andi t0, s1, SR_PIE
> beqz t0, 1f
> + li t0, EXC_BREAKPOINT
> + beq s4, t0, 1f
> #ifdef CONFIG_TRACE_IRQFLAGS
> call trace_hardirqs_on
> #endif
> --
> 2.31.0
>
>
--
Masami Hiramatsu <[email protected]>
On Tue, 30 Mar 2021 18:33:16 +0900
Masami Hiramatsu <[email protected]> wrote:
> Hi Jisheng,
Hi Masami,
>
> On Tue, 30 Mar 2021 02:16:24 +0800
> Jisheng Zhang <[email protected]> wrote:
>
> > From: Jisheng Zhang <[email protected]>
> >
> > Current riscv's kprobe handlers are run with both preemption and
> > interrupt enabled, this violates kprobe requirements. Fix this issue
> > by keeping interrupts disabled for BREAKPOINT exception.
>
> Not only while the breakpoint exception but also until the end of
> the single step (maybe you are using __BUG_INSN_32 ??) need to be
> disable interrupts. Can this do that?
>
interrupt is disabled during "single step" by kprobes_save_local_irqflag()
and kprobes_restore_local_irqflag(). The code flow looks like:
do_trap_break() // for bp
kprobe_breakpoint_handler()
setup_singlestep()
kprobes_restore_local_irqflag()
do_trap_break() // for ss
kprobe_single_step_handler()
kprobes_restore_local_irqflag()
Thanks
Hi,
On Wed, 31 Mar 2021 22:22:44 +0800
Jisheng Zhang <[email protected]> wrote:
> On Tue, 30 Mar 2021 18:33:16 +0900
> Masami Hiramatsu <[email protected]> wrote:
>
> > Hi Jisheng,
>
> Hi Masami,
>
> >
> > On Tue, 30 Mar 2021 02:16:24 +0800
> > Jisheng Zhang <[email protected]> wrote:
> >
> > > From: Jisheng Zhang <[email protected]>
> > >
> > > Current riscv's kprobe handlers are run with both preemption and
> > > interrupt enabled, this violates kprobe requirements. Fix this issue
> > > by keeping interrupts disabled for BREAKPOINT exception.
> >
> > Not only while the breakpoint exception but also until the end of
> > the single step (maybe you are using __BUG_INSN_32 ??) need to be
> > disable interrupts. Can this do that?
> >
>
> interrupt is disabled during "single step" by kprobes_save_local_irqflag()
> and kprobes_restore_local_irqflag(). The code flow looks like:
>
> do_trap_break() // for bp
> kprobe_breakpoint_handler()
> setup_singlestep()
> kprobes_restore_local_irqflag()
>
> do_trap_break() // for ss
> kprobe_single_step_handler()
> kprobes_restore_local_irqflag()
OK, thanks for the confirmation!
Reviewed-by: Masami Hiramatsu <[email protected]>
Thanks,
--
Masami Hiramatsu <[email protected]>
Hi Jisheng,
?? 2021/3/31 22:22, Jisheng Zhang д??:
> On Tue, 30 Mar 2021 18:33:16 +0900
> Masami Hiramatsu <[email protected]> wrote:
>
>> Hi Jisheng,
>
> Hi Masami,
>
>>
>> On Tue, 30 Mar 2021 02:16:24 +0800
>> Jisheng Zhang <[email protected]> wrote:
>>
>>> From: Jisheng Zhang <[email protected]>
>>>
>>> Current riscv's kprobe handlers are run with both preemption and
>>> interrupt enabled, this violates kprobe requirements. Fix this issue
>>> by keeping interrupts disabled for BREAKPOINT exception.
>>
>> Not only while the breakpoint exception but also until the end of
>> the single step (maybe you are using __BUG_INSN_32 ??) need to be
>> disable interrupts. Can this do that?
>>
>
> interrupt is disabled during "single step" by kprobes_save_local_irqflag()
> and kprobes_restore_local_irqflag(). The code flow looks like:
>
> do_trap_break() // for bp
> kprobe_breakpoint_handler()
> setup_singlestep()
> kprobes_restore_local_irqflag()
>
> do_trap_break() // for ss
> kprobe_single_step_handler()
> kprobes_restore_local_irqflag()
Recently, kernel hit BUG_ON() on QEMU after I install a probe at "sys_read" via kprobe,
accoriding to my debugging and analysis it looks like caused by the "irq disable" operation for single-stepping.
I present a detailed description about this problem in the mail with title "[PATCH] riscv/kprobe: fix kernel panic when invoking sys_read traced by kprobe".
Looking forward to some feedback,Thanks.
BR,
Liao Chang
>
> Thanks
>
>
> _______________________________________________
> linux-riscv mailing list
> [email protected]
> http://lists.infradead.org/mailman/listinfo/linux-riscv
> .
>
On Thu, 1 Apr 2021 16:49:47 +0800
"liaochang (A)" <[email protected]> wrote:
> Hi Jisheng,
Hi,
>
> 在 2021/3/31 22:22, Jisheng Zhang 写道:
> > On Tue, 30 Mar 2021 18:33:16 +0900
> > Masami Hiramatsu <[email protected]> wrote:
> >
> >> Hi Jisheng,
> >
> > Hi Masami,
> >
> >>
> >> On Tue, 30 Mar 2021 02:16:24 +0800
> >> Jisheng Zhang <[email protected]> wrote:
> >>
> >>> From: Jisheng Zhang <[email protected]>
> >>>
> >>> Current riscv's kprobe handlers are run with both preemption and
> >>> interrupt enabled, this violates kprobe requirements. Fix this issue
> >>> by keeping interrupts disabled for BREAKPOINT exception.
> >>
> >> Not only while the breakpoint exception but also until the end of
> >> the single step (maybe you are using __BUG_INSN_32 ??) need to be
> >> disable interrupts. Can this do that?
> >>
> >
> > interrupt is disabled during "single step" by kprobes_save_local_irqflag()
> > and kprobes_restore_local_irqflag(). The code flow looks like:
> >
> > do_trap_break() // for bp
> > kprobe_breakpoint_handler()
> > setup_singlestep()
> > kprobes_restore_local_irqflag()
> >
> > do_trap_break() // for ss
> > kprobe_single_step_handler()
> > kprobes_restore_local_irqflag()
>
> Recently, kernel hit BUG_ON() on QEMU after I install a probe at "sys_read" via kprobe,
TIPS: Each line should not exceed 80 chars
> accoriding to my debugging and analysis it looks like caused by the "irq disable" operation for single-stepping.
>
> I present a detailed description about this problem in the mail with title "[PATCH] riscv/kprobe: fix kernel panic when invoking sys_read traced by kprobe".
> Looking forward to some feedback,Thanks.
>
I will comment that patch.
thanks
On Thu, 1 Apr 2021, Masami Hiramatsu wrote:
> > > > Current riscv's kprobe handlers are run with both preemption and
> > > > interrupt enabled, this violates kprobe requirements. Fix this issue
> > > > by keeping interrupts disabled for BREAKPOINT exception.
> > >
> > > Not only while the breakpoint exception but also until the end of
> > > the single step (maybe you are using __BUG_INSN_32 ??) need to be
> > > disable interrupts. Can this do that?
> > >
> >
> > interrupt is disabled during "single step" by kprobes_save_local_irqflag()
> > and kprobes_restore_local_irqflag(). The code flow looks like:
> >
> > do_trap_break() // for bp
> > kprobe_breakpoint_handler()
> > setup_singlestep()
> > kprobes_restore_local_irqflag()
> >
> > do_trap_break() // for ss
> > kprobe_single_step_handler()
> > kprobes_restore_local_irqflag()
>
> OK, thanks for the confirmation!
Is this approach guaranteed to keep interrupt handling latency low enough
for the system not to be negatively affected, e.g. for the purpose of NTP
timekeeping?
Maciej
Hi Jisheng,
在 2021/4/2 21:32, Jisheng Zhang 写道:
> On Thu, 1 Apr 2021 16:49:47 +0800
> "liaochang (A)" <[email protected]> wrote:
>
>> Hi Jisheng,
>
> Hi,
>
>>
>> 在 2021/3/31 22:22, Jisheng Zhang 写道:
>>> On Tue, 30 Mar 2021 18:33:16 +0900
>>> Masami Hiramatsu <[email protected]> wrote:
>>>
>>>> Hi Jisheng,
>>>
>>> Hi Masami,
>>>
>>>>
>>>> On Tue, 30 Mar 2021 02:16:24 +0800
>>>> Jisheng Zhang <[email protected]> wrote:
>>>>
>>>>> From: Jisheng Zhang <[email protected]>
>>>>>
>>>>> Current riscv's kprobe handlers are run with both preemption and
>>>>> interrupt enabled, this violates kprobe requirements. Fix this issue
>>>>> by keeping interrupts disabled for BREAKPOINT exception.
>>>>
>>>> Not only while the breakpoint exception but also until the end of
>>>> the single step (maybe you are using __BUG_INSN_32 ??) need to be
>>>> disable interrupts. Can this do that?
>>>>
>>>
>>> interrupt is disabled during "single step" by kprobes_save_local_irqflag()
>>> and kprobes_restore_local_irqflag(). The code flow looks like:
>>>
>>> do_trap_break() // for bp
>>> kprobe_breakpoint_handler()
>>> setup_singlestep()
>>> kprobes_restore_local_irqflag()
>>>
>>> do_trap_break() // for ss
>>> kprobe_single_step_handler()
>>> kprobes_restore_local_irqflag()
>>
>> Recently, kernel hit BUG_ON() on QEMU after I install a probe at "sys_read" via kprobe,
>
> TIPS: Each line should not exceed 80 chars
>
>> accoriding to my debugging and analysis it looks like caused by the "irq disable" operation for single-stepping.
>>
>> I present a detailed description about this problem in the mail with title "[PATCH] riscv/kprobe: fix kernel panic when invoking sys_read traced by kprobe".
>> Looking forward to some feedback,Thanks.
>>
>
> I will comment that patch.
Thanks for your reminder.
>
> thanks
>
> .
>
On Sat, 3 Apr 2021 20:30:53 +0200 (CEST)
"Maciej W. Rozycki" <[email protected]> wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> On Thu, 1 Apr 2021, Masami Hiramatsu wrote:
>
> > > > > Current riscv's kprobe handlers are run with both preemption and
> > > > > interrupt enabled, this violates kprobe requirements. Fix this issue
> > > > > by keeping interrupts disabled for BREAKPOINT exception.
> > > >
> > > > Not only while the breakpoint exception but also until the end of
> > > > the single step (maybe you are using __BUG_INSN_32 ??) need to be
> > > > disable interrupts. Can this do that?
> > > >
> > >
> > > interrupt is disabled during "single step" by kprobes_save_local_irqflag()
> > > and kprobes_restore_local_irqflag(). The code flow looks like:
> > >
> > > do_trap_break() // for bp
> > > kprobe_breakpoint_handler()
> > > setup_singlestep()
> > > kprobes_restore_local_irqflag()
> > >
> > > do_trap_break() // for ss
> > > kprobe_single_step_handler()
> > > kprobes_restore_local_irqflag()
> >
> > OK, thanks for the confirmation!
>
> Is this approach guaranteed to keep interrupt handling latency low enough
> for the system not to be negatively affected, e.g. for the purpose of NTP
> timekeeping?
IMHO, interrupt latency can't be ensured if kprobes is triggered.
thanks
On Thu, 8 Apr 2021 19:23:48 +0800
Jisheng Zhang <[email protected]> wrote:
> On Sat, 3 Apr 2021 20:30:53 +0200 (CEST)
> "Maciej W. Rozycki" <[email protected]> wrote:
>
> > CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
> >
> >
> > On Thu, 1 Apr 2021, Masami Hiramatsu wrote:
> >
> > > > > > Current riscv's kprobe handlers are run with both preemption and
> > > > > > interrupt enabled, this violates kprobe requirements. Fix this issue
> > > > > > by keeping interrupts disabled for BREAKPOINT exception.
> > > > >
> > > > > Not only while the breakpoint exception but also until the end of
> > > > > the single step (maybe you are using __BUG_INSN_32 ??) need to be
> > > > > disable interrupts. Can this do that?
> > > > >
> > > >
> > > > interrupt is disabled during "single step" by kprobes_save_local_irqflag()
> > > > and kprobes_restore_local_irqflag(). The code flow looks like:
> > > >
> > > > do_trap_break() // for bp
> > > > kprobe_breakpoint_handler()
> > > > setup_singlestep()
> > > > kprobes_restore_local_irqflag()
> > > >
> > > > do_trap_break() // for ss
> > > > kprobe_single_step_handler()
> > > > kprobes_restore_local_irqflag()
> > >
> > > OK, thanks for the confirmation!
> >
> > Is this approach guaranteed to keep interrupt handling latency low enough
> > for the system not to be negatively affected, e.g. for the purpose of NTP
> > timekeeping?
>
> IMHO, interrupt latency can't be ensured if kprobes is triggered.
Indeed. The latency depends on what the kprobe user handler does. Of course
it must be as minimal as possible... On x86, it is less than a few microseconds.
Thus it may be a jitter on hard realtime system, but not a big issue on
usual system like NTP timekeeping.
Thank you,
--
Masami Hiramatsu <[email protected]>
On Fri, 9 Apr 2021, Masami Hiramatsu wrote:
> > > Is this approach guaranteed to keep interrupt handling latency low enough
> > > for the system not to be negatively affected, e.g. for the purpose of NTP
> > > timekeeping?
> >
> > IMHO, interrupt latency can't be ensured if kprobes is triggered.
>
> Indeed. The latency depends on what the kprobe user handler does. Of course
> it must be as minimal as possible... On x86, it is less than a few microseconds.
> Thus it may be a jitter on hard realtime system, but not a big issue on
> usual system like NTP timekeeping.
Ack. Assuming that the breakpoint exception will only disable interrupts
if kprobes are in use it seems reasonable to me.
Thanks for double-checking.
Maciej
On Mon, 29 Mar 2021 11:16:24 PDT (-0700), [email protected] wrote:
> From: Jisheng Zhang <[email protected]>
>
> Current riscv's kprobe handlers are run with both preemption and
> interrupt enabled, this violates kprobe requirements. Fix this issue
> by keeping interrupts disabled for BREAKPOINT exception.
>
> Fixes: c22b0bcb1dd0 ("riscv: Add kprobes supported")
> Signed-off-by: Jisheng Zhang <[email protected]>
> ---
> arch/riscv/kernel/entry.S | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
> index 744f3209c48d..4114b65698ec 100644
> --- a/arch/riscv/kernel/entry.S
> +++ b/arch/riscv/kernel/entry.S
> @@ -130,6 +130,8 @@ skip_context_tracking:
> */
> andi t0, s1, SR_PIE
> beqz t0, 1f
> + li t0, EXC_BREAKPOINT
> + beq s4, t0, 1f
> #ifdef CONFIG_TRACE_IRQFLAGS
> call trace_hardirqs_on
> #endif
This is on fixes, with a comment as otherwise I'm just going to forget
why.