On Tue, Jun 16, 2020 at 06:22:10AM -0700, Andy Lutomirski wrote:
>
> > On Jun 16, 2020, at 1:44 AM, Thomas Gleixner <[email protected]> wrote:
> >
> > kernel test robot <[email protected]> writes:
> >> FYI, we noticed the following commit (built with gcc-9):
> >>
> >> commit: 2bbc68f8373c0631ebf137f376fbea00e8086be7 ("x86/entry: Convert Debug exception to IDTENTRY_DB")
> >> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
> >
> > Is the head of linux.git exposing the same problem or is this an
> > intermittent failure, which only affects bisectability?
>
> It sure looks deterministic:
>
> ptrace08.c:62: BROK: Cannot find address of kernel symbol "do_debug"
ROFL
Hi!
> > >> FYI, we noticed the following commit (built with gcc-9):
> > >>
> > >> commit: 2bbc68f8373c0631ebf137f376fbea00e8086be7 ("x86/entry: Convert Debug exception to IDTENTRY_DB")
> > >> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
> > >
> > > Is the head of linux.git exposing the same problem or is this an
> > > intermittent failure, which only affects bisectability?
> >
> > It sure looks deterministic:
> >
> > ptrace08.c:62: BROK: Cannot find address of kernel symbol "do_debug"
>
> ROFL
It's nice to have a good laugh, however I would really appreciate if any
of you would help me to fix the test.
The test in question is a regression test for:
commit f67b15037a7a50c57f72e69a6d59941ad90a0f0f
Author: Linus Torvalds <[email protected]>
Date: Mon Mar 26 15:39:07 2018 -1000
perf/hwbp: Simplify the perf-hwbp code, fix documentation
Annoyingly, modify_user_hw_breakpoint() unnecessarily complicates the
modification of a breakpoint - simplify it and remove the pointless
local variables.
And as far as I can tell it uses ptrace() with PTRACE_POKEUSER in order to
trigger it. But I'm kind of lost on how exactly we trigger the kernel
crash.
What is does is to write:
(void*)1 to u_debugreg[0]
(void*)1 to u_debugreg[7]
do_debug addr to u_debugreg[0]
Looking at the kernel code the write to register 7 enables the breakpoints and
what we attempt here is to change an invalid address to a valid one after we
enabled the breakpoint but that's as far I can go.
So does anyone has an idea how to trigger the bug without the do_debug function
address? Would any valid kernel function address suffice?
--
Cyril Hrubis
[email protected]
On Wed, Jun 17, 2020 at 6:17 AM Cyril Hrubis <[email protected]> wrote:
>
> Hi!
> > > >> FYI, we noticed the following commit (built with gcc-9):
> > > >>
> > > >> commit: 2bbc68f8373c0631ebf137f376fbea00e8086be7 ("x86/entry: Convert Debug exception to IDTENTRY_DB")
> > > >> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
> > > >
> > > > Is the head of linux.git exposing the same problem or is this an
> > > > intermittent failure, which only affects bisectability?
> > >
> > > It sure looks deterministic:
> > >
> > > ptrace08.c:62: BROK: Cannot find address of kernel symbol "do_debug"
> >
> > ROFL
>
> It's nice to have a good laugh, however I would really appreciate if any
> of you would help me to fix the test.
>
> The test in question is a regression test for:
>
> commit f67b15037a7a50c57f72e69a6d59941ad90a0f0f
> Author: Linus Torvalds <[email protected]>
> Date: Mon Mar 26 15:39:07 2018 -1000
>
> perf/hwbp: Simplify the perf-hwbp code, fix documentation
>
> Annoyingly, modify_user_hw_breakpoint() unnecessarily complicates the
> modification of a breakpoint - simplify it and remove the pointless
> local variables.
>
> And as far as I can tell it uses ptrace() with PTRACE_POKEUSER in order to
> trigger it. But I'm kind of lost on how exactly we trigger the kernel
> crash.
>
> What is does is to write:
>
> (void*)1 to u_debugreg[0]
> (void*)1 to u_debugreg[7]
> do_debug addr to u_debugreg[0]
>
> Looking at the kernel code the write to register 7 enables the breakpoints and
> what we attempt here is to change an invalid address to a valid one after we
> enabled the breakpoint but that's as far I can go.
>
> So does anyone has an idea how to trigger the bug without the do_debug function
> address? Would any valid kernel function address suffice?
>
do_debug is a bit of a red herring here. ptrace should not be able to
put a breakpoint on a kernel address, period. I would just pick a
fixed address that's in the kernel text range or even just in the
pre-KASLR text range and make sure it gets rejected. Maybe try a few
different addresses for good measure.
--Andy
Cyril Hrubis <[email protected]> writes:
> What is does is to write:
>
> (void*)1 to u_debugreg[0]
> (void*)1 to u_debugreg[7]
> do_debug addr to u_debugreg[0]
>
> Looking at the kernel code the write to register 7 enables the breakpoints and
> what we attempt here is to change an invalid address to a valid one after we
> enabled the breakpoint but that's as far I can go.
>
> So does anyone has an idea how to trigger the bug without the do_debug function
> address? Would any valid kernel function address suffice?
According to https://www.openwall.com/lists/oss-security/2018/05/01/3
the trigger is to set the breakpoint to do_debug() and then execute
INT1, aka. ICEBP which ends up in do_debug() ....
In principle each kernel address is ok, but do_debug() is interesting
due to the recursion issue because user space can reach it by executing
INT1.
So you might check for exc_debug() if do_debug() is not available and
make the whole thing fail gracefully with a usefu error message.
Thanks,
tglx
On Fri, 19 Jun 2020 at 01:32, Thomas Gleixner <[email protected]> wrote:
>
> Cyril Hrubis <[email protected]> writes:
> > What is does is to write:
> >
> > (void*)1 to u_debugreg[0]
> > (void*)1 to u_debugreg[7]
> > do_debug addr to u_debugreg[0]
> >
> > Looking at the kernel code the write to register 7 enables the breakpoints and
> > what we attempt here is to change an invalid address to a valid one after we
> > enabled the breakpoint but that's as far I can go.
> >
> > So does anyone has an idea how to trigger the bug without the do_debug function
> > address? Would any valid kernel function address suffice?
>
> According to https://www.openwall.com/lists/oss-security/2018/05/01/3
> the trigger is to set the breakpoint to do_debug() and then execute
> INT1, aka. ICEBP which ends up in do_debug() ....
>
> In principle each kernel address is ok, but do_debug() is interesting
> due to the recursion issue because user space can reach it by executing
> INT1.
>
> So you might check for exc_debug() if do_debug() is not available and
> make the whole thing fail gracefully with a usefu error message.
My two cents,
LTP test case ptrace08 fails on x86_64 and i386.
ptrace08.c:62: BROK: Cannot find address of kernel symbol \"do_debug\"
This error is coming from test case setup
KERNEL_SYM = do_debug
if (strcmp(symname, KERNEL_SYM))
tst_brk(TBROK, "Cannot find address of kernel symbol \"%s\"",
KERNEL_SYM);
Test case got pass when DEBUG_INFO config enabled
CONFIG_DEBUG_INFO=y
ptrace08.c:68: INFO: Kernel symbol \"do_debug\" found at 0xd8898410
Full test log,
https://lkft.validation.linaro.org/scheduler/job/1483117#L1325
ref:
https://bugs.linaro.org/show_bug.cgi?id=5651#c1
Hi!
> do_debug is a bit of a red herring here. ptrace should not be able to
> put a breakpoint on a kernel address, period. I would just pick a
> fixed address that's in the kernel text range or even just in the
> pre-KASLR text range and make sure it gets rejected. Maybe try a few
> different addresses for good measure.
I've looked at the code and it seems like this would be a bit more
complicated since the breakpoint is set by an accident in a race and the
call still fails. Which is why the test triggers the breakpoint and
causes infinite loop in the kernel...
I guess that we could instead read back the address with
PTRACE_PEEKUSER, so something as:
break_addr = ptrace(PTRACE_PEEKUSER, child_pid,
(void *)offsetof(struct user, u_debugreg[0]),
NULL);
if (break_addr == kernel_addr)
tst_res(TFAIL, "ptrace() set break on a kernel address");
--
Cyril Hrubis
[email protected]
Hi!
> > do_debug is a bit of a red herring here. ptrace should not be able to
> > put a breakpoint on a kernel address, period. I would just pick a
> > fixed address that's in the kernel text range or even just in the
> > pre-KASLR text range and make sure it gets rejected. Maybe try a few
> > different addresses for good measure.
>
> I've looked at the code and it seems like this would be a bit more
> complicated since the breakpoint is set by an accident in a race and the
> call still fails. Which is why the test triggers the breakpoint and
> causes infinite loop in the kernel...
>
> I guess that we could instead read back the address with
> PTRACE_PEEKUSER, so something as:
>
>
> break_addr = ptrace(PTRACE_PEEKUSER, child_pid,
> (void *)offsetof(struct user, u_debugreg[0]),
> NULL);
>
> if (break_addr == kernel_addr)
> tst_res(TFAIL, "ptrace() set break on a kernel address");
So this works actually nicely, even better than the original code.
Any hints on how to select a fixed address in the kernel range as you
pointed out in one of the previous emails? I guess that this would end
up as a per-architecture mess of ifdefs if we wanted to hardcode it.
--
Cyril Hrubis
[email protected]
On Fri, Aug 14, 2020 at 7:58 AM Cyril Hrubis <[email protected]> wrote:
>
> Hi!
> > > do_debug is a bit of a red herring here. ptrace should not be able to
> > > put a breakpoint on a kernel address, period. I would just pick a
> > > fixed address that's in the kernel text range or even just in the
> > > pre-KASLR text range and make sure it gets rejected. Maybe try a few
> > > different addresses for good measure.
> >
> > I've looked at the code and it seems like this would be a bit more
> > complicated since the breakpoint is set by an accident in a race and the
> > call still fails. Which is why the test triggers the breakpoint and
> > causes infinite loop in the kernel...
> >
> > I guess that we could instead read back the address with
> > PTRACE_PEEKUSER, so something as:
> >
> >
> > break_addr = ptrace(PTRACE_PEEKUSER, child_pid,
> > (void *)offsetof(struct user, u_debugreg[0]),
> > NULL);
> >
> > if (break_addr == kernel_addr)
> > tst_res(TFAIL, "ptrace() set break on a kernel address");
>
> So this works actually nicely, even better than the original code.
>
> Any hints on how to select a fixed address in the kernel range as you
> pointed out in one of the previous emails? I guess that this would end
> up as a per-architecture mess of ifdefs if we wanted to hardcode it.
>
It's fundamentally architecture dependent. Sane architectures like
s390x don't even have this concept.
--Andy