2009-10-20 15:31:14

by wu zhangjin

[permalink] [raw]
Subject: Re: ftrace for MIPS

added CC to linux-mips and lkml.

> On Tue, 2009-10-20 at 07:39 +0800, Wu Zhangjin wrote:
>
> > 3. to handle the non-leaf function(hijack the return address), we need
> > to get the stack address of the return address, but it's not easy to get
> > it in MIPS, search the return address in the stack space is not
> > reliable, searching the text is dangerous(pagefault..., have tried
> > probe_kernel_read(), just hang there!), so, a clean solution maybe
> > hacking gcc via pushing ra to 0(sp) or another "fixed"(fixed offset)
> > stack address or recording the offset and transfer it to _mcount.
>
> Have you figured out why you can't find the text? If mcount is called,
> you most definitely must have stored ra somewhere.
>
> As for the hang with probe kernel read, I wonder if you need to disable
> tracing before using it. Or at least have a way not to recurs. I'm
> looking at probe_kernel_read, and it looks like it would also be traced.
>
> Looking at x86 and powerpc, we hand do the probing.
>

Just added tracing_stop() and tracing_start() around
probe_kernel_read(), it works(not hang again), and i can get the stack
address of the ra register(return address) now, but failed when trying
to hijack the return address via writing &return_to_handler in the stack
address:

I can write hijack some of the addresses, but failed with this error at
last:

Unable to handle kernel paging request at 0000000000000000, epc =
0000000000000000, ra = 000000000000.

Need to check which registers is missing when saving/restoring for
_mcount:


NESTED(ftrace_graph_caller, PT_SIZE, ra)
MCOUNT_SAVE_REGS
PTR_S v0, PT_R2(sp)

MCOUNT_SET_ARGS
jal prepare_ftrace_return
nop

/* overwrite the parent as &return_to_handler: v0 -> $1(at) */
move $1, v0

PTR_L v0, PT_R2(sp)
MCOUNT_RESTORE_REGS
RETURN_BACK
END(ftrace_graph_caller)

.align 2
.globl return_to_handler
return_to_handler:
PTR_SUBU sp, PT_SIZE
PTR_S v0, PT_R2(sp)

jal ftrace_return_to_handler
nop

/* restore the real parent address: v0 -> ra */
move ra, v0

PTR_L v0, PT_R2(sp)
PTR_ADDIU sp, PT_SIZE

jr ra

...

.macro MCOUNT_SAVE_REGS
PTR_SUBU sp, PT_SIZE
PTR_S ra, PT_R31(sp)
PTR_S AT, PT_R1(sp)
PTR_S a0, PT_R4(sp)
PTR_S a1, PT_R5(sp)
PTR_S a2, PT_R6(sp)
PTR_S a3, PT_R7(sp)
#ifdef CONFIG_64BIT
PTR_S a4, PT_R8(sp)
PTR_S a5, PT_R9(sp)
PTR_S a6, PT_R10(sp)
PTR_S a7, PT_R11(sp)
#endif
.endm

.macro MCOUNT_RESTORE_REGS
PTR_L ra, PT_R31(sp)
PTR_L AT, PT_R1(sp)
PTR_L a0, PT_R4(sp)
PTR_L a1, PT_R5(sp)
PTR_L a2, PT_R6(sp)
PTR_L a3, PT_R7(sp)
#ifdef CONFIG_64BIT
PTR_L a4, PT_R8(sp)
PTR_L a5, PT_R9(sp)
PTR_L a6, PT_R10(sp)
PTR_L a7, PT_R11(sp)
#endif
PTR_ADDIU sp, PT_SIZE

Regards,
Wu Zhangjin


2009-10-20 16:21:53

by Steven Rostedt

[permalink] [raw]
Subject: Re: ftrace for MIPS

On Tue, 2009-10-20 at 23:31 +0800, Wu Zhangjin wrote:

> Just added tracing_stop() and tracing_start() around

That seems a bit heavy handed. I still think writing it in "asm" the way
x86 and powerpc do is the best.

> probe_kernel_read(), it works(not hang again), and i can get the stack
> address of the ra register(return address) now, but failed when trying
> to hijack the return address via writing &return_to_handler in the stack
> address:
>
> I can write hijack some of the addresses, but failed with this error at
> last:
>
> Unable to handle kernel paging request at 0000000000000000, epc =
> 0000000000000000, ra = 000000000000.

hmm, looks like you jumped to "0"

>
> Need to check which registers is missing when saving/restoring for
> _mcount:
>
>
> NESTED(ftrace_graph_caller, PT_SIZE, ra)
> MCOUNT_SAVE_REGS
> PTR_S v0, PT_R2(sp)
>
> MCOUNT_SET_ARGS
> jal prepare_ftrace_return
> nop
>
> /* overwrite the parent as &return_to_handler: v0 -> $1(at) */
> move $1, v0

I'm confused here? I'm not exactly sure what the above is doing. Is $1 a
register (AT)? And how is this register used before calling mcount?

>
> PTR_L v0, PT_R2(sp)
> MCOUNT_RESTORE_REGS
> RETURN_BACK
> END(ftrace_graph_caller)
>
> .align 2
> .globl return_to_handler
> return_to_handler:
> PTR_SUBU sp, PT_SIZE
> PTR_S v0, PT_R2(sp)

BTW, is v0 the only return register? I know x86 can return two different
registers depending on what it returns. What happens if a function
returns a 64 bit value on a 32bit box? Does it use two registers for
that?

-- Steve

>
> jal ftrace_return_to_handler
> nop
>
> /* restore the real parent address: v0 -> ra */
> move ra, v0
>
> PTR_L v0, PT_R2(sp)
> PTR_ADDIU sp, PT_SIZE
>
> jr ra
>
> ...
>
> .macro MCOUNT_SAVE_REGS
> PTR_SUBU sp, PT_SIZE
> PTR_S ra, PT_R31(sp)
> PTR_S AT, PT_R1(sp)
> PTR_S a0, PT_R4(sp)
> PTR_S a1, PT_R5(sp)
> PTR_S a2, PT_R6(sp)
> PTR_S a3, PT_R7(sp)
> #ifdef CONFIG_64BIT
> PTR_S a4, PT_R8(sp)
> PTR_S a5, PT_R9(sp)
> PTR_S a6, PT_R10(sp)
> PTR_S a7, PT_R11(sp)
> #endif
> .endm
>
> .macro MCOUNT_RESTORE_REGS
> PTR_L ra, PT_R31(sp)
> PTR_L AT, PT_R1(sp)
> PTR_L a0, PT_R4(sp)
> PTR_L a1, PT_R5(sp)
> PTR_L a2, PT_R6(sp)
> PTR_L a3, PT_R7(sp)
> #ifdef CONFIG_64BIT
> PTR_L a4, PT_R8(sp)
> PTR_L a5, PT_R9(sp)
> PTR_L a6, PT_R10(sp)
> PTR_L a7, PT_R11(sp)
> #endif
> PTR_ADDIU sp, PT_SIZE
>
> Regards,
> Wu Zhangjin
>

2009-10-21 02:33:21

by wu zhangjin

[permalink] [raw]
Subject: Re: ftrace for MIPS

Hi, all

Just made it(function graph tracer for MIPS) work :-)

The problem is that: the stack offset should be from 0 to PT_SIZE(304),
but I mask it with 0xff(256), which is totally wrong.

Here is an example, the stack address of ra(return address) should be
(s8 + ffbf0128 & 0xfff).

ffffffff801dad10 <do_sync_read>:
ffffffff801dad10: 67bdfed0 daddiu sp,sp,-304
ffffffff801dad14: ffbe0120 sd s8,288(sp)
ffffffff801dad18: 03a0f02d move s8,sp
ffffffff801dad1c: ffbf0128 sd ra,296(sp)
ffffffff801dad20: ffb30118 sd s3,280(sp)
ffffffff801dad24: ffb20110 sd s2,272(sp)
ffffffff801dad28: ffb10108 sd s1,264(sp)
ffffffff801dad2c: ffb00100 sd s0,256(sp)
ffffffff801dad30: 03e0082d move at,ra
ffffffff801dad34: 0c042ab0 jal ffffffff8010aac0
<_mcount>
ffffffff801dad38: 00020021 nop

Thanks! will send the patches out later.

Regards,
Wu Zhangjin

On Tue, 2009-10-20 at 12:21 -0400, Steven Rostedt wrote:
> On Tue, 2009-10-20 at 23:31 +0800, Wu Zhangjin wrote:
>
> > Just added tracing_stop() and tracing_start() around
>
> That seems a bit heavy handed. I still think writing it in "asm" the way
> x86 and powerpc do is the best.
>
> > probe_kernel_read(), it works(not hang again), and i can get the stack
> > address of the ra register(return address) now, but failed when trying
> > to hijack the return address via writing &return_to_handler in the stack
> > address:
> >
> > I can write hijack some of the addresses, but failed with this error at
> > last:
> >
> > Unable to handle kernel paging request at 0000000000000000, epc =
> > 0000000000000000, ra = 000000000000.
>
> hmm, looks like you jumped to "0"
>
> >
> > Need to check which registers is missing when saving/restoring for
> > _mcount:
> >
> >
> > NESTED(ftrace_graph_caller, PT_SIZE, ra)
> > MCOUNT_SAVE_REGS
> > PTR_S v0, PT_R2(sp)
> >
> > MCOUNT_SET_ARGS
> > jal prepare_ftrace_return
> > nop
> >
> > /* overwrite the parent as &return_to_handler: v0 -> $1(at) */
> > move $1, v0
>
> I'm confused here? I'm not exactly sure what the above is doing. Is $1 a
> register (AT)? And how is this register used before calling mcount?
>
> >
> > PTR_L v0, PT_R2(sp)
> > MCOUNT_RESTORE_REGS
> > RETURN_BACK
> > END(ftrace_graph_caller)
> >
> > .align 2
> > .globl return_to_handler
> > return_to_handler:
> > PTR_SUBU sp, PT_SIZE
> > PTR_S v0, PT_R2(sp)
>
> BTW, is v0 the only return register? I know x86 can return two different
> registers depending on what it returns. What happens if a function
> returns a 64 bit value on a 32bit box? Does it use two registers for
> that?
>
> -- Steve
>
> >
> > jal ftrace_return_to_handler
> > nop
> >
> > /* restore the real parent address: v0 -> ra */
> > move ra, v0
> >
> > PTR_L v0, PT_R2(sp)
> > PTR_ADDIU sp, PT_SIZE
> >
> > jr ra
> >
> > ...
> >
> > .macro MCOUNT_SAVE_REGS
> > PTR_SUBU sp, PT_SIZE
> > PTR_S ra, PT_R31(sp)
> > PTR_S AT, PT_R1(sp)
> > PTR_S a0, PT_R4(sp)
> > PTR_S a1, PT_R5(sp)
> > PTR_S a2, PT_R6(sp)
> > PTR_S a3, PT_R7(sp)
> > #ifdef CONFIG_64BIT
> > PTR_S a4, PT_R8(sp)
> > PTR_S a5, PT_R9(sp)
> > PTR_S a6, PT_R10(sp)
> > PTR_S a7, PT_R11(sp)
> > #endif
> > .endm
> >
> > .macro MCOUNT_RESTORE_REGS
> > PTR_L ra, PT_R31(sp)
> > PTR_L AT, PT_R1(sp)
> > PTR_L a0, PT_R4(sp)
> > PTR_L a1, PT_R5(sp)
> > PTR_L a2, PT_R6(sp)
> > PTR_L a3, PT_R7(sp)
> > #ifdef CONFIG_64BIT
> > PTR_L a4, PT_R8(sp)
> > PTR_L a5, PT_R9(sp)
> > PTR_L a6, PT_R10(sp)
> > PTR_L a7, PT_R11(sp)
> > #endif
> > PTR_ADDIU sp, PT_SIZE
> >
> > Regards,
> > Wu Zhangjin
> >
>

2009-10-21 02:48:05

by Steven Rostedt

[permalink] [raw]
Subject: Re: ftrace for MIPS

On Wed, 2009-10-21 at 10:33 +0800, Wu Zhangjin wrote:
> Hi, all
>
> Just made it(function graph tracer for MIPS) work :-)
>
> The problem is that: the stack offset should be from 0 to PT_SIZE(304),
> but I mask it with 0xff(256), which is totally wrong.
>
> Here is an example, the stack address of ra(return address) should be
> (s8 + ffbf0128 & 0xfff).
>
> ffffffff801dad10 <do_sync_read>:
> ffffffff801dad10: 67bdfed0 daddiu sp,sp,-304
> ffffffff801dad14: ffbe0120 sd s8,288(sp)
> ffffffff801dad18: 03a0f02d move s8,sp
> ffffffff801dad1c: ffbf0128 sd ra,296(sp)
> ffffffff801dad20: ffb30118 sd s3,280(sp)
> ffffffff801dad24: ffb20110 sd s2,272(sp)
> ffffffff801dad28: ffb10108 sd s1,264(sp)
> ffffffff801dad2c: ffb00100 sd s0,256(sp)
> ffffffff801dad30: 03e0082d move at,ra
> ffffffff801dad34: 0c042ab0 jal ffffffff8010aac0
> <_mcount>
> ffffffff801dad38: 00020021 nop
>
> Thanks! will send the patches out later.

Great to hear that it works! When I get my cross compiling working I'll
test out your patches.

I'll also probably update them to use the asm() over the probe and
tracing_disable(). That method is very inefficient.

Good work!

-- Steve

2009-10-21 13:14:17

by Sergei Shtylyov

[permalink] [raw]
Subject: Re: ftrace for MIPS

Hello.

Steven Rostedt wrote:

>>Need to check which registers is missing when saving/restoring for
>>_mcount:

>>NESTED(ftrace_graph_caller, PT_SIZE, ra)
>> MCOUNT_SAVE_REGS
>> PTR_S v0, PT_R2(sp)
>>
>> MCOUNT_SET_ARGS
>> jal prepare_ftrace_return
>> nop
>>
>> /* overwrite the parent as &return_to_handler: v0 -> $1(at) */
>> move $1, v0

> I'm confused here? I'm not exactly sure what the above is doing. Is $1 a
> register (AT)?

Yes.

> And how is this register used before calling mcount?

>> PTR_L v0, PT_R2(sp)
>> MCOUNT_RESTORE_REGS
>> RETURN_BACK
>> END(ftrace_graph_caller)

>> .align 2
>> .globl return_to_handler
>>return_to_handler:
>> PTR_SUBU sp, PT_SIZE
>> PTR_S v0, PT_R2(sp)

> BTW, is v0 the only return register? I know x86 can return two different
> registers depending on what it returns. What happens if a function
> returns a 64 bit value on a 32bit box? Does it use two registers for
> that?

Yes, there's also v1 register.

WBR, Sergei

2009-10-21 13:25:32

by wu zhangjin

[permalink] [raw]
Subject: Re: ftrace for MIPS

On Wed, 2009-10-21 at 17:14 +0400, Sergei Shtylyov wrote:
> Hello.
>
> Steven Rostedt wrote:
>
> >>Need to check which registers is missing when saving/restoring for
> >>_mcount:
>
> >>NESTED(ftrace_graph_caller, PT_SIZE, ra)
> >> MCOUNT_SAVE_REGS
> >> PTR_S v0, PT_R2(sp)
> >>
> >> MCOUNT_SET_ARGS
> >> jal prepare_ftrace_return
> >> nop
> >>
> >> /* overwrite the parent as &return_to_handler: v0 -> $1(at) */
> >> move $1, v0
>
> > I'm confused here? I'm not exactly sure what the above is doing. Is $1 a
> > register (AT)?
>
> Yes.

Have replaced it by AT, thanks!

>
> > And how is this register used before calling mcount?
>
> >> PTR_L v0, PT_R2(sp)
> >> MCOUNT_RESTORE_REGS
> >> RETURN_BACK
> >> END(ftrace_graph_caller)
>
> >> .align 2
> >> .globl return_to_handler
> >>return_to_handler:
> >> PTR_SUBU sp, PT_SIZE
> >> PTR_S v0, PT_R2(sp)
>
> > BTW, is v0 the only return register? I know x86 can return two different
> > registers depending on what it returns. What happens if a function
> > returns a 64 bit value on a 32bit box? Does it use two registers for
> > that?
>
> Yes, there's also v1 register.
>

Thanks, added.

Regards,
Wu Zhangjin