hi,
sorry, I'm so late to reply your E-mail, a little busy these days.
On Sun, May 31, 2009 at 2:47 PM, Wang Liming <[email protected]> wrote:
> hi Wu,
>
> I have one question about dynamic ftrace support for modules on mips arch.
> I have also ported a dynamic ftrace of mips before, but when I using modules
> with dynamic ftrace enabled in mips, I have following error:
>
> # insmod ./usbcore
> module usbcore: relocation overflow
> insmod: cannot insert './usbcore.ko': invalid module format
> #
>
> I have no time to test your patch, but I think your patch also has this
> problem.
>
> Below I explain possibility of this problem in details:
>
> We know that on mips, kernel symbol will locate in 0x80000000~0x90000000,
> and modules' symbol will locate in >0xc0000000.
>
> On mips, gcc will insert following assembler code before every functions:
>
> move at,ra
> jal 0 <usb_ifnum_to_if>
> 10: R_MIPS_26 _mcount
>
>
> Here "_mcount" is a kernel symbol in 0x80000000~0x90000000, for
> example at 0x8027a740.
>
> When loading a module, kernel should relocate "jal 0" instruction to jump to
> the kernel's "_mcount" address, 0x8027a740. But "jal 0" is a 26 bits offset
> jump
> instruction, it only can be relocated to jump to modules' symbol address
> space(>0xc0000000) and it can't be relocated to jump to kernel symbol
> address
> space(such as 0x8027a740), because offset is 32 bits(is 0xc0000000 -
> 0x80000000). We can't modify this instruction to jump to a 32 offset address
> except that we modify gcc compiler to insert other assembler codes.
>
> So, when installing modules, the kernel will try to relocate "jal 0"
> instruction
> to instruction of jumping to >0xc0000000, but it finds that _mcount's
> address
> is 0x8027a740 and then print error indicated that it can't be reloacted.
>
> The following source code is the checking place:
>
>
> ----------arch/mips/kernel/module.c:apply_r_mips_26_rel()-------------------
>
> static int apply_r_mips_26_rel(struct module *me, u32 *location, Elf_Addr v)
> {
> ...
> if ((v & 0xf0000000) != (((unsigned long)location + 4) & 0xf0000000)) {
> printk(KERN_ERR
> "module %s: relocation overflow, v:%x, location:%x\n",
> me->name, v, (unsigned long)location);
> return -ENOEXEC;
> }
>
> *location = (*location & ~0x03ffffff) |
> ((*location + (v >> 2)) & 0x03ffffff);
>
> }
> ----------arch/mips/kernel/module.c:apply_r_mips_26_rel()-------------------
>
> v is kernel _mcount's address, location is the address of the instrution
> that should be relocated;
>
> To resolve this problem, we may need to do more work, either on gcc or on
> the kernel. So I want to hear your test result and if you have solution,
> please let me know.
>
yes, current version of mips-specific dynamic ftrace not support modules yet.
there is similar solution implemented in PowerPC(something named trampoline),
although I did not look into it, but I'm sure we can implement the
mips-specific one
via imitating it.
and currently, I'm planning to fix it in the -v3 of mips-specific
ftrace support.
best wishes,
Wu Zhangjin
wu zhangjin wrote:
> hi,
>
> sorry, I'm so late to reply your E-mail, a little busy these days.
>>
>> }
>> ----------arch/mips/kernel/module.c:apply_r_mips_26_rel()-------------------
>>
>> v is kernel _mcount's address, location is the address of the instrution
>> that should be relocated;
>>
>> To resolve this problem, we may need to do more work, either on gcc or on
>> the kernel. So I want to hear your test result and if you have solution,
>> please let me know.
>>
>
> yes, current version of mips-specific dynamic ftrace not support modules yet.
>
> there is similar solution implemented in PowerPC(something named trampoline),
> although I did not look into it, but I'm sure we can implement the
> mips-specific one
> via imitating it.
Good hit. I may have a look on Powerpc implementation.
>
> and currently, I'm planning to fix it in the -v3 of mips-specific
> ftrace support.
OK. I'm looking forward to your -v3 patches.
Thanks.
Liming Wang
>
> best wishes,
> Wu Zhangjin
>
On Wed, 3 Jun 2009, Wang Liming wrote:
> wu zhangjin wrote:
> > hi,
> >
> > sorry, I'm so late to reply your E-mail, a little busy these days.
> > >
> > > }
> > >
> > > ----------arch/mips/kernel/module.c:apply_r_mips_26_rel()-------------------
> > >
> > > v is kernel _mcount's address, location is the address of the instrution
> > > that should be relocated;
> > >
> > > To resolve this problem, we may need to do more work, either on gcc or on
> > > the kernel. So I want to hear your test result and if you have solution,
> > > please let me know.
> > >
> >
> > yes, current version of mips-specific dynamic ftrace not support modules
> > yet.
> >
> > there is similar solution implemented in PowerPC(something named
> > trampoline),
> > although I did not look into it, but I'm sure we can implement the
> > mips-specific one
> > via imitating it.
> Good hit. I may have a look on Powerpc implementation.
Note, PowerPC uses a trampoline from modules to kernel core. I think MIPS
just calls mcount differently. That is, it does a full 32bit address call
(64 bit for 64 bit archs?). Something like:
lui v1, _mcount
addiu v1, v1, _mcount
jalr v1
addiu sp, sp, -8
Then a nop would not do. Due to preemption, we can not modify more than
one line. But you could modify it to:
b 1f
addiu v1, v1, _mcount
jalr v1
addiu sp, sp, -8
1:
Clobbering v1 should not be an issue since it is already used to store
_mcount. That is, we still do the addiu v1,v1,_mcount with that branch.
But v1 should be ignored.
-- Steve
hi, Steven
> Note, PowerPC uses a trampoline from modules to kernel core. I think MIPS
> just calls mcount differently. That is, it does a full 32bit address call
> (64 bit for 64 bit archs?). Something like:
>
> lui v1, _mcount
> addiu v1, v1, _mcount
> jalr v1
> addiu sp, sp, -8
>
> Then a nop would not do. Due to preemption, we can not modify more than
> one line. But you could modify it to:
>
> b 1f
> addiu v1, v1, _mcount
> jalr v1
> addiu sp, sp, -8
> 1:
>
> Clobbering v1 should not be an issue since it is already used to store
> _mcount. That is, we still do the addiu v1,v1,_mcount with that branch.
> But v1 should be ignored.
>
sorry, I'm not clear what you said above, could you give more
explanation on it, thanks!
I think "jal _mcount" itself will not work in static and dynamic ftrace.
something like what you described above should be used instead. perhaps
a PATCH can be sent to gcc or we use some tricks in kernel.
if we not use the method of modifying the kernel code in run time(avoid
preemption?), can we link something as following to the modules:
_mcount_trampoline:
PTR_LA t0, _mcount
jalr t0
and then we replace the original "jal _mcount" in every module to "jal
_mcount_trampoline" with a perl script.
this solution maybe work for static ftrace. does it?
but in dynamic ftrace, when ftrace is enabled and set_filter_function is
set, the place of filtered function will be substituted to "jal
ftrace_caller", this will not work since ftrace_caller is a function
like _mcount, which will not reached in modules. so, this ftrace_caller
should be substituted to something like this:
ftrace_caller_trampoline:
PTR_LA t0, ftrace_caller
jalr t0
and ftrace_caller_trampoline should be linked to modules too. or we
substituted it to the above _mcount_trampoline, but change "PTR_LA t0,
_mcount" to "PTR_LA t0, ftrace_caller".
does this work?
thanks!
Wu Zhangjin