Hi,
during 'make modules' on s390x, I see lots of warnings about 'ignoring
changed section attributes for __ksymtab' that I have found to be the
result of changeset 1.373.196.1, where Kai changed the defaults for module
exports to 'no symbols exported'.
The problem is that there is a section '__ksymtab,"a"', while s390x
requires it to be '__ksymtab,"aw"' because modules must be compiled with
'-fpic' here, unlike afaics all the other architectures.
The patch below is a workaround for the Problem and should be
correct on all architectures, but I'd prefer if there was a nicer
way to do that.
Arnd <><
--- broken/include/linux/module.h 15 Oct 2002 07:55:01 -0000 1.8
+++ ugly/include/linux/module.h 15 Oct 2002 15:30:39 -0000
@@ -498,5 +498,9 @@
* "export all symbols" to modutils)
*/
+#ifndef __PIC__
__asm__(".section __ksymtab,\"a\"\n.previous");
+#else
+__asm__(".section __ksymtab,\"aw\"\n.previous");
+#endif
#endif
On Tue, 15 Oct 2002, Arnd Bergmann wrote:
> during 'make modules' on s390x, I see lots of warnings about 'ignoring
> changed section attributes for __ksymtab' that I have found to be the
> result of changeset 1.373.196.1, where Kai changed the defaults for module
> exports to 'no symbols exported'.
>
> The problem is that there is a section '__ksymtab,"a"', while s390x
> requires it to be '__ksymtab,"aw"' because modules must be compiled with
> '-fpic' here, unlike afaics all the other architectures.
Hmmh, there's a couple of things I don't understand, though they're most
likely there for a reason. First of all, why do you need -fpic at all?
kernel modules are not shared, they should get properly relocated when
insmod'ing them, so I don't see why you're doing that.
The next thing I do not understand is why -fpic has the effect of marking
the section writeable, does it make .text writeable as well? And what for?
> The patch below is a workaround for the Problem and should be
> correct on all architectures, but I'd prefer if there was a nicer
> way to do that.
>
> Arnd <><
>
> --- broken/include/linux/module.h 15 Oct 2002 07:55:01 -0000 1.8
> +++ ugly/include/linux/module.h 15 Oct 2002 15:30:39 -0000
> @@ -498,5 +498,9 @@
> * "export all symbols" to modutils)
> */
> +#ifndef __PIC__
> __asm__(".section __ksymtab,\"a\"\n.previous");
> +#else
> +__asm__(".section __ksymtab,\"aw\"\n.previous");
> +#endif
> #endif
Well, it's not too bad, and it's hidden away in a header, so it's fine
with me. Still, I don't understand yet why all this happens in the first
place.
--Kai
>> during 'make modules' on s390x, I see lots of warnings about 'ignoring
>> changed section attributes for __ksymtab' that I have found to be the
>> result of changeset 1.373.196.1, where Kai changed the defaults for
module
>> exports to 'no symbols exported'.
>>
>> The problem is that there is a section '__ksymtab,"a"', while s390x
>> requires it to be '__ksymtab,"aw"' because modules must be compiled with
>> '-fpic' here, unlike afaics all the other architectures.
>
>Hmmh, there's a couple of things I don't understand, though they're most
>likely there for a reason. First of all, why do you need -fpic at all?
>kernel modules are not shared, they should get properly relocated when
>insmod'ing them, so I don't see why you're doing that.
The reason for -fpic for module code lies in the compiler. To improve the
code we use the brasl and larl instructions for function calling and
addressing data. Unluckily these two instructions have a limited range
of +-4GB. For user space programs that means that a single shared object
may not be bigger than 4GB and that no non-fpic code is linked into
shared objects. With these two restrictions we are able to generate
much better code. There is one small problem though: kernel modules.
They get loaded into the vmalloc area which is located AFTER the main
memory. A machine with more than 4 GB of main memory therefore can't
load modules anymore because the calls and references to kernel structure
can't span the distance between vmalloc area and kernel image. To get
around this problem we compile kernel modules with -fpic and make the
modutils create plt stubs and got entries. Easy ?
>The next thing I do not understand is why -fpic has the effect of marking
>the section writeable, does it make .text writeable as well? And what for?
Because -fpic code likes to relocated absolute addresses.
blue skies,
Martin
Linux/390 Design & Development, IBM Deutschland Entwicklung GmbH
Sch?naicherstr. 220, D-71032 B?blingen, Telefon: 49 - (0)7031 - 16-2247
E-Mail: [email protected]
On Tue, 15 Oct 2002, Martin Schwidefsky wrote:
> The reason for -fpic for module code lies in the compiler. To improve the
> code we use the brasl and larl instructions for function calling and
> addressing data. Unluckily these two instructions have a limited range
> of +-4GB. For user space programs that means that a single shared object
> may not be bigger than 4GB and that no non-fpic code is linked into
> shared objects. With these two restrictions we are able to generate
> much better code. There is one small problem though: kernel modules.
> They get loaded into the vmalloc area which is located AFTER the main
> memory. A machine with more than 4 GB of main memory therefore can't
> load modules anymore because the calls and references to kernel structure
> can't span the distance between vmalloc area and kernel image. To get
> around this problem we compile kernel modules with -fpic and make the
> modutils create plt stubs and got entries. Easy ?
Makes me wonder how you deal with function pointers, where the functions
are possibly in a module - guess you have to use full 64 bit there?
Doesn't it possibly make sense to hack module loading to put modules into
< 4 GB as well, so you can use 32bit everywhere? Oh wait, or do use stub
functions at < 4 GB for function pointers as well? Well, whatever, I
obviously don't know much about .got, nor S390x for that matter.
> >The next thing I do not understand is why -fpic has the effect of marking
> >the section writeable, does it make .text writeable as well? And what for?
>
> Because -fpic code likes to relocated absolute addresses.
I still don't see why someone would want to muck with modifying .text in a
shared lib, and I'm pretty sure that __ksymtab apart from the initial
relocation should not need further modification, OTOH that's a moot point
anyway, since gcc apparently marks it writeable and I don't see an easy
way to change that.
So, as far as I'm concerned I don't really care, modify module.h as you
suggested or think about mapping module memory closer to the kernel, which
might be a win performance-wise anyway.
--Kai
On Tue, Oct 15, 2002 at 06:50:12PM +0200, Martin Schwidefsky wrote:
>
> >> during 'make modules' on s390x, I see lots of warnings about 'ignoring
> >> changed section attributes for __ksymtab' that I have found to be the
> >> result of changeset 1.373.196.1, where Kai changed the defaults for
> module
> >> exports to 'no symbols exported'.
> >>
> >> The problem is that there is a section '__ksymtab,"a"', while s390x
> >> requires it to be '__ksymtab,"aw"' because modules must be compiled with
> >> '-fpic' here, unlike afaics all the other architectures.
> >
> >Hmmh, there's a couple of things I don't understand, though they're most
> >likely there for a reason. First of all, why do you need -fpic at all?
> >kernel modules are not shared, they should get properly relocated when
> >insmod'ing them, so I don't see why you're doing that.
>
> The reason for -fpic for module code lies in the compiler. To improve the
> code we use the brasl and larl instructions for function calling and
> addressing data. Unluckily these two instructions have a limited range
> of +-4GB. For user space programs that means that a single shared object
> may not be bigger than 4GB and that no non-fpic code is linked into
> shared objects. With these two restrictions we are able to generate
> much better code. There is one small problem though: kernel modules.
> They get loaded into the vmalloc area which is located AFTER the main
> memory. A machine with more than 4 GB of main memory therefore can't
> load modules anymore because the calls and references to kernel structure
> can't span the distance between vmalloc area and kernel image. To get
> around this problem we compile kernel modules with -fpic and make the
> modutils create plt stubs and got entries. Easy ?
Is there any reason why you cannot implement module_map and module_unmap
on s390x like sparc64 or x86-64 does?
I don't think we'll have size of kernel + size of all modules approaching
2 or 4GB size any time soon on any architecture.
Jakub
Kai Germaschewski wrote:
>Makes me wonder how you deal with function pointers, where the functions
>are possibly in a module - guess you have to use full 64 bit there?
Indirect calls on s390x go like this: load 64-bit target address
into a register, call to the address in that register.
Direct calls go like this: call to the address that results from
adding this immediate 32-bit offset to the current instruction
address.
So function pointers are no problem, calls via the GOT are also no
problem (as the GOT basically just holds function pointers), but
direct calls without -fpic can't span more than 4 GB.
>Doesn't it possibly make sense to hack module loading to put modules into
>< 4 GB as well, so you can use 32bit everywhere?
Possibly; but just using -fpic was simple and worked ...
I'm not quite sure how to muck with the initial memory mapping
to free up a suitable range of addresses where to place the modules.
What happens to the physical memory that is already there?
>I still don't see why someone would want to muck with modifying .text in a
>shared lib, and I'm pretty sure that __ksymtab ***apart from the initial
>relocation*** should not need further modification
Well, modified is modified, even if only once ;-) If the section
were mapped read-only, the dynamic linker couldn't apply the relocation.
(At least on some platforms I guess; on Linux it would actually work
because ld.so is playing mprotect games in such cases.)
B.t.w. .text is still read-only (because there are no relocation to
the .text section of proper -fpic code).
Bye,
Ulrich
--
Dr. Ulrich Weigand
[email protected]