A while ago we reviewed the idea of whether or not to use a set of
distribution vanilla kernels to allow us to do compilation testing of
modules on the compat backport project [0] with the benefit being that
at least one Linux distribution was doing this work already and given
that this work was using plain vanilla kernels directly from
kernel.org, with no modifications. The other option that existed was
that each distro could provide their own set of kernel packages along
with their own kernel headers but it didn't seem like other Linux
distributions were doing this and the information was not easy to
gather. So we went with the Ubuntu vanilla kernels and Ozan later
figured out a way to extract deb packages in a distribution agnostic
manner [1] and later added an option to enable a way to install and
use this for ckmake which test compiles against 21 kernels without
requiring root [2].
This works swell as it allows anyone on any Linux distribution to test
compile against a series of older kernels, today 21 older kernels and
all vanilla kernels which means we get to focus and prioritize
upstream.
This has started working well except for the fact that as of the >=
v3.4 vanilla kernel releases, there are a few files which are binary
in the kernel headers which use dynamic libraries. Of interest to us
here are the ones required to build external modules. An example:
/lib/modules/3.4.4-030404-generic/build/scripts/genksyms/genksyms
It just so happens to be that the folks building these packages are
using a glibc >= 2.14 and although the *only* symbol I see being
required from glibc >= 2.14 being memcpy():
mcgrof@garbanzo ~/compat (git::master)$ objdump -T
/home/mcgrof/compat-ksrc/lib/modules/3.4.4-030404-generic/build/scripts/genksyms/genksyms
/home/mcgrof/compat-ksrc/lib/modules/3.4.4-030404-generic/build/scripts/genksyms/genksyms:
file format elf64-x86-64
DYNAMIC SYMBOL TABLE:
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.3.4 __snprintf_chk
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 free
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.3.4 __vfprintf_chk
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 __errno_location
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 ferror
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 isatty
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 fread
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 fclose
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 strlen
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.4 __stack_chk_fail
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 getopt_long
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 strchr
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 _IO_putc
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 fputs
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 fgetc
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 fputc
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 __strdup
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 __libc_start_main
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 strcmp
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 feof
0000000000000000 w D *UND* 0000000000000000 __gmon_start__
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 strtol
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 clearerr
Bleh:
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.14 memcpy
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 fileno
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 malloc
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 _IO_getc
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 ungetc
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 realloc
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.3.4 __printf_chk
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 fopen
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 perror
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 exit
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 fwrite
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.3.4 __fprintf_chk
000000000060b160 g DO .bss 0000000000000008 GLIBC_2.2.5 stdout
000000000060b168 g DO .bss 0000000000000008 GLIBC_2.2.5 stdin
000000000060b178 g DO .bss 0000000000000008 GLIBC_2.2.5 stderr
000000000060b170 g DO .bss 0000000000000008 GLIBC_2.2.5 optarg
It would be reasonable to simply require the users of ckmake to update
their libc version but even Debian sid does not yet seem to have libc
>= 2.14, not sure of others. The gains for requiring a glibc >= 2.14
only for memcpy() would seem rather small for this particular case so
I wanted to review the options to fix this.
---------------
1) Get the kernel headers to statically link against the required libraries
This options would enable any Linux distribution to use the packages provided
2) Add a LD_PRELOAD hack for mempcy() for running ckmake and friends
on the compat package
This doesn't seem so.. clean, but its the only thing I can currently control.
3) Simply require libc > 2.14 for folks who really want to help do
backport testing for kernels >= 3.4
I suppose this is the easiest but we likely could run into other
similar issues later.
---------------
Are there other options?
[0] https://lkml.org/lkml/2012/2/6/558
[1] https://github.com/mcgrof/compat/commit/fdbea5a6ce2cba6d6c8fb3eb998cffa5a9e3bf78
[2] https://github.com/mcgrof/compat/commit/e7d88d67040bbaa79720fa1b2dfc4fc24e4d004f
Luis
On Tue, Aug 7, 2012 at 10:28 PM, Luis R. Rodriguez <[email protected]> wrote:
> A while ago we reviewed the idea of whether or not to use a set of
> distribution vanilla kernels to allow us to do compilation testing of
> modules on the compat backport project [0] with the benefit being that
> at least one Linux distribution was doing this work already and given
> that this work was using plain vanilla kernels directly from
> kernel.org, with no modifications. The other option that existed was
> that each distro could provide their own set of kernel packages along
> with their own kernel headers but it didn't seem like other Linux
> distributions were doing this and the information was not easy to
> gather. So we went with the Ubuntu vanilla kernels and Ozan later
> figured out a way to extract deb packages in a distribution agnostic
> manner [1] and later added an option to enable a way to install and
> use this for ckmake which test compiles against 21 kernels without
> requiring root [2].
>
> This works swell as it allows anyone on any Linux distribution to test
> compile against a series of older kernels, today 21 older kernels and
> all vanilla kernels which means we get to focus and prioritize
> upstream.
>
> This has started working well except for the fact that as of the >=
> v3.4 vanilla kernel releases, there are a few files which are binary
> in the kernel headers which use dynamic libraries. Of interest to us
> here are the ones required to build external modules. An example:
>
> /lib/modules/3.4.4-030404-generic/build/scripts/genksyms/genksyms
>
> It just so happens to be that the folks building these packages are
> using a glibc >= 2.14 and although the *only* symbol I see being
> required from glibc >= 2.14 being memcpy():
>
> mcgrof@garbanzo ~/compat (git::master)$ objdump -T
> /home/mcgrof/compat-ksrc/lib/modules/3.4.4-030404-generic/build/scripts/genksyms/genksyms
>
> /home/mcgrof/compat-ksrc/lib/modules/3.4.4-030404-generic/build/scripts/genksyms/genksyms:
> file format elf64-x86-64
>
> DYNAMIC SYMBOL TABLE:
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.3.4 __snprintf_chk
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 free
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.3.4 __vfprintf_chk
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 __errno_location
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 ferror
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 isatty
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 fread
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 fclose
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 strlen
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.4 __stack_chk_fail
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 getopt_long
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 strchr
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 _IO_putc
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 fputs
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 fgetc
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 fputc
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 __strdup
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 __libc_start_main
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 strcmp
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 feof
> 0000000000000000 w D *UND* 0000000000000000 __gmon_start__
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 strtol
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 clearerr
>
>
> Bleh:
>
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.14 memcpy
>
>
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 fileno
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 malloc
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 _IO_getc
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 ungetc
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 realloc
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.3.4 __printf_chk
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 fopen
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 perror
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 exit
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 fwrite
> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.3.4 __fprintf_chk
> 000000000060b160 g DO .bss 0000000000000008 GLIBC_2.2.5 stdout
> 000000000060b168 g DO .bss 0000000000000008 GLIBC_2.2.5 stdin
> 000000000060b178 g DO .bss 0000000000000008 GLIBC_2.2.5 stderr
> 000000000060b170 g DO .bss 0000000000000008 GLIBC_2.2.5 optarg
>
> It would be reasonable to simply require the users of ckmake to update
> their libc version but even Debian sid does not yet seem to have libc
>>= 2.14, not sure of others. The gains for requiring a glibc >= 2.14
> only for memcpy() would seem rather small for this particular case so
> I wanted to review the options to fix this.
Can't we just rebuild genksyms in compat? I already have glibc 2.15 on
my system so I can't test this but normally this should adapt to the
build environment as it rebuilds and relinks the binary.
[root@ozzyfedora compat]# make -C
/lib/modules/2.6.36-02063604-generic/build M=scripts/genksyms
make: Entering directory `/usr/src/linux-headers-2.6.36-02063604-generic'
HOSTCC scripts/genksyms/genksyms.o
HOSTCC scripts/genksyms/lex.o
HOSTCC scripts/genksyms/parse.o
HOSTLD scripts/genksyms/genksyms
Building modules, stage 2.
MODPOST 0 modules
make: Leaving directory `/usr/src/linux-headers-2.6.36-02063604-generic'
One other thing that I have found is to force linking memcpy to a
hard-coded (namely GLIBC_2.2.5) memcpy() symbol. I didn't know about
this so I'm posting this as a reference:
make HOSTCFLAGS="-Wa,--defsym,memcpy=memcpy@GLIBC_2.2.5" -C
/lib/modules/2.6.36-02063604-generic/build M=scripts/genksyms
On Wed, Aug 8, 2012 at 5:43 AM, Ozan Çağlayan <[email protected]> wrote:
> On Tue, Aug 7, 2012 at 10:28 PM, Luis R. Rodriguez <[email protected]> wrote:
>> mcgrof@garbanzo ~/compat (git::master)$ objdump -T
>> /home/mcgrof/compat-ksrc/lib/modules/3.4.4-030404-generic/build/scripts/genksyms/genksyms
<-- snip -->
>> Bleh:
>>
>> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.14 memcpy
<-- snip -->
> Can't we just rebuild genksyms in compat? I already have glibc 2.15 on
> my system so I can't test this but normally this should adapt to the
> build environment as it rebuilds and relinks the binary.
Indeed! Implemented and pushed upstream:
https://github.com/mcgrof/compat/commit/42faf2dc8d8bbbdc5b8913183fcd021a27e953c9
So if you run into this issue all you have to do now is run:
./bin/get-compat-kernels -r
New users of the script will automatically have this run after it
downloads and installs the kernel headers.
Case closed, thanks :)
Luis
On Thu, Aug 9, 2012 at 5:30 AM, Luis R. Rodriguez <[email protected]> wrote:
> On Wed, Aug 8, 2012 at 5:43 AM, Ozan Çağlayan <[email protected]> wrote:
>> On Tue, Aug 7, 2012 at 10:28 PM, Luis R. Rodriguez <[email protected]> wrote:
>
>>> mcgrof@garbanzo ~/compat (git::master)$ objdump -T
>>> /home/mcgrof/compat-ksrc/lib/modules/3.4.4-030404-generic/build/scripts/genksyms/genksyms
>
> <-- snip -->
>
>>> Bleh:
>>>
>>> 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.14 memcpy
>
> <-- snip -->
>
>> Can't we just rebuild genksyms in compat? I already have glibc 2.15 on
>> my system so I can't test this but normally this should adapt to the
>> build environment as it rebuilds and relinks the binary.
>
> Indeed! Implemented and pushed upstream:
>
> https://github.com/mcgrof/compat/commit/42faf2dc8d8bbbdc5b8913183fcd021a27e953c9
>
I am not an English native but this sentence sounds a bit strange to
me ("used" followed by "require" and double "Linux"?).
I can imagine what you want to express but have no better wording right now.
180 + echo -e " This was added since kernels >= 3.4 used require"
181 + echo -e " a glibc >= 2.14 for memcpy(), and not all Linux"
182 + echo -e " Linux distributions have such glibc.
- Sedat -
> So if you run into this issue all you have to do now is run:
>
> ./bin/get-compat-kernels -r
>
> New users of the script will automatically have this run after it
> downloads and installs the kernel headers.
>
> Case closed, thanks :)
>
> Luis
> --
> To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Aug 9, 2012 at 12:27 AM, Sedat Dilek <[email protected]> wrote:
> I am not an English native but this sentence sounds a bit strange to
> me ("used" followed by "require" and double "Linux"?).
> I can imagine what you want to express but have no better wording right now.
>
> 180 + echo -e " This was added since kernels >= 3.4 used require"
> 181 + echo -e " a glibc >= 2.14 for memcpy(), and not all Linux"
> 182 + echo -e " Linux distributions have such glibc.
Fixed.
Luis
On Thu, Aug 9, 2012 at 11:01 AM, Luis R. Rodriguez <[email protected]> wrote:
> On Thu, Aug 9, 2012 at 12:27 AM, Sedat Dilek <[email protected]> wrote:
>> I am not an English native but this sentence sounds a bit strange to
>> me ("used" followed by "require" and double "Linux"?).
>> I can imagine what you want to express but have no better wording right now.
>>
>> 180 + echo -e " This was added since kernels >= 3.4 used require"
>> 181 + echo -e " a glibc >= 2.14 for memcpy(), and not all Linux"
>> 182 + echo -e " Linux distributions have such glibc.
>
> Fixed.
>
Thanks for the quick fix, sounds much better now.
- Sedat -
> Luis