On Tue, Jan 30, 2024 at 10:37:30AM -0500, Rich Felker wrote:
> On Tue, Jan 30, 2024 at 11:43:38AM +0100, Szabolcs Nagy wrote:
> > * Rich Felker <[email protected]> [2024-01-16 15:45:52 -0500]:
> >
> > > On Tue, Jan 16, 2024 at 07:29:18PM +0100, Szabolcs Nagy wrote:
> > > > * Cody Wetzel <[email protected]> [2024-01-16 09:21:05 -0600]:
> > > > > Here is the output for the old
> > > > > ....
> > > > > >
> > > > > > / # /tmp/ld-musl-armhf.so.1 /usr/bin/readelf -lW /tmp/ld-musl-armhf.so.1
> > > > > >
> > > > > > Elf file type is DYN (Shared object file)
> > > > > > Entry point 0x359cd
> > > > > > There are 6 program headers, starting at offset 52
> > > > > >
> > > > > > Program Headers:
> > > > > > Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
> > > > > > EXIDX 0x07acec 0x0007acec 0x0007acec 0x00008 0x00008 R 0x4
> > > > > > LOAD 0x000000 0x00000000 0x00000000 0x7acf4 0x7acf4 R E 0x10000
> > > > > > LOAD 0x07fd6c 0x0008fd6c 0x0008fd6c 0x0054a 0x02258 RW 0x10000
> > > >
> > > > this load segment is 64k aligned.
> > > >
> > > > > > DYNAMIC 0x07febc 0x0008febc 0x0008febc 0x000c0 0x000c0 RW 0x4
> > > > > > GNU_STACK 0x000000 0x00000000 0x00000000 0x00000 0x00000 RW 0x10
> > > > > > GNU_RELRO 0x07fd6c 0x0008fd6c 0x0008fd6c 0x00294 0x00294 R 0x1
> > > > > >
> > > > > > Section to Segment mapping:
> > > > > > Segment Sections...
> > > > > > 00 .ARM.exidx
> > > > > > 01 .hash .gnu.hash .dynsym .dynstr .rel.dyn .rel.plt .plt .text
> > > > > > .rodata .ARM.exidx
> > > > > > 02 .data.rel.ro .dynamic .got .data .bss
> > > > > > 03 .dynamic
> > > > > > 04
> > > > > > 05 .data.rel.ro .dynamic .got
> > > > > >
> > > > >
> > > > > And the new...
> > > > >
> > > > > / # /tmp/ld-musl-armhf.so.1 /usr/bin/readelf -lW /lib/ld-musl-armhf.so.1
> > > > > >
> > > > > > Elf file type is DYN (Shared object file)
> > > > > > Entry point 0x362f1
> > > > > > There are 6 program headers, starting at offset 52
> > > > > >
> > > > > > Program Headers:
> > > > > > Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
> > > > > > EXIDX 0x07b81c 0x0007b81c 0x0007b81c 0x00008 0x00008 R 0x4
> > > > > > LOAD 0x000000 0x00000000 0x00000000 0x7b824 0x7b824 R E 0x1000
> > > > > > LOAD 0x07bd74 0x0007cd74 0x0007cd74 0x0054a 0x0225c RW 0x1000
> > > >
> > > > this load segment is 4k aligned and offset vs addr is not congruent
> > > > modulo 64k, or 32k, so won't work on systems with such page size.
> > > >
> > > > > > DYNAMIC 0x07bebc 0x0007cebc 0x0007cebc 0x000c0 0x000c0 RW 0x4
> > > > > > GNU_STACK 0x000000 0x00000000 0x00000000 0x00000 0x00000 RW 0x10
> > > > > > GNU_RELRO 0x07bd74 0x0007cd74 0x0007cd74 0x0028c 0x0028c R 0x1
> > > > > >
> > > > > > Section to Segment mapping:
> > > > > > Segment Sections...
> > > > > > 00 .ARM.exidx
> > > > > > 01 .hash .gnu.hash .dynsym .dynstr .rel.dyn .rel.plt .plt .text
> > > > > > .rodata .ARM.exidx
> > > > > > 02 .data.rel.ro .dynamic .got .data .bss
> > > > > > 03 .dynamic
> > > > > > 04
> > > > > > 05 .data.rel.ro .dynamic .got
> > > > >
> > > > >
> > > > > I hope that helps.
> > > >
> > > > yes, this is a linking issue, not musl libc.
> > > >
> > > > alpine linux links binaries for 4k pagesize only.
> > > >
> > > > arm linkers were updated at some point to create binaries supporting
> > > > up to 64k pagesize. i suspect some ppl ran into issues in practice
> > > > and decided the larger binaries are not worth it, if they dont work
> > > > reliably and forced 4k page size at link time.
> > > >
> > > > you have to raise an issue with alpine linux, if you think 32k
> > > > oage size is useful and reliably supportable.
> > >
> > > Are they using -Wl,-z,separate-code? That incurs a large
> > > binary-size-on-disk penalty when supporting oversized pages, and IIRC
> > > something was done to make the linker default to not supporting
> > > oversized pages when that's used. It might be the reason, if arm
> > > linking is normally expected to use a larger max pagesize.
> >
> > i looked at this now, turns out they just changed the
> > pagesize back to 4k (i missed this change):
> >
> > https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=1a26a53a0dee39106ba58fcb15496c5f13074652
>
> This doesn't help immediately, but a major ingredient to fix this
> situation would be getting the kernel to stop doing the wrong thing.
> Right now, it's ignoring the fact that the ELF program header
> constraints are incompatible with mmap given the oversized system
> pagesize, and just incorrectly mapping the executable and trying to
> run it anyway, whereby it blows up.
>
> The right thing to do would be either to fail with ENOEXEC in this
> case, or when mmap with the required offset constraint fails, falling
> back to making an anonymous map and copying the whole content of the
> loadable segment into that (no COW sharing). The latter is really not
> all that bad for got/data/etc. mappings which you expect will be dirty
> (modified) anyway.
>
> BTW the former choice (ENOEXEC) would allow doing the latter in
> userspace with a binfmt_misc loader.
Completely untested draft patch showing the concept is attached.
Rich