Received-SPF: pass (google.com: domain of linux-kernel+bounces-10158-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161;
Precedence: bulk
MIME-Version: 1.0
References: <20231122221814.139916-1-deller@kernel.org> <CAK7LNAQZO0g-B7UUEvdJWh3FhdhmWaaSaJyyEUoVoSYG0j8v-Q@mail.gmail.com>
 <CAK7LNASk=A4aeMuhUt4NGi5RHedcQ_WQrdN3r7S_x0euvsPUXA@mail.gmail.com> <bb34147d-4e67-456a-b0d6-965699cda596@gmx.de>
In-Reply-To: <bb34147d-4e67-456a-b0d6-965699cda596@gmx.de>
From: Masahiro Yamada <masahiroy@kernel.org>
Date: Sat, 23 Dec 2023 10:32:58 +0900
Message-ID: <CAK7LNATaGSPK8dpE=Vq2EgjQJXKn+Ui_3GAYKVz4-bc8pir-ag@mail.gmail.com>
Subject: Re: [PATCH 0/4] Section alignment issues?
To: Helge Deller <deller@gmx.de>
Cc: deller@kernel.org, linux-kernel@vger.kernel.org, 
	Arnd Bergmann <arnd@arndb.de>, linux-modules@vger.kernel.org, linux-arch@vger.kernel.org, 
	Luis Chamberlain <mcgrof@kernel.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Fri, Dec 22, 2023 at 5:23=E2=80=AFPM Helge Deller <deller@gmx.de> wrote:
>
> On 12/21/23 16:42, Masahiro Yamada wrote:
> > On Thu, Dec 21, 2023 at 10:40=E2=80=AFPM Masahiro Yamada <masahiroy@ker=
nel.org> wrote:
> >>
> >> On Thu, Nov 23, 2023 at 7:18=E2=80=AFAM <deller@kernel.org> wrote:
> >>>
> >>> From: Helge Deller <deller@gmx.de>
> >>>
> >>> While working on the 64-bit parisc kernel, I noticed that the __ksymt=
ab[]
> >>> table was not correctly 64-bit aligned in many modules.
> >>> The following patches do fix some of those issues in the generic code=
.
> >>>
> >>> But further investigation shows that multiple sections in the kernel =
and in
> >>> modules are possibly not correctly aligned, and thus may lead to perf=
ormance
> >>> degregations at runtime (small on x86, huge on parisc, sparc and othe=
rs which
> >>> need exception handlers). Sometimes wrong alignments may also be simp=
ly hidden
> >>> by the linker or kernel module loader which pulls in the sections by =
luck with
> >>> a correct alignment (e.g. because the previous section was aligned al=
ready).
> >>>
> >>> An objdump on a x86 module shows e.g.:
> >>>
> >>> ./kernel/net/netfilter/nf_log_syslog.ko:     file format elf64-x86-64
> >>> Sections:
> >>> Idx Name          Size      VMA               LMA               File =
off  Algn
> >>>    0 .text         00001fdf  0000000000000000  0000000000000000  0000=
0040  2**4
> >>>                    CONTENTS, ALLOC, LOAD, RELOC, READONLY, CODE
> >>>    1 .init.text    000000f6  0000000000000000  0000000000000000  0000=
2020  2**4
> >>>                    CONTENTS, ALLOC, LOAD, RELOC, READONLY, CODE
> >>>    2 .exit.text    0000005c  0000000000000000  0000000000000000  0000=
2120  2**4
> >>>                    CONTENTS, ALLOC, LOAD, RELOC, READONLY, CODE
> >>>    3 .rodata.str1.8 000000dc  0000000000000000  0000000000000000  000=
02180  2**3
> >>>                    CONTENTS, ALLOC, LOAD, READONLY, DATA
> >>>    4 .rodata.str1.1 0000030a  0000000000000000  0000000000000000  000=
0225c  2**0
> >>>                    CONTENTS, ALLOC, LOAD, READONLY, DATA
> >>>    5 .rodata       000000b0  0000000000000000  0000000000000000  0000=
2580  2**5
> >>>                    CONTENTS, ALLOC, LOAD, READONLY, DATA
> >>>    6 .modinfo      0000019e  0000000000000000  0000000000000000  0000=
2630  2**0
> >>>                    CONTENTS, ALLOC, LOAD, READONLY, DATA
> >>>    7 .return_sites 00000034  0000000000000000  0000000000000000  0000=
27ce  2**0
> >>>                    CONTENTS, ALLOC, LOAD, RELOC, READONLY, DATA
> >>>    8 .call_sites   0000029c  0000000000000000  0000000000000000  0000=
2802  2**0
> >>>                    CONTENTS, ALLOC, LOAD, RELOC, READONLY, DATA
> >>>
> >>> In this example I believe the ".return_sites" and ".call_sites" shoul=
d have
> >>> an alignment of at least 32-bit (4 bytes).
> >>>
> >>> On other architectures or modules other sections like ".altinstructio=
ns" or
> >>> "__ex_table" may show up wrongly instead.
> >>>
> >>> In general I think it would be beneficial to search for wrong alignme=
nts at
> >>> link time, and maybe at runtime.
> >>>
> >>> The patch at the end of this cover letter
> >>> - adds compile time checks to the "modpost" tool, and
> >>> - adds a runtime check to the kernel module loader at runtime.
> >>> And it will possibly show false positives too (!!!)
> >>> I do understand that some of those sections are not performce critica=
l
> >>> and thus any alignment is OK.
> >>>
> >>> The modpost patch will emit at compile time such warnings (on x86-64 =
kernel build):
> >>>
> >>> WARNING: modpost: vmlinux: section .initcall7.init (type 1, flags 2) =
has alignment of 1, expected at least 4.
> >>> Maybe you need to add ALIGN() to the modules.lds file (or fix modpost=
) ?
> >>> WARNING: modpost: vmlinux: section .altinstructions (type 1, flags 2)=
 has alignment of 1, expected at least 2.
> >>> WARNING: modpost: vmlinux: section .initcall6.init (type 1, flags 2) =
has alignment of 1, expected at least 4.
> >>> WARNING: modpost: vmlinux: section .initcallearly.init (type 1, flags=
 2) has alignment of 1, expected at least 4.
> >>> WARNING: modpost: vmlinux: section .rodata.cst2 (type 1, flags 18) ha=
s alignment of 2, expected at least 64.
> >>> WARNING: modpost: vmlinux: section .static_call_tramp_key (type 1, fl=
ags 2) has alignment of 1, expected at least 8.
> >>> WARNING: modpost: vmlinux: section .con_initcall.init (type 1, flags =
2) has alignment of 1, expected at least 8.
> >>> WARNING: modpost: vmlinux: section __bug_table (type 1, flags 3) has =
alignment of 1, expected at least 4.
> >>> ...
> >>
> >>
> >>
> >>
> >> modpost acts on vmlinux.o instead of vmlinux.
> >>
> >>
> >> vmlinux.o is a relocatable ELF, which is not a real layout
> >> because no linker script has been considered yet at this
> >> point.
> >>
> >>
> >> vmlinux is an executable ELF, produced by a linker,
> >> with the linker script taken into consideration
> >> to determine the final section/symbol layout.
> >>
> >>
> >> So, checking this in modpost is meaningless.
> >>
> >>
> >>
> >> I did not check the module checking code, but
> >> modules are also relocatable ELF.
> >
> >
> >
> > Sorry, I replied too early.
> > (Actually I replied without reading your modpost code).
> >
> > Now, I understand what your checker is doing.
> >
> >
> > I did not test how many false positives are produced,
> > but it catches several suspicious mis-alignments.
>
> Yes.
>
> > However, I am not convinced with this warning.
> >
> >
> > +               warn("%s: section %s (type %d, flags %lu) has
> > alignment of %d, expected at least %d.\n"
> > +                    "Maybe you need to add ALIGN() to the modules.lds
> > file (or fix modpost) ?\n",
> > +                    modname, sec, sechdr->sh_type, sechdr->sh_flags,
> > is_shalign, should_shalign);
> > +       }
> >
> >
> > Adding ALGIN() hides the real problem.
>
> Right.
> It took me some time to understand the effects here too.
> See below...
>
> > I think the real problem is that not enough alignment was requested
> > in the code.
> >
> > For example, the right fix for ".initcall7.init" should be this:
> >
> > diff --git a/include/linux/init.h b/include/linux/init.h
> > index 3fa3f6241350..650311e4b215 100644
> > --- a/include/linux/init.h
> > +++ b/include/linux/init.h
> > @@ -264,6 +264,7 @@ extern struct module __this_module;
> >   #define ____define_initcall(fn, __stub, __name, __sec)         \
> >          __define_initcall_stub(__stub, fn)                      \
> >          asm(".section   \"" __sec "\", \"a\"            \n"     \
> > +           ".balign 4                                  \n"     \
> >              __stringify(__name) ":                      \n"     \
> >              ".long      " __stringify(__stub) " - .     \n"     \
> >              ".previous                                  \n");   \
> >
> > Then, "this section requires at least 4 byte alignment"
> > is recorded in the sh_addralign field.
>
> Yes, this is the important part.
>
> > Then, the rest is the linker's job.
> >
> > We should not tweak the linker script.
>
> That's right, but let's phrase it slightly different...
> There is *no need* to tweak the linker script, *if* the alignment
> gets correctly assigned by the inline assembly (like your
> initcall patch above).
> But on some platforms (e.g. on parisc) I noticed that this .balign
> was missing for some other sections, in which case the other (not preferr=
ed)
> possible option is to tweak the linker script.
>
> So I think we agree that fixing the inline assembly is the right
> way to go?


Yes, I think so.


> Either way, a link-time check like the proposed modpost patch
> may catch section issue for upcoming/newly added sections too.


Yes. This check seems to be useful.


--=20
Best Regards
Masahiro Yamada