Hi,
Boris complained he could no longer build allyesconfig on his 32G desktop
machine without having OOM terminate either objtool or chrome.
After talking about these patches on IRC, Nathan mentioned the linux-clang CI
was also having trouble of recent, and these patches appear to make it happy
again.
In total these patches shrink an allyesconfig run by about 6G:
pre: 5:58.22 real, 226.69 user, 131.22 sys, 26221520 mem
post: 5:03.34 real, 210.75 user, 88.80 sys, 20241232 mem
Also at:
https://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git/log/?h=objtool/core
On Wed, Feb 08, 2023 at 06:17:56PM +0100, Peter Zijlstra wrote:
> Hi,
>
> Boris complained he could no longer build allyesconfig on his 32G desktop
> machine without having OOM terminate either objtool or chrome.
>
> After talking about these patches on IRC, Nathan mentioned the linux-clang CI
> was also having trouble of recent, and these patches appear to make it happy
> again.
>
> In total these patches shrink an allyesconfig run by about 6G:
>
> pre: 5:58.22 real, 226.69 user, 131.22 sys, 26221520 mem
> post: 5:03.34 real, 210.75 user, 88.80 sys, 20241232 mem
>
> Also at:
>
> https://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git/log/?h=objtool/core
For patches 1-9:
Tested-by: Nathan Chancellor <[email protected]> # build only
Prior to this series:
[INFO] Memory used: 25.09GB
After this series:
[INFO] Memory used: 19.27GB
Our builds on TuxSuite were consistenly timing out after four hours and
they had no problem passing with this series (the worst time was 2.2h,
which is line with the VM specs that they use I believe):
https://tuxapi.tuxsuite.com/v1/groups/clangbuiltlinux/projects/nathan/plans/2LQbNuWRo3Xf62Yg3SINuA9d7cR
Thanks a lot!
Cheers,
Nathan
On Wed, 08. Feb 18:17, Peter Zijlstra wrote:
> Hi,
>
> Boris complained he could no longer build allyesconfig on his 32G desktop
> machine without having OOM terminate either objtool or chrome.
>
> After talking about these patches on IRC, Nathan mentioned the linux-clang CI
> was also having trouble of recent, and these patches appear to make it happy
> again.
>
> In total these patches shrink an allyesconfig run by about 6G:
>
> pre: 5:58.22 real, 226.69 user, 131.22 sys, 26221520 mem
> post: 5:03.34 real, 210.75 user, 88.80 sys, 20241232 mem
>
> Also at:
>
> https://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git/log/?h=objtool/core
>
Hello Peter,
with clang-17 the build failed:
In file included from weak.c:10:
In file included from /home/damian/kernel/linux/tools/objtool/include/objtool/objtool.h:13:
/home/damian/kernel/linux/tools/objtool/include/objtool/elf.h:88:86: error: '_Static_assert' with no message is a C2x extension [-Werror,-Wc2x-extensions]
static_assert(offsetof(struct reloc, rela.r_offset) == offsetof(struct reloc, offset));
^
, ""
In file included from arch/x86/special.c:4:
In file included from /home/damian/kernel/linux/tools/objtool/include/objtool/special.h:10:
In file included from /home/damian/kernel/linux/tools/objtool/include/objtool/check.h:11/home/damian/kernel/linux/tools/objtool/include/objtool/elf.h:
In file included from :89:86: error: '_Static_assert' with no message is a C2x extension [-Werror,-Wc2x-extensions]
/home/damian/kernel/linux/tools/objtool/include/objtool/arch.h:11:
In file included from static_assert(offsetof(struct reloc, rela.r_addend) == offsetof(struct reloc, addend));
^
, ""
/home/damian/kernel/linux/tools/objtool/include/objtool/objtool.h:13:
/home/damian/kernel/linux/tools/objtool/include/objtool/elf.h:88:86: error: '_Static_assert' with no message is a C2x extension [-Werror,-Wc2x-extensions]
static_assert(offsetof(struct reloc, rela.r_offset) == offsetof(struct reloc, offset));
^
, ""
/home/damian/kernel/linux/tools/objtool/include/objtool/elf.h:89:86: error: '_Static_assert' with no message is a C2x extension [-Werror,-Wc2x-extensions]
static_assert(offsetof(struct reloc, rela.r_addend) == offsetof(struct reloc, addend));
^
, ""
2 errors generated.
In file included from builtin-check.c:10:
In file included from /home/damian/kernel/linux/tools/objtool/include/objtool/objtool.h:13:
/home/damian/kernel/linux/tools/objtool/include/objtool/elf.h:88:86: error: '_Static_assert' with no message is a C2x extension [-Werror,-Wc2x-extensions]
static_assert(offsetof(struct reloc, rela.r_offset) == offsetof(struct reloc, offset));
...
Best regards
Damian
On Wed, Feb 08, 2023 at 09:22:02PM +0100, Damian Tometzki wrote:
> On Wed, 08. Feb 18:17, Peter Zijlstra wrote:
> > Hi,
> >
> > Boris complained he could no longer build allyesconfig on his 32G desktop
> > machine without having OOM terminate either objtool or chrome.
> >
> > After talking about these patches on IRC, Nathan mentioned the linux-clang CI
> > was also having trouble of recent, and these patches appear to make it happy
> > again.
> >
> > In total these patches shrink an allyesconfig run by about 6G:
> >
> > pre: 5:58.22 real, 226.69 user, 131.22 sys, 26221520 mem
> > post: 5:03.34 real, 210.75 user, 88.80 sys, 20241232 mem
> >
> > Also at:
> >
> > https://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git/log/?h=objtool/core
> >
> Hello Peter,
>
> with clang-17 the build failed:
> In file included from weak.c:10:
> In file included from /home/damian/kernel/linux/tools/objtool/include/objtool/objtool.h:13:
> /home/damian/kernel/linux/tools/objtool/include/objtool/elf.h:88:86: error: '_Static_assert' with no message is a C2x extension [-Werror,-Wc2x-extensions]
> static_assert(offsetof(struct reloc, rela.r_offset) == offsetof(struct reloc, offset));
Oh urgh. Apparently the kernel wrapper went missing in this userspace
project :-)
include/linux/build_bug.h:#define static_assert(expr, ...) __static_assert(expr, ##__VA_ARGS__, #expr)
Anyway, it's that last patch and I meant to do that in a slightly less
horrid way :-)
On Thu, 09. Feb 11:22, Peter Zijlstra wrote:
> On Wed, Feb 08, 2023 at 09:22:02PM +0100, Damian Tometzki wrote:
> > On Wed, 08. Feb 18:17, Peter Zijlstra wrote:
> > > Hi,
> > >
> > > Boris complained he could no longer build allyesconfig on his 32G desktop
> > > machine without having OOM terminate either objtool or chrome.
> > >
> > > After talking about these patches on IRC, Nathan mentioned the linux-clang CI
> > > was also having trouble of recent, and these patches appear to make it happy
> > > again.
> > >
> > > In total these patches shrink an allyesconfig run by about 6G:
> > >
> > > pre: 5:58.22 real, 226.69 user, 131.22 sys, 26221520 mem
> > > post: 5:03.34 real, 210.75 user, 88.80 sys, 20241232 mem
> > >
> > > Also at:
> > >
> > > https://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git/log/?h=objtool/core
> > >
> > Hello Peter,
> >
> > with clang-17 the build failed:
> > In file included from weak.c:10:
> > In file included from /home/damian/kernel/linux/tools/objtool/include/objtool/objtool.h:13:
> > /home/damian/kernel/linux/tools/objtool/include/objtool/elf.h:88:86: error: '_Static_assert' with no message is a C2x extension [-Werror,-Wc2x-extensions]
> > static_assert(offsetof(struct reloc, rela.r_offset) == offsetof(struct reloc, offset));
>
> Oh urgh. Apparently the kernel wrapper went missing in this userspace
> project :-)
>
> include/linux/build_bug.h:#define static_assert(expr, ...) __static_assert(expr, ##__VA_ARGS__, #expr)
>
> Anyway, it's that last patch and I meant to do that in a slightly less
> horrid way :-)
Hello Peter,
here a shortipossible fix:
Then the build with clang works.
diff --git a/tools/objtool/Makefile b/tools/objtool/Makefile
index 83b100c1e7f6..b7c8b476db95 100644
--- a/tools/objtool/Makefile
+++ b/tools/objtool/Makefile
@@ -31,7 +31,7 @@ INCLUDES := -I$(srctree)/tools/include \
-I$(LIBSUBCMD_OUTPUT)/include
# Note, EXTRA_WARNINGS here was determined for CC and not HOSTCC, it
# is passed here to match a legacy behavior.
-WARNINGS := $(EXTRA_WARNINGS) -Wno-switch-default -Wno-switch-enum -Wno-packed -Wno-nested-externs
+WARNINGS := $(EXTRA_WARNINGS) -Wno-switch-default -Wno-switch-enum -Wno-packed -Wno-nested-externs -Wno-c2x-extensions
OBJTOOL_CFLAGS := -Werror $(WARNINGS) $(KBUILD_HOSTCFLAGS) -g $(INCLUDES) $(LIBELF_FLAGS)
On Wed, Feb 08, 2023 at 06:17:56PM +0100, Peter Zijlstra wrote:
> Hi,
>
> Boris complained he could no longer build allyesconfig on his 32G desktop
> machine without having OOM terminate either objtool or chrome.
>
> After talking about these patches on IRC, Nathan mentioned the linux-clang CI
> was also having trouble of recent, and these patches appear to make it happy
> again.
>
> In total these patches shrink an allyesconfig run by about 6G:
>
> pre: 5:58.22 real, 226.69 user, 131.22 sys, 26221520 mem
> post: 5:03.34 real, 210.75 user, 88.80 sys, 20241232 mem
>
> Also at:
>
> https://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git/log/?h=objtool/core
For patches 1-9:
Acked-by: Josh Poimboeuf <[email protected]>
To make the last patch legit we could just have reloc_offset(),
reloc_addend(), reloc_type() helpers to abstract access to rela/rel
fields.
And I'm sure there are other savings we could do to struct reloc, like
single-linked lists, bitfields, etc.
Let me know if you want me to do it.
--
Josh
On Wed, Feb 08, 2023 at 06:17:56PM +0100, Peter Zijlstra wrote:
> Hi,
>
> Boris complained he could no longer build allyesconfig on his 32G desktop
> machine without having OOM terminate either objtool or chrome.
>
> After talking about these patches on IRC, Nathan mentioned the linux-clang CI
> was also having trouble of recent, and these patches appear to make it happy
> again.
>
> In total these patches shrink an allyesconfig run by about 6G:
>
> pre: 5:58.22 real, 226.69 user, 131.22 sys, 26221520 mem
> post: 5:03.34 real, 210.75 user, 88.80 sys, 20241232 mem
Thanks!
This makes a huge difference on my memory-starved travel machine.
Seems to be back to about 6.0 levels.
Tested-by: Thomas Weißschuh <[email protected]> # compile and run
> Also at:
>
> https://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git/log/?h=objtool/core
>
On Thu, Feb 09, 2023 at 11:57:40AM -0800, Josh Poimboeuf wrote:
> To make the last patch legit we could just have reloc_offset(),
> reloc_addend(), reloc_type() helpers to abstract access to rela/rel
> fields.
>
> And I'm sure there are other savings we could do to struct reloc, like
> single-linked lists, bitfields, etc.
>
> Let me know if you want me to do it.
I'll make a start today -- I've got snot pouring out of me, so thinking
isn't going to happen, might as well just do the busy work :-)
On Fri, Feb 10, 2023 at 09:59:05AM +0100, Peter Zijlstra wrote:
> On Thu, Feb 09, 2023 at 11:57:40AM -0800, Josh Poimboeuf wrote:
>
> > To make the last patch legit we could just have reloc_offset(),
> > reloc_addend(), reloc_type() helpers to abstract access to rela/rel
> > fields.
> >
> > And I'm sure there are other savings we could do to struct reloc, like
> > single-linked lists, bitfields, etc.
> >
> > Let me know if you want me to do it.
>
> I'll make a start today -- I've got snot pouring out of me, so thinking
> isn't going to happen, might as well just do the busy work :-)
(lightly tested...)
Shrinks reloc from 120 to 80 bytes:
https://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git/log/?h=objtool/shrink-reloc
pre: 5:03.34 real, 210.75 user, 88.80 sys, 20241232 mem
post: 4:12.71 real, 188.21 user, 64.49 sys, 18144848 mem
(still running on the exact same allyesconfig image)
I can probably get rid of symbol::reloc_list if I rewrite that whole
symtable trainwreck and I think I can also get rid of
instruction::alt_group, but both of those are a bit more work.