Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Tue, 12 May 2020 13:01:14 -0700
From:   Fangrui Song <maskray@google.com>
To:     Nick Desaulniers <ndesaulniers@google.com>
Cc:     Masahiro Yamada <masahiroy@kernel.org>,
        Sedat Dilek <sedat.dilek@gmail.com>,
        Nick Desaulniers <nick.desaulniers@gmail.com>,
        Michal Marek <michal.lkml@markovi.net>,
        Andrew Morton <akpm@linux-foundation.org>,
        Changbin Du <changbin.du@intel.com>,
        Randy Dunlap <rdunlap@infradead.org>,
        Krzysztof Kozlowski <krzk@kernel.org>,
        Linux Kbuild mailing list <linux-kbuild@vger.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        Clang-Built-Linux ML <clang-built-linux@googlegroups.com>
Subject: Re: [PATCH] Makefile: support compressed debug info
Message-ID: <20200512200114.64vo5lbl7wk2tzxk@google.com>
References: <20200504031340.7103-1-nick.desaulniers@gmail.com>
 <CA+icZUUOaqeKeh6n4BJq2k6XQWAfNghUj57j42ZX5qyd3iOmLw@mail.gmail.com>
 <20200505004738.ew2lcp27c2n4jqia@google.com>
 <CAK7LNAR7-VMEWBcJ_Wd+61ZDHEa0gD8FaSs63YPu7m_FgH8Htg@mail.gmail.com>
 <CAKwvOdmEP9Auuc+M+MqPoQmx+70DgdsPYZQ6pg=8oGnfCviqRA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Disposition: inline
In-Reply-To: <CAKwvOdmEP9Auuc+M+MqPoQmx+70DgdsPYZQ6pg=8oGnfCviqRA@mail.gmail.com>
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On 2020-05-12, Nick Desaulniers wrote:
>On Mon, May 11, 2020 at 10:54 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
>>
>> > >On Mon, May 4, 2020 at 5:13 AM Nick Desaulniers
>> > ><nick.desaulniers@gmail.com> wrote:
>> > >>
>> > >> As debug information gets larger and larger, it helps significantly save
>> > >> the size of vmlinux images to compress the information in the debug
>> > >> information sections. Note: this debug info is typically split off from
>> > >> the final compressed kernel image, which is why vmlinux is what's used
>> > >> in conjunction with GDB. Minimizing the debug info size should have no
>> > >> impact on boot times, or final compressed kernel image size.
>> > >>
>> Nick,
>>
>> I am OK with this patch.
>>
>> Fangrui provided the minimal requirement for
>> --compress-debug-sections=zlib
>>
>>
>> Is it worth recording in the help text?
>> Do you want to send v2?
>
>Yes I'd like to record that information.  I can also record Sedat's
>Tested-by tag.  Thank you for testing Sedat.
>
>I don't know what "linux-image-dbg file" are, or why they would be
>bigger.  The size of the debug info is the primary concern with this
>config.  It sounds like however that file is created might be
>problematic.
>
>Fangrui, I wasn't able to easily find what version of binutils first
>added support.  Can you please teach me how to fish?

I actually downloaded https://ftp.gnu.org/gnu/binutils/ archives and
located the sources... I think an easier way is:

% cd binutils-gdb
% git show binutils-2_26:./gas/as.c | grep compress-debug-sections
--compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi}]\n\
...

GNU as 2.25 only supports --compress-debug-sections which means "zlib-gnu" in
newer versions.

Similarly, for GNU ld:

% git show binutils-2_26:./ld/lexsup.c | grep compress-debug-sections
   --compress-debug-sections=[none|zlib|zlib-gnu|zlib-gabi]\n\

(I have spent a lot of time investigating GNU ld's behavior :)

>Another question I had for Fangrui is, if the linker can compress
>these sections, shouldn't we just have the linker do it, not the the
>compiler and assembler?  IIUC the debug info can contain relocations,
>so the linker would have to decompress these, perform relocations,
>then recompress these?  I guess having the compiler and assembler
>compress the debug info as well would minimize the size of the .o
>files on disk.

The linker will decompress debug info unconditionally. Because
input .debug_info sections need to be concatenated to form the output
.debug_info . Whether the output .debug_info is compressed is controlled
by the linker option --compress-debug-sections=zlib, which is not
affected by the compression state of object files.

Both GNU as and GNU ld name the option --compress-debug-sections=zlib.
In a compiler driver context, an unfamiliar user may find
-Wa,--compress-debug-sections=zlib -Wl,--compress-debug-sections=zlib
confusing:/

>Otherwise I should add this flag to the assembler invocation, too, in
>v2.  Thoughts?

Compressing object files along with the linked output should be fine. It
can save disk space. (It'd be great if you paste the comparison
with and w/o object files compressed)

Feel free to add:

Reviewed-by: Fangrui Song <maskray@google.com>

>I have a patch series that enables dwarf5 support in the kernel that
>I'm working up to.  I wanted to send this first.  Both roughly reduce
>the debug info size by 20% each, though I haven't measured them
>together, yet.  Requires ToT binutils because there have been many
>fixes from reports of mine recently.

This will be awesome! I also heard that enabling DWARF v5 for our object
files can easily make debug info size smaller by 20%. Glad that the
kernel can benefit it as well:)