Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <20210910130954.1392104-1-lucjan.lucjanov@gmail.com> <CAL2Yj0QvoNZ3VeYfk63oMUTenP3eypJx+fB6f09rprV-fCphGw@mail.gmail.com>
In-Reply-To: <CAL2Yj0QvoNZ3VeYfk63oMUTenP3eypJx+fB6f09rprV-fCphGw@mail.gmail.com>
From:   Masahiro Yamada <masahiroy@kernel.org>
Date:   Sun, 19 Sep 2021 20:36:11 +0900
Message-ID: <CAK7LNATBvZBmuznFaFQgZeOO_F_UsisPYBobDC7LYDMVW3Yd=w@mail.gmail.com>
Subject: Re: [PATCH] kbuild: use smaller dictionary size for zstd module compression
To:     =?UTF-8?Q?Piotr_G=C3=B3rski?= <lucjan.lucjanov@gmail.com>
Cc:     Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        Linux Kbuild mailing list <linux-kbuild@vger.kernel.org>,
        Michal Marek <michal.lkml@markovi.net>,
        Nick Desaulniers <ndesaulniers@google.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Sat, Sep 18, 2021 at 12:07 AM Piotr G=C3=B3rski <lucjan.lucjanov@gmail.c=
om> wrote:
>
> Ping...
>
> pt., 10 wrz 2021 o 15:10 Piotr Gorski <lucjan.lucjanov@gmail.com> napisa=
=C5=82(a):
> >
> > By default, zstd without parameters uses a dictionary size of 8 MB.
> > However, most modules are much smaller than that.
> > Use a dictionary size of 2 MB for module compression, resulting in
> > slightly higher compression speed while still maintaining a good
> > compression ratio.
> > The --zstd=3Dwlog=3D21 option is equivalent to --lzma2=3Ddict=3D2MiB us=
ed in XZ compression.


What is a benefit of this patch?

It claims "slightly higher compression", but I do not see a difference.
(15.0577 sec   --> 15.013 sec)


[1] build  allmodconfig
[2] collect *.ko files whose size is less than 2MB
[3] measure the time elapsed for compressing all of them

I did 'perf stat --repeat 10'.


The default zstd:


 Performance counter stats for '/tmp/zstd2/test.sh' (10 runs):

         19,250.23 msec task-clock                #    1.278 CPUs
utilized            ( +-  0.49% )
            65,424      context-switches          #    0.003 M/sec
               ( +-  0.03% )
             5,609      cpu-migrations            #    0.291 K/sec
               ( +-  0.25% )
         4,207,419      page-faults               #    0.219 M/sec
               ( +-  0.19% )
    54,349,495,038      cycles                    #    2.823 GHz
               ( +-  0.15% )  (85.93%)
     3,833,225,032      stalled-cycles-frontend   #    7.05% frontend
cycles idle     ( +-  0.47% )  (86.34%)
    20,789,203,221      stalled-cycles-backend    #   38.25% backend
cycles idle      ( +-  0.31% )  (85.92%)
    65,385,071,612      instructions              #    1.20  insn per
cycle
                                                  #    0.32  stalled
cycles per insn  ( +-  0.09% )  (86.54%)
    11,033,842,879      branches                  #  573.180 M/sec
               ( +-  0.10% )  (86.38%)
       288,314,097      branch-misses             #    2.61% of all
branches          ( +-  0.16% )  (86.33%)

           15.0577 +- 0.0728 seconds time elapsed  ( +-  0.48% )


zstd --zstd=3Dwlog=3D21:


 Performance counter stats for '/tmp/zstd2/test2.sh' (10 runs):

         19,203.08 msec task-clock                #    1.279 CPUs
utilized            ( +-  0.85% )
            65,404      context-switches          #    0.003 M/sec
               ( +-  0.05% )
             5,617      cpu-migrations            #    0.293 K/sec
               ( +-  0.15% )
         4,214,502      page-faults               #    0.219 M/sec
               ( +-  0.12% )
    54,495,218,169      cycles                    #    2.838 GHz
               ( +-  0.29% )  (85.55%)
     3,867,166,903      stalled-cycles-frontend   #    7.10% frontend
cycles idle     ( +-  0.26% )  (87.22%)
    20,950,577,113      stalled-cycles-backend    #   38.44% backend
cycles idle      ( +-  0.38% )  (86.51%)
    65,500,859,667      instructions              #    1.20  insn per
cycle
                                                  #    0.32  stalled
cycles per insn  ( +-  0.12% )  (86.65%)
    11,045,504,296      branches                  #  575.194 M/sec
               ( +-  0.12% )  (85.90%)
       288,729,169      branch-misses             #    2.61% of all
branches          ( +-  0.12% )  (85.59%)

            15.013 +- 0.129 seconds time elapsed  ( +-  0.86% )


> > Signed-off-by: Piotr Gorski <lucjan.lucjanov@gmail.com>
> > ---
> >  scripts/Makefile.modinst | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/scripts/Makefile.modinst b/scripts/Makefile.modinst
> > index ff9b09e4cfca..c3475e5aca7c 100644
> > --- a/scripts/Makefile.modinst
> > +++ b/scripts/Makefile.modinst
> > @@ -97,7 +97,7 @@ quiet_cmd_gzip =3D GZIP    $@
> >  quiet_cmd_xz =3D XZ      $@
> >        cmd_xz =3D $(XZ) --lzma2=3Ddict=3D2MiB -f $<
> >  quiet_cmd_zstd =3D ZSTD    $@
> > -      cmd_zstd =3D $(ZSTD) -T0 --rm -f -q $<
> > +      cmd_zstd =3D $(ZSTD) --zstd=3Dwlog=3D21 -T0 --rm -f -q $<
> >
> >  $(dst)/%.ko.gz: $(dst)/%.ko FORCE
> >         $(call cmd,gzip)
> > --
> > 2.33.0.142.ge0a2f5cbc5
> >


--
Best Regards
Masahiro Yamada