Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3;
MIME-Version: 1.0
References: <20231017103742.130927-1-masahiroy@kernel.org> <20231017103742.130927-2-masahiroy@kernel.org>
 <CAEf4Bzaxb1npVtH_CnFNrOJQxQF5t82_nZxqbaFLiE-rpk_jBg@mail.gmail.com>
In-Reply-To: <CAEf4Bzaxb1npVtH_CnFNrOJQxQF5t82_nZxqbaFLiE-rpk_jBg@mail.gmail.com>
From:   Masahiro Yamada <masahiroy@kernel.org>
Date:   Wed, 18 Oct 2023 14:00:42 +0900
Message-ID: <CAK7LNARRtCQOoWT0q5EjJhoMrfAVbCg_fREeA+NpUHBRbB5oww@mail.gmail.com>
Subject: Re: [PATCH 2/4] kbuild: avoid too many execution of scripts/pahole-flags.sh
To:     Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc:     linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org,
        Alex Gaynor <alex.gaynor@gmail.com>,
        Alexei Starovoitov <ast@kernel.org>,
        Alice Ryhl <aliceryhl@google.com>,
        Andreas Hindborg <a.hindborg@samsung.com>,
        Andrii Nakryiko <andrii@kernel.org>,
        Benno Lossin <benno.lossin@proton.me>,
        =?UTF-8?Q?Bj=C3=B6rn_Roy_Baron?= <bjorn3_gh@protonmail.com>,
        Boqun Feng <boqun.feng@gmail.com>,
        Daniel Borkmann <daniel@iogearbox.net>,
        Gary Guo <gary@garyguo.net>, Hao Luo <haoluo@google.com>,
        Jiri Olsa <jolsa@kernel.org>,
        John Fastabend <john.fastabend@gmail.com>,
        KP Singh <kpsingh@kernel.org>,
        Martin KaFai Lau <martin.lau@linux.dev>,
        Miguel Ojeda <ojeda@kernel.org>,
        Nathan Chancellor <nathan@kernel.org>,
        Nick Desaulniers <ndesaulniers@google.com>,
        Nicolas Schier <nicolas@fjasle.eu>, Song Liu <song@kernel.org>,
        Stanislav Fomichev <sdf@google.com>,
        Wedson Almeida Filho <wedsonaf@gmail.com>,
        Yonghong Song <yonghong.song@linux.dev>, bpf@vger.kernel.org,
        rust-for-linux@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Wed, Oct 18, 2023 at 4:57=E2=80=AFAM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Tue, Oct 17, 2023 at 3:38=E2=80=AFAM Masahiro Yamada <masahiroy@kernel=
.org> wrote:
> >
> > scripts/pahole-flags.sh is executed so many times.
> >
> > You can check how many times it is invoked during the build, as follows=
:
> >
> >   $ cat <<EOF >> scripts/pahole-flags.sh
> >   > echo "scripts/pahole-flags.sh was executed" >&2
> >   > EOF
> >
> >   $ make -s
> >   scripts/pahole-flags.sh was executed
> >   scripts/pahole-flags.sh was executed
> >   scripts/pahole-flags.sh was executed
> >   scripts/pahole-flags.sh was executed
> >   scripts/pahole-flags.sh was executed
> >     [ lots of repeated lines suppressed... ]
> >
> > This scripts is exectuted more than 20 times during the kernel build
> > because PAHOLE_FLAGS is a recursively expanded variable and exported
> > to sub-processes.
> >
> > With the GNU Make >=3D 4.4, it is executed more than 60 times because
> > exported variables are also passed to other $(shell ) invocations.
> > Without careful coding, it is known to cause an exponential fork
> > explosion. [1]
> >
> > The use of $(shell ) in an exported recursive variable is likely wrong
> > because $(shell ) is always evaluated due to the 'export' keyword, and
> > the evaluation can occur multiple times by the nature of recursive
> > variables.
> >
> > Convert the shell script to a Makefile, which is included only when
> > CONFIG_DEBUG_INFO_BTF=3Dy.
> >
> > [1]: https://savannah.gnu.org/bugs/index.php?64746
> >
> > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> > ---
> >
> >  Makefile                |  4 +---
> >  scripts/Makefile.btf    | 19 +++++++++++++++++++
> >  scripts/pahole-flags.sh | 30 ------------------------------
> >  3 files changed, 20 insertions(+), 33 deletions(-)
> >  create mode 100644 scripts/Makefile.btf
> >  delete mode 100755 scripts/pahole-flags.sh
> >
> > diff --git a/Makefile b/Makefile
> > index fed9a6cc3665..eaddec67e5e1 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -513,8 +513,6 @@ LZ4         =3D lz4c
> >  XZ             =3D xz
> >  ZSTD           =3D zstd
> >
> > -PAHOLE_FLAGS   =3D $(shell PAHOLE=3D$(PAHOLE) $(srctree)/scripts/pahol=
e-flags.sh)
>
> What if we just used :=3D here? Wouldn't it avoid unnecessary multiple ex=
ecutions?


Yeah, :=3D is less silly than =3D.


But, I do not like to run the script for non-build targets
such as 'make clean', 'make help', etc.

Also, when building with CONFIG_DEBUG_INFO_BTF=3Dn,
the shell is forked to compute PAHOLE_FLAGS,
which we know are unnecessary.


--
Best Regards

Masahiro Yamada