Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <CA+G9fYtFvJdtBknaDKR54HHMf4XsXKD4UD3qXkQ1KhgY19n3tw@mail.gmail.com>
 <CAHk-=wisUqoX5Njrnnpp0pDx+bxSAJdPxfgEUv82tZkvUqoN1w@mail.gmail.com>
 <CAHk-=whF9F89vsfH8E9TGc0tZA-yhzi2Di8wOtquNB5vRkFX5w@mail.gmail.com>
 <53ce8db-3372-b5e2-cee7-c0ebe9c45a9@tarent.de> <CANn89iJzyPbR-fS8S_oAMSJzUGTHAfx49CXVc6ZSckUk91Opvg@mail.gmail.com>
In-Reply-To: <CANn89iJzyPbR-fS8S_oAMSJzUGTHAfx49CXVc6ZSckUk91Opvg@mail.gmail.com>
From:   Linus Torvalds <torvalds@linux-foundation.org>
Date:   Wed, 8 Sep 2021 08:48:41 -0700
Message-ID: <CAHk-=whkOK2DTHMt1rQ7wCBCqW=itkihpQBcZ=T6vrciEE4ycA@mail.gmail.com>
Subject: Re: ipv4/tcp.c:4234:1: error: the frame size of 1152 bytes is larger
 than 1024 bytes [-Werror=frame-larger-than=]
To:     Eric Dumazet <edumazet@google.com>
Cc:     Thorsten Glaser <t.glaser@tarent.de>,
        Naresh Kamboju <naresh.kamboju@linaro.org>,
        Mathias Nyman <mathias.nyman@intel.com>,
        Johannes Berg <johannes@sipsolutions.net>,
        Jakub Kicinski <kuba@kernel.org>,
        Shuah Khan <skhan@linuxfoundation.org>,
        Brendan Higgins <brendanhiggins@google.com>,
        Ariel Elior <aelior@marvell.com>,
        GR-everest-linux-l2@marvell.com, Wei Liu <wei.liu@kernel.org>,
        Linux ARM <linux-arm-kernel@lists.infradead.org>,
        open list <linux-kernel@vger.kernel.org>,
        Netdev <netdev@vger.kernel.org>, lkft-triage@lists.linaro.org,
        Arnd Bergmann <arnd@arndb.de>,
        "David S. Miller" <davem@davemloft.net>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        Nick Desaulniers <ndesaulniers@google.com>,
        Nathan Chancellor <nathan@kernel.org>,
        Daniel Borkmann <daniel@iogearbox.net>,
        Alexei Starovoitov <ast@kernel.org>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Wed, Sep 8, 2021 at 7:50 AM Eric Dumazet <edumazet@google.com> wrote:
>
> At least on my builds,  do_tcp_getsockopt() uses less than 512 bytes of stack.
>
> Probably because tcp_zerocopy_receive() is _not_ inlined, by pure luck
> I suppose.
>
> Perhaps we should use noinline_for_stack here.

I agree that that is likely a good idea, but I also suspect that the
stack growth may be related to other issues. So it being less than 512
bytes for you may be related to other random noise than inlining.

In the past I've seen at least two patterns

 (a) not merging stack slots at all

 (b) some odd "pattern allocator" problems, where I think gcc ended up
re-using previous stack slots if they were the right size, but failing
when previous allocations were fragmented

that (a) thing is what -fconserve-stack is all about, and we also used
to have (iirc) -fno-defer-pop to avoid having function call argument
stacks stick around.

And (b) is one of those "random allocation pattern" things, which
depends on the phase of the moon, where gcc ends up treating the stack
frame as a series of fixed-size allocations, but isn't very smart
about it. Even if some allocations got free'd, they might be
surrounded by oithers that didn't, and then gcc wouldn't re-use them
if there's a bigger allocation afterwards. And similarly, I don't
think gcc ever even joins together two free'd stack frame allocations.

I also wouldn't be surprised at all if some of our hardening flags
ended up causing the stack frame reuse to entirely fail. IOW, I could
easily see things like INIT_STACK_ALL_ZERO might cause the compiler to
initialize all the stack frame allocations "early", so that their
lifetimes all overlap.

So it could easily be about very subtle and random code generation
choices that just change the order of allocation. A spill in the wrong
place, things like that.

Or it could be about not-so-subtle big config option things.

          Linus