Received-SPF: pass (google.com: domain of linux-kernel+bounces-66010-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
From: Michael Ellerman <mpe@ellerman.id.au>
To: Rahul Rameshbabu <rrameshbabu@nvidia.com>
Cc: linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, Gal
 Pressman <gal@nvidia.com>
Subject: Re: [PATCH] powerpc/64s: Increase default stack size to 32KB
In-Reply-To: <87ttmau5mg.fsf@nvidia.com>
References: <20231215124449.317597-1-mpe@ellerman.id.au>
 <87ttmau5mg.fsf@nvidia.com>
Date: Thu, 15 Feb 2024 08:56:59 +1100
Message-ID: <87cysy7lqs.fsf@mail.lhotse>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain

Rahul Rameshbabu <rrameshbabu@nvidia.com> writes:
> On Fri, 15 Dec, 2023 23:44:49 +1100 Michael Ellerman <mpe@ellerman.id.au> wrote:
>> There are reports of kernels crashing due to stack overflow while
>> running OpenShift (Kubernetes). The primary contributor to the stack
>> usage seems to be openvswitch, which is used by OVN-Kubernetes (based on
>> OVN (Open Virtual Network)), but NFS also contributes in some stack
>> traces.
>>
>> There may be some opportunities to reduce stack usage in the openvswitch
>> code, but doing so potentially require tradeoffs vs performance, and
>> also requires testing across architectures.
>>
>> Looking at stack usage across the kernel (using -fstack-usage), shows
>> that ppc64le stack frames are on average 50-100% larger than the
>> equivalent function built for x86-64. Which is not surprising given the
>> minimum stack frame size is 32 bytes on ppc64le vs 16 bytes on x86-64.
>>
>> So increase the default stack size to 32KB for the modern 64-bit Book3S
>> platforms, ie. pseries (virtualised) and powernv (bare metal). That
>> leaves the older systems like G5s, and the AmigaOne (pasemi) with a 16KB
>> stack which should be sufficient on those machines.
>>
>> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
>> ---
>
> We noticed this change is causing assembler issues for us when building
> the kernel.
>
>   make ARCH=powerpc KERNELRELEASE=6.8.0-rc2_for_upstream_debug_2024_02_06_20_01 KBUILD_BUILD_VERSION=1
>   arch/powerpc/kernel/switch.S: Assembler messages:
>   arch/powerpc/kernel/switch.S:249: Error: operand out of range (0x000000000000fe50 is not between 0xffffffffffff8000 and 0x0000000000007fff)
>   make[6]: *** [scripts/Makefile.build:361: arch/powerpc/kernel/switch.o] Error 1
>   make[5]: *** [scripts/Makefile.build:481: arch/powerpc/kernel] Error 2
>   make[5]: *** Waiting for unfinished jobs....
>   make[4]: *** [scripts/Makefile.build:481: arch/powerpc] Error 2
>   make[4]: *** Waiting for unfinished jobs....
>   make[3]: *** [Makefile:1921: .] Error 2

There's a fix in my fixes branch:
  https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git/commit/?h=fixes&id=f1acb109505d983779bbb7e20a1ee6244d2b5736

I'll send it to Linus this week.

cheers