Received-SPF: softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
Date:   Mon, 7 Feb 2022 23:27:46 +0800
From:   Jisheng Zhang <jszhang@kernel.org>
To:     Arnd Bergmann <arnd@arndb.de>
Cc:     Paul Walmsley <paul.walmsley@sifive.com>,
        Palmer Dabbelt <palmer@dabbelt.com>,
        Albert Ou <aou@eecs.berkeley.edu>,
        linux-riscv <linux-riscv@lists.infradead.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Subject: Re: [RFC PATCH] riscv: reduce THREAD_SIZE from 16KB to 8KB for RV64
Message-ID: <YgE6co0GR93s1LTQ@xhacker>
References: <20220206174359.2986-1-jszhang@kernel.org>
 <CAK8P3a3Nnw5gfPAYx5UGdW8Om16GDcBi1-SS9Bs+CyLzssgVbA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <CAK8P3a3Nnw5gfPAYx5UGdW8Om16GDcBi1-SS9Bs+CyLzssgVbA@mail.gmail.com>
Precedence: bulk

On Mon, Feb 07, 2022 at 08:35:54AM +0100, Arnd Bergmann wrote:
> On Sun, Feb 6, 2022 at 6:43 PM Jisheng Zhang <jszhang@kernel.org> wrote:
> >
> > After irq stack is supported, it's possible to use small THREAD_SIZE.
> > In fact, I tested this patch on a Lichee RV board, looks good so far.
> >
> > Signed-off-by: Jisheng Zhang <jszhang@kernel.org>
> 
> I can definitely see the value in this, in particular when you get hardware with
> small RAM configurations that would run a 32-bit kernel on other architectures.
> 
> However, it's worth pointing out that all other 64-bit architectures use 16KB or
> more, so it is rather dangerous to be the first architecture to try this out in
> a long time. Some on-stack structures have a lot of pointers and 'unsigned long'
> members, so they need twice the space, while other structures are the
> same size either way.
> 
> IRQ stacks obviously help here, but I don't think the 8KB size can be made
> the default without a lot of testing of some of the more rarely used code paths.
> 
> Here are a few things that would be worth doing on the way to a smaller
> kernel stack:
> 
> - do more compile-time testing with a lower CONFIG_FRAME_WARN value.
>   Historically, this defaults to 2048 bytes on 64-bit architectures. This is
>   much higher than we really want, but it takes work to find and eliminate
>   the outliers. I previously had a series to reduce the limit to 1280 bytes on
>   arm64 and still build all 'randconfig' configurations.
> 
> - Use a much lower limit during regression testing. There is already a config
>    option to randomize the start of the thread stack, but you can also try
>    adding a configurable offset to see how far you can push it for a given
>    workload before you run into the guard page.
> 
> - With vmap stacks, using 12KB may be an option as well, giving you
>    three pages for the stack plus one guard page, instead of 4 pages
>    stack plus four guard pages.
> 
> - If you can make a convincing case for using a lower THREAD_SIZE,
>   make this a compile-time option across all 64-bit architectures that
>   support both IRQ stacks and VMAP stacks. The actual frame size
>   does depend on the ISA a bit, and we know that parisc and ia64 are
>   particularly, possibly s390 as well, but I would expect risc-v to be
>   not much different from arm64 and x86 here.
> 

Hi Arnd

Thanks so much for all the suggestions. Item3 and Item4 look more
interesting to me.

Thanks a lot