Received-SPF: pass (google.com: domain of linux-kernel+bounces-104044-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Date: Thu, 14 Mar 2024 21:17:27 -0700
From: "H. Peter Anvin" <hpa@zytor.com>
To: Pasha Tatashin <pasha.tatashin@soleen.com>,
        Matthew Wilcox <willy@infradead.org>
CC: Kent Overstreet <kent.overstreet@linux.dev>, linux-kernel@vger.kernel.org,
        linux-mm@kvack.org, akpm@linux-foundation.org, x86@kernel.org,
        bp@alien8.de, brauner@kernel.org, bristot@redhat.com,
        bsegall@google.com, dave.hansen@linux.intel.com, dianders@chromium.org,
        dietmar.eggemann@arm.com, eric.devolder@oracle.com, hca@linux.ibm.com,
        hch@infradead.org, jacob.jun.pan@linux.intel.com, jgg@ziepe.ca,
        jpoimboe@kernel.org, jroedel@suse.de, juri.lelli@redhat.com,
        kinseyho@google.com, kirill.shutemov@linux.intel.com,
        lstoakes@gmail.com, luto@kernel.org, mgorman@suse.de, mic@digikod.net,
        michael.christie@oracle.com, mingo@redhat.com, mjguzik@gmail.com,
        mst@redhat.com, npiggin@gmail.com, peterz@infradead.org,
        pmladek@suse.com, rick.p.edgecombe@intel.com, rostedt@goodmis.org,
        surenb@google.com, tglx@linutronix.de, urezki@gmail.com,
        vincent.guittot@linaro.org, vschneid@redhat.com
Subject: Re: [RFC 00/14] Dynamic Kernel Stacks
User-Agent: K-9 Mail for Android
In-Reply-To: <CA+CK2bAmOj2J10szVijNikexFZ1gmA913vvxnqW4DJKWQikwqQ@mail.gmail.com>
References: <20240311164638.2015063-1-pasha.tatashin@soleen.com> <2cb8f02d-f21e-45d2-afe2-d1c6225240f3@zytor.com> <ZfNTSjfE_w50Otnz@casper.infradead.org> <2qp4uegb4kqkryihqyo6v3fzoc2nysuhltc535kxnh6ozpo5ni@isilzw7nth42> <ZfNWojLB7qjjB0Zw@casper.infradead.org> <CA+CK2bAmOj2J10szVijNikexFZ1gmA913vvxnqW4DJKWQikwqQ@mail.gmail.com>
Message-ID: <A5238E27-88DB-4758-B630-22F17501AFD5@zytor.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain;
 charset=utf-8
Content-Transfer-Encoding: quoted-printable

On March 14, 2024 8:13:56 PM PDT, Pasha Tatashin <pasha=2Etatashin@soleen=
=2Ecom> wrote:
>On Thu, Mar 14, 2024 at 3:57=E2=80=AFPM Matthew Wilcox <willy@infradead=
=2Eorg> wrote:
>>
>> On Thu, Mar 14, 2024 at 03:53:39PM -0400, Kent Overstreet wrote:
>> > On Thu, Mar 14, 2024 at 07:43:06PM +0000, Matthew Wilcox wrote:
>> > > On Tue, Mar 12, 2024 at 10:18:10AM -0700, H=2E Peter Anvin wrote:
>> > > > Second, non-dynamic kernel memory is one of the core design decis=
ions in
>> > > > Linux from early on=2E This means there are lot of deeply embedde=
d assumptions
>> > > > which would have to be untangled=2E
>> > >
>> > > I think there are other ways of getting the benefit that Pasha is s=
eeking
>> > > without moving to dynamically allocated kernel memory=2E  One icky =
thing
>> > > that XFS does is punt work over to a kernel thread in order to use =
more
>> > > stack!  That breaks a number of things including lockdep (because t=
he
>> > > kernel thread doesn't own the lock, the thread waiting for the kern=
el
>> > > thread owns the lock)=2E
>> > >
>> > > If we had segmented stacks, XFS could say "I need at least 6kB of s=
tack",
>> > > and if less than that was available, we could allocate a temporary
>> > > stack and switch to it=2E  I suspect Google would also be able to u=
se this
>> > > API for their rare cases when they need more than 8kB of kernel sta=
ck=2E
>> > > Who knows, we might all be able to use such a thing=2E
>> > >
>> > > I'd been thinking about this from the point of view of allocating m=
ore
>> > > stack elsewhere in kernel space, but combining what Pasha has done =
here
>> > > with this idea might lead to a hybrid approach that works better; a=
llocate
>> > > 32kB of vmap space per kernel thread, put 12kB of memory at the top=
 of it,
>> > > rely on people using this "I need more stack" API correctly, and fr=
ee the
>> > > excess pages on return to userspace=2E  No complicated "switch stac=
ks" API
>> > > needed, just an "ensure we have at least N bytes of stack remaining=
" API=2E
>
>I like this approach! I think we could also consider having permanent
>big stacks for some kernel only threads like kvm-vcpu=2E A cooperative
>stack increase framework could work well and wouldn't negatively
>impact the performance of context switching=2E However, thorough
>analysis would be necessary to proactively identify potential stack
>overflow situations=2E
>
>> > Why would we need an "I need more stack" API? Pasha's approach seems
>> > like everything we need for what you're talking about=2E
>>
>> Because double faults are hard, possibly impossible, and the FRED appro=
ach
>> Peter described has extra overhead?  This was all described up-thread=
=2E
>
>Handling faults in #DF is possible=2E It requires code inspection to
>handle race conditions such as what was shown by tglx=2E However, as
>Andy pointed out, this is not supported by SDM as it is an abort
>context (yet we return from it because of ESPFIX64, so return is
>possible)=2E
>
>My question, however, if we ignore memory savings and only consider
>reliability aspect of this feature=2E  What is better unconditionally
>crashing the machine because a guard page was reached, or printing a
>huge warning with a backtracing information about the offending stack,
>handling the fault, and survive? I know that historically Linus
>preferred WARN() to BUG() [1]=2E But, this is a somewhat different
>scenario compared to simple BUG vs WARN=2E
>
>Pasha
>
>[1] https://lore=2Ekernel=2Eorg/all/Pine=2ELNX=2E4=2E44=2E0209091832160=
=2E1714-100000@home=2Etransmeta=2Ecom
>

From=20a reliability point of view it is better to die than to proceed with =
possible data loss=2E The latter is extremely serious=2E

However, the one way that this could be made to work would be with stack p=
robes, which could be compiler-inserted=2E The point is that you touch an o=
ffset below the stack pointer that is large enough that you cover not only =
the maximum amount of stack the function needs, but with an additional marg=
in, which includes enough space that you can safely take the #PF on the rem=
aining stack=2E