Received-SPF: pass (google.com: domain of linux-kernel+bounces-103736-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Date: Thu, 14 Mar 2024 19:43:06 +0000
From: Matthew Wilcox <willy@infradead.org>
To: "H. Peter Anvin" <hpa@zytor.com>
Cc: Pasha Tatashin <pasha.tatashin@soleen.com>,
	linux-kernel@vger.kernel.org, linux-mm@kvack.org,
	akpm@linux-foundation.org, x86@kernel.org, bp@alien8.de,
	brauner@kernel.org, bristot@redhat.com, bsegall@google.com,
	dave.hansen@linux.intel.com, dianders@chromium.org,
	dietmar.eggemann@arm.com, eric.devolder@oracle.com,
	hca@linux.ibm.com, hch@infradead.org, jacob.jun.pan@linux.intel.com,
	jgg@ziepe.ca, jpoimboe@kernel.org, jroedel@suse.de,
	juri.lelli@redhat.com, kent.overstreet@linux.dev,
	kinseyho@google.com, kirill.shutemov@linux.intel.com,
	lstoakes@gmail.com, luto@kernel.org, mgorman@suse.de,
	mic@digikod.net, michael.christie@oracle.com, mingo@redhat.com,
	mjguzik@gmail.com, mst@redhat.com, npiggin@gmail.com,
	peterz@infradead.org, pmladek@suse.com, rick.p.edgecombe@intel.com,
	rostedt@goodmis.org, surenb@google.com, tglx@linutronix.de,
	urezki@gmail.com, vincent.guittot@linaro.org, vschneid@redhat.com
Subject: Re: [RFC 00/14] Dynamic Kernel Stacks
Message-ID: <ZfNTSjfE_w50Otnz@casper.infradead.org>
References: <20240311164638.2015063-1-pasha.tatashin@soleen.com>
 <2cb8f02d-f21e-45d2-afe2-d1c6225240f3@zytor.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <2cb8f02d-f21e-45d2-afe2-d1c6225240f3@zytor.com>

On Tue, Mar 12, 2024 at 10:18:10AM -0700, H. Peter Anvin wrote:
> Second, non-dynamic kernel memory is one of the core design decisions in
> Linux from early on. This means there are lot of deeply embedded assumptions
> which would have to be untangled.

I think there are other ways of getting the benefit that Pasha is seeking
without moving to dynamically allocated kernel memory.  One icky thing
that XFS does is punt work over to a kernel thread in order to use more
stack!  That breaks a number of things including lockdep (because the
kernel thread doesn't own the lock, the thread waiting for the kernel
thread owns the lock).

If we had segmented stacks, XFS could say "I need at least 6kB of stack",
and if less than that was available, we could allocate a temporary
stack and switch to it.  I suspect Google would also be able to use this
API for their rare cases when they need more than 8kB of kernel stack.
Who knows, we might all be able to use such a thing.

I'd been thinking about this from the point of view of allocating more
stack elsewhere in kernel space, but combining what Pasha has done here
with this idea might lead to a hybrid approach that works better; allocate
32kB of vmap space per kernel thread, put 12kB of memory at the top of it,
rely on people using this "I need more stack" API correctly, and free the
excess pages on return to userspace.  No complicated "switch stacks" API
needed, just an "ensure we have at least N bytes of stack remaining" API.