Received-SPF: pass (google.com: domain of linux-kernel+bounces-89451-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Precedence: bulk
MIME-Version: 1.0
References: <20240301084046.3370376-1-xin@zytor.com> <CAMzpN2j7xKcGx=+z8mu_2z2RsqjB-mpODdrOH7N1J2_OzuyEUQ@mail.gmail.com>
 <c4fb7c39-c99c-4e4c-bd85-470f5d0dc4dd@zytor.com>
In-Reply-To: <c4fb7c39-c99c-4e4c-bd85-470f5d0dc4dd@zytor.com>
From: Brian Gerst <brgerst@gmail.com>
Date: Sat, 2 Mar 2024 08:20:22 -0500
Message-ID: <CAMzpN2jpRov_zVp+8qXybS6dtm57uk1pfvRUUKeFGVGnC2G_Wg@mail.gmail.com>
Subject: Re: [PATCH v1 1/1] x86/fred: Fix init_task thread stack pointer initialization
To: Xin Li <xin@zytor.com>
Cc: linux-kernel@vger.kernel.org, tglx@linutronix.de, mingo@redhat.com, 
	bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Fri, Mar 1, 2024 at 11:18=E2=80=AFPM Xin Li <xin@zytor.com> wrote:
>
> On 3/1/2024 5:15 AM, Brian Gerst wrote:
> > On Fri, Mar 1, 2024 at 3:41=E2=80=AFAM Xin Li (Intel) <xin@zytor.com> w=
rote:
> > There is another spot in head_64.S that also needs this offset:
>
> I checked all references to __end_init_task before sending out this
> patch, and I doubt we need to make more similar changes.
>
> First of all, "movq    TASK_threadsp(%rcx), %rsp" you added in
> 3adee777ad0d ("x86/smpboot: Remove initial_stack on 64-bit") is exactly
> what we need to set up %rsp for the init task.
>
> > /* Set up the stack for verify_cpu() */
> > leaq (__end_init_task - PTREGS_SIZE)(%rip), %rsp
>
> As the comment says, it's a _temporary_ stack for calling verify_cpu()
> (but only for BSP, as APs use a different bring up stack), at which
> stage the concept of "task" has not formed. I'm thinking maybe it's
> better to do:
>
> /* Set up the stack for verify_cpu() */
> leaq __end_init_task(%rip), %rsp
>
> Previously it was "leaq    (__end_init_task - FRAME_SIZE)(%rip), %rsp",
> but the kernel unwinder goes up only to secondary_startup_64_no_verify()
> after the new way you introduced to set up %rsp for the init task, and
> it seems to me that there is no point to subtract FRAME_SIZE or
> PTREGS_SIZE.
>
> On the other hand, TOP_OF_KERNEL_STACK_PADDING is required for x86_32,
> but probably not for x86_64 (defined as 0 before FRED). The most
> important usage of TOP_OF_KERNEL_STACK_PADDING is to get the pt_regs
> pointer for a task, i.e., task_pt_regs(task), which assumes a fixed
> offset from the top of a task stack, but also limits the space that
> could be used by future hardware above the pt_regs structure. Thus I
> prefer to limit the usage of TOP_OF_KERNEL_STACK_PADDING on x86_64.

The point is to keep consistency with other kernel threads, which have
the pt_regs area cleared (see copy_thread()).  In particular, the CS
field can't have junk in it or else user_mode(regs) could return the
wrong result.  So the stack needs to start below pt_regs, or we need
to explicitly zero pt_regs later.

Ideally, the load from thread->sp should just shift RSP by phys_base,
pointing to the same memory location in the virtual mapping.

Brian Gerst