2021-09-29 15:11:50

by Dave Hansen

[permalink] [raw]
Subject: Re: [PATCH v2 2/4] x86/mm/64: Flush global TLB on AP bringup

On 9/29/21 7:54 AM, Joerg Roedel wrote:
> The AP bringup code uses the trampoline_pgd page-table, which
> establishes global mappings in the user range of the address space.
> Flush the global TLB entries after CR4 is setup for the AP to make sure
> no stale entries remain in the TLB.
...
> diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
> index 0f8885949e8c..0f71ea2e5680 100644
> --- a/arch/x86/kernel/cpu/common.c
> +++ b/arch/x86/kernel/cpu/common.c
> @@ -436,6 +436,12 @@ void cr4_init(void)
>
> /* Initialize cr4 shadow for this CPU. */
> this_cpu_write(cpu_tlbstate.cr4, cr4);
> +
> + /*
> + * Flush any global TLB entries that might be left from the
> + * trampline_pgd.
> + */
> + __flush_tlb_all();
> }

Is there a reason to do this flush here as opposed to doing it closer to
the CR3 write where we switch away from trampoline_pgd? cr4_init()
seems like an odd place.


2021-09-30 13:56:33

by Joerg Roedel

[permalink] [raw]
Subject: Re: [PATCH v2 2/4] x86/mm/64: Flush global TLB on AP bringup

On Wed, Sep 29, 2021 at 08:09:38AM -0700, Dave Hansen wrote:
> On 9/29/21 7:54 AM, Joerg Roedel wrote:
>
> > + __flush_tlb_all();
> > }
>
> Is there a reason to do this flush here as opposed to doing it closer to
> the CR3 write where we switch away from trampoline_pgd? cr4_init()
> seems like an odd place.

Yeah, the reason is that global flushing is done by toggling CR4.PGE and
I didn't want to do that before CR4 is set up.

The CR3 switch away from the trampoline_pgd for secondary CPUs on x86-64
happens in head_64.S already. I will add some asm to do a global flush
there right after the CR3 switch. Secondary CPUs are already on kernel
virtual addresses at this point.


Joerg