Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Thomas Gleixner <tglx@linutronix.de>
To:     Andrew Cooper <Andrew.Cooper3@citrix.com>,
        LKML <linux-kernel@vger.kernel.org>
Cc:     "x86@kernel.org" <x86@kernel.org>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        Tim Chen <tim.c.chen@linux.intel.com>,
        Josh Poimboeuf <jpoimboe@kernel.org>,
        Pawan Gupta <pawan.kumar.gupta@linux.intel.com>,
        Johannes Wikner <kwikner@ethz.ch>,
        Alyssa Milburn <alyssa.milburn@linux.intel.com>,
        Jann Horn <jannh@google.com>, "H.J. Lu" <hjl.tools@gmail.com>,
        Joao Moreira <joao.moreira@intel.com>,
        Joseph Nuzman <joseph.nuzman@intel.com>,
        Steven Rostedt <rostedt@goodmis.org>
Subject: Re: [patch 02/38] x86/cpu: Use native_wrmsrl() in
 load_percpu_segment()
In-Reply-To: <87zgh7wo91.ffs@tglx>
References: <20220716230344.239749011@linutronix.de>
 <20220716230952.787452088@linutronix.de>
 <0bec8fe2-d1e3-f01c-6e52-06ab542efdd8@citrix.com> <87zgh7wo91.ffs@tglx>
Date:   Sun, 17 Jul 2022 22:08:16 +0200
Message-ID: <87tu7fwlhr.ffs@tglx>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Sun, Jul 17 2022 at 21:08, Thomas Gleixner wrote:
> On Sun, Jul 17 2022 at 00:22, Andrew Cooper wrote:
>> =C2=A0#ifdef CONFIG_X86_32
>> =C2=A0/* The 32-bit entry code needs to find cpu_entry_area. */
>> =C2=A0DEFINE_PER_CPU(struct cpu_entry_area *, cpu_entry_area);
>> @@ -742,12 +732,15 @@ EXPORT_SYMBOL_GPL(load_fixmap_gdt);
>> =C2=A0 * Current gdt points %fs at the "master" per-cpu area: after this,
>> =C2=A0 * it's on the real one.
>> =C2=A0 */
>> -void switch_to_new_gdt(int cpu)
>> +void __noinstr switch_to_new_gdt(int cpu)
>> =C2=A0{
>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 /* Load the original GDT */
>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 load_direct_gdt(cpu);
>> +
>> +#ifdef CONFIG_X86_32
>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 /* Reload the per-cpu base */
>> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 load_percpu_segment(cpu);
>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 loadsegment(fs, __KERNEL_PERCPU);
>> +#endif
>> =C2=A0}
>> =C2=A0
>> =C2=A0static const struct cpu_dev *cpu_devs[X86_VENDOR_NUM] =3D {};
>>
>>
>> It's only 32bit where the percpu pointer is tied to the GDT.=C2=A0 On 64=
bit,
>> gsbase is good before this, and remains good after.
>>
>> With this change,
>>
>> # Make sure load_percpu_segment has no stackprotector
>> CFLAGS_common.o=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 :=3D -fn=
o-stack-protector
>>
>> comes up for re-evaluation too.
>
> Good point. Let me stare at it some more.

If it only would be that simple :)

loadsegment_simple() was a red herring. The gs segment is already zero.

So what explodes here is the early boot when switching from early per
CPU to the real per CPU area.

start_kernel()
        .....
        setup_per_cpu_areas();
        smp_prepare_boot_cpu()
          switch_to_new_gdt()
       	     load_direct_gdt(cpu);
          load_percpu_segment(cpu)
            GS: 0
            GS_BASE: 0xffffffff829d0000 (early PERCPU)=20
            wrmsrl()
            GS_BASE: 0xffff888237c00000 (real PERCPU)

So the explosion happens when accessing a per CPU variable after loading
the GDT and before GS_BASE is fixed up.

That's the only case AFAICT where this matters. In all other invocations
GS_BASE is already correct.

Let me fix this proper.

Thanks,

        tglx