Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Thomas Gleixner <tglx@linutronix.de>
To:     Andrew Cooper <Andrew.Cooper3@citrix.com>,
        LKML <linux-kernel@vger.kernel.org>
Cc:     "x86@kernel.org" <x86@kernel.org>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        Tim Chen <tim.c.chen@linux.intel.com>,
        Josh Poimboeuf <jpoimboe@kernel.org>,
        Pawan Gupta <pawan.kumar.gupta@linux.intel.com>,
        Johannes Wikner <kwikner@ethz.ch>,
        Alyssa Milburn <alyssa.milburn@linux.intel.com>,
        Jann Horn <jannh@google.com>, "H.J. Lu" <hjl.tools@gmail.com>,
        Joao Moreira <joao.moreira@intel.com>,
        Joseph Nuzman <joseph.nuzman@intel.com>,
        Steven Rostedt <rostedt@goodmis.org>
Subject: Re: [patch 02/38] x86/cpu: Use native_wrmsrl() in
 load_percpu_segment()
In-Reply-To: <0bec8fe2-d1e3-f01c-6e52-06ab542efdd8@citrix.com>
References: <20220716230344.239749011@linutronix.de>
 <20220716230952.787452088@linutronix.de>
 <0bec8fe2-d1e3-f01c-6e52-06ab542efdd8@citrix.com>
Date:   Sun, 17 Jul 2022 21:08:42 +0200
Message-ID: <87zgh7wo91.ffs@tglx>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Sun, Jul 17 2022 at 00:22, Andrew Cooper wrote:
>> -void load_percpu_segment(int cpu)
>> +static noinstr void load_percpu_segment(int cpu)
>>  {
>>  #ifdef CONFIG_X86_32
>>  	loadsegment(fs, __KERNEL_PERCPU);
>>  #else
>>  	__loadsegment_simple(gs, 0);
>> -	wrmsrl(MSR_GS_BASE, cpu_kernelmode_gs_base(cpu));
>> +	/*
>> +	 * Because of the __loadsegment_simple(gs, 0) above, any GS-prefixed
>> +	 * instruction will explode right about here. As such, we must not have
>> +	 * any CALL-thunks using per-cpu data.
>> +	 *
>> +	 * Therefore, use native_wrmsrl() and have XenPV take the fault and
>> +	 * emulate.
>> +	 */
>> +	native_wrmsrl(MSR_GS_BASE, cpu_kernelmode_gs_base(cpu));
>>  #endif
>
> Lovely :-/
>
> But I still don't see how that works, because __loadsegment_simple() is
> a memory clobber and cpu_kernelmode_gs_base() has a per-cpu lookup in
> it.

No. It uses an array lookup :)

> That said, this only has a sole caller, and in context, it's bogus for
> 64bit.=C2=A0 Can't we fix all the problems by just doing this:
>
> diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
> index 736262a76a12..6f393bc9d89d 100644
> --- a/arch/x86/kernel/cpu/common.c
> +++ b/arch/x86/kernel/cpu/common.c
> @@ -701,16 +701,6 @@ static const char *table_lookup_model(struct
> cpuinfo_x86 *c)
> =C2=A0__u32 cpu_caps_cleared[NCAPINTS + NBUGINTS] __aligned(sizeof(unsign=
ed
> long));
> =C2=A0__u32 cpu_caps_set[NCAPINTS + NBUGINTS] __aligned(sizeof(unsigned l=
ong));
> =C2=A0
> -void load_percpu_segment(int cpu)
> -{
> -#ifdef CONFIG_X86_32
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 loadsegment(fs, __KERNEL_PERCPU);
> -#else
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 __loadsegment_simple(gs, 0);
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 wrmsrl(MSR_GS_BASE, cpu_kernelmode_=
gs_base(cpu));
> -#endif
> -}
> -
> =C2=A0#ifdef CONFIG_X86_32
> =C2=A0/* The 32-bit entry code needs to find cpu_entry_area. */
> =C2=A0DEFINE_PER_CPU(struct cpu_entry_area *, cpu_entry_area);
> @@ -742,12 +732,15 @@ EXPORT_SYMBOL_GPL(load_fixmap_gdt);
> =C2=A0 * Current gdt points %fs at the "master" per-cpu area: after this,
> =C2=A0 * it's on the real one.
> =C2=A0 */
> -void switch_to_new_gdt(int cpu)
> +void __noinstr switch_to_new_gdt(int cpu)
> =C2=A0{
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 /* Load the original GDT */
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 load_direct_gdt(cpu);
> +
> +#ifdef CONFIG_X86_32
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 /* Reload the per-cpu base */
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 load_percpu_segment(cpu);
> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 loadsegment(fs, __KERNEL_PERCPU);
> +#endif
> =C2=A0}
> =C2=A0
> =C2=A0static const struct cpu_dev *cpu_devs[X86_VENDOR_NUM] =3D {};
>
>
> It's only 32bit where the percpu pointer is tied to the GDT.=C2=A0 On 64b=
it,
> gsbase is good before this, and remains good after.
>
> With this change,
>
> # Make sure load_percpu_segment has no stackprotector
> CFLAGS_common.o=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 :=3D -fno=
-stack-protector
>
> comes up for re-evaluation too.

Good point. Let me stare at it some more.

Thanks,

        tglx