Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Tue, 4 Jul 2023 11:22:22 +0200
From:   Borislav Petkov <bp@alien8.de>
To:     Peter Zijlstra <peterz@infradead.org>
Cc:     X86 ML <x86@kernel.org>,
        Kishon VijayAbraham <Kishon.VijayAbraham@amd.com>,
        LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] x86/barrier: Do not serialize MSR accesses on AMD
Message-ID: <20230704092222.GBZKPkzgdM8rbPe7zA@fat_crate.local>
References: <20230622095212.20940-1-bp@alien8.de>
 <20230703125419.GJ4253@hirez.programming.kicks-ass.net>
 <20230704074631.GAZKPOV/9BfqP0aU8v@fat_crate.local>
 <20230704090132.GP4253@hirez.programming.kicks-ass.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20230704090132.GP4253@hirez.programming.kicks-ass.net>
Precedence: bulk

On Tue, Jul 04, 2023 at 11:01:32AM +0200, Peter Zijlstra wrote:
> So they are normal MSRs like all other? AMD doesn't have any exceptions
> for MSRs, they all the same?

Yap, as far as I know.

> Dunno, code density, speculation, many raisons to avoid jumps :-)

Looking at x2apic_send_IPI asm:

	cmpb	$2, boot_cpu_data+1(%rip)	#, boot_cpu_data.x86_vendor
# arch/x86/kernel/apic/x2apic_phys.c:44: 	u32 dest = per_cpu(x86_cpu_to_apicid, cpu);
	movq	__per_cpu_offset(,%rdi,8), %rdx	# __per_cpu_offset[cpu_7(D)], __per_cpu_offset[cpu_7(D)]
	movzwl	(%rdx,%rax), %edx	# *_8,
# ./arch/x86/include/asm/processor.h:753: 	if (boot_cpu_data.x86_vendor != X86_VENDOR_AMD)
	je	.L114	#,
# ./arch/x86/include/asm/processor.h:754: 		asm volatile("mfence; lfence" : : : "memory");
#APP
# 754 "./arch/x86/include/asm/processor.h" 1
	mfence; lfence

So gcc already does mix unrelated insns so that they can all go in
parallel. So it is a

	CMP RIP-relative
	JE

So yeah, I guess, on the one hand we want to avoid conditional jumps
but, on the other, sprinkling alternatives everywhere without a good
reason is a waste. Especially if this branch is going to be
predicted-taken most of the time and it wouldn't matter.

So I'm still not convinced. We could measure it on my Coffeelake box
which says

"Switched APIC routing to cluster x2apic."

but I don't think it'll be visible.

> > > 	asm volatile (ALTERNATIVE("mfence; lfence;", "", X86_FEATURE_AMD));
> > 
> > There's no X86_FEATURE_AMD :)
> 
> I know, but that's easily fixed.

Yeah, there's X86_VENDOR_AMD too. I can see the confusion ensue.

I'm wondering if we could make:

	alternative("mfence; lfence;", "", X86_VENDOR_AMD);

work...

Might come in handy in the future.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette