Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
From:   Thomas Gleixner <tglx@linutronix.de>
To:     Jan Kiszka <jan.kiszka@siemens.com>, x86 <x86@kernel.org>
Cc:     Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Subject: Re: x2apic_wrmsr_fence vs. Intel manual
In-Reply-To: <783add60-f6c7-c8c6-b369-42e5ebfbf8c9@siemens.com>
References: <783add60-f6c7-c8c6-b369-42e5ebfbf8c9@siemens.com>
Date:   Mon, 02 Mar 2020 17:20:26 +0100
Message-ID: <87lfoienjp.fsf@nanos.tec.linutronix.de>
MIME-Version: 1.0
Content-Type: text/plain
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

Jan Kiszka <jan.kiszka@siemens.com> writes:
> as I generated a nice bug around fence vs. x2apic icr writes, I studied 
> the kernel code and the Intel manual in this regard more closely. But 
> there is a discrepancy:
>
> arch/x86/include/asm/apic.h:
>
> /*
>  * Make previous memory operations globally visible before
>  * sending the IPI through x2apic wrmsr. We need a serializing instruction or
>  * mfence for this.
>  */
> static inline void x2apic_wrmsr_fence(void)
> {
>         asm volatile("mfence" : : : "memory");
> }
>
> Intel SDM, 10.12.3 MSR Access in x2APIC Mode:
>
> "A WRMSR to an APIC register may complete before all preceding stores 
> are globally visible; software can prevent this by inserting a 
> serializing instruction or the sequence MFENCE;LFENCE before the WRMSR."
>
> The former dates back to ce4e240c279a, but that commit does not mention 
> why lfence is not needed. Did the manual read differently back then? Or 
> why are we safe? To my reading of lfence, it also has a certain 
> instruction serializing effect that mfence does not have.

The 2011 SDM says:

  A WRMSR to an APIC register may complete before all preceding stores
  are globally visible; software can prevent this by inserting a
  serializing instruction, an SFENCE, or an MFENCE before the WRMSR.

Sigh....