Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
To:     x86 <x86@kernel.org>
Cc:     Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
From:   Jan Kiszka <jan.kiszka@siemens.com>
Subject: x2apic_wrmsr_fence vs. Intel manual
Message-ID: <783add60-f6c7-c8c6-b369-42e5ebfbf8c9@siemens.com>
Date:   Mon, 2 Mar 2020 17:11:21 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.5.0
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

Hi all,

as I generated a nice bug around fence vs. x2apic icr writes, I studied 
the kernel code and the Intel manual in this regard more closely. But 
there is a discrepancy:

arch/x86/include/asm/apic.h:

/*
 * Make previous memory operations globally visible before
 * sending the IPI through x2apic wrmsr. We need a serializing instruction or
 * mfence for this.
 */
static inline void x2apic_wrmsr_fence(void)
{
        asm volatile("mfence" : : : "memory");
}

Intel SDM, 10.12.3 MSR Access in x2APIC Mode:

"A WRMSR to an APIC register may complete before all preceding stores 
are globally visible; software can prevent this by inserting a 
serializing instruction or the sequence MFENCE;LFENCE before the WRMSR."

The former dates back to ce4e240c279a, but that commit does not mention 
why lfence is not needed. Did the manual read differently back then? Or 
why are we safe? To my reading of lfence, it also has a certain 
instruction serializing effect that mfence does not have.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux