Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Thu, 27 May 2021 11:57:37 +0200
From:   Peter Zijlstra <peterz@infradead.org>
To:     Adrian Hunter <adrian.hunter@intel.com>
Cc:     Leo Yan <leo.yan@linaro.org>,
        Arnaldo Carvalho de Melo <acme@kernel.org>,
        Ingo Molnar <mingo@redhat.com>,
        Mark Rutland <mark.rutland@arm.com>,
        Alexander Shishkin <alexander.shishkin@linux.intel.com>,
        Namhyung Kim <namhyung@kernel.org>,
        Andi Kleen <ak@linux.intel.com>,
        linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v1 1/2] perf auxtrace: Change to use SMP memory barriers
Message-ID: <YK9tEQmk4Q+M5yJh@hirez.programming.kicks-ass.net>
References: <20210519140319.1673043-1-leo.yan@linaro.org>
 <d1fc3dd3-e79a-4e93-1083-6b08e0cabe59@intel.com>
 <YK9UFl7Grv/mwmVc@hirez.programming.kicks-ass.net>
 <3c7dcd5d-fddd-5d3b-81ac-cb7b615b0338@intel.com>
 <7cdc3578-a50e-89ef-477a-3dc1f84f96bb@intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <7cdc3578-a50e-89ef-477a-3dc1f84f96bb@intel.com>
Precedence: bulk

On Thu, May 27, 2021 at 12:24:15PM +0300, Adrian Hunter wrote:

> > If all we want is a compiler barrier, then shouldn't that be what we use?
> > i.e. barrier()
> 
> I guess you are saying we still need to stop potential re-ordering across
> CPUs, so please ignore my comments.

Right; so the ordering issue is real, consider:

	CPU0 (kernel)		CPU1 (user)

	write data		read head
	smp_wmb()		smp_rmb()
	write head		read data

Without explicit ordering (on either side), we might either read data
that isn't written yet:

			     ,--(read data)
	write data	     |
	smp_wmb()	     |
	write head ---.	     |
		      `-->   |	read head
			     `- read data

Where the head load observes the new head writte, but the data load is
speculated and loads data before it is written.

Or, we can write the head before the data write is visible:

    ,--	write data
    |	write head
    |				read head
    |				smp_rmb()
    |				read data
    `-> (data visible)

Where we read the head head, but still observe stale data because the
stores got committed out of order.

x86 is TSO, so neither reordering is actually possible, hence both
barriers being a compiler barrier (to ensure the compiler doesn't
reorder them for us). But weaker hardware *will* allow those orderings
and we very much need actual barriers there.

Welcome to the magical world of memory ordering and weak architectures.