Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Mon, 21 Jan 2019 13:25:26 +0100
From:   Peter Zijlstra <peterz@infradead.org>
To:     Andrea Parri <andrea.parri@amarulasolutions.com>
Cc:     linux-kernel@vger.kernel.org, Ingo Molnar <mingo@redhat.com>,
        "Paul E. McKenney" <paulmck@linux.ibm.com>,
        Alan Stern <stern@rowland.harvard.edu>,
        Will Deacon <will.deacon@arm.com>
Subject: Re: [PATCH] sched: Use READ_ONCE()/WRITE_ONCE() in
 task_cpu()/__set_task_cpu()
Message-ID: <20190121122526.GA13777@hirez.programming.kicks-ass.net>
References: <20190116184218.13757-1-andrea.parri@amarulasolutions.com>
 <20190121104707.GA5883@andrea>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20190121104707.GA5883@andrea>
User-Agent: Mutt/1.10.1 (2018-07-13)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Mon, Jan 21, 2019 at 11:51:21AM +0100, Andrea Parri wrote:
> On Wed, Jan 16, 2019 at 07:42:18PM +0100, Andrea Parri wrote:
> > The smp_wmb() in move_queued_task() (c.f., __set_task_cpu()) pairs with
> > the composition of the dependency and the ACQUIRE in task_rq_lock():
> > 
> > 	move_queued_task()		task_rq_lock()
> > 
> > 	[S] ->on_rq = MIGRATING		[L] rq = task_rq()
> > 	WMB (__set_task_cpu())		ACQUIRE (rq->lock);
> > 	[S] ->cpu = new_cpu		[L] ->on_rq
> > 
> > where "[L] rq = task_rq()" is ordered before "ACQUIRE (rq->lock)" by an
> > address dependency and, in turn, "ACQUIRE (rq->lock)" is ordered before
> > "[L] ->on_rq" by the ACQUIRE itself.
> > 
> > Use READ_ONCE() to load ->cpu in task_rq() (c.f., task_cpu()) to honour
> > this address dependency between loads; also, mark the store to ->cpu in
> > __set_task_cpu() by using WRITE_ONCE() in order to tell the compiler to
> > not mess/tear this (synchronizing) memory access.
> 
> In the light of the recent discussion about the integration of plain
> accesses in the LKMM (c.f., e.g., [1] and discussion thereof), I was
> considering even further changes to this in order to "reinforce" the
> above smp_wmb().  Here's two approaches (one of):
> 
>  1) replace this smp_wmb()+WRITE_ONCE() with an smp_store_release();
> 
>  2) or keep this smp_wmb()+WRITE_ONCE(), but use {WRITE,READ}_ONCE()
>     also for the accesses to ->on_rq.

That should be the least painful I think. Note that we never store a
value larger than a single byte in that word, so tearing shouldn't be a
problem, but yes, that makes it all neat and tidy.

> What do you think?  (maybe I'm just being too paranoid?)
> 
> Adding Will to the Cc:  ((1) should be "painless" for x86, not sure
> about arm64...)

ARM64 should be fine, it is 32bit ARM that will suffer, because it uses
smp_mb() to implement acquire/release.