Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Mon, 21 Jan 2019 16:34:39 +0100
From:   Andrea Parri <andrea.parri@amarulasolutions.com>
To:     Peter Zijlstra <peterz@infradead.org>
Cc:     linux-kernel@vger.kernel.org, Ingo Molnar <mingo@redhat.com>,
        "Paul E. McKenney" <paulmck@linux.ibm.com>,
        Alan Stern <stern@rowland.harvard.edu>,
        Will Deacon <will.deacon@arm.com>
Subject: Re: [PATCH] sched: Use READ_ONCE()/WRITE_ONCE() in
 task_cpu()/__set_task_cpu()
Message-ID: <20190121153439.GA12623@andrea>
References: <20190116184218.13757-1-andrea.parri@amarulasolutions.com>
 <20190121104707.GA5883@andrea>
 <20190121122526.GA13777@hirez.programming.kicks-ass.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20190121122526.GA13777@hirez.programming.kicks-ass.net>
User-Agent: Mutt/1.9.4 (2018-02-28)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Mon, Jan 21, 2019 at 01:25:26PM +0100, Peter Zijlstra wrote:
> On Mon, Jan 21, 2019 at 11:51:21AM +0100, Andrea Parri wrote:
> > On Wed, Jan 16, 2019 at 07:42:18PM +0100, Andrea Parri wrote:
> > > The smp_wmb() in move_queued_task() (c.f., __set_task_cpu()) pairs with
> > > the composition of the dependency and the ACQUIRE in task_rq_lock():
> > > 
> > > 	move_queued_task()		task_rq_lock()
> > > 
> > > 	[S] ->on_rq = MIGRATING		[L] rq = task_rq()
> > > 	WMB (__set_task_cpu())		ACQUIRE (rq->lock);
> > > 	[S] ->cpu = new_cpu		[L] ->on_rq
> > > 
> > > where "[L] rq = task_rq()" is ordered before "ACQUIRE (rq->lock)" by an
> > > address dependency and, in turn, "ACQUIRE (rq->lock)" is ordered before
> > > "[L] ->on_rq" by the ACQUIRE itself.
> > > 
> > > Use READ_ONCE() to load ->cpu in task_rq() (c.f., task_cpu()) to honour
> > > this address dependency between loads; also, mark the store to ->cpu in
> > > __set_task_cpu() by using WRITE_ONCE() in order to tell the compiler to
> > > not mess/tear this (synchronizing) memory access.
> > 
> > In the light of the recent discussion about the integration of plain
> > accesses in the LKMM (c.f., e.g., [1] and discussion thereof), I was
> > considering even further changes to this in order to "reinforce" the
> > above smp_wmb().  Here's two approaches (one of):
> > 
> >  1) replace this smp_wmb()+WRITE_ONCE() with an smp_store_release();
> > 
> >  2) or keep this smp_wmb()+WRITE_ONCE(), but use {WRITE,READ}_ONCE()
> >     also for the accesses to ->on_rq.
> 
> That should be the least painful I think. Note that we never store a
> value larger than a single byte in that word, so tearing shouldn't be a
> problem, but yes, that makes it all neat and tidy.

Thank you for the suggestion; I'll send the revisited patch shortly.

  Andrea