Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
	id S932327AbbBRSoA (ORCPT <rfc822;w@1wt.eu>);
	Wed, 18 Feb 2015 13:44:00 -0500
Received: from e39.co.us.ibm.com ([32.97.110.160]:40021 "EHLO
	e39.co.us.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
	with ESMTP id S1752554AbbBRSn5 (ORCPT
	<rfc822;linux-kernel@vger.kernel.org>);
	Wed, 18 Feb 2015 13:43:57 -0500
Date: Wed, 18 Feb 2015 10:43:52 -0800
From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Kirill Tkhai <ktkhai@parallels.com>, linux-kernel@vger.kernel.org,
        Ingo Molnar <mingo@redhat.com>, Josh Poimboeuf <jpoimboe@redhat.com>,
        oleg@redhat.com
Subject: Re: [PATCH 2/2] [PATCH] sched: Add smp_rmb() in task rq locking
 cycles
Message-ID: <20150218184352.GD5745@linux.vnet.ibm.com>
Reply-To: paulmck@linux.vnet.ibm.com
References: <20150217104516.12144.85911.stgit@tkhai>
 <1424170021.5749.22.camel@tkhai>
 <20150217121258.GM5029@twins.programming.kicks-ass.net>
 <20150217130523.GV24151@twins.programming.kicks-ass.net>
 <20150217160532.GW4166@linux.vnet.ibm.com>
 <20150217183636.GR5029@twins.programming.kicks-ass.net>
 <20150217215231.GK4166@linux.vnet.ibm.com>
 <20150218134706.GW5029@twins.programming.kicks-ass.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20150218134706.GW5029@twins.programming.kicks-ass.net>
User-Agent: Mutt/1.5.21 (2010-09-15)
X-TM-AS-MML: disable
X-Content-Scanned: Fidelis XPS MAILER
x-cbid: 15021818-0033-0000-0000-000003B431FC
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
Content-Length: 4000
Lines: 89

On Wed, Feb 18, 2015 at 02:47:06PM +0100, Peter Zijlstra wrote:
> On Tue, Feb 17, 2015 at 01:52:31PM -0800, Paul E. McKenney wrote:
> > I could do a table per communication style.  For example, message
> > passing looks like this (give or take likely errors in the table):
> > 
> > 	Side CPU	Top CPU
> > 	--------	-------
> > 	X = 1;		r1 = Y;
> > 	<some barrier>	<some barrier>
> > 	Y = 1;		r2 = X;
> > 
> > 	assert(r1 == 0 || r2 == 1);
> > 
> > 
> >       |   mb  |  wmb  |  rmb  |  rbd  |  acq  |  rel  |  ctl  |
> >  -----+-------+-------+-------+-------+-------+-------+-------+
> >    mb |   Y   |       |   Y   |   y   |   Y   |       |   Y   +
> >  -----+-------+-------+-------+-------+-------+-------+-------+
> >   wmb |   Y   |       |   Y   |   y   |   Y   |       |   Y   +
> >  -----+-------+-------+-------+-------+-------+-------+-------+
> >   rmb |       |       |       |       |       |       |       +
> >  -----+-------+-------+-------+-------+-------+-------+-------+
> >   rbd |       |       |       |       |       |       |       +
> >  -----+-------+-------+-------+-------+-------+-------+-------+
> >   acq |       |       |       |       |       |       |       +
> >  -----+-------+-------+-------+-------+-------+-------+-------+
> >   rel |   Y   |       |   Y   |   y   |   Y   |       |   Y   +
> >  -----+-------+-------+-------+-------+-------+-------+-------+
> >   ctl |       |       |       |       |       |       |       +
> >  -----+-------+-------+-------+-------+-------+-------+-------+
> > 
> > Here "Y" says that the barrier pair works, "y" says that it can
> > work but requires an artificial dependency, and " " says that
> > it does not work.
> 
> I would maybe do s/artificial/additional/, the pointer deref in RCU is
> not really artificial, is it?

Ah, but RCU is a slightly different pattern because you do have to have
a dependency, so you need a different litmus test:

	Initial state: X = 0, Y = &Z, Z = 2

	Side CPU		Top CPU
	--------		-------
	X = 1;			r1 = READ_ONCE(Y);
	<some barrier>		<some_barrier>
	WRITE_ONCE(Y, &X);	r2 = *r1;

	assert(r1 == &Z || r2 == 1);

      |   mb  |  wmb  |  rmb  |  rbd  |  acq  |  rel  |  ctl  |
 -----+-------+-------+-------+-------+-------+-------+-------+
   mb |   Y   |       |   Y   |   Y   |   Y   |       |   Y   +
 -----+-------+-------+-------+-------+-------+-------+-------+
  wmb |   Y   |       |   Y   |   Y   |   Y   |       |   Y   +
 -----+-------+-------+-------+-------+-------+-------+-------+
  rmb |       |       |       |       |       |       |       +
 -----+-------+-------+-------+-------+-------+-------+-------+
  rbd |       |       |       |       |       |       |       +
 -----+-------+-------+-------+-------+-------+-------+-------+
  acq |       |       |       |       |       |       |       +
 -----+-------+-------+-------+-------+-------+-------+-------+
  rel |   Y   |       |   Y   |   Y   |   Y   |       |   Y   +
 -----+-------+-------+-------+-------+-------+-------+-------+
  ctl |       |       |       |       |       |       |       +
 -----+-------+-------+-------+-------+-------+-------+-------+

But of course you should instead use rcu_assign_pointer() and
rcu_dereference().  And I should also have used READ_ONCE() and
WRITE_ONCE() in my earlier example.

> Also, how many communication styles do you envision to enumerate?

As few as possible, but yes, that is likely to be a bit of a problem.
To get an idea of how big it could get if not contained somehow, take
a look at http://www.cl.cam.ac.uk/~pes20/ppc-supplemental/test6.pdf.
20 of them on the first page alone!  ;-)

We most definitely need to stick to the lowest common denominator
if we are taking this approach.

							Thanx, Paul

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/