Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 16 Jan 2023 11:06:52 -0800
From:   "Paul E. McKenney" <paulmck@kernel.org>
To:     Alan Stern <stern@rowland.harvard.edu>
Cc:     Jonas Oberhauser <jonas.oberhauser@huawei.com>,
        Peter Zijlstra <peterz@infradead.org>,
        "parri.andrea" <parri.andrea@gmail.com>, will <will@kernel.org>,
        "boqun.feng" <boqun.feng@gmail.com>, npiggin <npiggin@gmail.com>,
        dhowells <dhowells@redhat.com>,
        "j.alglave" <j.alglave@ucl.ac.uk>,
        "luc.maranget" <luc.maranget@inria.fr>, akiyks <akiyks@gmail.com>,
        dlustig <dlustig@nvidia.com>, joel <joel@joelfernandes.org>,
        urezki <urezki@gmail.com>,
        quic_neeraju <quic_neeraju@quicinc.com>,
        frederic <frederic@kernel.org>,
        Kernel development list <linux-kernel@vger.kernel.org>
Subject: Re: Internal vs. external barriers (was: Re: Interesting LKMM litmus
 test)
Message-ID: <20230116190652.GZ2948950@paulmck-ThinkPad-P17-Gen-1>
Reply-To: paulmck@kernel.org
References: <136d019d8c8049f6b737627df830e66f@huawei.com>
 <20230114175343.GF2948950@paulmck-ThinkPad-P17-Gen-1>
 <20230114181537.GA493203@paulmck-ThinkPad-P17-Gen-1>
 <Y8MOOrrHntA9TyUk@rowland.harvard.edu>
 <20230115051510.GG2948950@paulmck-ThinkPad-P17-Gen-1>
 <Y8Qog8qf7wIx2Kve@rowland.harvard.edu>
 <20230115181052.GJ2948950@paulmck-ThinkPad-P17-Gen-1>
 <Y8RmEtBnwqOzNhsK@rowland.harvard.edu>
 <20230116042329.GN2948950@paulmck-ThinkPad-P17-Gen-1>
 <Y8WTXS73qTBpUzcI@rowland.harvard.edu>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <Y8WTXS73qTBpUzcI@rowland.harvard.edu>
Precedence: bulk

On Mon, Jan 16, 2023 at 01:11:41PM -0500, Alan Stern wrote:
> On Sun, Jan 15, 2023 at 08:23:29PM -0800, Paul E. McKenney wrote:
> > On Sun, Jan 15, 2023 at 03:46:10PM -0500, Alan Stern wrote:
> > > On Sun, Jan 15, 2023 at 10:10:52AM -0800, Paul E. McKenney wrote:
> > > > On Sun, Jan 15, 2023 at 11:23:31AM -0500, Alan Stern wrote:
> > > > > On Sat, Jan 14, 2023 at 09:15:10PM -0800, Paul E. McKenney wrote:
> > > > > > What am I missing here?
> > > > > 
> > > > > I don't think you're missing anything.  This is a matter for Boqun or 
> > > > > Luc; it must have something to do with the way herd treats the 
> > > > > srcu_read_lock() and srcu_read_unlock() primitives.
> > > > 
> > > > It looks like we need something that tracks (data | rf)* between
> > > > the return value of srcu_read_lock() and the second parameter of
> > > > srcu_read_unlock().  The reason for rf rather than rfi is the upcoming
> > > > srcu_down_read() and srcu_up_read().
> > > 
> > > Or just make herd treat srcu_read_lock(s) as an annotated equivalent of 
> > > READ_ONCE(&s) and srcu_read_unlock(s, v) as an annotated equivalent of 
> > > WRITE_ONCE(s, v).  But with some special accomodation to avoid 
> > > interaction with the new carry-dep relation.
> > 
> > This is a modification to herd7 you are suggesting?  Otherwise, I am
> > suffering a failure of imagination on how to properly sort it from the
> > other READ_ONCE() and WRITE_ONCE() instances.
> 
> srcu_read_lock and srcu_read_unlock events would be distinguished from 
> other marked loads and stores by belonging to the Srcu-lock and 
> Srcu-unlock sets.  But I don't know whether this result can be 
> accomplished just by modifying the .def file -- it might require changes 
> to herd7.  (In fact, as far as I know there is no documentation at all 
> for the double-underscore operations used in linux-kernel.def.  Hint 
> hint!)
> 
> As mentioned earlier, we should ask Luc or Boqun.

Good point, will do.

> > > > Or is there some better intermediate position that could be taken?
> > > 
> > > Do you mean go back to the current linux-kernel.bell?  The code you 
> > > wrote above is different, since it prohibits nesting.
> > 
> > Not to the current linux-kernel.bell, but, as you say, making the change
> > to obtain a better approximation by prohibiting nesting.
> 
> Why do you want to prohibit nesting?  Why would that be a better 
> approximation?

Because the current LKMM gives wrong answers for nested critical
sections.  For example, for the litmus test shown below, mainline
LKMM will incorrectly report "Never".  The two SRCU read-side critical
sections are independent, so the fact that P1()'s synchronize_srcu() is
guaranteed to wait for the first on to complete says nothing about the
second having completed.  Therefore, in Linux-kernel SRCU, the "exists"
clause could be satisfied.

In contrast, the proposed change flags this as having nesting.

							Thaxn, Paul

------------------------------------------------------------------------

C C-srcu-nest-5

(*
 * Result: Sometimes
 *
 * This demonstrates non-nesting of SRCU read-side critical sections.
 * Unlike RCU, SRCU critical sections do not nest.
 *)

{}

P0(int *x, int *y, struct srcu_struct *s1)
{
	int r1;
	int r2;
	int r3;
	int r4;

	r3 = srcu_read_lock(s1);
	r2 = READ_ONCE(*y);
	r4 = srcu_read_lock(s1);
	srcu_read_unlock(s1, r3);
	r1 = READ_ONCE(*x);
	srcu_read_unlock(s1, r4);
}

P1(int *x, int *y, struct srcu_struct *s1)
{
	WRITE_ONCE(*y, 1);
	synchronize_srcu(s1);
	WRITE_ONCE(*x, 1);
}

locations [0:r1]
exists (0:r1=1 /\ 0:r2=0)