Received-SPF: pass (google.com: domain of linux-kernel+bounces-184887-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161;
Message-ID: <a536d330-a3e9-f747-99d0-0375d7ce9afb@huaweicloud.com>
Date: Tue, 21 May 2024 13:38:41 +0200
Precedence: bulk
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Thunderbird/102.13.0
Subject: Re: LKMM: Making RMW barriers explicit
Content-Language: en-US
To: Alan Stern <stern@rowland.harvard.edu>
Cc: Jonas Oberhauser <jonas.oberhauser@huaweicloud.com>,
 "Paul E. McKenney" <paulmck@kernel.org>, linux-kernel@vger.kernel.org,
 linux-arch@vger.kernel.org, kernel-team@meta.com, parri.andrea@gmail.com,
 boqun.feng@gmail.com, j.alglave@ucl.ac.uk, luc.maranget@inria.fr,
 Joel Fernandes <joel@joelfernandes.org>
References: <72c804c8-2511-4349-a823-bc1de8bb729e@rowland.harvard.edu>
 <e030f7a4-97e7-4e91-bbae-230ee5c97763@huaweicloud.com>
 <a9bf972c-b5ee-f1c2-36bf-30ba62f419d7@huaweicloud.com>
 <2f20e7cf-7c67-4ad3-8a0c-3c1d01257ae4@rowland.harvard.edu>
From: Hernan Ponce de Leon <hernan.poncedeleon@huaweicloud.com>
In-Reply-To: <2f20e7cf-7c67-4ad3-8a0c-3c1d01257ae4@rowland.harvard.edu>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 5/18/2024 2:31 AM, Alan Stern wrote:
> On Thu, May 16, 2024 at 10:44:05AM +0200, Hernan Ponce de Leon wrote:
>> On 5/16/2024 10:31 AM, Jonas Oberhauser wrote:
>>>
>>>
>>> Am 5/16/2024 um 3:43 AM schrieb Alan Stern:
>>>> Hernan and Jonas:
>>>>
>>>> Can you explain more fully the changes you want to make to herd7 and/or
>>>> the LKMM?  The goal is to make the memory barriers currently implicit in
>>>> RMW operations explicit, but I couldn't understand how you propose to do
>>>> this.
>>>>
>>>> Are you going to change herd7 somehow, and if so, how?  It seems like
>>>> you should want to provide sufficient information so that the .bell
>>>> and .cat files can implement the appropriate memory barriers associated
>>>> with each RMW operation.  What additional information is needed?  And
>>>> how (explained in English, not by quoting source code) will the .bell
>>>> and .cat files make use of this information?
>>>>
>>>> Alan
>>>
>>>
>>> I don't know whether herd7 needs to be changed. Probably, herd7 does the
>>> following:
>>> - if a tag called Mb appears on an rmw instruction (by instruction I
>>> mean things like xchg(), atomic_inc_return_relaxed()), replace it with
>>> one of those things:
>>>     * full mb ; once (the rmw) ; full mb, if a value returning
>>> (successful) rmw
>>>     * once (the rmw)   otherwise
>>> - everything else gets translated 1:1 into some internal representation
>>
>> This is my understanding from reading the source code of CSem.ml in herd7's
>> repo.
>>
>> Also, this is exactly what dartagnan is currently doing.
>>
>>>
>>> What I'm proposing is:
>>> 1. remove this transpilation step,
>>> 2. and instead allow the Mb tag to actually appear on RMW instructions
>>> 3. change the cat file to explicitly define the behavior of the Mb tag
>>> on RMW instructions
>>
>> These are the exact 3 things I changed in dartagnan for testing what Jonas
>> proposed.
>>
>> I am not sure if further changes are needed for herd7.

I implemented these changes in herd7 and they seem enough.
I opened a PRs for discussion
	https://github.com/herd/herdtools7/pull/865

> 
> Okay, good.  This answers the first part of what I asked.  What about
> the second part?  That is, how will the changes to the .def, .bell, and
> .cat files achieve your goals?

At least the cat model needs to be updated. If I remove the fences from 
herd7, but keep the current model, these 4 tests fail.

  --- Summary:
  !!! Result changed: 
/litmus/manual/locked/SUW+or-ow+l-ow-or.litmus.out.new
  !!! Result changed: 
/litmus/manual/atomic/C-PaulEMcKenney-SB+adat-o+adat-o.litmus.out.new
  !!! Result changed: ./litmus/manual/atomic/C-atomic-01.litmus.out.new
  !!! Result changed: ./litmus/manual/atomic/C-atomic-02.litmus.out.new

Using this patch I get the correct results

diff --git a/tools/memory-model/linux-kernel.cat 
b/tools/memory-model/linux-kernel.cat
index adf3c4f41229..7e4787cdbd66 100644
--- a/tools/memory-model/linux-kernel.cat
+++ b/tools/memory-model/linux-kernel.cat
@@ -34,6 +34,12 @@ let R4rmb = R \ Noreturn     (* Reads for which rmb 
works *)
  let rmb = [R4rmb] ; fencerel(Rmb) ; [R4rmb]
  let wmb = [W] ; fencerel(Wmb) ; [W]
  let mb = ([M] ; fencerel(Mb) ; [M]) |
+       (* everything across a full barrier RMW is ordered. This 
includes up to one event inside the RMW. *)
+       ([M] ; po ; [RMW & Mb] ; po ; [M]) |
+       (* full barrier RMW writes are ordered with everything behind 
the RMW *)
+       ([W & RMW & Mb] ; po ; [M]) |
+       (* full barrier RMW reads are ordered with everything before the 
RMW *)
+       ([M] ; po ; [R & RMW & Mb]) |
         ([M] ; fencerel(Before-atomic) ; [RMW] ; po? ; [M]) |
         ([M] ; po? ; [RMW] ; fencerel(After-atomic) ; [M]) |
         ([M] ; po? ; [LKW] ; fencerel(After-spinlock) ; [M]) |

Hernan

> 
> Alan