Subject: Re: [PATCH] refcount_t: documentation for memory ordering differences
To: Andrea Parri <parri.andrea@gmail.com>
Cc: Elena Reshetova <elena.reshetova@intel.com>, peterz@infradead.org,
        linux-kernel@vger.kernel.org, keescook@chromium.org,
        david@fromorbit.com, Alan Stern <stern@rowland.harvard.edu>,
        "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
References: <1511958996-19501-1-git-send-email-elena.reshetova@intel.com>
 <1511958996-19501-2-git-send-email-elena.reshetova@intel.com>
 <2dbef3c8-a70f-9584-2de5-3440520f224a@infradead.org>
 <20171203062003.GA3020@andrea>
From: Randy Dunlap <rdunlap@infradead.org>
Message-ID: <221d1191-0d13-5c05-4337-5373023e8d27@infradead.org>
Date: Sun, 3 Dec 2017 09:27:51 -0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.4.0
MIME-Version: 1.0
In-Reply-To: <20171203062003.GA3020@andrea>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 3671
Lines: 80

On 12/02/2017 10:20 PM, Andrea Parri wrote:
> On Fri, Dec 01, 2017 at 12:34:23PM -0800, Randy Dunlap wrote:
>> On 11/29/2017 04:36 AM, Elena Reshetova wrote:
>>> Some functions from refcount_t API provide different
>>> memory ordering guarantees that their atomic counterparts.
>>> This adds a document outlining these differences.
>>>
>>> Signed-off-by: Elena Reshetova <elena.reshetova@intel.com>
>>> ---
>>>  Documentation/core-api/index.rst              |   1 +
>>>  Documentation/core-api/refcount-vs-atomic.rst | 129 ++++++++++++++++++++++++++
>>>  2 files changed, 130 insertions(+)
>>>  create mode 100644 Documentation/core-api/refcount-vs-atomic.rst
>>
>>> diff --git a/Documentation/core-api/refcount-vs-atomic.rst b/Documentation/core-api/refcount-vs-atomic.rst
>>> new file mode 100644
>>> index 0000000..5619d48
>>> --- /dev/null
>>> +++ b/Documentation/core-api/refcount-vs-atomic.rst
>>> @@ -0,0 +1,129 @@
>>> +===================================
>>> +refcount_t API compared to atomic_t
>>> +===================================
>>> +
>>> +The goal of refcount_t API is to provide a minimal API for implementing
>>> +an object's reference counters. While a generic architecture-independent
>>> +implementation from lib/refcount.c uses atomic operations underneath,
>>> +there are a number of differences between some of the refcount_*() and
>>> +atomic_*() functions with regards to the memory ordering guarantees.
>>> +This document outlines the differences and provides respective examples
>>> +in order to help maintainers validate their code against the change in
>>> +these memory ordering guarantees.
>>> +
>>> +memory-barriers.txt and atomic_t.txt provide more background to the
>>> +memory ordering in general and for atomic operations specifically.
>>> +
>>> +Relevant types of memory ordering
>>> +=================================
>>> +
>>> +**Note**: the following section only covers some of the memory
>>> +ordering types that are relevant for the atomics and reference
>>> +counters and used through this document. For a much broader picture
>>> +please consult memory-barriers.txt document.
>>> +
>>> +In the absence of any memory ordering guarantees (i.e. fully unordered)
>>> +atomics & refcounters only provide atomicity and
>>> +program order (po) relation (on the same CPU). It guarantees that
>>> +each atomic_*() and refcount_*() operation is atomic and instructions
>>> +are executed in program order on a single CPU.
>>> +This is implemented using READ_ONCE()/WRITE_ONCE() and
>>> +compare-and-swap primitives.
>>> +
>>> +A strong (full) memory ordering guarantees that all prior loads and
>>> +stores (all po-earlier instructions) on the same CPU are completed
>>> +before any po-later instruction is executed on the same CPU.
>>> +It also guarantees that all po-earlier stores on the same CPU
>>> +and all propagated stores from other CPUs must propagate to all
>>> +other CPUs before any po-later instruction is executed on the original
>>> +CPU (A-cumulative property). This is implemented using smp_mb().
>>
>> I don't know what "A-cumulative property" means, and google search didn't
>> either.
> 
> The description above seems to follow the (informal) definition given in:
> 
>   https://github.com/aparri/memory-model/blob/master/Documentation/explanation.txt
>   (c.f., in part., Sect. 13-14)
> 
> and formalized by the LKMM. (The notion of A-cumulativity also appears, in
> different contexts, in some memory consistency literature, e.g.,
> 
>   http://www.cl.cam.ac.uk/~pes20/ppc-supplemental/index.html
>   http://www.cl.cam.ac.uk/~pes20/armv8-mca/
>   https://arxiv.org/abs/1308.6810 )

Got it.  Thanks.


-- 
~Randy