Message-ID: <d1e28124-b7a7-ae19-87ec-b1dcd3701b61@redhat.com>
Date:   Tue, 24 Jan 2023 10:42:24 -0500
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.6.0
Subject: Re: [PATCH] Fix data race in mark_rt_mutex_waiters
Content-Language: en-US
To:     Hernan Ponce de Leon <hernan.poncedeleon@huaweicloud.com>,
        paulmck@kernel.org
Cc:     Arjan van de Ven <arjan@linux.intel.com>, peterz@infradead.org,
        mingo@redhat.com, will@kernel.org, boqun.feng@gmail.com,
        akpm@osdl.org, tglx@linutronix.de, joel@joelfernandes.org,
        stern@rowland.harvard.edu, diogo.behrens@huawei.com,
        jonas.oberhauser@huawei.com, linux-kernel@vger.kernel.org,
        Hernan Ponce de Leon <hernanl.leon@huawei.com>,
        stable@vger.kernel.org
References: <20230120135525.25561-1-hernan.poncedeleon@huaweicloud.com>
 <562c883b-b2c3-3a27-f045-97e7e3281e0b@linux.intel.com>
 <20230120155439.GI2948950@paulmck-ThinkPad-P17-Gen-1>
 <9a1c7959-4b8c-94df-a3e2-e69be72bfd7d@huaweicloud.com>
 <20230123164014.GN2948950@paulmck-ThinkPad-P17-Gen-1>
 <f17dcce0-d510-a112-3127-984e8e73f480@huaweicloud.com>
From:   Waiman Long <longman@redhat.com>
In-Reply-To: <f17dcce0-d510-a112-3127-984e8e73f480@huaweicloud.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk


On 1/24/23 09:57, Hernan Ponce de Leon wrote:
>> In the case, the value read is passed into cmpxchg_relaxed(), which
>> checks the value against memory.  In this case, as Arjan noted, the only
>> compiler-and-silicon difference between data_race() and READ_ONCE()
>> is that use of data_race() might allow the compiler to do things like
>> tear the load, thus forcing the occasional spurious cmpxchg_relaxed()
>> failure.  In contrast, LKMM (by design) throws up its hands when it sees
>> a data race.  Something about not being eager to track the 
>> idiosyncrasies
>> of many compiler versions.
>>
>> My approach in my own code is to use *_ONCE() unless it causes a visible
>> performance regression or if it confuses KCSAN.  An example of the 
>> latter
>> can be debug code, in which case use of data_race() avoids suppressing
>> KCSAN warnings (and also false positives, depending).
>
> I understand that *_ONCE() might avoid some compiler optimization and 
> reduce performance in the general case. However, if I understand your 
> first paragraph correctly, in this particular case data_race() could 
> allow the CAS to fail more often, resulting in more spinning 
> iterations and degraded performance. Am I right?
>
>>
>> Except that your other email seems to also be arguing that additional
>> ordering is required.  So is https://lkml.org/lkml/2023/1/20/702 really
>> sufficient just by itself, or is additional ordering required?
>
> I do not claim that we need to mark the read to add the ordering that 
> is needed for correctness (mutual exclusion). What I claim in this 
> patch is that there is a data race, and since it can affect ordering 
> constrains in subtle ways, I consider it harmful and thus I want to 
> fix it.
>
> What I explain in the other email is that if we fix the data race, 
> either the fence or the acquire store might be relaxed (because 
> marking the read gives us some extra ordering guarantees). If the race 
> is not fixed, both the fence and the acquire are needed according to 
> LKMM. The situation is different wrt hardware models. In that case the 
> tool cannot find any violation even if we don't fix the race and we 
> relax the store / remove the fence.

I would suggest to do it as suggested by PeterZ. Instead of set_bit(), 
however, it is probably better to use atomic_long_or() like

atomic_long_or_relaxed(RT_MUTEX_HAS_WAITERS, (atomic_long_t *)&lock->owner)

The mutex code stores the lock owner as atomic_long_t. So it is natural 
to treat &lock->owner as atomic_long_t here too.

Cheers,
Longman