Message-ID: <554137EC.4030700@hp.com>
Date: Wed, 29 Apr 2015 15:58:36 -0400
From: Waiman Long <waiman.long@hp.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0.12) Gecko/20130109 Thunderbird/10.0.12
MIME-Version: 1.0
To: Davidlohr Bueso <dave@stgolabs.net>
CC: Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@kernel.org>,
        linux-kernel@vger.kernel.org, Jason Low <jason.low2@hp.com>,
        Scott J Norton <scott.norton@hp.com>,
        Douglas Hatch <doug.hatch@hp.com>
Subject: Re: [PATCH v3] locking/rwsem: reduce spinlock contention in wakeup
 after up_read/up_write
References: <1429898069-28907-1-git-send-email-Waiman.Long@hp.com>  <1429907956.10273.65.camel@stgolabs.net> <553E9B25.4060802@hp.com> <1430245038.2004.21.camel@stgolabs.net>
In-Reply-To: <1430245038.2004.21.camel@stgolabs.net>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 1939
Lines: 45

On 04/28/2015 02:17 PM, Davidlohr Bueso wrote:
> On Mon, 2015-04-27 at 16:25 -0400, Waiman Long wrote:
>>>> +	/*
>>>> +	 * up_write() cleared the owner field before calling this function.
>>>> +	 * If that field is now set, a writer must have stolen the lock and
>>>> +	 * the wakeup operation should be aborted.
>>>> +	 */
>>>> +	if (rwsem_has_active_writer(sem))
>>>> +		goto out;
>>> We currently allow small races between rwsem owner and counter checks.
>>> And __rwsem_do_wake() can be called by checking the former -- and lock
>>> stealing is done with the counter as well. Please see below how we back
>>> out of such cases, as it is very much considered when granting the next
>>> reader. So nack to this as is, sorry.
>> If the first one in the queue is a writer, wake_up_process() may be
>> called directly which can be quite expensive if the lock has already
>> been stolen as the task will have to sleep again.
> But how can this occur? Lock stealing takes form in two places:
>
> 1) fastpath: only if the counter is 0 -- which, since we are discussing
> waking up waiter(s) code, obviously cannot occur.
>
> 2) With the cmpxchg() in rwsem_try_write_lock(), which is serialized
> with the wait_lock, so again this cannot occur.
>
> Which is why this is not considered in __rwsem_do_wake() when waking the
> writer fist in the queue.
>
> Thanks,
> Davidlohr
>

A write lock can also be acquired by a spinning writer in 
rwsem_try_write_lock_unqueued() where wait_lock isn't used. With 
multiple down_read's, it is possible that the first exiting reader wakes 
up a writer who acquires the write lock while the other readers are 
waiting for acquiring the wait_lock.

Cheers,
Longman
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/