Message-ID: <1430245038.2004.21.camel@stgolabs.net>
Subject: Re: [PATCH v3] locking/rwsem: reduce spinlock contention in wakeup
 after up_read/up_write
From: Davidlohr Bueso <dave@stgolabs.net>
To: Waiman Long <waiman.long@hp.com>
Cc: Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@kernel.org>,
        linux-kernel@vger.kernel.org, Jason Low <jason.low2@hp.com>,
        Scott J Norton <scott.norton@hp.com>,
        Douglas Hatch <doug.hatch@hp.com>
Date: Tue, 28 Apr 2015 11:17:18 -0700
In-Reply-To: <553E9B25.4060802@hp.com>
References: <1429898069-28907-1-git-send-email-Waiman.Long@hp.com>
	 <1429907956.10273.65.camel@stgolabs.net> <553E9B25.4060802@hp.com>
Content-Type: text/plain; charset="UTF-8"
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 1547
Lines: 37

On Mon, 2015-04-27 at 16:25 -0400, Waiman Long wrote:
> >> +	/*
> >> +	 * up_write() cleared the owner field before calling this function.
> >> +	 * If that field is now set, a writer must have stolen the lock and
> >> +	 * the wakeup operation should be aborted.
> >> +	 */
> >> +	if (rwsem_has_active_writer(sem))
> >> +		goto out;
> > We currently allow small races between rwsem owner and counter checks.
> > And __rwsem_do_wake() can be called by checking the former -- and lock
> > stealing is done with the counter as well. Please see below how we back
> > out of such cases, as it is very much considered when granting the next
> > reader. So nack to this as is, sorry.
> 
> If the first one in the queue is a writer, wake_up_process() may be 
> called directly which can be quite expensive if the lock has already 
> been stolen as the task will have to sleep again.

But how can this occur? Lock stealing takes form in two places:

1) fastpath: only if the counter is 0 -- which, since we are discussing
waking up waiter(s) code, obviously cannot occur.

2) With the cmpxchg() in rwsem_try_write_lock(), which is serialized
with the wait_lock, so again this cannot occur.

Which is why this is not considered in __rwsem_do_wake() when waking the
writer fist in the queue.

Thanks,
Davidlohr

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/