Date: Mon, 21 Nov 2016 13:29:35 +0100
From: Peter Zijlstra <peterz@infradead.org>
To: Oleg Nesterov <oleg@redhat.com>
Cc: Davidlohr Bueso <dave@stgolabs.net>, mingo@kernel.org,
        john.stultz@linaro.org, dimitrysh@google.com,
        linux-kernel@vger.kernel.org, Davidlohr Bueso <dbueso@suse.de>
Subject: Re: [PATCH 3/3] locking/percpu-rwsem: Avoid unnecessary writer
 wakeups
Message-ID: <20161121122935.GD3092@twins.programming.kicks-ass.net>
References: <1479495277-9075-1-git-send-email-dave@stgolabs.net>
 <1479495277-9075-4-git-send-email-dave@stgolabs.net>
 <20161121122343.GA635@redhat.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20161121122343.GA635@redhat.com>
User-Agent: Mutt/1.5.23.1 (2014-03-12)
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 1300
Lines: 38

On Mon, Nov 21, 2016 at 01:23:44PM +0100, Oleg Nesterov wrote:
> On 11/18, Davidlohr Bueso wrote:
> >
> > +static bool __readers_active_check(struct percpu_rw_semaphore *sem)
> > +{
> > +	return !(per_cpu_sum(*sem->read_count) !=0);
> > +}
> 
> Hmm,
> 
> 	return per_cpu_sum(*sem->read_count) == 0;
> 
> looks more clear, but this is minor,

Very much so; that must be one of the most convoluted statements
possible :-).

> 
> >  int __percpu_init_rwsem(struct percpu_rw_semaphore *sem,
> >  			const char *name, struct lock_class_key *rwsem_key)
> >  {
> > @@ -103,41 +141,11 @@ void __percpu_up_read(struct percpu_rw_semaphore *sem)
> >  	__this_cpu_dec(*sem->read_count);
> >  
> >  	/* Prod writer to recheck readers_active */
> > -	swake_up(&sem->writer);
> > +	if (__readers_active_check(sem))
> > +		swake_up(&sem->writer);
> 
> Suppose we have 2 active readers which call __percpu_up_read() at the same
> time and the pending writer sleeps.
> 
> What guarantees that one of these readers will observe per_cpu_sum() == 0 ?
> They both can read the old value of the remote per-cpu counter, no?

In particular, you're thinking of what provides the guarantee that the
woken CPU observes the same state the wakee saw? Isn't this one of the
Program-Order guarantees the scheduler _should_ provide?