Message-ID: <1147abb3-fb72-dd63-8e32-25ff8000972e@redhat.com>
Date:   Sun, 26 Feb 2023 16:31:15 -0500
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.6.0
Subject: Re: [PATCH 3/6] locking/rwsem: Rework writer wakeup
Content-Language: en-US
To:     Peter Zijlstra <peterz@infradead.org>
Cc:     mingo@redhat.com, will@kernel.org, linux-kernel@vger.kernel.org,
        boqun.feng@gmail.com
References: <20230223122642.491637862@infradead.org>
 <20230223123319.487908155@infradead.org>
 <bf2948c4-dd6a-1cf6-16b5-39e5e17ef72a@redhat.com>
 <Y/tJ2n1e22YhsZ17@hirez.programming.kicks-ass.net>
From:   Waiman Long <longman@redhat.com>
In-Reply-To: <Y/tJ2n1e22YhsZ17@hirez.programming.kicks-ass.net>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 2/26/23 07:00, Peter Zijlstra wrote:
> On Thu, Feb 23, 2023 at 04:38:08PM -0500, Waiman Long wrote:
>
>>> @@ -1143,54 +1138,36 @@ rwsem_down_write_slowpath(struct rw_sema
>>>    	} else {
>>>    		atomic_long_or(RWSEM_FLAG_WAITERS, &sem->count);
>>>    	}
>>> +	raw_spin_unlock_irq(&sem->wait_lock);
>>>    	/* wait until we successfully acquire the lock */
>>> -	set_current_state(state);
>>>    	trace_contention_begin(sem, LCB_F_WRITE);
>>>    	for (;;) {
>>> -		if (rwsem_try_write_lock(sem, &waiter)) {
>>> -			/* rwsem_try_write_lock() implies ACQUIRE on success */
>>> +		set_current_state(state);
>>> +		if (!smp_load_acquire(&waiter.task)) {
>>> +			/* Matches rwsem_waiter_wake()'s smp_store_release(). */
>>>    			break;
>>>    		}
>>> -
>>> -		raw_spin_unlock_irq(&sem->wait_lock);
>>> -
>>> -		if (signal_pending_state(state, current))
>>> -			goto out_nolock;
>>> -
>>> -		/*
>>> -		 * After setting the handoff bit and failing to acquire
>>> -		 * the lock, attempt to spin on owner to accelerate lock
>>> -		 * transfer. If the previous owner is a on-cpu writer and it
>>> -		 * has just released the lock, OWNER_NULL will be returned.
>>> -		 * In this case, we attempt to acquire the lock again
>>> -		 * without sleeping.
>>> -		 */
>>> -		if (waiter.handoff_set) {
>>> -			enum owner_state owner_state;
>>> -
>>> -			owner_state = rwsem_spin_on_owner(sem);
>>> -			if (owner_state == OWNER_NULL)
>>> -				goto trylock_again;
>>> +		if (signal_pending_state(state, current)) {
>>> +			raw_spin_lock_irq(&sem->wait_lock);
>>> +			if (waiter.task)
>>> +				goto out_nolock;
>>> +			raw_spin_unlock_irq(&sem->wait_lock);
>>> +			/* Ordered by sem->wait_lock against rwsem_mark_wake(). */
>>> +			break;
>>>    		}
>>> -
>>>    		schedule_preempt_disabled();
>>>    		lockevent_inc(rwsem_sleep_writer);
>>> -		set_current_state(state);
>>> -trylock_again:
>>> -		raw_spin_lock_irq(&sem->wait_lock);
>>>    	}
>>>    	__set_current_state(TASK_RUNNING);
>>> -	raw_spin_unlock_irq(&sem->wait_lock);
>>>    	lockevent_inc(rwsem_wlock);
>>>    	trace_contention_end(sem, 0);
>>>    	return sem;
>>>    out_nolock:
>>> -	__set_current_state(TASK_RUNNING);
>>> -	raw_spin_lock_irq(&sem->wait_lock);
>>>    	rwsem_del_wake_waiter(sem, &waiter, &wake_q);
>>> +	__set_current_state(TASK_RUNNING);
>>>    	lockevent_inc(rwsem_wlock_fail);
>>>    	trace_contention_end(sem, -EINTR);
>>>    	return ERR_PTR(-EINTR);
>> I believe it is better to change state inside the wait_lock critical section
>> to provide a release barrier for free.
> I can't follow... a release for what? Note that the reader slowpath has
> this exact form already.\

You are right. I forgot that we don't need synchronization when setting 
state to TASK_RUNNING.

Cheers,
Longman