Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757564AbcLPOUD (ORCPT ); Fri, 16 Dec 2016 09:20:03 -0500 Received: from mail-wj0-f194.google.com ([209.85.210.194]:36137 "EHLO mail-wj0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1761482AbcLPOTr (ORCPT ); Fri, 16 Dec 2016 09:19:47 -0500 Subject: Re: [PATCH v2 05/11] locking/ww_mutex: Add waiters in stamp order To: Peter Zijlstra References: <1480601214-26583-1-git-send-email-nhaehnle@gmail.com> <1480601214-26583-6-git-send-email-nhaehnle@gmail.com> <20161206165544.GX3045@worktop.programming.kicks-ass.net> Cc: linux-kernel@vger.kernel.org, =?UTF-8?Q?Nicolai_H=c3=a4hnle?= , Ingo Molnar , Maarten Lankhorst , Daniel Vetter , Chris Wilson , dri-devel@lists.freedesktop.org From: =?UTF-8?Q?Nicolai_H=c3=a4hnle?= Message-ID: Date: Fri, 16 Dec 2016 15:19:43 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <20161206165544.GX3045@worktop.programming.kicks-ass.net> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3723 Lines: 107 Hi Peter and Chris, (trying to combine the handoff discussion here) On 06.12.2016 17:55, Peter Zijlstra wrote: > On Thu, Dec 01, 2016 at 03:06:48PM +0100, Nicolai H?hnle wrote: >> @@ -693,8 +748,12 @@ __mutex_lock_common(struct mutex *lock, long state, unsigned int subclass, >> * mutex_unlock() handing the lock off to us, do a trylock >> * before testing the error conditions to make sure we pick up >> * the handoff. >> + * >> + * For w/w locks, we always need to do this even if we're not >> + * currently the first waiter, because we may have been the >> + * first waiter during the unlock. >> */ >> - if (__mutex_trylock(lock, first)) >> + if (__mutex_trylock(lock, use_ww_ctx || first)) >> goto acquired; > > So I'm somewhat uncomfortable with this. The point is that with the > .handoff logic it is very easy to accidentally allow: > > mutex_lock(&a); > mutex_lock(&a); > > And I'm not sure this doesn't make that happen for ww_mutexes. We get to > this __mutex_trylock() without first having blocked. Okay, took me a while, but I see the problem. If we have: ww_mutex_lock(&a, NULL); ww_mutex_lock(&a, ctx); then it's possible that another currently waiting task sets the HANDOFF flag between those calls and we'll allow the second ww_mutex_lock to go through. The concern about picking up a handoff that we didn't request is real, though it cannot happen in the first iteration. Perhaps this __mutex_trylock can be moved to the end of the loop? See below... > > >> /* >> @@ -716,7 +775,20 @@ __mutex_lock_common(struct mutex *lock, long state, unsigned int subclass, >> spin_unlock_mutex(&lock->wait_lock, flags); >> schedule_preempt_disabled(); >> >> - if (!first && __mutex_waiter_is_first(lock, &waiter)) { >> + if (use_ww_ctx && ww_ctx) { >> + /* >> + * Always re-check whether we're in first position. We >> + * don't want to spin if another task with a lower >> + * stamp has taken our position. >> + * >> + * We also may have to set the handoff flag again, if >> + * our position at the head was temporarily taken away. >> + */ >> + first = __mutex_waiter_is_first(lock, &waiter); >> + >> + if (first) >> + __mutex_set_flag(lock, MUTEX_FLAG_HANDOFF); >> + } else if (!first && __mutex_waiter_is_first(lock, &waiter)) { >> first = true; >> __mutex_set_flag(lock, MUTEX_FLAG_HANDOFF); >> } > > So the point is that !ww_ctx entries are 'skipped' during the insertion > and therefore, if one becomes first, it must stay first? Yes. Actually, it should be possible to replace all the cases of use_ww_ctx || first with ww_ctx. Similarly, all cases of use_ww_ctx && ww_ctx could be replaced by just ww_ctx. > >> @@ -728,7 +800,7 @@ __mutex_lock_common(struct mutex *lock, long state, unsigned int subclass, >> * or we must see its unlock and acquire. >> */ >> if ((first && mutex_optimistic_spin(lock, ww_ctx, use_ww_ctx, true)) || >> - __mutex_trylock(lock, first)) >> + __mutex_trylock(lock, use_ww_ctx || first)) >> break; >> >> spin_lock_mutex(&lock->wait_lock, flags); Change this code to: acquired = first && mutex_optimistic_spin(lock, ww_ctx, use_ww_ctx, &waiter); spin_lock_mutex(&lock->wait_lock, flags); if (acquired || __mutex_trylock(lock, use_ww_ctx || first)) break; } This changes the trylock to always be under the wait_lock, but we previously had that at the beginning of the loop anyway. It also removes back-to-back calls to __mutex_trylock when going through the loop; and for the first iteration, there is a __mutex_trylock under wait_lock already before adding ourselves to the wait list. What do you think? Nicolai