On 6/30/21 11:35 AM, Peter Zijlstra wrote:
> Yanfei reported that it is possible to loose HANDOFF when we race with
> mutex_unlock() and end up setting HANDOFF on an unlocked mutex. At
> that point anybody can steal it, loosing HANDOFF in the process.
>
> If this happens often enough, we can in fact starve the top waiter.
>
> Solve this by folding the 'set HANDOFF' operation into the trylock
> operation, such that either we acquire the lock, or it gets HANDOFF
> set. This avoids having HANDOFF set on an unlocked mutex.
>
> Reported-by: Yanfei Xu <[email protected]>
> Signed-off-by: Peter Zijlstra (Intel) <[email protected]>
> ---
> kernel/locking/mutex.c | 58 +++++++++++++++++++++++++++++--------------------
> 1 file changed, 35 insertions(+), 23 deletions(-)
>
> --- a/kernel/locking/mutex.c
> +++ b/kernel/locking/mutex.c
> @@ -91,10 +91,7 @@ static inline unsigned long __owner_flag
> return owner & MUTEX_FLAGS;
> }
>
> -/*
> - * Trylock variant that returns the owning task on failure.
> - */
> -static inline struct task_struct *__mutex_trylock_or_owner(struct mutex *lock)
> +static inline struct task_struct *__mutex_trylock_common(struct mutex *lock, bool handoff)
> {
> unsigned long owner, curr = (unsigned long)current;
>
> @@ -104,39 +101,56 @@ static inline struct task_struct *__mute
> unsigned long task = owner & ~MUTEX_FLAGS;
>
> if (task) {
> - if (likely(task != curr))
> + if (flags & MUTEX_FLAG_PICKUP) {
> + if (task != curr)
> + break;
> + flags &= ~MUTEX_FLAG_HANDOFF;
I think you mean "flags &= ~MUTEX_FLAG_PICKUP". Right:-)
Cheers,
Longman