Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Wed, 26 Jun 2019 12:35:58 +0200
From:   Peter Zijlstra <peterz@infradead.org>
To:     Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc:     minyard@acm.org, linux-rt-users@vger.kernel.org,
        Corey Minyard <cminyard@mvista.com>,
        linux-kernel@vger.kernel.org, tglx@linutronix.de,
        Steven Rostedt <rostedt@goodmis.org>
Subject: Re: [PATCH RT v2] Fix a lockup in wait_for_completion() and friends
Message-ID: <20190626103558.GL3419@hirez.programming.kicks-ass.net>
References: <20190508205728.25557-1-minyard@acm.org>
 <20190509161925.kul66w54wpjcinuc@linutronix.de>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20190509161925.kul66w54wpjcinuc@linutronix.de>
User-Agent: Mutt/1.10.1 (2018-07-13)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Thu, May 09, 2019 at 06:19:25PM +0200, Sebastian Andrzej Siewior wrote:
> One question for the upstream completion implementation:
> completion_done() returns true if there are no waiters. It acquires the
> wait.lock to ensure that complete()/complete_all() is done. However,
> once complete releases the lock it is guaranteed that the wake_up() (for
> the waiter) occurred. The waiter task still needs to be remove itself
> from the wait-queue before the completion can be removed.
> Do I miss something?

So you mean:

	init_completion(&done);


	wait_for_copmletion(&done)
	  spin_lock()
	   __add_wait_queue()
	  spin_unlock()
	  schedule()

					complete()
								completion_done()
	  spin_lock()
	  __remove_wait_queue()
	  spin_unlock()

Right?

I think that boils down to that whenever you have multiple waiters,
someone needs to be in charge of @done's lifetime.

The case that matters is:

	DECLARE_COMPLETION_ONSTACK(done)

	while (!completion_done(&done))
		cpu_relax();

Where there is but a single waiter, and that waiter is
completion_done(). In that case it must not return early.

Now, I've also seen a ton of code do:

	if (!completion_done(done))
		complete(done);

And that makes me itch... but I've not bothered to look into it hard
enough.