Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Mon, 1 Jul 2019 16:34:48 -0500
From:   Corey Minyard <minyard@acm.org>
To:     Steven Rostedt <rostedt@goodmis.org>
Cc:     Corey Minyard <cminyard@mvista.com>,
        Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
        linux-rt-users@vger.kernel.org, linux-kernel@vger.kernel.org,
        Peter Zijlstra <peterz@infradead.org>, tglx@linutronix.de
Subject: Re: [PATCH RT v2] Fix a lockup in wait_for_completion() and friends
Message-ID: <20190701213448.GC4336@minyard.net>
Reply-To: minyard@acm.org
References: <20190509193320.21105-1-minyard@acm.org>
 <20190510103318.6cieoifz27eph4n5@linutronix.de>
 <20190628214903.6f92a9ea@oasis.local.home>
 <20190701190949.GB4336@minyard.net>
 <20190701161840.1a53c9e4@gandalf.local.home>
 <20190701204325.GD5041@minyard.net>
 <20190701170602.2fdb35c2@gandalf.local.home>
 <20190701171333.37cc0567@gandalf.local.home>
 <20190701172825.7d861e85@gandalf.local.home>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20190701172825.7d861e85@gandalf.local.home>
User-Agent: Mutt/1.9.4 (2018-02-28)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Mon, Jul 01, 2019 at 05:28:25PM -0400, Steven Rostedt wrote:
> On Mon, 1 Jul 2019 17:13:33 -0400
> Steven Rostedt <rostedt@goodmis.org> wrote:
> 
> > On Mon, 1 Jul 2019 17:06:02 -0400
> > Steven Rostedt <rostedt@goodmis.org> wrote:
> > 
> > > On Mon, 1 Jul 2019 15:43:25 -0500
> > > Corey Minyard <cminyard@mvista.com> wrote:
> > > 
> > >   
> > > > I show that patch is already applied at
> > > > 
> > > >     1921ea799b7dc561c97185538100271d88ee47db
> > > >     sched/completion: Fix a lockup in wait_for_completion()
> > > > 
> > > > git describe --contains 1921ea799b7dc561c97185538100271d88ee47db
> > > > v4.19.37-rt20~1
> > > > 
> > > > So I'm not sure what is going on.    
> > > 
> > > Bah, I'm replying to the wrong commit that I'm having issues with.
> > > 
> > > I searched your name to find the patch that is of trouble, and picked
> > > this one.
> > > 
> > > I'll go find the problem patch, sorry for the noise on this one.
> > >   
> > 
> > No, I did reply to the right email, but it wasn't the top patch I was
> > having issues with. It was the patch I replied to:
> > 
> > This change below that Sebastian marked as stable-rt is what is causing
> > me an issue. Not the patch that started the thread.
> > 
> 
> In fact, my system doesn't boot with this commit in 5.0-rt.
> 
> If I revert 90e1b18eba2ae4a729 ("swait: Delete the task from after a
> wakeup occured") the machine boots again.
> 
> Sebastian, I think that's a bad commit, please revert it.

Yeah.  d_wait_lookup() does not use __SWAITQUEUE_INITIALIZER() to
intitialize it's queue item, but uses swake_up_all(), so it goes
into an infinite loop since it won't remove the item because remove
isn't set.

I'd suspect there are other places this is the case.

-corey

> 
> Thanks!
> 
> -- Steve
> 
> > 
> > 
> > > Now.. that will fix it, but I think it is also wrong.
> > > 
> > > The problem being that it violates FIFO, something that might be more
> > > important on -RT than elsewhere.
> > > 
> > > The regular wait API seems confused/inconsistent when it uses
> > > autoremove_wake_function and default_wake_function, which doesn't help,
> > > but we can easily support this with swait -- the problematic thing is
> > > the custom wake functions, we musn't do that.
> > > 
> > > (also, mingo went and renamed a whole bunch of wait_* crap and didn't do
> > > the same to swait_ so now its named all different :/)
> > > 
> > > Something like the below perhaps.
> > > 
> > > ---
> > > diff --git a/include/linux/swait.h b/include/linux/swait.h
> > > index 73e06e9986d4..f194437ae7d2 100644
> > > --- a/include/linux/swait.h
> > > +++ b/include/linux/swait.h
> > > @@ -61,11 +61,13 @@ struct swait_queue_head {
> > >  struct swait_queue {
> > >  	struct task_struct	*task;
> > >  	struct list_head	task_list;
> > > +	unsigned int		remove;
> > >  };
> > >  
> > >  #define __SWAITQUEUE_INITIALIZER(name) {				\
> > >  	.task		= current,					\
> > >  	.task_list	= LIST_HEAD_INIT((name).task_list),		\
> > > +	.remove		= 1,						\
> > >  }
> > >  
> > >  #define DECLARE_SWAITQUEUE(name)					\
> > > diff --git a/kernel/sched/swait.c b/kernel/sched/swait.c
> > > index e83a3f8449f6..86974ecbabfc 100644
> > > --- a/kernel/sched/swait.c
> > > +++ b/kernel/sched/swait.c
> > > @@ -28,7 +28,8 @@ void swake_up_locked(struct swait_queue_head *q)
> > >  
> > >  	curr = list_first_entry(&q->task_list, typeof(*curr), task_list);
> > >  	wake_up_process(curr->task);
> > > -	list_del_init(&curr->task_list);
> > > +	if (curr->remove)
> > > +		list_del_init(&curr->task_list);
> > >  }
> > >  EXPORT_SYMBOL(swake_up_locked);
> > >  
> > > @@ -57,7 +58,8 @@ void swake_up_all(struct swait_queue_head *q)
> > >  		curr = list_first_entry(&tmp, typeof(*curr), task_list);
> > >  
> > >  		wake_up_state(curr->task, TASK_NORMAL);
> > > -		list_del_init(&curr->task_list);
> > > +		if (curr->remove)
> > > +			list_del_init(&curr->task_list);
> > >  
> > >  		if (list_empty(&tmp))
> > >  			break;  
> > 
>