Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Wed, 2 Sep 2020 08:54:10 -0700
From:   "Paul E. McKenney" <paulmck@kernel.org>
To:     Davidlohr Bueso <dave@stgolabs.net>
Cc:     peterz@infradead.org, mingo@redhat.com, will@kernel.org,
        linux-kernel@vger.kernel.org
Subject: Re: Question on task_blocks_on_rt_mutex()
Message-ID: <20200902155410.GH29330@paulmck-ThinkPad-P72>
Reply-To: paulmck@kernel.org
References: <20200831224911.GA13114@paulmck-ThinkPad-P72>
 <20200831232130.GA28456@paulmck-ThinkPad-P72>
 <20200901174938.GA8158@paulmck-ThinkPad-P72>
 <20200901235821.GA8516@paulmck-ThinkPad-P72>
 <20200902015128.wsulcxhbo7dutcjz@linux-p48b>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20200902015128.wsulcxhbo7dutcjz@linux-p48b>
User-Agent: Mutt/1.9.4 (2018-02-28)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Tue, Sep 01, 2020 at 06:51:28PM -0700, Davidlohr Bueso wrote:
> On Tue, 01 Sep 2020, Paul E. McKenney wrote:
> 
> > And it appears that a default-niced CPU-bound SCHED_OTHER process is
> > not preempted by a newly awakened MAX_NICE SCHED_OTHER process.  OK,
> > OK, I never waited for more than 10 minutes, but on my 2.2GHz that is
> > close enough to a hang for most people.
> > 
> > Which means that the patch below prevents the hangs.  And maybe does
> > other things as well, firing rcutorture up on it to check.
> > 
> > But is this indefinite delay expected behavior?
> > 
> > This reproduces for me on current mainline as follows:
> > 
> > tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --torture lock --duration 3 --configs LOCK05
> > 
> > This hangs within a minute of boot on my setup.  Here "hangs" is defined
> > as stopping the per-15-second console output of:
> > 	Writes:  Total: 569906696 Max/Min: 81495031/63736508   Fail: 0
> 
> Ok this doesn't seem to be related to lockless wake_qs then. fyi there have
> been missed wakeups in the past where wake_q_add() fails the cmpxchg because
> the task is already pending a wakeup leading to the actual wakeup ocurring
> before its corresponding wake_up_q(). This is why we have wake_q_add_safe().
> But for rtmutexes, because there is no lock stealing only top-waiter is awoken
> as well as try_to_take_rt_mutex() is done under the lock->wait_lock I was not
> seeing an actual race here.

This problem is avoided if stutter_wait() does the occasional sleep.
I would have expected preemption to take effect, but even setting the
kthreads in stutter_wait() to MAX_NICE doesn't help.  The current fix
destroys intended instant-on nature of stutter_wait(), so the eventual
fix will need to use hrtimer-based sleeps or some such.

							Thanx, Paul