Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Thu, 28 Oct 2021 10:46:54 +0200
From:   Sebastian Andrzej Siewior <bigeasy@linutronix.de>
To:     Ronny Meeus <ronny.meeus@gmail.com>
Cc:     linux-kernel@vger.kernel.org, linux-rt-users@vger.kernel.org
Subject: Re: Unbounded priority inversion while assigning tasks into cgroups.
Message-ID: <20211028084654.bgtvnibvqnz2o5rh@linutronix.de>
References: <CAMJ=MEd9WuGA0MN+n0rGD6T+sgd=yciTmeEW9TjRjNXt+cF=qQ@mail.gmail.com>
 <20211027165800.md2gxbsku4avqjgt@linutronix.de>
 <CAMJ=MEfkQ9VaphaNS_qbWMOANo7P6h2Ln6iYg4JLWbWzxp85mA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <CAMJ=MEfkQ9VaphaNS_qbWMOANo7P6h2Ln6iYg4JLWbWzxp85mA@mail.gmail.com>
Precedence: bulk

On 2021-10-27 22:54:33 [+0200], Ronny Meeus wrote:
> > From a looking at percpu_rw_semaphore implementation, no new readers are
> > allowed as long as there is a writer pending. The writer has
> > (unfortunately) to wait until all readers are out. But then I doubt that
> > this takes up to two minutes for all existing readers to leave the
> > critical section.
> >
> 
> The readers can be running at low priority while there can be other threads
> with a medium priority will consume the complete cpu. So the low prio
> readers are just waiting to be scheduled and by that also block the high
> prio thread.

Hmm. So you have say, 5 reads stuck in the RW semaphore while preempted
be medium tasks and high-prio writer is then stuck on semaphore, waiting
for the MED tasks to finish so the low-prio threads can leave the
criticial section?

> Looking at v4.9.84, at least the RT implementation of rw_semaphore
> > allows new readers if a writer is pending. So this could be culprit as
> > you would have to wait until all reader are gone and the writer needs to
> > grab the lock before another reader shows up. But then this shouldn't be
> > the case for the generic implementation and new reader should wait until
> > the writer got its chance.
> >
> 
> So what do you suggest for the v4.9 kernel as a solution? Move to the RT
> version of the rw_semaphore and hope for the best?

I don't think it will help. Based on what you wrote above it appears
that the problem is that the readers are preempted and are not leaving
the critical section soon enough.

How many CPUs do you have? Maybe using a rtmutex here and allowing only
one reader at a time isn't that bad in your case. With one CPU for
instance, there isn't much space for multiple readers I guess.

Sebastian