From:   David Laight <David.Laight@ACULAB.COM>
To:     'Steven Rostedt' <rostedt@goodmis.org>,
        John Stultz <jstultz@google.com>
CC:     LKML <linux-kernel@vger.kernel.org>, Wei Wang <wvw@google.com>,
        "Midas Chien" <midaschieh@google.com>,
        Kees Cook <keescook@chromium.org>,
        "Anton Vorontsov" <anton@enomsg.org>,
        "Guilherme G. Piccoli" <gpiccoli@igalia.com>,
        Tony Luck <tony.luck@intel.com>,
        "kernel-team@android.com" <kernel-team@android.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        "Peter Zijlstra" <peterz@infradead.org>,
        Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Subject: RE: [PATCH] pstore: Revert pmsg_lock back to a normal mutex
Thread-Topic: [PATCH] pstore: Revert pmsg_lock back to a normal mutex
Thread-Index: AQHZTUSK7fWJqBV/U0q1fZmOSFQwN67oEXJw
Date:   Thu, 2 Mar 2023 22:41:36 +0000
Message-ID: <dcf2fa0bde1e49a286c57c1e7d4a78d3@AcuMS.aculab.com>
References: <20230302062741.483079-1-jstultz@google.com>
        <20230302082414.77613351@gandalf.local.home>
        <CANDhNCo4ruC4pP+iDe49b3e1nAcWtYQj4bx82+oZhyLFYkdFJQ@mail.gmail.com>
 <20230302152103.2618f1b7@gandalf.local.home>
In-Reply-To: <20230302152103.2618f1b7@gandalf.local.home>
Accept-Language: en-GB, en-US
MIME-Version: 1.0
Content-Language: en-US
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT
Precedence: bulk

From: Steven Rostedt
> Sent: 02 March 2023 20:21
...
> There's no harm in spinning, as the task can still be preempted, and
> there's no issue of priority inversion, because the spinners will not be on
> the same CPU as the owner and the top waiter, if they only spin if those
> two tasks are also running on a CPU.

ISTM that a spinlock should spin - even on an RT kernel.
If it might spin for longer than it takes to do a process
switch it shouldn't be a spinlock at all.
(I bet there are quite a few that do spin for ages...)
Adaptive spinning for a sleep lock (as an optimisation)
is entirely different.

I changed some very old driver code that used sema4 (which
always sleep) to mutex (which quite often spin) and got a
massive performance gain.

I've also had terrible problems trying to get a multithreaded
user program to work well [1].
Because you don't have spinlocks (userpace can always be preempted)
you can't bound the time mutex are held for.
So any vaguely 'hot' lock (maybe just used to remove an item
from a list) can get interrupted by a hardware interrupt.
The only way to make it work is to use atomic operations
instead of mutex.

I can't help feeing that the RT kernel suffers from the
same problems if the system is under any kind of load.
You might get slightly better RT response, but the overall
amount of 'work' a system can actually do will be lower.

[1] Test was 36 threads on a 40 cpu system that need to
spend about 90% of the time processing RTP (UDP) audio.
This also involves 500k ethernet packets/sec (tx and rx).
It is all possible, but there were a lot of pitfalls.

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)