Received-SPF: pass (google.com: domain of linux-kernel+bounces-50189-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161;
From: John Ogness <john.ogness@linutronix.de>
To: paulmck@kernel.org
Cc: Tejun Heo <tj@kernel.org>, Lai Jiangshan <jiangshanlai@gmail.com>, Petr
 Mladek <pmladek@suse.com>, Steven Rostedt <rostedt@goodmis.org>, Sergey
 Senozhatsky <senozhatsky@chromium.org>, Stephen Rothwell
 <sfr@canb.auug.org.au>, linux-kernel@vger.kernel.org, rcu@vger.kernel.org
Subject: Re: [BUG] workqueues and printk not playing nice since next-20240130
In-Reply-To: <7b5a386e-2dcf-49f1-a8e9-c46d2a9f8800@paulmck-laptop>
References: <410d6a87-bf34-457e-b714-1e6149d48532@paulmck-laptop>
 <87le83ara7.fsf@jogness.linutronix.de>
 <7b5a386e-2dcf-49f1-a8e9-c46d2a9f8800@paulmck-laptop>
Date: Fri, 02 Feb 2024 18:08:25 +0106
Message-ID: <87zfwibxzy.fsf@jogness.linutronix.de>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain

On 2024-02-02, "Paul E. McKenney" <paulmck@kernel.org> wrote:
>> The printk ringbuffer contents would certainly be interesting.
>> 
>> If you build the GDB scripts (CONFIG_GDB_SCRIPTS) then you will have:
>> 
>> (gdb) lx-dmesg
>
> This says no such command even though I do have CONFIG_GDB_SCRIPTS=y
> in my .config.

You actually need to build them as well. The target is "scripts_gdb"

And you probably need to add:

add-auto-load-safe-path /path/to/your/kernel/build/directory

to your .gdbinit

(This is documented in Documentation/dev-tools/gdb-kernel-debugging.rst)

>> As an alternative, you could copy the contents of
>> Documentation/admin-guide/kdump/gdbmacros.txt into your .gdbinit and
>> then will have:
>> 
>> (gdb) dmesg
>
> This one hangs.

:-/ I will look into this.

> On the other hand, next-20240202 doesn't show the problem.  No idea
> what might have changed.  :-/

Did you check the backtrace on all the "threads"? I would expect one of
them has tty in it and is probably deadlocked. There are known problems
that if a WARN or lockdep triggers while holding the port lock, that CPU
will deadlock itself. That has the effect that no output is generated,
but all the other CPUs will run fine. And even printk() calls will
happily store into the ringbuffer because they use trylock for printing
and the deadlocked CPU will be holding the lock.

John