Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) client-ip=23.128.96.32;
MIME-Version: 1.0
References: <CAD=FV=XhqDxmk=yTdujwtFyF23NZ13LORH0GMS5_iTAEVva_rA@mail.gmail.com>
 <20231122035304.57483-1-lizhe.67@bytedance.com> <ZWTORl4jMi9ODns1@alley>
In-Reply-To: <ZWTORl4jMi9ODns1@alley>
From:   Doug Anderson <dianders@chromium.org>
Date:   Mon, 27 Nov 2023 13:55:16 -0800
Message-ID: <CAD=FV=UJc3Ln7KDXVfXfpmbWmmbp2v=1o4T_11nv2R=bWttZeQ@mail.gmail.com>
Subject: Re: [RFC] softlockup: serialized softlockup's log
To:     Petr Mladek <pmladek@suse.com>
Cc:     lizhe.67@bytedance.com, akpm@linux-foundation.org,
        kernelfans@gmail.com, lecopzer.chen@mediatek.com,
        linux-kernel@vger.kernel.org, lizefan.x@bytedance.com,
        John Ogness <john.ogness@linutronix.de>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

Hi,

On Mon, Nov 27, 2023 at 9:13=E2=80=AFAM Petr Mladek <pmladek@suse.com> wrot=
e:
>
> > I've tried several times, but unfortunately I haven't been able to
> > reproduce the problem you mentioned. My concern is that if the lock
> > is shared, there will be potential deadlock issues because hardlockup
> > exploits nmi.
>
> Yes, one has to be very careful when using locks under NMI.
> I am aware of three approaches:
>
>   1. It should be safe to use a lock which is used only in NMI.
>      The only problem might be ABBA deadlocks.
>
>      But this is not usable to serialize softlockup and hardlockup.
>
>
>   2. Using try_lock() in NMI and skipping the report when the lock
>      could not be taken.
>
>     But we probably should not skip the detailed info printed by the
>     hardlockup detector just because another softlockup or
>     hardlockup report is running in parallel.
>
>
>   3. Using a lock which might be nested on the same CPU.
>
>     It does not prevent mixed output completely. The nested
>     context will print the messages in the middle of the outer
>     one on the same CPU. But it would serialize output against
>     other CPUs.
>
>     There already is one used to serialize some backtraces,
>     see printk_cpu_sync_get_irqsave() in dump_stack_lvl() and
>     nmi_cpu_backtrace().
>
>     It should be safe when the locked code does not wait for
>     other CPUs. And when it is a leaf lock so that there
>     is no need to think about ABBA deadlocks against
>     other locks.
>
>     IMHO, printk_cpu_sync_get_irqsave() probably should be
>     safe to serialize the softlockup and hardlockup
>     detectors.
>
>
> Adding John into Cc. He is familiar with
> printk_cpu_sync_get_irqsave(). He wanted to use it also
> for synchronizing console drivers. Well, it will be done
> another way in the end.

Thanks for all the detailed info! Using printk_cpu_sync_get_irqsave()
looks pretty promising. I'll try to do some testing with this and then
aim to send a patch later this week that builds upon Li Zhe's patch.
If everything works out I'll probably remove the newly added spinlock
and use printk_cpu_sync_get_irqsave() instead, but the spinlock should
work fine in the short term in any case.

-Doug