Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 20 Jun 2022 12:02:34 +0200
From:   Marek =?UTF-8?B?QmVow7pu?= <kabel@kernel.org>
To:     Sergey Senozhatsky <senozhatsky@chromium.org>
Cc:     John Ogness <john.ogness@linutronix.de>,
        Petr Mladek <pmladek@suse.com>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        linux-kernel@vger.kernel.org
Subject: Re: Boot stall regression from "printk for 5.19" merge
Message-ID: <20220620120234.5a65ac97@thinkpad>
In-Reply-To: <YrAEUM20n3Rc4aOn@google.com>
References: <20220619204949.50d9154d@thinkpad>
        <YrAEUM20n3Rc4aOn@google.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Mon, 20 Jun 2022 14:23:28 +0900
Sergey Senozhatsky <senozhatsky@chromium.org> wrote:

> On (22/06/19 20:49), Marek Beh=C3=BAn wrote:
> [..]
> > causes a regression on arm64 (Marvell CN9130-CRB board) where the
> > system boot freezes in most cases (and is unusable until restarted by
> > watchdog), or, in some cases boots, but the console output gets mangled
> > for a while (the serial console spits garbage characters). =20
>=20
> Can you please try disabling console kthreads and see how the boot
> process goes? Just `return 0` from printk_activate_kthreads() (I think
> this should do the trick).

This indeed makes the problem go away...

>=20
> > The stall example:
> >=20
> >   ...
> >   [    1.108141] io scheduler mq-deadline registered
> >   [    1.108143] io scheduler kyber registered
> >   [    1.163484] armada-ap806-pinctrl f06f4000.system-controller:pinctr=
l: registered pinctrl driver
> >   [
> >=20
> > Or another:
> >=20
> >   ...
> >   [    1.108155] io scheduler mq-deadline registered
> >   [    1.108156] io scheduler kyber registered
> >   [    1.162991] armada-ap806-pinctrl f06f4000.system-controller:pinctr=
l: registered pinctrl driver
> >   [    1.163361] armada-cp110-pinctrl f2440000.system-controller:pinctr=
l: registered pinctrl driver
> >   [    1.180827] mv_xor_v2 f0400000.xor: Marvell Version 2 XOR driver
> >   [ =20
> >=20
> > The garbage example:
> >=20
> >   ...
> >   [    0.920951] raid6: using neon recovery algorithm
> >   [    0.921228] iommu: Default domain type: Translated
> >   %
> >=20
> >           gb@k+cFL/[    4.954974] DSA: tree 0 setup
> >   [    4.955286] cfg80211: Loading compiled-in X.509 certificates for r=
egulatory database =20
>=20
> This is pretty suspicious. I don't see how console kthreads would
> corrupt the output. I suspect that something else is going on, some
> memory corruption, etc.

Maybe multiple threads are writing to serial registers, or something...

Marek