2023-08-29 19:11:55

by Marc Haber

[permalink] [raw]
Subject: Re: Linux 6.5 speed regression, boot VERY slow with anything systemd related

[Please keep me on Cc, I am only subscribed to linux-kernel]

Hi Bagas,

thanks for your quick answer.

On Tue, Aug 29, 2023 at 03:17:15PM +0700, Bagas Sanjaya wrote:
> In any case, bisecting kernel is highly appreciated in order to pin down
> the culprit.

Without having read the docs (that came too late, need to read up on
that again), my bisect came out at
84a9582fd203063cd4d301204971ff2cd8327f1a being the first bad commit.
This is a rather big one, that does not easily back out of the 6.5
release. Sadly, just transplanting drivers/tty/serial from a 6.4.12 tree
doesn't even build. I'm adding Tony Lindgren, the author of the commit,
to the Cc list.

But, since the commit is related to serial port, I began fiddling around
with the serial port setting on the misbehaving VM and found out that
running the VM without the serial console that I am using (thus removing
"console=ttyS0,57600n8" from the kernel command line) makes the machine
boot up just fine with the 6.5 kernel that I built yesterday. It is not
even necessary to remove the virtual serial port.

The issue is still somehow connected to the host the machine is running
on, since my VMs all have a serial console and the test VMs running on
different hosts are running fine with 6.5.

Greetings
Marc

--
-----------------------------------------------------------------------------
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany | lose things." Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421


2023-08-30 19:19:35

by Marc Haber

[permalink] [raw]
Subject: Re: Linux 6.5 speed regression, boot VERY slow with anything systemd related

Hi,

while debugging in the direction of qemu/kvm continues, I think that
this is an actual problem with the serial code.

On Tue, Aug 29, 2023 at 03:00:51PM +0200, Marc Haber wrote:
> my bisect came out at
> 84a9582fd203063cd4d301204971ff2cd8327f1a being the first bad commit.
> This is a rather big one, that does not easily back out of the 6.5
> release. Sadly, just transplanting drivers/tty/serial from a 6.4.12 tree
> doesn't even build. I'm adding Tony Lindgren, the author of the commit,
> to the Cc list.

Last night, I tried hard to roll back that commit and ended up with
doing

git checkout v6.5
git checkout 84a9582fd2^ -- drivers/tty/serial/
git checkout 84a9582fd2^ -- include/linux/serial_8250.h
git checkout 84a9582fd2^ -- include/linux/serial_core.h

which resulted in a 6.5 kernel that actually works without obvious
regression on the VM in question.

Thanks to Hilko Bengen and Bastian Blank who helped with the git
gymnastics and with understanding the patch in question. I wouldn't have
been able to roll that back without their help.

Greetings
Marc

--
-----------------------------------------------------------------------------
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany | lose things." Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421