2024-01-16 01:14:37

by Mirsad Todorovac

[permalink] [raw]
Subject: [BUG][BISECTED] Freeze at loading init ramdisk

Hi,

On the Ubuntu 22.04 LTS Jammy platform, on a mainline vanilla torvalds tree kernel, the boot
freezes upon first two lines and before any systemd messages.

(Please find the config attached.)

Bisecting the bug led to this result:

marvin@defiant:~/linux/kernel/linux_torvalds$ git bisect good
d97a78423c33f68ca6543de510a409167baed6f5 is the first bad commit
commit d97a78423c33f68ca6543de510a409167baed6f5
Merge: 61da593f4458 689237ab37c5
Author: Linus Torvalds <[email protected]>
Date: Fri Jan 12 14:38:08 2024 -0800

Merge tag 'fbdev-for-6.8-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/deller/linux-fbdev

Pull fbdev updates from Helge Deller:
"Three fbdev drivers (~8500 lines of code) removed. The Carillo Ranch
fbdev driver is for an Intel product which was never shipped, and for
the intelfb and the amba-clcd drivers the drm drivers can be used
instead.

The other code changes are minor: some fb_deferred_io flushing fixes,
imxfb margin fixes and stifb cleanups.

Summary:
- Remove intelfb fbdev driver (Thomas Zimmermann)
- Remove amba-clcd fbdev driver (Linus Walleij)
- Remove vmlfb Carillo Ranch fbdev driver (Matthew Wilcox)
- fb_deferred_io flushing fixes (Nam Cao)
- imxfb code fixes and cleanups (Dario Binacchi)
- stifb primary screen detection cleanups (Thomas Zimmermann)"

* tag 'fbdev-for-6.8-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/deller/linux-fbdev: (28 commits)
fbdev/intelfb: Remove driver
fbdev/hyperv_fb: Do not clear global screen_info
firmware/sysfb: Clear screen_info state after consuming it
fbdev/hyperv_fb: Remove firmware framebuffers with aperture helpers
drm/hyperv: Remove firmware framebuffers with aperture helper
fbdev/sis: Remove dependency on screen_info
video/logo: use %u format specifier for unsigned int values
video/sticore: Remove info field from STI struct
arch/parisc: Detect primary video device from device instance
fbdev/stifb: Allocate fb_info instance with framebuffer_alloc()
video/sticore: Store ROM device in STI struct
fbdev: flush deferred IO before closing
fbdev: flush deferred work in fb_deferred_io_fsync()
fbdev: amba-clcd: Delete the old CLCD driver
fbdev: Remove support for Carillo Ranch driver
fbdev: hgafb: fix kernel-doc comments
fbdev: mmp: Fix typo and wording in code comment
fbdev: fsl-diu-fb: Fix sparse warning due to virt_to_phys() prototype change
fbdev: imxfb: add '*/' on a separate line in block comment
fbdev: imxfb: use __func__ for function name
...

Documentation/fb/index.rst | 1 -
Documentation/fb/intelfb.rst | 155 --
Documentation/userspace-api/ioctl/ioctl-number.rst | 1 -
MAINTAINERS | 12 -
arch/parisc/video/fbdev.c | 2 +-
drivers/Makefile | 3 +-
drivers/firmware/sysfb.c | 14 +-
drivers/gpu/drm/hyperv/hyperv_drm_drv.c | 8 +-
drivers/video/backlight/Kconfig | 7 -
drivers/video/backlight/Makefile | 1 -
drivers/video/backlight/cr_bllcd.c | 264 ---
drivers/video/fbdev/Kconfig | 72 -
drivers/video/fbdev/Makefile | 2 -
drivers/video/fbdev/amba-clcd.c | 986 ---------
drivers/video/fbdev/core/fb_defio.c | 8 +-
drivers/video/fbdev/fsl-diu-fb.c | 2 +-
drivers/video/fbdev/hgafb.c | 13 +-
drivers/video/fbdev/hyperv_fb.c | 20 +-
drivers/video/fbdev/imxfb.c | 179 +-
drivers/video/fbdev/intelfb/Makefile | 8 -
drivers/video/fbdev/intelfb/intelfb.h | 382 ----
drivers/video/fbdev/intelfb/intelfb_i2c.c | 209 --
drivers/video/fbdev/intelfb/intelfbdrv.c | 1680 ----------------
drivers/video/fbdev/intelfb/intelfbhw.c | 2115 --------------------
drivers/video/fbdev/intelfb/intelfbhw.h | 609 ------
drivers/video/fbdev/mmp/hw/mmp_spi.c | 2 +-
drivers/video/fbdev/sis/sis_main.c | 37 -
drivers/video/fbdev/stifb.c | 109 +-
drivers/video/fbdev/vermilion/Makefile | 6 -
drivers/video/fbdev/vermilion/cr_pll.c | 195 --
drivers/video/fbdev/vermilion/vermilion.c | 1175 -----------
drivers/video/fbdev/vermilion/vermilion.h | 245 ---
drivers/video/logo/pnmtologo.c | 6 +-
drivers/video/sticore.c | 5 +
include/linux/amba/clcd-regs.h | 87 -
include/linux/amba/clcd.h | 290 ---
include/video/sticore.h | 6 +-
37 files changed, 208 insertions(+), 8708 deletions(-)
delete mode 100644 Documentation/fb/intelfb.rst
delete mode 100644 drivers/video/backlight/cr_bllcd.c
delete mode 100644 drivers/video/fbdev/amba-clcd.c
delete mode 100644 drivers/video/fbdev/intelfb/Makefile
delete mode 100644 drivers/video/fbdev/intelfb/intelfb.h
delete mode 100644 drivers/video/fbdev/intelfb/intelfb_i2c.c
delete mode 100644 drivers/video/fbdev/intelfb/intelfbdrv.c
delete mode 100644 drivers/video/fbdev/intelfb/intelfbhw.c
delete mode 100644 drivers/video/fbdev/intelfb/intelfbhw.h
delete mode 100644 drivers/video/fbdev/vermilion/Makefile
delete mode 100644 drivers/video/fbdev/vermilion/cr_pll.c
delete mode 100644 drivers/video/fbdev/vermilion/vermilion.c
delete mode 100644 drivers/video/fbdev/vermilion/vermilion.h
delete mode 100644 include/linux/amba/clcd-regs.h
delete mode 100644 include/linux/amba/clcd.h
marvin@defiant:~/linux/kernel/linux_torvalds$

Hope this helps.

Best regards,
Mirsad Todorovac


Attachments:
config-6.7.0-initrd-02743-gca6c080eef42.xz (57.21 kB)

2024-01-17 18:48:33

by Mirsad Todorovac

[permalink] [raw]
Subject: Re: [BUG][BISECTED] Freeze at loading init ramdisk

On 1/16/24 01:32, Mirsad Todorovac wrote:
> Hi,
>
> On the Ubuntu 22.04 LTS Jammy platform, on a mainline vanilla torvalds tree kernel, the boot
> freezes upon first two lines and before any systemd messages.
>
> (Please find the config attached.)
>
> Bisecting the bug led to this result:
>
> marvin@defiant:~/linux/kernel/linux_torvalds$ git bisect good
> d97a78423c33f68ca6543de510a409167baed6f5 is the first bad commit
> commit d97a78423c33f68ca6543de510a409167baed6f5
> Merge: 61da593f4458 689237ab37c5
> Author: Linus Torvalds <[email protected]>
> Date:   Fri Jan 12 14:38:08 2024 -0800
>
>     Merge tag 'fbdev-for-6.8-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/deller/linux-fbdev
>     Pull fbdev updates from Helge Deller:
>      "Three fbdev drivers (~8500 lines of code) removed. The Carillo Ranch
>       fbdev driver is for an Intel product which was never shipped, and for
>       the intelfb and the amba-clcd drivers the drm drivers can be used
>       instead.
>       The other code changes are minor: some fb_deferred_io flushing fixes,
>       imxfb margin fixes and stifb cleanups.
>       Summary:
>        - Remove intelfb fbdev driver (Thomas Zimmermann)
>        - Remove amba-clcd fbdev driver (Linus Walleij)
>        - Remove vmlfb Carillo Ranch fbdev driver (Matthew Wilcox)
>        - fb_deferred_io flushing fixes (Nam Cao)
>        - imxfb code fixes and cleanups (Dario Binacchi)
>        - stifb primary screen detection cleanups (Thomas Zimmermann)"
>     * tag 'fbdev-for-6.8-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/deller/linux-fbdev: (28 commits)
>       fbdev/intelfb: Remove driver
>       fbdev/hyperv_fb: Do not clear global screen_info
>       firmware/sysfb: Clear screen_info state after consuming it
>       fbdev/hyperv_fb: Remove firmware framebuffers with aperture helpers
>       drm/hyperv: Remove firmware framebuffers with aperture helper
>       fbdev/sis: Remove dependency on screen_info
>       video/logo: use %u format specifier for unsigned int values
>       video/sticore: Remove info field from STI struct
>       arch/parisc: Detect primary video device from device instance
>       fbdev/stifb: Allocate fb_info instance with framebuffer_alloc()
>       video/sticore: Store ROM device in STI struct
>       fbdev: flush deferred IO before closing
>       fbdev: flush deferred work in fb_deferred_io_fsync()
>       fbdev: amba-clcd: Delete the old CLCD driver
>       fbdev: Remove support for Carillo Ranch driver
>       fbdev: hgafb: fix kernel-doc comments
>       fbdev: mmp: Fix typo and wording in code comment
>       fbdev: fsl-diu-fb: Fix sparse warning due to virt_to_phys() prototype change
>       fbdev: imxfb: add '*/' on a separate line in block comment
>       fbdev: imxfb: use __func__ for function name
>       ...
>
>  Documentation/fb/index.rst                         |    1 -
>  Documentation/fb/intelfb.rst                       |  155 --
>  Documentation/userspace-api/ioctl/ioctl-number.rst |    1 -
>  MAINTAINERS                                        |   12 -
>  arch/parisc/video/fbdev.c                          |    2 +-
>  drivers/Makefile                                   |    3 +-
>  drivers/firmware/sysfb.c                           |   14 +-
>  drivers/gpu/drm/hyperv/hyperv_drm_drv.c            |    8 +-
>  drivers/video/backlight/Kconfig                    |    7 -
>  drivers/video/backlight/Makefile                   |    1 -
>  drivers/video/backlight/cr_bllcd.c                 |  264 ---
>  drivers/video/fbdev/Kconfig                        |   72 -
>  drivers/video/fbdev/Makefile                       |    2 -
>  drivers/video/fbdev/amba-clcd.c                    |  986 ---------
>  drivers/video/fbdev/core/fb_defio.c                |    8 +-
>  drivers/video/fbdev/fsl-diu-fb.c                   |    2 +-
>  drivers/video/fbdev/hgafb.c                        |   13 +-
>  drivers/video/fbdev/hyperv_fb.c                    |   20 +-
>  drivers/video/fbdev/imxfb.c                        |  179 +-
>  drivers/video/fbdev/intelfb/Makefile               |    8 -
>  drivers/video/fbdev/intelfb/intelfb.h              |  382 ----
>  drivers/video/fbdev/intelfb/intelfb_i2c.c          |  209 --
>  drivers/video/fbdev/intelfb/intelfbdrv.c           | 1680 ----------------
>  drivers/video/fbdev/intelfb/intelfbhw.c            | 2115 --------------------
>  drivers/video/fbdev/intelfb/intelfbhw.h            |  609 ------
>  drivers/video/fbdev/mmp/hw/mmp_spi.c               |    2 +-
>  drivers/video/fbdev/sis/sis_main.c                 |   37 -
>  drivers/video/fbdev/stifb.c                        |  109 +-
>  drivers/video/fbdev/vermilion/Makefile             |    6 -
>  drivers/video/fbdev/vermilion/cr_pll.c             |  195 --
>  drivers/video/fbdev/vermilion/vermilion.c          | 1175 -----------
>  drivers/video/fbdev/vermilion/vermilion.h          |  245 ---
>  drivers/video/logo/pnmtologo.c                     |    6 +-
>  drivers/video/sticore.c                            |    5 +
>  include/linux/amba/clcd-regs.h                     |   87 -
>  include/linux/amba/clcd.h                          |  290 ---
>  include/video/sticore.h                            |    6 +-
>  37 files changed, 208 insertions(+), 8708 deletions(-)
>  delete mode 100644 Documentation/fb/intelfb.rst
>  delete mode 100644 drivers/video/backlight/cr_bllcd.c
>  delete mode 100644 drivers/video/fbdev/amba-clcd.c
>  delete mode 100644 drivers/video/fbdev/intelfb/Makefile
>  delete mode 100644 drivers/video/fbdev/intelfb/intelfb.h
>  delete mode 100644 drivers/video/fbdev/intelfb/intelfb_i2c.c
>  delete mode 100644 drivers/video/fbdev/intelfb/intelfbdrv.c
>  delete mode 100644 drivers/video/fbdev/intelfb/intelfbhw.c
>  delete mode 100644 drivers/video/fbdev/intelfb/intelfbhw.h
>  delete mode 100644 drivers/video/fbdev/vermilion/Makefile
>  delete mode 100644 drivers/video/fbdev/vermilion/cr_pll.c
>  delete mode 100644 drivers/video/fbdev/vermilion/vermilion.c
>  delete mode 100644 drivers/video/fbdev/vermilion/vermilion.h
>  delete mode 100644 include/linux/amba/clcd-regs.h
>  delete mode 100644 include/linux/amba/clcd.h
> marvin@defiant:~/linux/kernel/linux_torvalds$
>
> Hope this helps.

P.S.

As I see that this is a larger merge commit, with 5K+ lines changed, I don't think I can
bisect further to determine the culprit.

But I thought later that it would be interesting to see why my hardware triggered the freeze
and probably others did not, or someone would complain already.

Both of the boxes were AMD Ryzen: Ryzen 7 5700G and Ryzen 9 7950X.

FWIW, I am attaching both hardware listings and the config used, so anyone knowledgeable with
fbdev could possibly narrow down the search.

Best regards,
Mirsad Todorovac


Attachments:
lshw.txt (57.20 kB)
lshw-ryzen-7.txt (37.94 kB)
config-6.7.0-initrd-02743-gca6c080eef42.xz (57.21 kB)
Download all attachments

2024-01-18 07:46:31

by Uwe Kleine-König

[permalink] [raw]
Subject: Re: [BUG][BISECTED] Freeze at loading init ramdisk

Hello Mirsad,

On Wed, Jan 17, 2024 at 07:47:49PM +0100, Mirsad Todorovac wrote:
> On 1/16/24 01:32, Mirsad Todorovac wrote:
> > On the Ubuntu 22.04 LTS Jammy platform, on a mainline vanilla torvalds tree kernel, the boot
> > freezes upon first two lines and before any systemd messages.
> >
> > (Please find the config attached.)
> >
> > Bisecting the bug led to this result:
> >
> > marvin@defiant:~/linux/kernel/linux_torvalds$ git bisect good
> > d97a78423c33f68ca6543de510a409167baed6f5 is the first bad commit
> > commit d97a78423c33f68ca6543de510a409167baed6f5
> > Merge: 61da593f4458 689237ab37c5
> > Author: Linus Torvalds <[email protected]>
> > Date:?? Fri Jan 12 14:38:08 2024 -0800
> >
> > [...]
> >
> > Hope this helps.
>
> P.S.
>
> As I see that this is a larger merge commit, with 5K+ lines changed, I don't think I can
> bisect further to determine the culprit.

Actually it's not that hard. If a merge commit is the first bad commit
for a bisection, either the merge wasn't done correctly (less likely,
looking at d97a78423c33f68ca6543de510a409167baed6f5 I'd bet this isn't
the problem); or changes on different sides conflict or you did
something wrong during bisection.

To rule out the third option, you can just retest d97a78423c33,
61da593f4458 and 689237ab37c5. If d97a78423c33 is the only bad one, you
did it right.

Then to further debug the second option you can find out the offending
commit on each side with a bisection as follows, here for the RHS (i.e.
689237ab37c5):

git bisect start 689237ab37c5 $(git merge-base 61da593f4458 689237ab37c5)

and then in each bisection step do:

git merge --no-commit 61da593f4458
test if the problem is present
git reset --hard
git bisect good/bad

In this case you get merge conflicts in drivers/video/fbdev/amba-clcd.c
and drivers/video/fbdev/vermilion/vermilion.c. In the assumption that
you don't have these enabled in your .config, you can just ignore these.

Side note: A problem during bisection can be that the .config changes
along the process. You should put your config into (say)
arch/x86/configs/lala_defconfig and do

make lala_defconfig

before building each step to prevent this.

Best regards
Uwe

--
Pengutronix e.K. | Uwe Kleine-K?nig |
Industrial Linux Solutions | https://www.pengutronix.de/ |


Attachments:
(No filename) (2.37 kB)
signature.asc (499.00 B)
Download all attachments

2024-01-18 20:04:35

by Mirsad Todorovac

[permalink] [raw]
Subject: Re: [BUG][BISECTED] Freeze at loading init ramdisk



On 1/18/24 08:45, Uwe Kleine-König wrote:
> Hello Mirsad,
>
> On Wed, Jan 17, 2024 at 07:47:49PM +0100, Mirsad Todorovac wrote:
>> On 1/16/24 01:32, Mirsad Todorovac wrote:
>>> On the Ubuntu 22.04 LTS Jammy platform, on a mainline vanilla torvalds tree kernel, the boot
>>> freezes upon first two lines and before any systemd messages.
>>>
>>> (Please find the config attached.)
>>>
>>> Bisecting the bug led to this result:
>>>
>>> marvin@defiant:~/linux/kernel/linux_torvalds$ git bisect good
>>> d97a78423c33f68ca6543de510a409167baed6f5 is the first bad commit
>>> commit d97a78423c33f68ca6543de510a409167baed6f5
>>> Merge: 61da593f4458 689237ab37c5
>>> Author: Linus Torvalds <[email protected]>
>>> Date:   Fri Jan 12 14:38:08 2024 -0800
>>>
>>> [...]
>>>
>>> Hope this helps.
>>
>> P.S.
>>
>> As I see that this is a larger merge commit, with 5K+ lines changed, I don't think I can
>> bisect further to determine the culprit.
>
> Actually it's not that hard. If a merge commit is the first bad commit
> for a bisection, either the merge wasn't done correctly (less likely,
> looking at d97a78423c33f68ca6543de510a409167baed6f5 I'd bet this isn't
> the problem); or changes on different sides conflict or you did
> something wrong during bisection.
>
> To rule out the third option, you can just retest d97a78423c33,
> 61da593f4458 and 689237ab37c5. If d97a78423c33 is the only bad one, you
> did it right.

This was confirmed.

> Then to further debug the second option you can find out the offending
> commit on each side with a bisection as follows, here for the RHS (i.e.
> 689237ab37c5):
>
> git bisect start 689237ab37c5 $(git merge-base 61da593f4458 689237ab37c5)
>
> and then in each bisection step do:
>
> git merge --no-commit 61da593f4458
> test if the problem is present
> git reset --hard
> git bisect good/bad
>
> In this case you get merge conflicts in drivers/video/fbdev/amba-clcd.c
> and drivers/video/fbdev/vermilion/vermilion.c. In the assumption that
> you don't have these enabled in your .config, you can just ignore these.
>
> Side note: A problem during bisection can be that the .config changes
> along the process. You should put your config into (say)
> arch/x86/configs/lala_defconfig and do
>
> make lala_defconfig
>
> before building each step to prevent this.

I must have done something wrong:

marvin@defiant:~/linux/kernel/linux_torvalds$ git bisect log
# bad: [689237ab37c59b9909bc9371d7fece3081683fba] fbdev/intelfb: Remove driver
# good: [de927f6c0b07d9e698416c5b287c521b07694cac] Merge tag 's390-6.8-1' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux
git bisect start '689237ab37c5' 'de927f6c0b07d9e698416c5b287c521b07694cac'
# good: [d9f25b59ed85ae45801cf45fe17eb269b0ef3038] fbdev: Remove support for Carillo Ranch driver
git bisect good d9f25b59ed85ae45801cf45fe17eb269b0ef3038
# good: [e2e0b838a1849f92612a8305c09aaf31bf824350] video/sticore: Remove info field from STI struct
git bisect good e2e0b838a1849f92612a8305c09aaf31bf824350
# good: [778e73d2411abc8f3a2d60dbf038acaec218792e] drm/hyperv: Remove firmware framebuffers with aperture helper
git bisect good 778e73d2411abc8f3a2d60dbf038acaec218792e
# good: [df67699c9cb0ceb70f6cc60630ca938c06773eda] firmware/sysfb: Clear screen_info state after consuming it
git bisect good df67699c9cb0ceb70f6cc60630ca938c06773eda
marvin@defiant:~/linux/kernel/linux_torvalds$

with the error:

marvin@defiant:~/linux/kernel/linux_torvalds$ git bisect good
Bisecting: 0 revisions left to test after this (roughly 0 steps)
drivers/video/fbdev/amba-clcd.c: needs merge
drivers/video/fbdev/vermilion/vermilion.c: needs merge
error: you need to resolve your current index first
marvin@defiant:~/linux/kernel/linux_torvalds$

Best regards,
Mirsad

> Best regards
> Uwe
>

2024-01-18 21:18:35

by Uwe Kleine-König

[permalink] [raw]
Subject: Re: [BUG][BISECTED] Freeze at loading init ramdisk

On Thu, Jan 18, 2024 at 09:04:05PM +0100, Mirsad Todorovac wrote:
>
>
> On 1/18/24 08:45, Uwe Kleine-K?nig wrote:
> > Hello Mirsad,
> >
> > On Wed, Jan 17, 2024 at 07:47:49PM +0100, Mirsad Todorovac wrote:
> > > On 1/16/24 01:32, Mirsad Todorovac wrote:
> > > > On the Ubuntu 22.04 LTS Jammy platform, on a mainline vanilla torvalds tree kernel, the boot
> > > > freezes upon first two lines and before any systemd messages.
> > > >
> > > > (Please find the config attached.)
> > > >
> > > > Bisecting the bug led to this result:
> > > >
> > > > marvin@defiant:~/linux/kernel/linux_torvalds$ git bisect good
> > > > d97a78423c33f68ca6543de510a409167baed6f5 is the first bad commit
> > > > commit d97a78423c33f68ca6543de510a409167baed6f5
> > > > Merge: 61da593f4458 689237ab37c5
> > > > Author: Linus Torvalds <[email protected]>
> > > > Date:?? Fri Jan 12 14:38:08 2024 -0800
> > > >
> > > > [...]
> > > >
> > > > Hope this helps.
> > >
> > > P.S.
> > >
> > > As I see that this is a larger merge commit, with 5K+ lines changed, I don't think I can
> > > bisect further to determine the culprit.
> >
> > Actually it's not that hard. If a merge commit is the first bad commit
> > for a bisection, either the merge wasn't done correctly (less likely,
> > looking at d97a78423c33f68ca6543de510a409167baed6f5 I'd bet this isn't
> > the problem); or changes on different sides conflict or you did
> > something wrong during bisection.
> >
> > To rule out the third option, you can just retest d97a78423c33,
> > 61da593f4458 and 689237ab37c5. If d97a78423c33 is the only bad one, you
> > did it right.
>
> This was confirmed.
>
> > Then to further debug the second option you can find out the offending
> > commit on each side with a bisection as follows, here for the RHS (i.e.
> > 689237ab37c5):
> >
> > git bisect start 689237ab37c5 $(git merge-base 61da593f4458 689237ab37c5)
> >
> > and then in each bisection step do:
> >
> > git merge --no-commit 61da593f4458
> > test if the problem is present
> > git reset --hard
> > git bisect good/bad
> >
> > In this case you get merge conflicts in drivers/video/fbdev/amba-clcd.c
> > and drivers/video/fbdev/vermilion/vermilion.c. In the assumption that
> > you don't have these enabled in your .config, you can just ignore these.
> >
> > Side note: A problem during bisection can be that the .config changes
> > along the process. You should put your config into (say)
> > arch/x86/configs/lala_defconfig and do
> >
> > make lala_defconfig
> >
> > before building each step to prevent this.
>
> I must have done something wrong:
>
> marvin@defiant:~/linux/kernel/linux_torvalds$ git bisect log
> # bad: [689237ab37c59b9909bc9371d7fece3081683fba] fbdev/intelfb: Remove driver
> # good: [de927f6c0b07d9e698416c5b287c521b07694cac] Merge tag 's390-6.8-1' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux
> git bisect start '689237ab37c5' 'de927f6c0b07d9e698416c5b287c521b07694cac'
> # good: [d9f25b59ed85ae45801cf45fe17eb269b0ef3038] fbdev: Remove support for Carillo Ranch driver
> git bisect good d9f25b59ed85ae45801cf45fe17eb269b0ef3038
> # good: [e2e0b838a1849f92612a8305c09aaf31bf824350] video/sticore: Remove info field from STI struct
> git bisect good e2e0b838a1849f92612a8305c09aaf31bf824350
> # good: [778e73d2411abc8f3a2d60dbf038acaec218792e] drm/hyperv: Remove firmware framebuffers with aperture helper
> git bisect good 778e73d2411abc8f3a2d60dbf038acaec218792e
> # good: [df67699c9cb0ceb70f6cc60630ca938c06773eda] firmware/sysfb: Clear screen_info state after consuming it
> git bisect good df67699c9cb0ceb70f6cc60630ca938c06773eda
> marvin@defiant:~/linux/kernel/linux_torvalds$
>
> with the error:
>
> marvin@defiant:~/linux/kernel/linux_torvalds$ git bisect good
> Bisecting: 0 revisions left to test after this (roughly 0 steps)
> drivers/video/fbdev/amba-clcd.c: needs merge
> drivers/video/fbdev/vermilion/vermilion.c: needs merge
> error: you need to resolve your current index first

It seems you forgot the "git reset --hard" step. Doing it in this state
should still be possible.

Best regards
Uwe

--
Pengutronix e.K. | Uwe Kleine-K?nig |
Industrial Linux Solutions | https://www.pengutronix.de/ |


Attachments:
(No filename) (4.27 kB)
signature.asc (499.00 B)
Download all attachments

2024-01-18 21:44:51

by Mirsad Todorovac

[permalink] [raw]
Subject: Re: [BUG][BISECTED] Freeze at loading init ramdisk



On 1/18/24 22:14, Uwe Kleine-König wrote:
> On Thu, Jan 18, 2024 at 09:04:05PM +0100, Mirsad Todorovac wrote:
>>
>>
>> On 1/18/24 08:45, Uwe Kleine-König wrote:
>>> Hello Mirsad,
>>>
>>> On Wed, Jan 17, 2024 at 07:47:49PM +0100, Mirsad Todorovac wrote:
>>>> On 1/16/24 01:32, Mirsad Todorovac wrote:
>>>>> On the Ubuntu 22.04 LTS Jammy platform, on a mainline vanilla torvalds tree kernel, the boot
>>>>> freezes upon first two lines and before any systemd messages.
>>>>>
>>>>> (Please find the config attached.)
>>>>>
>>>>> Bisecting the bug led to this result:
>>>>>
>>>>> marvin@defiant:~/linux/kernel/linux_torvalds$ git bisect good
>>>>> d97a78423c33f68ca6543de510a409167baed6f5 is the first bad commit
>>>>> commit d97a78423c33f68ca6543de510a409167baed6f5
>>>>> Merge: 61da593f4458 689237ab37c5
>>>>> Author: Linus Torvalds <[email protected]>
>>>>> Date:   Fri Jan 12 14:38:08 2024 -0800
>>>>>
>>>>> [...]
>>>>>
>>>>> Hope this helps.
>>>>
>>>> P.S.
>>>>
>>>> As I see that this is a larger merge commit, with 5K+ lines changed, I don't think I can
>>>> bisect further to determine the culprit.
>>>
>>> Actually it's not that hard. If a merge commit is the first bad commit
>>> for a bisection, either the merge wasn't done correctly (less likely,
>>> looking at d97a78423c33f68ca6543de510a409167baed6f5 I'd bet this isn't
>>> the problem); or changes on different sides conflict or you did
>>> something wrong during bisection.
>>>
>>> To rule out the third option, you can just retest d97a78423c33,
>>> 61da593f4458 and 689237ab37c5. If d97a78423c33 is the only bad one, you
>>> did it right.
>>
>> This was confirmed.
>>
>>> Then to further debug the second option you can find out the offending
>>> commit on each side with a bisection as follows, here for the RHS (i.e.
>>> 689237ab37c5):
>>>
>>> git bisect start 689237ab37c5 $(git merge-base 61da593f4458 689237ab37c5)
>>>
>>> and then in each bisection step do:
>>>
>>> git merge --no-commit 61da593f4458
>>> test if the problem is present
>>> git reset --hard
>>> git bisect good/bad
>>>
>>> In this case you get merge conflicts in drivers/video/fbdev/amba-clcd.c
>>> and drivers/video/fbdev/vermilion/vermilion.c. In the assumption that
>>> you don't have these enabled in your .config, you can just ignore these.
>>>
>>> Side note: A problem during bisection can be that the .config changes
>>> along the process. You should put your config into (say)
>>> arch/x86/configs/lala_defconfig and do
>>>
>>> make lala_defconfig
>>>
>>> before building each step to prevent this.
>>
>> I must have done something wrong:
>>
>> marvin@defiant:~/linux/kernel/linux_torvalds$ git bisect log
>> # bad: [689237ab37c59b9909bc9371d7fece3081683fba] fbdev/intelfb: Remove driver
>> # good: [de927f6c0b07d9e698416c5b287c521b07694cac] Merge tag 's390-6.8-1' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux
>> git bisect start '689237ab37c5' 'de927f6c0b07d9e698416c5b287c521b07694cac'
>> # good: [d9f25b59ed85ae45801cf45fe17eb269b0ef3038] fbdev: Remove support for Carillo Ranch driver
>> git bisect good d9f25b59ed85ae45801cf45fe17eb269b0ef3038
>> # good: [e2e0b838a1849f92612a8305c09aaf31bf824350] video/sticore: Remove info field from STI struct
>> git bisect good e2e0b838a1849f92612a8305c09aaf31bf824350
>> # good: [778e73d2411abc8f3a2d60dbf038acaec218792e] drm/hyperv: Remove firmware framebuffers with aperture helper
>> git bisect good 778e73d2411abc8f3a2d60dbf038acaec218792e
>> # good: [df67699c9cb0ceb70f6cc60630ca938c06773eda] firmware/sysfb: Clear screen_info state after consuming it
>> git bisect good df67699c9cb0ceb70f6cc60630ca938c06773eda
>> marvin@defiant:~/linux/kernel/linux_torvalds$
>>
>> with the error:
>>
>> marvin@defiant:~/linux/kernel/linux_torvalds$ git bisect good
>> Bisecting: 0 revisions left to test after this (roughly 0 steps)
>> drivers/video/fbdev/amba-clcd.c: needs merge
>> drivers/video/fbdev/vermilion/vermilion.c: needs merge
>> error: you need to resolve your current index first
>
> It seems you forgot the "git reset --hard" step. Doing it in this state
> should still be possible.

Well, it was possible, but I obviously got the wrong result:

marvin@defiant:~/linux/kernel/linux_torvalds$ git bisect log
# bad: [689237ab37c59b9909bc9371d7fece3081683fba] fbdev/intelfb: Remove driver
# good: [de927f6c0b07d9e698416c5b287c521b07694cac] Merge tag 's390-6.8-1' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux
git bisect start '689237ab37c5' 'de927f6c0b07d9e698416c5b287c521b07694cac'
# good: [d9f25b59ed85ae45801cf45fe17eb269b0ef3038] fbdev: Remove support for Carillo Ranch driver
git bisect good d9f25b59ed85ae45801cf45fe17eb269b0ef3038
# good: [e2e0b838a1849f92612a8305c09aaf31bf824350] video/sticore: Remove info field from STI struct
git bisect good e2e0b838a1849f92612a8305c09aaf31bf824350
# good: [778e73d2411abc8f3a2d60dbf038acaec218792e] drm/hyperv: Remove firmware framebuffers with aperture helper
git bisect good 778e73d2411abc8f3a2d60dbf038acaec218792e
# good: [df67699c9cb0ceb70f6cc60630ca938c06773eda] firmware/sysfb: Clear screen_info state after consuming it
git bisect good df67699c9cb0ceb70f6cc60630ca938c06773eda
# good: [df67699c9cb0ceb70f6cc60630ca938c06773eda] firmware/sysfb: Clear screen_info state after consuming it
git bisect good df67699c9cb0ceb70f6cc60630ca938c06773eda
# good: [c25a19afb81cfd73dab494ba64f9a434cf1a4499] fbdev/hyperv_fb: Do not clear global screen_info
git bisect good c25a19afb81cfd73dab494ba64f9a434cf1a4499
# first bad commit: [689237ab37c59b9909bc9371d7fece3081683fba] fbdev/intelfb: Remove driver
marvin@defiant:~/linux/kernel/linux_torvalds$ uname -rms
Linux 6.7.0-initrd-retest-02751-g689237ab37c5 x86_64
marvin@defiant:~/linux/kernel/linux_torvalds$

.. meaning, I get the 689237ab37c5 as "first bad commit" but it boots.

If you see an obvious error, it would be helpful, otherwise I might retry the bisect, but later.

I am puzzled, but bisecting a merge commit is my first attempt at this.

Best regards,
Mirsad

2024-01-20 11:25:55

by Bagas Sanjaya

[permalink] [raw]
Subject: Re: [BUG][BISECTED] Freeze at loading init ramdisk

On Wed, Jan 17, 2024 at 07:47:49PM +0100, Mirsad Todorovac wrote:
> On 1/16/24 01:32, Mirsad Todorovac wrote:
> > Hi,
> >
> > On the Ubuntu 22.04 LTS Jammy platform, on a mainline vanilla torvalds tree kernel, the boot
> > freezes upon first two lines and before any systemd messages.
> >
> > (Please find the config attached.)
> >
> > Bisecting the bug led to this result:
> >
> > marvin@defiant:~/linux/kernel/linux_torvalds$ git bisect good
> > d97a78423c33f68ca6543de510a409167baed6f5 is the first bad commit
> > commit d97a78423c33f68ca6543de510a409167baed6f5
> > Merge: 61da593f4458 689237ab37c5
> > Author: Linus Torvalds <[email protected]>
> > Date:   Fri Jan 12 14:38:08 2024 -0800
> >
> >     Merge tag 'fbdev-for-6.8-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/deller/linux-fbdev
> >     Pull fbdev updates from Helge Deller:
> >      "Three fbdev drivers (~8500 lines of code) removed. The Carillo Ranch
> >       fbdev driver is for an Intel product which was never shipped, and for
> >       the intelfb and the amba-clcd drivers the drm drivers can be used
> >       instead.
> >       The other code changes are minor: some fb_deferred_io flushing fixes,
> >       imxfb margin fixes and stifb cleanups.
> >       Summary:
> >        - Remove intelfb fbdev driver (Thomas Zimmermann)
> >        - Remove amba-clcd fbdev driver (Linus Walleij)
> >        - Remove vmlfb Carillo Ranch fbdev driver (Matthew Wilcox)
> >        - fb_deferred_io flushing fixes (Nam Cao)
> >        - imxfb code fixes and cleanups (Dario Binacchi)
> >        - stifb primary screen detection cleanups (Thomas Zimmermann)"
> >     * tag 'fbdev-for-6.8-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/deller/linux-fbdev: (28 commits)
> >       fbdev/intelfb: Remove driver
> >       fbdev/hyperv_fb: Do not clear global screen_info
> >       firmware/sysfb: Clear screen_info state after consuming it
> >       fbdev/hyperv_fb: Remove firmware framebuffers with aperture helpers
> >       drm/hyperv: Remove firmware framebuffers with aperture helper
> >       fbdev/sis: Remove dependency on screen_info
> >       video/logo: use %u format specifier for unsigned int values
> >       video/sticore: Remove info field from STI struct
> >       arch/parisc: Detect primary video device from device instance
> >       fbdev/stifb: Allocate fb_info instance with framebuffer_alloc()
> >       video/sticore: Store ROM device in STI struct
> >       fbdev: flush deferred IO before closing
> >       fbdev: flush deferred work in fb_deferred_io_fsync()
> >       fbdev: amba-clcd: Delete the old CLCD driver
> >       fbdev: Remove support for Carillo Ranch driver
> >       fbdev: hgafb: fix kernel-doc comments
> >       fbdev: mmp: Fix typo and wording in code comment
> >       fbdev: fsl-diu-fb: Fix sparse warning due to virt_to_phys() prototype change
> >       fbdev: imxfb: add '*/' on a separate line in block comment
> >       fbdev: imxfb: use __func__ for function name
> >       ...
> >
> >  Documentation/fb/index.rst                         |    1 -
> >  Documentation/fb/intelfb.rst                       |  155 --
> >  Documentation/userspace-api/ioctl/ioctl-number.rst |    1 -
> >  MAINTAINERS                                        |   12 -
> >  arch/parisc/video/fbdev.c                          |    2 +-
> >  drivers/Makefile                                   |    3 +-
> >  drivers/firmware/sysfb.c                           |   14 +-
> >  drivers/gpu/drm/hyperv/hyperv_drm_drv.c            |    8 +-
> >  drivers/video/backlight/Kconfig                    |    7 -
> >  drivers/video/backlight/Makefile                   |    1 -
> >  drivers/video/backlight/cr_bllcd.c                 |  264 ---
> >  drivers/video/fbdev/Kconfig                        |   72 -
> >  drivers/video/fbdev/Makefile                       |    2 -
> >  drivers/video/fbdev/amba-clcd.c                    |  986 ---------
> >  drivers/video/fbdev/core/fb_defio.c                |    8 +-
> >  drivers/video/fbdev/fsl-diu-fb.c                   |    2 +-
> >  drivers/video/fbdev/hgafb.c                        |   13 +-
> >  drivers/video/fbdev/hyperv_fb.c                    |   20 +-
> >  drivers/video/fbdev/imxfb.c                        |  179 +-
> >  drivers/video/fbdev/intelfb/Makefile               |    8 -
> >  drivers/video/fbdev/intelfb/intelfb.h              |  382 ----
> >  drivers/video/fbdev/intelfb/intelfb_i2c.c          |  209 --
> >  drivers/video/fbdev/intelfb/intelfbdrv.c           | 1680 ----------------
> >  drivers/video/fbdev/intelfb/intelfbhw.c            | 2115 --------------------
> >  drivers/video/fbdev/intelfb/intelfbhw.h            |  609 ------
> >  drivers/video/fbdev/mmp/hw/mmp_spi.c               |    2 +-
> >  drivers/video/fbdev/sis/sis_main.c                 |   37 -
> >  drivers/video/fbdev/stifb.c                        |  109 +-
> >  drivers/video/fbdev/vermilion/Makefile             |    6 -
> >  drivers/video/fbdev/vermilion/cr_pll.c             |  195 --
> >  drivers/video/fbdev/vermilion/vermilion.c          | 1175 -----------
> >  drivers/video/fbdev/vermilion/vermilion.h          |  245 ---
> >  drivers/video/logo/pnmtologo.c                     |    6 +-
> >  drivers/video/sticore.c                            |    5 +
> >  include/linux/amba/clcd-regs.h                     |   87 -
> >  include/linux/amba/clcd.h                          |  290 ---
> >  include/video/sticore.h                            |    6 +-
> >  37 files changed, 208 insertions(+), 8708 deletions(-)
> >  delete mode 100644 Documentation/fb/intelfb.rst
> >  delete mode 100644 drivers/video/backlight/cr_bllcd.c
> >  delete mode 100644 drivers/video/fbdev/amba-clcd.c
> >  delete mode 100644 drivers/video/fbdev/intelfb/Makefile
> >  delete mode 100644 drivers/video/fbdev/intelfb/intelfb.h
> >  delete mode 100644 drivers/video/fbdev/intelfb/intelfb_i2c.c
> >  delete mode 100644 drivers/video/fbdev/intelfb/intelfbdrv.c
> >  delete mode 100644 drivers/video/fbdev/intelfb/intelfbhw.c
> >  delete mode 100644 drivers/video/fbdev/intelfb/intelfbhw.h
> >  delete mode 100644 drivers/video/fbdev/vermilion/Makefile
> >  delete mode 100644 drivers/video/fbdev/vermilion/cr_pll.c
> >  delete mode 100644 drivers/video/fbdev/vermilion/vermilion.c
> >  delete mode 100644 drivers/video/fbdev/vermilion/vermilion.h
> >  delete mode 100644 include/linux/amba/clcd-regs.h
> >  delete mode 100644 include/linux/amba/clcd.h
> > marvin@defiant:~/linux/kernel/linux_torvalds$
> >
> > Hope this helps.
>
> P.S.
>
> As I see that this is a larger merge commit, with 5K+ lines changed, I don't think I can
> bisect further to determine the culprit.
>
> But I thought later that it would be interesting to see why my hardware triggered the freeze
> and probably others did not, or someone would complain already.
>
> Both of the boxes were AMD Ryzen: Ryzen 7 5700G and Ryzen 9 7950X.
>
> FWIW, I am attaching both hardware listings and the config used, so anyone knowledgeable with
> fbdev could possibly narrow down the search.
>

Hi Mirsad,

There is another report from Jens with similar symptom [1]. Can you check if
reverting df67699c9cb0ce ("firmware/sysfb: Clear screen_info state after
consuming it") fixes your regression?

Thanks.

[1]: https://lore.kernel.org/regressions/[email protected]/T/#t


--
An old man doll... just what I always wanted! - Clara


Attachments:
(No filename) (8.47 kB)
signature.asc (235.00 B)
Download all attachments

2024-01-20 18:15:29

by Mirsad Todorovac

[permalink] [raw]
Subject: Re: [BUG][BISECTED] Freeze at loading init ramdisk

On 1/20/24 12:25, Bagas Sanjaya wrote:
> On Wed, Jan 17, 2024 at 07:47:49PM +0100, Mirsad Todorovac wrote:
>> On 1/16/24 01:32, Mirsad Todorovac wrote:
>>> Hi,
>>>
>>> On the Ubuntu 22.04 LTS Jammy platform, on a mainline vanilla torvalds tree kernel, the boot
>>> freezes upon first two lines and before any systemd messages.
>>>
>>> (Please find the config attached.)
>>>
>>> Bisecting the bug led to this result:
>>>
>>> marvin@defiant:~/linux/kernel/linux_torvalds$ git bisect good
>>> d97a78423c33f68ca6543de510a409167baed6f5 is the first bad commit
>>> commit d97a78423c33f68ca6543de510a409167baed6f5
>>> Merge: 61da593f4458 689237ab37c5
>>> Author: Linus Torvalds <[email protected]>
>>> Date:   Fri Jan 12 14:38:08 2024 -0800
>>>
>>>     Merge tag 'fbdev-for-6.8-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/deller/linux-fbdev
>>>     Pull fbdev updates from Helge Deller:
>>>      "Three fbdev drivers (~8500 lines of code) removed. The Carillo Ranch
>>>       fbdev driver is for an Intel product which was never shipped, and for
>>>       the intelfb and the amba-clcd drivers the drm drivers can be used
>>>       instead.
>>>       The other code changes are minor: some fb_deferred_io flushing fixes,
>>>       imxfb margin fixes and stifb cleanups.
>>>       Summary:
>>>        - Remove intelfb fbdev driver (Thomas Zimmermann)
>>>        - Remove amba-clcd fbdev driver (Linus Walleij)
>>>        - Remove vmlfb Carillo Ranch fbdev driver (Matthew Wilcox)
>>>        - fb_deferred_io flushing fixes (Nam Cao)
>>>        - imxfb code fixes and cleanups (Dario Binacchi)
>>>        - stifb primary screen detection cleanups (Thomas Zimmermann)"
>>>     * tag 'fbdev-for-6.8-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/deller/linux-fbdev: (28 commits)
>>>       fbdev/intelfb: Remove driver
>>>       fbdev/hyperv_fb: Do not clear global screen_info
>>>       firmware/sysfb: Clear screen_info state after consuming it
>>>       fbdev/hyperv_fb: Remove firmware framebuffers with aperture helpers
>>>       drm/hyperv: Remove firmware framebuffers with aperture helper
>>>       fbdev/sis: Remove dependency on screen_info
>>>       video/logo: use %u format specifier for unsigned int values
>>>       video/sticore: Remove info field from STI struct
>>>       arch/parisc: Detect primary video device from device instance
>>>       fbdev/stifb: Allocate fb_info instance with framebuffer_alloc()
>>>       video/sticore: Store ROM device in STI struct
>>>       fbdev: flush deferred IO before closing
>>>       fbdev: flush deferred work in fb_deferred_io_fsync()
>>>       fbdev: amba-clcd: Delete the old CLCD driver
>>>       fbdev: Remove support for Carillo Ranch driver
>>>       fbdev: hgafb: fix kernel-doc comments
>>>       fbdev: mmp: Fix typo and wording in code comment
>>>       fbdev: fsl-diu-fb: Fix sparse warning due to virt_to_phys() prototype change
>>>       fbdev: imxfb: add '*/' on a separate line in block comment
>>>       fbdev: imxfb: use __func__ for function name
>>>       ...
>>>
>>>  Documentation/fb/index.rst                         |    1 -
>>>  Documentation/fb/intelfb.rst                       |  155 --
>>>  Documentation/userspace-api/ioctl/ioctl-number.rst |    1 -
>>>  MAINTAINERS                                        |   12 -
>>>  arch/parisc/video/fbdev.c                          |    2 +-
>>>  drivers/Makefile                                   |    3 +-
>>>  drivers/firmware/sysfb.c                           |   14 +-
>>>  drivers/gpu/drm/hyperv/hyperv_drm_drv.c            |    8 +-
>>>  drivers/video/backlight/Kconfig                    |    7 -
>>>  drivers/video/backlight/Makefile                   |    1 -
>>>  drivers/video/backlight/cr_bllcd.c                 |  264 ---
>>>  drivers/video/fbdev/Kconfig                        |   72 -
>>>  drivers/video/fbdev/Makefile                       |    2 -
>>>  drivers/video/fbdev/amba-clcd.c                    |  986 ---------
>>>  drivers/video/fbdev/core/fb_defio.c                |    8 +-
>>>  drivers/video/fbdev/fsl-diu-fb.c                   |    2 +-
>>>  drivers/video/fbdev/hgafb.c                        |   13 +-
>>>  drivers/video/fbdev/hyperv_fb.c                    |   20 +-
>>>  drivers/video/fbdev/imxfb.c                        |  179 +-
>>>  drivers/video/fbdev/intelfb/Makefile               |    8 -
>>>  drivers/video/fbdev/intelfb/intelfb.h              |  382 ----
>>>  drivers/video/fbdev/intelfb/intelfb_i2c.c          |  209 --
>>>  drivers/video/fbdev/intelfb/intelfbdrv.c           | 1680 ----------------
>>>  drivers/video/fbdev/intelfb/intelfbhw.c            | 2115 --------------------
>>>  drivers/video/fbdev/intelfb/intelfbhw.h            |  609 ------
>>>  drivers/video/fbdev/mmp/hw/mmp_spi.c               |    2 +-
>>>  drivers/video/fbdev/sis/sis_main.c                 |   37 -
>>>  drivers/video/fbdev/stifb.c                        |  109 +-
>>>  drivers/video/fbdev/vermilion/Makefile             |    6 -
>>>  drivers/video/fbdev/vermilion/cr_pll.c             |  195 --
>>>  drivers/video/fbdev/vermilion/vermilion.c          | 1175 -----------
>>>  drivers/video/fbdev/vermilion/vermilion.h          |  245 ---
>>>  drivers/video/logo/pnmtologo.c                     |    6 +-
>>>  drivers/video/sticore.c                            |    5 +
>>>  include/linux/amba/clcd-regs.h                     |   87 -
>>>  include/linux/amba/clcd.h                          |  290 ---
>>>  include/video/sticore.h                            |    6 +-
>>>  37 files changed, 208 insertions(+), 8708 deletions(-)
>>>  delete mode 100644 Documentation/fb/intelfb.rst
>>>  delete mode 100644 drivers/video/backlight/cr_bllcd.c
>>>  delete mode 100644 drivers/video/fbdev/amba-clcd.c
>>>  delete mode 100644 drivers/video/fbdev/intelfb/Makefile
>>>  delete mode 100644 drivers/video/fbdev/intelfb/intelfb.h
>>>  delete mode 100644 drivers/video/fbdev/intelfb/intelfb_i2c.c
>>>  delete mode 100644 drivers/video/fbdev/intelfb/intelfbdrv.c
>>>  delete mode 100644 drivers/video/fbdev/intelfb/intelfbhw.c
>>>  delete mode 100644 drivers/video/fbdev/intelfb/intelfbhw.h
>>>  delete mode 100644 drivers/video/fbdev/vermilion/Makefile
>>>  delete mode 100644 drivers/video/fbdev/vermilion/cr_pll.c
>>>  delete mode 100644 drivers/video/fbdev/vermilion/vermilion.c
>>>  delete mode 100644 drivers/video/fbdev/vermilion/vermilion.h
>>>  delete mode 100644 include/linux/amba/clcd-regs.h
>>>  delete mode 100644 include/linux/amba/clcd.h
>>> marvin@defiant:~/linux/kernel/linux_torvalds$
>>>
>>> Hope this helps.
>>
>> P.S.
>>
>> As I see that this is a larger merge commit, with 5K+ lines changed, I don't think I can
>> bisect further to determine the culprit.
>>
>> But I thought later that it would be interesting to see why my hardware triggered the freeze
>> and probably others did not, or someone would complain already.
>>
>> Both of the boxes were AMD Ryzen: Ryzen 7 5700G and Ryzen 9 7950X.
>>
>> FWIW, I am attaching both hardware listings and the config used, so anyone knowledgeable with
>> fbdev could possibly narrow down the search.
>>
>
> Hi Mirsad,
>
> There is another report from Jens with similar symptom [1]. Can you check if
> reverting df67699c9cb0ce ("firmware/sysfb: Clear screen_info state after
> consuming it") fixes your regression?
>
> Thanks.
>
> [1]: https://lore.kernel.org/regressions/[email protected]/T/#t

Thanks, Bagas, I confirm that it is the same issue:

1991 git checkout d97a78423c33
1992 git revert df67699c9cb0ce
1993 make clean; make olddefconfig
1994 time nice make -j 36 bindeb-pkg |& tee ../err-6.8-mrg-1.log; date
1995 sudo apt-get -s install ../linux-image-6.7.0-bagas-vanilla-rvt-09751-g6b082430adc8_6.7.0-09751-g6b082430adc8-26_amd64.deb
1996 sudo apt-get -y install ../linux-image-6.7.0-bagas-vanilla-rvt-09751-g6b082430adc8_6.7.0-09751-g6b082430adc8-26_amd64.deb

Reverting df67699c9cb0ce fixed it.

I don't have a slightest idea what is the side effect of this commit.

Tested-by: Mirsad Goran Todorovac <[email protected]>

Best regards,
Mirsad

2024-01-22 10:22:02

by Uwe Kleine-König

[permalink] [raw]
Subject: Re: [BUG][BISECTED] Freeze at loading init ramdisk

On Thu, Jan 18, 2024 at 09:04:05PM +0100, Mirsad Todorovac wrote:
>
>
> On 1/18/24 08:45, Uwe Kleine-K?nig wrote:
> > Hello Mirsad,
> >
> > On Wed, Jan 17, 2024 at 07:47:49PM +0100, Mirsad Todorovac wrote:
> > > On 1/16/24 01:32, Mirsad Todorovac wrote:
> > > > On the Ubuntu 22.04 LTS Jammy platform, on a mainline vanilla torvalds tree kernel, the boot
> > > > freezes upon first two lines and before any systemd messages.
> > > >
> > > > (Please find the config attached.)
> > > >
> > > > Bisecting the bug led to this result:
> > > >
> > > > marvin@defiant:~/linux/kernel/linux_torvalds$ git bisect good
> > > > d97a78423c33f68ca6543de510a409167baed6f5 is the first bad commit
> > > > commit d97a78423c33f68ca6543de510a409167baed6f5
> > > > Merge: 61da593f4458 689237ab37c5
> > > > Author: Linus Torvalds <[email protected]>
> > > > Date:?? Fri Jan 12 14:38:08 2024 -0800
> > > >
> > > > [...]
> > > >
> > > > Hope this helps.
> > >
> > > P.S.
> > >
> > > As I see that this is a larger merge commit, with 5K+ lines changed, I don't think I can
> > > bisect further to determine the culprit.
> >
> > Actually it's not that hard. If a merge commit is the first bad commit
> > for a bisection, either the merge wasn't done correctly (less likely,
> > looking at d97a78423c33f68ca6543de510a409167baed6f5 I'd bet this isn't
> > the problem); or changes on different sides conflict or you did
> > something wrong during bisection.
> >
> > To rule out the third option, you can just retest d97a78423c33,
> > 61da593f4458 and 689237ab37c5. If d97a78423c33 is the only bad one, you
> > did it right.
>
> This was confirmed.
>
> > Then to further debug the second option you can find out the offending
> > commit on each side with a bisection as follows, here for the RHS (i.e.
> > 689237ab37c5):
> >
> > git bisect start 689237ab37c5 $(git merge-base 61da593f4458 689237ab37c5)
> >
> > and then in each bisection step do:
> >
> > git merge --no-commit 61da593f4458
> > test if the problem is present
> > git reset --hard
> > git bisect good/bad
> >
> > In this case you get merge conflicts in drivers/video/fbdev/amba-clcd.c
> > and drivers/video/fbdev/vermilion/vermilion.c. In the assumption that
> > you don't have these enabled in your .config, you can just ignore these.
> >
> > Side note: A problem during bisection can be that the .config changes
> > along the process. You should put your config into (say)
> > arch/x86/configs/lala_defconfig and do
> >
> > make lala_defconfig
> >
> > before building each step to prevent this.
>
> I must have done something wrong:
>
> marvin@defiant:~/linux/kernel/linux_torvalds$ git bisect log
> # bad: [689237ab37c59b9909bc9371d7fece3081683fba] fbdev/intelfb: Remove driver
> # good: [de927f6c0b07d9e698416c5b287c521b07694cac] Merge tag 's390-6.8-1' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux
> git bisect start '689237ab37c5' 'de927f6c0b07d9e698416c5b287c521b07694cac'
> # good: [d9f25b59ed85ae45801cf45fe17eb269b0ef3038] fbdev: Remove support for Carillo Ranch driver
> git bisect good d9f25b59ed85ae45801cf45fe17eb269b0ef3038
> # good: [e2e0b838a1849f92612a8305c09aaf31bf824350] video/sticore: Remove info field from STI struct
> git bisect good e2e0b838a1849f92612a8305c09aaf31bf824350
> # good: [778e73d2411abc8f3a2d60dbf038acaec218792e] drm/hyperv: Remove firmware framebuffers with aperture helper
> git bisect good 778e73d2411abc8f3a2d60dbf038acaec218792e
> # good: [df67699c9cb0ceb70f6cc60630ca938c06773eda] firmware/sysfb: Clear screen_info state after consuming it
> git bisect good df67699c9cb0ceb70f6cc60630ca938c06773eda

FTR: Now that you identified df67699c9cb0ce as the culprit, calling
git bisect good on it was wrong, so something was fishy in your testing
and it's no surprise the bisection found a wrong result.

Best regards
Uwe

--
Pengutronix e.K. | Uwe Kleine-K?nig |
Industrial Linux Solutions | https://www.pengutronix.de/ |


Attachments:
(No filename) (4.00 kB)
signature.asc (499.00 B)
Download all attachments

2024-01-22 22:07:14

by Mirsad Todorovac

[permalink] [raw]
Subject: Re: [BUG][BISECTED] Freeze at loading init ramdisk



On 1/22/24 11:20, Uwe Kleine-König wrote:
> On Thu, Jan 18, 2024 at 09:04:05PM +0100, Mirsad Todorovac wrote:
>>
>>
>> On 1/18/24 08:45, Uwe Kleine-König wrote:
>>> Hello Mirsad,
>>>
>>> On Wed, Jan 17, 2024 at 07:47:49PM +0100, Mirsad Todorovac wrote:
>>>> On 1/16/24 01:32, Mirsad Todorovac wrote:
>>>>> On the Ubuntu 22.04 LTS Jammy platform, on a mainline vanilla torvalds tree kernel, the boot
>>>>> freezes upon first two lines and before any systemd messages.
>>>>>
>>>>> (Please find the config attached.)
>>>>>
>>>>> Bisecting the bug led to this result:
>>>>>
>>>>> marvin@defiant:~/linux/kernel/linux_torvalds$ git bisect good
>>>>> d97a78423c33f68ca6543de510a409167baed6f5 is the first bad commit
>>>>> commit d97a78423c33f68ca6543de510a409167baed6f5
>>>>> Merge: 61da593f4458 689237ab37c5
>>>>> Author: Linus Torvalds <[email protected]>
>>>>> Date:   Fri Jan 12 14:38:08 2024 -0800
>>>>>
>>>>> [...]
>>>>>
>>>>> Hope this helps.
>>>>
>>>> P.S.
>>>>
>>>> As I see that this is a larger merge commit, with 5K+ lines changed, I don't think I can
>>>> bisect further to determine the culprit.
>>>
>>> Actually it's not that hard. If a merge commit is the first bad commit
>>> for a bisection, either the merge wasn't done correctly (less likely,
>>> looking at d97a78423c33f68ca6543de510a409167baed6f5 I'd bet this isn't
>>> the problem); or changes on different sides conflict or you did
>>> something wrong during bisection.
>>>
>>> To rule out the third option, you can just retest d97a78423c33,
>>> 61da593f4458 and 689237ab37c5. If d97a78423c33 is the only bad one, you
>>> did it right.
>>
>> This was confirmed.
>>
>>> Then to further debug the second option you can find out the offending
>>> commit on each side with a bisection as follows, here for the RHS (i.e.
>>> 689237ab37c5):
>>>
>>> git bisect start 689237ab37c5 $(git merge-base 61da593f4458 689237ab37c5)
>>>
>>> and then in each bisection step do:
>>>
>>> git merge --no-commit 61da593f4458
>>> test if the problem is present
>>> git reset --hard
>>> git bisect good/bad
>>>
>>> In this case you get merge conflicts in drivers/video/fbdev/amba-clcd.c
>>> and drivers/video/fbdev/vermilion/vermilion.c. In the assumption that
>>> you don't have these enabled in your .config, you can just ignore these.
>>>
>>> Side note: A problem during bisection can be that the .config changes
>>> along the process. You should put your config into (say)
>>> arch/x86/configs/lala_defconfig and do
>>>
>>> make lala_defconfig
>>>
>>> before building each step to prevent this.
>>
>> I must have done something wrong:
>>
>> marvin@defiant:~/linux/kernel/linux_torvalds$ git bisect log
>> # bad: [689237ab37c59b9909bc9371d7fece3081683fba] fbdev/intelfb: Remove driver
>> # good: [de927f6c0b07d9e698416c5b287c521b07694cac] Merge tag 's390-6.8-1' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux
>> git bisect start '689237ab37c5' 'de927f6c0b07d9e698416c5b287c521b07694cac'
>> # good: [d9f25b59ed85ae45801cf45fe17eb269b0ef3038] fbdev: Remove support for Carillo Ranch driver
>> git bisect good d9f25b59ed85ae45801cf45fe17eb269b0ef3038
>> # good: [e2e0b838a1849f92612a8305c09aaf31bf824350] video/sticore: Remove info field from STI struct
>> git bisect good e2e0b838a1849f92612a8305c09aaf31bf824350
>> # good: [778e73d2411abc8f3a2d60dbf038acaec218792e] drm/hyperv: Remove firmware framebuffers with aperture helper
>> git bisect good 778e73d2411abc8f3a2d60dbf038acaec218792e
>> # good: [df67699c9cb0ceb70f6cc60630ca938c06773eda] firmware/sysfb: Clear screen_info state after consuming it
>> git bisect good df67699c9cb0ceb70f6cc60630ca938c06773eda
>
> FTR: Now that you identified df67699c9cb0ce as the culprit, calling
> git bisect good on it was wrong, so something was fishy in your testing
> and it's no surprise the bisection found a wrong result.

Copy that. But it is my first attempt on a bisect of a merge commit, so I will simply
ask to be forgiven. Maybe I forgot "git reset --hard" in some step.

I have to do a thorough homework on the merge commit magic.

Best regards,
Mirsad


> Best regards
> Uwe
>