Hi
Recently I've got access to ARM Juno R1 board and did some tests with
current mainline kernel on it. I'm a bit surprised that enabling
CONFIG_PROVE_LOCKING causes a boot failure on this board. After enabling
this Kconfig option, I get no single message from the kernel, although I
have earlycon enabled.
I've did my test with default defconfig and current linux-next,
v5.4-rc1, v5.3 and v4.19. In all cases the result is the same. I'm
booting kernel using a precompiled uboot from Linaro release and TFTP
download.
Is this a known issue? Other ARM64 boards I have access to (Samsung TM2e
and RaspberryPi3) boots fine with the same kernel image.
Best regards
--
Marek Szyprowski, PhD
Samsung R&D Institute Poland
Hi Marek,
On Fri, Oct 11, 2019 at 11:26:04AM +0200, Marek Szyprowski wrote:
> Hi
>
> Recently I've got access to ARM Juno R1 board and did some tests with
> current mainline kernel on it. I'm a bit surprised that enabling
> CONFIG_PROVE_LOCKING causes a boot failure on this board. After enabling
> this Kconfig option, I get no single message from the kernel, although I
> have earlycon enabled.
>
I don't have Juno R1 but I tried defconfig + CONFIG_PROVE_LOCKING and
it boots fine.
So if you disable CONFIG_PROVE_LOCKING(i.e. defconfig) boots fine ?
Are you using DTB from the mainline ?
> I've did my test with default defconfig and current linux-next,
> v5.4-rc1, v5.3 and v4.19. In all cases the result is the same. I'm
> booting kernel using a precompiled uboot from Linaro release and TFTP
> download.
>
OK, I use UEFI+GRUB but I don't think that should cause any issue.
> Is this a known issue? Other ARM64 boards I have access to (Samsung TM2e
> and RaspberryPi3) boots fine with the same kernel image.
>
Not that I am aware of. If you could send me the bootlog with defconfig
I can take a look and see if I get any clue.
--
Regards,
Sudeep
Hi Sudeep,
On 11.10.2019 12:05, Sudeep Holla wrote:
> Hi Marek,
>
> On Fri, Oct 11, 2019 at 11:26:04AM +0200, Marek Szyprowski wrote:
>> Hi
>>
>> Recently I've got access to ARM Juno R1 board and did some tests with
>> current mainline kernel on it. I'm a bit surprised that enabling
>> CONFIG_PROVE_LOCKING causes a boot failure on this board. After enabling
>> this Kconfig option, I get no single message from the kernel, although I
>> have earlycon enabled.
>>
> I don't have Juno R1 but I tried defconfig + CONFIG_PROVE_LOCKING and
> it boots fine.
>
> So if you disable CONFIG_PROVE_LOCKING(i.e. defconfig) boots fine ?
> Are you using DTB from the mainline ?
Yes, ARM Juno r1 boots fine with pure defconfig and mainline dtb.
However a few minutes ago I found that it boots with
v4.14+PROVE_LOCKING, so I will bisect it and share the results.
>> I've did my test with default defconfig and current linux-next,
>> v5.4-rc1, v5.3 and v4.19. In all cases the result is the same. I'm
>> booting kernel using a precompiled uboot from Linaro release and TFTP
>> download.
>>
> OK, I use UEFI+GRUB but I don't think that should cause any issue.
>
>> Is this a known issue? Other ARM64 boards I have access to (Samsung TM2e
>> and RaspberryPi3) boots fine with the same kernel image.
>>
> Not that I am aware of. If you could send me the bootlog with defconfig
> I can take a look and see if I get any clue.
>
Best regards
--
Marek Szyprowski, PhD
Samsung R&D Institute Poland
Hi guys,
On 11/10/2019 11:05, Sudeep Holla wrote:
> On Fri, Oct 11, 2019 at 11:26:04AM +0200, Marek Szyprowski wrote:
>> Recently I've got access to ARM Juno R1 board and did some tests with
>> current mainline kernel on it. I'm a bit surprised that enabling
>> CONFIG_PROVE_LOCKING causes a boot failure on this board. After enabling
>> this Kconfig option, I get no single message from the kernel, although I
>> have earlycon enabled.
> I don't have Juno R1 but I tried defconfig + CONFIG_PROVE_LOCKING and
> it boots fine.
I just tried this on my r1, v5.4-rc1 with this configuration worked just fine.
My cmdline is:
| root=/dev/sda6 loglevel=9 earlycon=pl011,0x7ff80000 hugepagesz=2M hugepages=512
| crashkernel=1G console=ttyAMA0 resume=/dev/sda2 no_console_suspend efi=debug
>> I've did my test with default defconfig and current linux-next,
>> v5.4-rc1, v5.3 and v4.19. In all cases the result is the same. I'm
>> booting kernel using a precompiled uboot from Linaro release and TFTP
>> download.
> OK, I use UEFI+GRUB but I don't think that should cause any issue.
... same ... this uboot binary looks like the main difference.
Is it using u-boots UEFI support? Is it possible to turn that off?
It may that lockdep is just perturbing the size of the binary. It adds an extra 4MB for me.
Thanks,
James
On Fri, Oct 11, 2019 at 11:38:17AM +0100, James Morse wrote:
> Hi guys,
>
> On 11/10/2019 11:05, Sudeep Holla wrote:
> > On Fri, Oct 11, 2019 at 11:26:04AM +0200, Marek Szyprowski wrote:
> >> Recently I've got access to ARM Juno R1 board and did some tests with
> >> current mainline kernel on it. I'm a bit surprised that enabling
> >> CONFIG_PROVE_LOCKING causes a boot failure on this board. After enabling
> >> this Kconfig option, I get no single message from the kernel, although I
> >> have earlycon enabled.
>
> > I don't have Juno R1 but I tried defconfig + CONFIG_PROVE_LOCKING and
> > it boots fine.
>
> I just tried this on my r1, v5.4-rc1 with this configuration worked just fine.
>
> My cmdline is:
> | root=/dev/sda6 loglevel=9 earlycon=pl011,0x7ff80000 hugepagesz=2M hugepages=512
> | crashkernel=1G console=ttyAMA0 resume=/dev/sda2 no_console_suspend efi=debug
>
>
> >> I've did my test with default defconfig and current linux-next,
> >> v5.4-rc1, v5.3 and v4.19. In all cases the result is the same. I'm
> >> booting kernel using a precompiled uboot from Linaro release and TFTP
> >> download.
>
> > OK, I use UEFI+GRUB but I don't think that should cause any issue.
>
> ... same ... this uboot binary looks like the main difference.
> Is it using u-boots UEFI support? Is it possible to turn that off?
>
Did give a quick try with mainline uboot on my Juno R2 and it boots fine.
Not sure if EFI support is default there. I am using vexpress_aemv8a_juno_defconfig
> It may that lockdep is just perturbing the size of the binary. It adds an
> extra 4MB for me.
The image size was 35MB.
I was thinking if it has some wrongly configured firmware, but since
defconfig works, it must have sane firmware.
--
Regards,
Sudeep
Hi James,
On 11.10.2019 12:38, James Morse wrote:
> Hi guys,
>
> On 11/10/2019 11:05, Sudeep Holla wrote:
>> On Fri, Oct 11, 2019 at 11:26:04AM +0200, Marek Szyprowski wrote:
>>> Recently I've got access to ARM Juno R1 board and did some tests with
>>> current mainline kernel on it. I'm a bit surprised that enabling
>>> CONFIG_PROVE_LOCKING causes a boot failure on this board. After enabling
>>> this Kconfig option, I get no single message from the kernel, although I
>>> have earlycon enabled.
>> I don't have Juno R1 but I tried defconfig + CONFIG_PROVE_LOCKING and
>> it boots fine.
> I just tried this on my r1, v5.4-rc1 with this configuration worked just fine.
>
> My cmdline is:
> | root=/dev/sda6 loglevel=9 earlycon=pl011,0x7ff80000 hugepagesz=2M hugepages=512
> | crashkernel=1G console=ttyAMA0 resume=/dev/sda2 no_console_suspend efi=debug
>
That is a bit strange. Here is a boot log from v5.4-rc1 with pure
defconfig: https://paste.debian.net/1105851/
The bisect lead me to the commit
c3bc8fd637a9623f5c507bd18f9677effbddf584 ("tracing: Centralize
preemptirq tracepoints and unify their usage"), which appeared in
v4.19-rc1. It cannot be easily reverted, but kernel built from earlier
versions boots fine here with PROVE_LOCKING enabled. I wonder what I do
in a different way than You...
>>> I've did my test with default defconfig and current linux-next,
>>> v5.4-rc1, v5.3 and v4.19. In all cases the result is the same. I'm
>>> booting kernel using a precompiled uboot from Linaro release and TFTP
>>> download.
>> OK, I use UEFI+GRUB but I don't think that should cause any issue.
> ... same ... this uboot binary looks like the main difference.
> Is it using u-boots UEFI support? Is it possible to turn that off?
>
> It may that lockdep is just perturbing the size of the binary. It adds an extra 4MB for me.
The size of the kernel binary doesn't matter. I've successfully booted
larger images, than the once compiled with PROVE_LOCKING enabled.
Best regards
--
Marek Szyprowski, PhD
Samsung R&D Institute Poland
On Fri, Oct 11, 2019 at 03:02:42PM +0200, Marek Szyprowski wrote:
> Hi James,
>
> On 11.10.2019 12:38, James Morse wrote:
> > Hi guys,
> >
> > On 11/10/2019 11:05, Sudeep Holla wrote:
> >> On Fri, Oct 11, 2019 at 11:26:04AM +0200, Marek Szyprowski wrote:
> >>> Recently I've got access to ARM Juno R1 board and did some tests with
> >>> current mainline kernel on it. I'm a bit surprised that enabling
> >>> CONFIG_PROVE_LOCKING causes a boot failure on this board. After enabling
> >>> this Kconfig option, I get no single message from the kernel, although I
> >>> have earlycon enabled.
> >> I don't have Juno R1 but I tried defconfig + CONFIG_PROVE_LOCKING and
> >> it boots fine.
> > I just tried this on my r1, v5.4-rc1 with this configuration worked just fine.
> >
> > My cmdline is:
> > | root=/dev/sda6 loglevel=9 earlycon=pl011,0x7ff80000 hugepagesz=2M hugepages=512
> > | crashkernel=1G console=ttyAMA0 resume=/dev/sda2 no_console_suspend efi=debug
> >
> That is a bit strange. Here is a boot log from v5.4-rc1 with pure
> defconfig: https://paste.debian.net/1105851/
>
I see from the boot log that both Image.gz and dtb being loaded at the
same address 0x82000000, will u-boot uncompress it elsewhere after loading
it ? Just for my understanding.
--
Regards,
Sudeep
Hi Sudeep
On 11.10.2019 15:10, Sudeep Holla wrote:
> On Fri, Oct 11, 2019 at 03:02:42PM +0200, Marek Szyprowski wrote:
>> Hi James,
>>
>> On 11.10.2019 12:38, James Morse wrote:
>>> Hi guys,
>>>
>>> On 11/10/2019 11:05, Sudeep Holla wrote:
>>>> On Fri, Oct 11, 2019 at 11:26:04AM +0200, Marek Szyprowski wrote:
>>>>> Recently I've got access to ARM Juno R1 board and did some tests with
>>>>> current mainline kernel on it. I'm a bit surprised that enabling
>>>>> CONFIG_PROVE_LOCKING causes a boot failure on this board. After enabling
>>>>> this Kconfig option, I get no single message from the kernel, although I
>>>>> have earlycon enabled.
>>>> I don't have Juno R1 but I tried defconfig + CONFIG_PROVE_LOCKING and
>>>> it boots fine.
>>> I just tried this on my r1, v5.4-rc1 with this configuration worked just fine.
>>>
>>> My cmdline is:
>>> | root=/dev/sda6 loglevel=9 earlycon=pl011,0x7ff80000 hugepagesz=2M hugepages=512
>>> | crashkernel=1G console=ttyAMA0 resume=/dev/sda2 no_console_suspend efi=debug
>>>
>> That is a bit strange. Here is a boot log from v5.4-rc1 with pure
>> defconfig: https://paste.debian.net/1105851/
>>
> I see from the boot log that both Image.gz and dtb being loaded at the
> same address 0x82000000, will u-boot uncompress it elsewhere after loading
> it ? Just for my understanding.
tftp downloads Image.gz to 0x82000000, then decompress it to
$kernel_addr to save transfer time
my bootcmd is:
tftp ${fdt_addr} juno/Image.gz; unzip ${fdt_addr} ${kernel_addr}; tftp
${fdt_addr} juno/juno-r1.dtb; booti ${kernel_addr} - ${fdt_addr};
Best regards
--
Marek Szyprowski, PhD
Samsung R&D Institute Poland
On Fri, Oct 11, 2019 at 03:15:32PM +0200, Marek Szyprowski wrote:
> Hi Sudeep
>
> On 11.10.2019 15:10, Sudeep Holla wrote:
> > On Fri, Oct 11, 2019 at 03:02:42PM +0200, Marek Szyprowski wrote:
> >> Hi James,
> >>
> >> On 11.10.2019 12:38, James Morse wrote:
> >>> Hi guys,
> >>>
> >>> On 11/10/2019 11:05, Sudeep Holla wrote:
> >>>> On Fri, Oct 11, 2019 at 11:26:04AM +0200, Marek Szyprowski wrote:
> >>>>> Recently I've got access to ARM Juno R1 board and did some tests with
> >>>>> current mainline kernel on it. I'm a bit surprised that enabling
> >>>>> CONFIG_PROVE_LOCKING causes a boot failure on this board. After enabling
> >>>>> this Kconfig option, I get no single message from the kernel, although I
> >>>>> have earlycon enabled.
> >>>> I don't have Juno R1 but I tried defconfig + CONFIG_PROVE_LOCKING and
> >>>> it boots fine.
> >>> I just tried this on my r1, v5.4-rc1 with this configuration worked just fine.
> >>>
> >>> My cmdline is:
> >>> | root=/dev/sda6 loglevel=9 earlycon=pl011,0x7ff80000 hugepagesz=2M hugepages=512
> >>> | crashkernel=1G console=ttyAMA0 resume=/dev/sda2 no_console_suspend efi=debug
> >>>
> >> That is a bit strange. Here is a boot log from v5.4-rc1 with pure
> >> defconfig: https://paste.debian.net/1105851/
> >>
> > I see from the boot log that both Image.gz and dtb being loaded at the
> > same address 0x82000000, will u-boot uncompress it elsewhere after loading
> > it ? Just for my understanding.
>
> tftp downloads Image.gz to 0x82000000, then decompress it to
> $kernel_addr to save transfer time
>
> my bootcmd is:
>
> tftp ${fdt_addr} juno/Image.gz; unzip ${fdt_addr} ${kernel_addr}; tftp
> ${fdt_addr} juno/juno-r1.dtb; booti ${kernel_addr} - ${fdt_addr};
>
Thanks for the info. I got hold of R1 board(not the one James has ;))
and it works fine on that too.
Further, with reference to the commit you mentioned make sure defconfig
works with that as it's a commit in the middle of merge window.
I am using gcc7 and I noticed yours is gcc6 not sure if that makes any
difference. Just listing the differences.
I will see if I can grab the exact linaro binaries.
--
Regards,
Sudeep
On Fri, Oct 11, 2019 at 02:43:54PM +0100, Sudeep Holla wrote:
> On Fri, Oct 11, 2019 at 03:15:32PM +0200, Marek Szyprowski wrote:
> > Hi Sudeep
> >
> > On 11.10.2019 15:10, Sudeep Holla wrote:
> > > On Fri, Oct 11, 2019 at 03:02:42PM +0200, Marek Szyprowski wrote:
> > >> Hi James,
> > >>
> > >> On 11.10.2019 12:38, James Morse wrote:
> > >>> Hi guys,
> > >>>
> > >>> On 11/10/2019 11:05, Sudeep Holla wrote:
> > >>>> On Fri, Oct 11, 2019 at 11:26:04AM +0200, Marek Szyprowski wrote:
> > >>>>> Recently I've got access to ARM Juno R1 board and did some tests with
> > >>>>> current mainline kernel on it. I'm a bit surprised that enabling
> > >>>>> CONFIG_PROVE_LOCKING causes a boot failure on this board. After enabling
> > >>>>> this Kconfig option, I get no single message from the kernel, although I
> > >>>>> have earlycon enabled.
> > >>>> I don't have Juno R1 but I tried defconfig + CONFIG_PROVE_LOCKING and
> > >>>> it boots fine.
> > >>> I just tried this on my r1, v5.4-rc1 with this configuration worked just fine.
> > >>>
> > >>> My cmdline is:
> > >>> | root=/dev/sda6 loglevel=9 earlycon=pl011,0x7ff80000 hugepagesz=2M hugepages=512
> > >>> | crashkernel=1G console=ttyAMA0 resume=/dev/sda2 no_console_suspend efi=debug
> > >>>
> > >> That is a bit strange. Here is a boot log from v5.4-rc1 with pure
> > >> defconfig: https://paste.debian.net/1105851/
> > >>
> > > I see from the boot log that both Image.gz and dtb being loaded at the
> > > same address 0x82000000, will u-boot uncompress it elsewhere after loading
> > > it ? Just for my understanding.
> >
> > tftp downloads Image.gz to 0x82000000, then decompress it to
> > $kernel_addr to save transfer time
> >
> > my bootcmd is:
> >
> > tftp ${fdt_addr} juno/Image.gz; unzip ${fdt_addr} ${kernel_addr}; tftp
> > ${fdt_addr} juno/juno-r1.dtb; booti ${kernel_addr} - ${fdt_addr};
> >
If your ${kernel_addr}=0x80000000 or within first 32MB, then it will override
DTB with the image size I had(35MB). Even if kernel fits 32MB, there is a
chance that .bss lies beyond 32MB and it will be cleared during boot resulting
in DTB corruption(Andre P reminded me this)
Can you try setting $${fdt_addr} to 0x84000000 to begin with ?
--
Regards,
Sudeep
Hi Sudeep,
On 11.10.2019 16:42, Sudeep Holla wrote:
> On Fri, Oct 11, 2019 at 02:43:54PM +0100, Sudeep Holla wrote:
>> On Fri, Oct 11, 2019 at 03:15:32PM +0200, Marek Szyprowski wrote:
>>> Hi Sudeep
>>>
>>> On 11.10.2019 15:10, Sudeep Holla wrote:
>>>> On Fri, Oct 11, 2019 at 03:02:42PM +0200, Marek Szyprowski wrote:
>>>>> Hi James,
>>>>>
>>>>> On 11.10.2019 12:38, James Morse wrote:
>>>>>> Hi guys,
>>>>>>
>>>>>> On 11/10/2019 11:05, Sudeep Holla wrote:
>>>>>>> On Fri, Oct 11, 2019 at 11:26:04AM +0200, Marek Szyprowski wrote:
>>>>>>>> Recently I've got access to ARM Juno R1 board and did some tests with
>>>>>>>> current mainline kernel on it. I'm a bit surprised that enabling
>>>>>>>> CONFIG_PROVE_LOCKING causes a boot failure on this board. After enabling
>>>>>>>> this Kconfig option, I get no single message from the kernel, although I
>>>>>>>> have earlycon enabled.
>>>>>>> I don't have Juno R1 but I tried defconfig + CONFIG_PROVE_LOCKING and
>>>>>>> it boots fine.
>>>>>> I just tried this on my r1, v5.4-rc1 with this configuration worked just fine.
>>>>>>
>>>>>> My cmdline is:
>>>>>> | root=/dev/sda6 loglevel=9 earlycon=pl011,0x7ff80000 hugepagesz=2M hugepages=512
>>>>>> | crashkernel=1G console=ttyAMA0 resume=/dev/sda2 no_console_suspend efi=debug
>>>>>>
>>>>> That is a bit strange. Here is a boot log from v5.4-rc1 with pure
>>>>> defconfig: https://paste.debian.net/1105851/
>>>>>
>>>> I see from the boot log that both Image.gz and dtb being loaded at the
>>>> same address 0x82000000, will u-boot uncompress it elsewhere after loading
>>>> it ? Just for my understanding.
>>> tftp downloads Image.gz to 0x82000000, then decompress it to
>>> $kernel_addr to save transfer time
>>>
>>> my bootcmd is:
>>>
>>> tftp ${fdt_addr} juno/Image.gz; unzip ${fdt_addr} ${kernel_addr}; tftp
>>> ${fdt_addr} juno/juno-r1.dtb; booti ${kernel_addr} - ${fdt_addr};
>>>
> If your ${kernel_addr}=0x80000000 or within first 32MB, then it will override
> DTB with the image size I had(35MB). Even if kernel fits 32MB, there is a
> chance that .bss lies beyond 32MB and it will be cleared during boot resulting
> in DTB corruption(Andre P reminded me this)
>
> Can you try setting $${fdt_addr} to 0x84000000 to begin with ?
Right, my fault. Changing fdt_addr to something higher than the default
0x82000000 fixed the boot issue. I wonder how I missed that, as I was
convinced that there is enough space for the kernel image. Sorry for the
noise...
Best regards
--
Marek Szyprowski, PhD
Samsung R&D Institute Poland
Hi Marek,
On 14/10/2019 10:02, Marek Szyprowski wrote:
> On 11.10.2019 16:42, Sudeep Holla wrote:
>> On Fri, Oct 11, 2019 at 02:43:54PM +0100, Sudeep Holla wrote:
>>> On Fri, Oct 11, 2019 at 03:15:32PM +0200, Marek Szyprowski wrote:
>>>> On 11.10.2019 15:10, Sudeep Holla wrote:
>>>>> On Fri, Oct 11, 2019 at 03:02:42PM +0200, Marek Szyprowski wrote:
>>>>>> On 11.10.2019 12:38, James Morse wrote:
>>>>>>> On 11/10/2019 11:05, Sudeep Holla wrote:
>>>>>>>> On Fri, Oct 11, 2019 at 11:26:04AM +0200, Marek Szyprowski wrote:
>>>>>>>>> Recently I've got access to ARM Juno R1 board and did some tests with
>>>>>>>>> current mainline kernel on it. I'm a bit surprised that enabling
>>>>>>>>> CONFIG_PROVE_LOCKING causes a boot failure on this board. After enabling
>>>>>>>>> this Kconfig option, I get no single message from the kernel, although I
>>>>>>>>> have earlycon enabled.
>>>> my bootcmd is:
>>>>
>>>> tftp ${fdt_addr} juno/Image.gz; unzip ${fdt_addr} ${kernel_addr}; tftp
>>>> ${fdt_addr} juno/juno-r1.dtb; booti ${kernel_addr} - ${fdt_addr};
>>>>
>> If your ${kernel_addr}=0x80000000 or within first 32MB, then it will override
>> DTB with the image size I had(35MB). Even if kernel fits 32MB, there is a
>> chance that .bss lies beyond 32MB and it will be cleared during boot resulting
>> in DTB corruption(Andre P reminded me this)
>>
>> Can you try setting $${fdt_addr} to 0x84000000 to begin with ?
>
> Right, my fault. Changing fdt_addr to something higher than the default
> 0x82000000 fixed the boot issue. I wonder how I missed that, as I was
> convinced that there is enough space for the kernel image. Sorry for the
> noise...
Is it possible for uboot's booti command to print a warning in this case?
The size of the BSS is in the header as the 'effective size' of the kernel'.
(it must have taken a while to bisect this, and it just happened to pick a believable
commit that modified start_kernel()...)
Thanks,
James