Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Subject: Re: RPI4: fail too boot with an initrd
To:     Matthias Brugger <mbrugger@suse.com>,
        LABBE Corentin <clabbe@baylibre.com>,
        Matthias Brugger <matthias.bgg@gmail.com>
Cc:     linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org,
        u-boot@lists.denx.de, James Morse <james.morse@arm.com>,
        linux-rpi-kernel@lists.infradead.org,
        bcm-kernel-feedback-list@broadcom.com
From:   Jaehoon Chung <jh80.chung@samsung.com>
Message-ID: <901d17c4-8265-a2a2-313b-cd9aeb7ecd63@samsung.com>
Date:   Tue, 18 Feb 2020 20:08:30 +0900
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
        Thunderbird/60.9.0
MIME-Version: 1.0
In-Reply-To: <d9727fac-e7fd-7dfb-ce39-da0c5b0d12a2@suse.com>
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="utf-8"
CMS-TYPE: 101P
DLP-Filter: Pass
References: <20200214132748.GA23276@Red>
        <b726290c-1038-3771-5187-6ac370bc92c9@arm.com> <20200217103733.GA11379@Red>
        <c5a959d7-44b2-fab9-8269-d8e858790925@gmail.com>
        <20200217125301.GA31847@Red>
        <982c710c-d606-2c04-03ee-604626230bbc@gmail.com>
        <20200218100128.GB10369@Red>
        <CGME20200218103651epcas1p4b6a9d599ec3577f1407ebc5b781c127f@epcas1p4.samsung.com>
        <d9727fac-e7fd-7dfb-ce39-da0c5b0d12a2@suse.com>
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On 2/18/20 7:36 PM, Matthias Brugger wrote:
> 
> 
> On 18/02/2020 11:01, LABBE Corentin wrote:
>> On Mon, Feb 17, 2020 at 02:07:09PM +0100, Matthias Brugger wrote:
>>>
>>>
>>> On 17/02/2020 13:53, LABBE Corentin wrote:
>>>> On Mon, Feb 17, 2020 at 11:50:04AM +0100, Matthias Brugger wrote:
>>>>>
>>>>>
>>>>> On 17/02/2020 11:37, LABBE Corentin wrote:
>>>>>> On Fri, Feb 14, 2020 at 06:15:27PM +0000, James Morse wrote:
>>>>>>> Hi Corentin,
>>>>>>>
>>>>>>> On 14/02/2020 13:27, LABBE Corentin wrote:
>>>>>>>> Since the inclusion of the "enable network support in RPi4 config" serie on uboot, I
>>>>>>>> have started to work on adding the rpi4 in kernelCI.
>>>>>>>> But I fail to succeed in using a kernel/dtb/ramdisk downloaded via tftp.
>>>>>>>>
>>>>>>>> Using booti I hit:
>>>>>>>> [    0.000000] Linux version 5.6.0-rc1-next-20200212 (clabbe@build2-bionic-1804) (gcc version 7.4.1 20181213 [linaro-7.4-2019.02 revision 56ec6f6b99cc167ff0c2f8e1a2eed33b1edc85d4] (Linaro    GCC 7.4-2019.02)) #66 SMP PREEMPT Wed Feb 12 10:14:20 UTC 2020
>>>>>>>> [    0.000000] Machine model: Raspberry Pi 4 Model B
>>>>>>>> [    0.000000] earlycon: uart0 at MMIO32 0x00000000fe215040 (options '')
>>>>>>>> [    0.000000] printk: bootconsole [uart0] enabled
>>>>>>>> [    0.000000] efi: Getting EFI parameters from FDT:
>>>>>>>> [    0.000000] efi: UEFI not found.
>>>>>>>
>>>>>>> So no EFI,
>>>>>>>
>>>>>>>> [    0.000000] OF: reserved mem: failed to allocate memory for node 'linux,cma'
>>>>>>>
>>>>>>> Out of memory.
>>>>>>>
>>>>>>>> [    0.000000] cma: Failed to reserve 32 MiB
>>>>>>>> [    0.000000] Kernel panic - not syncing: Failed to allocate page table page
>>>>>>>
>>>>>>> Out of memory...
>>>>>>>
>>>>>>>> [    0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 5.6.0-rc1-next-20200212 #66
>>>>>>>> [    0.000000] Hardware name: Raspberry Pi 4 Model B (DT)
>>>>>>>> [    0.000000] Call trace:
>>>>>>>> [    0.000000]  dump_backtrace+0x0/0x1a0
>>>>>>>> [    0.000000]  show_stack+0x14/0x20
>>>>>>>> [    0.000000]  dump_stack+0xbc/0x104
>>>>>>>> [    0.000000]  panic+0x16c/0x37c
>>>>>>>> [    0.000000]  early_pgtable_alloc+0x30/0xa0
>>>>>>>
>>>>>>> ... really early!
>>>>>>>
>>>>>>>> [    0.000000]  __create_pgd_mapping+0x36c/0x588
>>>>>>>> [    0.000000]  map_kernel_segment+0x70/0xa4
>>>>>>>> [    0.000000]  paging_init+0xf4/0x528
>>>>>>>> [    0.000000]  setup_arch+0x250/0x5d8
>>>>>>>> [    0.000000]  start_kernel+0x90/0x6d8
>>>>>>>>
>>>>>>>>  
>>>>>>>> Since the same kernel boot with bootefi and that bootefi lack ramdisk address,
>>>>>>>
>>>>>>> Booting with EFI will cause linux to use the EFI memory map.
>>>>>>>
>>>>>>> Does your DT have a memory node? (or does it expect EFI to provide the information)
>>>>>>>
>>>>>>>
>>>>>>>> I tried to add the address in the dtb via:
>>>>>>>> fdt addr 0x02400000; fdt resize; fdt set /chosen linux,initrd-start 0x02700000; fdt set /chosen linux,initrd-end 0x10000000; bootefi 0x00080000 0x02400000
>>>>>>>> But with that, I get:
>>>>>>>> initrd not fully accessible via the linear mapping -- please check your bootloader ...
>>>>>>>
>>>>>>> So this one is an EFI boot, but you can't find where to put the initramfs such that the
>>>>>>> kernel agrees its in memory.
>>>>>>>
>>>>>>> If you boot with 'efi=debug', linux will print the EFI memory map. Could you compare that
>>>>>>> to where U-Boot thinks memory is?
>>>>>>>
>>>>>>> (it sounds like your DT memory node is missing, and your EFI memory map is surprisingly small)
>>>>>>
>>>>>> Hello
>>>>>>
>>>>>> Thanks for your advices.
>>>>>>
>>>>>> In the dtb of mainline linux:
>>>>>> 	/* Will be filled by the bootloader */
>>>>>> 	memory@0 {
>>>>>> 		device_type = "memory";
>>>>>> 		reg = <0 0 0>;
>>>>>> 	};
>>>>>>
>>>>>> In uboot I have:
>>>>>> static struct mm_region bcm2711_mem_map[] = {
>>>>>>         {
>>>>>>                 .virt = 0x00000000UL,
>>>>>>                 .phys = 0x00000000UL,
>>>>>>                 .size = 0xfe000000UL,
>>>>>>                 .attrs = PTE_BLOCK_MEMTYPE(MT_NORMAL) |
>>>>>>                          PTE_BLOCK_INNER_SHARE
>>>>>>         }, {
>>>>>>                 .virt = 0xfc000000UL,
>>>>>>                 .phys = 0xfc000000UL,
>>>>>>                 .size = 0x03800000UL,
>>>>>>                 .attrs = PTE_BLOCK_MEMTYPE(MT_DEVICE_NGNRNE) |
>>>>>>                          PTE_BLOCK_NON_SHARE |
>>>>>>                          PTE_BLOCK_PXN | PTE_BLOCK_UXN
>>>>>>         }, {
>>>>>>                 /* List terminator */
>>>>>>                 0,
>>>>>>         }
>>>>>> };
>>>>>> But I dont know if uboot use that for filling the memory node.
>>>>>
>>>>> No it doesn't. U-Boot uses the DT from the firmware and passes this to the
>>>>> kernel. But it seems you pass instead your own device-tree to the kernel, so you
>>>>> will need to update the memory node to show the available memory on you board.
>>>>>
>>>>
>>>> I dont understand, in the Linux commit "ARM: dts: Add minimal Raspberry Pi 4 support" I read:
>>>> The RPi 4 is available in 3 different variants (1, 2 and 4 GB RAM), so leave the memory size to zero and let the bootloader take care of it.
>>>> But if uboot dont fill that...
>>>> So the DTB in mainline is wrong, right ?
>>>>
>>>
>>> How do you pass your DTB to the kernel? Does the FW uses your DTB by putting it
>>> as bcm2711-rpi-4-b.dtb in the first FAT partition? Or do you load it from U-Boot
>>> afterwards? In the latter case you have to take care to add the needed size of
>>> memory. In the first case you use what is the default behavior for U-Boot; the
>>> RPi FW updates the node and then passes it to U-Boot then it get's passed to the
>>> kernel.
>>>
>>
>> Hello
>>
>> I load the dtb via uboot/tftp.
>> On another thread, I got the hint to enable CONFIG_ARCH_FIXUP_FDT_MEMORY and it made my problem solved.
> 
> Hm, so should we enable that for all RPi3/4/arm64 configs then?

In u-boot, it's already enabled, except RPI4. I didn't check in more detail.

In RPI4's case, it doesn't set in config
# CONFIG_ARCH_FIXUP_FDT_MEMORY is not set

Best Regards,
Jaehoon Chung

> 
> Regards,
> Matthias
> 
> 
>