Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <20201226163037.43691-1-vitaly.wool@konsulko.com>
 <CAAhSdy1M5pMjYHNWdOicb3N3fjTfQLEgE8tFb74sqGbPE_9eyQ@mail.gmail.com>
 <CAM4kBBJ3Vbytx=dFK7+DMByV3zK=FVLATSwjyuuygkDK1MCQjA@mail.gmail.com>
 <CAAhSdy0E8xspb3epBdrTmjg7vSQanG9zwyMY19eeDDL8z=WJFg@mail.gmail.com> <CAM4kBBJLnrp8XB1OZnv2FEuKdEQuQfDWdgwsV--9mZYB=JmJPQ@mail.gmail.com>
In-Reply-To: <CAM4kBBJLnrp8XB1OZnv2FEuKdEQuQfDWdgwsV--9mZYB=JmJPQ@mail.gmail.com>
From:   Anup Patel <anup@brainfault.org>
Date:   Tue, 29 Dec 2020 10:35:20 +0530
Message-ID: <CAAhSdy1VMujreaaWCDjO9BPa0P2As7c0Hh3riZsY3ncU3NnsGg@mail.gmail.com>
Subject: Re: [PATCH] riscv: add BUILTIN_DTB support for MMU-enabled targets
To:     Vitaly Wool <vitaly.wool@konsulko.com>
Cc:     linux-riscv <linux-riscv@lists.infradead.org>,
        "linux-kernel@vger.kernel.org List" <linux-kernel@vger.kernel.org>,
        Bin Meng <bin.meng@windriver.com>,
        Palmer Dabbelt <palmerdabbelt@google.com>,
        Damien Le Moal <damien.lemoal@wdc.com>,
        devicetree@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Mon, Dec 28, 2020 at 10:08 PM Vitaly Wool <vitaly.wool@konsulko.com> wrote:
>
> On Mon, Dec 28, 2020 at 3:10 PM Anup Patel <anup@brainfault.org> wrote:
> >
> > On Mon, Dec 28, 2020 at 7:05 PM Vitaly Wool <vitaly.wool@konsulko.com> wrote:
> > >
> > > On Mon, Dec 28, 2020 at 12:59 PM Anup Patel <anup@brainfault.org> wrote:
> > > >
> > > > On Sat, Dec 26, 2020 at 10:03 PM Vitaly Wool <vitaly.wool@konsulko.com> wrote:
> > > > >
> > > > > Sometimes, especially in a production system we may not want to
> > > > > use a "smart bootloader" like u-boot to load kernel, ramdisk and
> > > > > device tree from a filesystem on eMMC, but rather load the kernel
> > > > > from a NAND partition and just run it as soon as we can, and in
> > > > > this case it is convenient to have device tree compiled into the
> > > > > kernel binary. Since this case is not limited to MMU-less systems,
> > > > > let's support it for these which have MMU enabled too.
> > > > >
> > > > > Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com>
> > > > > ---
> > > > >  arch/riscv/Kconfig   |  1 -
> > > > >  arch/riscv/mm/init.c | 12 ++++++++++--
> > > > >  2 files changed, 10 insertions(+), 3 deletions(-)
> > > > >
> > > > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > > > index 2b41f6d8e458..9464b4e3a71a 100644
> > > > > --- a/arch/riscv/Kconfig
> > > > > +++ b/arch/riscv/Kconfig
> > > > > @@ -419,7 +419,6 @@ endmenu
> > > > >
> > > > >  config BUILTIN_DTB
> > > > >         def_bool n
> > > > > -       depends on RISCV_M_MODE
> > > > >         depends on OF
> > > > >
> > > > >  menu "Power management options"
> > > > > diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c
> > > > > index 87c305c566ac..5d1c7a3ec01c 100644
> > > > > --- a/arch/riscv/mm/init.c
> > > > > +++ b/arch/riscv/mm/init.c
> > > > > @@ -194,12 +194,20 @@ void __init setup_bootmem(void)
> > > > >         setup_initrd();
> > > > >  #endif /* CONFIG_BLK_DEV_INITRD */
> > > > >
> > > > > +       /*
> > > > > +        * If DTB is built in, no need to reserve its memblock.
> > > > > +        * OTOH, initial_boot_params has to be set to properly copy DTB
> > > > > +        * before unflattening later on.
> > > > > +        */
> > > > > +       if (IS_ENABLED(CONFIG_BUILTIN_DTB))
> > > > > +               initial_boot_params = __va(dtb_early_pa);
> > > >
> > > > Don't assign initial_boot_params directly here because the
> > > > early_init_dt_scan() will do it.
> > >
> > > early_init_dt_scan will set initial_boot_params to dtb_early_va from
> > > the early mapping which will be gone by the time
> > > unflatten_and_copy_device_tree() is called.
> >
> > That's why we are doing early_init_dt_verify() again for the MMU-enabled
> > case which already takes care of your concern.
>
> I might be out in the woods here but... Do you mean the call to
> early_init_dt_verify() in setup_arch() which is compiled out
> completely in the CONFIG_BUILTIN_DTB case?
> Or is there any other call that I'm overlooking?

Sorry for the confusion, what I meant was that we are calling
early_init_dt_verify() from setup_arch() for the MMU-enabled
with built-in DTB disabled case to update "initial_boot_params"
after the boot CPU has switched from early_pg_dir to swapper_pg_dir.

For MMU-enabled with built-in DTB case, if setup_vm() sets the
dtb_early_va and dtb_early_pa correctly then early_init_dt_scan()
called from setup_arch() will automatically set correct value for
"initial_boot_params".

It is strange that early_init_dt_verify() is being compiled-out for you
because the early_init_dt_scan() called from setup_arch() also uses
early_init_dt_verify(). I quickly compiled the NoMMU kernel for K210
which also uses built-in DTB and I see that early_init_dt_verify()
is not being compiled-out when built-in DTB is enabled.

Regards,
Anup

>
> Best regards,
>    Vitaly
>
> > We use early_init_dt_verify() like most architectures to set the initial DTB.
> >
> > >
> > > > The setup_vm() is supposed to setup dtb_early_va and dtb_early_pa
> > > > for MMU-enabled case so please add a "#ifdef" over there for the
> > > > built-in DTB case.
> > > >
> > > > > +       else
> > > > > +               memblock_reserve(dtb_early_pa, fdt_totalsize(dtb_early_va));
> > > > > +
> > > > >         /*
> > > > >          * Avoid using early_init_fdt_reserve_self() since __pa() does
> > > > >          * not work for DTB pointers that are fixmap addresses
> > > > >          */
> > > >
> > > > This comment needs to be updated and moved along the memblock_reserve()
> > > > statement.
> > > >
> > > > > -       memblock_reserve(dtb_early_pa, fdt_totalsize(dtb_early_va));
> > > > > -
> > > > >         early_init_fdt_scan_reserved_mem();
> > > > >         dma_contiguous_reserve(dma32_phys_limit);
> > > > >         memblock_allow_resize();
> > > > > --
> > > > > 2.29.2
> > > > >
> > > >
> > > > This patch should be based upon Damiens builtin DTB patch.
> > > > Refer, https://www.spinics.net/lists/linux-gpio/msg56616.html
> > >
> > > Thanks for the pointer, however I don't think our patches have
> > > intersections. Besides, Damien is dealing with the MMU-less case
> > > there.
> >
> > Damien's patch is also trying to move to use generic BUILTIN_DTB
> > support for the MMU-less case so it is similar work hence the chance
> > of patch conflict.
> >
> > Regards,
> > Anup