Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
References: <20190321094710.16552-1-anup.patel@wdc.com> <20190321094710.16552-4-anup.patel@wdc.com>
 <20190323154012.GA25149@rapoport-lnx>
In-Reply-To: <20190323154012.GA25149@rapoport-lnx>
From:   Anup Patel <anup@brainfault.org>
Date:   Sun, 24 Mar 2019 09:02:31 +0530
Message-ID: <CAAhSdy1enG_9o1rtZcwhht6e_-t2G1-yUy8Q5ENjnQKKW4UTeA@mail.gmail.com>
Subject: Re: [PATCH v2 3/5] RISC-V: Allow booting kernel from any 4KB aligned address
To:     Mike Rapoport <rppt@linux.ibm.com>
Cc:     Anup Patel <Anup.Patel@wdc.com>,
        Palmer Dabbelt <palmer@sifive.com>,
        Albert Ou <aou@eecs.berkeley.edu>,
        Atish Patra <Atish.Patra@wdc.com>,
        Paul Walmsley <paul.walmsley@sifive.com>,
        Christoph Hellwig <hch@infradead.org>,
        "linux-riscv@lists.infradead.org" <linux-riscv@lists.infradead.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Sat, Mar 23, 2019 at 9:10 PM Mike Rapoport <rppt@linux.ibm.com> wrote:
>
> On Thu, Mar 21, 2019 at 09:47:51AM +0000, Anup Patel wrote:
> > Currently, we have to boot RISCV64 kernel from a 2MB aligned physical
> > address and RISCV32 kernel from a 4MB aligned physical address. This
> > constraint is because initial pagetable setup (i.e. setup_vm()) maps
> > entire RAM using hugepages (i.e. 2MB for 3-level pagetable and 4MB for
> > 2-level pagetable).
> >
> > Further, the above booting contraint also results in memory wastage
> > because if we boot kernel from some <xyz> address (which is not same as
> > RAM start address) then RISCV kernel will map PAGE_OFFSET virtual address
> > lineraly to <xyz> physical address and memory between RAM start and <xyz>
> > will be reserved/unusable.
> >
> > For example, RISCV64 kernel booted from 0x80200000 will waste 2MB of RAM
> > and RISCV32 kernel booted from 0x80400000 will waste 4MB of RAM.
> >
> > This patch re-writes the initial pagetable setup code to allow booting
> > RISV32 and RISCV64 kernel from any 4KB (i.e. PAGE_SIZE) aligned address.
> >
> > To achieve this:
> > 1. We add kconfig option BOOT_PAGE_ALIGNED. When it is enabled we use
> >    4KB mappings in initial page table setup otherwise we use 2MB/4MB
> >    mappings.
> > 2. We map kernel and dtb (few MBs) in setup_vm() (called from head.S)
> > 3. Once we reach paging_init() (called from setup_arch()) after
> >    memblock setup, we map all available memory banks.
> >
> > With this patch in-place, the booting constraint for RISCV32 and RISCV64
> > kernel is much more relaxed when CONFIG_BOOT_PAGE_ALIGNED=y and we can
> > now boot kernel very close to RAM start thereby minimizng memory wastage.
>
> I have no general objection, but I presume the patch will be significantly
> simplified if the addition of 4K pages support will follow the removal of
> the trampoline_pd_dir.
>
> That said, I didn't look into the details, since they will change
> substantially, only some comments on the Kconfig part.
>
> On the high level, have you considered using large pages in setup_vm() and
> the remapping everything with 4K pages in setup_vm_final()?  This might
> save you the whole ops-> churn.

Yes, it can save the ops churn in setup_vm() but we should let setup_vm()
choose best possible mapping size based on load address alignment hence
it's better have common page table setup code shared between early and
final page table setup which can create mapping of any size.

For example, if we are booted from 1G aligned load address then setup_vm()
should create 1G mapping thereby getting better performance compared to
2M mappings.

Regards,
Anup