Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Thu, 23 Jul 2020 16:18:37 +0300
From:   Mike Rapoport <rppt@linux.ibm.com>
To:     Catalin Marinas <catalin.marinas@arm.com>
Cc:     "liwei (CM)" <liwei213@huawei.com>,
        "will@kernel.org" <will@kernel.org>,
        "Xiaqing (A)" <saberlily.xia@hisilicon.com>,
        "Chenfeng (puck)" <puck.chen@hisilicon.com>,
        butao <butao@hisilicon.com>,
        fengbaopeng <fengbaopeng2@hisilicon.com>,
        "nsaenzjulienne@suse.de" <nsaenzjulienne@suse.de>,
        "steve.capper@arm.com" <steve.capper@arm.com>,
        "Song Bao Hua (Barry Song)" <song.bao.hua@hisilicon.com>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        sujunfei <sujunfei2@hisilicon.com>,
        zhaojiapeng <zhaojiapeng@huawei.com>
Subject: Re: =?utf-8?B?562U5aSNOiDnrZTlpI06IFtQQVRD?=
 =?utf-8?Q?H=5D_arm64=3A_mm=3A_fre?= =?utf-8?Q?e?= unused memmap for sparse
 memory model that define VMEMMAP
Message-ID: <20200723131837.GC1975360@linux.ibm.com>
References: <20200721073203.107862-1-liwei213@huawei.com>
 <20200722060705.GK802087@linux.ibm.com>
 <1699CE87DE933F49876AD744B5DC140F2312E948@dggemm526-mbx.china.huawei.com>
 <20200722124910.GE27540@gaia>
 <1699CE87DE933F49876AD744B5DC140F2312F0D6@dggemm526-mbx.china.huawei.com>
 <20200723112926.GB7315@gaia>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20200723112926.GB7315@gaia>
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Thu, Jul 23, 2020 at 12:29:26PM +0100, Catalin Marinas wrote:
> On Wed, Jul 22, 2020 at 01:40:34PM +0000, liwei (CM) wrote:
> > Catalin Marinas wrote:
> > > On Wed, Jul 22, 2020 at 08:41:17AM +0000, liwei (CM) wrote:
> > > > Mike Rapoport wrote:
> > > > > On Tue, Jul 21, 2020 at 03:32:03PM +0800, Wei Li wrote:
> > > > > > For the memory hole, sparse memory model that define 
> > > > > > SPARSEMEM_VMEMMAP do not free the reserved memory for the page
> > > > > > map, this patch do it.
> > > > > 
> > > > > Are there numbers showing how much memory is actually freed?
> > > > > 
> > > > > The freeing of empty memmap would become rather complex with these 
> > > > > changes, do the memory savings justify it?
> > > > 
> > > > In the sparse memory model, the size of a section is 1 GB 
> > > > (SECTION_SIZE_BITS 30) by default.
> > > 
> > > Can we reduce SECTION_SIZE_BITS instead? Say 26?
> > 
> > Yes, you are right, reduce SECTION_SIZE_BITS to 26 can save almost the
> > same memory as the patch. 
> > 
> > 1) However, it is not clear whether changing the section size has any
> > other impact.
> 
> Well, we should analyse this.
> 
> > 2) Just like the flat memory model and the sparse memory model that
> > does not define VMEMMAP, both of them have their own ways to free
> > unused memmap. I think we've given a similar way for sparse memory
> > define VMEMMAP.
> 
> I think we did it for flatmem initially (on arm32) and added support for
> sparsemem later on, so free_unused_memmap() had to cope with sparse
> sections. On arm64 we introduced vmemmap support and didn't bother with
> the freeing at all because of the added complexity of the vmemmap page
> tables.
> 
> I wonder whether we should just disallow flatmem and non-vmemmap
> sparsemem on arm64. Is there any value in keeping them around?

FLATMEM is useful for UMA systems with a single memory bank, so probably
it's worth keeping it for low end machines.

Non-vmemmap sparsemem is essentially disable in arch/arm64/Kconfig, so
for NUMA configurations SPARSEMEM_VMEMMAP is the only choice.
 
> > 3) This explicit free unused memmap method does reduce unnecessary
> > memory waste for users who do not notice the section size
> > modification.
> 
> But if we changed SECTION_SIZE_BITS in the mainline kernel, then we
> wouldn't need additional code to free the unused memmap.

Moreover if we reduce SECTION_SIZE_BITS, we can drop
free_unused_memmap() and since the arm64 memory map for sparse will not
differ from other arches we can drop custom pfn_valid() as well.
 
> -- 
> Catalin

-- 
Sincerely yours,
Mike.