Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Subject: Re: [PATCH v2] mm: hugetlb: optionally allocate gigantic hugepages
 using cma
To:     Rik van Riel <riel@surriel.com>, Michal Hocko <mhocko@kernel.org>,
        Roman Gushchin <guro@fb.com>
Cc:     Andrew Morton <akpm@linux-foundation.org>,
        Johannes Weiner <hannes@cmpxchg.org>, linux-mm@kvack.org,
        kernel-team@fb.com, linux-kernel@vger.kernel.org
References: <20200310002524.2291595-1-guro@fb.com>
 <5cfa9031-fc15-2bcc-adb9-9779285ef0f7@oracle.com>
 <20200310180558.GD85000@carbon.dhcp.thefacebook.com>
 <4b78a8a9-7b5a-eb62-acaa-2677e615bea1@oracle.com>
 <20200310191906.GA96999@carbon.dhcp.thefacebook.com>
 <20200310193622.GC8447@dhcp22.suse.cz>
 <43e2e8443288260aa305f39ba566f81bf065d010.camel@surriel.com>
From:   Mike Kravetz <mike.kravetz@oracle.com>
Message-ID: <57494a9c-5c24-20b6-0bda-dac8bbb6f731@oracle.com>
Date:   Tue, 10 Mar 2020 13:11:52 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.5.0
MIME-Version: 1.0
In-Reply-To: <43e2e8443288260aa305f39ba566f81bf065d010.camel@surriel.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On 3/10/20 12:46 PM, Rik van Riel wrote:
> On Tue, 2020-03-10 at 20:36 +0100, Michal Hocko wrote:
>> On Tue 10-03-20 12:19:06, Roman Gushchin wrote:
>> [...]
>>>> I found this out by testing code and specifying
>>>> hugetlb_cma=2M.  Messages
>>>> in log were:
>>>> 	kernel: hugetlb_cma: reserve 2097152, 1048576 per node
>>>> 	kernel: hugetlb_cma: successfully reserved 1048576 on node 0
>>>> 	kernel: hugetlb_cma: successfully reserved 1048576 on node 1
>>>> But, it really reserved 1GB per node.
>>>
>>> Good point! In the passed size is too small to cover a single huge
>>> page,
>>> we should probably print a warning and bail out.
>>
>> Or maybe you just want to make the interface the unit size rather
>> than
>> overall size oriented. E.g. I want 10G pages per each numa node.
> 
> How would that work for architectures that have multiple
> possible hugetlbfs gigantic page sizes, where the admin
> can allocate different numbers of differently sized pages
> after bootup?

For hugetlb page reservations at boot today, pairs specifying size and
quantity are put on the command line.  For example,
hugepagesz=2M hugepages=512 hugepagesz=1G hugepages=64

We could do something similiar for CMA.
hugepagesz=512M hugepages_cma=256 hugepagesz=1G hugepages_cma=64

That would make things much more complicated (implies separate CMA
reservations per size) and may be overkill for the first implementation.

Perhaps we limit CMA reservations to one gigantic huge page size.  The
architectures would need to define the default and there could be a
command line option to override.  Something like,
default_cmapagesz=  analogous to today's default_hugepagesz=.  Then
hugepages_cma= is only associated with that default gigantic huge page
size.

The more I think about it, the more I like limiting CMA reservations to
only one gigantic huge page size (per arch).
-- 
Mike Kravetz