After commit 42d7395f ("mm: support more pagesizes for MAP_HUGETLB/SHM_HUGETLB")
be merged, kernel permit multiple huge page sizes, and when the system administrator
has configured the system to provide huge page pools of different sizes, application
can choose the page size used for their allocation. However, just default size of
huge page pool is statistical when memory overcommit accouting, the bad is that this
will result in innocent processes be killed by oom-killer later. Fix it by statistic
all huge page pools of different sizes provided by administrator.
Testcase:
boot: hugepagesz=1G hugepages=1
before patch:
egrep 'CommitLimit' /proc/meminfo
CommitLimit: 55434168 kB
after patch:
egrep 'CommitLimit' /proc/meminfo
CommitLimit: 54909880 kB
Signed-off-by: Wanpeng Li <[email protected]>
---
mm/hugetlb.c | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index cdb64e4..9e25040 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -2124,8 +2124,11 @@ int hugetlb_report_node_meminfo(int nid, char *buf)
/* Return the number pages of memory we physically have, in PAGE_SIZE units. */
unsigned long hugetlb_total_pages(void)
{
- struct hstate *h = &default_hstate;
- return h->nr_huge_pages * pages_per_huge_page(h);
+ struct hstate *h;
+ unsigned long nr_total_pages = 0;
+ for_each_hstate(h)
+ nr_total_pages += h->nr_huge_pages * pages_per_huge_page(h);
+ return nr_total_pages;
}
static int hugetlb_acct_memory(struct hstate *h, long delta)
--
1.7.11.7
[cc Andi]
On Wed, Mar 13, 2013 at 3:08 PM, Wanpeng Li <[email protected]> wrote:
> After commit 42d7395f ("mm: support more pagesizes for MAP_HUGETLB/SHM_HUGETLB")
> be merged, kernel permit multiple huge page sizes, and when the system administrator
> has configured the system to provide huge page pools of different sizes, application
> can choose the page size used for their allocation. However, just default size of
> huge page pool is statistical when memory overcommit accouting, the bad is that this
> will result in innocent processes be killed by oom-killer later. Fix it by statistic
> all huge page pools of different sizes provided by administrator.
>
Can we enrich the output of hugetlb_report_meminfo() ?
thanks
Hillf
> Testcase:
> boot: hugepagesz=1G hugepages=1
> before patch:
> egrep 'CommitLimit' /proc/meminfo
> CommitLimit: 55434168 kB
> after patch:
> egrep 'CommitLimit' /proc/meminfo
> CommitLimit: 54909880 kB
>
> Signed-off-by: Wanpeng Li <[email protected]>
> ---
> mm/hugetlb.c | 7 +++++--
> 1 file changed, 5 insertions(+), 2 deletions(-)
>
> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
> index cdb64e4..9e25040 100644
> --- a/mm/hugetlb.c
> +++ b/mm/hugetlb.c
> @@ -2124,8 +2124,11 @@ int hugetlb_report_node_meminfo(int nid, char *buf)
> /* Return the number pages of memory we physically have, in PAGE_SIZE units. */
> unsigned long hugetlb_total_pages(void)
> {
> - struct hstate *h = &default_hstate;
> - return h->nr_huge_pages * pages_per_huge_page(h);
> + struct hstate *h;
> + unsigned long nr_total_pages = 0;
> + for_each_hstate(h)
> + nr_total_pages += h->nr_huge_pages * pages_per_huge_page(h);
> + return nr_total_pages;
> }
>
> static int hugetlb_acct_memory(struct hstate *h, long delta)
> --
> 1.7.11.7
>
> Can we enrich the output of hugetlb_report_meminfo() ?
The data is reported separately in sysfs. It was originally decided
to not extend meminfo for them.
-Andi
On Wed 13-03-13 15:08:31, Wanpeng Li wrote:
> After commit 42d7395f ("mm: support more pagesizes for MAP_HUGETLB/SHM_HUGETLB")
> be merged, kernel permit multiple huge page sizes,
multiple huge page sizes were possible long before this commit. The
above mentioned patch just made their usage via IPC much easier. You
could do the same previously (since a137e1cc) by mounting hugetlbfs with
a specific page size as a parameter and using mmap.
> and when the system administrator has configured the system to provide
> huge page pools of different sizes, application can choose the page
> size used for their allocation.
> However, just default size of huge page pool is statistical when
> memory overcommit accouting, the bad is that this will result in
> innocent processes be killed by oom-killer later.
Why would an innnocent process be killed? The overcommit calculation
is incorrect, that is true, but this just means that an unexpected
ENOMEM/EFAULT or SIGSEGV would be returned, no? How an OOM could be a
result?
> Fix it by statistic all huge page pools of different sizes provided by
> administrator.
The patch makes sense but the description is misleading AFAICS.
> Testcase:
> boot: hugepagesz=1G hugepages=1
> before patch:
> egrep 'CommitLimit' /proc/meminfo
> CommitLimit: 55434168 kB
> after patch:
> egrep 'CommitLimit' /proc/meminfo
> CommitLimit: 54909880 kB
>
> Signed-off-by: Wanpeng Li <[email protected]>
> ---
> mm/hugetlb.c | 7 +++++--
> 1 file changed, 5 insertions(+), 2 deletions(-)
>
> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
> index cdb64e4..9e25040 100644
> --- a/mm/hugetlb.c
> +++ b/mm/hugetlb.c
> @@ -2124,8 +2124,11 @@ int hugetlb_report_node_meminfo(int nid, char *buf)
> /* Return the number pages of memory we physically have, in PAGE_SIZE units. */
> unsigned long hugetlb_total_pages(void)
> {
> - struct hstate *h = &default_hstate;
> - return h->nr_huge_pages * pages_per_huge_page(h);
> + struct hstate *h;
> + unsigned long nr_total_pages = 0;
> + for_each_hstate(h)
> + nr_total_pages += h->nr_huge_pages * pages_per_huge_page(h);
> + return nr_total_pages;
> }
>
> static int hugetlb_acct_memory(struct hstate *h, long delta)
> --
> 1.7.11.7
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to [email protected]. For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
--
Michal Hocko
SUSE Labs