This patchset is based on the series of "Free some vmemmap pages of HugeTLB
page". More details can refer to the below link.
https://lkml.kernel.org/r/[email protected]
I often received some feedback (We want to test this feature on arm64) before.
Because the previous code has been reviewed for 18 versions and is merged
into mm tree, I think that it is time to release this patchset. If you want
to test then you can start now :-). And I also hope someone can review this.
Thanks.
Muchun Song (3):
mm: bootmem_info: mark register_page_bootmem_info_section __init
mm: hugetlb: introduce arch_free_vmemmap_page
arm64: mm: hugetlb: add support for free vmemmap pages of HugeTLB
arch/arm64/mm/mmu.c | 5 +++++
arch/x86/mm/init_64.c | 5 +++++
fs/Kconfig | 4 ++--
mm/bootmem_info.c | 4 ++--
mm/sparse-vmemmap.c | 9 +++++++--
5 files changed, 21 insertions(+), 6 deletions(-)
--
2.11.0
The preparation of supporting freeing vmemmap associated with each
infrastructure is ready, so now we can support it for arm64.
Signed-off-by: Muchun Song <[email protected]>
---
arch/arm64/mm/mmu.c | 5 +++++
fs/Kconfig | 4 ++--
2 files changed, 7 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
index 3802cfbdd20d..6051a120c906 100644
--- a/arch/arm64/mm/mmu.c
+++ b/arch/arm64/mm/mmu.c
@@ -22,6 +22,7 @@
#include <linux/io.h>
#include <linux/mm.h>
#include <linux/vmalloc.h>
+#include <linux/hugetlb.h>
#include <asm/barrier.h>
#include <asm/cputype.h>
@@ -1109,6 +1110,10 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node,
pmd_t *pmdp;
WARN_ON((start < VMEMMAP_START) || (end > VMEMMAP_END));
+
+ if (is_hugetlb_free_vmemmap_enabled() && !altmap)
+ return vmemmap_populate_basepages(start, end, node, altmap);
+
do {
next = pmd_addr_end(addr, end);
diff --git a/fs/Kconfig b/fs/Kconfig
index b5dcc68aab25..d90860135844 100644
--- a/fs/Kconfig
+++ b/fs/Kconfig
@@ -239,9 +239,9 @@ config HUGETLB_PAGE
config HUGETLB_PAGE_FREE_VMEMMAP
def_bool HUGETLB_PAGE
- depends on X86_64
+ select HAVE_BOOTMEM_INFO_NODE if X86_64
depends on SPARSEMEM_VMEMMAP
- depends on HAVE_BOOTMEM_INFO_NODE
+ depends on X86_64 || ARM64
config MEMFD_CREATE
def_bool TMPFS || HUGETLBFS
--
2.11.0
Hi Muchun,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on next-20210310]
[cannot apply to hnaz-linux-mm/master arm64/for-next/core tip/x86/mm linus/master v5.12-rc2 v5.12-rc1 v5.11 v5.12-rc2]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Muchun-Song/mm-bootmem_info-mark-register_page_bootmem_info_section-__init/20210310-161619
base: b01d57bfdc41c8f635b08b8a5af8a31217d46936
config: x86_64-randconfig-r002-20210309 (attached as .config)
compiler: clang version 13.0.0 (https://github.com/llvm/llvm-project cd9a69289c7825d54450cb6829fef2c8e0f1963a)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# install x86_64 cross compiling tool for clang build
# apt-get install binutils-x86-64-linux-gnu
# https://github.com/0day-ci/linux/commit/d0b8d83bb423fd25a85be4a62a8bed9f8d8a0d96
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Muchun-Song/mm-bootmem_info-mark-register_page_bootmem_info_section-__init/20210310-161619
git checkout d0b8d83bb423fd25a85be4a62a8bed9f8d8a0d96
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=x86_64
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <[email protected]>
All warnings (new ones prefixed by >>):
arch/x86/mm/init_64.c:1585:6: error: implicit declaration of function 'vmemmap_use_new_sub_pmd' [-Werror,-Wimplicit-function-declaration]
vmemmap_use_new_sub_pmd(addr, next);
^
arch/x86/mm/init_64.c:1591:4: error: implicit declaration of function 'vmemmap_use_sub_pmd' [-Werror,-Wimplicit-function-declaration]
vmemmap_use_sub_pmd(addr, next);
^
>> arch/x86/mm/init_64.c:1625:6: warning: no previous prototype for function 'arch_free_vmemmap_page' [-Wmissing-prototypes]
void arch_free_vmemmap_page(struct page *page)
^
arch/x86/mm/init_64.c:1625:1: note: declare 'static' if the function is not intended to be used outside of this translation unit
void arch_free_vmemmap_page(struct page *page)
^
static
1 warning and 2 errors generated.
--
mm/bootmem_info.c:87:11: error: implicit declaration of function 'sparse_decode_mem_map' [-Werror,-Wimplicit-function-declaration]
memmap = sparse_decode_mem_map(ms->section_mem_map, section_nr);
^
>> mm/bootmem_info.c:87:9: warning: incompatible integer to pointer conversion assigning to 'struct page *' from 'int' [-Wint-conversion]
memmap = sparse_decode_mem_map(ms->section_mem_map, section_nr);
^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning and 1 error generated.
vim +/arch_free_vmemmap_page +1625 arch/x86/mm/init_64.c
c2b91e2eec9678 Yinghai Lu 2008-04-12 1623
cd28b1a6791da0 Muchun Song 2021-03-09 1624 #ifdef CONFIG_HAVE_BOOTMEM_INFO_NODE
92924b23a9f6de Muchun Song 2021-03-10 @1625 void arch_free_vmemmap_page(struct page *page)
92924b23a9f6de Muchun Song 2021-03-10 1626 {
92924b23a9f6de Muchun Song 2021-03-10 1627 free_bootmem_page(page);
92924b23a9f6de Muchun Song 2021-03-10 1628 }
92924b23a9f6de Muchun Song 2021-03-10 1629
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/[email protected]
在 2021/3/10 15:15, Muchun Song 写道:
> This patchset is based on the series of "Free some vmemmap pages of HugeTLB
> page". More details can refer to the below link.
>
> https://lkml.kernel.org/r/[email protected]
>
> I often received some feedback (We want to test this feature on arm64) before.
> Because the previous code has been reviewed for 18 versions and is merged
> into mm tree, I think that it is time to release this patchset. If you want
> to test then you can start now :-). And I also hope someone can review this.
>
> Thanks.
>
> Muchun Song (3):
> mm: bootmem_info: mark register_page_bootmem_info_section __init
> mm: hugetlb: introduce arch_free_vmemmap_page
> arm64: mm: hugetlb: add support for free vmemmap pages of HugeTLB
>
> arch/arm64/mm/mmu.c | 5 +++++
> arch/x86/mm/init_64.c | 5 +++++
> fs/Kconfig | 4 ++--
> mm/bootmem_info.c | 4 ++--
> mm/sparse-vmemmap.c | 9 +++++++--
> 5 files changed, 21 insertions(+), 6 deletions(-)
>
Tested-by: Chen Huang <[email protected]>
I have tested the patch and the result is same as the last time.
Chen, is your testing steps documented somewhere, can you please point us to the same. I followed some steps for testing the x86 patches, just wanted to make sure I am covering your tests as well. We are actively working on building and testing these patches for ARM.
On 11/03/21, 9:44 AM, "Chen Huang" <[email protected]> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
在 2021/3/10 15:15, Muchun Song 写道:
> This patchset is based on the series of "Free some vmemmap pages of HugeTLB
> page". More details can refer to the below link.
>
> https://lkml.kernel.org/r/[email protected]
>
> I often received some feedback (We want to test this feature on arm64) before.
> Because the previous code has been reviewed for 18 versions and is merged
> into mm tree, I think that it is time to release this patchset. If you want
> to test then you can start now :-). And I also hope someone can review this.
>
> Thanks.
>
> Muchun Song (3):
> mm: bootmem_info: mark register_page_bootmem_info_section __init
> mm: hugetlb: introduce arch_free_vmemmap_page
> arm64: mm: hugetlb: add support for free vmemmap pages of HugeTLB
>
> arch/arm64/mm/mmu.c | 5 +++++
> arch/x86/mm/init_64.c | 5 +++++
> fs/Kconfig | 4 ++--
> mm/bootmem_info.c | 4 ++--
> mm/sparse-vmemmap.c | 9 +++++++--
> 5 files changed, 21 insertions(+), 6 deletions(-)
>
Tested-by: Chen Huang <[email protected]>
I have tested the patch and the result is same as the last time.
在 2021/3/11 13:00, Bodeddula, Balasubramaniam 写道:
> Chen, is your testing steps documented somewhere, can you please point us to the same. I followed some steps for testing the x86 patches, just wanted to make sure I am covering your tests as well. We are actively working on building and testing these patches for ARM.
>
> On 11/03/21, 9:44 AM, "Chen Huang" <[email protected]> wrote:
>
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
> 在 2021/3/10 15:15, Muchun Song 写道:
> > This patchset is based on the series of "Free some vmemmap pages of HugeTLB
> > page". More details can refer to the below link.
> >
> > https://lkml.kernel.org/r/[email protected]
> >
> > I often received some feedback (We want to test this feature on arm64) before.
> > Because the previous code has been reviewed for 18 versions and is merged
> > into mm tree, I think that it is time to release this patchset. If you want
> > to test then you can start now :-). And I also hope someone can review this.
> >
> > Thanks.
> >
> > Muchun Song (3):
> > mm: bootmem_info: mark register_page_bootmem_info_section __init
> > mm: hugetlb: introduce arch_free_vmemmap_page
> > arm64: mm: hugetlb: add support for free vmemmap pages of HugeTLB
> >
> > arch/arm64/mm/mmu.c | 5 +++++
> > arch/x86/mm/init_64.c | 5 +++++
> > fs/Kconfig | 4 ++--
> > mm/bootmem_info.c | 4 ++--
> > mm/sparse-vmemmap.c | 9 +++++++--
> > 5 files changed, 21 insertions(+), 6 deletions(-)
> >
>
> Tested-by: Chen Huang <[email protected]>
>
> I have tested the patch and the result is same as the last time.
>
The test work is that: I set the total memory of 40G, and use 10G for hugepages.
First I reserve 10G hugepages from the command line and the result is that:
------------------------------------------------------------------------------------------------
2M page | 1G page |
----------------------|------------------------|----------------------|------------------------|
enable | disable | enable | disable |
----------------------|------------------------|----------------------|------------------------|
total | used | free | total | used | free |total | used | free | total | used | free |
39,697 | 10279 |29,415| 39580 | 10279 | 29,297|39,739 | 10279 |29,455| 39580 | 10279 | 29,296|
------------------------------------------------------------------------------------------------
For 2M hugepage, we can save 118M memory which is correspoinding to the expected 120M memory.
For 1G hugepage, we can save 159M memory which is correspoinding to the expected 160M memory.
Then I alloc 10G hugepages using "echo XX > /sys/kernel/mm/hugepages/hugepages-XXkB/nr_hugepages",
and get the result:
------------------------------------------------------------------------------------------------
2M page | 1G page |
----------------------|------------------------|----------------------|------------------------|
enable | disable | enable | disable |
----------------------|------------------------|----------------------|------------------------|
total | used | free | total | used | free |total | used | free | total | used | free |
39,699 | 10279 |29,415| 39580 | 10279 | 29,297|39,739 | 10279 |29,455| 39580 | 10279 | 29,296|
------------------------------------------------------------------------------------------------
For 2M hugepage, we can save 118M memory which is correspoinding to the expected 120M memory.
For 1G hugepage, we can save 159M memory which is correspoinding to the expected 160M memory.
Hey, thanks for the testing steps.
I tried applying these patches on 5.11 source tree. These patches were applied on top of the x86 patch, which worked fine. But in this case we don't see the same improvements in our testing. We made sure we set CONFIG_HUGETLB_PAGE_FREE_VMEMMAP=y in our .config file.
Are we missing any more configuration settings to get this patch to work on ARM? Can you please help with general troubleshooting steps to debug what could be going wrong.
On 11/03/21, 11:32 AM, "Chen Huang" <[email protected]> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
在 2021/3/11 13:00, Bodeddula, Balasubramaniam 写道:
> Chen, is your testing steps documented somewhere, can you please point us to the same. I followed some steps for testing the x86 patches, just wanted to make sure I am covering your tests as well. We are actively working on building and testing these patches for ARM.
>
> On 11/03/21, 9:44 AM, "Chen Huang" <[email protected]> wrote:
>
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
> 在 2021/3/10 15:15, Muchun Song 写道:
> > This patchset is based on the series of "Free some vmemmap pages of HugeTLB
> > page". More details can refer to the below link.
> >
> > https://lkml.kernel.org/r/[email protected]
> >
> > I often received some feedback (We want to test this feature on arm64) before.
> > Because the previous code has been reviewed for 18 versions and is merged
> > into mm tree, I think that it is time to release this patchset. If you want
> > to test then you can start now :-). And I also hope someone can review this.
> >
> > Thanks.
> >
> > Muchun Song (3):
> > mm: bootmem_info: mark register_page_bootmem_info_section __init
> > mm: hugetlb: introduce arch_free_vmemmap_page
> > arm64: mm: hugetlb: add support for free vmemmap pages of HugeTLB
> >
> > arch/arm64/mm/mmu.c | 5 +++++
> > arch/x86/mm/init_64.c | 5 +++++
> > fs/Kconfig | 4 ++--
> > mm/bootmem_info.c | 4 ++--
> > mm/sparse-vmemmap.c | 9 +++++++--
> > 5 files changed, 21 insertions(+), 6 deletions(-)
> >
>
> Tested-by: Chen Huang <[email protected]>
>
> I have tested the patch and the result is same as the last time.
>
The test work is that: I set the total memory of 40G, and use 10G for hugepages.
First I reserve 10G hugepages from the command line and the result is that:
------------------------------------------------------------------------------------------------
2M page | 1G page |
----------------------|------------------------|----------------------|------------------------|
enable | disable | enable | disable |
----------------------|------------------------|----------------------|------------------------|
total | used | free | total | used | free |total | used | free | total | used | free |
39,697 | 10279 |29,415| 39580 | 10279 | 29,297|39,739 | 10279 |29,455| 39580 | 10279 | 29,296|
------------------------------------------------------------------------------------------------
For 2M hugepage, we can save 118M memory which is correspoinding to the expected 120M memory.
For 1G hugepage, we can save 159M memory which is correspoinding to the expected 160M memory.
Then I alloc 10G hugepages using "echo XX > /sys/kernel/mm/hugepages/hugepages-XXkB/nr_hugepages",
and get the result:
------------------------------------------------------------------------------------------------
2M page | 1G page |
----------------------|------------------------|----------------------|------------------------|
enable | disable | enable | disable |
----------------------|------------------------|----------------------|------------------------|
total | used | free | total | used | free |total | used | free | total | used | free |
39,699 | 10279 |29,415| 39580 | 10279 | 29,297|39,739 | 10279 |29,455| 39580 | 10279 | 29,296|
------------------------------------------------------------------------------------------------
For 2M hugepage, we can save 118M memory which is correspoinding to the expected 120M memory.
For 1G hugepage, we can save 159M memory which is correspoinding to the expected 160M memory.
Are you saying that the patch works well in x86, but doesn't in arm64? If so, my suggestion is that first you can check the dumpstack to check whether the function free_vmemmap_page has been called.
Then you can dump the page list of vmemmap_pages and the refcount of pages to verify the pages has been freed into the buddy system.
Hope this will help you!
在 2021/3/12 2:00, Bodeddula, Balasubramaniam 写道:
> Hey, thanks for the testing steps.
>
> I tried applying these patches on 5.11 source tree. These patches were applied on top of the x86 patch, which worked fine. But in this case we don't see the same improvements in our testing. We made sure we set CONFIG_HUGETLB_PAGE_FREE_VMEMMAP=y in our .config file.
>
> Are we missing any more configuration settings to get this patch to work on ARM? Can you please help with general troubleshooting steps to debug what could be going wrong.
>
> On 11/03/21, 11:32 AM, "Chen Huang" <[email protected]> wrote:
>
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
> 在 2021/3/11 13:00, Bodeddula, Balasubramaniam 写道:
> > Chen, is your testing steps documented somewhere, can you please point us to the same. I followed some steps for testing the x86 patches, just wanted to make sure I am covering your tests as well. We are actively working on building and testing these patches for ARM.
> >
> > On 11/03/21, 9:44 AM, "Chen Huang" <[email protected]> wrote:
> >
> > CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
> >
> >
> >
> > 在 2021/3/10 15:15, Muchun Song 写道:
> > > This patchset is based on the series of "Free some vmemmap pages of HugeTLB
> > > page". More details can refer to the below link.
> > >
> > > https://lkml.kernel.org/r/[email protected]
> > >
> > > I often received some feedback (We want to test this feature on arm64) before.
> > > Because the previous code has been reviewed for 18 versions and is merged
> > > into mm tree, I think that it is time to release this patchset. If you want
> > > to test then you can start now :-). And I also hope someone can review this.
> > >
> > > Thanks.
> > >
> > > Muchun Song (3):
> > > mm: bootmem_info: mark register_page_bootmem_info_section __init
> > > mm: hugetlb: introduce arch_free_vmemmap_page
> > > arm64: mm: hugetlb: add support for free vmemmap pages of HugeTLB
> > >
> > > arch/arm64/mm/mmu.c | 5 +++++
> > > arch/x86/mm/init_64.c | 5 +++++
> > > fs/Kconfig | 4 ++--
> > > mm/bootmem_info.c | 4 ++--
> > > mm/sparse-vmemmap.c | 9 +++++++--
> > > 5 files changed, 21 insertions(+), 6 deletions(-)
> > >
> >
> > Tested-by: Chen Huang <[email protected]>
> >
> > I have tested the patch and the result is same as the last time.
> >
>
> The test work is that: I set the total memory of 40G, and use 10G for hugepages.
> First I reserve 10G hugepages from the command line and the result is that:
> ------------------------------------------------------------------------------------------------
> 2M page | 1G page |
> ----------------------|------------------------|----------------------|------------------------|
> enable | disable | enable | disable |
> ----------------------|------------------------|----------------------|------------------------|
> total | used | free | total | used | free |total | used | free | total | used | free |
> 39,697 | 10279 |29,415| 39580 | 10279 | 29,297|39,739 | 10279 |29,455| 39580 | 10279 | 29,296|
> ------------------------------------------------------------------------------------------------
> For 2M hugepage, we can save 118M memory which is correspoinding to the expected 120M memory.
> For 1G hugepage, we can save 159M memory which is correspoinding to the expected 160M memory.
>
> Then I alloc 10G hugepages using "echo XX > /sys/kernel/mm/hugepages/hugepages-XXkB/nr_hugepages",
> and get the result:
> ------------------------------------------------------------------------------------------------
> 2M page | 1G page |
> ----------------------|------------------------|----------------------|------------------------|
> enable | disable | enable | disable |
> ----------------------|------------------------|----------------------|------------------------|
> total | used | free | total | used | free |total | used | free | total | used | free |
> 39,699 | 10279 |29,415| 39580 | 10279 | 29,297|39,739 | 10279 |29,455| 39580 | 10279 | 29,296|
> ------------------------------------------------------------------------------------------------
> For 2M hugepage, we can save 118M memory which is correspoinding to the expected 120M memory.
> For 1G hugepage, we can save 159M memory which is correspoinding to the expected 160M memory.
>
On 3/11/21 10:00 AM, Bodeddula, Balasubramaniam wrote:
> Hey, thanks for the testing steps.
>
> I tried applying these patches on 5.11 source tree. These patches were applied on top of the x86 patch, which worked fine. But in this case we don't see the same improvements in our testing. We made sure we set CONFIG_HUGETLB_PAGE_FREE_VMEMMAP=y in our .config file.
>
> Are we missing any more configuration settings to get this patch to work on ARM? Can you please help with general troubleshooting steps to debug what could be going wrong.
>
Are you specifying 'hugetlb_free_vmemmap=on' on the kernel command line?
This feature is only enabled if you 'opt in' via the command line option.
--
Mike Kravetz