This series enables vmemmap backing memory allocation from device memory
ranges on arm64. But before that, it enables vmemmap_populate_basepages()
and vmemmap_alloc_block_buf() to accommodate struct vmem_altmap based
alocation requests.
This series applies on 5.8-rc1.
Pending Question:
altmap_alloc_block_buf() does not have any other remaining users in the
tree after this change. Should it be converted into a static function and
it's declaration be dropped from the header (include/linux/mm.h). Avoided
doing so because I was not sure if there are any off-tree users or not.
Changes in V3:
- Dropped comment from free_hotplug_page_range() per Robin
- Modified comment in unmap_hotplug_range() per Robin
- Enabled altmap support in vmemmap_alloc_block_buf() per Robin
Changes in V2: (https://lkml.org/lkml/2020/3/4/475)
- Rebased on latest hot-remove series (v14) adding P4D page table support
Changes in V1: (https://lkml.org/lkml/2020/1/23/12)
- Added an WARN_ON() in unmap_hotplug_range() when altmap is
provided without the page table backing memory being freed
Changes in RFC V2: (https://lkml.org/lkml/2019/10/21/11)
- Changed the commit message on 1/2 patch per Will
- Changed the commit message on 2/2 patch as well
- Rebased on arm64 memory hot remove series (v10)
RFC V1: (https://lkml.org/lkml/2019/6/28/32)
Cc: Catalin Marinas <[email protected]>
Cc: Will Deacon <[email protected]>
Cc: Mark Rutland <[email protected]>
Cc: Paul Walmsley <[email protected]>
Cc: Palmer Dabbelt <[email protected]>
Cc: Tony Luck <[email protected]>
Cc: Fenghua Yu <[email protected]>
Cc: Dave Hansen <[email protected]>
Cc: Andy Lutomirski <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: David Hildenbrand <[email protected]>
Cc: Mike Rapoport <[email protected]>
Cc: Michal Hocko <[email protected]>
Cc: "Matthew Wilcox (Oracle)" <[email protected]>
Cc: "Kirill A. Shutemov" <[email protected]>
Cc: Andrew Morton <[email protected]>
Cc: Dan Williams <[email protected]>
Cc: Pavel Tatashin <[email protected]>
Cc: Benjamin Herrenschmidt <[email protected]>
Cc: Paul Mackerras <[email protected]>
Cc: Michael Ellerman <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Anshuman Khandual (3):
mm/sparsemem: Enable vmem_altmap support in vmemmap_populate_basepages()
mm/sparsemem: Enable vmem_altmap support in vmemmap_alloc_block_buf()
arm64/mm: Enable vmem_altmap support for vmemmap mappings
arch/arm64/mm/mmu.c | 59 ++++++++++++++++++++++++++-------------
arch/ia64/mm/discontig.c | 2 +-
arch/powerpc/mm/init_64.c | 10 +++----
arch/riscv/mm/init.c | 2 +-
arch/x86/mm/init_64.c | 12 ++++----
include/linux/mm.h | 8 ++++--
mm/sparse-vmemmap.c | 38 ++++++++++++++++++++-----
7 files changed, 87 insertions(+), 44 deletions(-)
--
2.20.1
There are many instances where vmemap allocation is often switched between
regular memory and device memory just based on whether altmap is available
or not. vmemmap_alloc_block_buf() is used in various platforms to allocate
vmemmap mappings. Lets also enable it to handle altmap based device memory
allocation along with existing regular memory allocations. This will help
in avoiding the altmap based allocation switch in many places.
While here also implement a regular memory allocation fallback mechanism
when the first preferred device memory allocation fails. This will ensure
preserving the existing semantics on powerpc platform. To summarize there
are three different methods to call vmemmap_alloc_block_buf().
(., NULL, false) /* Allocate from system RAM */
(., altmap, false) /* Allocate from altmap without any fallback */
(., altmap, true) /* Allocate from altmap with fallback (system RAM) */
Cc: Catalin Marinas <[email protected]>
Cc: Will Deacon <[email protected]>
Cc: Benjamin Herrenschmidt <[email protected]>
Cc: Paul Mackerras <[email protected]>
Cc: Michael Ellerman <[email protected]>
Cc: Dave Hansen <[email protected]>
Cc: Andy Lutomirski <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: "H. Peter Anvin" <[email protected]>
Cc: Andrew Morton <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Tested-by: Jia He <[email protected]>
Suggested-by: Robin Murphy <[email protected]>
Signed-off-by: Anshuman Khandual <[email protected]>
---
arch/arm64/mm/mmu.c | 3 ++-
arch/powerpc/mm/init_64.c | 10 +++++-----
arch/x86/mm/init_64.c | 6 ++----
include/linux/mm.h | 3 ++-
mm/sparse-vmemmap.c | 30 ++++++++++++++++++++++++------
5 files changed, 35 insertions(+), 17 deletions(-)
diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
index 0adad8859393..7ca21adb4412 100644
--- a/arch/arm64/mm/mmu.c
+++ b/arch/arm64/mm/mmu.c
@@ -1100,7 +1100,8 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node,
if (pmd_none(READ_ONCE(*pmdp))) {
void *p = NULL;
- p = vmemmap_alloc_block_buf(PMD_SIZE, node);
+ p = vmemmap_alloc_block_buf(PMD_SIZE, node,
+ NULL, false);
if (!p)
return -ENOMEM;
diff --git a/arch/powerpc/mm/init_64.c b/arch/powerpc/mm/init_64.c
index bc73abf0bc25..01e25b56eccb 100644
--- a/arch/powerpc/mm/init_64.c
+++ b/arch/powerpc/mm/init_64.c
@@ -225,12 +225,12 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node,
* fall back to system memory if the altmap allocation fail.
*/
if (altmap && !altmap_cross_boundary(altmap, start, page_size)) {
- p = altmap_alloc_block_buf(page_size, altmap);
- if (!p)
- pr_debug("altmap block allocation failed, falling back to system memory");
+ p = vmemmap_alloc_block_buf(page_size, node,
+ altmap, true);
+ } else {
+ p = vmemmap_alloc_block_buf(page_size, node,
+ NULL, false);
}
- if (!p)
- p = vmemmap_alloc_block_buf(page_size, node);
if (!p)
return -ENOMEM;
diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c
index 19c0ed3271a3..4ae4f767c004 100644
--- a/arch/x86/mm/init_64.c
+++ b/arch/x86/mm/init_64.c
@@ -1463,10 +1463,8 @@ static int __meminit vmemmap_populate_hugepages(unsigned long start,
if (pmd_none(*pmd)) {
void *p;
- if (altmap)
- p = altmap_alloc_block_buf(PMD_SIZE, altmap);
- else
- p = vmemmap_alloc_block_buf(PMD_SIZE, node);
+ p = vmemmap_alloc_block_buf(PMD_SIZE, node,
+ altmap, false);
if (p) {
pte_t entry;
diff --git a/include/linux/mm.h b/include/linux/mm.h
index e40ac543d248..dade7c3f634d 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -3015,7 +3015,8 @@ pte_t *vmemmap_pte_populate(pmd_t *pmd, unsigned long addr, int node,
struct vmem_altmap *altmap);
void *vmemmap_alloc_block(unsigned long size, int node);
struct vmem_altmap;
-void *vmemmap_alloc_block_buf(unsigned long size, int node);
+void *vmemmap_alloc_block_buf(unsigned long size, int node,
+ struct vmem_altmap *altmap, bool sysram_fallback);
void *altmap_alloc_block_buf(unsigned long size, struct vmem_altmap *altmap);
void vmemmap_verify(pte_t *, int, unsigned long, unsigned long);
int vmemmap_populate_basepages(unsigned long start, unsigned long end,
diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c
index ceed10dec31e..388121c5bbcb 100644
--- a/mm/sparse-vmemmap.c
+++ b/mm/sparse-vmemmap.c
@@ -70,10 +70,31 @@ void * __meminit vmemmap_alloc_block(unsigned long size, int node)
}
/* need to make sure size is all the same during early stage */
-void * __meminit vmemmap_alloc_block_buf(unsigned long size, int node)
+void * __meminit vmemmap_alloc_block_buf(unsigned long size, int node,
+ struct vmem_altmap *altmap,
+ bool sysram_fallback)
{
- void *ptr = sparse_buffer_alloc(size);
+ void *ptr;
+ /*
+ * There is no point in asking for sysram fallback
+ * without an altmap request to begin with. So just
+ * warn here to catch potential call sites that may
+ * be violating this.
+ */
+ WARN_ON(!altmap && sysram_fallback);
+
+ if (altmap) {
+ ptr = altmap_alloc_block_buf(size, altmap);
+ if (ptr)
+ return ptr;
+ pr_debug("altmap block allocation failed\n");
+ if (!sysram_fallback)
+ return NULL;
+ pr_debug("falling back to system memory\n");
+ }
+
+ ptr = sparse_buffer_alloc(size);
if (!ptr)
ptr = vmemmap_alloc_block(size, node);
return ptr;
@@ -147,10 +168,7 @@ pte_t * __meminit vmemmap_pte_populate(pmd_t *pmd, unsigned long addr, int node,
pte_t entry;
void *p;
- if (altmap)
- p = altmap_alloc_block_buf(PAGE_SIZE, altmap);
- else
- p = vmemmap_alloc_block_buf(PAGE_SIZE, node);
+ p = vmemmap_alloc_block_buf(PAGE_SIZE, node, altmap, false);
if (!p)
return NULL;
entry = pfn_pte(__pa(p) >> PAGE_SHIFT, PAGE_KERNEL);
--
2.20.1
On Thu, Jun 18, 2020 at 06:45:27AM +0530, Anshuman Khandual wrote:
> This series enables vmemmap backing memory allocation from device memory
> ranges on arm64. But before that, it enables vmemmap_populate_basepages()
> and vmemmap_alloc_block_buf() to accommodate struct vmem_altmap based
> alocation requests.
>
> This series applies on 5.8-rc1.
>
> Pending Question:
>
> altmap_alloc_block_buf() does not have any other remaining users in
> the tree after this change. Should it be converted into a static
> function and it's declaration be dropped from the header
> (include/linux/mm.h). Avoided doing so because I was not sure if there
> are any off-tree users or not.
Well, off-tree users probably have an active fork anyway so they could
switch to vmemmap_alloc_block_buf()...
Regardless, can you please update Documentation/vm/memory-model.rst to
keep it in sync with the code?
> Changes in V3:
>
> - Dropped comment from free_hotplug_page_range() per Robin
> - Modified comment in unmap_hotplug_range() per Robin
> - Enabled altmap support in vmemmap_alloc_block_buf() per Robin
>
> Changes in V2: (https://lkml.org/lkml/2020/3/4/475)
>
> - Rebased on latest hot-remove series (v14) adding P4D page table support
>
> Changes in V1: (https://lkml.org/lkml/2020/1/23/12)
>
> - Added an WARN_ON() in unmap_hotplug_range() when altmap is
> provided without the page table backing memory being freed
>
> Changes in RFC V2: (https://lkml.org/lkml/2019/10/21/11)
>
> - Changed the commit message on 1/2 patch per Will
> - Changed the commit message on 2/2 patch as well
> - Rebased on arm64 memory hot remove series (v10)
>
> RFC V1: (https://lkml.org/lkml/2019/6/28/32)
>
> Cc: Catalin Marinas <[email protected]>
> Cc: Will Deacon <[email protected]>
> Cc: Mark Rutland <[email protected]>
> Cc: Paul Walmsley <[email protected]>
> Cc: Palmer Dabbelt <[email protected]>
> Cc: Tony Luck <[email protected]>
> Cc: Fenghua Yu <[email protected]>
> Cc: Dave Hansen <[email protected]>
> Cc: Andy Lutomirski <[email protected]>
> Cc: Peter Zijlstra <[email protected]>
> Cc: Thomas Gleixner <[email protected]>
> Cc: Ingo Molnar <[email protected]>
> Cc: David Hildenbrand <[email protected]>
> Cc: Mike Rapoport <[email protected]>
> Cc: Michal Hocko <[email protected]>
> Cc: "Matthew Wilcox (Oracle)" <[email protected]>
> Cc: "Kirill A. Shutemov" <[email protected]>
> Cc: Andrew Morton <[email protected]>
> Cc: Dan Williams <[email protected]>
> Cc: Pavel Tatashin <[email protected]>
> Cc: Benjamin Herrenschmidt <[email protected]>
> Cc: Paul Mackerras <[email protected]>
> Cc: Michael Ellerman <[email protected]>
> Cc: [email protected]
> Cc: [email protected]
> Cc: [email protected]
> Cc: [email protected]
> Cc: [email protected]
> Cc: [email protected]
> Cc: [email protected]
>
> Anshuman Khandual (3):
> mm/sparsemem: Enable vmem_altmap support in vmemmap_populate_basepages()
> mm/sparsemem: Enable vmem_altmap support in vmemmap_alloc_block_buf()
> arm64/mm: Enable vmem_altmap support for vmemmap mappings
>
> arch/arm64/mm/mmu.c | 59 ++++++++++++++++++++++++++-------------
> arch/ia64/mm/discontig.c | 2 +-
> arch/powerpc/mm/init_64.c | 10 +++----
> arch/riscv/mm/init.c | 2 +-
> arch/x86/mm/init_64.c | 12 ++++----
> include/linux/mm.h | 8 ++++--
> mm/sparse-vmemmap.c | 38 ++++++++++++++++++++-----
> 7 files changed, 87 insertions(+), 44 deletions(-)
>
> --
> 2.20.1
>
--
Sincerely yours,
Mike.
On 06/18/2020 02:26 PM, Mike Rapoport wrote:
> On Thu, Jun 18, 2020 at 06:45:27AM +0530, Anshuman Khandual wrote:
>> This series enables vmemmap backing memory allocation from device memory
>> ranges on arm64. But before that, it enables vmemmap_populate_basepages()
>> and vmemmap_alloc_block_buf() to accommodate struct vmem_altmap based
>> alocation requests.
>>
>> This series applies on 5.8-rc1.
>>
>> Pending Question:
>>
>> altmap_alloc_block_buf() does not have any other remaining users in
>> the tree after this change. Should it be converted into a static
>> function and it's declaration be dropped from the header
>> (include/linux/mm.h). Avoided doing so because I was not sure if there
>> are any off-tree users or not.
>
> Well, off-tree users probably have an active fork anyway so they could
> switch to vmemmap_alloc_block_buf()...
Sure, will make the function a static and remove it's declaration
from the header.
>
> Regardless, can you please update Documentation/vm/memory-model.rst to
> keep it in sync with the code?
Sure, will do.
On Thu, Jun 18, 2020 at 06:45:29AM +0530, Anshuman Khandual wrote:
> There are many instances where vmemap allocation is often switched between
> regular memory and device memory just based on whether altmap is available
> or not. vmemmap_alloc_block_buf() is used in various platforms to allocate
> vmemmap mappings. Lets also enable it to handle altmap based device memory
> allocation along with existing regular memory allocations. This will help
> in avoiding the altmap based allocation switch in many places.
>
> While here also implement a regular memory allocation fallback mechanism
> when the first preferred device memory allocation fails. This will ensure
> preserving the existing semantics on powerpc platform. To summarize there
> are three different methods to call vmemmap_alloc_block_buf().
>
> (., NULL, false) /* Allocate from system RAM */
> (., altmap, false) /* Allocate from altmap without any fallback */
> (., altmap, true) /* Allocate from altmap with fallback (system RAM) */
[...]
> diff --git a/arch/powerpc/mm/init_64.c b/arch/powerpc/mm/init_64.c
> index bc73abf0bc25..01e25b56eccb 100644
> --- a/arch/powerpc/mm/init_64.c
> +++ b/arch/powerpc/mm/init_64.c
> @@ -225,12 +225,12 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node,
> * fall back to system memory if the altmap allocation fail.
> */
> if (altmap && !altmap_cross_boundary(altmap, start, page_size)) {
> - p = altmap_alloc_block_buf(page_size, altmap);
> - if (!p)
> - pr_debug("altmap block allocation failed, falling back to system memory");
> + p = vmemmap_alloc_block_buf(page_size, node,
> + altmap, true);
> + } else {
> + p = vmemmap_alloc_block_buf(page_size, node,
> + NULL, false);
> }
> - if (!p)
> - p = vmemmap_alloc_block_buf(page_size, node);
> if (!p)
> return -ENOMEM;
Is the fallback argument actually necessary. It may be cleaner to just
leave the code as is with the choice between altmap and NULL. If an arch
needs a fallback (only powerpc), they have the fallback in place
already. I don't see the powerpc code any better after this change.
I'm fine with the altmap argument though.
--
Catalin
On 07/02/2020 07:37 PM, Catalin Marinas wrote:
> On Thu, Jun 18, 2020 at 06:45:29AM +0530, Anshuman Khandual wrote:
>> There are many instances where vmemap allocation is often switched between
>> regular memory and device memory just based on whether altmap is available
>> or not. vmemmap_alloc_block_buf() is used in various platforms to allocate
>> vmemmap mappings. Lets also enable it to handle altmap based device memory
>> allocation along with existing regular memory allocations. This will help
>> in avoiding the altmap based allocation switch in many places.
>>
>> While here also implement a regular memory allocation fallback mechanism
>> when the first preferred device memory allocation fails. This will ensure
>> preserving the existing semantics on powerpc platform. To summarize there
>> are three different methods to call vmemmap_alloc_block_buf().
>>
>> (., NULL, false) /* Allocate from system RAM */
>> (., altmap, false) /* Allocate from altmap without any fallback */
>> (., altmap, true) /* Allocate from altmap with fallback (system RAM) */
> [...]
>> diff --git a/arch/powerpc/mm/init_64.c b/arch/powerpc/mm/init_64.c
>> index bc73abf0bc25..01e25b56eccb 100644
>> --- a/arch/powerpc/mm/init_64.c
>> +++ b/arch/powerpc/mm/init_64.c
>> @@ -225,12 +225,12 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node,
>> * fall back to system memory if the altmap allocation fail.
>> */
>> if (altmap && !altmap_cross_boundary(altmap, start, page_size)) {
>> - p = altmap_alloc_block_buf(page_size, altmap);
>> - if (!p)
>> - pr_debug("altmap block allocation failed, falling back to system memory");
>> + p = vmemmap_alloc_block_buf(page_size, node,
>> + altmap, true);
>> + } else {
>> + p = vmemmap_alloc_block_buf(page_size, node,
>> + NULL, false);
>> }
>> - if (!p)
>> - p = vmemmap_alloc_block_buf(page_size, node);
>> if (!p)
>> return -ENOMEM;
>
> Is the fallback argument actually necessary. It may be cleaner to just
> leave the code as is with the choice between altmap and NULL. If an arch
> needs a fallback (only powerpc), they have the fallback in place
> already. I don't see the powerpc code any better after this change.
>
> I'm fine with the altmap argument though.
Okay. Will drop 'fallback' from vmemmap_alloc_block_buf() and update the
callers. There will also be a single change in the subsequent patch i.e
vmemmap_alloc_block_buf(PMD_SIZE, node, altmap).
Catalin Marinas <[email protected]> writes:
> On Thu, Jun 18, 2020 at 06:45:29AM +0530, Anshuman Khandual wrote:
>> There are many instances where vmemap allocation is often switched between
>> regular memory and device memory just based on whether altmap is available
>> or not. vmemmap_alloc_block_buf() is used in various platforms to allocate
>> vmemmap mappings. Lets also enable it to handle altmap based device memory
>> allocation along with existing regular memory allocations. This will help
>> in avoiding the altmap based allocation switch in many places.
>>
>> While here also implement a regular memory allocation fallback mechanism
>> when the first preferred device memory allocation fails. This will ensure
>> preserving the existing semantics on powerpc platform. To summarize there
>> are three different methods to call vmemmap_alloc_block_buf().
>>
>> (., NULL, false) /* Allocate from system RAM */
>> (., altmap, false) /* Allocate from altmap without any fallback */
>> (., altmap, true) /* Allocate from altmap with fallback (system RAM) */
> [...]
>> diff --git a/arch/powerpc/mm/init_64.c b/arch/powerpc/mm/init_64.c
>> index bc73abf0bc25..01e25b56eccb 100644
>> --- a/arch/powerpc/mm/init_64.c
>> +++ b/arch/powerpc/mm/init_64.c
>> @@ -225,12 +225,12 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node,
>> * fall back to system memory if the altmap allocation fail.
>> */
>> if (altmap && !altmap_cross_boundary(altmap, start, page_size)) {
>> - p = altmap_alloc_block_buf(page_size, altmap);
>> - if (!p)
>> - pr_debug("altmap block allocation failed, falling back to system memory");
>> + p = vmemmap_alloc_block_buf(page_size, node,
>> + altmap, true);
>> + } else {
>> + p = vmemmap_alloc_block_buf(page_size, node,
>> + NULL, false);
>> }
>> - if (!p)
>> - p = vmemmap_alloc_block_buf(page_size, node);
>> if (!p)
>> return -ENOMEM;
>
> Is the fallback argument actually necessary. It may be cleaner to just
> leave the code as is with the choice between altmap and NULL. If an arch
> needs a fallback (only powerpc), they have the fallback in place
> already. I don't see the powerpc code any better after this change.
Yeah I agree.
cheers