Commit bcec54bf3118 ("mips: switch to NO_BOOTMEM") broke SGI-IP27
and NUMA enabled loongson3 by doing memblock_set_current_limit()
before max_low_pfn has been evaluated. Both platforms need to do the
memblock_set_current_limit() in platform specific code. For
consistency the call to memblock_set_current_limit() is moved
to the common bootmem_init(), where max_low_pfn is calculated
for non NUMA enabled platforms.
Fixes: bcec54bf3118 ("mips: switch to NO_BOOTMEM")
Signed-off-by: Thomas Bogendoerfer <[email protected]>
---
arch/mips/kernel/setup.c | 18 +++++++++---------
arch/mips/loongson64/loongson-3/numa.c | 1 +
arch/mips/sgi-ip27/ip27-memory.c | 1 +
3 files changed, 11 insertions(+), 9 deletions(-)
diff --git a/arch/mips/kernel/setup.c b/arch/mips/kernel/setup.c
index ea09ed6a80a9..b5167b198231 100644
--- a/arch/mips/kernel/setup.c
+++ b/arch/mips/kernel/setup.c
@@ -576,6 +576,15 @@ static void __init bootmem_init(void)
* Reserve initrd memory if needed.
*/
finalize_initrd();
+
+ /*
+ * Prevent memblock from allocating high memory.
+ * This cannot be done before max_low_pfn is detected, so up
+ * to this point is possible to only reserve physical memory
+ * with memblock_reserve; memblock_alloc* can be used
+ * only after this point
+ */
+ memblock_set_current_limit(PFN_PHYS(max_low_pfn));
}
#endif /* CONFIG_SGI_IP27 */
@@ -854,15 +863,6 @@ static void __init arch_mem_init(char **cmdline_p)
bootmem_init();
- /*
- * Prevent memblock from allocating high memory.
- * This cannot be done before max_low_pfn is detected, so up
- * to this point is possible to only reserve physical memory
- * with memblock_reserve; memblock_alloc* can be used
- * only after this point
- */
- memblock_set_current_limit(PFN_PHYS(max_low_pfn));
-
#ifdef CONFIG_PROC_VMCORE
if (setup_elfcorehdr && setup_elfcorehdr_size) {
printk(KERN_INFO "kdump reserved memory at %lx-%lx\n",
diff --git a/arch/mips/loongson64/loongson-3/numa.c b/arch/mips/loongson64/loongson-3/numa.c
index 622761878cd1..5593a8e3cd88 100644
--- a/arch/mips/loongson64/loongson-3/numa.c
+++ b/arch/mips/loongson64/loongson-3/numa.c
@@ -265,6 +265,7 @@ void __init paging_init(void)
zones_size[ZONE_DMA32] = MAX_DMA32_PFN;
#endif
zones_size[ZONE_NORMAL] = max_low_pfn;
+ memblock_set_current_limit(PFN_PHYS(max_low_pfn));
free_area_init_nodes(zones_size);
}
diff --git a/arch/mips/sgi-ip27/ip27-memory.c b/arch/mips/sgi-ip27/ip27-memory.c
index d8b8444d6795..0d0deeae1f47 100644
--- a/arch/mips/sgi-ip27/ip27-memory.c
+++ b/arch/mips/sgi-ip27/ip27-memory.c
@@ -468,6 +468,7 @@ void __init paging_init(void)
max_low_pfn = end_pfn;
}
zones_size[ZONE_NORMAL] = max_low_pfn;
+ memblock_set_current_limit(PFN_PHYS(max_low_pfn));
free_area_init_nodes(zones_size);
}
--
2.13.7
On Thu, Nov 08, 2018 at 03:44:28PM +0100, Thomas Bogendoerfer wrote:
> Commit bcec54bf3118 ("mips: switch to NO_BOOTMEM") broke SGI-IP27
> and NUMA enabled loongson3 by doing memblock_set_current_limit()
> before max_low_pfn has been evaluated. Both platforms need to do the
> memblock_set_current_limit() in platform specific code. For
> consistency the call to memblock_set_current_limit() is moved
> to the common bootmem_init(), where max_low_pfn is calculated
> for non NUMA enabled platforms.
As far as I can tell loongsoon3 initially sets max_low_pfn in
node_mem_init(0) and then re-evaluates it in paging_init(). So it seems
that in this case the limit would be wrong for a system with more than one
node, but it would be set to some usable value.
As for SGI-IP27, the initialization of max_low_pfn as late as in
paging_init() seems to be broken because it's value is used in
arch_mem_init() and in finalize_initrd() anyway.
AFAIU, both platforms set max_low_pfn to last available pfn, so it seems we
can simply do
max_low_pfn = PFN_PHYS(memblock_end_of_DRAM())
in the prom_meminit() function for both platforms and drop the loop
evaluating max_low_pfn in paging_init().
> Fixes: bcec54bf3118 ("mips: switch to NO_BOOTMEM")
> Signed-off-by: Thomas Bogendoerfer <[email protected]>
> ---
> arch/mips/kernel/setup.c | 18 +++++++++---------
> arch/mips/loongson64/loongson-3/numa.c | 1 +
> arch/mips/sgi-ip27/ip27-memory.c | 1 +
> 3 files changed, 11 insertions(+), 9 deletions(-)
>
> diff --git a/arch/mips/kernel/setup.c b/arch/mips/kernel/setup.c
> index ea09ed6a80a9..b5167b198231 100644
> --- a/arch/mips/kernel/setup.c
> +++ b/arch/mips/kernel/setup.c
> @@ -576,6 +576,15 @@ static void __init bootmem_init(void)
> * Reserve initrd memory if needed.
> */
> finalize_initrd();
> +
> + /*
> + * Prevent memblock from allocating high memory.
> + * This cannot be done before max_low_pfn is detected, so up
> + * to this point is possible to only reserve physical memory
> + * with memblock_reserve; memblock_alloc* can be used
> + * only after this point
> + */
> + memblock_set_current_limit(PFN_PHYS(max_low_pfn));
> }
>
> #endif /* CONFIG_SGI_IP27 */
> @@ -854,15 +863,6 @@ static void __init arch_mem_init(char **cmdline_p)
>
> bootmem_init();
>
> - /*
> - * Prevent memblock from allocating high memory.
> - * This cannot be done before max_low_pfn is detected, so up
> - * to this point is possible to only reserve physical memory
> - * with memblock_reserve; memblock_alloc* can be used
> - * only after this point
> - */
> - memblock_set_current_limit(PFN_PHYS(max_low_pfn));
> -
> #ifdef CONFIG_PROC_VMCORE
> if (setup_elfcorehdr && setup_elfcorehdr_size) {
> printk(KERN_INFO "kdump reserved memory at %lx-%lx\n",
> diff --git a/arch/mips/loongson64/loongson-3/numa.c b/arch/mips/loongson64/loongson-3/numa.c
> index 622761878cd1..5593a8e3cd88 100644
> --- a/arch/mips/loongson64/loongson-3/numa.c
> +++ b/arch/mips/loongson64/loongson-3/numa.c
> @@ -265,6 +265,7 @@ void __init paging_init(void)
> zones_size[ZONE_DMA32] = MAX_DMA32_PFN;
> #endif
> zones_size[ZONE_NORMAL] = max_low_pfn;
> + memblock_set_current_limit(PFN_PHYS(max_low_pfn));
> free_area_init_nodes(zones_size);
> }
>
> diff --git a/arch/mips/sgi-ip27/ip27-memory.c b/arch/mips/sgi-ip27/ip27-memory.c
> index d8b8444d6795..0d0deeae1f47 100644
> --- a/arch/mips/sgi-ip27/ip27-memory.c
> +++ b/arch/mips/sgi-ip27/ip27-memory.c
> @@ -468,6 +468,7 @@ void __init paging_init(void)
> max_low_pfn = end_pfn;
> }
> zones_size[ZONE_NORMAL] = max_low_pfn;
> + memblock_set_current_limit(PFN_PHYS(max_low_pfn));
> free_area_init_nodes(zones_size);
> }
>
> --
> 2.13.7
>
--
Sincerely yours,
Mike.
On Thu, 8 Nov 2018 18:18:23 +0200
Mike Rapoport <[email protected]> wrote:
> On Thu, Nov 08, 2018 at 03:44:28PM +0100, Thomas Bogendoerfer wrote:
> > Commit bcec54bf3118 ("mips: switch to NO_BOOTMEM") broke SGI-IP27
> > and NUMA enabled loongson3 by doing memblock_set_current_limit()
> > before max_low_pfn has been evaluated. Both platforms need to do the
> > memblock_set_current_limit() in platform specific code. For
> > consistency the call to memblock_set_current_limit() is moved
> > to the common bootmem_init(), where max_low_pfn is calculated
> > for non NUMA enabled platforms.
> [..]
>
> As for SGI-IP27, the initialization of max_low_pfn as late as in
> paging_init() seems to be broken because it's value is used in
> arch_mem_init() and in finalize_initrd() anyway.
well, the patch is tested on real hardware and the first caller of
a memblock_alloc* function is in a function called by free_area_init_nodes().
> AFAIU, both platforms set max_low_pfn to last available pfn, so it seems we
> can simply do
>
> max_low_pfn = PFN_PHYS(memblock_end_of_DRAM())
>
> in the prom_meminit() function for both platforms and drop the loop
> evaluating max_low_pfn in paging_init().
sounds like a better plan. I'll prepare a new patch.
Thomas.
--
SUSE Linux GmbH
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
On November 8, 2018 6:52:17 PM GMT+02:00, Thomas Bogendoerfer <[email protected]> wrote:
>On Thu, 8 Nov 2018 18:18:23 +0200
>Mike Rapoport <[email protected]> wrote:
>
>> On Thu, Nov 08, 2018 at 03:44:28PM +0100, Thomas Bogendoerfer wrote:
>> > Commit bcec54bf3118 ("mips: switch to NO_BOOTMEM") broke SGI-IP27
>> > and NUMA enabled loongson3 by doing memblock_set_current_limit()
>> > before max_low_pfn has been evaluated. Both platforms need to do
>the
>> > memblock_set_current_limit() in platform specific code. For
>> > consistency the call to memblock_set_current_limit() is moved
>> > to the common bootmem_init(), where max_low_pfn is calculated
>> > for non NUMA enabled platforms.
>> [..]
>>
>> As for SGI-IP27, the initialization of max_low_pfn as late as in
>> paging_init() seems to be broken because it's value is used in
>> arch_mem_init() and in finalize_initrd() anyway.
>
>well, the patch is tested on real hardware and the first caller of
>a memblock_alloc* function is in a function called by
>free_area_init_nodes().
Then, apparently, I've missed something else.
The Onyx2 I worked on is dead for a couple of years now ;-)
>> AFAIU, both platforms set max_low_pfn to last available pfn, so it
>seems we
>> can simply do
>>
>> max_low_pfn = PFN_PHYS(memblock_end_of_DRAM())
>>
Should have been PHYS_PFN, sorry.
>> in the prom_meminit() function for both platforms and drop the loop
>> evaluating max_low_pfn in paging_init().
>
>sounds like a better plan. I'll prepare a new patch.
>
>Thomas.
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
Hi,
It seems the patch below can solve many problems after switched to NO_BOOTMEM, because the memory allocation behavior is more similar as before.
diff --git a/arch/mips/kernel/setup.c b/arch/mips/kernel/setup.c
index 070234b..7a449d9 100644
--- a/arch/mips/kernel/setup.c
+++ b/arch/mips/kernel/setup.c
@@ -839,6 +839,7 @@ static void __init arch_mem_init(char **cmdline_p)
/* call board setup routine */
plat_mem_setup();
+ memblock_set_bottom_up(true);
/*
* Make sure all kernel memory is in the maps. The "UP" and
diff --git a/arch/mips/kernel/traps.c b/arch/mips/kernel/traps.c
index 0f852e1..15e103c 100644
--- a/arch/mips/kernel/traps.c
+++ b/arch/mips/kernel/traps.c
@@ -2260,10 +2260,8 @@ void __init trap_init(void)
unsigned long size = 0x200 + VECTORSPACING*64;
phys_addr_t ebase_pa;
- memblock_set_bottom_up(true);
ebase = (unsigned long)
memblock_alloc_from(size, 1 << fls(size), 0);
- memblock_set_bottom_up(false);
/*
* Try to ensure ebase resides in KSeg0 if possible.
@@ -2307,6 +2305,7 @@ void __init trap_init(void)
if (board_ebase_setup)
board_ebase_setup();
per_cpu_trap_init(true);
+ memblock_set_bottom_up(false);
/*
* Copy the generic exception handlers to their final destination.
------------------ Original ------------------
From: "Mike Rapoport"<[email protected]>;
Date: Fri, Nov 9, 2018 02:01 AM
To: "Thomas Bogendoerfer"<[email protected]>;
Cc: "Ralf Baechle"<[email protected]>; "Paul Burton"<[email protected]>; "James Hogan"<[email protected]>; "Huacai Chen"<[email protected]>; "linux-mips"<[email protected]>; "linux-kernel"<[email protected]>; "rppt"<[email protected]>;
Subject: Re: [[PATCH]] mips: Fix switch to NO_BOOTMEM for SGI-IP27/loongons3 NUMA
On November 8, 2018 6:52:17 PM GMT+02:00, Thomas Bogendoerfer <[email protected]> wrote:
>On Thu, 8 Nov 2018 18:18:23 +0200
>Mike Rapoport <[email protected]> wrote:
>
>> On Thu, Nov 08, 2018 at 03:44:28PM +0100, Thomas Bogendoerfer wrote:
>> > Commit bcec54bf3118 ("mips: switch to NO_BOOTMEM") broke SGI-IP27
>> > and NUMA enabled loongson3 by doing memblock_set_current_limit()
>> > before max_low_pfn has been evaluated. Both platforms need to do
>the
>> > memblock_set_current_limit() in platform specific code. For
>> > consistency the call to memblock_set_current_limit() is moved
>> > to the common bootmem_init(), where max_low_pfn is calculated
>> > for non NUMA enabled platforms.
>> [..]
>>
>> As for SGI-IP27, the initialization of max_low_pfn as late as in
>> paging_init() seems to be broken because it's value is used in
>> arch_mem_init() and in finalize_initrd() anyway.
>
>well, the patch is tested on real hardware and the first caller of
>a memblock_alloc* function is in a function called by
>free_area_init_nodes().
Then, apparently, I've missed something else.
The Onyx2 I worked on is dead for a couple of years now ;-)
>> AFAIU, both platforms set max_low_pfn to last available pfn, so it
>seems we
>> can simply do
>>
>> max_low_pfn = PFN_PHYS(memblock_end_of_DRAM())
>>
Should have been PHYS_PFN, sorry.
>> in the prom_meminit() function for both platforms and drop the loop
>> evaluating max_low_pfn in paging_init().
>
>sounds like a better plan. I'll prepare a new patch.
>
>Thomas.
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
On Fri, Nov 09, 2018 at 10:54:26AM +0800, 陈华才 wrote:
> Hi,
>
> It seems the patch below can solve many problems after switched to NO_BOOTMEM, because the memory allocation behavior is more similar as before.
Yes, this should work.
Still, simplifying the max_low_pfn evaluation has it's value regardless.
> diff --git a/arch/mips/kernel/setup.c b/arch/mips/kernel/setup.c
> index 070234b..7a449d9 100644
> --- a/arch/mips/kernel/setup.c
> +++ b/arch/mips/kernel/setup.c
> @@ -839,6 +839,7 @@ static void __init arch_mem_init(char **cmdline_p)
>
> /* call board setup routine */
> plat_mem_setup();
> + memblock_set_bottom_up(true);
>
> /*
> * Make sure all kernel memory is in the maps. The "UP" and
> diff --git a/arch/mips/kernel/traps.c b/arch/mips/kernel/traps.c
> index 0f852e1..15e103c 100644
> --- a/arch/mips/kernel/traps.c
> +++ b/arch/mips/kernel/traps.c
> @@ -2260,10 +2260,8 @@ void __init trap_init(void)
> unsigned long size = 0x200 + VECTORSPACING*64;
> phys_addr_t ebase_pa;
>
> - memblock_set_bottom_up(true);
> ebase = (unsigned long)
> memblock_alloc_from(size, 1 << fls(size), 0);
> - memblock_set_bottom_up(false);
>
> /*
> * Try to ensure ebase resides in KSeg0 if possible.
> @@ -2307,6 +2305,7 @@ void __init trap_init(void)
> if (board_ebase_setup)
> board_ebase_setup();
> per_cpu_trap_init(true);
> + memblock_set_bottom_up(false);
>
> /*
> * Copy the generic exception handlers to their final destination.
>
>
> ------------------ Original ------------------
> From: "Mike Rapoport"<[email protected]>;
> Date: Fri, Nov 9, 2018 02:01 AM
> To: "Thomas Bogendoerfer"<[email protected]>;
> Cc: "Ralf Baechle"<[email protected]>; "Paul Burton"<[email protected]>; "James Hogan"<[email protected]>; "Huacai Chen"<[email protected]>; "linux-mips"<[email protected]>; "linux-kernel"<[email protected]>; "rppt"<[email protected]>;
> Subject: Re: [[PATCH]] mips: Fix switch to NO_BOOTMEM for SGI-IP27/loongons3 NUMA
>
>
>
> On November 8, 2018 6:52:17 PM GMT+02:00, Thomas Bogendoerfer <[email protected]> wrote:
> >On Thu, 8 Nov 2018 18:18:23 +0200
> >Mike Rapoport <[email protected]> wrote:
> >
> >> On Thu, Nov 08, 2018 at 03:44:28PM +0100, Thomas Bogendoerfer wrote:
> >> > Commit bcec54bf3118 ("mips: switch to NO_BOOTMEM") broke SGI-IP27
> >> > and NUMA enabled loongson3 by doing memblock_set_current_limit()
> >> > before max_low_pfn has been evaluated. Both platforms need to do
> >the
> >> > memblock_set_current_limit() in platform specific code. For
> >> > consistency the call to memblock_set_current_limit() is moved
> >> > to the common bootmem_init(), where max_low_pfn is calculated
> >> > for non NUMA enabled platforms.
> >> [..]
> >>
> >> As for SGI-IP27, the initialization of max_low_pfn as late as in
> >> paging_init() seems to be broken because it's value is used in
> >> arch_mem_init() and in finalize_initrd() anyway.
> >
> >well, the patch is tested on real hardware and the first caller of
> >a memblock_alloc* function is in a function called by
> >free_area_init_nodes().
>
> Then, apparently, I've missed something else.
> The Onyx2 I worked on is dead for a couple of years now ;-)
>
> >> AFAIU, both platforms set max_low_pfn to last available pfn, so it
> >seems we
> >> can simply do
> >>
> >> max_low_pfn = PFN_PHYS(memblock_end_of_DRAM())
> >>
>
> Should have been PHYS_PFN, sorry.
>
> >> in the prom_meminit() function for both platforms and drop the loop
> >> evaluating max_low_pfn in paging_init().
> >
> >sounds like a better plan. I'll prepare a new patch.
> >
> >Thomas.
>
> --
> Sent from my Android device with K-9 Mail. Please excuse my brevity.
--
Sincerely yours,
Mike.