LinuxLists.cc - [PATCH v4 0/4] MIPS: Modify mem= and memmap= parameter

[permalink] [raw]

Subject: Re: [PATCH v4 0/4] MIPS: Modify mem= and memmap= parameter

Hi,

On Tue, Mar 01, 2022 at 12:28:57PM +0800, Tiezhu Yang wrote:
> In the current code, the kernel command-line parameter mem= and memmap=
> can not work well on MIPS, this patchset refactors the related code to
> fix them.
>
> For kdump on MIPS, if the users want to limit the memory region for the
> capture kernel to avoid corrupting the memory image of the panic kernel,
> use the parameter memmap=limit@base is the proper way, I will submit a
> patch to use memmap=limit@base for kexec-tools after this patchset is
> applied.

Sorry, apparently I misread the prevoius version.
What's wrong with the current implementation of mem=limit@base for the
kdump case?

> v4: Fix some build warnings reported by kernel test robot
>
> v3: Modify patch #3 to maintain compatibility for memmap=limit{$,#,!}base,
> commented by Mike Rapoport, thank you
>
> v2: Add some new patches to support memmap=limit@base
>
> Tiezhu Yang (4):
> MIPS: Refactor early_parse_mem() to fix mem= parameter
> memblock: Introduce memblock_mem_range_remove_map()
> MIPS: Refactor early_parse_memmap() to fix memmap= parameter
> MIPS: Remove not used variable usermem
>
> arch/mips/kernel/setup.c | 69 ++++++++++++++++++++++--------------------------
> include/linux/memblock.h | 1 +
> mm/memblock.c | 9 +++++--
> 3 files changed, 40 insertions(+), 39 deletions(-)
>
> --
> 2.1.0
>

--
Sincerely yours,
Mike.

2022-03-01 12:11:57

[permalink] [raw]

Subject: Re: [PATCH v4 0/4] MIPS: Modify mem= and memmap= parameter

On 03/01/2022 05:55 PM, Mike Rapoport wrote:
> Hi,
>
> On Tue, Mar 01, 2022 at 12:28:57PM +0800, Tiezhu Yang wrote:
>> In the current code, the kernel command-line parameter mem= and memmap=
>> can not work well on MIPS, this patchset refactors the related code to
>> fix them.
>>
>> For kdump on MIPS, if the users want to limit the memory region for the
>> capture kernel to avoid corrupting the memory image of the panic kernel,
>> use the parameter memmap=limit@base is the proper way, I will submit a
>> patch to use memmap=limit@base for kexec-tools after this patchset is
>> applied.
>
> Sorry, apparently I misread the prevoius version.
> What's wrong with the current implementation of mem=limit@base for the
> kdump case?

In the current code, without this patchset, kernel boot hangs when add
mem=3G, mem=3G@64M or memmap=3G@64M to the command-line, it means that
the parameter mem= and memmap= have bug on mips.

Thanks,
Tiezhu

>
>> v4: Fix some build warnings reported by kernel test robot
>>
>> v3: Modify patch #3 to maintain compatibility for memmap=limit{$,#,!}base,
>> commented by Mike Rapoport, thank you
>>
>> v2: Add some new patches to support memmap=limit@base
>>
>> Tiezhu Yang (4):
>> MIPS: Refactor early_parse_mem() to fix mem= parameter
>> memblock: Introduce memblock_mem_range_remove_map()
>> MIPS: Refactor early_parse_memmap() to fix memmap= parameter
>> MIPS: Remove not used variable usermem
>>
>> arch/mips/kernel/setup.c | 69 ++++++++++++++++++++++--------------------------
>> include/linux/memblock.h | 1 +
>> mm/memblock.c | 9 +++++--
>> 3 files changed, 40 insertions(+), 39 deletions(-)
>>
>> --
>> 2.1.0
>>
>

2022-03-01 17:53:12

[permalink] [raw]

Subject: Re: [PATCH v4 0/4] MIPS: Modify mem= and memmap= parameter

On Tue, Mar 01, 2022 at 07:51:23PM +0800, Tiezhu Yang wrote:
>
>
> On 03/01/2022 05:55 PM, Mike Rapoport wrote:
> > Hi,
> >
> > On Tue, Mar 01, 2022 at 12:28:57PM +0800, Tiezhu Yang wrote:
> > > In the current code, the kernel command-line parameter mem= and memmap=
> > > can not work well on MIPS, this patchset refactors the related code to
> > > fix them.
> > >
> > > For kdump on MIPS, if the users want to limit the memory region for the
> > > capture kernel to avoid corrupting the memory image of the panic kernel,
> > > use the parameter memmap=limit@base is the proper way, I will submit a
> > > patch to use memmap=limit@base for kexec-tools after this patchset is
> > > applied.
> >
> > Sorry, apparently I misread the prevoius version.
> > What's wrong with the current implementation of mem=limit@base for the
> > kdump case?
>
> In the current code, without this patchset, kernel boot hangs when add
> mem=3G, mem=3G@64M or memmap=3G@64M to the command-line, it means that
> the parameter mem= and memmap= have bug on mips.

I can see how mem=3G may be wrong when the memory does not start at 0, but
it seems to do the right thing of mem=3G@64M.

Do you see system hangs with mem=3G@64M?

Do you have the logs before the hang?

As for memmap= option, it does not specify the memory map but rather alters
the memory map passed by the firmware. Particularity in MIPS implementation
it allows to add a single range of available or reserved memory.

AFAIU, for the kdump use-case mem=X@Y should suffice.

> Thanks,
> Tiezhu
>
> >
> > > v4: Fix some build warnings reported by kernel test robot
> > >
> > > v3: Modify patch #3 to maintain compatibility for memmap=limit{$,#,!}base,
> > > commented by Mike Rapoport, thank you
> > >
> > > v2: Add some new patches to support memmap=limit@base
> > >
> > > Tiezhu Yang (4):
> > > MIPS: Refactor early_parse_mem() to fix mem= parameter
> > > memblock: Introduce memblock_mem_range_remove_map()
> > > MIPS: Refactor early_parse_memmap() to fix memmap= parameter
> > > MIPS: Remove not used variable usermem
> > >
> > > arch/mips/kernel/setup.c | 69 ++++++++++++++++++++++--------------------------
> > > include/linux/memblock.h | 1 +
> > > mm/memblock.c | 9 +++++--
> > > 3 files changed, 40 insertions(+), 39 deletions(-)
> > >
> > > --
> > > 2.1.0
> > >
> >
>

--
Sincerely yours,
Mike.

2022-03-02 11:43:41

[permalink] [raw]

Subject: Re: [PATCH v4 0/4] MIPS: Modify mem= and memmap= parameter

On 03/02/2022 04:03 PM, Mike Rapoport wrote:
> On Wed, Mar 02, 2022 at 09:50:49AM +0800, Tiezhu Yang wrote:
>>
>>
>> On 03/01/2022 10:31 PM, Mike Rapoport wrote:
>>> On Tue, Mar 01, 2022 at 07:51:23PM +0800, Tiezhu Yang wrote:
>>>>
>>>>
>>>> On 03/01/2022 05:55 PM, Mike Rapoport wrote:
>>>>> Hi,
>>>>>
>>>>> On Tue, Mar 01, 2022 at 12:28:57PM +0800, Tiezhu Yang wrote:
>>>>>> In the current code, the kernel command-line parameter mem= and memmap=
>>>>>> can not work well on MIPS, this patchset refactors the related code to
>>>>>> fix them.
>>>>>>
>>>>>> For kdump on MIPS, if the users want to limit the memory region for the
>>>>>> capture kernel to avoid corrupting the memory image of the panic kernel,
>>>>>> use the parameter memmap=limit@base is the proper way, I will submit a
>>>>>> patch to use memmap=limit@base for kexec-tools after this patchset is
>>>>>> applied.
>>>>>
>>>>> Sorry, apparently I misread the prevoius version.
>>>>> What's wrong with the current implementation of mem=limit@base for the
>>>>> kdump case?
>>>>
>>>> In the current code, without this patchset, kernel boot hangs when add
>>>> mem=3G, mem=3G@64M or memmap=3G@64M to the command-line, it means that
>>>> the parameter mem= and memmap= have bug on mips.
>>>
>>> I can see how mem=3G may be wrong when the memory does not start at 0, but
>>> it seems to do the right thing of mem=3G@64M.
>>>
>>> Do you see system hangs with mem=3G@64M?
>>
>> Yes.
>>
>>>
>>> Do you have the logs before the hang?
>>
>> Here are the logs:
>>
>> [ 0.000000] Linux version 5.17.0-rc3+ (loongson@linux) (gcc (GCC) 7.3.1
>> 20180303 (Red Hat 7.3.1-6), GNU ld version 2.28-13.fc21.loongson.6) #1 SMP
>> PREEMPT Wed Mar 2 09:07:39 CST 2022
>> [ 0.000000] CpuClock = 1800000000
>> [ 0.000000] The bridge chip is LS7A
>> [ 0.000000] CP0_Config3: CP0 16.3 (0xdc8030a0)
>> [ 0.000000] CP0_PageGrain: CP0 5.1 (0x28000000)
>> [ 0.000000] NUMA: Discovered 4 cpus on 1 nodes
>> [ 0.000000] Node0: mem_type:1, mem_start:0x200000, mem_size:0xee MB
>> [ 0.000000] start_pfn:0x80, end_pfn:0x3c00, num_physpages:0x3b80
>> [ 0.000000] Node0: mem_type:2, mem_start:0x90200000, mem_size:0x6fe MB
>> [ 0.000000] start_pfn:0x24080, end_pfn:0x40000,
>> num_physpages:0x1fb00
>> [ 0.000000] Node0: mem_type:2, mem_start:0x120000000, mem_size:0x1600 MB
>> [ 0.000000] start_pfn:0x48000, end_pfn:0xa0000,
>> num_physpages:0x77b00
>> [ 0.000000] Node0's addrspace_offset is 0x0
>> [ 0.000000] Node0: start_pfn=0x80, end_pfn=0xa0000
>> [ 0.000000] NUMA: set cpumask cpu 0 on node 0
>> [ 0.000000] NUMA: set cpumask cpu 1 on node 0
>> [ 0.000000] NUMA: set cpumask cpu 2 on node 0
>> [ 0.000000] NUMA: set cpumask cpu 3 on node 0
>> [ 0.000000] printk: bootconsole [early0] enabled
>> [ 0.000000] CPU0 revision is: 0014c001 (ICT Loongson-3)
>> [ 0.000000] FPU revision is: 00f70501
>> [ 0.000000] MSA revision is: 00060140
>> [ 0.000000] OF: fdt: No chosen node found, continuing without
>> [ 0.000000] MIPS: machine is loongson,loongson64g-4core-ls7a
>> [ 0.000000] User-defined physical RAM map overwrite
>> [ 0.000000] Kernel sections are not in the memory maps
>> [ 0.000000] Initrd not found or empty - disabling initrd
>
> Can you please also send the log with "memblock=debug" added to the kernel
> command line?

[ 0.000000] Linux version 5.17.0-rc3+ (loongson@linux) (gcc (GCC)
7.3.1 20180303 (Red Hat 7.3.1-6), GNU ld version
2.28-13.fc21.loongson.6) #1 SMP PREEMPT Wed Mar 2 09:07:39 CST 2022
[ 0.000000] CpuClock = 1800000000
[ 0.000000] The bridge chip is LS7A
[ 0.000000] CP0_Config3: CP0 16.3 (0xdc8030a0)
[ 0.000000] CP0_PageGrain: CP0 5.1 (0x28000000)
[ 0.000000] NUMA: Discovered 4 cpus on 1 nodes
[ 0.000000] Node0: mem_type:1, mem_start:0x200000, mem_size:0xee MB
[ 0.000000] start_pfn:0x80, end_pfn:0x3c00, num_physpages:0x3b80
[ 0.000000] Node0: mem_type:2, mem_start:0x90200000, mem_size:0x6fe MB
[ 0.000000] start_pfn:0x24080, end_pfn:0x40000,
num_physpages:0x1fb00
[ 0.000000] Node0: mem_type:2, mem_start:0x120000000, mem_size:0x1600 MB
[ 0.000000] start_pfn:0x48000, end_pfn:0xa0000,
num_physpages:0x77b00
[ 0.000000] Node0's addrspace_offset is 0x0
[ 0.000000] Node0: start_pfn=0x80, end_pfn=0xa0000
[ 0.000000] NUMA: set cpumask cpu 0 on node 0
[ 0.000000] NUMA: set cpumask cpu 1 on node 0
[ 0.000000] NUMA: set cpumask cpu 2 on node 0
[ 0.000000] NUMA: set cpumask cpu 3 on node 0
[ 0.000000] printk: bootconsole [early0] enabled
[ 0.000000] CPU0 revision is: 0014c001 (ICT Loongson-3)
[ 0.000000] FPU revision is: 00f70501
[ 0.000000] MSA revision is: 00060140
[ 0.000000] OF: fdt: No chosen node found, continuing without
[ 0.000000] MIPS: machine is loongson,loongson64g-4core-ls7a
[ 0.000000] User-defined physical RAM map overwrite
[ 0.000000] Kernel sections are not in the memory maps
[ 0.000000] memblock_add: [0x0000000000200000-0x000000000231185f]
setup_arch+0x140/0x794
[ 0.000000] memblock_reserve: [0x0000000001260520-0x0000000001262560]
setup_arch+0x148/0x794
[ 0.000000] Initrd not found or empty - disabling initrd
[ 0.000000] memblock_alloc_try_nid: 8257 bytes align=0x40 nid=-1
from=0x0000000000000000 max_addr=0x0000000000000000
early_init_dt_alloc_memory_arch+0x30/0x60
[ 0.000000] memblock_reserve: [0x0000000004000000-0x0000000004002040]
memblock_alloc_range_nid+0xf0/0x178
[ 0.000000] memblock_alloc_try_nid: 37972 bytes align=0x8 nid=-1
from=0x0000000000000000 max_addr=0x0000000000000000
early_init_dt_alloc_memory_arch+0x30/0x60
[ 0.000000] memblock_reserve: [0x0000000004002048-0x000000000400b49b]
memblock_alloc_range_nid+0xf0/0x178

>
>>> As for memmap= option, it does not specify the memory map but rather alters
>>> the memory map passed by the firmware. Particularity in MIPS implementation
>>> it allows to add a single range of available or reserved memory.
>>>
>>> AFAIU, for the kdump use-case mem=X@Y should suffice.
>>
>> We can modify some code to make mem=X@Y work well,
>> but according to Documentation/admin-guide/kernel-parameters.txt,
>> the common way is mem=X and memmap=X@Y, so mem=X@Y for mips seems
>> odd, the intention of this patchset is to make mem= and memmap=
>> work well and consistent with the other archs.
>
> These options are anyway not consistent on different architectures.
> arm, mips and x86 implement mem= one way and arm64, powerpc and riscv in
> another so there is no common way to use mem=.
>
> Your changes will break the existing systems that use mem= and memmap=
> options because they change the semantics of their MIPS implementation.

Thank you, I know your opinion.
Let's wait for Thomas's review to get some more feedbacks.

Thanks,
Tiezhu

>
> For kexec/kdump use-cases modern architectures usually do not pass mem= but
> rather prepare the memory map for the kexeced kernel to use. I believe this
> would be the right solution.
>
>> Thanks,
>> Tiezhu
>>
>>>
>>>> Thanks,
>>>> Tiezhu
>>>>
>>>>>
>>>>>> v4: Fix some build warnings reported by kernel test robot
>>>>>>
>>>>>> v3: Modify patch #3 to maintain compatibility for memmap=limit{$,#,!}base,
>>>>>> commented by Mike Rapoport, thank you
>>>>>>
>>>>>> v2: Add some new patches to support memmap=limit@base
>>>>>>
>>>>>> Tiezhu Yang (4):
>>>>>> MIPS: Refactor early_parse_mem() to fix mem= parameter
>>>>>> memblock: Introduce memblock_mem_range_remove_map()
>>>>>> MIPS: Refactor early_parse_memmap() to fix memmap= parameter
>>>>>> MIPS: Remove not used variable usermem
>>>>>>
>>>>>> arch/mips/kernel/setup.c | 69 ++++++++++++++++++++++--------------------------
>>>>>> include/linux/memblock.h | 1 +
>>>>>> mm/memblock.c | 9 +++++--
>>>>>> 3 files changed, 40 insertions(+), 39 deletions(-)
>>>>>>
>>>>>> --
>>>>>> 2.1.0
>>>>>>
>>>>>
>>>>
>>>
>>
>

2022-03-02 14:10:15

[permalink] [raw]

Subject: Re: [PATCH v4 0/4] MIPS: Modify mem= and memmap= parameter

On 03/01/2022 10:31 PM, Mike Rapoport wrote:
> On Tue, Mar 01, 2022 at 07:51:23PM +0800, Tiezhu Yang wrote:
>>
>>
>> On 03/01/2022 05:55 PM, Mike Rapoport wrote:
>>> Hi,
>>>
>>> On Tue, Mar 01, 2022 at 12:28:57PM +0800, Tiezhu Yang wrote:
>>>> In the current code, the kernel command-line parameter mem= and memmap=
>>>> can not work well on MIPS, this patchset refactors the related code to
>>>> fix them.
>>>>
>>>> For kdump on MIPS, if the users want to limit the memory region for the
>>>> capture kernel to avoid corrupting the memory image of the panic kernel,
>>>> use the parameter memmap=limit@base is the proper way, I will submit a
>>>> patch to use memmap=limit@base for kexec-tools after this patchset is
>>>> applied.
>>>
>>> Sorry, apparently I misread the prevoius version.
>>> What's wrong with the current implementation of mem=limit@base for the
>>> kdump case?
>>
>> In the current code, without this patchset, kernel boot hangs when add
>> mem=3G, mem=3G@64M or memmap=3G@64M to the command-line, it means that
>> the parameter mem= and memmap= have bug on mips.
>
> I can see how mem=3G may be wrong when the memory does not start at 0, but
> it seems to do the right thing of mem=3G@64M.
>
> Do you see system hangs with mem=3G@64M?

Yes.

>
> Do you have the logs before the hang?

Here are the logs:

[ 0.000000] Linux version 5.17.0-rc3+ (loongson@linux) (gcc (GCC)
7.3.1 20180303 (Red Hat 7.3.1-6), GNU ld version
2.28-13.fc21.loongson.6) #1 SMP PREEMPT Wed Mar 2 09:07:39 CST 2022
[ 0.000000] CpuClock = 1800000000
[ 0.000000] The bridge chip is LS7A
[ 0.000000] CP0_Config3: CP0 16.3 (0xdc8030a0)
[ 0.000000] CP0_PageGrain: CP0 5.1 (0x28000000)
[ 0.000000] NUMA: Discovered 4 cpus on 1 nodes
[ 0.000000] Node0: mem_type:1, mem_start:0x200000, mem_size:0xee MB
[ 0.000000] start_pfn:0x80, end_pfn:0x3c00, num_physpages:0x3b80
[ 0.000000] Node0: mem_type:2, mem_start:0x90200000, mem_size:0x6fe MB
[ 0.000000] start_pfn:0x24080, end_pfn:0x40000,
num_physpages:0x1fb00
[ 0.000000] Node0: mem_type:2, mem_start:0x120000000, mem_size:0x1600 MB
[ 0.000000] start_pfn:0x48000, end_pfn:0xa0000,
num_physpages:0x77b00
[ 0.000000] Node0's addrspace_offset is 0x0
[ 0.000000] Node0: start_pfn=0x80, end_pfn=0xa0000
[ 0.000000] NUMA: set cpumask cpu 0 on node 0
[ 0.000000] NUMA: set cpumask cpu 1 on node 0
[ 0.000000] NUMA: set cpumask cpu 2 on node 0
[ 0.000000] NUMA: set cpumask cpu 3 on node 0
[ 0.000000] printk: bootconsole [early0] enabled
[ 0.000000] CPU0 revision is: 0014c001 (ICT Loongson-3)
[ 0.000000] FPU revision is: 00f70501
[ 0.000000] MSA revision is: 00060140
[ 0.000000] OF: fdt: No chosen node found, continuing without
[ 0.000000] MIPS: machine is loongson,loongson64g-4core-ls7a
[ 0.000000] User-defined physical RAM map overwrite
[ 0.000000] Kernel sections are not in the memory maps
[ 0.000000] Initrd not found or empty - disabling initrd

>
> As for memmap= option, it does not specify the memory map but rather alters
> the memory map passed by the firmware. Particularity in MIPS implementation
> it allows to add a single range of available or reserved memory.
>
> AFAIU, for the kdump use-case mem=X@Y should suffice.

We can modify some code to make mem=X@Y work well,
but according to Documentation/admin-guide/kernel-parameters.txt,
the common way is mem=X and memmap=X@Y, so mem=X@Y for mips seems
odd, the intention of this patchset is to make mem= and memmap=
work well and consistent with the other archs.

Thanks,
Tiezhu

>
>> Thanks,
>> Tiezhu
>>
>>>
>>>> v4: Fix some build warnings reported by kernel test robot
>>>>
>>>> v3: Modify patch #3 to maintain compatibility for memmap=limit{$,#,!}base,
>>>> commented by Mike Rapoport, thank you
>>>>
>>>> v2: Add some new patches to support memmap=limit@base
>>>>
>>>> Tiezhu Yang (4):
>>>> MIPS: Refactor early_parse_mem() to fix mem= parameter
>>>> memblock: Introduce memblock_mem_range_remove_map()
>>>> MIPS: Refactor early_parse_memmap() to fix memmap= parameter
>>>> MIPS: Remove not used variable usermem
>>>>
>>>> arch/mips/kernel/setup.c | 69 ++++++++++++++++++++++--------------------------
>>>> include/linux/memblock.h | 1 +
>>>> mm/memblock.c | 9 +++++--
>>>> 3 files changed, 40 insertions(+), 39 deletions(-)
>>>>
>>>> --
>>>> 2.1.0
>>>>
>>>
>>
>

2022-03-02 23:39:17

[permalink] [raw]

Subject: Re: [PATCH v4 0/4] MIPS: Modify mem= and memmap= parameter

On Wed, Mar 02, 2022 at 09:50:49AM +0800, Tiezhu Yang wrote:
>
>
> On 03/01/2022 10:31 PM, Mike Rapoport wrote:
> > On Tue, Mar 01, 2022 at 07:51:23PM +0800, Tiezhu Yang wrote:
> > >
> > >
> > > On 03/01/2022 05:55 PM, Mike Rapoport wrote:
> > > > Hi,
> > > >
> > > > On Tue, Mar 01, 2022 at 12:28:57PM +0800, Tiezhu Yang wrote:
> > > > > In the current code, the kernel command-line parameter mem= and memmap=
> > > > > can not work well on MIPS, this patchset refactors the related code to
> > > > > fix them.
> > > > >
> > > > > For kdump on MIPS, if the users want to limit the memory region for the
> > > > > capture kernel to avoid corrupting the memory image of the panic kernel,
> > > > > use the parameter memmap=limit@base is the proper way, I will submit a
> > > > > patch to use memmap=limit@base for kexec-tools after this patchset is
> > > > > applied.
> > > >
> > > > Sorry, apparently I misread the prevoius version.
> > > > What's wrong with the current implementation of mem=limit@base for the
> > > > kdump case?
> > >
> > > In the current code, without this patchset, kernel boot hangs when add
> > > mem=3G, mem=3G@64M or memmap=3G@64M to the command-line, it means that
> > > the parameter mem= and memmap= have bug on mips.
> >
> > I can see how mem=3G may be wrong when the memory does not start at 0, but
> > it seems to do the right thing of mem=3G@64M.
> >
> > Do you see system hangs with mem=3G@64M?
>
> Yes.
>
> >
> > Do you have the logs before the hang?
>
> Here are the logs:
>
> [ 0.000000] Linux version 5.17.0-rc3+ (loongson@linux) (gcc (GCC) 7.3.1
> 20180303 (Red Hat 7.3.1-6), GNU ld version 2.28-13.fc21.loongson.6) #1 SMP
> PREEMPT Wed Mar 2 09:07:39 CST 2022
> [ 0.000000] CpuClock = 1800000000
> [ 0.000000] The bridge chip is LS7A
> [ 0.000000] CP0_Config3: CP0 16.3 (0xdc8030a0)
> [ 0.000000] CP0_PageGrain: CP0 5.1 (0x28000000)
> [ 0.000000] NUMA: Discovered 4 cpus on 1 nodes
> [ 0.000000] Node0: mem_type:1, mem_start:0x200000, mem_size:0xee MB
> [ 0.000000] start_pfn:0x80, end_pfn:0x3c00, num_physpages:0x3b80
> [ 0.000000] Node0: mem_type:2, mem_start:0x90200000, mem_size:0x6fe MB
> [ 0.000000] start_pfn:0x24080, end_pfn:0x40000,
> num_physpages:0x1fb00
> [ 0.000000] Node0: mem_type:2, mem_start:0x120000000, mem_size:0x1600 MB
> [ 0.000000] start_pfn:0x48000, end_pfn:0xa0000,
> num_physpages:0x77b00
> [ 0.000000] Node0's addrspace_offset is 0x0
> [ 0.000000] Node0: start_pfn=0x80, end_pfn=0xa0000
> [ 0.000000] NUMA: set cpumask cpu 0 on node 0
> [ 0.000000] NUMA: set cpumask cpu 1 on node 0
> [ 0.000000] NUMA: set cpumask cpu 2 on node 0
> [ 0.000000] NUMA: set cpumask cpu 3 on node 0
> [ 0.000000] printk: bootconsole [early0] enabled
> [ 0.000000] CPU0 revision is: 0014c001 (ICT Loongson-3)
> [ 0.000000] FPU revision is: 00f70501
> [ 0.000000] MSA revision is: 00060140
> [ 0.000000] OF: fdt: No chosen node found, continuing without
> [ 0.000000] MIPS: machine is loongson,loongson64g-4core-ls7a
> [ 0.000000] User-defined physical RAM map overwrite
> [ 0.000000] Kernel sections are not in the memory maps
> [ 0.000000] Initrd not found or empty - disabling initrd

Can you please also send the log with "memblock=debug" added to the kernel
command line?

> > As for memmap= option, it does not specify the memory map but rather alters
> > the memory map passed by the firmware. Particularity in MIPS implementation
> > it allows to add a single range of available or reserved memory.
> >
> > AFAIU, for the kdump use-case mem=X@Y should suffice.
>
> We can modify some code to make mem=X@Y work well,
> but according to Documentation/admin-guide/kernel-parameters.txt,
> the common way is mem=X and memmap=X@Y, so mem=X@Y for mips seems
> odd, the intention of this patchset is to make mem= and memmap=
> work well and consistent with the other archs.

These options are anyway not consistent on different architectures.
arm, mips and x86 implement mem= one way and arm64, powerpc and riscv in
another so there is no common way to use mem=.

Your changes will break the existing systems that use mem= and memmap=
options because they change the semantics of their MIPS implementation.

For kexec/kdump use-cases modern architectures usually do not pass mem= but
rather prepare the memory map for the kexeced kernel to use. I believe this
would be the right solution.

> Thanks,
> Tiezhu
>
> >
> > > Thanks,
> > > Tiezhu
> > >
> > > >
> > > > > v4: Fix some build warnings reported by kernel test robot
> > > > >
> > > > > v3: Modify patch #3 to maintain compatibility for memmap=limit{$,#,!}base,
> > > > > commented by Mike Rapoport, thank you
> > > > >
> > > > > v2: Add some new patches to support memmap=limit@base
> > > > >
> > > > > Tiezhu Yang (4):
> > > > > MIPS: Refactor early_parse_mem() to fix mem= parameter
> > > > > memblock: Introduce memblock_mem_range_remove_map()
> > > > > MIPS: Refactor early_parse_memmap() to fix memmap= parameter
> > > > > MIPS: Remove not used variable usermem
> > > > >
> > > > > arch/mips/kernel/setup.c | 69 ++++++++++++++++++++++--------------------------
> > > > > include/linux/memblock.h | 1 +
> > > > > mm/memblock.c | 9 +++++--
> > > > > 3 files changed, 40 insertions(+), 39 deletions(-)
> > > > >
> > > > > --
> > > > > 2.1.0
> > > > >
> > > >
> > >
> >
>

--
Sincerely yours,
Mike.

2022-03-03 00:13:46

[permalink] [raw]

Subject: Re: [PATCH v4 0/4] MIPS: Modify mem= and memmap= parameter

On Wed, Mar 02, 2022 at 05:28:27PM +0800, Tiezhu Yang wrote:
>
> On 03/02/2022 04:03 PM, Mike Rapoport wrote:
> > On Wed, Mar 02, 2022 at 09:50:49AM +0800, Tiezhu Yang wrote:
>
> [ 0.000000] Linux version 5.17.0-rc3+ (loongson@linux) (gcc (GCC) 7.3.1
> 20180303 (Red Hat 7.3.1-6), GNU ld version 2.28-13.fc21.loongson.6) #1 SMP
> PREEMPT Wed Mar 2 09:07:39 CST 2022
> [ 0.000000] CpuClock = 1800000000
> [ 0.000000] The bridge chip is LS7A
> [ 0.000000] CP0_Config3: CP0 16.3 (0xdc8030a0)
> [ 0.000000] CP0_PageGrain: CP0 5.1 (0x28000000)
> [ 0.000000] NUMA: Discovered 4 cpus on 1 nodes
> [ 0.000000] Node0: mem_type:1, mem_start:0x200000, mem_size:0xee MB
> [ 0.000000] start_pfn:0x80, end_pfn:0x3c00, num_physpages:0x3b80
> [ 0.000000] Node0: mem_type:2, mem_start:0x90200000, mem_size:0x6fe MB
> [ 0.000000] start_pfn:0x24080, end_pfn:0x40000,
> num_physpages:0x1fb00
> [ 0.000000] Node0: mem_type:2, mem_start:0x120000000, mem_size:0x1600 MB
> [ 0.000000] start_pfn:0x48000, end_pfn:0xa0000,
> num_physpages:0x77b00
> [ 0.000000] Node0's addrspace_offset is 0x0
> [ 0.000000] Node0: start_pfn=0x80, end_pfn=0xa0000
> [ 0.000000] NUMA: set cpumask cpu 0 on node 0
> [ 0.000000] NUMA: set cpumask cpu 1 on node 0
> [ 0.000000] NUMA: set cpumask cpu 2 on node 0
> [ 0.000000] NUMA: set cpumask cpu 3 on node 0
> [ 0.000000] printk: bootconsole [early0] enabled
> [ 0.000000] CPU0 revision is: 0014c001 (ICT Loongson-3)
> [ 0.000000] FPU revision is: 00f70501
> [ 0.000000] MSA revision is: 00060140
> [ 0.000000] OF: fdt: No chosen node found, continuing without
> [ 0.000000] MIPS: machine is loongson,loongson64g-4core-ls7a
> [ 0.000000] User-defined physical RAM map overwrite
> [ 0.000000] Kernel sections are not in the memory maps
> [ 0.000000] memblock_add: [0x0000000000200000-0x000000000231185f]
> setup_arch+0x140/0x794
> [ 0.000000] memblock_reserve: [0x0000000001260520-0x0000000001262560]
> setup_arch+0x148/0x794
> [ 0.000000] Initrd not found or empty - disabling initrd
> [ 0.000000] memblock_alloc_try_nid: 8257 bytes align=0x40 nid=-1
> from=0x0000000000000000 max_addr=0x0000000000000000
> early_init_dt_alloc_memory_arch+0x30/0x60
> [ 0.000000] memblock_reserve: [0x0000000004000000-0x0000000004002040]
> memblock_alloc_range_nid+0xf0/0x178
> [ 0.000000] memblock_alloc_try_nid: 37972 bytes align=0x8 nid=-1
> from=0x0000000000000000 max_addr=0x0000000000000000
> early_init_dt_alloc_memory_arch+0x30/0x60
> [ 0.000000] memblock_reserve: [0x0000000004002048-0x000000000400b49b]
> memblock_alloc_range_nid+0xf0/0x178

As far as I can tell, the kernel lives in 0x200000 and using mem=3G@64M
removes the memory with the kernel and also makes the kernel think there is
memory between 0x400000 and 0xf000000 while there seem to be a hole up to
0x90200000.

This definitely can be reason for the hangs.

--
Sincerely yours,
Mike.

2022-03-04 17:32:23

[permalink] [raw]

Subject: Re: [PATCH v4 0/4] MIPS: Modify mem= and memmap= parameter

On Wed, 2 Mar 2022, Tiezhu Yang wrote:

> > As for memmap= option, it does not specify the memory map but rather alters
> > the memory map passed by the firmware. Particularity in MIPS implementation
> > it allows to add a single range of available or reserved memory.
> >
> > AFAIU, for the kdump use-case mem=X@Y should suffice.
>
> We can modify some code to make mem=X@Y work well,
> but according to Documentation/admin-guide/kernel-parameters.txt,
> the common way is mem=X and memmap=X@Y, so mem=X@Y for mips seems
> odd, the intention of this patchset is to make mem= and memmap=
> work well and consistent with the other archs.

It is not the MIPS implementation that is odd, it is the others that have
changed the semantics that are.

When I added `mem=...' support to the MIPS platform, back on Dec 11th,
2000, which I needed for a system with with memory holes until I got
proper memory probing implemented, AFAIR the only other implementation was
for the x86 and naturally what I did for the MIPS platform was exactly the
same. It used to be documented too, but the documentation was removed
sometime back in 2003 when someone has changed the x86 semantics for
reasons unknown to me and without letting people working on other
platforms know, so things diverged.

Please review:

<https://lore.kernel.org/linux-mips/[email protected]/>

as it has been already discussed.

If you have a system that hangs with `mem=3G' and which does have
contiguous RAM available for the kernel to use from 0 through to 3GiB,
then please either bisect the problem or try finding the root cause as it
used to work at least those 21 years ago. Conversely if your system does
*not* have such RAM available, then use the correct option(s) instead that
reflect your memory map.

It is preferable that the memory map be determined automatically either
by the firmware and then passed to the kernel somehow, or a device tree
entry, or probed by the kernel itself. You shouldn't have to specify
`mem=...' by hand except for debugging or as a temporary workaround.

For example I have an x86 system that Linux does not how to interrogate
for RAM beyond 64MiB, so I do use `memmap=128M@0' (for legacy reasons the
x86 platform has a special exception to always exclude area between 640K
and 1M from being used even if not explicitly specified, but we do not
have a need for such legacy such legacy concerns with the MIPS port). I
consider it an interim measure however until the kernel has been fixed.

Maciej

2022-03-05 15:03:08

[permalink] [raw]

Subject: Re: [PATCH v4 0/4] MIPS: Modify mem= and memmap= parameter

On 03/05/2022 01:05 AM, Maciej W. Rozycki wrote:
> On Wed, 2 Mar 2022, Tiezhu Yang wrote:
>
>>> As for memmap= option, it does not specify the memory map but rather alters
>>> the memory map passed by the firmware. Particularity in MIPS implementation
>>> it allows to add a single range of available or reserved memory.
>>>
>>> AFAIU, for the kdump use-case mem=X@Y should suffice.
>>
>> We can modify some code to make mem=X@Y work well,
>> but according to Documentation/admin-guide/kernel-parameters.txt,
>> the common way is mem=X and memmap=X@Y, so mem=X@Y for mips seems
>> odd, the intention of this patchset is to make mem= and memmap=
>> work well and consistent with the other archs.
>
> It is not the MIPS implementation that is odd, it is the others that have
> changed the semantics that are.
>
> When I added `mem=...' support to the MIPS platform, back on Dec 11th,
> 2000, which I needed for a system with with memory holes until I got
> proper memory probing implemented, AFAIR the only other implementation was
> for the x86 and naturally what I did for the MIPS platform was exactly the
> same. It used to be documented too, but the documentation was removed
> sometime back in 2003 when someone has changed the x86 semantics for
> reasons unknown to me and without letting people working on other
> platforms know, so things diverged.
>
> Please review:
>
> <https://lore.kernel.org/linux-mips/[email protected]/>
>
> as it has been already discussed.
>
> If you have a system that hangs with `mem=3G' and which does have
> contiguous RAM available for the kernel to use from 0 through to 3GiB,
> then please either bisect the problem or try finding the root cause as it
> used to work at least those 21 years ago. Conversely if your system does
> *not* have such RAM available, then use the correct option(s) instead that
> reflect your memory map.
>
> It is preferable that the memory map be determined automatically either
> by the firmware and then passed to the kernel somehow, or a device tree
> entry, or probed by the kernel itself. You shouldn't have to specify
> `mem=...' by hand except for debugging or as a temporary workaround.
>
> For example I have an x86 system that Linux does not how to interrogate
> for RAM beyond 64MiB, so I do use `memmap=128M@0' (for legacy reasons the
> x86 platform has a special exception to always exclude area between 640K
> and 1M from being used even if not explicitly specified, but we do not
> have a need for such legacy such legacy concerns with the MIPS port). I
> consider it an interim measure however until the kernel has been fixed.
>
> Maciej
>

Hi Mike, Thomas and Maciej,

Thank you very much for your feedbacks and discussions.

To be frank, I think mem= and memmap= are used for debugging and testing
in most cases, the intention of this patchset is to refactor the related
code to make them work well on mips.

Now, if put the current patch #2 as the first patch, and then modify the
current patch #1 to support both mem=limit and mem=limit@base (if @base
is omitted, it is equivalent to mem=limit), the other patches #3 and #4
remain unchanged, make sense?

I will send v5 for your review.

Thanks,
Tiezhu

2022-03-05 20:29:19

[permalink] [raw]

Subject: Re: [PATCH v4 0/4] MIPS: Modify mem= and memmap= parameter

On Sat, Mar 05, 2022 at 07:21:15PM +0000, Maciej W. Rozycki wrote:
> On Sat, 5 Mar 2022, Mike Rapoport wrote:
>
> > > > For example I have an x86 system that Linux does not how to interrogate
> > > > for RAM beyond 64MiB, so I do use `memmap=128M@0' (for legacy reasons the
> > > > x86 platform has a special exception to always exclude area between 640K
> > > > and 1M from being used even if not explicitly specified, but we do not
> > > > have a need for such legacy such legacy concerns with the MIPS port). I
> > > > consider it an interim measure however until the kernel has been fixed.
> > > >
> > > > Maciej
> > > >
> > >
> > > Hi Mike, Thomas and Maciej,
> > >
> > > Thank you very much for your feedbacks and discussions.
> > >
> > > To be frank, I think mem= and memmap= are used for debugging and testing
> > > in most cases, the intention of this patchset is to refactor the related
> > > code to make them work well on mips.
> >
> > mem= works fine on mips and there is no need to change it.
> >
> > If you must supply complex memory layout on the command line, consider
> > implementing support for memmap=exact and multiple memmap= parameters on
> > the kernel command line, like x86 does.
>
> There's nothing to implement as the MIPS port has supported arbitrary
> memory maps since Dec 11th, 2000; that's almost 22 years now. C.f.:
> <https://lore.kernel.org/linux-mips/[email protected]/>,
> <https://git.kernel.org/pub/scm/linux/kernel/git/ralf/linux.git/commit/?id=97b7ae4257ef>.

You are right, and providing mem=X@Y for each contiguous memory range
should work even after 22 years.
I missed the fact that mem= can be repeated several times.

> Sadly commit a09fc446fb6d ("[MIPS] setup.c: use early_param() for early
> command line parsing") removed last pieces of inline documentation; I
> don't know why things like that get approved, but neither I can take
> responsibility.

This is sad indeed, but we still can update the kernel-parameters.txt with
a MIPS paragraph.

> Maciej

--
Sincerely yours,
Mike.

2022-03-06 00:18:48

[permalink] [raw]

Subject: Re: [PATCH v4 0/4] MIPS: Modify mem= and memmap= parameter

On Sat, Mar 05, 2022 at 02:19:41PM +0800, Tiezhu Yang wrote:
>
>
> On 03/05/2022 01:05 AM, Maciej W. Rozycki wrote:
> > On Wed, 2 Mar 2022, Tiezhu Yang wrote:
> >
> > > > As for memmap= option, it does not specify the memory map but rather alters
> > > > the memory map passed by the firmware. Particularity in MIPS implementation
> > > > it allows to add a single range of available or reserved memory.
> > > >
> > > > AFAIU, for the kdump use-case mem=X@Y should suffice.
> > >
> > > We can modify some code to make mem=X@Y work well,
> > > but according to Documentation/admin-guide/kernel-parameters.txt,
> > > the common way is mem=X and memmap=X@Y, so mem=X@Y for mips seems
> > > odd, the intention of this patchset is to make mem= and memmap=
> > > work well and consistent with the other archs.
> >
> > It is not the MIPS implementation that is odd, it is the others that have
> > changed the semantics that are.
> >
> > When I added `mem=...' support to the MIPS platform, back on Dec 11th,
> > 2000, which I needed for a system with with memory holes until I got
> > proper memory probing implemented, AFAIR the only other implementation was
> > for the x86 and naturally what I did for the MIPS platform was exactly the
> > same. It used to be documented too, but the documentation was removed
> > sometime back in 2003 when someone has changed the x86 semantics for
> > reasons unknown to me and without letting people working on other
> > platforms know, so things diverged.
> >
> > Please review:
> >
> > <https://lore.kernel.org/linux-mips/[email protected]/>
> >
> > as it has been already discussed.
> >
> > If you have a system that hangs with `mem=3G' and which does have
> > contiguous RAM available for the kernel to use from 0 through to 3GiB,
> > then please either bisect the problem or try finding the root cause as it
> > used to work at least those 21 years ago. Conversely if your system does
> > *not* have such RAM available, then use the correct option(s) instead that
> > reflect your memory map.
> >
> > It is preferable that the memory map be determined automatically either
> > by the firmware and then passed to the kernel somehow, or a device tree
> > entry, or probed by the kernel itself. You shouldn't have to specify
> > `mem=...' by hand except for debugging or as a temporary workaround.
> >
> > For example I have an x86 system that Linux does not how to interrogate
> > for RAM beyond 64MiB, so I do use `memmap=128M@0' (for legacy reasons the
> > x86 platform has a special exception to always exclude area between 640K
> > and 1M from being used even if not explicitly specified, but we do not
> > have a need for such legacy such legacy concerns with the MIPS port). I
> > consider it an interim measure however until the kernel has been fixed.
> >
> > Maciej
> >
>
> Hi Mike, Thomas and Maciej,
>
> Thank you very much for your feedbacks and discussions.
>
> To be frank, I think mem= and memmap= are used for debugging and testing
> in most cases, the intention of this patchset is to refactor the related
> code to make them work well on mips.

mem= works fine on mips and there is no need to change it.

If you must supply complex memory layout on the command line, consider
implementing support for memmap=exact and multiple memmap= parameters on
the kernel command line, like x86 does.

> Now, if put the current patch #2 as the first patch, and then modify the
> current patch #1 to support both mem=limit and mem=limit@base (if @base
> is omitted, it is equivalent to mem=limit), the other patches #3 and #4
> remain unchanged, make sense?
>
> I will send v5 for your review.
>
> Thanks,
> Tiezhu
>

--
Sincerely yours,
Mike.

2022-03-06 03:20:04

[permalink] [raw]

Subject: Re: [PATCH v4 0/4] MIPS: Modify mem= and memmap= parameter

On Sat, 5 Mar 2022, Mike Rapoport wrote:

> > There's nothing to implement as the MIPS port has supported arbitrary
> > memory maps since Dec 11th, 2000; that's almost 22 years now. C.f.:
> > <https://lore.kernel.org/linux-mips/[email protected]/>,
> > <https://git.kernel.org/pub/scm/linux/kernel/git/ralf/linux.git/commit/?id=97b7ae4257ef>.
>
> You are right, and providing mem=X@Y for each contiguous memory range
> should work even after 22 years.
> I missed the fact that mem= can be repeated several times.

Yep, I have now finally tracked down the original bootstrap log for the
first try of that change:

This DECstation is a DS5000/2x0
Loading R[23]00 MMU routines.
CPU revision is: 00000230
Primary instruction cache 64kb, linesize 4 bytes
Primary data cache 64kb, linesize 4 bytes
Linux version 2.4.0-test5 ([email protected]) (gcc version 2.95.3 19991030 (prerelease)) #16 Sun Aug 13 16:43:22 CEST 2000
PROM-provided physical RAM map:
memory: 06800000 @ 00000000
memory: 00800000 @ 08000000
memory: 00800000 @ 0a000000
User-defined physical RAM map:
memory: 06800000 @ 00000000
memory: 00800000 @ 08000000
memory: 00800000 @ 0a000000
memory: 00800000 @ 0c000000
On node 0 totalpages: 51200
zone(0): 51200 pages.
zone(1): 0 pages.
zone(2): 0 pages.
Kernel command line: rw console=ttyS2 mem=104M@0M mem=8M@128M mem=8M@160M mem=8M@192M
Calibrating delay loop... 39.85 BogoMIPS
Memory: 127852k/204800k available (2403k kernel code, 76948k reserved, 610k data, 72k init)
[...]

NB that system is still in service, with more RAM since added.

> > Sadly commit a09fc446fb6d ("[MIPS] setup.c: use early_param() for early
> > command line parsing") removed last pieces of inline documentation; I
> > don't know why things like that get approved, but neither I can take
> > responsibility.
>
> This is sad indeed, but we still can update the kernel-parameters.txt with
> a MIPS paragraph.

Sure, patches are welcome.

Maciej

2022-03-06 11:41:14

[permalink] [raw]

Subject: Re: [PATCH v4 0/4] MIPS: Modify mem= and memmap= parameter

On Sat, 5 Mar 2022, Maciej W. Rozycki wrote:

> Also to say (in said commit):
>
> "There's no point to rewrite some logic to parse command line
> to pass initrd parameters or to declare a user memory area.
> We could use instead parse_early_param() that does the same
> thing."
>
> is IMHO unfair given that the "rewrite" was there in place almost six
> years before `parse_early_param' even started to exist! Why do people
> assume things have always been like they see them at the time they look?

Self-correction for an incorrect last-moment edit: four years before
rather than six (`parse_early_param' went in upstream shortly before:
<https://git.kernel.org/pub/scm/linux/kernel/git/ralf/linux.git/commit/?id=5145764263ab>).

Maciej

2022-03-07 01:00:10