On riscv, the current crash kernel allocation logic is trying to
allocate within 32bit addressible memory region by default, if
failed, try to allocate without 4G restriction.
In need of saving DMA zone memory while allocating a relatively large
crash kernel region, allocating the reserved memory top down in
high memory, without overlapping the DMA zone, is a mature solution.
Hence this patchset introduces the parameter option crashkernel=X,[high,low].
One can reserve the crash kernel from high memory above DMA zone range
by explicitly passing "crashkernel=X,high"; or reserve a memory range
below 4G with "crashkernel=X,low". Besides, there are few rules need
to take notice:
1. "crashkernel=X,[high,low]" will be ignored if "crashkernel=size"
is specified.
2. "crashkernel=X,low" is valid only when "crashkernel=X,high" is passed
and there is enough memory to be allocated under 4G.
3. When allocating crashkernel above 4G and no "crashkernel=X,low" is
specified, a 128M low memory will be allocated automatically for
swiotlb bounce buffer.
See Documentation/admin-guide/kernel-parameters.txt for more information.
To verify loading the crashkernel, adapted kexec-tools is attached below:
https://github.com/chenjh005/kexec-tools/tree/build-test-riscv-v2
Following test cases have been performed as expected:
1) crashkernel=256M //low=256M
2) crashkernel=1G //low=1G
3) crashkernel=4G //high=4G, low=128M(default)
4) crashkernel=4G crashkernel=256M,high //high=4G, low=128M(default), high is ignored
5) crashkernel=4G crashkernel=256M,low //high=4G, low=128M(default), low is ignored
6) crashkernel=4G,high //high=4G, low=128M(default)
7) crashkernel=256M,low //low=0M, invalid
8) crashkernel=4G,high crashkernel=256M,low //high=4G, low=256M
9) crashkernel=4G,high crashkernel=4G,low //high=0M, low=0M, invalid
10) crashkernel=512M@0xd0000000 //low=512M
Changes since [v6]:
1. Introduce the "high" flag to mark whether "crashkernel=X,high"
is passed. Fix the retrying logic between "crashkernel=X,high"
case and others when the first allocation attempt fails.
Changes since [v5]:
1. Update the crashkernel allocation logic when crashkernel=X,high
is specified. In this case, region above 4G will directly get
reserved as crashkernel, rather than trying lower 32bit allocation
first.
Changes since [v4]:
1. Update some imprecise code comments for cmdline parsing.
Changes since [v3]:
1. Update to print warning and return explicitly on failure when
crashkernel=size@offset is specified. Not changing the result
in this case but making the logic more straightforward.
2. Some minor cleanup.
Changes since [v2]:
1. Update the allocation logic to ensure the high crashkernel
region is reserved strictly above dma32_phys_limit.
2. Clean up some minor format problems.
Chen Jiahao (2):
riscv: kdump: Implement crashkernel=X,[high,low]
docs: kdump: Update the crashkernel description for riscv
.../admin-guide/kernel-parameters.txt | 15 ++--
arch/riscv/kernel/setup.c | 5 ++
arch/riscv/mm/init.c | 84 +++++++++++++++++--
3 files changed, 90 insertions(+), 14 deletions(-)
--
2.34.1
Now "crashkernel=" parameter on riscv has been updated to support
crashkernel=X,[high,low]. Through which we can reserve memory region
above/within 32bit addressible DMA zone.
Here update the parameter description accordingly.
Signed-off-by: Chen Jiahao <[email protected]>
Reviewed-by: Guo Ren <[email protected]>
Reviewed-by: Simon Horman <[email protected]>
Reviewed-by: Zhen Lei <[email protected]>
---
Documentation/admin-guide/kernel-parameters.txt | 15 ++++++++-------
1 file changed, 8 insertions(+), 7 deletions(-)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 24477a616cd2..991f436ee39d 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -862,7 +862,7 @@
memory region [offset, offset + size] for that kernel
image. If '@offset' is omitted, then a suitable offset
is selected automatically.
- [KNL, X86-64, ARM64] Select a region under 4G first, and
+ [KNL, X86-64, ARM64, RISCV] Select a region under 4G first, and
fall back to reserve region above 4G when '@offset'
hasn't been specified.
See Documentation/admin-guide/kdump/kdump.rst for further details.
@@ -875,14 +875,14 @@
Documentation/admin-guide/kdump/kdump.rst for an example.
crashkernel=size[KMG],high
- [KNL, X86-64, ARM64] range could be above 4G. Allow kernel
- to allocate physical memory region from top, so could
- be above 4G if system have more than 4G ram installed.
- Otherwise memory region will be allocated below 4G, if
- available.
+ [KNL, X86-64, ARM64, RISCV] range could be above 4G.
+ Allow kernel to allocate physical memory region from top,
+ so could be above 4G if system have more than 4G ram
+ installed. Otherwise memory region will be allocated
+ below 4G, if available.
It will be ignored if crashkernel=X is specified.
crashkernel=size[KMG],low
- [KNL, X86-64, ARM64] range under 4G. When crashkernel=X,high
+ [KNL, X86-64, ARM64, RISCV] range under 4G. When crashkernel=X,high
is passed, kernel could allocate physical memory region
above 4G, that cause second kernel crash on system
that require some amount of low memory, e.g. swiotlb
@@ -893,6 +893,7 @@
size is platform dependent.
--> x86: max(swiotlb_size_or_default() + 8MiB, 256MiB)
--> arm64: 128MiB
+ --> riscv: 128MiB
This one lets the user specify own low range under 4G
for second kernel instead.
0: to disable low allocation.
--
2.34.1
On Sat, Jul 1, 2023 at 5:12 PM Chen Jiahao <[email protected]> wrote:
>
> On riscv, the current crash kernel allocation logic is trying to
> allocate within 32bit addressible memory region by default, if
> failed, try to allocate without 4G restriction.
>
> In need of saving DMA zone memory while allocating a relatively large
> crash kernel region, allocating the reserved memory top down in
> high memory, without overlapping the DMA zone, is a mature solution.
> Hence this patchset introduces the parameter option crashkernel=X,[high,low].
>
> One can reserve the crash kernel from high memory above DMA zone range
> by explicitly passing "crashkernel=X,high"; or reserve a memory range
> below 4G with "crashkernel=X,low". Besides, there are few rules need
> to take notice:
> 1. "crashkernel=X,[high,low]" will be ignored if "crashkernel=size"
> is specified.
> 2. "crashkernel=X,low" is valid only when "crashkernel=X,high" is passed
> and there is enough memory to be allocated under 4G.
> 3. When allocating crashkernel above 4G and no "crashkernel=X,low" is
> specified, a 128M low memory will be allocated automatically for
> swiotlb bounce buffer.
> See Documentation/admin-guide/kernel-parameters.txt for more information.
>
> To verify loading the crashkernel, adapted kexec-tools is attached below:
> https://github.com/chenjh005/kexec-tools/tree/build-test-riscv-v2
>
> Following test cases have been performed as expected:
> 1) crashkernel=256M //low=256M
> 2) crashkernel=1G //low=1G
Have you tried 1GB memory? we found a pud mapping problem on Sv39 of kexec, See:
https://lore.kernel.org/linux-riscv/[email protected]/
> 3) crashkernel=4G //high=4G, low=128M(default)
> 4) crashkernel=4G crashkernel=256M,high //high=4G, low=128M(default), high is ignored
> 5) crashkernel=4G crashkernel=256M,low //high=4G, low=128M(default), low is ignored
> 6) crashkernel=4G,high //high=4G, low=128M(default)
> 7) crashkernel=256M,low //low=0M, invalid
> 8) crashkernel=4G,high crashkernel=256M,low //high=4G, low=256M
> 9) crashkernel=4G,high crashkernel=4G,low //high=0M, low=0M, invalid
> 10) crashkernel=512M@0xd0000000 //low=512M
>
> Changes since [v6]:
> 1. Introduce the "high" flag to mark whether "crashkernel=X,high"
> is passed. Fix the retrying logic between "crashkernel=X,high"
> case and others when the first allocation attempt fails.
>
> Changes since [v5]:
> 1. Update the crashkernel allocation logic when crashkernel=X,high
> is specified. In this case, region above 4G will directly get
> reserved as crashkernel, rather than trying lower 32bit allocation
> first.
>
> Changes since [v4]:
> 1. Update some imprecise code comments for cmdline parsing.
>
> Changes since [v3]:
> 1. Update to print warning and return explicitly on failure when
> crashkernel=size@offset is specified. Not changing the result
> in this case but making the logic more straightforward.
> 2. Some minor cleanup.
>
> Changes since [v2]:
> 1. Update the allocation logic to ensure the high crashkernel
> region is reserved strictly above dma32_phys_limit.
> 2. Clean up some minor format problems.
>
> Chen Jiahao (2):
> riscv: kdump: Implement crashkernel=X,[high,low]
> docs: kdump: Update the crashkernel description for riscv
>
> .../admin-guide/kernel-parameters.txt | 15 ++--
> arch/riscv/kernel/setup.c | 5 ++
> arch/riscv/mm/init.c | 84 +++++++++++++++++--
> 3 files changed, 90 insertions(+), 14 deletions(-)
>
> --
> 2.34.1
>
--
Best Regards
Guo Ren
On 2023/7/1 21:45, Guo Ren wrote:
> On Sat, Jul 1, 2023 at 5:12 PM Chen Jiahao <[email protected]> wrote:
>> On riscv, the current crash kernel allocation logic is trying to
>> allocate within 32bit addressible memory region by default, if
>> failed, try to allocate without 4G restriction.
>>
>> In need of saving DMA zone memory while allocating a relatively large
>> crash kernel region, allocating the reserved memory top down in
>> high memory, without overlapping the DMA zone, is a mature solution.
>> Hence this patchset introduces the parameter option crashkernel=X,[high,low].
>>
>> One can reserve the crash kernel from high memory above DMA zone range
>> by explicitly passing "crashkernel=X,high"; or reserve a memory range
>> below 4G with "crashkernel=X,low". Besides, there are few rules need
>> to take notice:
>> 1. "crashkernel=X,[high,low]" will be ignored if "crashkernel=size"
>> is specified.
>> 2. "crashkernel=X,low" is valid only when "crashkernel=X,high" is passed
>> and there is enough memory to be allocated under 4G.
>> 3. When allocating crashkernel above 4G and no "crashkernel=X,low" is
>> specified, a 128M low memory will be allocated automatically for
>> swiotlb bounce buffer.
>> See Documentation/admin-guide/kernel-parameters.txt for more information.
>>
>> To verify loading the crashkernel, adapted kexec-tools is attached below:
>> https://github.com/chenjh005/kexec-tools/tree/build-test-riscv-v2
>>
>> Following test cases have been performed as expected:
>> 1) crashkernel=256M //low=256M
>> 2) crashkernel=1G //low=1G
> Have you tried 1GB memory? we found a pud mapping problem on Sv39 of kexec, See:
> https://lore.kernel.org/linux-riscv/[email protected]/
I have tested on QEMU with sv57 mmu, so it seems the synchronization problem
was not reproduce when reserving 1G memory and loading the capture kernel.
Thanks,
Jiahao
>
>> 3) crashkernel=4G //high=4G, low=128M(default)
>> 4) crashkernel=4G crashkernel=256M,high //high=4G, low=128M(default), high is ignored
>> 5) crashkernel=4G crashkernel=256M,low //high=4G, low=128M(default), low is ignored
>> 6) crashkernel=4G,high //high=4G, low=128M(default)
>> 7) crashkernel=256M,low //low=0M, invalid
>> 8) crashkernel=4G,high crashkernel=256M,low //high=4G, low=256M
>> 9) crashkernel=4G,high crashkernel=4G,low //high=0M, low=0M, invalid
>> 10) crashkernel=512M@0xd0000000 //low=512M
>>
>> Changes since [v6]:
>> 1. Introduce the "high" flag to mark whether "crashkernel=X,high"
>> is passed. Fix the retrying logic between "crashkernel=X,high"
>> case and others when the first allocation attempt fails.
>>
>> Changes since [v5]:
>> 1. Update the crashkernel allocation logic when crashkernel=X,high
>> is specified. In this case, region above 4G will directly get
>> reserved as crashkernel, rather than trying lower 32bit allocation
>> first.
>>
>> Changes since [v4]:
>> 1. Update some imprecise code comments for cmdline parsing.
>>
>> Changes since [v3]:
>> 1. Update to print warning and return explicitly on failure when
>> crashkernel=size@offset is specified. Not changing the result
>> in this case but making the logic more straightforward.
>> 2. Some minor cleanup.
>>
>> Changes since [v2]:
>> 1. Update the allocation logic to ensure the high crashkernel
>> region is reserved strictly above dma32_phys_limit.
>> 2. Clean up some minor format problems.
>>
>> Chen Jiahao (2):
>> riscv: kdump: Implement crashkernel=X,[high,low]
>> docs: kdump: Update the crashkernel description for riscv
>>
>> .../admin-guide/kernel-parameters.txt | 15 ++--
>> arch/riscv/kernel/setup.c | 5 ++
>> arch/riscv/mm/init.c | 84 +++++++++++++++++--
>> 3 files changed, 90 insertions(+), 14 deletions(-)
>>
>> --
>> 2.34.1
>>
>
On Mon, Jul 3, 2023 at 9:07 PM chenjiahao (C) <[email protected]> wrote:
>
>
> On 2023/7/1 21:45, Guo Ren wrote:
> > On Sat, Jul 1, 2023 at 5:12 PM Chen Jiahao <[email protected]> wrote:
> >> On riscv, the current crash kernel allocation logic is trying to
> >> allocate within 32bit addressible memory region by default, if
> >> failed, try to allocate without 4G restriction.
> >>
> >> In need of saving DMA zone memory while allocating a relatively large
> >> crash kernel region, allocating the reserved memory top down in
> >> high memory, without overlapping the DMA zone, is a mature solution.
> >> Hence this patchset introduces the parameter option crashkernel=X,[high,low].
> >>
> >> One can reserve the crash kernel from high memory above DMA zone range
> >> by explicitly passing "crashkernel=X,high"; or reserve a memory range
> >> below 4G with "crashkernel=X,low". Besides, there are few rules need
> >> to take notice:
> >> 1. "crashkernel=X,[high,low]" will be ignored if "crashkernel=size"
> >> is specified.
> >> 2. "crashkernel=X,low" is valid only when "crashkernel=X,high" is passed
> >> and there is enough memory to be allocated under 4G.
> >> 3. When allocating crashkernel above 4G and no "crashkernel=X,low" is
> >> specified, a 128M low memory will be allocated automatically for
> >> swiotlb bounce buffer.
> >> See Documentation/admin-guide/kernel-parameters.txt for more information.
> >>
> >> To verify loading the crashkernel, adapted kexec-tools is attached below:
> >> https://github.com/chenjh005/kexec-tools/tree/build-test-riscv-v2
> >>
> >> Following test cases have been performed as expected:
> >> 1) crashkernel=256M //low=256M
> >> 2) crashkernel=1G //low=1G
> > Have you tried 1GB memory? we found a pud mapping problem on Sv39 of kexec, See:
> > https://lore.kernel.org/linux-riscv/[email protected]/
>
> I have tested on QEMU with sv57 mmu, so it seems the synchronization problem
> was not reproduce when reserving 1G memory and loading the capture kernel.
Yes, the pud of sv57 is not the pgd entries, so you didn't get problem.
>
>
> Thanks,
> Jiahao
>
> >
> >> 3) crashkernel=4G //high=4G, low=128M(default)
> >> 4) crashkernel=4G crashkernel=256M,high //high=4G, low=128M(default), high is ignored
> >> 5) crashkernel=4G crashkernel=256M,low //high=4G, low=128M(default), low is ignored
> >> 6) crashkernel=4G,high //high=4G, low=128M(default)
> >> 7) crashkernel=256M,low //low=0M, invalid
> >> 8) crashkernel=4G,high crashkernel=256M,low //high=4G, low=256M
> >> 9) crashkernel=4G,high crashkernel=4G,low //high=0M, low=0M, invalid
> >> 10) crashkernel=512M@0xd0000000 //low=512M
> >>
> >> Changes since [v6]:
> >> 1. Introduce the "high" flag to mark whether "crashkernel=X,high"
> >> is passed. Fix the retrying logic between "crashkernel=X,high"
> >> case and others when the first allocation attempt fails.
> >>
> >> Changes since [v5]:
> >> 1. Update the crashkernel allocation logic when crashkernel=X,high
> >> is specified. In this case, region above 4G will directly get
> >> reserved as crashkernel, rather than trying lower 32bit allocation
> >> first.
> >>
> >> Changes since [v4]:
> >> 1. Update some imprecise code comments for cmdline parsing.
> >>
> >> Changes since [v3]:
> >> 1. Update to print warning and return explicitly on failure when
> >> crashkernel=size@offset is specified. Not changing the result
> >> in this case but making the logic more straightforward.
> >> 2. Some minor cleanup.
> >>
> >> Changes since [v2]:
> >> 1. Update the allocation logic to ensure the high crashkernel
> >> region is reserved strictly above dma32_phys_limit.
> >> 2. Clean up some minor format problems.
> >>
> >> Chen Jiahao (2):
> >> riscv: kdump: Implement crashkernel=X,[high,low]
> >> docs: kdump: Update the crashkernel description for riscv
> >>
> >> .../admin-guide/kernel-parameters.txt | 15 ++--
> >> arch/riscv/kernel/setup.c | 5 ++
> >> arch/riscv/mm/init.c | 84 +++++++++++++++++--
> >> 3 files changed, 90 insertions(+), 14 deletions(-)
> >>
> >> --
> >> 2.34.1
> >>
> >
--
Best Regards
Guo Ren