LinuxLists.cc - [PATCH v3] mm/mm_init: use node's number of cpus in deferred_page_init_max

2024-05-28 19:00:53

Subject: [PATCH v3] mm/mm_init: use node's number of cpus in deferred_page_init_max_threads

When DEFERRED_STRUCT_PAGE_INIT=y, use a node's cpu count as maximum
thread count for the deferred initialization of struct pages via padata.
This should result in shorter boot times for these configurations by
going through page_alloc_init_late() faster as systems tend not to be
under heavy load that early in the bootstrap.

Only x86_64 does that now. Make it archs agnostic when
DEFERRED_STRUCT_PAGE_INIT is set. With the default defconfigs, that
includes powerpc and s390.

It used to be so before offering archs to override the function for
tuning with commit ecd096506922 ("mm: make deferred init's max threads
arch-specific").

Setting DEFERRED_STRUCT_PAGE_INIT and testing on a few arm64 platforms
shows faster deferred_init_memmap completions:
| | x13s | SA8775p-ride | Ampere R137-P31 | Ampere HR330 |
| | Metal, 32GB | VM, 36GB | VM, 58GB | Metal, 128GB |
| | 8cpus | 8cpus | 8cpus | 32cpus |
|---------|-------------|--------------|-----------------|--------------|
| threads | ms (%) | ms (%) | ms (%) | ms (%) |
|---------|-------------|--------------|-----------------|--------------|
| 1 | 108 (0%) | 72 (0%) | 224 (0%) | 324 (0%) |
| cpus | 24 (-77%) | 36 (-50%) | 40 (-82%) | 56 (-82%) |

Michael Ellerman on a powerpc machine (1TB, 40 cores, 4KB pages) reports
faster deferred_init_memmap from 210-240ms to 90-110ms between nodes.

Signed-off-by: Eric Chanudet <[email protected]>
Tested-by: Michael Ellerman <[email protected]> (powerpc)

---
- v1: https://lore.kernel.org/linux-arm-kernel/[email protected]
- Changes since v1:
- Make the generic function return the number of cpus of the node as
max threads limit instead overriding it for arm64.
- Drop Baoquan He's R-b on v1 since the logic changed.
- Add CCs according to patch changes (ppc and s390 set
DEFERRED_STRUCT_PAGE_INIT by default).

- v2: https://lore.kernel.org/linux-arm-kernel/[email protected]/
- Changes since v2:
- deferred_page_init_max_threads returns unsigned and use max instead
of max_t.
- Make deferred_page_init_max_threads static since there are no more
override.
- Rephrase description.
- Add T-b and report from Michael Ellerman.

arch/x86/mm/init_64.c | 12 ------------
include/linux/memblock.h | 2 --
mm/mm_init.c | 5 ++---
3 files changed, 2 insertions(+), 17 deletions(-)

diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c
index 7e177856ee4f..adec42928ec1 100644
--- a/arch/x86/mm/init_64.c
+++ b/arch/x86/mm/init_64.c
@@ -1354,18 +1354,6 @@ void __init mem_init(void)
preallocate_vmalloc_pages();
}

-#ifdef CONFIG_DEFERRED_STRUCT_PAGE_INIT
-int __init deferred_page_init_max_threads(const struct cpumask *node_cpumask)
-{
- /*
- * More CPUs always led to greater speedups on tested systems, up to
- * all the nodes' CPUs. Use all since the system is otherwise idle
- * now.
- */
- return max_t(int, cpumask_weight(node_cpumask), 1);
-}
-#endif
-
int kernel_set_to_readonly;

void mark_rodata_ro(void)
diff --git a/include/linux/memblock.h b/include/linux/memblock.h
index e2082240586d..40c62aca36ec 100644
--- a/include/linux/memblock.h
+++ b/include/linux/memblock.h
@@ -335,8 +335,6 @@ void __next_mem_pfn_range_in_zone(u64 *idx, struct zone *zone,
for (; i != U64_MAX; \
__next_mem_pfn_range_in_zone(&i, zone, p_start, p_end))

-int __init deferred_page_init_max_threads(const struct cpumask *node_cpumask);
-
#endif /* CONFIG_DEFERRED_STRUCT_PAGE_INIT */

/**
diff --git a/mm/mm_init.c b/mm/mm_init.c
index f72b852bd5b8..acfeba508796 100644
--- a/mm/mm_init.c
+++ b/mm/mm_init.c
@@ -2122,11 +2122,10 @@ deferred_init_memmap_chunk(unsigned long start_pfn, unsigned long end_pfn,
}
}

-/* An arch may override for more concurrency. */
-__weak int __init
+static unsigned int __init
deferred_page_init_max_threads(const struct cpumask *node_cpumask)
{
- return 1;
+ return max(cpumask_weight(node_cpumask), 1U);
}

/* Initialise remaining memory on a node */
--
2.44.0

2024-05-29 05:33:10

by Mike Rapoport

[permalink] [raw]

Subject: Re: [PATCH v3] mm/mm_init: use node's number of cpus in deferred_page_init_max_threads

On Tue, May 28, 2024 at 02:54:58PM -0400, Eric Chanudet wrote:
> When DEFERRED_STRUCT_PAGE_INIT=y, use a node's cpu count as maximum
> thread count for the deferred initialization of struct pages via padata.
> This should result in shorter boot times for these configurations by
> going through page_alloc_init_late() faster as systems tend not to be
> under heavy load that early in the bootstrap.
>
> Only x86_64 does that now. Make it archs agnostic when
> DEFERRED_STRUCT_PAGE_INIT is set. With the default defconfigs, that
> includes powerpc and s390.
>
> It used to be so before offering archs to override the function for
> tuning with commit ecd096506922 ("mm: make deferred init's max threads
> arch-specific").
>
> Setting DEFERRED_STRUCT_PAGE_INIT and testing on a few arm64 platforms
> shows faster deferred_init_memmap completions:
> | | x13s | SA8775p-ride | Ampere R137-P31 | Ampere HR330 |
> | | Metal, 32GB | VM, 36GB | VM, 58GB | Metal, 128GB |
> | | 8cpus | 8cpus | 8cpus | 32cpus |
> |---------|-------------|--------------|-----------------|--------------|
> | threads | ms (%) | ms (%) | ms (%) | ms (%) |
> |---------|-------------|--------------|-----------------|--------------|
> | 1 | 108 (0%) | 72 (0%) | 224 (0%) | 324 (0%) |
> | cpus | 24 (-77%) | 36 (-50%) | 40 (-82%) | 56 (-82%) |
>
> Michael Ellerman on a powerpc machine (1TB, 40 cores, 4KB pages) reports
> faster deferred_init_memmap from 210-240ms to 90-110ms between nodes.
>
> Signed-off-by: Eric Chanudet <[email protected]>
> Tested-by: Michael Ellerman <[email protected]> (powerpc)

Acked-by: Mike Rapoport (IBM) <[email protected]>

> ---
> - v1: https://lore.kernel.org/linux-arm-kernel/[email protected]
> - Changes since v1:
> - Make the generic function return the number of cpus of the node as
> max threads limit instead overriding it for arm64.
> - Drop Baoquan He's R-b on v1 since the logic changed.
> - Add CCs according to patch changes (ppc and s390 set
> DEFERRED_STRUCT_PAGE_INIT by default).
>
> - v2: https://lore.kernel.org/linux-arm-kernel/[email protected]/
> - Changes since v2:
> - deferred_page_init_max_threads returns unsigned and use max instead
> of max_t.
> - Make deferred_page_init_max_threads static since there are no more
> override.
> - Rephrase description.
> - Add T-b and report from Michael Ellerman.
>
> arch/x86/mm/init_64.c | 12 ------------
> include/linux/memblock.h | 2 --
> mm/mm_init.c | 5 ++---
> 3 files changed, 2 insertions(+), 17 deletions(-)
>
> diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c
> index 7e177856ee4f..adec42928ec1 100644
> --- a/arch/x86/mm/init_64.c
> +++ b/arch/x86/mm/init_64.c
> @@ -1354,18 +1354,6 @@ void __init mem_init(void)
> preallocate_vmalloc_pages();
> }
>
> -#ifdef CONFIG_DEFERRED_STRUCT_PAGE_INIT
> -int __init deferred_page_init_max_threads(const struct cpumask *node_cpumask)
> -{
> - /*
> - * More CPUs always led to greater speedups on tested systems, up to
> - * all the nodes' CPUs. Use all since the system is otherwise idle
> - * now.
> - */
> - return max_t(int, cpumask_weight(node_cpumask), 1);
> -}
> -#endif
> -
> int kernel_set_to_readonly;
>
> void mark_rodata_ro(void)
> diff --git a/include/linux/memblock.h b/include/linux/memblock.h
> index e2082240586d..40c62aca36ec 100644
> --- a/include/linux/memblock.h
> +++ b/include/linux/memblock.h
> @@ -335,8 +335,6 @@ void __next_mem_pfn_range_in_zone(u64 *idx, struct zone *zone,
> for (; i != U64_MAX; \
> __next_mem_pfn_range_in_zone(&i, zone, p_start, p_end))
>
> -int __init deferred_page_init_max_threads(const struct cpumask *node_cpumask);
> -
> #endif /* CONFIG_DEFERRED_STRUCT_PAGE_INIT */
>
> /**
> diff --git a/mm/mm_init.c b/mm/mm_init.c
> index f72b852bd5b8..acfeba508796 100644
> --- a/mm/mm_init.c
> +++ b/mm/mm_init.c
> @@ -2122,11 +2122,10 @@ deferred_init_memmap_chunk(unsigned long start_pfn, unsigned long end_pfn,
> }
> }
>
> -/* An arch may override for more concurrency. */
> -__weak int __init
> +static unsigned int __init
> deferred_page_init_max_threads(const struct cpumask *node_cpumask)
> {
> - return 1;
> + return max(cpumask_weight(node_cpumask), 1U);
> }
>
> /* Initialise remaining memory on a node */
> --
> 2.44.0
>

--
Sincerely yours,
Mike.

2024-05-29 06:29:10

by Alexander Gordeev

[permalink] [raw]

Subject: Re: [PATCH v3] mm/mm_init: use node's number of cpus in deferred_page_init_max_threads

2024-05-29 08:52:27

by Baoquan He

[permalink] [raw]

Subject: Re: [PATCH v3] mm/mm_init: use node's number of cpus in deferred_page_init_max_threads

On 05/28/24 at 02:54pm, Eric Chanudet wrote:
> When DEFERRED_STRUCT_PAGE_INIT=y, use a node's cpu count as maximum
> thread count for the deferred initialization of struct pages via padata.
> This should result in shorter boot times for these configurations by
> going through page_alloc_init_late() faster as systems tend not to be
> under heavy load that early in the bootstrap.
>
> Only x86_64 does that now. Make it archs agnostic when
> DEFERRED_STRUCT_PAGE_INIT is set. With the default defconfigs, that
> includes powerpc and s390.
>
> It used to be so before offering archs to override the function for
> tuning with commit ecd096506922 ("mm: make deferred init's max threads
> arch-specific").
>
> Setting DEFERRED_STRUCT_PAGE_INIT and testing on a few arm64 platforms
> shows faster deferred_init_memmap completions:
> | | x13s | SA8775p-ride | Ampere R137-P31 | Ampere HR330 |
> | | Metal, 32GB | VM, 36GB | VM, 58GB | Metal, 128GB |
> | | 8cpus | 8cpus | 8cpus | 32cpus |
> |---------|-------------|--------------|-----------------|--------------|
> | threads | ms (%) | ms (%) | ms (%) | ms (%) |
> |---------|-------------|--------------|-----------------|--------------|
> | 1 | 108 (0%) | 72 (0%) | 224 (0%) | 324 (0%) |
> | cpus | 24 (-77%) | 36 (-50%) | 40 (-82%) | 56 (-82%) |
>
> Michael Ellerman on a powerpc machine (1TB, 40 cores, 4KB pages) reports
> faster deferred_init_memmap from 210-240ms to 90-110ms between nodes.
>
> Signed-off-by: Eric Chanudet <[email protected]>
> Tested-by: Michael Ellerman <[email protected]> (powerpc)
>
> ---
> - v1: https://lore.kernel.org/linux-arm-kernel/[email protected]
> - Changes since v1:
> - Make the generic function return the number of cpus of the node as
> max threads limit instead overriding it for arm64.
> - Drop Baoquan He's R-b on v1 since the logic changed.
> - Add CCs according to patch changes (ppc and s390 set
> DEFERRED_STRUCT_PAGE_INIT by default).
>
> - v2: https://lore.kernel.org/linux-arm-kernel/[email protected]/
> - Changes since v2:
> - deferred_page_init_max_threads returns unsigned and use max instead
> of max_t.
> - Make deferred_page_init_max_threads static since there are no more
> override.
> - Rephrase description.
> - Add T-b and report from Michael Ellerman.
>
> arch/x86/mm/init_64.c | 12 ------------
> include/linux/memblock.h | 2 --
> mm/mm_init.c | 5 ++---
> 3 files changed, 2 insertions(+), 17 deletions(-)

Reviewed-by: Baoquan He <[email protected]>

>
> diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c
> index 7e177856ee4f..adec42928ec1 100644
> --- a/arch/x86/mm/init_64.c
> +++ b/arch/x86/mm/init_64.c
> @@ -1354,18 +1354,6 @@ void __init mem_init(void)
> preallocate_vmalloc_pages();
> }
>
> -#ifdef CONFIG_DEFERRED_STRUCT_PAGE_INIT
> -int __init deferred_page_init_max_threads(const struct cpumask *node_cpumask)
> -{
> - /*
> - * More CPUs always led to greater speedups on tested systems, up to
> - * all the nodes' CPUs. Use all since the system is otherwise idle
> - * now.
> - */
> - return max_t(int, cpumask_weight(node_cpumask), 1);
> -}
> -#endif
> -
> int kernel_set_to_readonly;
>
> void mark_rodata_ro(void)
> diff --git a/include/linux/memblock.h b/include/linux/memblock.h
> index e2082240586d..40c62aca36ec 100644
> --- a/include/linux/memblock.h
> +++ b/include/linux/memblock.h
> @@ -335,8 +335,6 @@ void __next_mem_pfn_range_in_zone(u64 *idx, struct zone *zone,
> for (; i != U64_MAX; \
> __next_mem_pfn_range_in_zone(&i, zone, p_start, p_end))
>
> -int __init deferred_page_init_max_threads(const struct cpumask *node_cpumask);
> -
> #endif /* CONFIG_DEFERRED_STRUCT_PAGE_INIT */
>
> /**
> diff --git a/mm/mm_init.c b/mm/mm_init.c
> index f72b852bd5b8..acfeba508796 100644
> --- a/mm/mm_init.c
> +++ b/mm/mm_init.c
> @@ -2122,11 +2122,10 @@ deferred_init_memmap_chunk(unsigned long start_pfn, unsigned long end_pfn,
> }
> }
>
> -/* An arch may override for more concurrency. */
> -__weak int __init
> +static unsigned int __init
> deferred_page_init_max_threads(const struct cpumask *node_cpumask)
> {
> - return 1;
> + return max(cpumask_weight(node_cpumask), 1U);
> }
>
> /* Initialise remaining memory on a node */
> --
> 2.44.0
>