From: Sergei Trofimovich <[email protected]>
To reproduce the failure we need the following system:
- kernel command: page_poison=1 init_on_free=0 init_on_alloc=0
- kernel config:
* CONFIG_INIT_ON_ALLOC_DEFAULT_ON=y
* CONFIG_INIT_ON_FREE_DEFAULT_ON=y
* CONFIG_PAGE_POISONING=y
0000000085629bdd: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0000000022861832: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00000000c597f5b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
CPU: 11 PID: 15195 Comm: bash Kdump: loaded Tainted: G U O 5.13.1-gentoo-x86_64 #1
Hardware name: System manufacturer System Product Name/PRIME Z370-A, BIOS 2801 01/13/2021
Call Trace:
dump_stack+0x64/0x7c
__kernel_unpoison_pages.cold+0x48/0x84
post_alloc_hook+0x60/0xa0
get_page_from_freelist+0xdb8/0x1000
__alloc_pages+0x163/0x2b0
__get_free_pages+0xc/0x30
pgd_alloc+0x2e/0x1a0
? dup_mm+0x37/0x4f0
mm_init+0x185/0x270
dup_mm+0x6b/0x4f0
? __lock_task_sighand+0x35/0x70
copy_process+0x190d/0x1b10
kernel_clone+0xba/0x3b0
__do_sys_clone+0x8f/0xb0
do_syscall_64+0x68/0x80
? do_syscall_64+0x11/0x80
entry_SYSCALL_64_after_hwframe+0x44/0xae
Before the 51cba1eb ("init_on_alloc: Optimize static branches")
init_on_alloc never enabled static branch by default. It could
only be enabed explicitly by init_mem_debugging_and_hardening().
But after the 51cba1eb static branch could already be enabled
by default. There was no code to ever disable it. That caused
page_poison=1 / init_on_free=1 conflict.
This change extends init_mem_debugging_and_hardening() to also
disable static branch disabling.
CC: Alexander Potapenko <[email protected]>
CC: Thomas Gleixner <[email protected]>
CC: Vlastimil Babka <[email protected]>
CC: [email protected]
Reported-by: [email protected]
Reported-by: Mikhail Morfikov <[email protected]>
Fixes: 51cba1ebc60d ("init_on_alloc: Optimize static branches")
Cc: [email protected]
Signed-off-by: Sergei Trofimovich <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Co-developed-by: Kees Cook <[email protected]>
Signed-off-by: Kees Cook <[email protected]>
---
mm/page_alloc.c | 29 ++++++++++++++++-------------
1 file changed, 16 insertions(+), 13 deletions(-)
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 3b97e17806be..1f19365bc158 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -840,21 +840,24 @@ void init_mem_debugging_and_hardening(void)
}
#endif
- if (_init_on_alloc_enabled_early) {
- if (page_poisoning_requested)
- pr_info("mem auto-init: CONFIG_PAGE_POISONING is on, "
- "will take precedence over init_on_alloc\n");
- else
- static_branch_enable(&init_on_alloc);
- }
- if (_init_on_free_enabled_early) {
- if (page_poisoning_requested)
- pr_info("mem auto-init: CONFIG_PAGE_POISONING is on, "
- "will take precedence over init_on_free\n");
- else
- static_branch_enable(&init_on_free);
+ if ((_init_on_alloc_enabled_early || _init_on_free_enabled_early) &&
+ page_poisoning_requested) {
+ pr_info("mem auto-init: CONFIG_PAGE_POISONING is on, "
+ "will take precedence over init_on_alloc and init_on_free\n");
+ _init_on_alloc_enabled_early = false;
+ _init_on_free_enabled_early = false;
}
+ if (_init_on_alloc_enabled_early)
+ static_branch_enable(&init_on_alloc);
+ else
+ static_branch_disable(&init_on_alloc);
+
+ if (_init_on_free_enabled_early)
+ static_branch_enable(&init_on_free);
+ else
+ static_branch_disable(&init_on_free);
+
#ifdef CONFIG_DEBUG_PAGEALLOC
if (!debug_pagealloc_enabled())
return;
--
2.30.2
On Tue, Jul 13, 2021 at 08:19:35PM -0700, Kees Cook wrote:
> From: Sergei Trofimovich <[email protected]>
>
> To reproduce the failure we need the following system:
> - kernel command: page_poison=1 init_on_free=0 init_on_alloc=0
> - kernel config:
> * CONFIG_INIT_ON_ALLOC_DEFAULT_ON=y
> * CONFIG_INIT_ON_FREE_DEFAULT_ON=y
> * CONFIG_PAGE_POISONING=y
>
> 0000000085629bdd: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 0000000022861832: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000000c597f5b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> CPU: 11 PID: 15195 Comm: bash Kdump: loaded Tainted: G U O 5.13.1-gentoo-x86_64 #1
> Hardware name: System manufacturer System Product Name/PRIME Z370-A, BIOS 2801 01/13/2021
> Call Trace:
> dump_stack+0x64/0x7c
> __kernel_unpoison_pages.cold+0x48/0x84
> post_alloc_hook+0x60/0xa0
> get_page_from_freelist+0xdb8/0x1000
> __alloc_pages+0x163/0x2b0
> __get_free_pages+0xc/0x30
> pgd_alloc+0x2e/0x1a0
> ? dup_mm+0x37/0x4f0
> mm_init+0x185/0x270
> dup_mm+0x6b/0x4f0
> ? __lock_task_sighand+0x35/0x70
> copy_process+0x190d/0x1b10
> kernel_clone+0xba/0x3b0
> __do_sys_clone+0x8f/0xb0
> do_syscall_64+0x68/0x80
> ? do_syscall_64+0x11/0x80
> entry_SYSCALL_64_after_hwframe+0x44/0xae
>
> Before the 51cba1eb ("init_on_alloc: Optimize static branches")
> init_on_alloc never enabled static branch by default. It could
> only be enabed explicitly by init_mem_debugging_and_hardening().
>
> But after the 51cba1eb static branch could already be enabled
> by default. There was no code to ever disable it. That caused
> page_poison=1 / init_on_free=1 conflict.
>
> This change extends init_mem_debugging_and_hardening() to also
> disable static branch disabling.
>
> CC: Alexander Potapenko <[email protected]>
> CC: Thomas Gleixner <[email protected]>
> CC: Vlastimil Babka <[email protected]>
> CC: [email protected]
> Reported-by: [email protected]
> Reported-by: Mikhail Morfikov <[email protected]>
> Fixes: 51cba1ebc60d ("init_on_alloc: Optimize static branches")
> Cc: [email protected]
yay typo "kernelo". Andrew, are you able to clean this up, or should I send a v3?
Sorry!
-Kees
> Signed-off-by: Sergei Trofimovich <[email protected]>
> Link: https://lore.kernel.org/r/[email protected]
> Co-developed-by: Kees Cook <[email protected]>
> Signed-off-by: Kees Cook <[email protected]>
> ---
> mm/page_alloc.c | 29 ++++++++++++++++-------------
> 1 file changed, 16 insertions(+), 13 deletions(-)
>
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 3b97e17806be..1f19365bc158 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -840,21 +840,24 @@ void init_mem_debugging_and_hardening(void)
> }
> #endif
>
> - if (_init_on_alloc_enabled_early) {
> - if (page_poisoning_requested)
> - pr_info("mem auto-init: CONFIG_PAGE_POISONING is on, "
> - "will take precedence over init_on_alloc\n");
> - else
> - static_branch_enable(&init_on_alloc);
> - }
> - if (_init_on_free_enabled_early) {
> - if (page_poisoning_requested)
> - pr_info("mem auto-init: CONFIG_PAGE_POISONING is on, "
> - "will take precedence over init_on_free\n");
> - else
> - static_branch_enable(&init_on_free);
> + if ((_init_on_alloc_enabled_early || _init_on_free_enabled_early) &&
> + page_poisoning_requested) {
> + pr_info("mem auto-init: CONFIG_PAGE_POISONING is on, "
> + "will take precedence over init_on_alloc and init_on_free\n");
> + _init_on_alloc_enabled_early = false;
> + _init_on_free_enabled_early = false;
> }
>
> + if (_init_on_alloc_enabled_early)
> + static_branch_enable(&init_on_alloc);
> + else
> + static_branch_disable(&init_on_alloc);
> +
> + if (_init_on_free_enabled_early)
> + static_branch_enable(&init_on_free);
> + else
> + static_branch_disable(&init_on_free);
> +
> #ifdef CONFIG_DEBUG_PAGEALLOC
> if (!debug_pagealloc_enabled())
> return;
> --
> 2.30.2
>
--
Kees Cook
On 14.07.21 05:19, Kees Cook wrote:
> From: Sergei Trofimovich <[email protected]>
>
> To reproduce the failure we need the following system:
> - kernel command: page_poison=1 init_on_free=0 init_on_alloc=0
> - kernel config:
> * CONFIG_INIT_ON_ALLOC_DEFAULT_ON=y
> * CONFIG_INIT_ON_FREE_DEFAULT_ON=y
> * CONFIG_PAGE_POISONING=y
>
> 0000000085629bdd: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 0000000022861832: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000000c597f5b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> CPU: 11 PID: 15195 Comm: bash Kdump: loaded Tainted: G U O 5.13.1-gentoo-x86_64 #1
> Hardware name: System manufacturer System Product Name/PRIME Z370-A, BIOS 2801 01/13/2021
> Call Trace:
> dump_stack+0x64/0x7c
> __kernel_unpoison_pages.cold+0x48/0x84
> post_alloc_hook+0x60/0xa0
> get_page_from_freelist+0xdb8/0x1000
> __alloc_pages+0x163/0x2b0
> __get_free_pages+0xc/0x30
> pgd_alloc+0x2e/0x1a0
> ? dup_mm+0x37/0x4f0
> mm_init+0x185/0x270
> dup_mm+0x6b/0x4f0
> ? __lock_task_sighand+0x35/0x70
> copy_process+0x190d/0x1b10
> kernel_clone+0xba/0x3b0
> __do_sys_clone+0x8f/0xb0
> do_syscall_64+0x68/0x80
> ? do_syscall_64+0x11/0x80
> entry_SYSCALL_64_after_hwframe+0x44/0xae
>
> Before the 51cba1eb ("init_on_alloc: Optimize static branches")
> init_on_alloc never enabled static branch by default. It could
> only be enabed explicitly by init_mem_debugging_and_hardening().
>
> But after the 51cba1eb static branch could already be enabled
> by default. There was no code to ever disable it. That caused
> page_poison=1 / init_on_free=1 conflict.
>
> This change extends init_mem_debugging_and_hardening() to also
> disable static branch disabling.
>
> CC: Alexander Potapenko <[email protected]>
> CC: Thomas Gleixner <[email protected]>
> CC: Vlastimil Babka <[email protected]>
> CC: [email protected]
> Reported-by: [email protected]
> Reported-by: Mikhail Morfikov <[email protected]>
> Fixes: 51cba1ebc60d ("init_on_alloc: Optimize static branches")
> Cc: [email protected]
> Signed-off-by: Sergei Trofimovich <[email protected]>
> Link: https://lore.kernel.org/r/[email protected]
> Co-developed-by: Kees Cook <[email protected]>
> Signed-off-by: Kees Cook <[email protected]>
> ---
> mm/page_alloc.c | 29 ++++++++++++++++-------------
> 1 file changed, 16 insertions(+), 13 deletions(-)
>
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 3b97e17806be..1f19365bc158 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -840,21 +840,24 @@ void init_mem_debugging_and_hardening(void)
> }
> #endif
>
> - if (_init_on_alloc_enabled_early) {
> - if (page_poisoning_requested)
> - pr_info("mem auto-init: CONFIG_PAGE_POISONING is on, "
> - "will take precedence over init_on_alloc\n");
> - else
> - static_branch_enable(&init_on_alloc);
> - }
> - if (_init_on_free_enabled_early) {
> - if (page_poisoning_requested)
> - pr_info("mem auto-init: CONFIG_PAGE_POISONING is on, "
> - "will take precedence over init_on_free\n");
> - else
> - static_branch_enable(&init_on_free);
> + if ((_init_on_alloc_enabled_early || _init_on_free_enabled_early) &&
> + page_poisoning_requested) {
> + pr_info("mem auto-init: CONFIG_PAGE_POISONING is on, "
> + "will take precedence over init_on_alloc and init_on_free\n");
> + _init_on_alloc_enabled_early = false;
> + _init_on_free_enabled_early = false;
> }
>
> + if (_init_on_alloc_enabled_early)
> + static_branch_enable(&init_on_alloc);
> + else
> + static_branch_disable(&init_on_alloc);
> +
> + if (_init_on_free_enabled_early)
> + static_branch_enable(&init_on_free);
> + else
> + static_branch_disable(&init_on_free);
> +
> #ifdef CONFIG_DEBUG_PAGEALLOC
> if (!debug_pagealloc_enabled())
> return;
>
LGTM
Reviewed-by: David Hildenbrand <[email protected]>
--
Thanks,
David / dhildenb