The page table check trigger BUG_ON() unexpectedly when split hugepage:
------------[ cut here ]------------
kernel BUG at mm/page_table_check.c:119!
Internal error: Oops - BUG: 00000000f2000800 [#1] SMP
Dumping ftrace buffer:
(ftrace buffer empty)
Modules linked in:
CPU: 7 PID: 210 Comm: transhuge-stres Not tainted 6.1.0-rc3+ #748
Hardware name: linux,dummy-virt (DT)
pstate: 20000005 (nzCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
pc : page_table_check_set.isra.0+0x398/0x468
lr : page_table_check_set.isra.0+0x1c0/0x468
[...]
Call trace:
page_table_check_set.isra.0+0x398/0x468
__page_table_check_pte_set+0x160/0x1c0
__split_huge_pmd_locked+0x900/0x1648
__split_huge_pmd+0x28c/0x3b8
unmap_page_range+0x428/0x858
unmap_single_vma+0xf4/0x1c8
zap_page_range+0x2b0/0x410
madvise_vma_behavior+0xc44/0xe78
do_madvise+0x280/0x698
__arm64_sys_madvise+0x90/0xe8
invoke_syscall.constprop.0+0xdc/0x1d8
do_el0_svc+0xf4/0x3f8
el0_svc+0x58/0x120
el0t_64_sync_handler+0xb8/0xc0
el0t_64_sync+0x19c/0x1a0
[...]
On arm64, pmd_leaf() will return true even if the pmd is invalid due to
pmd_present_invalid() check. So in pmdp_invalidate() the file_map_count
will not only decrease once but also increase once. Then in set_pte_at(),
the file_map_count increase again, and so trigger BUG_ON() unexpectedly.
Fix this problem by adding pmd_valid() in pmd_user_accessible_page().
Moreover, add pud_valid() for pud_user_accessible_page() too.
Fixes: 42b2547137f5 ("arm64/mm: enable ARCH_SUPPORTS_PAGE_TABLE_CHECK")
Reported-by: Denys Vlasenko <[email protected]>
Signed-off-by: Liu Shixin <[email protected]>
Acked-by: Pasha Tatashin <[email protected]>
---
arch/arm64/include/asm/pgtable.h | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
index edf6625ce965..3bc64199aa2e 100644
--- a/arch/arm64/include/asm/pgtable.h
+++ b/arch/arm64/include/asm/pgtable.h
@@ -863,12 +863,12 @@ static inline bool pte_user_accessible_page(pte_t pte)
static inline bool pmd_user_accessible_page(pmd_t pmd)
{
- return pmd_leaf(pmd) && (pmd_user(pmd) || pmd_user_exec(pmd));
+ return pmd_valid(pmd) && pmd_leaf(pmd) && (pmd_user(pmd) || pmd_user_exec(pmd));
}
static inline bool pud_user_accessible_page(pud_t pud)
{
- return pud_leaf(pud) && pud_user(pud);
+ return pud_valid(pud) && pud_leaf(pud) && pud_user(pud);
}
#endif
--
2.25.1
On 17.11.22 08:56, Liu Shixin wrote:
> The page table check trigger BUG_ON() unexpectedly when split hugepage:
>
> ------------[ cut here ]------------
> kernel BUG at mm/page_table_check.c:119!
> Internal error: Oops - BUG: 00000000f2000800 [#1] SMP
> Dumping ftrace buffer:
> (ftrace buffer empty)
> Modules linked in:
> CPU: 7 PID: 210 Comm: transhuge-stres Not tainted 6.1.0-rc3+ #748
> Hardware name: linux,dummy-virt (DT)
> pstate: 20000005 (nzCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> pc : page_table_check_set.isra.0+0x398/0x468
> lr : page_table_check_set.isra.0+0x1c0/0x468
> [...]
> Call trace:
> page_table_check_set.isra.0+0x398/0x468
> __page_table_check_pte_set+0x160/0x1c0
> __split_huge_pmd_locked+0x900/0x1648
> __split_huge_pmd+0x28c/0x3b8
> unmap_page_range+0x428/0x858
> unmap_single_vma+0xf4/0x1c8
> zap_page_range+0x2b0/0x410
> madvise_vma_behavior+0xc44/0xe78
> do_madvise+0x280/0x698
> __arm64_sys_madvise+0x90/0xe8
> invoke_syscall.constprop.0+0xdc/0x1d8
> do_el0_svc+0xf4/0x3f8
> el0_svc+0x58/0x120
> el0t_64_sync_handler+0xb8/0xc0
> el0t_64_sync+0x19c/0x1a0
> [...]
>
> On arm64, pmd_leaf() will return true even if the pmd is invalid due to
> pmd_present_invalid() check. So in pmdp_invalidate() the file_map_count
> will not only decrease once but also increase once. Then in set_pte_at(),
> the file_map_count increase again, and so trigger BUG_ON() unexpectedly.
>
> Fix this problem by adding pmd_valid() in pmd_user_accessible_page().
> Moreover, add pud_valid() for pud_user_accessible_page() too.
>
> Fixes: 42b2547137f5 ("arm64/mm: enable ARCH_SUPPORTS_PAGE_TABLE_CHECK")
> Reported-by: Denys Vlasenko <[email protected]>
> Signed-off-by: Liu Shixin <[email protected]>
> Acked-by: Pasha Tatashin <[email protected]>
> ---
> arch/arm64/include/asm/pgtable.h | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
> index edf6625ce965..3bc64199aa2e 100644
> --- a/arch/arm64/include/asm/pgtable.h
> +++ b/arch/arm64/include/asm/pgtable.h
> @@ -863,12 +863,12 @@ static inline bool pte_user_accessible_page(pte_t pte)
>
> static inline bool pmd_user_accessible_page(pmd_t pmd)
> {
> - return pmd_leaf(pmd) && (pmd_user(pmd) || pmd_user_exec(pmd));
> + return pmd_valid(pmd) && pmd_leaf(pmd) && (pmd_user(pmd) || pmd_user_exec(pmd));
> }
>
> static inline bool pud_user_accessible_page(pud_t pud)
> {
> - return pud_leaf(pud) && pud_user(pud);
> + return pud_valid(pud) && pud_leaf(pud) && pud_user(pud);
> }
> #endif
>
Acked-by: David Hildenbrand <[email protected]>
--
Thanks,
David / dhildenb
On 2022/11/17 15:56, Liu Shixin wrote:
> The page table check trigger BUG_ON() unexpectedly when split hugepage:
>
> ------------[ cut here ]------------
> kernel BUG at mm/page_table_check.c:119!
> Internal error: Oops - BUG: 00000000f2000800 [#1] SMP
> Dumping ftrace buffer:
> (ftrace buffer empty)
> Modules linked in:
> CPU: 7 PID: 210 Comm: transhuge-stres Not tainted 6.1.0-rc3+ #748
> Hardware name: linux,dummy-virt (DT)
> pstate: 20000005 (nzCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> pc : page_table_check_set.isra.0+0x398/0x468
> lr : page_table_check_set.isra.0+0x1c0/0x468
> [...]
> Call trace:
> page_table_check_set.isra.0+0x398/0x468
> __page_table_check_pte_set+0x160/0x1c0
> __split_huge_pmd_locked+0x900/0x1648
> __split_huge_pmd+0x28c/0x3b8
> unmap_page_range+0x428/0x858
> unmap_single_vma+0xf4/0x1c8
> zap_page_range+0x2b0/0x410
> madvise_vma_behavior+0xc44/0xe78
> do_madvise+0x280/0x698
> __arm64_sys_madvise+0x90/0xe8
> invoke_syscall.constprop.0+0xdc/0x1d8
> do_el0_svc+0xf4/0x3f8
> el0_svc+0x58/0x120
> el0t_64_sync_handler+0xb8/0xc0
> el0t_64_sync+0x19c/0x1a0
> [...]
>
> On arm64, pmd_leaf() will return true even if the pmd is invalid due to
> pmd_present_invalid() check. So in pmdp_invalidate() the file_map_count
> will not only decrease once but also increase once. Then in set_pte_at(),
> the file_map_count increase again, and so trigger BUG_ON() unexpectedly.
>
> Fix this problem by adding pmd_valid() in pmd_user_accessible_page().
> Moreover, add pud_valid() for pud_user_accessible_page() too.
>
> Fixes: 42b2547137f5 ("arm64/mm: enable ARCH_SUPPORTS_PAGE_TABLE_CHECK")
> Reported-by: Denys Vlasenko <[email protected]>
> Signed-off-by: Liu Shixin <[email protected]>
> Acked-by: Pasha Tatashin <[email protected]>
Reviewed-by: Kefeng Wang <[email protected]>
On Thu, Nov 17, 2022 at 03:56:02PM +0800, Liu Shixin wrote:
> The page table check trigger BUG_ON() unexpectedly when split hugepage:
>
> ------------[ cut here ]------------
> kernel BUG at mm/page_table_check.c:119!
> Internal error: Oops - BUG: 00000000f2000800 [#1] SMP
> Dumping ftrace buffer:
> (ftrace buffer empty)
> Modules linked in:
> CPU: 7 PID: 210 Comm: transhuge-stres Not tainted 6.1.0-rc3+ #748
> Hardware name: linux,dummy-virt (DT)
> pstate: 20000005 (nzCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> pc : page_table_check_set.isra.0+0x398/0x468
> lr : page_table_check_set.isra.0+0x1c0/0x468
> [...]
> Call trace:
> page_table_check_set.isra.0+0x398/0x468
> __page_table_check_pte_set+0x160/0x1c0
> __split_huge_pmd_locked+0x900/0x1648
> __split_huge_pmd+0x28c/0x3b8
> unmap_page_range+0x428/0x858
> unmap_single_vma+0xf4/0x1c8
> zap_page_range+0x2b0/0x410
> madvise_vma_behavior+0xc44/0xe78
> do_madvise+0x280/0x698
> __arm64_sys_madvise+0x90/0xe8
> invoke_syscall.constprop.0+0xdc/0x1d8
> do_el0_svc+0xf4/0x3f8
> el0_svc+0x58/0x120
> el0t_64_sync_handler+0xb8/0xc0
> el0t_64_sync+0x19c/0x1a0
> [...]
>
> On arm64, pmd_leaf() will return true even if the pmd is invalid due to
> pmd_present_invalid() check. So in pmdp_invalidate() the file_map_count
> will not only decrease once but also increase once. Then in set_pte_at(),
> the file_map_count increase again, and so trigger BUG_ON() unexpectedly.
>
> Fix this problem by adding pmd_valid() in pmd_user_accessible_page().
> Moreover, add pud_valid() for pud_user_accessible_page() too.
>
> Fixes: 42b2547137f5 ("arm64/mm: enable ARCH_SUPPORTS_PAGE_TABLE_CHECK")
> Reported-by: Denys Vlasenko <[email protected]>
> Signed-off-by: Liu Shixin <[email protected]>
> Acked-by: Pasha Tatashin <[email protected]>
> ---
> arch/arm64/include/asm/pgtable.h | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
> index edf6625ce965..3bc64199aa2e 100644
> --- a/arch/arm64/include/asm/pgtable.h
> +++ b/arch/arm64/include/asm/pgtable.h
> @@ -863,12 +863,12 @@ static inline bool pte_user_accessible_page(pte_t pte)
>
> static inline bool pmd_user_accessible_page(pmd_t pmd)
> {
> - return pmd_leaf(pmd) && (pmd_user(pmd) || pmd_user_exec(pmd));
> + return pmd_valid(pmd) && pmd_leaf(pmd) && (pmd_user(pmd) || pmd_user_exec(pmd));
Hmm, doesn't this have a funny interaction with PROT_NONE where the pmd is
invalid but present? If you don't care about PROT_NONE, then you could just
do:
pmd_valid(pmd) && !pmd_table(pmd) && (pmd_user(pmd) || pmd_user_exec(pmd))
but if you do care then you could do:
pmd_leaf(pmd) && !pmd_present_invalid(pmd) && (pmd_user(pmd) || pmd_user_exec(pmd))
> static inline bool pud_user_accessible_page(pud_t pud)
> {
> - return pud_leaf(pud) && pud_user(pud);
> + return pud_valid(pud) && pud_leaf(pud) && pud_user(pud);
Not caused by this patch, but why don't we have something like a
pud_user_exec() check here like we do for the pte and pmd levels?
Will
On 2022/11/18 22:34, Will Deacon wrote:
> On Thu, Nov 17, 2022 at 03:56:02PM +0800, Liu Shixin wrote:
>> The page table check trigger BUG_ON() unexpectedly when split hugepage:
>>
>> ------------[ cut here ]------------
>> kernel BUG at mm/page_table_check.c:119!
>> Internal error: Oops - BUG: 00000000f2000800 [#1] SMP
>> Dumping ftrace buffer:
>> (ftrace buffer empty)
>> Modules linked in:
>> CPU: 7 PID: 210 Comm: transhuge-stres Not tainted 6.1.0-rc3+ #748
>> Hardware name: linux,dummy-virt (DT)
>> pstate: 20000005 (nzCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
>> pc : page_table_check_set.isra.0+0x398/0x468
>> lr : page_table_check_set.isra.0+0x1c0/0x468
>> [...]
>> Call trace:
>> page_table_check_set.isra.0+0x398/0x468
>> __page_table_check_pte_set+0x160/0x1c0
>> __split_huge_pmd_locked+0x900/0x1648
>> __split_huge_pmd+0x28c/0x3b8
>> unmap_page_range+0x428/0x858
>> unmap_single_vma+0xf4/0x1c8
>> zap_page_range+0x2b0/0x410
>> madvise_vma_behavior+0xc44/0xe78
>> do_madvise+0x280/0x698
>> __arm64_sys_madvise+0x90/0xe8
>> invoke_syscall.constprop.0+0xdc/0x1d8
>> do_el0_svc+0xf4/0x3f8
>> el0_svc+0x58/0x120
>> el0t_64_sync_handler+0xb8/0xc0
>> el0t_64_sync+0x19c/0x1a0
>> [...]
>>
>> On arm64, pmd_leaf() will return true even if the pmd is invalid due to
>> pmd_present_invalid() check. So in pmdp_invalidate() the file_map_count
>> will not only decrease once but also increase once. Then in set_pte_at(),
>> the file_map_count increase again, and so trigger BUG_ON() unexpectedly.
>>
>> Fix this problem by adding pmd_valid() in pmd_user_accessible_page().
>> Moreover, add pud_valid() for pud_user_accessible_page() too.
>>
>> Fixes: 42b2547137f5 ("arm64/mm: enable ARCH_SUPPORTS_PAGE_TABLE_CHECK")
>> Reported-by: Denys Vlasenko <[email protected]>
>> Signed-off-by: Liu Shixin <[email protected]>
>> Acked-by: Pasha Tatashin <[email protected]>
>> ---
>> arch/arm64/include/asm/pgtable.h | 4 ++--
>> 1 file changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
>> index edf6625ce965..3bc64199aa2e 100644
>> --- a/arch/arm64/include/asm/pgtable.h
>> +++ b/arch/arm64/include/asm/pgtable.h
>> @@ -863,12 +863,12 @@ static inline bool pte_user_accessible_page(pte_t pte)
>>
>> static inline bool pmd_user_accessible_page(pmd_t pmd)
>> {
>> - return pmd_leaf(pmd) && (pmd_user(pmd) || pmd_user_exec(pmd));
>> + return pmd_valid(pmd) && pmd_leaf(pmd) && (pmd_user(pmd) || pmd_user_exec(pmd));
> Hmm, doesn't this have a funny interaction with PROT_NONE where the pmd is
> invalid but present? If you don't care about PROT_NONE, then you could just
> do:
>
> pmd_valid(pmd) && !pmd_table(pmd) && (pmd_user(pmd) || pmd_user_exec(pmd))
>
> but if you do care then you could do:
>
> pmd_leaf(pmd) && !pmd_present_invalid(pmd) && (pmd_user(pmd) || pmd_user_exec(pmd))
I prefer the latter. I will fix and resend later.
>> static inline bool pud_user_accessible_page(pud_t pud)
>> {
>> - return pud_leaf(pud) && pud_user(pud);
>> + return pud_valid(pud) && pud_leaf(pud) && pud_user(pud);
> Not caused by this patch, but why don't we have something like a
> pud_user_exec() check here like we do for the pte and pmd levels?
As far as I know, there is no user use the user executable pud on arm64, so didn't define pud_user_exec().
Thanks,
>
> Will
> .
>
On Mon, Nov 21, 2022 at 11:15:49AM +0800, Liu Shixin wrote:
> On 2022/11/18 22:34, Will Deacon wrote:
> > On Thu, Nov 17, 2022 at 03:56:02PM +0800, Liu Shixin wrote:
> >> The page table check trigger BUG_ON() unexpectedly when split hugepage:
> >>
> >> ------------[ cut here ]------------
> >> kernel BUG at mm/page_table_check.c:119!
> >> Internal error: Oops - BUG: 00000000f2000800 [#1] SMP
> >> Dumping ftrace buffer:
> >> (ftrace buffer empty)
> >> Modules linked in:
> >> CPU: 7 PID: 210 Comm: transhuge-stres Not tainted 6.1.0-rc3+ #748
> >> Hardware name: linux,dummy-virt (DT)
> >> pstate: 20000005 (nzCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> >> pc : page_table_check_set.isra.0+0x398/0x468
> >> lr : page_table_check_set.isra.0+0x1c0/0x468
> >> [...]
> >> Call trace:
> >> page_table_check_set.isra.0+0x398/0x468
> >> __page_table_check_pte_set+0x160/0x1c0
> >> __split_huge_pmd_locked+0x900/0x1648
> >> __split_huge_pmd+0x28c/0x3b8
> >> unmap_page_range+0x428/0x858
> >> unmap_single_vma+0xf4/0x1c8
> >> zap_page_range+0x2b0/0x410
> >> madvise_vma_behavior+0xc44/0xe78
> >> do_madvise+0x280/0x698
> >> __arm64_sys_madvise+0x90/0xe8
> >> invoke_syscall.constprop.0+0xdc/0x1d8
> >> do_el0_svc+0xf4/0x3f8
> >> el0_svc+0x58/0x120
> >> el0t_64_sync_handler+0xb8/0xc0
> >> el0t_64_sync+0x19c/0x1a0
> >> [...]
> >>
> >> On arm64, pmd_leaf() will return true even if the pmd is invalid due to
> >> pmd_present_invalid() check. So in pmdp_invalidate() the file_map_count
> >> will not only decrease once but also increase once. Then in set_pte_at(),
> >> the file_map_count increase again, and so trigger BUG_ON() unexpectedly.
> >>
> >> Fix this problem by adding pmd_valid() in pmd_user_accessible_page().
> >> Moreover, add pud_valid() for pud_user_accessible_page() too.
> >>
> >> Fixes: 42b2547137f5 ("arm64/mm: enable ARCH_SUPPORTS_PAGE_TABLE_CHECK")
> >> Reported-by: Denys Vlasenko <[email protected]>
> >> Signed-off-by: Liu Shixin <[email protected]>
> >> Acked-by: Pasha Tatashin <[email protected]>
> >> ---
> >> arch/arm64/include/asm/pgtable.h | 4 ++--
> >> 1 file changed, 2 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
> >> index edf6625ce965..3bc64199aa2e 100644
> >> --- a/arch/arm64/include/asm/pgtable.h
> >> +++ b/arch/arm64/include/asm/pgtable.h
> >> @@ -863,12 +863,12 @@ static inline bool pte_user_accessible_page(pte_t pte)
> >>
> >> static inline bool pmd_user_accessible_page(pmd_t pmd)
> >> {
> >> - return pmd_leaf(pmd) && (pmd_user(pmd) || pmd_user_exec(pmd));
> >> + return pmd_valid(pmd) && pmd_leaf(pmd) && (pmd_user(pmd) || pmd_user_exec(pmd));
> > Hmm, doesn't this have a funny interaction with PROT_NONE where the pmd is
> > invalid but present? If you don't care about PROT_NONE, then you could just
> > do:
> >
> > pmd_valid(pmd) && !pmd_table(pmd) && (pmd_user(pmd) || pmd_user_exec(pmd))
> >
> > but if you do care then you could do:
> >
> > pmd_leaf(pmd) && !pmd_present_invalid(pmd) && (pmd_user(pmd) || pmd_user_exec(pmd))
> I prefer the latter. I will fix and resend later.
> >> static inline bool pud_user_accessible_page(pud_t pud)
> >> {
> >> - return pud_leaf(pud) && pud_user(pud);
> >> + return pud_valid(pud) && pud_leaf(pud) && pud_user(pud);
> > Not caused by this patch, but why don't we have something like a
> > pud_user_exec() check here like we do for the pte and pmd levels?
> As far as I know, there is no user use the user executable pud on arm64, so didn't define pud_user_exec().
I can believe they don't get exposed to userspace at all, but exposing only
as non-executable doesn't sound right. So I would have thought that either
pud_user_accessible_page() would always return false or it would need to
check for the executable case too.
Will
On 2022/11/22 2:16, Will Deacon wrote:
> On Mon, Nov 21, 2022 at 11:15:49AM +0800, Liu Shixin wrote:
>> On 2022/11/18 22:34, Will Deacon wrote:
>>> On Thu, Nov 17, 2022 at 03:56:02PM +0800, Liu Shixin wrote:
>>>> static inline bool pud_user_accessible_page(pud_t pud)
>>>> {
>>>> - return pud_leaf(pud) && pud_user(pud);
>>>> + return pud_valid(pud) && pud_leaf(pud) && pud_user(pud);
>>> Not caused by this patch, but why don't we have something like a
>>> pud_user_exec() check here like we do for the pte and pmd levels?
>> As far as I know, there is no user use the user executable pud on arm64, so didn't define pud_user_exec().
> I can believe they don't get exposed to userspace at all, but exposing only
> as non-executable doesn't sound right. So I would have thought that either
> pud_user_accessible_page() would always return false or it would need to
> check for the executable case too.
Thanks for your advice, I will add the check for the executable case too.
>
> Will
> .
>