Similar to commit bf587caae305 ("riscv: mm: synchronize MMU after pte change")
For those riscv implementations whose TLB cannot synchronize with dcache,
an SFENCE.VMA is necessary after page table update.
This patch fixed two functions:
1. pgd_alloc
During fork, a parent process prepares pgd for its child and updates satp
later, but they may not run on the same core. Adding a remote SFENCE.VMA to
invalidate TLB in other cores is needed. Thus use flush_tlb_all() instead
of local_flush_tlb_all() here.
Similar approaches can be found in arm and csky.
2. __set_fixmap
Add a SFENCE.VMA after fixmap pte update.
Similar approaches can be found in arm and sh.
Signed-off-by: Nick Hu <[email protected]>
Signed-off-by: Nylon Chen <[email protected]>
Cc: Alan Kao <[email protected]>
---
arch/riscv/include/asm/pgalloc.h | 1 +
arch/riscv/mm/init.c | 2 +-
2 files changed, 2 insertions(+), 1 deletion(-)
diff --git a/arch/riscv/include/asm/pgalloc.h b/arch/riscv/include/asm/pgalloc.h
index 3f601ee8233f..071468fa14b7 100644
--- a/arch/riscv/include/asm/pgalloc.h
+++ b/arch/riscv/include/asm/pgalloc.h
@@ -51,6 +51,7 @@ static inline pgd_t *pgd_alloc(struct mm_struct *mm)
memcpy(pgd + USER_PTRS_PER_PGD,
init_mm.pgd + USER_PTRS_PER_PGD,
(PTRS_PER_PGD - USER_PTRS_PER_PGD) * sizeof(pgd_t));
+ flush_tlb_all();
}
return pgd;
}
diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c
index fab855963c73..a7f329503ed0 100644
--- a/arch/riscv/mm/init.c
+++ b/arch/riscv/mm/init.c
@@ -203,8 +203,8 @@ void __set_fixmap(enum fixed_addresses idx, phys_addr_t phys, pgprot_t prot)
set_pte(ptep, pfn_pte(phys >> PAGE_SHIFT, prot));
} else {
pte_clear(&init_mm, addr, ptep);
- local_flush_tlb_page(addr);
}
+ local_flush_tlb_page(addr);
}
static pte_t *__init get_pte_virt(phys_addr_t pa)
--
2.17.0
On Tue, Mar 24, 2020 at 1:50 PM Nick Hu <[email protected]> wrote:
>
> Similar to commit bf587caae305 ("riscv: mm: synchronize MMU after pte change")
>
> For those riscv implementations whose TLB cannot synchronize with dcache,
> an SFENCE.VMA is necessary after page table update.
> This patch fixed two functions:
>
> 1. pgd_alloc
> During fork, a parent process prepares pgd for its child and updates satp
> later, but they may not run on the same core. Adding a remote SFENCE.VMA to
> invalidate TLB in other cores is needed. Thus use flush_tlb_all() instead
> of local_flush_tlb_all() here.
> Similar approaches can be found in arm and csky.
>
> 2. __set_fixmap
> Add a SFENCE.VMA after fixmap pte update.
> Similar approaches can be found in arm and sh.
>
> Signed-off-by: Nick Hu <[email protected]>
> Signed-off-by: Nylon Chen <[email protected]>
> Cc: Alan Kao <[email protected]>
> ---
> arch/riscv/include/asm/pgalloc.h | 1 +
> arch/riscv/mm/init.c | 2 +-
> 2 files changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/arch/riscv/include/asm/pgalloc.h b/arch/riscv/include/asm/pgalloc.h
> index 3f601ee8233f..071468fa14b7 100644
> --- a/arch/riscv/include/asm/pgalloc.h
> +++ b/arch/riscv/include/asm/pgalloc.h
> @@ -51,6 +51,7 @@ static inline pgd_t *pgd_alloc(struct mm_struct *mm)
> memcpy(pgd + USER_PTRS_PER_PGD,
> init_mm.pgd + USER_PTRS_PER_PGD,
> (PTRS_PER_PGD - USER_PTRS_PER_PGD) * sizeof(pgd_t));
> + flush_tlb_all();
Are you sure to put a tlb flush operation here ? I think it should be
a dcache flush ops (but there is no solution for riscv ISA :P)
Ref to csky's implementation, just some old cores need sync dcache
with tlb by a simple dcache flush range ops and tlb ops should be put
in another place due to Linux infrastructure.
static inline pgd_t *pgd_alloc(struct mm_struct *mm)
{
pgd_t *ret;
pgd_t *init;
ret = (pgd_t *) __get_free_pages(GFP_KERNEL, PGD_ORDER);
if (ret) {
init = pgd_offset(&init_mm, 0UL);
pgd_init((unsigned long *)ret);
memcpy(ret + USER_PTRS_PER_PGD, init + USER_PTRS_PER_PGD,
(PTRS_PER_PGD - USER_PTRS_PER_PGD) * sizeof(pgd_t));
/* prevent out of order excute */
smp_mb();
#ifdef CONFIG_CPU_NEED_TLBSYNC
dcache_wb_range((unsigned int)ret,
(unsigned int)(ret + PTRS_PER_PGD));
#endif
}
return ret;
}
> }
> return pgd;
> }
> diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c
> index fab855963c73..a7f329503ed0 100644
> --- a/arch/riscv/mm/init.c
> +++ b/arch/riscv/mm/init.c
> @@ -203,8 +203,8 @@ void __set_fixmap(enum fixed_addresses idx, phys_addr_t phys, pgprot_t prot)
> set_pte(ptep, pfn_pte(phys >> PAGE_SHIFT, prot));
> } else {
> pte_clear(&init_mm, addr, ptep);
> - local_flush_tlb_page(addr);
> }
> + local_flush_tlb_page(addr);
> }
>
> static pte_t *__init get_pte_virt(phys_addr_t pa)
> --
> 2.17.0
>
>
--
Best Regards
Guo Ren
ML: https://lore.kernel.org/linux-csky/
On Mon, 23 Mar 2020 22:49:45 PDT (-0700), [email protected] wrote:
> Similar to commit bf587caae305 ("riscv: mm: synchronize MMU after pte change")
>
> For those riscv implementations whose TLB cannot synchronize with dcache,
> an SFENCE.VMA is necessary after page table update.
> This patch fixed two functions:
>
> 1. pgd_alloc
> During fork, a parent process prepares pgd for its child and updates satp
> later, but they may not run on the same core. Adding a remote SFENCE.VMA to
> invalidate TLB in other cores is needed. Thus use flush_tlb_all() instead
> of local_flush_tlb_all() here.
> Similar approaches can be found in arm and csky.
>
> 2. __set_fixmap
> Add a SFENCE.VMA after fixmap pte update.
> Similar approaches can be found in arm and sh.
>
> Signed-off-by: Nick Hu <[email protected]>
> Signed-off-by: Nylon Chen <[email protected]>
> Cc: Alan Kao <[email protected]>
> ---
> arch/riscv/include/asm/pgalloc.h | 1 +
> arch/riscv/mm/init.c | 2 +-
> 2 files changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/arch/riscv/include/asm/pgalloc.h b/arch/riscv/include/asm/pgalloc.h
> index 3f601ee8233f..071468fa14b7 100644
> --- a/arch/riscv/include/asm/pgalloc.h
> +++ b/arch/riscv/include/asm/pgalloc.h
> @@ -51,6 +51,7 @@ static inline pgd_t *pgd_alloc(struct mm_struct *mm)
> memcpy(pgd + USER_PTRS_PER_PGD,
> init_mm.pgd + USER_PTRS_PER_PGD,
> (PTRS_PER_PGD - USER_PTRS_PER_PGD) * sizeof(pgd_t));
> + flush_tlb_all();
> }
> return pgd;
> }
> diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c
> index fab855963c73..a7f329503ed0 100644
> --- a/arch/riscv/mm/init.c
> +++ b/arch/riscv/mm/init.c
> @@ -203,8 +203,8 @@ void __set_fixmap(enum fixed_addresses idx, phys_addr_t phys, pgprot_t prot)
> set_pte(ptep, pfn_pte(phys >> PAGE_SHIFT, prot));
> } else {
> pte_clear(&init_mm, addr, ptep);
> - local_flush_tlb_page(addr);
> }
> + local_flush_tlb_page(addr);
> }
>
> static pte_t *__init get_pte_virt(phys_addr_t pa)
As a general rule, any fences are supposed to have a comment describing why
they're there. The generic fences will be caught by checkpatch, but I don't
remember if these are.