There are couple of issues with current implementations and this series
tries to resolve the issues:
(a) All needed information are scattered in variables, passed to various
test functions. The code is organized in pretty much relaxed fashion.
(b) The page isn't allocated from buddy during page table entry modifying
tests. The page can be invalid, conflicting to the implementations
of set_xxx_at() on ARM64. The target page is accessed so that the iCache
can be flushed when execution permission is given on ARM64. Besides,
the target page can be unmapped and access to it causes kernel crash.
"struct pgtable_debug_args" is introduced to address issue (a). For issue
(b), the used page is allocated from buddy in page table entry modifying
tests. The corresponding tets will be skipped if we fail to allocate the
(huge) page. For other test cases, the original page around to kernel
symbol (@start_kernel) is still used.
The patches are organized as below. PATCH[2-10] could be combined to one
patch, but it will make the review harder:
PATCH[1] introduces "struct pgtable_debug_args" as place holder of all
needed information. With it, the old and new implementation
can coexist.
PATCH[2-10] uses "struct pgtable_debug_args" in various test functions.
PATCH[11] removes the unused code for old implementation.
PATCH[12] fixes the issue of corrupted page flag for ARM64
Changelog
=========
v4:
* Determine the page allocation method according to
MAX_ORDER (Anshuman)
* Move existing comments to init_args() (Anshuman)
* Code refactoring as suggested by Anshuman (Anshuman)
* Improved commit log and add comments for flush_dcache_page()
in PATCH[v4 12/12] (Anshuman)
* Address misc comments (Anshuman)
v3:
* Fix the warning caused by allocating more pages than
(1 << (MAX_ORDER - 1)) in init_args() (syzbot)
* Fix build warning by dropping unused variables in separate
patches (0-day)
* Missed "WARN_ON(!pud_none(pud))" in pud_huge_tests() in
PATCH[v2 09/12] (0-day)
* Fix the subjects for PATCH[05/12] and PATCH[09/12] (Gavin)
v2:
* Rename struct vm_pgtable_debug to struct pgtable_debug_args.
The parameter name to various test functions are renamed
to "@args" (Anshuman)
* Code changes as suggested by Anshuman (Anshuman)
Gavin Shan (12):
mm/debug_vm_pgtable: Introduce struct pgtable_debug_args
mm/debug_vm_pgtable: Use struct pgtable_debug_args in basic tests
mm/debug_vm_pgtable: Use struct pgtable_debug_args in leaf and
savewrite tests
mm/debug_vm_pgtable: Use struct pgtable_debug_args in protnone and
devmap tests
mm/debug_vm_pgtable: Use struct pgtable_debug_args in soft_dirty and
swap tests
mm/debug_vm_pgtable: Use struct pgtable_debug_args in migration and
thp tests
mm/debug_vm_pgtable: Use struct pgtable_debug_args in PTE modifying
tests
mm/debug_vm_pgtable: Use struct pgtable_debug_args in PMD modifying
tests
mm/debug_vm_pgtable: Use struct pgtable_debug_args in PUD modifying
tests
mm/debug_vm_pgtable: Use struct pgtable_debug_args in PGD and P4D
modifying tests
mm/debug_vm_pgtable: Remove unused code
mm/debug_vm_pgtable: Fix corrupted page flag
mm/debug_vm_pgtable.c | 919 ++++++++++++++++++++++++++----------------
1 file changed, 566 insertions(+), 353 deletions(-)
--
2.23.0
In debug_vm_pgtable(), there are many local variables introduced to
track the needed information and they are passed to the functions for
various test cases. It'd better to introduce a struct as place holder
for these information. With it, what the tests functions need is the
struct. In this way, the code is simplified and easier to be maintained.
Besides, set_xxx_at() could access the data on the corresponding pages
in the page table modifying tests. So the accessed pages in the tests
should have been allocated from buddy. Otherwise, we're accessing pages
that aren't owned by us. This causes issues like page flag corruption
or kernel crash on accessing unmapped page when CONFIG_DEBUG_PAGEALLOC
is enabled.
This introduces "struct pgtable_debug_args". The struct is initialized
and destroyed, but the information in the struct isn't used yet. It will
be used in subsequent patches.
Signed-off-by: Gavin Shan <[email protected]>
---
mm/debug_vm_pgtable.c | 280 +++++++++++++++++++++++++++++++++++++++++-
1 file changed, 279 insertions(+), 1 deletion(-)
diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
index 1c922691aa61..8c7361643166 100644
--- a/mm/debug_vm_pgtable.c
+++ b/mm/debug_vm_pgtable.c
@@ -58,6 +58,37 @@
#define RANDOM_ORVALUE (GENMASK(BITS_PER_LONG - 1, 0) & ~ARCH_SKIP_MASK)
#define RANDOM_NZVALUE GENMASK(7, 0)
+struct pgtable_debug_args {
+ struct mm_struct *mm;
+ struct vm_area_struct *vma;
+
+ pgd_t *pgdp;
+ p4d_t *p4dp;
+ pud_t *pudp;
+ pmd_t *pmdp;
+ pte_t *ptep;
+
+ p4d_t *start_p4dp;
+ pud_t *start_pudp;
+ pmd_t *start_pmdp;
+ pgtable_t start_ptep;
+
+ unsigned long vaddr;
+ pgprot_t page_prot;
+ pgprot_t page_prot_none;
+
+ bool is_contiguous_page;
+ unsigned long pud_pfn;
+ unsigned long pmd_pfn;
+ unsigned long pte_pfn;
+
+ unsigned long fixed_pgd_pfn;
+ unsigned long fixed_p4d_pfn;
+ unsigned long fixed_pud_pfn;
+ unsigned long fixed_pmd_pfn;
+ unsigned long fixed_pte_pfn;
+};
+
static void __init pte_basic_tests(unsigned long pfn, int idx)
{
pgprot_t prot = protection_map[idx];
@@ -955,8 +986,249 @@ static unsigned long __init get_random_vaddr(void)
return random_vaddr;
}
+static void __init destroy_args(struct pgtable_debug_args *args)
+{
+ struct page *page = NULL;
+
+ /* Free (huge) page */
+ if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) &&
+ IS_ENABLED(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD) &&
+ has_transparent_hugepage() &&
+ args->pud_pfn != ULONG_MAX) {
+ if (args->is_contiguous_page) {
+ free_contig_range(args->pud_pfn,
+ (1 << (HPAGE_PUD_SHIFT - PAGE_SHIFT)));
+ } else {
+ page = pfn_to_page(args->pud_pfn);
+ __free_pages(page, HPAGE_PUD_SHIFT - PAGE_SHIFT);
+ }
+
+ args->pud_pfn = ULONG_MAX;
+ args->pmd_pfn = ULONG_MAX;
+ args->pte_pfn = ULONG_MAX;
+ }
+
+ if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) &&
+ has_transparent_hugepage() &&
+ args->pmd_pfn != ULONG_MAX) {
+ if (args->is_contiguous_page) {
+ free_contig_range(args->pmd_pfn, (1 << HPAGE_PMD_ORDER));
+ } else {
+ page = pfn_to_page(args->pmd_pfn);
+ __free_pages(page, HPAGE_PMD_ORDER);
+ }
+
+ args->pmd_pfn = ULONG_MAX;
+ args->pte_pfn = ULONG_MAX;
+ }
+
+ if (args->pte_pfn != ULONG_MAX) {
+ page = pfn_to_page(args->pte_pfn);
+ __free_pages(page, 0);
+ }
+
+ /* Free page table entries */
+ if (args->start_ptep) {
+ pte_free(args->mm, args->start_ptep);
+ mm_dec_nr_ptes(args->mm);
+ }
+
+ if (args->start_pmdp) {
+ pmd_free(args->mm, args->start_pmdp);
+ mm_dec_nr_pmds(args->mm);
+ }
+
+ if (args->start_pudp) {
+ pud_free(args->mm, args->start_pudp);
+ mm_dec_nr_puds(args->mm);
+ }
+
+ if (args->start_p4dp)
+ p4d_free(args->mm, args->p4dp);
+
+ /* Free vma and mm struct */
+ if (args->vma)
+ vm_area_free(args->vma);
+ if (args->mm)
+ mmdrop(args->mm);
+}
+
+static int __init init_args(struct pgtable_debug_args *args)
+{
+ struct page *page = NULL;
+ phys_addr_t phys;
+ int ret = 0;
+
+ /*
+ * Initialize the debugging data.
+ *
+ * __P000 (or even __S000) will help create page table entries with
+ * PROT_NONE permission as required for pxx_protnone_tests().
+ */
+ memset(args, 0, sizeof(*args));
+ args->vaddr = get_random_vaddr();
+ args->page_prot = vm_get_page_prot(VMFLAGS);
+ args->page_prot_none = __P000;
+ args->is_contiguous_page = false;
+ args->pud_pfn = ULONG_MAX;
+ args->pmd_pfn = ULONG_MAX;
+ args->pte_pfn = ULONG_MAX;
+ args->fixed_pgd_pfn = ULONG_MAX;
+ args->fixed_p4d_pfn = ULONG_MAX;
+ args->fixed_pud_pfn = ULONG_MAX;
+ args->fixed_pmd_pfn = ULONG_MAX;
+ args->fixed_pte_pfn = ULONG_MAX;
+
+ /* Allocate mm and vma */
+ args->mm = mm_alloc();
+ if (!args->mm) {
+ pr_err("Failed to allocate mm struct\n");
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ args->vma = vm_area_alloc(args->mm);
+ if (!args->vma) {
+ pr_err("Failed to allocate vma\n");
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ /*
+ * Allocate page table entries. They will be modified in the tests.
+ * Lets save the page table entries so that they can be released
+ * when the tests are completed.
+ */
+ args->pgdp = pgd_offset(args->mm, args->vaddr);
+ args->p4dp = p4d_alloc(args->mm, args->pgdp, args->vaddr);
+ if (!args->p4dp) {
+ pr_err("Failed to allocate p4d entries\n");
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ args->start_p4dp = p4d_offset(args->pgdp, 0UL);
+ WARN_ON(!args->start_p4dp);
+ args->pudp = pud_alloc(args->mm, args->p4dp, args->vaddr);
+ if (!args->pudp) {
+ pr_err("Failed to allocate pud entries\n");
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ args->start_pudp = pud_offset(args->p4dp, 0UL);
+ WARN_ON(!args->start_pudp);
+ args->pmdp = pmd_alloc(args->mm, args->pudp, args->vaddr);
+ if (!args->pmdp) {
+ pr_err("Failed to allocate pmd entries\n");
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ args->start_pmdp = pmd_offset(args->pudp, 0UL);
+ WARN_ON(!args->start_pmdp);
+ args->ptep = pte_alloc_map(args->mm, args->pmdp, args->vaddr);
+ if (!args->ptep) {
+ pr_err("Failed to allocate pte entries\n");
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ args->start_ptep = pmd_pgtable(READ_ONCE(*args->pmdp));
+ WARN_ON(!args->start_ptep);
+
+ /*
+ * PFN for mapping at PTE level is determined from a standard kernel
+ * text symbol. But pfns for higher page table levels are derived by
+ * masking lower bits of this real pfn. These derived pfns might not
+ * exist on the platform but that does not really matter as pfn_pxx()
+ * helpers will still create appropriate entries for the test. This
+ * helps avoid large memory block allocations to be used for mapping
+ * at higher page table levels in some of the tests.
+ */
+ phys = __pa_symbol(&start_kernel);
+ args->fixed_pgd_pfn = __phys_to_pfn(phys & PGDIR_MASK);
+ args->fixed_p4d_pfn = __phys_to_pfn(phys & P4D_MASK);
+ args->fixed_pud_pfn = __phys_to_pfn(phys & PUD_MASK);
+ args->fixed_pmd_pfn = __phys_to_pfn(phys & PMD_MASK);
+ args->fixed_pte_pfn = __phys_to_pfn(phys & PAGE_MASK);
+ WARN_ON(!pfn_valid(args->fixed_pte_pfn));
+
+ /*
+ * Allocate (huge) pages because some of the tests need to access
+ * the data in the pages. The corresponding tests will be skipped
+ * if we fail to allocate (huge) pages.
+ */
+#ifdef CONFIG_CONTIG_ALLOC
+ if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) &&
+ IS_ENABLED(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD) &&
+ has_transparent_hugepage() &&
+ (HPAGE_PUD_SHIFT - PAGE_SHIFT) >= MAX_ORDER) {
+ page = alloc_contig_pages((1 << (HPAGE_PUD_SHIFT - PAGE_SHIFT)),
+ GFP_KERNEL, first_online_node, NULL);
+ if (page) {
+ args->is_contiguous_page = true;
+ args->pud_pfn = page_to_pfn(page);
+ args->pmd_pfn = args->pud_pfn;
+ args->pte_pfn = args->pud_pfn;
+ return 0;
+ }
+ }
+#endif
+
+ if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) &&
+ IS_ENABLED(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD) &&
+ has_transparent_hugepage() &&
+ (HPAGE_PUD_SHIFT - PAGE_SHIFT) < MAX_ORDER) {
+ page = alloc_pages(GFP_KERNEL, HPAGE_PUD_SHIFT - PAGE_SHIFT);
+ if (page) {
+ args->pud_pfn = page_to_pfn(page);
+ args->pmd_pfn = args->pud_pfn;
+ args->pte_pfn = args->pud_pfn;
+ return 0;
+ }
+ }
+
+#ifdef CONFIG_CONTIG_ALLOC
+ if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) &&
+ has_transparent_hugepage() &&
+ HPAGE_PMD_ORDER >= MAX_ORDER) {
+ page = alloc_contig_pages((1 << HPAGE_PMD_ORDER), GFP_KERNEL,
+ first_online_node, NULL);
+ if (page) {
+ args->is_contiguous_page = true;
+ args->pmd_pfn = page_to_pfn(page);
+ args->pte_pfn = args->pmd_pfn;
+ return 0;
+ }
+ }
+#endif
+
+ if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) &&
+ has_transparent_hugepage() &&
+ HPAGE_PMD_ORDER < MAX_ORDER) {
+ page = alloc_pages(GFP_KERNEL, HPAGE_PMD_ORDER);
+ if (page) {
+ args->pmd_pfn = page_to_pfn(page);
+ args->pte_pfn = args->pmd_pfn;
+ return 0;
+ }
+ }
+
+ page = alloc_pages(GFP_KERNEL, 0);
+ if (page)
+ args->pte_pfn = page_to_pfn(page);
+
+ return 0;
+
+error:
+ destroy_args(args);
+ return ret;
+}
+
static int __init debug_vm_pgtable(void)
{
+ struct pgtable_debug_args args;
struct vm_area_struct *vma;
struct mm_struct *mm;
pgd_t *pgdp;
@@ -970,9 +1242,13 @@ static int __init debug_vm_pgtable(void)
unsigned long vaddr, pte_aligned, pmd_aligned;
unsigned long pud_aligned, p4d_aligned, pgd_aligned;
spinlock_t *ptl = NULL;
- int idx;
+ int idx, ret;
pr_info("Validating architecture page table helpers\n");
+ ret = init_args(&args);
+ if (ret)
+ return ret;
+
prot = vm_get_page_prot(VMFLAGS);
vaddr = get_random_vaddr();
mm = mm_alloc();
@@ -1127,6 +1403,8 @@ static int __init debug_vm_pgtable(void)
mm_dec_nr_pmds(mm);
mm_dec_nr_ptes(mm);
mmdrop(mm);
+
+ destroy_args(&args);
return 0;
}
late_initcall(debug_vm_pgtable);
--
2.23.0
This uses struct pgtable_debug_args in protnone and devmap test
functions. After that, the unused variable @protnone in debug_vm_pgtable()
is dropped.
Signed-off-by: Gavin Shan <[email protected]>
---
mm/debug_vm_pgtable.c | 58 +++++++++++++++++++------------------------
1 file changed, 26 insertions(+), 32 deletions(-)
diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
index 0fa98083f7d5..650e35656660 100644
--- a/mm/debug_vm_pgtable.c
+++ b/mm/debug_vm_pgtable.c
@@ -662,9 +662,9 @@ static void __init pmd_populate_tests(struct mm_struct *mm, pmd_t *pmdp,
WARN_ON(pmd_bad(pmd));
}
-static void __init pte_special_tests(unsigned long pfn, pgprot_t prot)
+static void __init pte_special_tests(struct pgtable_debug_args *args)
{
- pte_t pte = pfn_pte(pfn, prot);
+ pte_t pte = pfn_pte(args->fixed_pte_pfn, args->page_prot);
if (!IS_ENABLED(CONFIG_ARCH_HAS_PTE_SPECIAL))
return;
@@ -673,9 +673,9 @@ static void __init pte_special_tests(unsigned long pfn, pgprot_t prot)
WARN_ON(!pte_special(pte_mkspecial(pte)));
}
-static void __init pte_protnone_tests(unsigned long pfn, pgprot_t prot)
+static void __init pte_protnone_tests(struct pgtable_debug_args *args)
{
- pte_t pte = pfn_pte(pfn, prot);
+ pte_t pte = pfn_pte(args->fixed_pte_pfn, args->page_prot_none);
if (!IS_ENABLED(CONFIG_NUMA_BALANCING))
return;
@@ -686,7 +686,7 @@ static void __init pte_protnone_tests(unsigned long pfn, pgprot_t prot)
}
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
-static void __init pmd_protnone_tests(unsigned long pfn, pgprot_t prot)
+static void __init pmd_protnone_tests(struct pgtable_debug_args *args)
{
pmd_t pmd;
@@ -697,25 +697,25 @@ static void __init pmd_protnone_tests(unsigned long pfn, pgprot_t prot)
return;
pr_debug("Validating PMD protnone\n");
- pmd = pmd_mkhuge(pfn_pmd(pfn, prot));
+ pmd = pmd_mkhuge(pfn_pmd(args->fixed_pmd_pfn, args->page_prot_none));
WARN_ON(!pmd_protnone(pmd));
WARN_ON(!pmd_present(pmd));
}
#else /* !CONFIG_TRANSPARENT_HUGEPAGE */
-static void __init pmd_protnone_tests(unsigned long pfn, pgprot_t prot) { }
+static void __init pmd_protnone_tests(struct pgtable_debug_args *args) { }
#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
#ifdef CONFIG_ARCH_HAS_PTE_DEVMAP
-static void __init pte_devmap_tests(unsigned long pfn, pgprot_t prot)
+static void __init pte_devmap_tests(struct pgtable_debug_args *args)
{
- pte_t pte = pfn_pte(pfn, prot);
+ pte_t pte = pfn_pte(args->fixed_pte_pfn, args->page_prot);
pr_debug("Validating PTE devmap\n");
WARN_ON(!pte_devmap(pte_mkdevmap(pte)));
}
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
-static void __init pmd_devmap_tests(unsigned long pfn, pgprot_t prot)
+static void __init pmd_devmap_tests(struct pgtable_debug_args *args)
{
pmd_t pmd;
@@ -723,12 +723,12 @@ static void __init pmd_devmap_tests(unsigned long pfn, pgprot_t prot)
return;
pr_debug("Validating PMD devmap\n");
- pmd = pfn_pmd(pfn, prot);
+ pmd = pfn_pmd(args->fixed_pmd_pfn, args->page_prot);
WARN_ON(!pmd_devmap(pmd_mkdevmap(pmd)));
}
#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
-static void __init pud_devmap_tests(unsigned long pfn, pgprot_t prot)
+static void __init pud_devmap_tests(struct pgtable_debug_args *args)
{
pud_t pud;
@@ -736,20 +736,20 @@ static void __init pud_devmap_tests(unsigned long pfn, pgprot_t prot)
return;
pr_debug("Validating PUD devmap\n");
- pud = pfn_pud(pfn, prot);
+ pud = pfn_pud(args->fixed_pud_pfn, args->page_prot);
WARN_ON(!pud_devmap(pud_mkdevmap(pud)));
}
#else /* !CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */
-static void __init pud_devmap_tests(unsigned long pfn, pgprot_t prot) { }
+static void __init pud_devmap_tests(struct pgtable_debug_args *args) { }
#endif /* CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */
#else /* CONFIG_TRANSPARENT_HUGEPAGE */
-static void __init pmd_devmap_tests(unsigned long pfn, pgprot_t prot) { }
-static void __init pud_devmap_tests(unsigned long pfn, pgprot_t prot) { }
+static void __init pmd_devmap_tests(struct pgtable_debug_args *args) { }
+static void __init pud_devmap_tests(struct pgtable_debug_args *args) { }
#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
#else
-static void __init pte_devmap_tests(unsigned long pfn, pgprot_t prot) { }
-static void __init pmd_devmap_tests(unsigned long pfn, pgprot_t prot) { }
-static void __init pud_devmap_tests(unsigned long pfn, pgprot_t prot) { }
+static void __init pte_devmap_tests(struct pgtable_debug_args *args) { }
+static void __init pmd_devmap_tests(struct pgtable_debug_args *args) { }
+static void __init pud_devmap_tests(struct pgtable_debug_args *args) { }
#endif /* CONFIG_ARCH_HAS_PTE_DEVMAP */
static void __init pte_soft_dirty_tests(unsigned long pfn, pgprot_t prot)
@@ -1237,7 +1237,7 @@ static int __init debug_vm_pgtable(void)
pmd_t *pmdp, *saved_pmdp, pmd;
pte_t *ptep;
pgtable_t saved_ptep;
- pgprot_t prot, protnone;
+ pgprot_t prot;
phys_addr_t paddr;
unsigned long vaddr, pte_aligned, pmd_aligned;
unsigned long pud_aligned;
@@ -1257,12 +1257,6 @@ static int __init debug_vm_pgtable(void)
return 1;
}
- /*
- * __P000 (or even __S000) will help create page table entries with
- * PROT_NONE permission as required for pxx_protnone_tests().
- */
- protnone = __P000;
-
vma = vm_area_alloc(mm);
if (!vma) {
pr_err("vma allocation failed\n");
@@ -1337,13 +1331,13 @@ static int __init debug_vm_pgtable(void)
pte_savedwrite_tests(&args);
pmd_savedwrite_tests(&args);
- pte_special_tests(pte_aligned, prot);
- pte_protnone_tests(pte_aligned, protnone);
- pmd_protnone_tests(pmd_aligned, protnone);
+ pte_special_tests(&args);
+ pte_protnone_tests(&args);
+ pmd_protnone_tests(&args);
- pte_devmap_tests(pte_aligned, prot);
- pmd_devmap_tests(pmd_aligned, prot);
- pud_devmap_tests(pud_aligned, prot);
+ pte_devmap_tests(&args);
+ pmd_devmap_tests(&args);
+ pud_devmap_tests(&args);
pte_soft_dirty_tests(pte_aligned, prot);
pmd_soft_dirty_tests(pmd_aligned, prot);
--
2.23.0
This uses struct pgtable_debug_args in the basic test functions. The
unused variables @pgd_aligned and @p4d_aligned in debug_vm_pgtable()
are dropped.
Signed-off-by: Gavin Shan <[email protected]>
---
mm/debug_vm_pgtable.c | 50 +++++++++++++++++++++----------------------
1 file changed, 24 insertions(+), 26 deletions(-)
diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
index 8c7361643166..8498aa180ebc 100644
--- a/mm/debug_vm_pgtable.c
+++ b/mm/debug_vm_pgtable.c
@@ -89,10 +89,10 @@ struct pgtable_debug_args {
unsigned long fixed_pte_pfn;
};
-static void __init pte_basic_tests(unsigned long pfn, int idx)
+static void __init pte_basic_tests(struct pgtable_debug_args *args, int idx)
{
pgprot_t prot = protection_map[idx];
- pte_t pte = pfn_pte(pfn, prot);
+ pte_t pte = pfn_pte(args->fixed_pte_pfn, prot);
unsigned long val = idx, *ptr = &val;
pr_debug("Validating PTE basic (%pGv)\n", ptr);
@@ -174,7 +174,7 @@ static void __init pte_savedwrite_tests(unsigned long pfn, pgprot_t prot)
}
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
-static void __init pmd_basic_tests(unsigned long pfn, int idx)
+static void __init pmd_basic_tests(struct pgtable_debug_args *args, int idx)
{
pgprot_t prot = protection_map[idx];
unsigned long val = idx, *ptr = &val;
@@ -184,7 +184,7 @@ static void __init pmd_basic_tests(unsigned long pfn, int idx)
return;
pr_debug("Validating PMD basic (%pGv)\n", ptr);
- pmd = pfn_pmd(pfn, prot);
+ pmd = pfn_pmd(args->fixed_pmd_pfn, prot);
/*
* This test needs to be executed after the given page table entry
@@ -296,7 +296,7 @@ static void __init pmd_savedwrite_tests(unsigned long pfn, pgprot_t prot)
}
#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
-static void __init pud_basic_tests(struct mm_struct *mm, unsigned long pfn, int idx)
+static void __init pud_basic_tests(struct pgtable_debug_args *args, int idx)
{
pgprot_t prot = protection_map[idx];
unsigned long val = idx, *ptr = &val;
@@ -306,7 +306,7 @@ static void __init pud_basic_tests(struct mm_struct *mm, unsigned long pfn, int
return;
pr_debug("Validating PUD basic (%pGv)\n", ptr);
- pud = pfn_pud(pfn, prot);
+ pud = pfn_pud(args->fixed_pud_pfn, prot);
/*
* This test needs to be executed after the given page table entry
@@ -327,7 +327,7 @@ static void __init pud_basic_tests(struct mm_struct *mm, unsigned long pfn, int
WARN_ON(pud_dirty(pud_wrprotect(pud_mkclean(pud))));
WARN_ON(!pud_dirty(pud_wrprotect(pud_mkdirty(pud))));
- if (mm_pmd_folded(mm))
+ if (mm_pmd_folded(args->mm))
return;
/*
@@ -404,7 +404,7 @@ static void __init pud_leaf_tests(unsigned long pfn, pgprot_t prot)
WARN_ON(!pud_leaf(pud));
}
#else /* !CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */
-static void __init pud_basic_tests(struct mm_struct *mm, unsigned long pfn, int idx) { }
+static void __init pud_basic_tests(struct pgtable_debug_args *args, int idx) { }
static void __init pud_advanced_tests(struct mm_struct *mm,
struct vm_area_struct *vma, pud_t *pudp,
unsigned long pfn, unsigned long vaddr,
@@ -414,8 +414,8 @@ static void __init pud_advanced_tests(struct mm_struct *mm,
static void __init pud_leaf_tests(unsigned long pfn, pgprot_t prot) { }
#endif /* CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */
#else /* !CONFIG_TRANSPARENT_HUGEPAGE */
-static void __init pmd_basic_tests(unsigned long pfn, int idx) { }
-static void __init pud_basic_tests(struct mm_struct *mm, unsigned long pfn, int idx) { }
+static void __init pmd_basic_tests(struct pgtable_debug_args *args, int idx) { }
+static void __init pud_basic_tests(struct pgtable_debug_args *args, int idx) { }
static void __init pmd_advanced_tests(struct mm_struct *mm,
struct vm_area_struct *vma, pmd_t *pmdp,
unsigned long pfn, unsigned long vaddr,
@@ -476,7 +476,7 @@ static void __init pmd_huge_tests(pmd_t *pmdp, unsigned long pfn, pgprot_t prot)
static void __init pud_huge_tests(pud_t *pudp, unsigned long pfn, pgprot_t prot) { }
#endif /* CONFIG_HAVE_ARCH_HUGE_VMAP */
-static void __init p4d_basic_tests(unsigned long pfn, pgprot_t prot)
+static void __init p4d_basic_tests(struct pgtable_debug_args *args)
{
p4d_t p4d;
@@ -485,7 +485,7 @@ static void __init p4d_basic_tests(unsigned long pfn, pgprot_t prot)
WARN_ON(!p4d_same(p4d, p4d));
}
-static void __init pgd_basic_tests(unsigned long pfn, pgprot_t prot)
+static void __init pgd_basic_tests(struct pgtable_debug_args *args)
{
pgd_t pgd;
@@ -890,7 +890,7 @@ static void __init swap_migration_tests(void)
}
#ifdef CONFIG_HUGETLB_PAGE
-static void __init hugetlb_basic_tests(unsigned long pfn, pgprot_t prot)
+static void __init hugetlb_basic_tests(struct pgtable_debug_args *args)
{
struct page *page;
pte_t pte;
@@ -900,21 +900,21 @@ static void __init hugetlb_basic_tests(unsigned long pfn, pgprot_t prot)
* Accessing the page associated with the pfn is safe here,
* as it was previously derived from a real kernel symbol.
*/
- page = pfn_to_page(pfn);
- pte = mk_huge_pte(page, prot);
+ page = pfn_to_page(args->fixed_pmd_pfn);
+ pte = mk_huge_pte(page, args->page_prot);
WARN_ON(!huge_pte_dirty(huge_pte_mkdirty(pte)));
WARN_ON(!huge_pte_write(huge_pte_mkwrite(huge_pte_wrprotect(pte))));
WARN_ON(huge_pte_write(huge_pte_wrprotect(huge_pte_mkwrite(pte))));
#ifdef CONFIG_ARCH_WANT_GENERAL_HUGETLB
- pte = pfn_pte(pfn, prot);
+ pte = pfn_pte(args->fixed_pmd_pfn, args->page_prot);
WARN_ON(!pte_huge(pte_mkhuge(pte)));
#endif /* CONFIG_ARCH_WANT_GENERAL_HUGETLB */
}
#else /* !CONFIG_HUGETLB_PAGE */
-static void __init hugetlb_basic_tests(unsigned long pfn, pgprot_t prot) { }
+static void __init hugetlb_basic_tests(struct pgtable_debug_args *args) { }
#endif /* CONFIG_HUGETLB_PAGE */
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
@@ -1240,7 +1240,7 @@ static int __init debug_vm_pgtable(void)
pgprot_t prot, protnone;
phys_addr_t paddr;
unsigned long vaddr, pte_aligned, pmd_aligned;
- unsigned long pud_aligned, p4d_aligned, pgd_aligned;
+ unsigned long pud_aligned;
spinlock_t *ptl = NULL;
int idx, ret;
@@ -1283,8 +1283,6 @@ static int __init debug_vm_pgtable(void)
pte_aligned = (paddr & PAGE_MASK) >> PAGE_SHIFT;
pmd_aligned = (paddr & PMD_MASK) >> PAGE_SHIFT;
pud_aligned = (paddr & PUD_MASK) >> PAGE_SHIFT;
- p4d_aligned = (paddr & P4D_MASK) >> PAGE_SHIFT;
- pgd_aligned = (paddr & PGDIR_MASK) >> PAGE_SHIFT;
WARN_ON(!pfn_valid(pte_aligned));
pgdp = pgd_offset(mm, vaddr);
@@ -1318,9 +1316,9 @@ static int __init debug_vm_pgtable(void)
* given page table entry.
*/
for (idx = 0; idx < ARRAY_SIZE(protection_map); idx++) {
- pte_basic_tests(pte_aligned, idx);
- pmd_basic_tests(pmd_aligned, idx);
- pud_basic_tests(mm, pud_aligned, idx);
+ pte_basic_tests(&args, idx);
+ pmd_basic_tests(&args, idx);
+ pud_basic_tests(&args, idx);
}
/*
@@ -1330,8 +1328,8 @@ static int __init debug_vm_pgtable(void)
* the above iteration for now to save some test execution
* time.
*/
- p4d_basic_tests(p4d_aligned, prot);
- pgd_basic_tests(pgd_aligned, prot);
+ p4d_basic_tests(&args);
+ pgd_basic_tests(&args);
pmd_leaf_tests(pmd_aligned, prot);
pud_leaf_tests(pud_aligned, prot);
@@ -1360,7 +1358,7 @@ static int __init debug_vm_pgtable(void)
pmd_thp_tests(pmd_aligned, prot);
pud_thp_tests(pud_aligned, prot);
- hugetlb_basic_tests(pte_aligned, prot);
+ hugetlb_basic_tests(&args);
/*
* Page table modifying tests. They need to hold
--
2.23.0
This uses struct pgtable_debug_args in the leaf and savewrite test
functions.
Signed-off-by: Gavin Shan <[email protected]>
---
mm/debug_vm_pgtable.c | 32 ++++++++++++++++----------------
1 file changed, 16 insertions(+), 16 deletions(-)
diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
index 8498aa180ebc..0fa98083f7d5 100644
--- a/mm/debug_vm_pgtable.c
+++ b/mm/debug_vm_pgtable.c
@@ -161,9 +161,9 @@ static void __init pte_advanced_tests(struct mm_struct *mm,
WARN_ON(pte_young(pte));
}
-static void __init pte_savedwrite_tests(unsigned long pfn, pgprot_t prot)
+static void __init pte_savedwrite_tests(struct pgtable_debug_args *args)
{
- pte_t pte = pfn_pte(pfn, prot);
+ pte_t pte = pfn_pte(args->fixed_pte_pfn, args->page_prot_none);
if (!IS_ENABLED(CONFIG_NUMA_BALANCING))
return;
@@ -262,7 +262,7 @@ static void __init pmd_advanced_tests(struct mm_struct *mm,
pgtable = pgtable_trans_huge_withdraw(mm, pmdp);
}
-static void __init pmd_leaf_tests(unsigned long pfn, pgprot_t prot)
+static void __init pmd_leaf_tests(struct pgtable_debug_args *args)
{
pmd_t pmd;
@@ -270,7 +270,7 @@ static void __init pmd_leaf_tests(unsigned long pfn, pgprot_t prot)
return;
pr_debug("Validating PMD leaf\n");
- pmd = pfn_pmd(pfn, prot);
+ pmd = pfn_pmd(args->fixed_pmd_pfn, args->page_prot);
/*
* PMD based THP is a leaf entry.
@@ -279,7 +279,7 @@ static void __init pmd_leaf_tests(unsigned long pfn, pgprot_t prot)
WARN_ON(!pmd_leaf(pmd));
}
-static void __init pmd_savedwrite_tests(unsigned long pfn, pgprot_t prot)
+static void __init pmd_savedwrite_tests(struct pgtable_debug_args *args)
{
pmd_t pmd;
@@ -290,7 +290,7 @@ static void __init pmd_savedwrite_tests(unsigned long pfn, pgprot_t prot)
return;
pr_debug("Validating PMD saved write\n");
- pmd = pfn_pmd(pfn, prot);
+ pmd = pfn_pmd(args->fixed_pmd_pfn, args->page_prot_none);
WARN_ON(!pmd_savedwrite(pmd_mk_savedwrite(pmd_clear_savedwrite(pmd))));
WARN_ON(pmd_savedwrite(pmd_clear_savedwrite(pmd_mk_savedwrite(pmd))));
}
@@ -388,7 +388,7 @@ static void __init pud_advanced_tests(struct mm_struct *mm,
pudp_huge_get_and_clear(mm, vaddr, pudp);
}
-static void __init pud_leaf_tests(unsigned long pfn, pgprot_t prot)
+static void __init pud_leaf_tests(struct pgtable_debug_args *args)
{
pud_t pud;
@@ -396,7 +396,7 @@ static void __init pud_leaf_tests(unsigned long pfn, pgprot_t prot)
return;
pr_debug("Validating PUD leaf\n");
- pud = pfn_pud(pfn, prot);
+ pud = pfn_pud(args->fixed_pud_pfn, args->page_prot);
/*
* PUD based THP is a leaf entry.
*/
@@ -411,7 +411,7 @@ static void __init pud_advanced_tests(struct mm_struct *mm,
pgprot_t prot)
{
}
-static void __init pud_leaf_tests(unsigned long pfn, pgprot_t prot) { }
+static void __init pud_leaf_tests(struct pgtable_debug_args *args) { }
#endif /* CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */
#else /* !CONFIG_TRANSPARENT_HUGEPAGE */
static void __init pmd_basic_tests(struct pgtable_debug_args *args, int idx) { }
@@ -428,9 +428,9 @@ static void __init pud_advanced_tests(struct mm_struct *mm,
pgprot_t prot)
{
}
-static void __init pmd_leaf_tests(unsigned long pfn, pgprot_t prot) { }
-static void __init pud_leaf_tests(unsigned long pfn, pgprot_t prot) { }
-static void __init pmd_savedwrite_tests(unsigned long pfn, pgprot_t prot) { }
+static void __init pmd_leaf_tests(struct pgtable_debug_args *args) { }
+static void __init pud_leaf_tests(struct pgtable_debug_args *args) { }
+static void __init pmd_savedwrite_tests(struct pgtable_debug_args *args) { }
#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
#ifdef CONFIG_HAVE_ARCH_HUGE_VMAP
@@ -1331,11 +1331,11 @@ static int __init debug_vm_pgtable(void)
p4d_basic_tests(&args);
pgd_basic_tests(&args);
- pmd_leaf_tests(pmd_aligned, prot);
- pud_leaf_tests(pud_aligned, prot);
+ pmd_leaf_tests(&args);
+ pud_leaf_tests(&args);
- pte_savedwrite_tests(pte_aligned, protnone);
- pmd_savedwrite_tests(pmd_aligned, protnone);
+ pte_savedwrite_tests(&args);
+ pmd_savedwrite_tests(&args);
pte_special_tests(pte_aligned, prot);
pte_protnone_tests(pte_aligned, protnone);
--
2.23.0
This uses struct pgtable_debug_args in the soft_dirty and swap test
functions.
Signed-off-by: Gavin Shan <[email protected]>
---
mm/debug_vm_pgtable.c | 48 +++++++++++++++++++++----------------------
1 file changed, 23 insertions(+), 25 deletions(-)
diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
index 650e35656660..bc153cad9045 100644
--- a/mm/debug_vm_pgtable.c
+++ b/mm/debug_vm_pgtable.c
@@ -752,9 +752,9 @@ static void __init pmd_devmap_tests(struct pgtable_debug_args *args) { }
static void __init pud_devmap_tests(struct pgtable_debug_args *args) { }
#endif /* CONFIG_ARCH_HAS_PTE_DEVMAP */
-static void __init pte_soft_dirty_tests(unsigned long pfn, pgprot_t prot)
+static void __init pte_soft_dirty_tests(struct pgtable_debug_args *args)
{
- pte_t pte = pfn_pte(pfn, prot);
+ pte_t pte = pfn_pte(args->fixed_pte_pfn, args->page_prot);
if (!IS_ENABLED(CONFIG_MEM_SOFT_DIRTY))
return;
@@ -764,9 +764,9 @@ static void __init pte_soft_dirty_tests(unsigned long pfn, pgprot_t prot)
WARN_ON(pte_soft_dirty(pte_clear_soft_dirty(pte)));
}
-static void __init pte_swap_soft_dirty_tests(unsigned long pfn, pgprot_t prot)
+static void __init pte_swap_soft_dirty_tests(struct pgtable_debug_args *args)
{
- pte_t pte = pfn_pte(pfn, prot);
+ pte_t pte = pfn_pte(args->fixed_pte_pfn, args->page_prot);
if (!IS_ENABLED(CONFIG_MEM_SOFT_DIRTY))
return;
@@ -777,7 +777,7 @@ static void __init pte_swap_soft_dirty_tests(unsigned long pfn, pgprot_t prot)
}
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
-static void __init pmd_soft_dirty_tests(unsigned long pfn, pgprot_t prot)
+static void __init pmd_soft_dirty_tests(struct pgtable_debug_args *args)
{
pmd_t pmd;
@@ -788,12 +788,12 @@ static void __init pmd_soft_dirty_tests(unsigned long pfn, pgprot_t prot)
return;
pr_debug("Validating PMD soft dirty\n");
- pmd = pfn_pmd(pfn, prot);
+ pmd = pfn_pmd(args->fixed_pmd_pfn, args->page_prot);
WARN_ON(!pmd_soft_dirty(pmd_mksoft_dirty(pmd)));
WARN_ON(pmd_soft_dirty(pmd_clear_soft_dirty(pmd)));
}
-static void __init pmd_swap_soft_dirty_tests(unsigned long pfn, pgprot_t prot)
+static void __init pmd_swap_soft_dirty_tests(struct pgtable_debug_args *args)
{
pmd_t pmd;
@@ -805,31 +805,29 @@ static void __init pmd_swap_soft_dirty_tests(unsigned long pfn, pgprot_t prot)
return;
pr_debug("Validating PMD swap soft dirty\n");
- pmd = pfn_pmd(pfn, prot);
+ pmd = pfn_pmd(args->fixed_pmd_pfn, args->page_prot);
WARN_ON(!pmd_swp_soft_dirty(pmd_swp_mksoft_dirty(pmd)));
WARN_ON(pmd_swp_soft_dirty(pmd_swp_clear_soft_dirty(pmd)));
}
#else /* !CONFIG_TRANSPARENT_HUGEPAGE */
-static void __init pmd_soft_dirty_tests(unsigned long pfn, pgprot_t prot) { }
-static void __init pmd_swap_soft_dirty_tests(unsigned long pfn, pgprot_t prot)
-{
-}
+static void __init pmd_soft_dirty_tests(struct pgtable_debug_args *args) { }
+static void __init pmd_swap_soft_dirty_tests(struct pgtable_debug_args *args) { }
#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
-static void __init pte_swap_tests(unsigned long pfn, pgprot_t prot)
+static void __init pte_swap_tests(struct pgtable_debug_args *args)
{
swp_entry_t swp;
pte_t pte;
pr_debug("Validating PTE swap\n");
- pte = pfn_pte(pfn, prot);
+ pte = pfn_pte(args->fixed_pte_pfn, args->page_prot);
swp = __pte_to_swp_entry(pte);
pte = __swp_entry_to_pte(swp);
- WARN_ON(pfn != pte_pfn(pte));
+ WARN_ON(args->fixed_pte_pfn != pte_pfn(pte));
}
#ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION
-static void __init pmd_swap_tests(unsigned long pfn, pgprot_t prot)
+static void __init pmd_swap_tests(struct pgtable_debug_args *args)
{
swp_entry_t swp;
pmd_t pmd;
@@ -838,13 +836,13 @@ static void __init pmd_swap_tests(unsigned long pfn, pgprot_t prot)
return;
pr_debug("Validating PMD swap\n");
- pmd = pfn_pmd(pfn, prot);
+ pmd = pfn_pmd(args->fixed_pmd_pfn, args->page_prot);
swp = __pmd_to_swp_entry(pmd);
pmd = __swp_entry_to_pmd(swp);
- WARN_ON(pfn != pmd_pfn(pmd));
+ WARN_ON(args->fixed_pmd_pfn != pmd_pfn(pmd));
}
#else /* !CONFIG_ARCH_ENABLE_THP_MIGRATION */
-static void __init pmd_swap_tests(unsigned long pfn, pgprot_t prot) { }
+static void __init pmd_swap_tests(struct pgtable_debug_args *args) { }
#endif /* CONFIG_ARCH_ENABLE_THP_MIGRATION */
static void __init swap_migration_tests(void)
@@ -1339,13 +1337,13 @@ static int __init debug_vm_pgtable(void)
pmd_devmap_tests(&args);
pud_devmap_tests(&args);
- pte_soft_dirty_tests(pte_aligned, prot);
- pmd_soft_dirty_tests(pmd_aligned, prot);
- pte_swap_soft_dirty_tests(pte_aligned, prot);
- pmd_swap_soft_dirty_tests(pmd_aligned, prot);
+ pte_soft_dirty_tests(&args);
+ pmd_soft_dirty_tests(&args);
+ pte_swap_soft_dirty_tests(&args);
+ pmd_swap_soft_dirty_tests(&args);
- pte_swap_tests(pte_aligned, prot);
- pmd_swap_tests(pmd_aligned, prot);
+ pte_swap_tests(&args);
+ pmd_swap_tests(&args);
swap_migration_tests();
--
2.23.0
This uses struct pgtable_debug_args in the migration and thp test
functions. It's notable that the pre-allocated page is used in
swap_migration_tests() as set_pte_at() is used there.
Signed-off-by: Gavin Shan <[email protected]>
---
mm/debug_vm_pgtable.c | 28 ++++++++++++++--------------
1 file changed, 14 insertions(+), 14 deletions(-)
diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
index bc153cad9045..9136195efde3 100644
--- a/mm/debug_vm_pgtable.c
+++ b/mm/debug_vm_pgtable.c
@@ -845,7 +845,7 @@ static void __init pmd_swap_tests(struct pgtable_debug_args *args)
static void __init pmd_swap_tests(struct pgtable_debug_args *args) { }
#endif /* CONFIG_ARCH_ENABLE_THP_MIGRATION */
-static void __init swap_migration_tests(void)
+static void __init swap_migration_tests(struct pgtable_debug_args *args)
{
struct page *page;
swp_entry_t swp;
@@ -861,9 +861,10 @@ static void __init swap_migration_tests(void)
* problematic. Lets allocate a dedicated page explicitly for this
* purpose that will be freed subsequently.
*/
- page = alloc_page(GFP_KERNEL);
+ page = (args->pte_pfn != ULONG_MAX) ?
+ pfn_to_page(args->pte_pfn) : NULL;
if (!page) {
- pr_err("page allocation failed\n");
+ pr_err("no page available\n");
return;
}
@@ -884,7 +885,6 @@ static void __init swap_migration_tests(void)
WARN_ON(!is_migration_entry(swp));
WARN_ON(is_writable_migration_entry(swp));
__ClearPageLocked(page);
- __free_page(page);
}
#ifdef CONFIG_HUGETLB_PAGE
@@ -916,7 +916,7 @@ static void __init hugetlb_basic_tests(struct pgtable_debug_args *args) { }
#endif /* CONFIG_HUGETLB_PAGE */
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
-static void __init pmd_thp_tests(unsigned long pfn, pgprot_t prot)
+static void __init pmd_thp_tests(struct pgtable_debug_args *args)
{
pmd_t pmd;
@@ -935,7 +935,7 @@ static void __init pmd_thp_tests(unsigned long pfn, pgprot_t prot)
* needs to return true. pmd_present() should be true whenever
* pmd_trans_huge() returns true.
*/
- pmd = pfn_pmd(pfn, prot);
+ pmd = pfn_pmd(args->fixed_pmd_pfn, args->page_prot);
WARN_ON(!pmd_trans_huge(pmd_mkhuge(pmd)));
#ifndef __HAVE_ARCH_PMDP_INVALIDATE
@@ -945,7 +945,7 @@ static void __init pmd_thp_tests(unsigned long pfn, pgprot_t prot)
}
#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
-static void __init pud_thp_tests(unsigned long pfn, pgprot_t prot)
+static void __init pud_thp_tests(struct pgtable_debug_args *args)
{
pud_t pud;
@@ -953,7 +953,7 @@ static void __init pud_thp_tests(unsigned long pfn, pgprot_t prot)
return;
pr_debug("Validating PUD based THP\n");
- pud = pfn_pud(pfn, prot);
+ pud = pfn_pud(args->fixed_pud_pfn, args->page_prot);
WARN_ON(!pud_trans_huge(pud_mkhuge(pud)));
/*
@@ -965,11 +965,11 @@ static void __init pud_thp_tests(unsigned long pfn, pgprot_t prot)
*/
}
#else /* !CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */
-static void __init pud_thp_tests(unsigned long pfn, pgprot_t prot) { }
+static void __init pud_thp_tests(struct pgtable_debug_args *args) { }
#endif /* CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */
#else /* !CONFIG_TRANSPARENT_HUGEPAGE */
-static void __init pmd_thp_tests(unsigned long pfn, pgprot_t prot) { }
-static void __init pud_thp_tests(unsigned long pfn, pgprot_t prot) { }
+static void __init pmd_thp_tests(struct pgtable_debug_args *args) { }
+static void __init pud_thp_tests(struct pgtable_debug_args *args) { }
#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
static unsigned long __init get_random_vaddr(void)
@@ -1345,10 +1345,10 @@ static int __init debug_vm_pgtable(void)
pte_swap_tests(&args);
pmd_swap_tests(&args);
- swap_migration_tests();
+ swap_migration_tests(&args);
- pmd_thp_tests(pmd_aligned, prot);
- pud_thp_tests(pud_aligned, prot);
+ pmd_thp_tests(&args);
+ pud_thp_tests(&args);
hugetlb_basic_tests(&args);
--
2.23.0
This uses struct pgtable_debug_args in PGD/P4D modifying tests. No
allocated huge page is used in these tests. Besides, the unused
variable @saved_p4dp and @saved_pudp are dropped.
Signed-off-by: Gavin Shan <[email protected]>
---
mm/debug_vm_pgtable.c | 86 +++++++++++++++++++------------------------
1 file changed, 38 insertions(+), 48 deletions(-)
diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
index 863871e70268..0611b5f31e89 100644
--- a/mm/debug_vm_pgtable.c
+++ b/mm/debug_vm_pgtable.c
@@ -518,27 +518,26 @@ static void __init pud_populate_tests(struct pgtable_debug_args *args) { }
#endif /* PAGETABLE_PUD_FOLDED */
#ifndef __PAGETABLE_P4D_FOLDED
-static void __init p4d_clear_tests(struct mm_struct *mm, p4d_t *p4dp)
+static void __init p4d_clear_tests(struct pgtable_debug_args *args)
{
- p4d_t p4d = READ_ONCE(*p4dp);
+ p4d_t p4d = READ_ONCE(*args->p4dp);
- if (mm_pud_folded(mm))
+ if (mm_pud_folded(args->mm))
return;
pr_debug("Validating P4D clear\n");
p4d = __p4d(p4d_val(p4d) | RANDOM_ORVALUE);
- WRITE_ONCE(*p4dp, p4d);
- p4d_clear(p4dp);
- p4d = READ_ONCE(*p4dp);
+ WRITE_ONCE(*args->p4dp, p4d);
+ p4d_clear(args->p4dp);
+ p4d = READ_ONCE(*args->p4dp);
WARN_ON(!p4d_none(p4d));
}
-static void __init p4d_populate_tests(struct mm_struct *mm, p4d_t *p4dp,
- pud_t *pudp)
+static void __init p4d_populate_tests(struct pgtable_debug_args *args)
{
p4d_t p4d;
- if (mm_pud_folded(mm))
+ if (mm_pud_folded(args->mm))
return;
pr_debug("Validating P4D populate\n");
@@ -546,34 +545,33 @@ static void __init p4d_populate_tests(struct mm_struct *mm, p4d_t *p4dp,
* This entry points to next level page table page.
* Hence this must not qualify as p4d_bad().
*/
- pud_clear(pudp);
- p4d_clear(p4dp);
- p4d_populate(mm, p4dp, pudp);
- p4d = READ_ONCE(*p4dp);
+ pud_clear(args->pudp);
+ p4d_clear(args->p4dp);
+ p4d_populate(args->mm, args->p4dp, args->start_pudp);
+ p4d = READ_ONCE(*args->p4dp);
WARN_ON(p4d_bad(p4d));
}
-static void __init pgd_clear_tests(struct mm_struct *mm, pgd_t *pgdp)
+static void __init pgd_clear_tests(struct pgtable_debug_args *args)
{
- pgd_t pgd = READ_ONCE(*pgdp);
+ pgd_t pgd = READ_ONCE(*(args->pgdp));
- if (mm_p4d_folded(mm))
+ if (mm_p4d_folded(args->mm))
return;
pr_debug("Validating PGD clear\n");
pgd = __pgd(pgd_val(pgd) | RANDOM_ORVALUE);
- WRITE_ONCE(*pgdp, pgd);
- pgd_clear(pgdp);
- pgd = READ_ONCE(*pgdp);
+ WRITE_ONCE(*args->pgdp, pgd);
+ pgd_clear(args->pgdp);
+ pgd = READ_ONCE(*args->pgdp);
WARN_ON(!pgd_none(pgd));
}
-static void __init pgd_populate_tests(struct mm_struct *mm, pgd_t *pgdp,
- p4d_t *p4dp)
+static void __init pgd_populate_tests(struct pgtable_debug_args *args)
{
pgd_t pgd;
- if (mm_p4d_folded(mm))
+ if (mm_p4d_folded(args->mm))
return;
pr_debug("Validating PGD populate\n");
@@ -581,23 +579,17 @@ static void __init pgd_populate_tests(struct mm_struct *mm, pgd_t *pgdp,
* This entry points to next level page table page.
* Hence this must not qualify as pgd_bad().
*/
- p4d_clear(p4dp);
- pgd_clear(pgdp);
- pgd_populate(mm, pgdp, p4dp);
- pgd = READ_ONCE(*pgdp);
+ p4d_clear(args->p4dp);
+ pgd_clear(args->pgdp);
+ pgd_populate(args->mm, args->pgdp, args->start_p4dp);
+ pgd = READ_ONCE(*args->pgdp);
WARN_ON(pgd_bad(pgd));
}
#else /* !__PAGETABLE_P4D_FOLDED */
-static void __init p4d_clear_tests(struct mm_struct *mm, p4d_t *p4dp) { }
-static void __init pgd_clear_tests(struct mm_struct *mm, pgd_t *pgdp) { }
-static void __init p4d_populate_tests(struct mm_struct *mm, p4d_t *p4dp,
- pud_t *pudp)
-{
-}
-static void __init pgd_populate_tests(struct mm_struct *mm, pgd_t *pgdp,
- p4d_t *p4dp)
-{
-}
+static void __init p4d_clear_tests(struct pgtable_debug_args *args) { }
+static void __init pgd_clear_tests(struct pgtable_debug_args *args) { }
+static void __init p4d_populate_tests(struct pgtable_debug_args *args) { }
+static void __init pgd_populate_tests(struct pgtable_debug_args *args) { }
#endif /* PAGETABLE_P4D_FOLDED */
static void __init pte_clear_tests(struct pgtable_debug_args *args)
@@ -1212,8 +1204,8 @@ static int __init debug_vm_pgtable(void)
struct vm_area_struct *vma;
struct mm_struct *mm;
pgd_t *pgdp;
- p4d_t *p4dp, *saved_p4dp;
- pud_t *pudp, *saved_pudp;
+ p4d_t *p4dp;
+ pud_t *pudp;
pmd_t *pmdp, *saved_pmdp, pmd;
pte_t *ptep;
pgtable_t saved_ptep;
@@ -1258,8 +1250,6 @@ static int __init debug_vm_pgtable(void)
* page table pages.
*/
pmd = READ_ONCE(*pmdp);
- saved_p4dp = p4d_offset(pgdp, 0UL);
- saved_pudp = pud_offset(p4dp, 0UL);
saved_pmdp = pmd_offset(pudp, 0UL);
saved_ptep = pmd_pgtable(pmd);
@@ -1338,15 +1328,15 @@ static int __init debug_vm_pgtable(void)
pud_populate_tests(&args);
spin_unlock(ptl);
- spin_lock(&mm->page_table_lock);
- p4d_clear_tests(mm, p4dp);
- pgd_clear_tests(mm, pgdp);
- p4d_populate_tests(mm, p4dp, saved_pudp);
- pgd_populate_tests(mm, pgdp, saved_p4dp);
- spin_unlock(&mm->page_table_lock);
+ spin_lock(&(args.mm->page_table_lock));
+ p4d_clear_tests(&args);
+ pgd_clear_tests(&args);
+ p4d_populate_tests(&args);
+ pgd_populate_tests(&args);
+ spin_unlock(&(args.mm->page_table_lock));
- p4d_free(mm, saved_p4dp);
- pud_free(mm, saved_pudp);
+ p4d_free(mm, p4d_offset(pgdp, 0UL));
+ pud_free(mm, pud_offset(p4dp, 0UL));
pmd_free(mm, saved_pmdp);
pte_free(mm, saved_ptep);
--
2.23.0
In page table entry modifying tests, set_xxx_at() are used to populate
the page table entries. On ARM64, PG_arch_1 (PG_dcache_clean) flag is
set to the target page flag if execution permission is given. The logic
exits since commit 4f04d8f00545 ("arm64: MMU definitions"). The page
flag is kept when the page is free'd to buddy's free area list. However,
it will trigger page checking failure when it's pulled from the buddy's
free area list, as the following warning messages indicate.
BUG: Bad page state in process memhog pfn:08000
page:0000000015c0a628 refcount:0 mapcount:0 \
mapping:0000000000000000 index:0x1 pfn:0x8000
flags: 0x7ffff8000000800(arch_1|node=0|zone=0|lastcpupid=0xfffff)
raw: 07ffff8000000800 dead000000000100 dead000000000122 0000000000000000
raw: 0000000000000001 0000000000000000 00000000ffffffff 0000000000000000
page dumped because: PAGE_FLAGS_CHECK_AT_PREP flag(s) set
This fixes the issue by clearing PG_arch_1 through flush_dcache_page()
after set_xxx_at() is called. For architectures other than ARM64, the
unexpected overhead of cache flushing is acceptable.
Signed-off-by: Gavin Shan <[email protected]>
---
mm/debug_vm_pgtable.c | 55 +++++++++++++++++++++++++++++++++++++++----
1 file changed, 51 insertions(+), 4 deletions(-)
diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
index 162ff6329f7b..d2c2d23e542e 100644
--- a/mm/debug_vm_pgtable.c
+++ b/mm/debug_vm_pgtable.c
@@ -29,6 +29,8 @@
#include <linux/start_kernel.h>
#include <linux/sched/mm.h>
#include <linux/io.h>
+
+#include <asm/cacheflush.h>
#include <asm/pgalloc.h>
#include <asm/tlbflush.h>
@@ -119,19 +121,28 @@ static void __init pte_basic_tests(struct pgtable_debug_args *args, int idx)
static void __init pte_advanced_tests(struct pgtable_debug_args *args)
{
+ struct page *page;
pte_t pte;
/*
* Architectures optimize set_pte_at by avoiding TLB flush.
* This requires set_pte_at to be not used to update an
* existing pte entry. Clear pte before we do set_pte_at
+ *
+ * flush_dcache_page() is called after set_pte_at() to clear
+ * PG_arch_1 for the page on ARM64. The page flag isn't cleared
+ * when it's released and page allocation check will fail when
+ * the page is allocated again. For architectures other than ARM64,
+ * the unexpected overhead of cache flushing is acceptable.
*/
- if (args->pte_pfn == ULONG_MAX)
+ page = (args->pte_pfn != ULONG_MAX) ? pfn_to_page(args->pte_pfn) : NULL;
+ if (!page)
return;
pr_debug("Validating PTE advanced\n");
pte = pfn_pte(args->pte_pfn, args->page_prot);
set_pte_at(args->mm, args->vaddr, args->ptep, pte);
+ flush_dcache_page(page);
ptep_set_wrprotect(args->mm, args->vaddr, args->ptep);
pte = ptep_get(args->ptep);
WARN_ON(pte_write(pte));
@@ -143,6 +154,7 @@ static void __init pte_advanced_tests(struct pgtable_debug_args *args)
pte = pte_wrprotect(pte);
pte = pte_mkclean(pte);
set_pte_at(args->mm, args->vaddr, args->ptep, pte);
+ flush_dcache_page(page);
pte = pte_mkwrite(pte);
pte = pte_mkdirty(pte);
ptep_set_access_flags(args->vma, args->vaddr, args->ptep, pte, 1);
@@ -155,6 +167,7 @@ static void __init pte_advanced_tests(struct pgtable_debug_args *args)
pte = pfn_pte(args->pte_pfn, args->page_prot);
pte = pte_mkyoung(pte);
set_pte_at(args->mm, args->vaddr, args->ptep, pte);
+ flush_dcache_page(page);
ptep_test_and_clear_young(args->vma, args->vaddr, args->ptep);
pte = ptep_get(args->ptep);
WARN_ON(pte_young(pte));
@@ -213,15 +226,24 @@ static void __init pmd_basic_tests(struct pgtable_debug_args *args, int idx)
static void __init pmd_advanced_tests(struct pgtable_debug_args *args)
{
+ struct page *page;
pmd_t pmd;
unsigned long vaddr = args->vaddr;
if (!has_transparent_hugepage())
return;
- if (args->pmd_pfn == ULONG_MAX)
+ page = (args->pmd_pfn != ULONG_MAX) ? pfn_to_page(args->pmd_pfn) : NULL;
+ if (!page)
return;
+ /*
+ * flush_dcache_page() is called after set_pmd_at() to clear
+ * PG_arch_1 for the page on ARM64. The page flag isn't cleared
+ * when it's released and page allocation check will fail when
+ * the page is allocated again. For architectures other than ARM64,
+ * the unexpected overhead of cache flushing is acceptable.
+ */
pr_debug("Validating PMD advanced\n");
/* Align the address wrt HPAGE_PMD_SIZE */
vaddr &= HPAGE_PMD_MASK;
@@ -230,6 +252,7 @@ static void __init pmd_advanced_tests(struct pgtable_debug_args *args)
pmd = pfn_pmd(args->pmd_pfn, args->page_prot);
set_pmd_at(args->mm, vaddr, args->pmdp, pmd);
+ flush_dcache_page(page);
pmdp_set_wrprotect(args->mm, vaddr, args->pmdp);
pmd = READ_ONCE(*args->pmdp);
WARN_ON(pmd_write(pmd));
@@ -241,6 +264,7 @@ static void __init pmd_advanced_tests(struct pgtable_debug_args *args)
pmd = pmd_wrprotect(pmd);
pmd = pmd_mkclean(pmd);
set_pmd_at(args->mm, vaddr, args->pmdp, pmd);
+ flush_dcache_page(page);
pmd = pmd_mkwrite(pmd);
pmd = pmd_mkdirty(pmd);
pmdp_set_access_flags(args->vma, vaddr, args->pmdp, pmd, 1);
@@ -253,6 +277,7 @@ static void __init pmd_advanced_tests(struct pgtable_debug_args *args)
pmd = pmd_mkhuge(pfn_pmd(args->pmd_pfn, args->page_prot));
pmd = pmd_mkyoung(pmd);
set_pmd_at(args->mm, vaddr, args->pmdp, pmd);
+ flush_dcache_page(page);
pmdp_test_and_clear_young(args->vma, vaddr, args->pmdp);
pmd = READ_ONCE(*args->pmdp);
WARN_ON(pmd_young(pmd));
@@ -339,21 +364,31 @@ static void __init pud_basic_tests(struct pgtable_debug_args *args, int idx)
static void __init pud_advanced_tests(struct pgtable_debug_args *args)
{
+ struct page *page;
unsigned long vaddr = args->vaddr;
pud_t pud;
if (!has_transparent_hugepage())
return;
- if (args->pud_pfn == ULONG_MAX)
+ page = (args->pud_pfn != ULONG_MAX) ? pfn_to_page(args->pud_pfn) : NULL;
+ if (!page)
return;
+ /*
+ * flush_dcache_page() is called after set_pud_at() to clear
+ * PG_arch_1 for the page on ARM64. The page flag isn't cleared
+ * when it's released and page allocation check will fail when
+ * the page is allocated again. For architectures other than ARM64,
+ * the unexpected overhead of cache flushing is acceptable.
+ */
pr_debug("Validating PUD advanced\n");
/* Align the address wrt HPAGE_PUD_SIZE */
vaddr &= HPAGE_PUD_MASK;
pud = pfn_pud(args->pud_pfn, args->page_prot);
set_pud_at(args->mm, vaddr, args->pudp, pud);
+ flush_dcache_page(page);
pudp_set_wrprotect(args->mm, vaddr, args->pudp);
pud = READ_ONCE(*args->pudp);
WARN_ON(pud_write(pud));
@@ -367,6 +402,7 @@ static void __init pud_advanced_tests(struct pgtable_debug_args *args)
pud = pud_wrprotect(pud);
pud = pud_mkclean(pud);
set_pud_at(args->mm, vaddr, args->pudp, pud);
+ flush_dcache_page(page);
pud = pud_mkwrite(pud);
pud = pud_mkdirty(pud);
pudp_set_access_flags(args->vma, vaddr, args->pudp, pud, 1);
@@ -382,6 +418,7 @@ static void __init pud_advanced_tests(struct pgtable_debug_args *args)
pud = pfn_pud(args->pud_pfn, args->page_prot);
pud = pud_mkyoung(pud);
set_pud_at(args->mm, vaddr, args->pudp, pud);
+ flush_dcache_page(page);
pudp_test_and_clear_young(args->vma, vaddr, args->pudp);
pud = READ_ONCE(*args->pudp);
WARN_ON(pud_young(pud));
@@ -594,16 +631,26 @@ static void __init pgd_populate_tests(struct pgtable_debug_args *args) { }
static void __init pte_clear_tests(struct pgtable_debug_args *args)
{
+ struct page *page;
pte_t pte = pfn_pte(args->pte_pfn, args->page_prot);
- if (args->pte_pfn == ULONG_MAX)
+ page = (args->pte_pfn != ULONG_MAX) ? pfn_to_page(args->pte_pfn) : NULL;
+ if (!page)
return;
+ /*
+ * flush_dcache_page() is called after set_pte_at() to clear
+ * PG_arch_1 for the page on ARM64. The page flag isn't cleared
+ * when it's released and page allocation check will fail when
+ * the page is allocated again. For architectures other than ARM64,
+ * the unexpected overhead of cache flushing is acceptable.
+ */
pr_debug("Validating PTE clear\n");
#ifndef CONFIG_RISCV
pte = __pte(pte_val(pte) | RANDOM_ORVALUE);
#endif
set_pte_at(args->mm, args->vaddr, args->ptep, pte);
+ flush_dcache_page(page);
barrier();
pte_clear(args->mm, args->vaddr, args->ptep);
pte = ptep_get(args->ptep);
--
2.23.0
The variables used by old implementation isn't needed as we switched
to "struct pgtable_debug_args". Lets remove them and related code in
debug_vm_pgtable().
Signed-off-by: Gavin Shan <[email protected]>
---
mm/debug_vm_pgtable.c | 54 -------------------------------------------
1 file changed, 54 deletions(-)
diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
index 0611b5f31e89..162ff6329f7b 100644
--- a/mm/debug_vm_pgtable.c
+++ b/mm/debug_vm_pgtable.c
@@ -1201,15 +1201,7 @@ static int __init init_args(struct pgtable_debug_args *args)
static int __init debug_vm_pgtable(void)
{
struct pgtable_debug_args args;
- struct vm_area_struct *vma;
- struct mm_struct *mm;
- pgd_t *pgdp;
- p4d_t *p4dp;
- pud_t *pudp;
- pmd_t *pmdp, *saved_pmdp, pmd;
pte_t *ptep;
- pgtable_t saved_ptep;
- unsigned long vaddr;
spinlock_t *ptl = NULL;
int idx, ret;
@@ -1218,41 +1210,6 @@ static int __init debug_vm_pgtable(void)
if (ret)
return ret;
- vaddr = get_random_vaddr();
- mm = mm_alloc();
- if (!mm) {
- pr_err("mm_struct allocation failed\n");
- return 1;
- }
-
- vma = vm_area_alloc(mm);
- if (!vma) {
- pr_err("vma allocation failed\n");
- return 1;
- }
-
- pgdp = pgd_offset(mm, vaddr);
- p4dp = p4d_alloc(mm, pgdp, vaddr);
- pudp = pud_alloc(mm, p4dp, vaddr);
- pmdp = pmd_alloc(mm, pudp, vaddr);
- /*
- * Allocate pgtable_t
- */
- if (pte_alloc(mm, pmdp)) {
- pr_err("pgtable allocation failed\n");
- return 1;
- }
-
- /*
- * Save all the page table page addresses as the page table
- * entries will be used for testing with random or garbage
- * values. These saved addresses will be used for freeing
- * page table pages.
- */
- pmd = READ_ONCE(*pmdp);
- saved_pmdp = pmd_offset(pudp, 0UL);
- saved_ptep = pmd_pgtable(pmd);
-
/*
* Iterate over the protection_map[] to make sure that all
* the basic page table transformation validations just hold
@@ -1335,17 +1292,6 @@ static int __init debug_vm_pgtable(void)
pgd_populate_tests(&args);
spin_unlock(&(args.mm->page_table_lock));
- p4d_free(mm, p4d_offset(pgdp, 0UL));
- pud_free(mm, pud_offset(p4dp, 0UL));
- pmd_free(mm, saved_pmdp);
- pte_free(mm, saved_ptep);
-
- vm_area_free(vma);
- mm_dec_nr_puds(mm);
- mm_dec_nr_pmds(mm);
- mm_dec_nr_ptes(mm);
- mmdrop(mm);
-
destroy_args(&args);
return 0;
}
--
2.23.0
This uses struct pgtable_debug_args in PMD modifying tests. The allocated
huge page is used when set_pmd_at() is used. The corresponding tests
are skipped if the huge page doesn't exist. Besides, the unused variable
@pmd_aligned in debug_vm_pgtable() is dropped.
Signed-off-by: Gavin Shan <[email protected]>
---
mm/debug_vm_pgtable.c | 98 ++++++++++++++++++++-----------------------
1 file changed, 46 insertions(+), 52 deletions(-)
diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
index b5b22e666954..2a7c810b56b4 100644
--- a/mm/debug_vm_pgtable.c
+++ b/mm/debug_vm_pgtable.c
@@ -211,54 +211,55 @@ static void __init pmd_basic_tests(struct pgtable_debug_args *args, int idx)
WARN_ON(!pmd_bad(pmd_mkhuge(pmd)));
}
-static void __init pmd_advanced_tests(struct mm_struct *mm,
- struct vm_area_struct *vma, pmd_t *pmdp,
- unsigned long pfn, unsigned long vaddr,
- pgprot_t prot, pgtable_t pgtable)
+static void __init pmd_advanced_tests(struct pgtable_debug_args *args)
{
pmd_t pmd;
+ unsigned long vaddr = args->vaddr;
if (!has_transparent_hugepage())
return;
+ if (args->pmd_pfn == ULONG_MAX)
+ return;
+
pr_debug("Validating PMD advanced\n");
/* Align the address wrt HPAGE_PMD_SIZE */
vaddr &= HPAGE_PMD_MASK;
- pgtable_trans_huge_deposit(mm, pmdp, pgtable);
+ pgtable_trans_huge_deposit(args->mm, args->pmdp, args->start_ptep);
- pmd = pfn_pmd(pfn, prot);
- set_pmd_at(mm, vaddr, pmdp, pmd);
- pmdp_set_wrprotect(mm, vaddr, pmdp);
- pmd = READ_ONCE(*pmdp);
+ pmd = pfn_pmd(args->pmd_pfn, args->page_prot);
+ set_pmd_at(args->mm, vaddr, args->pmdp, pmd);
+ pmdp_set_wrprotect(args->mm, vaddr, args->pmdp);
+ pmd = READ_ONCE(*args->pmdp);
WARN_ON(pmd_write(pmd));
- pmdp_huge_get_and_clear(mm, vaddr, pmdp);
- pmd = READ_ONCE(*pmdp);
+ pmdp_huge_get_and_clear(args->mm, vaddr, args->pmdp);
+ pmd = READ_ONCE(*args->pmdp);
WARN_ON(!pmd_none(pmd));
- pmd = pfn_pmd(pfn, prot);
+ pmd = pfn_pmd(args->pmd_pfn, args->page_prot);
pmd = pmd_wrprotect(pmd);
pmd = pmd_mkclean(pmd);
- set_pmd_at(mm, vaddr, pmdp, pmd);
+ set_pmd_at(args->mm, vaddr, args->pmdp, pmd);
pmd = pmd_mkwrite(pmd);
pmd = pmd_mkdirty(pmd);
- pmdp_set_access_flags(vma, vaddr, pmdp, pmd, 1);
- pmd = READ_ONCE(*pmdp);
+ pmdp_set_access_flags(args->vma, vaddr, args->pmdp, pmd, 1);
+ pmd = READ_ONCE(*args->pmdp);
WARN_ON(!(pmd_write(pmd) && pmd_dirty(pmd)));
- pmdp_huge_get_and_clear_full(vma, vaddr, pmdp, 1);
- pmd = READ_ONCE(*pmdp);
+ pmdp_huge_get_and_clear_full(args->vma, vaddr, args->pmdp, 1);
+ pmd = READ_ONCE(*args->pmdp);
WARN_ON(!pmd_none(pmd));
- pmd = pmd_mkhuge(pfn_pmd(pfn, prot));
+ pmd = pmd_mkhuge(pfn_pmd(args->pmd_pfn, args->page_prot));
pmd = pmd_mkyoung(pmd);
- set_pmd_at(mm, vaddr, pmdp, pmd);
- pmdp_test_and_clear_young(vma, vaddr, pmdp);
- pmd = READ_ONCE(*pmdp);
+ set_pmd_at(args->mm, vaddr, args->pmdp, pmd);
+ pmdp_test_and_clear_young(args->vma, vaddr, args->pmdp);
+ pmd = READ_ONCE(*args->pmdp);
WARN_ON(pmd_young(pmd));
/* Clear the pte entries */
- pmdp_huge_get_and_clear(mm, vaddr, pmdp);
- pgtable = pgtable_trans_huge_withdraw(mm, pmdp);
+ pmdp_huge_get_and_clear(args->mm, vaddr, args->pmdp);
+ pgtable_trans_huge_withdraw(args->mm, args->pmdp);
}
static void __init pmd_leaf_tests(struct pgtable_debug_args *args)
@@ -415,12 +416,7 @@ static void __init pud_leaf_tests(struct pgtable_debug_args *args) { }
#else /* !CONFIG_TRANSPARENT_HUGEPAGE */
static void __init pmd_basic_tests(struct pgtable_debug_args *args, int idx) { }
static void __init pud_basic_tests(struct pgtable_debug_args *args, int idx) { }
-static void __init pmd_advanced_tests(struct mm_struct *mm,
- struct vm_area_struct *vma, pmd_t *pmdp,
- unsigned long pfn, unsigned long vaddr,
- pgprot_t prot, pgtable_t pgtable)
-{
-}
+static void __init pmd_advanced_tests(struct pgtable_debug_args *args) { }
static void __init pud_advanced_tests(struct mm_struct *mm,
struct vm_area_struct *vma, pud_t *pudp,
unsigned long pfn, unsigned long vaddr,
@@ -433,11 +429,11 @@ static void __init pmd_savedwrite_tests(struct pgtable_debug_args *args) { }
#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
#ifdef CONFIG_HAVE_ARCH_HUGE_VMAP
-static void __init pmd_huge_tests(pmd_t *pmdp, unsigned long pfn, pgprot_t prot)
+static void __init pmd_huge_tests(struct pgtable_debug_args *args)
{
pmd_t pmd;
- if (!arch_vmap_pmd_supported(prot))
+ if (!arch_vmap_pmd_supported(args->page_prot))
return;
pr_debug("Validating PMD huge\n");
@@ -445,10 +441,10 @@ static void __init pmd_huge_tests(pmd_t *pmdp, unsigned long pfn, pgprot_t prot)
* X86 defined pmd_set_huge() verifies that the given
* PMD is not a populated non-leaf entry.
*/
- WRITE_ONCE(*pmdp, __pmd(0));
- WARN_ON(!pmd_set_huge(pmdp, __pfn_to_phys(pfn), prot));
- WARN_ON(!pmd_clear_huge(pmdp));
- pmd = READ_ONCE(*pmdp);
+ WRITE_ONCE(*args->pmdp, __pmd(0));
+ WARN_ON(!pmd_set_huge(args->pmdp, __pfn_to_phys(args->fixed_pmd_pfn), args->page_prot));
+ WARN_ON(!pmd_clear_huge(args->pmdp));
+ pmd = READ_ONCE(*args->pmdp);
WARN_ON(!pmd_none(pmd));
}
@@ -471,7 +467,7 @@ static void __init pud_huge_tests(pud_t *pudp, unsigned long pfn, pgprot_t prot)
WARN_ON(!pud_none(pud));
}
#else /* !CONFIG_HAVE_ARCH_HUGE_VMAP */
-static void __init pmd_huge_tests(pmd_t *pmdp, unsigned long pfn, pgprot_t prot) { }
+static void __init pmd_huge_tests(struct pgtable_debug_args *args) { }
static void __init pud_huge_tests(pud_t *pudp, unsigned long pfn, pgprot_t prot) { }
#endif /* CONFIG_HAVE_ARCH_HUGE_VMAP */
@@ -635,20 +631,19 @@ static void __init pte_clear_tests(struct pgtable_debug_args *args)
WARN_ON(!pte_none(pte));
}
-static void __init pmd_clear_tests(struct mm_struct *mm, pmd_t *pmdp)
+static void __init pmd_clear_tests(struct pgtable_debug_args *args)
{
- pmd_t pmd = READ_ONCE(*pmdp);
+ pmd_t pmd = READ_ONCE(*args->pmdp);
pr_debug("Validating PMD clear\n");
pmd = __pmd(pmd_val(pmd) | RANDOM_ORVALUE);
- WRITE_ONCE(*pmdp, pmd);
- pmd_clear(pmdp);
- pmd = READ_ONCE(*pmdp);
+ WRITE_ONCE(*args->pmdp, pmd);
+ pmd_clear(args->pmdp);
+ pmd = READ_ONCE(*args->pmdp);
WARN_ON(!pmd_none(pmd));
}
-static void __init pmd_populate_tests(struct mm_struct *mm, pmd_t *pmdp,
- pgtable_t pgtable)
+static void __init pmd_populate_tests(struct pgtable_debug_args *args)
{
pmd_t pmd;
@@ -657,8 +652,8 @@ static void __init pmd_populate_tests(struct mm_struct *mm, pmd_t *pmdp,
* This entry points to next level page table page.
* Hence this must not qualify as pmd_bad().
*/
- pmd_populate(mm, pmdp, pgtable);
- pmd = READ_ONCE(*pmdp);
+ pmd_populate(args->mm, args->pmdp, args->start_ptep);
+ pmd = READ_ONCE(*args->pmdp);
WARN_ON(pmd_bad(pmd));
}
@@ -1237,7 +1232,7 @@ static int __init debug_vm_pgtable(void)
pgtable_t saved_ptep;
pgprot_t prot;
phys_addr_t paddr;
- unsigned long vaddr, pmd_aligned;
+ unsigned long vaddr;
unsigned long pud_aligned;
spinlock_t *ptl = NULL;
int idx, ret;
@@ -1272,7 +1267,6 @@ static int __init debug_vm_pgtable(void)
*/
paddr = __pa_symbol(&start_kernel);
- pmd_aligned = (paddr & PMD_MASK) >> PAGE_SHIFT;
pud_aligned = (paddr & PUD_MASK) >> PAGE_SHIFT;
pgdp = pgd_offset(mm, vaddr);
@@ -1360,11 +1354,11 @@ static int __init debug_vm_pgtable(void)
pte_advanced_tests(&args);
pte_unmap_unlock(ptep, ptl);
- ptl = pmd_lock(mm, pmdp);
- pmd_clear_tests(mm, pmdp);
- pmd_advanced_tests(mm, vma, pmdp, pmd_aligned, vaddr, prot, saved_ptep);
- pmd_huge_tests(pmdp, pmd_aligned, prot);
- pmd_populate_tests(mm, pmdp, saved_ptep);
+ ptl = pmd_lock(args.mm, args.pmdp);
+ pmd_clear_tests(&args);
+ pmd_advanced_tests(&args);
+ pmd_huge_tests(&args);
+ pmd_populate_tests(&args);
spin_unlock(ptl);
ptl = pud_lock(mm, pudp);
--
2.23.0
This uses struct pgtable_debug_args in PTE modifying tests. The allocated
page is used as set_pte_at() is used there. The tests are skipped if
the allocated page doesn't exist. Besides, the unused variable @ptep
and @pte_aligned in debug_vm_pgtable() are dropped.
Signed-off-by: Gavin Shan <[email protected]>
---
mm/debug_vm_pgtable.c | 64 +++++++++++++++++++++----------------------
1 file changed, 31 insertions(+), 33 deletions(-)
diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
index 9136195efde3..b5b22e666954 100644
--- a/mm/debug_vm_pgtable.c
+++ b/mm/debug_vm_pgtable.c
@@ -117,10 +117,7 @@ static void __init pte_basic_tests(struct pgtable_debug_args *args, int idx)
WARN_ON(!pte_dirty(pte_wrprotect(pte_mkdirty(pte))));
}
-static void __init pte_advanced_tests(struct mm_struct *mm,
- struct vm_area_struct *vma, pte_t *ptep,
- unsigned long pfn, unsigned long vaddr,
- pgprot_t prot)
+static void __init pte_advanced_tests(struct pgtable_debug_args *args)
{
pte_t pte;
@@ -129,35 +126,37 @@ static void __init pte_advanced_tests(struct mm_struct *mm,
* This requires set_pte_at to be not used to update an
* existing pte entry. Clear pte before we do set_pte_at
*/
+ if (args->pte_pfn == ULONG_MAX)
+ return;
pr_debug("Validating PTE advanced\n");
- pte = pfn_pte(pfn, prot);
- set_pte_at(mm, vaddr, ptep, pte);
- ptep_set_wrprotect(mm, vaddr, ptep);
- pte = ptep_get(ptep);
+ pte = pfn_pte(args->pte_pfn, args->page_prot);
+ set_pte_at(args->mm, args->vaddr, args->ptep, pte);
+ ptep_set_wrprotect(args->mm, args->vaddr, args->ptep);
+ pte = ptep_get(args->ptep);
WARN_ON(pte_write(pte));
- ptep_get_and_clear(mm, vaddr, ptep);
- pte = ptep_get(ptep);
+ ptep_get_and_clear(args->mm, args->vaddr, args->ptep);
+ pte = ptep_get(args->ptep);
WARN_ON(!pte_none(pte));
- pte = pfn_pte(pfn, prot);
+ pte = pfn_pte(args->pte_pfn, args->page_prot);
pte = pte_wrprotect(pte);
pte = pte_mkclean(pte);
- set_pte_at(mm, vaddr, ptep, pte);
+ set_pte_at(args->mm, args->vaddr, args->ptep, pte);
pte = pte_mkwrite(pte);
pte = pte_mkdirty(pte);
- ptep_set_access_flags(vma, vaddr, ptep, pte, 1);
- pte = ptep_get(ptep);
+ ptep_set_access_flags(args->vma, args->vaddr, args->ptep, pte, 1);
+ pte = ptep_get(args->ptep);
WARN_ON(!(pte_write(pte) && pte_dirty(pte)));
- ptep_get_and_clear_full(mm, vaddr, ptep, 1);
- pte = ptep_get(ptep);
+ ptep_get_and_clear_full(args->mm, args->vaddr, args->ptep, 1);
+ pte = ptep_get(args->ptep);
WARN_ON(!pte_none(pte));
- pte = pfn_pte(pfn, prot);
+ pte = pfn_pte(args->pte_pfn, args->page_prot);
pte = pte_mkyoung(pte);
- set_pte_at(mm, vaddr, ptep, pte);
- ptep_test_and_clear_young(vma, vaddr, ptep);
- pte = ptep_get(ptep);
+ set_pte_at(args->mm, args->vaddr, args->ptep, pte);
+ ptep_test_and_clear_young(args->vma, args->vaddr, args->ptep);
+ pte = ptep_get(args->ptep);
WARN_ON(pte_young(pte));
}
@@ -618,20 +617,21 @@ static void __init pgd_populate_tests(struct mm_struct *mm, pgd_t *pgdp,
}
#endif /* PAGETABLE_P4D_FOLDED */
-static void __init pte_clear_tests(struct mm_struct *mm, pte_t *ptep,
- unsigned long pfn, unsigned long vaddr,
- pgprot_t prot)
+static void __init pte_clear_tests(struct pgtable_debug_args *args)
{
- pte_t pte = pfn_pte(pfn, prot);
+ pte_t pte = pfn_pte(args->pte_pfn, args->page_prot);
+
+ if (args->pte_pfn == ULONG_MAX)
+ return;
pr_debug("Validating PTE clear\n");
#ifndef CONFIG_RISCV
pte = __pte(pte_val(pte) | RANDOM_ORVALUE);
#endif
- set_pte_at(mm, vaddr, ptep, pte);
+ set_pte_at(args->mm, args->vaddr, args->ptep, pte);
barrier();
- pte_clear(mm, vaddr, ptep);
- pte = ptep_get(ptep);
+ pte_clear(args->mm, args->vaddr, args->ptep);
+ pte = ptep_get(args->ptep);
WARN_ON(!pte_none(pte));
}
@@ -1237,7 +1237,7 @@ static int __init debug_vm_pgtable(void)
pgtable_t saved_ptep;
pgprot_t prot;
phys_addr_t paddr;
- unsigned long vaddr, pte_aligned, pmd_aligned;
+ unsigned long vaddr, pmd_aligned;
unsigned long pud_aligned;
spinlock_t *ptl = NULL;
int idx, ret;
@@ -1272,10 +1272,8 @@ static int __init debug_vm_pgtable(void)
*/
paddr = __pa_symbol(&start_kernel);
- pte_aligned = (paddr & PAGE_MASK) >> PAGE_SHIFT;
pmd_aligned = (paddr & PMD_MASK) >> PAGE_SHIFT;
pud_aligned = (paddr & PUD_MASK) >> PAGE_SHIFT;
- WARN_ON(!pfn_valid(pte_aligned));
pgdp = pgd_offset(mm, vaddr);
p4dp = p4d_alloc(mm, pgdp, vaddr);
@@ -1357,9 +1355,9 @@ static int __init debug_vm_pgtable(void)
* proper page table lock.
*/
- ptep = pte_offset_map_lock(mm, pmdp, vaddr, &ptl);
- pte_clear_tests(mm, ptep, pte_aligned, vaddr, prot);
- pte_advanced_tests(mm, vma, ptep, pte_aligned, vaddr, prot);
+ ptep = pte_offset_map_lock(args.mm, args.pmdp, args.vaddr, &ptl);
+ pte_clear_tests(&args);
+ pte_advanced_tests(&args);
pte_unmap_unlock(ptep, ptl);
ptl = pmd_lock(mm, pmdp);
--
2.23.0
This uses struct pgtable_debug_args in PUD modifying tests. The allocated
huge page is used when set_pud_at() is used. The corresponding tests
are skipped if the huge page doesn't exist. Besides, the following unused
variables in debug_vm_pgtable() are dropped: @prot, @paddr, @pud_aligned.
Signed-off-by: Gavin Shan <[email protected]>
---
mm/debug_vm_pgtable.c | 126 ++++++++++++++++--------------------------
1 file changed, 48 insertions(+), 78 deletions(-)
diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
index 2a7c810b56b4..863871e70268 100644
--- a/mm/debug_vm_pgtable.c
+++ b/mm/debug_vm_pgtable.c
@@ -337,55 +337,56 @@ static void __init pud_basic_tests(struct pgtable_debug_args *args, int idx)
WARN_ON(!pud_bad(pud_mkhuge(pud)));
}
-static void __init pud_advanced_tests(struct mm_struct *mm,
- struct vm_area_struct *vma, pud_t *pudp,
- unsigned long pfn, unsigned long vaddr,
- pgprot_t prot)
+static void __init pud_advanced_tests(struct pgtable_debug_args *args)
{
+ unsigned long vaddr = args->vaddr;
pud_t pud;
if (!has_transparent_hugepage())
return;
+ if (args->pud_pfn == ULONG_MAX)
+ return;
+
pr_debug("Validating PUD advanced\n");
/* Align the address wrt HPAGE_PUD_SIZE */
vaddr &= HPAGE_PUD_MASK;
- pud = pfn_pud(pfn, prot);
- set_pud_at(mm, vaddr, pudp, pud);
- pudp_set_wrprotect(mm, vaddr, pudp);
- pud = READ_ONCE(*pudp);
+ pud = pfn_pud(args->pud_pfn, args->page_prot);
+ set_pud_at(args->mm, vaddr, args->pudp, pud);
+ pudp_set_wrprotect(args->mm, vaddr, args->pudp);
+ pud = READ_ONCE(*args->pudp);
WARN_ON(pud_write(pud));
#ifndef __PAGETABLE_PMD_FOLDED
- pudp_huge_get_and_clear(mm, vaddr, pudp);
- pud = READ_ONCE(*pudp);
+ pudp_huge_get_and_clear(args->mm, vaddr, args->pudp);
+ pud = READ_ONCE(*args->pudp);
WARN_ON(!pud_none(pud));
#endif /* __PAGETABLE_PMD_FOLDED */
- pud = pfn_pud(pfn, prot);
+ pud = pfn_pud(args->pud_pfn, args->page_prot);
pud = pud_wrprotect(pud);
pud = pud_mkclean(pud);
- set_pud_at(mm, vaddr, pudp, pud);
+ set_pud_at(args->mm, vaddr, args->pudp, pud);
pud = pud_mkwrite(pud);
pud = pud_mkdirty(pud);
- pudp_set_access_flags(vma, vaddr, pudp, pud, 1);
- pud = READ_ONCE(*pudp);
+ pudp_set_access_flags(args->vma, vaddr, args->pudp, pud, 1);
+ pud = READ_ONCE(*args->pudp);
WARN_ON(!(pud_write(pud) && pud_dirty(pud)));
#ifndef __PAGETABLE_PMD_FOLDED
- pudp_huge_get_and_clear_full(mm, vaddr, pudp, 1);
- pud = READ_ONCE(*pudp);
+ pudp_huge_get_and_clear_full(args->mm, vaddr, args->pudp, 1);
+ pud = READ_ONCE(*args->pudp);
WARN_ON(!pud_none(pud));
#endif /* __PAGETABLE_PMD_FOLDED */
- pud = pfn_pud(pfn, prot);
+ pud = pfn_pud(args->pud_pfn, args->page_prot);
pud = pud_mkyoung(pud);
- set_pud_at(mm, vaddr, pudp, pud);
- pudp_test_and_clear_young(vma, vaddr, pudp);
- pud = READ_ONCE(*pudp);
+ set_pud_at(args->mm, vaddr, args->pudp, pud);
+ pudp_test_and_clear_young(args->vma, vaddr, args->pudp);
+ pud = READ_ONCE(*args->pudp);
WARN_ON(pud_young(pud));
- pudp_huge_get_and_clear(mm, vaddr, pudp);
+ pudp_huge_get_and_clear(args->mm, vaddr, args->pudp);
}
static void __init pud_leaf_tests(struct pgtable_debug_args *args)
@@ -405,24 +406,14 @@ static void __init pud_leaf_tests(struct pgtable_debug_args *args)
}
#else /* !CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */
static void __init pud_basic_tests(struct pgtable_debug_args *args, int idx) { }
-static void __init pud_advanced_tests(struct mm_struct *mm,
- struct vm_area_struct *vma, pud_t *pudp,
- unsigned long pfn, unsigned long vaddr,
- pgprot_t prot)
-{
-}
+static void __init pud_advanced_tests(struct pgtable_debug_args *args) { }
static void __init pud_leaf_tests(struct pgtable_debug_args *args) { }
#endif /* CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */
#else /* !CONFIG_TRANSPARENT_HUGEPAGE */
static void __init pmd_basic_tests(struct pgtable_debug_args *args, int idx) { }
static void __init pud_basic_tests(struct pgtable_debug_args *args, int idx) { }
static void __init pmd_advanced_tests(struct pgtable_debug_args *args) { }
-static void __init pud_advanced_tests(struct mm_struct *mm,
- struct vm_area_struct *vma, pud_t *pudp,
- unsigned long pfn, unsigned long vaddr,
- pgprot_t prot)
-{
-}
+static void __init pud_advanced_tests(struct pgtable_debug_args *args) { }
static void __init pmd_leaf_tests(struct pgtable_debug_args *args) { }
static void __init pud_leaf_tests(struct pgtable_debug_args *args) { }
static void __init pmd_savedwrite_tests(struct pgtable_debug_args *args) { }
@@ -448,11 +439,11 @@ static void __init pmd_huge_tests(struct pgtable_debug_args *args)
WARN_ON(!pmd_none(pmd));
}
-static void __init pud_huge_tests(pud_t *pudp, unsigned long pfn, pgprot_t prot)
+static void __init pud_huge_tests(struct pgtable_debug_args *args)
{
pud_t pud;
- if (!arch_vmap_pud_supported(prot))
+ if (!arch_vmap_pud_supported(args->page_prot))
return;
pr_debug("Validating PUD huge\n");
@@ -460,15 +451,15 @@ static void __init pud_huge_tests(pud_t *pudp, unsigned long pfn, pgprot_t prot)
* X86 defined pud_set_huge() verifies that the given
* PUD is not a populated non-leaf entry.
*/
- WRITE_ONCE(*pudp, __pud(0));
- WARN_ON(!pud_set_huge(pudp, __pfn_to_phys(pfn), prot));
- WARN_ON(!pud_clear_huge(pudp));
- pud = READ_ONCE(*pudp);
+ WRITE_ONCE(*args->pudp, __pud(0));
+ WARN_ON(!pud_set_huge(args->pudp, __pfn_to_phys(args->fixed_pud_pfn), args->page_prot));
+ WARN_ON(!pud_clear_huge(args->pudp));
+ pud = READ_ONCE(*args->pudp);
WARN_ON(!pud_none(pud));
}
#else /* !CONFIG_HAVE_ARCH_HUGE_VMAP */
static void __init pmd_huge_tests(struct pgtable_debug_args *args) { }
-static void __init pud_huge_tests(pud_t *pudp, unsigned long pfn, pgprot_t prot) { }
+static void __init pud_huge_tests(struct pgtable_debug_args *args) { }
#endif /* CONFIG_HAVE_ARCH_HUGE_VMAP */
static void __init p4d_basic_tests(struct pgtable_debug_args *args)
@@ -490,27 +481,26 @@ static void __init pgd_basic_tests(struct pgtable_debug_args *args)
}
#ifndef __PAGETABLE_PUD_FOLDED
-static void __init pud_clear_tests(struct mm_struct *mm, pud_t *pudp)
+static void __init pud_clear_tests(struct pgtable_debug_args *args)
{
- pud_t pud = READ_ONCE(*pudp);
+ pud_t pud = READ_ONCE(*args->pudp);
- if (mm_pmd_folded(mm))
+ if (mm_pmd_folded(args->mm))
return;
pr_debug("Validating PUD clear\n");
pud = __pud(pud_val(pud) | RANDOM_ORVALUE);
- WRITE_ONCE(*pudp, pud);
- pud_clear(pudp);
- pud = READ_ONCE(*pudp);
+ WRITE_ONCE(*args->pudp, pud);
+ pud_clear(args->pudp);
+ pud = READ_ONCE(*args->pudp);
WARN_ON(!pud_none(pud));
}
-static void __init pud_populate_tests(struct mm_struct *mm, pud_t *pudp,
- pmd_t *pmdp)
+static void __init pud_populate_tests(struct pgtable_debug_args *args)
{
pud_t pud;
- if (mm_pmd_folded(mm))
+ if (mm_pmd_folded(args->mm))
return;
pr_debug("Validating PUD populate\n");
@@ -518,16 +508,13 @@ static void __init pud_populate_tests(struct mm_struct *mm, pud_t *pudp,
* This entry points to next level page table page.
* Hence this must not qualify as pud_bad().
*/
- pud_populate(mm, pudp, pmdp);
- pud = READ_ONCE(*pudp);
+ pud_populate(args->mm, args->pudp, args->start_pmdp);
+ pud = READ_ONCE(*args->pudp);
WARN_ON(pud_bad(pud));
}
#else /* !__PAGETABLE_PUD_FOLDED */
-static void __init pud_clear_tests(struct mm_struct *mm, pud_t *pudp) { }
-static void __init pud_populate_tests(struct mm_struct *mm, pud_t *pudp,
- pmd_t *pmdp)
-{
-}
+static void __init pud_clear_tests(struct pgtable_debug_args *args) { }
+static void __init pud_populate_tests(struct pgtable_debug_args *args) { }
#endif /* PAGETABLE_PUD_FOLDED */
#ifndef __PAGETABLE_P4D_FOLDED
@@ -1230,10 +1217,7 @@ static int __init debug_vm_pgtable(void)
pmd_t *pmdp, *saved_pmdp, pmd;
pte_t *ptep;
pgtable_t saved_ptep;
- pgprot_t prot;
- phys_addr_t paddr;
unsigned long vaddr;
- unsigned long pud_aligned;
spinlock_t *ptl = NULL;
int idx, ret;
@@ -1242,7 +1226,6 @@ static int __init debug_vm_pgtable(void)
if (ret)
return ret;
- prot = vm_get_page_prot(VMFLAGS);
vaddr = get_random_vaddr();
mm = mm_alloc();
if (!mm) {
@@ -1256,19 +1239,6 @@ static int __init debug_vm_pgtable(void)
return 1;
}
- /*
- * PFN for mapping at PTE level is determined from a standard kernel
- * text symbol. But pfns for higher page table levels are derived by
- * masking lower bits of this real pfn. These derived pfns might not
- * exist on the platform but that does not really matter as pfn_pxx()
- * helpers will still create appropriate entries for the test. This
- * helps avoid large memory block allocations to be used for mapping
- * at higher page table levels.
- */
- paddr = __pa_symbol(&start_kernel);
-
- pud_aligned = (paddr & PUD_MASK) >> PAGE_SHIFT;
-
pgdp = pgd_offset(mm, vaddr);
p4dp = p4d_alloc(mm, pgdp, vaddr);
pudp = pud_alloc(mm, p4dp, vaddr);
@@ -1361,11 +1331,11 @@ static int __init debug_vm_pgtable(void)
pmd_populate_tests(&args);
spin_unlock(ptl);
- ptl = pud_lock(mm, pudp);
- pud_clear_tests(mm, pudp);
- pud_advanced_tests(mm, vma, pudp, pud_aligned, vaddr, prot);
- pud_huge_tests(pudp, pud_aligned, prot);
- pud_populate_tests(mm, pudp, saved_pmdp);
+ ptl = pud_lock(args.mm, args.pudp);
+ pud_clear_tests(&args);
+ pud_advanced_tests(&args);
+ pud_huge_tests(&args);
+ pud_populate_tests(&args);
spin_unlock(ptl);
spin_lock(&mm->page_table_lock);
--
2.23.0
On 7/27/21 11:43 AM, Gavin Shan wrote:
> In debug_vm_pgtable(), there are many local variables introduced to
> track the needed information and they are passed to the functions for
> various test cases. It'd better to introduce a struct as place holder
> for these information. With it, what the tests functions need is the
> struct. In this way, the code is simplified and easier to be maintained.
>
> Besides, set_xxx_at() could access the data on the corresponding pages
> in the page table modifying tests. So the accessed pages in the tests
> should have been allocated from buddy. Otherwise, we're accessing pages
> that aren't owned by us. This causes issues like page flag corruption
> or kernel crash on accessing unmapped page when CONFIG_DEBUG_PAGEALLOC
> is enabled.
>
> This introduces "struct pgtable_debug_args". The struct is initialized
> and destroyed, but the information in the struct isn't used yet. It will
> be used in subsequent patches.
>
> Signed-off-by: Gavin Shan <[email protected]>
> ---
> mm/debug_vm_pgtable.c | 280 +++++++++++++++++++++++++++++++++++++++++-
> 1 file changed, 279 insertions(+), 1 deletion(-)
>
> diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
> index 1c922691aa61..8c7361643166 100644
> --- a/mm/debug_vm_pgtable.c
> +++ b/mm/debug_vm_pgtable.c
> @@ -58,6 +58,37 @@
> #define RANDOM_ORVALUE (GENMASK(BITS_PER_LONG - 1, 0) & ~ARCH_SKIP_MASK)
> #define RANDOM_NZVALUE GENMASK(7, 0)
>
> +struct pgtable_debug_args {
> + struct mm_struct *mm;
> + struct vm_area_struct *vma;
> +
> + pgd_t *pgdp;
> + p4d_t *p4dp;
> + pud_t *pudp;
> + pmd_t *pmdp;
> + pte_t *ptep;
> +
> + p4d_t *start_p4dp;
> + pud_t *start_pudp;
> + pmd_t *start_pmdp;
> + pgtable_t start_ptep;
> +
> + unsigned long vaddr;
> + pgprot_t page_prot;
> + pgprot_t page_prot_none;
> +
> + bool is_contiguous_page;
> + unsigned long pud_pfn;
> + unsigned long pmd_pfn;
> + unsigned long pte_pfn;
> +
> + unsigned long fixed_pgd_pfn;
> + unsigned long fixed_p4d_pfn;
> + unsigned long fixed_pud_pfn;
> + unsigned long fixed_pmd_pfn;
> + unsigned long fixed_pte_pfn;
> +};
> +
> static void __init pte_basic_tests(unsigned long pfn, int idx)
> {
> pgprot_t prot = protection_map[idx];
> @@ -955,8 +986,249 @@ static unsigned long __init get_random_vaddr(void)
> return random_vaddr;
> }
>
> +static void __init destroy_args(struct pgtable_debug_args *args)
> +{
> + struct page *page = NULL;
> +
> + /* Free (huge) page */
> + if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) &&
> + IS_ENABLED(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD) &&
> + has_transparent_hugepage() &&
> + args->pud_pfn != ULONG_MAX) {
> + if (args->is_contiguous_page) {
> + free_contig_range(args->pud_pfn,
> + (1 << (HPAGE_PUD_SHIFT - PAGE_SHIFT)));
> + } else {
> + page = pfn_to_page(args->pud_pfn);
> + __free_pages(page, HPAGE_PUD_SHIFT - PAGE_SHIFT);
> + }
> +
> + args->pud_pfn = ULONG_MAX;
> + args->pmd_pfn = ULONG_MAX;
> + args->pte_pfn = ULONG_MAX;
> + }
> +
> + if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) &&
> + has_transparent_hugepage() &&
> + args->pmd_pfn != ULONG_MAX) {
> + if (args->is_contiguous_page) {
> + free_contig_range(args->pmd_pfn, (1 << HPAGE_PMD_ORDER));
> + } else {
> + page = pfn_to_page(args->pmd_pfn);
> + __free_pages(page, HPAGE_PMD_ORDER);
> + }
> +
> + args->pmd_pfn = ULONG_MAX;
> + args->pte_pfn = ULONG_MAX;
> + }
> +
> + if (args->pte_pfn != ULONG_MAX) {
> + page = pfn_to_page(args->pte_pfn);
> + __free_pages(page, 0);
> + }
> +
> + /* Free page table entries */
> + if (args->start_ptep) {
> + pte_free(args->mm, args->start_ptep);
> + mm_dec_nr_ptes(args->mm);
> + }
> +
> + if (args->start_pmdp) {
> + pmd_free(args->mm, args->start_pmdp);
> + mm_dec_nr_pmds(args->mm);
> + }
> +
> + if (args->start_pudp) {
> + pud_free(args->mm, args->start_pudp);
> + mm_dec_nr_puds(args->mm);
> + }
> +
> + if (args->start_p4dp)
> + p4d_free(args->mm, args->p4dp);
^^^^ s/args->p4dp/args->start_p4dp ?
Hi Anshuman,
On 7/28/21 5:32 PM, Anshuman Khandual wrote:
> On 7/27/21 11:43 AM, Gavin Shan wrote:
>> In debug_vm_pgtable(), there are many local variables introduced to
>> track the needed information and they are passed to the functions for
>> various test cases. It'd better to introduce a struct as place holder
>> for these information. With it, what the tests functions need is the
>> struct. In this way, the code is simplified and easier to be maintained.
>>
>> Besides, set_xxx_at() could access the data on the corresponding pages
>> in the page table modifying tests. So the accessed pages in the tests
>> should have been allocated from buddy. Otherwise, we're accessing pages
>> that aren't owned by us. This causes issues like page flag corruption
>> or kernel crash on accessing unmapped page when CONFIG_DEBUG_PAGEALLOC
>> is enabled.
>>
>> This introduces "struct pgtable_debug_args". The struct is initialized
>> and destroyed, but the information in the struct isn't used yet. It will
>> be used in subsequent patches.
>>
>> Signed-off-by: Gavin Shan <[email protected]>
>> ---
>> mm/debug_vm_pgtable.c | 280 +++++++++++++++++++++++++++++++++++++++++-
>> 1 file changed, 279 insertions(+), 1 deletion(-)
>>
>> diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
>> index 1c922691aa61..8c7361643166 100644
>> --- a/mm/debug_vm_pgtable.c
>> +++ b/mm/debug_vm_pgtable.c
>> @@ -58,6 +58,37 @@
>> #define RANDOM_ORVALUE (GENMASK(BITS_PER_LONG - 1, 0) & ~ARCH_SKIP_MASK)
>> #define RANDOM_NZVALUE GENMASK(7, 0)
>>
>> +struct pgtable_debug_args {
>> + struct mm_struct *mm;
>> + struct vm_area_struct *vma;
>> +
>> + pgd_t *pgdp;
>> + p4d_t *p4dp;
>> + pud_t *pudp;
>> + pmd_t *pmdp;
>> + pte_t *ptep;
>> +
>> + p4d_t *start_p4dp;
>> + pud_t *start_pudp;
>> + pmd_t *start_pmdp;
>> + pgtable_t start_ptep;
>> +
>> + unsigned long vaddr;
>> + pgprot_t page_prot;
>> + pgprot_t page_prot_none;
>> +
>> + bool is_contiguous_page;
>> + unsigned long pud_pfn;
>> + unsigned long pmd_pfn;
>> + unsigned long pte_pfn;
>> +
>> + unsigned long fixed_pgd_pfn;
>> + unsigned long fixed_p4d_pfn;
>> + unsigned long fixed_pud_pfn;
>> + unsigned long fixed_pmd_pfn;
>> + unsigned long fixed_pte_pfn;
>> +};
>> +
>> static void __init pte_basic_tests(unsigned long pfn, int idx)
>> {
>> pgprot_t prot = protection_map[idx];
>> @@ -955,8 +986,249 @@ static unsigned long __init get_random_vaddr(void)
>> return random_vaddr;
>> }
>>
>> +static void __init destroy_args(struct pgtable_debug_args *args)
>> +{
>> + struct page *page = NULL;
>> +
>> + /* Free (huge) page */
>> + if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) &&
>> + IS_ENABLED(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD) &&
>> + has_transparent_hugepage() &&
>> + args->pud_pfn != ULONG_MAX) {
>> + if (args->is_contiguous_page) {
>> + free_contig_range(args->pud_pfn,
>> + (1 << (HPAGE_PUD_SHIFT - PAGE_SHIFT)));
>> + } else {
>> + page = pfn_to_page(args->pud_pfn);
>> + __free_pages(page, HPAGE_PUD_SHIFT - PAGE_SHIFT);
>> + }
>> +
>> + args->pud_pfn = ULONG_MAX;
>> + args->pmd_pfn = ULONG_MAX;
>> + args->pte_pfn = ULONG_MAX;
>> + }
>> +
>> + if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) &&
>> + has_transparent_hugepage() &&
>> + args->pmd_pfn != ULONG_MAX) {
>> + if (args->is_contiguous_page) {
>> + free_contig_range(args->pmd_pfn, (1 << HPAGE_PMD_ORDER));
>> + } else {
>> + page = pfn_to_page(args->pmd_pfn);
>> + __free_pages(page, HPAGE_PMD_ORDER);
>> + }
>> +
>> + args->pmd_pfn = ULONG_MAX;
>> + args->pte_pfn = ULONG_MAX;
>> + }
>> +
>> + if (args->pte_pfn != ULONG_MAX) {
>> + page = pfn_to_page(args->pte_pfn);
>> + __free_pages(page, 0);
>> + }
>> +
>> + /* Free page table entries */
>> + if (args->start_ptep) {
>> + pte_free(args->mm, args->start_ptep);
>> + mm_dec_nr_ptes(args->mm);
>> + }
>> +
>> + if (args->start_pmdp) {
>> + pmd_free(args->mm, args->start_pmdp);
>> + mm_dec_nr_pmds(args->mm);
>> + }
>> +
>> + if (args->start_pudp) {
>> + pud_free(args->mm, args->start_pudp);
>> + mm_dec_nr_puds(args->mm);
>> + }
>> +
>> + if (args->start_p4dp)
>> + p4d_free(args->mm, args->p4dp);
>
> ^^^^ s/args->p4dp/args->start_p4dp ?
>
Yes, It should be "args->start_p4dp" here. I will correct it in v5
after you finish review on v4.
Thanks,
Gavin
Gavin Shan <[email protected]> a écrit :
> In page table entry modifying tests, set_xxx_at() are used to populate
> the page table entries. On ARM64, PG_arch_1 (PG_dcache_clean) flag is
> set to the target page flag if execution permission is given. The logic
> exits since commit 4f04d8f00545 ("arm64: MMU definitions"). The page
> flag is kept when the page is free'd to buddy's free area list. However,
> it will trigger page checking failure when it's pulled from the buddy's
> free area list, as the following warning messages indicate.
>
> BUG: Bad page state in process memhog pfn:08000
> page:0000000015c0a628 refcount:0 mapcount:0 \
> mapping:0000000000000000 index:0x1 pfn:0x8000
> flags: 0x7ffff8000000800(arch_1|node=0|zone=0|lastcpupid=0xfffff)
> raw: 07ffff8000000800 dead000000000100 dead000000000122 0000000000000000
> raw: 0000000000000001 0000000000000000 00000000ffffffff 0000000000000000
> page dumped because: PAGE_FLAGS_CHECK_AT_PREP flag(s) set
>
> This fixes the issue by clearing PG_arch_1 through flush_dcache_page()
> after set_xxx_at() is called. For architectures other than ARM64, the
> unexpected overhead of cache flushing is acceptable.
>
> Signed-off-by: Gavin Shan <[email protected]>
Maybe a Fixes: tag would be good to have
And would it be possible to have this fix as first patch of the series
so that it can be applied to stable without applying the whole series ?
Christophe
> ---
> mm/debug_vm_pgtable.c | 55 +++++++++++++++++++++++++++++++++++++++----
> 1 file changed, 51 insertions(+), 4 deletions(-)
>
> diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
> index 162ff6329f7b..d2c2d23e542e 100644
> --- a/mm/debug_vm_pgtable.c
> +++ b/mm/debug_vm_pgtable.c
> @@ -29,6 +29,8 @@
> #include <linux/start_kernel.h>
> #include <linux/sched/mm.h>
> #include <linux/io.h>
> +
> +#include <asm/cacheflush.h>
> #include <asm/pgalloc.h>
> #include <asm/tlbflush.h>
>
> @@ -119,19 +121,28 @@ static void __init pte_basic_tests(struct
> pgtable_debug_args *args, int idx)
>
> static void __init pte_advanced_tests(struct pgtable_debug_args *args)
> {
> + struct page *page;
> pte_t pte;
>
> /*
> * Architectures optimize set_pte_at by avoiding TLB flush.
> * This requires set_pte_at to be not used to update an
> * existing pte entry. Clear pte before we do set_pte_at
> + *
> + * flush_dcache_page() is called after set_pte_at() to clear
> + * PG_arch_1 for the page on ARM64. The page flag isn't cleared
> + * when it's released and page allocation check will fail when
> + * the page is allocated again. For architectures other than ARM64,
> + * the unexpected overhead of cache flushing is acceptable.
> */
> - if (args->pte_pfn == ULONG_MAX)
> + page = (args->pte_pfn != ULONG_MAX) ? pfn_to_page(args->pte_pfn) : NULL;
> + if (!page)
> return;
>
> pr_debug("Validating PTE advanced\n");
> pte = pfn_pte(args->pte_pfn, args->page_prot);
> set_pte_at(args->mm, args->vaddr, args->ptep, pte);
> + flush_dcache_page(page);
> ptep_set_wrprotect(args->mm, args->vaddr, args->ptep);
> pte = ptep_get(args->ptep);
> WARN_ON(pte_write(pte));
> @@ -143,6 +154,7 @@ static void __init pte_advanced_tests(struct
> pgtable_debug_args *args)
> pte = pte_wrprotect(pte);
> pte = pte_mkclean(pte);
> set_pte_at(args->mm, args->vaddr, args->ptep, pte);
> + flush_dcache_page(page);
> pte = pte_mkwrite(pte);
> pte = pte_mkdirty(pte);
> ptep_set_access_flags(args->vma, args->vaddr, args->ptep, pte, 1);
> @@ -155,6 +167,7 @@ static void __init pte_advanced_tests(struct
> pgtable_debug_args *args)
> pte = pfn_pte(args->pte_pfn, args->page_prot);
> pte = pte_mkyoung(pte);
> set_pte_at(args->mm, args->vaddr, args->ptep, pte);
> + flush_dcache_page(page);
> ptep_test_and_clear_young(args->vma, args->vaddr, args->ptep);
> pte = ptep_get(args->ptep);
> WARN_ON(pte_young(pte));
> @@ -213,15 +226,24 @@ static void __init pmd_basic_tests(struct
> pgtable_debug_args *args, int idx)
>
> static void __init pmd_advanced_tests(struct pgtable_debug_args *args)
> {
> + struct page *page;
> pmd_t pmd;
> unsigned long vaddr = args->vaddr;
>
> if (!has_transparent_hugepage())
> return;
>
> - if (args->pmd_pfn == ULONG_MAX)
> + page = (args->pmd_pfn != ULONG_MAX) ? pfn_to_page(args->pmd_pfn) : NULL;
> + if (!page)
> return;
>
> + /*
> + * flush_dcache_page() is called after set_pmd_at() to clear
> + * PG_arch_1 for the page on ARM64. The page flag isn't cleared
> + * when it's released and page allocation check will fail when
> + * the page is allocated again. For architectures other than ARM64,
> + * the unexpected overhead of cache flushing is acceptable.
> + */
> pr_debug("Validating PMD advanced\n");
> /* Align the address wrt HPAGE_PMD_SIZE */
> vaddr &= HPAGE_PMD_MASK;
> @@ -230,6 +252,7 @@ static void __init pmd_advanced_tests(struct
> pgtable_debug_args *args)
>
> pmd = pfn_pmd(args->pmd_pfn, args->page_prot);
> set_pmd_at(args->mm, vaddr, args->pmdp, pmd);
> + flush_dcache_page(page);
> pmdp_set_wrprotect(args->mm, vaddr, args->pmdp);
> pmd = READ_ONCE(*args->pmdp);
> WARN_ON(pmd_write(pmd));
> @@ -241,6 +264,7 @@ static void __init pmd_advanced_tests(struct
> pgtable_debug_args *args)
> pmd = pmd_wrprotect(pmd);
> pmd = pmd_mkclean(pmd);
> set_pmd_at(args->mm, vaddr, args->pmdp, pmd);
> + flush_dcache_page(page);
> pmd = pmd_mkwrite(pmd);
> pmd = pmd_mkdirty(pmd);
> pmdp_set_access_flags(args->vma, vaddr, args->pmdp, pmd, 1);
> @@ -253,6 +277,7 @@ static void __init pmd_advanced_tests(struct
> pgtable_debug_args *args)
> pmd = pmd_mkhuge(pfn_pmd(args->pmd_pfn, args->page_prot));
> pmd = pmd_mkyoung(pmd);
> set_pmd_at(args->mm, vaddr, args->pmdp, pmd);
> + flush_dcache_page(page);
> pmdp_test_and_clear_young(args->vma, vaddr, args->pmdp);
> pmd = READ_ONCE(*args->pmdp);
> WARN_ON(pmd_young(pmd));
> @@ -339,21 +364,31 @@ static void __init pud_basic_tests(struct
> pgtable_debug_args *args, int idx)
>
> static void __init pud_advanced_tests(struct pgtable_debug_args *args)
> {
> + struct page *page;
> unsigned long vaddr = args->vaddr;
> pud_t pud;
>
> if (!has_transparent_hugepage())
> return;
>
> - if (args->pud_pfn == ULONG_MAX)
> + page = (args->pud_pfn != ULONG_MAX) ? pfn_to_page(args->pud_pfn) : NULL;
> + if (!page)
> return;
>
> + /*
> + * flush_dcache_page() is called after set_pud_at() to clear
> + * PG_arch_1 for the page on ARM64. The page flag isn't cleared
> + * when it's released and page allocation check will fail when
> + * the page is allocated again. For architectures other than ARM64,
> + * the unexpected overhead of cache flushing is acceptable.
> + */
> pr_debug("Validating PUD advanced\n");
> /* Align the address wrt HPAGE_PUD_SIZE */
> vaddr &= HPAGE_PUD_MASK;
>
> pud = pfn_pud(args->pud_pfn, args->page_prot);
> set_pud_at(args->mm, vaddr, args->pudp, pud);
> + flush_dcache_page(page);
> pudp_set_wrprotect(args->mm, vaddr, args->pudp);
> pud = READ_ONCE(*args->pudp);
> WARN_ON(pud_write(pud));
> @@ -367,6 +402,7 @@ static void __init pud_advanced_tests(struct
> pgtable_debug_args *args)
> pud = pud_wrprotect(pud);
> pud = pud_mkclean(pud);
> set_pud_at(args->mm, vaddr, args->pudp, pud);
> + flush_dcache_page(page);
> pud = pud_mkwrite(pud);
> pud = pud_mkdirty(pud);
> pudp_set_access_flags(args->vma, vaddr, args->pudp, pud, 1);
> @@ -382,6 +418,7 @@ static void __init pud_advanced_tests(struct
> pgtable_debug_args *args)
> pud = pfn_pud(args->pud_pfn, args->page_prot);
> pud = pud_mkyoung(pud);
> set_pud_at(args->mm, vaddr, args->pudp, pud);
> + flush_dcache_page(page);
> pudp_test_and_clear_young(args->vma, vaddr, args->pudp);
> pud = READ_ONCE(*args->pudp);
> WARN_ON(pud_young(pud));
> @@ -594,16 +631,26 @@ static void __init pgd_populate_tests(struct
> pgtable_debug_args *args) { }
>
> static void __init pte_clear_tests(struct pgtable_debug_args *args)
> {
> + struct page *page;
> pte_t pte = pfn_pte(args->pte_pfn, args->page_prot);
>
> - if (args->pte_pfn == ULONG_MAX)
> + page = (args->pte_pfn != ULONG_MAX) ? pfn_to_page(args->pte_pfn) : NULL;
> + if (!page)
> return;
>
> + /*
> + * flush_dcache_page() is called after set_pte_at() to clear
> + * PG_arch_1 for the page on ARM64. The page flag isn't cleared
> + * when it's released and page allocation check will fail when
> + * the page is allocated again. For architectures other than ARM64,
> + * the unexpected overhead of cache flushing is acceptable.
> + */
> pr_debug("Validating PTE clear\n");
> #ifndef CONFIG_RISCV
> pte = __pte(pte_val(pte) | RANDOM_ORVALUE);
> #endif
> set_pte_at(args->mm, args->vaddr, args->ptep, pte);
> + flush_dcache_page(page);
> barrier();
> pte_clear(args->mm, args->vaddr, args->ptep);
> pte = ptep_get(args->ptep);
> --
> 2.23.0
On 7/28/21 1:23 PM, Christophe Leroy wrote:
> Gavin Shan <[email protected]> a écrit :
>
>> In page table entry modifying tests, set_xxx_at() are used to populate
>> the page table entries. On ARM64, PG_arch_1 (PG_dcache_clean) flag is
>> set to the target page flag if execution permission is given. The logic
>> exits since commit 4f04d8f00545 ("arm64: MMU definitions"). The page
>> flag is kept when the page is free'd to buddy's free area list. However,
>> it will trigger page checking failure when it's pulled from the buddy's
>> free area list, as the following warning messages indicate.
>>
>> BUG: Bad page state in process memhog pfn:08000
>> page:0000000015c0a628 refcount:0 mapcount:0 \
>> mapping:0000000000000000 index:0x1 pfn:0x8000
>> flags: 0x7ffff8000000800(arch_1|node=0|zone=0|lastcpupid=0xfffff)
>> raw: 07ffff8000000800 dead000000000100 dead000000000122 0000000000000000
>> raw: 0000000000000001 0000000000000000 00000000ffffffff 0000000000000000
>> page dumped because: PAGE_FLAGS_CHECK_AT_PREP flag(s) set
>>
>> This fixes the issue by clearing PG_arch_1 through flush_dcache_page()
>> after set_xxx_at() is called. For architectures other than ARM64, the
>> unexpected overhead of cache flushing is acceptable.
>>
>> Signed-off-by: Gavin Shan <[email protected]>
>
> Maybe a Fixes: tag would be good to have
Agreed.
Fixes: a5c3b9ffb0f4 ("mm/debug_vm_pgtable: add tests validating advanced arch page table helpers")
>
> And would it be possible to have this fix as first patch of the series so that it can be applied to stable without applying the whole series ?
Changing the allocation scheme does solve another problem (using non-owned pages)
but is achieved via the entire series applied. But this particular patch could be
moved to the beginning without much problem.
On 7/27/21 11:43 AM, Gavin Shan wrote:
> This uses struct pgtable_debug_args in the migration and thp test
> functions. It's notable that the pre-allocated page is used in
> swap_migration_tests() as set_pte_at() is used there.
>
> Signed-off-by: Gavin Shan <[email protected]>
> ---
> mm/debug_vm_pgtable.c | 28 ++++++++++++++--------------
> 1 file changed, 14 insertions(+), 14 deletions(-)
>
> diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
> index bc153cad9045..9136195efde3 100644
> --- a/mm/debug_vm_pgtable.c
> +++ b/mm/debug_vm_pgtable.c
> @@ -845,7 +845,7 @@ static void __init pmd_swap_tests(struct pgtable_debug_args *args)
> static void __init pmd_swap_tests(struct pgtable_debug_args *args) { }
> #endif /* CONFIG_ARCH_ENABLE_THP_MIGRATION */
>
> -static void __init swap_migration_tests(void)
> +static void __init swap_migration_tests(struct pgtable_debug_args *args)
> {
> struct page *page;
> swp_entry_t swp;
> @@ -861,9 +861,10 @@ static void __init swap_migration_tests(void)
> * problematic. Lets allocate a dedicated page explicitly for this
> * purpose that will be freed subsequently.
> */
> - page = alloc_page(GFP_KERNEL);
> + page = (args->pte_pfn != ULONG_MAX) ?
> + pfn_to_page(args->pte_pfn) : NULL;
> if (!page) {
> - pr_err("page allocation failed\n");
> + pr_err("no page available\n");
> return;
> }
Please check for a valid page earlier in the function and return. Otherwise
this calls out the page unavailability (after starting the test), which is
inconsistent with all other functions like pxx_advanced_tests().
[ 1.051633] debug_vm_pgtable: [pte_swap_tests ]: Validating PTE swap
[ 1.052697] debug_vm_pgtable: [pmd_swap_tests ]: Validating PMD swap
[ 1.053765] debug_vm_pgtable: [swap_migration_tests ]: Validating swap migration <=====
[ 1.054900] debug_vm_pgtable: [swap_migration_tests ]: no page available <=====
Should do this just before pr_info("Validating swap migration\n").
......
page = (args->pte_pfn != ULONG_MAX) ? pfn_to_page(args->pte_pfn) : NULL;
if (!page)
return;
.....
Hi Anshuman,
On 7/28/21 9:08 PM, Anshuman Khandual wrote:
> On 7/27/21 11:43 AM, Gavin Shan wrote:
>> This uses struct pgtable_debug_args in the migration and thp test
>> functions. It's notable that the pre-allocated page is used in
>> swap_migration_tests() as set_pte_at() is used there.
>>
>> Signed-off-by: Gavin Shan <[email protected]>
>> ---
>> mm/debug_vm_pgtable.c | 28 ++++++++++++++--------------
>> 1 file changed, 14 insertions(+), 14 deletions(-)
>>
>> diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
>> index bc153cad9045..9136195efde3 100644
>> --- a/mm/debug_vm_pgtable.c
>> +++ b/mm/debug_vm_pgtable.c
>> @@ -845,7 +845,7 @@ static void __init pmd_swap_tests(struct pgtable_debug_args *args)
>> static void __init pmd_swap_tests(struct pgtable_debug_args *args) { }
>> #endif /* CONFIG_ARCH_ENABLE_THP_MIGRATION */
>>
>> -static void __init swap_migration_tests(void)
>> +static void __init swap_migration_tests(struct pgtable_debug_args *args)
>> {
>> struct page *page;
>> swp_entry_t swp;
>> @@ -861,9 +861,10 @@ static void __init swap_migration_tests(void)
>> * problematic. Lets allocate a dedicated page explicitly for this
>> * purpose that will be freed subsequently.
>> */
>> - page = alloc_page(GFP_KERNEL);
>> + page = (args->pte_pfn != ULONG_MAX) ?
>> + pfn_to_page(args->pte_pfn) : NULL;
>> if (!page) {
>> - pr_err("page allocation failed\n");
>> + pr_err("no page available\n");
>> return;
>> }
>
> Please check for a valid page earlier in the function and return. Otherwise
> this calls out the page unavailability (after starting the test), which is
> inconsistent with all other functions like pxx_advanced_tests().
>
> [ 1.051633] debug_vm_pgtable: [pte_swap_tests ]: Validating PTE swap
> [ 1.052697] debug_vm_pgtable: [pmd_swap_tests ]: Validating PMD swap
> [ 1.053765] debug_vm_pgtable: [swap_migration_tests ]: Validating swap migration <=====
> [ 1.054900] debug_vm_pgtable: [swap_migration_tests ]: no page available <=====
>
> Should do this just before pr_info("Validating swap migration\n").
>
> ......
> page = (args->pte_pfn != ULONG_MAX) ? pfn_to_page(args->pte_pfn) : NULL;
> if (!page)
> return;
> .....
>
Yes. The order of error messages are sticky to original implementation, but
it'd better to be consistent with the new order we have in this series. I
will adjust in v5.
Thanks,
Gavin
Hi Christophe and Anshuman,
On 7/28/21 8:05 PM, Anshuman Khandual wrote:
> On 7/28/21 1:23 PM, Christophe Leroy wrote:
>> Gavin Shan <[email protected]> a écrit :
>>> In page table entry modifying tests, set_xxx_at() are used to populate
>>> the page table entries. On ARM64, PG_arch_1 (PG_dcache_clean) flag is
>>> set to the target page flag if execution permission is given. The logic
>>> exits since commit 4f04d8f00545 ("arm64: MMU definitions"). The page
>>> flag is kept when the page is free'd to buddy's free area list. However,
>>> it will trigger page checking failure when it's pulled from the buddy's
>>> free area list, as the following warning messages indicate.
>>>
>>> BUG: Bad page state in process memhog pfn:08000
>>> page:0000000015c0a628 refcount:0 mapcount:0 \
>>> mapping:0000000000000000 index:0x1 pfn:0x8000
>>> flags: 0x7ffff8000000800(arch_1|node=0|zone=0|lastcpupid=0xfffff)
>>> raw: 07ffff8000000800 dead000000000100 dead000000000122 0000000000000000
>>> raw: 0000000000000001 0000000000000000 00000000ffffffff 0000000000000000
>>> page dumped because: PAGE_FLAGS_CHECK_AT_PREP flag(s) set
>>>
>>> This fixes the issue by clearing PG_arch_1 through flush_dcache_page()
>>> after set_xxx_at() is called. For architectures other than ARM64, the
>>> unexpected overhead of cache flushing is acceptable.
>>>
>>> Signed-off-by: Gavin Shan <[email protected]>
>>
>> Maybe a Fixes: tag would be good to have
>
> Agreed.
>
> Fixes: a5c3b9ffb0f4 ("mm/debug_vm_pgtable: add tests validating advanced arch page table helpers")
>
Yep, I will add the tag in v5.
>>
>> And would it be possible to have this fix as first patch of the series so that it can be applied to stable without applying the whole series ?
> Changing the allocation scheme does solve another problem (using non-owned pages)
> but is achieved via the entire series applied. But this particular patch could be
> moved to the beginning without much problem.
>
I prefer to keep current layout as explained before. Firstly, all
code changes included in this series are affecting only one source
file. It's hard to apply the whole series to stable kernel. I also
need apply this series to our downstream kernel once it hits upstream.
Secondly, applying PATCH[v4 12/12] can't resolve all issues we have.
It means we need to apply the whole series to resolve all issues.
Lastly, moving PATCH[v4 12/12] to PATCH[v4 01/12] will introduce
unnecessary code for subsequent patches. I don't think it's worthy
to do.
So lets keep current layout we have.
Thanks,
Gavin
On 7/27/21 11:43 AM, Gavin Shan wrote:
> In debug_vm_pgtable(), there are many local variables introduced to
> track the needed information and they are passed to the functions for
> various test cases. It'd better to introduce a struct as place holder
> for these information. With it, what the tests functions need is the
> struct. In this way, the code is simplified and easier to be maintained.
>
> Besides, set_xxx_at() could access the data on the corresponding pages
> in the page table modifying tests. So the accessed pages in the tests
> should have been allocated from buddy. Otherwise, we're accessing pages
> that aren't owned by us. This causes issues like page flag corruption
> or kernel crash on accessing unmapped page when CONFIG_DEBUG_PAGEALLOC
> is enabled.
>
> This introduces "struct pgtable_debug_args". The struct is initialized
> and destroyed, but the information in the struct isn't used yet. It will
> be used in subsequent patches.
>
> Signed-off-by: Gavin Shan <[email protected]>
> ---
> mm/debug_vm_pgtable.c | 280 +++++++++++++++++++++++++++++++++++++++++-
> 1 file changed, 279 insertions(+), 1 deletion(-)
>
> diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
> index 1c922691aa61..8c7361643166 100644
> --- a/mm/debug_vm_pgtable.c
> +++ b/mm/debug_vm_pgtable.c
> @@ -58,6 +58,37 @@
> #define RANDOM_ORVALUE (GENMASK(BITS_PER_LONG - 1, 0) & ~ARCH_SKIP_MASK)
> #define RANDOM_NZVALUE GENMASK(7, 0)
>
> +struct pgtable_debug_args {
> + struct mm_struct *mm;
> + struct vm_area_struct *vma;
> +
> + pgd_t *pgdp;
> + p4d_t *p4dp;
> + pud_t *pudp;
> + pmd_t *pmdp;
> + pte_t *ptep;
> +
> + p4d_t *start_p4dp;
> + pud_t *start_pudp;
> + pmd_t *start_pmdp;
> + pgtable_t start_ptep;
> +
> + unsigned long vaddr;
> + pgprot_t page_prot;
> + pgprot_t page_prot_none;
> +
> + bool is_contiguous_page;
> + unsigned long pud_pfn;
> + unsigned long pmd_pfn;
> + unsigned long pte_pfn;
> +
> + unsigned long fixed_pgd_pfn;
> + unsigned long fixed_p4d_pfn;
> + unsigned long fixed_pud_pfn;
> + unsigned long fixed_pmd_pfn;
> + unsigned long fixed_pte_pfn;
> +};
> +
> static void __init pte_basic_tests(unsigned long pfn, int idx)
> {
> pgprot_t prot = protection_map[idx];
> @@ -955,8 +986,249 @@ static unsigned long __init get_random_vaddr(void)
> return random_vaddr;
> }
>
> +static void __init destroy_args(struct pgtable_debug_args *args)
> +{
> + struct page *page = NULL;
> +
> + /* Free (huge) page */
> + if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) &&
> + IS_ENABLED(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD) &&
> + has_transparent_hugepage() &&
> + args->pud_pfn != ULONG_MAX) {
> + if (args->is_contiguous_page) {
> + free_contig_range(args->pud_pfn,
> + (1 << (HPAGE_PUD_SHIFT - PAGE_SHIFT)));
> + } else {
> + page = pfn_to_page(args->pud_pfn);
> + __free_pages(page, HPAGE_PUD_SHIFT - PAGE_SHIFT);
> + }
> +
> + args->pud_pfn = ULONG_MAX;
> + args->pmd_pfn = ULONG_MAX;
> + args->pte_pfn = ULONG_MAX;
> + }
> +
> + if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) &&
> + has_transparent_hugepage() &&
> + args->pmd_pfn != ULONG_MAX) {
> + if (args->is_contiguous_page) {
> + free_contig_range(args->pmd_pfn, (1 << HPAGE_PMD_ORDER));
> + } else {
> + page = pfn_to_page(args->pmd_pfn);
> + __free_pages(page, HPAGE_PMD_ORDER);
> + }
> +
> + args->pmd_pfn = ULONG_MAX;
> + args->pte_pfn = ULONG_MAX;
> + }
> +
> + if (args->pte_pfn != ULONG_MAX) {
> + page = pfn_to_page(args->pte_pfn);
> + __free_pages(page, 0);
> + }
> +
> + /* Free page table entries */
> + if (args->start_ptep) {
> + pte_free(args->mm, args->start_ptep);
> + mm_dec_nr_ptes(args->mm);
> + }
> +
> + if (args->start_pmdp) {
> + pmd_free(args->mm, args->start_pmdp);
> + mm_dec_nr_pmds(args->mm);
> + }
> +
> + if (args->start_pudp) {
> + pud_free(args->mm, args->start_pudp);
> + mm_dec_nr_puds(args->mm);
> + }
> +
> + if (args->start_p4dp)
> + p4d_free(args->mm, args->p4dp);
> +
> + /* Free vma and mm struct */
> + if (args->vma)
> + vm_area_free(args->vma);
Small nit, needs an extra line here.
> + if (args->mm)
> + mmdrop(args->mm);
> +}
> +
> +static int __init init_args(struct pgtable_debug_args *args)
> +{
> + struct page *page = NULL;
> + phys_addr_t phys;
> + int ret = 0;
> +
> + /*
> + * Initialize the debugging data.
> + *
> + * __P000 (or even __S000) will help create page table entries with
> + * PROT_NONE permission as required for pxx_protnone_tests().
> + */
> + memset(args, 0, sizeof(*args));
> + args->vaddr = get_random_vaddr();
> + args->page_prot = vm_get_page_prot(VMFLAGS);
> + args->page_prot_none = __P000;
> + args->is_contiguous_page = false;
> + args->pud_pfn = ULONG_MAX;
> + args->pmd_pfn = ULONG_MAX;
> + args->pte_pfn = ULONG_MAX;
> + args->fixed_pgd_pfn = ULONG_MAX;
> + args->fixed_p4d_pfn = ULONG_MAX;
> + args->fixed_pud_pfn = ULONG_MAX;
> + args->fixed_pmd_pfn = ULONG_MAX;
> + args->fixed_pte_pfn = ULONG_MAX;
> +
> + /* Allocate mm and vma */
> + args->mm = mm_alloc();
> + if (!args->mm) {
> + pr_err("Failed to allocate mm struct\n");
> + ret = -ENOMEM;
> + goto error;
> + }
> +
> + args->vma = vm_area_alloc(args->mm);
> + if (!args->vma) {
> + pr_err("Failed to allocate vma\n");
> + ret = -ENOMEM;
> + goto error;
> + }
> +
> + /*
> + * Allocate page table entries. They will be modified in the tests.
> + * Lets save the page table entries so that they can be released
> + * when the tests are completed.
> + */
> + args->pgdp = pgd_offset(args->mm, args->vaddr);
> + args->p4dp = p4d_alloc(args->mm, args->pgdp, args->vaddr);
> + if (!args->p4dp) {
> + pr_err("Failed to allocate p4d entries\n");
> + ret = -ENOMEM;
> + goto error;
> + }
> +
> + args->start_p4dp = p4d_offset(args->pgdp, 0UL);
> + WARN_ON(!args->start_p4dp);
Please move these two lines up to the previous block as args->start_p4dp
is primarily derived from args->pgdp. Also please do the same for all
others blocks down here.
> + args->pudp = pud_alloc(args->mm, args->p4dp, args->vaddr);
> + if (!args->pudp) {
> + pr_err("Failed to allocate pud entries\n");
> + ret = -ENOMEM;
> + goto error;
> + }
> +
> + args->start_pudp = pud_offset(args->p4dp, 0UL);
> + WARN_ON(!args->start_pudp);
> + args->pmdp = pmd_alloc(args->mm, args->pudp, args->vaddr);
> + if (!args->pmdp) {
> + pr_err("Failed to allocate pmd entries\n");
> + ret = -ENOMEM;
> + goto error;
> + }
> +
> + args->start_pmdp = pmd_offset(args->pudp, 0UL);
> + WARN_ON(!args->start_pmdp);
> + args->ptep = pte_alloc_map(args->mm, args->pmdp, args->vaddr);
> + if (!args->ptep) {
> + pr_err("Failed to allocate pte entries\n");
> + ret = -ENOMEM;
> + goto error;
> + }
> +
> + args->start_ptep = pmd_pgtable(READ_ONCE(*args->pmdp));
> + WARN_ON(!args->start_ptep);
> +
> + /*
> + * PFN for mapping at PTE level is determined from a standard kernel
> + * text symbol. But pfns for higher page table levels are derived by
> + * masking lower bits of this real pfn. These derived pfns might not
> + * exist on the platform but that does not really matter as pfn_pxx()
> + * helpers will still create appropriate entries for the test. This
> + * helps avoid large memory block allocations to be used for mapping
> + * at higher page table levels in some of the tests.
> + */
> + phys = __pa_symbol(&start_kernel);
> + args->fixed_pgd_pfn = __phys_to_pfn(phys & PGDIR_MASK);
> + args->fixed_p4d_pfn = __phys_to_pfn(phys & P4D_MASK);
> + args->fixed_pud_pfn = __phys_to_pfn(phys & PUD_MASK);
> + args->fixed_pmd_pfn = __phys_to_pfn(phys & PMD_MASK);
> + args->fixed_pte_pfn = __phys_to_pfn(phys & PAGE_MASK);
> + WARN_ON(!pfn_valid(args->fixed_pte_pfn));
> +
> + /*
> + * Allocate (huge) pages because some of the tests need to access
> + * the data in the pages. The corresponding tests will be skipped
> + * if we fail to allocate (huge) pages.
> + */
> +#ifdef CONFIG_CONTIG_ALLOC
> + if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) &&
> + IS_ENABLED(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD) &&
> + has_transparent_hugepage() &&
> + (HPAGE_PUD_SHIFT - PAGE_SHIFT) >= MAX_ORDER) {
> + page = alloc_contig_pages((1 << (HPAGE_PUD_SHIFT - PAGE_SHIFT)),
> + GFP_KERNEL, first_online_node, NULL);
> + if (page) {
> + args->is_contiguous_page = true;
> + args->pud_pfn = page_to_pfn(page);
> + args->pmd_pfn = args->pud_pfn;
> + args->pte_pfn = args->pud_pfn;
> + return 0;
> + }
> + }
> +#endif
> +
> + if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) &&
> + IS_ENABLED(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD) &&
> + has_transparent_hugepage() &&
> + (HPAGE_PUD_SHIFT - PAGE_SHIFT) < MAX_ORDER) {
> + page = alloc_pages(GFP_KERNEL, HPAGE_PUD_SHIFT - PAGE_SHIFT);
> + if (page) {
> + args->pud_pfn = page_to_pfn(page);
> + args->pmd_pfn = args->pud_pfn;
> + args->pte_pfn = args->pud_pfn;
> + return 0;
> + }
> + }
> +
> +#ifdef CONFIG_CONTIG_ALLOC
> + if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) &&
> + has_transparent_hugepage() &&
> + HPAGE_PMD_ORDER >= MAX_ORDER) {
> + page = alloc_contig_pages((1 << HPAGE_PMD_ORDER), GFP_KERNEL,
> + first_online_node, NULL);
> + if (page) {
> + args->is_contiguous_page = true;
> + args->pmd_pfn = page_to_pfn(page);
> + args->pte_pfn = args->pmd_pfn;
> + return 0;
> + }
> + }
> +#endif
> +
> + if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) &&
> + has_transparent_hugepage() &&
> + HPAGE_PMD_ORDER < MAX_ORDER) {
> + page = alloc_pages(GFP_KERNEL, HPAGE_PMD_ORDER);
> + if (page) {
> + args->pmd_pfn = page_to_pfn(page);
> + args->pte_pfn = args->pmd_pfn;
> + return 0;
> + }
> + }
This can be simplified further. Could you please define a helper alloc_huge_page()
which compares the order against MAX_ORDER and calls either alloc_contig_pages()
when CONFIG_CONTIG_ALLOC or alloc_pages(). This will result in reduced code and
CONFIG_CONTIG_ALLOC will move into the helper as well.
> +
> + page = alloc_pages(GFP_KERNEL, 0);
> + if (page)
> + args->pte_pfn = page_to_pfn(page);
> +
> + return 0;
> +
> +error:
> + destroy_args(args);
> + return ret;
> +}
> +
> static int __init debug_vm_pgtable(void)
> {
> + struct pgtable_debug_args args;
> struct vm_area_struct *vma;
> struct mm_struct *mm;
> pgd_t *pgdp;
> @@ -970,9 +1242,13 @@ static int __init debug_vm_pgtable(void)
> unsigned long vaddr, pte_aligned, pmd_aligned;
> unsigned long pud_aligned, p4d_aligned, pgd_aligned;
> spinlock_t *ptl = NULL;
> - int idx;
> + int idx, ret;
>
> pr_info("Validating architecture page table helpers\n");
> + ret = init_args(&args);
> + if (ret)
> + return ret;
> +
> prot = vm_get_page_prot(VMFLAGS);
> vaddr = get_random_vaddr();
> mm = mm_alloc();
> @@ -1127,6 +1403,8 @@ static int __init debug_vm_pgtable(void)
> mm_dec_nr_pmds(mm);
> mm_dec_nr_ptes(mm);
> mmdrop(mm);
> +
> + destroy_args(&args);
> return 0;
> }
> late_initcall(debug_vm_pgtable);
>
Otherwise LGTM.
On 7/27/21 11:43 AM, Gavin Shan wrote:
> This uses struct pgtable_debug_args in the basic test functions. The
> unused variables @pgd_aligned and @p4d_aligned in debug_vm_pgtable()
> are dropped.
>
> Signed-off-by: Gavin Shan <[email protected]>
> ---
> mm/debug_vm_pgtable.c | 50 +++++++++++++++++++++----------------------
> 1 file changed, 24 insertions(+), 26 deletions(-)
>
> diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
> index 8c7361643166..8498aa180ebc 100644
> --- a/mm/debug_vm_pgtable.c
> +++ b/mm/debug_vm_pgtable.c
> @@ -89,10 +89,10 @@ struct pgtable_debug_args {
> unsigned long fixed_pte_pfn;
> };
>
> -static void __init pte_basic_tests(unsigned long pfn, int idx)
> +static void __init pte_basic_tests(struct pgtable_debug_args *args, int idx)
> {
> pgprot_t prot = protection_map[idx];
> - pte_t pte = pfn_pte(pfn, prot);
> + pte_t pte = pfn_pte(args->fixed_pte_pfn, prot);
> unsigned long val = idx, *ptr = &val;
>
> pr_debug("Validating PTE basic (%pGv)\n", ptr);
> @@ -174,7 +174,7 @@ static void __init pte_savedwrite_tests(unsigned long pfn, pgprot_t prot)
> }
>
> #ifdef CONFIG_TRANSPARENT_HUGEPAGE
> -static void __init pmd_basic_tests(unsigned long pfn, int idx)
> +static void __init pmd_basic_tests(struct pgtable_debug_args *args, int idx)
> {
> pgprot_t prot = protection_map[idx];
> unsigned long val = idx, *ptr = &val;
> @@ -184,7 +184,7 @@ static void __init pmd_basic_tests(unsigned long pfn, int idx)
> return;
>
> pr_debug("Validating PMD basic (%pGv)\n", ptr);
> - pmd = pfn_pmd(pfn, prot);
> + pmd = pfn_pmd(args->fixed_pmd_pfn, prot);
>
> /*
> * This test needs to be executed after the given page table entry
> @@ -296,7 +296,7 @@ static void __init pmd_savedwrite_tests(unsigned long pfn, pgprot_t prot)
> }
>
> #ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
> -static void __init pud_basic_tests(struct mm_struct *mm, unsigned long pfn, int idx)
> +static void __init pud_basic_tests(struct pgtable_debug_args *args, int idx)
> {
> pgprot_t prot = protection_map[idx];
> unsigned long val = idx, *ptr = &val;
> @@ -306,7 +306,7 @@ static void __init pud_basic_tests(struct mm_struct *mm, unsigned long pfn, int
> return;
>
> pr_debug("Validating PUD basic (%pGv)\n", ptr);
> - pud = pfn_pud(pfn, prot);
> + pud = pfn_pud(args->fixed_pud_pfn, prot);
>
> /*
> * This test needs to be executed after the given page table entry
> @@ -327,7 +327,7 @@ static void __init pud_basic_tests(struct mm_struct *mm, unsigned long pfn, int
> WARN_ON(pud_dirty(pud_wrprotect(pud_mkclean(pud))));
> WARN_ON(!pud_dirty(pud_wrprotect(pud_mkdirty(pud))));
>
> - if (mm_pmd_folded(mm))
> + if (mm_pmd_folded(args->mm))
> return;
>
> /*
> @@ -404,7 +404,7 @@ static void __init pud_leaf_tests(unsigned long pfn, pgprot_t prot)
> WARN_ON(!pud_leaf(pud));
> }
> #else /* !CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */
> -static void __init pud_basic_tests(struct mm_struct *mm, unsigned long pfn, int idx) { }
> +static void __init pud_basic_tests(struct pgtable_debug_args *args, int idx) { }
> static void __init pud_advanced_tests(struct mm_struct *mm,
> struct vm_area_struct *vma, pud_t *pudp,
> unsigned long pfn, unsigned long vaddr,
> @@ -414,8 +414,8 @@ static void __init pud_advanced_tests(struct mm_struct *mm,
> static void __init pud_leaf_tests(unsigned long pfn, pgprot_t prot) { }
> #endif /* CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */
> #else /* !CONFIG_TRANSPARENT_HUGEPAGE */
> -static void __init pmd_basic_tests(unsigned long pfn, int idx) { }
> -static void __init pud_basic_tests(struct mm_struct *mm, unsigned long pfn, int idx) { }
> +static void __init pmd_basic_tests(struct pgtable_debug_args *args, int idx) { }
> +static void __init pud_basic_tests(struct pgtable_debug_args *args, int idx) { }
> static void __init pmd_advanced_tests(struct mm_struct *mm,
> struct vm_area_struct *vma, pmd_t *pmdp,
> unsigned long pfn, unsigned long vaddr,
> @@ -476,7 +476,7 @@ static void __init pmd_huge_tests(pmd_t *pmdp, unsigned long pfn, pgprot_t prot)
> static void __init pud_huge_tests(pud_t *pudp, unsigned long pfn, pgprot_t prot) { }
> #endif /* CONFIG_HAVE_ARCH_HUGE_VMAP */
>
> -static void __init p4d_basic_tests(unsigned long pfn, pgprot_t prot)
> +static void __init p4d_basic_tests(struct pgtable_debug_args *args)
> {
> p4d_t p4d;
>
> @@ -485,7 +485,7 @@ static void __init p4d_basic_tests(unsigned long pfn, pgprot_t prot)
> WARN_ON(!p4d_same(p4d, p4d));
> }
>
> -static void __init pgd_basic_tests(unsigned long pfn, pgprot_t prot)
> +static void __init pgd_basic_tests(struct pgtable_debug_args *args)
> {
> pgd_t pgd;
>
> @@ -890,7 +890,7 @@ static void __init swap_migration_tests(void)
> }
>
> #ifdef CONFIG_HUGETLB_PAGE
> -static void __init hugetlb_basic_tests(unsigned long pfn, pgprot_t prot)
> +static void __init hugetlb_basic_tests(struct pgtable_debug_args *args)
> {
> struct page *page;
> pte_t pte;
> @@ -900,21 +900,21 @@ static void __init hugetlb_basic_tests(unsigned long pfn, pgprot_t prot)
> * Accessing the page associated with the pfn is safe here,
> * as it was previously derived from a real kernel symbol.
> */
> - page = pfn_to_page(pfn);
> - pte = mk_huge_pte(page, prot);
> + page = pfn_to_page(args->fixed_pmd_pfn);
> + pte = mk_huge_pte(page, args->page_prot);
>
> WARN_ON(!huge_pte_dirty(huge_pte_mkdirty(pte)));
> WARN_ON(!huge_pte_write(huge_pte_mkwrite(huge_pte_wrprotect(pte))));
> WARN_ON(huge_pte_write(huge_pte_wrprotect(huge_pte_mkwrite(pte))));
>
> #ifdef CONFIG_ARCH_WANT_GENERAL_HUGETLB
> - pte = pfn_pte(pfn, prot);
> + pte = pfn_pte(args->fixed_pmd_pfn, args->page_prot);
>
> WARN_ON(!pte_huge(pte_mkhuge(pte)));
> #endif /* CONFIG_ARCH_WANT_GENERAL_HUGETLB */
> }
> #else /* !CONFIG_HUGETLB_PAGE */
> -static void __init hugetlb_basic_tests(unsigned long pfn, pgprot_t prot) { }
> +static void __init hugetlb_basic_tests(struct pgtable_debug_args *args) { }
> #endif /* CONFIG_HUGETLB_PAGE */
>
> #ifdef CONFIG_TRANSPARENT_HUGEPAGE
> @@ -1240,7 +1240,7 @@ static int __init debug_vm_pgtable(void)
> pgprot_t prot, protnone;
> phys_addr_t paddr;
> unsigned long vaddr, pte_aligned, pmd_aligned;
> - unsigned long pud_aligned, p4d_aligned, pgd_aligned;
> + unsigned long pud_aligned;
> spinlock_t *ptl = NULL;
> int idx, ret;
>
> @@ -1283,8 +1283,6 @@ static int __init debug_vm_pgtable(void)
> pte_aligned = (paddr & PAGE_MASK) >> PAGE_SHIFT;
> pmd_aligned = (paddr & PMD_MASK) >> PAGE_SHIFT;
> pud_aligned = (paddr & PUD_MASK) >> PAGE_SHIFT;
> - p4d_aligned = (paddr & P4D_MASK) >> PAGE_SHIFT;
> - pgd_aligned = (paddr & PGDIR_MASK) >> PAGE_SHIFT;
> WARN_ON(!pfn_valid(pte_aligned));
>
> pgdp = pgd_offset(mm, vaddr);
> @@ -1318,9 +1316,9 @@ static int __init debug_vm_pgtable(void)
> * given page table entry.
> */
> for (idx = 0; idx < ARRAY_SIZE(protection_map); idx++) {
> - pte_basic_tests(pte_aligned, idx);
> - pmd_basic_tests(pmd_aligned, idx);
> - pud_basic_tests(mm, pud_aligned, idx);
> + pte_basic_tests(&args, idx);
> + pmd_basic_tests(&args, idx);
> + pud_basic_tests(&args, idx);
> }
>
> /*
> @@ -1330,8 +1328,8 @@ static int __init debug_vm_pgtable(void)
> * the above iteration for now to save some test execution
> * time.
> */
> - p4d_basic_tests(p4d_aligned, prot);
> - pgd_basic_tests(pgd_aligned, prot);
> + p4d_basic_tests(&args);
> + pgd_basic_tests(&args);
>
> pmd_leaf_tests(pmd_aligned, prot);
> pud_leaf_tests(pud_aligned, prot);
> @@ -1360,7 +1358,7 @@ static int __init debug_vm_pgtable(void)
> pmd_thp_tests(pmd_aligned, prot);
> pud_thp_tests(pud_aligned, prot);
>
> - hugetlb_basic_tests(pte_aligned, prot);
> + hugetlb_basic_tests(&args);
>
> /*
> * Page table modifying tests. They need to hold
>
Reviewed-by: Anshuman Khandual <[email protected]>
On 7/27/21 11:43 AM, Gavin Shan wrote:
> This uses struct pgtable_debug_args in the leaf and savewrite test
> functions.
>
> Signed-off-by: Gavin Shan <[email protected]>
> ---
> mm/debug_vm_pgtable.c | 32 ++++++++++++++++----------------
> 1 file changed, 16 insertions(+), 16 deletions(-)
>
> diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
> index 8498aa180ebc..0fa98083f7d5 100644
> --- a/mm/debug_vm_pgtable.c
> +++ b/mm/debug_vm_pgtable.c
> @@ -161,9 +161,9 @@ static void __init pte_advanced_tests(struct mm_struct *mm,
> WARN_ON(pte_young(pte));
> }
>
> -static void __init pte_savedwrite_tests(unsigned long pfn, pgprot_t prot)
> +static void __init pte_savedwrite_tests(struct pgtable_debug_args *args)
> {
> - pte_t pte = pfn_pte(pfn, prot);
> + pte_t pte = pfn_pte(args->fixed_pte_pfn, args->page_prot_none);
>
> if (!IS_ENABLED(CONFIG_NUMA_BALANCING))
> return;
> @@ -262,7 +262,7 @@ static void __init pmd_advanced_tests(struct mm_struct *mm,
> pgtable = pgtable_trans_huge_withdraw(mm, pmdp);
> }
>
> -static void __init pmd_leaf_tests(unsigned long pfn, pgprot_t prot)
> +static void __init pmd_leaf_tests(struct pgtable_debug_args *args)
> {
> pmd_t pmd;
>
> @@ -270,7 +270,7 @@ static void __init pmd_leaf_tests(unsigned long pfn, pgprot_t prot)
> return;
>
> pr_debug("Validating PMD leaf\n");
> - pmd = pfn_pmd(pfn, prot);
> + pmd = pfn_pmd(args->fixed_pmd_pfn, args->page_prot);
>
> /*
> * PMD based THP is a leaf entry.
> @@ -279,7 +279,7 @@ static void __init pmd_leaf_tests(unsigned long pfn, pgprot_t prot)
> WARN_ON(!pmd_leaf(pmd));
> }
>
> -static void __init pmd_savedwrite_tests(unsigned long pfn, pgprot_t prot)
> +static void __init pmd_savedwrite_tests(struct pgtable_debug_args *args)
> {
> pmd_t pmd;
>
> @@ -290,7 +290,7 @@ static void __init pmd_savedwrite_tests(unsigned long pfn, pgprot_t prot)
> return;
>
> pr_debug("Validating PMD saved write\n");
> - pmd = pfn_pmd(pfn, prot);
> + pmd = pfn_pmd(args->fixed_pmd_pfn, args->page_prot_none);
> WARN_ON(!pmd_savedwrite(pmd_mk_savedwrite(pmd_clear_savedwrite(pmd))));
> WARN_ON(pmd_savedwrite(pmd_clear_savedwrite(pmd_mk_savedwrite(pmd))));
> }
> @@ -388,7 +388,7 @@ static void __init pud_advanced_tests(struct mm_struct *mm,
> pudp_huge_get_and_clear(mm, vaddr, pudp);
> }
>
> -static void __init pud_leaf_tests(unsigned long pfn, pgprot_t prot)
> +static void __init pud_leaf_tests(struct pgtable_debug_args *args)
> {
> pud_t pud;
>
> @@ -396,7 +396,7 @@ static void __init pud_leaf_tests(unsigned long pfn, pgprot_t prot)
> return;
>
> pr_debug("Validating PUD leaf\n");
> - pud = pfn_pud(pfn, prot);
> + pud = pfn_pud(args->fixed_pud_pfn, args->page_prot);
> /*
> * PUD based THP is a leaf entry.
> */
> @@ -411,7 +411,7 @@ static void __init pud_advanced_tests(struct mm_struct *mm,
> pgprot_t prot)
> {
> }
> -static void __init pud_leaf_tests(unsigned long pfn, pgprot_t prot) { }
> +static void __init pud_leaf_tests(struct pgtable_debug_args *args) { }
> #endif /* CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */
> #else /* !CONFIG_TRANSPARENT_HUGEPAGE */
> static void __init pmd_basic_tests(struct pgtable_debug_args *args, int idx) { }
> @@ -428,9 +428,9 @@ static void __init pud_advanced_tests(struct mm_struct *mm,
> pgprot_t prot)
> {
> }
> -static void __init pmd_leaf_tests(unsigned long pfn, pgprot_t prot) { }
> -static void __init pud_leaf_tests(unsigned long pfn, pgprot_t prot) { }
> -static void __init pmd_savedwrite_tests(unsigned long pfn, pgprot_t prot) { }
> +static void __init pmd_leaf_tests(struct pgtable_debug_args *args) { }
> +static void __init pud_leaf_tests(struct pgtable_debug_args *args) { }
> +static void __init pmd_savedwrite_tests(struct pgtable_debug_args *args) { }
> #endif /* CONFIG_TRANSPARENT_HUGEPAGE */
>
> #ifdef CONFIG_HAVE_ARCH_HUGE_VMAP
> @@ -1331,11 +1331,11 @@ static int __init debug_vm_pgtable(void)
> p4d_basic_tests(&args);
> pgd_basic_tests(&args);
>
> - pmd_leaf_tests(pmd_aligned, prot);
> - pud_leaf_tests(pud_aligned, prot);
> + pmd_leaf_tests(&args);
> + pud_leaf_tests(&args);
>
> - pte_savedwrite_tests(pte_aligned, protnone);
> - pmd_savedwrite_tests(pmd_aligned, protnone);
> + pte_savedwrite_tests(&args);
> + pmd_savedwrite_tests(&args);
>
> pte_special_tests(pte_aligned, prot);
> pte_protnone_tests(pte_aligned, protnone);
>
Reviewed-by: Anshuman Khandual <[email protected]>
On 7/27/21 11:43 AM, Gavin Shan wrote:
> This uses struct pgtable_debug_args in protnone and devmap test
> functions. After that, the unused variable @protnone in debug_vm_pgtable()
> is dropped.
>
> Signed-off-by: Gavin Shan <[email protected]>
> ---
> mm/debug_vm_pgtable.c | 58 +++++++++++++++++++------------------------
> 1 file changed, 26 insertions(+), 32 deletions(-)
>
> diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
> index 0fa98083f7d5..650e35656660 100644
> --- a/mm/debug_vm_pgtable.c
> +++ b/mm/debug_vm_pgtable.c
> @@ -662,9 +662,9 @@ static void __init pmd_populate_tests(struct mm_struct *mm, pmd_t *pmdp,
> WARN_ON(pmd_bad(pmd));
> }
>
> -static void __init pte_special_tests(unsigned long pfn, pgprot_t prot)
> +static void __init pte_special_tests(struct pgtable_debug_args *args)
> {
> - pte_t pte = pfn_pte(pfn, prot);
> + pte_t pte = pfn_pte(args->fixed_pte_pfn, args->page_prot);
>
> if (!IS_ENABLED(CONFIG_ARCH_HAS_PTE_SPECIAL))
> return;
> @@ -673,9 +673,9 @@ static void __init pte_special_tests(unsigned long pfn, pgprot_t prot)
> WARN_ON(!pte_special(pte_mkspecial(pte)));
> }
>
> -static void __init pte_protnone_tests(unsigned long pfn, pgprot_t prot)
> +static void __init pte_protnone_tests(struct pgtable_debug_args *args)
> {
> - pte_t pte = pfn_pte(pfn, prot);
> + pte_t pte = pfn_pte(args->fixed_pte_pfn, args->page_prot_none);
>
> if (!IS_ENABLED(CONFIG_NUMA_BALANCING))
> return;
> @@ -686,7 +686,7 @@ static void __init pte_protnone_tests(unsigned long pfn, pgprot_t prot)
> }
>
> #ifdef CONFIG_TRANSPARENT_HUGEPAGE
> -static void __init pmd_protnone_tests(unsigned long pfn, pgprot_t prot)
> +static void __init pmd_protnone_tests(struct pgtable_debug_args *args)
> {
> pmd_t pmd;
>
> @@ -697,25 +697,25 @@ static void __init pmd_protnone_tests(unsigned long pfn, pgprot_t prot)
> return;
>
> pr_debug("Validating PMD protnone\n");
> - pmd = pmd_mkhuge(pfn_pmd(pfn, prot));
> + pmd = pmd_mkhuge(pfn_pmd(args->fixed_pmd_pfn, args->page_prot_none));
> WARN_ON(!pmd_protnone(pmd));
> WARN_ON(!pmd_present(pmd));
> }
> #else /* !CONFIG_TRANSPARENT_HUGEPAGE */
> -static void __init pmd_protnone_tests(unsigned long pfn, pgprot_t prot) { }
> +static void __init pmd_protnone_tests(struct pgtable_debug_args *args) { }
> #endif /* CONFIG_TRANSPARENT_HUGEPAGE */
>
> #ifdef CONFIG_ARCH_HAS_PTE_DEVMAP
> -static void __init pte_devmap_tests(unsigned long pfn, pgprot_t prot)
> +static void __init pte_devmap_tests(struct pgtable_debug_args *args)
> {
> - pte_t pte = pfn_pte(pfn, prot);
> + pte_t pte = pfn_pte(args->fixed_pte_pfn, args->page_prot);
>
> pr_debug("Validating PTE devmap\n");
> WARN_ON(!pte_devmap(pte_mkdevmap(pte)));
> }
>
> #ifdef CONFIG_TRANSPARENT_HUGEPAGE
> -static void __init pmd_devmap_tests(unsigned long pfn, pgprot_t prot)
> +static void __init pmd_devmap_tests(struct pgtable_debug_args *args)
> {
> pmd_t pmd;
>
> @@ -723,12 +723,12 @@ static void __init pmd_devmap_tests(unsigned long pfn, pgprot_t prot)
> return;
>
> pr_debug("Validating PMD devmap\n");
> - pmd = pfn_pmd(pfn, prot);
> + pmd = pfn_pmd(args->fixed_pmd_pfn, args->page_prot);
> WARN_ON(!pmd_devmap(pmd_mkdevmap(pmd)));
> }
>
> #ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
> -static void __init pud_devmap_tests(unsigned long pfn, pgprot_t prot)
> +static void __init pud_devmap_tests(struct pgtable_debug_args *args)
> {
> pud_t pud;
>
> @@ -736,20 +736,20 @@ static void __init pud_devmap_tests(unsigned long pfn, pgprot_t prot)
> return;
>
> pr_debug("Validating PUD devmap\n");
> - pud = pfn_pud(pfn, prot);
> + pud = pfn_pud(args->fixed_pud_pfn, args->page_prot);
> WARN_ON(!pud_devmap(pud_mkdevmap(pud)));
> }
> #else /* !CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */
> -static void __init pud_devmap_tests(unsigned long pfn, pgprot_t prot) { }
> +static void __init pud_devmap_tests(struct pgtable_debug_args *args) { }
> #endif /* CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */
> #else /* CONFIG_TRANSPARENT_HUGEPAGE */
> -static void __init pmd_devmap_tests(unsigned long pfn, pgprot_t prot) { }
> -static void __init pud_devmap_tests(unsigned long pfn, pgprot_t prot) { }
> +static void __init pmd_devmap_tests(struct pgtable_debug_args *args) { }
> +static void __init pud_devmap_tests(struct pgtable_debug_args *args) { }
> #endif /* CONFIG_TRANSPARENT_HUGEPAGE */
> #else
> -static void __init pte_devmap_tests(unsigned long pfn, pgprot_t prot) { }
> -static void __init pmd_devmap_tests(unsigned long pfn, pgprot_t prot) { }
> -static void __init pud_devmap_tests(unsigned long pfn, pgprot_t prot) { }
> +static void __init pte_devmap_tests(struct pgtable_debug_args *args) { }
> +static void __init pmd_devmap_tests(struct pgtable_debug_args *args) { }
> +static void __init pud_devmap_tests(struct pgtable_debug_args *args) { }
> #endif /* CONFIG_ARCH_HAS_PTE_DEVMAP */
>
> static void __init pte_soft_dirty_tests(unsigned long pfn, pgprot_t prot)
> @@ -1237,7 +1237,7 @@ static int __init debug_vm_pgtable(void)
> pmd_t *pmdp, *saved_pmdp, pmd;
> pte_t *ptep;
> pgtable_t saved_ptep;
> - pgprot_t prot, protnone;
> + pgprot_t prot;
> phys_addr_t paddr;
> unsigned long vaddr, pte_aligned, pmd_aligned;
> unsigned long pud_aligned;
> @@ -1257,12 +1257,6 @@ static int __init debug_vm_pgtable(void)
> return 1;
> }
>
> - /*
> - * __P000 (or even __S000) will help create page table entries with
> - * PROT_NONE permission as required for pxx_protnone_tests().
> - */
> - protnone = __P000;
> -
> vma = vm_area_alloc(mm);
> if (!vma) {
> pr_err("vma allocation failed\n");
> @@ -1337,13 +1331,13 @@ static int __init debug_vm_pgtable(void)
> pte_savedwrite_tests(&args);
> pmd_savedwrite_tests(&args);
>
> - pte_special_tests(pte_aligned, prot);
> - pte_protnone_tests(pte_aligned, protnone);
> - pmd_protnone_tests(pmd_aligned, protnone);
> + pte_special_tests(&args);
> + pte_protnone_tests(&args);
> + pmd_protnone_tests(&args);
>
> - pte_devmap_tests(pte_aligned, prot);
> - pmd_devmap_tests(pmd_aligned, prot);
> - pud_devmap_tests(pud_aligned, prot);
> + pte_devmap_tests(&args);
> + pmd_devmap_tests(&args);
> + pud_devmap_tests(&args);
>
> pte_soft_dirty_tests(pte_aligned, prot);
> pmd_soft_dirty_tests(pmd_aligned, prot);
>
Reviewed-by: Anshuman Khandual <[email protected]>
On 7/27/21 11:43 AM, Gavin Shan wrote:
> This uses struct pgtable_debug_args in the soft_dirty and swap test
> functions.
>
> Signed-off-by: Gavin Shan <[email protected]>
> ---
> mm/debug_vm_pgtable.c | 48 +++++++++++++++++++++----------------------
> 1 file changed, 23 insertions(+), 25 deletions(-)
>
> diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
> index 650e35656660..bc153cad9045 100644
> --- a/mm/debug_vm_pgtable.c
> +++ b/mm/debug_vm_pgtable.c
> @@ -752,9 +752,9 @@ static void __init pmd_devmap_tests(struct pgtable_debug_args *args) { }
> static void __init pud_devmap_tests(struct pgtable_debug_args *args) { }
> #endif /* CONFIG_ARCH_HAS_PTE_DEVMAP */
>
> -static void __init pte_soft_dirty_tests(unsigned long pfn, pgprot_t prot)
> +static void __init pte_soft_dirty_tests(struct pgtable_debug_args *args)
> {
> - pte_t pte = pfn_pte(pfn, prot);
> + pte_t pte = pfn_pte(args->fixed_pte_pfn, args->page_prot);
>
> if (!IS_ENABLED(CONFIG_MEM_SOFT_DIRTY))
> return;
> @@ -764,9 +764,9 @@ static void __init pte_soft_dirty_tests(unsigned long pfn, pgprot_t prot)
> WARN_ON(pte_soft_dirty(pte_clear_soft_dirty(pte)));
> }
>
> -static void __init pte_swap_soft_dirty_tests(unsigned long pfn, pgprot_t prot)
> +static void __init pte_swap_soft_dirty_tests(struct pgtable_debug_args *args)
> {
> - pte_t pte = pfn_pte(pfn, prot);
> + pte_t pte = pfn_pte(args->fixed_pte_pfn, args->page_prot);
>
> if (!IS_ENABLED(CONFIG_MEM_SOFT_DIRTY))
> return;
> @@ -777,7 +777,7 @@ static void __init pte_swap_soft_dirty_tests(unsigned long pfn, pgprot_t prot)
> }
>
> #ifdef CONFIG_TRANSPARENT_HUGEPAGE
> -static void __init pmd_soft_dirty_tests(unsigned long pfn, pgprot_t prot)
> +static void __init pmd_soft_dirty_tests(struct pgtable_debug_args *args)
> {
> pmd_t pmd;
>
> @@ -788,12 +788,12 @@ static void __init pmd_soft_dirty_tests(unsigned long pfn, pgprot_t prot)
> return;
>
> pr_debug("Validating PMD soft dirty\n");
> - pmd = pfn_pmd(pfn, prot);
> + pmd = pfn_pmd(args->fixed_pmd_pfn, args->page_prot);
> WARN_ON(!pmd_soft_dirty(pmd_mksoft_dirty(pmd)));
> WARN_ON(pmd_soft_dirty(pmd_clear_soft_dirty(pmd)));
> }
>
> -static void __init pmd_swap_soft_dirty_tests(unsigned long pfn, pgprot_t prot)
> +static void __init pmd_swap_soft_dirty_tests(struct pgtable_debug_args *args)
> {
> pmd_t pmd;
>
> @@ -805,31 +805,29 @@ static void __init pmd_swap_soft_dirty_tests(unsigned long pfn, pgprot_t prot)
> return;
>
> pr_debug("Validating PMD swap soft dirty\n");
> - pmd = pfn_pmd(pfn, prot);
> + pmd = pfn_pmd(args->fixed_pmd_pfn, args->page_prot);
> WARN_ON(!pmd_swp_soft_dirty(pmd_swp_mksoft_dirty(pmd)));
> WARN_ON(pmd_swp_soft_dirty(pmd_swp_clear_soft_dirty(pmd)));
> }
> #else /* !CONFIG_TRANSPARENT_HUGEPAGE */
> -static void __init pmd_soft_dirty_tests(unsigned long pfn, pgprot_t prot) { }
> -static void __init pmd_swap_soft_dirty_tests(unsigned long pfn, pgprot_t prot)
> -{
> -}
> +static void __init pmd_soft_dirty_tests(struct pgtable_debug_args *args) { }
> +static void __init pmd_swap_soft_dirty_tests(struct pgtable_debug_args *args) { }
> #endif /* CONFIG_TRANSPARENT_HUGEPAGE */
>
> -static void __init pte_swap_tests(unsigned long pfn, pgprot_t prot)
> +static void __init pte_swap_tests(struct pgtable_debug_args *args)
> {
> swp_entry_t swp;
> pte_t pte;
>
> pr_debug("Validating PTE swap\n");
> - pte = pfn_pte(pfn, prot);
> + pte = pfn_pte(args->fixed_pte_pfn, args->page_prot);
> swp = __pte_to_swp_entry(pte);
> pte = __swp_entry_to_pte(swp);
> - WARN_ON(pfn != pte_pfn(pte));
> + WARN_ON(args->fixed_pte_pfn != pte_pfn(pte));
> }
>
> #ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION
> -static void __init pmd_swap_tests(unsigned long pfn, pgprot_t prot)
> +static void __init pmd_swap_tests(struct pgtable_debug_args *args)
> {
> swp_entry_t swp;
> pmd_t pmd;
> @@ -838,13 +836,13 @@ static void __init pmd_swap_tests(unsigned long pfn, pgprot_t prot)
> return;
>
> pr_debug("Validating PMD swap\n");
> - pmd = pfn_pmd(pfn, prot);
> + pmd = pfn_pmd(args->fixed_pmd_pfn, args->page_prot);
> swp = __pmd_to_swp_entry(pmd);
> pmd = __swp_entry_to_pmd(swp);
> - WARN_ON(pfn != pmd_pfn(pmd));
> + WARN_ON(args->fixed_pmd_pfn != pmd_pfn(pmd));
> }
> #else /* !CONFIG_ARCH_ENABLE_THP_MIGRATION */
> -static void __init pmd_swap_tests(unsigned long pfn, pgprot_t prot) { }
> +static void __init pmd_swap_tests(struct pgtable_debug_args *args) { }
> #endif /* CONFIG_ARCH_ENABLE_THP_MIGRATION */
>
> static void __init swap_migration_tests(void)
> @@ -1339,13 +1337,13 @@ static int __init debug_vm_pgtable(void)
> pmd_devmap_tests(&args);
> pud_devmap_tests(&args);
>
> - pte_soft_dirty_tests(pte_aligned, prot);
> - pmd_soft_dirty_tests(pmd_aligned, prot);
> - pte_swap_soft_dirty_tests(pte_aligned, prot);
> - pmd_swap_soft_dirty_tests(pmd_aligned, prot);
> + pte_soft_dirty_tests(&args);
> + pmd_soft_dirty_tests(&args);
> + pte_swap_soft_dirty_tests(&args);
> + pmd_swap_soft_dirty_tests(&args);
>
> - pte_swap_tests(pte_aligned, prot);
> - pmd_swap_tests(pmd_aligned, prot);
> + pte_swap_tests(&args);
> + pmd_swap_tests(&args);
>
> swap_migration_tests();
>
>
Reviewed-by: Anshuman Khandual <[email protected]>
On 7/27/21 11:43 AM, Gavin Shan wrote:
> This uses struct pgtable_debug_args in PTE modifying tests. The allocated
> page is used as set_pte_at() is used there. The tests are skipped if
> the allocated page doesn't exist. Besides, the unused variable @ptep
> and @pte_aligned in debug_vm_pgtable() are dropped.
A small nit. @ptep is not getting dropped here, it's still around and used.
>
> Signed-off-by: Gavin Shan <[email protected]>
> ---
> mm/debug_vm_pgtable.c | 64 +++++++++++++++++++++----------------------
> 1 file changed, 31 insertions(+), 33 deletions(-)
>
> diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
> index 9136195efde3..b5b22e666954 100644
> --- a/mm/debug_vm_pgtable.c
> +++ b/mm/debug_vm_pgtable.c
> @@ -117,10 +117,7 @@ static void __init pte_basic_tests(struct pgtable_debug_args *args, int idx)
> WARN_ON(!pte_dirty(pte_wrprotect(pte_mkdirty(pte))));
> }
>
> -static void __init pte_advanced_tests(struct mm_struct *mm,
> - struct vm_area_struct *vma, pte_t *ptep,
> - unsigned long pfn, unsigned long vaddr,
> - pgprot_t prot)
> +static void __init pte_advanced_tests(struct pgtable_debug_args *args)
> {
> pte_t pte;
>
> @@ -129,35 +126,37 @@ static void __init pte_advanced_tests(struct mm_struct *mm,
> * This requires set_pte_at to be not used to update an
> * existing pte entry. Clear pte before we do set_pte_at
> */
> + if (args->pte_pfn == ULONG_MAX)
> + return;
>
> pr_debug("Validating PTE advanced\n");
> - pte = pfn_pte(pfn, prot);
> - set_pte_at(mm, vaddr, ptep, pte);
> - ptep_set_wrprotect(mm, vaddr, ptep);
> - pte = ptep_get(ptep);
> + pte = pfn_pte(args->pte_pfn, args->page_prot);
> + set_pte_at(args->mm, args->vaddr, args->ptep, pte);
> + ptep_set_wrprotect(args->mm, args->vaddr, args->ptep);
> + pte = ptep_get(args->ptep);
> WARN_ON(pte_write(pte));
> - ptep_get_and_clear(mm, vaddr, ptep);
> - pte = ptep_get(ptep);
> + ptep_get_and_clear(args->mm, args->vaddr, args->ptep);
> + pte = ptep_get(args->ptep);
> WARN_ON(!pte_none(pte));
>
> - pte = pfn_pte(pfn, prot);
> + pte = pfn_pte(args->pte_pfn, args->page_prot);
> pte = pte_wrprotect(pte);
> pte = pte_mkclean(pte);
> - set_pte_at(mm, vaddr, ptep, pte);
> + set_pte_at(args->mm, args->vaddr, args->ptep, pte);
> pte = pte_mkwrite(pte);
> pte = pte_mkdirty(pte);
> - ptep_set_access_flags(vma, vaddr, ptep, pte, 1);
> - pte = ptep_get(ptep);
> + ptep_set_access_flags(args->vma, args->vaddr, args->ptep, pte, 1);
> + pte = ptep_get(args->ptep);
> WARN_ON(!(pte_write(pte) && pte_dirty(pte)));
> - ptep_get_and_clear_full(mm, vaddr, ptep, 1);
> - pte = ptep_get(ptep);
> + ptep_get_and_clear_full(args->mm, args->vaddr, args->ptep, 1);
> + pte = ptep_get(args->ptep);
> WARN_ON(!pte_none(pte));
>
> - pte = pfn_pte(pfn, prot);
> + pte = pfn_pte(args->pte_pfn, args->page_prot);
> pte = pte_mkyoung(pte);
> - set_pte_at(mm, vaddr, ptep, pte);
> - ptep_test_and_clear_young(vma, vaddr, ptep);
> - pte = ptep_get(ptep);
> + set_pte_at(args->mm, args->vaddr, args->ptep, pte);
> + ptep_test_and_clear_young(args->vma, args->vaddr, args->ptep);
> + pte = ptep_get(args->ptep);
> WARN_ON(pte_young(pte));
> }
>
> @@ -618,20 +617,21 @@ static void __init pgd_populate_tests(struct mm_struct *mm, pgd_t *pgdp,
> }
> #endif /* PAGETABLE_P4D_FOLDED */
>
> -static void __init pte_clear_tests(struct mm_struct *mm, pte_t *ptep,
> - unsigned long pfn, unsigned long vaddr,
> - pgprot_t prot)
> +static void __init pte_clear_tests(struct pgtable_debug_args *args)
> {
> - pte_t pte = pfn_pte(pfn, prot);
> + pte_t pte = pfn_pte(args->pte_pfn, args->page_prot);
> +
> + if (args->pte_pfn == ULONG_MAX)
> + return;
>
> pr_debug("Validating PTE clear\n");
> #ifndef CONFIG_RISCV
> pte = __pte(pte_val(pte) | RANDOM_ORVALUE);
> #endif
> - set_pte_at(mm, vaddr, ptep, pte);
> + set_pte_at(args->mm, args->vaddr, args->ptep, pte);
> barrier();
> - pte_clear(mm, vaddr, ptep);
> - pte = ptep_get(ptep);
> + pte_clear(args->mm, args->vaddr, args->ptep);
> + pte = ptep_get(args->ptep);
> WARN_ON(!pte_none(pte));
> }
>
> @@ -1237,7 +1237,7 @@ static int __init debug_vm_pgtable(void)
> pgtable_t saved_ptep;
> pgprot_t prot;
> phys_addr_t paddr;
> - unsigned long vaddr, pte_aligned, pmd_aligned;
> + unsigned long vaddr, pmd_aligned;
> unsigned long pud_aligned;
> spinlock_t *ptl = NULL;
> int idx, ret;
> @@ -1272,10 +1272,8 @@ static int __init debug_vm_pgtable(void)
> */
> paddr = __pa_symbol(&start_kernel);
>
> - pte_aligned = (paddr & PAGE_MASK) >> PAGE_SHIFT;
> pmd_aligned = (paddr & PMD_MASK) >> PAGE_SHIFT;
> pud_aligned = (paddr & PUD_MASK) >> PAGE_SHIFT;
> - WARN_ON(!pfn_valid(pte_aligned));
>
> pgdp = pgd_offset(mm, vaddr);
> p4dp = p4d_alloc(mm, pgdp, vaddr);
> @@ -1357,9 +1355,9 @@ static int __init debug_vm_pgtable(void)
> * proper page table lock.
> */
>
> - ptep = pte_offset_map_lock(mm, pmdp, vaddr, &ptl);
> - pte_clear_tests(mm, ptep, pte_aligned, vaddr, prot);
> - pte_advanced_tests(mm, vma, ptep, pte_aligned, vaddr, prot);
> + ptep = pte_offset_map_lock(args.mm, args.pmdp, args.vaddr, &ptl);
> + pte_clear_tests(&args);
> + pte_advanced_tests(&args);
> pte_unmap_unlock(ptep, ptl);
>
> ptl = pmd_lock(mm, pmdp);
>
Reviewed-by: Anshuman Khandual <[email protected]>
On 7/27/21 11:43 AM, Gavin Shan wrote:
> This uses struct pgtable_debug_args in PMD modifying tests. The allocated
> huge page is used when set_pmd_at() is used. The corresponding tests
> are skipped if the huge page doesn't exist. Besides, the unused variable
> @pmd_aligned in debug_vm_pgtable() is dropped.
>
> Signed-off-by: Gavin Shan <[email protected]>
> ---
> mm/debug_vm_pgtable.c | 98 ++++++++++++++++++++-----------------------
> 1 file changed, 46 insertions(+), 52 deletions(-)
>
> diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
> index b5b22e666954..2a7c810b56b4 100644
> --- a/mm/debug_vm_pgtable.c
> +++ b/mm/debug_vm_pgtable.c
> @@ -211,54 +211,55 @@ static void __init pmd_basic_tests(struct pgtable_debug_args *args, int idx)
> WARN_ON(!pmd_bad(pmd_mkhuge(pmd)));
> }
>
> -static void __init pmd_advanced_tests(struct mm_struct *mm,
> - struct vm_area_struct *vma, pmd_t *pmdp,
> - unsigned long pfn, unsigned long vaddr,
> - pgprot_t prot, pgtable_t pgtable)
> +static void __init pmd_advanced_tests(struct pgtable_debug_args *args)
> {
> pmd_t pmd;
> + unsigned long vaddr = args->vaddr;
>
> if (!has_transparent_hugepage())
> return;
>
> + if (args->pmd_pfn == ULONG_MAX)
> + return;
> +
> pr_debug("Validating PMD advanced\n");
> /* Align the address wrt HPAGE_PMD_SIZE */
> vaddr &= HPAGE_PMD_MASK;
>
> - pgtable_trans_huge_deposit(mm, pmdp, pgtable);
> + pgtable_trans_huge_deposit(args->mm, args->pmdp, args->start_ptep);
>
> - pmd = pfn_pmd(pfn, prot);
> - set_pmd_at(mm, vaddr, pmdp, pmd);
> - pmdp_set_wrprotect(mm, vaddr, pmdp);
> - pmd = READ_ONCE(*pmdp);
> + pmd = pfn_pmd(args->pmd_pfn, args->page_prot);
> + set_pmd_at(args->mm, vaddr, args->pmdp, pmd);
> + pmdp_set_wrprotect(args->mm, vaddr, args->pmdp);
> + pmd = READ_ONCE(*args->pmdp);
> WARN_ON(pmd_write(pmd));
> - pmdp_huge_get_and_clear(mm, vaddr, pmdp);
> - pmd = READ_ONCE(*pmdp);
> + pmdp_huge_get_and_clear(args->mm, vaddr, args->pmdp);
> + pmd = READ_ONCE(*args->pmdp);
> WARN_ON(!pmd_none(pmd));
>
> - pmd = pfn_pmd(pfn, prot);
> + pmd = pfn_pmd(args->pmd_pfn, args->page_prot);
> pmd = pmd_wrprotect(pmd);
> pmd = pmd_mkclean(pmd);
> - set_pmd_at(mm, vaddr, pmdp, pmd);
> + set_pmd_at(args->mm, vaddr, args->pmdp, pmd);
> pmd = pmd_mkwrite(pmd);
> pmd = pmd_mkdirty(pmd);
> - pmdp_set_access_flags(vma, vaddr, pmdp, pmd, 1);
> - pmd = READ_ONCE(*pmdp);
> + pmdp_set_access_flags(args->vma, vaddr, args->pmdp, pmd, 1);
> + pmd = READ_ONCE(*args->pmdp);
> WARN_ON(!(pmd_write(pmd) && pmd_dirty(pmd)));
> - pmdp_huge_get_and_clear_full(vma, vaddr, pmdp, 1);
> - pmd = READ_ONCE(*pmdp);
> + pmdp_huge_get_and_clear_full(args->vma, vaddr, args->pmdp, 1);
> + pmd = READ_ONCE(*args->pmdp);
> WARN_ON(!pmd_none(pmd));
>
> - pmd = pmd_mkhuge(pfn_pmd(pfn, prot));
> + pmd = pmd_mkhuge(pfn_pmd(args->pmd_pfn, args->page_prot));
> pmd = pmd_mkyoung(pmd);
> - set_pmd_at(mm, vaddr, pmdp, pmd);
> - pmdp_test_and_clear_young(vma, vaddr, pmdp);
> - pmd = READ_ONCE(*pmdp);
> + set_pmd_at(args->mm, vaddr, args->pmdp, pmd);
> + pmdp_test_and_clear_young(args->vma, vaddr, args->pmdp);
> + pmd = READ_ONCE(*args->pmdp);
> WARN_ON(pmd_young(pmd));
>
> /* Clear the pte entries */
> - pmdp_huge_get_and_clear(mm, vaddr, pmdp);
> - pgtable = pgtable_trans_huge_withdraw(mm, pmdp);
> + pmdp_huge_get_and_clear(args->mm, vaddr, args->pmdp);
> + pgtable_trans_huge_withdraw(args->mm, args->pmdp);
> }
>
> static void __init pmd_leaf_tests(struct pgtable_debug_args *args)
> @@ -415,12 +416,7 @@ static void __init pud_leaf_tests(struct pgtable_debug_args *args) { }
> #else /* !CONFIG_TRANSPARENT_HUGEPAGE */
> static void __init pmd_basic_tests(struct pgtable_debug_args *args, int idx) { }
> static void __init pud_basic_tests(struct pgtable_debug_args *args, int idx) { }
> -static void __init pmd_advanced_tests(struct mm_struct *mm,
> - struct vm_area_struct *vma, pmd_t *pmdp,
> - unsigned long pfn, unsigned long vaddr,
> - pgprot_t prot, pgtable_t pgtable)
> -{
> -}
> +static void __init pmd_advanced_tests(struct pgtable_debug_args *args) { }
> static void __init pud_advanced_tests(struct mm_struct *mm,
> struct vm_area_struct *vma, pud_t *pudp,
> unsigned long pfn, unsigned long vaddr,
> @@ -433,11 +429,11 @@ static void __init pmd_savedwrite_tests(struct pgtable_debug_args *args) { }
> #endif /* CONFIG_TRANSPARENT_HUGEPAGE */
>
> #ifdef CONFIG_HAVE_ARCH_HUGE_VMAP
> -static void __init pmd_huge_tests(pmd_t *pmdp, unsigned long pfn, pgprot_t prot)
> +static void __init pmd_huge_tests(struct pgtable_debug_args *args)
> {
> pmd_t pmd;
>
> - if (!arch_vmap_pmd_supported(prot))
> + if (!arch_vmap_pmd_supported(args->page_prot))
> return;
>
> pr_debug("Validating PMD huge\n");
> @@ -445,10 +441,10 @@ static void __init pmd_huge_tests(pmd_t *pmdp, unsigned long pfn, pgprot_t prot)
> * X86 defined pmd_set_huge() verifies that the given
> * PMD is not a populated non-leaf entry.
> */
> - WRITE_ONCE(*pmdp, __pmd(0));
> - WARN_ON(!pmd_set_huge(pmdp, __pfn_to_phys(pfn), prot));
> - WARN_ON(!pmd_clear_huge(pmdp));
> - pmd = READ_ONCE(*pmdp);
> + WRITE_ONCE(*args->pmdp, __pmd(0));
> + WARN_ON(!pmd_set_huge(args->pmdp, __pfn_to_phys(args->fixed_pmd_pfn), args->page_prot));
> + WARN_ON(!pmd_clear_huge(args->pmdp));
> + pmd = READ_ONCE(*args->pmdp);
> WARN_ON(!pmd_none(pmd));
> }
>
> @@ -471,7 +467,7 @@ static void __init pud_huge_tests(pud_t *pudp, unsigned long pfn, pgprot_t prot)
> WARN_ON(!pud_none(pud));
> }
> #else /* !CONFIG_HAVE_ARCH_HUGE_VMAP */
> -static void __init pmd_huge_tests(pmd_t *pmdp, unsigned long pfn, pgprot_t prot) { }
> +static void __init pmd_huge_tests(struct pgtable_debug_args *args) { }
> static void __init pud_huge_tests(pud_t *pudp, unsigned long pfn, pgprot_t prot) { }
> #endif /* CONFIG_HAVE_ARCH_HUGE_VMAP */
>
> @@ -635,20 +631,19 @@ static void __init pte_clear_tests(struct pgtable_debug_args *args)
> WARN_ON(!pte_none(pte));
> }
>
> -static void __init pmd_clear_tests(struct mm_struct *mm, pmd_t *pmdp)
> +static void __init pmd_clear_tests(struct pgtable_debug_args *args)
> {
> - pmd_t pmd = READ_ONCE(*pmdp);
> + pmd_t pmd = READ_ONCE(*args->pmdp);
>
> pr_debug("Validating PMD clear\n");
> pmd = __pmd(pmd_val(pmd) | RANDOM_ORVALUE);
> - WRITE_ONCE(*pmdp, pmd);
> - pmd_clear(pmdp);
> - pmd = READ_ONCE(*pmdp);
> + WRITE_ONCE(*args->pmdp, pmd);
> + pmd_clear(args->pmdp);
> + pmd = READ_ONCE(*args->pmdp);
> WARN_ON(!pmd_none(pmd));
> }
>
> -static void __init pmd_populate_tests(struct mm_struct *mm, pmd_t *pmdp,
> - pgtable_t pgtable)
> +static void __init pmd_populate_tests(struct pgtable_debug_args *args)
> {
> pmd_t pmd;
>
> @@ -657,8 +652,8 @@ static void __init pmd_populate_tests(struct mm_struct *mm, pmd_t *pmdp,
> * This entry points to next level page table page.
> * Hence this must not qualify as pmd_bad().
> */
> - pmd_populate(mm, pmdp, pgtable);
> - pmd = READ_ONCE(*pmdp);
> + pmd_populate(args->mm, args->pmdp, args->start_ptep);
> + pmd = READ_ONCE(*args->pmdp);
> WARN_ON(pmd_bad(pmd));
> }
>
> @@ -1237,7 +1232,7 @@ static int __init debug_vm_pgtable(void)
> pgtable_t saved_ptep;
> pgprot_t prot;
> phys_addr_t paddr;
> - unsigned long vaddr, pmd_aligned;
> + unsigned long vaddr;
> unsigned long pud_aligned;
> spinlock_t *ptl = NULL;
> int idx, ret;
> @@ -1272,7 +1267,6 @@ static int __init debug_vm_pgtable(void)
> */
> paddr = __pa_symbol(&start_kernel);
>
> - pmd_aligned = (paddr & PMD_MASK) >> PAGE_SHIFT;
> pud_aligned = (paddr & PUD_MASK) >> PAGE_SHIFT;
>
> pgdp = pgd_offset(mm, vaddr);
> @@ -1360,11 +1354,11 @@ static int __init debug_vm_pgtable(void)
> pte_advanced_tests(&args);
> pte_unmap_unlock(ptep, ptl);
>
> - ptl = pmd_lock(mm, pmdp);
> - pmd_clear_tests(mm, pmdp);
> - pmd_advanced_tests(mm, vma, pmdp, pmd_aligned, vaddr, prot, saved_ptep);
> - pmd_huge_tests(pmdp, pmd_aligned, prot);
> - pmd_populate_tests(mm, pmdp, saved_ptep);
> + ptl = pmd_lock(args.mm, args.pmdp);
> + pmd_clear_tests(&args);
> + pmd_advanced_tests(&args);
> + pmd_huge_tests(&args);
> + pmd_populate_tests(&args);
> spin_unlock(ptl);
>
> ptl = pud_lock(mm, pudp);
>
Reviewed-by: Anshuman Khandual <[email protected]>
On 7/27/21 11:43 AM, Gavin Shan wrote:
> This uses struct pgtable_debug_args in PUD modifying tests. The allocated
> huge page is used when set_pud_at() is used. The corresponding tests
> are skipped if the huge page doesn't exist. Besides, the following unused
> variables in debug_vm_pgtable() are dropped: @prot, @paddr, @pud_aligned.
>
> Signed-off-by: Gavin Shan <[email protected]>
> ---
> mm/debug_vm_pgtable.c | 126 ++++++++++++++++--------------------------
> 1 file changed, 48 insertions(+), 78 deletions(-)
>
> diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
> index 2a7c810b56b4..863871e70268 100644
> --- a/mm/debug_vm_pgtable.c
> +++ b/mm/debug_vm_pgtable.c
> @@ -337,55 +337,56 @@ static void __init pud_basic_tests(struct pgtable_debug_args *args, int idx)
> WARN_ON(!pud_bad(pud_mkhuge(pud)));
> }
>
> -static void __init pud_advanced_tests(struct mm_struct *mm,
> - struct vm_area_struct *vma, pud_t *pudp,
> - unsigned long pfn, unsigned long vaddr,
> - pgprot_t prot)
> +static void __init pud_advanced_tests(struct pgtable_debug_args *args)
> {
> + unsigned long vaddr = args->vaddr;
> pud_t pud;
>
> if (!has_transparent_hugepage())
> return;
>
> + if (args->pud_pfn == ULONG_MAX)
> + return;
> +
> pr_debug("Validating PUD advanced\n");
> /* Align the address wrt HPAGE_PUD_SIZE */
> vaddr &= HPAGE_PUD_MASK;
>
> - pud = pfn_pud(pfn, prot);
> - set_pud_at(mm, vaddr, pudp, pud);
> - pudp_set_wrprotect(mm, vaddr, pudp);
> - pud = READ_ONCE(*pudp);
> + pud = pfn_pud(args->pud_pfn, args->page_prot);
> + set_pud_at(args->mm, vaddr, args->pudp, pud);
> + pudp_set_wrprotect(args->mm, vaddr, args->pudp);
> + pud = READ_ONCE(*args->pudp);
> WARN_ON(pud_write(pud));
>
> #ifndef __PAGETABLE_PMD_FOLDED
> - pudp_huge_get_and_clear(mm, vaddr, pudp);
> - pud = READ_ONCE(*pudp);
> + pudp_huge_get_and_clear(args->mm, vaddr, args->pudp);
> + pud = READ_ONCE(*args->pudp);
> WARN_ON(!pud_none(pud));
> #endif /* __PAGETABLE_PMD_FOLDED */
> - pud = pfn_pud(pfn, prot);
> + pud = pfn_pud(args->pud_pfn, args->page_prot);
> pud = pud_wrprotect(pud);
> pud = pud_mkclean(pud);
> - set_pud_at(mm, vaddr, pudp, pud);
> + set_pud_at(args->mm, vaddr, args->pudp, pud);
> pud = pud_mkwrite(pud);
> pud = pud_mkdirty(pud);
> - pudp_set_access_flags(vma, vaddr, pudp, pud, 1);
> - pud = READ_ONCE(*pudp);
> + pudp_set_access_flags(args->vma, vaddr, args->pudp, pud, 1);
> + pud = READ_ONCE(*args->pudp);
> WARN_ON(!(pud_write(pud) && pud_dirty(pud)));
>
> #ifndef __PAGETABLE_PMD_FOLDED
> - pudp_huge_get_and_clear_full(mm, vaddr, pudp, 1);
> - pud = READ_ONCE(*pudp);
> + pudp_huge_get_and_clear_full(args->mm, vaddr, args->pudp, 1);
> + pud = READ_ONCE(*args->pudp);
> WARN_ON(!pud_none(pud));
> #endif /* __PAGETABLE_PMD_FOLDED */
>
> - pud = pfn_pud(pfn, prot);
> + pud = pfn_pud(args->pud_pfn, args->page_prot);
> pud = pud_mkyoung(pud);
> - set_pud_at(mm, vaddr, pudp, pud);
> - pudp_test_and_clear_young(vma, vaddr, pudp);
> - pud = READ_ONCE(*pudp);
> + set_pud_at(args->mm, vaddr, args->pudp, pud);
> + pudp_test_and_clear_young(args->vma, vaddr, args->pudp);
> + pud = READ_ONCE(*args->pudp);
> WARN_ON(pud_young(pud));
>
> - pudp_huge_get_and_clear(mm, vaddr, pudp);
> + pudp_huge_get_and_clear(args->mm, vaddr, args->pudp);
> }
>
> static void __init pud_leaf_tests(struct pgtable_debug_args *args)
> @@ -405,24 +406,14 @@ static void __init pud_leaf_tests(struct pgtable_debug_args *args)
> }
> #else /* !CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */
> static void __init pud_basic_tests(struct pgtable_debug_args *args, int idx) { }
> -static void __init pud_advanced_tests(struct mm_struct *mm,
> - struct vm_area_struct *vma, pud_t *pudp,
> - unsigned long pfn, unsigned long vaddr,
> - pgprot_t prot)
> -{
> -}
> +static void __init pud_advanced_tests(struct pgtable_debug_args *args) { }
> static void __init pud_leaf_tests(struct pgtable_debug_args *args) { }
> #endif /* CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */
> #else /* !CONFIG_TRANSPARENT_HUGEPAGE */
> static void __init pmd_basic_tests(struct pgtable_debug_args *args, int idx) { }
> static void __init pud_basic_tests(struct pgtable_debug_args *args, int idx) { }
> static void __init pmd_advanced_tests(struct pgtable_debug_args *args) { }
> -static void __init pud_advanced_tests(struct mm_struct *mm,
> - struct vm_area_struct *vma, pud_t *pudp,
> - unsigned long pfn, unsigned long vaddr,
> - pgprot_t prot)
> -{
> -}
> +static void __init pud_advanced_tests(struct pgtable_debug_args *args) { }
> static void __init pmd_leaf_tests(struct pgtable_debug_args *args) { }
> static void __init pud_leaf_tests(struct pgtable_debug_args *args) { }
> static void __init pmd_savedwrite_tests(struct pgtable_debug_args *args) { }
> @@ -448,11 +439,11 @@ static void __init pmd_huge_tests(struct pgtable_debug_args *args)
> WARN_ON(!pmd_none(pmd));
> }
>
> -static void __init pud_huge_tests(pud_t *pudp, unsigned long pfn, pgprot_t prot)
> +static void __init pud_huge_tests(struct pgtable_debug_args *args)
> {
> pud_t pud;
>
> - if (!arch_vmap_pud_supported(prot))
> + if (!arch_vmap_pud_supported(args->page_prot))
> return;
>
> pr_debug("Validating PUD huge\n");
> @@ -460,15 +451,15 @@ static void __init pud_huge_tests(pud_t *pudp, unsigned long pfn, pgprot_t prot)
> * X86 defined pud_set_huge() verifies that the given
> * PUD is not a populated non-leaf entry.
> */
> - WRITE_ONCE(*pudp, __pud(0));
> - WARN_ON(!pud_set_huge(pudp, __pfn_to_phys(pfn), prot));
> - WARN_ON(!pud_clear_huge(pudp));
> - pud = READ_ONCE(*pudp);
> + WRITE_ONCE(*args->pudp, __pud(0));
> + WARN_ON(!pud_set_huge(args->pudp, __pfn_to_phys(args->fixed_pud_pfn), args->page_prot));
> + WARN_ON(!pud_clear_huge(args->pudp));
> + pud = READ_ONCE(*args->pudp);
> WARN_ON(!pud_none(pud));
> }
> #else /* !CONFIG_HAVE_ARCH_HUGE_VMAP */
> static void __init pmd_huge_tests(struct pgtable_debug_args *args) { }
> -static void __init pud_huge_tests(pud_t *pudp, unsigned long pfn, pgprot_t prot) { }
> +static void __init pud_huge_tests(struct pgtable_debug_args *args) { }
> #endif /* CONFIG_HAVE_ARCH_HUGE_VMAP */
>
> static void __init p4d_basic_tests(struct pgtable_debug_args *args)
> @@ -490,27 +481,26 @@ static void __init pgd_basic_tests(struct pgtable_debug_args *args)
> }
>
> #ifndef __PAGETABLE_PUD_FOLDED
> -static void __init pud_clear_tests(struct mm_struct *mm, pud_t *pudp)
> +static void __init pud_clear_tests(struct pgtable_debug_args *args)
> {
> - pud_t pud = READ_ONCE(*pudp);
> + pud_t pud = READ_ONCE(*args->pudp);
>
> - if (mm_pmd_folded(mm))
> + if (mm_pmd_folded(args->mm))
> return;
>
> pr_debug("Validating PUD clear\n");
> pud = __pud(pud_val(pud) | RANDOM_ORVALUE);
> - WRITE_ONCE(*pudp, pud);
> - pud_clear(pudp);
> - pud = READ_ONCE(*pudp);
> + WRITE_ONCE(*args->pudp, pud);
> + pud_clear(args->pudp);
> + pud = READ_ONCE(*args->pudp);
> WARN_ON(!pud_none(pud));
> }
>
> -static void __init pud_populate_tests(struct mm_struct *mm, pud_t *pudp,
> - pmd_t *pmdp)
> +static void __init pud_populate_tests(struct pgtable_debug_args *args)
> {
> pud_t pud;
>
> - if (mm_pmd_folded(mm))
> + if (mm_pmd_folded(args->mm))
> return;
>
> pr_debug("Validating PUD populate\n");
> @@ -518,16 +508,13 @@ static void __init pud_populate_tests(struct mm_struct *mm, pud_t *pudp,
> * This entry points to next level page table page.
> * Hence this must not qualify as pud_bad().
> */
> - pud_populate(mm, pudp, pmdp);
> - pud = READ_ONCE(*pudp);
> + pud_populate(args->mm, args->pudp, args->start_pmdp);
> + pud = READ_ONCE(*args->pudp);
> WARN_ON(pud_bad(pud));
> }
> #else /* !__PAGETABLE_PUD_FOLDED */
> -static void __init pud_clear_tests(struct mm_struct *mm, pud_t *pudp) { }
> -static void __init pud_populate_tests(struct mm_struct *mm, pud_t *pudp,
> - pmd_t *pmdp)
> -{
> -}
> +static void __init pud_clear_tests(struct pgtable_debug_args *args) { }
> +static void __init pud_populate_tests(struct pgtable_debug_args *args) { }
> #endif /* PAGETABLE_PUD_FOLDED */
>
> #ifndef __PAGETABLE_P4D_FOLDED
> @@ -1230,10 +1217,7 @@ static int __init debug_vm_pgtable(void)
> pmd_t *pmdp, *saved_pmdp, pmd;
> pte_t *ptep;
> pgtable_t saved_ptep;
> - pgprot_t prot;
> - phys_addr_t paddr;
> unsigned long vaddr;
> - unsigned long pud_aligned;
> spinlock_t *ptl = NULL;
> int idx, ret;
>
> @@ -1242,7 +1226,6 @@ static int __init debug_vm_pgtable(void)
> if (ret)
> return ret;
>
> - prot = vm_get_page_prot(VMFLAGS);
> vaddr = get_random_vaddr();
> mm = mm_alloc();
> if (!mm) {
> @@ -1256,19 +1239,6 @@ static int __init debug_vm_pgtable(void)
> return 1;
> }
>
> - /*
> - * PFN for mapping at PTE level is determined from a standard kernel
> - * text symbol. But pfns for higher page table levels are derived by
> - * masking lower bits of this real pfn. These derived pfns might not
> - * exist on the platform but that does not really matter as pfn_pxx()
> - * helpers will still create appropriate entries for the test. This
> - * helps avoid large memory block allocations to be used for mapping
> - * at higher page table levels.
> - */
> - paddr = __pa_symbol(&start_kernel);
> -
> - pud_aligned = (paddr & PUD_MASK) >> PAGE_SHIFT;
> -
> pgdp = pgd_offset(mm, vaddr);
> p4dp = p4d_alloc(mm, pgdp, vaddr);
> pudp = pud_alloc(mm, p4dp, vaddr);
> @@ -1361,11 +1331,11 @@ static int __init debug_vm_pgtable(void)
> pmd_populate_tests(&args);
> spin_unlock(ptl);
>
> - ptl = pud_lock(mm, pudp);
> - pud_clear_tests(mm, pudp);
> - pud_advanced_tests(mm, vma, pudp, pud_aligned, vaddr, prot);
> - pud_huge_tests(pudp, pud_aligned, prot);
> - pud_populate_tests(mm, pudp, saved_pmdp);
> + ptl = pud_lock(args.mm, args.pudp);
> + pud_clear_tests(&args);
> + pud_advanced_tests(&args);
> + pud_huge_tests(&args);
> + pud_populate_tests(&args);
> spin_unlock(ptl);
>
> spin_lock(&mm->page_table_lock);
>
Reviewed-by: Anshuman Khandual <[email protected]>
On 7/27/21 11:43 AM, Gavin Shan wrote:
> This uses struct pgtable_debug_args in PGD/P4D modifying tests. No
> allocated huge page is used in these tests. Besides, the unused
> variable @saved_p4dp and @saved_pudp are dropped.
>
> Signed-off-by: Gavin Shan <[email protected]>
> ---
> mm/debug_vm_pgtable.c | 86 +++++++++++++++++++------------------------
> 1 file changed, 38 insertions(+), 48 deletions(-)
>
> diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
> index 863871e70268..0611b5f31e89 100644
> --- a/mm/debug_vm_pgtable.c
> +++ b/mm/debug_vm_pgtable.c
> @@ -518,27 +518,26 @@ static void __init pud_populate_tests(struct pgtable_debug_args *args) { }
> #endif /* PAGETABLE_PUD_FOLDED */
>
> #ifndef __PAGETABLE_P4D_FOLDED
> -static void __init p4d_clear_tests(struct mm_struct *mm, p4d_t *p4dp)
> +static void __init p4d_clear_tests(struct pgtable_debug_args *args)
> {
> - p4d_t p4d = READ_ONCE(*p4dp);
> + p4d_t p4d = READ_ONCE(*args->p4dp);
>
> - if (mm_pud_folded(mm))
> + if (mm_pud_folded(args->mm))
> return;
>
> pr_debug("Validating P4D clear\n");
> p4d = __p4d(p4d_val(p4d) | RANDOM_ORVALUE);
> - WRITE_ONCE(*p4dp, p4d);
> - p4d_clear(p4dp);
> - p4d = READ_ONCE(*p4dp);
> + WRITE_ONCE(*args->p4dp, p4d);
> + p4d_clear(args->p4dp);
> + p4d = READ_ONCE(*args->p4dp);
> WARN_ON(!p4d_none(p4d));
> }
>
> -static void __init p4d_populate_tests(struct mm_struct *mm, p4d_t *p4dp,
> - pud_t *pudp)
> +static void __init p4d_populate_tests(struct pgtable_debug_args *args)
> {
> p4d_t p4d;
>
> - if (mm_pud_folded(mm))
> + if (mm_pud_folded(args->mm))
> return;
>
> pr_debug("Validating P4D populate\n");
> @@ -546,34 +545,33 @@ static void __init p4d_populate_tests(struct mm_struct *mm, p4d_t *p4dp,
> * This entry points to next level page table page.
> * Hence this must not qualify as p4d_bad().
> */
> - pud_clear(pudp);
> - p4d_clear(p4dp);
> - p4d_populate(mm, p4dp, pudp);
> - p4d = READ_ONCE(*p4dp);
> + pud_clear(args->pudp);
> + p4d_clear(args->p4dp);
> + p4d_populate(args->mm, args->p4dp, args->start_pudp);
> + p4d = READ_ONCE(*args->p4dp);
> WARN_ON(p4d_bad(p4d));
> }
>
> -static void __init pgd_clear_tests(struct mm_struct *mm, pgd_t *pgdp)
> +static void __init pgd_clear_tests(struct pgtable_debug_args *args)
> {
> - pgd_t pgd = READ_ONCE(*pgdp);
> + pgd_t pgd = READ_ONCE(*(args->pgdp));
>
> - if (mm_p4d_folded(mm))
> + if (mm_p4d_folded(args->mm))
> return;
>
> pr_debug("Validating PGD clear\n");
> pgd = __pgd(pgd_val(pgd) | RANDOM_ORVALUE);
> - WRITE_ONCE(*pgdp, pgd);
> - pgd_clear(pgdp);
> - pgd = READ_ONCE(*pgdp);
> + WRITE_ONCE(*args->pgdp, pgd);
> + pgd_clear(args->pgdp);
> + pgd = READ_ONCE(*args->pgdp);
> WARN_ON(!pgd_none(pgd));
> }
>
> -static void __init pgd_populate_tests(struct mm_struct *mm, pgd_t *pgdp,
> - p4d_t *p4dp)
> +static void __init pgd_populate_tests(struct pgtable_debug_args *args)
> {
> pgd_t pgd;
>
> - if (mm_p4d_folded(mm))
> + if (mm_p4d_folded(args->mm))
> return;
>
> pr_debug("Validating PGD populate\n");
> @@ -581,23 +579,17 @@ static void __init pgd_populate_tests(struct mm_struct *mm, pgd_t *pgdp,
> * This entry points to next level page table page.
> * Hence this must not qualify as pgd_bad().
> */
> - p4d_clear(p4dp);
> - pgd_clear(pgdp);
> - pgd_populate(mm, pgdp, p4dp);
> - pgd = READ_ONCE(*pgdp);
> + p4d_clear(args->p4dp);
> + pgd_clear(args->pgdp);
> + pgd_populate(args->mm, args->pgdp, args->start_p4dp);
> + pgd = READ_ONCE(*args->pgdp);
> WARN_ON(pgd_bad(pgd));
> }
> #else /* !__PAGETABLE_P4D_FOLDED */
> -static void __init p4d_clear_tests(struct mm_struct *mm, p4d_t *p4dp) { }
> -static void __init pgd_clear_tests(struct mm_struct *mm, pgd_t *pgdp) { }
> -static void __init p4d_populate_tests(struct mm_struct *mm, p4d_t *p4dp,
> - pud_t *pudp)
> -{
> -}
> -static void __init pgd_populate_tests(struct mm_struct *mm, pgd_t *pgdp,
> - p4d_t *p4dp)
> -{
> -}
> +static void __init p4d_clear_tests(struct pgtable_debug_args *args) { }
> +static void __init pgd_clear_tests(struct pgtable_debug_args *args) { }
> +static void __init p4d_populate_tests(struct pgtable_debug_args *args) { }
> +static void __init pgd_populate_tests(struct pgtable_debug_args *args) { }
> #endif /* PAGETABLE_P4D_FOLDED */
>
> static void __init pte_clear_tests(struct pgtable_debug_args *args)
> @@ -1212,8 +1204,8 @@ static int __init debug_vm_pgtable(void)
> struct vm_area_struct *vma;
> struct mm_struct *mm;
> pgd_t *pgdp;
> - p4d_t *p4dp, *saved_p4dp;
> - pud_t *pudp, *saved_pudp;
> + p4d_t *p4dp;
> + pud_t *pudp;
> pmd_t *pmdp, *saved_pmdp, pmd;
> pte_t *ptep;
> pgtable_t saved_ptep;
> @@ -1258,8 +1250,6 @@ static int __init debug_vm_pgtable(void)
> * page table pages.
> */
> pmd = READ_ONCE(*pmdp);
> - saved_p4dp = p4d_offset(pgdp, 0UL);
> - saved_pudp = pud_offset(p4dp, 0UL);
> saved_pmdp = pmd_offset(pudp, 0UL);
> saved_ptep = pmd_pgtable(pmd);
>
> @@ -1338,15 +1328,15 @@ static int __init debug_vm_pgtable(void)
> pud_populate_tests(&args);
> spin_unlock(ptl);
>
> - spin_lock(&mm->page_table_lock);
> - p4d_clear_tests(mm, p4dp);
> - pgd_clear_tests(mm, pgdp);
> - p4d_populate_tests(mm, p4dp, saved_pudp);
> - pgd_populate_tests(mm, pgdp, saved_p4dp);
> - spin_unlock(&mm->page_table_lock);
> + spin_lock(&(args.mm->page_table_lock));
> + p4d_clear_tests(&args);
> + pgd_clear_tests(&args);
> + p4d_populate_tests(&args);
> + pgd_populate_tests(&args);
> + spin_unlock(&(args.mm->page_table_lock));
>
> - p4d_free(mm, saved_p4dp);
> - pud_free(mm, saved_pudp);
> + p4d_free(mm, p4d_offset(pgdp, 0UL));
> + pud_free(mm, pud_offset(p4dp, 0UL));
> pmd_free(mm, saved_pmdp);
> pte_free(mm, saved_ptep);
>
>
Reviewed-by: Anshuman Khandual <[email protected]>
On 7/27/21 11:44 AM, Gavin Shan wrote:
> The variables used by old implementation isn't needed as we switched
> to "struct pgtable_debug_args". Lets remove them and related code in
> debug_vm_pgtable().
>
> Signed-off-by: Gavin Shan <[email protected]>
> ---
> mm/debug_vm_pgtable.c | 54 -------------------------------------------
> 1 file changed, 54 deletions(-)
>
> diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
> index 0611b5f31e89..162ff6329f7b 100644
> --- a/mm/debug_vm_pgtable.c
> +++ b/mm/debug_vm_pgtable.c
> @@ -1201,15 +1201,7 @@ static int __init init_args(struct pgtable_debug_args *args)
> static int __init debug_vm_pgtable(void)
> {
> struct pgtable_debug_args args;
> - struct vm_area_struct *vma;
> - struct mm_struct *mm;
> - pgd_t *pgdp;
> - p4d_t *p4dp;
> - pud_t *pudp;
> - pmd_t *pmdp, *saved_pmdp, pmd;
> pte_t *ptep;
> - pgtable_t saved_ptep;
> - unsigned long vaddr;
> spinlock_t *ptl = NULL;
> int idx, ret;
>
> @@ -1218,41 +1210,6 @@ static int __init debug_vm_pgtable(void)
> if (ret)
> return ret;
>
> - vaddr = get_random_vaddr();
> - mm = mm_alloc();
> - if (!mm) {
> - pr_err("mm_struct allocation failed\n");
> - return 1;
> - }
> -
> - vma = vm_area_alloc(mm);
> - if (!vma) {
> - pr_err("vma allocation failed\n");
> - return 1;
> - }
> -
> - pgdp = pgd_offset(mm, vaddr);
> - p4dp = p4d_alloc(mm, pgdp, vaddr);
> - pudp = pud_alloc(mm, p4dp, vaddr);
> - pmdp = pmd_alloc(mm, pudp, vaddr);
> - /*
> - * Allocate pgtable_t
> - */
> - if (pte_alloc(mm, pmdp)) {
> - pr_err("pgtable allocation failed\n");
> - return 1;
> - }
> -
> - /*
> - * Save all the page table page addresses as the page table
> - * entries will be used for testing with random or garbage
> - * values. These saved addresses will be used for freeing
> - * page table pages.
> - */
> - pmd = READ_ONCE(*pmdp);
> - saved_pmdp = pmd_offset(pudp, 0UL);
> - saved_ptep = pmd_pgtable(pmd);
> -
> /*
> * Iterate over the protection_map[] to make sure that all
> * the basic page table transformation validations just hold
> @@ -1335,17 +1292,6 @@ static int __init debug_vm_pgtable(void)
> pgd_populate_tests(&args);
> spin_unlock(&(args.mm->page_table_lock));
>
> - p4d_free(mm, p4d_offset(pgdp, 0UL));
> - pud_free(mm, pud_offset(p4dp, 0UL));
Small nit. saved_p4dp and saved_pudp could have been dropped in this
patch rather than the previous one. Previous build error with W=1
might not have been a problem as there was a remaining usage after
the assignment.
> - pmd_free(mm, saved_pmdp);
> - pte_free(mm, saved_ptep);
> -
> - vm_area_free(vma);
> - mm_dec_nr_puds(mm);
> - mm_dec_nr_pmds(mm);
> - mm_dec_nr_ptes(mm);
> - mmdrop(mm);
> -
> destroy_args(&args);
> return 0;
> }
>
With or without the above change.
Reviewed-by: Anshuman Khandual <[email protected]>
On 7/27/21 11:44 AM, Gavin Shan wrote:
> In page table entry modifying tests, set_xxx_at() are used to populate
> the page table entries. On ARM64, PG_arch_1 (PG_dcache_clean) flag is
> set to the target page flag if execution permission is given. The logic
> exits since commit 4f04d8f00545 ("arm64: MMU definitions"). The page
> flag is kept when the page is free'd to buddy's free area list. However,
> it will trigger page checking failure when it's pulled from the buddy's
> free area list, as the following warning messages indicate.
>
> BUG: Bad page state in process memhog pfn:08000
> page:0000000015c0a628 refcount:0 mapcount:0 \
> mapping:0000000000000000 index:0x1 pfn:0x8000
> flags: 0x7ffff8000000800(arch_1|node=0|zone=0|lastcpupid=0xfffff)
> raw: 07ffff8000000800 dead000000000100 dead000000000122 0000000000000000
> raw: 0000000000000001 0000000000000000 00000000ffffffff 0000000000000000
> page dumped because: PAGE_FLAGS_CHECK_AT_PREP flag(s) set
>
> This fixes the issue by clearing PG_arch_1 through flush_dcache_page()
> after set_xxx_at() is called. For architectures other than ARM64, the
> unexpected overhead of cache flushing is acceptable.
>
> Signed-off-by: Gavin Shan <[email protected]>
> ---
> mm/debug_vm_pgtable.c | 55 +++++++++++++++++++++++++++++++++++++++----
> 1 file changed, 51 insertions(+), 4 deletions(-)
>
> diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
> index 162ff6329f7b..d2c2d23e542e 100644
> --- a/mm/debug_vm_pgtable.c
> +++ b/mm/debug_vm_pgtable.c
> @@ -29,6 +29,8 @@
> #include <linux/start_kernel.h>
> #include <linux/sched/mm.h>
> #include <linux/io.h>
> +
> +#include <asm/cacheflush.h>
> #include <asm/pgalloc.h>
> #include <asm/tlbflush.h>
>
> @@ -119,19 +121,28 @@ static void __init pte_basic_tests(struct pgtable_debug_args *args, int idx)
>
> static void __init pte_advanced_tests(struct pgtable_debug_args *args)
> {
> + struct page *page;
> pte_t pte;
>
> /*
> * Architectures optimize set_pte_at by avoiding TLB flush.
> * This requires set_pte_at to be not used to update an
> * existing pte entry. Clear pte before we do set_pte_at
> + *
> + * flush_dcache_page() is called after set_pte_at() to clear
> + * PG_arch_1 for the page on ARM64. The page flag isn't cleared
> + * when it's released and page allocation check will fail when
> + * the page is allocated again. For architectures other than ARM64,
> + * the unexpected overhead of cache flushing is acceptable.
> */
> - if (args->pte_pfn == ULONG_MAX)
> + page = (args->pte_pfn != ULONG_MAX) ? pfn_to_page(args->pte_pfn) : NULL;
> + if (!page)
> return;
>
> pr_debug("Validating PTE advanced\n");
> pte = pfn_pte(args->pte_pfn, args->page_prot);
> set_pte_at(args->mm, args->vaddr, args->ptep, pte);
> + flush_dcache_page(page);
> ptep_set_wrprotect(args->mm, args->vaddr, args->ptep);
> pte = ptep_get(args->ptep);
> WARN_ON(pte_write(pte));
> @@ -143,6 +154,7 @@ static void __init pte_advanced_tests(struct pgtable_debug_args *args)
> pte = pte_wrprotect(pte);
> pte = pte_mkclean(pte);
> set_pte_at(args->mm, args->vaddr, args->ptep, pte);
> + flush_dcache_page(page);
> pte = pte_mkwrite(pte);
> pte = pte_mkdirty(pte);
> ptep_set_access_flags(args->vma, args->vaddr, args->ptep, pte, 1);
> @@ -155,6 +167,7 @@ static void __init pte_advanced_tests(struct pgtable_debug_args *args)
> pte = pfn_pte(args->pte_pfn, args->page_prot);
> pte = pte_mkyoung(pte);
> set_pte_at(args->mm, args->vaddr, args->ptep, pte);
> + flush_dcache_page(page);
> ptep_test_and_clear_young(args->vma, args->vaddr, args->ptep);
> pte = ptep_get(args->ptep);
> WARN_ON(pte_young(pte));
> @@ -213,15 +226,24 @@ static void __init pmd_basic_tests(struct pgtable_debug_args *args, int idx)
>
> static void __init pmd_advanced_tests(struct pgtable_debug_args *args)
> {
> + struct page *page;
> pmd_t pmd;
> unsigned long vaddr = args->vaddr;
>
> if (!has_transparent_hugepage())
> return;
>
> - if (args->pmd_pfn == ULONG_MAX)
> + page = (args->pmd_pfn != ULONG_MAX) ? pfn_to_page(args->pmd_pfn) : NULL;
> + if (!page)
> return;
>
> + /*
> + * flush_dcache_page() is called after set_pmd_at() to clear
> + * PG_arch_1 for the page on ARM64. The page flag isn't cleared
> + * when it's released and page allocation check will fail when
> + * the page is allocated again. For architectures other than ARM64,
> + * the unexpected overhead of cache flushing is acceptable.
> + */
> pr_debug("Validating PMD advanced\n");
> /* Align the address wrt HPAGE_PMD_SIZE */
> vaddr &= HPAGE_PMD_MASK;
> @@ -230,6 +252,7 @@ static void __init pmd_advanced_tests(struct pgtable_debug_args *args)
>
> pmd = pfn_pmd(args->pmd_pfn, args->page_prot);
> set_pmd_at(args->mm, vaddr, args->pmdp, pmd);
> + flush_dcache_page(page);
> pmdp_set_wrprotect(args->mm, vaddr, args->pmdp);
> pmd = READ_ONCE(*args->pmdp);
> WARN_ON(pmd_write(pmd));
> @@ -241,6 +264,7 @@ static void __init pmd_advanced_tests(struct pgtable_debug_args *args)
> pmd = pmd_wrprotect(pmd);
> pmd = pmd_mkclean(pmd);
> set_pmd_at(args->mm, vaddr, args->pmdp, pmd);
> + flush_dcache_page(page);
> pmd = pmd_mkwrite(pmd);
> pmd = pmd_mkdirty(pmd);
> pmdp_set_access_flags(args->vma, vaddr, args->pmdp, pmd, 1);
> @@ -253,6 +277,7 @@ static void __init pmd_advanced_tests(struct pgtable_debug_args *args)
> pmd = pmd_mkhuge(pfn_pmd(args->pmd_pfn, args->page_prot));
> pmd = pmd_mkyoung(pmd);
> set_pmd_at(args->mm, vaddr, args->pmdp, pmd);
> + flush_dcache_page(page);
> pmdp_test_and_clear_young(args->vma, vaddr, args->pmdp);
> pmd = READ_ONCE(*args->pmdp);
> WARN_ON(pmd_young(pmd));
> @@ -339,21 +364,31 @@ static void __init pud_basic_tests(struct pgtable_debug_args *args, int idx)
>
> static void __init pud_advanced_tests(struct pgtable_debug_args *args)
> {
> + struct page *page;
> unsigned long vaddr = args->vaddr;
> pud_t pud;
>
> if (!has_transparent_hugepage())
> return;
>
> - if (args->pud_pfn == ULONG_MAX)
> + page = (args->pud_pfn != ULONG_MAX) ? pfn_to_page(args->pud_pfn) : NULL;
> + if (!page)
> return;
>
> + /*
> + * flush_dcache_page() is called after set_pud_at() to clear
> + * PG_arch_1 for the page on ARM64. The page flag isn't cleared
> + * when it's released and page allocation check will fail when
> + * the page is allocated again. For architectures other than ARM64,
> + * the unexpected overhead of cache flushing is acceptable.
> + */
> pr_debug("Validating PUD advanced\n");
> /* Align the address wrt HPAGE_PUD_SIZE */
> vaddr &= HPAGE_PUD_MASK;
>
> pud = pfn_pud(args->pud_pfn, args->page_prot);
> set_pud_at(args->mm, vaddr, args->pudp, pud);
> + flush_dcache_page(page);
> pudp_set_wrprotect(args->mm, vaddr, args->pudp);
> pud = READ_ONCE(*args->pudp);
> WARN_ON(pud_write(pud));
> @@ -367,6 +402,7 @@ static void __init pud_advanced_tests(struct pgtable_debug_args *args)
> pud = pud_wrprotect(pud);
> pud = pud_mkclean(pud);
> set_pud_at(args->mm, vaddr, args->pudp, pud);
> + flush_dcache_page(page);
> pud = pud_mkwrite(pud);
> pud = pud_mkdirty(pud);
> pudp_set_access_flags(args->vma, vaddr, args->pudp, pud, 1);
> @@ -382,6 +418,7 @@ static void __init pud_advanced_tests(struct pgtable_debug_args *args)
> pud = pfn_pud(args->pud_pfn, args->page_prot);
> pud = pud_mkyoung(pud);
> set_pud_at(args->mm, vaddr, args->pudp, pud);
> + flush_dcache_page(page);
> pudp_test_and_clear_young(args->vma, vaddr, args->pudp);
> pud = READ_ONCE(*args->pudp);
> WARN_ON(pud_young(pud));
> @@ -594,16 +631,26 @@ static void __init pgd_populate_tests(struct pgtable_debug_args *args) { }
>
> static void __init pte_clear_tests(struct pgtable_debug_args *args)
> {
> + struct page *page;
> pte_t pte = pfn_pte(args->pte_pfn, args->page_prot);
>
> - if (args->pte_pfn == ULONG_MAX)
> + page = (args->pte_pfn != ULONG_MAX) ? pfn_to_page(args->pte_pfn) : NULL;
> + if (!page)
> return;
>
> + /*
> + * flush_dcache_page() is called after set_pte_at() to clear
> + * PG_arch_1 for the page on ARM64. The page flag isn't cleared
> + * when it's released and page allocation check will fail when
> + * the page is allocated again. For architectures other than ARM64,
> + * the unexpected overhead of cache flushing is acceptable.
> + */
> pr_debug("Validating PTE clear\n");
> #ifndef CONFIG_RISCV
> pte = __pte(pte_val(pte) | RANDOM_ORVALUE);
> #endif
> set_pte_at(args->mm, args->vaddr, args->ptep, pte);
> + flush_dcache_page(page);
> barrier();
> pte_clear(args->mm, args->vaddr, args->ptep);
> pte = ptep_get(args->ptep);
>
Reviewed-by: Anshuman Khandual <[email protected]>
On 7/27/21 11:43 AM, Gavin Shan wrote:
> There are couple of issues with current implementations and this series
> tries to resolve the issues:
>
> (a) All needed information are scattered in variables, passed to various
> test functions. The code is organized in pretty much relaxed fashion.
>
> (b) The page isn't allocated from buddy during page table entry modifying
> tests. The page can be invalid, conflicting to the implementations
> of set_xxx_at() on ARM64. The target page is accessed so that the iCache
> can be flushed when execution permission is given on ARM64. Besides,
> the target page can be unmapped and access to it causes kernel crash.
>
> "struct pgtable_debug_args" is introduced to address issue (a). For issue
> (b), the used page is allocated from buddy in page table entry modifying
> tests. The corresponding tets will be skipped if we fail to allocate the
> (huge) page. For other test cases, the original page around to kernel
> symbol (@start_kernel) is still used.
>
> The patches are organized as below. PATCH[2-10] could be combined to one
> patch, but it will make the review harder:
>
> PATCH[1] introduces "struct pgtable_debug_args" as place holder of all
> needed information. With it, the old and new implementation
> can coexist.
> PATCH[2-10] uses "struct pgtable_debug_args" in various test functions.
> PATCH[11] removes the unused code for old implementation.
> PATCH[12] fixes the issue of corrupted page flag for ARM64
>
> Changelog
> =========
> v4:
> * Determine the page allocation method according to
> MAX_ORDER (Anshuman)
> * Move existing comments to init_args() (Anshuman)
> * Code refactoring as suggested by Anshuman (Anshuman)
> * Improved commit log and add comments for flush_dcache_page()
> in PATCH[v4 12/12] (Anshuman)
> * Address misc comments (Anshuman
This series builds ok on all supported platforms but should also be tested
on them as well i.e s390, ppc, arc and riscv. I was able to do some tests
on arm64 and x86 platforms.
Hi Anshuman,
On 7/29/21 2:45 PM, Anshuman Khandual wrote:
> On 7/27/21 11:43 AM, Gavin Shan wrote:
>> In debug_vm_pgtable(), there are many local variables introduced to
>> track the needed information and they are passed to the functions for
>> various test cases. It'd better to introduce a struct as place holder
>> for these information. With it, what the tests functions need is the
>> struct. In this way, the code is simplified and easier to be maintained.
>>
>> Besides, set_xxx_at() could access the data on the corresponding pages
>> in the page table modifying tests. So the accessed pages in the tests
>> should have been allocated from buddy. Otherwise, we're accessing pages
>> that aren't owned by us. This causes issues like page flag corruption
>> or kernel crash on accessing unmapped page when CONFIG_DEBUG_PAGEALLOC
>> is enabled.
>>
>> This introduces "struct pgtable_debug_args". The struct is initialized
>> and destroyed, but the information in the struct isn't used yet. It will
>> be used in subsequent patches.
>>
>> Signed-off-by: Gavin Shan <[email protected]>
>> ---
>> mm/debug_vm_pgtable.c | 280 +++++++++++++++++++++++++++++++++++++++++-
>> 1 file changed, 279 insertions(+), 1 deletion(-)
>>
>> diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
>> index 1c922691aa61..8c7361643166 100644
>> --- a/mm/debug_vm_pgtable.c
>> +++ b/mm/debug_vm_pgtable.c
>> @@ -58,6 +58,37 @@
>> #define RANDOM_ORVALUE (GENMASK(BITS_PER_LONG - 1, 0) & ~ARCH_SKIP_MASK)
>> #define RANDOM_NZVALUE GENMASK(7, 0)
>>
>> +struct pgtable_debug_args {
>> + struct mm_struct *mm;
>> + struct vm_area_struct *vma;
>> +
>> + pgd_t *pgdp;
>> + p4d_t *p4dp;
>> + pud_t *pudp;
>> + pmd_t *pmdp;
>> + pte_t *ptep;
>> +
>> + p4d_t *start_p4dp;
>> + pud_t *start_pudp;
>> + pmd_t *start_pmdp;
>> + pgtable_t start_ptep;
>> +
>> + unsigned long vaddr;
>> + pgprot_t page_prot;
>> + pgprot_t page_prot_none;
>> +
>> + bool is_contiguous_page;
>> + unsigned long pud_pfn;
>> + unsigned long pmd_pfn;
>> + unsigned long pte_pfn;
>> +
>> + unsigned long fixed_pgd_pfn;
>> + unsigned long fixed_p4d_pfn;
>> + unsigned long fixed_pud_pfn;
>> + unsigned long fixed_pmd_pfn;
>> + unsigned long fixed_pte_pfn;
>> +};
>> +
>> static void __init pte_basic_tests(unsigned long pfn, int idx)
>> {
>> pgprot_t prot = protection_map[idx];
>> @@ -955,8 +986,249 @@ static unsigned long __init get_random_vaddr(void)
>> return random_vaddr;
>> }
>>
>> +static void __init destroy_args(struct pgtable_debug_args *args)
>> +{
>> + struct page *page = NULL;
>> +
>> + /* Free (huge) page */
>> + if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) &&
>> + IS_ENABLED(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD) &&
>> + has_transparent_hugepage() &&
>> + args->pud_pfn != ULONG_MAX) {
>> + if (args->is_contiguous_page) {
>> + free_contig_range(args->pud_pfn,
>> + (1 << (HPAGE_PUD_SHIFT - PAGE_SHIFT)));
>> + } else {
>> + page = pfn_to_page(args->pud_pfn);
>> + __free_pages(page, HPAGE_PUD_SHIFT - PAGE_SHIFT);
>> + }
>> +
>> + args->pud_pfn = ULONG_MAX;
>> + args->pmd_pfn = ULONG_MAX;
>> + args->pte_pfn = ULONG_MAX;
>> + }
>> +
>> + if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) &&
>> + has_transparent_hugepage() &&
>> + args->pmd_pfn != ULONG_MAX) {
>> + if (args->is_contiguous_page) {
>> + free_contig_range(args->pmd_pfn, (1 << HPAGE_PMD_ORDER));
>> + } else {
>> + page = pfn_to_page(args->pmd_pfn);
>> + __free_pages(page, HPAGE_PMD_ORDER);
>> + }
>> +
>> + args->pmd_pfn = ULONG_MAX;
>> + args->pte_pfn = ULONG_MAX;
>> + }
>> +
>> + if (args->pte_pfn != ULONG_MAX) {
>> + page = pfn_to_page(args->pte_pfn);
>> + __free_pages(page, 0);
>> + }
>> +
>> + /* Free page table entries */
>> + if (args->start_ptep) {
>> + pte_free(args->mm, args->start_ptep);
>> + mm_dec_nr_ptes(args->mm);
>> + }
>> +
>> + if (args->start_pmdp) {
>> + pmd_free(args->mm, args->start_pmdp);
>> + mm_dec_nr_pmds(args->mm);
>> + }
>> +
>> + if (args->start_pudp) {
>> + pud_free(args->mm, args->start_pudp);
>> + mm_dec_nr_puds(args->mm);
>> + }
>> +
>> + if (args->start_p4dp)
>> + p4d_free(args->mm, args->p4dp);
>> +
>> + /* Free vma and mm struct */
>> + if (args->vma)
>> + vm_area_free(args->vma);
>
> Small nit, needs an extra line here.
>
Yes.
>> + if (args->mm)
>> + mmdrop(args->mm);
>> +}
>> +
>> +static int __init init_args(struct pgtable_debug_args *args)
>> +{
>> + struct page *page = NULL;
>> + phys_addr_t phys;
>> + int ret = 0;
>> +
>> + /*
>> + * Initialize the debugging data.
>> + *
>> + * __P000 (or even __S000) will help create page table entries with
>> + * PROT_NONE permission as required for pxx_protnone_tests().
>> + */
>> + memset(args, 0, sizeof(*args));
>> + args->vaddr = get_random_vaddr();
>> + args->page_prot = vm_get_page_prot(VMFLAGS);
>> + args->page_prot_none = __P000;
>> + args->is_contiguous_page = false;
>> + args->pud_pfn = ULONG_MAX;
>> + args->pmd_pfn = ULONG_MAX;
>> + args->pte_pfn = ULONG_MAX;
>> + args->fixed_pgd_pfn = ULONG_MAX;
>> + args->fixed_p4d_pfn = ULONG_MAX;
>> + args->fixed_pud_pfn = ULONG_MAX;
>> + args->fixed_pmd_pfn = ULONG_MAX;
>> + args->fixed_pte_pfn = ULONG_MAX;
>> +
>> + /* Allocate mm and vma */
>> + args->mm = mm_alloc();
>> + if (!args->mm) {
>> + pr_err("Failed to allocate mm struct\n");
>> + ret = -ENOMEM;
>> + goto error;
>> + }
>> +
>> + args->vma = vm_area_alloc(args->mm);
>> + if (!args->vma) {
>> + pr_err("Failed to allocate vma\n");
>> + ret = -ENOMEM;
>> + goto error;
>> + }
>> +
>> + /*
>> + * Allocate page table entries. They will be modified in the tests.
>> + * Lets save the page table entries so that they can be released
>> + * when the tests are completed.
>> + */
>> + args->pgdp = pgd_offset(args->mm, args->vaddr);
>> + args->p4dp = p4d_alloc(args->mm, args->pgdp, args->vaddr);
>> + if (!args->p4dp) {
>> + pr_err("Failed to allocate p4d entries\n");
>> + ret = -ENOMEM;
>> + goto error;
>> + }
>> +
>> + args->start_p4dp = p4d_offset(args->pgdp, 0UL);
>> + WARN_ON(!args->start_p4dp);
>
> Please move these two lines up to the previous block as args->start_p4dp
> is primarily derived from args->pgdp. Also please do the same for all
> others blocks down here.
>
Good point. I will do in v5.
>> + args->pudp = pud_alloc(args->mm, args->p4dp, args->vaddr);
>> + if (!args->pudp) {
>> + pr_err("Failed to allocate pud entries\n");
>> + ret = -ENOMEM;
>> + goto error;
>> + }
>> +
>> + args->start_pudp = pud_offset(args->p4dp, 0UL);
>> + WARN_ON(!args->start_pudp);
>> + args->pmdp = pmd_alloc(args->mm, args->pudp, args->vaddr);
>> + if (!args->pmdp) {
>> + pr_err("Failed to allocate pmd entries\n");
>> + ret = -ENOMEM;
>> + goto error;
>> + }
>> +
>> + args->start_pmdp = pmd_offset(args->pudp, 0UL);
>> + WARN_ON(!args->start_pmdp);
>> + args->ptep = pte_alloc_map(args->mm, args->pmdp, args->vaddr);
>> + if (!args->ptep) {
>> + pr_err("Failed to allocate pte entries\n");
>> + ret = -ENOMEM;
>> + goto error;
>> + }
>> +
>> + args->start_ptep = pmd_pgtable(READ_ONCE(*args->pmdp));
>> + WARN_ON(!args->start_ptep);
>> +
>> + /*
>> + * PFN for mapping at PTE level is determined from a standard kernel
>> + * text symbol. But pfns for higher page table levels are derived by
>> + * masking lower bits of this real pfn. These derived pfns might not
>> + * exist on the platform but that does not really matter as pfn_pxx()
>> + * helpers will still create appropriate entries for the test. This
>> + * helps avoid large memory block allocations to be used for mapping
>> + * at higher page table levels in some of the tests.
>> + */
>> + phys = __pa_symbol(&start_kernel);
>> + args->fixed_pgd_pfn = __phys_to_pfn(phys & PGDIR_MASK);
>> + args->fixed_p4d_pfn = __phys_to_pfn(phys & P4D_MASK);
>> + args->fixed_pud_pfn = __phys_to_pfn(phys & PUD_MASK);
>> + args->fixed_pmd_pfn = __phys_to_pfn(phys & PMD_MASK);
>> + args->fixed_pte_pfn = __phys_to_pfn(phys & PAGE_MASK);
>> + WARN_ON(!pfn_valid(args->fixed_pte_pfn));
>> +
>> + /*
>> + * Allocate (huge) pages because some of the tests need to access
>> + * the data in the pages. The corresponding tests will be skipped
>> + * if we fail to allocate (huge) pages.
>> + */
>> +#ifdef CONFIG_CONTIG_ALLOC
>> + if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) &&
>> + IS_ENABLED(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD) &&
>> + has_transparent_hugepage() &&
>> + (HPAGE_PUD_SHIFT - PAGE_SHIFT) >= MAX_ORDER) {
>> + page = alloc_contig_pages((1 << (HPAGE_PUD_SHIFT - PAGE_SHIFT)),
>> + GFP_KERNEL, first_online_node, NULL);
>> + if (page) {
>> + args->is_contiguous_page = true;
>> + args->pud_pfn = page_to_pfn(page);
>> + args->pmd_pfn = args->pud_pfn;
>> + args->pte_pfn = args->pud_pfn;
>> + return 0;
>> + }
>> + }
>> +#endif
>> +
>> + if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) &&
>> + IS_ENABLED(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD) &&
>> + has_transparent_hugepage() &&
>> + (HPAGE_PUD_SHIFT - PAGE_SHIFT) < MAX_ORDER) {
>> + page = alloc_pages(GFP_KERNEL, HPAGE_PUD_SHIFT - PAGE_SHIFT);
>> + if (page) {
>> + args->pud_pfn = page_to_pfn(page);
>> + args->pmd_pfn = args->pud_pfn;
>> + args->pte_pfn = args->pud_pfn;
>> + return 0;
>> + }
>> + }
>> +
>> +#ifdef CONFIG_CONTIG_ALLOC
>> + if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) &&
>> + has_transparent_hugepage() &&
>> + HPAGE_PMD_ORDER >= MAX_ORDER) {
>> + page = alloc_contig_pages((1 << HPAGE_PMD_ORDER), GFP_KERNEL,
>> + first_online_node, NULL);
>> + if (page) {
>> + args->is_contiguous_page = true;
>> + args->pmd_pfn = page_to_pfn(page);
>> + args->pte_pfn = args->pmd_pfn;
>> + return 0;
>> + }
>> + }
>> +#endif
>> +
>> + if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) &&
>> + has_transparent_hugepage() &&
>> + HPAGE_PMD_ORDER < MAX_ORDER) {
>> + page = alloc_pages(GFP_KERNEL, HPAGE_PMD_ORDER);
>> + if (page) {
>> + args->pmd_pfn = page_to_pfn(page);
>> + args->pte_pfn = args->pmd_pfn;
>> + return 0;
>> + }
>> + }
>
> This can be simplified further. Could you please define a helper alloc_huge_page()
> which compares the order against MAX_ORDER and calls either alloc_contig_pages()
> when CONFIG_CONTIG_ALLOC or alloc_pages(). This will result in reduced code and
> CONFIG_CONTIG_ALLOC will move into the helper as well.
>
Yes, I will introduce a helper for this, but the function name will be
debug_vm_pgtable_alloc_huge_page() because alloc_huge_page() has been
declared in include/linux/hugetlb.h
>> +
>> + page = alloc_pages(GFP_KERNEL, 0);
>> + if (page)
>> + args->pte_pfn = page_to_pfn(page);
>> +
>> + return 0;
>> +
>> +error:
>> + destroy_args(args);
>> + return ret;
>> +}
>> +
>> static int __init debug_vm_pgtable(void)
>> {
>> + struct pgtable_debug_args args;
>> struct vm_area_struct *vma;
>> struct mm_struct *mm;
>> pgd_t *pgdp;
>> @@ -970,9 +1242,13 @@ static int __init debug_vm_pgtable(void)
>> unsigned long vaddr, pte_aligned, pmd_aligned;
>> unsigned long pud_aligned, p4d_aligned, pgd_aligned;
>> spinlock_t *ptl = NULL;
>> - int idx;
>> + int idx, ret;
>>
>> pr_info("Validating architecture page table helpers\n");
>> + ret = init_args(&args);
>> + if (ret)
>> + return ret;
>> +
>> prot = vm_get_page_prot(VMFLAGS);
>> vaddr = get_random_vaddr();
>> mm = mm_alloc();
>> @@ -1127,6 +1403,8 @@ static int __init debug_vm_pgtable(void)
>> mm_dec_nr_pmds(mm);
>> mm_dec_nr_ptes(mm);
>> mmdrop(mm);
>> +
>> + destroy_args(&args);
>> return 0;
>> }
>> late_initcall(debug_vm_pgtable);
>>
>
> Otherwise LGTM.
>
I will treat this as r-b in v5.
Thanks,
Gavin
Hi Anshuman,
On 7/29/21 5:14 PM, Anshuman Khandual wrote:
> On 7/27/21 11:43 AM, Gavin Shan wrote:
>> There are couple of issues with current implementations and this series
>> tries to resolve the issues:
>>
>> (a) All needed information are scattered in variables, passed to various
>> test functions. The code is organized in pretty much relaxed fashion.
>>
>> (b) The page isn't allocated from buddy during page table entry modifying
>> tests. The page can be invalid, conflicting to the implementations
>> of set_xxx_at() on ARM64. The target page is accessed so that the iCache
>> can be flushed when execution permission is given on ARM64. Besides,
>> the target page can be unmapped and access to it causes kernel crash.
>>
>> "struct pgtable_debug_args" is introduced to address issue (a). For issue
>> (b), the used page is allocated from buddy in page table entry modifying
>> tests. The corresponding tets will be skipped if we fail to allocate the
>> (huge) page. For other test cases, the original page around to kernel
>> symbol (@start_kernel) is still used.
>>
>> The patches are organized as below. PATCH[2-10] could be combined to one
>> patch, but it will make the review harder:
>>
>> PATCH[1] introduces "struct pgtable_debug_args" as place holder of all
>> needed information. With it, the old and new implementation
>> can coexist.
>> PATCH[2-10] uses "struct pgtable_debug_args" in various test functions.
>> PATCH[11] removes the unused code for old implementation.
>> PATCH[12] fixes the issue of corrupted page flag for ARM64
>>
>> Changelog
>> =========
>> v4:
>> * Determine the page allocation method according to
>> MAX_ORDER (Anshuman)
>> * Move existing comments to init_args() (Anshuman)
>> * Code refactoring as suggested by Anshuman (Anshuman)
>> * Improved commit log and add comments for flush_dcache_page()
>> in PATCH[v4 12/12] (Anshuman)
>> * Address misc comments (Anshuman
>
> This series builds ok on all supported platforms but should also be tested
> on them as well i.e s390, ppc, arc and riscv. I was able to do some tests
> on arm64 and x86 platforms.
>
I had tests on the platforms except arc as it's not supported by QEMU yet.
The test is pretty simple to boot the kernel and see if there is any warning
raised from mm/debug_vm_pgtable. Everything looks good.
The test was done on v5, which will be posted pretty soon. Thanks again for
your continuous review.
Thanks,
Gavin
On 8/2/21 5:39 AM, Gavin Shan wrote:
> Hi Anshuman,
>
> On 7/29/21 5:14 PM, Anshuman Khandual wrote:
>> On 7/27/21 11:43 AM, Gavin Shan wrote:
>>> There are couple of issues with current implementations and this series
>>> tries to resolve the issues:
>>>
>>> (a) All needed information are scattered in variables, passed to various
>>> test functions. The code is organized in pretty much relaxed fashion.
>>>
>>> (b) The page isn't allocated from buddy during page table entry modifying
>>> tests. The page can be invalid, conflicting to the implementations
>>> of set_xxx_at() on ARM64. The target page is accessed so that the iCache
>>> can be flushed when execution permission is given on ARM64. Besides,
>>> the target page can be unmapped and access to it causes kernel crash.
>>>
>>> "struct pgtable_debug_args" is introduced to address issue (a). For issue
>>> (b), the used page is allocated from buddy in page table entry modifying
>>> tests. The corresponding tets will be skipped if we fail to allocate the
>>> (huge) page. For other test cases, the original page around to kernel
>>> symbol (@start_kernel) is still used.
>>>
>>> The patches are organized as below. PATCH[2-10] could be combined to one
>>> patch, but it will make the review harder:
>>>
>>> PATCH[1] introduces "struct pgtable_debug_args" as place holder of all
>>> needed information. With it, the old and new implementation
>>> can coexist.
>>> PATCH[2-10] uses "struct pgtable_debug_args" in various test functions.
>>> PATCH[11] removes the unused code for old implementation.
>>> PATCH[12] fixes the issue of corrupted page flag for ARM64
>>>
>>> Changelog
>>> =========
>>> v4:
>>> * Determine the page allocation method according to
>>> MAX_ORDER (Anshuman)
>>> * Move existing comments to init_args() (Anshuman)
>>> * Code refactoring as suggested by Anshuman (Anshuman)
>>> * Improved commit log and add comments for flush_dcache_page()
>>> in PATCH[v4 12/12] (Anshuman)
>>> * Address misc comments (Anshuman
>>
>> This series builds ok on all supported platforms but should also be tested
>> on them as well i.e s390, ppc, arc and riscv. I was able to do some tests
>> on arm64 and x86 platforms.
>>
>
> I had tests on the platforms except arc as it's not supported by QEMU yet.
> The test is pretty simple to boot the kernel and see if there is any warning
> raised from mm/debug_vm_pgtable. Everything looks good.
Sure, that should be enough.
>
> The test was done on v5, which will be posted pretty soon. Thanks again for
> your continuous review.
Please also do copy Vineet Gupta <[email protected]> for ARC in the next version.
- Anshuman