Received: by 10.213.65.68 with SMTP id h4csp654967imn; Tue, 27 Mar 2018 06:26:34 -0700 (PDT) X-Google-Smtp-Source: AG47ELuidLLy5cEDiYfPVkbYNpiJrnw5nFdX9z7qHN4l3wzemivr4IpVGg6zHG4pU5r+txN9LUKg X-Received: by 10.101.97.200 with SMTP id j8mr9603764pgv.443.1522157194842; Tue, 27 Mar 2018 06:26:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1522157194; cv=none; d=google.com; s=arc-20160816; b=dSoK7s6SByjyWNrEOi93fKKx2hBwr+cg8mjbwwnQG85PoecApP1biMpJgwjyqAMiOc xoTvbh3330+ZleU4KmETuHR1FMFlNDkR/QKCbGENbMedCemYB+0fSCuPzePYI/dD3CF4 YVVJCdNeat1J5zDG6QeVws1fNIfRZWa9hSpETg2Kj8R/stDMiBOwoMkD67ydKzKoKiAJ i5zcCiyH9aUU9bePpyKOquFyolcS2hG+kkW7wOX3rqhsqrHbRSx+9x6p4KuKhOyEljq9 k4c4FqbZRyMga9l0B7875xjU2Qi94sLh6KPEw4cTak6zSAfn5WE6w1RwdlGC7ffP4oKf USMg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=qtBafllB6zQluOfek20f32/BCn+U4svx9BD0cx1wNCc=; b=d5SGlwvdL1DYolE7cCt0DIVyDeNg5frk9OvUwywKhSJtbYX8ZtOnChTXFCW+i+T9gt 75ft6Qc2QBMacbUKyCnXO0dPhCUaMyi0UFbwGReDsLY2XGtmD+lYr0MpxRaNV4OxPouU 8xvLPpjs9WgbnwXn+iqwW/e0uo56xT+Dx0pxDegLuaIwhEsOzYkzfGNPU6E+O5mZB/cu zTaWsUBcpj5+djOt5ZghGHoAWHW3oKTOumcsQkb4N8KXi4BBR8BuDc1tIyvuf81aMM7z ysjqHLc0B1SQ5hWWCGCeo/PjTA7Sk/onUSR2BAMACy/dRiYfXBD0LkhYKoEkhtgSqS0t xFFQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 3-v6si1274615plo.475.2018.03.27.06.26.20; Tue, 27 Mar 2018 06:26:34 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752905AbeC0NYj (ORCPT + 99 others); Tue, 27 Mar 2018 09:24:39 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:55000 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752165AbeC0NQK (ORCPT ); Tue, 27 Mar 2018 09:16:10 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D11DE1650; Tue, 27 Mar 2018 06:16:09 -0700 (PDT) Received: from en101.cambridge.arm.com (en101.cambridge.arm.com [10.1.206.73]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 0DF193F24A; Tue, 27 Mar 2018 06:16:06 -0700 (PDT) From: Suzuki K Poulose To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, cdall@kernel.org, marc.zyngier@arm.com, punit.agrawal@arm.com, will.deacon@arm.com, catalin.marinas@arm.com, pbonzini@redhat.com, rkrcmar@redhat.com, ard.biesheuvel@linaro.org, peter.maydell@linaro.org, kristina.martsenko@arm.com, mark.rutland@arm.com, Suzuki K Poulose , Steve Capper Subject: [PATCH v2 03/17] arm64: Make page table helpers reusable Date: Tue, 27 Mar 2018 14:15:13 +0100 Message-Id: <1522156531-28348-4-git-send-email-suzuki.poulose@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1522156531-28348-1-git-send-email-suzuki.poulose@arm.com> References: <1522156531-28348-1-git-send-email-suzuki.poulose@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This patch rearranges the page table level helpers so that it can be reused for a page table with different number of levels (e.g, stage2 page table for a VM) than the kernel page tables. As such there is no functional change with this patch. The page table helpers are defined to do the right thing for the fixed page table levels set for the kernel. This patch tries to refactor the code such that, we can have helpers for each level, which should be used when the caller knows that the level exists for the page table dealt with. Since the kernel defines helpers p.d_action and __p.d_action, for consistency, we name the raw page table action helpers __raw_p.d_action. Cc: Catalin Marinas Cc: Mark Rutland Cc: Will Deacon Cc: Steve Capper Cc: Marc Zyngier Cc: Christoffer Dall Signed-off-by: Suzuki K Poulose --- arch/arm64/include/asm/pgalloc.h | 34 +++++++++++++++++++---- arch/arm64/include/asm/pgtable.h | 60 +++++++++++++++++++++++++--------------- 2 files changed, 66 insertions(+), 28 deletions(-) diff --git a/arch/arm64/include/asm/pgalloc.h b/arch/arm64/include/asm/pgalloc.h index 2e05bcd..ab4662c 100644 --- a/arch/arm64/include/asm/pgalloc.h +++ b/arch/arm64/include/asm/pgalloc.h @@ -29,6 +29,30 @@ #define PGALLOC_GFP (GFP_KERNEL | __GFP_ZERO) #define PGD_SIZE (PTRS_PER_PGD * sizeof(pgd_t)) +static inline void __raw_pmd_free(pmd_t *pmdp) +{ + BUG_ON((unsigned long)pmdp & (PAGE_SIZE-1)); + free_page((unsigned long)pmdp); +} + +static inline void +__raw_pud_populate(pud_t *pudp, phys_addr_t pmdp, pudval_t prot) +{ + __raw_set_pud(pudp, __pud(__phys_to_pud_val(pmdp) | prot)); +} + +static inline void __raw_pud_free(pud_t *pudp) +{ + BUG_ON((unsigned long)pudp & (PAGE_SIZE-1)); + free_page((unsigned long)pudp); +} + +static inline void +__raw_pgd_populate(pgd_t *pgdp, phys_addr_t pudp, pgdval_t prot) +{ + __raw_set_pgd(pgdp, __pgd(__phys_to_pgd_val(pudp) | prot)); +} + #if CONFIG_PGTABLE_LEVELS > 2 static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long addr) @@ -38,13 +62,12 @@ static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long addr) static inline void pmd_free(struct mm_struct *mm, pmd_t *pmdp) { - BUG_ON((unsigned long)pmdp & (PAGE_SIZE-1)); - free_page((unsigned long)pmdp); + __raw_pmd_free(pmdp); } static inline void __pud_populate(pud_t *pudp, phys_addr_t pmdp, pudval_t prot) { - set_pud(pudp, __pud(__phys_to_pud_val(pmdp) | prot)); + __raw_pud_populate(pudp, pmdp, prot); } static inline void pud_populate(struct mm_struct *mm, pud_t *pudp, pmd_t *pmdp) @@ -67,13 +90,12 @@ static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long addr) static inline void pud_free(struct mm_struct *mm, pud_t *pudp) { - BUG_ON((unsigned long)pudp & (PAGE_SIZE-1)); - free_page((unsigned long)pudp); + __raw_pud_free(pudp); } static inline void __pgd_populate(pgd_t *pgdp, phys_addr_t pudp, pgdval_t prot) { - set_pgd(pgdp, __pgd(__phys_to_pgd_val(pudp) | prot)); + __raw_pgd_populate(pgdp, pudp, prot); } static inline void pgd_populate(struct mm_struct *mm, pgd_t *pgdp, pud_t *pudp) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 7e2c27e..5e22503 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -475,31 +475,39 @@ static inline phys_addr_t pmd_page_paddr(pmd_t pmd) */ #define mk_pte(page,prot) pfn_pte(page_to_pfn(page),prot) -#if CONFIG_PGTABLE_LEVELS > 2 +#define __raw_pud_none(pud) (!pud_val(pud)) +#define __raw_pud_bad(pud) (!(pud_val(pud) & PUD_TABLE_BIT)) +#define __raw_pud_present(pud) pte_present(pud_pte(pud)) -#define pmd_ERROR(pmd) __pmd_error(__FILE__, __LINE__, pmd_val(pmd)) - -#define pud_none(pud) (!pud_val(pud)) -#define pud_bad(pud) (!(pud_val(pud) & PUD_TABLE_BIT)) -#define pud_present(pud) pte_present(pud_pte(pud)) - -static inline void set_pud(pud_t *pudp, pud_t pud) +static inline void __raw_set_pud(pud_t *pudp, pud_t pud) { WRITE_ONCE(*pudp, pud); dsb(ishst); isb(); } -static inline void pud_clear(pud_t *pudp) +static inline void __raw_pud_clear(pud_t *pudp) { - set_pud(pudp, __pud(0)); + __raw_set_pud(pudp, __pud(0)); } -static inline phys_addr_t pud_page_paddr(pud_t pud) +static inline phys_addr_t __raw_pud_page_paddr(pud_t pud) { return __pud_to_phys(pud); } +#if CONFIG_PGTABLE_LEVELS > 2 + +#define pmd_ERROR(pmd) __pmd_error(__FILE__, __LINE__, pmd_val(pmd)) + +#define pud_none(pud) __raw_pud_none(pud) +#define pud_bad(pud) __raw_pud_bad(pud) +#define pud_present(pud) __raw_pud_present(pud) + +#define set_pud(pudp, pud) __raw_set_pud((pudp), (pud)) +#define pud_clear(pudp) __raw_pud_clear((pudp)) +#define pud_page_paddr(pud) __raw_pud_page_paddr((pud)) + /* Find an entry in the second-level page table. */ #define pmd_index(addr) (((addr) >> PMD_SHIFT) & (PTRS_PER_PMD - 1)) @@ -528,30 +536,38 @@ static inline phys_addr_t pud_page_paddr(pud_t pud) #endif /* CONFIG_PGTABLE_LEVELS > 2 */ -#if CONFIG_PGTABLE_LEVELS > 3 +#define __raw_pgd_none(pgd) (!pgd_val(pgd)) +#define __raw_pgd_bad(pgd) (!(pgd_val(pgd) & 2)) +#define __raw_pgd_present(pgd) (pgd_val(pgd)) -#define pud_ERROR(pud) __pud_error(__FILE__, __LINE__, pud_val(pud)) - -#define pgd_none(pgd) (!pgd_val(pgd)) -#define pgd_bad(pgd) (!(pgd_val(pgd) & 2)) -#define pgd_present(pgd) (pgd_val(pgd)) - -static inline void set_pgd(pgd_t *pgdp, pgd_t pgd) +static inline void __raw_set_pgd(pgd_t *pgdp, pgd_t pgd) { WRITE_ONCE(*pgdp, pgd); dsb(ishst); } -static inline void pgd_clear(pgd_t *pgdp) +static inline void __raw_pgd_clear(pgd_t *pgdp) { - set_pgd(pgdp, __pgd(0)); + __raw_set_pgd(pgdp, __pgd(0)); } -static inline phys_addr_t pgd_page_paddr(pgd_t pgd) +static inline phys_addr_t __raw_pgd_page_paddr(pgd_t pgd) { return __pgd_to_phys(pgd); } +#if CONFIG_PGTABLE_LEVELS > 3 + +#define pud_ERROR(pud) __pud_error(__FILE__, __LINE__, pud_val(pud)) + +#define pgd_none(pgd) __raw_pgd_none((pgd)) +#define pgd_bad(pgd) __raw_pgd_bad((pgd)) +#define pgd_present(pgd) __raw_pgd_present((pgd)) + +#define set_pgd(pgdp, pgd) __raw_set_pgd((pgdp), (pgd)) +#define pgd_clear(pgdp) __raw_pgd_clear((pgdp)) +#define pgd_page_paddr(pgd) __raw_pgd_page_paddr((pgd)) + /* Find an entry in the frst-level page table. */ #define pud_index(addr) (((addr) >> PUD_SHIFT) & (PTRS_PER_PUD - 1)) -- 2.7.4