Received: by 10.213.65.68 with SMTP id h4csp884459imn; Tue, 27 Mar 2018 10:33:17 -0700 (PDT) X-Google-Smtp-Source: AIpwx4+/QAErYcVh3ycy8NMS2nJP5GSi889QUieo05yDm7AhhtYzaTDtxg/KpwRyEr1sqTCXRGCW X-Received: by 2002:a17:902:161:: with SMTP id 88-v6mr232704plb.212.1522171997358; Tue, 27 Mar 2018 10:33:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1522171997; cv=none; d=google.com; s=arc-20160816; b=PQ0GPSV+gyZriyS8tEuMfa05N6JjDqCwqTbBBWLS04jo7UDaeVA9PEbBuHuj+vyp7G jl+T8pyKrxf3kAZ40LTph7DFmK5jRaDEM4sCEtpZvDIbNLwpIdp8FGvWPD+H+z0HaSwI lA/vkERvacpQuyHACaP0yCjb2uQGcECdqColpip9SzIpHEwjDKibXrOI498rJxmOXQB5 0+3fWr+obfx/6r4G0Q0S2ZqdFBQib88WPRjL2vvJW+E5rxIkVUahnJ1vGkLwS1TE/vw/ Bn8gEw/xyoJPqH66hdAAM9Y5DPYGbgOisP5RbgWEvVIY3h/rwNaVpvPYMQg4mlE/NTOn ovpQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:user-agent:references :in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=j94aMi0uL2HmcIlzWa64u5wYu62eMdcimVUOYFFHhpQ=; b=cxGnB92FYz9qTAZy2SNEtoEt8x9wKpXkdI2u3cCJ7DoydZOr49plphojJntpnV+Xgi kH8GJ6tlF4n3cRixCz+xkB4RMa2SEk4pLIAFGZ+ERGJwmXNkZmYnBVImB9yQuJ8RAsws gQOopORUbtQe6Z5P1bWJltSq1SZArAcOUeAi/6GKMu1cz8HcvGr+xbYjp1nREPys09kI orihSeIsp0QTC7HjL4C2Pe+shoqB4FGr1YuZchVMw9EQYn8Fui4XjDDnsuEUMps8ONZD fgnECksPmwOdM7+YgHGaKiwLe52rrWDrsLcSk1XvPoXX2tqL6kXgGSpzrViJd96aQOcV C4Hw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g11si1112261pgs.153.2018.03.27.10.33.02; Tue, 27 Mar 2018 10:33:17 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754570AbeC0Qfp (ORCPT + 99 others); Tue, 27 Mar 2018 12:35:45 -0400 Received: from mail.linuxfoundation.org ([140.211.169.12]:44932 "EHLO mail.linuxfoundation.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754065AbeC0Qfm (ORCPT ); Tue, 27 Mar 2018 12:35:42 -0400 Received: from localhost (LFbn-1-12247-202.w90-92.abo.wanadoo.fr [90.92.61.202]) by mail.linuxfoundation.org (Postfix) with ESMTPSA id 6F3261264; Tue, 27 Mar 2018 16:35:41 +0000 (UTC) From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Lei Li , Toshi Kani , Catalin Marinas , Wang Xuefeng , Will Deacon , Hanjun Guo , Michal Hocko , Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , Borislav Petkov , Matthew Wilcox , Chintan Pandya , Andrew Morton , Linus Torvalds Subject: [PATCH 4.14 047/101] mm/vmalloc: add interfaces to free unmapped page table Date: Tue, 27 Mar 2018 18:27:19 +0200 Message-Id: <20180327162752.926350491@linuxfoundation.org> X-Mailer: git-send-email 2.16.3 In-Reply-To: <20180327162749.993880276@linuxfoundation.org> References: <20180327162749.993880276@linuxfoundation.org> User-Agent: quilt/0.65 X-stable: review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org 4.14-stable review patch. If anyone has any objections, please let me know. ------------------ From: Toshi Kani commit b6bdb7517c3d3f41f20e5c2948d6bc3f8897394e upstream. On architectures with CONFIG_HAVE_ARCH_HUGE_VMAP set, ioremap() may create pud/pmd mappings. A kernel panic was observed on arm64 systems with Cortex-A75 in the following steps as described by Hanjun Guo. 1. ioremap a 4K size, valid page table will build, 2. iounmap it, pte0 will set to 0; 3. ioremap the same address with 2M size, pgd/pmd is unchanged, then set the a new value for pmd; 4. pte0 is leaked; 5. CPU may meet exception because the old pmd is still in TLB, which will lead to kernel panic. This panic is not reproducible on x86. INVLPG, called from iounmap, purges all levels of entries associated with purged address on x86. x86 still has memory leak. The patch changes the ioremap path to free unmapped page table(s) since doing so in the unmap path has the following issues: - The iounmap() path is shared with vunmap(). Since vmap() only supports pte mappings, making vunmap() to free a pte page is an overhead for regular vmap users as they do not need a pte page freed up. - Checking if all entries in a pte page are cleared in the unmap path is racy, and serializing this check is expensive. - The unmap path calls free_vmap_area_noflush() to do lazy TLB purges. Clearing a pud/pmd entry before the lazy TLB purges needs extra TLB purge. Add two interfaces, pud_free_pmd_page() and pmd_free_pte_page(), which clear a given pud/pmd entry and free up a page for the lower level entries. This patch implements their stub functions on x86 and arm64, which work as workaround. [akpm@linux-foundation.org: fix typo in pmd_free_pte_page() stub] Link: http://lkml.kernel.org/r/20180314180155.19492-2-toshi.kani@hpe.com Fixes: e61ce6ade404e ("mm: change ioremap to set up huge I/O mappings") Reported-by: Lei Li Signed-off-by: Toshi Kani Cc: Catalin Marinas Cc: Wang Xuefeng Cc: Will Deacon Cc: Hanjun Guo Cc: Michal Hocko Cc: Thomas Gleixner Cc: Ingo Molnar Cc: "H. Peter Anvin" Cc: Borislav Petkov Cc: Matthew Wilcox Cc: Chintan Pandya Cc: Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Greg Kroah-Hartman --- arch/arm64/mm/mmu.c | 10 ++++++++++ arch/x86/mm/pgtable.c | 24 ++++++++++++++++++++++++ include/asm-generic/pgtable.h | 10 ++++++++++ lib/ioremap.c | 6 ++++-- 4 files changed, 48 insertions(+), 2 deletions(-) --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -937,3 +937,13 @@ int pmd_clear_huge(pmd_t *pmd) pmd_clear(pmd); return 1; } + +int pud_free_pmd_page(pud_t *pud) +{ + return pud_none(*pud); +} + +int pmd_free_pte_page(pmd_t *pmd) +{ + return pmd_none(*pmd); +} --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -702,4 +702,28 @@ int pmd_clear_huge(pmd_t *pmd) return 0; } + +/** + * pud_free_pmd_page - Clear pud entry and free pmd page. + * @pud: Pointer to a PUD. + * + * Context: The pud range has been unmaped and TLB purged. + * Return: 1 if clearing the entry succeeded. 0 otherwise. + */ +int pud_free_pmd_page(pud_t *pud) +{ + return pud_none(*pud); +} + +/** + * pmd_free_pte_page - Clear pmd entry and free pte page. + * @pmd: Pointer to a PMD. + * + * Context: The pmd range has been unmaped and TLB purged. + * Return: 1 if clearing the entry succeeded. 0 otherwise. + */ +int pmd_free_pte_page(pmd_t *pmd) +{ + return pmd_none(*pmd); +} #endif /* CONFIG_HAVE_ARCH_HUGE_VMAP */ --- a/include/asm-generic/pgtable.h +++ b/include/asm-generic/pgtable.h @@ -976,6 +976,8 @@ int pud_set_huge(pud_t *pud, phys_addr_t int pmd_set_huge(pmd_t *pmd, phys_addr_t addr, pgprot_t prot); int pud_clear_huge(pud_t *pud); int pmd_clear_huge(pmd_t *pmd); +int pud_free_pmd_page(pud_t *pud); +int pmd_free_pte_page(pmd_t *pmd); #else /* !CONFIG_HAVE_ARCH_HUGE_VMAP */ static inline int p4d_set_huge(p4d_t *p4d, phys_addr_t addr, pgprot_t prot) { @@ -1001,6 +1003,14 @@ static inline int pmd_clear_huge(pmd_t * { return 0; } +static inline int pud_free_pmd_page(pud_t *pud) +{ + return 0; +} +static inline int pmd_free_pte_page(pmd_t *pmd) +{ + return 0; +} #endif /* CONFIG_HAVE_ARCH_HUGE_VMAP */ #ifndef __HAVE_ARCH_FLUSH_PMD_TLB_RANGE --- a/lib/ioremap.c +++ b/lib/ioremap.c @@ -91,7 +91,8 @@ static inline int ioremap_pmd_range(pud_ if (ioremap_pmd_enabled() && ((next - addr) == PMD_SIZE) && - IS_ALIGNED(phys_addr + addr, PMD_SIZE)) { + IS_ALIGNED(phys_addr + addr, PMD_SIZE) && + pmd_free_pte_page(pmd)) { if (pmd_set_huge(pmd, phys_addr + addr, prot)) continue; } @@ -117,7 +118,8 @@ static inline int ioremap_pud_range(p4d_ if (ioremap_pud_enabled() && ((next - addr) == PUD_SIZE) && - IS_ALIGNED(phys_addr + addr, PUD_SIZE)) { + IS_ALIGNED(phys_addr + addr, PUD_SIZE) && + pud_free_pmd_page(pud)) { if (pud_set_huge(pud, phys_addr + addr, prot)) continue; }