Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752786AbbF3JAq (ORCPT ); Tue, 30 Jun 2015 05:00:46 -0400 Received: from mailout1.w1.samsung.com ([210.118.77.11]:36577 "EHLO mailout1.w1.samsung.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750940AbbF3JAU (ORCPT ); Tue, 30 Jun 2015 05:00:20 -0400 X-AuditID: cbfec7f5-f794b6d000001495-99-55925aa1aa95 From: Andrey Ryabinin To: Ingo Molnar , "H. Peter Anvin" , Thomas Gleixner , x86@kernel.org Cc: Andrey Konovalov , Andrew Morton , Borislav Petkov , Alexander Popov , Dmitry Vyukov , Alexander Potapenko , linux-kernel@vger.kernel.org, Andrey Ryabinin , stable@vger.kernel.org Subject: [PATCH 1/5] x86_64: fix kasan shadow region page tables Date: Tue, 30 Jun 2015 12:00:07 +0300 Message-id: <1435654811-8915-2-git-send-email-a.ryabinin@samsung.com> X-Mailer: git-send-email 2.4.4 In-reply-to: <1435654811-8915-1-git-send-email-a.ryabinin@samsung.com> References: <1435654466-8714-1-git-send-email-a.ryabinin@samsung.com> <1435654811-8915-1-git-send-email-a.ryabinin@samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrPLMWRmVeSWpSXmKPExsVy+t/xy7oLoyaFGhw8om2x7dcjNovfe2ey WsxZv4bNYvqOPnaLzxv+sVlMeNjGbtH+cS+zxbSN4haXd81hs7h0YAGTxYKNjxgtNm+aymzx Y8NjVgdej++tfSweO2fdZfdYsKnUY9OqTjaPd+fOsXucmPGbxWP7o6ksHu/3XWXz6NuyitHj 8yY5jxMtX1gDuKO4bFJSczLLUov07RK4Mnp2X2AruKVfMbu5g72BsU2ji5GTQ0LAROLr41uM ELaYxIV769m6GLk4hASWMkpcnTeFCcJpYpJ4desrM0gVm4CexL9Z29lAbBGBGomO1m5mkCJm gctMEo+n9rN3MXJwCAs4SNz85Q1SwyKgKvHpyEoWEJtXwFXi3a4NUNvkJK5cnw42h1PATWL5 q2MsEMsaGSV+3b7DPoGRdwEjwypG0dTS5ILipPRcI73ixNzi0rx0veT83E2MkGD+uoNx6TGr Q4wCHIxKPLwJTyeGCrEmlhVX5h5ilOBgVhLhrZOcFCrEm5JYWZValB9fVJqTWnyIUZqDRUmc d+au9yFCAumJJanZqakFqUUwWSYOTqkGRoYny5kOzTiv+nf7XvOb7+x0+mJmvvp54VPfyntz Xy6ZHHyMN9J6jspz1z9Sh0R+3Gcs7G26x8wf05dyzZf1p2G/z9F/QU5H8lyfVv7QKip+/Zqt WqNgQ+SZdZJz5bjynjTM2Tijvajw24JbbNstPbZOyT1tPEXhR811U63wBr2Aawk3mNe/CFdi Kc5INNRiLipOBABlOkLTYgIAAA== Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 6163 Lines: 193 From: Alexander Popov Currently kasan shadow region page tables created without respect of physical offset (phys_base). This causes kernel halt when phys_base is not zero. So let's initialize kasan shadow region page tables in kasan_early_init() using __pa_nodebug() which considers phys_base. This patch also cleans up x86_64_start_kernel() from kasan low level details by moving kasan_map_early_shadow(init_level4_pgt) into kasan_early_init(). That requires to move clear_page(init_level4_pgt) before kasan_early_init(). Remove the comment before clear_bss() which stopped bringing much profit to the code readability. Otherwise describing all the new order dependencies would be too verbose. Signed-off-by: Alexander Popov Signed-off-by: Andrey Ryabinin Cc: # 4.0 --- arch/x86/include/asm/kasan.h | 8 ++------ arch/x86/kernel/head64.c | 10 ++++------ arch/x86/kernel/head_64.S | 29 ----------------------------- arch/x86/mm/kasan_init_64.c | 36 ++++++++++++++++++++++++++++++++++-- 4 files changed, 40 insertions(+), 43 deletions(-) diff --git a/arch/x86/include/asm/kasan.h b/arch/x86/include/asm/kasan.h index 8b22422..74a2a8d 100644 --- a/arch/x86/include/asm/kasan.h +++ b/arch/x86/include/asm/kasan.h @@ -14,15 +14,11 @@ #ifndef __ASSEMBLY__ -extern pte_t kasan_zero_pte[]; -extern pte_t kasan_zero_pmd[]; -extern pte_t kasan_zero_pud[]; - #ifdef CONFIG_KASAN -void __init kasan_map_early_shadow(pgd_t *pgd); +void __init kasan_early_init(void); void __init kasan_init(void); #else -static inline void kasan_map_early_shadow(pgd_t *pgd) { } +static inline void kasan_early_init(void) { } static inline void kasan_init(void) { } #endif diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index 5a46681..f129a9a 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -161,11 +161,12 @@ asmlinkage __visible void __init x86_64_start_kernel(char * real_mode_data) /* Kill off the identity-map trampoline */ reset_early_page_tables(); - kasan_map_early_shadow(early_level4_pgt); - - /* clear bss before set_intr_gate with early_idt_handler */ clear_bss(); + clear_page(init_level4_pgt); + + kasan_early_init(); + for (i = 0; i < NUM_EXCEPTION_VECTORS; i++) set_intr_gate(i, early_idt_handler_array[i]); load_idt((const struct desc_ptr *)&idt_descr); @@ -177,12 +178,9 @@ asmlinkage __visible void __init x86_64_start_kernel(char * real_mode_data) */ load_ucode_bsp(); - clear_page(init_level4_pgt); /* set init_level4_pgt kernel high mapping*/ init_level4_pgt[511] = early_level4_pgt[511]; - kasan_map_early_shadow(init_level4_pgt); - x86_64_start_reservations(real_mode_data); } diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index df7e780..7e5da2c 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -516,38 +516,9 @@ ENTRY(phys_base) /* This must match the first entry in level2_kernel_pgt */ .quad 0x0000000000000000 -#ifdef CONFIG_KASAN -#define FILL(VAL, COUNT) \ - .rept (COUNT) ; \ - .quad (VAL) ; \ - .endr - -NEXT_PAGE(kasan_zero_pte) - FILL(kasan_zero_page - __START_KERNEL_map + _KERNPG_TABLE, 512) -NEXT_PAGE(kasan_zero_pmd) - FILL(kasan_zero_pte - __START_KERNEL_map + _KERNPG_TABLE, 512) -NEXT_PAGE(kasan_zero_pud) - FILL(kasan_zero_pmd - __START_KERNEL_map + _KERNPG_TABLE, 512) - -#undef FILL -#endif - - #include "../../x86/xen/xen-head.S" __PAGE_ALIGNED_BSS NEXT_PAGE(empty_zero_page) .skip PAGE_SIZE -#ifdef CONFIG_KASAN -/* - * This page used as early shadow. We don't use empty_zero_page - * at early stages, stack instrumentation could write some garbage - * to this page. - * Latter we reuse it as zero shadow for large ranges of memory - * that allowed to access, but not instrumented by kasan - * (vmalloc/vmemmap ...). - */ -NEXT_PAGE(kasan_zero_page) - .skip PAGE_SIZE -#endif diff --git a/arch/x86/mm/kasan_init_64.c b/arch/x86/mm/kasan_init_64.c index 4860906..0e4a05f 100644 --- a/arch/x86/mm/kasan_init_64.c +++ b/arch/x86/mm/kasan_init_64.c @@ -11,7 +11,19 @@ extern pgd_t early_level4_pgt[PTRS_PER_PGD]; extern struct range pfn_mapped[E820_X_MAX]; -extern unsigned char kasan_zero_page[PAGE_SIZE]; +static pud_t kasan_zero_pud[PTRS_PER_PUD] __page_aligned_bss; +static pmd_t kasan_zero_pmd[PTRS_PER_PMD] __page_aligned_bss; +static pte_t kasan_zero_pte[PTRS_PER_PTE] __page_aligned_bss; + +/* + * This page used as early shadow. We don't use empty_zero_page + * at early stages, stack instrumentation could write some garbage + * to this page. + * Latter we reuse it as zero shadow for large ranges of memory + * that allowed to access, but not instrumented by kasan + * (vmalloc/vmemmap ...). + */ +static unsigned char kasan_zero_page[PAGE_SIZE] __page_aligned_bss; static int __init map_range(struct range *range) { @@ -36,7 +48,7 @@ static void __init clear_pgds(unsigned long start, pgd_clear(pgd_offset_k(start)); } -void __init kasan_map_early_shadow(pgd_t *pgd) +static void __init kasan_map_early_shadow(pgd_t *pgd) { int i; unsigned long start = KASAN_SHADOW_START; @@ -166,6 +178,26 @@ static struct notifier_block kasan_die_notifier = { }; #endif +void __init kasan_early_init(void) +{ + int i; + pteval_t pte_val = __pa_nodebug(kasan_zero_page) | __PAGE_KERNEL; + pmdval_t pmd_val = __pa_nodebug(kasan_zero_pte) | _KERNPG_TABLE; + pudval_t pud_val = __pa_nodebug(kasan_zero_pmd) | _KERNPG_TABLE; + + for (i = 0; i < PTRS_PER_PTE; i++) + kasan_zero_pte[i] = __pte(pte_val); + + for (i = 0; i < PTRS_PER_PMD; i++) + kasan_zero_pmd[i] = __pmd(pmd_val); + + for (i = 0; i < PTRS_PER_PUD; i++) + kasan_zero_pud[i] = __pud(pud_val); + + kasan_map_early_shadow(early_level4_pgt); + kasan_map_early_shadow(init_level4_pgt); +} + void __init kasan_init(void) { int i; -- 2.4.4 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/