Received: by 2002:ac0:a5a7:0:0:0:0:0 with SMTP id m36-v6csp8866imm; Tue, 17 Jul 2018 19:26:08 -0700 (PDT) X-Google-Smtp-Source: AAOMgpeIMW5fxu36jKn4jd59DzH8kZ230NXGSrL3IcHii9vcBfiNou5I9nf2WijtNnp4Xzp8JKiv X-Received: by 2002:a63:555:: with SMTP id 82-v6mr3968352pgf.25.1531880768925; Tue, 17 Jul 2018 19:26:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1531880768; cv=none; d=google.com; s=arc-20160816; b=grfss6SGUrSF3G0tKylzEclI/3DXnVNA5yz0XOQ1SZBzw1s1SS48mUevb8e4ormh9/ 6MU9OoyV5xAWxRKGE2i1/ILaM0GPH9/xNJpDC4DAzq7SJAYSMnd0RIGh5yALtfAuMfZC Lia4zfRirI+tSfQ5MVMRrW1ZyF07rDnbPoTJ1Y+oLi566gAr5Z4p7FRYnPlPy6c2zDiq GG/CkVmabrMVp8UJ5A5oeMaUqOHNCPk84t1HPM9hV2RffnBBN8LthINQIU9ZXVDClhRH yu4MHme4jYvISNmLGM5Yxvh/Ugd7/0buKWrIyzuzQXZ4zQkBJFjhGdN/O0hPn3E374p0 de/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=uAuMWI2Ms5FAflkGTyvXqXMrpyGjdu4sicwkzYvF4DY=; b=eD4e5rRlg3L7pOvpAvhRoktGmBa/f5qDJiIBY+3tBHkRv7im0KE87owVot/ThDJhD4 V2+UZJtdOR7/ej3mzCPl33+T+MMVueBJae3aogy/TDQ1rpGKOLzcVqCT/+DYbYcQcS5+ 4mf5zvWJhu1GwB9mDStc+/zYE37Efmd9ZuoUTq+ix7TcEmsLWU9mUW9wQ47KTGCM4PAV 5H20YfSu/wlP5f9gVOKp+JipEmpr+iib2Qlq69BVQH+czuDI108b3fhkd3BOpZXjx7eW ekNuhn6iDhpgShZi9W1GyePIw+3Lnr2N5/fdbLdk4OpZ+1dJbw9SNHcRjsCbBG2tiFeb GD8w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=fIMcmMXu; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id v36-v6si2311459pga.336.2018.07.17.19.25.53; Tue, 17 Jul 2018 19:26:08 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=fIMcmMXu; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731910AbeGRC7n (ORCPT + 99 others); Tue, 17 Jul 2018 22:59:43 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:42484 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731733AbeGRC7m (ORCPT ); Tue, 17 Jul 2018 22:59:42 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w6I2JJ82006858; Wed, 18 Jul 2018 02:22:43 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=uAuMWI2Ms5FAflkGTyvXqXMrpyGjdu4sicwkzYvF4DY=; b=fIMcmMXufVFXNU6p6BQlxvepEHDFN8fbuoke3FWwoe4eyoyqvod+N/c8v+CpbpI2ogw2 QZfRcJyXvrNWPlVIhhdD261l2uNvnPX5xL6aUxLHSjp2LbP7v1JKf/g70NT9hpLMjTAb itWo5ljqARFMVoSyNfCZK8OFvQk90sw62ftnq6qB4j8FypKmQ8+us8nCSZIKnyENDFzD WMleOKpD4FlESY74cHHDR+qUMQn07qcb5Z8ffbMQZaoLpX64taGR1AFiqfEHxlbV+/XO bpeKO3HTPnBCgOTn2KJ5Phe290vizWVNIX7l3czCrG2+V1CiMqlSQy/BKClqo0Nu1FIw UQ== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by aserp2120.oracle.com with ESMTP id 2k7a3432se-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 18 Jul 2018 02:22:43 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id w6I2MgFR023694 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 18 Jul 2018 02:22:43 GMT Received: from abhmp0017.oracle.com (abhmp0017.oracle.com [141.146.116.23]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id w6I2Mfu7020585; Wed, 18 Jul 2018 02:22:41 GMT Received: from localhost.localdomain (/73.69.118.222) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 17 Jul 2018 19:22:41 -0700 From: Pavel Tatashin To: steven.sistare@oracle.com, daniel.m.jordan@oracle.com, linux@armlinux.org.uk, schwidefsky@de.ibm.com, heiko.carstens@de.ibm.com, john.stultz@linaro.org, sboyd@codeaurora.org, x86@kernel.org, linux-kernel@vger.kernel.org, mingo@redhat.com, tglx@linutronix.de, hpa@zytor.com, douly.fnst@cn.fujitsu.com, peterz@infradead.org, prarit@redhat.com, feng.tang@intel.com, pmladek@suse.com, gnomes@lxorguk.ukuu.org.uk, linux-s390@vger.kernel.org, pasha.tatashin@oracle.com, boris.ostrovsky@oracle.com, jgross@suse.com, pbonzini@redhat.com Subject: [PATCH v14 09/25] x86: text_poke() may access uninitialized struct pages Date: Tue, 17 Jul 2018 22:21:55 -0400 Message-Id: <20180718022211.6259-10-pasha.tatashin@oracle.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180718022211.6259-1-pasha.tatashin@oracle.com> References: <20180718022211.6259-1-pasha.tatashin@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=8957 signatures=668706 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1806210000 definitions=main-1807180026 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org It supposed to be safe to modify static branches after jump_label_init(). But, because static key modifying code eventually calls text_poke() we may end up with accessing struct page that have not been initialized. Here is how to quickly reproduce the problem. Insert code like this into init/main.c: | +static DEFINE_STATIC_KEY_FALSE(__test); | asmlinkage __visible void __init start_kernel(void) | { | char *command_line; |@@ -587,6 +609,10 @@ asmlinkage __visible void __init start_kernel(void) | vfs_caches_init_early(); | sort_main_extable(); | trap_init(); |+ { |+ static_branch_enable(&__test); |+ WARN_ON(!static_branch_likely(&__test)); |+ } | mm_init(); The following warnings show-up: WARNING: CPU: 0 PID: 0 at arch/x86/kernel/alternative.c:701 text_poke+0x20d/0x230 Modules linked in: CPU: 0 PID: 0 Comm: swapper Not tainted 4.18.0-rc1_pt_t1 #30 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.11.0-20171110_100015-anatol 04/01/2014 RIP: 0010:text_poke+0x20d/0x230 Code: 0f 0b 4c 89 e2 4c 89 ee 4c 89 f7 e8 7d 4b 9b 00 31 d2 31 f6 bf 86 02 00 00 48 8b 05 95 8e 24 01 e8 78 18 d8 00 e9 55 ff ff ff <0f> 0b e9 54 fe ff ff 48 8b 05 75 a8 38 01 e9 64 fe ff ff 48 8b 1d RSP: 0000:ffffffff94e03e30 EFLAGS: 00010046 RAX: 0100000000000000 RBX: fffff7b2c011f300 RCX: ffffffff94fcccf4 RDX: 0000000000000001 RSI: ffffffff94e03e77 RDI: ffffffff94fcccef RBP: ffffffff94fcccef R08: 00000000fffffe00 R09: 00000000000000a0 R10: 0000000000000000 R11: 0000000000000040 R12: 0000000000000001 R13: ffffffff94e03e77 R14: ffffffff94fcdcef R15: fffff7b2c0000000 FS: 0000000000000000(0000) GS:ffff9adc87c00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: ffff9adc8499d000 CR3: 000000000460a001 CR4: 00000000000606b0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: ? start_kernel+0x23e/0x4c8 ? start_kernel+0x23f/0x4c8 ? text_poke_bp+0x50/0xda ? arch_jump_label_transform+0x89/0xe0 ? __jump_label_update+0x78/0xb0 ? static_key_enable_cpuslocked+0x4d/0x80 ? static_key_enable+0x11/0x20 ? start_kernel+0x23e/0x4c8 ? secondary_startup_64+0xa5/0xb0 ---[ end trace abdc99c031b8a90a ]--- If the code above is moved after mm_init(), no warning is shown, as struct pages are initialized during handover from memblock. Use text_poke_early() in static branching until early boot IRQs are enabled, at which time switch to text_poke. Also, ensure text_poke() is never invoked when unitialized memory access may happen by using: BUG_ON(!after_bootmem); assertion. Signed-off-by: Pavel Tatashin --- arch/x86/include/asm/text-patching.h | 1 + arch/x86/kernel/alternative.c | 7 +++++++ arch/x86/kernel/jump_label.c | 11 +++++++---- 3 files changed, 15 insertions(+), 4 deletions(-) diff --git a/arch/x86/include/asm/text-patching.h b/arch/x86/include/asm/text-patching.h index 2ecd34e2d46c..e85ff65c43c3 100644 --- a/arch/x86/include/asm/text-patching.h +++ b/arch/x86/include/asm/text-patching.h @@ -37,5 +37,6 @@ extern void *text_poke_early(void *addr, const void *opcode, size_t len); extern void *text_poke(void *addr, const void *opcode, size_t len); extern int poke_int3_handler(struct pt_regs *regs); extern void *text_poke_bp(void *addr, const void *opcode, size_t len, void *handler); +extern int after_bootmem; #endif /* _ASM_X86_TEXT_PATCHING_H */ diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index a481763a3776..014f214da581 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -668,6 +668,7 @@ void *__init_or_module text_poke_early(void *addr, const void *opcode, local_irq_save(flags); memcpy(addr, opcode, len); local_irq_restore(flags); + sync_core(); /* Could also do a CLFLUSH here to speed up CPU recovery; but that causes hangs on some VIA CPUs. */ return addr; @@ -693,6 +694,12 @@ void *text_poke(void *addr, const void *opcode, size_t len) struct page *pages[2]; int i; + /* + * While boot memory allocator is runnig we cannot use struct + * pages as they are not yet initialized. + */ + BUG_ON(!after_bootmem); + if (!core_kernel_text((unsigned long)addr)) { pages[0] = vmalloc_to_page(addr); pages[1] = vmalloc_to_page(addr + PAGE_SIZE); diff --git a/arch/x86/kernel/jump_label.c b/arch/x86/kernel/jump_label.c index e56c95be2808..eeea935e9bb5 100644 --- a/arch/x86/kernel/jump_label.c +++ b/arch/x86/kernel/jump_label.c @@ -37,15 +37,18 @@ static void bug_at(unsigned char *ip, int line) BUG(); } -static void __jump_label_transform(struct jump_entry *entry, - enum jump_label_type type, - void *(*poker)(void *, const void *, size_t), - int init) +static void __ref __jump_label_transform(struct jump_entry *entry, + enum jump_label_type type, + void *(*poker)(void *, const void *, size_t), + int init) { union jump_code_union code; const unsigned char default_nop[] = { STATIC_KEY_INIT_NOP }; const unsigned char *ideal_nop = ideal_nops[NOP_ATOMIC5]; + if (early_boot_irqs_disabled) + poker = text_poke_early; + if (type == JUMP_LABEL_JMP) { if (init) { /* -- 2.18.0