Received: by 10.192.165.148 with SMTP id m20csp590455imm; Wed, 2 May 2018 05:46:02 -0700 (PDT) X-Google-Smtp-Source: AB8JxZq7zwDJtDyerDVJHTsa3BKvRzPRZaGHLLuL+fBguSNKim5Dxyr/uA37YLXB9CRCkE3lzUND X-Received: by 2002:a17:902:a585:: with SMTP id az5-v6mr13804083plb.79.1525265162535; Wed, 02 May 2018 05:46:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1525265162; cv=none; d=google.com; s=arc-20160816; b=KIlJTAE7bc0nf9Lq1F3aU6+yt13+V6s/ZRUxmxG/rWnuSUcWw4Huw9G6lQB63PlPBj gF3afe2Bu8bvnU80pq86yFMuok5lIQrZU1sdgPMJ5G2L6J66ZchNeQj0zovF/xFTHcsj 9oQbvGmJwtQDeTCnrqfnmIyNa2qFdxnI4lup+jPRhEQhAuASg+n+Yvd9o0jdeStIYCnC le1WTv3SaO8ipeLZvIjffRVBAJ2VxFn2obLpwUmc9KGtnOQQ4dnR5pYcQ5atoqDbyPpp FeD3nijmNZ3GFEBWRhggn0+PAnQq9AguPuw1bZeB6aMqJ0b/g/CnPQIqu4ckAkI3zgyG DKZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:user-agent:in-reply-to :content-disposition:mime-version:references:message-id:subject:cc :to:from:date:arc-authentication-results; bh=0A1uo88ndRuB1x899MvZpzDvzKcq2jz4zL8gWfXVdYE=; b=JpgO1O72WzTqjFxl+8u5Z+Zlv1mZPR5soppV7jCAwdXaJMhHUebOdJKhEdO0QVFc2q rA4IjMm7aSgYnJhh8Uh7RnIGAcZaqiTYCzM1IXlZr4ddvY7v2jFn6iHxbb1gybGi+QwS EflDF4Fs97JenUECUvfqd6ZmhgntnFWB+iTN70liXKz5CZ2TuT+MyQk35wGtmCzf4ikp k1wV5+F/a2I+oNFAca1A+SkxD9PH5NIroab7+FxqgnSZ9wspl8rEquSj6xgPdBoIE+FJ 6rnFsuZPB6r+ycR9WJLsj76Th169uwlX6lpPWPUY0zov4nvAc10mdp/ys44r4efkGfDd sk8g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id z187-v6si9725689pgd.646.2018.05.02.05.45.48; Wed, 02 May 2018 05:46:02 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751246AbeEBMoY (ORCPT + 99 others); Wed, 2 May 2018 08:44:24 -0400 Received: from mga12.intel.com ([192.55.52.136]:11794 "EHLO mga12.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751020AbeEBMoW (ORCPT ); Wed, 2 May 2018 08:44:22 -0400 X-Amp-Result: UNKNOWN X-Amp-Original-Verdict: FILE UNKNOWN X-Amp-File-Uploaded: False Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 02 May 2018 05:44:22 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.49,354,1520924400"; d="scan'208";a="46175470" Received: from jshi-mobl1.ccr.corp.intel.com (HELO wfg-t540p.sh.intel.com) ([10.254.210.216]) by FMSMGA003.fm.intel.com with ESMTP; 02 May 2018 05:44:19 -0700 Received: from wfg by wfg-t540p.sh.intel.com with local (Exim 4.89) (envelope-from ) id 1fDr7J-00073o-VF; Wed, 02 May 2018 20:44:17 +0800 Date: Wed, 2 May 2018 20:44:17 +0800 From: Fengguang Wu To: Dennis Zhou Cc: Pavel Tatashin , linux-mm@kvack.org, Tejun Heo , Christoph Lameter , Linus Torvalds , Josef Bacik , linux-kernel@vger.kernel.org, lkp@01.org Subject: Re: [per_cpu_ptr_to_phys] PANIC: early exception 0x0d IP 10:ffffffffa892f15f error 0 cr2 0xffff88001fbff000 Message-ID: <20180502124417.du2ytsnrulevihp4@wfg-t540p.sh.intel.com> References: <20180418135300.inazvpxjxowogyge@wfg-t540p.sh.intel.com> <20180418135553.zvw3loh52gbr7e2b@wfg-t540p.sh.intel.com> <20180418233825.GA33106@big-sky.local> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: <20180418233825.GA33106@big-sky.local> User-Agent: NeoMutt/20170609 (1.8.3) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi all, On Wed, Apr 18, 2018 at 06:38:25PM -0500, Dennis Zhou wrote: >Hi, > >On Wed, Apr 18, 2018 at 09:55:53PM +0800, Fengguang Wu wrote: >> >> Hello, >> >> FYI here is a slightly different boot error in mainline kernel 4.17.0-rc1. >> It also dates back to v4.16 . Now I find 2 more occurrances in v4.15 kernel. Here are the statistics: kernel count error-id v4.15: 2 RIP:per_cpu_ptr_to_phys v4.16: 12 RIP:per_cpu_ptr_to_phys v4.16: 1 BUG:KASAN:null-ptr-deref-in-per_cpu_ptr_to_phys v4.16-rc7: 2 RIP:per_cpu_ptr_to_phys v4.17-rc1: 217 RIP:per_cpu_ptr_to_phys v4.17-rc1: 5 BUG:KASAN:null-ptr-deref-in-per_cpu_ptr_to_phys v4.17-rc2: 46 RIP:per_cpu_ptr_to_phys v4.17-rc2: 15 BUG:KASAN:null-ptr-deref-in-per_cpu_ptr_to_phys v4.17-rc3: 12 RIP:per_cpu_ptr_to_phys >> It occurs in 4 out of 4 boots. >> >> [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 128873 >> [ 0.000000] Kernel command line: root=/dev/ram0 hung_task_panic=1 debug apic=debug sysrq_always_enabled rcupdate.rcu_cpu_stall_timeout=100 net.ifnames=0 printk.devkmsg=on panic=-1 softlockup_panic=1 nmi_watchdog=panic oops=panic load_ramdisk=2 prompt_ramdisk=0 drbd.minor_count=8 systemd.log_level=err ignore_loglevel console=tty0 earlyprintk=ttyS0,115200 console=ttyS0,115200 vga=normal rw link=/kbuild-tests/run-queue/kvm/x86_64-randconfig-a0-04172313/linux-devel:devel-hourly-2018041714:60cc43fc888428bb2f18f08997432d426a243338/.vmlinuz-60cc43fc888428bb2f18f08997432d426a243338-20180418000325-19:yocto-lkp-nhm-dp2-4 branch=linux-devel/devel-hourly-2018041714 BOOT_IMAGE=/pkg/linux/x86_64-randconfig-a0-04172313/gcc-7/60cc43fc888428bb2f18f08997432d426a243338/vmlinuz-4.17.0-rc1 drbd.minor_count=8 rcuperf.shutdown=0 >> [ 0.000000] sysrq: sysrq always enabled. >> [ 0.000000] Dentry cache hash table entries: 65536 (order: 7, 524288 bytes) >> [ 0.000000] Inode-cache hash table entries: 32768 (order: 6, 262144 bytes) >> PANIC: early exception 0x0d IP 10:ffffffffa892f15f error 0 cr2 0xffff88001fbff000 >> [ 0.000000] CPU: 0 PID: 0 Comm: swapper Tainted: G T 4.17.0-rc1 #238 >> [ 0.000000] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014 >> [ 0.000000] RIP: 0010:per_cpu_ptr_to_phys+0x16a/0x298: >> __section_mem_map_addr at include/linux/mmzone.h:1188 >> (inlined by) per_cpu_ptr_to_phys at mm/percpu.c:1849 >> [ 0.000000] RSP: 0000:ffffffffab407e50 EFLAGS: 00010046 ORIG_RAX: 0000000000000000 >> [ 0.000000] RAX: dffffc0000000000 RBX: ffff88001f17c340 RCX: 000000000000000f >> [ 0.000000] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffffffffacfbf580 >> [ 0.000000] RBP: ffffffffab40d000 R08: fffffbfff57c4eca R09: 0000000000000000 >> [ 0.000000] R10: ffff880015421000 R11: fffffbfff57c4ec9 R12: 0000000000000000 >> [ 0.000000] R13: ffff88001fb03ff8 R14: ffff88001fc051c0 R15: 0000000000000000 >> [ 0.000000] FS: 0000000000000000(0000) GS:ffffffffab4c5000(0000) knlGS:0000000000000000 >> [ 0.000000] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 >> [ 0.000000] CR2: ffff88001fbff000 CR3: 000000001a06c000 CR4: 00000000000006b0 >> [ 0.000000] Call Trace: >> [ 0.000000] setup_cpu_entry_areas+0x7b/0x27b: >> setup_cpu_entry_area at arch/x86/mm/cpu_entry_area.c:104 >> (inlined by) setup_cpu_entry_areas at arch/x86/mm/cpu_entry_area.c:177 >> [ 0.000000] trap_init+0xb/0x13d: >> trap_init at arch/x86/kernel/traps.c:949 >> [ 0.000000] start_kernel+0x2a5/0x91d: >> mm_init at init/main.c:519 >> (inlined by) start_kernel at init/main.c:589 >> [ 0.000000] ? thread_stack_cache_init+0x6/0x6 >> [ 0.000000] ? memcpy_orig+0x16/0x110: >> memcpy_orig at arch/x86/lib/memcpy_64.S:77 >> [ 0.000000] ? x86_family+0x5/0x1d: >> x86_family at arch/x86/lib/cpu.c:8 >> [ 0.000000] ? load_ucode_bsp+0x42/0x13e: >> load_ucode_bsp at arch/x86/kernel/cpu/microcode/core.c:183 >> [ 0.000000] secondary_startup_64+0xa5/0xb0: >> secondary_startup_64 at arch/x86/kernel/head_64.S:242 >> [ 0.000000] Code: 78 06 00 49 8b 45 00 48 85 c0 74 a5 49 c1 ec 28 41 81 e4 e0 0f 00 00 49 01 c4 4c 89 e2 48 b8 00 00 00 00 00 fc ff df 48 c1 ea 03 <80> 3c 02 00 74 08 4c 89 e7 e8 63 78 06 00 49 8b 04 24 81 e5 ff >> BUG: kernel hang in boot stage >> > >I spent some time bisecting this one and it seemse to be an intermittent >issue starting with this commit for me: >c9e97a1997, mm: initialize pages on demand during boot. The prior >commit, 3a2d7fa8a3, did not run into this issue after 10+ boots. That commit is post-4.16, so probably not the root cause. >I don't have that much time right now, nor the expertise with this code. >Pavel could you take a look at this? Thanks, Fengguang