Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752965AbbGIA7y (ORCPT ); Wed, 8 Jul 2015 20:59:54 -0400 Received: from mail-lb0-f179.google.com ([209.85.217.179]:33226 "EHLO mail-lb0-f179.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751676AbbGIA7p (ORCPT ); Wed, 8 Jul 2015 20:59:45 -0400 MIME-Version: 1.0 In-Reply-To: <559A9821.4010600@oracle.com> References: <1433752501-15901-1-git-send-email-mingo@kernel.org> <1433752501-15901-5-git-send-email-mingo@kernel.org> <5575954F.5070408@redhat.com> <20150608185106.GA28747@gmail.com> <559A9821.4010600@oracle.com> From: Andy Lutomirski Date: Wed, 8 Jul 2015 17:59:21 -0700 Message-ID: Subject: Re: [PATCH] x86/asm/entry/64: Clean up entry_64.S To: Sasha Levin Cc: Ingo Molnar , Denys Vlasenko , "linux-kernel@vger.kernel.org" , Brian Gerst , Peter Zijlstra , Borislav Petkov , "H. Peter Anvin" , Linus Torvalds , Oleg Nesterov , Thomas Gleixner Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 4671 Lines: 80 Having failed to bisect, let's look at the trace: On Mon, Jul 6, 2015 at 8:00 AM, Sasha Levin wrote: > [3157054.661763] ------------[ cut here ]------------ > [3157054.662552] kernel BUG at arch/x86/kernel/nmi.c:533! > [3157054.663277] invalid opcode: 0000 [#1] PREEMPT SMP KASAN > [3157054.664164] Dumping ftrace buffer: > [3157054.664740] (ftrace buffer empty) > [3157054.665274] Modules linked in: > [3157054.665768] CPU: 16 PID: 11446 Comm: trinity-main Not tainted 4.1.0-next-20150703-sasha-00040-gd868f14-dirty #2292 > [3157054.667203] task: ffff880408813000 ti: ffff8803d29c8000 task.ti: ffff8803d29c8000 > [3157054.668256] RIP: do_nmi (arch/x86/kernel/nmi.c:533 (discriminator 1)) > [3157054.669378] RSP: 0018:ffff88077800bed8 EFLAGS: 00010006 > [3157054.670141] ================================================================== > [3157054.671268] BUG: KASan: out of bounds on stack in __show_regs+0x7f6/0x940 at addr ffff88077800be50 I bet that__show_regs interacts poorly with KASan for some reason. But that's not the underlying bug. In fact, the bad read is quite close the RSP, so this is almost certainly a bug in KASan or __show_regs. > [3157054.674604] Read of size 8 by task trinity-main/11446 > [3157054.676521] page:ffffea001de002c0 count:1 mapcount:0 mapping: (null) index:0x0 > [3157054.679451] flags: 0x42fffff80000400(reserved) > [3157054.681237] page dumped because: kasan: bad access detected > [3157054.683326] CPU: 16 PID: 11446 Comm: trinity-main Not tainted 4.1.0-next-20150703-sasha-00040-gd868f14-dirty #2292 > [3157054.687097] ffff88077800be50 000000009c65e33f ffff88077800b9f8 ffffffffa0ac8938 > [3157054.690303] 1ffffd4003bc0058 ffff88077800ba88 ffff88077800ba78 ffffffff9759796e > [3157054.693365] ffff88077800bab8 ffffffffa0abe0b3 0000000000000082 ffffffffa2fe39e4 > [3157054.696209] Call Trace: > [3157054.697180] dump_stack (lib/dump_stack.c:52) > [3157054.699390] kasan_report_error (mm/kasan/report.c:132 mm/kasan/report.c:193) > [3157054.701663] ? printk (kernel/printk/printk.c:1896) > [3157054.703531] ? bitmap_weight (include/linux/bitmap.h:303) > [3157054.705553] __asan_report_load8_noabort (mm/kasan/report.c:230 mm/kasan/report.c:251) > [3157054.708014] ? __show_regs (arch/x86/kernel/process_64.c:68) > [3157054.710046] __show_regs (arch/x86/kernel/process_64.c:68) > [3157054.712066] ? printk (kernel/printk/printk.c:1896) > [3157054.713878] ? bitmap_weight (include/linux/bitmap.h:303) > [3157054.715875] ? start_thread_common.constprop.0 (arch/x86/kernel/process_64.c:58) > [3157054.718336] ? dump_stack_print_info (kernel/printk/printk.c:3121) > [3157054.720619] show_regs (arch/x86/kernel/dumpstack_64.c:313) > [3157054.722530] __die (arch/x86/kernel/dumpstack.c:294) > [3157054.724290] die (arch/x86/kernel/dumpstack.c:316) > [3157054.725962] do_trap (arch/x86/kernel/traps.c:214 arch/x86/kernel/traps.c:260) > [3157054.727805] do_error_trap (arch/x86/kernel/traps.c:298 include/linux/jump_label.h:125 include/linux/context_tracking_state.h:29 include/linux/context_tracking.h:46 arch/x86/kernel/traps.c:302) > [3157054.729843] ? do_device_not_available (arch/x86/kernel/traps.c:291) > [3157054.732211] ? do_nmi (arch/x86/kernel/nmi.c:533 (discriminator 1)) > [3157054.734101] ? kvm_clock_read (./arch/x86/include/asm/preempt.h:87 arch/x86/kernel/kvmclock.c:86) > [3157054.736165] ? sched_clock (arch/x86/kernel/tsc.c:305) > [3157054.738126] ? nmi_handle (arch/x86/kernel/nmi.c:134) > [3157054.740133] ? trace_hardirqs_off_thunk (arch/x86/entry/thunk_64.S:40) > [3157054.742997] do_invalid_op (arch/x86/kernel/traps.c:313) > [3157054.744991] invalid_op (arch/x86/entry/entry_64.S:925) So we got #UD somewhere... > [3157054.746873] ? do_nmi (arch/x86/kernel/nmi.c:533 (discriminator 1)) > [3157054.748769] ? do_nmi (arch/x86/kernel/nmi.c:515 arch/x86/kernel/nmi.c:531) > [3157054.750658] end_repeat_nmi (arch/x86/entry/entry_64.S:1435) ...here, perhaps? Do you know what line 1435 was in the version you tested? There shouldn't be funny instructions in end_repeat_nmi, though. Did we end up off an instruction boundary? Here's my wild guess. The repeat_nmi thing is really rare. What if there's a CPU or emulator that can't do mov %cr2, %r12 or vice versa? mov from cr has a somewhat unusual encoding. What platform is this? Does KASan play games that would cause KVM to emulate a mov to or from cr2? --Andy -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/