Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Content-Type: text/plain;
        charset=us-ascii
Mime-Version: 1.0 (1.0)
Subject: Re: [crash] PANIC: double fault, error_code: 0x0
From:   Andy Lutomirski <luto@amacapital.net>
In-Reply-To: <20171124220934.q7ovq4yzaihevqls@gmail.com>
Date:   Fri, 24 Nov 2017 15:35:43 -0700
Cc:     linux-kernel@vger.kernel.org, Andy Lutomirski <luto@kernel.org>,
        Dave Hansen <dave.hansen@linux.intel.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        "H . Peter Anvin" <hpa@zytor.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Borislav Petkov <bp@alien8.de>,
        Linus Torvalds <torvalds@linux-foundation.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <464B14E7-EC38-4A5A-8BF6-B086F437C6D1@amacapital.net>
References: <20171124172411.19476-1-mingo@kernel.org> <20171124202237.oytdkqq25s3ak2ul@gmail.com> <20171124220934.q7ovq4yzaihevqls@gmail.com>
To:     Ingo Molnar <mingo@kernel.org>
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk


> On Nov 24, 2017, at 3:09 PM, Ingo Molnar <mingo@kernel.org> wrote:
>=20
>=20
> * Ingo Molnar <mingo@kernel.org> wrote:
>=20
>>=20
>> * Ingo Molnar <mingo@kernel.org> wrote:
>>=20
>>> This is a repost of the latest entry-stack plus Kaiser bits from Andy Lu=
tomirski
>>> (v3 series from today) and Dave Hansen (kaiser-414-tipwip-20171123 versi=
on),
>>> on top of latest tip:x86/urgent (12a78d43de76).
>>>=20
>>> This version is pretty well tested, at least on the usual x86 tree test s=
ystems.
>>> It has a couple of merge mistakes fixed, the biggest difference is in pa=
tch #22:
>>>=20
>>>  x86/mm/kaiser: Prepare assembly for entry/exit CR3 switching
>>>=20
>>> The other patches are identical or very close to what I posted earlier t=
oday.
>>=20
>> Here's a new bug, on a testsystem I get the double fault boot crash attac=
hed=20
>> below. The same bzImage crashes on other systems as well, so it's not CPU=
=20
>> dependent.
>>=20
>> Via Kconfig-bisection I have narrowed it down to the following .config de=
tail:=20
>> it's triggered by _disabling_ CONFIG_DEBUG_ENTRY and enabling CONFIG_KAIS=
ER=3Dy.
>>=20
>> I.e. one of the sanity checks of CONFIG_DEBUG_ENTRY has some positive sid=
e effect.=20
>> I'll try to track down which one it is - any ideas meanwhile?
>>=20
>> Thanks,
>>=20
>>    Ingo
>>=20
>> [    8.797733] calling  pt_dump_init+0x0/0x3b @ 1
>> [    8.803144] initcall pt_dump_init+0x0/0x3b returned 0 after 1 usecs
>> [    8.810589] calling  aes_init+0x0/0x11 @ 1
>> [    8.815757] initcall aes_init+0x0/0x11 returned 0 after 141 usecs
>> [    8.823020] calling  ghash_pclmulqdqni_mod_init+0x0/0x54 @ 1
>> [    8.831002] PANIC: double fault, error_code: 0x0
>> [    8.831002] CPU: 11 PID: 260 Comm: modprobe Not tainted 4.14.0-01419-g=
1b46550a680d-dirty #17
>> [    8.831002] Hardware name: Intel Corporation S2600GZ/S2600GZ, BIOS SE5=
C600.86B.02.02.0002.122320131210 12/23/2013
>> [    8.831002] task: ffff880828ba8000 task.stack: ffffc90004444000
>> [    8.831002] RIP: 0010:page_fault+0x11/0x60
>> [    8.831002] RSP: 0000:ffffffffff0e7fc8 EFLAGS: 00010046
>> [    8.831002] RAX: 00000000819d4d77 RBX: 0000000000000001 RCX: ffffffff8=
19d4d77
>=20
> After much more debugging, the patch below 'fixes' the crash as well, when=
=20
> CONFIG_DEBUG_ENTRY is disabled.
>=20
> Note that if *any* of those 4 padding sequences is removed, the kernel sta=
rts=20
> crashing again. Also note that the exact size of the padding appears to be=
 not=20
> material - it could be larger as well, i.e. it's not an alignment bug I th=
ink.
>=20
> In any case it's not a problem in the actual assembly code paths itself it=
=20
> appears.
>=20
> One guess would be tha it's some sort of sizing bug: maybe the padding for=
ces a=20
> key piece of data or code on another page - but I'm too tired to root caus=
e it=20
> right now.
>=20
> Any ideas?

This smells like a pagerable setup bug. Either the pagetables are a bit brok=
en or they're totally busted and the passing gets something in a more TLB-fr=
iendly place.

>=20
> Thanks,
>=20
>    Ingo
> ---
> arch/x86/entry/entry_64.S | 8 ++++++++
> 1 file changed, 8 insertions(+)
>=20
> diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S
> index 4ac952080869..e83029892017 100644
> --- a/arch/x86/entry/entry_64.S
> +++ b/arch/x86/entry/entry_64.S
> @@ -547,6 +547,8 @@ END(irq_entries_start)
>    ud2
> .Lokay_\@:
>    addq $8, %rsp
> +#else
> +    .rep 16; nop; .endr
> #endif
> .endm
>=20
> @@ -597,6 +599,8 @@ END(irq_entries_start)
>    je    .Lirq_stack_okay\@
>    ud2
>    .Lirq_stack_okay\@:
> +#else
> +    .rep 16; nop; .endr
> #endif
>=20
> .Lirq_stack_push_old_rsp_\@:
> @@ -707,6 +711,8 @@ GLOBAL(swapgs_restore_regs_and_return_to_usermode)
>    jnz    1f
>    ud2
> 1:
> +#else
> +    .rep 16; nop; .endr
> #endif
>    POP_EXTRA_REGS
>    popq    %r11
> @@ -773,6 +779,8 @@ GLOBAL(restore_regs_and_return_to_kernel)
>    jz    1f
>    ud2
> 1:
> +#else
> +    .rep 16; nop; .endr
> #endif
>    POP_EXTRA_REGS
>    POP_C_REGS

From 1584987037405793264@xxx Fri Nov 24 22:10:29 +0000 2017
X-GM-THRID: 1584980346327414425
X-Gmail-Labels: Inbox,Category Forums,HistoricalUnread