Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Wed, 29 Aug 2018 14:12:12 -0700
From:   Guenter Roeck <linux@roeck-us.net>
To:     Dave Hansen <dave.hansen@intel.com>
Cc:     linux-kernel@vger.kernel.org, Thomas Gleixner <tglx@linutronix.de>,
        Michal Hocko <mhocko@suse.com>,
        Andi Kleen <ak@linux.intel.com>,
        Linus Torvalds <torvalds@linux-foundation.org>, x86@kernel.org,
        Joerg Roedel <jroedel@suse.de>, Pavel Machek <pavel@ucw.cz>
Subject: Re: efi boot failures due to PTI with 32 bit builds and Intel CPUs
Message-ID: <20180829211212.GA18036@roeck-us.net>
References: <20180829201659.GA29801@roeck-us.net>
 <cd62e6b9-aec4-b8a5-be71-7edc10734955@intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <cd62e6b9-aec4-b8a5-be71-7edc10734955@intel.com>
User-Agent: Mutt/1.5.24 (2015-08-30)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Wed, Aug 29, 2018 at 01:28:16PM -0700, Dave Hansen wrote:
> On 08/29/2018 01:16 PM, Guenter Roeck wrote:
> > 
> > I see boot failures on mainline when trying to boot x86 images with an efi
> > bios on Intel CPUs in qemu. Behavior is quite unusual: qemu dies silently
> > after the kernel displays "Run /sbin/init as init process". With debugging
> > enabled, qemu reports a CR3 update followed by a triple fault.
> 
> My first thought would be that the EFI pgd is broken somehow.
> 
> Is 0e39b000 in your kernel binary, or was it dynamically allocated?
> 
No idea. In the log below (taken after I recompiled with the latest upstream
kernel) the CR3 value is completely different. Where is the value expected
to come from ?

> What was CR2 when things went bad?  Could you just share a full register
> dump?

Here you are. I have a complete log file, but its size is about 1.7GB
(21MB compressed). Let me know if you need it, and I'll publish it
somewhere.

Thanks,
Guenter

---
0xce1f1cd9:  66 90                    nop      
0xce1f1cdb:  8b 44 24 38              movl     0x38(%esp), %eax
0xce1f1cdf:  8a 64 24 40              movb     0x40(%esp), %ah
0xce1f1ce3:  8a 44 24 34              movb     0x34(%esp), %al
0xce1f1ce7:  25 03 04 02 00           andl     $0x20403, %eax
0xce1f1cec:  3d 03 04 00 00           cmpl     $0x403, %eax
0xce1f1cf1:  75 27                    jne      0xce1f1d1a

EAX=f60b8000 EBX=ff8020bc ECX=00000000 EDX=00000000
ESI=f60c1ff8 EDI=ff802100 EBP=00000000 ESP=f60c1fb4
EIP=ce1f1cd7 EFL=00200006 [-----P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =007b 00000000 ffffffff 00cff300 DPL=3 DS   [-WA]
CS =0060 00000000 ffffffff 00cf9a00 DPL=0 CS32 [-R-]
SS =0068 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
DS =007b 00000000 ffffffff 00cff300 DPL=3 DS   [-WA]
FS =00d8 2819b000 ffffffff 008f9300 DPL=0 DS16 [-WA]
GS =00e0 f67f29c0 00000018 00409100 DPL=0 DS   [--A]
LDT=0000 00000000 00000000 00008200 DPL=0 LDT
TR =0080 ff803000 0000206b 00008900 DPL=0 TSS32-avl
GDT=     f67e2000 000000ff
IDT=     ff800000 000007ff
CR0=80050033 CR2=b7f30854 CR3=35402000 CR4=000006d0
DR0=00000000 DR1=00000000 DR2=00000000 DR3=00000000 
DR6=ffff0ff0 DR7=00000400
CCS=00000022 CCD=00000011 CCO=SARL    
EFER=0000000000000000
----------------
IN: 
0xce1f1d1a:  66 90                    nop      
0xce1f1d1c:  0f 20 d8                 movl     %cr3, %eax
0xce1f1d1f:  0d 00 10 00 00           orl      $0x1000, %eax
0xce1f1d24:  0f 22 d8                 movl     %eax, %cr3

EAX=00000003 EBX=ff8020bc ECX=00000000 EDX=00000000
ESI=f60c1ff8 EDI=ff802100 EBP=00000000 ESP=ff8020bc
EIP=ce1f1d1a EFL=00200087 [--S--PC] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =007b 00000000 ffffffff 00cff300 DPL=3 DS   [-WA]
CS =0060 00000000 ffffffff 00cf9a00 DPL=0 CS32 [-R-]
SS =0068 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
DS =007b 00000000 ffffffff 00cff300 DPL=3 DS   [-WA]
FS =00d8 2819b000 ffffffff 008f9300 DPL=0 DS16 [-WA]
GS =00e0 f67f29c0 00000018 00409100 DPL=0 DS   [--A]
LDT=0000 00000000 00000000 00008200 DPL=0 LDT
TR =0080 ff803000 0000206b 00008900 DPL=0 TSS32-avl
GDT=     f67e2000 000000ff
IDT=     ff800000 000007ff
CR0=80050033 CR2=b7f30854 CR3=35402000 CR4=000006d0
DR0=00000000 DR1=00000000 DR2=00000000 DR3=00000000 
DR6=ffff0ff0 DR7=00000400
CCS=00000403 CCD=fffffc00 CCO=SUBL    
EFER=0000000000000000
CR3 update: CR3=35403000
----------------
IN: 
0xce1f1d27:  5b                       popl     %ebx
0xce1f1d28:  59                       popl     %ecx
0xce1f1d29:  5a                       popl     %edx
0xce1f1d2a:  5e                       popl     %esi
0xce1f1d2b:  5f                       popl     %edi
0xce1f1d2c:  5d                       popl     %ebp
0xce1f1d2d:  58                       popl     %eax
0xce1f1d2e:  1f                       popl     %ds

EAX=35403000 EBX=ff8020bc ECX=00000000 EDX=00000000
ESI=f60c1ff8 EDI=ff802100 EBP=00000000 ESP=ff8020bc
EIP=ce1f1d27 EFL=00200006 [-----P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =007b 00000000 ffffffff 00cff300 DPL=3 DS   [-WA]
CS =0060 00000000 ffffffff 00cf9a00 DPL=0 CS32 [-R-]
SS =0068 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
DS =007b 00000000 ffffffff 00cff300 DPL=3 DS   [-WA]
FS =00d8 2819b000 ffffffff 008f9300 DPL=0 DS16 [-WA]
GS =00e0 f67f29c0 00000018 00409100 DPL=0 DS   [--A]
LDT=0000 00000000 00000000 00008200 DPL=0 LDT
TR =0080 ff803000 0000206b 00008900 DPL=0 TSS32-avl
GDT=     f67e2000 000000ff
IDT=     ff800000 000007ff
CR0=80050033 CR2=b7f30854 CR3=35403000 CR4=000006d0
DR0=00000000 DR1=00000000 DR2=00000000 DR3=00000000 
DR6=ffff0ff0 DR7=00000400
CCS=00000403 CCD=35403000 CCO=LOGICL  
EFER=0000000000000000
EAX=35403000 EBX=ff8020bc ECX=00000000 EDX=00000000
ESI=f60c1ff8 EDI=ff802100 EBP=00000000 ESP=ff8020bc
EIP=ce1f1d27 EFL=00200006 [-----P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =007b 00000000 ffffffff 00cff300 DPL=3 DS   [-WA]
CS =0060 00000000 ffffffff 00cf9a00 DPL=0 CS32 [-R-]
SS =0068 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
DS =007b 00000000 ffffffff 00cff300 DPL=3 DS   [-WA]
FS =00d8 2819b000 ffffffff 008f9300 DPL=0 DS16 [-WA]
GS =00e0 f67f29c0 00000018 00409100 DPL=0 DS   [--A]
LDT=0000 00000000 00000000 00008200 DPL=0 LDT
TR =0080 ff803000 0000206b 00008900 DPL=0 TSS32-avl
GDT=     f67e2000 000000ff
IDT=     ff800000 000007ff
CR0=80050033 CR2=b7f30854 CR3=35403000 CR4=000006d0
DR0=00000000 DR1=00000000 DR2=00000000 DR3=00000000 
DR6=ffff0ff0 DR7=00000400
CCS=00000004 CCD=35403000 CCO=EFLAGS  
EFER=0000000000000000
Triple fault