Linux crashes on boot on SGI UV systems. git bisect tracked it
down to commit cc5a080c5d40c36089bb08a8a16fa3fc7047fe0f. Undoing
that patch fixes the problem.
http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=cc5a080c5d40c36089bb08a8a16fa3fc7047fe0f
-----------------------------------------------------------
linux$ git bisect bad
cc5a080c5d40c36089bb08a8a16fa3fc7047fe0f is first bad commit
commit cc5a080c5d40c36089bb08a8a16fa3fc7047fe0f
Author: Matthew Garrett <[email protected]>
Date: Mon Apr 15 13:09:46 2013 -0700
efi: Pass boot services variable info to runtime code
EFI variables can be flagged as being accessible only within boot services.
This makes it awkward for us to figure out how much space they use at
runtime. In theory we could figure this out by simply comparing the results
from QueryVariableInfo() to the space used by all of our variables, but
that fails if the platform doesn't garbage collect on every boot. Thankfully,
calling QueryVariableInfo() while still inside boot services gives a more
reliable answer. This patch passes that information from the EFI boot stub
up to the efi platform code.
Signed-off-by: Matthew Garrett <[email protected]>
Signed-off-by: Matt Fleming <[email protected]>
:040000 040000 eaaca954c158017931b383d8a7799f0372118fa4 5a2816a34b5393ca1ad49a9ce240e2e5caee3aca M arch
-----------------------------------------------------------
The failing output:
-----------------------------------------------------------
[ 6.038007] Intel P-state driver initializing.
[ 6.043006] Intel pstate controlling: cpu 0
[ 6.047700] Intel pstate controlling: cpu 1
[ 6.052456] cpuidle: using governor ladder
[ 6.057085] cpuidle: using governor menu
[ 6.062157] EFI Variables Facility v0.08 2004-May-17
[ 6.067731] BUG: unable to handle kernel paging request at 000000007ca95b10
[ 6.075519] IP: [<ffff88007dbf2140>] 0xffff88007dbf213f
[ 6.081363] PGD 0
[ 6.083616] Oops: 0000 [#1] SMP
[ 6.087240] Modules linked in:
[ 6.090656] CPU 1
[ 6.092706] Pid: 1, comm: swapper/0 Not tainted 3.9.0-0.55.el7.x86_64 #1 SGI UV2000/ROMLEY
[ 6.102129] RIP: 0010:[<ffff88007dbf2140>] [<ffff88007dbf2140>] 0xffff88007dbf213f
[ 6.110681] RSP: 0000:ffff880815913990 EFLAGS: 00010202
[ 6.116603] RAX: 000000007ca95ad0 RBX: 000000000000006a RCX: 0000000090000002
[ 6.124562] RDX: 0000000003050007 RSI: 0000000000000000 RDI: ffff88007dbf6ea8
[ 6.132520] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
[ 6.140478] R10: 0000000000003638 R11: ffff880815913b8e R12: 0000000003050007
[ 6.148436] R13: 0000000090000002 R14: ffff880815913e00 R15: 0000000000000000
[ 6.156394] FS: 0000000000000000(0000) GS:ffff8817bec00000(0000) knlGS:0000000000000000
[ 6.165418] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 6.171826] CR2: 000000007ca95b10 CR3: 00000000018e6000 CR4: 00000000000407e0
[ 6.179785] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 6.187743] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 6.195702] Process swapper/0 (pid: 1, threadinfo ffff880815912000, task ffff880815928000)
[ 6.204919] Stack:
[ 6.207160] 00000000ffffffff 0000000000000010 0000000000000041 0000000000000040
[ 6.215456] ffffea005e447dc0 0000000000000001 002212d000000000 0000000000000246
[ 6.223752] ffff8817befd4d80 0000000000000002 0000000000000000 00000000002012d0
[ 6.232048] Call Trace:
[ 6.234799] [<ffffffff8113c647>] ? __alloc_pages_nodemask+0x157/0xa00
[ 6.242096] [<ffffffff812f02df>] ? idr_get_empty_slot+0x16f/0x3c0
[ 6.248997] [<ffffffff812f7972>] ? put_dec+0x72/0x90
[ 6.254633] [<ffffffff812f076c>] ? ida_get_new_above+0x6c/0x290
[ 6.261340] [<ffffffff8120d0b5>] ? sysfs_link_sibling+0xb5/0xe0
[ 6.268040] [<ffffffff8120d7b1>] ? __sysfs_add_one+0x71/0x110
[ 6.274545] [<ffffffff8120d9d3>] ? sysfs_addrm_finish+0x33/0xc0
[ 6.281245] [<ffffffff8120dadd>] ? create_dir+0x7d/0xd0
[ 6.287171] [<ffffffff8120de72>] ? sysfs_create_dir+0x92/0xf0
[ 6.293690] [<ffffffff81058a23>] ? efi_call3+0x43/0x80
[ 6.299517] [<ffffffff8105832a>] ? virt_efi_get_next_variable+0x3a/0x1a0
[ 6.307095] [<ffffffff814b27e3>] ? register_efivars+0xd3/0x530
[ 6.313710] [<ffffffff81a4d042>] ? dmi_sysfs_register_handle+0x1c7/0x1c7
[ 6.321280] [<ffffffff81a4d0f5>] ? efivars_init+0xb3/0xff
[ 6.327405] [<ffffffff8100210a>] ? do_one_initcall+0x10a/0x160
[ 6.334026] [<ffffffff81a0e064>] ? kernel_init_freeable+0x181/0x207
[ 6.341115] [<ffffffff81a0d887>] ? do_early_param+0x88/0x88
[ 6.347445] [<ffffffff815e0340>] ? rest_init+0x80/0x80
[ 6.353274] [<ffffffff815e034e>] ? kernel_init+0xe/0x180
[ 6.359307] [<ffffffff8160acac>] ? ret_from_fork+0x7c/0xb0
[ 6.365522] [<ffffffff815e0340>] ? rest_init+0x80/0x80
[ 6.371347] Code: 8b d8 e8 e4 fa ff ff 84 c0 75 0f 48 8b 15 61 58 00 00 48 8b 4c 24 30 ff 52 48 48 8b c3 eb 58 48 8b 05 4d 58 00 00 48 85 c0 74 42 <48> 83 78 40 00 74 3b 48 83 78 48 00 74 34 48 8d 53 14 4c 8d 44
[ 6.393044] RIP [<ffff88007dbf2140>] 0xffff88007dbf213f
[ 6.398978] RSP <ffff880815913990>
[ 6.406605] ---[ end trace 18f487bf56d8bf90 ]---
[ 6.411785] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000009
-----------------------------------------------------------
--
Russ Anderson, OS RAS/Partitioning Project Lead
SGI - Silicon Graphics Inc [email protected]
On Wed, 22 May, at 11:27:47AM, Russ Anderson wrote:
> [ 6.062157] EFI Variables Facility v0.08 2004-May-17
> [ 6.067731] BUG: unable to handle kernel paging request at 000000007ca95b10
> [ 6.075519] IP: [<ffff88007dbf2140>] 0xffff88007dbf213f
This is a bit of a head scratcher. Could you paste the EFI memmap
entries in your dmesg for the regions that cover 0x7ca95b10 and
0x7dbf2140? My guess would be that they're EFI runtime code regions,
which would at least explain why we seem to be executing code in the
direct mapping region (0xffff8800xxxxxxxx).
Are you booting via the EFI boot stub?
--
Matt Fleming, Intel Open Source Technology Center
On Thu, May 23, 2013 at 12:58:01PM +0100, Matt Fleming wrote:
> On Wed, 22 May, at 11:27:47AM, Russ Anderson wrote:
> > [ 6.062157] EFI Variables Facility v0.08 2004-May-17
> > [ 6.067731] BUG: unable to handle kernel paging request at 000000007ca95b10
> > [ 6.075519] IP: [<ffff88007dbf2140>] 0xffff88007dbf213f
>
> This is a bit of a head scratcher. Could you paste the EFI memmap
> entries in your dmesg for the regions that cover 0x7ca95b10 and
> 0x7dbf2140? My guess would be that they're EFI runtime code regions,
> which would at least explain why we seem to be executing code in the
> direct mapping region (0xffff8800xxxxxxxx).
Yes. The full e820 and EFI memmap are below.
> Are you booting via the EFI boot stub?
grub2.
FWIW, the EFI related config settings.
gulag1$ grep _EFI .config
CONFIG_EFI_PARTITION=y
CONFIG_EFI=y
CONFIG_EFI_STUB=y
CONFIG_FB_EFI=y
CONFIG_EFI_VARS=y
CONFIG_EFI_VARS_PSTORE=y
CONFIG_EFI_VARS_PSTORE_DEFAULT_DISABLE=y
# CONFIG_EFIVAR_FS is not set
----------------------------------------------------------------------
57.923378 ( 0.023626)| [ 0.000000] e820: BIOS-provided physical RAM map:
57.927705 ( 0.004327)| [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000007efff] usable
57.987450 ( 0.059745)| [ 0.000000] BIOS-e820: [mem 0x000000000007f000-0x000000000007ffff] reserved
57.987563 ( 0.000113)| [ 0.000000] BIOS-e820: [mem 0x0000000000080000-0x000000000009ffff] usable
57.987647 ( 0.000084)| [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007ca9bfff] usable
57.987734 ( 0.000087)| [ 0.000000] BIOS-e820: [mem 0x000000007ca9c000-0x000000007ca9cfff] reserved
57.987827 ( 0.000093)| [ 0.000000] BIOS-e820: [mem 0x000000007ca9d000-0x000000007caaffff] usable
57.987914 ( 0.000087)| [ 0.000000] BIOS-e820: [mem 0x000000007cab0000-0x000000007cceffff] reserved
57.988002 ( 0.000088)| [ 0.000000] BIOS-e820: [mem 0x000000007ccf0000-0x000000007d9fefff] usable
58.003483 ( 0.015481)| [ 0.000000] BIOS-e820: [mem 0x000000007d9ff000-0x000000007dcfefff] reserved
58.003590 ( 0.000107)| [ 0.000000] BIOS-e820: [mem 0x000000007dcff000-0x000000007eefefff] ACPI NVS
58.007785 ( 0.004195)| [ 0.000000] BIOS-e820: [mem 0x000000007eeff000-0x000000007efe9fff] ACPI data
58.015532 ( 0.007747)| [ 0.000000] BIOS-e820: [mem 0x000000007efea000-0x000000007effffff] usable
58.023658 ( 0.008126)| [ 0.000000] BIOS-e820: [mem 0x0000000080000000-0x000000008fffffff] reserved
58.027770 ( 0.004112)| [ 0.000000] BIOS-e820: [mem 0x00000000f8000000-0x00000000fdffffff] reserved
58.035511 ( 0.007741)| [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved
58.043636 ( 0.008125)| [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000083fffffff] usable
58.051382 ( 0.007746)| [ 0.000000] BIOS-e820: [mem 0x0000001000000000-0x00000017beffffff] usable
58.059752 ( 0.008370)| [ 0.000000] BIOS-e820: [mem 0x000000ff00000000-0x000000ff03dfffff] reserved
58.067496 ( 0.007744)| [ 0.000000] BIOS-e820: [mem 0x000000ff03f00000-0x000000ff03ffffff] reserved
58.075619 ( 0.008123)| [ 0.000000] BIOS-e820: [mem 0x000000ff07f00000-0x000000ff07ffffff] reserved
58.083368 ( 0.007749)| [ 0.000000] BIOS-e820: [mem 0x000000ff40000000-0x000000ff42ffffff] reserved
58.091738 ( 0.008370)| [ 0.000000] BIOS-e820: [mem 0x000000ff44000000-0x000000ff46ffffff] reserved
58.099485 ( 0.007747)| [ 0.000000] BIOS-e820: [mem 0x000000ff48000000-0x000000ff4affffff] reserved
58.107609 ( 0.008124)| [ 0.000000] BIOS-e820: [mem 0x000000ff4c000000-0x000000ff4effffff] reserved
58.115358 ( 0.007749)| [ 0.000000] e820: update [mem 0x693c5018-0x693cd057] usable ==> usable
58.123721 ( 0.008363)| [ 0.000000] extended physical RAM map:
58.127414 ( 0.003693)| [ 0.000000] reserve setup_data: [mem 0x0000000000000000-0x000000000007efff] usable
58.135786 ( 0.008372)| [ 0.000000] reserve setup_data: [mem 0x000000000007f000-0x000000000007ffff] reserved
58.143544 ( 0.007758)| [ 0.000000] reserve setup_data: [mem 0x0000000000080000-0x000000000009ffff] usable
58.151681 ( 0.008137)| [ 0.000000] reserve setup_data: [mem 0x0000000000100000-0x00000000693c5017] usable
58.159436 ( 0.007755)| [ 0.000000] reserve setup_data: [mem 0x00000000693c5018-0x00000000693cd057] usable
58.167810 ( 0.008374)| [ 0.000000] reserve setup_data: [mem 0x00000000693cd058-0x000000007ca9bfff] usable
58.175565 ( 0.007755)| [ 0.000000] reserve setup_data: [mem 0x000000007ca9c000-0x000000007ca9cfff] reserved
58.183704 ( 0.008139)| [ 0.000000] reserve setup_data: [mem 0x000000007ca9d000-0x000000007caaffff] usable
58.195472 ( 0.011768)| [ 0.000000] reserve setup_data: [mem 0x000000007cab0000-0x000000007cceffff] reserved
58.203607 ( 0.008135)| [ 0.000000] reserve setup_data: [mem 0x000000007ccf0000-0x000000007d9fefff] usable
58.211363 ( 0.007756)| [ 0.000000] reserve setup_data: [mem 0x000000007d9ff000-0x000000007dcfefff] reserved
58.219746 ( 0.008383)| [ 0.000000] reserve setup_data: [mem 0x000000007dcff000-0x000000007eefefff] ACPI NVS
58.227501 ( 0.007755)| [ 0.000000] reserve setup_data: [mem 0x000000007eeff000-0x000000007efe9fff] ACPI data
58.235636 ( 0.008135)| [ 0.000000] reserve setup_data: [mem 0x000000007efea000-0x000000007effffff] usable
58.243393 ( 0.007757)| [ 0.000000] reserve setup_data: [mem 0x0000000080000000-0x000000008fffffff] reserved
58.255414 ( 0.012021)| [ 0.000000] reserve setup_data: [mem 0x00000000f8000000-0x00000000fdffffff] reserved
58.263360 ( 0.007946)| [ 0.000000] reserve setup_data: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved
58.271798 ( 0.008438)| [ 0.000000] reserve setup_data: [mem 0x0000000100000000-0x000000083fffffff] usable
58.279745 ( 0.007947)| [ 0.000000] reserve setup_data: [mem 0x0000001000000000-0x00000017beffffff] usable
58.287506 ( 0.007761)| [ 0.000000] reserve setup_data: [mem 0x000000ff00000000-0x000000ff03dfffff] reserved
58.295634 ( 0.008128)| [ 0.000000] reserve setup_data: [mem 0x000000ff03f00000-0x000000ff03ffffff] reserved
58.303393 ( 0.007759)| [ 0.000000] reserve setup_data: [mem 0x000000ff07f00000-0x000000ff07ffffff] reserved
58.315415 ( 0.012022)| [ 0.000000] reserve setup_data: [mem 0x000000ff40000000-0x000000ff42ffffff] reserved
58.323792 ( 0.008377)| [ 0.000000] reserve setup_data: [mem 0x000000ff44000000-0x000000ff46ffffff] reserved
58.331548 ( 0.007756)| [ 0.000000] reserve setup_data: [mem 0x000000ff48000000-0x000000ff4affffff] reserved
58.339687 ( 0.008139)| [ 0.000000] reserve setup_data: [mem 0x000000ff4c000000-0x000000ff4effffff] reserved
58.347445 ( 0.007758)| [ 0.000000] NX (Execute Disable) protection: active
58.355780 ( 0.008335)| [ 0.000000] efi: EFI v2.30 by EDK II
58.359602 ( 0.003822)| [ 0.000000] efi: ACPI=0x7efe9000 ACPI 2.0=0x7efe9014 SMBIOS=0x7dcfb000 UVsystab=0x7dab7000
58.367374 ( 0.007772)| [ 0.000000] efi: mem00: type=3, attr=0xf, range=[0x0000000000000000-0x0000000000001000) (0MB)
58.375767 ( 0.008393)| [ 0.000000] efi: mem01: type=2, attr=0xf, range=[0x0000000000001000-0x0000000000004000) (0MB)
58.387792 ( 0.012025)| [ 0.000000] efi: mem02: type=7, attr=0xf, range=[0x0000000000004000-0x000000000005f000) (0MB)
58.395560 ( 0.007768)| [ 0.000000] efi: mem03: type=4, attr=0xf, range=[0x000000000005f000-0x0000000000060000) (0MB)
58.407723 ( 0.012163)| [ 0.000000] efi: mem04: type=3, attr=0xf, range=[0x0000000000060000-0x000000000007f000) (0MB)
58.415490 ( 0.007767)| [ 0.000000] efi: mem05: type=0, attr=0xf, range=[0x000000000007f000-0x0000000000080000) (0MB)
58.424635 ( 0.009145)| [ 0.000000] efi: mem06: type=3, attr=0xf, range=[0x0000000000080000-0x00000000000a0000) (0MB)
58.435668 ( 0.011033)| [ 0.000000] efi: mem07: type=7, attr=0xf, range=[0x0000000000100000-0x0000000000c00000) (11MB)
58.443437 ( 0.007769)| [ 0.000000] efi: mem08: type=3, attr=0xf, range=[0x0000000000c00000-0x0000000001000000) (4MB)
58.451822 ( 0.008385)| [ 0.000000] efi: mem09: type=2, attr=0xf, range=[0x0000000001000000-0x0000000001fab000) (15MB)
58.463366 ( 0.011544)| [ 0.000000] efi: mem10: type=7, attr=0xf, range=[0x0000000001fab000-0x000000003ebb0000) (972MB)
58.471761 ( 0.008395)| [ 0.000000] efi: mem11: type=2, attr=0xf, range=[0x000000003ebb0000-0x0000000040000000) (20MB)
58.484788 ( 0.013027)| [ 0.000000] efi: mem12: type=7, attr=0xf, range=[0x0000000040000000-0x000000004eeb1000) (238MB)
58.494746 ( 0.009958)| [ 0.000000] efi: mem13: type=2, attr=0xf, range=[0x000000004eeb1000-0x0000000069231000) (419MB)
58.513531 ( 0.018785)| [ 0.000000] efi: mem14: type=1, attr=0xf, range=[0x0000000069231000-0x00000000692ff000) (0MB)
58.525416 ( 0.011885)| [ 0.000000] efi: mem15: type=7, attr=0xf, range=[0x00000000692ff000-0x00000000693c5000) (0MB)
58.536698 ( 0.011282)| [ 0.000000] efi: mem16: type=2, attr=0xf, range=[0x00000000693c5000-0x00000000693ce000) (0MB)
58.548787 ( 0.012089)| [ 0.000000] efi: mem17: type=4, attr=0xf, range=[0x00000000693ce000-0x000000006941a000) (0MB)
58.548922 ( 0.000135)| [ 0.000000] efi: mem18: type=7, attr=0xf, range=[0x000000006941a000-0x000000006941c000) (0MB)
58.560687 ( 0.011765)| [ 0.000000] efi: mem19: type=4, attr=0xf, range=[0x000000006941c000-0x0000000069421000) (0MB)
58.572589 ( 0.011902)| [ 0.000000] efi: mem20: type=7, attr=0xf, range=[0x0000000069421000-0x0000000069423000) (0MB)
58.575380 ( 0.002791)| [ 0.000000] efi: mem21: type=4, attr=0xf, range=[0x0000000069423000-0x0000000069424000) (0MB)
58.579662 ( 0.004282)| [ 0.000000] efi: mem22: type=7, attr=0xf, range=[0x0000000069424000-0x0000000069426000) (0MB)
58.587433 ( 0.007771)| [ 0.000000] efi: mem23: type=4, attr=0xf, range=[0x0000000069426000-0x000000006944e000) (0MB)
58.595816 ( 0.008383)| [ 0.000000] efi: mem24: type=2, attr=0xf, range=[0x000000006944e000-0x0000000069453000) (0MB)
58.607358 ( 0.011542)| [ 0.000000] efi: mem25: type=4, attr=0xf, range=[0x0000000069453000-0x0000000069457000) (0MB)
58.615751 ( 0.008393)| [ 0.000000] efi: mem26: type=2, attr=0xf, range=[0x0000000069457000-0x0000000069466000) (0MB)
58.623515 ( 0.007764)| [ 0.000000] efi: mem27: type=4, attr=0xf, range=[0x0000000069466000-0x0000000069523000) (0MB)
58.635545 ( 0.012030)| [ 0.000000] efi: mem28: type=1, attr=0xf, range=[0x0000000069523000-0x00000000695e0000) (0MB)
58.643693 ( 0.008148)| [ 0.000000] efi: mem29: type=4, attr=0xf, range=[0x00000000695e0000-0x00000000695f0000) (0MB)
58.655475 ( 0.011782)| [ 0.000000] efi: mem30: type=2, attr=0xf, range=[0x00000000695f0000-0x00000000695f7000) (0MB)
58.663619 ( 0.008144)| [ 0.000000] efi: mem31: type=4, attr=0xf, range=[0x00000000695f7000-0x00000000695fd000) (0MB)
58.671389 ( 0.007770)| [ 0.000000] efi: mem32: type=3, attr=0xf, range=[0x00000000695fd000-0x0000000069600000) (0MB)
58.683421 ( 0.012032)| [ 0.000000] efi: mem33: type=4, attr=0xf, range=[0x0000000069600000-0x0000000069603000) (0MB)
58.691807 ( 0.008386)| [ 0.000000] efi: mem34: type=3, attr=0xf, range=[0x0000000069603000-0x0000000069617000) (0MB)
58.703346 ( 0.011539)| [ 0.000000] efi: mem35: type=4, attr=0xf, range=[0x0000000069617000-0x0000000069619000) (0MB)
58.711738 ( 0.008392)| [ 0.000000] efi: mem36: type=3, attr=0xf, range=[0x0000000069619000-0x000000006961f000) (0MB)
58.719736 ( 0.007998)| [ 0.000000] efi: mem37: type=4, attr=0xf, range=[0x000000006961f000-0x000000006afdc000) (25MB)
58.731784 ( 0.012048)| [ 0.000000] efi: mem38: type=3, attr=0xf, range=[0x000000006afdc000-0x000000006b045000) (0MB)
58.739553 ( 0.007769)| [ 0.000000] efi: mem39: type=4, attr=0xf, range=[0x000000006b045000-0x000000006b18d000) (1MB)
58.747702 ( 0.008149)| [ 0.000000] efi: mem40: type=3, attr=0xf, range=[0x000000006b18d000-0x000000006b190000) (0MB)
58.759485 ( 0.011783)| [ 0.000000] efi: mem41: type=4, attr=0xf, range=[0x000000006b190000-0x000000006b1ae000) (0MB)
58.767629 ( 0.008144)| [ 0.000000] efi: mem42: type=3, attr=0xf, range=[0x000000006b1ae000-0x000000006b1df000) (0MB)
58.779661 ( 0.012032)| [ 0.000000] efi: mem43: type=4, attr=0xf, range=[0x000000006b1df000-0x000000006b20c000) (0MB)
58.786817 ( 0.007156)| [ 0.000000] efi: mem44: type=3, attr=0xf, range=[0x000000006b20c000-0x000000006b22c000) (0MB)
58.795757 ( 0.008940)| [ 0.000000] efi: mem45: type=4, attr=0xf, range=[0x000000006b22c000-0x000000006b22f000) (0MB)
58.807786 ( 0.012029)| [ 0.000000] efi: mem46: type=3, attr=0xf, range=[0x000000006b22f000-0x000000006b232000) (0MB)
58.815554 ( 0.007768)| [ 0.000000] efi: mem47: type=4, attr=0xf, range=[0x000000006b232000-0x000000006b237000) (0MB)
58.823704 ( 0.008150)| [ 0.000000] efi: mem48: type=3, attr=0xf, range=[0x000000006b237000-0x000000006b242000) (0MB)
58.835485 ( 0.011781)| [ 0.000000] efi: mem49: type=4, attr=0xf, range=[0x000000006b242000-0x000000006b245000) (0MB)
58.843628 ( 0.008143)| [ 0.000000] efi: mem50: type=3, attr=0xf, range=[0x000000006b245000-0x000000006b247000) (0MB)
58.858639 ( 0.015011)| [ 0.000000] efi: mem51: type=4, attr=0xf, range=[0x000000006b247000-0x000000006b24a000) (0MB)
58.863431 ( 0.004792)| [ 0.000000] efi: mem52: type=3, attr=0xf, range=[0x000000006b24a000-0x000000006b25f000) (0MB)
58.872817 ( 0.009386)| [ 0.000000] efi: mem53: type=4, attr=0xf, range=[0x000000006b25f000-0x000000006b263000) (0MB)
58.883357 ( 0.010540)| [ 0.000000] efi: mem54: type=3, attr=0xf, range=[0x000000006b263000-0x000000006b268000) (0MB)
58.891749 ( 0.008392)| [ 0.000000] efi: mem55: type=4, attr=0xf, range=[0x000000006b268000-0x000000006b28b000) (0MB)
58.900516 ( 0.008767)| [ 0.000000] efi: mem56: type=3, attr=0xf, range=[0x000000006b28b000-0x000000006b292000) (0MB)
58.911565 ( 0.011049)| [ 0.000000] efi: mem57: type=4, attr=0xf, range=[0x000000006b292000-0x000000006b293000) (0MB)
58.919706 ( 0.008141)| [ 0.000000] efi: mem58: type=3, attr=0xf, range=[0x000000006b293000-0x000000006b2af000) (0MB)
58.932482 ( 0.012776)| [ 0.000000] efi: mem59: type=4, attr=0xf, range=[0x000000006b2af000-0x000000006b2b0000) (0MB)
58.939601 ( 0.007119)| [ 0.000000] efi: mem60: type=3, attr=0xf, range=[0x000000006b2b0000-0x000000006b2b5000) (0MB)
58.947387 ( 0.007786)| [ 0.000000] efi: mem61: type=4, attr=0xf, range=[0x000000006b2b5000-0x000000006b2b7000) (0MB)
58.959421 ( 0.012034)| [ 0.000000] efi: mem62: type=3, attr=0xf, range=[0x000000006b2b7000-0x000000006b2e6000) (0MB)
58.967806 ( 0.008385)| [ 0.000000] efi: mem63: type=4, attr=0xf, range=[0x000000006b2e6000-0x000000006b2e8000) (0MB)
58.975575 ( 0.007769)| [ 0.000000] efi: mem64: type=3, attr=0xf, range=[0x000000006b2e8000-0x000000006b2ee000) (0MB)
58.987737 ( 0.012162)| [ 0.000000] efi: mem65: type=4, attr=0xf, range=[0x000000006b2ee000-0x000000006b2f0000) (0MB)
58.995506 ( 0.007769)| [ 0.000000] efi: mem66: type=3, attr=0xf, range=[0x000000006b2f0000-0x000000006b304000) (0MB)
59.007534 ( 0.012028)| [ 0.000000] efi: mem67: type=4, attr=0xf, range=[0x000000006b304000-0x000000006b306000) (0MB)
59.015684 ( 0.008150)| [ 0.000000] efi: mem68: type=3, attr=0xf, range=[0x000000006b306000-0x000000006b30c000) (0MB)
59.023451 ( 0.007767)| [ 0.000000] efi: mem69: type=4, attr=0xf, range=[0x000000006b30c000-0x000000006b30d000) (0MB)
59.038608 ( 0.015157)| [ 0.000000] efi: mem70: type=3, attr=0xf, range=[0x000000006b30d000-0x000000006b310000) (0MB)
59.043807 ( 0.005199)| [ 0.000000] efi: mem71: type=4, attr=0xf, range=[0x000000006b310000-0x000000006b311000) (0MB)
59.055345 ( 0.011538)| [ 0.000000] efi: mem72: type=3, attr=0xf, range=[0x000000006b311000-0x000000006b31a000) (0MB)
59.063738 ( 0.008393)| [ 0.000000] efi: mem73: type=4, attr=0xf, range=[0x000000006b31a000-0x000000006b31c000) (0MB)
59.071507 ( 0.007769)| [ 0.000000] efi: mem74: type=3, attr=0xf, range=[0x000000006b31c000-0x000000006b333000) (0MB)
59.097647 ( 0.026140)| [ 0.000000] efi: mem75: type=4, attr=0xf, range=[0x000000006b333000-0x000000006b337000) (0MB)
59.108554 ( 0.010907)| [ 0.000000] efi: mem76: type=3, attr=0xf, range=[0x000000006b337000-0x000000006b345000) (0MB)
59.108685 ( 0.000131)| [ 0.000000] efi: mem77: type=4, attr=0xf, range=[0x000000006b345000-0x000000006b348000) (0MB)
59.120455 ( 0.011770)| [ 0.000000] efi: mem78: type=3, attr=0xf, range=[0x000000006b348000-0x000000006b3bc000) (0MB)
59.132549 ( 0.012094)| [ 0.000000] efi: mem79: type=4, attr=0xf, range=[0x000000006b3bc000-0x000000006b3c6000) (0MB)
59.144828 ( 0.012279)| [ 0.000000] efi: mem80: type=3, attr=0xf, range=[0x000000006b3c6000-0x000000006b3cd000) (0MB)
59.147614 ( 0.002786)| [ 0.000000] efi: mem81: type=4, attr=0xf, range=[0x000000006b3cd000-0x000000006b3ce000) (0MB)
59.147747 ( 0.000133)| [ 0.000000] efi: mem82: type=3, attr=0xf, range=[0x000000006b3ce000-0x000000006b3d9000) (0MB)
59.159667 ( 0.011920)| [ 0.000000] efi: mem83: type=4, attr=0xf, range=[0x000000006b3d9000-0x000000006b3da000) (0MB)
59.167436 ( 0.007769)| [ 0.000000] efi: mem84: type=3, attr=0xf, range=[0x000000006b3da000-0x000000006b3e1000) (0MB)
59.185822 ( 0.018386)| [ 0.000000] efi: mem85: type=4, attr=0xf, range=[0x000000006b3e1000-0x000000006b3e2000) (0MB)
59.187424 ( 0.001602)| [ 0.000000] efi: mem86: type=3, attr=0xf, range=[0x000000006b3e2000-0x000000006b3f4000) (0MB)
59.196755 ( 0.009331)| [ 0.000000] efi: mem87: type=4, attr=0xf, range=[0x000000006b3f4000-0x000000006b3f5000) (0MB)
59.207349 ( 0.010594)| [ 0.000000] efi: mem88: type=3, attr=0xf, range=[0x000000006b3f5000-0x000000006b408000) (0MB)
59.215740 ( 0.008391)| [ 0.000000] efi: mem89: type=4, attr=0xf, range=[0x000000006b408000-0x000000006b40c000) (0MB)
59.223509 ( 0.007769)| [ 0.000000] efi: mem90: type=3, attr=0xf, range=[0x000000006b40c000-0x000000006b40e000) (0MB)
59.235537 ( 0.012028)| [ 0.000000] efi: mem91: type=4, attr=0xf, range=[0x000000006b40e000-0x000000006b411000) (0MB)
59.243686 ( 0.008149)| [ 0.000000] efi: mem92: type=3, attr=0xf, range=[0x000000006b411000-0x000000006b4cf000) (0MB)
59.251454 ( 0.007768)| [ 0.000000] efi: mem93: type=4, attr=0xf, range=[0x000000006b4cf000-0x000000006b4d8000) (0MB)
59.263486 ( 0.012032)| [ 0.000000] efi: mem94: type=3, attr=0xf, range=[0x000000006b4d8000-0x000000006b526000) (0MB)
59.272628 ( 0.009142)| [ 0.000000] efi: mem95: type=4, attr=0xf, range=[0x000000006b526000-0x000000006b52c000) (0MB)
59.283663 ( 0.011035)| [ 0.000000] efi: mem96: type=3, attr=0xf, range=[0x000000006b52c000-0x000000006b52d000) (0MB)
59.291431 ( 0.007768)| [ 0.000000] efi: mem97: type=4, attr=0xf, range=[0x000000006b52d000-0x000000006b531000) (0MB)
59.307560 ( 0.016129)| [ 0.000000] efi: mem98: type=3, attr=0xf, range=[0x000000006b531000-0x000000006b53c000) (0MB)
59.311357 ( 0.003797)| [ 0.000000] efi: mem99: type=4, attr=0xf, range=[0x000000006b53c000-0x000000006b53f000) (0MB)
59.319479 ( 0.008122)| [ 0.000000] efi: mem100: type=3, attr=0xf, range=[0x000000006b53f000-0x000000006b56e000) (0MB)
59.331718 ( 0.012239)| [ 0.000000] efi: mem101: type=4, attr=0xf, range=[0x000000006b56e000-0x000000006b56f000) (0MB)
59.339489 ( 0.007771)| [ 0.000000] efi: mem102: type=3, attr=0xf, range=[0x000000006b56f000-0x000000006b578000) (0MB)
59.347632 ( 0.008143)| [ 0.000000] efi: mem103: type=4, attr=0xf, range=[0x000000006b578000-0x000000006b57b000) (0MB)
59.359670 ( 0.012038)| [ 0.000000] efi: mem104: type=3, attr=0xf, range=[0x000000006b57b000-0x000000006b590000) (0MB)
59.367439 ( 0.007769)| [ 0.000000] efi: mem105: type=4, attr=0xf, range=[0x000000006b590000-0x000000006b594000) (0MB)
59.379593 ( 0.012154)| [ 0.000000] efi: mem106: type=3, attr=0xf, range=[0x000000006b594000-0x000000006b5a5000) (0MB)
59.387368 ( 0.007775)| [ 0.000000] efi: mem107: type=4, attr=0xf, range=[0x000000006b5a5000-0x000000006b5b2000) (0MB)
59.395761 ( 0.008393)| [ 0.000000] efi: mem108: type=3, attr=0xf, range=[0x000000006b5b2000-0x000000006b5b7000) (0MB)
59.407788 ( 0.012027)| [ 0.000000] efi: mem109: type=4, attr=0xf, range=[0x000000006b5b7000-0x000000006b5b8000) (0MB)
59.415558 ( 0.007770)| [ 0.000000] efi: mem110: type=3, attr=0xf, range=[0x000000006b5b8000-0x000000006b5ba000) (0MB)
59.427719 ( 0.012161)| [ 0.000000] efi: mem111: type=4, attr=0xf, range=[0x000000006b5ba000-0x000000006b5bb000) (0MB)
59.435491 ( 0.007772)| [ 0.000000] efi: mem112: type=3, attr=0xf, range=[0x000000006b5bb000-0x000000006b5d8000) (0MB)
59.443634 ( 0.008143)| [ 0.000000] efi: mem113: type=4, attr=0xf, range=[0x000000006b5d8000-0x000000006b9d8000) (4MB)
59.455669 ( 0.012035)| [ 0.000000] efi: mem114: type=3, attr=0xf, range=[0x000000006b9d8000-0x000000006b9de000) (0MB)
59.463439 ( 0.007770)| [ 0.000000] efi: mem115: type=4, attr=0xf, range=[0x000000006b9de000-0x000000006b9e1000) (0MB)
59.475596 ( 0.012157)| [ 0.000000] efi: mem116: type=3, attr=0xf, range=[0x000000006b9e1000-0x000000006b9e5000) (0MB)
59.483369 ( 0.007773)| [ 0.000000] efi: mem117: type=4, attr=0xf, range=[0x000000006b9e5000-0x000000006b9e6000) (0MB)
59.491762 ( 0.008393)| [ 0.000000] efi: mem118: type=3, attr=0xf, range=[0x000000006b9e6000-0x000000006b9e8000) (0MB)
59.503790 ( 0.012028)| [ 0.000000] efi: mem119: type=4, attr=0xf, range=[0x000000006b9e8000-0x000000006b9ea000) (0MB)
59.511560 ( 0.007770)| [ 0.000000] efi: mem120: type=3, attr=0xf, range=[0x000000006b9ea000-0x000000006ba00000) (0MB)
59.523721 ( 0.012161)| [ 0.000000] efi: mem121: type=4, attr=0xf, range=[0x000000006ba00000-0x000000006ba47000) (0MB)
59.531549 ( 0.007828)| [ 0.000000] efi: mem122: type=3, attr=0xf, range=[0x000000006ba47000-0x000000006ba84000) (0MB)
59.539698 ( 0.008149)| [ 0.000000] efi: mem123: type=4, attr=0xf, range=[0x000000006ba84000-0x000000006bad2000) (0MB)
59.551482 ( 0.011784)| [ 0.000000] efi: mem124: type=3, attr=0xf, range=[0x000000006bad2000-0x000000006bae6000) (0MB)
59.559690 ( 0.008208)| [ 0.000000] efi: mem125: type=4, attr=0xf, range=[0x000000006bae6000-0x000000006bb1e000) (0MB)
59.574471 ( 0.014781)| [ 0.000000] efi: mem126: type=3, attr=0xf, range=[0x000000006bb1e000-0x000000006bb22000) (0MB)
59.579618 ( 0.005147)| [ 0.000000] efi: mem127: type=4, attr=0xf, range=[0x000000006bb22000-0x000000007ca9c000) (271MB)
59.587390 ( 0.007772)| [ 0.000000] efi: mem128: type=6, attr=0x800000000000000f, range=[0x000000007ca9c000-0x000000007ca9d000) (0MB)
59.599441 ( 0.012051)| [ 0.000000] efi: mem129: type=4, attr=0xf, range=[0x000000007ca9d000-0x000000007cab0000) (0MB)
59.611598 ( 0.012157)| [ 0.000000] efi: mem130: type=6, attr=0x800000000000000f, range=[0x000000007cab0000-0x000000007ccf0000) (2MB)
59.659388 ( 0.047790)| [ 0.000000] efi: mem131: type=4, attr=0xf, range=[0x000000007ccf0000-0x000000007d9ff000) (13MB)
59.659520 ( 0.000132)| [ 0.000000] efi: mem132: type=6, attr=0x800000000000000f, range=[0x000000007d9ff000-0x000000007daff000) (1MB)
59.659639 ( 0.000119)| [ 0.000000] efi: mem133: type=5, attr=0x800000000000000f, range=[0x000000007daff000-0x000000007dbff000) (1MB)
59.672694 ( 0.013055)| [ 0.000000] efi: mem134: type=0, attr=0xf, range=[0x000000007dbff000-0x000000007dcff000) (1MB)
59.672828 ( 0.000134)| [ 0.000000] efi: mem135: type=10, attr=0xf, range=[0x000000007dcff000-0x000000007eeff000) (18MB)
59.684801 ( 0.011973)| [ 0.000000] efi: mem136: type=9, attr=0xf, range=[0x000000007eeff000-0x000000007efea000) (0MB)
59.696460 ( 0.011659)| [ 0.000000] efi: mem137: type=4, attr=0xf, range=[0x000000007efea000-0x000000007f000000) (0MB)
59.708492 ( 0.012032)| [ 0.000000] efi: mem138: type=7, attr=0xf, range=[0x0000000100000000-0x0000000840000000) (29696MB)
59.719825 ( 0.011333)| [ 0.000000] efi: mem139: type=7, attr=0xf, range=[0x0000001000000000-0x00000017bf000000) (31728MB)
59.719960 ( 0.000135)| [ 0.000000] efi: mem140: type=11, attr=0x8000000000000000, range=[0x0000000080000000-0x0000000090000000) (256MB)
59.723610 ( 0.003650)| [ 0.000000] efi: mem141: type=11, attr=0x8000000000000001, range=[0x00000000f8000000-0x00000000fe000000) (96MB)
59.735667 ( 0.012057)| [ 0.000000] efi: mem142: type=11, attr=0x8000000000000001, range=[0x00000000fed1c000-0x00000000fed20000) (0MB)
59.743450 ( 0.007783)| [ 0.000000] efi: mem143: type=11, attr=0x8000000000000001, range=[0x000000ff00000000-0x000000ff03e00000) (62MB)
59.755628 ( 0.012178)| [ 0.000000] efi: mem144: type=11, attr=0x8000000000000001, range=[0x000000ff03f00000-0x000000ff04000000) (1MB)
59.767682 ( 0.012054)| [ 0.000000] efi: mem145: type=11, attr=0x8000000000000001, range=[0x000000ff07f00000-0x000000ff08000000) (1MB)
59.779483 ( 0.011801)| [ 0.000000] efi: mem146: type=11, attr=0x8000000000000001, range=[0x000000ff40000000-0x000000ff43000000) (48MB)
59.791533 ( 0.012050)| [ 0.000000] efi: mem147: type=11, attr=0x8000000000000001, range=[0x000000ff44000000-0x000000ff47000000) (48MB)
59.799702 ( 0.008169)| [ 0.000000] efi: mem148: type=11, attr=0x8000000000000001, range=[0x000000ff48000000-0x000000ff4b000000) (48MB)
59.811505 ( 0.011803)| [ 0.000000] efi: mem149: type=11, attr=0x8000000000000001, range=[0x000000ff4c000000-0x000000ff4f000000) (48MB)
[...]
67.055714 ( 0.008325)| [ 5.971040] EFI Variables Facility v0.08 2004-May-17
67.059441 ( 0.003727)| [ 5.976599] BUG: unable to handle kernel paging request at 000000007ca95b10
67.067807 ( 0.008366)| [ 5.984386] IP: [<ffff88007dbf2140>] 0xffff88007dbf213f
67.075529 ( 0.007722)| [ 5.990230] PGD 0
67.075570 ( 0.000041)| [ 5.992484] Oops: 0000 [#1] SMP
67.079368 ( 0.003798)| [ 5.996107] Modules linked in:
67.083673 ( 0.004305)| [ 5.999516] CPU: 1 PID: 1 Comm: swapper/0 Not tainted 3.10.0-rc2-0.55.el7-rja+ #1
67.091427 ( 0.007754)| [ 6.007861] Hardware name: SGI UV2000/ROMLEY, BIOS SGI UV 2000/3000 series BIOS 01/15/2013
67.099815 ( 0.008388)| [ 6.017082] task: ffff880815938000 ti: ffff880815932000 task.ti: ffff880815932000
67.107751 ( 0.007936)| [ 6.025428] RIP: 0010:[<ffff88007dbf2140>] [<ffff88007dbf2140>] 0xffff88007dbf213f
67.115507 ( 0.007756)| [ 6.033978] RSP: 0018:ffff8808159338d0 EFLAGS: 00010002
67.123613 ( 0.008106)| [ 6.039903] RAX: 000000007ca95ad0 RBX: 000000000000006a RCX: 0000000090000002
67.131365 ( 0.007752)| [ 6.047860] RDX: 0000000003050007 RSI: 0000000000000000 RDI: ffff88007dbf6ea8
67.139730 ( 0.008365)| [ 6.055818] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
67.147482 ( 0.007752)| [ 6.063776] R10: ffff880820801500 R11: 0000000000000000 R12: 0000000003050007
67.155614 ( 0.008132)| [ 6.071732] R13: 0000000090000002 R14: ffff880815933db8 R15: 0000000000000000
67.163362 ( 0.007748)| [ 6.079690] FS: 0000000000000000(0000) GS:ffff8817bec00000(0000) knlGS:0000000000000000
67.171744 ( 0.008382)| [ 6.088715] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
67.179475 ( 0.007731)| [ 6.095122] CR2: 000000007ca95b10 CR3: 00000000018fd000 CR4: 00000000000407e0
67.187605 ( 0.008130)| [ 6.103082] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
67.195355 ( 0.007750)| [ 6.111039] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
67.203724 ( 0.008369)| [ 6.118997] Stack:
67.203764 ( 0.000040)| [ 6.121238] 0000000000000010 0000000000000246 ffffffff20815a80 ffff88083ffd9d80
67.221504 ( 0.017740)| [ 6.129527] 0000000000000010 ffff88083ffdab08 0000000000000000 ffff88083ffd9d80
67.233872 ( 0.012368)| [ 6.137819] 0000000000000010 0000000000000002 0000000000000000 0000000000000000
67.245638 ( 0.011766)| [ 6.146110] Call Trace:
67.245689 ( 0.000051)| [ 6.148847] [<ffffffff81139a34>] ? __alloc_pages_nodemask+0x154/0x2f0
67.247488 ( 0.001799)| [ 6.156125] [<ffffffff81174f7d>] ? alloc_page_interleave+0x9d/0xa0
67.247582 ( 0.000094)| [ 6.163120] [<ffffffff812fe192>] ? put_dec+0x72/0x90
67.281377 ( 0.033795)| [ 6.168756] [<ffffffff812f6d53>] ? ida_get_new_above+0xb3/0x220
67.281469 ( 0.000092)| [ 6.175449] [<ffffffff812f6174>] ? sub_alloc+0x74/0x1d0
67.281550 ( 0.000081)| [ 6.181372] [<ffffffff812f6174>] ? sub_alloc+0x74/0x1d0
67.281619 ( 0.000069)| [ 6.187305] [<ffffffff812f6d53>] ? ida_get_new_above+0xb3/0x220
67.283631 ( 0.002012)| [ 6.194010] [<ffffffff814c8cc0>] ? create_efivars_bin_attributes+0x150/0x150
67.283748 ( 0.000117)| [ 6.201974] [<ffffffff810499b3>] ? efi_call3+0x43/0x80
67.291432 ( 0.007684)| [ 6.207803] [<ffffffff810492a7>] ? virt_efi_get_next_variable+0x47/0x1c0
67.299795 ( 0.008363)| [ 6.215374] [<ffffffff814c8cc0>] ? create_efivars_bin_attributes+0x150/0x150
67.307545 ( 0.007750)| [ 6.223340] [<ffffffff814c7b55>] ? efivar_init+0xd5/0x390
67.311384 ( 0.003839)| [ 6.229458] [<ffffffff814c8ae0>] ? efivar_update_sysfs_entries+0x90/0x90
67.319748 ( 0.008364)| [ 6.237032] [<ffffffff812f906b>] ? kobject_uevent+0xb/0x10
67.327478 ( 0.007730)| [ 6.243247] [<ffffffff812f812b>] ? kset_register+0x5b/0x70
67.331822 ( 0.004344)| [ 6.249469] [<ffffffff814c8cc0>] ? create_efivars_bin_attributes+0x150/0x150
67.339570 ( 0.007748)| [ 6.257428] [<ffffffff814c8d47>] ? efivars_sysfs_init+0x87/0xf0
67.347684 ( 0.008114)| [ 6.264129] [<ffffffff8100032a>] ? do_one_initcall+0x15a/0x1b0
67.355415 ( 0.007731)| [ 6.270738] [<ffffffff81a17831>] ? do_basic_setup+0xad/0xce
67.359500 ( 0.004085)| [ 6.277051] [<ffffffff81a17ae3>] ? kernel_init_freeable+0x291/0x291
67.367616 ( 0.008116)| [ 6.284142] [<ffffffff81a3708a>] ? sched_init_smp+0x15b/0x162
67.375352 ( 0.007736)| [ 6.290648] [<ffffffff81a17a5f>] ? kernel_init_freeable+0x20d/0x291
67.379705 ( 0.004353)| [ 6.297739] [<ffffffff81601eb0>] ? rest_init+0x80/0x80
67.387425 ( 0.007720)| [ 6.303566] [<ffffffff81601ebe>] ? kernel_init+0xe/0x180
67.391507 ( 0.004082)| [ 6.309588] [<ffffffff8162179c>] ? ret_from_fork+0x7c/0xb0
67.399616 ( 0.008109)| [ 6.315803] [<ffffffff81601eb0>] ? rest_init+0x80/0x80
67.403574 ( 0.003958)| [ 6.321628] Code: 8b d8 e8 e4 fa ff ff 84 c0 75 0f 48 8b 15 61 58 00 00 48 8b 4c 24 30 ff 52 48 48 8b c3 eb 58 48 8b 05 4d 58 00 00 48 85 c0 74 42 <48> 83 78 40 00 74 3b 48 83 78 48 00 74 34 48 8d 53 14 4c 8d 44
67.427797 ( 0.024223)| [ 6.343375] RIP [<ffff88007dbf2140>] 0xffff88007dbf213f
67.431635 ( 0.003838)| [ 6.349310] RSP <ffff8808159338d0>
67.435568 ( 0.003933)| [ 6.353198] CR2: 000000007ca95b10
67.439379 ( 0.003811)| [ 6.356901] ---[ end trace 0d9ca8269f302599 ]---
67.443710 ( 0.004331)| [ 6.362116] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000009
----------------------------------------------------------------------
--
Russ Anderson, OS RAS/Partitioning Project Lead
SGI - Silicon Graphics Inc [email protected]
On Thu, May 23, 2013 at 12:58:01PM +0100, Matt Fleming wrote:
> On Wed, 22 May, at 11:27:47AM, Russ Anderson wrote:
> > [ 6.062157] EFI Variables Facility v0.08 2004-May-17
> > [ 6.067731] BUG: unable to handle kernel paging request at 000000007ca95b10
> > [ 6.075519] IP: [<ffff88007dbf2140>] 0xffff88007dbf213f
>
> This is a bit of a head scratcher. Could you paste the EFI memmap
> entries in your dmesg for the regions that cover 0x7ca95b10 and
> 0x7dbf2140? My guess would be that they're EFI runtime code regions,
> which would at least explain why we seem to be executing code in the
> direct mapping region (0xffff8800xxxxxxxx).
>
> Are you booting via the EFI boot stub?
Interesting data point. The failure is on a rhel7/grub2 root.
The identical kernel on a rhel6/grub root boots. So maybe
grub2 brings out the failure? I suspect Fedora19/grub2 on
EFI should hit the problem (for someone looking to reproduce
it).
In both cases the kernel boot line options are the same.
--
Russ Anderson, OS RAS/Partitioning Project Lead
SGI - Silicon Graphics Inc [email protected]
On Thu, 23 May, at 03:32:34PM, Russ Anderson wrote:
> efi: mem127: type=4, attr=0xf, range=[0x000000006bb22000-0x000000007ca9c000) (271MB)
EFI_BOOT_SERVICES_CODE
> efi: mem133: type=5, attr=0x800000000000000f, range=[0x000000007daff000-0x000000007dbff000) (1MB)
EFI_RUNTIME_SERVICES_CODE
> EFI Variables Facility v0.08 2004-May-17
> BUG: unable to handle kernel paging request at 000000007ca95b10
> IP: [<ffff88007dbf2140>] 0xffff88007dbf213f
This...
> Call Trace:
> [<ffffffff81139a34>] ? __alloc_pages_nodemask+0x154/0x2f0
> [<ffffffff81174f7d>] ? alloc_page_interleave+0x9d/0xa0
> [<ffffffff812fe192>] ? put_dec+0x72/0x90
> [<ffffffff812f6d53>] ? ida_get_new_above+0xb3/0x220
> [<ffffffff812f6174>] ? sub_alloc+0x74/0x1d0
> [<ffffffff812f6174>] ? sub_alloc+0x74/0x1d0
> [<ffffffff812f6d53>] ? ida_get_new_above+0xb3/0x220
> [<ffffffff814c8cc0>] ? create_efivars_bin_attributes+0x150/0x150
is junk on the stack.
> [<ffffffff810499b3>] ? efi_call3+0x43/0x80
> [<ffffffff810492a7>] ? virt_efi_get_next_variable+0x47/0x1c0
> [<ffffffff814c8cc0>] ? create_efivars_bin_attributes+0x150/0x150
> [<ffffffff814c7b55>] ? efivar_init+0xd5/0x390
> [<ffffffff814c8ae0>] ? efivar_update_sysfs_entries+0x90/0x90
> [<ffffffff812f906b>] ? kobject_uevent+0xb/0x10
> [<ffffffff812f812b>] ? kset_register+0x5b/0x70
> [<ffffffff814c8cc0>] ? create_efivars_bin_attributes+0x150/0x150
> [<ffffffff814c8d47>] ? efivars_sysfs_init+0x87/0xf0
> [<ffffffff8100032a>] ? do_one_initcall+0x15a/0x1b0
> [<ffffffff81a17831>] ? do_basic_setup+0xad/0xce
> [<ffffffff81a17ae3>] ? kernel_init_freeable+0x291/0x291
> [<ffffffff81a3708a>] ? sched_init_smp+0x15b/0x162
> [<ffffffff81a17a5f>] ? kernel_init_freeable+0x20d/0x291
> [<ffffffff81601eb0>] ? rest_init+0x80/0x80
> [<ffffffff81601ebe>] ? kernel_init+0xe/0x180
> [<ffffffff8162179c>] ? ret_from_fork+0x7c/0xb0
> [<ffffffff81601eb0>] ? rest_init+0x80/0x80
Here's the real call stack leading up to the crash.
What appears to be happening is that your the EFI runtime services code
is calling into the EFI boot services code, which is definitely a bug in
your firmware because we're at runtime, but we've seen other machines
that do similar things so we usually handle it just fine. However, what
makes your case different, and the reason you see the above splat, is
that it's using the physical address of the EFI boot services region,
not the virtual one we setup with SetVirtualAddressMap(). Which is a
second firmware bug. Again, we have seen other machines that access
physical addresses after SetVirtualAddressMap(), but until now we
haven't had any non-optional code that triggered them.
The only reason I can see that the offending commit would introduce this
problem is because it calls QueryVariableInfo() at boot time. I notice
that your machine is an SGI UV one, is there any chance you could get a
firmware fix for this? If possible, it would be also good to confirm
that it's this chunk of code in setup_efi_vars(),
status = efi_call_phys4(sys_table->runtime->query_variable_info,
EFI_VARIABLE_NON_VOLATILE |
EFI_VARIABLE_BOOTSERVICE_ACCESS |
EFI_VARIABLE_RUNTIME_ACCESS, &store_size,
&remaining_size, &var_size);
that later makes GetNextVariable() jump to the physical address of the
EFI Boot Services region. Because if not, we need to do some more
digging.
Borislav, how are your 1:1 mapping patches coming along? In theory, once
those are merged we can gracefully workaround these kinds of issues.
--
Matt Fleming, Intel Open Source Technology Center
On Thu, 23 May, at 05:23:21PM, Russ Anderson wrote:
> Interesting data point. The failure is on a rhel7/grub2 root.
> The identical kernel on a rhel6/grub root boots. So maybe
> grub2 brings out the failure? I suspect Fedora19/grub2 on
> EFI should hit the problem (for someone looking to reproduce
> it).
>
> In both cases the kernel boot line options are the same.
I'll bet that rhel7 is using the EFI handover protocol which uses the
internal mechanisms of the EFI boot stub.
I don't know whether anyone will be able to reproduce it, it looks like
it's a bug that's specific to your firmware.
--
Matt Fleming, Intel Open Source Technology Center
On Fri, May 24, 2013 at 08:43:31AM +0100, Matt Fleming wrote:
> What appears to be happening is that your the EFI runtime services
> code is calling into the EFI boot services code, which is definitely
> a bug in your firmware because we're at runtime, but we've seen
> other machines that do similar things so we usually handle it just
> fine. However, what makes your case different, and the reason you
> see the above splat, is that it's using the physical address of
> the EFI boot services region, not the virtual one we setup with
> SetVirtualAddressMap(). Which is a second firmware bug.
I'm speechless. Let's have someone else do the ranting this time:
http://www.happyassassin.net/2013/05/03/a-day-in-the-life-of-a-firmware-engineer/
> Again, we have seen other machines that access
> physical addresses after SetVirtualAddressMap(), but until now we
> haven't had any non-optional code that triggered them.
>
> The only reason I can see that the offending commit would introduce this
> problem is because it calls QueryVariableInfo() at boot time. I notice
> that your machine is an SGI UV one, is there any chance you could get a
> firmware fix for this? If possible, it would be also good to confirm
> that it's this chunk of code in setup_efi_vars(),
>
> status = efi_call_phys4(sys_table->runtime->query_variable_info,
> EFI_VARIABLE_NON_VOLATILE |
> EFI_VARIABLE_BOOTSERVICE_ACCESS |
> EFI_VARIABLE_RUNTIME_ACCESS, &store_size,
> &remaining_size, &var_size);
>
> that later makes GetNextVariable() jump to the physical address of the
> EFI Boot Services region. Because if not, we need to do some more
> digging.
>
> Borislav, how are your 1:1 mapping patches coming along? In theory, once
> those are merged we can gracefully workaround these kinds of issues.
What do you mean, map boot time functions 1:1 too?
In any case, I think I have an idea about the bug I was discussing with
hpa recently but I need to do more experimenting. I have the next week
off, though, so don't hold your breath just yet :).
On Fri, 24 May, at 01:09:11PM, Borislav Petkov wrote:
> What do you mean, map boot time functions 1:1 too?
Yep, that's what I meant.
--
Matt Fleming, Intel Open Source Technology Center
Russ,
Can we open a bug for the BIOS folks and see if we can get this addressed?
Robin
On Fri, May 24, 2013 at 08:43:31AM +0100, Matt Fleming wrote:
> On Thu, 23 May, at 03:32:34PM, Russ Anderson wrote:
> > efi: mem127: type=4, attr=0xf, range=[0x000000006bb22000-0x000000007ca9c000) (271MB)
>
> EFI_BOOT_SERVICES_CODE
>
> > efi: mem133: type=5, attr=0x800000000000000f, range=[0x000000007daff000-0x000000007dbff000) (1MB)
>
> EFI_RUNTIME_SERVICES_CODE
>
> > EFI Variables Facility v0.08 2004-May-17
> > BUG: unable to handle kernel paging request at 000000007ca95b10
> > IP: [<ffff88007dbf2140>] 0xffff88007dbf213f
>
> This...
>
> > Call Trace:
> > [<ffffffff81139a34>] ? __alloc_pages_nodemask+0x154/0x2f0
> > [<ffffffff81174f7d>] ? alloc_page_interleave+0x9d/0xa0
> > [<ffffffff812fe192>] ? put_dec+0x72/0x90
> > [<ffffffff812f6d53>] ? ida_get_new_above+0xb3/0x220
> > [<ffffffff812f6174>] ? sub_alloc+0x74/0x1d0
> > [<ffffffff812f6174>] ? sub_alloc+0x74/0x1d0
> > [<ffffffff812f6d53>] ? ida_get_new_above+0xb3/0x220
> > [<ffffffff814c8cc0>] ? create_efivars_bin_attributes+0x150/0x150
>
> is junk on the stack.
>
> > [<ffffffff810499b3>] ? efi_call3+0x43/0x80
> > [<ffffffff810492a7>] ? virt_efi_get_next_variable+0x47/0x1c0
> > [<ffffffff814c8cc0>] ? create_efivars_bin_attributes+0x150/0x150
> > [<ffffffff814c7b55>] ? efivar_init+0xd5/0x390
> > [<ffffffff814c8ae0>] ? efivar_update_sysfs_entries+0x90/0x90
> > [<ffffffff812f906b>] ? kobject_uevent+0xb/0x10
> > [<ffffffff812f812b>] ? kset_register+0x5b/0x70
> > [<ffffffff814c8cc0>] ? create_efivars_bin_attributes+0x150/0x150
> > [<ffffffff814c8d47>] ? efivars_sysfs_init+0x87/0xf0
> > [<ffffffff8100032a>] ? do_one_initcall+0x15a/0x1b0
> > [<ffffffff81a17831>] ? do_basic_setup+0xad/0xce
> > [<ffffffff81a17ae3>] ? kernel_init_freeable+0x291/0x291
> > [<ffffffff81a3708a>] ? sched_init_smp+0x15b/0x162
> > [<ffffffff81a17a5f>] ? kernel_init_freeable+0x20d/0x291
> > [<ffffffff81601eb0>] ? rest_init+0x80/0x80
> > [<ffffffff81601ebe>] ? kernel_init+0xe/0x180
> > [<ffffffff8162179c>] ? ret_from_fork+0x7c/0xb0
> > [<ffffffff81601eb0>] ? rest_init+0x80/0x80
>
> Here's the real call stack leading up to the crash.
>
> What appears to be happening is that your the EFI runtime services code
> is calling into the EFI boot services code, which is definitely a bug in
> your firmware because we're at runtime, but we've seen other machines
> that do similar things so we usually handle it just fine. However, what
> makes your case different, and the reason you see the above splat, is
> that it's using the physical address of the EFI boot services region,
> not the virtual one we setup with SetVirtualAddressMap(). Which is a
> second firmware bug. Again, we have seen other machines that access
> physical addresses after SetVirtualAddressMap(), but until now we
> haven't had any non-optional code that triggered them.
>
> The only reason I can see that the offending commit would introduce this
> problem is because it calls QueryVariableInfo() at boot time. I notice
> that your machine is an SGI UV one, is there any chance you could get a
> firmware fix for this? If possible, it would be also good to confirm
> that it's this chunk of code in setup_efi_vars(),
>
> status = efi_call_phys4(sys_table->runtime->query_variable_info,
> EFI_VARIABLE_NON_VOLATILE |
> EFI_VARIABLE_BOOTSERVICE_ACCESS |
> EFI_VARIABLE_RUNTIME_ACCESS, &store_size,
> &remaining_size, &var_size);
>
> that later makes GetNextVariable() jump to the physical address of the
> EFI Boot Services region. Because if not, we need to do some more
> digging.
>
> Borislav, how are your 1:1 mapping patches coming along? In theory, once
> those are merged we can gracefully workaround these kinds of issues.
>
> --
> Matt Fleming, Intel Open Source Technology Center
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
On Fri, May 24, 2013 at 11:11:11AM -0500, Robin Holt wrote:
> Russ,
>
> Can we open a bug for the BIOS folks and see if we can get this addressed?
I already talked with them. It is not in an area that we
normally change, so if there is a bug may be in the Intel
reference code. More investigation is needed to track down
the actual problem, and that could take help from Intel.
Regardless of that, it is a kernel patch that triggers the
problem. This isn't the first time a kernel change does
the "right thing" but trips across questionable bios/EFI/bootloader
implementation. That still makes it a kernel bug.
I'm still digging to better understand the root problem.
> Robin
>
> On Fri, May 24, 2013 at 08:43:31AM +0100, Matt Fleming wrote:
> > On Thu, 23 May, at 03:32:34PM, Russ Anderson wrote:
> > > efi: mem127: type=4, attr=0xf, range=[0x000000006bb22000-0x000000007ca9c000) (271MB)
> >
> > EFI_BOOT_SERVICES_CODE
> >
> > > efi: mem133: type=5, attr=0x800000000000000f, range=[0x000000007daff000-0x000000007dbff000) (1MB)
> >
> > EFI_RUNTIME_SERVICES_CODE
> >
> > > EFI Variables Facility v0.08 2004-May-17
> > > BUG: unable to handle kernel paging request at 000000007ca95b10
> > > IP: [<ffff88007dbf2140>] 0xffff88007dbf213f
> >
> > This...
> >
> > > Call Trace:
> > > [<ffffffff81139a34>] ? __alloc_pages_nodemask+0x154/0x2f0
> > > [<ffffffff81174f7d>] ? alloc_page_interleave+0x9d/0xa0
> > > [<ffffffff812fe192>] ? put_dec+0x72/0x90
> > > [<ffffffff812f6d53>] ? ida_get_new_above+0xb3/0x220
> > > [<ffffffff812f6174>] ? sub_alloc+0x74/0x1d0
> > > [<ffffffff812f6174>] ? sub_alloc+0x74/0x1d0
> > > [<ffffffff812f6d53>] ? ida_get_new_above+0xb3/0x220
> > > [<ffffffff814c8cc0>] ? create_efivars_bin_attributes+0x150/0x150
> >
> > is junk on the stack.
> >
> > > [<ffffffff810499b3>] ? efi_call3+0x43/0x80
> > > [<ffffffff810492a7>] ? virt_efi_get_next_variable+0x47/0x1c0
> > > [<ffffffff814c8cc0>] ? create_efivars_bin_attributes+0x150/0x150
> > > [<ffffffff814c7b55>] ? efivar_init+0xd5/0x390
> > > [<ffffffff814c8ae0>] ? efivar_update_sysfs_entries+0x90/0x90
> > > [<ffffffff812f906b>] ? kobject_uevent+0xb/0x10
> > > [<ffffffff812f812b>] ? kset_register+0x5b/0x70
> > > [<ffffffff814c8cc0>] ? create_efivars_bin_attributes+0x150/0x150
> > > [<ffffffff814c8d47>] ? efivars_sysfs_init+0x87/0xf0
> > > [<ffffffff8100032a>] ? do_one_initcall+0x15a/0x1b0
> > > [<ffffffff81a17831>] ? do_basic_setup+0xad/0xce
> > > [<ffffffff81a17ae3>] ? kernel_init_freeable+0x291/0x291
> > > [<ffffffff81a3708a>] ? sched_init_smp+0x15b/0x162
> > > [<ffffffff81a17a5f>] ? kernel_init_freeable+0x20d/0x291
> > > [<ffffffff81601eb0>] ? rest_init+0x80/0x80
> > > [<ffffffff81601ebe>] ? kernel_init+0xe/0x180
> > > [<ffffffff8162179c>] ? ret_from_fork+0x7c/0xb0
> > > [<ffffffff81601eb0>] ? rest_init+0x80/0x80
> >
> > Here's the real call stack leading up to the crash.
> >
> > What appears to be happening is that your the EFI runtime services code
> > is calling into the EFI boot services code, which is definitely a bug in
> > your firmware because we're at runtime, but we've seen other machines
> > that do similar things so we usually handle it just fine. However, what
> > makes your case different, and the reason you see the above splat, is
> > that it's using the physical address of the EFI boot services region,
> > not the virtual one we setup with SetVirtualAddressMap(). Which is a
> > second firmware bug. Again, we have seen other machines that access
> > physical addresses after SetVirtualAddressMap(), but until now we
> > haven't had any non-optional code that triggered them.
> >
> > The only reason I can see that the offending commit would introduce this
> > problem is because it calls QueryVariableInfo() at boot time. I notice
> > that your machine is an SGI UV one, is there any chance you could get a
> > firmware fix for this? If possible, it would be also good to confirm
> > that it's this chunk of code in setup_efi_vars(),
> >
> > status = efi_call_phys4(sys_table->runtime->query_variable_info,
> > EFI_VARIABLE_NON_VOLATILE |
> > EFI_VARIABLE_BOOTSERVICE_ACCESS |
> > EFI_VARIABLE_RUNTIME_ACCESS, &store_size,
> > &remaining_size, &var_size);
> >
> > that later makes GetNextVariable() jump to the physical address of the
> > EFI Boot Services region. Because if not, we need to do some more
> > digging.
> >
> > Borislav, how are your 1:1 mapping patches coming along? In theory, once
> > those are merged we can gracefully workaround these kinds of issues.
> >
> > --
> > Matt Fleming, Intel Open Source Technology Center
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> > the body of a message to [email protected]
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
> > Please read the FAQ at http://www.tux.org/lkml/
--
Russ Anderson, OS RAS/Partitioning Project Lead
SGI - Silicon Graphics Inc [email protected]
On Fri, May 24, 2013 at 08:43:31AM +0100, Matt Fleming wrote:
> On Thu, 23 May, at 03:32:34PM, Russ Anderson wrote:
> > efi: mem127: type=4, attr=0xf, range=[0x000000006bb22000-0x000000007ca9c000) (271MB)
>
> EFI_BOOT_SERVICES_CODE
>
> > efi: mem133: type=5, attr=0x800000000000000f, range=[0x000000007daff000-0x000000007dbff000) (1MB)
>
> EFI_RUNTIME_SERVICES_CODE
>
> > EFI Variables Facility v0.08 2004-May-17
> > BUG: unable to handle kernel paging request at 000000007ca95b10
> > IP: [<ffff88007dbf2140>] 0xffff88007dbf213f
>
> > [<ffffffff810499b3>] ? efi_call3+0x43/0x80
> > [<ffffffff810492a7>] ? virt_efi_get_next_variable+0x47/0x1c0
> > [<ffffffff814c8cc0>] ? create_efivars_bin_attributes+0x150/0x150
> > [<ffffffff814c7b55>] ? efivar_init+0xd5/0x390
> > [<ffffffff814c8ae0>] ? efivar_update_sysfs_entries+0x90/0x90
> > [<ffffffff812f906b>] ? kobject_uevent+0xb/0x10
> > [<ffffffff812f812b>] ? kset_register+0x5b/0x70
> > [<ffffffff814c8cc0>] ? create_efivars_bin_attributes+0x150/0x150
> > [<ffffffff814c8d47>] ? efivars_sysfs_init+0x87/0xf0
> > [<ffffffff8100032a>] ? do_one_initcall+0x15a/0x1b0
> > [<ffffffff81a17831>] ? do_basic_setup+0xad/0xce
> > [<ffffffff81a17ae3>] ? kernel_init_freeable+0x291/0x291
> > [<ffffffff81a3708a>] ? sched_init_smp+0x15b/0x162
> > [<ffffffff81a17a5f>] ? kernel_init_freeable+0x20d/0x291
> > [<ffffffff81601eb0>] ? rest_init+0x80/0x80
> > [<ffffffff81601ebe>] ? kernel_init+0xe/0x180
> > [<ffffffff8162179c>] ? ret_from_fork+0x7c/0xb0
> > [<ffffffff81601eb0>] ? rest_init+0x80/0x80
>
> Here's the real call stack leading up to the crash.
>
> What appears to be happening is that your the EFI runtime services code
> is calling into the EFI boot services code, which is definitely a bug in
> your firmware because we're at runtime, but we've seen other machines
> that do similar things so we usually handle it just fine. However, what
> makes your case different, and the reason you see the above splat, is
> that it's using the physical address of the EFI boot services region,
> not the virtual one we setup with SetVirtualAddressMap(). Which is a
> second firmware bug. Again, we have seen other machines that access
> physical addresses after SetVirtualAddressMap(), but until now we
> haven't had any non-optional code that triggered them.
>
> The only reason I can see that the offending commit would introduce this
> problem is because it calls QueryVariableInfo() at boot time. I notice
> that your machine is an SGI UV one, is there any chance you could get a
> firmware fix for this? If possible, it would be also good to confirm
> that it's this chunk of code in setup_efi_vars(),
>
> status = efi_call_phys4(sys_table->runtime->query_variable_info,
> EFI_VARIABLE_NON_VOLATILE |
> EFI_VARIABLE_BOOTSERVICE_ACCESS |
> EFI_VARIABLE_RUNTIME_ACCESS, &store_size,
> &remaining_size, &var_size);
This call is failing, but not returning a valid EFI_* return status.
setup_efi_vars() returns at that point. Maybe it is not set up
to do GetNextVariable() later on??? Why call GetNextVariable() if the
earlier call failed?
> that later makes GetNextVariable() jump to the physical address of the
> EFI Boot Services region. Because if not, we need to do some more
> digging.
One other data point is if the query_variable_info call is hacked to
remove one of the EFI flags (ie comment out EFI_VARIABLE_BOOTSERVICE_ACCESS)
the efi_call_phys4() call fails with EFI_INVALID_PARAMETER and
the system boots. Of course it does not create /sys/firmware/efivars
entries and complains "[Firmware Bug]: efi: Inconsistent initial sizes".
But at least it boots.
One of the BIOS guys will build a debug bios next week to help see
what is going on in the query_variable_info() call.
> Borislav, how are your 1:1 mapping patches coming along? In theory, once
> those are merged we can gracefully workaround these kinds of issues.
>
> --
> Matt Fleming, Intel Open Source Technology Center
--
Russ Anderson, OS RAS/Partitioning Project Lead
SGI - Silicon Graphics Inc [email protected]
On Fri, 2013-05-24 at 15:05 -0500, Russ Anderson wrote:
> One other data point is if the query_variable_info call is hacked to
> remove one of the EFI flags (ie comment out EFI_VARIABLE_BOOTSERVICE_ACCESS)
> the efi_call_phys4() call fails with EFI_INVALID_PARAMETER and
> the system boots. Of course it does not create /sys/firmware/efivars
> entries and complains "[Firmware Bug]: efi: Inconsistent initial sizes".
> But at least it boots.
EFI_VARIABLE_RUNTIME_ACCESS is only legal if
EFI_VARIABLE_BOOTSERVICE_ACCESS is set, so it's correct to throw
EFI_INVALID_PARAMETER there.
--
Matthew Garrett | [email protected]
????{.n?+???????+%?????ݶ??w??{.n?+????{??G?????{ay? ʇڙ?,j??f???h?????????z_??(?階?ݢj"???m??????G????????????&???~???iO???z??v?^?m???? ????????I?
On Fri, May 24, 2013 at 08:11:01PM +0000, Matthew Garrett wrote:
> On Fri, 2013-05-24 at 15:05 -0500, Russ Anderson wrote:
>
> > One other data point is if the query_variable_info call is hacked to
> > remove one of the EFI flags (ie comment out EFI_VARIABLE_BOOTSERVICE_ACCESS)
> > the efi_call_phys4() call fails with EFI_INVALID_PARAMETER and
> > the system boots. Of course it does not create /sys/firmware/efivars
> > entries and complains "[Firmware Bug]: efi: Inconsistent initial sizes".
> > But at least it boots.
>
> EFI_VARIABLE_RUNTIME_ACCESS is only legal if
> EFI_VARIABLE_BOOTSERVICE_ACCESS is set, so it's correct to throw
> EFI_INVALID_PARAMETER there.
Yes. The point of the experiment was to see if it returned a
valid failure status (which it did) and see if the boot still
failed (which it didn't). So something about going deeper
into that call seems to trigger the failure.
Why does the kernel still try to create /sys/firmware/efivars/
entries in the original failure even though efi_call_phys4()
failed? Or does it always try to create those entries
and GetNextVariable() blows up in the original failure
but not in my experiment?
Thanks,
--
Russ Anderson, OS RAS/Partitioning Project Lead
SGI - Silicon Graphics Inc [email protected]
On Fri, May 24, 2013 at 12:02:15PM -0500, Russ Anderson wrote:
> On Fri, May 24, 2013 at 11:11:11AM -0500, Robin Holt wrote:
> > Russ,
> >
> > Can we open a bug for the BIOS folks and see if we can get this addressed?
>
> I already talked with them. It is not in an area that we
> normally change, so if there is a bug may be in the Intel
> reference code. More investigation is needed to track down
> the actual problem, and that could take help from Intel.
>
> Regardless of that, it is a kernel patch that triggers the
> problem. This isn't the first time a kernel change does
> the "right thing" but trips across questionable bios/EFI/bootloader
> implementation. That still makes it a kernel bug.
>
> I'm still digging to better understand the root problem.
When we rebased the Fedora 18 kernel to 3.9 we had a bunch of reports
from people who can no longer boot with what looks like similar symptoms.
https://bugzilla.redhat.com/show_bug.cgi?id=964335
Dave
Hi Dave,
於 五，2013-05-24 於 17:05 -0400，Dave Jones 提到：
> On Fri, May 24, 2013 at 12:02:15PM -0500, Russ Anderson wrote:
> > On Fri, May 24, 2013 at 11:11:11AM -0500, Robin Holt wrote:
> > > Russ,
> > >
> > > Can we open a bug for the BIOS folks and see if we can get this addressed?
> >
> > I already talked with them. It is not in an area that we
> > normally change, so if there is a bug may be in the Intel
> > reference code. More investigation is needed to track down
> > the actual problem, and that could take help from Intel.
> >
> > Regardless of that, it is a kernel patch that triggers the
> > problem. This isn't the first time a kernel change does
> > the "right thing" but trips across questionable bios/EFI/bootloader
> > implementation. That still makes it a kernel bug.
> >
> > I'm still digging to better understand the root problem.
>
> When we rebased the Fedora 18 kernel to 3.9 we had a bunch of reports
> from people who can no longer boot with what looks like similar symptoms.
>
> https://bugzilla.redhat.com/show_bug.cgi?id=964335
>
> Dave
>
The oops on the above bug is similar to my problem on Acer machine,
could you please ask the reporter try the eccaf52f patch in urgent
branch on Matt's efi git tree:
https://git.kernel.org/cgit/linux/kernel/git/mfleming/efi.git/commit/?h=urgent&id=eccaf52fee8305d5207ff110950a82c100e459bc
Thanks!
Joey Lee
於 一，2013-05-27 於 12:27 +0800，joeyli 提到：
> Hi Dave,
>
> 於 五，2013-05-24 於 17:05 -0400，Dave Jones 提到：
> > On Fri, May 24, 2013 at 12:02:15PM -0500, Russ Anderson wrote:
> > > On Fri, May 24, 2013 at 11:11:11AM -0500, Robin Holt wrote:
> > > > Russ,
> > > >
> > > > Can we open a bug for the BIOS folks and see if we can get this addressed?
> > >
> > > I already talked with them. It is not in an area that we
> > > normally change, so if there is a bug may be in the Intel
> > > reference code. More investigation is needed to track down
> > > the actual problem, and that could take help from Intel.
> > >
> > > Regardless of that, it is a kernel patch that triggers the
> > > problem. This isn't the first time a kernel change does
> > > the "right thing" but trips across questionable bios/EFI/bootloader
> > > implementation. That still makes it a kernel bug.
> > >
> > > I'm still digging to better understand the root problem.
> >
> > When we rebased the Fedora 18 kernel to 3.9 we had a bunch of reports
> > from people who can no longer boot with what looks like similar symptoms.
> >
> > https://bugzilla.redhat.com/show_bug.cgi?id=964335
> >
> > Dave
> >
>
> The oops on the above bug is similar to my problem on Acer machine,
> could you please ask the reporter try the eccaf52f patch in urgent
> branch on Matt's efi git tree:
>
> https://git.kernel.org/cgit/linux/kernel/git/mfleming/efi.git/commit/?h=urgent&id=eccaf52fee8305d5207ff110950a82c100e459bc
>
>
> Thanks!
> Joey Lee
Looks there have a couple of different machines on this bug. The oops
similar to my machine is reported by josephhenryblack.
Thanks
Joey Lee
On Mon, May 27, 2013 at 12:27:12PM +0800, joeyli wrote:
> Hi Dave,
>
> 於 五，2013-05-24 於 17:05 -0400，Dave Jones 提到：
> > On Fri, May 24, 2013 at 12:02:15PM -0500, Russ Anderson wrote:
> > > On Fri, May 24, 2013 at 11:11:11AM -0500, Robin Holt wrote:
> > > > Russ,
> > > >
> > > > Can we open a bug for the BIOS folks and see if we can get this addressed?
> > >
> > > I already talked with them. It is not in an area that we
> > > normally change, so if there is a bug may be in the Intel
> > > reference code. More investigation is needed to track down
> > > the actual problem, and that could take help from Intel.
> > >
> > > Regardless of that, it is a kernel patch that triggers the
> > > problem. This isn't the first time a kernel change does
> > > the "right thing" but trips across questionable bios/EFI/bootloader
> > > implementation. That still makes it a kernel bug.
> > >
> > > I'm still digging to better understand the root problem.
> >
> > When we rebased the Fedora 18 kernel to 3.9 we had a bunch of reports
> > from people who can no longer boot with what looks like similar symptoms.
> >
> > https://bugzilla.redhat.com/show_bug.cgi?id=964335
> >
> > Dave
> >
>
> The oops on the above bug is similar to my problem on Acer machine,
> could you please ask the reporter try the eccaf52f patch in urgent
> branch on Matt's efi git tree:
>
> https://git.kernel.org/cgit/linux/kernel/git/mfleming/efi.git/commit/?h=urgent&id=eccaf52fee8305d5207ff110950a82c100e459bc
I tried that patch, but it does not fix the problem.
Thanks for the suggestion.
--
Russ Anderson, OS RAS/Partitioning Project Lead
SGI - Silicon Graphics Inc [email protected]
* Matt Fleming <[email protected]> wrote:
> What appears to be happening is that your the EFI runtime services code
> is calling into the EFI boot services code, which is definitely a bug in
> your firmware because we're at runtime, but we've seen other machines
> that do similar things so we usually handle it just fine. However, what
> makes your case different, and the reason you see the above splat, is
> that it's using the physical address of the EFI boot services region,
> not the virtual one we setup with SetVirtualAddressMap(). Which is a
> second firmware bug. Again, we have seen other machines that access
> physical addresses after SetVirtualAddressMap(), but until now we
> haven't had any non-optional code that triggered them.
>
> The only reason I can see that the offending commit would introduce this
> problem is because it calls QueryVariableInfo() at boot time. I notice
> that your machine is an SGI UV one, is there any chance you could get a
> firmware fix for this? If possible, it would be also good to confirm
> that it's this chunk of code in setup_efi_vars(),
>
> status = efi_call_phys4(sys_table->runtime->query_variable_info,
> EFI_VARIABLE_NON_VOLATILE |
> EFI_VARIABLE_BOOTSERVICE_ACCESS |
> EFI_VARIABLE_RUNTIME_ACCESS, &store_size,
> &remaining_size, &var_size);
>
> that later makes GetNextVariable() jump to the physical address of the
> EFI Boot Services region. Because if not, we need to do some more
> digging.
>
> Borislav, how are your 1:1 mapping patches coming along? In theory, once
> those are merged we can gracefully workaround these kinds of issues.
Handling these gracefully without crashing boxes or expecting firmware to
be sane (which is wishful thinking) would be _SO_ preferred ...
I suspect 1:1 mapped is what Windows does - and we simply need to provide
a Windows-EFI compatible environment (which is reality), not just an
EFI-spec environment (which is a fiction).
Thanks,
Ingo
On Fri, 24 May, at 03:49:38PM, Russ Anderson wrote:
> Why does the kernel still try to create /sys/firmware/efivars/
> entries in the original failure even though efi_call_phys4()
> failed? Or does it always try to create those entries
> and GetNextVariable() blows up in the original failure
> but not in my experiment?
CONFIG_EFI_VARS will try to create /sys/firmware/efivars even without
the additional info from the early EFI boot variable code. The early
variable code is only a heuristic that is supposed to improve the
anti-bricking algorithm in the EFI variable code.
--
Matt Fleming, Intel Open Source Technology Center
On Fri, 24 May, at 03:05:39PM, Russ Anderson wrote:
> One of the BIOS guys will build a debug bios next week to help see
> what is going on in the query_variable_info() call.
That would be incredibly useful, thanks.
--
Matt Fleming, Intel Open Source Technology Center
On Fri, May 24, 2013 at 08:45:44AM +0100, Matt Fleming wrote:
> On Thu, 23 May, at 05:23:21PM, Russ Anderson wrote:
> > Interesting data point. The failure is on a rhel7/grub2 root.
> > The identical kernel on a rhel6/grub root boots. So maybe
> > grub2 brings out the failure? I suspect Fedora19/grub2 on
> > EFI should hit the problem (for someone looking to reproduce
> > it).
> >
> > In both cases the kernel boot line options are the same.
>
> I'll bet that rhel7 is using the EFI handover protocol which uses the
> internal mechanisms of the EFI boot stub.
To be more specific (now that I've dug through the code), grub2 (used
by rhel7) uses EFI boot stubs. grub and elilo apparently do not use
EFI boot stubs, so they don't hit the problem (at least on my test
systems).
--
Russ Anderson, OS RAS/Partitioning Project Lead
SGI - Silicon Graphics Inc [email protected]
On Fri, May 24, 2013 at 08:43:31AM +0100, Matt Fleming wrote:
> On Thu, 23 May, at 03:32:34PM, Russ Anderson wrote:
> > efi: mem127: type=4, attr=0xf, range=[0x000000006bb22000-0x000000007ca9c000) (271MB)
>
> EFI_BOOT_SERVICES_CODE
>
> > efi: mem133: type=5, attr=0x800000000000000f, range=[0x000000007daff000-0x000000007dbff000) (1MB)
>
> EFI_RUNTIME_SERVICES_CODE
>
> > EFI Variables Facility v0.08 2004-May-17
> > BUG: unable to handle kernel paging request at 000000007ca95b10
> > IP: [<ffff88007dbf2140>] 0xffff88007dbf213f
>
> > [<ffffffff810499b3>] ? efi_call3+0x43/0x80
> > [<ffffffff810492a7>] ? virt_efi_get_next_variable+0x47/0x1c0
> > [<ffffffff814c8cc0>] ? create_efivars_bin_attributes+0x150/0x150
> > [<ffffffff814c7b55>] ? efivar_init+0xd5/0x390
> > [<ffffffff814c8ae0>] ? efivar_update_sysfs_entries+0x90/0x90
> > [<ffffffff812f906b>] ? kobject_uevent+0xb/0x10
> > [<ffffffff812f812b>] ? kset_register+0x5b/0x70
> > [<ffffffff814c8cc0>] ? create_efivars_bin_attributes+0x150/0x150
> > [<ffffffff814c8d47>] ? efivars_sysfs_init+0x87/0xf0
> > [<ffffffff8100032a>] ? do_one_initcall+0x15a/0x1b0
> > [<ffffffff81a17831>] ? do_basic_setup+0xad/0xce
> > [<ffffffff81a17ae3>] ? kernel_init_freeable+0x291/0x291
> > [<ffffffff81a3708a>] ? sched_init_smp+0x15b/0x162
> > [<ffffffff81a17a5f>] ? kernel_init_freeable+0x20d/0x291
> > [<ffffffff81601eb0>] ? rest_init+0x80/0x80
> > [<ffffffff81601ebe>] ? kernel_init+0xe/0x180
> > [<ffffffff8162179c>] ? ret_from_fork+0x7c/0xb0
> > [<ffffffff81601eb0>] ? rest_init+0x80/0x80
>
> Here's the real call stack leading up to the crash.
>
> What appears to be happening is that your the EFI runtime services code
> is calling into the EFI boot services code, which is definitely a bug in
> your firmware because we're at runtime, but we've seen other machines
> that do similar things so we usually handle it just fine. However, what
> makes your case different, and the reason you see the above splat, is
> that it's using the physical address of the EFI boot services region,
> not the virtual one we setup with SetVirtualAddressMap(). Which is a
> second firmware bug. Again, we have seen other machines that access
> physical addresses after SetVirtualAddressMap(), but until now we
> haven't had any non-optional code that triggered them.
>
> The only reason I can see that the offending commit would introduce this
> problem is because it calls QueryVariableInfo() at boot time. I notice
> that your machine is an SGI UV one, is there any chance you could get a
> firmware fix for this? If possible, it would be also good to confirm
> that it's this chunk of code in setup_efi_vars(),
>
> status = efi_call_phys4(sys_table->runtime->query_variable_info,
> EFI_VARIABLE_NON_VOLATILE |
> EFI_VARIABLE_BOOTSERVICE_ACCESS |
> EFI_VARIABLE_RUNTIME_ACCESS, &store_size,
> &remaining_size, &var_size);
This does trigger the problem. Note that the definition of
QueryVariableInfo() in the UEFI spec says:
The returned MaximumVariableStorageSize, RemainingVariableStorageSize,
MaximumVariableSize information may change immediately after the call
based on other runtime activities including asynchronous error events.
Also, these values associated with different attributes are not
additive in nature.
Note the values may be accurate at the point in time when returned,
but may not be after that.
After the system has transitioned into runtime (after
ExitBootServices() is called), an implementation may not be able to
accurately return information about the Boot Services variable store.
In such cases, EFI_INVALID_PARAMETER should be returned.
It is not clear to me exactly when ExitBootServices() is called.
Our bios is returning a failing indication on the call.
The description from commit cc5a080c5d40c36089bb08a8a16fa3fc7047fe0f is:
efi: Pass boot services variable info to runtime code
EFI variables can be flagged as being accessible only within boot
services. This makes it awkward for us to figure out how much space
they use at runtime. In theory we could figure this out by simply
comparing the results from QueryVariableInfo() to the space used by
all of our variables, but that fails if the platform doesn't garbage
collect on every boot. Thankfully, calling QueryVariableInfo() while
still inside boot services gives a more reliable answer. This patch
passes that information from the EFI boot stub up to the efi platform
code.
"more reliable" is a relative answer. Saving those values for later
use seems inconsistent with the UEFI spec.
> that later makes GetNextVariable() jump to the physical address of the
> EFI Boot Services region. Because if not, we need to do some more
> digging.
>
> Borislav, how are your 1:1 mapping patches coming along? In theory, once
> those are merged we can gracefully workaround these kinds of issues.
>
> --
> Matt Fleming, Intel Open Source Technology Center
--
Russ Anderson, OS RAS/Partitioning Project Lead
SGI - Silicon Graphics Inc [email protected]
On Wed, 29 May 2013, Russ Anderson wrote:
> > What appears to be happening is that your the EFI runtime services code
> > is calling into the EFI boot services code, which is definitely a bug in
> > your firmware because we're at runtime, but we've seen other machines
> > that do similar things so we usually handle it just fine. However, what
> > makes your case different, and the reason you see the above splat, is
> > that it's using the physical address of the EFI boot services region,
> > not the virtual one we setup with SetVirtualAddressMap(). Which is a
> > second firmware bug. Again, we have seen other machines that access
> > physical addresses after SetVirtualAddressMap(), but until now we
> > haven't had any non-optional code that triggered them.
> >
> > The only reason I can see that the offending commit would introduce this
> > problem is because it calls QueryVariableInfo() at boot time. I notice
> > that your machine is an SGI UV one, is there any chance you could get a
> > firmware fix for this? If possible, it would be also good to confirm
> > that it's this chunk of code in setup_efi_vars(),
> >
> > status = efi_call_phys4(sys_table->runtime->query_variable_info,
> > EFI_VARIABLE_NON_VOLATILE |
> > EFI_VARIABLE_BOOTSERVICE_ACCESS |
> > EFI_VARIABLE_RUNTIME_ACCESS, &store_size,
> > &remaining_size, &var_size);
>
> This does trigger the problem. Note that the definition of
> QueryVariableInfo() in the UEFI spec says:
>
> The returned MaximumVariableStorageSize, RemainingVariableStorageSize,
> MaximumVariableSize information may change immediately after the call
> based on other runtime activities including asynchronous error events.
> Also, these values associated with different attributes are not
> additive in nature.
>
> Note the values may be accurate at the point in time when returned,
> but may not be after that.
>
> After the system has transitioned into runtime (after
> ExitBootServices() is called), an implementation may not be able to
> accurately return information about the Boot Services variable store.
> In such cases, EFI_INVALID_PARAMETER should be returned.
>
> It is not clear to me exactly when ExitBootServices() is called.
> Our bios is returning a failing indication on the call.
Yes, but this call is clearly happening way before ExitBootServices() --
see the surrounding code, see for example this in efi_main():
[ ... snip ... ]
setup_efi_vars(boot_params);
setup_efi_pci(boot_params);
status = efi_call_phys3(sys_table->boottime->allocate_pool,
EFI_LOADER_DATA, sizeof(*gdt),
(void **)&gdt);
if (status != EFI_SUCCESS) {
efi_printk("Failed to alloc mem for gdt structure\n");
goto fail;
}
[ ... snip ... ]
We are calling QueryVariableInfo() in setup_efi_vars(), and later on
AllocatePool is being called (through boot table). That'd inherently fail
if we were calling it after ExitBootServices() has happened, but I believe
that call is succeeding on your affected system as well.
--
Jiri Kosina
SUSE Labs
On Thu, May 30, 2013 at 12:22:13AM +0200, Jiri Kosina wrote:
> On Wed, 29 May 2013, Russ Anderson wrote:
>
> > > What appears to be happening is that your the EFI runtime services code
> > > is calling into the EFI boot services code, which is definitely a bug in
> > > your firmware because we're at runtime, but we've seen other machines
> > > that do similar things so we usually handle it just fine. However, what
> > > makes your case different, and the reason you see the above splat, is
> > > that it's using the physical address of the EFI boot services region,
> > > not the virtual one we setup with SetVirtualAddressMap(). Which is a
> > > second firmware bug. Again, we have seen other machines that access
> > > physical addresses after SetVirtualAddressMap(), but until now we
> > > haven't had any non-optional code that triggered them.
> > >
> > > The only reason I can see that the offending commit would introduce this
> > > problem is because it calls QueryVariableInfo() at boot time. I notice
> > > that your machine is an SGI UV one, is there any chance you could get a
> > > firmware fix for this? If possible, it would be also good to confirm
> > > that it's this chunk of code in setup_efi_vars(),
> > >
> > > status = efi_call_phys4(sys_table->runtime->query_variable_info,
> > > EFI_VARIABLE_NON_VOLATILE |
> > > EFI_VARIABLE_BOOTSERVICE_ACCESS |
> > > EFI_VARIABLE_RUNTIME_ACCESS, &store_size,
> > > &remaining_size, &var_size);
> >
> > This does trigger the problem. Note that the definition of
> > QueryVariableInfo() in the UEFI spec says:
> >
> > The returned MaximumVariableStorageSize, RemainingVariableStorageSize,
> > MaximumVariableSize information may change immediately after the call
> > based on other runtime activities including asynchronous error events.
> > Also, these values associated with different attributes are not
> > additive in nature.
> >
> > Note the values may be accurate at the point in time when returned,
> > but may not be after that.
> >
> > After the system has transitioned into runtime (after
> > ExitBootServices() is called), an implementation may not be able to
> > accurately return information about the Boot Services variable store.
> > In such cases, EFI_INVALID_PARAMETER should be returned.
> >
> > It is not clear to me exactly when ExitBootServices() is called.
> > Our bios is returning a failing indication on the call.
>
> Yes, but this call is clearly happening way before ExitBootServices() --
> see the surrounding code, see for example this in efi_main():
>
> [ ... snip ... ]
> setup_efi_vars(boot_params);
>
> setup_efi_pci(boot_params);
>
> status = efi_call_phys3(sys_table->boottime->allocate_pool,
> EFI_LOADER_DATA, sizeof(*gdt),
> (void **)&gdt);
> if (status != EFI_SUCCESS) {
> efi_printk("Failed to alloc mem for gdt structure\n");
> goto fail;
> }
> [ ... snip ... ]
Yes. Note the failing call is sys_table->runtime while all the
other calls are sys_table->boottime and seem to work. Not sure
why the sys_table->runtime call has a problem but it may be
a clue. Could something in the runtime path not be set up???
> We are calling QueryVariableInfo() in setup_efi_vars(), and later on
> AllocatePool is being called (through boot table).
On my system the QueryVariableInfo() call fails, so AllocatePool()
is not called in setup_efi_vars().
> That'd inherently fail
> if we were calling it after ExitBootServices() has happened, but I believe
> that call is succeeding on your affected system as well.
>
> --
> Jiri Kosina
> SUSE Labs
--
Russ Anderson, OS RAS/Partitioning Project Lead
SGI - Silicon Graphics Inc [email protected]
On Wed, 29 May 2013, Russ Anderson wrote:
> > Yes, but this call is clearly happening way before ExitBootServices() --
> > see the surrounding code, see for example this in efi_main():
> >
> > [ ... snip ... ]
> > setup_efi_vars(boot_params);
> >
> > setup_efi_pci(boot_params);
> >
> > status = efi_call_phys3(sys_table->boottime->allocate_pool,
> > EFI_LOADER_DATA, sizeof(*gdt),
> > (void **)&gdt);
> > if (status != EFI_SUCCESS) {
> > efi_printk("Failed to alloc mem for gdt structure\n");
> > goto fail;
> > }
> > [ ... snip ... ]
>
> Yes. Note the failing call is sys_table->runtime while all the
> other calls are sys_table->boottime and seem to work. Not sure
> why the sys_table->runtime call has a problem but it may be
> a clue. Could something in the runtime path not be set up???
That was my original idea early today as well. My understanding of the
UEFI spec is admittedly limited, but afaics calling runtime method from
boot environment should be a valid thing to do ... ?
> > We are calling QueryVariableInfo() in setup_efi_vars(), and later on
> > AllocatePool is being called (through boot table).
>
> On my system the QueryVariableInfo() call fails, so AllocatePool()
> is not called in setup_efi_vars().
But it's being called later on coming back to efi_main(). That was just a
poor man's demonstration attempt why this code is running before
ExitBootServices() has been called.
--
Jiri Kosina
SUSE Labs
於 四，2013-05-30 於 00:53 +0200，Jiri Kosina 提到：
> On Wed, 29 May 2013, Russ Anderson wrote:
>
> > > Yes, but this call is clearly happening way before ExitBootServices() --
> > > see the surrounding code, see for example this in efi_main():
> > >
> > > [ ... snip ... ]
> > > setup_efi_vars(boot_params);
> > >
> > > setup_efi_pci(boot_params);
> > >
> > > status = efi_call_phys3(sys_table->boottime->allocate_pool,
> > > EFI_LOADER_DATA, sizeof(*gdt),
> > > (void **)&gdt);
> > > if (status != EFI_SUCCESS) {
> > > efi_printk("Failed to alloc mem for gdt structure\n");
> > > goto fail;
> > > }
> > > [ ... snip ... ]
> >
> > Yes. Note the failing call is sys_table->runtime while all the
> > other calls are sys_table->boottime and seem to work. Not sure
> > why the sys_table->runtime call has a problem but it may be
> > a clue. Could something in the runtime path not be set up???
>
> That was my original idea early today as well. My understanding of the
> UEFI spec is admittedly limited, but afaics calling runtime method from
> boot environment should be a valid thing to do ... ?
QueryVariableInfo() is a runtime services, all runtime services should
available bother on boot time and runtime:
UEFI spec 2.3.1 P.109:
Runtime Services
Functions that are available before and after any call to
ExitBootServices(). These functions are described in Section 7.
>
> > > We are calling QueryVariableInfo() in setup_efi_vars(), and later on
> > > AllocatePool is being called (through boot table).
> >
> > On my system the QueryVariableInfo() call fails, so AllocatePool()
> > is not called in setup_efi_vars().
>
> But it's being called later on coming back to efi_main(). That was just a
> poor man's demonstration attempt why this code is running before
> ExitBootServices() has been called.
>
Yes, I agreed your point, the space information of
EFI_VARIABLE_BOOTSERVICE_ACCESS should still return by
QueryVariableInfo() because we call it before ExitBootServices():
arch/x86/boot/compressed/eboot.c
efi_main(void *handle, efi_system_table_t *_table, struct boot_params *boot_params)
..
sys_table = _table;
/* Check if we were booted by the EFI firmware */
if (sys_table->hdr.signature != EFI_SYSTEM_TABLE_SIGNATURE)
goto fail;
boot_params->secure_boot = get_secure_boot(sys_table) /* check does BIOS in secure boot mode */
setup_graphics(boot_params);
setup_efi_vars(boot_params); /* Pass boot services variable info to runtime code, call QueryVariableInfo() */
...
status = exit_boot(boot_params, handle); /* call ExitBootServices() */
...
Thanks a lot!
Joey Lee
於 三，2013-05-29 於 17:46 -0500，Russ Anderson 提到：
> On Thu, May 30, 2013 at 12:22:13AM +0200, Jiri Kosina wrote:
> > On Wed, 29 May 2013, Russ Anderson wrote:
> >
> > > > What appears to be happening is that your the EFI runtime services code
> > > > is calling into the EFI boot services code, which is definitely a bug in
> > > > your firmware because we're at runtime, but we've seen other machines
> > > > that do similar things so we usually handle it just fine. However, what
> > > > makes your case different, and the reason you see the above splat, is
> > > > that it's using the physical address of the EFI boot services region,
> > > > not the virtual one we setup with SetVirtualAddressMap(). Which is a
> > > > second firmware bug. Again, we have seen other machines that access
> > > > physical addresses after SetVirtualAddressMap(), but until now we
> > > > haven't had any non-optional code that triggered them.
> > > >
> > > > The only reason I can see that the offending commit would introduce this
> > > > problem is because it calls QueryVariableInfo() at boot time. I notice
> > > > that your machine is an SGI UV one, is there any chance you could get a
> > > > firmware fix for this? If possible, it would be also good to confirm
> > > > that it's this chunk of code in setup_efi_vars(),
> > > >
> > > > status = efi_call_phys4(sys_table->runtime->query_variable_info,
> > > > EFI_VARIABLE_NON_VOLATILE |
> > > > EFI_VARIABLE_BOOTSERVICE_ACCESS |
> > > > EFI_VARIABLE_RUNTIME_ACCESS, &store_size,
> > > > &remaining_size, &var_size);
> > >
> > > This does trigger the problem. Note that the definition of
> > > QueryVariableInfo() in the UEFI spec says:
> > >
> > > The returned MaximumVariableStorageSize, RemainingVariableStorageSize,
> > > MaximumVariableSize information may change immediately after the call
> > > based on other runtime activities including asynchronous error events.
> > > Also, these values associated with different attributes are not
> > > additive in nature.
> > >
> > > Note the values may be accurate at the point in time when returned,
> > > but may not be after that.
> > >
> > > After the system has transitioned into runtime (after
> > > ExitBootServices() is called), an implementation may not be able to
> > > accurately return information about the Boot Services variable store.
> > > In such cases, EFI_INVALID_PARAMETER should be returned.
> > >
> > > It is not clear to me exactly when ExitBootServices() is called.
> > > Our bios is returning a failing indication on the call.
> >
> > Yes, but this call is clearly happening way before ExitBootServices() --
> > see the surrounding code, see for example this in efi_main():
> >
> > [ ... snip ... ]
> > setup_efi_vars(boot_params);
> >
> > setup_efi_pci(boot_params);
> >
> > status = efi_call_phys3(sys_table->boottime->allocate_pool,
> > EFI_LOADER_DATA, sizeof(*gdt),
> > (void **)&gdt);
> > if (status != EFI_SUCCESS) {
> > efi_printk("Failed to alloc mem for gdt structure\n");
> > goto fail;
> > }
> > [ ... snip ... ]
>
> Yes. Note the failing call is sys_table->runtime while all the
> other calls are sys_table->boottime and seem to work. Not sure
> why the sys_table->runtime call has a problem but it may be
> a clue. Could something in the runtime path not be set up???
>
Per UEFI spec Section 6, all runtime services should a available both on
boot time and runtime. And, we query the EFI_VARIABLE_BOOTSERVICE_ACCESS
space information before ExitBootServices(), that means we call it in
boot time, so, QueryVariableInfo() should return information to us.
Back to the kernel oops, the oops happened in runtime environment when
efivar_init running. As Matt's point out as following:
於 五，2013-05-24 於 08:43 +0100，Matt Fleming 提到：
> On Thu, 23 May, at 03:32:34PM, Russ Anderson wrote:
> > efi: mem127: type=4, attr=0xf, range=[0x000000006bb22000-0x000000007ca9c000) (271MB)
>
> EFI_BOOT_SERVICES_CODE
Per UEFI 2.3.1 Section 6.2, type 4 is EfiBootServicesCode. This area available for
OS using after ExitBootServices():
UEFI 2.3.1 P.133
EfiBootServicesData Memory available for general use.
So, any runtime services should not access this area, it's the first thing need to check
the BIOS code for why.
> > efi: mem133: type=5, attr=0x800000000000000f, range=[0x000000007daff000-0x000000007dbff000) (1MB)
>
> EFI_RUNTIME_SERVICES_CODE
>
> > EFI Variables Facility v0.08 2004-May-17
> > BUG: unable to handle kernel paging request at 000000007ca95b10
> > IP: [<ffff88007dbf2140>] 0xffff88007dbf213f
[...]
What appears to be happening is that your the EFI runtime services code
> is calling into the EFI boot services code, which is definitely a bug in
> your firmware because we're at runtime, but we've seen other machines
> that do similar things so we usually handle it just fine. However, what
> makes your case different, and the reason you see the above splat, is
> that it's using the physical address of the EFI boot services region,
> not the virtual one we setup with SetVirtualAddressMap(). Which is a
> second firmware bug. Again, we have seen other machines that access
> physical addresses after SetVirtualAddressMap(), but until now we
> haven't had any non-optional code that triggered them.
Actually, when efivars doing init, system is already in virtual mode not just runtime
mode, that means OS already call SetVirtualAddressMap() to set the EfiBootServicesCode
and EfiBootServicesData to virtual address.
As Matt's point out, the second things need to check is why does paging fault
happen on physical address after SetVirtualAddressMap().
Currently, the only clue is we call QueryVariableInfo() in boottime for grab the space
information of EFI_VARIABLE_BOOTSERVICE_ACCESS, but it should permitted because it's
in boot time:
於 五，2013-05-24 於 08:43 +0100，Matt Fleming 提到：
> status = efi_call_phys4(sys_table->runtime->query_variable_info,
> EFI_VARIABLE_NON_VOLATILE |
> EFI_VARIABLE_BOOTSERVICE_ACCESS |
> EFI_VARIABLE_RUNTIME_ACCESS, &store_size,
> &remaining_size, &var_size);
Thanks a lot!
Joey Lee
On Thu, May 30, 2013 at 10:16:12AM +0800, joeyli wrote:
> 於 四，2013-05-30 於 00:53 +0200，Jiri Kosina 提到：
> > On Wed, 29 May 2013, Russ Anderson wrote:
> >
> > > > Yes, but this call is clearly happening way before ExitBootServices() --
> > > > see the surrounding code, see for example this in efi_main():
> > > >
> > > > [ ... snip ... ]
> > > > setup_efi_vars(boot_params);
> > > >
> > > > setup_efi_pci(boot_params);
> > > >
> > > > status = efi_call_phys3(sys_table->boottime->allocate_pool,
> > > > EFI_LOADER_DATA, sizeof(*gdt),
> > > > (void **)&gdt);
> > > > if (status != EFI_SUCCESS) {
> > > > efi_printk("Failed to alloc mem for gdt structure\n");
> > > > goto fail;
> > > > }
> > > > [ ... snip ... ]
> > >
> > > Yes. Note the failing call is sys_table->runtime while all the
> > > other calls are sys_table->boottime and seem to work. Not sure
> > > why the sys_table->runtime call has a problem but it may be
> > > a clue. Could something in the runtime path not be set up???
> >
> > That was my original idea early today as well. My understanding of the
> > UEFI spec is admittedly limited, but afaics calling runtime method from
> > boot environment should be a valid thing to do ... ?
>
> QueryVariableInfo() is a runtime services, all runtime services should
> available bother on boot time and runtime:
>
> UEFI spec 2.3.1 P.109:
> Runtime Services
> Functions that are available before and after any call to
> ExitBootServices(). These functions are described in Section 7.
That's a great idea. This patch moves the QueryVariableInfo()
call from bootime to runtime, in efi_late_init(). The attached
patch is consistent with the UEFI spec and avoids the problem.
Thanks,
--
Russ Anderson, OS RAS/Partitioning Project Lead
SGI - Silicon Graphics Inc [email protected]
On Thu, 2013-05-30 at 17:17 -0500, Russ Anderson wrote:
> That's a great idea. This patch moves the QueryVariableInfo()
> call from bootime to runtime, in efi_late_init(). The attached
> patch is consistent with the UEFI spec and avoids the problem.
No, that defeats the entire point of the original patch.
--
Matthew Garrett | [email protected]
????{.n?+???????+%?????ݶ??w??{.n?+????{??G?????{ay? ʇڙ?,j??f???h?????????z_??(?階?ݢj"???m??????G????????????&???~???iO???z??v?^?m???? ????????I?
On Thu, 30 May 2013, Russ Anderson wrote:
> > > > > Yes, but this call is clearly happening way before ExitBootServices() --
> > > > > see the surrounding code, see for example this in efi_main():
> > > > >
> > > > > [ ... snip ... ]
> > > > > setup_efi_vars(boot_params);
> > > > >
> > > > > setup_efi_pci(boot_params);
> > > > >
> > > > > status = efi_call_phys3(sys_table->boottime->allocate_pool,
> > > > > EFI_LOADER_DATA, sizeof(*gdt),
> > > > > (void **)&gdt);
> > > > > if (status != EFI_SUCCESS) {
> > > > > efi_printk("Failed to alloc mem for gdt structure\n");
> > > > > goto fail;
> > > > > }
> > > > > [ ... snip ... ]
> > > >
> > > > Yes. Note the failing call is sys_table->runtime while all the
> > > > other calls are sys_table->boottime and seem to work. Not sure
> > > > why the sys_table->runtime call has a problem but it may be
> > > > a clue. Could something in the runtime path not be set up???
> > >
> > > That was my original idea early today as well. My understanding of the
> > > UEFI spec is admittedly limited, but afaics calling runtime method from
> > > boot environment should be a valid thing to do ... ?
> >
> > QueryVariableInfo() is a runtime services, all runtime services should
> > available bother on boot time and runtime:
> >
> > UEFI spec 2.3.1 P.109:
> > Runtime Services
> > Functions that are available before and after any call to
> > ExitBootServices(). These functions are described in Section 7.
>
> That's a great idea. This patch moves the QueryVariableInfo()
> call from bootime to runtime, in efi_late_init(). The attached
> patch is consistent with the UEFI spec and avoids the problem.
Unfortunately that means that you can as well throw the patch away
completely.
The sole point is to run the QueryVariableInfo() from the boot
environment, in order to obtain more accurate information.
And it's a valid thing to do, according to UEFI specification.
--
Jiri Kosina
SUSE Labs
On Thu, May 30, 2013 at 10:21:53PM +0000, Matthew Garrett wrote:
> On Thu, 2013-05-30 at 17:17 -0500, Russ Anderson wrote:
>
> > That's a great idea. This patch moves the QueryVariableInfo()
> > call from bootime to runtime, in efi_late_init(). The attached
> > patch is consistent with the UEFI spec and avoids the problem.
>
> No, that defeats the entire point of the original patch.
How so? It is still calling QueryVariableInfo()
before the data is used.
--
Russ Anderson, OS RAS/Partitioning Project Lead
SGI - Silicon Graphics Inc [email protected]
On Thu, 30 May 2013, Russ Anderson wrote:
> > > That's a great idea. This patch moves the QueryVariableInfo()
> > > call from bootime to runtime, in efi_late_init(). The attached
> > > patch is consistent with the UEFI spec and avoids the problem.
> >
> > No, that defeats the entire point of the original patch.
>
> How so? It is still calling QueryVariableInfo()
> before the data is used.
You lose information provided by QueryVariableInfo() about boot-only
variables once the transition boot -> runtime has happened.
--
Jiri Kosina
SUSE Labs
On Thu, 2013-05-30 at 17:28 -0500, Russ Anderson wrote:
> On Thu, May 30, 2013 at 10:21:53PM +0000, Matthew Garrett wrote:
> > On Thu, 2013-05-30 at 17:17 -0500, Russ Anderson wrote:
> >
> > > That's a great idea. This patch moves the QueryVariableInfo()
> > > call from bootime to runtime, in efi_late_init(). The attached
> > > patch is consistent with the UEFI spec and avoids the problem.
> >
> > No, that defeats the entire point of the original patch.
>
> How so? It is still calling QueryVariableInfo()
> before the data is used.
We want to know how much space is used by variables that aren't visible
at runtime.
--
Matthew Garrett | [email protected]
????{.n?+???????+%?????ݶ??w??{.n?+????{??G?????{ay? ʇڙ?,j??f???h?????????z_??(?階?ݢj"???m??????G????????????&???~???iO???z??v?^?m???? ????????I?
On Fri, May 31, 2013 at 12:30:43AM +0200, Jiri Kosina wrote:
> On Thu, 30 May 2013, Russ Anderson wrote:
>
> > > > That's a great idea. This patch moves the QueryVariableInfo()
> > > > call from bootime to runtime, in efi_late_init(). The attached
> > > > patch is consistent with the UEFI spec and avoids the problem.
> > >
> > > No, that defeats the entire point of the original patch.
> >
> > How so? It is still calling QueryVariableInfo()
> > before the data is used.
>
> You lose information provided by QueryVariableInfo() about boot-only
> variables once the transition boot -> runtime has happened.
Is that information really more important than the ability
to boot?
Correct me if I'm wrong, but linux was able to boot without
the boottime QueryVariableInfo() call up until 3.9-rc7,
and it still does on systems that do not use EFI stubs (ie
grub and elilo). It is only when linux uses EFI stubs (ie
grub2) that linux makes the boottime QueryVariableInfo()
call. So why is that call, or whatever is dependent on it,
more important than booting?
Thanks,
--
Russ Anderson, OS RAS/Partitioning Project Lead
SGI - Silicon Graphics Inc [email protected]
On Thu, May 30, 2013 at 10:32:09PM +0000, Matthew Garrett wrote:
> On Thu, 2013-05-30 at 17:28 -0500, Russ Anderson wrote:
> > On Thu, May 30, 2013 at 10:21:53PM +0000, Matthew Garrett wrote:
> > > On Thu, 2013-05-30 at 17:17 -0500, Russ Anderson wrote:
> > >
> > > > That's a great idea. This patch moves the QueryVariableInfo()
> > > > call from bootime to runtime, in efi_late_init(). The attached
> > > > patch is consistent with the UEFI spec and avoids the problem.
> > >
> > > No, that defeats the entire point of the original patch.
> >
> > How so? It is still calling QueryVariableInfo()
> > before the data is used.
>
> We want to know how much space is used by variables that aren't visible
> at runtime.
We want to boot. We could boot up through 3.9-rc7.
Knowing how much space is used by variables that aren't
visible at runtime it moot if you can't boot.
And again, maybe this is a bios bug - we have bios people
looking into it - and maybe that call _should_ work, but
the fact is the kernel booted without that change[1] and does
not boot with it.
[1] commit cc5a080c5d40c36089bb08a8a16fa3fc7047fe0f
Thanks,
--
Russ Anderson, OS RAS/Partitioning Project Lead
SGI - Silicon Graphics Inc [email protected]
於 四，2013-05-30 於 21:17 -0500，Russ Anderson 提到：
> On Fri, May 31, 2013 at 12:30:43AM +0200, Jiri Kosina wrote:
> > On Thu, 30 May 2013, Russ Anderson wrote:
> >
> > > > > That's a great idea. This patch moves the QueryVariableInfo()
> > > > > call from bootime to runtime, in efi_late_init(). The attached
> > > > > patch is consistent with the UEFI spec and avoids the problem.
> > > >
> > > > No, that defeats the entire point of the original patch.
> > >
> > > How so? It is still calling QueryVariableInfo()
> > > before the data is used.
> >
> > You lose information provided by QueryVariableInfo() about boot-only
> > variables once the transition boot -> runtime has happened.
>
> Is that information really more important than the ability
> to boot?
>
> Correct me if I'm wrong, but linux was able to boot without
> the boottime QueryVariableInfo() call up until 3.9-rc7,
> and it still does on systems that do not use EFI stubs (ie
> grub and elilo). It is only when linux uses EFI stubs (ie
> grub2) that linux makes the boottime QueryVariableInfo()
> call. So why is that call, or whatever is dependent on it,
> more important than booting?
>
>
>
> Thanks,
It related to BIOS's garbage collection behavior of UEFI variable
storage.
The used space of non volatile boottime variables is useful to us for
calculate the active_size, please reference 31ff2f2 patch:
https://lkml.org/lkml/2013/4/15/476
An earlier 68d9298 patch to avoid some machines bricked when more than
50% of the system flash is in use, because the garbage collection will
not trigger on those machines.
We need find out the size of system flash space indeed usage for avoid
this problem. So, cc5a080c5 patch call QueryVariableInfo() to grab the
usage information in boot time.
Calling QueryVariableInfo() at boot time should not causes side effect
until your issue show up. Before this issue happen, avoid bricking some
machines is also important.
Thanks a lot!
Joey Lee
* Russ Anderson <[email protected]> wrote:
> On Thu, May 30, 2013 at 10:32:09PM +0000, Matthew Garrett wrote:
> > On Thu, 2013-05-30 at 17:28 -0500, Russ Anderson wrote:
> > > On Thu, May 30, 2013 at 10:21:53PM +0000, Matthew Garrett wrote:
> > > > On Thu, 2013-05-30 at 17:17 -0500, Russ Anderson wrote:
> > > >
> > > > > That's a great idea. This patch moves the QueryVariableInfo()
> > > > > call from bootime to runtime, in efi_late_init(). The attached
> > > > > patch is consistent with the UEFI spec and avoids the problem.
> > > >
> > > > No, that defeats the entire point of the original patch.
> > >
> > > How so? It is still calling QueryVariableInfo()
> > > before the data is used.
> >
> > We want to know how much space is used by variables that aren't visible
> > at runtime.
>
> We want to boot. We could boot up through 3.9-rc7.
>
> Knowing how much space is used by variables that aren't
> visible at runtime it moot if you can't boot.
Exactly - fixing a boot regression is _WAY_ more important than pretty
much any other concern.
and the boot breakage is not limited to UV systems - the thread mentioned
a couple of other systems as well.
So it's an absolute no-brainer that this change should be reverted or
fixed via your patch.
Once a safer mechanism is implemented to call QueryVariableInfo() earlier
(Boris's patches?) the change can be reintroduced.
Thanks,
Ingo
* Jiri Kosina <[email protected]> wrote:
> On Thu, 30 May 2013, Russ Anderson wrote:
>
> > > > > > Yes, but this call is clearly happening way before ExitBootServices() --
> > > > > > see the surrounding code, see for example this in efi_main():
> > > > > >
> > > > > > [ ... snip ... ]
> > > > > > setup_efi_vars(boot_params);
> > > > > >
> > > > > > setup_efi_pci(boot_params);
> > > > > >
> > > > > > status = efi_call_phys3(sys_table->boottime->allocate_pool,
> > > > > > EFI_LOADER_DATA, sizeof(*gdt),
> > > > > > (void **)&gdt);
> > > > > > if (status != EFI_SUCCESS) {
> > > > > > efi_printk("Failed to alloc mem for gdt structure\n");
> > > > > > goto fail;
> > > > > > }
> > > > > > [ ... snip ... ]
> > > > >
> > > > > Yes. Note the failing call is sys_table->runtime while all the
> > > > > other calls are sys_table->boottime and seem to work. Not sure
> > > > > why the sys_table->runtime call has a problem but it may be
> > > > > a clue. Could something in the runtime path not be set up???
> > > >
> > > > That was my original idea early today as well. My understanding of the
> > > > UEFI spec is admittedly limited, but afaics calling runtime method from
> > > > boot environment should be a valid thing to do ... ?
> > >
> > > QueryVariableInfo() is a runtime services, all runtime services should
> > > available bother on boot time and runtime:
> > >
> > > UEFI spec 2.3.1 P.109:
> > > Runtime Services
> > > Functions that are available before and after any call to
> > > ExitBootServices(). These functions are described in Section 7.
> >
> > That's a great idea. This patch moves the QueryVariableInfo()
> > call from bootime to runtime, in efi_late_init(). The attached
> > patch is consistent with the UEFI spec and avoids the problem.
>
> Unfortunately that means that you can as well throw the patch away
> completely.
>
> The sole point is to run the QueryVariableInfo() from the boot
> environment, in order to obtain more accurate information. And it's a
> valid thing to do, according to UEFI specification.
Argh!
The 'UEFI specification', if contradicted by reality, has _NO VALUE_
whatsoever. Full stop.
Any specification is only valuable as long as it's followed by everyone:
but once that is not the case, it does not matter much what the written
wishes on the paper express.
What matters is that a system that booted fine with v3.8 does not boot
anymore due to a change we did to the EFI code. And the only valid
response to that is to make it work. Full stop!
And this is not only about SGI/UV systems: in this thread there were
multiple systems mentioned that _DO NOT BOOT_ due to this change.
So this change needs to be reverted or fixed.
Thanks,
Ingo
On Fri, 31 May 2013, Ingo Molnar wrote:
> So this change needs to be reverted or fixed.
I don't think anyone is arguing against that.
My remark was purely to describe the current status quo and help to
understand what exactly is happening, i.e.:
- QueryVariableInfo() should be a valid thing to do from inside boot
environment, according to the spec
- now we see that at least SGI bios (an probably other incarnations)
think otherwise
- if we are not able to fix / work around the bug in BIOS (*), we have to
make a choice between two evils -- either increase likelyhood of
bricking certain machines due to filling the EFI storage space, or
break booting on broken BIOSen
- the theory is that Borislav's 1:1 mapping patches will work this around;
one of the supporting arguments being that it's probably what Windows is
doing. I believe Borislav is in the process of testing this. But the
patches are not ready for mainline yet.
(*) If one would be naive enough, he'd believe that the pressure should be
put on the BIOS writers instead of OS to fix the bug :)
--
Jiri Kosina
SUSE Labs
* Jiri Kosina <[email protected]> wrote:
> On Fri, 31 May 2013, Ingo Molnar wrote:
>
> > > Unfortunately that means that you can as well throw the patch away
> > > completely.
> > >
> > > The sole point is to run the QueryVariableInfo() from the boot
> > > environment, in order to obtain more accurate information. And it's
> > > a valid thing to do, according to UEFI specification.
> >
> > So this change needs to be reverted or fixed.
>
> I don't think anyone is arguing against that.
The statement I replied to above can be fairly read as arguing against it.
Anyway, given that the fix (workaround) from Boris seems elaborate so late
in the v3.10 cycle, does everyone else agree with a revert as well?
Thanks,
Ingo
On Fri, May 31, 2013 at 7:40 AM, Ingo Molnar <[email protected]> wrote:
>
> * Jiri Kosina <[email protected]> wrote:
>
>> On Fri, 31 May 2013, Ingo Molnar wrote:
>>
>> > > Unfortunately that means that you can as well throw the patch away
>> > > completely.
>> > >
>> > > The sole point is to run the QueryVariableInfo() from the boot
>> > > environment, in order to obtain more accurate information. And it's
>> > > a valid thing to do, according to UEFI specification.
>> >
>> > So this change needs to be reverted or fixed.
>>
>> I don't think anyone is arguing against that.
>
> The statement I replied to above can be fairly read as arguing against it.
>
> Anyway, given that the fix (workaround) from Boris seems elaborate so late
> in the v3.10 cycle, does everyone else agree with a revert as well?
If you go with a revert, please make sure to tag it for stable to it
gets picked up with 3.9.y.
josh
On Fri, May 31, 2013 at 01:06:09PM +0200, Jiri Kosina wrote:
> (*) If one would be naive enough, he'd believe that the pressure
> should be put on the BIOS writers instead of OS to fix the bug :)
Oh, definitely pressure on BIOS dudes. If they're in violation of the
spec and they still can fix it in time, they better. I'm sick and tired
of having to deal with BIOS idiocy in kernel code.
* Borislav Petkov <[email protected]> wrote:
> On Fri, May 31, 2013 at 01:06:09PM +0200, Jiri Kosina wrote:
> >
> > (*) If one would be naive enough, he'd believe that the pressure
> > should be put on the BIOS writers instead of OS to fix the bug :)
>
> Oh, definitely pressure on BIOS dudes. If they're in violation of the
> spec and they still can fix it in time, they better. I'm sick and tired
> of having to deal with BIOS idiocy in kernel code.
I'm all for some BIOS quality bashing, but the reality is:
1) It's not just about SGI/UV systems but apparently about several
different types of x86 laptops produce the same boot crash pattern:
most of which come from manufacturers that simply don't care about
Linux all that much.
So by not reverting we'd screw our users, not put any recognizable
pressure on any BIOS writers or manufacturers.
2) Obviously Windows does not crash, and that's what most laptops test.
So our realistic 'spec target' is not some sort of pure 'EFI spec',
but EFI implementations _tested under Windows_. Consider it an
'extended EFI compatibility spec'.
3) There's a better, more robust firmware environment approach being
worked on (by you?) that avoids such 1:1 physical mapping assumption
crashes. That's something worth doing anyway, so why not delay the
early QueryVariableInfo() call change to when that enviroment is
properly implemented?
4) The revert is easy, and the functionality the original patch provided
was a marginal increase in debug output to begin with...
So to me the right approach seems to be:
A: revert now for v3.10
B: implement 1:1 mappings environment for firmware, for v3.11
C: reintroduce the early QueryVariableInfo() call again, in v3.11
Thanks,
Ingo
On Fri, May 31, 2013 at 02:43:56PM +0200, Ingo Molnar wrote:
> 4) The revert is easy, and the functionality the original patch provided
> was a marginal increase in debug output to begin with...
I agree that a revert is probably the right thing to do here, but the
original patch was there to permit a more accurate calculation of the
amount of nvram in use, not to provide additional debug information.
Reverting it is going to differently break a different set of systems
--
Matthew Garrett | [email protected]
On 05/29/2013 01:16 PM, Russ Anderson wrote:
>
> To be more specific (now that I've dug through the code), grub2 (used
> by rhel7) uses EFI boot stubs. grub and elilo apparently do not use
> EFI boot stubs, so they don't hit the problem (at least on my test
> systems).
>
Grub2 *can* use the EFI boot stub... as far as I know it doesn't by
default, but the way it is used in RHEL does.
Either way, we need to fix this and fix it soon.
-hpa
On Fri, 2013-05-31 at 15:34 +0100, Matthew Garrett wrote:
> On Fri, May 31, 2013 at 02:43:56PM +0200, Ingo Molnar wrote:
>
> > 4) The revert is easy, and the functionality the original patch provided
> > was a marginal increase in debug output to begin with...
>
> I agree that a revert is probably the right thing to do here, but the
> original patch was there to permit a more accurate calculation of the
> amount of nvram in use, not to provide additional debug information.
> Reverting it is going to differently break a different set of systems
The only ones that are broken are the Samsung ones. Samsung claims to
have fixed their UEFI firmware, so we could refer any problems to them.
The signature of the Samsung failure, which this is guarding against is
that the laptop gets bricked, so it really is a nasty choice of poisons
we have to pick...
Could we hedge the QueryVariableInfo checks with a test for Samsung in
the UEFI identity strings?
James
On 05/31/2013 07:42 AM, James Bottomley wrote:
>
> The only ones that are broken are the Samsung ones. Samsung claims to
> have fixed their UEFI firmware, so we could refer any problems to them.
>
> The signature of the Samsung failure, which this is guarding against is
> that the laptop gets bricked, so it really is a nasty choice of poisons
> we have to pick...
>
Are we sure it is only Samsung? I was under the impression there were
other systems with similar failures?
-hpa
On Fri, May 31, 2013 at 07:42:37AM -0700, James Bottomley wrote:
> On Fri, 2013-05-31 at 15:34 +0100, Matthew Garrett wrote:
> > I agree that a revert is probably the right thing to do here, but the
> > original patch was there to permit a more accurate calculation of the
> > amount of nvram in use, not to provide additional debug information.
> > Reverting it is going to differently break a different set of systems
>
> The only ones that are broken are the Samsung ones. Samsung claims to
> have fixed their UEFI firmware, so we could refer any problems to them.
No, reverting this gets us back to the old state of refusing any writes
if more than 50% of the variable store *appears* to be used, regardless
of whether it's actually used. Which, unfortunately, makes it impossible
to install Linux on most UEFI machines. In any case, Samsung clearly
haven't fixed this problem on a pile of machines that have already
shipped.
> Could we hedge the QueryVariableInfo checks with a test for Samsung in
> the UEFI identity strings?
We could, but apparently some Lenovos also have a similar problem. We
just don't have the information we need to implement a comprehensive
blacklist, and if we get it wrong we're back to destroying people's
hardware.
--
Matthew Garrett | [email protected]
On Fri, May 31, 2013 at 03:48:26PM +0100, Matthew Garrett wrote:
> On Fri, May 31, 2013 at 07:42:37AM -0700, James Bottomley wrote:
> > On Fri, 2013-05-31 at 15:34 +0100, Matthew Garrett wrote:
> > > I agree that a revert is probably the right thing to do here, but the
> > > original patch was there to permit a more accurate calculation of the
> > > amount of nvram in use, not to provide additional debug information.
> > > Reverting it is going to differently break a different set of systems
> >
> > The only ones that are broken are the Samsung ones. Samsung claims to
> > have fixed their UEFI firmware, so we could refer any problems to them.
>
> No, reverting this gets us back to the old state of refusing any writes
> if more than 50% of the variable store *appears* to be used, regardless
> of whether it's actually used. Which, unfortunately, makes it impossible
> to install Linux on most UEFI machines.
When did writing EFI variables to nvram become necessary to boot
on UEFI? And if it is necessary, why is it that only linux boot
loaders that use EFI stubs (generally grub2) need it? The current
kernel boots using EFI/grub and EFI/elilo. It is just when
EFI stubs are used that the boot fails.
I'm missing the background on why linux needs to write so many
EFI variables to nvram that it fills up nvram. What is that
all about?
> In any case, Samsung clearly
> haven't fixed this problem on a pile of machines that have already
> shipped.
Which means the previous patch(es) that caused the bricking should
get pulled, too.
--
Russ Anderson, OS RAS/Partitioning Project Lead
SGI - Silicon Graphics Inc [email protected]
On Fri, May 31, 2013 at 10:43:49AM -0500, Russ Anderson wrote:
> When did writing EFI variables to nvram become necessary to boot on
> UEFI? And if it is necessary, why is it that only linux boot loaders
> that use EFI stubs (generally grub2) need it? The current kernel
> boots using EFI/grub and EFI/elilo. It is just when EFI stubs are
> used that the boot fails.
I think you've misunderstood the problem. If nvram becaomes full, some
systems crash during firmware initialisation. So we can't let nvram
become full. The obvious thing to do here is to look at the values from
QueryVariableInfo, but many systems won't perform any garbage collection
until they're almost out of space and so variables that have been
deleted still show up as used space. We can work around that by adding
up the size of the variables ourselves, but that only gives us the value
for runtime-visible variables. We also need to know how much space is
used by variables that are only visible during boot, hence calling
QueryVariableInfo before ExitBootServices.
> Which means the previous patch(es) that caused the bricking should
> get pulled, too.
There are no patches that cause the bricking.
--
Matthew Garrett | [email protected]
On Fri, 2013-05-31 at 17:28 +0100, Matthew Garrett wrote:
> On Fri, May 31, 2013 at 10:43:49AM -0500, Russ Anderson wrote:
>
> > When did writing EFI variables to nvram become necessary to boot on
> > UEFI? And if it is necessary, why is it that only linux boot loaders
> > that use EFI stubs (generally grub2) need it? The current kernel
> > boots using EFI/grub and EFI/elilo. It is just when EFI stubs are
> > used that the boot fails.
>
> I think you've misunderstood the problem. If nvram becaomes full, some
> systems crash during firmware initialisation. So we can't let nvram
> become full. The obvious thing to do here is to look at the values from
> QueryVariableInfo, but many systems won't perform any garbage collection
> until they're almost out of space and so variables that have been
> deleted still show up as used space. We can work around that by adding
> up the size of the variables ourselves, but that only gives us the value
> for runtime-visible variables. We also need to know how much space is
> used by variables that are only visible during boot, hence calling
> QueryVariableInfo before ExitBootServices.
>
> > Which means the previous patch(es) that caused the bricking should
> > get pulled, too.
>
> There are no patches that cause the bricking.
This is the description of the original problem:
http://www.h-online.com/open/news/item/Booting-Linux-using-UEFI-can-brick-Samsung-laptops-1793958.html
And the further investigation:
http://mjg59.dreamwidth.org/22855.html
If you read the latter, it shows you why we have to use
QueryVariableInfo to try to work out how much space we have available.
James
On Fri, May 31, 2013 at 05:28:16PM +0100, Matthew Garrett wrote:
> On Fri, May 31, 2013 at 10:43:49AM -0500, Russ Anderson wrote:
>
> > When did writing EFI variables to nvram become necessary to boot on
> > UEFI? And if it is necessary, why is it that only linux boot loaders
> > that use EFI stubs (generally grub2) need it? The current kernel
> > boots using EFI/grub and EFI/elilo. It is just when EFI stubs are
> > used that the boot fails.
>
> I think you've misunderstood the problem.
The problem for me is the system not booting. I was not
familiar with the problem you are trying to solve, but
have now looked at the links.
> If nvram becaomes full, some
> systems crash during firmware initialisation. So we can't let nvram
> become full. The obvious thing to do here is to look at the values from
> QueryVariableInfo, but many systems won't perform any garbage collection
> until they're almost out of space and so variables that have been
> deleted still show up as used space.
OK. I get nvram looks full due to lack of garbage collection
on some systems. Does QueryVariableInfo (at runtime) tell you
it is full? Is the problem that it says it is full when it
is not, or does not tell you it is full when it is?
> We can work around that by adding
> up the size of the variables ourselves, but that only gives us the value
> for runtime-visible variables. We also need to know how much space is
> used by variables that are only visible during boot,
Is it valid to assume that only the kernel writes to nvram at
runtime? Can bios log error information to nvram on an SMM
interrupt (for example)?
> hence calling
> QueryVariableInfo before ExitBootServices.
Correct me if I am wrong, but that is called from EFI stubs,
which is only used by some bootloaders (sometimes grub2).
Does that mean EFI/grub and EFI/elilo will not hit the problem,
or will not have your fix?
> > Which means the previous patch(es) that caused the bricking should
> > get pulled, too.
>
> There are no patches that cause the bricking.
I thought that was the problem you were trying to avoid.
--
Russ Anderson, OS RAS/Partitioning Project Lead
SGI - Silicon Graphics Inc [email protected]
On 05/31/2013 03:57 PM, Russ Anderson wrote:
>
>>> Which means the previous patch(es) that caused the bricking should
>>> get pulled, too.
>>
>> There are no patches that cause the bricking.
>
> I thought that was the problem you were trying to avoid.
>
There are no *patches* that cause the bricking. We have been trying to
figure out how to *avoid* the bricking.
-hpa
On Fri, 31 May 2013, Russ Anderson wrote:
> OK. I get nvram looks full due to lack of garbage collection
> on some systems. Does QueryVariableInfo (at runtime) tell you
> it is full? Is the problem that it says it is full when it
> is not, or does not tell you it is full when it is?
We are trying to count the size occupied by existing UEFI variables.
QueryVariableInfo() called from runtime environment doesn't give a full
picture, as it doesn't provide information about boot-only variables.
Hence the patch.
> > up the size of the variables ourselves, but that only gives us the value
> > for runtime-visible variables. We also need to know how much space is
> > used by variables that are only visible during boot,
>
> Is it valid to assume that only the kernel writes to nvram at
> runtime?
Well, mostly yes. But if the kernel doesn't know what the initial state
is, it can't count the size properly.
> > > Which means the previous patch(es) that caused the bricking should
> > > get pulled, too.
> >
> > There are no patches that cause the bricking.
>
> I thought that was the problem you were trying to avoid.
The problem that needs to be avoided is machines turning into bricks if
EFI storage nvram is getting full.
--
Jiri Kosina
SUSE Labs
On Fri, May 31, 2013 at 05:57:31PM -0500, Russ Anderson wrote:
> On Fri, May 31, 2013 at 05:28:16PM +0100, Matthew Garrett wrote:
> > If nvram becaomes full, some
> > systems crash during firmware initialisation. So we can't let nvram
> > become full. The obvious thing to do here is to look at the values from
> > QueryVariableInfo, but many systems won't perform any garbage collection
> > until they're almost out of space and so variables that have been
> > deleted still show up as used space.
>
> OK. I get nvram looks full due to lack of garbage collection
> on some systems. Does QueryVariableInfo (at runtime) tell you
> it is full? Is the problem that it says it is full when it
> is not, or does not tell you it is full when it is?
QueryVariableInfo reports the amount used *including garbage*. This
makes it useless for determining how much space is available for the
firmware to use.
> > We can work around that by adding
> > up the size of the variables ourselves, but that only gives us the value
> > for runtime-visible variables. We also need to know how much space is
> > used by variables that are only visible during boot,
>
> Is it valid to assume that only the kernel writes to nvram at
> runtime? Can bios log error information to nvram on an SMM
> interrupt (for example)?
There's a limited number of situations where the firmware can write to
nvram, but they're basically uninteresting. Practically speaking, it's
always via the kernel once ExitBootServices() has been called.
> > hence calling
> > QueryVariableInfo before ExitBootServices.
>
> Correct me if I am wrong, but that is called from EFI stubs,
> which is only used by some bootloaders (sometimes grub2).
> Does that mean EFI/grub and EFI/elilo will not hit the problem,
> or will not have your fix?
Will not have the fix. Boot EFI systems via the EFI stub.
--
Matthew Garrett | [email protected]
On Sat, Jun 01, 2013 at 01:03:11AM +0100, Matthew Garrett wrote:
> On Fri, May 31, 2013 at 05:57:31PM -0500, Russ Anderson wrote:
> > On Fri, May 31, 2013 at 05:28:16PM +0100, Matthew Garrett wrote:
> > > If nvram becaomes full, some
> > > systems crash during firmware initialisation. So we can't let nvram
> > > become full. The obvious thing to do here is to look at the values from
> > > QueryVariableInfo, but many systems won't perform any garbage collection
> > > until they're almost out of space and so variables that have been
> > > deleted still show up as used space.
> >
> > OK. I get nvram looks full due to lack of garbage collection
> > on some systems. Does QueryVariableInfo (at runtime) tell you
> > it is full? Is the problem that it says it is full when it
> > is not, or does not tell you it is full when it is?
>
> QueryVariableInfo reports the amount used *including garbage*. This
> makes it useless for determining how much space is available for the
> firmware to use.
So when QueryVariableInfo reports there is space available, the
write works and everything is fine.
And when QueryVariableInfo reports insufficient space, don't write
and everything is fine.
But when QueryVariableInfo reports insufficient space and you
write anyway, the system can get bricked. Is that the problem?
According to the UEFI spec for QueryVariableInfo()
RemainingVariableStorageSize
Returns the remaining size of the storage
space available for EFI variables associated
with the attributes specified.
It also says:
The returned MaximumVariableStorageSize, RemainingVariableStorageSize,
MaximumVariableSize information may change immediately after
the call based on other runtime activities including asynchronous
error events. Also, these values associated with different
attributes are not additive in nature.
Note that "other runtime activities including asynchronous error
events." That means runtime services may be using nvram without
the OS knowing about it.
Some bios implementation may be "*including garbage*", but can
you definitively say ALL do? The spec explicitly says "the
implementation of variable storage is not defined in this
specification". You can't assume any form of garbage collection.
7.2 Variable Services
Although the implementation of variable storage is not defined
in this specification, variables must be persistent in most cases.
This implies that the EFI implementation on a platform must arrange
it so that variables passed in for storage are retained and
available for use each time the system boots, at least until they
are explicitly deleted or overwritten. Provision of this type of
nonvolatile storage may be very limited on some platforms, so
variables should be used sparingly in cases where other means of
communicating information cannot be used.
I don't see anything in here about the OS being free to use
more nvram than QueryVariableInfo() reported as being available.
What happened to following the spec?
> > > We can work around that by adding
> > > up the size of the variables ourselves, but that only gives us the value
> > > for runtime-visible variables. We also need to know how much space is
> > > used by variables that are only visible during boot,
> >
> > Is it valid to assume that only the kernel writes to nvram at
> > runtime? Can bios log error information to nvram on an SMM
> > interrupt (for example)?
>
> There's a limited number of situations where the firmware can write to
> nvram, but they're basically uninteresting. Practically speaking, it's
> always via the kernel once ExitBootServices() has been called.
How can you say "they're basically uninteresting"???
Do you know what EVERY bios implementation does???
> > > hence calling
> > > QueryVariableInfo before ExitBootServices.
> >
> > Correct me if I am wrong, but that is called from EFI stubs,
> > which is only used by some bootloaders (sometimes grub2).
> > Does that mean EFI/grub and EFI/elilo will not hit the problem,
> > or will not have your fix?
>
> Will not have the fix. Boot EFI systems via the EFI stub.
Thanks for that clarification.
> Boot EFI systems via the EFI stub.
Fortunately all our shipping systems are EFI/grub and EFI/elilo
right now, so they boot.
--
Russ Anderson, OS RAS/Partitioning Project Lead
SGI - Silicon Graphics Inc [email protected]
On Fri, May 31, 2013 at 11:20:59PM -0500, Russ Anderson wrote:
> And when QueryVariableInfo reports insufficient space, don't write
> and everything is fine.
And then installs fail, which is why we implemented this additional
code.
> But when QueryVariableInfo reports insufficient space and you
> write anyway, the system can get bricked. Is that the problem?
No.
> Some bios implementation may be "*including garbage*", but can
> you definitively say ALL do? The spec explicitly says "the
> implementation of variable storage is not defined in this
> specification". You can't assume any form of garbage collection.
No. We can't assume that. But nor can we rely upon the spec, because
behaving according to the spec results in dead computers. So we're
forced to write code that behaves according to reality rather than
paper, which is unfortunate, but if *your* firmware behaved according to
the specification then you wouldn't be seeing any problems with the
current code, so I think there's a lesson for us all here.
> I don't see anything in here about the OS being free to use
> more nvram than QueryVariableInfo() reported as being available.
> What happened to following the spec?
That would be the same spec that tells you not to use physical addresses
after SetVirtualAddressMap() has been called, right? Regardless, we
don't use more space than QueryVariableInfo() reports as being
available. We would expect that any attempt to do so would fail. But nor
can we assume that we're free to use as much space as
QueryVariableInfo() *does* report, because doing so results in dead
computers. So instead we're forced to rely on workarounds that happen to
break your broken firmware, so now we're going to have to find a
different set of workarounds and hope that they don't break someone
else's computer instead.
> > There's a limited number of situations where the firmware can write to
> > nvram, but they're basically uninteresting. Practically speaking, it's
> > always via the kernel once ExitBootServices() has been called.
>
> How can you say "they're basically uninteresting"???
> Do you know what EVERY bios implementation does???
We have to assume that firmware doesn't behave in a pathological manner,
because the alternative is to give up entirely and declare that Linux
doesn't run on UEFI systems. I suspect that that would be problematic
for your employers.
--
Matthew Garrett | [email protected]
On Fri, 31 May 2013, Matthew Garrett wrote:
>
> I agree that a revert is probably the right thing to do here, but the
> original patch was there to permit a more accurate calculation of the
> amount of nvram in use, not to provide additional debug information.
> Reverting it is going to differently break a different set of systems
So "differently break" doesn't matter, if it's old breakage, and people
thus don't really expect it to work. We need to fix bugs without *new*
breakage, and quite frankly, I have been distressed by hearing the EFI
"specifications" mentioned so many times in this thread.
Firmware specs are pure and utter garbage. They are irrelevant. Firmware
is buggy, and will always be buggy. The "spec" doesn't matter. We should
use firmware for loading the kernel, and as little else as humanly
possible.
I'm very disappointed in how the EFI code doesn't seem to understand that.
There's tons of these stupid EFI variable crap that simply shouldn't
matter. Quite frankly, we'd be better off ignoring as much of it by
default as at all possible. Exactly because the more of an EFI interface
we have, the more we open us up to th einevitable firmware bugs.
Anyway, I'm traveling with absolutely horrendous internet access, so can
somebody please send a description of the revert with the relevant
information, because I literally have a hard time extracting it all from
this thread because my email access is so slow and flaky... Make it easy
for me to do the revert with a good explanation message, please,
Linus
On Sat, Jun 01, 2013 at 08:01:59PM +0900, Linus Torvalds wrote:
> On Fri, 31 May 2013, Matthew Garrett wrote:
> >
> > I agree that a revert is probably the right thing to do here, but the
> > original patch was there to permit a more accurate calculation of the
> > amount of nvram in use, not to provide additional debug information.
> > Reverting it is going to differently break a different set of systems
>
> So "differently break" doesn't matter, if it's old breakage, and people
> thus don't really expect it to work. We need to fix bugs without *new*
> breakage, and quite frankly, I have been distressed by hearing the EFI
> "specifications" mentioned so many times in this thread.
The old breakage has only ever been present in -rcs, not in released
kernels. We worked around the worst of it with the patchset that's
causing the problems. We can avoid that old breakage by reverting the
patches that introduced it, but that gets us back to bricking some
machines.
> Anyway, I'm traveling with absolutely horrendous internet access, so can
> somebody please send a description of the revert with the relevant
> information, because I literally have a hard time extracting it all from
> this thread because my email access is so slow and flaky... Make it easy
> for me to do the revert with a good explanation message, please,
I've finally got some information from Samsung that *should* let us fix
this without anything else breaking in the process, but since we're
dealing with firmware there's obviously a risk that that won't actually
be true. I'll send that later today.
--
Matthew Garrett | [email protected]