2018-04-09 09:10:48

by Jean-Baptiste Théou

[permalink] [raw]
Subject: Linux 4.9.93

Hi,

After this patchset, a kernel built with CFI fails. Disabling UNMAP_KERNEL_AT_EL0 fix the issue obviously.

Wondering if there is one of the test suite used on the review patchset that covers the CFI usecase.

Best regards,

[ 0.249191] CPU features: detected feature: GIC system register CPU interface
[ 0.256391] CPU features: detected feature: Privileged Access Never
[ 0.262719] CPU features: detected feature: User Access Override
[ 0.268791] CPU features: detected feature: 32-bit EL0 Support
[ 0.274683] CPU features: detected feature: Kernel page table isolation (KPTI)
[ 0.282166] CFI failure:
[ 0.282169] CFI failure:
[ 0.282172] CFI failure:
[ 0.282173] CFI failure:
[ 0.282175] CFI failure:
[ 0.282176] CFI failure:
[ 0.282177] CFI failure:
[ 0.282178] CFI failure:
[ 0.282188] ------------[ cut here ]------------
[ 0.282189] ------------[ cut here ]------------
[ 0.282190] ------------[ cut here ]------------
[ 0.282191] ------------[ cut here ]------------
[ 0.282193] ------------[ cut here ]------------
[ 0.282196] kernel BUG at kernel/cfi.c:32!
[ 0.282198] ------------[ cut here ]------------
[ 0.282201] kernel BUG at kernel/cfi.c:32!
[ 0.282202] ------------[ cut here ]------------
[ 0.282204] kernel BUG at kernel/cfi.c:32!
[ 0.282207] kernel BUG at kernel/cfi.c:32!
[ 0.282209] kernel BUG at kernel/cfi.c:32!
[ 0.282211] kernel BUG at kernel/cfi.c:32!
[ 0.282214] kernel BUG at kernel/cfi.c:32!
[ 0.282215] ------------[ cut here ]------------
[ 0.282216] kernel BUG at kernel/cfi.c:32!
[ 0.282218] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
[ 0.282224] Modules linked in:
[ 0.282230] CPU: 2 PID: 25 Comm: migration/2 Not tainted 4.9.93-perf+ #39
[ 0.282232] Hardware name: <REMOVED>
[ 0.282235] task: fffffffbb3b36580 task.stack: fffffffbb30cc000
[ 0.282250] PC is at __cfi_check_fail+0x14/0x1c
[ 0.282253] LR is at __cfi_check_fail+0x14/0x1c
[ 0.282255] pc : [<ffffff93b3f03d90>] lr : [<ffffff93b3f03d90>] pstate: 60c00085
[ 0.282256] sp : fffffffbb30cfc30
[ 0.282259] x29: fffffffbb30cfc30 x28: ffffff93b6415000
[ 0.282261] x27: 00000013b65c1000 x26: ffffff93b5ce6000
[ 0.282264] x25: ffffff93b5ce6000 x24: ffffff93b6419000
[ 0.282266] x23: ffffff93b65c1000 x22: ffffff93b65c4000
[ 0.282268] x21: 9d12f8172cb2f296 x20: 000000008180e3e0
[ 0.282271] x19: 0000000000000000 x18: 000000000000002c
[ 0.282274] x17: 00000000000fd054 x16: 0000000000000000
[ 0.282276] x15: ffffff93b65ec000 x14: 000000000000000c
[ 0.282279] x13: 0000000000000004 x12: 0000000000000000
[ 0.282281] x11: 0000000000000000 x10: 0000000001440144
[ 0.282283] x9 : 260822e8751d5000 x8 : 260822e8751d5000
[ 0.282286] x7 : 0000000000000000 x6 : fffffffbbac75b60
[ 0.282288] x5 : 0000000000000000 x4 : 0000000000000000
[ 0.282290] x3 : 000000003a657275 x2 : 0000000000000000
[ 0.282292] x1 : 0000000000000000 x0 : 000000000000000c
[ 0.282294]
[ 0.282294] PC: 0xffffff93b3f03d50:
[ 0.282308] 3d50 b9001ac8 f94002c8 370ffec8 17ffffbe d4210000 14000000 aa1603e0 f90007e8
[ 0.282315] 3d70 94536017 f94007e8 17ffffe2 a9bf7bfd 910003fd d000d100 913ee400 94533cc7
[ 0.282322] 3d90 d4210000 14000000 b0013788 2a1f03e0 f901c51f d65f03c0 f940406b 2a0203e8
[ 0.282329] 3db0 2a0103e9 aa0003ea b400008b f9000145 f94000cb b40001ab a9bf7bfd 910003fd
[ 0.282330]
[ 0.282330] LR: 0xffffff93b3f03d50:
[ 0.282336] 3d50 b9001ac8 f94002c8 370ffec8 17ffffbe d4210000 14000000 aa1603e0 f90007e8
[ 0.282343] 3d70 94536017 f94007e8 17ffffe2 a9bf7bfd 910003fd d000d100 913ee400 94533cc7
[ 0.282350] 3d90 d4210000 14000000 b0013788 2a1f03e0 f901c51f d65f03c0 f940406b 2a0203e8
[ 0.282357] 3db0 2a0103e9 aa0003ea b400008b f9000145 f94000cb b40001ab a9bf7bfd 910003fd
[ 0.282358]
[ 0.282358] SP: 0xfffffffbb30cfbf0:
[ 0.282365] fbf0 b3f03d90 ffffff93 b30cfc30 fffffffb b3f03d90 ffffff93 60c00085 00000000
[ 0.282372] fc10 b6415000 ffffff93 b642fa00 ffffff93 ffffffff ffffffff b3f03d90 ffffff93
[ 0.282378] fc30 b30cfc70 fffffffb b3d458c0 ffffff93 00000080 00000000 00000001 00000000
[ 0.282385] fc50 b65c4000 ffffff93 b64420f0 ffffff93 8180e3e0 00000000 00000002 00000000
[ 0.282387] Process migration/2 (pid: 25, stack limit = 0xfffffffbb30cc000)
[ 0.282389] Call trace:
[ 0.282391] Exception stack(0xfffffffbb30cfb00 to 0xfffffffbb30cfc30)
[ 0.282395] fb00: 000000000000000c 0000000000000000 0000000000000000 000000003a657275
[ 0.282397] fb20: 0000000000000000 0000000000000000 fffffffbbac75b60 0000000000000000
[ 0.282400] fb40: 260822e8751d5000 260822e8751d5000 0000000001440144 0000000000000000
[ 0.282403] fb60: 0000000000000000 0000000000000004 000000000000000c ffffff93b65ec000
[ 0.282405] fb80: 0000000000000000 00000000000fd054 000000000000002c 0000000000000000
[ 0.282408] fba0: 000000008180e3e0 9d12f8172cb2f296 ffffff93b65c4000 ffffff93b65c1000
[ 0.282411] fbc0: ffffff93b6419000 ffffff93b5ce6000 ffffff93b5ce6000 00000013b65c1000
[ 0.282413] fbe0: ffffff93b6415000 fffffffbb30cfc30 ffffff93b3f03d90 fffffffbb30cfc30
[ 0.282416] fc00: ffffff93b3f03d90 0000000060c00085 ffffff93b6415000 ffffff93b642fa00
[ 0.282418] fc20: ffffffffffffffff ffffff93b3f03d90
[ 0.282421] [<ffffff93b3f03d90>] __cfi_check_fail+0x14/0x1c
[ 0.282430] [<ffffff93b3d458c0>] name_to_dev_t+0x0/0x47c
[ 0.282436] [<ffffff93b3d51b80>] kpti_install_ng_mappings+0x178/0x2e0
[ 0.282443] [<ffffff93b3eae950>] multi_cpu_stop+0x114/0x170
[ 0.282445] [<ffffff93b3eaf08c>] cpu_stopper_thread+0x128/0x2e8
[ 0.282452] [<ffffff93b3db5504>] smpboot_thread_fn+0x230/0x558
[ 0.282455] [<ffffff93b3dae848>] kthread+0x21c/0x238
[ 0.282459] [<ffffff93b3c838f0>] ret_from_fork+0x10/0x20
[ 0.282464] Code: 910003fd d000d100 913ee400 94533cc7 (d4210000)




2018-04-09 09:34:25

by Greg Kroah-Hartman

[permalink] [raw]
Subject: Re: Linux 4.9.93

On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:
> Hi,
>
> After this patchset, a kernel built with CFI fails. Disabling
> UNMAP_KERNEL_AT_EL0 fix the issue obviously.

Is this a "clean" 4.9.93 tree or a "4.9.93 merged into
android-common-4.9?

> Wondering if there is one of the test suite used on the review patchset that covers the CFI usecase.
>
> Best regards,
>
> [ 0.249191] CPU features: detected feature: GIC system register CPU interface
> [ 0.256391] CPU features: detected feature: Privileged Access Never
> [ 0.262719] CPU features: detected feature: User Access Override
> [ 0.268791] CPU features: detected feature: 32-bit EL0 Support
> [ 0.274683] CPU features: detected feature: Kernel page table isolation (KPTI)
> [ 0.282166] CFI failure:
> [ 0.282169] CFI failure:
> [ 0.282172] CFI failure:
> [ 0.282173] CFI failure:
> [ 0.282175] CFI failure:
> [ 0.282176] CFI failure:
> [ 0.282177] CFI failure:
> [ 0.282178] CFI failure:
> [ 0.282188] ------------[ cut here ]------------
> [ 0.282189] ------------[ cut here ]------------
> [ 0.282190] ------------[ cut here ]------------
> [ 0.282191] ------------[ cut here ]------------
> [ 0.282193] ------------[ cut here ]------------
> [ 0.282196] kernel BUG at kernel/cfi.c:32!
> [ 0.282198] ------------[ cut here ]------------
> [ 0.282201] kernel BUG at kernel/cfi.c:32!
> [ 0.282202] ------------[ cut here ]------------
> [ 0.282204] kernel BUG at kernel/cfi.c:32!
> [ 0.282207] kernel BUG at kernel/cfi.c:32!
> [ 0.282209] kernel BUG at kernel/cfi.c:32!
> [ 0.282211] kernel BUG at kernel/cfi.c:32!
> [ 0.282214] kernel BUG at kernel/cfi.c:32!
> [ 0.282215] ------------[ cut here ]------------
> [ 0.282216] kernel BUG at kernel/cfi.c:32!
> [ 0.282218] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
> [ 0.282224] Modules linked in:
> [ 0.282230] CPU: 2 PID: 25 Comm: migration/2 Not tainted 4.9.93-perf+ #39
> [ 0.282232] Hardware name: <REMOVED>
> [ 0.282235] task: fffffffbb3b36580 task.stack: fffffffbb30cc000
> [ 0.282250] PC is at __cfi_check_fail+0x14/0x1c
> [ 0.282253] LR is at __cfi_check_fail+0x14/0x1c
> [ 0.282255] pc : [<ffffff93b3f03d90>] lr : [<ffffff93b3f03d90>] pstate: 60c00085
> [ 0.282256] sp : fffffffbb30cfc30
> [ 0.282259] x29: fffffffbb30cfc30 x28: ffffff93b6415000
> [ 0.282261] x27: 00000013b65c1000 x26: ffffff93b5ce6000
> [ 0.282264] x25: ffffff93b5ce6000 x24: ffffff93b6419000
> [ 0.282266] x23: ffffff93b65c1000 x22: ffffff93b65c4000
> [ 0.282268] x21: 9d12f8172cb2f296 x20: 000000008180e3e0
> [ 0.282271] x19: 0000000000000000 x18: 000000000000002c
> [ 0.282274] x17: 00000000000fd054 x16: 0000000000000000
> [ 0.282276] x15: ffffff93b65ec000 x14: 000000000000000c
> [ 0.282279] x13: 0000000000000004 x12: 0000000000000000
> [ 0.282281] x11: 0000000000000000 x10: 0000000001440144
> [ 0.282283] x9 : 260822e8751d5000 x8 : 260822e8751d5000
> [ 0.282286] x7 : 0000000000000000 x6 : fffffffbbac75b60
> [ 0.282288] x5 : 0000000000000000 x4 : 0000000000000000
> [ 0.282290] x3 : 000000003a657275 x2 : 0000000000000000
> [ 0.282292] x1 : 0000000000000000 x0 : 000000000000000c
> [ 0.282294]
> [ 0.282294] PC: 0xffffff93b3f03d50:
> [ 0.282308] 3d50 b9001ac8 f94002c8 370ffec8 17ffffbe d4210000 14000000 aa1603e0 f90007e8
> [ 0.282315] 3d70 94536017 f94007e8 17ffffe2 a9bf7bfd 910003fd d000d100 913ee400 94533cc7
> [ 0.282322] 3d90 d4210000 14000000 b0013788 2a1f03e0 f901c51f d65f03c0 f940406b 2a0203e8
> [ 0.282329] 3db0 2a0103e9 aa0003ea b400008b f9000145 f94000cb b40001ab a9bf7bfd 910003fd
> [ 0.282330]
> [ 0.282330] LR: 0xffffff93b3f03d50:
> [ 0.282336] 3d50 b9001ac8 f94002c8 370ffec8 17ffffbe d4210000 14000000 aa1603e0 f90007e8
> [ 0.282343] 3d70 94536017 f94007e8 17ffffe2 a9bf7bfd 910003fd d000d100 913ee400 94533cc7
> [ 0.282350] 3d90 d4210000 14000000 b0013788 2a1f03e0 f901c51f d65f03c0 f940406b 2a0203e8
> [ 0.282357] 3db0 2a0103e9 aa0003ea b400008b f9000145 f94000cb b40001ab a9bf7bfd 910003fd
> [ 0.282358]
> [ 0.282358] SP: 0xfffffffbb30cfbf0:
> [ 0.282365] fbf0 b3f03d90 ffffff93 b30cfc30 fffffffb b3f03d90 ffffff93 60c00085 00000000
> [ 0.282372] fc10 b6415000 ffffff93 b642fa00 ffffff93 ffffffff ffffffff b3f03d90 ffffff93
> [ 0.282378] fc30 b30cfc70 fffffffb b3d458c0 ffffff93 00000080 00000000 00000001 00000000
> [ 0.282385] fc50 b65c4000 ffffff93 b64420f0 ffffff93 8180e3e0 00000000 00000002 00000000
> [ 0.282387] Process migration/2 (pid: 25, stack limit = 0xfffffffbb30cc000)
> [ 0.282389] Call trace:
> [ 0.282391] Exception stack(0xfffffffbb30cfb00 to 0xfffffffbb30cfc30)
> [ 0.282395] fb00: 000000000000000c 0000000000000000 0000000000000000 000000003a657275
> [ 0.282397] fb20: 0000000000000000 0000000000000000 fffffffbbac75b60 0000000000000000
> [ 0.282400] fb40: 260822e8751d5000 260822e8751d5000 0000000001440144 0000000000000000
> [ 0.282403] fb60: 0000000000000000 0000000000000004 000000000000000c ffffff93b65ec000
> [ 0.282405] fb80: 0000000000000000 00000000000fd054 000000000000002c 0000000000000000
> [ 0.282408] fba0: 000000008180e3e0 9d12f8172cb2f296 ffffff93b65c4000 ffffff93b65c1000
> [ 0.282411] fbc0: ffffff93b6419000 ffffff93b5ce6000 ffffff93b5ce6000 00000013b65c1000
> [ 0.282413] fbe0: ffffff93b6415000 fffffffbb30cfc30 ffffff93b3f03d90 fffffffbb30cfc30
> [ 0.282416] fc00: ffffff93b3f03d90 0000000060c00085 ffffff93b6415000 ffffff93b642fa00
> [ 0.282418] fc20: ffffffffffffffff ffffff93b3f03d90
> [ 0.282421] [<ffffff93b3f03d90>] __cfi_check_fail+0x14/0x1c
> [ 0.282430] [<ffffff93b3d458c0>] name_to_dev_t+0x0/0x47c
> [ 0.282436] [<ffffff93b3d51b80>] kpti_install_ng_mappings+0x178/0x2e0
> [ 0.282443] [<ffffff93b3eae950>] multi_cpu_stop+0x114/0x170
> [ 0.282445] [<ffffff93b3eaf08c>] cpu_stopper_thread+0x128/0x2e8
> [ 0.282452] [<ffffff93b3db5504>] smpboot_thread_fn+0x230/0x558
> [ 0.282455] [<ffffff93b3dae848>] kthread+0x21c/0x238
> [ 0.282459] [<ffffff93b3c838f0>] ret_from_fork+0x10/0x20
> [ 0.282464] Code: 910003fd d000d100 913ee400 94533cc7 (d4210000)

Oh fun :(

Dragging in a bunch more people to the cc: and to: lines to have them
look at this...

thanks,

greg k-h

2018-04-09 09:49:48

by Marc Zyngier

[permalink] [raw]
Subject: Re: Linux 4.9.93

On 09/04/18 10:30, Greg KH wrote:
> On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:
>> Hi,
>>
>> After this patchset, a kernel built with CFI fails. Disabling
>> UNMAP_KERNEL_AT_EL0 fix the issue obviously.
>
> Is this a "clean" 4.9.93 tree or a "4.9.93 merged into
> android-common-4.9?
>
>> Wondering if there is one of the test suite used on the review patchset that covers the CFI usecase.
>>
>> Best regards,
>>
>> [ 0.249191] CPU features: detected feature: GIC system register CPU interface
>> [ 0.256391] CPU features: detected feature: Privileged Access Never
>> [ 0.262719] CPU features: detected feature: User Access Override
>> [ 0.268791] CPU features: detected feature: 32-bit EL0 Support
>> [ 0.274683] CPU features: detected feature: Kernel page table isolation (KPTI)
>> [ 0.282166] CFI failure:
>> [ 0.282169] CFI failure:
>> [ 0.282172] CFI failure:
>> [ 0.282173] CFI failure:
>> [ 0.282175] CFI failure:
>> [ 0.282176] CFI failure:
>> [ 0.282177] CFI failure:
>> [ 0.282178] CFI failure:
>> [ 0.282188] ------------[ cut here ]------------
>> [ 0.282189] ------------[ cut here ]------------
>> [ 0.282190] ------------[ cut here ]------------
>> [ 0.282191] ------------[ cut here ]------------
>> [ 0.282193] ------------[ cut here ]------------
>> [ 0.282196] kernel BUG at kernel/cfi.c:32!
>> [ 0.282198] ------------[ cut here ]------------
>> [ 0.282201] kernel BUG at kernel/cfi.c:32!
>> [ 0.282202] ------------[ cut here ]------------
>> [ 0.282204] kernel BUG at kernel/cfi.c:32!
>> [ 0.282207] kernel BUG at kernel/cfi.c:32!
>> [ 0.282209] kernel BUG at kernel/cfi.c:32!
>> [ 0.282211] kernel BUG at kernel/cfi.c:32!
>> [ 0.282214] kernel BUG at kernel/cfi.c:32!
>> [ 0.282215] ------------[ cut here ]------------
>> [ 0.282216] kernel BUG at kernel/cfi.c:32!
>> [ 0.282218] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
>> [ 0.282224] Modules linked in:
>> [ 0.282230] CPU: 2 PID: 25 Comm: migration/2 Not tainted 4.9.93-perf+ #39
>> [ 0.282232] Hardware name: <REMOVED>
>> [ 0.282235] task: fffffffbb3b36580 task.stack: fffffffbb30cc000
>> [ 0.282250] PC is at __cfi_check_fail+0x14/0x1c
>> [ 0.282253] LR is at __cfi_check_fail+0x14/0x1c
>> [ 0.282255] pc : [<ffffff93b3f03d90>] lr : [<ffffff93b3f03d90>] pstate: 60c00085
>> [ 0.282256] sp : fffffffbb30cfc30
>> [ 0.282259] x29: fffffffbb30cfc30 x28: ffffff93b6415000
>> [ 0.282261] x27: 00000013b65c1000 x26: ffffff93b5ce6000
>> [ 0.282264] x25: ffffff93b5ce6000 x24: ffffff93b6419000
>> [ 0.282266] x23: ffffff93b65c1000 x22: ffffff93b65c4000
>> [ 0.282268] x21: 9d12f8172cb2f296 x20: 000000008180e3e0
>> [ 0.282271] x19: 0000000000000000 x18: 000000000000002c
>> [ 0.282274] x17: 00000000000fd054 x16: 0000000000000000
>> [ 0.282276] x15: ffffff93b65ec000 x14: 000000000000000c
>> [ 0.282279] x13: 0000000000000004 x12: 0000000000000000
>> [ 0.282281] x11: 0000000000000000 x10: 0000000001440144
>> [ 0.282283] x9 : 260822e8751d5000 x8 : 260822e8751d5000
>> [ 0.282286] x7 : 0000000000000000 x6 : fffffffbbac75b60
>> [ 0.282288] x5 : 0000000000000000 x4 : 0000000000000000
>> [ 0.282290] x3 : 000000003a657275 x2 : 0000000000000000
>> [ 0.282292] x1 : 0000000000000000 x0 : 000000000000000c
>> [ 0.282294]
>> [ 0.282294] PC: 0xffffff93b3f03d50:
>> [ 0.282308] 3d50 b9001ac8 f94002c8 370ffec8 17ffffbe d4210000 14000000 aa1603e0 f90007e8
>> [ 0.282315] 3d70 94536017 f94007e8 17ffffe2 a9bf7bfd 910003fd d000d100 913ee400 94533cc7
>> [ 0.282322] 3d90 d4210000 14000000 b0013788 2a1f03e0 f901c51f d65f03c0 f940406b 2a0203e8
>> [ 0.282329] 3db0 2a0103e9 aa0003ea b400008b f9000145 f94000cb b40001ab a9bf7bfd 910003fd
>> [ 0.282330]
>> [ 0.282330] LR: 0xffffff93b3f03d50:
>> [ 0.282336] 3d50 b9001ac8 f94002c8 370ffec8 17ffffbe d4210000 14000000 aa1603e0 f90007e8
>> [ 0.282343] 3d70 94536017 f94007e8 17ffffe2 a9bf7bfd 910003fd d000d100 913ee400 94533cc7
>> [ 0.282350] 3d90 d4210000 14000000 b0013788 2a1f03e0 f901c51f d65f03c0 f940406b 2a0203e8
>> [ 0.282357] 3db0 2a0103e9 aa0003ea b400008b f9000145 f94000cb b40001ab a9bf7bfd 910003fd
>> [ 0.282358]
>> [ 0.282358] SP: 0xfffffffbb30cfbf0:
>> [ 0.282365] fbf0 b3f03d90 ffffff93 b30cfc30 fffffffb b3f03d90 ffffff93 60c00085 00000000
>> [ 0.282372] fc10 b6415000 ffffff93 b642fa00 ffffff93 ffffffff ffffffff b3f03d90 ffffff93
>> [ 0.282378] fc30 b30cfc70 fffffffb b3d458c0 ffffff93 00000080 00000000 00000001 00000000
>> [ 0.282385] fc50 b65c4000 ffffff93 b64420f0 ffffff93 8180e3e0 00000000 00000002 00000000
>> [ 0.282387] Process migration/2 (pid: 25, stack limit = 0xfffffffbb30cc000)
>> [ 0.282389] Call trace:
>> [ 0.282391] Exception stack(0xfffffffbb30cfb00 to 0xfffffffbb30cfc30)
>> [ 0.282395] fb00: 000000000000000c 0000000000000000 0000000000000000 000000003a657275
>> [ 0.282397] fb20: 0000000000000000 0000000000000000 fffffffbbac75b60 0000000000000000
>> [ 0.282400] fb40: 260822e8751d5000 260822e8751d5000 0000000001440144 0000000000000000
>> [ 0.282403] fb60: 0000000000000000 0000000000000004 000000000000000c ffffff93b65ec000
>> [ 0.282405] fb80: 0000000000000000 00000000000fd054 000000000000002c 0000000000000000
>> [ 0.282408] fba0: 000000008180e3e0 9d12f8172cb2f296 ffffff93b65c4000 ffffff93b65c1000
>> [ 0.282411] fbc0: ffffff93b6419000 ffffff93b5ce6000 ffffff93b5ce6000 00000013b65c1000
>> [ 0.282413] fbe0: ffffff93b6415000 fffffffbb30cfc30 ffffff93b3f03d90 fffffffbb30cfc30
>> [ 0.282416] fc00: ffffff93b3f03d90 0000000060c00085 ffffff93b6415000 ffffff93b642fa00
>> [ 0.282418] fc20: ffffffffffffffff ffffff93b3f03d90
>> [ 0.282421] [<ffffff93b3f03d90>] __cfi_check_fail+0x14/0x1c

Where is this coming from? Out of tree patches?

>> [ 0.282430] [<ffffff93b3d458c0>] name_to_dev_t+0x0/0x47c
>> [ 0.282436] [<ffffff93b3d51b80>] kpti_install_ng_mappings+0x178/0x2e0
>> [ 0.282443] [<ffffff93b3eae950>] multi_cpu_stop+0x114/0x170
>> [ 0.282445] [<ffffff93b3eaf08c>] cpu_stopper_thread+0x128/0x2e8
>> [ 0.282452] [<ffffff93b3db5504>] smpboot_thread_fn+0x230/0x558
>> [ 0.282455] [<ffffff93b3dae848>] kthread+0x21c/0x238
>> [ 0.282459] [<ffffff93b3c838f0>] ret_from_fork+0x10/0x20
>> [ 0.282464] Code: 910003fd d000d100 913ee400 94533cc7 (d4210000)
>
> Oh fun :(
>
> Dragging in a bunch more people to the cc: and to: lines to have them
> look at this...

Thanks,

M.
--
Jazz is not dead. It just smells funny...

2018-04-09 09:52:54

by Ard Biesheuvel

[permalink] [raw]
Subject: Re: Linux 4.9.93

On 9 April 2018 at 11:30, Greg KH <[email protected]> wrote:
> On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:
>> Hi,
>>
>> After this patchset, a kernel built with CFI fails. Disabling
>> UNMAP_KERNEL_AT_EL0 fix the issue obviously.

How does one 'build a kernel with CFI' for arm64?

>
> Is this a "clean" 4.9.93 tree or a "4.9.93 merged into
> android-common-4.9?
>
>> Wondering if there is one of the test suite used on the review patchset that covers the CFI usecase.
>>
>> Best regards,
>>
>> [ 0.249191] CPU features: detected feature: GIC system register CPU interface
>> [ 0.256391] CPU features: detected feature: Privileged Access Never
>> [ 0.262719] CPU features: detected feature: User Access Override
>> [ 0.268791] CPU features: detected feature: 32-bit EL0 Support
>> [ 0.274683] CPU features: detected feature: Kernel page table isolation (KPTI)
>> [ 0.282166] CFI failure:
>> [ 0.282169] CFI failure:
>> [ 0.282172] CFI failure:
>> [ 0.282173] CFI failure:
>> [ 0.282175] CFI failure:
>> [ 0.282176] CFI failure:
>> [ 0.282177] CFI failure:
>> [ 0.282178] CFI failure:
>> [ 0.282188] ------------[ cut here ]------------
>> [ 0.282189] ------------[ cut here ]------------
>> [ 0.282190] ------------[ cut here ]------------
>> [ 0.282191] ------------[ cut here ]------------
>> [ 0.282193] ------------[ cut here ]------------
>> [ 0.282196] kernel BUG at kernel/cfi.c:32!
>> [ 0.282198] ------------[ cut here ]------------
>> [ 0.282201] kernel BUG at kernel/cfi.c:32!
>> [ 0.282202] ------------[ cut here ]------------
>> [ 0.282204] kernel BUG at kernel/cfi.c:32!
>> [ 0.282207] kernel BUG at kernel/cfi.c:32!
>> [ 0.282209] kernel BUG at kernel/cfi.c:32!
>> [ 0.282211] kernel BUG at kernel/cfi.c:32!
>> [ 0.282214] kernel BUG at kernel/cfi.c:32!
>> [ 0.282215] ------------[ cut here ]------------
>> [ 0.282216] kernel BUG at kernel/cfi.c:32!
>> [ 0.282218] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
>> [ 0.282224] Modules linked in:
>> [ 0.282230] CPU: 2 PID: 25 Comm: migration/2 Not tainted 4.9.93-perf+ #39
>> [ 0.282232] Hardware name: <REMOVED>
>> [ 0.282235] task: fffffffbb3b36580 task.stack: fffffffbb30cc000
>> [ 0.282250] PC is at __cfi_check_fail+0x14/0x1c
>> [ 0.282253] LR is at __cfi_check_fail+0x14/0x1c
>> [ 0.282255] pc : [<ffffff93b3f03d90>] lr : [<ffffff93b3f03d90>] pstate: 60c00085
>> [ 0.282256] sp : fffffffbb30cfc30
>> [ 0.282259] x29: fffffffbb30cfc30 x28: ffffff93b6415000
>> [ 0.282261] x27: 00000013b65c1000 x26: ffffff93b5ce6000
>> [ 0.282264] x25: ffffff93b5ce6000 x24: ffffff93b6419000
>> [ 0.282266] x23: ffffff93b65c1000 x22: ffffff93b65c4000
>> [ 0.282268] x21: 9d12f8172cb2f296 x20: 000000008180e3e0
>> [ 0.282271] x19: 0000000000000000 x18: 000000000000002c
>> [ 0.282274] x17: 00000000000fd054 x16: 0000000000000000
>> [ 0.282276] x15: ffffff93b65ec000 x14: 000000000000000c
>> [ 0.282279] x13: 0000000000000004 x12: 0000000000000000
>> [ 0.282281] x11: 0000000000000000 x10: 0000000001440144
>> [ 0.282283] x9 : 260822e8751d5000 x8 : 260822e8751d5000
>> [ 0.282286] x7 : 0000000000000000 x6 : fffffffbbac75b60
>> [ 0.282288] x5 : 0000000000000000 x4 : 0000000000000000
>> [ 0.282290] x3 : 000000003a657275 x2 : 0000000000000000
>> [ 0.282292] x1 : 0000000000000000 x0 : 000000000000000c
>> [ 0.282294]
>> [ 0.282294] PC: 0xffffff93b3f03d50:
>> [ 0.282308] 3d50 b9001ac8 f94002c8 370ffec8 17ffffbe d4210000 14000000 aa1603e0 f90007e8
>> [ 0.282315] 3d70 94536017 f94007e8 17ffffe2 a9bf7bfd 910003fd d000d100 913ee400 94533cc7
>> [ 0.282322] 3d90 d4210000 14000000 b0013788 2a1f03e0 f901c51f d65f03c0 f940406b 2a0203e8
>> [ 0.282329] 3db0 2a0103e9 aa0003ea b400008b f9000145 f94000cb b40001ab a9bf7bfd 910003fd
>> [ 0.282330]
>> [ 0.282330] LR: 0xffffff93b3f03d50:
>> [ 0.282336] 3d50 b9001ac8 f94002c8 370ffec8 17ffffbe d4210000 14000000 aa1603e0 f90007e8
>> [ 0.282343] 3d70 94536017 f94007e8 17ffffe2 a9bf7bfd 910003fd d000d100 913ee400 94533cc7
>> [ 0.282350] 3d90 d4210000 14000000 b0013788 2a1f03e0 f901c51f d65f03c0 f940406b 2a0203e8
>> [ 0.282357] 3db0 2a0103e9 aa0003ea b400008b f9000145 f94000cb b40001ab a9bf7bfd 910003fd
>> [ 0.282358]
>> [ 0.282358] SP: 0xfffffffbb30cfbf0:
>> [ 0.282365] fbf0 b3f03d90 ffffff93 b30cfc30 fffffffb b3f03d90 ffffff93 60c00085 00000000
>> [ 0.282372] fc10 b6415000 ffffff93 b642fa00 ffffff93 ffffffff ffffffff b3f03d90 ffffff93
>> [ 0.282378] fc30 b30cfc70 fffffffb b3d458c0 ffffff93 00000080 00000000 00000001 00000000
>> [ 0.282385] fc50 b65c4000 ffffff93 b64420f0 ffffff93 8180e3e0 00000000 00000002 00000000
>> [ 0.282387] Process migration/2 (pid: 25, stack limit = 0xfffffffbb30cc000)
>> [ 0.282389] Call trace:
>> [ 0.282391] Exception stack(0xfffffffbb30cfb00 to 0xfffffffbb30cfc30)
>> [ 0.282395] fb00: 000000000000000c 0000000000000000 0000000000000000 000000003a657275
>> [ 0.282397] fb20: 0000000000000000 0000000000000000 fffffffbbac75b60 0000000000000000
>> [ 0.282400] fb40: 260822e8751d5000 260822e8751d5000 0000000001440144 0000000000000000
>> [ 0.282403] fb60: 0000000000000000 0000000000000004 000000000000000c ffffff93b65ec000
>> [ 0.282405] fb80: 0000000000000000 00000000000fd054 000000000000002c 0000000000000000
>> [ 0.282408] fba0: 000000008180e3e0 9d12f8172cb2f296 ffffff93b65c4000 ffffff93b65c1000
>> [ 0.282411] fbc0: ffffff93b6419000 ffffff93b5ce6000 ffffff93b5ce6000 00000013b65c1000
>> [ 0.282413] fbe0: ffffff93b6415000 fffffffbb30cfc30 ffffff93b3f03d90 fffffffbb30cfc30
>> [ 0.282416] fc00: ffffff93b3f03d90 0000000060c00085 ffffff93b6415000 ffffff93b642fa00
>> [ 0.282418] fc20: ffffffffffffffff ffffff93b3f03d90
>> [ 0.282421] [<ffffff93b3f03d90>] __cfi_check_fail+0x14/0x1c
>> [ 0.282430] [<ffffff93b3d458c0>] name_to_dev_t+0x0/0x47c
>> [ 0.282436] [<ffffff93b3d51b80>] kpti_install_ng_mappings+0x178/0x2e0
>> [ 0.282443] [<ffffff93b3eae950>] multi_cpu_stop+0x114/0x170
>> [ 0.282445] [<ffffff93b3eaf08c>] cpu_stopper_thread+0x128/0x2e8
>> [ 0.282452] [<ffffff93b3db5504>] smpboot_thread_fn+0x230/0x558
>> [ 0.282455] [<ffffff93b3dae848>] kthread+0x21c/0x238
>> [ 0.282459] [<ffffff93b3c838f0>] ret_from_fork+0x10/0x20
>> [ 0.282464] Code: 910003fd d000d100 913ee400 94533cc7 (d4210000)
>
> Oh fun :(
>
> Dragging in a bunch more people to the cc: and to: lines to have them
> look at this...
>
> thanks,
>
> greg k-h

2018-04-09 09:59:33

by Mark Rutland

[permalink] [raw]
Subject: Re: Linux 4.9.93

On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:
> Hi,

Hi,

> After this patchset, a kernel built with CFI fails. Disabling
> UNMAP_KERNEL_AT_EL0 fix the issue obviously.
>
> Wondering if there is one of the test suite used on the review
> patchset that covers the CFI usecase.

IIUC, you're saying that after my v4.9.y meltdown backport, your own
backport of an out-of-tree patchset does not function.

There's no support for CFI in upsteam (at least as of v4.16), nor in
v4.9.93. I have no idea as to how KPTI interacts with an out-of-tree
patchset, so could you elaborate on the issue?

I've not seen any CFI patches on the linux-arm-kernel mailing list, so
it's not clear to me how this interacts with KPTI.

Which patches do you have applied that implement CFI? Which compiler are
you using?

[...]

> [ 0.282196] kernel BUG at kernel/cfi.c:32!

This file doesn't exit in v4.9.93...

> [ 0.282230] CPU: 2 PID: 25 Comm: migration/2 Not tainted 4.9.93-perf+ #39

... and here I see you have local changes applied.

It would be best if you tried to get CFI supported upstream before
trying to backport it. That would be best discussed on the
linux-arm-kernel mailing list.

Thanks,
Mark.

2018-04-09 10:02:55

by Jean-Baptiste Théou

[permalink] [raw]
Subject: Re: Linux 4.9.93

On Mon, 9 Apr 2018 11:49:37 +0200
Ard Biesheuvel <[email protected]> wrote:

> On 9 April 2018 at 11:30, Greg KH <[email protected]> wrote:
> > On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:
> >> Hi,
> >>
> >> After this patchset, a kernel built with CFI fails. Disabling
> >> UNMAP_KERNEL_AT_EL0 fix the issue obviously.
>
> How does one 'build a kernel with CFI' for arm64?

From Google work on Android-4.9

https://android.googlesource.com/kernel/common/+/00a195e7c0752ff5d65c9caadfbcc226270ca232

I am not sure what is the plan on their side to upstream (Greg?), but definitely
useful to isolate actual issues.

>
> >
> > Is this a "clean" 4.9.93 tree or a "4.9.93 merged into
> > android-common-4.9?

It's a "clean 4.9.93" + whatever is needed for Clang/CFI support

My take is that CFI doesn't like

* void __kpti_install_ng_mappings(int cpu, int num_cpus, phys_addr_t swapper)

and

remap_fn = (void *)__pa_symbol(idmap_kpti_install_ng_mappings);

Maybe just flag this function to not use CFI? I remember that Sami Tolvanen did
similar changes.

I know it's a bit out of context since CFI support for ARM64 is not upstream yet,
but unfortunate that an stable patchset trigger such failures.

Thanks a lot

Best regards

> >
> >> Wondering if there is one of the test suite used on the review patchset that covers the CFI usecase.
> >>
> >> Best regards,
> >>
> >> [ 0.249191] CPU features: detected feature: GIC system register CPU interface
> >> [ 0.256391] CPU features: detected feature: Privileged Access Never
> >> [ 0.262719] CPU features: detected feature: User Access Override
> >> [ 0.268791] CPU features: detected feature: 32-bit EL0 Support
> >> [ 0.274683] CPU features: detected feature: Kernel page table isolation (KPTI)
> >> [ 0.282166] CFI failure:
> >> [ 0.282169] CFI failure:
> >> [ 0.282172] CFI failure:
> >> [ 0.282173] CFI failure:
> >> [ 0.282175] CFI failure:
> >> [ 0.282176] CFI failure:
> >> [ 0.282177] CFI failure:
> >> [ 0.282178] CFI failure:
> >> [ 0.282188] ------------[ cut here ]------------
> >> [ 0.282189] ------------[ cut here ]------------
> >> [ 0.282190] ------------[ cut here ]------------
> >> [ 0.282191] ------------[ cut here ]------------
> >> [ 0.282193] ------------[ cut here ]------------
> >> [ 0.282196] kernel BUG at kernel/cfi.c:32!
> >> [ 0.282198] ------------[ cut here ]------------
> >> [ 0.282201] kernel BUG at kernel/cfi.c:32!
> >> [ 0.282202] ------------[ cut here ]------------
> >> [ 0.282204] kernel BUG at kernel/cfi.c:32!
> >> [ 0.282207] kernel BUG at kernel/cfi.c:32!
> >> [ 0.282209] kernel BUG at kernel/cfi.c:32!
> >> [ 0.282211] kernel BUG at kernel/cfi.c:32!
> >> [ 0.282214] kernel BUG at kernel/cfi.c:32!
> >> [ 0.282215] ------------[ cut here ]------------
> >> [ 0.282216] kernel BUG at kernel/cfi.c:32!
> >> [ 0.282218] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
> >> [ 0.282224] Modules linked in:
> >> [ 0.282230] CPU: 2 PID: 25 Comm: migration/2 Not tainted 4.9.93-perf+ #39
> >> [ 0.282232] Hardware name: <REMOVED>
> >> [ 0.282235] task: fffffffbb3b36580 task.stack: fffffffbb30cc000
> >> [ 0.282250] PC is at __cfi_check_fail+0x14/0x1c
> >> [ 0.282253] LR is at __cfi_check_fail+0x14/0x1c
> >> [ 0.282255] pc : [<ffffff93b3f03d90>] lr : [<ffffff93b3f03d90>] pstate: 60c00085
> >> [ 0.282256] sp : fffffffbb30cfc30
> >> [ 0.282259] x29: fffffffbb30cfc30 x28: ffffff93b6415000
> >> [ 0.282261] x27: 00000013b65c1000 x26: ffffff93b5ce6000
> >> [ 0.282264] x25: ffffff93b5ce6000 x24: ffffff93b6419000
> >> [ 0.282266] x23: ffffff93b65c1000 x22: ffffff93b65c4000
> >> [ 0.282268] x21: 9d12f8172cb2f296 x20: 000000008180e3e0
> >> [ 0.282271] x19: 0000000000000000 x18: 000000000000002c
> >> [ 0.282274] x17: 00000000000fd054 x16: 0000000000000000
> >> [ 0.282276] x15: ffffff93b65ec000 x14: 000000000000000c
> >> [ 0.282279] x13: 0000000000000004 x12: 0000000000000000
> >> [ 0.282281] x11: 0000000000000000 x10: 0000000001440144
> >> [ 0.282283] x9 : 260822e8751d5000 x8 : 260822e8751d5000
> >> [ 0.282286] x7 : 0000000000000000 x6 : fffffffbbac75b60
> >> [ 0.282288] x5 : 0000000000000000 x4 : 0000000000000000
> >> [ 0.282290] x3 : 000000003a657275 x2 : 0000000000000000
> >> [ 0.282292] x1 : 0000000000000000 x0 : 000000000000000c
> >> [ 0.282294]
> >> [ 0.282294] PC: 0xffffff93b3f03d50:
> >> [ 0.282308] 3d50 b9001ac8 f94002c8 370ffec8 17ffffbe d4210000 14000000 aa1603e0 f90007e8
> >> [ 0.282315] 3d70 94536017 f94007e8 17ffffe2 a9bf7bfd 910003fd d000d100 913ee400 94533cc7
> >> [ 0.282322] 3d90 d4210000 14000000 b0013788 2a1f03e0 f901c51f d65f03c0 f940406b 2a0203e8
> >> [ 0.282329] 3db0 2a0103e9 aa0003ea b400008b f9000145 f94000cb b40001ab a9bf7bfd 910003fd
> >> [ 0.282330]
> >> [ 0.282330] LR: 0xffffff93b3f03d50:
> >> [ 0.282336] 3d50 b9001ac8 f94002c8 370ffec8 17ffffbe d4210000 14000000 aa1603e0 f90007e8
> >> [ 0.282343] 3d70 94536017 f94007e8 17ffffe2 a9bf7bfd 910003fd d000d100 913ee400 94533cc7
> >> [ 0.282350] 3d90 d4210000 14000000 b0013788 2a1f03e0 f901c51f d65f03c0 f940406b 2a0203e8
> >> [ 0.282357] 3db0 2a0103e9 aa0003ea b400008b f9000145 f94000cb b40001ab a9bf7bfd 910003fd
> >> [ 0.282358]
> >> [ 0.282358] SP: 0xfffffffbb30cfbf0:
> >> [ 0.282365] fbf0 b3f03d90 ffffff93 b30cfc30 fffffffb b3f03d90 ffffff93 60c00085 00000000
> >> [ 0.282372] fc10 b6415000 ffffff93 b642fa00 ffffff93 ffffffff ffffffff b3f03d90 ffffff93
> >> [ 0.282378] fc30 b30cfc70 fffffffb b3d458c0 ffffff93 00000080 00000000 00000001 00000000
> >> [ 0.282385] fc50 b65c4000 ffffff93 b64420f0 ffffff93 8180e3e0 00000000 00000002 00000000
> >> [ 0.282387] Process migration/2 (pid: 25, stack limit = 0xfffffffbb30cc000)
> >> [ 0.282389] Call trace:
> >> [ 0.282391] Exception stack(0xfffffffbb30cfb00 to 0xfffffffbb30cfc30)
> >> [ 0.282395] fb00: 000000000000000c 0000000000000000 0000000000000000 000000003a657275
> >> [ 0.282397] fb20: 0000000000000000 0000000000000000 fffffffbbac75b60 0000000000000000
> >> [ 0.282400] fb40: 260822e8751d5000 260822e8751d5000 0000000001440144 0000000000000000
> >> [ 0.282403] fb60: 0000000000000000 0000000000000004 000000000000000c ffffff93b65ec000
> >> [ 0.282405] fb80: 0000000000000000 00000000000fd054 000000000000002c 0000000000000000
> >> [ 0.282408] fba0: 000000008180e3e0 9d12f8172cb2f296 ffffff93b65c4000 ffffff93b65c1000
> >> [ 0.282411] fbc0: ffffff93b6419000 ffffff93b5ce6000 ffffff93b5ce6000 00000013b65c1000
> >> [ 0.282413] fbe0: ffffff93b6415000 fffffffbb30cfc30 ffffff93b3f03d90 fffffffbb30cfc30
> >> [ 0.282416] fc00: ffffff93b3f03d90 0000000060c00085 ffffff93b6415000 ffffff93b642fa00
> >> [ 0.282418] fc20: ffffffffffffffff ffffff93b3f03d90
> >> [ 0.282421] [<ffffff93b3f03d90>] __cfi_check_fail+0x14/0x1c
> >> [ 0.282430] [<ffffff93b3d458c0>] name_to_dev_t+0x0/0x47c
> >> [ 0.282436] [<ffffff93b3d51b80>] kpti_install_ng_mappings+0x178/0x2e0
> >> [ 0.282443] [<ffffff93b3eae950>] multi_cpu_stop+0x114/0x170
> >> [ 0.282445] [<ffffff93b3eaf08c>] cpu_stopper_thread+0x128/0x2e8
> >> [ 0.282452] [<ffffff93b3db5504>] smpboot_thread_fn+0x230/0x558
> >> [ 0.282455] [<ffffff93b3dae848>] kthread+0x21c/0x238
> >> [ 0.282459] [<ffffff93b3c838f0>] ret_from_fork+0x10/0x20
> >> [ 0.282464] Code: 910003fd d000d100 913ee400 94533cc7 (d4210000)
> >
> > Oh fun :(
> >
> > Dragging in a bunch more people to the cc: and to: lines to have them
> > look at this...
> >
> > thanks,
> >
> > greg k-h

2018-04-09 10:11:01

by Mark Rutland

[permalink] [raw]
Subject: Re: Linux 4.9.93

On Mon, Apr 09, 2018 at 06:57:51PM +0900, Jean-Baptiste Theou wrote:
> On Mon, 9 Apr 2018 11:49:37 +0200
> Ard Biesheuvel <[email protected]> wrote:
>
> > On 9 April 2018 at 11:30, Greg KH <[email protected]> wrote:
> > > On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:
> > >> Hi,
> > >>
> > >> After this patchset, a kernel built with CFI fails. Disabling
> > >> UNMAP_KERNEL_AT_EL0 fix the issue obviously.
> >
> > How does one 'build a kernel with CFI' for arm64?
>
> From Google work on Android-4.9
>
> https://android.googlesource.com/kernel/common/+/00a195e7c0752ff5d65c9caadfbcc226270ca232
>
> I am not sure what is the plan on their side to upstream (Greg?), but definitely
> useful to isolate actual issues.
>
> > > Is this a "clean" 4.9.93 tree or a "4.9.93 merged into
> > > android-common-4.9?
>
> It's a "clean 4.9.93" + whatever is needed for Clang/CFI support
>
> My take is that CFI doesn't like
>
> * void __kpti_install_ng_mappings(int cpu, int num_cpus, phys_addr_t swapper)
>
> and
>
> remap_fn = (void *)__pa_symbol(idmap_kpti_install_ng_mappings);
>
> Maybe just flag this function to not use CFI? I remember that Sami Tolvanen did
> similar changes.

From a quick scan, it looks like CFI uses shadow memory for function
prologues. Since we're taking the PA of a function pointer, presumably
this no longer maps to valid shadow.

I'd expect the same to apply to uses of cpu_replace_ttbr1(), but it
looks like the only user of that is marked as __init, and that patch
adds __nocfi to __init functions.

So you probably need to mark kpti_install_ng_mappings() as __nocfi.

> I know it's a bit out of context since CFI support for ARM64 is not upstream yet,
> but unfortunate that an stable patchset trigger such failures.

This is simply the nature of out-of-tree code.

In future, it would be very helpful if you could provide context for
out-of-tree patches in the initial report.

Thanks,
Mark.

2018-04-09 10:29:07

by Ard Biesheuvel

[permalink] [raw]
Subject: Re: Linux 4.9.93



> On 9 Apr 2018, at 11:57, Jean-Baptiste Theou <[email protected]> wrote:
>
> On Mon, 9 Apr 2018 11:49:37 +0200
> Ard Biesheuvel <[email protected]> wrote:
>
>>> On 9 April 2018 at 11:30, Greg KH <[email protected]> wrote:
>>>> On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:
>>>> Hi,
>>>>
>>>> After this patchset, a kernel built with CFI fails. Disabling
>>>> UNMAP_KERNEL_AT_EL0 fix the issue obviously.
>>
>> How does one 'build a kernel with CFI' for arm64?
>
> From Google work on Android-4.9
>
> https://android.googlesource.com/kernel/common/+/00a195e7c0752ff5d65c9caadfbcc226270ca232
>
> I am not sure what is the plan on their side to upstream (Greg?), but definitely
> useful to isolate actual issues.
>
>>
>>>
>>> Is this a "clean" 4.9.93 tree or a "4.9.93 merged into
>>> android-common-4.9?
>
> It's a "clean 4.9.93" + whatever is needed for Clang/CFI support
>
> My take is that CFI doesn't like
>
> * void __kpti_install_ng_mappings(int cpu, int num_cpus, phys_addr_t swapper)
>
> and
>
> remap_fn = (void *)__pa_symbol(idmap_kpti_install_ng_mappings);
>
> Maybe just flag this function to not use CFI? I remember that Sami Tolvanen did
> similar changes.
>
> I know it's a bit out of context since CFI support for ARM64 is not upstream yet,
> but unfortunate that an stable patchset trigger such failures.
>

I am sorry but if you are implying that we should have tested these patches against the out of tree CFI code, I have to disappoint you: that is simply not upstream’s job, and if the Google engineers merged this into their v4.9 tree without proper testing, may I suggest that you report it to them instead?

OTOH, if that is not what you are implying, please ignore the rant :-)


> Thanks a lot
>
> Best regards
>
>>>
>>>> Wondering if there is one of the test suite used on the review patchset that covers the CFI usecase.
>>>>
>>>> Best regards,
>>>>
>>>> [ 0.249191] CPU features: detected feature: GIC system register CPU interface
>>>> [ 0.256391] CPU features: detected feature: Privileged Access Never
>>>> [ 0.262719] CPU features: detected feature: User Access Override
>>>> [ 0.268791] CPU features: detected feature: 32-bit EL0 Support
>>>> [ 0.274683] CPU features: detected feature: Kernel page table isolation (KPTI)
>>>> [ 0.282166] CFI failure:
>>>> [ 0.282169] CFI failure:
>>>> [ 0.282172] CFI failure:
>>>> [ 0.282173] CFI failure:
>>>> [ 0.282175] CFI failure:
>>>> [ 0.282176] CFI failure:
>>>> [ 0.282177] CFI failure:
>>>> [ 0.282178] CFI failure:
>>>> [ 0.282188] ------------[ cut here ]------------
>>>> [ 0.282189] ------------[ cut here ]------------
>>>> [ 0.282190] ------------[ cut here ]------------
>>>> [ 0.282191] ------------[ cut here ]------------
>>>> [ 0.282193] ------------[ cut here ]------------
>>>> [ 0.282196] kernel BUG at kernel/cfi.c:32!
>>>> [ 0.282198] ------------[ cut here ]------------
>>>> [ 0.282201] kernel BUG at kernel/cfi.c:32!
>>>> [ 0.282202] ------------[ cut here ]------------
>>>> [ 0.282204] kernel BUG at kernel/cfi.c:32!
>>>> [ 0.282207] kernel BUG at kernel/cfi.c:32!
>>>> [ 0.282209] kernel BUG at kernel/cfi.c:32!
>>>> [ 0.282211] kernel BUG at kernel/cfi.c:32!
>>>> [ 0.282214] kernel BUG at kernel/cfi.c:32!
>>>> [ 0.282215] ------------[ cut here ]------------
>>>> [ 0.282216] kernel BUG at kernel/cfi.c:32!
>>>> [ 0.282218] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
>>>> [ 0.282224] Modules linked in:
>>>> [ 0.282230] CPU: 2 PID: 25 Comm: migration/2 Not tainted 4.9.93-perf+ #39
>>>> [ 0.282232] Hardware name: <REMOVED>
>>>> [ 0.282235] task: fffffffbb3b36580 task.stack: fffffffbb30cc000
>>>> [ 0.282250] PC is at __cfi_check_fail+0x14/0x1c
>>>> [ 0.282253] LR is at __cfi_check_fail+0x14/0x1c
>>>> [ 0.282255] pc : [<ffffff93b3f03d90>] lr : [<ffffff93b3f03d90>] pstate: 60c00085
>>>> [ 0.282256] sp : fffffffbb30cfc30
>>>> [ 0.282259] x29: fffffffbb30cfc30 x28: ffffff93b6415000
>>>> [ 0.282261] x27: 00000013b65c1000 x26: ffffff93b5ce6000
>>>> [ 0.282264] x25: ffffff93b5ce6000 x24: ffffff93b6419000
>>>> [ 0.282266] x23: ffffff93b65c1000 x22: ffffff93b65c4000
>>>> [ 0.282268] x21: 9d12f8172cb2f296 x20: 000000008180e3e0
>>>> [ 0.282271] x19: 0000000000000000 x18: 000000000000002c
>>>> [ 0.282274] x17: 00000000000fd054 x16: 0000000000000000
>>>> [ 0.282276] x15: ffffff93b65ec000 x14: 000000000000000c
>>>> [ 0.282279] x13: 0000000000000004 x12: 0000000000000000
>>>> [ 0.282281] x11: 0000000000000000 x10: 0000000001440144
>>>> [ 0.282283] x9 : 260822e8751d5000 x8 : 260822e8751d5000
>>>> [ 0.282286] x7 : 0000000000000000 x6 : fffffffbbac75b60
>>>> [ 0.282288] x5 : 0000000000000000 x4 : 0000000000000000
>>>> [ 0.282290] x3 : 000000003a657275 x2 : 0000000000000000
>>>> [ 0.282292] x1 : 0000000000000000 x0 : 000000000000000c
>>>> [ 0.282294]
>>>> [ 0.282294] PC: 0xffffff93b3f03d50:
>>>> [ 0.282308] 3d50 b9001ac8 f94002c8 370ffec8 17ffffbe d4210000 14000000 aa1603e0 f90007e8
>>>> [ 0.282315] 3d70 94536017 f94007e8 17ffffe2 a9bf7bfd 910003fd d000d100 913ee400 94533cc7
>>>> [ 0.282322] 3d90 d4210000 14000000 b0013788 2a1f03e0 f901c51f d65f03c0 f940406b 2a0203e8
>>>> [ 0.282329] 3db0 2a0103e9 aa0003ea b400008b f9000145 f94000cb b40001ab a9bf7bfd 910003fd
>>>> [ 0.282330]
>>>> [ 0.282330] LR: 0xffffff93b3f03d50:
>>>> [ 0.282336] 3d50 b9001ac8 f94002c8 370ffec8 17ffffbe d4210000 14000000 aa1603e0 f90007e8
>>>> [ 0.282343] 3d70 94536017 f94007e8 17ffffe2 a9bf7bfd 910003fd d000d100 913ee400 94533cc7
>>>> [ 0.282350] 3d90 d4210000 14000000 b0013788 2a1f03e0 f901c51f d65f03c0 f940406b 2a0203e8
>>>> [ 0.282357] 3db0 2a0103e9 aa0003ea b400008b f9000145 f94000cb b40001ab a9bf7bfd 910003fd
>>>> [ 0.282358]
>>>> [ 0.282358] SP: 0xfffffffbb30cfbf0:
>>>> [ 0.282365] fbf0 b3f03d90 ffffff93 b30cfc30 fffffffb b3f03d90 ffffff93 60c00085 00000000
>>>> [ 0.282372] fc10 b6415000 ffffff93 b642fa00 ffffff93 ffffffff ffffffff b3f03d90 ffffff93
>>>> [ 0.282378] fc30 b30cfc70 fffffffb b3d458c0 ffffff93 00000080 00000000 00000001 00000000
>>>> [ 0.282385] fc50 b65c4000 ffffff93 b64420f0 ffffff93 8180e3e0 00000000 00000002 00000000
>>>> [ 0.282387] Process migration/2 (pid: 25, stack limit = 0xfffffffbb30cc000)
>>>> [ 0.282389] Call trace:
>>>> [ 0.282391] Exception stack(0xfffffffbb30cfb00 to 0xfffffffbb30cfc30)
>>>> [ 0.282395] fb00: 000000000000000c 0000000000000000 0000000000000000 000000003a657275
>>>> [ 0.282397] fb20: 0000000000000000 0000000000000000 fffffffbbac75b60 0000000000000000
>>>> [ 0.282400] fb40: 260822e8751d5000 260822e8751d5000 0000000001440144 0000000000000000
>>>> [ 0.282403] fb60: 0000000000000000 0000000000000004 000000000000000c ffffff93b65ec000
>>>> [ 0.282405] fb80: 0000000000000000 00000000000fd054 000000000000002c 0000000000000000
>>>> [ 0.282408] fba0: 000000008180e3e0 9d12f8172cb2f296 ffffff93b65c4000 ffffff93b65c1000
>>>> [ 0.282411] fbc0: ffffff93b6419000 ffffff93b5ce6000 ffffff93b5ce6000 00000013b65c1000
>>>> [ 0.282413] fbe0: ffffff93b6415000 fffffffbb30cfc30 ffffff93b3f03d90 fffffffbb30cfc30
>>>> [ 0.282416] fc00: ffffff93b3f03d90 0000000060c00085 ffffff93b6415000 ffffff93b642fa00
>>>> [ 0.282418] fc20: ffffffffffffffff ffffff93b3f03d90
>>>> [ 0.282421] [<ffffff93b3f03d90>] __cfi_check_fail+0x14/0x1c
>>>> [ 0.282430] [<ffffff93b3d458c0>] name_to_dev_t+0x0/0x47c
>>>> [ 0.282436] [<ffffff93b3d51b80>] kpti_install_ng_mappings+0x178/0x2e0
>>>> [ 0.282443] [<ffffff93b3eae950>] multi_cpu_stop+0x114/0x170
>>>> [ 0.282445] [<ffffff93b3eaf08c>] cpu_stopper_thread+0x128/0x2e8
>>>> [ 0.282452] [<ffffff93b3db5504>] smpboot_thread_fn+0x230/0x558
>>>> [ 0.282455] [<ffffff93b3dae848>] kthread+0x21c/0x238
>>>> [ 0.282459] [<ffffff93b3c838f0>] ret_from_fork+0x10/0x20
>>>> [ 0.282464] Code: 910003fd d000d100 913ee400 94533cc7 (d4210000)
>>>
>>> Oh fun :(
>>>
>>> Dragging in a bunch more people to the cc: and to: lines to have them
>>> look at this...
>>>
>>> thanks,
>>>
>>> greg k-h

2018-04-09 10:32:36

by Jean-Baptiste Théou

[permalink] [raw]
Subject: Re: Linux 4.9.93

On Mon, 9 Apr 2018 12:25:07 +0200
Ard Biesheuvel <[email protected]> wrote:

> > On 9 Apr 2018, at 11:57, Jean-Baptiste Theou <[email protected]> wrote:
> >
> > On Mon, 9 Apr 2018 11:49:37 +0200
> > Ard Biesheuvel <[email protected]> wrote:
> >
> >>> On 9 April 2018 at 11:30, Greg KH <[email protected]> wrote:
> >>>> On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:
> >>>> Hi,
> >>>>
> >>>> After this patchset, a kernel built with CFI fails. Disabling
> >>>> UNMAP_KERNEL_AT_EL0 fix the issue obviously.
> >>
> >> How does one 'build a kernel with CFI' for arm64?
> >
> > From Google work on Android-4.9
> >
> > https://android.googlesource.com/kernel/common/+/00a195e7c0752ff5d65c9caadfbcc226270ca232
> >
> > I am not sure what is the plan on their side to upstream (Greg?), but definitely
> > useful to isolate actual issues.
> >
> >>
> >>>
> >>> Is this a "clean" 4.9.93 tree or a "4.9.93 merged into
> >>> android-common-4.9?
> >
> > It's a "clean 4.9.93" + whatever is needed for Clang/CFI support
> >
> > My take is that CFI doesn't like
> >
> > * void __kpti_install_ng_mappings(int cpu, int num_cpus, phys_addr_t swapper)
> >
> > and
> >
> > remap_fn = (void *)__pa_symbol(idmap_kpti_install_ng_mappings);
> >
> > Maybe just flag this function to not use CFI? I remember that Sami Tolvanen did
> > similar changes.
> >
> > I know it's a bit out of context since CFI support for ARM64 is not upstream yet,
> > but unfortunate that an stable patchset trigger such failures.
> >
>
> I am sorry but if you are implying that we should have tested these patches against the out of tree CFI code, I have to disappoint you: that is simply not upstream’s job, and if the Google engineers merged this into their v4.9 tree without proper testing, may I suggest that you report it to them instead?
>
> OTOH, if that is not what you are implying, please ignore the rant :-)
>

To be perfectly honest, I forgot that CFI wasn't an upstream feature. Indeed, I don't expect upstream test farm to run out of tree.
The real answer here is to upstream CFI ;-)

Thanks a lot

Best regards

>
> > Thanks a lot
> >
> > Best regards
> >
> >>>
> >>>> Wondering if there is one of the test suite used on the review patchset that covers the CFI usecase.
> >>>>
> >>>> Best regards,
> >>>>
> >>>> [ 0.249191] CPU features: detected feature: GIC system register CPU interface
> >>>> [ 0.256391] CPU features: detected feature: Privileged Access Never
> >>>> [ 0.262719] CPU features: detected feature: User Access Override
> >>>> [ 0.268791] CPU features: detected feature: 32-bit EL0 Support
> >>>> [ 0.274683] CPU features: detected feature: Kernel page table isolation (KPTI)
> >>>> [ 0.282166] CFI failure:
> >>>> [ 0.282169] CFI failure:
> >>>> [ 0.282172] CFI failure:
> >>>> [ 0.282173] CFI failure:
> >>>> [ 0.282175] CFI failure:
> >>>> [ 0.282176] CFI failure:
> >>>> [ 0.282177] CFI failure:
> >>>> [ 0.282178] CFI failure:
> >>>> [ 0.282188] ------------[ cut here ]------------
> >>>> [ 0.282189] ------------[ cut here ]------------
> >>>> [ 0.282190] ------------[ cut here ]------------
> >>>> [ 0.282191] ------------[ cut here ]------------
> >>>> [ 0.282193] ------------[ cut here ]------------
> >>>> [ 0.282196] kernel BUG at kernel/cfi.c:32!
> >>>> [ 0.282198] ------------[ cut here ]------------
> >>>> [ 0.282201] kernel BUG at kernel/cfi.c:32!
> >>>> [ 0.282202] ------------[ cut here ]------------
> >>>> [ 0.282204] kernel BUG at kernel/cfi.c:32!
> >>>> [ 0.282207] kernel BUG at kernel/cfi.c:32!
> >>>> [ 0.282209] kernel BUG at kernel/cfi.c:32!
> >>>> [ 0.282211] kernel BUG at kernel/cfi.c:32!
> >>>> [ 0.282214] kernel BUG at kernel/cfi.c:32!
> >>>> [ 0.282215] ------------[ cut here ]------------
> >>>> [ 0.282216] kernel BUG at kernel/cfi.c:32!
> >>>> [ 0.282218] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
> >>>> [ 0.282224] Modules linked in:
> >>>> [ 0.282230] CPU: 2 PID: 25 Comm: migration/2 Not tainted 4.9.93-perf+ #39
> >>>> [ 0.282232] Hardware name: <REMOVED>
> >>>> [ 0.282235] task: fffffffbb3b36580 task.stack: fffffffbb30cc000
> >>>> [ 0.282250] PC is at __cfi_check_fail+0x14/0x1c
> >>>> [ 0.282253] LR is at __cfi_check_fail+0x14/0x1c
> >>>> [ 0.282255] pc : [<ffffff93b3f03d90>] lr : [<ffffff93b3f03d90>] pstate: 60c00085
> >>>> [ 0.282256] sp : fffffffbb30cfc30
> >>>> [ 0.282259] x29: fffffffbb30cfc30 x28: ffffff93b6415000
> >>>> [ 0.282261] x27: 00000013b65c1000 x26: ffffff93b5ce6000
> >>>> [ 0.282264] x25: ffffff93b5ce6000 x24: ffffff93b6419000
> >>>> [ 0.282266] x23: ffffff93b65c1000 x22: ffffff93b65c4000
> >>>> [ 0.282268] x21: 9d12f8172cb2f296 x20: 000000008180e3e0
> >>>> [ 0.282271] x19: 0000000000000000 x18: 000000000000002c
> >>>> [ 0.282274] x17: 00000000000fd054 x16: 0000000000000000
> >>>> [ 0.282276] x15: ffffff93b65ec000 x14: 000000000000000c
> >>>> [ 0.282279] x13: 0000000000000004 x12: 0000000000000000
> >>>> [ 0.282281] x11: 0000000000000000 x10: 0000000001440144
> >>>> [ 0.282283] x9 : 260822e8751d5000 x8 : 260822e8751d5000
> >>>> [ 0.282286] x7 : 0000000000000000 x6 : fffffffbbac75b60
> >>>> [ 0.282288] x5 : 0000000000000000 x4 : 0000000000000000
> >>>> [ 0.282290] x3 : 000000003a657275 x2 : 0000000000000000
> >>>> [ 0.282292] x1 : 0000000000000000 x0 : 000000000000000c
> >>>> [ 0.282294]
> >>>> [ 0.282294] PC: 0xffffff93b3f03d50:
> >>>> [ 0.282308] 3d50 b9001ac8 f94002c8 370ffec8 17ffffbe d4210000 14000000 aa1603e0 f90007e8
> >>>> [ 0.282315] 3d70 94536017 f94007e8 17ffffe2 a9bf7bfd 910003fd d000d100 913ee400 94533cc7
> >>>> [ 0.282322] 3d90 d4210000 14000000 b0013788 2a1f03e0 f901c51f d65f03c0 f940406b 2a0203e8
> >>>> [ 0.282329] 3db0 2a0103e9 aa0003ea b400008b f9000145 f94000cb b40001ab a9bf7bfd 910003fd
> >>>> [ 0.282330]
> >>>> [ 0.282330] LR: 0xffffff93b3f03d50:
> >>>> [ 0.282336] 3d50 b9001ac8 f94002c8 370ffec8 17ffffbe d4210000 14000000 aa1603e0 f90007e8
> >>>> [ 0.282343] 3d70 94536017 f94007e8 17ffffe2 a9bf7bfd 910003fd d000d100 913ee400 94533cc7
> >>>> [ 0.282350] 3d90 d4210000 14000000 b0013788 2a1f03e0 f901c51f d65f03c0 f940406b 2a0203e8
> >>>> [ 0.282357] 3db0 2a0103e9 aa0003ea b400008b f9000145 f94000cb b40001ab a9bf7bfd 910003fd
> >>>> [ 0.282358]
> >>>> [ 0.282358] SP: 0xfffffffbb30cfbf0:
> >>>> [ 0.282365] fbf0 b3f03d90 ffffff93 b30cfc30 fffffffb b3f03d90 ffffff93 60c00085 00000000
> >>>> [ 0.282372] fc10 b6415000 ffffff93 b642fa00 ffffff93 ffffffff ffffffff b3f03d90 ffffff93
> >>>> [ 0.282378] fc30 b30cfc70 fffffffb b3d458c0 ffffff93 00000080 00000000 00000001 00000000
> >>>> [ 0.282385] fc50 b65c4000 ffffff93 b64420f0 ffffff93 8180e3e0 00000000 00000002 00000000
> >>>> [ 0.282387] Process migration/2 (pid: 25, stack limit = 0xfffffffbb30cc000)
> >>>> [ 0.282389] Call trace:
> >>>> [ 0.282391] Exception stack(0xfffffffbb30cfb00 to 0xfffffffbb30cfc30)
> >>>> [ 0.282395] fb00: 000000000000000c 0000000000000000 0000000000000000 000000003a657275
> >>>> [ 0.282397] fb20: 0000000000000000 0000000000000000 fffffffbbac75b60 0000000000000000
> >>>> [ 0.282400] fb40: 260822e8751d5000 260822e8751d5000 0000000001440144 0000000000000000
> >>>> [ 0.282403] fb60: 0000000000000000 0000000000000004 000000000000000c ffffff93b65ec000
> >>>> [ 0.282405] fb80: 0000000000000000 00000000000fd054 000000000000002c 0000000000000000
> >>>> [ 0.282408] fba0: 000000008180e3e0 9d12f8172cb2f296 ffffff93b65c4000 ffffff93b65c1000
> >>>> [ 0.282411] fbc0: ffffff93b6419000 ffffff93b5ce6000 ffffff93b5ce6000 00000013b65c1000
> >>>> [ 0.282413] fbe0: ffffff93b6415000 fffffffbb30cfc30 ffffff93b3f03d90 fffffffbb30cfc30
> >>>> [ 0.282416] fc00: ffffff93b3f03d90 0000000060c00085 ffffff93b6415000 ffffff93b642fa00
> >>>> [ 0.282418] fc20: ffffffffffffffff ffffff93b3f03d90
> >>>> [ 0.282421] [<ffffff93b3f03d90>] __cfi_check_fail+0x14/0x1c
> >>>> [ 0.282430] [<ffffff93b3d458c0>] name_to_dev_t+0x0/0x47c
> >>>> [ 0.282436] [<ffffff93b3d51b80>] kpti_install_ng_mappings+0x178/0x2e0
> >>>> [ 0.282443] [<ffffff93b3eae950>] multi_cpu_stop+0x114/0x170
> >>>> [ 0.282445] [<ffffff93b3eaf08c>] cpu_stopper_thread+0x128/0x2e8
> >>>> [ 0.282452] [<ffffff93b3db5504>] smpboot_thread_fn+0x230/0x558
> >>>> [ 0.282455] [<ffffff93b3dae848>] kthread+0x21c/0x238
> >>>> [ 0.282459] [<ffffff93b3c838f0>] ret_from_fork+0x10/0x20
> >>>> [ 0.282464] Code: 910003fd d000d100 913ee400 94533cc7 (d4210000)
> >>>
> >>> Oh fun :(
> >>>
> >>> Dragging in a bunch more people to the cc: and to: lines to have them
> >>> look at this...
> >>>
> >>> thanks,
> >>>
> >>> greg k-h

2018-04-09 10:45:09

by Jean-Baptiste Théou

[permalink] [raw]
Subject: Re: Linux 4.9.93

On Mon, 9 Apr 2018 11:07:41 +0100
Mark Rutland <[email protected]> wrote:

> On Mon, Apr 09, 2018 at 06:57:51PM +0900, Jean-Baptiste Theou wrote:
> > On Mon, 9 Apr 2018 11:49:37 +0200
> > Ard Biesheuvel <[email protected]> wrote:
> >
> > > On 9 April 2018 at 11:30, Greg KH <[email protected]> wrote:
> > > > On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:
> > > >> Hi,
> > > >>
> > > >> After this patchset, a kernel built with CFI fails. Disabling
> > > >> UNMAP_KERNEL_AT_EL0 fix the issue obviously.
> > >
> > > How does one 'build a kernel with CFI' for arm64?
> >
> > From Google work on Android-4.9
> >
> > https://android.googlesource.com/kernel/common/+/00a195e7c0752ff5d65c9caadfbcc226270ca232
> >
> > I am not sure what is the plan on their side to upstream (Greg?), but definitely
> > useful to isolate actual issues.
> >
> > > > Is this a "clean" 4.9.93 tree or a "4.9.93 merged into
> > > > android-common-4.9?
> >
> > It's a "clean 4.9.93" + whatever is needed for Clang/CFI support
> >
> > My take is that CFI doesn't like
> >
> > * void __kpti_install_ng_mappings(int cpu, int num_cpus, phys_addr_t swapper)
> >
> > and
> >
> > remap_fn = (void *)__pa_symbol(idmap_kpti_install_ng_mappings);
> >
> > Maybe just flag this function to not use CFI? I remember that Sami Tolvanen did
> > similar changes.
>
> From a quick scan, it looks like CFI uses shadow memory for function
> prologues. Since we're taking the PA of a function pointer, presumably
> this no longer maps to valid shadow.
>
> I'd expect the same to apply to uses of cpu_replace_ttbr1(), but it
> looks like the only user of that is marked as __init, and that patch
> adds __nocfi to __init functions.
>
> So you probably need to mark kpti_install_ng_mappings() as __nocfi.
>
> > I know it's a bit out of context since CFI support for ARM64 is not upstream yet,
> > but unfortunate that an stable patchset trigger such failures.
>
> This is simply the nature of out-of-tree code.
>
> In future, it would be very helpful if you could provide context for
> out-of-tree patches in the initial report.
>

I can pass the initial CFI failures by tagging the function with __nocfi, but still face issues down the road.
That said, it's out of tree, so my problem.

Will investigate.

Thanks a lot for the quick support.

Best regards

> Thanks,
> Mark.

2018-04-09 11:45:34

by Greg Kroah-Hartman

[permalink] [raw]
Subject: Re: Linux 4.9.93

On Mon, Apr 09, 2018 at 06:57:51PM +0900, Jean-Baptiste Theou wrote:
> On Mon, 9 Apr 2018 11:49:37 +0200
> Ard Biesheuvel <[email protected]> wrote:
>
> > On 9 April 2018 at 11:30, Greg KH <[email protected]> wrote:
> > > On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:
> > >> Hi,
> > >>
> > >> After this patchset, a kernel built with CFI fails. Disabling
> > >> UNMAP_KERNEL_AT_EL0 fix the issue obviously.
> >
> > How does one 'build a kernel with CFI' for arm64?
>
> From Google work on Android-4.9
>
> https://android.googlesource.com/kernel/common/+/00a195e7c0752ff5d65c9caadfbcc226270ca232
>
> I am not sure what is the plan on their side to upstream (Greg?), but definitely
> useful to isolate actual issues.

Ah, yeah, if you are taking anything from the Android tree, all bets are
off :)

Greg Hackmann can probably answer the questions about CFI and these
patches, as he's working on merging this stable release into the
android-4.9 branch this week. I'll let him deal with this...

thanks,

greg k-h

2018-04-09 17:06:33

by Greg Hackmann

[permalink] [raw]
Subject: Re: Linux 4.9.93

On 04/09/2018 04:41 AM, Greg KH wrote:
> On Mon, Apr 09, 2018 at 06:57:51PM +0900, Jean-Baptiste Theou wrote:
>> On Mon, 9 Apr 2018 11:49:37 +0200
>> Ard Biesheuvel <[email protected]> wrote:
>>
>>> On 9 April 2018 at 11:30, Greg KH <[email protected]> wrote:
>>>> On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:
>>>>> Hi,
>>>>>
>>>>> After this patchset, a kernel built with CFI fails. Disabling
>>>>> UNMAP_KERNEL_AT_EL0 fix the issue obviously.
>>>
>>> How does one 'build a kernel with CFI' for arm64?
>>
>> From Google work on Android-4.9
>>
>> https://android.googlesource.com/kernel/common/+/00a195e7c0752ff5d65c9caadfbcc226270ca232
>>
>> I am not sure what is the plan on their side to upstream (Greg?), but definitely
>> useful to isolate actual issues.
>
> Ah, yeah, if you are taking anything from the Android tree, all bets are
> off :)
>
> Greg Hackmann can probably answer the questions about CFI and these
> patches, as he's working on merging this stable release into the
> android-4.9 branch this week. I'll let him deal with this...
>
> thanks,
>
> greg k-h
>

I've added Sami Tolvanen, who's handling CFI in the kernel.

Sami, what are the plans for upstreaming this work?

2018-04-09 20:36:27

by Sami Tolvanen

[permalink] [raw]
Subject: Re: Linux 4.9.93

On Mon, Apr 9, 2018 at 10:02 AM Greg Hackmann <[email protected]> wrote:
> Sami, what are the plans for upstreaming this work?

CFI is a clang-specific feature that depends on LTO. Based on the earlier
LTO discussion, we decided to collect some more evidence that clang's LTO
doesn't actually break anything in the kernel before sending out the next
revision of these patches. I will look into upstreaming this code again
once we have sufficient test coverage, hopefully by the end of the year.

Sami