2018-10-05 22:24:26

by syzbot

[permalink] [raw]
Subject: general protection fault in __handle_mm_fault

Hello,

syzbot found the following crash on:

HEAD commit: 25bcda3e8b9f Add linux-next specific files for 20181004
git tree: linux-next
console output: https://syzkaller.appspot.com/x/log.txt?x=130e3bf1400000
kernel config: https://syzkaller.appspot.com/x/.config?x=603d7f9140c3368a
dashboard link: https://syzkaller.appspot.com/bug?extid=1577fbe983d20fe2e88f
compiler: gcc (GCC) 8.0.1 20180413 (experimental)
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=127e88d6400000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=13cdb67e400000

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: [email protected]

8021q: adding VLAN 0 to HW filter on device team0
IPv6: ADDRCONF(NETDEV_CHANGE): team0: link becomes ready
nf_conntrack: default automatic helper assignment has been turned off for
security reasons and CT-based firewall rule not found. Use the iptables CT
target to attach helpers instead.
kasan: CONFIG_KASAN_INLINE enabled
kasan: GPF could be caused by NULL-ptr deref or user memory access
general protection fault: 0000 [#1] PREEMPT SMP KASAN
CPU: 0 PID: 6015 Comm: syz-executor196 Not tainted
4.19.0-rc6-next-20181004+ #87
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
RIP: 0010:do_fault mm/memory.c:3524 [inline]
RIP: 0010:handle_pte_fault mm/memory.c:3762 [inline]
RIP: 0010:__handle_mm_fault+0x2d72/0x55c0 mm/memory.c:3886
Code: 48 89 fa 48 c1 ea 03 80 3c 02 00 0f 85 87 20 00 00 49 8b 9c 24 b0 fe
ff ff 48 b8 00 00 00 00 00 fc ff df 48 89 da 48 c1 ea 03 <80> 3c 02 00 0f
85 57 20 00 00 48 8b 1b 31 ff 48 83 e3 9f 48 89 de
RSP: 0018:ffff8801bbe66fc0 EFLAGS: 00010246
RAX: dffffc0000000000 RBX: 0000000000000000 RCX: ffffffff8160f791
RDX: 0000000000000000 RSI: 0000000000000004 RDI: ffff8801bbe670f8
RBP: ffff8801bbe67270 R08: ffffed0039d485ae R09: ffffed0039d485ad
R10: ffffed0039d485ad R11: ffff8801cea42d6b R12: ffff8801bbe67248
R13: ffff8801bbc7e800 R14: ffff8801d1759f40 R15: 0000000000000067
FS: 0000000001c6a880(0000) GS:ffff8801dae00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000020077000 CR3: 00000001d79be000 CR4: 00000000001406f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
handle_mm_fault+0x54f/0xc70 mm/memory.c:3923
__do_page_fault+0x567/0xd10 arch/x86/mm/fault.c:1355
do_page_fault+0xed/0x7d1 arch/x86/mm/fault.c:1430
page_fault+0x1e/0x30 arch/x86/entry/entry_64.S:1139
RIP: 0010:copy_user_enhanced_fast_string+0xe/0x20
arch/x86/lib/copy_user_64.S:180
Code: 89 d1 c1 e9 03 83 e2 07 f3 48 a5 89 d1 f3 a4 31 c0 0f 1f 00 c3 0f 1f
80 00 00 00 00 0f 1f 00 83 fa 40 0f 82 70 ff ff ff 89 d1 <f3> a4 31 c0 0f
1f 00 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 83
RSP: 0018:ffff8801bbe675b8 EFLAGS: 00010202
RAX: 0000000000000000 RBX: 0000000000007a50 RCX: 0000000000001b40
RDX: 0000000000007a50 RSI: 0000000020077000 RDI: ffff8801ce615f10
RBP: ffff8801bbe675f0 R08: ffffed0039cc2f4a R09: ffffed0039cc2f4a
R10: ffffed0039cc2f49 R11: ffff8801ce617a4f R12: 0000000020078b40
R13: 00000000200710f0 R14: ffff8801ce610000 R15: 00007ffffffff000
_copy_from_iter_full+0x263/0xc20 lib/iov_iter.c:724
copy_from_iter_full include/linux/uio.h:124 [inline]
skb_do_copy_data_nocache include/net/sock.h:1951 [inline]
skb_copy_to_page_nocache include/net/sock.h:1977 [inline]
tcp_sendmsg_locked+0x159e/0x3f90 net/ipv4/tcp.c:1338
tcp_sendmsg+0x2f/0x50 net/ipv4/tcp.c:1443
inet_sendmsg+0x19c/0x690 net/ipv4/af_inet.c:798
sock_sendmsg_nosec net/socket.c:622 [inline]
sock_sendmsg+0xd5/0x120 net/socket.c:632
__sys_sendto+0x3d7/0x670 net/socket.c:1789
__do_sys_sendto net/socket.c:1801 [inline]
__se_sys_sendto net/socket.c:1797 [inline]
__x64_sys_sendto+0xe1/0x1a0 net/socket.c:1797
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x441159
Code: 18 89 d0 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 48 89 f8 48 89 f7
48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff
ff 0f 83 db 0a fc ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007ffc1def2948 EFLAGS: 00000217 ORIG_RAX: 000000000000002c
RAX: ffffffffffffffda RBX: 00000000004002c8 RCX: 0000000000441159
RDX: fffffffffffffee3 RSI: 0000000020000b40 RDI: 0000000000000003
RBP: 00000000006cc018 R08: 0000000000000000 R09: ffffffffffffff61
R10: 00000000040000cb R11: 0000000000000217 R12: 00000000004020c0
R13: 0000000000402150 R14: 0000000000000000 R15: 0000000000000000
Modules linked in:
---[ end trace d2b868b55677c602 ]---
RIP: 0010:do_fault mm/memory.c:3524 [inline]
RIP: 0010:handle_pte_fault mm/memory.c:3762 [inline]
RIP: 0010:__handle_mm_fault+0x2d72/0x55c0 mm/memory.c:3886
Code: 48 89 fa 48 c1 ea 03 80 3c 02 00 0f 85 87 20 00 00 49 8b 9c 24 b0 fe
ff ff 48 b8 00 00 00 00 00 fc ff df 48 89 da 48 c1 ea 03 <80> 3c 02 00 0f
85 57 20 00 00 48 8b 1b 31 ff 48 83 e3 9f 48 89 de
RSP: 0018:ffff8801bbe66fc0 EFLAGS: 00010246
RAX: dffffc0000000000 RBX: 0000000000000000 RCX: ffffffff8160f791
RDX: 0000000000000000 RSI: 0000000000000004 RDI: ffff8801bbe670f8
RBP: ffff8801bbe67270 R08: ffffed0039d485ae R09: ffffed0039d485ad
R10: ffffed0039d485ad R11: ffff8801cea42d6b R12: ffff8801bbe67248
R13: ffff8801bbc7e800 R14: ffff8801d1759f40 R15: 0000000000000067
FS: 0000000001c6a880(0000) GS:ffff8801dae00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000020077000 CR3: 00000001d79be000 CR4: 00000000001406f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400


---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at [email protected].

syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#bug-status-tracking for how to communicate with
syzbot.
syzbot can test patches for this bug, for details see:
https://goo.gl/tpsmEJ#testing-patches


2018-10-08 16:11:27

by Willem de Bruijn

[permalink] [raw]
Subject: Re: general protection fault in __handle_mm_fault

On Fri, Oct 5, 2018 at 6:27 PM syzbot
<[email protected]> wrote:
>
> Hello,
>
> syzbot found the following crash on:
>
> HEAD commit: 25bcda3e8b9f Add linux-next specific files for 20181004
> git tree: linux-next
> console output: https://syzkaller.appspot.com/x/log.txt?x=130e3bf1400000
> kernel config: https://syzkaller.appspot.com/x/.config?x=603d7f9140c3368a
> dashboard link: https://syzkaller.appspot.com/bug?extid=1577fbe983d20fe2e88f
> compiler: gcc (GCC) 8.0.1 20180413 (experimental)
> syz repro: https://syzkaller.appspot.com/x/repro.syz?x=127e88d6400000
> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=13cdb67e400000

> RIP: 0010:copy_user_enhanced_fast_string+0xe/0x20
> arch/x86/lib/copy_user_64.S:180
> Code: 89 d1 c1 e9 03 83 e2 07 f3 48 a5 89 d1 f3 a4 31 c0 0f 1f 00 c3 0f 1f
> 80 00 00 00 00 0f 1f 00 83 fa 40 0f 82 70 ff ff ff 89 d1 <f3> a4 31 c0 0f
> 1f 00 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 83
> RSP: 0018:ffff8801bbe675b8 EFLAGS: 00010202
> RAX: 0000000000000000 RBX: 0000000000007a50 RCX: 0000000000001b40
> RDX: 0000000000007a50 RSI: 0000000020077000 RDI: ffff8801ce615f10
> RBP: ffff8801bbe675f0 R08: ffffed0039cc2f4a R09: ffffed0039cc2f4a
> R10: ffffed0039cc2f49 R11: ffff8801ce617a4f R12: 0000000020078b40
> R13: 00000000200710f0 R14: ffff8801ce610000 R15: 00007ffffffff000
> _copy_from_iter_full+0x263/0xc20 lib/iov_iter.c:724
> copy_from_iter_full include/linux/uio.h:124 [inline]
> skb_do_copy_data_nocache include/net/sock.h:1951 [inline]
> skb_copy_to_page_nocache include/net/sock.h:1977 [inline]
> tcp_sendmsg_locked+0x159e/0x3f90 net/ipv4/tcp.c:1338

This started on next-20181004. It still happens as of next-20181008.

It does not trigger on next 20181003. It does not occur if
CONFIG_DEBUG_KOBJECT is disabled.

2018-10-08 20:34:52

by Willem de Bruijn

[permalink] [raw]
Subject: Re: general protection fault in __handle_mm_fault

On Mon, Oct 8, 2018 at 12:10 PM Willem de Bruijn
<[email protected]> wrote:
>
> On Fri, Oct 5, 2018 at 6:27 PM syzbot
> <[email protected]> wrote:
> >
> > Hello,
> >
> > syzbot found the following crash on:
> >
> > HEAD commit: 25bcda3e8b9f Add linux-next specific files for 20181004
> > git tree: linux-next
> > console output: https://syzkaller.appspot.com/x/log.txt?x=130e3bf1400000
> > kernel config: https://syzkaller.appspot.com/x/.config?x=603d7f9140c3368a
> > dashboard link: https://syzkaller.appspot.com/bug?extid=1577fbe983d20fe2e88f
> > compiler: gcc (GCC) 8.0.1 20180413 (experimental)
> > syz repro: https://syzkaller.appspot.com/x/repro.syz?x=127e88d6400000
> > C reproducer: https://syzkaller.appspot.com/x/repro.c?x=13cdb67e400000
>
> > RIP: 0010:copy_user_enhanced_fast_string+0xe/0x20
> > arch/x86/lib/copy_user_64.S:180
> > Code: 89 d1 c1 e9 03 83 e2 07 f3 48 a5 89 d1 f3 a4 31 c0 0f 1f 00 c3 0f 1f
> > 80 00 00 00 00 0f 1f 00 83 fa 40 0f 82 70 ff ff ff 89 d1 <f3> a4 31 c0 0f
> > 1f 00 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 83
> > RSP: 0018:ffff8801bbe675b8 EFLAGS: 00010202
> > RAX: 0000000000000000 RBX: 0000000000007a50 RCX: 0000000000001b40
> > RDX: 0000000000007a50 RSI: 0000000020077000 RDI: ffff8801ce615f10
> > RBP: ffff8801bbe675f0 R08: ffffed0039cc2f4a R09: ffffed0039cc2f4a
> > R10: ffffed0039cc2f49 R11: ffff8801ce617a4f R12: 0000000020078b40
> > R13: 00000000200710f0 R14: ffff8801ce610000 R15: 00007ffffffff000
> > _copy_from_iter_full+0x263/0xc20 lib/iov_iter.c:724
> > copy_from_iter_full include/linux/uio.h:124 [inline]
> > skb_do_copy_data_nocache include/net/sock.h:1951 [inline]
> > skb_copy_to_page_nocache include/net/sock.h:1977 [inline]
> > tcp_sendmsg_locked+0x159e/0x3f90 net/ipv4/tcp.c:1338
>
> This started on next-20181004. It still happens as of next-20181008.
>
> It does not trigger on next 20181003. It does not occur if
> CONFIG_DEBUG_KOBJECT is disabled.

Bisected to commit e4d0c281a4c9 ("mm/memory.c: recheck page table
entry with page table lock held").

Verified to not trigger on next-20181008 after reverting that commit.

2018-10-09 08:55:05

by Aneesh Kumar K.V

[permalink] [raw]
Subject: Re: general protection fault in __handle_mm_fault

Willem de Bruijn <[email protected]> writes:

> On Mon, Oct 8, 2018 at 12:10 PM Willem de Bruijn
> <[email protected]> wrote:
>>
>> On Fri, Oct 5, 2018 at 6:27 PM syzbot
>> <[email protected]> wrote:
>> >
>> > Hello,
>> >
>> > syzbot found the following crash on:
>> >
>> > HEAD commit: 25bcda3e8b9f Add linux-next specific files for 20181004
>> > git tree: linux-next
>> > console output: https://syzkaller.appspot.com/x/log.txt?x=130e3bf1400000
>> > kernel config: https://syzkaller.appspot.com/x/.config?x=603d7f9140c3368a
>> > dashboard link: https://syzkaller.appspot.com/bug?extid=1577fbe983d20fe2e88f
>> > compiler: gcc (GCC) 8.0.1 20180413 (experimental)
>> > syz repro: https://syzkaller.appspot.com/x/repro.syz?x=127e88d6400000
>> > C reproducer: https://syzkaller.appspot.com/x/repro.c?x=13cdb67e400000
>>
>> > RIP: 0010:copy_user_enhanced_fast_string+0xe/0x20
>> > arch/x86/lib/copy_user_64.S:180
>> > Code: 89 d1 c1 e9 03 83 e2 07 f3 48 a5 89 d1 f3 a4 31 c0 0f 1f 00 c3 0f 1f
>> > 80 00 00 00 00 0f 1f 00 83 fa 40 0f 82 70 ff ff ff 89 d1 <f3> a4 31 c0 0f
>> > 1f 00 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 83
>> > RSP: 0018:ffff8801bbe675b8 EFLAGS: 00010202
>> > RAX: 0000000000000000 RBX: 0000000000007a50 RCX: 0000000000001b40
>> > RDX: 0000000000007a50 RSI: 0000000020077000 RDI: ffff8801ce615f10
>> > RBP: ffff8801bbe675f0 R08: ffffed0039cc2f4a R09: ffffed0039cc2f4a
>> > R10: ffffed0039cc2f49 R11: ffff8801ce617a4f R12: 0000000020078b40
>> > R13: 00000000200710f0 R14: ffff8801ce610000 R15: 00007ffffffff000
>> > _copy_from_iter_full+0x263/0xc20 lib/iov_iter.c:724
>> > copy_from_iter_full include/linux/uio.h:124 [inline]
>> > skb_do_copy_data_nocache include/net/sock.h:1951 [inline]
>> > skb_copy_to_page_nocache include/net/sock.h:1977 [inline]
>> > tcp_sendmsg_locked+0x159e/0x3f90 net/ipv4/tcp.c:1338
>>
>> This started on next-20181004. It still happens as of next-20181008.
>>
>> It does not trigger on next 20181003. It does not occur if
>> CONFIG_DEBUG_KOBJECT is disabled.
>
> Bisected to commit e4d0c281a4c9 ("mm/memory.c: recheck page table
> entry with page table lock held").
>
> Verified to not trigger on next-20181008 after reverting that commit.


Can you check with this patch

diff --git a/mm/memory.c b/mm/memory.c
index fa8894c70575..15c417e8e31d 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -3505,14 +3505,17 @@ static vm_fault_t do_fault(struct vm_fault *vmf)
* The VMA was not fully populated on mmap() or missing VM_DONTEXPAND
*/
if (!vma->vm_ops->fault) {
-
/*
- * pmd entries won't be marked none during a R/M/W cycle.
+ * If we find a migration pmd entry or a none pmd entry, which
+ * should never happen, return SIGBUS
*/
- if (unlikely(pmd_none(*vmf->pmd)))
+ if (unlikely(!pmd_present(*vmf->pmd)))
ret = VM_FAULT_SIGBUS;
else {
- vmf->ptl = pte_lockptr(vmf->vma->vm_mm, vmf->pmd);
+ vmf->pte = pte_offset_map_lock(vmf->vma->vm_mm,
+ vmf->pmd,
+ vmf->address,
+ &vmf->ptl);
/*
* Make sure this is not a temporary clearing of pte
* by holding ptl and checking again. A R/M/W update
@@ -3520,12 +3523,12 @@ static vm_fault_t do_fault(struct vm_fault *vmf)
* we don't have concurrent modification by hardware
* followed by an update.
*/
- spin_lock(vmf->ptl);
if (unlikely(pte_none(*vmf->pte)))
ret = VM_FAULT_SIGBUS;
else
ret = VM_FAULT_NOPAGE;
- spin_unlock(vmf->ptl);
+
+ pte_unmap_unlock(vmf->pte, vmf->ptl);
}
} else if (!(vmf->flags & FAULT_FLAG_WRITE))
ret = do_read_fault(vmf);


2018-10-09 15:01:04

by Eric Dumazet

[permalink] [raw]
Subject: Re: general protection fault in __handle_mm_fault



On 10/09/2018 01:53 AM, Aneesh Kumar K.V wrote:
...

>>>> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=13cdb67e400000
...

>
> Can you check with this patch

Well, this is a C repro, you can test this yourself instead of asking Willem who
already did a painful bisection.

Thanks.

2018-10-09 16:04:41

by Willem de Bruijn

[permalink] [raw]
Subject: Re: general protection fault in __handle_mm_fault

On Tue, Oct 9, 2018 at 11:00 AM Eric Dumazet <[email protected]> wrote:
>
>
>
> On 10/09/2018 01:53 AM, Aneesh Kumar K.V wrote:
> ...
>
> >>>> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=13cdb67e400000
> ...
>
> >
> > Can you check with this patch

Thanks. With that patch the crash indeed does not occur.

>
> Well, this is a C repro, you can test this yourself instead of asking Willem who
> already did a painful bisection.

Thanks Eric. It does take some effort to get the syzkaller environment
up and running [1] and I happen to have it ready, so I don't mind testing
a few patches.

It just had to wait until I got to the office. Somehow the debug kernel
produces so much output that it consistently locked up my shell over ssh.

[1] for reference:
https://github.com/google/syzkaller/blob/master/docs/linux/setup_ubuntu-host_qemu-vm_x86-64-kernel.md

2018-10-09 17:11:26

by Eric Dumazet

[permalink] [raw]
Subject: Re: general protection fault in __handle_mm_fault

On Tue, Oct 9, 2018 at 10:05 AM Ido Schimmel <[email protected]> wrote:
>
> On Tue, Oct 09, 2018 at 12:02:58PM -0400, Willem de Bruijn wrote:
> > On Tue, Oct 9, 2018 at 11:00 AM Eric Dumazet <[email protected]> wrote:
> > > Well, this is a C repro, you can test this yourself instead of asking Willem who
> > > already did a painful bisection.
> >
> > Thanks Eric. It does take some effort to get the syzkaller environment
> > up and running [1] and I happen to have it ready, so I don't mind testing
> > a few patches.
>
> It is possible to ask syzbot to test the patch. I sent a couple of
> patches and got a response in 20-30 minutes. Very convenient.
>
> It is mentioned at the bottom of the report:
>
> "
> syzbot can test patches for this bug, for details see:
> https://goo.gl/tpsmEJ#testing-patches
> "

One day syzbot will do the bisection as well maybe, once it has a
repro, this could be really nice.

2018-10-09 17:12:41

by Ido Schimmel

[permalink] [raw]
Subject: Re: general protection fault in __handle_mm_fault

On Tue, Oct 09, 2018 at 12:02:58PM -0400, Willem de Bruijn wrote:
> On Tue, Oct 9, 2018 at 11:00 AM Eric Dumazet <[email protected]> wrote:
> > Well, this is a C repro, you can test this yourself instead of asking Willem who
> > already did a painful bisection.
>
> Thanks Eric. It does take some effort to get the syzkaller environment
> up and running [1] and I happen to have it ready, so I don't mind testing
> a few patches.

It is possible to ask syzbot to test the patch. I sent a couple of
patches and got a response in 20-30 minutes. Very convenient.

It is mentioned at the bottom of the report:

"
syzbot can test patches for this bug, for details see:
https://goo.gl/tpsmEJ#testing-patches
"