Hello,
syzbot found the following issue on:
HEAD commit: ee3f96b16468 Merge tag 'nfsd-6.3-1' of git://git.kernel.or..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=1312bba8c80000
kernel config: https://syzkaller.appspot.com/x/.config?x=f5733ca1757172ad
dashboard link: https://syzkaller.appspot.com/bug?extid=68662811b3d5f6695bcb
compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=105393a8c80000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=1113917f480000
Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/29e7966ab711/disk-ee3f96b1.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/ae21b8e855de/vmlinux-ee3f96b1.xz
kernel image: https://storage.googleapis.com/syzbot-assets/803ee0425ad6/bzImage-ee3f96b1.xz
IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: [email protected]
executing program
BUG: memory leak
unreferenced object 0xffff88810a86f7a0 (size 32):
comm "syz-executor325", pid 5099, jiffies 4294978342 (age 119.240s)
hex dump (first 32 bytes):
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:
[<ffffffff81533d64>] kmalloc_trace+0x24/0x90 mm/slab_common.c:1061
[<ffffffff840edaa0>] kmalloc include/linux/slab.h:580 [inline]
[<ffffffff840edaa0>] tcp_md5sig_info_add net/ipv4/tcp_ipv4.c:1169 [inline]
[<ffffffff840edaa0>] tcp_md5_do_add+0xa0/0x150 net/ipv4/tcp_ipv4.c:1240
[<ffffffff84262c73>] tcp_v6_parse_md5_keys+0x253/0x4a0 net/ipv6/tcp_ipv6.c:671
[<ffffffff840c720e>] do_tcp_setsockopt+0x40e/0x1360 net/ipv4/tcp.c:3720
[<ffffffff840c81fb>] tcp_setsockopt+0x9b/0xa0 net/ipv4/tcp.c:3806
[<ffffffff83d72a8b>] __sys_setsockopt+0x1ab/0x330 net/socket.c:2274
[<ffffffff83d72c36>] __do_sys_setsockopt net/socket.c:2285 [inline]
[<ffffffff83d72c36>] __se_sys_setsockopt net/socket.c:2282 [inline]
[<ffffffff83d72c36>] __x64_sys_setsockopt+0x26/0x30 net/socket.c:2282
[<ffffffff849ad699>] do_syscall_x64 arch/x86/entry/common.c:50 [inline]
[<ffffffff849ad699>] do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
[<ffffffff84a0008b>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
BUG: memory leak
unreferenced object 0xffff88811225ccc0 (size 192):
comm "syz-executor325", pid 5099, jiffies 4294978342 (age 119.240s)
hex dump (first 32 bytes):
00 00 00 00 00 00 00 00 22 01 00 00 00 00 ad de ........".......
22 0a 80 00 fe 80 00 00 00 00 00 00 00 00 00 00 "...............
backtrace:
[<ffffffff8153444a>] __do_kmalloc_node mm/slab_common.c:966 [inline]
[<ffffffff8153444a>] __kmalloc+0x4a/0x120 mm/slab_common.c:980
[<ffffffff83d75c15>] kmalloc include/linux/slab.h:584 [inline]
[<ffffffff83d75c15>] sock_kmalloc net/core/sock.c:2635 [inline]
[<ffffffff83d75c15>] sock_kmalloc+0x65/0xa0 net/core/sock.c:2624
[<ffffffff840eb9bb>] __tcp_md5_do_add+0xcb/0x300 net/ipv4/tcp_ipv4.c:1212
[<ffffffff840eda67>] tcp_md5_do_add+0x67/0x150 net/ipv4/tcp_ipv4.c:1253
[<ffffffff84262c73>] tcp_v6_parse_md5_keys+0x253/0x4a0 net/ipv6/tcp_ipv6.c:671
[<ffffffff840c720e>] do_tcp_setsockopt+0x40e/0x1360 net/ipv4/tcp.c:3720
[<ffffffff840c81fb>] tcp_setsockopt+0x9b/0xa0 net/ipv4/tcp.c:3806
[<ffffffff83d72a8b>] __sys_setsockopt+0x1ab/0x330 net/socket.c:2274
[<ffffffff83d72c36>] __do_sys_setsockopt net/socket.c:2285 [inline]
[<ffffffff83d72c36>] __se_sys_setsockopt net/socket.c:2282 [inline]
[<ffffffff83d72c36>] __x64_sys_setsockopt+0x26/0x30 net/socket.c:2282
[<ffffffff849ad699>] do_syscall_x64 arch/x86/entry/common.c:50 [inline]
[<ffffffff849ad699>] do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
[<ffffffff84a0008b>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at [email protected].
syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
If the bug is already fixed, let syzbot know by replying with:
#syz fix: exact-commit-title
If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.
If you want to overwrite bug's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)
If the bug is a duplicate of another bug, reply with:
#syz dup: exact-subject-of-another-report
If you want to undo deduplication, reply with:
#syz undup
On Thu, Sep 21, 2023 at 6:56 PM syzbot
<[email protected]> wrote:
>
> Hello,
>
> syzbot found the following issue on:
>
> HEAD commit: ee3f96b16468 Merge tag 'nfsd-6.3-1' of git://git.kernel.or..
> git tree: upstream
> console output: https://syzkaller.appspot.com/x/log.txt?x=1312bba8c80000
> kernel config: https://syzkaller.appspot.com/x/.config?x=f5733ca1757172ad
> dashboard link: https://syzkaller.appspot.com/bug?extid=68662811b3d5f6695bcb
> compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2
> syz repro: https://syzkaller.appspot.com/x/repro.syz?x=105393a8c80000
> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=1113917f480000
>
> Downloadable assets:
> disk image: https://storage.googleapis.com/syzbot-assets/29e7966ab711/disk-ee3f96b1.raw.xz
> vmlinux: https://storage.googleapis.com/syzbot-assets/ae21b8e855de/vmlinux-ee3f96b1.xz
> kernel image: https://storage.googleapis.com/syzbot-assets/803ee0425ad6/bzImage-ee3f96b1.xz
>
> IMPORTANT: if you fix the issue, please add the following tag to the commit:
> Reported-by: [email protected]
>
> executing program
> BUG: memory leak
> unreferenced object 0xffff88810a86f7a0 (size 32):
> comm "syz-executor325", pid 5099, jiffies 4294978342 (age 119.240s)
> hex dump (first 32 bytes):
> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> backtrace:
> [<ffffffff81533d64>] kmalloc_trace+0x24/0x90 mm/slab_common.c:1061
> [<ffffffff840edaa0>] kmalloc include/linux/slab.h:580 [inline]
> [<ffffffff840edaa0>] tcp_md5sig_info_add net/ipv4/tcp_ipv4.c:1169 [inline]
> [<ffffffff840edaa0>] tcp_md5_do_add+0xa0/0x150 net/ipv4/tcp_ipv4.c:1240
> [<ffffffff84262c73>] tcp_v6_parse_md5_keys+0x253/0x4a0 net/ipv6/tcp_ipv6.c:671
> [<ffffffff840c720e>] do_tcp_setsockopt+0x40e/0x1360 net/ipv4/tcp.c:3720
> [<ffffffff840c81fb>] tcp_setsockopt+0x9b/0xa0 net/ipv4/tcp.c:3806
> [<ffffffff83d72a8b>] __sys_setsockopt+0x1ab/0x330 net/socket.c:2274
> [<ffffffff83d72c36>] __do_sys_setsockopt net/socket.c:2285 [inline]
> [<ffffffff83d72c36>] __se_sys_setsockopt net/socket.c:2282 [inline]
> [<ffffffff83d72c36>] __x64_sys_setsockopt+0x26/0x30 net/socket.c:2282
> [<ffffffff849ad699>] do_syscall_x64 arch/x86/entry/common.c:50 [inline]
> [<ffffffff849ad699>] do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
> [<ffffffff84a0008b>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
>
> BUG: memory leak
> unreferenced object 0xffff88811225ccc0 (size 192):
> comm "syz-executor325", pid 5099, jiffies 4294978342 (age 119.240s)
> hex dump (first 32 bytes):
> 00 00 00 00 00 00 00 00 22 01 00 00 00 00 ad de ........".......
> 22 0a 80 00 fe 80 00 00 00 00 00 00 00 00 00 00 "...............
> backtrace:
> [<ffffffff8153444a>] __do_kmalloc_node mm/slab_common.c:966 [inline]
> [<ffffffff8153444a>] __kmalloc+0x4a/0x120 mm/slab_common.c:980
> [<ffffffff83d75c15>] kmalloc include/linux/slab.h:584 [inline]
> [<ffffffff83d75c15>] sock_kmalloc net/core/sock.c:2635 [inline]
> [<ffffffff83d75c15>] sock_kmalloc+0x65/0xa0 net/core/sock.c:2624
> [<ffffffff840eb9bb>] __tcp_md5_do_add+0xcb/0x300 net/ipv4/tcp_ipv4.c:1212
> [<ffffffff840eda67>] tcp_md5_do_add+0x67/0x150 net/ipv4/tcp_ipv4.c:1253
> [<ffffffff84262c73>] tcp_v6_parse_md5_keys+0x253/0x4a0 net/ipv6/tcp_ipv6.c:671
> [<ffffffff840c720e>] do_tcp_setsockopt+0x40e/0x1360 net/ipv4/tcp.c:3720
> [<ffffffff840c81fb>] tcp_setsockopt+0x9b/0xa0 net/ipv4/tcp.c:3806
> [<ffffffff83d72a8b>] __sys_setsockopt+0x1ab/0x330 net/socket.c:2274
> [<ffffffff83d72c36>] __do_sys_setsockopt net/socket.c:2285 [inline]
> [<ffffffff83d72c36>] __se_sys_setsockopt net/socket.c:2282 [inline]
> [<ffffffff83d72c36>] __x64_sys_setsockopt+0x26/0x30 net/socket.c:2282
> [<ffffffff849ad699>] do_syscall_x64 arch/x86/entry/common.c:50 [inline]
> [<ffffffff849ad699>] do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
> [<ffffffff84a0008b>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
>
>
>
> ---
> This report is generated by a bot. It may contain errors.
> See https://goo.gl/tpsmEJ for more information about syzbot.
> syzbot engineers can be reached at [email protected].
>
> syzbot will keep track of this issue. See:
> https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
>
> If the bug is already fixed, let syzbot know by replying with:
> #syz fix: exact-commit-title
>
> If you want syzbot to run the reproducer, reply with:
> #syz test: git://repo/address.git branch-or-commit-hash
> If you attach or paste a git patch, syzbot will apply it before testing.
>
> If you want to overwrite bug's subsystems, reply with:
> #syz set subsystems: new-subsystem
> (See the list of subsystem names on the web dashboard)
>
> If the bug is a duplicate of another bug, reply with:
> #syz dup: exact-subject-of-another-report
>
> If you want to undo deduplication, reply with:
> #syz undup
Dmitry, please take a look at this bug, we need to fix it before your
patch series.
Thank you.
On 9/21/23 17:59, Eric Dumazet wrote:
> On Thu, Sep 21, 2023 at 6:56 PM syzbot
> <[email protected]> wrote:
>>
>> Hello,
>>
>> syzbot found the following issue on:
>>
>> HEAD commit: ee3f96b16468 Merge tag 'nfsd-6.3-1' of git://git.kernel.or..
>> git tree: upstream
>> console output: https://syzkaller.appspot.com/x/log.txt?x=1312bba8c80000
>> kernel config: https://syzkaller.appspot.com/x/.config?x=f5733ca1757172ad
>> dashboard link: https://syzkaller.appspot.com/bug?extid=68662811b3d5f6695bcb
>> compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2
>> syz repro: https://syzkaller.appspot.com/x/repro.syz?x=105393a8c80000
>> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=1113917f480000
>>
>> Downloadable assets:
>> disk image: https://storage.googleapis.com/syzbot-assets/29e7966ab711/disk-ee3f96b1.raw.xz
>> vmlinux: https://storage.googleapis.com/syzbot-assets/ae21b8e855de/vmlinux-ee3f96b1.xz
>> kernel image: https://storage.googleapis.com/syzbot-assets/803ee0425ad6/bzImage-ee3f96b1.xz
>>
>> IMPORTANT: if you fix the issue, please add the following tag to the commit:
>> Reported-by: [email protected]
>>
>> executing program
>> BUG: memory leak
>> unreferenced object 0xffff88810a86f7a0 (size 32):
>> comm "syz-executor325", pid 5099, jiffies 4294978342 (age 119.240s)
>> hex dump (first 32 bytes):
>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>> backtrace:
>> [<ffffffff81533d64>] kmalloc_trace+0x24/0x90 mm/slab_common.c:1061
>> [<ffffffff840edaa0>] kmalloc include/linux/slab.h:580 [inline]
>> [<ffffffff840edaa0>] tcp_md5sig_info_add net/ipv4/tcp_ipv4.c:1169 [inline]
>> [<ffffffff840edaa0>] tcp_md5_do_add+0xa0/0x150 net/ipv4/tcp_ipv4.c:1240
>> [<ffffffff84262c73>] tcp_v6_parse_md5_keys+0x253/0x4a0 net/ipv6/tcp_ipv6.c:671
>> [<ffffffff840c720e>] do_tcp_setsockopt+0x40e/0x1360 net/ipv4/tcp.c:3720
>> [<ffffffff840c81fb>] tcp_setsockopt+0x9b/0xa0 net/ipv4/tcp.c:3806
>> [<ffffffff83d72a8b>] __sys_setsockopt+0x1ab/0x330 net/socket.c:2274
>> [<ffffffff83d72c36>] __do_sys_setsockopt net/socket.c:2285 [inline]
>> [<ffffffff83d72c36>] __se_sys_setsockopt net/socket.c:2282 [inline]
>> [<ffffffff83d72c36>] __x64_sys_setsockopt+0x26/0x30 net/socket.c:2282
>> [<ffffffff849ad699>] do_syscall_x64 arch/x86/entry/common.c:50 [inline]
>> [<ffffffff849ad699>] do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
>> [<ffffffff84a0008b>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
>>
>> BUG: memory leak
>> unreferenced object 0xffff88811225ccc0 (size 192):
>> comm "syz-executor325", pid 5099, jiffies 4294978342 (age 119.240s)
>> hex dump (first 32 bytes):
>> 00 00 00 00 00 00 00 00 22 01 00 00 00 00 ad de ........".......
>> 22 0a 80 00 fe 80 00 00 00 00 00 00 00 00 00 00 "...............
>> backtrace:
>> [<ffffffff8153444a>] __do_kmalloc_node mm/slab_common.c:966 [inline]
>> [<ffffffff8153444a>] __kmalloc+0x4a/0x120 mm/slab_common.c:980
>> [<ffffffff83d75c15>] kmalloc include/linux/slab.h:584 [inline]
>> [<ffffffff83d75c15>] sock_kmalloc net/core/sock.c:2635 [inline]
>> [<ffffffff83d75c15>] sock_kmalloc+0x65/0xa0 net/core/sock.c:2624
>> [<ffffffff840eb9bb>] __tcp_md5_do_add+0xcb/0x300 net/ipv4/tcp_ipv4.c:1212
>> [<ffffffff840eda67>] tcp_md5_do_add+0x67/0x150 net/ipv4/tcp_ipv4.c:1253
>> [<ffffffff84262c73>] tcp_v6_parse_md5_keys+0x253/0x4a0 net/ipv6/tcp_ipv6.c:671
>> [<ffffffff840c720e>] do_tcp_setsockopt+0x40e/0x1360 net/ipv4/tcp.c:3720
>> [<ffffffff840c81fb>] tcp_setsockopt+0x9b/0xa0 net/ipv4/tcp.c:3806
>> [<ffffffff83d72a8b>] __sys_setsockopt+0x1ab/0x330 net/socket.c:2274
>> [<ffffffff83d72c36>] __do_sys_setsockopt net/socket.c:2285 [inline]
>> [<ffffffff83d72c36>] __se_sys_setsockopt net/socket.c:2282 [inline]
>> [<ffffffff83d72c36>] __x64_sys_setsockopt+0x26/0x30 net/socket.c:2282
>> [<ffffffff849ad699>] do_syscall_x64 arch/x86/entry/common.c:50 [inline]
>> [<ffffffff849ad699>] do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
>> [<ffffffff84a0008b>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
>>
>>
>>
>> ---
>> This report is generated by a bot. It may contain errors.
>> See https://goo.gl/tpsmEJ for more information about syzbot.
>> syzbot engineers can be reached at [email protected].
>>
>> syzbot will keep track of this issue. See:
>> https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
>>
>> If the bug is already fixed, let syzbot know by replying with:
>> #syz fix: exact-commit-title
>>
>> If you want syzbot to run the reproducer, reply with:
>> #syz test: git://repo/address.git branch-or-commit-hash
>> If you attach or paste a git patch, syzbot will apply it before testing.
>>
>> If you want to overwrite bug's subsystems, reply with:
>> #syz set subsystems: new-subsystem
>> (See the list of subsystem names on the web dashboard)
>>
>> If the bug is a duplicate of another bug, reply with:
>> #syz dup: exact-subject-of-another-report
>>
>> If you want to undo deduplication, reply with:
>> #syz undup
>
> Dmitry, please take a look at this bug, we need to fix it before your
> patch series.
Sure, seems reasonable to me to fix before merging something on top.
> Thank you.
Thanks,
Dmitry
Hi Eric,
On 9/21/23 18:01, Dmitry Safonov wrote:
> On 9/21/23 17:59, Eric Dumazet wrote:
>> On Thu, Sep 21, 2023 at 6:56 PM syzbot
>> <[email protected]> wrote:
>>>
>>> Hello,
>>>
>>> syzbot found the following issue on:
>>>
>>> HEAD commit: ee3f96b16468 Merge tag 'nfsd-6.3-1' of git://git.kernel.or..
>>> git tree: upstream
>>> console output: https://syzkaller.appspot.com/x/log.txt?x=1312bba8c80000
>>> kernel config: https://syzkaller.appspot.com/x/.config?x=f5733ca1757172ad
>>> dashboard link: https://syzkaller.appspot.com/bug?extid=68662811b3d5f6695bcb
>>> compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2
>>> syz repro: https://syzkaller.appspot.com/x/repro.syz?x=105393a8c80000
>>> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=1113917f480000
>>>
>>> Downloadable assets:
>>> disk image: https://storage.googleapis.com/syzbot-assets/29e7966ab711/disk-ee3f96b1.raw.xz
>>> vmlinux: https://storage.googleapis.com/syzbot-assets/ae21b8e855de/vmlinux-ee3f96b1.xz
>>> kernel image: https://storage.googleapis.com/syzbot-assets/803ee0425ad6/bzImage-ee3f96b1.xz
>>>
>>> IMPORTANT: if you fix the issue, please add the following tag to the commit:
>>> Reported-by: [email protected]
>>>
>>> executing program
>>> BUG: memory leak
>>> unreferenced object 0xffff88810a86f7a0 (size 32):
>>> comm "syz-executor325", pid 5099, jiffies 4294978342 (age 119.240s)
>>> hex dump (first 32 bytes):
>>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> backtrace:
>>> [<ffffffff81533d64>] kmalloc_trace+0x24/0x90 mm/slab_common.c:1061
>>> [<ffffffff840edaa0>] kmalloc include/linux/slab.h:580 [inline]
>>> [<ffffffff840edaa0>] tcp_md5sig_info_add net/ipv4/tcp_ipv4.c:1169 [inline]
>>> [<ffffffff840edaa0>] tcp_md5_do_add+0xa0/0x150 net/ipv4/tcp_ipv4.c:1240
>>> [<ffffffff84262c73>] tcp_v6_parse_md5_keys+0x253/0x4a0 net/ipv6/tcp_ipv6.c:671
>>> [<ffffffff840c720e>] do_tcp_setsockopt+0x40e/0x1360 net/ipv4/tcp.c:3720
>>> [<ffffffff840c81fb>] tcp_setsockopt+0x9b/0xa0 net/ipv4/tcp.c:3806
>>> [<ffffffff83d72a8b>] __sys_setsockopt+0x1ab/0x330 net/socket.c:2274
>>> [<ffffffff83d72c36>] __do_sys_setsockopt net/socket.c:2285 [inline]
>>> [<ffffffff83d72c36>] __se_sys_setsockopt net/socket.c:2282 [inline]
>>> [<ffffffff83d72c36>] __x64_sys_setsockopt+0x26/0x30 net/socket.c:2282
>>> [<ffffffff849ad699>] do_syscall_x64 arch/x86/entry/common.c:50 [inline]
>>> [<ffffffff849ad699>] do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
>>> [<ffffffff84a0008b>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
>>>
>>> BUG: memory leak
>>> unreferenced object 0xffff88811225ccc0 (size 192):
>>> comm "syz-executor325", pid 5099, jiffies 4294978342 (age 119.240s)
>>> hex dump (first 32 bytes):
>>> 00 00 00 00 00 00 00 00 22 01 00 00 00 00 ad de ........".......
>>> 22 0a 80 00 fe 80 00 00 00 00 00 00 00 00 00 00 "...............
>>> backtrace:
>>> [<ffffffff8153444a>] __do_kmalloc_node mm/slab_common.c:966 [inline]
>>> [<ffffffff8153444a>] __kmalloc+0x4a/0x120 mm/slab_common.c:980
>>> [<ffffffff83d75c15>] kmalloc include/linux/slab.h:584 [inline]
>>> [<ffffffff83d75c15>] sock_kmalloc net/core/sock.c:2635 [inline]
>>> [<ffffffff83d75c15>] sock_kmalloc+0x65/0xa0 net/core/sock.c:2624
>>> [<ffffffff840eb9bb>] __tcp_md5_do_add+0xcb/0x300 net/ipv4/tcp_ipv4.c:1212
>>> [<ffffffff840eda67>] tcp_md5_do_add+0x67/0x150 net/ipv4/tcp_ipv4.c:1253
>>> [<ffffffff84262c73>] tcp_v6_parse_md5_keys+0x253/0x4a0 net/ipv6/tcp_ipv6.c:671
>>> [<ffffffff840c720e>] do_tcp_setsockopt+0x40e/0x1360 net/ipv4/tcp.c:3720
>>> [<ffffffff840c81fb>] tcp_setsockopt+0x9b/0xa0 net/ipv4/tcp.c:3806
>>> [<ffffffff83d72a8b>] __sys_setsockopt+0x1ab/0x330 net/socket.c:2274
>>> [<ffffffff83d72c36>] __do_sys_setsockopt net/socket.c:2285 [inline]
>>> [<ffffffff83d72c36>] __se_sys_setsockopt net/socket.c:2282 [inline]
>>> [<ffffffff83d72c36>] __x64_sys_setsockopt+0x26/0x30 net/socket.c:2282
>>> [<ffffffff849ad699>] do_syscall_x64 arch/x86/entry/common.c:50 [inline]
>>> [<ffffffff849ad699>] do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
>>> [<ffffffff84a0008b>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
>>>
>>>
>>>
>>> ---
>>> This report is generated by a bot. It may contain errors.
>>> See https://goo.gl/tpsmEJ for more information about syzbot.
>>> syzbot engineers can be reached at [email protected].
>>>
>>> syzbot will keep track of this issue. See:
>>> https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
>>>
>>> If the bug is already fixed, let syzbot know by replying with:
>>> #syz fix: exact-commit-title
>>>
>>> If you want syzbot to run the reproducer, reply with:
>>> #syz test: git://repo/address.git branch-or-commit-hash
>>> If you attach or paste a git patch, syzbot will apply it before testing.
>>>
>>> If you want to overwrite bug's subsystems, reply with:
>>> #syz set subsystems: new-subsystem
>>> (See the list of subsystem names on the web dashboard)
>>>
>>> If the bug is a duplicate of another bug, reply with:
>>> #syz dup: exact-subject-of-another-report
>>>
>>> If you want to undo deduplication, reply with:
>>> #syz undup
>>
>> Dmitry, please take a look at this bug, we need to fix it before your
>> patch series.
>
> Sure, seems reasonable to me to fix before merging something on top.
It seems to me that it's related to a race between RCU grace period and
kmemleak scan period. There seems to be a patch [1] that likely fixes
that, albeit I couldn't verify it as all my attempts to reproduce syzbot
issue produced only unrelated to TCP-MD5 log:
> [ 263.201211] kmemleak: unreferenced object 0xffff9ceb047d9948 (size 192):
> [ 263.201781] kmemleak: comm "ip", pid 730, jiffies 4294937874 (age 257.270s)
> [ 263.202460] kmemleak: hex dump (first 32 bytes):
> [ 263.202921] kmemleak: 00 c8 e9 01 eb 9c ff ff e0 00 00 01 00 00 00 00 ................
> [ 263.203700] kmemleak: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> [ 263.204484] kmemleak: backtrace:
> [ 263.204814] kmemleak: [<ffffffff830a2946>] kmalloc_trace+0x26/0x90
> [ 263.205440] kmemleak: [<ffffffff837e8310>] ____ip_mc_inc_group+0xa0/0x240
> [ 263.206134] kmemleak: [<ffffffff837e9a9b>] ip_mc_up+0x4b/0xb0
> [ 263.206725] kmemleak: [<ffffffff837e28fb>] inetdev_event+0xbb/0x5c0
> [ 263.207358] kmemleak: [<ffffffff82f3caf6>] notifier_call_chain+0x56/0xc0
> [ 263.208070] kmemleak: [<ffffffff836f1818>] __dev_notify_flags+0x58/0xf0
> [ 263.208784] kmemleak: [<ffffffff836f2210>] dev_change_flags+0x50/0x60
> [ 263.209471] kmemleak: [<ffffffff837e1718>] devinet_ioctl+0x378/0x770
> [ 263.210152] kmemleak: [<ffffffff837e34a7>] inet_ioctl+0x187/0x1d0
> [ 263.210805] kmemleak: [<ffffffff836c40ed>] sock_do_ioctl+0x3d/0x100
> [ 263.211482] kmemleak: [<ffffffff836c4293>] sock_ioctl+0xe3/0x2b0
> [ 263.212131] kmemleak: [<ffffffff8313cbec>] __x64_sys_ioctl+0x8c/0xc0
> [ 263.212789] kmemleak: [<ffffffff83a2ad75>] do_syscall_64+0x35/0x80
> [ 263.213438] kmemleak: [<ffffffff83c0006a>] entry_SYSCALL_64_after_hwframe+0x46/0xb0
> [ 263.214283] kmemleak: unreferenced object 0xffff9ceb03ad5400 (size 512):
> [ 263.214982] kmemleak: comm "ip", pid 730, jiffies 4294937874 (age 257.290s)
> [ 263.215728] kmemleak: hex dump (first 32 bytes):
> [ 263.216231] kmemleak: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 ................
> [ 263.217106] kmemleak: 80 00 00 00 00 00 00 00 ff ff ff ff ff ff ff ff ................
> [ 263.218041] kmemleak: backtrace:
> [ 263.218438] kmemleak: [<ffffffff830a2946>] kmalloc_trace+0x26/0x90
> [ 263.219181] kmemleak: [<ffffffff8384b90b>] ipv6_add_addr+0x13b/0x6c0
> [ 263.219931] kmemleak: [<ffffffff8384d4b5>] add_addr+0x75/0x150
> [ 263.220627] kmemleak: [<ffffffff8385357d>] addrconf_notify+0x53d/0x730
> [ 263.221377] kmemleak: [<ffffffff82f3caf6>] notifier_call_chain+0x56/0xc0
> [ 263.222104] kmemleak: [<ffffffff836f1818>] __dev_notify_flags+0x58/0xf0
> [ 263.222844] kmemleak: [<ffffffff836f2210>] dev_change_flags+0x50/0x60
> [ 263.223581] kmemleak: [<ffffffff837e1718>] devinet_ioctl+0x378/0x770
> [ 263.224293] kmemleak: [<ffffffff837e34a7>] inet_ioctl+0x187/0x1d0
> [ 263.224961] kmemleak: [<ffffffff836c40ed>] sock_do_ioctl+0x3d/0x100
> [ 263.225660] kmemleak: [<ffffffff836c4293>] sock_ioctl+0xe3/0x2b0
> [ 263.226331] kmemleak: [<ffffffff8313cbec>] __x64_sys_ioctl+0x8c/0xc0
> [ 263.227039] kmemleak: [<ffffffff83a2ad75>] do_syscall_64+0x35/0x80
> [ 263.227747] kmemleak: [<ffffffff83c0006a>] entry_SYSCALL_64_after_hwframe+0x46/0xb0
> [ 263.228708] kmemleak: 2 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
This seems to be quite the same issue: inet6_ifa_finish_destroy()
destroys inet6_ifaddr with kfree_rcu().
[1]
https://lore.kernel.org/linux-mm/[email protected]/T/#ma4a68fdc44793e2594c9e7cadefa8ea40da5807d
Thanks,
Dmitry
On Fri, Sep 22, 2023 at 1:15 AM Dmitry Safonov <[email protected]> wrote:
>
> Hi Eric,
>
> On 9/21/23 18:01, Dmitry Safonov wrote:
> > On 9/21/23 17:59, Eric Dumazet wrote:
> >> On Thu, Sep 21, 2023 at 6:56 PM syzbot
> >> <[email protected]> wrote:
> >>>
> >>> Hello,
> >>>
> >>> syzbot found the following issue on:
> >>>
> >>> HEAD commit: ee3f96b16468 Merge tag 'nfsd-6.3-1' of git://git.kernel.or..
> >>> git tree: upstream
> >>> console output: https://syzkaller.appspot.com/x/log.txt?x=1312bba8c80000
> >>> kernel config: https://syzkaller.appspot.com/x/.config?x=f5733ca1757172ad
> >>> dashboard link: https://syzkaller.appspot.com/bug?extid=68662811b3d5f6695bcb
> >>> compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2
> >>> syz repro: https://syzkaller.appspot.com/x/repro.syz?x=105393a8c80000
> >>> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=1113917f480000
> >>>
> >>> Downloadable assets:
> >>> disk image: https://storage.googleapis.com/syzbot-assets/29e7966ab711/disk-ee3f96b1.raw.xz
> >>> vmlinux: https://storage.googleapis.com/syzbot-assets/ae21b8e855de/vmlinux-ee3f96b1.xz
> >>> kernel image: https://storage.googleapis.com/syzbot-assets/803ee0425ad6/bzImage-ee3f96b1.xz
> >>>
> >>> IMPORTANT: if you fix the issue, please add the following tag to the commit:
> >>> Reported-by: [email protected]
> >>>
> >>> executing program
> >>> BUG: memory leak
> >>> unreferenced object 0xffff88810a86f7a0 (size 32):
> >>> comm "syz-executor325", pid 5099, jiffies 4294978342 (age 119.240s)
> >>> hex dump (first 32 bytes):
> >>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> >>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> >>> backtrace:
> >>> [<ffffffff81533d64>] kmalloc_trace+0x24/0x90 mm/slab_common.c:1061
> >>> [<ffffffff840edaa0>] kmalloc include/linux/slab.h:580 [inline]
> >>> [<ffffffff840edaa0>] tcp_md5sig_info_add net/ipv4/tcp_ipv4.c:1169 [inline]
> >>> [<ffffffff840edaa0>] tcp_md5_do_add+0xa0/0x150 net/ipv4/tcp_ipv4.c:1240
> >>> [<ffffffff84262c73>] tcp_v6_parse_md5_keys+0x253/0x4a0 net/ipv6/tcp_ipv6.c:671
> >>> [<ffffffff840c720e>] do_tcp_setsockopt+0x40e/0x1360 net/ipv4/tcp.c:3720
> >>> [<ffffffff840c81fb>] tcp_setsockopt+0x9b/0xa0 net/ipv4/tcp.c:3806
> >>> [<ffffffff83d72a8b>] __sys_setsockopt+0x1ab/0x330 net/socket.c:2274
> >>> [<ffffffff83d72c36>] __do_sys_setsockopt net/socket.c:2285 [inline]
> >>> [<ffffffff83d72c36>] __se_sys_setsockopt net/socket.c:2282 [inline]
> >>> [<ffffffff83d72c36>] __x64_sys_setsockopt+0x26/0x30 net/socket.c:2282
> >>> [<ffffffff849ad699>] do_syscall_x64 arch/x86/entry/common.c:50 [inline]
> >>> [<ffffffff849ad699>] do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
> >>> [<ffffffff84a0008b>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
> >>>
> >>> BUG: memory leak
> >>> unreferenced object 0xffff88811225ccc0 (size 192):
> >>> comm "syz-executor325", pid 5099, jiffies 4294978342 (age 119.240s)
> >>> hex dump (first 32 bytes):
> >>> 00 00 00 00 00 00 00 00 22 01 00 00 00 00 ad de ........".......
> >>> 22 0a 80 00 fe 80 00 00 00 00 00 00 00 00 00 00 "...............
> >>> backtrace:
> >>> [<ffffffff8153444a>] __do_kmalloc_node mm/slab_common.c:966 [inline]
> >>> [<ffffffff8153444a>] __kmalloc+0x4a/0x120 mm/slab_common.c:980
> >>> [<ffffffff83d75c15>] kmalloc include/linux/slab.h:584 [inline]
> >>> [<ffffffff83d75c15>] sock_kmalloc net/core/sock.c:2635 [inline]
> >>> [<ffffffff83d75c15>] sock_kmalloc+0x65/0xa0 net/core/sock.c:2624
> >>> [<ffffffff840eb9bb>] __tcp_md5_do_add+0xcb/0x300 net/ipv4/tcp_ipv4.c:1212
> >>> [<ffffffff840eda67>] tcp_md5_do_add+0x67/0x150 net/ipv4/tcp_ipv4.c:1253
> >>> [<ffffffff84262c73>] tcp_v6_parse_md5_keys+0x253/0x4a0 net/ipv6/tcp_ipv6.c:671
> >>> [<ffffffff840c720e>] do_tcp_setsockopt+0x40e/0x1360 net/ipv4/tcp.c:3720
> >>> [<ffffffff840c81fb>] tcp_setsockopt+0x9b/0xa0 net/ipv4/tcp.c:3806
> >>> [<ffffffff83d72a8b>] __sys_setsockopt+0x1ab/0x330 net/socket.c:2274
> >>> [<ffffffff83d72c36>] __do_sys_setsockopt net/socket.c:2285 [inline]
> >>> [<ffffffff83d72c36>] __se_sys_setsockopt net/socket.c:2282 [inline]
> >>> [<ffffffff83d72c36>] __x64_sys_setsockopt+0x26/0x30 net/socket.c:2282
> >>> [<ffffffff849ad699>] do_syscall_x64 arch/x86/entry/common.c:50 [inline]
> >>> [<ffffffff849ad699>] do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
> >>> [<ffffffff84a0008b>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
> >>>
> >>>
> >>>
> >>> ---
> >>> This report is generated by a bot. It may contain errors.
> >>> See https://goo.gl/tpsmEJ for more information about syzbot.
> >>> syzbot engineers can be reached at [email protected].
> >>>
> >>> syzbot will keep track of this issue. See:
> >>> https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
> >>>
> >>> If the bug is already fixed, let syzbot know by replying with:
> >>> #syz fix: exact-commit-title
> >>>
> >>> If you want syzbot to run the reproducer, reply with:
> >>> #syz test: git://repo/address.git branch-or-commit-hash
> >>> If you attach or paste a git patch, syzbot will apply it before testing.
> >>>
> >>> If you want to overwrite bug's subsystems, reply with:
> >>> #syz set subsystems: new-subsystem
> >>> (See the list of subsystem names on the web dashboard)
> >>>
> >>> If the bug is a duplicate of another bug, reply with:
> >>> #syz dup: exact-subject-of-another-report
> >>>
> >>> If you want to undo deduplication, reply with:
> >>> #syz undup
> >>
> >> Dmitry, please take a look at this bug, we need to fix it before your
> >> patch series.
> >
> > Sure, seems reasonable to me to fix before merging something on top.
>
> It seems to me that it's related to a race between RCU grace period and
> kmemleak scan period. There seems to be a patch [1] that likely fixes
> that, albeit I couldn't verify it as all my attempts to reproduce syzbot
> issue produced only unrelated to TCP-MD5 log:
>
I doubt this, looking at the repro, which seems to abuse a not often
used feature of TCP (self connect)
# https://syzkaller.appspot.com/bug?id=323165b5fe193114de7a3a6a8bd16cf3a3c36ecf
# See https://goo.gl/kgGztJ for information about syzkaller reproducers.
#{"repeat":true,"procs":1,"slowdown":1,"sandbox":"none","sandbox_arg":0,"leak":true,"netdev":true,"close_fds":true,"usb":true}
r0 = socket$inet6_tcp(0xa, 0x1, 0x0)
setsockopt$inet6_tcp_TCP_MD5SIG(r0, 0x6, 0xe,
&(0x7f0000000040)={@in6={{0xa, 0x0, 0x0, @local}}, 0x0, 0x0, 0x22,
0x0, "b05423587c18814d6b1a5f25671d09815a4687d637ffc958defc671aad3d4de8ac7d88560c759d600ab650c07ef0ef162b199da0d017fe6f0ae40cfb4e241cf9a990f20f6b8c2c070a61cfad8a2d2600"},
0xd8)
connect$inet6(r0, &(0x7f0000000180)={0xa, 0x4001, 0x0, @ipv4={'\x00',
'\xff\xff', @remote}}, 0x1c)
dup(0xffffffffffffffff)
setsockopt$SO_BINDTODEVICE(r0, 0x1, 0x19,
&(0x7f00000001c0)='ip6_vti0\x00', 0xff4a)
You could not have KMEMLEAK in the kernel, and run the repro a thousand times.
Then compare /proc/slabinfo before/after.
From: Eric Dumazet <[email protected]>
Date: Fri, 22 Sep 2023 05:20:34 +0200
> On Fri, Sep 22, 2023 at 1:15 AM Dmitry Safonov <[email protected]> wrote:
> >
> > Hi Eric,
> >
> > On 9/21/23 18:01, Dmitry Safonov wrote:
> > > On 9/21/23 17:59, Eric Dumazet wrote:
> > >> On Thu, Sep 21, 2023 at 6:56 PM syzbot
> > >> <[email protected]> wrote:
> > >>>
> > >>> Hello,
> > >>>
> > >>> syzbot found the following issue on:
> > >>>
> > >>> HEAD commit: ee3f96b16468 Merge tag 'nfsd-6.3-1' of git://git.kernel.or..
> > >>> git tree: upstream
> > >>> console output: https://syzkaller.appspot.com/x/log.txt?x=1312bba8c80000
> > >>> kernel config: https://syzkaller.appspot.com/x/.config?x=f5733ca1757172ad
> > >>> dashboard link: https://syzkaller.appspot.com/bug?extid=68662811b3d5f6695bcb
> > >>> compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2
> > >>> syz repro: https://syzkaller.appspot.com/x/repro.syz?x=105393a8c80000
> > >>> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=1113917f480000
> > >>>
> > >>> Downloadable assets:
> > >>> disk image: https://storage.googleapis.com/syzbot-assets/29e7966ab711/disk-ee3f96b1.raw.xz
> > >>> vmlinux: https://storage.googleapis.com/syzbot-assets/ae21b8e855de/vmlinux-ee3f96b1.xz
> > >>> kernel image: https://storage.googleapis.com/syzbot-assets/803ee0425ad6/bzImage-ee3f96b1.xz
> > >>>
> > >>> IMPORTANT: if you fix the issue, please add the following tag to the commit:
> > >>> Reported-by: [email protected]
> > >>>
> > >>> executing program
> > >>> BUG: memory leak
> > >>> unreferenced object 0xffff88810a86f7a0 (size 32):
> > >>> comm "syz-executor325", pid 5099, jiffies 4294978342 (age 119.240s)
> > >>> hex dump (first 32 bytes):
> > >>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > >>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > >>> backtrace:
> > >>> [<ffffffff81533d64>] kmalloc_trace+0x24/0x90 mm/slab_common.c:1061
> > >>> [<ffffffff840edaa0>] kmalloc include/linux/slab.h:580 [inline]
> > >>> [<ffffffff840edaa0>] tcp_md5sig_info_add net/ipv4/tcp_ipv4.c:1169 [inline]
> > >>> [<ffffffff840edaa0>] tcp_md5_do_add+0xa0/0x150 net/ipv4/tcp_ipv4.c:1240
> > >>> [<ffffffff84262c73>] tcp_v6_parse_md5_keys+0x253/0x4a0 net/ipv6/tcp_ipv6.c:671
> > >>> [<ffffffff840c720e>] do_tcp_setsockopt+0x40e/0x1360 net/ipv4/tcp.c:3720
> > >>> [<ffffffff840c81fb>] tcp_setsockopt+0x9b/0xa0 net/ipv4/tcp.c:3806
> > >>> [<ffffffff83d72a8b>] __sys_setsockopt+0x1ab/0x330 net/socket.c:2274
> > >>> [<ffffffff83d72c36>] __do_sys_setsockopt net/socket.c:2285 [inline]
> > >>> [<ffffffff83d72c36>] __se_sys_setsockopt net/socket.c:2282 [inline]
> > >>> [<ffffffff83d72c36>] __x64_sys_setsockopt+0x26/0x30 net/socket.c:2282
> > >>> [<ffffffff849ad699>] do_syscall_x64 arch/x86/entry/common.c:50 [inline]
> > >>> [<ffffffff849ad699>] do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
> > >>> [<ffffffff84a0008b>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
> > >>>
> > >>> BUG: memory leak
> > >>> unreferenced object 0xffff88811225ccc0 (size 192):
> > >>> comm "syz-executor325", pid 5099, jiffies 4294978342 (age 119.240s)
> > >>> hex dump (first 32 bytes):
> > >>> 00 00 00 00 00 00 00 00 22 01 00 00 00 00 ad de ........".......
> > >>> 22 0a 80 00 fe 80 00 00 00 00 00 00 00 00 00 00 "...............
> > >>> backtrace:
> > >>> [<ffffffff8153444a>] __do_kmalloc_node mm/slab_common.c:966 [inline]
> > >>> [<ffffffff8153444a>] __kmalloc+0x4a/0x120 mm/slab_common.c:980
> > >>> [<ffffffff83d75c15>] kmalloc include/linux/slab.h:584 [inline]
> > >>> [<ffffffff83d75c15>] sock_kmalloc net/core/sock.c:2635 [inline]
> > >>> [<ffffffff83d75c15>] sock_kmalloc+0x65/0xa0 net/core/sock.c:2624
> > >>> [<ffffffff840eb9bb>] __tcp_md5_do_add+0xcb/0x300 net/ipv4/tcp_ipv4.c:1212
> > >>> [<ffffffff840eda67>] tcp_md5_do_add+0x67/0x150 net/ipv4/tcp_ipv4.c:1253
> > >>> [<ffffffff84262c73>] tcp_v6_parse_md5_keys+0x253/0x4a0 net/ipv6/tcp_ipv6.c:671
> > >>> [<ffffffff840c720e>] do_tcp_setsockopt+0x40e/0x1360 net/ipv4/tcp.c:3720
> > >>> [<ffffffff840c81fb>] tcp_setsockopt+0x9b/0xa0 net/ipv4/tcp.c:3806
> > >>> [<ffffffff83d72a8b>] __sys_setsockopt+0x1ab/0x330 net/socket.c:2274
> > >>> [<ffffffff83d72c36>] __do_sys_setsockopt net/socket.c:2285 [inline]
> > >>> [<ffffffff83d72c36>] __se_sys_setsockopt net/socket.c:2282 [inline]
> > >>> [<ffffffff83d72c36>] __x64_sys_setsockopt+0x26/0x30 net/socket.c:2282
> > >>> [<ffffffff849ad699>] do_syscall_x64 arch/x86/entry/common.c:50 [inline]
> > >>> [<ffffffff849ad699>] do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
> > >>> [<ffffffff84a0008b>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
> > >>>
> > >>>
> > >>>
> > >>> ---
> > >>> This report is generated by a bot. It may contain errors.
> > >>> See https://goo.gl/tpsmEJ for more information about syzbot.
> > >>> syzbot engineers can be reached at [email protected].
> > >>>
> > >>> syzbot will keep track of this issue. See:
> > >>> https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
> > >>>
> > >>> If the bug is already fixed, let syzbot know by replying with:
> > >>> #syz fix: exact-commit-title
> > >>>
> > >>> If you want syzbot to run the reproducer, reply with:
> > >>> #syz test: git://repo/address.git branch-or-commit-hash
> > >>> If you attach or paste a git patch, syzbot will apply it before testing.
> > >>>
> > >>> If you want to overwrite bug's subsystems, reply with:
> > >>> #syz set subsystems: new-subsystem
> > >>> (See the list of subsystem names on the web dashboard)
> > >>>
> > >>> If the bug is a duplicate of another bug, reply with:
> > >>> #syz dup: exact-subject-of-another-report
> > >>>
> > >>> If you want to undo deduplication, reply with:
> > >>> #syz undup
> > >>
> > >> Dmitry, please take a look at this bug, we need to fix it before your
> > >> patch series.
> > >
> > > Sure, seems reasonable to me to fix before merging something on top.
> >
> > It seems to me that it's related to a race between RCU grace period and
> > kmemleak scan period. There seems to be a patch [1] that likely fixes
> > that, albeit I couldn't verify it as all my attempts to reproduce syzbot
> > issue produced only unrelated to TCP-MD5 log:
> >
>
> I doubt this, looking at the repro, which seems to abuse a not often
> used feature of TCP (self connect)
>
> # https://syzkaller.appspot.com/bug?id=323165b5fe193114de7a3a6a8bd16cf3a3c36ecf
> # See https://goo.gl/kgGztJ for information about syzkaller reproducers.
> #{"repeat":true,"procs":1,"slowdown":1,"sandbox":"none","sandbox_arg":0,"leak":true,"netdev":true,"close_fds":true,"usb":true}
> r0 = socket$inet6_tcp(0xa, 0x1, 0x0)
> setsockopt$inet6_tcp_TCP_MD5SIG(r0, 0x6, 0xe,
> &(0x7f0000000040)={@in6={{0xa, 0x0, 0x0, @local}}, 0x0, 0x0, 0x22,
> 0x0, "b05423587c18814d6b1a5f25671d09815a4687d637ffc958defc671aad3d4de8ac7d88560c759d600ab650c07ef0ef162b199da0d017fe6f0ae40cfb4e241cf9a990f20f6b8c2c070a61cfad8a2d2600"},
> 0xd8)
> connect$inet6(r0, &(0x7f0000000180)={0xa, 0x4001, 0x0, @ipv4={'\x00',
> '\xff\xff', @remote}}, 0x1c)
> dup(0xffffffffffffffff)
> setsockopt$SO_BINDTODEVICE(r0, 0x1, 0x19,
> &(0x7f00000001c0)='ip6_vti0\x00', 0xff4a)
FWIW, I had the same report and another report for twsk and MD5.
syzkaller did not find repro though.
---8<---
BUG: memory leak
unreferenced object 0xffff888038513480 (size 192):
comm "syz-executor.0", pid 36537, jiffies 4295853096 (age 63.376s)
hex dump (first 32 bytes):
00 00 00 00 00 00 00 00 00 3e fc 43 80 88 ff ff .........>.C....
06 02 20 00 ac 14 14 aa 00 00 00 00 00 00 00 00 .. .............
backtrace:
[<0000000003e890c3>] __do_kmalloc_node mm/slab_common.c:984 [inline]
[<0000000003e890c3>] __kmalloc_node_track_caller+0x4b/0x130 mm/slab_common.c:1005
[<0000000026777435>] kmemdup+0x2c/0x60 mm/util.c:131
[<000000000318308e>] kmemdup include/linux/fortify-string.h:765 [inline]
[<000000000318308e>] tcp_time_wait_init net/ipv4/tcp_minisocks.c:261 [inline]
[<000000000318308e>] tcp_time_wait+0x25c/0x3b0 net/ipv4/tcp_minisocks.c:318
[<00000000bb86ba54>] tcp_rcv_state_process+0xb36/0x1990 net/ipv4/tcp_input.c:6668
[<00000000a26563d5>] tcp_v4_do_rcv+0x18b/0x4a0 net/ipv4/tcp_ipv4.c:1751
[<00000000b158e1f0>] sk_backlog_rcv include/net/sock.h:1115 [inline]
[<00000000b158e1f0>] __release_sock+0x177/0x1a0 net/core/sock.c:2982
[<000000000e8687d8>] __tcp_close+0x252/0x630 net/ipv4/tcp.c:2846
[<000000006b8a2f7d>] tcp_close+0x2d/0xc0 net/ipv4/tcp.c:2922
[<00000000d4c1915c>] inet_release+0x82/0xf0 net/ipv4/af_inet.c:433
[<00000000590c8ed6>] __sock_release+0x4b/0xf0 net/socket.c:657
[<00000000d49971a8>] sock_close+0x19/0x30 net/socket.c:1399
[<0000000097cacf4d>] __fput+0x1d0/0x4b0 fs/file_table.c:384
[<000000006a98802f>] __fput_sync+0x37/0x40 fs/file_table.c:465
[<00000000a6ebd3a7>] __do_sys_close fs/open.c:1572 [inline]
[<00000000a6ebd3a7>] __se_sys_close fs/open.c:1557 [inline]
[<00000000a6ebd3a7>] __x64_sys_close+0x4a/0xc0 fs/open.c:1557
[<000000004060032b>] do_syscall_x64 arch/x86/entry/common.c:50 [inline]
[<000000004060032b>] do_syscall_64+0x3c/0x90 arch/x86/entry/common.c:80
[<00000000e8d61c9b>] entry_SYSCALL_64_after_hwframe+0x6e/0xd8
---8<---
In my syzkaller log, only this program had the MD5 operation.
I ran this overnight but had no luck for now.
---8<---
23:51:30 executing program 0:
r0 = socket$inet(0x2, 0x4000000000000001, 0x0)
setsockopt$inet_tcp_TCP_MD5SIG(r0, 0x6, 0xe, &(0x7f0000000780)={@in={{0x2, 0x0, @local}}, 0x0, 0x9, 0x6, 0x0, "0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000030cf00"}, 0xd8) (async)
bind$inet(r0, &(0x7f0000deb000)={0x2, 0x4e23, @multicast1}, 0x10) (async, rerun: 64)
sendto$inet(r0, 0x0, 0x0, 0x200007b9, &(0x7f0000000040)={0x2, 0x4e23, @local}, 0x10) (async, rerun: 64)
socket$inet6(0xa, 0x0, 0x0) (async)
getsockopt$EBT_SO_GET_INIT_ENTRIES(0xffffffffffffffff, 0x0, 0x83, &(0x7f0000000080)={'filter\x00', 0x0, 0x4, 0x1000, [0x0, 0x8, 0x1, 0x1, 0x0, 0x7fffffff], 0x4, &(0x7f0000000000)=[{}, {}, {}, {}], &(0x7f0000000880)=""/4096}, 0x0) (async, rerun: 32)
socket(0x0, 0x0, 0x0) (async, rerun: 32)
r1 = openat2(0xffffffffffffffff, &(0x7f0000000100)='./file0\x00', &(0x7f0000000140)={0x20000, 0x8, 0x14}, 0x18)
bind$inet(r1, &(0x7f0000000180)={0x2, 0x4e22, @remote}, 0x10) (async)
sendmsg$nl_route(0xffffffffffffffff, 0x0, 0x0)
---8<---
>
> You could not have KMEMLEAK in the kernel, and run the repro a thousand times.
>
> Then compare /proc/slabinfo before/after.
On 9/22/23 04:20, Eric Dumazet wrote:
> On Fri, Sep 22, 2023 at 1:15 AM Dmitry Safonov <[email protected]> wrote:
>>
>> Hi Eric,
>>
>> On 9/21/23 18:01, Dmitry Safonov wrote:
>>> On 9/21/23 17:59, Eric Dumazet wrote:
[..]
>>>> Dmitry, please take a look at this bug, we need to fix it before your
>>>> patch series.
>>>
>>> Sure, seems reasonable to me to fix before merging something on top.
>>
>> It seems to me that it's related to a race between RCU grace period and
>> kmemleak scan period. There seems to be a patch [1] that likely fixes
>> that, albeit I couldn't verify it as all my attempts to reproduce syzbot
>> issue produced only unrelated to TCP-MD5 log:
>>
>
> I doubt this, looking at the repro, which seems to abuse a not often
> used feature of TCP (self connect)
>
> # https://syzkaller.appspot.com/bug?id=323165b5fe193114de7a3a6a8bd16cf3a3c36ecf
> # See https://goo.gl/kgGztJ for information about syzkaller reproducers.
> #{"repeat":true,"procs":1,"slowdown":1,"sandbox":"none","sandbox_arg":0,"leak":true,"netdev":true,"close_fds":true,"usb":true}
> r0 = socket$inet6_tcp(0xa, 0x1, 0x0)
> setsockopt$inet6_tcp_TCP_MD5SIG(r0, 0x6, 0xe,
> &(0x7f0000000040)={@in6={{0xa, 0x0, 0x0, @local}}, 0x0, 0x0, 0x22,
> 0x0, "b05423587c18814d6b1a5f25671d09815a4687d637ffc958defc671aad3d4de8ac7d88560c759d600ab650c07ef0ef162b199da0d017fe6f0ae40cfb4e241cf9a990f20f6b8c2c070a61cfad8a2d2600"},
> 0xd8)
> connect$inet6(r0, &(0x7f0000000180)={0xa, 0x4001, 0x0, @ipv4={'\x00',
> '\xff\xff', @remote}}, 0x1c)
> dup(0xffffffffffffffff)
> setsockopt$SO_BINDTODEVICE(r0, 0x1, 0x19,
> &(0x7f00000001c0)='ip6_vti0\x00', 0xff4a)
>
>
>
> You could not have KMEMLEAK in the kernel, and run the repro a thousand times.
>
> Then compare /proc/slabinfo before/after.
Eric!
After some experiments, I'm still standing on the same RCU/kmemleak
race. Here's what I did:
1. I couldn't reproduce it on locally-build kernel with the same .config
in order to verify that the patch does fix this. Probably the toolchain
or any other small bits make a difference.
2. I can easily reproduce it with bzImage from syzcaller.
3. slabinfo does fluctuate for kmalloc-192 even without running any
reproducer, so it'd be hard to tell the difference.
4. I went running the reproducer in loop a hundred times:
# for i in $(seq 1 100) ; do ./repro ; dmesg -c >> ./dmesg.log ; done
the dmesg clean here is because pre-compiled ring-buffer is quite small
(I attach the full logs to this mail)
5. `cat /sys/kernel/debug/kmemleak` is empty, I presume it means that
the addresses that were reported later got removed from kmemleak
lists/tables on kfree().
6. Curious about addresses reported, I can see that some addresses were
reported multiple times, which means that slab got reused, rather than
leaked:
# grep tcp_md5_do_add -B9 dmesg.log | sed -n 's/.*unreferenced object
\([^ ]\+\) .*/\1/p' | sort | uniq -c | sort -n | tail -n 5
2 0xffff8880174b96c0
2 0xffff888017686900
2 0xffff8880179fd720
3 0xffff8880146a8600
3 0xffff888015c226c0
See the logs for 0xffff888015c226c0 address (you can view them in dmesg
that I attach):
https://gist.github.com/0x7f454c46/dcc7936392a51a789a235eb73df1598c
7. Well, OK, maybe at least one of the addresses reported was leaked?
In order to check that, I did:
# cat dmesg.log | grep tcp_md5_do_add -B9 | sed -n 's/.*unreferenced
object \([^ ]\+\) .*/\1/p' | sort | uniq > addresses
# dmesg -c ; for i in $(cat addresses) ; do echo "dump=$i" >
/sys/kernel/debug/kmemleak ; dmesg -c >> addresses.kmemleak ; done
I attach addresses.kmemleak: the slabs were reused or weren't allocated
(I presume, the "Unknown object at 0xffff888019cdcdc0" means that slab
is free).
8. Now that I verified that kmemleak was misreporting those addresses, I
went on my regular TCP-AO selftests, added one for TCP self-connect and
with ftrace I can clearly see that md5 keys/info is deallocated:
# cat trace
# tracer: function
#
# entries-in-buffer/entries-written: 5/5 #P:2
#
# _-----=> irqs-off/BH-disabled
# / _----=> need-resched
# | / _---=> hardirq/softirq
# || / _--=> preempt-depth
# ||| / _-=> migrate-disable
# |||| / delay
# TASK-PID CPU# ||||| TIMESTAMP FUNCTION
# | | | ||||| | |
self-connect_ip-2125 [000] ...1. 6108.468401: tcp_md5_do_add
<-tcp_v4_parse_md5_keys
self-connect_ip-2125 [000] ...1. 6108.468727: __tcp_md5_do_add
<-tcp_md5_do_add
self-connect_ip-2125 [000] ...1. 6108.471276: tcp_clear_md5_list
<-tcp_v4_destroy_sock
kworker/u5:1-2108 [000] ..s1. 6108.475633:
tcp_md5sig_info_free_rcu <-rcu_core
<idle>-0 [000] ..s2. 6108.598342: tcp_md5_twsk_free_rcu
<-rcu_core
So, from source code point of view: the test opens socket, sends SYN,
receives SYN straight away and kernel decides that it's
simultaneous/fast open case. It sends SYN-ACK and establishes as normal.
On socket destruction, TCP-MD5 key gets destroyed the regular way.
Nothing seems special in this TCP self-connect case.
9. The only interesting part in this experiment is that now I have a TCP
self-connect selftest, that by its nature tests simultaneous open, which
as I expected works with TCP-MD5, but not with TCP-AO:
[ 3412.559472] TCP: AO hash mismatch for (127.0.0.1, 7010)->(127.0.0.1,
7010) SA
[ 4115.964926] TCP: AO hash mismatch for [::1]:7010->[::1]:7010 SA
L3index: 0
It was expected as tcp_inbound_ao_hash() has
: /* Fast-path */
: /* TODO: fix fastopen and simultaneous open (TCPF_SYN_RECV) */
Going to fix this for TCP-AO-v13.
Please, let me know if this explanation/investigation looks good to you,
so that I can proceed with v13.
Thanks,
Dmitry