2019-11-07 18:56:50

by syzbot

[permalink] [raw]
Subject: WARNING in iov_iter_pipe

Hello,

syzbot found the following crash on:

HEAD commit: c68c5373 Add linux-next specific files for 20191107
git tree: linux-next
console output: https://syzkaller.appspot.com/x/log.txt?x=13d6bcfce00000
kernel config: https://syzkaller.appspot.com/x/.config?x=742545dcdea21726
dashboard link: https://syzkaller.appspot.com/bug?extid=991400e8eba7e00a26e1
compiler: gcc (GCC) 9.0.0 20181231 (experimental)
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=1529829ae00000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=16a55c0ce00000

The bug was bisected to:

commit b1b4705d54abedfd69dcdf42779c521aa1e0fbd3
Author: Matthew Bobrowski <[email protected]>
Date: Tue Nov 5 12:01:37 2019 +0000

ext4: introduce direct I/O read using iomap infrastructure

bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=1176cffae00000
final crash: https://syzkaller.appspot.com/x/report.txt?x=1376cffae00000
console output: https://syzkaller.appspot.com/x/log.txt?x=1576cffae00000

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: [email protected]
Fixes: b1b4705d54ab ("ext4: introduce direct I/O read using iomap
infrastructure")

------------[ cut here ]------------
WARNING: CPU: 1 PID: 8715 at lib/iov_iter.c:1162 iov_iter_pipe+0x25b/0x2f0
lib/iov_iter.c:1162
Kernel panic - not syncing: panic_on_warn set ...
CPU: 1 PID: 8715 Comm: syz-executor719 Not tainted 5.4.0-rc6-next-20191107
#0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x197/0x210 lib/dump_stack.c:118
panic+0x2e3/0x75c kernel/panic.c:221
__warn.cold+0x2f/0x35 kernel/panic.c:582
report_bug+0x289/0x300 lib/bug.c:195
fixup_bug arch/x86/kernel/traps.c:174 [inline]
fixup_bug arch/x86/kernel/traps.c:169 [inline]
do_error_trap+0x11b/0x200 arch/x86/kernel/traps.c:267
do_invalid_op+0x37/0x50 arch/x86/kernel/traps.c:286
invalid_op+0x23/0x30 arch/x86/entry/entry_64.S:1027
RIP: 0010:iov_iter_pipe+0x25b/0x2f0 lib/iov_iter.c:1162
Code: 83 c0 03 38 d0 7c 04 84 d2 75 33 44 89 63 24 48 83 c4 10 5b 41 5c 41
5d 41 5e 41 5f 5d c3 e8 ac ba 2e fe 0f 0b e8 a5 ba 2e fe <0f> 0b e9 53 fe
ff ff 4c 89 f7 e8 46 e5 6a fe e9 f5 fd ff ff e8 dc
RSP: 0018:ffff8880a0b4f988 EFLAGS: 00010293
RAX: ffff88808f1602c0 RBX: ffff8880a0b4fa18 RCX: ffffffff8344ac89
RDX: 0000000000000000 RSI: ffffffff8344ae3b RDI: 0000000000000004
RBP: ffff8880a0b4f9c0 R08: ffff88808f1602c0 R09: 0000000000000000
R10: fffffbfff1390168 R11: ffffffff89c80b47 R12: ffff88808f121c00
R13: 0000000000000010 R14: ffff88808f121cc8 R15: ffff88808f121cd0
generic_file_splice_read+0xa7/0x800 fs/splice.c:303
do_splice_to+0x127/0x180 fs/splice.c:877
splice_direct_to_actor+0x2d3/0x970 fs/splice.c:955
do_splice_direct+0x1da/0x2a0 fs/splice.c:1064
do_sendfile+0x597/0xd00 fs/read_write.c:1464
__do_sys_sendfile64 fs/read_write.c:1525 [inline]
__se_sys_sendfile64 fs/read_write.c:1511 [inline]
__x64_sys_sendfile64+0x1dd/0x220 fs/read_write.c:1511
do_syscall_64+0xfa/0x760 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x446969
Code: e8 4c b4 02 00 48 83 c4 18 c3 0f 1f 80 00 00 00 00 48 89 f8 48 89 f7
48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff
ff 0f 83 0b 08 fc ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007f05c41b8ce8 EFLAGS: 00000246 ORIG_RAX: 0000000000000028
RAX: ffffffffffffffda RBX: 00000000006dbc58 RCX: 0000000000446969
RDX: 0000000000000000 RSI: 0000000000000003 RDI: 0000000000000005
RBP: 00000000006dbc50 R08: 0000000000000000 R09: 0000000000000000
R10: 00000000ffffffff R11: 0000000000000246 R12: 00000000006dbc5c
R13: 00007fffb77353df R14: 00007f05c41b99c0 R15: 20c49ba5e353f7cf
Kernel Offset: disabled
Rebooting in 86400 seconds..


---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at [email protected].

syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
For information about bisection process see: https://goo.gl/tpsmEJ#bisection
syzbot can test patches for this bug, for details see:
https://goo.gl/tpsmEJ#testing-patches


2019-11-08 10:33:32

by Jan Kara

[permalink] [raw]
Subject: Re: WARNING in iov_iter_pipe

Hi!

On Thu 07-11-19 10:54:10, syzbot wrote:
> syzbot found the following crash on:
>
> HEAD commit: c68c5373 Add linux-next specific files for 20191107
> git tree: linux-next
> console output: https://syzkaller.appspot.com/x/log.txt?x=13d6bcfce00000
> kernel config: https://syzkaller.appspot.com/x/.config?x=742545dcdea21726
> dashboard link: https://syzkaller.appspot.com/bug?extid=991400e8eba7e00a26e1
> compiler: gcc (GCC) 9.0.0 20181231 (experimental)
> syz repro: https://syzkaller.appspot.com/x/repro.syz?x=1529829ae00000
> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=16a55c0ce00000
>
> The bug was bisected to:
>
> commit b1b4705d54abedfd69dcdf42779c521aa1e0fbd3
> Author: Matthew Bobrowski <[email protected]>
> Date: Tue Nov 5 12:01:37 2019 +0000
>
> ext4: introduce direct I/O read using iomap infrastructure

Hum, interesting and from the first looks the problem looks real.
Deciphered reproducer is:

int fd0 = open("./file0", O_RDWR | O_CREAT | O_EXCL | O_DIRECT, 0);
int fd1 = open("./file0, O_RDONLY);
write(fd0, "some_data...", 512);
sendfile(fd0, fd1, NULL, 0x7fffffa7);
-> this is interesting as it will result in reading data from 'file0' at
offset X with buffered read and writing them with direct write to
offset X+512. So this way we'll grow the file up to those ~2GB in
512-byte chunks.
- not sure if we ever get there but the remainder of the reproducer is:
fd2 = open("./file0", O_RDWR | O_CREAT | O_NOATIME | O_SYNC, 0);
sendfile(fd2, fd0, NULL, 0xffffffff)
-> doesn't seem too interesting as fd0 is at EOF so this shouldn't do
anything.

Matthew, can you have a look?

Honza

> bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=1176cffae00000
> final crash: https://syzkaller.appspot.com/x/report.txt?x=1376cffae00000
> console output: https://syzkaller.appspot.com/x/log.txt?x=1576cffae00000
>
> IMPORTANT: if you fix the bug, please add the following tag to the commit:
> Reported-by: [email protected]
> Fixes: b1b4705d54ab ("ext4: introduce direct I/O read using iomap
> infrastructure")
>
> ------------[ cut here ]------------
> WARNING: CPU: 1 PID: 8715 at lib/iov_iter.c:1162 iov_iter_pipe+0x25b/0x2f0
> lib/iov_iter.c:1162
> Kernel panic - not syncing: panic_on_warn set ...
> CPU: 1 PID: 8715 Comm: syz-executor719 Not tainted 5.4.0-rc6-next-20191107
> #0
> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
> Google 01/01/2011
> Call Trace:
> __dump_stack lib/dump_stack.c:77 [inline]
> dump_stack+0x197/0x210 lib/dump_stack.c:118
> panic+0x2e3/0x75c kernel/panic.c:221
> __warn.cold+0x2f/0x35 kernel/panic.c:582
> report_bug+0x289/0x300 lib/bug.c:195
> fixup_bug arch/x86/kernel/traps.c:174 [inline]
> fixup_bug arch/x86/kernel/traps.c:169 [inline]
> do_error_trap+0x11b/0x200 arch/x86/kernel/traps.c:267
> do_invalid_op+0x37/0x50 arch/x86/kernel/traps.c:286
> invalid_op+0x23/0x30 arch/x86/entry/entry_64.S:1027
> RIP: 0010:iov_iter_pipe+0x25b/0x2f0 lib/iov_iter.c:1162
> Code: 83 c0 03 38 d0 7c 04 84 d2 75 33 44 89 63 24 48 83 c4 10 5b 41 5c 41
> 5d 41 5e 41 5f 5d c3 e8 ac ba 2e fe 0f 0b e8 a5 ba 2e fe <0f> 0b e9 53 fe ff
> ff 4c 89 f7 e8 46 e5 6a fe e9 f5 fd ff ff e8 dc
> RSP: 0018:ffff8880a0b4f988 EFLAGS: 00010293
> RAX: ffff88808f1602c0 RBX: ffff8880a0b4fa18 RCX: ffffffff8344ac89
> RDX: 0000000000000000 RSI: ffffffff8344ae3b RDI: 0000000000000004
> RBP: ffff8880a0b4f9c0 R08: ffff88808f1602c0 R09: 0000000000000000
> R10: fffffbfff1390168 R11: ffffffff89c80b47 R12: ffff88808f121c00
> R13: 0000000000000010 R14: ffff88808f121cc8 R15: ffff88808f121cd0
> generic_file_splice_read+0xa7/0x800 fs/splice.c:303
> do_splice_to+0x127/0x180 fs/splice.c:877
> splice_direct_to_actor+0x2d3/0x970 fs/splice.c:955
> do_splice_direct+0x1da/0x2a0 fs/splice.c:1064
> do_sendfile+0x597/0xd00 fs/read_write.c:1464
> __do_sys_sendfile64 fs/read_write.c:1525 [inline]
> __se_sys_sendfile64 fs/read_write.c:1511 [inline]
> __x64_sys_sendfile64+0x1dd/0x220 fs/read_write.c:1511
> do_syscall_64+0xfa/0x760 arch/x86/entry/common.c:290
> entry_SYSCALL_64_after_hwframe+0x49/0xbe
> RIP: 0033:0x446969
> Code: e8 4c b4 02 00 48 83 c4 18 c3 0f 1f 80 00 00 00 00 48 89 f8 48 89 f7
> 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff
> 0f 83 0b 08 fc ff c3 66 2e 0f 1f 84 00 00 00 00
> RSP: 002b:00007f05c41b8ce8 EFLAGS: 00000246 ORIG_RAX: 0000000000000028
> RAX: ffffffffffffffda RBX: 00000000006dbc58 RCX: 0000000000446969
> RDX: 0000000000000000 RSI: 0000000000000003 RDI: 0000000000000005
> RBP: 00000000006dbc50 R08: 0000000000000000 R09: 0000000000000000
> R10: 00000000ffffffff R11: 0000000000000246 R12: 00000000006dbc5c
> R13: 00007fffb77353df R14: 00007f05c41b99c0 R15: 20c49ba5e353f7cf
> Kernel Offset: disabled
> Rebooting in 86400 seconds..
>
>
> ---
> This bug is generated by a bot. It may contain errors.
> See https://goo.gl/tpsmEJ for more information about syzbot.
> syzbot engineers can be reached at [email protected].
>
> syzbot will keep track of this bug report. See:
> https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
> For information about bisection process see: https://goo.gl/tpsmEJ#bisection
> syzbot can test patches for this bug, for details see:
> https://goo.gl/tpsmEJ#testing-patches
--
Jan Kara <[email protected]>
SUSE Labs, CR

2019-11-11 08:17:35

by Matthew Bobrowski

[permalink] [raw]
Subject: Re: WARNING in iov_iter_pipe

On Fri, Nov 08, 2019 at 11:31:48AM +0100, Jan Kara wrote:
> On Thu 07-11-19 10:54:10, syzbot wrote:
> > syzbot found the following crash on:
> >
> > HEAD commit: c68c5373 Add linux-next specific files for 20191107
> > git tree: linux-next
> > console output: https://syzkaller.appspot.com/x/log.txt?x=13d6bcfce00000
> > kernel config: https://syzkaller.appspot.com/x/.config?x=742545dcdea21726
> > dashboard link: https://syzkaller.appspot.com/bug?extid=991400e8eba7e00a26e1
> > compiler: gcc (GCC) 9.0.0 20181231 (experimental)
> > syz repro: https://syzkaller.appspot.com/x/repro.syz?x=1529829ae00000
> > C reproducer: https://syzkaller.appspot.com/x/repro.c?x=16a55c0ce00000
> >
> > The bug was bisected to:
> >
> > commit b1b4705d54abedfd69dcdf42779c521aa1e0fbd3
> > Author: Matthew Bobrowski <[email protected]>
> > Date: Tue Nov 5 12:01:37 2019 +0000
> >
> > ext4: introduce direct I/O read using iomap infrastructure
>
> Hum, interesting and from the first looks the problem looks real.
> Deciphered reproducer is:
>
> int fd0 = open("./file0", O_RDWR | O_CREAT | O_EXCL | O_DIRECT, 0);
> int fd1 = open("./file0, O_RDONLY);
> write(fd0, "some_data...", 512);
> sendfile(fd0, fd1, NULL, 0x7fffffa7);
> -> this is interesting as it will result in reading data from 'file0' at
> offset X with buffered read and writing them with direct write to
> offset X+512. So this way we'll grow the file up to those ~2GB in
> 512-byte chunks.
> - not sure if we ever get there but the remainder of the reproducer is:
> fd2 = open("./file0", O_RDWR | O_CREAT | O_NOATIME | O_SYNC, 0);
> sendfile(fd2, fd0, NULL, 0xffffffff)
> -> doesn't seem too interesting as fd0 is at EOF so this shouldn't do
> anything.
>
> Matthew, can you have a look?

Sorry Jan, I've been crazy busy lately and I'm out at training this
week. Let me take a look at this and see whether I can determine
what's happening here.

/M

2019-11-19 03:13:01

by Eric Biggers

[permalink] [raw]
Subject: Re: WARNING in iov_iter_pipe

On Mon, Nov 11, 2019 at 07:16:29PM +1100, Matthew Bobrowski wrote:
> On Fri, Nov 08, 2019 at 11:31:48AM +0100, Jan Kara wrote:
> > On Thu 07-11-19 10:54:10, syzbot wrote:
> > > syzbot found the following crash on:
> > >
> > > HEAD commit: c68c5373 Add linux-next specific files for 20191107
> > > git tree: linux-next
> > > console output: https://syzkaller.appspot.com/x/log.txt?x=13d6bcfce00000
> > > kernel config: https://syzkaller.appspot.com/x/.config?x=742545dcdea21726
> > > dashboard link: https://syzkaller.appspot.com/bug?extid=991400e8eba7e00a26e1
> > > compiler: gcc (GCC) 9.0.0 20181231 (experimental)
> > > syz repro: https://syzkaller.appspot.com/x/repro.syz?x=1529829ae00000
> > > C reproducer: https://syzkaller.appspot.com/x/repro.c?x=16a55c0ce00000
> > >
> > > The bug was bisected to:
> > >
> > > commit b1b4705d54abedfd69dcdf42779c521aa1e0fbd3
> > > Author: Matthew Bobrowski <[email protected]>
> > > Date: Tue Nov 5 12:01:37 2019 +0000
> > >
> > > ext4: introduce direct I/O read using iomap infrastructure
> >
> > Hum, interesting and from the first looks the problem looks real.
> > Deciphered reproducer is:
> >
> > int fd0 = open("./file0", O_RDWR | O_CREAT | O_EXCL | O_DIRECT, 0);
> > int fd1 = open("./file0, O_RDONLY);
> > write(fd0, "some_data...", 512);
> > sendfile(fd0, fd1, NULL, 0x7fffffa7);
> > -> this is interesting as it will result in reading data from 'file0' at
> > offset X with buffered read and writing them with direct write to
> > offset X+512. So this way we'll grow the file up to those ~2GB in
> > 512-byte chunks.
> > - not sure if we ever get there but the remainder of the reproducer is:
> > fd2 = open("./file0", O_RDWR | O_CREAT | O_NOATIME | O_SYNC, 0);
> > sendfile(fd2, fd0, NULL, 0xffffffff)
> > -> doesn't seem too interesting as fd0 is at EOF so this shouldn't do
> > anything.
> >
> > Matthew, can you have a look?
>
> Sorry Jan, I've been crazy busy lately and I'm out at training this
> week. Let me take a look at this and see whether I can determine
> what's happening here.
>

FYI, syzbot is still seeing this on linux-next.

Also, a new thread was started to discuss this:
https://lkml.kernel.org/linux-ext4/[email protected]/T/#u
(Mentioning this in case anyone is following this thread only.)

- Eric

2019-11-19 10:10:46

by Matthew Bobrowski

[permalink] [raw]
Subject: Re: WARNING in iov_iter_pipe

On Mon, Nov 18, 2019 at 07:10:21PM -0800, Eric Biggers wrote:
> On Mon, Nov 11, 2019 at 07:16:29PM +1100, Matthew Bobrowski wrote:
> > On Fri, Nov 08, 2019 at 11:31:48AM +0100, Jan Kara wrote:
> > > On Thu 07-11-19 10:54:10, syzbot wrote:
> > > > syzbot found the following crash on:
> > > >
> > > > HEAD commit: c68c5373 Add linux-next specific files for 20191107
> > > > git tree: linux-next
> > > > console output: https://syzkaller.appspot.com/x/log.txt?x=13d6bcfce00000
> > > > kernel config: https://syzkaller.appspot.com/x/.config?x=742545dcdea21726
> > > > dashboard link: https://syzkaller.appspot.com/bug?extid=991400e8eba7e00a26e1
> > > > compiler: gcc (GCC) 9.0.0 20181231 (experimental)
> > > > syz repro: https://syzkaller.appspot.com/x/repro.syz?x=1529829ae00000
> > > > C reproducer: https://syzkaller.appspot.com/x/repro.c?x=16a55c0ce00000
> > > >
> > > > The bug was bisected to:
> > > >
> > > > commit b1b4705d54abedfd69dcdf42779c521aa1e0fbd3
> > > > Author: Matthew Bobrowski <[email protected]>
> > > > Date: Tue Nov 5 12:01:37 2019 +0000
> > > >
> > > > ext4: introduce direct I/O read using iomap infrastructure
> > >
> > > Hum, interesting and from the first looks the problem looks real.
> > > Deciphered reproducer is:
> > >
> > > int fd0 = open("./file0", O_RDWR | O_CREAT | O_EXCL | O_DIRECT, 0);
> > > int fd1 = open("./file0, O_RDONLY);
> > > write(fd0, "some_data...", 512);
> > > sendfile(fd0, fd1, NULL, 0x7fffffa7);
> > > -> this is interesting as it will result in reading data from 'file0' at
> > > offset X with buffered read and writing them with direct write to
> > > offset X+512. So this way we'll grow the file up to those ~2GB in
> > > 512-byte chunks.
> > > - not sure if we ever get there but the remainder of the reproducer is:
> > > fd2 = open("./file0", O_RDWR | O_CREAT | O_NOATIME | O_SYNC, 0);
> > > sendfile(fd2, fd0, NULL, 0xffffffff)
> > > -> doesn't seem too interesting as fd0 is at EOF so this shouldn't do
> > > anything.
> > >
> > > Matthew, can you have a look?
> >
> > Sorry Jan, I've been crazy busy lately and I'm out at training this
> > week. Let me take a look at this and see whether I can determine
> > what's happening here.
> >
>
> FYI, syzbot is still seeing this on linux-next.
>
> Also, a new thread was started to discuss this:
> https://lkml.kernel.org/linux-ext4/[email protected]/T/#u
> (Mentioning this in case anyone is following this thread only.)

Understood. I suppose this issue will get some more traction this
week.

/M