Received-SPF: pass (google.com: domain of linux-ext4+bounces-2057-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Precedence: bulk
MIME-Version: 1.0
References: <00000000000042c9190615cdb315@google.com> <20240411121319.adhz4ylacbv6ocuu@quack3>
 <CAOQ4uxi9L_Rs7q=fcLGqJMx15jLAArOWGwGfdCL8LOUCPR3L+w@mail.gmail.com>
 <875xwn8zxa.fsf@mailhost.krisman.be> <CACGdZYLJESbS6VCAza_V6PAeNb8k9nU2wYhBi-KmeYqYJ337mA@mail.gmail.com>
In-Reply-To: <CACGdZYLJESbS6VCAza_V6PAeNb8k9nU2wYhBi-KmeYqYJ337mA@mail.gmail.com>
From: Amir Goldstein <amir73il@gmail.com>
Date: Fri, 12 Apr 2024 13:24:50 +0300
Message-ID: <CAOQ4uxiw9+0ZPNgbby0_WJzKrcCk79n-Xd1fOMtvsr022Z8EwA@mail.gmail.com>
Subject: Re: [syzbot] [ext4?] KASAN: slab-use-after-free Read in fsnotify
To: Khazhy Kumykov <khazhy@chromium.org>
Cc: Gabriel Krisman Bertazi <krisman@suse.de>, Jan Kara <jack@suse.cz>, 
	syzbot <syzbot+5e3f9b2a67b45f16d4e6@syzkaller.appspotmail.com>, 
	linux-ext4@vger.kernel.org, linux-fsdevel@vger.kernel.org, 
	linux-kernel@vger.kernel.org, repnop@google.com, 
	syzkaller-bugs@googlegroups.com
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Apr 11, 2024 at 11:27=E2=80=AFPM Khazhy Kumykov <khazhy@chromium.or=
g> wrote:
>
> On Thu, Apr 11, 2024 at 12:25=E2=80=AFPM Gabriel Krisman Bertazi
> <krisman@suse.de> wrote:
> >
> > Amir Goldstein <amir73il@gmail.com> writes:
> >
> > > On Thu, Apr 11, 2024 at 3:13=E2=80=AFPM Jan Kara <jack@suse.cz> wrote=
:
> > >>
> > >> On Thu 11-04-24 01:11:20, syzbot wrote:
> > >> > Hello,
> > >> >
> > >> > syzbot found the following issue on:
> > >> >
> > >> > HEAD commit:    6ebf211bb11d Add linux-next specific files for 202=
40410
> > >> > git tree:       linux-next
> > >> > console+strace: https://syzkaller.appspot.com/x/log.txt?x=3D12be95=
5d180000
> > >> > kernel config:  https://syzkaller.appspot.com/x/.config?x=3D16ca15=
8ef7e08662
> > >> > dashboard link: https://syzkaller.appspot.com/bug?extid=3D5e3f9b2a=
67b45f16d4e6
> > >> > compiler:       Debian clang version 15.0.6, GNU ld (GNU Binutils =
for Debian) 2.40
> > >> > syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=3D13c9=
1175180000
> > >> > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=3D1621af=
9d180000
> > >> >
> > >> > Downloadable assets:
> > >> > disk image: https://storage.googleapis.com/syzbot-assets/b050f81f7=
3ed/disk-6ebf211b.raw.xz
> > >> > vmlinux: https://storage.googleapis.com/syzbot-assets/412c9b9a536e=
/vmlinux-6ebf211b.xz
> > >> > kernel image: https://storage.googleapis.com/syzbot-assets/0165272=
16c47/bzImage-6ebf211b.xz
> > >> > mounted in repro: https://storage.googleapis.com/syzbot-assets/75a=
d050c9945/mount_0.gz
> > >> >
> > >> > IMPORTANT: if you fix the issue, please add the following tag to t=
he commit:
> > >> > Reported-by: syzbot+5e3f9b2a67b45f16d4e6@syzkaller.appspotmail.com
> > >> >
> > >> > Quota error (device loop0): do_check_range: Getting block 0 out of=
 range 1-5
> > >> > EXT4-fs error (device loop0): ext4_release_dquot:6905: comm kworke=
r/u8:4: Failed to release dquot type 1
> > >> > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> > >> > BUG: KASAN: slab-use-after-free in fsnotify+0x2a4/0x1f70 fs/notify=
/fsnotify.c:539
> > >> > Read of size 8 at addr ffff88802f1dce80 by task kworker/u8:4/62
> > >> >
> > >> > CPU: 0 PID: 62 Comm: kworker/u8:4 Not tainted 6.9.0-rc3-next-20240=
410-syzkaller #0
> > >> > Hardware name: Google Google Compute Engine/Google Compute Engine,=
 BIOS Google 03/27/2024
> > >> > Workqueue: events_unbound quota_release_workfn
> > >> > Call Trace:
> > >> >  <TASK>
> > >> >  __dump_stack lib/dump_stack.c:88 [inline]
> > >> >  dump_stack_lvl+0x241/0x360 lib/dump_stack.c:114
> > >> >  print_address_description mm/kasan/report.c:377 [inline]
> > >> >  print_report+0x169/0x550 mm/kasan/report.c:488
> > >> >  kasan_report+0x143/0x180 mm/kasan/report.c:601
> > >> >  fsnotify+0x2a4/0x1f70 fs/notify/fsnotify.c:539
> > >> >  fsnotify_sb_error include/linux/fsnotify.h:456 [inline]
> > >> >  __ext4_error+0x255/0x3b0 fs/ext4/super.c:843
> > >> >  ext4_release_dquot+0x326/0x450 fs/ext4/super.c:6903
> > >> >  quota_release_workfn+0x39f/0x650 fs/quota/dquot.c:840
> > >> >  process_one_work kernel/workqueue.c:3218 [inline]
> > >> >  process_scheduled_works+0xa2c/0x1830 kernel/workqueue.c:3299
> > >> >  worker_thread+0x86d/0xd70 kernel/workqueue.c:3380
> > >> >  kthread+0x2f0/0x390 kernel/kthread.c:389
> > >> >  ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
> > >> >  ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
> > >> >  </TASK>
> > >>
> > >> Amir, I believe this happens on umount when the filesystem calls
> > >> fsnotify_sb_error() after calling fsnotify_sb_delete().
> Hmm, so we're releasing dquots after already shutting down the
> filesystem? Is that expected? This "Failed to release dquot type"
> error message only appears if we have an open handle from
> ext4_journal_start (although this filesystem was mounted without a
> journal, so we hit ext4_get_nojournal()...)
> > In theory these two
> > >> calls can even run in parallel and fsnotify() can be holding
> > >> fsnotify_sb_info pointer while fsnotify_sb_delete() is freeing it so=
 we
> > >> need to figure out some proper synchronization for that...
> > >
> > > Is it really needed to handle any for non SB_ACTIVE sb?
> >
> > I think it should be fine to exclude volumes being teared down.  Cc'ing
> > Khazhy, who sponsored this work at the time and owned the use-case.
> In terms of real-world use case... not sure there is one - you'll
> notice the errors when you fsck/mount again later on, and any users
> who care about notifs should be gone by the time we're unmounting. But
> it seems weird to me that we can get write errors shutting everything
> down.

That's what I thought.
Anyway, my naive fix breaks IN_UNMOUNT event as well as some other
fanotify tests, so I will need to work on another fix.

Thanks,
Amir.