2024-02-09 01:29:06

by Salvatore Bonaccorso

[permalink] [raw]
Subject: [regression 6.1.y] f2fs: invalid zstd compress level: 6

Hi Jaegeuk Kim, Chao Yu,

In Debian the following regression was reported after a Dhya updated
to 6.1.76:

On Wed, Feb 07, 2024 at 10:43:47PM -0500, Dhya wrote:
> Package: src:linux
> Version: 6.1.76-1
> Severity: critical
> Justification: breaks the whole system
>
> Dear Maintainer,
>
> After upgrade to linux-image-6.1.0-18-amd64 6.1.76-1 F2FS filesystem
> fails to mount rw. Message in the boot journal:
>
> kernel: F2FS-fs (nvme0n1p6): invalid zstd compress level: 6
>
> There was recently an f2fs patch to the 6.1 kernel tree which might be
> related: https://www.spinics.net/lists/stable-commits/msg329957.html
>
> Was able to recover the system by doing:
>
> sudo mount -o remount,rw,relatime,lazytime,background_gc=on,discard,no_heap,user_xattr,inline_xattr,acl,inline_data,inline_dentry,extent_cache,mode=adaptive,active_logs=6,alloc_mode=default,checkpoint_merge,fsync_mode=posix,compress_algorithm=lz4,compress_log_size=2,compress_mode=fs,atgc,discard_unit=block,memory=normal /dev/nvme0n1p6 /
>
> under the running bad 6.1.0-18-amd64 kernel, then editing
> /etc/default/grub:
>
> GRUB_DEFAULT="Advanced options for Debian GNU/Linux>Debian GNU/Linux, with Linux 6.1.0-17-amd64"
>
> and running 'update-grub' and rebooting to boot the 6.1.0-17-amd64
> kernel.

The issue is easily reproducible by:

# dd if=/dev/zero of=test.img count=100 bs=1M
# mkfs.f2fs -f -O compression,extra_attr ./test.img
# mount -t f2fs -o compress_algorithm=zstd:6,compress_chksum,atgc,gc_merge,lazytime ./test.img /mnt

resulting in

[ 60.789982] F2FS-fs (loop0): invalid zstd compress level: 6

A bugzilla report has been submitted in
https://bugzilla.kernel.org/show_bug.cgi?id=218471

#regzbot introduced: v6.1.69..v6.1.76
#regzbot link: https://bugs.debian.org/1063422
#regzbot link: https://bugzilla.kernel.org/show_bug.cgi?id=218471

Regards,
Salvatore


2024-02-09 01:43:33

by Jaegeuk Kim

[permalink] [raw]
Subject: Re: [regression 6.1.y] f2fs: invalid zstd compress level: 6

Hi,

Let me check this soon.

Thanks,

On 02/08, Salvatore Bonaccorso wrote:
> Hi Jaegeuk Kim, Chao Yu,
>
> In Debian the following regression was reported after a Dhya updated
> to 6.1.76:
>
> On Wed, Feb 07, 2024 at 10:43:47PM -0500, Dhya wrote:
> > Package: src:linux
> > Version: 6.1.76-1
> > Severity: critical
> > Justification: breaks the whole system
> >
> > Dear Maintainer,
> >
> > After upgrade to linux-image-6.1.0-18-amd64 6.1.76-1 F2FS filesystem
> > fails to mount rw. Message in the boot journal:
> >
> > kernel: F2FS-fs (nvme0n1p6): invalid zstd compress level: 6
> >
> > There was recently an f2fs patch to the 6.1 kernel tree which might be
> > related: https://www.spinics.net/lists/stable-commits/msg329957.html
> >
> > Was able to recover the system by doing:
> >
> > sudo mount -o remount,rw,relatime,lazytime,background_gc=on,discard,no_heap,user_xattr,inline_xattr,acl,inline_data,inline_dentry,extent_cache,mode=adaptive,active_logs=6,alloc_mode=default,checkpoint_merge,fsync_mode=posix,compress_algorithm=lz4,compress_log_size=2,compress_mode=fs,atgc,discard_unit=block,memory=normal /dev/nvme0n1p6 /
> >
> > under the running bad 6.1.0-18-amd64 kernel, then editing
> > /etc/default/grub:
> >
> > GRUB_DEFAULT="Advanced options for Debian GNU/Linux>Debian GNU/Linux, with Linux 6.1.0-17-amd64"
> >
> > and running 'update-grub' and rebooting to boot the 6.1.0-17-amd64
> > kernel.
>
> The issue is easily reproducible by:
>
> # dd if=/dev/zero of=test.img count=100 bs=1M
> # mkfs.f2fs -f -O compression,extra_attr ./test.img
> # mount -t f2fs -o compress_algorithm=zstd:6,compress_chksum,atgc,gc_merge,lazytime ./test.img /mnt
>
> resulting in
>
> [ 60.789982] F2FS-fs (loop0): invalid zstd compress level: 6
>
> A bugzilla report has been submitted in
> https://bugzilla.kernel.org/show_bug.cgi?id=218471
>
> #regzbot introduced: v6.1.69..v6.1.76
> #regzbot link: https://bugs.debian.org/1063422
> #regzbot link: https://bugzilla.kernel.org/show_bug.cgi?id=218471
>
> Regards,
> Salvatore

2024-02-19 02:35:26

by Chao Yu

[permalink] [raw]
Subject: Re: [regression 6.1.y] f2fs: invalid zstd compress level: 6

On 2024/2/9 4:19, Salvatore Bonaccorso wrote:
> Hi Jaegeuk Kim, Chao Yu,
>
> In Debian the following regression was reported after a Dhya updated
> to 6.1.76:
>
> On Wed, Feb 07, 2024 at 10:43:47PM -0500, Dhya wrote:
>> Package: src:linux
>> Version: 6.1.76-1
>> Severity: critical
>> Justification: breaks the whole system
>>
>> Dear Maintainer,
>>
>> After upgrade to linux-image-6.1.0-18-amd64 6.1.76-1 F2FS filesystem
>> fails to mount rw. Message in the boot journal:
>>
>> kernel: F2FS-fs (nvme0n1p6): invalid zstd compress level: 6
>>
>> There was recently an f2fs patch to the 6.1 kernel tree which might be
>> related: https://www.spinics.net/lists/stable-commits/msg329957.html
>>
>> Was able to recover the system by doing:
>>
>> sudo mount -o remount,rw,relatime,lazytime,background_gc=on,discard,no_heap,user_xattr,inline_xattr,acl,inline_data,inline_dentry,extent_cache,mode=adaptive,active_logs=6,alloc_mode=default,checkpoint_merge,fsync_mode=posix,compress_algorithm=lz4,compress_log_size=2,compress_mode=fs,atgc,discard_unit=block,memory=normal /dev/nvme0n1p6 /
>>
>> under the running bad 6.1.0-18-amd64 kernel, then editing
>> /etc/default/grub:
>>
>> GRUB_DEFAULT="Advanced options for Debian GNU/Linux>Debian GNU/Linux, with Linux 6.1.0-17-amd64"
>>
>> and running 'update-grub' and rebooting to boot the 6.1.0-17-amd64
>> kernel.
>
> The issue is easily reproducible by:
>
> # dd if=/dev/zero of=test.img count=100 bs=1M
> # mkfs.f2fs -f -O compression,extra_attr ./test.img
> # mount -t f2fs -o compress_algorithm=zstd:6,compress_chksum,atgc,gc_merge,lazytime ./test.img /mnt
>
> resulting in
>
> [ 60.789982] F2FS-fs (loop0): invalid zstd compress level: 6

Hi Salvatore,

Can you please try below fixes:

[PATCH 6.1] f2fs: add helper to check compression level
https://lore.kernel.org/linux-f2fs-devel/[email protected]

[PATCH] f2fs: compress: fix to check zstd compress level correctly in mount option
https://lore.kernel.org/linux-f2fs-devel/[email protected]

Thanks,

>
> A bugzilla report has been submitted in
> https://bugzilla.kernel.org/show_bug.cgi?id=218471
>
> #regzbot introduced: v6.1.69..v6.1.76
> #regzbot link: https://bugs.debian.org/1063422
> #regzbot link: https://bugzilla.kernel.org/show_bug.cgi?id=218471
>
> Regards,
> Salvatore

2024-02-19 19:55:48

by Salvatore Bonaccorso

[permalink] [raw]
Subject: Re: [regression 6.1.y] f2fs: invalid zstd compress level: 6

Hi,

On Mon, Feb 19, 2024 at 10:35:13AM +0800, Chao Yu wrote:
> On 2024/2/9 4:19, Salvatore Bonaccorso wrote:
> > Hi Jaegeuk Kim, Chao Yu,
> >
> > In Debian the following regression was reported after a Dhya updated
> > to 6.1.76:
> >
> > On Wed, Feb 07, 2024 at 10:43:47PM -0500, Dhya wrote:
> > > Package: src:linux
> > > Version: 6.1.76-1
> > > Severity: critical
> > > Justification: breaks the whole system
> > >
> > > Dear Maintainer,
> > >
> > > After upgrade to linux-image-6.1.0-18-amd64 6.1.76-1 F2FS filesystem
> > > fails to mount rw. Message in the boot journal:
> > >
> > > kernel: F2FS-fs (nvme0n1p6): invalid zstd compress level: 6
> > >
> > > There was recently an f2fs patch to the 6.1 kernel tree which might be
> > > related: https://www.spinics.net/lists/stable-commits/msg329957.html
> > >
> > > Was able to recover the system by doing:
> > >
> > > sudo mount -o remount,rw,relatime,lazytime,background_gc=on,discard,no_heap,user_xattr,inline_xattr,acl,inline_data,inline_dentry,extent_cache,mode=adaptive,active_logs=6,alloc_mode=default,checkpoint_merge,fsync_mode=posix,compress_algorithm=lz4,compress_log_size=2,compress_mode=fs,atgc,discard_unit=block,memory=normal /dev/nvme0n1p6 /
> > >
> > > under the running bad 6.1.0-18-amd64 kernel, then editing
> > > /etc/default/grub:
> > >
> > > GRUB_DEFAULT="Advanced options for Debian GNU/Linux>Debian GNU/Linux, with Linux 6.1.0-17-amd64"
> > >
> > > and running 'update-grub' and rebooting to boot the 6.1.0-17-amd64
> > > kernel.
> >
> > The issue is easily reproducible by:
> >
> > # dd if=/dev/zero of=test.img count=100 bs=1M
> > # mkfs.f2fs -f -O compression,extra_attr ./test.img
> > # mount -t f2fs -o compress_algorithm=zstd:6,compress_chksum,atgc,gc_merge,lazytime ./test.img /mnt
> >
> > resulting in
> >
> > [ 60.789982] F2FS-fs (loop0): invalid zstd compress level: 6
>
> Hi Salvatore,
>
> Can you please try below fixes:
>
> [PATCH 6.1] f2fs: add helper to check compression level
> https://lore.kernel.org/linux-f2fs-devel/[email protected]

Confirmed that this fixes the reported issue as it was reported to us
in Debian in https://bugs.debian.org/1063422 . Thanks a lot!
(note just tested with the first commit as it landed in 6.1.78 to
confirm the immediate regression).

#regzbot fixed-by: cf3d57ad6ff8b566deba3544b9ad3384781fb604

Regards,
Salvatore

2024-02-20 02:39:18

by Chao Yu

[permalink] [raw]
Subject: Re: [regression 6.1.y] f2fs: invalid zstd compress level: 6

On 2024/2/20 3:54, Salvatore Bonaccorso wrote:
> Hi,
>
> On Mon, Feb 19, 2024 at 10:35:13AM +0800, Chao Yu wrote:
>> On 2024/2/9 4:19, Salvatore Bonaccorso wrote:
>>> Hi Jaegeuk Kim, Chao Yu,
>>>
>>> In Debian the following regression was reported after a Dhya updated
>>> to 6.1.76:
>>>
>>> On Wed, Feb 07, 2024 at 10:43:47PM -0500, Dhya wrote:
>>>> Package: src:linux
>>>> Version: 6.1.76-1
>>>> Severity: critical
>>>> Justification: breaks the whole system
>>>>
>>>> Dear Maintainer,
>>>>
>>>> After upgrade to linux-image-6.1.0-18-amd64 6.1.76-1 F2FS filesystem
>>>> fails to mount rw. Message in the boot journal:
>>>>
>>>> kernel: F2FS-fs (nvme0n1p6): invalid zstd compress level: 6
>>>>
>>>> There was recently an f2fs patch to the 6.1 kernel tree which might be
>>>> related: https://www.spinics.net/lists/stable-commits/msg329957.html
>>>>
>>>> Was able to recover the system by doing:
>>>>
>>>> sudo mount -o remount,rw,relatime,lazytime,background_gc=on,discard,no_heap,user_xattr,inline_xattr,acl,inline_data,inline_dentry,extent_cache,mode=adaptive,active_logs=6,alloc_mode=default,checkpoint_merge,fsync_mode=posix,compress_algorithm=lz4,compress_log_size=2,compress_mode=fs,atgc,discard_unit=block,memory=normal /dev/nvme0n1p6 /
>>>>
>>>> under the running bad 6.1.0-18-amd64 kernel, then editing
>>>> /etc/default/grub:
>>>>
>>>> GRUB_DEFAULT="Advanced options for Debian GNU/Linux>Debian GNU/Linux, with Linux 6.1.0-17-amd64"
>>>>
>>>> and running 'update-grub' and rebooting to boot the 6.1.0-17-amd64
>>>> kernel.
>>>
>>> The issue is easily reproducible by:
>>>
>>> # dd if=/dev/zero of=test.img count=100 bs=1M
>>> # mkfs.f2fs -f -O compression,extra_attr ./test.img
>>> # mount -t f2fs -o compress_algorithm=zstd:6,compress_chksum,atgc,gc_merge,lazytime ./test.img /mnt
>>>
>>> resulting in
>>>
>>> [ 60.789982] F2FS-fs (loop0): invalid zstd compress level: 6
>>
>> Hi Salvatore,
>>
>> Can you please try below fixes:
>>
>> [PATCH 6.1] f2fs: add helper to check compression level
>> https://lore.kernel.org/linux-f2fs-devel/[email protected]
>
> Confirmed that this fixes the reported issue as it was reported to us
> in Debian in https://bugs.debian.org/1063422 . Thanks a lot!
> (note just tested with the first commit as it landed in 6.1.78 to
> confirm the immediate regression).
>
> #regzbot fixed-by: cf3d57ad6ff8b566deba3544b9ad3384781fb604

Hi,

Thank you for confirmation.

Thanks,

>
> Regards,
> Salvatore