2021-08-11 02:57:43

by Wu Bo

[permalink] [raw]
Subject: f2fs do DIO write make file corruption

I use the following command to create a file, the file may got corruption:
f2fs_io write 2 0 512 inc_num dio $path

And when I use bio or to set the chunk size to 1 block, the file is
normal. The commands as following:
f2fs_io write 2 0 512 inc_num buffered $path
f2fs_io write 1 0 512 inc_num dio $path

I find this bug on old kernel version 4.14.117, and not find on version
4.19.152. So this bug is fixed. Can anyone can tell me which patch fixed
this bug?


2021-08-11 03:06:13

by Chao Yu

[permalink] [raw]
Subject: Re: [f2fs-dev] f2fs do DIO write make file corruption

On 2021/8/11 10:48, Wu Bo wrote:
> I use the following command to create a file, the file may got corruption:
> f2fs_io write 2 0 512 inc_num dio $path
>
> And when I use bio or to set the chunk size to 1 block, the file is
> normal. The commands as following:
> f2fs_io write 2 0 512 inc_num buffered $path
> f2fs_io write 1 0 512 inc_num dio $path
>
> I find this bug on old kernel version 4.14.117, and not find on version
> 4.19.152. So this bug is fixed. Can anyone can tell me which patch fixed
> this bug?

Not sure,

https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=linux-4.19.y&id=7bae8b6b73e46c307fa355ce086800b7ad6610f8

Thanks,

>
>
> _______________________________________________
> Linux-f2fs-devel mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
>

2021-08-11 03:30:53

by Wu Bo

[permalink] [raw]
Subject: Re: [f2fs-dev] f2fs do DIO write make file corruption


在 2021/8/11 11:03, Chao Yu 写道:
> On 2021/8/11 10:48, Wu Bo wrote:
>> I use the following command to create a file, the file may got
>> corruption:
>>     f2fs_io write 2 0 512 inc_num dio $path
>>
>> And when I use bio or to set the chunk size to 1 block, the file is
>> normal. The commands as following:
>>     f2fs_io write 2 0 512 inc_num buffered $path
>>     f2fs_io write 1 0 512 inc_num dio $path
>>
>> I find this bug on old kernel version 4.14.117, and not find on version
>> 4.19.152. So this bug is fixed. Can anyone can tell me which patch fixed
>> this bug?
>
> Not sure,
>
> https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=linux-4.19.y&id=7bae8b6b73e46c307fa355ce086800b7ad6610f8
>

This patch is applied. The issue occurs when f2fs dio try to preallocate
multiple blocks and got scattered disk blocks. The log as following:
my_f2fs_io-13425 [002] .... 395.583637: el0_irq_naked:
1:type:1,ino:40132,off:768,old_blk:0,new_blk:185764
my_f2fs_io-13425 [002] .... 395.583710: el0_irq_naked:
1:type:1,ino:40132,off:769,old_blk:0,new_blk:205824
my_f2fs_io-13425 [002] .... 395.583721: f2fs_map_blocks: dev =
(259,23), ino = 40132, file offset = 768, start blkaddr = 0x2d5a4, len =
0x2, err = 0
my_f2fs_io-13425 [002] .... 395.583735: f2fs_map_blocks: dev =
(259,23), ino = 40132, file offset = 768, start blkaddr = 0x2d5a4, len =
0x1, err = 0
my_f2fs_io-13425 [002] .... 395.583741: f2fs_map_blocks: dev =
(259,23), ino = 40132, file offset = 769, start blkaddr = 0x32400, len =
0x1, err = 0

And if the blocks are continuously, the file data is normal:
my_f2fs_io-13425 [002] .... 395.584037: el0_irq_naked:
1:type:1,ino:40132,off:770,old_blk:0,new_blk:205825
my_f2fs_io-13425 [002] .... 395.584066: el0_irq_naked:
1:type:1,ino:40132,off:771,old_blk:0,new_blk:205826
my_f2fs_io-13425 [002] .... 395.584077: f2fs_map_blocks: dev =
(259,23), ino = 40132, file offset = 770, start blkaddr = 0x32401, len =
0x2, err = 0
my_f2fs_io-13425 [002] .... 395.584091: f2fs_map_blocks: dev =
(259,23), ino = 40132, file offset = 770, start blkaddr = 0x32401, len =
0x2, err = 0

>
> Thanks,
>
>>
>>
>> _______________________________________________
>> Linux-f2fs-devel mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
>>
>

2021-08-11 12:59:19

by Chao Yu

[permalink] [raw]
Subject: Re: [f2fs-dev] f2fs do DIO write make file corruption

On 2021/8/11 11:28, Wu Bo wrote:
>
> 在 2021/8/11 11:03, Chao Yu 写道:
>> On 2021/8/11 10:48, Wu Bo wrote:
>>> I use the following command to create a file, the file may got
>>> corruption:
>>>     f2fs_io write 2 0 512 inc_num dio $path
>>>
>>> And when I use bio or to set the chunk size to 1 block, the file is
>>> normal. The commands as following:
>>>     f2fs_io write 2 0 512 inc_num buffered $path
>>>     f2fs_io write 1 0 512 inc_num dio $path
>>>
>>> I find this bug on old kernel version 4.14.117, and not find on version
>>> 4.19.152. So this bug is fixed. Can anyone can tell me which patch fixed
>>> this bug?
>>
>> Not sure,
>>
>> https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=linux-4.19.y&id=7bae8b6b73e46c307fa355ce086800b7ad6610f8

I didn't see this patch in 4.14 stable kernel of mailine, so do you mean 4.14 kernel
maintained by Android?

If so, f2fs codes in between 4.14 and 4.19 Android kernel are almost the same,
see below link:

https://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs-stable.git/

Thanks,

>>
>
> This patch is applied. The issue occurs when f2fs dio try to preallocate
> multiple blocks and got scattered disk blocks. The log as following:
> my_f2fs_io-13425 [002] .... 395.583637: el0_irq_naked:
> 1:type:1,ino:40132,off:768,old_blk:0,new_blk:185764
> my_f2fs_io-13425 [002] .... 395.583710: el0_irq_naked:
> 1:type:1,ino:40132,off:769,old_blk:0,new_blk:205824
> my_f2fs_io-13425 [002] .... 395.583721: f2fs_map_blocks: dev =
> (259,23), ino = 40132, file offset = 768, start blkaddr = 0x2d5a4, len =
> 0x2, err = 0
> my_f2fs_io-13425 [002] .... 395.583735: f2fs_map_blocks: dev =
> (259,23), ino = 40132, file offset = 768, start blkaddr = 0x2d5a4, len =
> 0x1, err = 0
> my_f2fs_io-13425 [002] .... 395.583741: f2fs_map_blocks: dev =
> (259,23), ino = 40132, file offset = 769, start blkaddr = 0x32400, len =
> 0x1, err = 0
>
> And if the blocks are continuously, the file data is normal:
> my_f2fs_io-13425 [002] .... 395.584037: el0_irq_naked:
> 1:type:1,ino:40132,off:770,old_blk:0,new_blk:205825
> my_f2fs_io-13425 [002] .... 395.584066: el0_irq_naked:
> 1:type:1,ino:40132,off:771,old_blk:0,new_blk:205826
> my_f2fs_io-13425 [002] .... 395.584077: f2fs_map_blocks: dev =
> (259,23), ino = 40132, file offset = 770, start blkaddr = 0x32401, len =
> 0x2, err = 0
> my_f2fs_io-13425 [002] .... 395.584091: f2fs_map_blocks: dev =
> (259,23), ino = 40132, file offset = 770, start blkaddr = 0x32401, len =
> 0x2, err = 0
>
>>
>> Thanks,
>>
>>>
>>>
>>> _______________________________________________
>>> Linux-f2fs-devel mailing list
>>> [email protected]
>>> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
>>>
>>
>
>
> _______________________________________________
> Linux-f2fs-devel mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
>

2021-08-12 02:59:08

by Wu Bo

[permalink] [raw]
Subject: Re: [f2fs-dev] f2fs do DIO write make file corruption


在 2021/8/11 20:57, Chao Yu 写道:
> On 2021/8/11 11:28, Wu Bo wrote:
>>
>> 在 2021/8/11 11:03, Chao Yu 写道:
>>> On 2021/8/11 10:48, Wu Bo wrote:
>>>> I use the following command to create a file, the file may got
>>>> corruption:
>>>>      f2fs_io write 2 0 512 inc_num dio $path
>>>>
>>>> And when I use bio or to set the chunk size to 1 block, the file is
>>>> normal. The commands as following:
>>>>      f2fs_io write 2 0 512 inc_num buffered $path
>>>>      f2fs_io write 1 0 512 inc_num dio $path
>>>>
>>>> I find this bug on old kernel version 4.14.117, and not find on version
>>>> 4.19.152. So this bug is fixed. Can anyone can tell me which patch
>>>> fixed
>>>> this bug?
>>>
>>> Not sure,
>>>
>>> https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=linux-4.19.y&id=7bae8b6b73e46c307fa355ce086800b7ad6610f8
>>>
>
> I didn't see this patch in 4.14 stable kernel of mailine, so do you mean
> 4.14 kernel
> maintained by Android?
>
> If so, f2fs codes in between 4.14 and 4.19 Android kernel are almost the
> same,
> see below link:
>
> https://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs-stable.git/
>

Sorry, it's our old Android project code.So the kernel version may have
no reference value.
This project code last sync to Android common kernel is:
9f892496fc0b:f2fs: readahead encrypted block during GC[2018-09-05]
The code link is:
https://android.googlesource.com/kernel/common/+/9f892496fc0b
And after that, we applied some fixup patches.

> Thanks,
>
>>>
>>
>> This patch is applied. The issue occurs when f2fs dio try to preallocate
>> multiple blocks and got scattered disk blocks. The log as following:
>>         my_f2fs_io-13425 [002] ....   395.583637: el0_irq_naked:
>> 1:type:1,ino:40132,off:768,old_blk:0,new_blk:185764
>>         my_f2fs_io-13425 [002] ....   395.583710: el0_irq_naked:
>> 1:type:1,ino:40132,off:769,old_blk:0,new_blk:205824
>>         my_f2fs_io-13425 [002] ....   395.583721: f2fs_map_blocks: dev =
>> (259,23), ino = 40132, file offset = 768, start blkaddr = 0x2d5a4, len =
>> 0x2, err = 0
>>         my_f2fs_io-13425 [002] ....   395.583735: f2fs_map_blocks: dev =
>> (259,23), ino = 40132, file offset = 768, start blkaddr = 0x2d5a4, len =
>> 0x1, err = 0
>>         my_f2fs_io-13425 [002] ....   395.583741: f2fs_map_blocks: dev =
>> (259,23), ino = 40132, file offset = 769, start blkaddr = 0x32400, len =
>> 0x1, err = 0
>>
>> And if the blocks are continuously, the file data is normal:
>>       my_f2fs_io-13425 [002] ....   395.584037: el0_irq_naked:
>> 1:type:1,ino:40132,off:770,old_blk:0,new_blk:205825
>>         my_f2fs_io-13425 [002] ....   395.584066: el0_irq_naked:
>> 1:type:1,ino:40132,off:771,old_blk:0,new_blk:205826
>>         my_f2fs_io-13425 [002] ....   395.584077: f2fs_map_blocks: dev =
>> (259,23), ino = 40132, file offset = 770, start blkaddr = 0x32401, len =
>> 0x2, err = 0
>>         my_f2fs_io-13425 [002] ....   395.584091: f2fs_map_blocks: dev =
>> (259,23), ino = 40132, file offset = 770, start blkaddr = 0x32401, len =
>> 0x2, err = 0
>>
>>>
>>> Thanks,
>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Linux-f2fs-devel mailing list
>>>> [email protected]
>>>> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
>>>>
>>>
>>
>>
>> _______________________________________________
>> Linux-f2fs-devel mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
>>
>

2021-08-12 12:58:58

by Chao Yu

[permalink] [raw]
Subject: Re: [f2fs-dev] f2fs do DIO write make file corruption

On 2021/8/12 10:55, Wu Bo wrote:
>
> 在 2021/8/11 20:57, Chao Yu 写道:
>> On 2021/8/11 11:28, Wu Bo wrote:
>>>
>>> 在 2021/8/11 11:03, Chao Yu 写道:
>>>> On 2021/8/11 10:48, Wu Bo wrote:
>>>>> I use the following command to create a file, the file may got
>>>>> corruption:
>>>>>      f2fs_io write 2 0 512 inc_num dio $path
>>>>>
>>>>> And when I use bio or to set the chunk size to 1 block, the file is
>>>>> normal. The commands as following:
>>>>>      f2fs_io write 2 0 512 inc_num buffered $path
>>>>>      f2fs_io write 1 0 512 inc_num dio $path
>>>>>
>>>>> I find this bug on old kernel version 4.14.117, and not find on version
>>>>> 4.19.152. So this bug is fixed. Can anyone can tell me which patch
>>>>> fixed
>>>>> this bug?
>>>>
>>>> Not sure,
>>>>
>>>> https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=linux-4.19.y&id=7bae8b6b73e46c307fa355ce086800b7ad6610f8
>>>>
>>
>> I didn't see this patch in 4.14 stable kernel of mailine, so do you mean
>> 4.14 kernel
>> maintained by Android?
>>
>> If so, f2fs codes in between 4.14 and 4.19 Android kernel are almost the
>> same,
>> see below link:
>>
>> https://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs-stable.git/
>>
>
> Sorry, it's our old Android project code.So the kernel version may have
> no reference value.
> This project code last sync to Android common kernel is:
> 9f892496fc0b:f2fs: readahead encrypted block during GC[2018-09-05]
> The code link is:
> https://android.googlesource.com/kernel/common/+/9f892496fc0b
> And after that, we applied some fixup patches.

I think you can pick up patches from your 4.19 f2fs codes which are not in
your 4.14 f2fs codes, and check each patch you picked to see whether there
is a patch can fix that problem.

Thanks,

>
>> Thanks,
>>
>>>>
>>>
>>> This patch is applied. The issue occurs when f2fs dio try to preallocate
>>> multiple blocks and got scattered disk blocks. The log as following:
>>>         my_f2fs_io-13425 [002] ....   395.583637: el0_irq_naked:
>>> 1:type:1,ino:40132,off:768,old_blk:0,new_blk:185764
>>>         my_f2fs_io-13425 [002] ....   395.583710: el0_irq_naked:
>>> 1:type:1,ino:40132,off:769,old_blk:0,new_blk:205824
>>>         my_f2fs_io-13425 [002] ....   395.583721: f2fs_map_blocks: dev =
>>> (259,23), ino = 40132, file offset = 768, start blkaddr = 0x2d5a4, len =
>>> 0x2, err = 0
>>>         my_f2fs_io-13425 [002] ....   395.583735: f2fs_map_blocks: dev =
>>> (259,23), ino = 40132, file offset = 768, start blkaddr = 0x2d5a4, len =
>>> 0x1, err = 0
>>>         my_f2fs_io-13425 [002] ....   395.583741: f2fs_map_blocks: dev =
>>> (259,23), ino = 40132, file offset = 769, start blkaddr = 0x32400, len =
>>> 0x1, err = 0
>>>
>>> And if the blocks are continuously, the file data is normal:
>>>       my_f2fs_io-13425 [002] ....   395.584037: el0_irq_naked:
>>> 1:type:1,ino:40132,off:770,old_blk:0,new_blk:205825
>>>         my_f2fs_io-13425 [002] ....   395.584066: el0_irq_naked:
>>> 1:type:1,ino:40132,off:771,old_blk:0,new_blk:205826
>>>         my_f2fs_io-13425 [002] ....   395.584077: f2fs_map_blocks: dev =
>>> (259,23), ino = 40132, file offset = 770, start blkaddr = 0x32401, len =
>>> 0x2, err = 0
>>>         my_f2fs_io-13425 [002] ....   395.584091: f2fs_map_blocks: dev =
>>> (259,23), ino = 40132, file offset = 770, start blkaddr = 0x32401, len =
>>> 0x2, err = 0
>>>
>>>>
>>>> Thanks,
>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Linux-f2fs-devel mailing list
>>>>> [email protected]
>>>>> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
>>>>>
>>>>
>>>
>>>
>>> _______________________________________________
>>> Linux-f2fs-devel mailing list
>>> [email protected]
>>> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
>>>
>>