Hi Ted and other ext4 gurus,
I found fiemap may be broken for sparse files in ext4. Here is a simple
example.
dd if=/dev/zero of=testfile1 bs=1M count=1
using fiemap shows that it has a delalloc extent.
Logical: 0 Ext length: 1048576 Physical: 0 flags: 7
flags 7 means FIEMAP_EXTENT_LAST, FIEMAP_EXTENT_UNKNOWN and
FIEMAP_EXTENT_DELALLOC,
while if we create a sparse file, fiemap will not show the delalloc extent.
dd if=/dev/zero of=testfile1 bs=1M count=1 seek=1
using fiemap shows that it has no extent for the file. while we should
have some output like:
Logical: 1048576 Ext length: 1048576 Physical: 0 flags: 7
So we have different output with sparse and non-sparse file. Is it a bug
for ext4?
Regards,
Tao
Tao Ma wrote:
> Hi Ted and other ext4 gurus,
> I found fiemap may be broken for sparse files in ext4. Here is a
> simple example.
>
> dd if=/dev/zero of=testfile1 bs=1M count=1
> using fiemap shows that it has a delalloc extent.
> Logical: 0 Ext length: 1048576 Physical: 0 flags: 7
>
> flags 7 means FIEMAP_EXTENT_LAST, FIEMAP_EXTENT_UNKNOWN and
> FIEMAP_EXTENT_DELALLOC,
>
> while if we create a sparse file, fiemap will not show the delalloc extent.
> dd if=/dev/zero of=testfile1 bs=1M count=1 seek=1
> using fiemap shows that it has no extent for the file. while we should
> have some output like:
> Logical: 1048576 Ext length: 1048576 Physical: 0 flags: 7
>
> So we have different output with sparse and non-sparse file. Is it a bug
> for ext4?
What are you using to call fiemap? Here it seems to be working:
# dd if=/dev/zero of=testfile1 bs=1M count=1 seek=1;
# filefrag -v testfile1
Filesystem type is: ef53
Filesystem cylinder groups is approximately 119
File size of testfile1 is 2097152 (512 blocks, blocksize 4096)
ext logical physical expected length flags
0 256 151946 1 merged
1 257 151951 151946 2 merged
2 259 152434 151952 253 merged,eof
testfile1: 4 extents found
And with my homegrown fiemap tester:
# ./fiemap-test testfile1
ext: 0 logical: [ 256.. 256] phys: 151946.. 151946 flags: 0x1000 tot: 1
ext: 1 logical: [ 257.. 258] phys: 151951.. 151952 flags: 0x1000 tot: 2
ext: 2 logical: [ 259.. 511] phys: 152434.. 152686 flags: 0x1001 tot: 253
... seems ok here, aside from the fragmentation :) ...
-Eric
Hi Eric,
Thanks for the quick response.
On 06/11/2010 12:08 PM, Eric Sandeen wrote:
> Tao Ma wrote:
>> Hi Ted and other ext4 gurus,
>> I found fiemap may be broken for sparse files in ext4. Here is a
>> simple example.
>>
>> dd if=/dev/zero of=testfile1 bs=1M count=1
>> using fiemap shows that it has a delalloc extent.
>> Logical: 0 Ext length: 1048576 Physical: 0 flags: 7
>>
>> flags 7 means FIEMAP_EXTENT_LAST, FIEMAP_EXTENT_UNKNOWN and
>> FIEMAP_EXTENT_DELALLOC,
>>
>> while if we create a sparse file, fiemap will not show the delalloc extent.
>> dd if=/dev/zero of=testfile1 bs=1M count=1 seek=1
>> using fiemap shows that it has no extent for the file. while we should
>> have some output like:
>> Logical: 1048576 Ext length: 1048576 Physical: 0 flags: 7
>>
>> So we have different output with sparse and non-sparse file. Is it a bug
>> for ext4?
>
> What are you using to call fiemap? Here it seems to be working:
I just wrote a simple test program by calling ioctl. It is attached.
btw, you need to call it immediately after dd so that we have a chance
that ext4 don't have time to allocate extents. ;)
>
> # dd if=/dev/zero of=testfile1 bs=1M count=1 seek=1;
>
> # filefrag -v testfile1
> Filesystem type is: ef53
> Filesystem cylinder groups is approximately 119
> File size of testfile1 is 2097152 (512 blocks, blocksize 4096)
> ext logical physical expected length flags
> 0 256 151946 1 merged
> 1 257 151951 151946 2 merged
> 2 259 152434 151952 253 merged,eof
> testfile1: 4 extents found
I guess maybe filefrag use the diffrent ioctl flag, maybe
FIEMAP_FLAG_SYNC to let ext4 sync first.
Regards,
Tao
Tao Ma wrote:
> Hi Eric,
> Thanks for the quick response.
>
> On 06/11/2010 12:08 PM, Eric Sandeen wrote:
...
>> What are you using to call fiemap? Here it seems to be working:
> I just wrote a simple test program by calling ioctl. It is attached.
> btw, you need to call it immediately after dd so that we have a chance
> that ext4 don't have time to allocate extents. ;)
>>
>> # dd if=/dev/zero of=testfile1 bs=1M count=1 seek=1;
>>
>> # filefrag -v testfile1
>> Filesystem type is: ef53
>> Filesystem cylinder groups is approximately 119
>> File size of testfile1 is 2097152 (512 blocks, blocksize 4096)
>> ext logical physical expected length flags
>> 0 256 151946 1 merged
>> 1 257 151951 151946 2 merged
>> 2 259 152434 151952 253 merged,eof
>> testfile1: 4 extents found
> I guess maybe filefrag use the diffrent ioctl flag, maybe
> FIEMAP_FLAG_SYNC to let ext4 sync first.
My tester isn't calling sync or using the sync flag, AFAIK.
I'll take a look at yours.
Thanks,
-Eric
> Regards,
> Tao
>
Tao Ma wrote:
>> What are you using to call fiemap? Here it seems to be working:
> I just wrote a simple test program by calling ioctl. It is attached.
> btw, you need to call it immediately after dd so that we have a chance
> that ext4 don't have time to allocate extents. ;)
OK I take it back, I do see it. *cough* too many different filesystems
on this box ... ;)
Yes, it does look like a bug.
-Eric
On Fri, Jun 11, 2010 at 11:46 AM, Eric Sandeen <[email protected]> wrote:
> Tao Ma wrote:
>
>>> What are you using to call fiemap? ?Here it seems to be working:
>> I just wrote a simple test program by calling ioctl. It is attached.
>> btw, you need to call it immediately after dd so that we have a chance
>> that ext4 don't have time to allocate extents. ;)
>
> OK I take it back, I do see it. ?*cough* too many different filesystems
> on this box ... ;)
>
> Yes, it does look like a bug.
>
> -Eric
I'm not clear how this is supposed to work.
Is this just a userspace bug in that FIEMAP_FLAG_SYNC is needed in the
fiemap call, or is there a kernel bug as well?
If fiemap is called with the FIEMAP_FLAG_SYNC flag is there currently a problem?
Thanks
Greg
Greg Freemyer wrote:
> On Fri, Jun 11, 2010 at 11:46 AM, Eric Sandeen <[email protected]> wrote:
>> Tao Ma wrote:
>>
>>>> What are you using to call fiemap? Here it seems to be working:
>>> I just wrote a simple test program by calling ioctl. It is attached.
>>> btw, you need to call it immediately after dd so that we have a chance
>>> that ext4 don't have time to allocate extents. ;)
>> OK I take it back, I do see it. *cough* too many different filesystems
>> on this box ... ;)
>>
>> Yes, it does look like a bug.
>>
>> -Eric
>
> I'm not clear how this is supposed to work.
>
> Is this just a userspace bug in that FIEMAP_FLAG_SYNC is needed in the
> fiemap call, or is there a kernel bug as well?
The flag is optional, though maybe filefrag should use it.
Without it, we should get the proper logical offset and a delalloc-flagged
extent returned
> If fiemap is called with the FIEMAP_FLAG_SYNC flag is there currently a problem?
yep, we should get one delalloc extent in the results and we don't.
-Eric
> Thanks
> Greg
Hi Eric,
Eric Sandeen wrote:
> Greg Freemyer wrote:
>
>> On Fri, Jun 11, 2010 at 11:46 AM, Eric Sandeen <[email protected]> wrote:
>>
>>> Tao Ma wrote:
>>>
>>>
>>>>> What are you using to call fiemap? Here it seems to be working:
>>>>>
>>>> I just wrote a simple test program by calling ioctl. It is attached.
>>>> btw, you need to call it immediately after dd so that we have a chance
>>>> that ext4 don't have time to allocate extents. ;)
>>>>
>>> OK I take it back, I do see it. *cough* too many different filesystems
>>> on this box ... ;)
>>>
>>> Yes, it does look like a bug.
>>>
>>> -Eric
>>>
>> I'm not clear how this is supposed to work.
>>
>> Is this just a userspace bug in that FIEMAP_FLAG_SYNC is needed in the
>> fiemap call, or is there a kernel bug as well?
>>
>
> The flag is optional, though maybe filefrag should use it.
>
> Without it, we should get the proper logical offset and a delalloc-flagged
> extent returned
>
>
>> If fiemap is called with the FIEMAP_FLAG_SYNC flag is there currently a problem?
>>
>
> yep, we should get one delalloc extent in the results and we don't.
>
With FIEMAP_FLAG_SYNC set, my test shows that we get one extent without
dealloc flags. So no problem with it.
We have another different test result? ;)
Regards,
Tao
Tao Ma wrote:
> Hi Eric,
> Eric Sandeen wrote:
>> Greg Freemyer wrote:
>>
>>> On Fri, Jun 11, 2010 at 11:46 AM, Eric Sandeen <[email protected]>
>>> wrote:
>>>
>>>> Tao Ma wrote:
>>>>
>>>>
>>>>>> What are you using to call fiemap? Here it seems to be working:
>>>>>>
>>>>> I just wrote a simple test program by calling ioctl. It is attached.
>>>>> btw, you need to call it immediately after dd so that we have a chance
>>>>> that ext4 don't have time to allocate extents. ;)
>>>>>
>>>> OK I take it back, I do see it. *cough* too many different filesystems
>>>> on this box ... ;)
>>>>
>>>> Yes, it does look like a bug.
>>>>
>>>> -Eric
>>>>
>>> I'm not clear how this is supposed to work.
>>>
>>> Is this just a userspace bug in that FIEMAP_FLAG_SYNC is needed in the
>>> fiemap call, or is there a kernel bug as well?
>>>
>>
>> The flag is optional, though maybe filefrag should use it.
>>
>> Without it, we should get the proper logical offset and a
>> delalloc-flagged
>> extent returned
>>
>>
>>> If fiemap is called with the FIEMAP_FLAG_SYNC flag is there currently
>>> a problem?
>>>
>>
>> yep, we should get one delalloc extent in the results and we don't.
>>
> With FIEMAP_FLAG_SYNC set, my test shows that we get one extent without
> dealloc flags. So no problem with it.
> We have another different test result? ;)
No, I misread. :( Calling with FIEMAP_FLAG_SYNC works as expected.
-Eric
> Regards,
> Tao
>