From: Tao Ma <[email protected]>
Now we can use fallocate to create a large file while keep the size
to be small. It will cause the e2fsck complain about it. The test
script is simple and I have pasted it here.
DEVICE=/dev/sdb1
mount -t ext4 $DEVICE /mnt/ext4
for((i=0;i<10;i++))do fallocate -n -o $[$i*8192] -l 4096 /mnt/ext4/a;done
umount $DEVICE
e2fsck -fn $DEVICE
The error message will be like this:
e2fsck 1.42.3 (14-May-2012)
Pass 1: Checking inodes, blocks, and sizes
Inode 12 has zero length extent
(invalid logical block 0, physical block 32775)
Clear? no
Inode 12, i_blocks is 88, should be 0. Fix? no
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
Block bitmap differences: -(8231--8232) -(32770--32778)
Fix? no
Now actually the end_blk can be any value which is less than
u32, so make end_blk be the maximum value of u32.
Cc: Theodore Ts'o <[email protected]>
Signed-off-by: Tao Ma <[email protected]>
---
lib/ext2fs/extent.c | 4 +---
1 files changed, 1 insertions(+), 3 deletions(-)
diff --git a/lib/ext2fs/extent.c b/lib/ext2fs/extent.c
index eb096d6..e2815c2 100644
--- a/lib/ext2fs/extent.c
+++ b/lib/ext2fs/extent.c
@@ -253,9 +253,7 @@ extern errcode_t ext2fs_extent_open2(ext2_filsys fs, ext2_ino_t ino,
ext2fs_le16_to_cpu(eh->eh_entries);
handle->path[0].max_entries = ext2fs_le16_to_cpu(eh->eh_max);
handle->path[0].curr = 0;
- handle->path[0].end_blk =
- (EXT2_I_SIZE(handle->inode) + fs->blocksize - 1) >>
- EXT2_BLOCK_SIZE_BITS(fs->super);
+ handle->path[0].end_blk = ((((unsigned long long) 1) << 32) - 1);
handle->path[0].visit_num = 1;
handle->level = 0;
handle->magic = EXT2_ET_MAGIC_EXTENT_HANDLE;
--
1.7.1
On 5/16/12 3:50 AM, Tao Ma wrote:
> From: Tao Ma <[email protected]>
>
> Now we can use fallocate to create a large file while keep the size
> to be small. It will cause the e2fsck complain about it. The test
> script is simple and I have pasted it here.
>
> DEVICE=/dev/sdb1
> mount -t ext4 $DEVICE /mnt/ext4
> for((i=0;i<10;i++))do fallocate -n -o $[$i*8192] -l 4096 /mnt/ext4/a;done
> umount $DEVICE
> e2fsck -fn $DEVICE
Should this be put into an e2fsprogs regression test?
> The error message will be like this:
> e2fsck 1.42.3 (14-May-2012)
> Pass 1: Checking inodes, blocks, and sizes
> Inode 12 has zero length extent
> (invalid logical block 0, physical block 32775)
> Clear? no
>
> Inode 12, i_blocks is 88, should be 0. Fix? no
>
> Pass 2: Checking directory structure
> Pass 3: Checking directory connectivity
> Pass 4: Checking reference counts
> Pass 5: Checking group summary information
> Block bitmap differences: -(8231--8232) -(32770--32778)
> Fix? no
>
> Now actually the end_blk can be any value which is less than
> u32, so make end_blk be the maximum value of u32.
>
> Cc: Theodore Ts'o <[email protected]>
> Signed-off-by: Tao Ma <[email protected]>
> ---
> lib/ext2fs/extent.c | 4 +---
> 1 files changed, 1 insertions(+), 3 deletions(-)
>
> diff --git a/lib/ext2fs/extent.c b/lib/ext2fs/extent.c
> index eb096d6..e2815c2 100644
> --- a/lib/ext2fs/extent.c
> +++ b/lib/ext2fs/extent.c
> @@ -253,9 +253,7 @@ extern errcode_t ext2fs_extent_open2(ext2_filsys fs, ext2_ino_t ino,
> ext2fs_le16_to_cpu(eh->eh_entries);
> handle->path[0].max_entries = ext2fs_le16_to_cpu(eh->eh_max);
> handle->path[0].curr = 0;
> - handle->path[0].end_blk =
> - (EXT2_I_SIZE(handle->inode) + fs->blocksize - 1) >>
> - EXT2_BLOCK_SIZE_BITS(fs->super);
Hm, so this picked the actual last block of the file, whereas
> + handle->path[0].end_blk = ((((unsigned long long) 1) << 32) - 1);
this gives it an upper bound... why is that ok? It's been a long time since
I looked at this code, but some explanation in the commit and in code
comments would be helpful.
If end_blk can be any value less than u32, what is its purpose?
-Eric
> handle->path[0].visit_num = 1;
> handle->level = 0;
> handle->magic = EXT2_ET_MAGIC_EXTENT_HANDLE;
On 05/16/2012 10:04 PM, Eric Sandeen wrote:
> On 5/16/12 3:50 AM, Tao Ma wrote:
>> From: Tao Ma <[email protected]>
>>
>> Now we can use fallocate to create a large file while keep the size
>> to be small. It will cause the e2fsck complain about it. The test
>> script is simple and I have pasted it here.
>>
>> DEVICE=/dev/sdb1
>> mount -t ext4 $DEVICE /mnt/ext4
>> for((i=0;i<10;i++))do fallocate -n -o $[$i*8192] -l 4096 /mnt/ext4/a;done
>> umount $DEVICE
>> e2fsck -fn $DEVICE
>
> Should this be put into an e2fsprogs regression test?
sure, but could you please tell me where I can find the repo?
>
>> The error message will be like this:
>> e2fsck 1.42.3 (14-May-2012)
>> Pass 1: Checking inodes, blocks, and sizes
>> Inode 12 has zero length extent
>> (invalid logical block 0, physical block 32775)
>> Clear? no
>>
>> Inode 12, i_blocks is 88, should be 0. Fix? no
>>
>> Pass 2: Checking directory structure
>> Pass 3: Checking directory connectivity
>> Pass 4: Checking reference counts
>> Pass 5: Checking group summary information
>> Block bitmap differences: -(8231--8232) -(32770--32778)
>> Fix? no
>>
>> Now actually the end_blk can be any value which is less than
>> u32, so make end_blk be the maximum value of u32.
>>
>> Cc: Theodore Ts'o <[email protected]>
>> Signed-off-by: Tao Ma <[email protected]>
>> ---
>> lib/ext2fs/extent.c | 4 +---
>> 1 files changed, 1 insertions(+), 3 deletions(-)
>>
>> diff --git a/lib/ext2fs/extent.c b/lib/ext2fs/extent.c
>> index eb096d6..e2815c2 100644
>> --- a/lib/ext2fs/extent.c
>> +++ b/lib/ext2fs/extent.c
>> @@ -253,9 +253,7 @@ extern errcode_t ext2fs_extent_open2(ext2_filsys fs, ext2_ino_t ino,
>> ext2fs_le16_to_cpu(eh->eh_entries);
>> handle->path[0].max_entries = ext2fs_le16_to_cpu(eh->eh_max);
>> handle->path[0].curr = 0;
>> - handle->path[0].end_blk =
>> - (EXT2_I_SIZE(handle->inode) + fs->blocksize - 1) >>
>> - EXT2_BLOCK_SIZE_BITS(fs->super);
>
> Hm, so this picked the actual last block of the file, whereas
No, it doesn't. With fallocate(FALLOC_FL_KEEP_SIZE), we have no idea of
what is the last block until we iterate the last leaf ext4_extent.
>
>> + handle->path[0].end_blk = ((((unsigned long long) 1) << 32) - 1);
>
> this gives it an upper bound... why is that ok? It's been a long time since
> I looked at this code, but some explanation in the commit and in code
> comments would be helpful.
>
> If end_blk can be any value less than u32, what is its purpose?
As I have mentioned above, now there is no way for us to tell the end
block of a file at the very beginning of ext2fs_extent_open2, so
actually any value less than u32 could be OK if we have a sparse file
while the last block is fallocated near the end of u32 logical block
offset. Actually path[0]->end_blk is only used when we have no idea of
the length of the last ext4_extent_idx. See ext2fs_extent_get.
if (path->left > 0) {
ix++;
newpath->end_blk = ext2fs_le32_to_cpu(ix->ei_block);
} else
newpath->end_blk = path->end_blk;
Having said that, I have to admit that I didn't think of the case of
ext3 and I am not sure whether this change will affect it or not.
Thanks
Tao
>
> -Eric
>
>> handle->path[0].visit_num = 1;
>> handle->level = 0;
>> handle->magic = EXT2_ET_MAGIC_EXTENT_HANDLE;
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
On 5/16/12 10:07 AM, Tao Ma wrote:
> On 05/16/2012 10:04 PM, Eric Sandeen wrote:
>> On 5/16/12 3:50 AM, Tao Ma wrote:
>>> From: Tao Ma <[email protected]>
>>>
>>> Now we can use fallocate to create a large file while keep the size
>>> to be small. It will cause the e2fsck complain about it. The test
>>> script is simple and I have pasted it here.
>>>
>>> DEVICE=/dev/sdb1
>>> mount -t ext4 $DEVICE /mnt/ext4
>>> for((i=0;i<10;i++))do fallocate -n -o $[$i*8192] -l 4096 /mnt/ext4/a;done
>>> umount $DEVICE
>>> e2fsck -fn $DEVICE
>>
>> Should this be put into an e2fsprogs regression test?
> sure, but could you please tell me where I can find the repo?
there are regression tests in the e2fsprogs git tree itself.
I think a prepared filesytem image may be the way to go for this one.
>>> The error message will be like this:
>>> e2fsck 1.42.3 (14-May-2012)
>>> Pass 1: Checking inodes, blocks, and sizes
>>> Inode 12 has zero length extent
>>> (invalid logical block 0, physical block 32775)
>>> Clear? no
>>>
>>> Inode 12, i_blocks is 88, should be 0. Fix? no
>>>
>>> Pass 2: Checking directory structure
>>> Pass 3: Checking directory connectivity
>>> Pass 4: Checking reference counts
>>> Pass 5: Checking group summary information
>>> Block bitmap differences: -(8231--8232) -(32770--32778)
>>> Fix? no
>>>
>>> Now actually the end_blk can be any value which is less than
>>> u32, so make end_blk be the maximum value of u32.
>>>
>>> Cc: Theodore Ts'o <[email protected]>
>>> Signed-off-by: Tao Ma <[email protected]>
>>> ---
>>> lib/ext2fs/extent.c | 4 +---
>>> 1 files changed, 1 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/lib/ext2fs/extent.c b/lib/ext2fs/extent.c
>>> index eb096d6..e2815c2 100644
>>> --- a/lib/ext2fs/extent.c
>>> +++ b/lib/ext2fs/extent.c
>>> @@ -253,9 +253,7 @@ extern errcode_t ext2fs_extent_open2(ext2_filsys fs, ext2_ino_t ino,
>>> ext2fs_le16_to_cpu(eh->eh_entries);
>>> handle->path[0].max_entries = ext2fs_le16_to_cpu(eh->eh_max);
>>> handle->path[0].curr = 0;
>>> - handle->path[0].end_blk =
>>> - (EXT2_I_SIZE(handle->inode) + fs->blocksize - 1) >>
>>> - EXT2_BLOCK_SIZE_BITS(fs->super);
>>
>> Hm, so this picked the actual last block of the file, whereas
> No, it doesn't. With fallocate(FALLOC_FL_KEEP_SIZE), we have no idea of
> what is the last block until we iterate the last leaf ext4_extent.
Oh, sorry - this "tried to pick" is what I meant to say :)
>>> + handle->path[0].end_blk = ((((unsigned long long) 1) << 32) - 1);
>>
>> this gives it an upper bound... why is that ok? It's been a long time since
>> I looked at this code, but some explanation in the commit and in code
>> comments would be helpful.
>>
>> If end_blk can be any value less than u32, what is its purpose?
> As I have mentioned above, now there is no way for us to tell the end
> block of a file at the very beginning of ext2fs_extent_open2, so
> actually any value less than u32 could be OK if we have a sparse file
> while the last block is fallocated near the end of u32 logical block
> offset. Actually path[0]->end_blk is only used when we have no idea of
> the length of the last ext4_extent_idx. See ext2fs_extent_get.
Ok. Thanks.
-Eric
> if (path->left > 0) {
> ix++;
> newpath->end_blk = ext2fs_le32_to_cpu(ix->ei_block);
> } else
> newpath->end_blk = path->end_blk;
>
> Having said that, I have to admit that I didn't think of the case of
> ext3 and I am not sure whether this change will affect it or not.
>
> Thanks
> Tao
>>
>> -Eric
>>
>>> handle->path[0].visit_num = 1;
>>> handle->level = 0;
>>> handle->magic = EXT2_ET_MAGIC_EXTENT_HANDLE;
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
>> the body of a message to [email protected]
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
On 2012-05-16, at 2:50 AM, Tao Ma wrote:
> Now we can use fallocate to create a large file while keep the size
> to be small. It will cause the e2fsck complain about it. The test
> script is simple and I have pasted it here.
We had previously added the EXT4_EOFBLOCKS_FL to distinguish the
case between "corrupt i_size should be extended to match i_blocks"
and "valid fallocate case of blocks allocated beyond i_size".
This was removed recently, but I didn't really understand the reason
why it was removed.
Cheers, Andreas
> DEVICE=/dev/sdb1
> mount -t ext4 $DEVICE /mnt/ext4
> for((i=0;i<10;i++))do fallocate -n -o $[$i*8192] -l 4096 /mnt/ext4/a;done
> umount $DEVICE
> e2fsck -fn $DEVICE
>
> The error message will be like this:
> e2fsck 1.42.3 (14-May-2012)
> Pass 1: Checking inodes, blocks, and sizes
> Inode 12 has zero length extent
> (invalid logical block 0, physical block 32775)
> Clear? no
>
> Inode 12, i_blocks is 88, should be 0. Fix? no
>
> Pass 2: Checking directory structure
> Pass 3: Checking directory connectivity
> Pass 4: Checking reference counts
> Pass 5: Checking group summary information
> Block bitmap differences: -(8231--8232) -(32770--32778)
> Fix? no
>
> Now actually the end_blk can be any value which is less than
> u32, so make end_blk be the maximum value of u32.
>
> Cc: Theodore Ts'o <[email protected]>
> Signed-off-by: Tao Ma <[email protected]>
> ---
> lib/ext2fs/extent.c | 4 +---
> 1 files changed, 1 insertions(+), 3 deletions(-)
>
> diff --git a/lib/ext2fs/extent.c b/lib/ext2fs/extent.c
> index eb096d6..e2815c2 100644
> --- a/lib/ext2fs/extent.c
> +++ b/lib/ext2fs/extent.c
> @@ -253,9 +253,7 @@ extern errcode_t ext2fs_extent_open2(ext2_filsys fs, ext2_ino_t ino,
> ext2fs_le16_to_cpu(eh->eh_entries);
> handle->path[0].max_entries = ext2fs_le16_to_cpu(eh->eh_max);
> handle->path[0].curr = 0;
> - handle->path[0].end_blk =
> - (EXT2_I_SIZE(handle->inode) + fs->blocksize - 1) >>
> - EXT2_BLOCK_SIZE_BITS(fs->super);
> + handle->path[0].end_blk = ((((unsigned long long) 1) << 32) - 1);
> handle->path[0].visit_num = 1;
> handle->level = 0;
> handle->magic = EXT2_ET_MAGIC_EXTENT_HANDLE;
> --
> 1.7.1
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
Cheers, Andreas