2009-08-19 22:05:46

by Marcin Miroslaw

[permalink] [raw]
Subject: fsck.ext4 -D corrupts fs?

Hello,
I've noticed fs curruption after using fsck.ext4 -D
## uname -a
Linux dom 2.6.30-gentoo-r5 #1 SMP Wed Aug 19 20:38:18 CEST 2009 x86_64
Intel(R) Core(TM)2 CPU 4300 @ 1.80GHz GenuineIntel GNU/Linux


# mkfs.ext4 -q -L tmp2 -i 2048 -m 2 /dev/sdb11
# mount /dev/sdb11 /mnt/gentoo/
# rsync -aX / /mnt/gentoo/
# umount /mnt/gentoo
# LANGUAGE="en_US" fsck.ext4 -f /dev/sdb11
e2fsck 1.41.8 (11-July-2009)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
tmp2: 216739/3909120 files (0.2% non-contiguous), 1335716/1953897 blocks

Everything is OK until next:
# LANGUAGE="en_US" fsck.ext4 -f -D /dev/sdb11
e2fsck 1.41.8 (11-July-2009)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 3A: Optimizing directories
Failed to optimize directory /usr/share/man/man3 (96019): Extent not found
Pass 4: Checking reference counts
Pass 5: Checking group summary information
Free blocks count wrong for group #0 (4, counted=3).
Fix<y>? yes

Free blocks count wrong for group #2 (5, counted=3).
Fix<y>? yes

Free blocks count wrong for group #4 (4, counted=3).
Fix<y>? yes

Free blocks count wrong for group #9 (6, counted=5).
Fix<y>? yes

Free blocks count wrong for group #11 (15, counted=13).
Fix<y>? yes

Free blocks count wrong (618219, counted=618212).
Fix<y>? yes


tmp2: ***** FILE SYSTEM WAS MODIFIED *****
tmp2: 216739/3909120 files (0.2% non-contiguous), 1335685/1953897 blocks

Fsck can't fix fs now (even run many times):

# LANGUAGE="en_US" fsck.ext4 -fy /dev/sdb11
e2fsck 1.41.8 (11-July-2009)
Pass 1: Checking inodes, blocks, and sizes
HTREE directory inode 43092 has an invalid root node.
Clear HTree index? yes

HTREE directory inode 89999 has an invalid root node.
Clear HTree index? yes

Inode 96019, i_size is 208896, should be 225280. Fix? yes

Inode 96019, i_blocks is 496, should be 416. Fix? yes

HTREE directory inode 260626 has an invalid root node.
Clear HTree index? yes

HTREE directory inode 260630 has an invalid root node.
Clear HTree index? yes

HTREE directory inode 293397 has an invalid root node.
Clear HTree index? yes

Pass 2: Checking directory structure
Problem in HTREE directory inode 96019: node (51) not referenced
Problem in HTREE directory inode 96019: node (52) not referenced
Problem in HTREE directory inode 96019: node (53) not referenced
Problem in HTREE directory inode 96019: node (54) not referenced
Invalid HTREE directory inode 96019 (/usr/share/man/man3). Clear HTree
index? yes

Pass 3: Checking directory connectivity
'..' in .../python2.6/test (43092) is <The NULL inode> (0), should be
.../python2.6 (260836).
Fix? yes


Couldn't fix parent of inode 43092: Couldn't find parent directory entry

'..' in /usr/share/idl/xulrunner-1.9.1.2/unstable (89999) is <The NULL
inode> (0), should be /usr/share/idl/xulrunner-1.9.1.2 (89997).
Fix? yes


Couldn't fix parent of inode 89999: Couldn't find parent directory entry

'..' in /usr/bin (260626) is <The NULL inode> (0), should be /usr (260610).
Fix? yes

Couldn't fix parent of inode 260626: Couldn't find parent directory entry

'..' in /usr/lib64 (260630) is <The NULL inode> (0), should be /usr
(260610).
Fix? yes


Couldn't fix parent of inode 260630: Couldn't find parent directory entry

'..' in .../mozilla-thunderbird/idl (293397) is <The NULL inode> (0),
should be .../mozilla-thunderbird (260823).
Fix? yes

Couldn't fix parent of inode 293397: Couldn't find parent directory entry

Pass 3A: Optimizing directories
Pass 4: Checking reference counts
Inode 43092 ref count is 3, should be 2. Fix? yes

Inode 89997 ref count is 4, should be 3. Fix? yes

Inode 89999 ref count is 2, should be 1. Fix? yes

Inode 260610 ref count is 14, should be 12. Fix? yes

Inode 260626 ref count is 2, should be 1. Fix? yes

Inode 260630 ref count is 73, should be 72. Fix? yes

Inode 260823 ref count is 16, should be 15. Fix? yes

Inode 260836 ref count is 23, should be 22. Fix? yes

Inode 293397 ref count is 2, should be 1. Fix? yes

Pass 5: Checking group summary information
Block bitmap differences: -15480 -15484 -43618 -70438 -158694 -180514
-180585
Fix? yes

Free blocks count wrong for group #0 (3, counted=5).
Fix? yes

Free blocks count wrong for group #2 (3, counted=4).
Fix? yes

Free blocks count wrong for group #4 (3, counted=4).
Fix? yes

Free blocks count wrong for group #9 (5, counted=6).
Fix? yes

Free blocks count wrong for group #11 (13, counted=15).
Fix? yes

Free blocks count wrong (618212, counted=618219).
Fix? yes


tmp2: ***** FILE SYSTEM WAS MODIFIED *****

tmp2: ********** WARNING: Filesystem still has errors **********

tmp2: 216739/3909120 files (0.2% non-contiguous), 1335678/1953897 blocks


May i do something more to help You fix this problem?
With regards
Marcin Miroslaw



2009-08-20 02:44:22

by Theodore Ts'o

[permalink] [raw]
Subject: Re: fsck.ext4 -D corrupts fs?

On Wed, Aug 19, 2009 at 10:13:14PM +0200, Marcin Miroslaw wrote:
> Hello,
> I've noticed fs curruption after using fsck.ext4 -D

Yeah, I really should get a new release of e2fsprogs out. It's fixed
in the maint branch.

- Ted

commit 16e470e65010ba3f0516cf9cdf1bd6c98f064e3f
Author: Theodore Ts'o <[email protected]>
Date: Sun Jul 19 22:43:33 2009 -0400

libext2fs: Fix regression in ext2fs_extent_set_bmap()

Commit 0dc291611 introduced a regression when unmapping the first
block in an extent. This caused e2fsck -fD to corrupt large
directories if the directory has to shrink by more than one block.
The problem was set_bmap should only go to a next leaf when setting a
first block in an extent, and not when it is unmapping the first block
in an extent.

Addresses-Debian-Bug: #537510

Signed-off-by: "Theodore Ts'o" <[email protected]>

diff --git a/lib/ext2fs/extent.c b/lib/ext2fs/extent.c
index 7fcc2cf..9d7b7de 100644
--- a/lib/ext2fs/extent.c
+++ b/lib/ext2fs/extent.c
@@ -1296,11 +1296,12 @@ errcode_t ext2fs_extent_set_bmap(ext2_extent_handle_t handle,
}
if (retval)
goto done;
+ retval = ext2fs_extent_get(handle,
+ EXT2_EXTENT_NEXT_LEAF,
+ &extent);
+ if (retval)
+ goto done;
}
- retval = ext2fs_extent_get(handle, EXT2_EXTENT_NEXT_LEAF,
- &extent);
- if (retval)
- goto done;
extent.e_pblk++;
extent.e_lblk++;
extent.e_len--;

2009-08-20 08:54:44

by Thierry Vignaud

[permalink] [raw]
Subject: Re: fsck.ext4 -D corrupts fs?

Theodore Tso <[email protected]> writes:

> > I've noticed fs curruption after using fsck.ext4 -D
>
> Yeah, I really should get a new release of e2fsprogs out. It's fixed
> in the maint branch.

/me wait for your announce :-)

2009-08-20 19:50:07

by Marcin Miroslaw

[permalink] [raw]
Subject: Re: fsck.ext4 -D corrupts fs?

Theodore Tso pisze:
> On Wed, Aug 19, 2009 at 10:13:14PM +0200, Marcin Miroslaw wrote:
>> Hello,
>> I've noticed fs curruption after using fsck.ext4 -D
>
> Yeah, I really should get a new release of e2fsprogs out. It's fixed
> in the maint branch.

Thank you for quick reply.
I'm waitnig impatient for new release :)
With regards,
Marcin Miros?aw

2009-08-23 14:52:18

by Theodore Ts'o

[permalink] [raw]
Subject: Re: fsck.ext4 -D corrupts fs?

On Thu, Aug 20, 2009 at 10:54:30AM +0200, Thierry Vignaud wrote:
> > Yeah, I really should get a new release of e2fsprogs out. It's fixed
> > in the maint branch.
>
> /me wait for your announce :-)

I've released e2fsprogs 1.41.9.

- Ted

E2fsprogs 1.41.9 (August 22, 2009)
==================================

Fix a bug in e2fsck routines for reallocating an inode table which
could cause it to loop forever on an ext4 filesystem with the FLEX_BG
filesystem feature with a relatively rare (and specific) filesystem
corruption. This fix causes e2fsck to try to find space for a new
portion of the inode table in the containing flex_bg, and if that
fails, the new portion of the inode table will be allocated in any
free space available in the filesystem.

Make e2fsck less annoying by only asking for permission to relocate a
block group's inode table once, instead of for every overlapping
block. Similarly, only ask once to recompute the block group
checksums, instead of once for each corrupted block group's checksum.

Fix filefrag to avoid print the extent header if the FIEMAP ioctl is
not present, and it needs to fall back to using the FIBMAP ioctl.

Fix filefrag to correctly print the number of extents for zero-length
files. (Addresses Debian Bug: #540376)

Filefrag now has a -B option which forces the use of the FIBMAP ioctl
to more easily debug the FIBMAP code.

Fixed filefrag for non-extent based files.

Add a new program, e2freefrag, which displays information about the
free space fragmentation in an ext2/3/4 filesystem.

Fix inode resizing via tune2fs -I so that it works correctly in the
face of non-empty bad blocks inodes, and if the filesystem was
formatted using the "mke2fs -E stride=N" option for RAID arrays.

Fix regression in ext2fs_extent_set_bmap() caused e2fsck -fD to fail
and corrupt large directories if the directory needs to shrink by more
than one block. (Addresses Debian Bug: #537510)

Fix e2fsck's buggy_init_scritps=1 so that the if the last write and/or
last mount times are in the future, they are corrected even if
buggy_init_scripts is set. This is needed because otherwise resize2fs
will refuse to resize the filesystem, even after running "e2fsck -f".
(Addresses Launchpad bug: #373409)

E2fsck will now print much fuller information when the last mount time
or last written time is in the future, since most people can't seem to
believe their distribution has buggy init scripts, or they have a
failed CMOS/RTS clock battery.

Enhance dumpe2fs to dump the extent information via the 'stat'
command, and more detailed extent information via the new command
'dump_extents'.

Update French, Polish, Czech, and Sweedish translation from the
Translation Project.

Fixed various Debian packaging issues --- see debian/changelog for
details.

Programmer's Notes
------------------

Fixed miscellaneous gcc -Wall warnings.

Fixed memory leak in error path in ext2fs_block_iterate2()

Fixed non-Linux build of the intl directory by adding support for the
E/Q/V macros.

The bitmap read/write functions now treat uninitialized bitmaps as
unallocated; this fixes a number of problems in all e2fsprogs for ext4
filesystems when there is a need to allocate new blocks or inodes, and
there aren't any free blocks or inodes in the already-used block
groups.

Improve ext2fs_extent_set_bmap() to avoid creating new extents which
get inserted into the extent tree when they are not needed.