LinuxLists.cc - [PATCH][0/28] Lustre e2fsprogs patch series

2008-02-02 07:59:47

Subject: [PATCH][0/28] Lustre e2fsprogs patch series

The following series of emails will contain the large part of the
e2fsprogs patch series that is used for Lustre. It will not contain
the regression tests for EXTENTS nor the DIR_NLINK features, as those
are very large and were previously submitted.

A full tarball that includes the patches, series, and regression tests
will be uploaded to ftp://ftp.lustre.org/pub/lustre/other/e2fsprogs/

Patch series:

e2fsprogs-specdotin.patch
e2fsprogs-eacheck.patch
e2fsprogs-extended_ops.patch
e2fsprogs-tests-f_unsorted_EAs.patch
e2fsprogs-tests-f_ea_checks.patch
e2fsprogs-nlinks.patch
e2fsprogs-extents.patch
e2fsprogs-config-before-cmdline.patch
e2fsprogs-SLES10--m-support.patch
e2fsprogs-uninit.patch
e2fsprogs-nlinks-flag.patch
e2fsprogs-expand-extra-isize.patch
e2fsprogs-tests-f_expisize.patch
e2fsprogs-tests-f_expisize_ea_del.patch
e2fsprogs-ibadness-counter.patch
e2fsprogs-tests-f_ibadness.patch
e2fsprogs-tests-f_ibadness_bad_extents.patch
e2fsprogs-tests-f_random_corruption.patch
e2fsprogs-stride_option.patch
e2fsprogs-mmp.patch
e2fsprogs-journal_chksum.patch
e2fsprogs-tests-f_jchksum_bblk.patch
e2fsprogs-tests-f_jchksum_blast_trans.patch
e2fsprogs-tests-f_jchksum_remount.patch
e2fsprogs-i_size-corruption.patch
e2fsprogs-fiemap.patch
e2fsprogs-debugfs-supported_features.patch
e2fsprogs-lts-make_rpms.patch

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.

2008-02-02 08:14:16

by Andreas Dilger

[permalink] [raw]

Subject: [PATCH][1/28] e2fsprogs-specdotin.patch

Add the distro type to the RPM release number, so that it is
possible release multiple distro packages without having conflicting
RPM package names.

Allow the RPM built from upstream to replace the split packages provided
by the distros. At some point in the future it may be desirable to also
split the RPM built by this spec file, but this is complicated by the
fact that SLES and RHEL have different splits.

Signed-off-by: Girish Shilamkar <[email protected]>
Signed-off-by: Andreas Dilger <[email protected]>

Index: e2fsprogs-1.40.5/e2fsprogs.spec.in
===================================================================
--- e2fsprogs-1.40.5.orig/e2fsprogs.spec.in
+++ e2fsprogs-1.40.5/e2fsprogs.spec.in
@@ -6,13 +6,22 @@
Summary: Utilities for managing the second extended (ext2) filesystem.
Name: e2fsprogs
Version: @E2FSPROGS_PKGVER@
-Release: 0
+Release: 0%{_vendor}
License: GPLv2
Group: System Environment/Base
Source: ftp://download.sourceforge.net/pub/sourceforge/e2fsprogs/e2fsprogs-%{version}.tar.gz
Url: http://e2fsprogs.sourceforge.net/
Prereq: /sbin/ldconfig
BuildRoot: %{_tmppath}/%{name}-root
+%if %{_vendor} == "suse"
+Group: System/Filesystems
+Provides: e2fsbn ext2fs libcom_err = %{version}
+Obsoletes: ext2fs libcom_err < %{version}
+%else
+Group: System Environment/Base
+Obsoletes: e2fsprogs-libs < %{version}
+Provides: e2fsprogs-libs = %{version}
+%endif

%description
The e2fsprogs package contains a number of utilities for creating,

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.

2008-02-02 08:16:36

by Andreas Dilger

[permalink] [raw]

Subject: [PATCH] [2/28] e2fsprogs-eacheck.patch

Verify in-inode EA structure.
Allow in-inode EAs to have a checksum.
Connect zero-length inodes that have an EA to lost+found.

Signed-off-by: Andreas Dilger <[email protected]>

Index: e2fsprogs-1.40.5/e2fsck/e2fsck.h
===================================================================
--- e2fsprogs-1.40.5.orig/e2fsck/e2fsck.h
+++ e2fsprogs-1.40.5/e2fsck/e2fsck.h
@@ -478,6 +478,9 @@ extern void init_resource_track(struct r
extern int inode_has_valid_blocks(struct ext2_inode *inode);
extern void e2fsck_read_inode(e2fsck_t ctx, unsigned long ino,
struct ext2_inode * inode, const char * proc);
+extern void e2fsck_read_inode_full(e2fsck_t ctx, unsigned long ino,
+ struct ext2_inode *inode,
+ const int bufsize, const char *proc);
extern void e2fsck_write_inode(e2fsck_t ctx, unsigned long ino,
struct ext2_inode * inode, const char * proc);
extern void e2fsck_write_inode_full(e2fsck_t ctx, unsigned long ino,
Index: e2fsprogs-1.40.5/e2fsck/pass1.c
===================================================================
--- e2fsprogs-1.40.5.orig/e2fsck/pass1.c
+++ e2fsprogs-1.40.5/e2fsck/pass1.c
@@ -264,6 +264,7 @@ static void check_ea_in_inode(e2fsck_t c
remain = storage_size - sizeof(__u32);

while (!EXT2_EXT_IS_LAST_ENTRY(entry)) {
+ __u32 hash;

/* header eats this space */
remain -= sizeof(struct ext2_ext_attr_entry);
@@ -291,9 +292,12 @@ static void check_ea_in_inode(e2fsck_t c
problem = PR_1_ATTR_VALUE_BLOCK;
goto fix;
}
-
- /* e_hash must be 0 in inode's ea */
- if (entry->e_hash != 0) {
+
+ hash = ext2fs_ext_attr_hash_entry(entry,
+ start + entry->e_value_offs);
+
+ /* e_hash may be 0 in older inode's ea */
+ if (entry->e_hash != 0 && entry->e_hash != hash) {
pctx->num = entry->e_hash;
problem = PR_1_ATTR_HASH;
goto fix;
@@ -308,15 +312,12 @@ fix:
* it seems like a corruption. it's very unlikely we could repair
* EA(s) in automatic fashion -bzzz
*/
-#if 0
- problem = PR_1_ATTR_HASH;
-#endif
if (problem == 0 || !fix_problem(ctx, problem, pctx))
return;

- /* simple remove all possible EA(s) */
+ /* simply remove all remaining EA(s) */
*((__u32 *)start) = 0UL;
- e2fsck_write_inode_full(ctx, pctx->ino, pctx->inode,
+ e2fsck_write_inode_full(ctx, pctx->ino,(struct ext2_inode *)pctx->inode,
EXT2_INODE_SIZE(sb), "pass1");
}

@@ -1360,10 +1361,13 @@ static int check_ext_attr(e2fsck_t ctx,
entry = (struct ext2_ext_attr_entry *)(header+1);
end = block_buf + fs->blocksize;
while ((char *)entry < end && *(__u32 *)entry) {
+ __u32 hash;
+
if (region_allocate(region, (char *)entry - (char *)header,
EXT2_EXT_ATTR_LEN(entry->e_name_len))) {
if (fix_problem(ctx, PR_1_EA_ALLOC_COLLISION, pctx))
goto clear_extattr;
+ break;
}
if ((ctx->ext_attr_ver == 1 &&
(entry->e_name_len == 0 || entry->e_name_index != 0)) ||
@@ -1371,6 +1375,7 @@ static int check_ext_attr(e2fsck_t ctx,
entry->e_name_index == 0)) {
if (fix_problem(ctx, PR_1_EA_BAD_NAME, pctx))
goto clear_extattr;
+ break;
}
if (entry->e_value_block != 0) {
if (fix_problem(ctx, PR_1_EA_BAD_VALUE, pctx))
@@ -1387,6 +1392,17 @@ static int check_ext_attr(e2fsck_t ctx,
if (fix_problem(ctx, PR_1_EA_ALLOC_COLLISION, pctx))
goto clear_extattr;
}
+
+ hash = ext2fs_ext_attr_hash_entry(entry, block_buf +
+ entry->e_value_offs);
+
+ if (entry->e_hash != hash) {
+ pctx->num = entry->e_hash;
+ if (fix_problem(ctx, PR_1_ATTR_HASH, pctx))
+ goto clear_extattr;
+ entry->e_hash = hash;
+ }
+
entry = EXT2_EXT_ATTR_NEXT(entry);
}
if (region_allocate(region, (char *)entry - (char *)header, 4)) {
@@ -1508,8 +1524,11 @@ static void check_blocks(e2fsck_t ctx, s
}
}

- if (inode->i_file_acl && check_ext_attr(ctx, pctx, block_buf))
+ if (inode->i_file_acl && check_ext_attr(ctx, pctx, block_buf)) {
+ if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+ goto out;
pb.num_blocks++;
+ }

if (ext2fs_inode_has_valid_blocks(inode))
pctx->errcode = ext2fs_block_iterate2(fs, ino,
Index: e2fsprogs-1.40.5/e2fsck/pass4.c
===================================================================
--- e2fsprogs-1.40.5.orig/e2fsck/pass4.c
+++ e2fsprogs-1.40.5/e2fsck/pass4.c
@@ -15,6 +15,7 @@

#include "e2fsck.h"
#include "problem.h"
+#include <ext2fs/ext2_ext_attr.h>

/*
* This routine is called when an inode is not connected to the
@@ -23,31 +24,41 @@
* This subroutine returns 1 then the caller shouldn't bother with the
* rest of the pass 4 tests.
*/
-static int disconnect_inode(e2fsck_t ctx, ext2_ino_t i)
+static int disconnect_inode(e2fsck_t ctx, ext2_ino_t i,
+ struct ext2_inode *inode)
{
ext2_filsys fs = ctx->fs;
- struct ext2_inode inode;
struct problem_context pctx;
+ __u32 eamagic = 0;
+ int extra_size = 0;

- e2fsck_read_inode(ctx, i, &inode, "pass4: disconnect_inode");
+ if (EXT2_INODE_SIZE(fs->super) > EXT2_GOOD_OLD_INODE_SIZE) {
+ e2fsck_read_inode_full(ctx, i, inode,EXT2_INODE_SIZE(fs->super),
+ "pass4: disconnect_inode");
+ extra_size = ((struct ext2_inode_large *)inode)->i_extra_isize;
+ } else {
+ e2fsck_read_inode(ctx, i, inode, "pass4: disconnect_inode");
+ }
clear_problem_context(&pctx);
pctx.ino = i;
- pctx.inode = &inode;
+ pctx.inode = inode;

+ if (EXT2_INODE_SIZE(fs->super) -EXT2_GOOD_OLD_INODE_SIZE -extra_size >0)
+ eamagic = *(__u32 *)(((char *)inode) +EXT2_GOOD_OLD_INODE_SIZE +
+ extra_size);
/*
* Offer to delete any zero-length files that does not have
* blocks. If there is an EA block, it might have useful
* information, so we won't prompt to delete it, but let it be
* reconnected to lost+found.
*/
- if (!inode.i_blocks && (LINUX_S_ISREG(inode.i_mode) ||
- LINUX_S_ISDIR(inode.i_mode))) {
+ if (!inode->i_blocks && eamagic != EXT2_EXT_ATTR_MAGIC &&
+ (LINUX_S_ISREG(inode->i_mode) || LINUX_S_ISDIR(inode->i_mode))) {
if (fix_problem(ctx, PR_4_ZERO_LEN_INODE, &pctx)) {
ext2fs_icount_store(ctx->inode_link_info, i, 0);
- inode.i_links_count = 0;
- inode.i_dtime = ctx->now;
- e2fsck_write_inode(ctx, i, &inode,
- "disconnect_inode");
+ inode->i_links_count = 0;
+ inode->i_dtime = ctx->now;
+ e2fsck_write_inode(ctx, i, inode, "disconnect_inode");
/*
* Fix up the bitmaps...
*/
@@ -55,7 +66,7 @@ static int disconnect_inode(e2fsck_t ctx
ext2fs_unmark_inode_bitmap(ctx->inode_used_map, i);
ext2fs_unmark_inode_bitmap(ctx->inode_dir_map, i);
ext2fs_inode_alloc_stats2(fs, i, -1,
- LINUX_S_ISDIR(inode.i_mode));
+ LINUX_S_ISDIR(inode->i_mode));
return 0;
}
}
@@ -83,7 +94,7 @@ void e2fsck_pass4(e2fsck_t ctx)
{
ext2_filsys fs = ctx->fs;
ext2_ino_t i;
- struct ext2_inode inode;
+ struct ext2_inode *inode;
#ifdef RESOURCE_TRACK
struct resource_track rtrack;
#endif
@@ -111,6 +122,9 @@ void e2fsck_pass4(e2fsck_t ctx)
if ((ctx->progress)(ctx, 4, 0, maxgroup))
return;

+ inode = e2fsck_allocate_memory(ctx, EXT2_INODE_SIZE(fs->super),
+ "scratch inode");
+
/* Protect loop from wrap-around if s_inodes_count maxed */
for (i=1; i <= fs->super->s_inodes_count && i > 0; i++) {
if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
@@ -138,7 +152,7 @@ void e2fsck_pass4(e2fsck_t ctx)
fs->blocksize, "bad_inode buffer");
if (e2fsck_process_bad_inode(ctx, 0, i, buf))
continue;
- if (disconnect_inode(ctx, i))
+ if (disconnect_inode(ctx, i, inode))
continue;
ext2fs_icount_fetch(ctx->inode_link_info, i,
&link_count);
@@ -146,18 +160,18 @@ void e2fsck_pass4(e2fsck_t ctx)
&link_counted);
}
if (link_counted != link_count) {
- e2fsck_read_inode(ctx, i, &inode, "pass4");
+ e2fsck_read_inode(ctx, i, inode, "pass4");
pctx.ino = i;
- pctx.inode = &inode;
- if (link_count != inode.i_links_count) {
+ pctx.inode = inode;
+ if (link_count != inode->i_links_count) {
pctx.num = link_count;
fix_problem(ctx,
PR_4_INCONSISTENT_COUNT, &pctx);
}
pctx.num = link_counted;
if (fix_problem(ctx, PR_4_BAD_REF_COUNT, &pctx)) {
- inode.i_links_count = link_counted;
- e2fsck_write_inode(ctx, i, &inode, "pass4");
+ inode->i_links_count = link_counted;
+ e2fsck_write_inode(ctx, i, inode, "pass4");
}
}
}
@@ -170,6 +184,8 @@ void e2fsck_pass4(e2fsck_t ctx)
errout:
if (buf)
ext2fs_free_mem(&buf);
+
+ ext2fs_free_mem(&inode);
#ifdef RESOURCE_TRACK
if (ctx->options & E2F_OPT_TIME2) {
e2fsck_clear_progbar(ctx);
Index: e2fsprogs-1.40.5/e2fsck/problem.c
===================================================================
--- e2fsprogs-1.40.5.orig/e2fsck/problem.c
+++ e2fsprogs-1.40.5/e2fsck/problem.c
@@ -776,7 +776,7 @@ static struct e2fsck_problem problem_tab

/* invalid ea entry->e_hash */
{ PR_1_ATTR_HASH,
- N_("@a in @i %i has a hash (%N) which is @n (must be 0)\n"),
+ N_("@a in @i %i has a hash (%N) which is @n\n"),
PROMPT_CLEAR, PR_PREEN_OK },

/* inode appears to be a directory */
Index: e2fsprogs-1.40.5/e2fsck/util.c
===================================================================
--- e2fsprogs-1.40.5.orig/e2fsck/util.c
+++ e2fsprogs-1.40.5/e2fsck/util.c
@@ -393,6 +393,20 @@ void e2fsck_read_inode(e2fsck_t ctx, uns
}
}

+void e2fsck_read_inode_full(e2fsck_t ctx, unsigned long ino,
+ struct ext2_inode *inode, int bufsize,
+ const char *proc)
+{
+ int retval;
+
+ retval = ext2fs_read_inode_full(ctx->fs, ino, inode, bufsize);
+ if (retval) {
+ com_err("ext2fs_read_inode_full", retval,
+ _("while reading inode %ld in %s"), ino, proc);
+ fatal_error(ctx, 0);
+ }
+}
+
extern void e2fsck_write_inode_full(e2fsck_t ctx, unsigned long ino,
struct ext2_inode * inode, int bufsize,
const char *proc)
Index: e2fsprogs-1.40.5/lib/ext2fs/ext2_ext_attr.h
===================================================================
--- e2fsprogs-1.40.5.orig/lib/ext2fs/ext2_ext_attr.h
+++ e2fsprogs-1.40.5/lib/ext2fs/ext2_ext_attr.h
@@ -30,7 +30,7 @@ struct ext2_ext_attr_entry {
__u32 e_value_block; /* disk block attribute is stored on (n/i) */
__u32 e_value_size; /* size of attribute value */
__u32 e_hash; /* hash value of name and value */
-#if 0
+#if 1
char e_name[0]; /* attribute name */
#endif
};
Index: e2fsprogs-1.40.5/lib/ext2fs/ext2fs.h
===================================================================
--- e2fsprogs-1.40.5.orig/lib/ext2fs/ext2fs.h
+++ e2fsprogs-1.40.5/lib/ext2fs/ext2fs.h
@@ -75,10 +75,12 @@ typedef __u32 ext2_dirhash_t;
#include "com_err.h"
#include "ext2_io.h"
#include "ext2_err.h"
+#include "ext2_ext_attr.h"
#else
#include <et/com_err.h>
#include <ext2fs/ext2_io.h>
#include <ext2fs/ext2_err.h>
+#include <ext2fs/ext2_ext_attr.h>
#endif

/*
@@ -711,6 +713,8 @@ extern errcode_t ext2fs_dup_handle(ext2_
extern errcode_t ext2fs_expand_dir(ext2_filsys fs, ext2_ino_t dir);

/* ext_attr.c */
+extern __u32 ext2fs_ext_attr_hash_entry(struct ext2_ext_attr_entry *entry,
+ void *data);
extern errcode_t ext2fs_read_ext_attr(ext2_filsys fs, blk_t block, void *buf);
extern errcode_t ext2fs_write_ext_attr(ext2_filsys fs, blk_t block,
void *buf);
@@ -961,6 +965,10 @@ extern errcode_t ext2fs_create_resize_in
/* swapfs.c */
extern void ext2fs_swap_ext_attr(char *to, char *from, int bufsize,
int has_header);
+extern void ext2fs_swap_ext_attr_header(struct ext2_ext_attr_header *to_header,
+ struct ext2_ext_attr_header *from_hdr);
+extern void ext2fs_swap_ext_attr_entry(struct ext2_ext_attr_entry *to_entry,
+ struct ext2_ext_attr_entry *from_entry);
extern void ext2fs_swap_super(struct ext2_super_block * super);
extern void ext2fs_swap_group_desc(struct ext2_group_desc *gdp);
extern void ext2fs_swap_inode_full(ext2_filsys fs, struct ext2_inode_large *t,
Index: e2fsprogs-1.40.5/lib/ext2fs/ext_attr.c
===================================================================
--- e2fsprogs-1.40.5.orig/lib/ext2fs/ext_attr.c
+++ e2fsprogs-1.40.5/lib/ext2fs/ext_attr.c
@@ -23,6 +23,43 @@

#include "ext2fs.h"

+#define NAME_HASH_SHIFT 5
+#define VALUE_HASH_SHIFT 16
+
+/*
+ * ext2_xattr_hash_entry()
+ *
+ * Compute the hash of an extended attribute.
+ */
+__u32 ext2fs_ext_attr_hash_entry(struct ext2_ext_attr_entry *entry, void *data)
+{
+ __u32 hash = 0;
+ char *name = entry->e_name;
+ int n;
+
+ for (n = 0; n < entry->e_name_len; n++) {
+ hash = (hash << NAME_HASH_SHIFT) ^
+ (hash >> (8*sizeof(hash) - NAME_HASH_SHIFT)) ^
+ *name++;
+ }
+
+ /* The hash needs to be calculated on the data in little-endian. */
+ if (entry->e_value_block == 0 && entry->e_value_size != 0) {
+ __u32 *value = (__u32 *)data;
+ for (n = (entry->e_value_size + EXT2_EXT_ATTR_ROUND) >>
+ EXT2_EXT_ATTR_PAD_BITS; n; n--) {
+ hash = (hash << VALUE_HASH_SHIFT) ^
+ (hash >> (8*sizeof(hash) - VALUE_HASH_SHIFT)) ^
+ ext2fs_le32_to_cpu(*value++);
+ }
+ }
+
+ return hash;
+}
+
+#undef NAME_HASH_SHIFT
+#undef VALUE_HASH_SHIFT
+
errcode_t ext2fs_read_ext_attr(ext2_filsys fs, blk_t block, void *buf)
{
errcode_t retval;
Index: e2fsprogs-1.40.5/lib/ext2fs/swapfs.c
===================================================================
--- e2fsprogs-1.40.5.orig/lib/ext2fs/swapfs.c
+++ e2fsprogs-1.40.5/lib/ext2fs/swapfs.c
@@ -90,6 +90,29 @@ void ext2fs_swap_group_desc(struct ext2_
gdp->bg_checksum = ext2fs_swab16(gdp->bg_checksum);
}

+void ext2fs_swap_ext_attr_header(struct ext2_ext_attr_header *to_header,
+ struct ext2_ext_attr_header *from_header)
+{
+ int n;
+
+ to_header->h_magic = ext2fs_swab32(from_header->h_magic);
+ to_header->h_blocks = ext2fs_swab32(from_header->h_blocks);
+ to_header->h_refcount = ext2fs_swab32(from_header->h_refcount);
+ to_header->h_hash = ext2fs_swab32(from_header->h_hash);
+ for (n = 0; n < 4; n++)
+ to_header->h_reserved[n] =
+ ext2fs_swab32(from_header->h_reserved[n]);
+}
+
+void ext2fs_swap_ext_attr_entry(struct ext2_ext_attr_entry *to_entry,
+ struct ext2_ext_attr_entry *from_entry)
+{
+ to_entry->e_value_offs = ext2fs_swab16(from_entry->e_value_offs);
+ to_entry->e_value_block = ext2fs_swab32(from_entry->e_value_block);
+ to_entry->e_value_size = ext2fs_swab32(from_entry->e_value_size);
+ to_entry->e_hash = ext2fs_swab32(from_entry->e_hash);
+}
+
void ext2fs_swap_ext_attr(char *to, char *from, int bufsize, int has_header)
{
struct ext2_ext_attr_header *from_header =
@@ -98,32 +121,22 @@ void ext2fs_swap_ext_attr(char *to, char
(struct ext2_ext_attr_header *)to;
struct ext2_ext_attr_entry *from_entry, *to_entry;
char *from_end = (char *)from_header + bufsize;
- int n;

if (to_header != from_header)
memcpy(to_header, from_header, bufsize);

- from_entry = (struct ext2_ext_attr_entry *)from_header;
- to_entry = (struct ext2_ext_attr_entry *)to_header;
-
if (has_header) {
- to_header->h_magic = ext2fs_swab32(from_header->h_magic);
- to_header->h_blocks = ext2fs_swab32(from_header->h_blocks);
- to_header->h_refcount = ext2fs_swab32(from_header->h_refcount);
- for (n=0; n<4; n++)
- to_header->h_reserved[n] =
- ext2fs_swab32(from_header->h_reserved[n]);
+ ext2fs_swap_ext_attr_header(to_header, from_header);
+
from_entry = (struct ext2_ext_attr_entry *)(from_header+1);
to_entry = (struct ext2_ext_attr_entry *)(to_header+1);
+ } else {
+ from_entry = (struct ext2_ext_attr_entry *)from_header;
+ to_entry = (struct ext2_ext_attr_entry *)to_header;
}

while ((char *)from_entry < from_end && *(__u32 *)from_entry) {
- to_entry->e_value_offs =
- ext2fs_swab16(from_entry->e_value_offs);
- to_entry->e_value_block =
- ext2fs_swab32(from_entry->e_value_block);
- to_entry->e_value_size =
- ext2fs_swab32(from_entry->e_value_size);
+ ext2fs_swap_ext_attr_entry(to_entry, from_entry);
from_entry = EXT2_EXT_ATTR_NEXT(from_entry);
to_entry = EXT2_EXT_ATTR_NEXT(to_entry);
}

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.

2008-02-02 08:17:47

by Andreas Dilger

[permalink] [raw]

Subject: [PATCH][3/28] e2fsprogs-extended_ops.patch

Minor reformatting patch to make applying later patches easier.

Signed-off-by: Andreas Dilger <[email protected]>

Index: e2fsprogs-cfs/e2fsck/unix.c
===================================================================
--- e2fsprogs-cfs.orig/e2fsck/unix.c
+++ e2fsprogs-cfs/e2fsck/unix.c
@@ -537,14 +537,13 @@ static void parse_extended_opts(e2fsck_t
continue;
}
ea_ver = strtoul(arg, &p, 0);
- if (*p ||
- ((ea_ver != 1) && (ea_ver != 2))) {
- fprintf(stderr,
- _("Invalid EA version.\n"));
+ if (*p == '\0' && (ea_ver == 1 || ea_ver == 2)) {
+ ctx->ext_attr_ver = ea_ver;
+ } else {
+ fprintf(stderr, _("Invalid EA version.\n"));
extended_usage++;
continue;
}
- ctx->ext_attr_ver = ea_ver;
} else {
fprintf(stderr, _("Unknown extended option: %s\n"),
token);
@@ -558,7 +557,8 @@ static void parse_extended_opts(e2fsck_t
"and may take an argument which\n"
"is set off by an equals ('=') sign. "
"Valid extended options are:\n"
- "\tea_ver=<ea_version (1 or 2)>\n\n"), stderr);
+ "\tea_ver=<ea_version (1 or 2)>\n"
+ "\n"), stderr);
exit(1);
}
}

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.

2008-02-02 08:20:52

by Andreas Dilger

[permalink] [raw]

Subject: Re: [PATCH][4/28] e2fsprogs-tests-f_unsorted_EAs.patch

Attached binary patch.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.

Attachments:

(No filename) (125.00 B)
e2fsprogs-tests-f_unsorted_EAs.patch (16.85 kB)
Download all attachments

2008-02-02 08:22:09

Test case for expanding inode size where there is not enough room
for the requested new inode size. Prompt user to delete one or more
EAs (default is to abort).

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.

Attachments:

(No filename) (264.00 B)
e2fsprogs-tests-f_expisize_ea_del.patch (13.92 kB)
Download all attachments

2008-02-02 08:41:04

by Andreas Dilger

[permalink] [raw]

Subject: Re: [PATCH][15/28] e2fsprogs-ibadness-counter.patch

The present e2fsck code checks the inode, per field basis. It doesn't
take into consideration to total sanity of the inode. This may cause
e2fsck turning a garbage inode into an apparently sane inode ("It is a
vessel of fertilizer, and none may abide its strength.").

The following patch adds a heuristics to detect the degree of badness of
an inode. icount mechanism is used to keep track of the badness of every
inode. The badness is increased as various fields in inode are found to
be corrupt. Badness above a certain threshold value results in deletion
of the inode. The default threshold value is 7, it can be specified to
e2fsck using "-E inode_badness_threshold=<value>"

This can avoid lengthy pass1b shared block processing, where a corrupt
chunk of the inode table has resulted in a bunch of garbage inodes
suddenly having shared blocks with a lot of good inodes (or each other).

Signed-off-by: Andreas Dilger <[email protected]>
Signed-off-by: Girish Shilamkar <[email protected]>

Index: e2fsprogs-1.40.4/e2fsck/e2fsck.h
===================================================================
--- e2fsprogs-1.40.4.orig/e2fsck/e2fsck.h
+++ e2fsprogs-1.40.4/e2fsck/e2fsck.h
@@ -11,6 +11,7 @@

#include <stdio.h>
#include <string.h>
+#include <stddef.h>
#ifdef HAVE_UNISTD_H
#include <unistd.h>
#endif
@@ -195,6 +196,18 @@ typedef enum {
E2F_CLONE_ZERO
} clone_opt_t;

+#define EXT4_FITS_IN_INODE(ext4_inode, einode, field) \
+ ((offsetof(typeof(*ext4_inode), field) + \
+ sizeof(ext4_inode->field)) \
+ <= (EXT2_GOOD_OLD_INODE_SIZE + \
+ (einode)->i_extra_isize)) \
+
+#define BADNESS_NORMAL 1
+#define BADNESS_HIGH 2
+#define BADNESS_THRESHOLD 8
+#define BADNESS_BAD_MODE 100
+#define BADNESS_LARGE_FILE 2199023255552ULL
+
/*
* Define the extended attribute refcount structure
*/
@@ -229,7 +242,6 @@ struct e2fsck_struct {
unsigned long max);

ext2fs_inode_bitmap inode_used_map; /* Inodes which are in use */
- ext2fs_inode_bitmap inode_bad_map; /* Inodes which are bad somehow */
ext2fs_inode_bitmap inode_dir_map; /* Inodes which are directories */
ext2fs_inode_bitmap inode_bb_map; /* Inodes which are in bad blocks */
ext2fs_inode_bitmap inode_imagic_map; /* AFS inodes */
@@ -244,6 +256,8 @@ struct e2fsck_struct {
*/
ext2_icount_t inode_count;
ext2_icount_t inode_link_info;
+ ext2_icount_t inode_badness;
+ int inode_badness_threshold;

ext2_refcount_t refcount;
ext2_refcount_t refcount_extra;
@@ -344,6 +358,7 @@ struct e2fsck_struct {
/* misc fields */
time_t now;
time_t time_fudge; /* For working around buggy init scripts */
+ time_t now_tolerance_val;
int ext_attr_ver;
shared_opt_t shared;
clone_opt_t clone;
@@ -454,6 +469,8 @@ extern int e2fsck_pass1_check_device_ino
struct ext2_inode *inode);
extern int e2fsck_pass1_check_symlink(ext2_filsys fs,
struct ext2_inode *inode, char *buf);
+extern void e2fsck_mark_inode_bad(e2fsck_t ctx, ino_t ino, int count);
+extern int is_inode_bad(e2fsck_t ctx, ino_t ino);

/* pass2.c */
extern int e2fsck_process_bad_inode(e2fsck_t ctx, ext2_ino_t dir,
Index: e2fsprogs-1.40.4/e2fsck/pass1.c
===================================================================
--- e2fsprogs-1.40.4.orig/e2fsck/pass1.c
+++ e2fsprogs-1.40.4/e2fsck/pass1.c
@@ -20,7 +20,8 @@
* - A bitmap of which inodes are in use. (inode_used_map)
* - A bitmap of which inodes are directories. (inode_dir_map)
* - A bitmap of which inodes are regular files. (inode_reg_map)
- * - A bitmap of which inodes have bad fields. (inode_bad_map)
+ * - An icount mechanism is used to keep track of
+ * inodes with bad fields and its badness (ctx->inode_badness)
* - A bitmap of which inodes are in bad blocks. (inode_bb_map)
* - A bitmap of which inodes are imagic inodes. (inode_imagic_map)
* - A bitmap of which inodes need to be expanded (expand_eisize_map)
@@ -68,7 +69,6 @@ static void check_blocks(e2fsck_t ctx, s
static void mark_table_blocks(e2fsck_t ctx);
static void alloc_bb_map(e2fsck_t ctx);
static void alloc_imagic_map(e2fsck_t ctx);
-static void mark_inode_bad(e2fsck_t ctx, ino_t ino);
static void handle_fs_bad_blocks(e2fsck_t ctx);
static void process_inodes(e2fsck_t ctx, char *block_buf);
static EXT2_QSORT_TYPE process_inode_cmp(const void *a, const void *b);
@@ -220,6 +220,7 @@ static void check_immutable(e2fsck_t ctx
if (!(pctx->inode->i_flags & BAD_SPECIAL_FLAGS))
return;

+ e2fsck_mark_inode_bad(ctx, pctx->ino, BADNESS_NORMAL);
if (!fix_problem(ctx, PR_1_SET_IMMUTABLE, pctx))
return;

@@ -238,6 +239,7 @@ static void check_size(e2fsck_t ctx, str
if ((inode->i_size == 0) && (inode->i_size_high == 0))
return;

+ e2fsck_mark_inode_bad(ctx, pctx->ino, BADNESS_NORMAL);
if (!fix_problem(ctx, PR_1_SET_NONZSIZE, pctx))
return;

@@ -352,6 +354,7 @@ static void check_inode_extra_space(e2fs
*/
if (inode->i_extra_isize &&
(inode->i_extra_isize < min || inode->i_extra_isize > max)) {
+ e2fsck_mark_inode_bad(ctx, pctx->ino, BADNESS_NORMAL);
if (!fix_problem(ctx, PR_1_EXTRA_ISIZE, pctx))
return;
inode->i_extra_isize = ctx->want_extra_isize;
@@ -441,6 +444,7 @@ static void check_is_really_dir(e2fsck_t
(dirent->rec_len % 4))
return;

+ e2fsck_mark_inode_bad(ctx, pctx->ino, BADNESS_NORMAL);
if (fix_problem(ctx, PR_1_TREAT_AS_DIRECTORY, pctx)) {
inode->i_mode = (inode->i_mode & 07777) | LINUX_S_IFDIR;
e2fsck_write_inode_full(ctx, pctx->ino, inode,
@@ -637,6 +641,7 @@ void e2fsck_pass1(e2fsck_t ctx)
ext2_filsys fs = ctx->fs;
ext2_ino_t ino;
struct ext2_inode *inode;
+ struct ext2_inode_large *inode_large;
ext2_inode_scan scan;
char *block_buf;
#ifdef RESOURCE_TRACK
@@ -873,8 +878,10 @@ void e2fsck_pass1(e2fsck_t ctx)
ino, 0);
e2fsck_write_inode(ctx, ino, inode,
"pass1");
+ } else {
+ e2fsck_mark_inode_bad(ctx, ino,
+ BADNESS_NORMAL);
}
-
}
/*
* If dtime is set, offer to clear it. mke2fs
@@ -891,6 +898,7 @@ void e2fsck_pass1(e2fsck_t ctx)
e2fsck_write_inode(ctx, ino, inode,
"pass1");
}
+ e2fsck_mark_inode_bad(ctx, ino, BADNESS_NORMAL);
}
} else if (ino == EXT2_JOURNAL_INO) {
ext2fs_mark_inode_bitmap(ctx->inode_used_map, ino);
@@ -997,6 +1005,7 @@ void e2fsck_pass1(e2fsck_t ctx)
inode->i_dtime = 0;
e2fsck_write_inode(ctx, ino, inode, "pass1");
}
+ e2fsck_mark_inode_bad(ctx, ino, BADNESS_NORMAL);
}

ext2fs_mark_inode_bitmap(ctx->inode_used_map, ino);
@@ -1013,14 +1022,16 @@ void e2fsck_pass1(e2fsck_t ctx)
frag = fsize = 0;
}

+ /* Fixed in pass2, e2fsck_process_bad_inode(). */
if (inode->i_faddr || frag || fsize ||
(LINUX_S_ISDIR(inode->i_mode) && inode->i_dir_acl))
- mark_inode_bad(ctx, ino);
+ e2fsck_mark_inode_bad(ctx, ino, BADNESS_NORMAL);
+ /* Fixed in pass2, e2fsck_process_bad_inode(). */
if ((fs->super->s_creator_os == EXT2_OS_LINUX) &&
!(fs->super->s_feature_ro_compat &
EXT4_FEATURE_RO_COMPAT_HUGE_FILE) &&
(inode->osd2.linux2.l_i_blocks_hi != 0))
- mark_inode_bad(ctx, ino);
+ e2fsck_mark_inode_bad(ctx, ino, BADNESS_NORMAL);
if (inode->i_flags & EXT2_IMAGIC_FL) {
if (imagic_fs) {
if (!ctx->inode_imagic_map)
@@ -1033,6 +1044,7 @@ void e2fsck_pass1(e2fsck_t ctx)
e2fsck_write_inode(ctx, ino,
inode, "pass1");
}
+ e2fsck_mark_inode_bad(ctx, ino, BADNESS_NORMAL);
}
}

@@ -1075,8 +1087,41 @@ void e2fsck_pass1(e2fsck_t ctx)
check_immutable(ctx, &pctx);
check_size(ctx, &pctx);
ctx->fs_sockets_count++;
- } else
- mark_inode_bad(ctx, ino);
+ } else {
+ e2fsck_mark_inode_bad(ctx, ino, BADNESS_NORMAL);
+ }
+
+ if (inode->i_atime > ctx->now + ctx->now_tolerance_val ||
+ inode->i_mtime > ctx->now + ctx->now_tolerance_val)
+ e2fsck_mark_inode_bad(ctx, ino, BADNESS_NORMAL);
+
+ if (inode->i_ctime < sb->s_mkfs_time ||
+ inode->i_ctime > ctx->now + ctx->now_tolerance_val)
+ e2fsck_mark_inode_bad(ctx, ino, BADNESS_HIGH);
+
+ if (EXT4_FITS_IN_INODE(inode_large,
+ (struct ext2_inode_large *)inode, i_crtime)) {
+ if (((struct ext2_inode_large *)inode)->i_crtime <
+ sb->s_mkfs_time ||
+ ((struct ext2_inode_large *)inode)->i_crtime >
+ ctx->now + ctx->now_tolerance_val) {
+ e2fsck_mark_inode_bad(ctx, ino, BADNESS_HIGH);
+ }
+ }
+
+ /* Is it a regular file */
+ if ((LINUX_S_ISREG(inode->i_mode)) &&
+ /* File size > 2TB */
+ ((((long long)inode->i_size_high << 32) +
+ inode->i_size) > BADNESS_LARGE_FILE) &&
+ /* fs does not have huge file feature */
+ ((fs->super->s_creator_os == EXT2_OS_LINUX) &&
+ !(fs->super->s_feature_ro_compat &
+ EXT4_FEATURE_RO_COMPAT_HUGE_FILE) &&
+ /* inode does not have enough blocks for size */
+ (inode->osd2.linux2.l_i_blocks_hi != 0))) {
+ e2fsck_mark_inode_bad(ctx, ino, BADNESS_NORMAL);
+ }

eh = (struct ext3_extent_header *)inode->i_block;
if ((inode->i_flags & EXT4_EXTENTS_FL)) {
@@ -1091,19 +1136,28 @@ void e2fsck_pass1(e2fsck_t ctx)
ext2fs_mark_super_dirty(fs);
extent_fs = 1;
}
- } else if (fix_problem(ctx, PR_1_SET_EXTENT_FL, &pctx)){
- inode->i_flags &= ~EXT4_EXTENTS_FL;
- e2fsck_write_inode(ctx, ino, inode, "pass1");
- goto check_ind_inode;
+ } else {
+ e2fsck_mark_inode_bad(ctx, ino, BADNESS_NORMAL);
+ if (fix_problem(ctx, PR_1_SET_EXTENT_FL,
+ &pctx)) {
+ inode->i_flags &= ~EXT4_EXTENTS_FL;
+ e2fsck_write_inode(ctx, ino,
+ inode,"pass1");
+ goto check_ind_inode;
+ }
}
} else if (extent_fs &&
(LINUX_S_ISREG(inode->i_mode) ||
LINUX_S_ISDIR(inode->i_mode)) &&
ext2fs_extent_header_verify(eh, EXT2_N_BLOCKS *
- sizeof(__u32)) == 0 &&
- fix_problem(ctx, PR_1_UNSET_EXTENT_FL, &pctx)) {
- inode->i_flags |= EXT4_EXTENTS_FL;
- e2fsck_write_inode(ctx, ino, inode, "pass1");
+ sizeof(__u32)) == 0) {
+ if (fix_problem(ctx, PR_1_UNSET_EXTENT_FL,
+ &pctx)) {
+ inode->i_flags |= EXT4_EXTENTS_FL;
+ e2fsck_write_inode(ctx, ino, inode,
+ "pass1");
+ }
+ e2fsck_mark_inode_bad(ctx, ino, BADNESS_NORMAL);
}
if (extent_fs && inode->i_flags & EXT4_EXTENTS_FL) {
ctx->extent_files++;
@@ -1344,29 +1398,27 @@ static EXT2_QSORT_TYPE process_inode_cmp
}

/*
- * Mark an inode as being bad in some what
+ * Mark an inode as being bad and increment its badness counter.
*/
-static void mark_inode_bad(e2fsck_t ctx, ino_t ino)
+void e2fsck_mark_inode_bad(e2fsck_t ctx, ino_t ino, int count)
{
- struct problem_context pctx;
-
- if (!ctx->inode_bad_map) {
- clear_problem_context(&pctx);
+ struct problem_context pctx;
+ __u16 result;

- pctx.errcode = ext2fs_allocate_inode_bitmap(ctx->fs,
- _("bad inode map"), &ctx->inode_bad_map);
+ if (!ctx->inode_badness) {
+ clear_problem_context(&pctx);
+ pctx.errcode = ext2fs_create_icount2(ctx->fs, 0, 0, NULL,
+ &ctx->inode_badness);
if (pctx.errcode) {
- pctx.num = 3;
- fix_problem(ctx, PR_1_ALLOCATE_IBITMAP_ERROR, &pctx);
- /* Should never get here */
+ fix_problem(ctx, PR_1_ALLOCATE_ICOUNT, &pctx);
ctx->flags |= E2F_FLAG_ABORT;
return;
}
}
- ext2fs_mark_inode_bitmap(ctx->inode_bad_map, ino);
+ ext2fs_icount_fetch(ctx->inode_badness, ino, &result);
+ ext2fs_icount_store(ctx->inode_badness, ino, count + result);
}

-
/*
* This procedure will allocate the inode "bb" (badblock) map table
*/
@@ -1521,7 +1573,8 @@ static int check_ext_attr(e2fsck_t ctx,
if (!(fs->super->s_feature_compat & EXT2_FEATURE_COMPAT_EXT_ATTR) ||
(blk < fs->super->s_first_data_block) ||
(blk >= fs->super->s_blocks_count)) {
- mark_inode_bad(ctx, ino);
+ /* Fixed in pass2, e2fsck_process_bad_inode(). */
+ e2fsck_mark_inode_bad(ctx, ino, BADNESS_NORMAL);
return 0;
}

@@ -1701,21 +1754,28 @@ static int handle_htree(e2fsck_t ctx, st

if ((!LINUX_S_ISDIR(inode->i_mode) &&
fix_problem(ctx, PR_1_HTREE_NODIR, pctx)) ||
- (!(fs->super->s_feature_compat & EXT2_FEATURE_COMPAT_DIR_INDEX) &&
- fix_problem(ctx, PR_1_HTREE_SET, pctx)))
- return 1;
+ (!(fs->super->s_feature_compat & EXT2_FEATURE_COMPAT_DIR_INDEX))) {
+ e2fsck_mark_inode_bad(ctx, ino, BADNESS_NORMAL);
+ if (fix_problem(ctx, PR_1_HTREE_SET, pctx))
+ return 1;
+ }

ext2fs_block_iterate2(fs, ino, BLOCK_FLAG_DATA_ONLY | BLOCK_FLAG_HOLE,
block_buf, htree_blk_iter_cb, &blk);
if (((blk == 0) ||
(blk < fs->super->s_first_data_block) ||
- (blk >= fs->super->s_blocks_count)) &&
- fix_problem(ctx, PR_1_HTREE_BADROOT, pctx))
- return 1;
+ (blk >= fs->super->s_blocks_count))) {
+ e2fsck_mark_inode_bad(ctx, ino, BADNESS_NORMAL);
+ if (fix_problem(ctx, PR_1_HTREE_BADROOT, pctx))
+ return 1;
+ }

retval = io_channel_read_blk(fs->io, blk, 1, block_buf);
- if (retval && fix_problem(ctx, PR_1_HTREE_BADROOT, pctx))
- return 1;
+ if (retval) {
+ e2fsck_mark_inode_bad(ctx, ino, BADNESS_NORMAL);
+ if (fix_problem(ctx, PR_1_HTREE_BADROOT, pctx))
+ return 1;
+ }

/* XXX should check that beginning matches a directory */
root = (struct ext2_dx_root_info *) (block_buf + 24);
@@ -1785,6 +1845,9 @@ static int e2fsck_ind_block_verify(struc
bad++;
}

+ if (num_indir <= EXT2_N_BLOCKS)
+ e2fsck_mark_inode_bad(p->ctx, p->ino, bad);
+
if ((num_indir <= EXT2_N_BLOCKS && bad > 4) || bad > 8)
return PR_1_INDIRECT_BAD;

@@ -1828,6 +1891,10 @@ static int e2fsck_ext_block_verify(struc
pctx->blkcount = ex->ee_start;
pctx->num = ex->ee_len;
pctx->blk = ex->ee_block;
+ /* To ensure that extent is in inode */
+ if (eh->eh_max == 4)
+ e2fsck_mark_inode_bad(p->ctx, p->ino,
+ BADNESS_HIGH);
if (fix_problem(ctx, PR_1_EXTENT_BAD, pctx)) {
ext2fs_extent_remove(eh, ex);
i--; ex--; /* check next (moved) item */
@@ -1854,6 +1921,10 @@ static int e2fsck_ext_block_verify(struc
pctx->blkcount = ix->ei_leaf;;
pctx->num = i;
pctx->blk = ix->ei_block;
+ /* To ensure that extent_idx is in inode */
+ if (eh->eh_max == 4)
+ e2fsck_mark_inode_bad(p->ctx, p->ino,
+ BADNESS_HIGH);
if (fix_problem(ctx, PR_1_EXTENT_IDX_BAD,pctx)){
ext2fs_extent_index_remove(eh, ix);
i--; ix--; /* check next (moved) item */
@@ -1861,7 +1932,6 @@ static int e2fsck_ext_block_verify(struc
continue;
}
}
-
ix_prev = ix;
}
}
@@ -1916,6 +1986,7 @@ static void check_blocks(e2fsck_t ctx, s
inode->i_flags &= ~EXT2_COMPRBLK_FL;
dirty_inode++;
}
+ e2fsck_mark_inode_bad(ctx, ino, BADNESS_NORMAL);
}
}

@@ -1961,6 +2032,7 @@ static void check_blocks(e2fsck_t ctx, s
ext2fs_icount_store(ctx->inode_link_info, ino, 0);
inode->i_dtime = ctx->now;
dirty_inode++;
+ ext2fs_icount_store(ctx->inode_badness, ino, 0);
ext2fs_unmark_inode_bitmap(ctx->inode_dir_map, ino);
ext2fs_unmark_inode_bitmap(ctx->inode_reg_map, ino);
ext2fs_unmark_inode_bitmap(ctx->inode_used_map, ino);
@@ -2000,6 +2072,11 @@ static void check_blocks(e2fsck_t ctx, s
ctx->fs_directory_count--;
goto out;
}
+ /*
+ * The mode might be in-correct. Increasing the badness by
+ * small amount won't hurt much.
+ */
+ e2fsck_mark_inode_bad(ctx, ino, BADNESS_NORMAL);
}

pb.num_blocks *= (fs->blocksize / 512);
@@ -2039,6 +2116,7 @@ static void check_blocks(e2fsck_t ctx, s
inode->i_size_high = pctx->num >> 32;
dirty_inode++;
}
+ e2fsck_mark_inode_bad(ctx, ino, BADNESS_NORMAL);
pctx->num = 0;
}
if (LINUX_S_ISREG(inode->i_mode) &&
@@ -2050,6 +2128,7 @@ static void check_blocks(e2fsck_t ctx, s
inode->i_blocks = pb.num_blocks;
dirty_inode++;
}
+ e2fsck_mark_inode_bad(ctx, ino, BADNESS_NORMAL);
pctx->num = 0;
}
out:
Index: e2fsprogs-1.40.4/e2fsck/pass4.c
===================================================================
--- e2fsprogs-1.40.4.orig/e2fsck/pass4.c
+++ e2fsprogs-1.40.4/e2fsck/pass4.c
@@ -185,6 +185,7 @@ void e2fsck_pass4(e2fsck_t ctx)
}
ext2fs_free_icount(ctx->inode_link_info); ctx->inode_link_info = 0;
ext2fs_free_icount(ctx->inode_count); ctx->inode_count = 0;
+ ext2fs_free_icount(ctx->inode_badness); ctx->inode_badness = 0;
ext2fs_free_inode_bitmap(ctx->inode_bb_map);
ctx->inode_bb_map = 0;
ext2fs_free_inode_bitmap(ctx->inode_imagic_map);
Index: e2fsprogs-1.40.4/e2fsck/pass2.c
===================================================================
--- e2fsprogs-1.40.4.orig/e2fsck/pass2.c
+++ e2fsprogs-1.40.4/e2fsck/pass2.c
@@ -251,10 +251,6 @@ void e2fsck_pass2(e2fsck_t ctx)
ext2fs_free_mem(&buf);
ext2fs_free_dblist(fs->dblist);

- if (ctx->inode_bad_map) {
- ext2fs_free_inode_bitmap(ctx->inode_bad_map);
- ctx->inode_bad_map = 0;
- }
if (ctx->inode_reg_map) {
ext2fs_free_inode_bitmap(ctx->inode_reg_map);
ctx->inode_reg_map = 0;
@@ -501,6 +497,7 @@ static _INLINE_ int check_filetype(e2fsc
{
int filetype = dirent->name_len >> 8;
int should_be = EXT2_FT_UNKNOWN;
+ __u32 result;
struct ext2_inode inode;

if (!(ctx->fs->super->s_feature_incompat &
@@ -512,16 +509,18 @@ static _INLINE_ int check_filetype(e2fsc
return 1;
}

+ if (ctx->inode_badness)
+ ext2fs_icount_fetch32(ctx->inode_badness, dirent->inode,
+ &result);
+
if (ext2fs_test_inode_bitmap(ctx->inode_dir_map, dirent->inode)) {
should_be = EXT2_FT_DIR;
} else if (ext2fs_test_inode_bitmap(ctx->inode_reg_map,
dirent->inode)) {
should_be = EXT2_FT_REG_FILE;
- } else if (ctx->inode_bad_map &&
- ext2fs_test_inode_bitmap(ctx->inode_bad_map,
- dirent->inode))
+ } else if (ctx->inode_badness && result >= BADNESS_BAD_MODE) {
should_be = 0;
- else {
+ } else {
e2fsck_read_inode(ctx, dirent->inode, &inode,
"check_filetype");
should_be = ext2_file_type(inode.i_mode);
@@ -956,12 +955,10 @@ static int check_dir_block(ext2_filsys f
* (We wait until now so that we can display the
* pathname to the user.)
*/
- if (ctx->inode_bad_map &&
- ext2fs_test_inode_bitmap(ctx->inode_bad_map,
- dirent->inode)) {
- if (e2fsck_process_bad_inode(ctx, ino,
- dirent->inode,
- buf + fs->blocksize)) {
+ if ((ctx->inode_badness) &&
+ ext2fs_icount_is_set(ctx->inode_badness, dirent->inode)) {
+ if (e2fsck_process_bad_inode(ctx, ino, dirent->inode,
+ buf + fs->blocksize)) {
dirent->inode = 0;
dir_modified++;
goto next;
@@ -1195,8 +1192,8 @@ static void deallocate_inode(e2fsck_t ct
e2fsck_read_bitmaps(ctx);
ext2fs_unmark_inode_bitmap(ctx->inode_used_map, ino);
ext2fs_unmark_inode_bitmap(ctx->inode_dir_map, ino);
- if (ctx->inode_bad_map)
- ext2fs_unmark_inode_bitmap(ctx->inode_bad_map, ino);
+ if (ctx->inode_badness)
+ ext2fs_icount_store(ctx->inode_badness, ino, 0);
ext2fs_inode_alloc_stats2(fs, ino, -1, LINUX_S_ISDIR(inode.i_mode));

if (inode.i_file_acl &&
@@ -1261,8 +1258,10 @@ extern int e2fsck_process_bad_inode(e2fs
int not_fixed = 0;
unsigned char *frag, *fsize;
struct problem_context pctx;
- int problem = 0;
+ int problem = 0;
+ __u16 badness;

+ ext2fs_icount_fetch(ctx->inode_badness, ino, &badness);
e2fsck_read_inode(ctx, ino, &inode, "process_bad_inode");

clear_problem_context(&pctx);
@@ -1277,6 +1276,7 @@ extern int e2fsck_process_bad_inode(e2fs
inode_modified++;
} else
not_fixed++;
+ badness += BADNESS_NORMAL;
}

if (!LINUX_S_ISDIR(inode.i_mode) && !LINUX_S_ISREG(inode.i_mode) &&
@@ -1310,6 +1310,11 @@ extern int e2fsck_process_bad_inode(e2fs
} else
not_fixed++;
problem = 0;
+ /*
+ * A high value is associated with bad mode in order to detect
+ * that mode was corrupt in check_filetype()
+ */
+ badness += BADNESS_BAD_MODE;
}

if (inode.i_faddr) {
@@ -1318,6 +1323,7 @@ extern int e2fsck_process_bad_inode(e2fs
inode_modified++;
} else
not_fixed++;
+ badness += BADNESS_NORMAL;
}

switch (fs->super->s_creator_os) {
@@ -1339,6 +1345,7 @@ extern int e2fsck_process_bad_inode(e2fs
inode_modified++;
} else
not_fixed++;
+ badness += BADNESS_NORMAL;
pctx.num = 0;
}
if (fsize && *fsize) {
@@ -1348,11 +1355,28 @@ extern int e2fsck_process_bad_inode(e2fs
inode_modified++;
} else
not_fixed++;
+ badness += BADNESS_NORMAL;
pctx.num = 0;
}

+ /* In pass1 these conditions were used to mark inode bad so that
+ * it calls e2fsck_process_bad_inode and make an extensive check
+ * plus prompt for action to be taken. To compensate for badness
+ * incremented in pass1 by this condition, decrease it.
+ */
+ if ((inode.i_faddr || frag || fsize ||
+ (LINUX_S_ISDIR(inode.i_mode) && inode.i_dir_acl)) ||
+ (inode.i_file_acl &&
+ (!(fs->super->s_feature_compat & EXT2_FEATURE_COMPAT_EXT_ATTR) ||
+ (inode.i_file_acl < fs->super->s_first_data_block) ||
+ (inode.i_file_acl >= fs->super->s_blocks_count)))) {
+ /* badness can be 0 if called from pass4. */
+ if (badness)
+ badness -= BADNESS_NORMAL;
+ }
+
if ((fs->super->s_creator_os == EXT2_OS_LINUX) &&
- !(fs->super->s_feature_ro_compat &
+ !(fs->super->s_feature_ro_compat &
EXT4_FEATURE_RO_COMPAT_HUGE_FILE) &&
(inode.osd2.linux2.l_i_blocks_hi != 0)) {
pctx.num = inode.osd2.linux2.l_i_blocks_hi;
@@ -1360,6 +1384,8 @@ extern int e2fsck_process_bad_inode(e2fs
inode.osd2.linux2.l_i_blocks_hi = 0;
inode_modified++;
}
+ /* Badness was increased in pass1 for this condition */
+ /* badness += BADNESS_NORMAL; */
}

if (inode.i_file_acl &&
@@ -1370,6 +1396,7 @@ extern int e2fsck_process_bad_inode(e2fs
inode_modified++;
} else
not_fixed++;
+ badness += BADNESS_NORMAL;
}
if (inode.i_dir_acl &&
LINUX_S_ISDIR(inode.i_mode)) {
@@ -1378,12 +1405,28 @@ extern int e2fsck_process_bad_inode(e2fs
inode_modified++;
} else
not_fixed++;
+ badness += BADNESS_NORMAL;
+ }
+
+ /*
+ * The high value due to BADNESS_BAD_MODE should not delete the inode.
+ */
+ if ((badness - ((badness >= BADNESS_BAD_MODE) ? BADNESS_BAD_MODE : 0))>=
+ ctx->inode_badness_threshold) {
+ pctx.num = badness;
+ if (fix_problem(ctx, PR_2_INODE_TOOBAD, &pctx)) {
+ deallocate_inode(ctx, ino, 0);
+ if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+ return 0;
+ return 1;
+ }
+ not_fixed++;
}

if (inode_modified)
e2fsck_write_inode(ctx, ino, &inode, "process_bad_inode");
- if (!not_fixed && ctx->inode_bad_map)
- ext2fs_unmark_inode_bitmap(ctx->inode_bad_map, ino);
+ if (ctx->inode_badness)
+ ext2fs_icount_store(ctx->inode_badness, ino, 0);
return 0;
}

Index: e2fsprogs-1.40.4/e2fsck/problem.c
===================================================================
--- e2fsprogs-1.40.4.orig/e2fsck/problem.c
+++ e2fsprogs-1.40.4/e2fsck/problem.c
@@ -1316,6 +1316,11 @@ static struct e2fsck_problem problem_tab
N_("@i %i found in @g %g unused inodes area. "),
PROMPT_FIX, PR_PREEN_OK },

+ /* Inode too bad */
+ { PR_2_INODE_TOOBAD,
+ N_("@i %i is badly corrupt (badness value = %N). "),
+ PROMPT_CLEAR, PR_PREEN_OK },
+
/* Pass 3 errors */

/* Pass 3: Checking directory connectivity */
Index: e2fsprogs-1.40.4/e2fsck/problem.h
===================================================================
--- e2fsprogs-1.40.4.orig/e2fsck/problem.h
+++ e2fsprogs-1.40.4/e2fsck/problem.h
@@ -792,6 +792,9 @@ struct problem_context {
/* Inode found in group unused inodes area */
#define PR_2_INOREF_IN_UNUSED 0x020046

+/* Inode completely corrupt */
+#define PR_2_INODE_TOOBAD 0x020047
+
/*
* Pass 3 errors
*/
Index: e2fsprogs-1.40.4/lib/ext2fs/icount.c
===================================================================
--- e2fsprogs-1.40.4.orig/lib/ext2fs/icount.c
+++ e2fsprogs-1.40.4/lib/ext2fs/icount.c
@@ -462,6 +462,23 @@ static errcode_t get_inode_count(ext2_ic
return 0;
}

+int ext2fs_icount_is_set(ext2_icount_t icount, ext2_ino_t ino)
+{
+ __u16 result;
+
+ if (ext2fs_test_inode_bitmap(icount->single, ino))
+ return 1;
+ else if (icount->multiple) {
+ if (ext2fs_test_inode_bitmap(icount->multiple, ino))
+ return 1;
+ return 0;
+ }
+ ext2fs_icount_fetch(icount, ino, &result);
+ if (result)
+ return 1;
+ return 0;
+}
+
errcode_t ext2fs_icount_validate(ext2_icount_t icount, FILE *out)
{
errcode_t ret = 0;
@@ -501,6 +518,7 @@ errcode_t ext2fs_icount_fetch32(ext2_ico
*ret = 0;
return 0;
}
+
get_inode_count(icount, ino, ret);
return 0;
}
Index: e2fsprogs-1.40.4/e2fsck/pass1b.c
===================================================================
--- e2fsprogs-1.40.4.orig/e2fsck/pass1b.c
+++ e2fsprogs-1.40.4/e2fsck/pass1b.c
@@ -613,8 +613,8 @@ static void delete_file(e2fsck_t ctx, ex
fix_problem(ctx, PR_1B_BLOCK_ITERATE, &pctx);
ext2fs_unmark_inode_bitmap(ctx->inode_used_map, ino);
ext2fs_unmark_inode_bitmap(ctx->inode_dir_map, ino);
- if (ctx->inode_bad_map)
- ext2fs_unmark_inode_bitmap(ctx->inode_bad_map, ino);
+ if (ctx->inode_badness)
+ e2fsck_mark_inode_bad(ctx, ino, 0);
ext2fs_inode_alloc_stats2(fs, ino, -1, LINUX_S_ISDIR(inode.i_mode));

/* Inode may have changed by block_iterate, so reread it */
Index: e2fsprogs-1.40.4/e2fsck/unix.c
===================================================================
--- e2fsprogs-1.40.4.orig/e2fsck/unix.c
+++ e2fsprogs-1.40.4/e2fsck/unix.c
@@ -624,6 +624,18 @@ static void parse_extended_opts(e2fsck_t
extended_usage++;
continue;
}
+ /* -E inode_badness_threshold=<value> */
+ } else if (strcmp(token, "inode_badness_threshold") == 0) {
+ if (!arg) {
+ extended_usage++;
+ continue;
+ }
+ ctx->inode_badness_threshold = strtoul(arg, &p, 0);
+ if (*p != '\0' || (ctx->inode_badness_threshold > 200)){
+ fprintf(stderr, _("Invalid badness value.\n"));
+ extended_usage++;
+ continue;
+ }
} else {
fprintf(stderr, _("Unknown extended option: %s\n"),
token);
@@ -639,6 +646,7 @@ static void parse_extended_opts(e2fsck_t
"\tshared=<preserve|lost+found|delete>\n"
"\tclone=<dup|zero>\n"
"\tea_ver=<ea_version (1 or 2)>\n"
+ "\tinode_badness_threhold=(value)\n"
"\texpand_extra_isize\n"
"\n"), stderr);
exit(1);
@@ -703,6 +711,9 @@ static errcode_t PRS(int argc, char *arg
profile_init(config_fn, &ctx->profile);
initialize_profile_options(ctx);

+ ctx->inode_badness_threshold = BADNESS_THRESHOLD;
+ ctx->now_tolerance_val = 172800; /* Two days */
+
while ((c = getopt (argc, argv, "panyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDk")) != EOF)
switch (c) {
case 'C':
Index: e2fsprogs-1.40.4/e2fsck/e2fsck.c
===================================================================
--- e2fsprogs-1.40.4.orig/e2fsck/e2fsck.c
+++ e2fsprogs-1.40.4/e2fsck/e2fsck.c
@@ -105,10 +105,6 @@ errcode_t e2fsck_reset_context(e2fsck_t
ext2fs_free_inode_bitmap(ctx->inode_bb_map);
ctx->inode_bb_map = 0;
}
- if (ctx->inode_bad_map) {
- ext2fs_free_inode_bitmap(ctx->inode_bad_map);
- ctx->inode_bad_map = 0;
- }
if (ctx->inode_imagic_map) {
ext2fs_free_inode_bitmap(ctx->inode_imagic_map);
ctx->inode_imagic_map = 0;
Index: e2fsprogs-1.40.4/e2fsck/e2fsck.8.in
===================================================================
--- e2fsprogs-1.40.4.orig/e2fsck/e2fsck.8.in
+++ e2fsprogs-1.40.4/e2fsck/e2fsck.8.in
@@ -191,6 +191,13 @@ in place (preserve);
Assume the format of the extended attribute blocks in the filesystem is
the specified version number. The version number may be 1 or 2. The
default extended attribute version format is 2.
+.TP
+.BI inode_badness_threshold= threshold_value
+A badness counter is associated with every inode, which determines the degree
+of inode corruption. Each error found in the inode will increase the badness by
+1 or 2, and inodes with a badness at or above
+.I threshold_value will be prompted for deletion. The default
+.I threshold_value is 7.
.RE
.TP
.B \-f
Index: e2fsprogs-1.40.4/tests/f_bad_disconnected_inode/expect.1
===================================================================
--- e2fsprogs-1.40.4.orig/tests/f_bad_disconnected_inode/expect.1
+++ e2fsprogs-1.40.4/tests/f_bad_disconnected_inode/expect.1
@@ -39,10 +39,7 @@ Clear? yes
i_blocks_hi for inode 16 (...) is 62762, should be zero.
Clear? yes

-Unattached inode 16
-Connect to /lost+found? yes
-
-Inode 16 ref count is 5925, should be 1. Fix? yes
+Inode 16 is badly corrupt (badness value = 10). Clear? yes

Pass 5: Checking group summary information
Block bitmap differences: -(9--19)
@@ -54,19 +51,16 @@ Fix? yes
Free blocks count wrong (79, counted=91).
Fix? yes

-Inode bitmap differences: +16
-Fix? yes
-
-Free inodes count wrong for group #0 (7, counted=4).
+Free inodes count wrong for group #0 (8, counted=5).
Fix? yes

Directories count wrong for group #0 (3, counted=2).
Fix? yes

-Free inodes count wrong (7, counted=4).
+Free inodes count wrong (8, counted=5).
Fix? yes

test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
-test_filesys: 12/16 files (0.0% non-contiguous), 9/100 blocks
+test_filesys: 11/16 files (0.0% non-contiguous), 9/100 blocks
Exit status is 1
Index: e2fsprogs-1.40.4/tests/f_bad_disconnected_inode/expect.2
===================================================================
--- e2fsprogs-1.40.4.orig/tests/f_bad_disconnected_inode/expect.2
+++ e2fsprogs-1.40.4/tests/f_bad_disconnected_inode/expect.2
@@ -3,5 +3,5 @@ Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
-test_filesys: 12/16 files (0.0% non-contiguous), 9/100 blocks
+test_filesys: 11/16 files (0.0% non-contiguous), 9/100 blocks
Exit status is 0
Index: e2fsprogs-1.40.4/lib/ext2fs/ext2fs.h
===================================================================
--- e2fsprogs-1.40.4.orig/lib/ext2fs/ext2fs.h
+++ e2fsprogs-1.40.4/lib/ext2fs/ext2fs.h
@@ -831,6 +831,7 @@ extern errcode_t ext2fs_initialize(const

/* icount.c */
extern void ext2fs_free_icount(ext2_icount_t icount);
+extern int ext2fs_icount_is_set(ext2_icount_t icount, ext2_ino_t ino);
extern errcode_t ext2fs_create_icount_tdb(ext2_filsys fs, char *tdb_dir,
int flags, ext2_icount_t *ret);
extern errcode_t ext2fs_create_icount2(ext2_filsys fs, int flags,

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.

2008-02-02 08:43:46

On Mar 15, 2008 15:41 -0400, Theodore Ts'o wrote:
> What is the intended use of the SF_DO_CSUM flag? I see where it is
> defined, and where it gets sets, but as far as I can tell nothing
> actually tests for it or uses it.

Probably a hold-over from a previous version of the code. It sets the
SF_DO_LAZY flag if GDT_CSUM is set, causing ext2fs_get_next_inode_full()
to skip the BG_INODE_UNINT groups. It's up to you if it would be better
to keep SF_DO_CSUM and check for it explicitly (possibly using it for
something else later), or to overload SF_DO_LAZY as we do currently.

> On Sat, Feb 02, 2008 at 01:34:44AM -0700, Andreas Dilger wrote:
> > Index: e2fsprogs-1.40.5/lib/ext2fs/ext2fs.h
> >
> > +#define EXT2_SF_DO_CSUM 0x0020
> >
>
> > Index: e2fsprogs-1.40.5/lib/ext2fs/inode.c
> > ===================================================================
> > --- e2fsprogs-1.40.5.orig/lib/ext2fs/inode.c
> > +++ e2fsprogs-1.40.5/lib/ext2fs/inode.c
> > @@ -167,6 +167,9 @@ errcode_t ext2fs_open_inode_scan(ext2_fi
> > if (EXT2_HAS_COMPAT_FEATURE(fs->super,
> > EXT2_FEATURE_COMPAT_LAZY_BG))
> > scan->scan_flags |= EXT2_SF_DO_LAZY;
> > + if (EXT2_HAS_RO_COMPAT_FEATURE(fs->super,
> > + EXT4_FEATURE_RO_COMPAT_GDT_CSUM))
> > + scan->scan_flags |= EXT2_SF_DO_LAZY | EXT2_SF_DO_CSUM;

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.

2008-03-17 12:33:51

by Theodore Ts'o

[permalink] [raw]

Subject: Re: [PATCH][10/28] e2fsprogs-uninit.patch

On Sat, Feb 02, 2008 at 01:34:44AM -0700, Andreas Dilger wrote:
> Index: e2fsprogs-1.40.5/e2fsck/super.c
> ===================================================================
> @@ -626,6 +631,50 @@ void check_super_block(e2fsck_t ctx)
...
> + if (!ext2fs_group_desc_csum_verify(sb, i, gd)) {
> + if (fix_problem(ctx, PR_0_GDT_CSUM, &pctx)) {
> + gd->bg_flags &= ~(EXT2_BG_BLOCK_UNINIT |
> + EXT2_BG_INODE_UNINIT);
> + gd->bg_itable_unused = 0;
> + }
> + ext2fs_unmark_valid(fs);
> + }
> +
...
> +
> + gd->bg_checksum = ext2fs_group_desc_csum(fs->super, i, gd);

This last looks horribly wrong. check_super_block() is merely
supposed to check to see if the superblock and block gorup descriptos
looks OK, and to mark the filesystem as invalid if anything looks
insane. It should *not* modifying the block group descriptor, and it
certainly should not be doing so without first checking to see if the
filesystem has been opened read/only or calling
ext2fs_mark_super_dirty(fs) after making a change (that it shouldn't
do).

In fact there already is a check to see if the checksum has verified
correctly (see the first part of the patch which I quoted), so I think
the best thing to do is to remove that last bit. I'm not sure why
it's there at all, in fact....

- Ted