From: Wang Shilong <[email protected]>
From: Wang Shilong <[email protected]>
Currently it has been popular that single disk could be more than TiB,
etc 16Tib with only one single disk, with this trend, one single
filesystem could be larger and larger and easily reach PiB with LUN system.
The journal filesystem like ext4 need be offline to do regular
check and repair from time to time, however the problem is e2fsck
still do this using single thread, this could be challenging at scale
for two reasons:
1) even with readahead, IO speed still limits several tens MiB per second.
2) could not utilize CPU cores.
It could be challenging to try multh-threads for all phase of e2fsck, but as
first step, we might try this for most time-consuming pass1, according to
our benchmarking it cost of 80% time for whole e2fck phase.
Pass1 is trying to scanning all valid inode of filesystem and check it one by
one, and the patchset idea is trying to split these to different threads and
trying to do this at the same time, we try to merge these inodes and corresponding
inode's extent information after threads finish.
To simplify complexity and make it less error-prone, the fix is still serialized,
since most of time there will be only minor errors for filesystem, what's important
for us is parallel reading and checking.
Here is a benchmarking on our Lustre filesystem with 1.2 PiB OSD ext4 based
filesystem:
DDN SFA18KE StorageServer
DCR(DeClustering RAID) with 162 x HGST 10TB NL-SAS
Tested Server
A Virtual Machine running on SFA18KE
8 x CPU cores (Xeon(R) Gold 6140)
150GB memory
CentoOS7.7 (Lustre patched kernel)
Created 600 Million x 32K byte files.
Without Patch With Patch thr=64
pass1: 13079.66 488.57 seconds
Total: 15673.33 3188.42
We have 5x total time reduction of total time which is very inspiring.
I've tested the whole patch series using 'make test' of e2fsck itself, and i
manually set default threads to 4 which still pass almost of test suite,
failure cases are below:
f_h_badroot f_multithread f_multithread_logfile f_multithread_no
h_h_badroot failed because out of order checking output, and others are because
of extra multiple threads log output.
The other test i've done to verify is:
1) filled a filesystem with different features enabled(encryption, shared xattr)
2) copied the above image out using e2image.
3) using e2fuzz to randomly corrupt above image, and then have another copy of
corrupted image.
4) running e2fsck with single thread(without patches) and multiple threads with
corrupted image respectively.
5) using valgrind to check any memory leak.
6) compare single thread and multiple threads fix results by mounting two images
back and compare the directores and files.(size and name match)
So the whole series is reasonably stable if you are intrested testing it on
different platforms, i've pushed it to github:
https://github.com/wangshilong/e2fsprogs pfsck_pass1_v2
It is definitely in early stage, any review or comments are welcomed!
Thanks you very much!
Shilong
changelog v1->v2:
1) fixed shared xattr handling merging.
2) fix serveral memory leaks problem which was detected by valgrind.
3) stop any checking while there is fixing thread active.
4) fix the OOM problem when runing 128 threads on 3TiB NVME filesystem.
5) limit multiple threads memory usage.
6) some other minor extra fixes
Li Xi (25):
e2fsck: add -m option for multithread
e2fsck: copy context when using multi-thread fsck
e2fsck: copy fs when using multi-thread fsck
e2fsck: copy dblist when using multi-thread fsck
e2fsck: clear icache when using multi-thread fsck
e2fsck: add assert when copying context
e2fsck: copy bitmaps when copying context
e2fsck: copy badblocks when copying fs
e2fsck: open io-channel when copying fs
e2fsck: create logs for mult-threads
e2fsck: create one thread to fsck
e2fsck: add start/end group for thread
e2fsck: split groups to different threads
e2fsck: print thread log properly
e2fsck: merge bitmaps after thread completes
e2fsck: do not change global variables
e2fsck: optimize the inserting of dir_info_db
e2fsck: merge dir_info after thread finishes
e2fsck: rbtree bitmap for dir
e2fsck: merge badblocks after thread finishes
e2fsck: merge icounts after thread finishes
e2fsck: merge dblist after thread finishes
e2fsck: add debug codes for multiple threds
e2fsck: merge counts when threads finish
LU-8465 e2fsck: merge fs flags when threads finish
Wang Shilong (26):
e2fsck: cleanup struct e2fsck_struct
e2fsck: remove unused fs_ext_attr_inodes/blocks
e2fsck: merge dx_dir_info
e2fsck: make threads splitting aware of flex_bg
e2fsck: merge dirs_to_hash when threads finish
e2fsck: merge context flags properly
e2fsck: split and merge quota context
e2fsck: serialize fix operations
e2fsck: move some fixes out of parallel pthreads
e2fsck: split and merge invalid bitmaps
e2fsck: fix to protect EA checking
e2fsck: allow admin specify number of threads
e2fsck: kickoff mutex lock for block found map
e2fsck: fix readahead for pfsck of pass1
e2fsck: kick off ea mutex lock from pfsck
e2fsck: merge encrypted_files after threads finish
e2fsck: merge inode_bad_map after threads finish
e2fsck: simplify e2fsck context merging codes
e2fsck: merge options after threads finish
e2fsck: reset lost_and_found after threads finish
LU-8465 e2fsck: merge extent depth count after threads finish
e2fsck: only set E2F_FLAG_ALLOC_OK if all threads succeed
e2fsck: only setup threads if -m option required
e2fsck: wait fix thread finish before checking
e2fsck: fix to free icache leak
e2fsck: fix to avoid too much memory allocation for pfsck
configure.ac | 6 +
e2fsck/dirinfo.c | 237 ++-
e2fsck/dx_dirinfo.c | 65 +
e2fsck/e2fsck.h | 328 ++--
e2fsck/encrypted_files.c | 175 ++-
e2fsck/logfile.c | 12 +-
e2fsck/pass1.c | 1912 ++++++++++++++++++++---
e2fsck/problem.c | 9 +
e2fsck/problem.h | 3 +
e2fsck/unix.c | 33 +-
e2fsck/util.c | 56 +-
lib/ext2fs/badblocks.c | 75 +-
lib/ext2fs/bitmaps.c | 10 +
lib/ext2fs/bitops.h | 2 +
lib/ext2fs/blkmap64_rb.c | 63 +
lib/ext2fs/bmap64.h | 4 +
lib/ext2fs/dblist.c | 38 +
lib/ext2fs/ext2_err.et.in | 3 +
lib/ext2fs/ext2_io.h | 2 +
lib/ext2fs/ext2fs.h | 16 +-
lib/ext2fs/ext2fsP.h | 1 -
lib/ext2fs/gen_bitmap64.c | 74 +-
lib/ext2fs/icount.c | 108 ++
lib/ext2fs/undo_io.c | 19 +
lib/ext2fs/unix_io.c | 24 +-
lib/support/mkquota.c | 19 +
lib/support/quotaio.h | 2 +
tests/f_itable_collision/expect.1 | 3 -
tests/f_multithread/expect.1 | 25 +
tests/f_multithread/expect.2 | 7 +
tests/f_multithread/image.gz | 1 +
tests/f_multithread/name | 1 +
tests/f_multithread/script | 4 +
tests/f_multithread_completion/expect.1 | 2 +
tests/f_multithread_completion/expect.2 | 23 +
tests/f_multithread_completion/image.gz | 1 +
tests/f_multithread_completion/name | 1 +
tests/f_multithread_completion/script | 4 +
tests/f_multithread_logfile/expect.1 | 25 +
tests/f_multithread_logfile/image.gz | 1 +
tests/f_multithread_logfile/name | 1 +
tests/f_multithread_logfile/script | 32 +
tests/f_multithread_no/expect.1 | 26 +
tests/f_multithread_no/expect.2 | 23 +
tests/f_multithread_no/image.gz | 1 +
tests/f_multithread_no/name | 1 +
tests/f_multithread_no/script | 4 +
tests/f_multithread_preen/expect.1 | 11 +
tests/f_multithread_preen/expect.2 | 23 +
tests/f_multithread_preen/image.gz | 1 +
tests/f_multithread_preen/name | 1 +
tests/f_multithread_preen/script | 4 +
tests/f_multithread_yes/expect.1 | 2 +
tests/f_multithread_yes/expect.2 | 23 +
tests/f_multithread_yes/image.gz | 1 +
tests/f_multithread_yes/name | 1 +
tests/f_multithread_yes/script | 4 +
57 files changed, 3049 insertions(+), 504 deletions(-)
create mode 100644 tests/f_multithread/expect.1
create mode 100644 tests/f_multithread/expect.2
create mode 120000 tests/f_multithread/image.gz
create mode 100644 tests/f_multithread/name
create mode 100644 tests/f_multithread/script
create mode 100644 tests/f_multithread_completion/expect.1
create mode 100644 tests/f_multithread_completion/expect.2
create mode 120000 tests/f_multithread_completion/image.gz
create mode 100644 tests/f_multithread_completion/name
create mode 100644 tests/f_multithread_completion/script
create mode 100644 tests/f_multithread_logfile/expect.1
create mode 120000 tests/f_multithread_logfile/image.gz
create mode 100644 tests/f_multithread_logfile/name
create mode 100644 tests/f_multithread_logfile/script
create mode 100644 tests/f_multithread_no/expect.1
create mode 100644 tests/f_multithread_no/expect.2
create mode 120000 tests/f_multithread_no/image.gz
create mode 100644 tests/f_multithread_no/name
create mode 100644 tests/f_multithread_no/script
create mode 100644 tests/f_multithread_preen/expect.1
create mode 100644 tests/f_multithread_preen/expect.2
create mode 120000 tests/f_multithread_preen/image.gz
create mode 100644 tests/f_multithread_preen/name
create mode 100644 tests/f_multithread_preen/script
create mode 100644 tests/f_multithread_yes/expect.1
create mode 100644 tests/f_multithread_yes/expect.2
create mode 120000 tests/f_multithread_yes/image.gz
create mode 100644 tests/f_multithread_yes/name
create mode 100644 tests/f_multithread_yes/script
--
2.25.4
From: Wang Shilong <[email protected]>
Only define but not used, remove them.
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/e2fsck.h | 2 --
1 file changed, 2 deletions(-)
diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
index f403360e..b9e2f06e 100644
--- a/e2fsck/e2fsck.h
+++ b/e2fsck/e2fsck.h
@@ -419,8 +419,6 @@ struct e2fsck_struct {
__u32 fs_fragmented;
__u32 fs_fragmented_dir;
__u32 large_files;
- __u32 fs_ext_attr_inodes;
- __u32 fs_ext_attr_blocks;
__u32 extent_depth_count[MAX_EXTENT_DEPTH_COUNT];
};
--
2.25.4
From: Wang Shilong <[email protected]>
Fields of "struct e2fsck_struct" are seperated into
different types according to how these fields will be
used when parallel fsck is enabled.
remove unused @abort_code
Signed-off-by: Li Xi <[email protected]>
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/e2fsck.h | 274 ++++++++++++++++++++++++------------------------
1 file changed, 139 insertions(+), 135 deletions(-)
diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
index 9b2b9ce8..f403360e 100644
--- a/e2fsck/e2fsck.h
+++ b/e2fsck/e2fsck.h
@@ -227,91 +227,153 @@ typedef struct e2fsck_struct *e2fsck_t;
#define MAX_EXTENT_DEPTH_COUNT 5
struct e2fsck_struct {
- ext2_filsys fs;
- const char *program_name;
- char *filesystem_name;
- char *device_name;
- char *io_options;
- FILE *logf;
- char *log_fn;
- FILE *problem_logf;
- char *problem_log_fn;
- int flags; /* E2fsck internal flags */
- int options;
- unsigned blocksize; /* blocksize */
- blk64_t use_superblock; /* sb requested by user */
- blk64_t superblock; /* sb used to open fs */
- blk64_t num_blocks; /* Total number of blocks */
- blk64_t free_blocks;
- ext2_ino_t free_inodes;
- int mount_flags;
- int openfs_flags;
- blkid_cache blkid; /* blkid cache */
-
+ /* ---- Following fields are never updated during the pass1 ---- */
+ const char *program_name;
+ char *filesystem_name;
+ char *device_name;
+ char *io_options;
+ int options; /* E2F_OPT_* flags */
+ int blocksize; /* blocksize */
+ blk64_t use_superblock; /* sb requested by user */
+ blk64_t superblock; /* sb used to open fs */
+ blk64_t num_blocks; /* Total number of blocks */
+ blk64_t free_blocks;
+ ext2_ino_t free_inodes;
+ int mount_flags;
+ int openfs_flags;
+ blkid_cache blkid; /* blkid cache */
#ifdef HAVE_SETJMP_H
- jmp_buf abort_loc;
+ jmp_buf abort_loc;
#endif
- unsigned long abort_code;
+#ifdef RESOURCE_TRACK
+ /*
+ * For timing purposes
+ */
+ struct resource_track global_rtrack;
+#endif
+ int bad_lost_and_found;
+ /*
+ * Tuning parameters
+ */
+ int process_inode_size;
+ int inode_buffer_blocks;
+ unsigned int htree_slack_percentage;
+
+ /*
+ * ext3 journal support
+ */
+ io_channel journal_io;
+ char *journal_name;
+ /* misc fields */
+ time_t now;
+ /* For working around buggy init scripts */
+ time_t time_fudge;
+
+ int ext_attr_ver;
+ int blocks_per_page;
+ /* Are we connected directly to a tty? */
+ int interactive;
+ char start_meta[2], stop_meta[2];
+ /*
+ * For the use of callers of the e2fsck functions; not used by
+ * e2fsck functions themselves.
+ */
+ void *priv_data;
+ /* Undo file */
+ char *undo_file;
+ /* How much are we allowed to readahead? */
+ unsigned long long readahead_kb;
+ /* ---- Following fields are shared by different threads for pass1 -*/
+ /* E2fsck internal flags */
+ int flags;
+ /*
+ * How we display the progress update (for unix)
+ */
+ int progress_fd;
+ int progress_pos;
+ int progress_last_percent;
+ unsigned int progress_last_time;
int (*progress)(e2fsck_t ctx, int pass, unsigned long cur,
unsigned long max);
-
- ext2fs_inode_bitmap inode_used_map; /* Inodes which are in use */
- ext2fs_inode_bitmap inode_bad_map; /* Inodes which are bad somehow */
- ext2fs_inode_bitmap inode_dir_map; /* Inodes which are directories */
- ext2fs_inode_bitmap inode_bb_map; /* Inodes which are in bad blocks */
- ext2fs_inode_bitmap inode_imagic_map; /* AFS inodes */
- ext2fs_inode_bitmap inode_reg_map; /* Inodes which are regular files*/
-
- ext2fs_block_bitmap block_found_map; /* Blocks which are in use */
- ext2fs_block_bitmap block_dup_map; /* Blks referenced more than once */
- ext2fs_block_bitmap block_ea_map; /* Blocks which are used by EA's */
+ /* Metadata blocks */
+ ext2fs_block_bitmap block_metadata_map;
+ profile_t profile;
+ /* Reserve blocks for root and l+f re-creation */
+ blk64_t root_repair_block, lnf_repair_block;
+ /*
+ * Location of the lost and found directory
+ */
+ ext2_ino_t lost_and_found;
+
+ /* ---- Following fields are seperated for each thread for pass1- */
+ ext2_filsys fs;
+ FILE *logf;
+ char *log_fn;
+ FILE *problem_logf;
+ char *problem_log_fn;
+
+ /* Inodes which are in use */
+ ext2fs_inode_bitmap inode_used_map;
+ /* Inodes which are bad somehow */
+ ext2fs_inode_bitmap inode_bad_map;
+ /* Inodes which are directories */
+ ext2fs_inode_bitmap inode_dir_map;
+ /* Inodes which are in bad blocks */
+ ext2fs_inode_bitmap inode_bb_map;
+ /* AFS inodes */
+ ext2fs_inode_bitmap inode_imagic_map;
+ /* Inodes which are regular files */
+ ext2fs_inode_bitmap inode_reg_map;
+ /* Inodes to rebuild extent trees */
+ ext2fs_inode_bitmap inodes_to_rebuild;
+ /* Blocks which are in use */
+ ext2fs_block_bitmap block_found_map;
+ /* Blks referenced more than once */
+ ext2fs_block_bitmap block_dup_map;
+ /* Blocks which are used by EA's */
+ ext2fs_block_bitmap block_ea_map;
/*
* Inode count arrays
*/
- ext2_icount_t inode_count;
- ext2_icount_t inode_link_info;
+ ext2_icount_t inode_count;
+ ext2_icount_t inode_link_info;
- ext2_refcount_t refcount;
- ext2_refcount_t refcount_extra;
+ ext2_refcount_t refcount;
+ ext2_refcount_t refcount_extra;
/*
* Quota blocks and inodes to be charged for each ea block.
*/
- ext2_refcount_t ea_block_quota_blocks;
- ext2_refcount_t ea_block_quota_inodes;
+ ext2_refcount_t ea_block_quota_blocks;
+ ext2_refcount_t ea_block_quota_inodes;
/*
* ea_inode references from attr entries.
*/
- ext2_refcount_t ea_inode_refs;
+ ext2_refcount_t ea_inode_refs;
/*
* Array of flags indicating whether an inode bitmap, block
* bitmap, or inode table is invalid
*/
- int *invalid_inode_bitmap_flag;
- int *invalid_block_bitmap_flag;
- int *invalid_inode_table_flag;
- int invalid_bitmaps; /* There are invalid bitmaps/itable */
+ int *invalid_inode_bitmap_flag;
+ int *invalid_block_bitmap_flag;
+ int *invalid_inode_table_flag;
+ /* There are invalid bitmaps/itable */
+ int invalid_bitmaps;
/*
* Block buffer
*/
- char *block_buf;
+ char *block_buf;
/*
* For pass1_check_directory and pass1_get_blocks
*/
- ext2_ino_t stashed_ino;
- struct ext2_inode *stashed_inode;
-
- /*
- * Location of the lost and found directory
- */
- ext2_ino_t lost_and_found;
- int bad_lost_and_found;
+ ext2_ino_t stashed_ino;
+ struct ext2_inode *stashed_inode;
/*
* Directory information
@@ -328,96 +390,38 @@ struct e2fsck_struct {
/*
* Directories to hash
*/
- ext2_u32_list dirs_to_hash;
+ ext2_u32_list dirs_to_hash;
/*
* Encrypted file information
*/
struct encrypted_file_info *encrypted_files;
- /*
- * Tuning parameters
- */
- int process_inode_size;
- int inode_buffer_blocks;
- unsigned int htree_slack_percentage;
-
- /*
- * ext3 journal support
- */
- io_channel journal_io;
- char *journal_name;
-
/*
* Ext4 quota support
*/
- quota_ctx_t qctx;
-#ifdef RESOURCE_TRACK
- /*
- * For timing purposes
- */
- struct resource_track global_rtrack;
-#endif
-
- /*
- * How we display the progress update (for unix)
- */
- int progress_fd;
- int progress_pos;
- int progress_last_percent;
- unsigned int progress_last_time;
- int interactive; /* Are we connected directly to a tty? */
- char start_meta[2], stop_meta[2];
-
- /* File counts */
- __u32 fs_directory_count;
- __u32 fs_regular_count;
- __u32 fs_blockdev_count;
- __u32 fs_chardev_count;
- __u32 fs_links_count;
- __u32 fs_symlinks_count;
- __u32 fs_fast_symlinks_count;
- __u32 fs_fifo_count;
- __u32 fs_total_count;
- __u32 fs_badblocks_count;
- __u32 fs_sockets_count;
- __u32 fs_ind_count;
- __u32 fs_dind_count;
- __u32 fs_tind_count;
- __u32 fs_fragmented;
- __u32 fs_fragmented_dir;
- __u32 large_files;
- __u32 fs_ext_attr_inodes;
- __u32 fs_ext_attr_blocks;
- __u32 extent_depth_count[MAX_EXTENT_DEPTH_COUNT];
-
- /* misc fields */
- time_t now;
- time_t time_fudge; /* For working around buggy init scripts */
- int ext_attr_ver;
- profile_t profile;
- int blocks_per_page;
-
- /* Reserve blocks for root and l+f re-creation */
- blk64_t root_repair_block, lnf_repair_block;
-
- /*
- * For the use of callers of the e2fsck functions; not used by
- * e2fsck functions themselves.
- */
- void *priv_data;
- ext2fs_block_bitmap block_metadata_map; /* Metadata blocks */
-
- /* How much are we allowed to readahead? */
- unsigned long long readahead_kb;
-
- /*
- * Inodes to rebuild extent trees
- */
- ext2fs_inode_bitmap inodes_to_rebuild;
-
- /* Undo file */
- char *undo_file;
+ quota_ctx_t qctx;
+
+ __u32 fs_directory_count;
+ __u32 fs_regular_count;
+ __u32 fs_blockdev_count;
+ __u32 fs_chardev_count;
+ __u32 fs_links_count;
+ __u32 fs_symlinks_count;
+ __u32 fs_fast_symlinks_count;
+ __u32 fs_fifo_count;
+ __u32 fs_total_count;
+ __u32 fs_badblocks_count;
+ __u32 fs_sockets_count;
+ __u32 fs_ind_count;
+ __u32 fs_dind_count;
+ __u32 fs_tind_count;
+ __u32 fs_fragmented;
+ __u32 fs_fragmented_dir;
+ __u32 large_files;
+ __u32 fs_ext_attr_inodes;
+ __u32 fs_ext_attr_blocks;
+ __u32 extent_depth_count[MAX_EXTENT_DEPTH_COUNT];
};
/* Data structures to evaluate whether an extent tree needs rebuilding. */
--
2.25.4
From: Li Xi <[email protected]>
This patch only copy the context to a new one when -m is enabled.
It doesn't actually start any thread. When pass1 test finishes,
the new context is copied back to the original context.
Since the signal handler only changes the original context, so
add global_ctx in "struct e2fsck_struct" and use that to check
whether there is any signal of canceling.
This patch handles the long jump properly so that all the existing
tests can be passed even the context has been copied. Otherwise,
test f_expisize_ea_del would fail when aborting.
Signed-off-by: Li Xi <[email protected]>
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/e2fsck.h | 2 +
e2fsck/pass1.c | 116 ++++++++++++++++++++++++++++++++++++++++++++----
e2fsck/unix.c | 1 +
3 files changed, 110 insertions(+), 9 deletions(-)
diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
index 8b7e1276..46792c9b 100644
--- a/e2fsck/e2fsck.h
+++ b/e2fsck/e2fsck.h
@@ -229,6 +229,8 @@ typedef struct e2fsck_struct *e2fsck_t;
struct e2fsck_struct {
/* ---- Following fields are never updated during the pass1 ---- */
+ /* Global context to get the cancel flag */
+ e2fsck_t global_ctx;
const char *program_name;
char *filesystem_name;
char *device_name;
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 8eecd958..a9244201 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -1144,7 +1144,22 @@ static int quota_inum_is_reserved(ext2_filsys fs, ext2_ino_t ino)
return 0;
}
-void e2fsck_pass1(e2fsck_t ctx)
+static int e2fsck_should_abort(e2fsck_t ctx)
+{
+ e2fsck_t global_ctx;
+
+ if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+ return 1;
+
+ if (ctx->global_ctx) {
+ global_ctx = ctx->global_ctx;
+ if (global_ctx->flags & E2F_FLAG_SIGNAL_MASK)
+ return 1;
+ }
+ return 0;
+}
+
+void e2fsck_pass1_thread(e2fsck_t ctx)
{
int i;
__u64 max_sizes;
@@ -1360,7 +1375,7 @@ void e2fsck_pass1(e2fsck_t ctx)
if (ino > ino_threshold)
pass1_readahead(ctx, &ra_group, &ino_threshold);
ehandler_operation(old_op);
- if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+ if (e2fsck_should_abort(ctx))
goto endit;
if (pctx.errcode == EXT2_ET_BAD_BLOCK_IN_INODE_TABLE) {
/*
@@ -1949,13 +1964,13 @@ void e2fsck_pass1(e2fsck_t ctx)
FINISH_INODE_LOOP(ctx, ino, &pctx, failed_csum);
- if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+ if (e2fsck_should_abort(ctx))
goto endit;
if (process_inode_count >= ctx->process_inode_size) {
process_inodes(ctx, block_buf);
- if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+ if (e2fsck_should_abort(ctx))
goto endit;
}
}
@@ -2068,6 +2083,89 @@ endit:
else
ctx->invalid_bitmaps++;
}
+
+static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thread_ctx)
+{
+ errcode_t retval;
+ e2fsck_t thread_context;
+
+ retval = ext2fs_get_mem(sizeof(struct e2fsck_struct), &thread_context);
+ if (retval) {
+ com_err(global_ctx->program_name, retval, "while allocating memory");
+ return retval;
+ }
+ memcpy(thread_context, global_ctx, sizeof(struct e2fsck_struct));
+ thread_context->fs->priv_data = thread_context;
+ thread_context->global_ctx = global_ctx;
+
+ *thread_ctx = thread_context;
+ return 0;
+}
+
+static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
+{
+ int flags = global_ctx->flags;
+#ifdef HAVE_SETJMP_H
+ jmp_buf old_jmp;
+
+ memcpy(old_jmp, global_ctx->abort_loc, sizeof(jmp_buf));
+#endif
+ memcpy(global_ctx, thread_ctx, sizeof(struct e2fsck_struct));
+#ifdef HAVE_SETJMP_H
+ memcpy(global_ctx->abort_loc, old_jmp, sizeof(jmp_buf));
+#endif
+ /* Keep the global singal flags*/
+ global_ctx->flags |= (flags & E2F_FLAG_SIGNAL_MASK) |
+ (global_ctx->flags & E2F_FLAG_SIGNAL_MASK);
+
+ global_ctx->fs->priv_data = global_ctx;
+ ext2fs_free_mem(&thread_ctx);
+ return 0;
+}
+
+void e2fsck_pass1_multithread(e2fsck_t ctx)
+{
+ errcode_t retval;
+ e2fsck_t thread_ctx;
+
+ retval = e2fsck_pass1_thread_prepare(ctx, &thread_ctx);
+ if (retval) {
+ com_err(ctx->program_name, 0,
+ _("while preparing pass1 thread\n"));
+ ctx->flags |= E2F_FLAG_ABORT;
+ return;
+ }
+
+#ifdef HAVE_SETJMP_H
+ /*
+ * When fatal_error() happens, jump to here. The thread
+ * context's flags will be saved, but its abort_loc will
+ * be overwritten by original jump buffer for the later
+ * tests.
+ */
+ if (setjmp(thread_ctx->abort_loc)) {
+ thread_ctx->flags &= ~E2F_FLAG_SETJMP_OK;
+ e2fsck_pass1_thread_join(ctx, thread_ctx);
+ return;
+ }
+ thread_ctx->flags |= E2F_FLAG_SETJMP_OK;
+#endif
+
+ e2fsck_pass1_thread(thread_ctx);
+ retval = e2fsck_pass1_thread_join(ctx, thread_ctx);
+ if (retval) {
+ com_err(ctx->program_name, 0,
+ _("while joining pass1 thread\n"));
+ ctx->flags |= E2F_FLAG_ABORT;
+ return;
+ }
+}
+
+void e2fsck_pass1(e2fsck_t ctx)
+{
+ e2fsck_pass1_multithread(ctx);
+}
+
#undef FINISH_INODE_LOOP
/*
@@ -2130,7 +2228,7 @@ static void process_inodes(e2fsck_t ctx, char *block_buf)
ehandler_operation(buf);
check_blocks(ctx, &pctx, block_buf,
&inodes_to_process[i].ea_ibody_quota);
- if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+ if (e2fsck_should_abort(ctx))
break;
}
ctx->stashed_inode = old_stashed_inode;
@@ -3300,7 +3398,7 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx,
inlinedata_fs = ext2fs_has_feature_inline_data(ctx->fs->super);
if (check_ext_attr(ctx, pctx, block_buf, &ea_block_quota)) {
- if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+ if (e2fsck_should_abort(ctx))
goto out;
pb.num_blocks += EXT2FS_B2C(ctx->fs, ea_block_quota.blocks);
}
@@ -3355,7 +3453,7 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx,
}
end_problem_latch(ctx, PR_LATCH_BLOCK);
end_problem_latch(ctx, PR_LATCH_TOOBIG);
- if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+ if (e2fsck_should_abort(ctx))
goto out;
if (pctx->errcode)
fix_problem(ctx, PR_1_BLOCK_ITERATE, pctx);
@@ -3836,7 +3934,7 @@ static int process_bad_block(ext2_filsys fs,
*block_nr = 0;
return BLOCK_CHANGED;
}
- if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+ if (e2fsck_should_abort(ctx))
return BLOCK_ABORT;
} else
mark_block_used(ctx, blk);
@@ -3933,7 +4031,7 @@ static int process_bad_block(ext2_filsys fs,
*block_nr = 0;
return BLOCK_CHANGED;
}
- if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+ if (e2fsck_should_abort(ctx))
return BLOCK_ABORT;
return 0;
}
diff --git a/e2fsck/unix.c b/e2fsck/unix.c
index 0a027be6..83b62032 100644
--- a/e2fsck/unix.c
+++ b/e2fsck/unix.c
@@ -1444,6 +1444,7 @@ int main (int argc, char *argv[])
}
reserve_stdio_fds();
+ ctx->global_ctx = NULL;
set_up_logging(ctx);
if (ctx->logf) {
int i;
--
2.25.4
From: Li Xi <[email protected]>
This patch only copy the dblist of the fs to a new one when -m
is specified.
Signed-off-by: Li Xi <[email protected]>
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/pass1.c | 33 ++++++++++++++++++++++++++++++---
1 file changed, 30 insertions(+), 3 deletions(-)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index c0df4330..b212cdde 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -2086,8 +2086,29 @@ endit:
ctx->invalid_bitmaps++;
}
-static void e2fsck_pass1_copy_fs(ext2_filsys dest, ext2_filsys src)
+static errcode_t e2fsck_pass1_copy_fs(ext2_filsys dest, ext2_filsys src)
{
+ errcode_t retval;
+
+ memcpy(dest, src, sizeof(struct struct_ext2_filsys));
+ if (dest->dblist) {
+ retval = ext2fs_copy_dblist(src->dblist, &dest->dblist);
+ if (retval)
+ return retval;
+ /* The ext2fs_copy_dblist() uses the src->fs as the fs */
+ dest->dblist->fs = dest;
+ }
+ if (dest->inode_map)
+ dest->inode_map->fs = dest;
+ if (dest->block_map)
+ dest->block_map->fs = dest;
+ return 0;
+}
+
+static void e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
+{
+ if (dest->dblist)
+ ext2fs_free_dblist(dest->dblist);
memcpy(dest, src, sizeof(struct struct_ext2_filsys));
if (dest->dblist)
dest->dblist->fs = dest;
@@ -2118,12 +2139,18 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre
goto out_context;
}
- e2fsck_pass1_copy_fs(thread_fs, global_fs);
+ retval = e2fsck_pass1_copy_fs(thread_fs, global_fs);
+ if (retval) {
+ com_err(global_ctx->program_name, retval, "while copying fs");
+ goto out_fs;
+ }
thread_fs->priv_data = thread_context;
thread_context->fs = thread_fs;
*thread_ctx = thread_context;
return 0;
+out_fs:
+ ext2fs_free_mem(&thread_fs);
out_context:
ext2fs_free_mem(&thread_context);
return retval;
@@ -2147,7 +2174,7 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
global_ctx->flags |= (flags & E2F_FLAG_SIGNAL_MASK) |
(global_ctx->flags & E2F_FLAG_SIGNAL_MASK);
- e2fsck_pass1_copy_fs(global_fs, thread_fs);
+ e2fsck_pass1_merge_fs(global_fs, thread_fs);
global_fs->priv_data = global_ctx;
global_ctx->fs = global_fs;
--
2.25.4
From: Li Xi <[email protected]>
This patch copies bitmap when the copying context. In the
multi-thread fsck, each thread use different bitmap that copied
from the glboal bitmap. And Bitmaps from multiple threads will
be merged into a global one after the pass1 finishes.
Signed-off-by: Li Xi <[email protected]>
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/pass1.c | 121 +++++++++++++++++++++++++++++++++++++++----------
1 file changed, 98 insertions(+), 23 deletions(-)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 1ee6b5bc..b836e666 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -2087,22 +2087,45 @@ endit:
ctx->invalid_bitmaps++;
}
+#define PASS1_COPY_FS_BITMAP(_dest, _src, _map_filed) \
+do { \
+ errcode_t _ret; \
+ if (_src->_map_filed) { \
+ _ret = ext2fs_copy_bitmap(_src->_map_filed, &_dest->_map_filed);\
+ if (_ret) \
+ return _ret; \
+ _dest->_map_filed->fs = _dest; \
+ \
+ ext2fs_free_generic_bmap(_src->_map_filed); \
+ _src->_map_filed = NULL; \
+ } \
+} while (0)
+
+#define PASS1_COPY_CTX_BITMAP(_dest, _src, _map_filed) \
+do { \
+ errcode_t _ret; \
+ if (_src->_map_filed) { \
+ _ret = ext2fs_copy_bitmap(_src->_map_filed, &_dest->_map_filed);\
+ if (_ret) \
+ return _ret; \
+ _dest->_map_filed->fs = _dest->fs; \
+ \
+ ext2fs_free_generic_bmap(_src->_map_filed); \
+ _src->_map_filed = NULL; \
+ } \
+} while (0)
+
static errcode_t e2fsck_pass1_copy_fs(ext2_filsys dest, ext2_filsys src)
{
errcode_t retval;
memcpy(dest, src, sizeof(struct struct_ext2_filsys));
- if (dest->dblist) {
- retval = ext2fs_copy_dblist(src->dblist, &dest->dblist);
- if (retval)
- return retval;
- /* The ext2fs_copy_dblist() uses the src->fs as the fs */
- dest->dblist->fs = dest;
- }
- if (dest->inode_map)
- dest->inode_map->fs = dest;
- if (dest->block_map)
- dest->block_map->fs = dest;
+ /*
+ * PASS1_COPY_FS_BITMAP might return directly from this function,
+ * so please do NOT leave any garbage behind after returning.
+ */
+ PASS1_COPY_FS_BITMAP(dest, src, inode_map);
+ PASS1_COPY_FS_BITMAP(dest, src, block_map);
/* icache will be rebuilt if needed, so do not copy from @src */
if (src->icache) {
@@ -2110,20 +2133,32 @@ static errcode_t e2fsck_pass1_copy_fs(ext2_filsys dest, ext2_filsys src)
src->icache = NULL;
}
dest->icache = NULL;
+
+ if (src->dblist) {
+ retval = ext2fs_copy_dblist(src->dblist, &dest->dblist);
+ if (retval)
+ return retval;
+ /* The ext2fs_copy_dblist() uses the src->fs as the fs */
+ dest->dblist->fs = dest;
+
+ ext2fs_free_dblist(src->dblist);
+ src->dblist = NULL;
+ }
+
return 0;
}
-static void e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
+static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
{
- if (dest->dblist)
- ext2fs_free_dblist(dest->dblist);
+ errcode_t retval = 0;
+
memcpy(dest, src, sizeof(struct struct_ext2_filsys));
- if (dest->dblist)
- dest->dblist->fs = dest;
- if (dest->inode_map)
- dest->inode_map->fs = dest;
- if (dest->block_map)
- dest->block_map->fs = dest;
+ /*
+ * PASS1_COPY_FS_BITMAP might return directly from this function,
+ * so please do NOT leave any garbage behind after returning.
+ */
+ PASS1_COPY_FS_BITMAP(dest, src, inode_map);
+ PASS1_COPY_FS_BITMAP(dest, src, block_map);
/* icache will be rebuilt if needed, so do not copy from @src */
if (src->icache) {
@@ -2131,6 +2166,18 @@ static void e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
src->icache = NULL;
}
dest->icache = NULL;
+
+ if (dest->dblist) {
+ retval = ext2fs_copy_dblist(src->dblist, &dest->dblist);
+ if (retval == 0) {
+ /* The ext2fs_copy_dblist() uses the src->fs as the fs */
+ dest->dblist->fs = dest;
+ }
+
+ ext2fs_free_dblist(src->dblist);
+ src->dblist = NULL;
+ }
+ return retval;
}
static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thread_ctx)
@@ -2183,8 +2230,9 @@ out_context:
return retval;
}
-static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
+static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx)
{
+ errcode_t retval;
int flags = global_ctx->flags;
ext2_filsys thread_fs = thread_ctx->fs;
ext2_filsys global_fs = global_ctx->fs;
@@ -2201,13 +2249,40 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
global_ctx->flags |= (flags & E2F_FLAG_SIGNAL_MASK) |
(global_ctx->flags & E2F_FLAG_SIGNAL_MASK);
- e2fsck_pass1_merge_fs(global_fs, thread_fs);
+ retval = e2fsck_pass1_merge_fs(global_fs, thread_fs);
+ if (retval) {
+ com_err(global_ctx->program_name, 0, _("while merging fs\n"));
+ return retval;
+ }
global_fs->priv_data = global_ctx;
global_ctx->fs = global_fs;
+ /*
+ * PASS1_COPY_CTX_BITMAP might return directly from this function,
+ * so please do NOT leave any garbage behind after returning.
+ */
+ PASS1_COPY_CTX_BITMAP(global_ctx, thread_ctx, inode_used_map);
+ PASS1_COPY_CTX_BITMAP(global_ctx, thread_ctx, inode_dir_map);
+ PASS1_COPY_CTX_BITMAP(global_ctx, thread_ctx, inode_bb_map);
+ PASS1_COPY_CTX_BITMAP(global_ctx, thread_ctx, inode_imagic_map);
+ PASS1_COPY_CTX_BITMAP(global_ctx, thread_ctx, inode_reg_map);
+ PASS1_COPY_CTX_BITMAP(global_ctx, thread_ctx, inodes_to_rebuild);
+ PASS1_COPY_CTX_BITMAP(global_ctx, thread_ctx, block_found_map);
+ PASS1_COPY_CTX_BITMAP(global_ctx, thread_ctx, block_dup_map);
+ PASS1_COPY_CTX_BITMAP(global_ctx, thread_ctx, block_ea_map);
+ PASS1_COPY_CTX_BITMAP(global_ctx, thread_ctx, block_metadata_map);
+ return 0;
+}
+
+static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
+{
+ errcode_t retval;
+
+ retval = e2fsck_pass1_thread_join_one(global_ctx, thread_ctx);
ext2fs_free_mem(&thread_ctx->fs);
ext2fs_free_mem(&thread_ctx);
- return 0;
+
+ return retval;
}
void e2fsck_pass1_multithread(e2fsck_t ctx)
--
2.25.4
From: Li Xi <[email protected]>
When multi-threads are used, different logs should be created
for different threads. Each thread has log files with subfix
of ".$THREAD_INDEX".
And this patch adds f_multithread_logfile test case.
Signed-off-by: Li Xi <[email protected]>
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/e2fsck.h | 2 ++
e2fsck/logfile.c | 11 +++++++++-
e2fsck/pass1.c | 24 ++++++++++++++++-----
tests/f_multithread_logfile/expect.1 | 23 ++++++++++++++++++++
tests/f_multithread_logfile/image.gz | 1 +
tests/f_multithread_logfile/name | 1 +
tests/f_multithread_logfile/script | 32 ++++++++++++++++++++++++++++
7 files changed, 88 insertions(+), 6 deletions(-)
create mode 100644 tests/f_multithread_logfile/expect.1
create mode 120000 tests/f_multithread_logfile/image.gz
create mode 100644 tests/f_multithread_logfile/name
create mode 100644 tests/f_multithread_logfile/script
diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
index 8925b5d8..b6cfcbb5 100644
--- a/e2fsck/e2fsck.h
+++ b/e2fsck/e2fsck.h
@@ -378,6 +378,8 @@ struct e2fsck_struct {
*/
ext2_ino_t stashed_ino;
struct ext2_inode *stashed_inode;
+ /* Thread index, if global_ctx is null, this field is useless */
+ int thread_index;
/*
* Directory information
diff --git a/e2fsck/logfile.c b/e2fsck/logfile.c
index 63e9a12f..17bfc86e 100644
--- a/e2fsck/logfile.c
+++ b/e2fsck/logfile.c
@@ -16,6 +16,7 @@
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
+#include <assert.h>
#include "e2fsck.h"
#include <pwd.h>
@@ -291,6 +292,8 @@ static FILE *set_up_log_file(e2fsck_t ctx, const char *key, const char *fn)
struct string s, s1, s2;
char *s0 = 0, *log_dir = 0, *log_fn = 0;
int log_dir_wait = 0;
+ int string_size;
+ char string_index[4];
s.s = s1.s = s2.s = 0;
@@ -307,6 +310,13 @@ static FILE *set_up_log_file(e2fsck_t ctx, const char *key, const char *fn)
goto out;
expand_logfn(ctx, log_fn, &s);
+ if (ctx->global_ctx) {
+ assert(ctx->thread_index < 1000);
+ sprintf(string_index, "%03d", ctx->thread_index);
+ append_string(&s, ".", 1);
+ append_string(&s, string_index, 0);
+ }
+
if ((log_fn[0] == '/') || !log_dir || !log_dir[0])
s0 = s.s;
@@ -325,7 +335,6 @@ static FILE *set_up_log_file(e2fsck_t ctx, const char *key, const char *fn)
append_string(&s2, log_dir, 0);
append_string(&s2, "/", 1);
append_string(&s2, s.s, 0);
- printf("%s\n", s2.s);
}
if (s0)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 013c8478..bf843bb9 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -2324,6 +2324,9 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre
}
thread_fs->priv_data = thread_context;
+ thread_context->thread_index = 0;
+ set_up_logging(thread_context);
+
thread_context->fs = thread_fs;
*thread_ctx = thread_context;
return 0;
@@ -2336,12 +2339,14 @@ out_context:
static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx)
{
- errcode_t retval;
- int flags = global_ctx->flags;
- ext2_filsys thread_fs = thread_ctx->fs;
- ext2_filsys global_fs = global_ctx->fs;
+ errcode_t retval;
+ int flags = global_ctx->flags;
+ ext2_filsys thread_fs = thread_ctx->fs;
+ ext2_filsys global_fs = global_ctx->fs;
+ FILE *global_logf = global_ctx->logf;
+ FILE *global_problem_logf = global_ctx->problem_logf;
#ifdef HAVE_SETJMP_H
- jmp_buf old_jmp;
+ jmp_buf old_jmp;
memcpy(old_jmp, global_ctx->abort_loc, sizeof(jmp_buf));
#endif
@@ -2360,6 +2365,8 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
}
global_fs->priv_data = global_ctx;
global_ctx->fs = global_fs;
+ global_ctx->logf = global_logf;
+ global_ctx->problem_logf = global_problem_logf;
/*
* PASS1_COPY_CTX_BITMAP might return directly from this function,
@@ -2375,6 +2382,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
PASS1_COPY_CTX_BITMAP(global_ctx, thread_ctx, block_dup_map);
PASS1_COPY_CTX_BITMAP(global_ctx, thread_ctx, block_ea_map);
PASS1_COPY_CTX_BITMAP(global_ctx, thread_ctx, block_metadata_map);
+
return 0;
}
@@ -2384,6 +2392,12 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
retval = e2fsck_pass1_thread_join_one(global_ctx, thread_ctx);
ext2fs_free_mem(&thread_ctx->fs);
+ if (thread_ctx->logf)
+ fclose(thread_ctx->logf);
+ if (thread_ctx->problem_logf) {
+ fputs("</problem_log>\n", thread_ctx->problem_logf);
+ fclose(thread_ctx->problem_logf);
+ }
ext2fs_free_mem(&thread_ctx);
return retval;
diff --git a/tests/f_multithread_logfile/expect.1 b/tests/f_multithread_logfile/expect.1
new file mode 100644
index 00000000..e2b954d0
--- /dev/null
+++ b/tests/f_multithread_logfile/expect.1
@@ -0,0 +1,23 @@
+ext2fs_open2: Bad magic number in super-block
+../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+Free blocks count wrong for group #0 (7987, counted=7982).
+Fix? yes
+
+Free blocks count wrong (11602, counted=11597).
+Fix? yes
+
+Free inodes count wrong for group #0 (1493, counted=1488).
+Fix? yes
+
+Free inodes count wrong (2997, counted=2992).
+Fix? yes
+
+
+test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
+test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks
+Exit status is 1
diff --git a/tests/f_multithread_logfile/image.gz b/tests/f_multithread_logfile/image.gz
new file mode 120000
index 00000000..0fd40018
--- /dev/null
+++ b/tests/f_multithread_logfile/image.gz
@@ -0,0 +1 @@
+../f_zero_super/image.gz
\ No newline at end of file
diff --git a/tests/f_multithread_logfile/name b/tests/f_multithread_logfile/name
new file mode 100644
index 00000000..b8fcb997
--- /dev/null
+++ b/tests/f_multithread_logfile/name
@@ -0,0 +1 @@
+test "e2fsck -m" option works with "--E log_filename="
diff --git a/tests/f_multithread_logfile/script b/tests/f_multithread_logfile/script
new file mode 100644
index 00000000..d7042a03
--- /dev/null
+++ b/tests/f_multithread_logfile/script
@@ -0,0 +1,32 @@
+LOG_FNAME="f_multithread_logfile_xxx"
+FSCK_OPT="-fy -m -y -E log_filename=$LOG_FNAME"
+SKIP_VERIFY="true"
+ONE_PASS_ONLY="true"
+SKIP_CLEANUP="true"
+
+rm -f $LOG_FNAME.* $LOG_FNAME
+
+. $cmd_dir/run_e2fsck
+
+rm -f $test_name.ok $test_name.failed
+cmp -s $OUT1 $EXP1
+status1=$?
+
+if [ "$status1" -eq 0 ]; then
+ if [ ! -f $LOG_FNAME -o ! -f $LOG_FNAME.000 ]; then
+ echo "$LOG_FNAME or $LOG_FNAME.000 is not created" > $test_name.failed
+ echo "$test_name: $test_description: failed"
+ else
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+ fi
+else
+ diff $DIFF_OPTS $test_dir/expect.1 \
+ $test_name.1.log >> $test_name.failed
+ echo "$test_name: $test_description: failed"
+fi
+
+unset IMAGE FSCK_OPT SECOND_FSCK_OPT OUT1 OUT2 EXP1 EXP2
+unset SKIP_VERIFY SKIP_CLEANUP SKIP_GUNZIP ONE_PASS_ONLY PREP_CMD
+unset DESCRIPTION SKIP_UNLINK AFTER_CMD PASS_ZERO
+unset LOG_FINAME
--
2.25.4
From: Li Xi <[email protected]>
This patch also add writethrough flag to the thread io-channel.
When multiple threads write the same disk, we don't want the
data being saved in memory cache. This will be useful in the
future, but even without that flag, the tests can be passed too.
This patch also cleanup the io channel cache of the global
context. Otherwise, after pass1 step, the next steps would old
data saved in the cache. And the cached data might have already
been overwritten in the pass1 step.
Signed-off-by: Li Xi <[email protected]>
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/e2fsck.h | 1 +
e2fsck/pass1.c | 71 ++++++++++++++++++++++++++++++++++++++++++--
e2fsck/unix.c | 1 +
lib/ext2fs/ext2_io.h | 2 ++
lib/ext2fs/undo_io.c | 19 ++++++++++++
lib/ext2fs/unix_io.c | 24 +++++++++++++--
6 files changed, 112 insertions(+), 6 deletions(-)
diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
index 46792c9b..8925b5d8 100644
--- a/e2fsck/e2fsck.h
+++ b/e2fsck/e2fsck.h
@@ -244,6 +244,7 @@ struct e2fsck_struct {
ext2_ino_t free_inodes;
int mount_flags;
int openfs_flags;
+ io_manager io_manager;
blkid_cache blkid; /* blkid cache */
#ifdef HAVE_SETJMP_H
jmp_buf abort_loc;
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index a1feb166..013c8478 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -2115,7 +2115,37 @@ do { \
} \
} while (0)
-static errcode_t e2fsck_pass1_copy_fs(ext2_filsys dest, ext2_filsys src)
+static errcode_t pass1_open_io_channel(ext2_filsys fs,
+ const char *io_options,
+ io_manager manager, int flags)
+{
+ errcode_t retval;
+ unsigned int io_flags = 0;
+
+ if (flags & EXT2_FLAG_RW)
+ io_flags |= IO_FLAG_RW;
+ if (flags & EXT2_FLAG_EXCLUSIVE)
+ io_flags |= IO_FLAG_EXCLUSIVE;
+ if (flags & EXT2_FLAG_DIRECT_IO)
+ io_flags |= IO_FLAG_DIRECT_IO;
+ retval = manager->open(fs->device_name, io_flags, &fs->io);
+ if (retval)
+ return retval;
+
+ if (io_options &&
+ (retval = io_channel_set_options(fs->io, io_options)))
+ goto out_close;
+ fs->image_io = fs->io;
+ fs->io->app_data = fs;
+
+ return 0;
+out_close:
+ io_channel_close(fs->io);
+ return retval;
+}
+
+static errcode_t e2fsck_pass1_copy_fs(ext2_filsys dest, e2fsck_t src_context,
+ ext2_filsys src)
{
errcode_t retval;
@@ -2153,6 +2183,33 @@ static errcode_t e2fsck_pass1_copy_fs(ext2_filsys dest, ext2_filsys src)
ext2fs_badblocks_list_free(src->badblocks);
src->badblocks = NULL;
}
+
+ /* disable it for now */
+ src_context->openfs_flags &= ~EXT2_FLAG_EXCLUSIVE;
+ retval = pass1_open_io_channel(dest, src_context->io_options,
+ src_context->io_manager,
+ src_context->openfs_flags);
+ if (retval)
+ goto out_dblist;
+
+ /* Block size might not be default */
+ io_channel_set_blksize(dest->io, src->io->block_size);
+ ehandler_init(dest->io);
+
+ assert(dest->io->magic == src->io->magic);
+ assert(dest->io->manager == src->io->manager);
+ assert(strcmp(dest->io->name, src->io->name) == 0);
+ assert(dest->io->block_size == src->io->block_size);
+ assert(dest->io->read_error == src->io->read_error);
+ assert(dest->io->write_error == src->io->write_error);
+ assert(dest->io->refcount == src->io->refcount);
+ assert(dest->io->flags == src->io->flags);
+ assert(dest->io->app_data == dest);
+ assert(src->io->app_data == src);
+ assert(dest->io->align == src->io->align);
+
+ /* The data should be written to disk immediately */
+ dest->io->flags |= CHANNEL_FLAGS_WRITETHROUGH;
return 0;
out_dblist:
@@ -2163,9 +2220,15 @@ out_dblist:
static int _e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
{
- errcode_t retval = 0;
+ errcode_t retval = 0;
+ io_channel dest_io;
+ io_channel dest_image_io;
+ dest_io = dest->io;
+ dest_image_io = dest->image_io;
memcpy(dest, src, sizeof(struct struct_ext2_filsys));
+ dest->io = dest_io;
+ dest->image_io = dest_image_io;
/*
* PASS1_COPY_FS_BITMAP might return directly from this function,
* so please do NOT leave any garbage behind after returning.
@@ -2186,6 +2249,7 @@ static int _e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
if (retval)
goto out_dblist;
}
+ io_channel_close(src->io);
return 0;
out_dblist:
ext2fs_free_dblist(dest->dblist);
@@ -2252,7 +2316,8 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre
goto out_context;
}
- retval = e2fsck_pass1_copy_fs(thread_fs, global_fs);
+ io_channel_flush_cleanup(global_fs->io);
+ retval = e2fsck_pass1_copy_fs(thread_fs, global_ctx, global_fs);
if (retval) {
com_err(global_ctx->program_name, retval, "while copying fs");
goto out_fs;
diff --git a/e2fsck/unix.c b/e2fsck/unix.c
index 83b62032..3124019a 100644
--- a/e2fsck/unix.c
+++ b/e2fsck/unix.c
@@ -1524,6 +1524,7 @@ restart:
}
ctx->openfs_flags = flags;
+ ctx->io_manager = io_ptr;
retval = try_open_fs(ctx, flags, io_ptr, &fs);
if (!ctx->superblock && !(ctx->options & E2F_OPT_PREEN) &&
diff --git a/lib/ext2fs/ext2_io.h b/lib/ext2fs/ext2_io.h
index 5540900a..4ad2fec8 100644
--- a/lib/ext2fs/ext2_io.h
+++ b/lib/ext2fs/ext2_io.h
@@ -81,6 +81,7 @@ struct struct_io_manager {
errcode_t (*write_blk)(io_channel channel, unsigned long block,
int count, const void *data);
errcode_t (*flush)(io_channel channel);
+ errcode_t (*flush_cleanup)(io_channel channel);
errcode_t (*write_byte)(io_channel channel, unsigned long offset,
int count, const void *data);
errcode_t (*set_option)(io_channel channel, const char *option,
@@ -113,6 +114,7 @@ struct struct_io_manager {
#define io_channel_read_blk(c,b,n,d) ((c)->manager->read_blk((c),b,n,d))
#define io_channel_write_blk(c,b,n,d) ((c)->manager->write_blk((c),b,n,d))
#define io_channel_flush(c) ((c)->manager->flush((c)))
+#define io_channel_flush_cleanup(c) ((c)->manager->flush_cleanup((c)))
#define io_channel_bumpcount(c) ((c)->refcount++)
/* io_manager.c */
diff --git a/lib/ext2fs/undo_io.c b/lib/ext2fs/undo_io.c
index 19862414..1391a534 100644
--- a/lib/ext2fs/undo_io.c
+++ b/lib/ext2fs/undo_io.c
@@ -1020,6 +1020,24 @@ static errcode_t undo_flush(io_channel channel)
return retval;
}
+/*
+ * Flush data buffers to disk and cleanup the cache.
+ */
+static errcode_t undo_flush_cleanup(io_channel channel)
+{
+ errcode_t retval = 0;
+ struct undo_private_data *data;
+
+ EXT2_CHECK_MAGIC(channel, EXT2_ET_MAGIC_IO_CHANNEL);
+ data = (struct undo_private_data *) channel->private_data;
+ EXT2_CHECK_MAGIC(data, EXT2_ET_MAGIC_UNIX_IO_CHANNEL);
+
+ if (data->real)
+ retval = io_channel_flush_cleanup(data->real);
+
+ return retval;
+}
+
static errcode_t undo_set_option(io_channel channel, const char *option,
const char *arg)
{
@@ -1091,6 +1109,7 @@ static struct struct_io_manager struct_undo_manager = {
.read_blk = undo_read_blk,
.write_blk = undo_write_blk,
.flush = undo_flush,
+ .flush_cleanup = undo_flush_cleanup,
.write_byte = undo_write_byte,
.set_option = undo_set_option,
.get_stats = undo_get_stats,
diff --git a/lib/ext2fs/unix_io.c b/lib/ext2fs/unix_io.c
index 628e60c3..c9defd4b 100644
--- a/lib/ext2fs/unix_io.c
+++ b/lib/ext2fs/unix_io.c
@@ -1030,9 +1030,9 @@ static errcode_t unix_write_byte(io_channel channel, unsigned long offset,
}
/*
- * Flush data buffers to disk.
+ * Flush data buffers to disk and invalidate cache if needed
*/
-static errcode_t unix_flush(io_channel channel)
+static errcode_t _unix_flush(io_channel channel, int invalidate)
{
struct unix_private_data *data;
errcode_t retval = 0;
@@ -1042,7 +1042,7 @@ static errcode_t unix_flush(io_channel channel)
EXT2_CHECK_MAGIC(data, EXT2_ET_MAGIC_UNIX_IO_CHANNEL);
#ifndef NO_IO_CACHE
- retval = flush_cached_blocks(channel, data, 0);
+ retval = flush_cached_blocks(channel, data, invalidate);
#endif
#ifdef HAVE_FSYNC
if (!retval && fsync(data->dev) != 0)
@@ -1051,6 +1051,22 @@ static errcode_t unix_flush(io_channel channel)
return retval;
}
+/*
+ * Flush data buffers to disk.
+ */
+static errcode_t unix_flush(io_channel channel)
+{
+ return _unix_flush(channel, 0);
+}
+
+/*
+ * Flush data buffers to disk and invalidate cache.
+ */
+static errcode_t unix_flush_cleanup(io_channel channel)
+{
+ return _unix_flush(channel, 1);
+}
+
static errcode_t unix_set_option(io_channel channel, const char *option,
const char *arg)
{
@@ -1225,6 +1241,7 @@ static struct struct_io_manager struct_unix_manager = {
.discard = unix_discard,
.cache_readahead = unix_cache_readahead,
.zeroout = unix_zeroout,
+ .flush_cleanup = unix_flush_cleanup,
};
io_manager unix_io_manager = &struct_unix_manager;
@@ -1246,6 +1263,7 @@ static struct struct_io_manager struct_unixfd_manager = {
.discard = unix_discard,
.cache_readahead = unix_cache_readahead,
.zeroout = unix_zeroout,
+ .flush_cleanup = unix_flush_cleanup,
};
io_manager unixfd_io_manager = &struct_unixfd_manager;
--
2.25.4
From: Li Xi <[email protected]>
This patch creates only one thread to do pass1 check. The same
codes can be used to create multiple threads, but other functions
need to be modified to get ready for that.
Signed-off-by: Li Xi <[email protected]>
Signed-off-by: Wang Shilong <[email protected]>
---
configure.ac | 6 ++
e2fsck/e2fsck.h | 11 ++++
e2fsck/pass1.c | 144 ++++++++++++++++++++++++++++++++++++++++++------
3 files changed, 143 insertions(+), 18 deletions(-)
diff --git a/configure.ac b/configure.ac
index 18e434bc..a8d3784c 100644
--- a/configure.ac
+++ b/configure.ac
@@ -107,6 +107,12 @@ if test "$GCC" = yes; then
fi
AC_PROG_CPP
dnl
+dnl Add pthread to the CFLAGS/LDFLAGS
+dnl
+CFLAGS="$CFLAGS -pthread"
+LDFLAGS="$CFLAGS -pthread"
+LDFLAGS_STATIC="$LDFLAGS_STATIC -pthread"
+dnl
dnl Alpha computers use fast and imprecise floating point code that may
dnl miss exceptions by default. Force sane options if we're using GCC.
AC_MSG_CHECKING(for additional special compiler flags)
diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
index b6cfcbb5..93387bd6 100644
--- a/e2fsck/e2fsck.h
+++ b/e2fsck/e2fsck.h
@@ -428,6 +428,17 @@ struct e2fsck_struct {
__u32 extent_depth_count[MAX_EXTENT_DEPTH_COUNT];
};
+struct e2fsck_thread_info {
+ /* ID returned by pthread_create() */
+ pthread_t eti_thread_id;
+ /* Application-defined thread index */
+ int eti_thread_index;
+ /* Thread has been started */
+ int eti_started;
+ /* Context used for this thread */
+ e2fsck_t eti_thread_ctx;
+};
+
/* Data structures to evaluate whether an extent tree needs rebuilding. */
struct extent_tree_level {
unsigned int num_extents;
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index bf843bb9..35806f29 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -47,6 +47,7 @@
#include <errno.h>
#endif
#include <assert.h>
+#include <pthread.h>
#include "e2fsck.h"
#include <ext2fs/ext2_ext_attr.h>
@@ -1162,7 +1163,7 @@ static int e2fsck_should_abort(e2fsck_t ctx)
return 0;
}
-void e2fsck_pass1_thread(e2fsck_t ctx)
+void _e2fsck_pass1(e2fsck_t ctx)
{
int i;
__u64 max_sizes;
@@ -2403,18 +2404,38 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
return retval;
}
-void e2fsck_pass1_multithread(e2fsck_t ctx)
+static int e2fsck_pass1_threads_join(struct e2fsck_thread_info *infos,
+ int num_threads, e2fsck_t global_ctx)
{
- errcode_t retval;
- e2fsck_t thread_ctx;
+ errcode_t rc;
+ errcode_t ret = 0;
+ int i;
+ struct e2fsck_thread_info *pinfo;
- retval = e2fsck_pass1_thread_prepare(ctx, &thread_ctx);
- if (retval) {
- com_err(ctx->program_name, 0,
- _("while preparing pass1 thread\n"));
- ctx->flags |= E2F_FLAG_ABORT;
- return;
+ for (i = 0; i < num_threads; i++) {
+ pinfo = &infos[i];
+
+ if (!pinfo->eti_started)
+ continue;
+
+ rc = pthread_join(pinfo->eti_thread_id, NULL);
+ if (rc) {
+ com_err(global_ctx->program_name, rc,
+ _("while joining thread\n"));
+ if (ret == 0)
+ ret = rc;
+ }
+ e2fsck_pass1_thread_join(global_ctx, infos[i].eti_thread_ctx);
}
+ free(infos);
+
+ return ret;
+}
+
+static void *e2fsck_pass1_thread(void *arg)
+{
+ struct e2fsck_thread_info *info = arg;
+ e2fsck_t thread_ctx = info->eti_thread_ctx;
#ifdef HAVE_SETJMP_H
/*
@@ -2425,20 +2446,107 @@ void e2fsck_pass1_multithread(e2fsck_t ctx)
*/
if (setjmp(thread_ctx->abort_loc)) {
thread_ctx->flags &= ~E2F_FLAG_SETJMP_OK;
- e2fsck_pass1_thread_join(ctx, thread_ctx);
- return;
+ goto out;
}
thread_ctx->flags |= E2F_FLAG_SETJMP_OK;
#endif
- e2fsck_pass1_thread(thread_ctx);
- retval = e2fsck_pass1_thread_join(ctx, thread_ctx);
+ _e2fsck_pass1(thread_ctx);
+
+out:
+ return NULL;
+}
+
+static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo,
+ int num_threads, e2fsck_t global_ctx)
+{
+ struct e2fsck_thread_info *infos;
+ pthread_attr_t attr;
+ errcode_t retval;
+ errcode_t ret;
+ struct e2fsck_thread_info *tmp_pinfo;
+ int i;
+ e2fsck_t thread_ctx;
+
+ retval = pthread_attr_init(&attr);
if (retval) {
- com_err(ctx->program_name, 0,
- _("while joining pass1 thread\n"));
- ctx->flags |= E2F_FLAG_ABORT;
- return;
+ com_err(global_ctx->program_name, retval,
+ _("while setting pthread attribute\n"));
+ return retval;
+ }
+
+ infos = calloc(num_threads, sizeof(struct e2fsck_thread_info));
+ if (infos == NULL) {
+ retval = -ENOMEM;
+ com_err(global_ctx->program_name, retval,
+ _("while allocating memory for threads\n"));
+ pthread_attr_destroy(&attr);
+ return retval;
}
+
+ for (i = 0; i < num_threads; i++) {
+ tmp_pinfo = &infos[i];
+ tmp_pinfo->eti_thread_index = i;
+ retval = e2fsck_pass1_thread_prepare(global_ctx, &thread_ctx);
+ if (retval) {
+ com_err(global_ctx->program_name, retval,
+ _("while preparing pass1 thread\n"));
+ break;
+ }
+ tmp_pinfo->eti_thread_ctx = thread_ctx;
+
+ retval = pthread_create(&tmp_pinfo->eti_thread_id, &attr,
+ &e2fsck_pass1_thread, tmp_pinfo);
+ if (retval) {
+ com_err(global_ctx->program_name, retval,
+ _("while creating thread\n"));
+ e2fsck_pass1_thread_join(global_ctx, thread_ctx);
+ break;
+ }
+
+ tmp_pinfo->eti_started = 1;
+ }
+
+ /* destroy the thread attribute object, since it is no longer needed */
+ ret = pthread_attr_destroy(&attr);
+ if (ret) {
+ com_err(global_ctx->program_name, ret,
+ _("while destroying thread attribute\n"));
+ if (retval == 0)
+ retval = ret;
+ }
+
+ if (retval) {
+ e2fsck_pass1_threads_join(infos, num_threads, global_ctx);
+ return retval;
+ }
+ *pinfo = infos;
+ return 0;
+}
+
+static void e2fsck_pass1_multithread(e2fsck_t global_ctx)
+{
+ struct e2fsck_thread_info *infos = NULL;
+ int num_threads = 1;
+ errcode_t retval;
+
+ retval = e2fsck_pass1_threads_start(&infos, num_threads, global_ctx);
+ if (retval) {
+ com_err(global_ctx->program_name, retval,
+ _("while starting pass1 threads\n"));
+ goto out_abort;
+ }
+
+ retval = e2fsck_pass1_threads_join(infos, num_threads, global_ctx);
+ if (retval) {
+ com_err(global_ctx->program_name, retval,
+ _("while joining pass1 threads\n"));
+ goto out_abort;
+ }
+ return;
+out_abort:
+ global_ctx->flags |= E2F_FLAG_ABORT;
+ return;
}
void e2fsck_pass1(e2fsck_t ctx)
--
2.25.4
From: Li Xi <[email protected]>
When multi-thread fsck is enabled, logs printed from multiple
threads could overlap with each other. The overlap sometimes
makes the logs unreadable because log_out() is used multiple times
for a single line.
This patch adds leading [Thread XXX] to each logs if multi-thread
is enabed by -m option.
This patch also adds message to show the group ranges and inode
numbers for each thread, which is useful for debuging multi-thread
check.
Signed-off-by: Li Xi <[email protected]>
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/e2fsck.h | 4 ++
e2fsck/pass1.c | 17 ++++++++-
e2fsck/problem.c | 4 ++
e2fsck/util.c | 56 ++++++++++++++++++++++++++--
tests/f_multithread/expect.1 | 4 +-
tests/f_multithread_logfile/expect.1 | 4 +-
tests/f_multithread_no/expect.1 | 4 +-
7 files changed, 86 insertions(+), 7 deletions(-)
diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
index 896f5f39..121d1b9b 100644
--- a/e2fsck/e2fsck.h
+++ b/e2fsck/e2fsck.h
@@ -239,6 +239,10 @@ struct e2fsck_thread {
dgrp_t et_group_next;
/* Thread index */
int et_thread_index;
+ /* Scanned inode number */
+ ext2_ino_t et_inode_number;
+ char et_log_buf[2048];
+ char et_log_length;
};
struct e2fsck_struct {
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index c676cbbc..64666663 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -1435,6 +1435,8 @@ void _e2fsck_pass1(e2fsck_t ctx)
}
if (!ino)
break;
+ if (ctx->global_ctx)
+ ctx->thread_info.et_inode_number++;
pctx.ino = ino;
pctx.inode = inode;
ctx->stashed_ino = ino;
@@ -2362,7 +2364,12 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx,
else
tinfo->et_group_end = average_group * (thread_index + 1);
tinfo->et_group_next = tinfo->et_group_start;
-
+ tinfo->et_inode_number = 0;
+ tinfo->et_log_buf[0] = '\0';
+ tinfo->et_log_length = 0;
+ if (thread_context->options & E2F_OPT_MULTITHREAD)
+ log_out(thread_context, _("Scan group range [%d, %d)\n"),
+ tinfo->et_group_start, tinfo->et_group_end);
thread_context->fs = thread_fs;
*thread_ctx = thread_context;
return 0;
@@ -2403,6 +2410,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
global_ctx->fs = global_fs;
global_ctx->logf = global_logf;
global_ctx->problem_logf = global_problem_logf;
+ global_ctx->global_ctx = NULL;
/*
* PASS1_COPY_CTX_BITMAP might return directly from this function,
@@ -2489,6 +2497,12 @@ static void *e2fsck_pass1_thread(void *arg)
_e2fsck_pass1(thread_ctx);
out:
+ if (thread_ctx->options & E2F_OPT_MULTITHREAD)
+ log_out(thread_ctx,
+ _("Scanned group range [%lu, %lu), inodes %lu\n"),
+ thread_ctx->thread_info.et_group_start,
+ thread_ctx->thread_info.et_group_end,
+ thread_ctx->thread_info.et_inode_number);
return NULL;
}
@@ -2616,6 +2630,7 @@ static errcode_t scan_callback(ext2_filsys fs,
if (ctx->global_ctx) {
tinfo = &ctx->thread_info;
+ //printf("iii group %d finished\n", tinfo->et_group_next);
tinfo->et_group_next++;
if (tinfo->et_group_next >= tinfo->et_group_end)
return EXT2_ET_SCAN_FINISHED;
diff --git a/e2fsck/problem.c b/e2fsck/problem.c
index 22c2652c..ff52099f 100644
--- a/e2fsck/problem.c
+++ b/e2fsck/problem.c
@@ -2481,6 +2481,10 @@ int fix_problem(e2fsck_t ctx, problem_t code, struct problem_context *pctx)
if (*message)
message = _(message);
if (!suppress) {
+ if ((ctx->options & E2F_OPT_MULTITHREAD) && ctx->global_ctx)
+ printf("[Thread %d] ",
+ ctx->thread_info.et_thread_index);
+
if ((ctx->options & E2F_OPT_PREEN) &&
!(ptr->flags & PR_PREEN_NOHDR)) {
printf("%s: ", ctx->device_name ?
diff --git a/e2fsck/util.c b/e2fsck/util.c
index 8cebd95a..ffa5c64b 100644
--- a/e2fsck/util.c
+++ b/e2fsck/util.c
@@ -11,6 +11,7 @@
#include "config.h"
#include <stdlib.h>
+#include <assert.h>
#include <stdio.h>
#include <unistd.h>
#include <string.h>
@@ -88,13 +89,62 @@ out:
exit(exit_value);
}
+static void thread_log_out(struct e2fsck_thread *tinfo)
+{
+ printf("[Thread %d] %s", tinfo->et_thread_index,
+ tinfo->et_log_buf);
+ tinfo->et_log_length = 0;
+ tinfo->et_log_buf[0] = '\0';
+}
+
void log_out(e2fsck_t ctx, const char *fmt, ...)
{
va_list pvar;
+ struct e2fsck_thread *tinfo;
+ int buf_size;
+ int msg_size;
+ int left_size;
+ int fmt_length = strlen(fmt);
+
+ if ((ctx->options & E2F_OPT_MULTITHREAD) && ctx->global_ctx) {
+ tinfo = &ctx->thread_info;
+ buf_size = sizeof(tinfo->et_log_buf);
+ left_size = buf_size - tinfo->et_log_length;
+
+ va_start(pvar, fmt);
+ msg_size = vsnprintf(tinfo->et_log_buf + tinfo->et_log_length,
+ left_size, fmt, pvar);
+ va_end(pvar);
+
+ if (msg_size >= left_size) {
+ tinfo->et_log_buf[tinfo->et_log_length] = '\0';
+
+ assert(msg_size < buf_size);
+ if (msg_size < buf_size) {
+ thread_log_out(tinfo);
+
+ va_start(pvar, fmt);
+ msg_size = vsnprintf(tinfo->et_log_buf, buf_size,
+ fmt, pvar);
+ va_end(pvar);
+
+ tinfo->et_log_length += msg_size;
+ tinfo->et_log_buf[tinfo->et_log_length] = '\0';
+ }
+ } else {
+ tinfo->et_log_length += msg_size;
+ tinfo->et_log_buf[tinfo->et_log_length] = '\0';
+ }
+
+ if (tinfo->et_log_length > 0 &&
+ tinfo->et_log_buf[tinfo->et_log_length - 1] == '\n')
+ thread_log_out(tinfo);
+ } else {
+ va_start(pvar, fmt);
+ vprintf(fmt, pvar);
+ va_end(pvar);
+ }
- va_start(pvar, fmt);
- vprintf(fmt, pvar);
- va_end(pvar);
if (ctx->logf) {
va_start(pvar, fmt);
vfprintf(ctx->logf, fmt, pvar);
diff --git a/tests/f_multithread/expect.1 b/tests/f_multithread/expect.1
index e2b954d0..8d2acd2b 100644
--- a/tests/f_multithread/expect.1
+++ b/tests/f_multithread/expect.1
@@ -1,6 +1,8 @@
ext2fs_open2: Bad magic number in super-block
../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
-Pass 1: Checking inodes, blocks, and sizes
+[Thread 0] Scan group range [0, 2)
+[Thread 0] Pass 1: Checking inodes, blocks, and sizes
+[Thread 0] Scanned group range [0, 2), inodes 3008
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
diff --git a/tests/f_multithread_logfile/expect.1 b/tests/f_multithread_logfile/expect.1
index e2b954d0..8d2acd2b 100644
--- a/tests/f_multithread_logfile/expect.1
+++ b/tests/f_multithread_logfile/expect.1
@@ -1,6 +1,8 @@
ext2fs_open2: Bad magic number in super-block
../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
-Pass 1: Checking inodes, blocks, and sizes
+[Thread 0] Scan group range [0, 2)
+[Thread 0] Pass 1: Checking inodes, blocks, and sizes
+[Thread 0] Scanned group range [0, 2), inodes 3008
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
diff --git a/tests/f_multithread_no/expect.1 b/tests/f_multithread_no/expect.1
index d14c4083..f85a3382 100644
--- a/tests/f_multithread_no/expect.1
+++ b/tests/f_multithread_no/expect.1
@@ -1,6 +1,8 @@
ext2fs_open2: Bad magic number in super-block
../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
-Pass 1: Checking inodes, blocks, and sizes
+[Thread 0] Scan group range [0, 2)
+[Thread 0] Pass 1: Checking inodes, blocks, and sizes
+[Thread 0] Scanned group range [0, 2), inodes 3008
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
--
2.25.4
From: Li Xi <[email protected]>
A new method merge_bmap has been added to bitmap operations. But
only red-black bitmap has that operation now.
Signed-off-by: Li Xi <[email protected]>
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/pass1.c | 149 +++++++++++++++++++++++++++++---------
lib/ext2fs/bitmaps.c | 7 ++
lib/ext2fs/blkmap64_rb.c | 25 +++++++
lib/ext2fs/bmap64.h | 2 +
lib/ext2fs/ext2fs.h | 4 +
lib/ext2fs/gen_bitmap64.c | 21 ++++++
6 files changed, 173 insertions(+), 35 deletions(-)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 64666663..8f24df0e 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -2105,31 +2105,57 @@ endit:
ctx->invalid_bitmaps++;
}
-#define PASS1_COPY_FS_BITMAP(_dest, _src, _map_filed) \
+#define PASS1_COPY_FS_BITMAP(_dest, _src, _map_filed) \
+do { \
+ errcode_t _ret; \
+ if (_src->_map_filed) { \
+ _ret = ext2fs_copy_bitmap(_src->_map_filed, \
+ &_dest->_map_filed); \
+ if (_ret) \
+ return _ret; \
+ _dest->_map_filed->fs = _dest; \
+ } \
+} while (0)
+
+#define PASS1_MERGE_FS_BITMAP(_dest, _src, _map_field) \
do { \
- errcode_t _ret; \
- if (_src->_map_filed) { \
- _ret = ext2fs_copy_bitmap(_src->_map_filed, &_dest->_map_filed);\
- if (_ret) \
- return _ret; \
- _dest->_map_filed->fs = _dest; \
- \
- ext2fs_free_generic_bmap(_src->_map_filed); \
- _src->_map_filed = NULL; \
- } \
+ errcode_t _ret = 0; \
+ if (_src->_map_field) { \
+ if (_dest->_map_field == NULL) { \
+ _dest->_map_field = _src->_map_field; \
+ _src->_map_field = NULL; \
+ } else { \
+ _ret = ext2fs_merge_bitmap(_src->_map_field, \
+ _dest->_map_field); \
+ if (_ret) \
+ return _ret; \
+ } \
+ _dest->_map_field->fs = _dest; \
+ } \
} while (0)
-#define PASS1_COPY_CTX_BITMAP(_dest, _src, _map_filed) \
+#define PASS1_MERGE_CTX_BITMAP(_dest, _src, _map_field) \
do { \
- errcode_t _ret; \
- if (_src->_map_filed) { \
- _ret = ext2fs_copy_bitmap(_src->_map_filed, &_dest->_map_filed);\
- if (_ret) \
- return _ret; \
- _dest->_map_filed->fs = _dest->fs; \
- \
- ext2fs_free_generic_bmap(_src->_map_filed); \
- _src->_map_filed = NULL; \
+ errcode_t _ret = 0; \
+ if (_src->_map_field) { \
+ if (_dest->_map_field == NULL) { \
+ _dest->_map_field = _src->_map_field; \
+ _src->_map_field = NULL; \
+ } else { \
+ _ret = ext2fs_merge_bitmap(_src->_map_field, \
+ _dest->_map_field); \
+ if (_ret) \
+ return _ret; \
+ } \
+ _dest->_map_field->fs = _dest->fs; \
+ } \
+} while (0)
+
+#define PASS1_FREE_CTX_BITMAP(_src, _map_field) \
+do { \
+ if (_src->_map_field) { \
+ ext2fs_free_generic_bmap(_src->_map_field); \
+ _src->_map_field = NULL; \
} \
} while (0)
@@ -2168,6 +2194,9 @@ static errcode_t e2fsck_pass1_copy_fs(ext2_filsys dest, e2fsck_t src_context,
errcode_t retval;
memcpy(dest, src, sizeof(struct struct_ext2_filsys));
+ dest->inode_map = NULL;
+ dest->block_map = NULL;
+
/*
* PASS1_COPY_FS_BITMAP might return directly from this function,
* so please do NOT leave any garbage behind after returning.
@@ -2241,18 +2270,24 @@ static int _e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
errcode_t retval = 0;
io_channel dest_io;
io_channel dest_image_io;
+ ext2fs_inode_bitmap inode_map;
+ ext2fs_block_bitmap block_map;
dest_io = dest->io;
dest_image_io = dest->image_io;
+ inode_map = dest->inode_map;
+ block_map = dest->block_map;
memcpy(dest, src, sizeof(struct struct_ext2_filsys));
dest->io = dest_io;
dest->image_io = dest_image_io;
+ dest->inode_map = inode_map;
+ dest->block_map = block_map;
/*
- * PASS1_COPY_FS_BITMAP might return directly from this function,
+ * PASS1_MERGE_FS_BITMAP might return directly from this function,
* so please do NOT leave any garbage behind after returning.
*/
- PASS1_COPY_FS_BITMAP(dest, src, inode_map);
- PASS1_COPY_FS_BITMAP(dest, src, block_map);
+ PASS1_MERGE_FS_BITMAP(dest, src, inode_map);
+ PASS1_MERGE_FS_BITMAP(dest, src, block_map);
if (src->dblist) {
retval = ext2fs_copy_dblist(src->dblist, &dest->dblist);
@@ -2281,6 +2316,11 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
retval = _e2fsck_pass1_merge_fs(dest, src);
+ if (src->inode_map)
+ ext2fs_free_generic_bmap(src->inode_map);
+ if (src->block_map)
+ ext2fs_free_generic_bmap(src->block_map);
+
/* icache will be rebuilt if needed, so do not copy from @src */
if (src->icache) {
ext2fs_free_inode_cache(src->icache);
@@ -2388,6 +2428,17 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
ext2_filsys global_fs = global_ctx->fs;
FILE *global_logf = global_ctx->logf;
FILE *global_problem_logf = global_ctx->problem_logf;
+ ext2fs_inode_bitmap inode_used_map = global_ctx->inode_used_map;
+ ext2fs_inode_bitmap inode_dir_map = global_ctx->inode_dir_map;
+ ext2fs_inode_bitmap inode_bb_map = global_ctx->inode_bb_map;
+ ext2fs_inode_bitmap inode_imagic_map = global_ctx->inode_imagic_map;
+ ext2fs_inode_bitmap inode_reg_map = global_ctx->inode_reg_map;
+ ext2fs_block_bitmap block_found_map = global_ctx->block_found_map;
+ ext2fs_block_bitmap block_dup_map = global_ctx->block_dup_map;
+ ext2fs_block_bitmap block_ea_map = global_ctx->block_ea_map;
+ ext2fs_block_bitmap block_metadata_map = global_ctx->block_metadata_map;
+ ext2fs_block_bitmap inodes_to_rebuild = global_ctx->inodes_to_rebuild;
+
#ifdef HAVE_SETJMP_H
jmp_buf old_jmp;
@@ -2397,6 +2448,18 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
#ifdef HAVE_SETJMP_H
memcpy(global_ctx->abort_loc, old_jmp, sizeof(jmp_buf));
#endif
+
+ global_ctx->inode_used_map = inode_used_map;
+ global_ctx->inode_dir_map = inode_dir_map;
+ global_ctx->inode_bb_map = inode_bb_map;
+ global_ctx->inode_imagic_map = inode_imagic_map;
+ global_ctx->inodes_to_rebuild = inodes_to_rebuild;
+ global_ctx->inode_reg_map = inode_reg_map;
+ global_ctx->block_found_map = block_found_map;
+ global_ctx->block_dup_map = block_dup_map;
+ global_ctx->block_ea_map = block_ea_map;
+ global_ctx->block_metadata_map = block_metadata_map;
+
/* Keep the global singal flags*/
global_ctx->flags |= (flags & E2F_FLAG_SIGNAL_MASK) |
(global_ctx->flags & E2F_FLAG_SIGNAL_MASK);
@@ -2416,16 +2479,16 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
* PASS1_COPY_CTX_BITMAP might return directly from this function,
* so please do NOT leave any garbage behind after returning.
*/
- PASS1_COPY_CTX_BITMAP(global_ctx, thread_ctx, inode_used_map);
- PASS1_COPY_CTX_BITMAP(global_ctx, thread_ctx, inode_dir_map);
- PASS1_COPY_CTX_BITMAP(global_ctx, thread_ctx, inode_bb_map);
- PASS1_COPY_CTX_BITMAP(global_ctx, thread_ctx, inode_imagic_map);
- PASS1_COPY_CTX_BITMAP(global_ctx, thread_ctx, inode_reg_map);
- PASS1_COPY_CTX_BITMAP(global_ctx, thread_ctx, inodes_to_rebuild);
- PASS1_COPY_CTX_BITMAP(global_ctx, thread_ctx, block_found_map);
- PASS1_COPY_CTX_BITMAP(global_ctx, thread_ctx, block_dup_map);
- PASS1_COPY_CTX_BITMAP(global_ctx, thread_ctx, block_ea_map);
- PASS1_COPY_CTX_BITMAP(global_ctx, thread_ctx, block_metadata_map);
+ PASS1_MERGE_CTX_BITMAP(global_ctx, thread_ctx, inode_used_map);
+ PASS1_MERGE_CTX_BITMAP(global_ctx, thread_ctx, inode_dir_map);
+ PASS1_MERGE_CTX_BITMAP(global_ctx, thread_ctx, inode_bb_map);
+ PASS1_MERGE_CTX_BITMAP(global_ctx, thread_ctx, inode_imagic_map);
+ PASS1_MERGE_CTX_BITMAP(global_ctx, thread_ctx, inode_reg_map);
+ PASS1_MERGE_CTX_BITMAP(global_ctx, thread_ctx, inodes_to_rebuild);
+ PASS1_MERGE_CTX_BITMAP(global_ctx, thread_ctx, block_found_map);
+ PASS1_MERGE_CTX_BITMAP(global_ctx, thread_ctx, block_dup_map);
+ PASS1_MERGE_CTX_BITMAP(global_ctx, thread_ctx, block_ea_map);
+ PASS1_MERGE_CTX_BITMAP(global_ctx, thread_ctx, block_metadata_map);
return 0;
}
@@ -2442,6 +2505,16 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
fputs("</problem_log>\n", thread_ctx->problem_logf);
fclose(thread_ctx->problem_logf);
}
+ PASS1_FREE_CTX_BITMAP(thread_ctx, inode_used_map);
+ PASS1_FREE_CTX_BITMAP(thread_ctx, inode_dir_map);
+ PASS1_FREE_CTX_BITMAP(thread_ctx, inode_bb_map);
+ PASS1_FREE_CTX_BITMAP(thread_ctx, inode_imagic_map);
+ PASS1_FREE_CTX_BITMAP(thread_ctx, inode_reg_map);
+ PASS1_FREE_CTX_BITMAP(thread_ctx, inodes_to_rebuild);
+ PASS1_FREE_CTX_BITMAP(thread_ctx, block_found_map);
+ PASS1_FREE_CTX_BITMAP(thread_ctx, block_dup_map);
+ PASS1_FREE_CTX_BITMAP(thread_ctx, block_ea_map);
+ PASS1_FREE_CTX_BITMAP(thread_ctx, block_metadata_map);
ext2fs_free_mem(&thread_ctx);
return retval;
@@ -2468,7 +2541,13 @@ static int e2fsck_pass1_threads_join(struct e2fsck_thread_info *infos,
if (ret == 0)
ret = rc;
}
- e2fsck_pass1_thread_join(global_ctx, infos[i].eti_thread_ctx);
+ rc = e2fsck_pass1_thread_join(global_ctx, infos[i].eti_thread_ctx);
+ if (rc) {
+ com_err(global_ctx->program_name, rc,
+ _("while joining pass1 thread\n"));
+ if (ret == 0)
+ ret = rc;
+ }
}
free(infos);
diff --git a/lib/ext2fs/bitmaps.c b/lib/ext2fs/bitmaps.c
index 834a3962..74a8c347 100644
--- a/lib/ext2fs/bitmaps.c
+++ b/lib/ext2fs/bitmaps.c
@@ -45,6 +45,13 @@ errcode_t ext2fs_copy_bitmap(ext2fs_generic_bitmap src,
{
return (ext2fs_copy_generic_bmap(src, dest));
}
+
+errcode_t ext2fs_merge_bitmap(ext2fs_generic_bitmap src,
+ ext2fs_generic_bitmap dest)
+{
+ return ext2fs_merge_generic_bmap(src, dest);
+}
+
void ext2fs_set_bitmap_padding(ext2fs_generic_bitmap map)
{
ext2fs_set_generic_bmap_padding(map);
diff --git a/lib/ext2fs/blkmap64_rb.c b/lib/ext2fs/blkmap64_rb.c
index 1fd55274..42a10536 100644
--- a/lib/ext2fs/blkmap64_rb.c
+++ b/lib/ext2fs/blkmap64_rb.c
@@ -968,11 +968,36 @@ static void rb_print_stats(ext2fs_generic_bitmap_64 bitmap EXT2FS_ATTR((unused))
}
#endif
+static errcode_t rb_merge_bmap(ext2fs_generic_bitmap_64 src,
+ ext2fs_generic_bitmap_64 dest)
+{
+ struct ext2fs_rb_private *src_bp, *dest_bp;
+ struct bmap_rb_extent *src_ext;
+ struct rb_node *src_node;
+ errcode_t retval = 0;
+
+ src_bp = (struct ext2fs_rb_private *) src->private;
+ dest_bp = (struct ext2fs_rb_private *) dest->private;
+ src_bp->rcursor = NULL;
+ dest_bp->rcursor = NULL;
+
+ src_node = ext2fs_rb_first(&src_bp->root);
+ while (src_node) {
+ src_ext = node_to_extent(src_node);
+ rb_insert_extent(src_ext->start, src_ext->count, dest_bp);
+
+ src_node = ext2fs_rb_next(src_node);
+ }
+
+ return retval;
+}
+
struct ext2_bitmap_ops ext2fs_blkmap64_rbtree = {
.type = EXT2FS_BMAP64_RBTREE,
.new_bmap = rb_new_bmap,
.free_bmap = rb_free_bmap,
.copy_bmap = rb_copy_bmap,
+ .merge_bmap = rb_merge_bmap,
.resize_bmap = rb_resize_bmap,
.mark_bmap = rb_mark_bmap,
.unmark_bmap = rb_unmark_bmap,
diff --git a/lib/ext2fs/bmap64.h b/lib/ext2fs/bmap64.h
index de334548..09a5886b 100644
--- a/lib/ext2fs/bmap64.h
+++ b/lib/ext2fs/bmap64.h
@@ -72,6 +72,8 @@ struct ext2_bitmap_ops {
void (*free_bmap)(ext2fs_generic_bitmap_64 bitmap);
errcode_t (*copy_bmap)(ext2fs_generic_bitmap_64 src,
ext2fs_generic_bitmap_64 dest);
+ errcode_t (*merge_bmap)(ext2fs_generic_bitmap_64 src,
+ ext2fs_generic_bitmap_64 dest);
errcode_t (*resize_bmap)(ext2fs_generic_bitmap_64 bitmap,
__u64 new_end,
__u64 new_real_end);
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index 69c8a3ff..d5b1af2f 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -837,6 +837,8 @@ extern void ext2fs_free_block_bitmap(ext2fs_block_bitmap bitmap);
extern void ext2fs_free_inode_bitmap(ext2fs_inode_bitmap bitmap);
extern errcode_t ext2fs_copy_bitmap(ext2fs_generic_bitmap src,
ext2fs_generic_bitmap *dest);
+errcode_t ext2fs_merge_bitmap(ext2fs_generic_bitmap src,
+ ext2fs_generic_bitmap dest);
extern errcode_t ext2fs_write_inode_bitmap(ext2_filsys fs);
extern errcode_t ext2fs_write_block_bitmap (ext2_filsys fs);
extern errcode_t ext2fs_read_inode_bitmap (ext2_filsys fs);
@@ -1433,6 +1435,8 @@ void ext2fs_set_generic_bmap_padding(ext2fs_generic_bitmap bmap);
errcode_t ext2fs_resize_generic_bmap(ext2fs_generic_bitmap bmap,
__u64 new_end,
__u64 new_real_end);
+errcode_t ext2fs_merge_generic_bmap(ext2fs_generic_bitmap gen_src,
+ ext2fs_generic_bitmap gen_dest);
errcode_t ext2fs_compare_generic_bmap(errcode_t neq,
ext2fs_generic_bitmap bm1,
ext2fs_generic_bitmap bm2);
diff --git a/lib/ext2fs/gen_bitmap64.c b/lib/ext2fs/gen_bitmap64.c
index b2370667..2617ac22 100644
--- a/lib/ext2fs/gen_bitmap64.c
+++ b/lib/ext2fs/gen_bitmap64.c
@@ -344,6 +344,27 @@ errcode_t ext2fs_copy_generic_bmap(ext2fs_generic_bitmap gen_src,
return 0;
}
+errcode_t ext2fs_merge_generic_bmap(ext2fs_generic_bitmap gen_src,
+ ext2fs_generic_bitmap gen_dest)
+{
+ ext2fs_generic_bitmap_64 src = (ext2fs_generic_bitmap_64) gen_src;
+ ext2fs_generic_bitmap_64 dest = (ext2fs_generic_bitmap_64) gen_dest;
+
+ if (!src || !dest)
+ return EINVAL;
+
+ if (!EXT2FS_IS_64_BITMAP(src) || !EXT2FS_IS_64_BITMAP(dest))
+ return EINVAL;
+
+ if (src->bitmap_ops != dest->bitmap_ops)
+ return EINVAL;
+
+ if (src->bitmap_ops->merge_bmap == NULL)
+ return EOPNOTSUPP;
+
+ return src->bitmap_ops->merge_bmap(src, dest);
+}
+
errcode_t ext2fs_resize_generic_bmap(ext2fs_generic_bitmap gen_bmap,
__u64 new_end,
__u64 new_real_end)
--
2.25.4
From: Li Xi <[email protected]>
Global variables used in pass1 check are changed to local variables
in this patch. This will avoid conflict between threads.
Signed-off-by: Li Xi <[email protected]>
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/pass1.c | 75 +++++++++++++++++++++++++++++++-------------------
1 file changed, 47 insertions(+), 28 deletions(-)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 8f24df0e..85444421 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -84,7 +84,6 @@ static void alloc_bb_map(e2fsck_t ctx);
static void alloc_imagic_map(e2fsck_t ctx);
static void mark_inode_bad(e2fsck_t ctx, ino_t ino);
static void handle_fs_bad_blocks(e2fsck_t ctx);
-static void process_inodes(e2fsck_t ctx, char *block_buf);
static EXT2_QSORT_TYPE process_inode_cmp(const void *a, const void *b);
static errcode_t scan_callback(ext2_filsys fs, ext2_inode_scan scan,
dgrp_t group, void * priv_data);
@@ -119,15 +118,15 @@ struct process_inode_block {
};
struct scan_callback_struct {
- e2fsck_t ctx;
- char *block_buf;
+ e2fsck_t ctx;
+ char *block_buf;
+ struct process_inode_block *inodes_to_process;
+ int *process_inode_count;
};
-/*
- * For the inodes to process list.
- */
-static struct process_inode_block *inodes_to_process;
-static int process_inode_count;
+static void process_inodes(e2fsck_t ctx, char *block_buf,
+ struct process_inode_block *inodes_to_process,
+ int *process_inode_count);
static __u64 ext2_max_sizes[EXT2_MAX_BLOCK_LOG_SIZE -
EXT2_MIN_BLOCK_LOG_SIZE + 1];
@@ -1166,7 +1165,6 @@ static int e2fsck_should_abort(e2fsck_t ctx)
void _e2fsck_pass1(e2fsck_t ctx)
{
int i;
- __u64 max_sizes;
ext2_filsys fs = ctx->fs;
ext2_ino_t ino = 0;
struct ext2_inode *inode = NULL;
@@ -1189,6 +1187,8 @@ void _e2fsck_pass1(e2fsck_t ctx)
ext2_ino_t ino_threshold = 0;
dgrp_t ra_group = 0;
struct ea_quota ea_ibody_quota;
+ struct process_inode_block *inodes_to_process;
+ int process_inode_count;
init_resource_track(&rtrack, ctx->fs->io);
clear_problem_context(&pctx);
@@ -1213,17 +1213,6 @@ void _e2fsck_pass1(e2fsck_t ctx)
mtrace_print("Pass 1");
#endif
-#define EXT2_BPP(bits) (1ULL << ((bits) - 2))
-
- for (i = EXT2_MIN_BLOCK_LOG_SIZE; i <= EXT2_MAX_BLOCK_LOG_SIZE; i++) {
- max_sizes = EXT2_NDIR_BLOCKS + EXT2_BPP(i);
- max_sizes = max_sizes + EXT2_BPP(i) * EXT2_BPP(i);
- max_sizes = max_sizes + EXT2_BPP(i) * EXT2_BPP(i) * EXT2_BPP(i);
- max_sizes = (max_sizes * (1UL << i));
- ext2_max_sizes[i - EXT2_MIN_BLOCK_LOG_SIZE] = max_sizes;
- }
-#undef EXT2_BPP
-
imagic_fs = ext2fs_has_feature_imagic_inodes(sb);
extent_fs = ext2fs_has_feature_extents(sb);
inlinedata_fs = ext2fs_has_feature_inline_data(sb);
@@ -1347,6 +1336,8 @@ void _e2fsck_pass1(e2fsck_t ctx)
ctx->stashed_inode = inode;
scan_struct.ctx = ctx;
scan_struct.block_buf = block_buf;
+ scan_struct.inodes_to_process = inodes_to_process;
+ scan_struct.process_inode_count = &process_inode_count;
ext2fs_set_inode_callback(scan, scan_callback, &scan_struct);
if (ctx->progress && ((ctx->progress)(ctx, 1, 0,
ctx->fs->group_desc_count)))
@@ -1989,13 +1980,15 @@ void _e2fsck_pass1(e2fsck_t ctx)
goto endit;
if (process_inode_count >= ctx->process_inode_size) {
- process_inodes(ctx, block_buf);
+ process_inodes(ctx, block_buf, inodes_to_process,
+ &process_inode_count);
if (e2fsck_should_abort(ctx))
goto endit;
}
}
- process_inodes(ctx, block_buf);
+ process_inodes(ctx, block_buf, inodes_to_process,
+ &process_inode_count);
ext2fs_close_inode_scan(scan);
scan = NULL;
@@ -2653,12 +2646,34 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo,
return 0;
}
+static void init_ext2_max_sizes()
+{
+ int i;
+ __u64 max_sizes;
+
+ /*
+ * Init ext2_max_sizes which will be immutable and shared between
+ * threads
+ */
+#define EXT2_BPP(bits) (1ULL << ((bits) - 2))
+
+ for (i = EXT2_MIN_BLOCK_LOG_SIZE; i <= EXT2_MAX_BLOCK_LOG_SIZE; i++) {
+ max_sizes = EXT2_NDIR_BLOCKS + EXT2_BPP(i);
+ max_sizes = max_sizes + EXT2_BPP(i) * EXT2_BPP(i);
+ max_sizes = max_sizes + EXT2_BPP(i) * EXT2_BPP(i) * EXT2_BPP(i);
+ max_sizes = (max_sizes * (1UL << i));
+ ext2_max_sizes[i - EXT2_MIN_BLOCK_LOG_SIZE] = max_sizes;
+ }
+#undef EXT2_BPP
+}
+
static void e2fsck_pass1_multithread(e2fsck_t global_ctx)
{
struct e2fsck_thread_info *infos = NULL;
int num_threads = 1;
errcode_t retval;
+ init_ext2_max_sizes();
retval = e2fsck_pass1_threads_start(&infos, num_threads, global_ctx);
if (retval) {
com_err(global_ctx->program_name, retval,
@@ -2700,7 +2715,9 @@ static errcode_t scan_callback(ext2_filsys fs,
scan_struct = (struct scan_callback_struct *) priv_data;
ctx = scan_struct->ctx;
- process_inodes((e2fsck_t) fs->priv_data, scan_struct->block_buf);
+ process_inodes((e2fsck_t) fs->priv_data, scan_struct->block_buf,
+ scan_struct->inodes_to_process,
+ scan_struct->process_inode_count);
if (ctx->progress)
if ((ctx->progress)(ctx, 1, group+1,
@@ -2721,7 +2738,9 @@ static errcode_t scan_callback(ext2_filsys fs,
/*
* Process the inodes in the "inodes to process" list.
*/
-static void process_inodes(e2fsck_t ctx, char *block_buf)
+static void process_inodes(e2fsck_t ctx, char *block_buf,
+ struct process_inode_block *inodes_to_process,
+ int *process_inode_count)
{
int i;
struct ext2_inode *old_stashed_inode;
@@ -2733,15 +2752,15 @@ static void process_inodes(e2fsck_t ctx, char *block_buf)
#if 0
printf("begin process_inodes: ");
#endif
- if (process_inode_count == 0)
+ if (*process_inode_count == 0)
return;
old_operation = ehandler_operation(0);
old_stashed_inode = ctx->stashed_inode;
old_stashed_ino = ctx->stashed_ino;
- qsort(inodes_to_process, process_inode_count,
+ qsort(inodes_to_process, *process_inode_count,
sizeof(struct process_inode_block), process_inode_cmp);
clear_problem_context(&pctx);
- for (i=0; i < process_inode_count; i++) {
+ for (i=0; i < *process_inode_count; i++) {
pctx.inode = ctx->stashed_inode =
(struct ext2_inode *) &inodes_to_process[i].inode;
pctx.ino = ctx->stashed_ino = inodes_to_process[i].ino;
@@ -2759,7 +2778,7 @@ static void process_inodes(e2fsck_t ctx, char *block_buf)
}
ctx->stashed_inode = old_stashed_inode;
ctx->stashed_ino = old_stashed_ino;
- process_inode_count = 0;
+ *process_inode_count = 0;
#if 0
printf("end process inodes\n");
#endif
--
2.25.4
From: Li Xi <[email protected]>
The start/end groups of a thread is calculated according to the
thread number. But still, only one thread is used to check.
Signed-off-by: Li Xi <[email protected]>
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/pass1.c | 24 ++++++++++++++++++------
1 file changed, 18 insertions(+), 6 deletions(-)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 45915513..c676cbbc 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -2301,13 +2301,15 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx,
e2fsck_t *thread_ctx,
- int thread_index)
+ int thread_index,
+ int num_threads)
{
errcode_t retval;
e2fsck_t thread_context;
ext2_filsys thread_fs;
ext2_filsys global_fs = global_ctx->fs;
struct e2fsck_thread *tinfo;
+ dgrp_t average_group;
assert(global_ctx->inode_used_map == NULL);
assert(global_ctx->inode_dir_map == NULL);
@@ -2346,11 +2348,20 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx,
thread_context->thread_info.et_thread_index = thread_index;
set_up_logging(thread_context);
- assert(thread_index == 0);
+ /*
+ * Distribute work to multiple threads:
+ * Each thread work on fs->group_desc_count / nthread groups.
+ */
tinfo = &thread_context->thread_info;
- tinfo->et_group_start = 0;
- tinfo->et_group_next = 0;
- tinfo->et_group_end = thread_fs->group_desc_count;
+ average_group = thread_fs->group_desc_count / num_threads;
+ if (average_group == 0)
+ average_group = 1;
+ tinfo->et_group_start = average_group * thread_index;
+ if (thread_index == num_threads - 1)
+ tinfo->et_group_end = thread_fs->group_desc_count;
+ else
+ tinfo->et_group_end = average_group * (thread_index + 1);
+ tinfo->et_group_next = tinfo->et_group_start;
thread_context->fs = thread_fs;
*thread_ctx = thread_context;
@@ -2511,7 +2522,8 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo,
for (i = 0; i < num_threads; i++) {
tmp_pinfo = &infos[i];
tmp_pinfo->eti_thread_index = i;
- retval = e2fsck_pass1_thread_prepare(global_ctx, &thread_ctx, i);
+ retval = e2fsck_pass1_thread_prepare(global_ctx, &thread_ctx,
+ i, num_threads);
if (retval) {
com_err(global_ctx->program_name, retval,
_("while preparing pass1 thread\n"));
--
2.25.4
From: Li Xi <[email protected]>
Signed-off-by: Li Xi <[email protected]>
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/dirinfo.c | 66 ++++++++++++++++++++++++++++++++++++++++++++++++
e2fsck/e2fsck.h | 2 ++
e2fsck/pass1.c | 20 +++++++++++++++
3 files changed, 88 insertions(+)
diff --git a/e2fsck/dirinfo.c b/e2fsck/dirinfo.c
index f299620f..ea2fc75b 100644
--- a/e2fsck/dirinfo.c
+++ b/e2fsck/dirinfo.c
@@ -168,6 +168,72 @@ e2fsck_dir_info_min_larger_equal(struct dir_info_db *dir_info,
return -ENOENT;
}
+/*
+ * Merge two sorted dir info to @dest
+ */
+void e2fsck_merge_dir_info(e2fsck_t ctx, struct dir_info_db *src,
+ struct dir_info_db *dest)
+{
+ int size_dir_info = sizeof(struct dir_info);
+ ext2_ino_t size = dest->size;
+ struct dir_info *src_array = src->array;
+ struct dir_info *dest_array = dest->array;
+ ext2_ino_t src_count = src->count;
+ ext2_ino_t dest_count = dest->count;
+ ext2_ino_t total_count = src_count + dest_count;
+ struct dir_info *array;
+ struct dir_info *array_ptr;
+ ext2_ino_t src_index = 0;
+ ext2_ino_t dest_index = 0;
+
+ if (src->count == 0)
+ return;
+
+ if (size < total_count)
+ size = total_count;
+
+ if (size < src->size)
+ size = src->size;
+
+ array = e2fsck_allocate_memory(ctx, size * size_dir_info,
+ "directory map");
+ array_ptr = array;
+ /*
+ * This can be improved by binary search and memcpy, but codes
+ * would be complexer. And if the groups distributed to each
+ * thread are stided, this implementation won't be too bad comparing
+ * to the optimiztion.
+ */
+ while (src_index < src_count || dest_index < dest_count) {
+ if (src_index >= src_count) {
+ memcpy(array_ptr, &dest_array[dest_index],
+ (dest_count - dest_index) * size_dir_info);
+ break;
+ }
+ if (dest_index >= dest_count) {
+ memcpy(array_ptr, &src_array[src_index],
+ (src_count - src_index) * size_dir_info);
+ break;
+ }
+ if (src_array[src_index].ino < dest_array[dest_index].ino) {
+ *array_ptr = src_array[src_index];
+ src_index++;
+ } else {
+ assert(src_array[src_index].ino >
+ dest_array[dest_index].ino);
+ *array_ptr = dest_array[dest_index];
+ dest_index++;
+ }
+ array_ptr++;
+ }
+
+ if (dest->array)
+ ext2fs_free_mem(&dest->array);
+ dest->array = array;
+ dest->size = size;
+ dest->count = total_count;
+}
+
/*
*
* Insert an inode into the sorted array. The array should have at least one
diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
index 121d1b9b..24f164a7 100644
--- a/e2fsck/e2fsck.h
+++ b/e2fsck/e2fsck.h
@@ -502,6 +502,8 @@ extern void read_bad_blocks_file(e2fsck_t ctx, const char *bad_blocks_file,
/* dirinfo.c */
extern void e2fsck_add_dir_info(e2fsck_t ctx, ext2_ino_t ino, ext2_ino_t parent);
+void e2fsck_merge_dir_info(e2fsck_t ctx, struct dir_info_db *src,
+ struct dir_info_db *dest);
extern void e2fsck_free_dir_info(e2fsck_t ctx);
extern int e2fsck_get_num_dirinfo(e2fsck_t ctx);
extern struct dir_info_iter *e2fsck_dir_info_iter_begin(e2fsck_t ctx);
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 85444421..7a839d4b 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -2413,6 +2413,22 @@ out_context:
return retval;
}
+static void e2fsck_pass1_merge_dir_info(e2fsck_t global_ctx, e2fsck_t thread_ctx)
+{
+ if (thread_ctx->dir_info == NULL)
+ return;
+
+ if (global_ctx->dir_info == NULL) {
+ /* TODO: tdb needs to be handled properly */
+ global_ctx->dir_info = thread_ctx->dir_info;
+ thread_ctx->dir_info = NULL;
+ return;
+ }
+
+ e2fsck_merge_dir_info(global_ctx, thread_ctx->dir_info,
+ global_ctx->dir_info);
+}
+
static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx)
{
errcode_t retval;
@@ -2421,6 +2437,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
ext2_filsys global_fs = global_ctx->fs;
FILE *global_logf = global_ctx->logf;
FILE *global_problem_logf = global_ctx->problem_logf;
+ struct dir_info_db *dir_info = global_ctx->dir_info;
ext2fs_inode_bitmap inode_used_map = global_ctx->inode_used_map;
ext2fs_inode_bitmap inode_dir_map = global_ctx->inode_dir_map;
ext2fs_inode_bitmap inode_bb_map = global_ctx->inode_bb_map;
@@ -2452,6 +2469,8 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
global_ctx->block_dup_map = block_dup_map;
global_ctx->block_ea_map = block_ea_map;
global_ctx->block_metadata_map = block_metadata_map;
+ global_ctx->dir_info = dir_info;
+ e2fsck_pass1_merge_dir_info(global_ctx, thread_ctx);
/* Keep the global singal flags*/
global_ctx->flags |= (flags & E2F_FLAG_SIGNAL_MASK) |
@@ -2508,6 +2527,7 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
PASS1_FREE_CTX_BITMAP(thread_ctx, block_dup_map);
PASS1_FREE_CTX_BITMAP(thread_ctx, block_ea_map);
PASS1_FREE_CTX_BITMAP(thread_ctx, block_metadata_map);
+ e2fsck_free_dir_info(thread_ctx);
ext2fs_free_mem(&thread_ctx);
return retval;
--
2.25.4
From: Li Xi <[email protected]>
Binary search is now used when inserting an dir info to the array.
Memmove is now used when moving array. Both of them improves
the performance of inserting.
This patch is also a prepartion for the merging of two dir db
arrays.
Signed-off-by: Li Xi <[email protected]>
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/dirinfo.c | 171 ++++++++++++++++++++++++++++++-----------------
1 file changed, 111 insertions(+), 60 deletions(-)
diff --git a/e2fsck/dirinfo.c b/e2fsck/dirinfo.c
index 49d624c5..f299620f 100644
--- a/e2fsck/dirinfo.c
+++ b/e2fsck/dirinfo.c
@@ -7,6 +7,7 @@
#undef DIRINFO_DEBUG
+#include <assert.h>
#include "config.h"
#include "e2fsck.h"
#include <sys/stat.h>
@@ -122,6 +123,103 @@ static void setup_db(e2fsck_t ctx)
"directory map");
}
+/*
+ * Return the min index that has ino larger or equal to @ino
+ * If not found, return -ENOENT
+ */
+static int
+e2fsck_dir_info_min_larger_equal(struct dir_info_db *dir_info,
+ ext2_ino_t ino, ext2_ino_t *index)
+{
+ ext2_ino_t low = 0;
+ ext2_ino_t mid, high;
+ ext2_ino_t tmp_ino;
+ int found = 0;
+
+ if (dir_info->count == 0)
+ return -ENOENT;
+
+ high = dir_info->count - 1;
+ while (low <= high) {
+ mid = (low + high) / 2;
+ tmp_ino = dir_info->array[mid].ino;
+ if (ino == tmp_ino) {
+ *index = mid;
+ found = 1;
+ return 0;
+ } else if (ino < tmp_ino) {
+ /*
+ * The mid ino is larger than @ino, remember the index
+ * here so we won't miss this ino
+ */
+ *index = mid;
+ found = 1;
+ if (mid == 0)
+ break;
+ high = mid - 1;
+ } else {
+ low = mid + 1;
+ }
+ }
+
+ if (found)
+ return 0;
+
+ return -ENOENT;
+}
+
+/*
+ *
+ * Insert an inode into the sorted array. The array should have at least one
+ * free slot.
+ *
+ * Normally, add_dir_info is called with each inode in
+ * sequential order; but once in a while (like when pass 3
+ * needs to recreate the root directory or lost+found
+ * directory) it is called out of order. In those cases, we
+ * need to move the dir_info entries down to make room, since
+ * the dir_info array needs to be sorted by inode number for
+ * get_dir_info()'s sake.
+ */
+static void e2fsck_insert_dir_info(struct dir_info_db *dir_info, ext2_ino_t ino, ext2_ino_t parent)
+{
+ ext2_ino_t index;
+ struct dir_info *dir;
+ int dir_size = sizeof(*dir);
+ struct dir_info *array = dir_info->array;
+ ext2_ino_t array_count = dir_info->count;
+ int err;
+
+ /*
+ * Removing this check won't break anything. But since seqential ino
+ * inserting happens a lot, this check avoids binary search.
+ */
+ if (array_count == 0 || array[array_count - 1].ino < ino) {
+ dir = &array[array_count];
+ dir_info->count++;
+ goto out;
+ }
+
+ err = e2fsck_dir_info_min_larger_equal(dir_info, ino, &index);
+ if (err >= 0 && array[index].ino == ino) {
+ dir = &array[index];
+ goto out;
+ }
+ if (err < 0) {
+ dir = &array[array_count];
+ dir_info->count++;
+ goto out;
+ }
+
+ dir = &array[index];
+ memmove((char *)dir + dir_size, dir, dir_size * (array_count - index));
+ dir_info->count++;
+out:
+ dir->ino = ino;
+ dir->dotdot = parent;
+ dir->parent = parent;
+}
+
/*
* This subroutine is called during pass1 to create a directory info
* entry. During pass1, the passed-in parent is 0; it will get filled
@@ -171,30 +269,7 @@ void e2fsck_add_dir_info(e2fsck_t ctx, ext2_ino_t ino, ext2_ino_t parent)
}
#endif
- /*
- * Normally, add_dir_info is called with each inode in
- * sequential order; but once in a while (like when pass 3
- * needs to recreate the root directory or lost+found
- * directory) it is called out of order. In those cases, we
- * need to move the dir_info entries down to make room, since
- * the dir_info array needs to be sorted by inode number for
- * get_dir_info()'s sake.
- */
- if (ctx->dir_info->count &&
- ctx->dir_info->array[ctx->dir_info->count-1].ino >= ino) {
- for (i = ctx->dir_info->count-1; i > 0; i--)
- if (ctx->dir_info->array[i-1].ino < ino)
- break;
- dir = &ctx->dir_info->array[i];
- if (dir->ino != ino)
- for (j = ctx->dir_info->count++; j > i; j--)
- ctx->dir_info->array[j] = ctx->dir_info->array[j-1];
- } else
- dir = &ctx->dir_info->array[ctx->dir_info->count++];
-
- dir->ino = ino;
- dir->dotdot = parent;
- dir->parent = parent;
+ e2fsck_insert_dir_info(ctx->dir_info, ino, parent);
}
/*
@@ -204,7 +279,8 @@ void e2fsck_add_dir_info(e2fsck_t ctx, ext2_ino_t ino, ext2_ino_t parent)
static struct dir_info *e2fsck_get_dir_info(e2fsck_t ctx, ext2_ino_t ino)
{
struct dir_info_db *db = ctx->dir_info;
- ext2_ino_t low, high, mid;
+ ext2_ino_t index;
+ int err;
if (!db)
return 0;
@@ -245,44 +321,19 @@ static struct dir_info *e2fsck_get_dir_info(e2fsck_t ctx, ext2_ino_t ino)
if (db->last_lookup && db->last_lookup->ino == ino)
return db->last_lookup;
- low = 0;
- high = ctx->dir_info->count - 1;
- if (ino == ctx->dir_info->array[low].ino) {
-#ifdef DIRINFO_DEBUG
- printf("(%u,%u,%u)\n", ino,
- ctx->dir_info->array[low].dotdot,
- ctx->dir_info->array[low].parent);
-#endif
- return &ctx->dir_info->array[low];
- }
- if (ino == ctx->dir_info->array[high].ino) {
+ err = e2fsck_dir_info_min_larger_equal(ctx->dir_info, ino, &index);
+ if (err < 0)
+ return NULL;
+ assert(ino <= ctx->dir_info->array[index].ino);
+ if (ino == ctx->dir_info->array[index].ino) {
#ifdef DIRINFO_DEBUG
- printf("(%u,%u,%u)\n", ino,
- ctx->dir_info->array[high].dotdot,
- ctx->dir_info->array[high].parent);
+ printf("(%d,%d,%d)\n", ino,
+ ctx->dir_info->array[index].dotdot,
+ ctx->dir_info->array[index].parent);
#endif
- return &ctx->dir_info->array[high];
+ return &ctx->dir_info->array[index];
}
-
- while (low < high) {
- /* sum may overflow, but result will fit into mid again */
- mid = (unsigned long long)(low + high) / 2;
- if (mid == low || mid == high)
- break;
- if (ino == ctx->dir_info->array[mid].ino) {
-#ifdef DIRINFO_DEBUG
- printf("(%u,%u,%u)\n", ino,
- ctx->dir_info->array[mid].dotdot,
- ctx->dir_info->array[mid].parent);
-#endif
- return &ctx->dir_info->array[mid];
- }
- if (ino < ctx->dir_info->array[mid].ino)
- high = mid;
- else
- low = mid;
- }
- return 0;
+ return NULL;
}
static void e2fsck_put_dir_info(e2fsck_t ctx EXT2FS_NO_TDB_UNUSED,
--
2.25.4
From: Li Xi <[email protected]>
Only rbtree supports merge, so use it for all bitmaps.
Change-Id: I863687ce275f9c891cd2d18c115cb75c6e24f4e4
Signed-off-by: Li Xi <[email protected]>
Signed-off-by: Wang Shilong <[email protected]>
---
lib/ext2fs/gen_bitmap64.c | 7 +------
1 file changed, 1 insertion(+), 6 deletions(-)
diff --git a/lib/ext2fs/gen_bitmap64.c b/lib/ext2fs/gen_bitmap64.c
index 2617ac22..7bae443f 100644
--- a/lib/ext2fs/gen_bitmap64.c
+++ b/lib/ext2fs/gen_bitmap64.c
@@ -86,7 +86,6 @@ static void warn_bitmap(ext2fs_generic_bitmap_64 bitmap,
#define INC_STAT(map, name) ;;
#endif
-
errcode_t ext2fs_alloc_generic_bmap(ext2_filsys fs, errcode_t magic,
int type, __u64 start, __u64 end,
__u64 real_end,
@@ -109,11 +108,7 @@ errcode_t ext2fs_alloc_generic_bmap(ext2_filsys fs, errcode_t magic,
ops = &ext2fs_blkmap64_rbtree;
break;
case EXT2FS_BMAP64_AUTODIR:
- retval = ext2fs_get_num_dirs(fs, &num_dirs);
- if (retval || num_dirs > (fs->super->s_inodes_count / 320))
- ops = &ext2fs_blkmap64_bitarray;
- else
- ops = &ext2fs_blkmap64_rbtree;
+ ops = &ext2fs_blkmap64_rbtree;
break;
default:
return EINVAL;
--
2.25.4
From: Li Xi <[email protected]>
Signed-off-by: Li Xi <[email protected]>
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/pass1.c | 8 ++++-
lib/ext2fs/badblocks.c | 75 ++++++++++++++++++++++++++++++++++++++----
lib/ext2fs/ext2fs.h | 2 ++
lib/ext2fs/ext2fsP.h | 1 -
4 files changed, 77 insertions(+), 9 deletions(-)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 7a839d4b..e343ec00 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -2265,16 +2265,19 @@ static int _e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
io_channel dest_image_io;
ext2fs_inode_bitmap inode_map;
ext2fs_block_bitmap block_map;
+ ext2_badblocks_list badblocks;
dest_io = dest->io;
dest_image_io = dest->image_io;
inode_map = dest->inode_map;
block_map = dest->block_map;
+ badblocks = dest->badblocks;
memcpy(dest, src, sizeof(struct struct_ext2_filsys));
dest->io = dest_io;
dest->image_io = dest_image_io;
dest->inode_map = inode_map;
dest->block_map = block_map;
+ dest->badblocks = badblocks;
/*
* PASS1_MERGE_FS_BITMAP might return directly from this function,
* so please do NOT leave any garbage behind after returning.
@@ -2291,7 +2294,10 @@ static int _e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
}
if (src->badblocks) {
- retval = ext2fs_badblocks_copy(src->badblocks, &dest->badblocks);
+ if (dest->badblocks == NULL)
+ retval = ext2fs_badblocks_copy(src->badblocks, &dest->badblocks);
+ else
+ retval = ext2fs_badblocks_merge(src->badblocks, dest->badblocks);
if (retval)
goto out_dblist;
}
diff --git a/lib/ext2fs/badblocks.c b/lib/ext2fs/badblocks.c
index 0f23983b..addc3d26 100644
--- a/lib/ext2fs/badblocks.c
+++ b/lib/ext2fs/badblocks.c
@@ -11,6 +11,7 @@
#include "config.h"
#include <stdio.h>
+#include <assert.h>
#include <string.h>
#if HAVE_UNISTD_H
#include <unistd.h>
@@ -56,6 +57,65 @@ static errcode_t make_u32_list(int size, int num, __u32 *list,
return 0;
}
+/*
+ * Merge list from src to dest
+ */
+static errcode_t merge_u32_list(ext2_u32_list src, ext2_u32_list dest)
+{
+ errcode_t retval;
+ int src_count = src->num;
+ int dest_count = dest->num;
+ int size = src_count + dest_count;
+ int size_entry = sizeof(blk_t);
+ blk_t *array;
+ blk_t *array_ptr;
+ blk_t *src_array = src->list;
+ blk_t *dest_array = dest->list;
+ int src_index = 0;
+ int dest_index = 0;
+
+ if (src->num == 0)
+ return 0;
+
+ retval = ext2fs_get_array(size, size_entry, &array);
+ if (retval)
+ return retval;
+
+ array_ptr = array;
+ /*
+ * This can be improved by binary search and memcpy, but codes would
+ * be complexer. And if number of bad blocks is small, the optimization
+ * won't improve performance a lot.
+ */
+ while (src_index < src_count || dest_index < dest_count) {
+ if (src_index >= src_count) {
+ memcpy(array_ptr, &dest_array[dest_index],
+ (dest_count - dest_index) * size_entry);
+ break;
+ }
+ if (dest_index >= dest_count) {
+ memcpy(array_ptr, &src_array[src_index],
+ (src_count - src_index) * size_entry);
+ break;
+ }
+ if (src_array[src_index] < dest_array[dest_index]) {
+ *array_ptr = src_array[src_index];
+ src_index++;
+ } else {
+ assert(src_array[src_index] > dest_array[dest_index]);
+ *array_ptr = dest_array[dest_index];
+ dest_index++;
+ }
+ array_ptr++;
+ }
+
+ ext2fs_free_mem(&dest->list);
+ dest->list = array;
+ dest->num = src_count + dest_count;
+ dest->size = size;
+ return 0;
+}
+
/*
* This procedure creates an empty u32 list.
@@ -79,13 +139,7 @@ errcode_t ext2fs_badblocks_list_create(ext2_badblocks_list *ret, int size)
*/
errcode_t ext2fs_u32_copy(ext2_u32_list src, ext2_u32_list *dest)
{
- errcode_t retval;
-
- retval = make_u32_list(src->size, src->num, src->list, dest);
- if (retval)
- return retval;
- (*dest)->badblocks_flags = src->badblocks_flags;
- return 0;
+ return make_u32_list(src->size, src->num, src->list, dest);
}
errcode_t ext2fs_badblocks_copy(ext2_badblocks_list src,
@@ -95,6 +149,13 @@ errcode_t ext2fs_badblocks_copy(ext2_badblocks_list src,
(ext2_u32_list *) dest);
}
+errcode_t ext2fs_badblocks_merge(ext2_badblocks_list src,
+ ext2_badblocks_list dest)
+{
+ return merge_u32_list((ext2_u32_list) src,
+ (ext2_u32_list) dest);
+}
+
/*
* This procedure frees a badblocks list.
*
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index d5b1af2f..bdb72251 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -814,6 +814,8 @@ extern int ext2fs_badblocks_list_iterate(ext2_badblocks_iterate iter,
extern void ext2fs_badblocks_list_iterate_end(ext2_badblocks_iterate iter);
extern errcode_t ext2fs_badblocks_copy(ext2_badblocks_list src,
ext2_badblocks_list *dest);
+extern errcode_t ext2fs_badblocks_merge(ext2_badblocks_list src,
+ ext2_badblocks_list dest);
extern int ext2fs_badblocks_equal(ext2_badblocks_list bb1,
ext2_badblocks_list bb2);
extern int ext2fs_u32_list_count(ext2_u32_list bb);
diff --git a/lib/ext2fs/ext2fsP.h b/lib/ext2fs/ext2fsP.h
index ad8b7d52..02df759a 100644
--- a/lib/ext2fs/ext2fsP.h
+++ b/lib/ext2fs/ext2fsP.h
@@ -34,7 +34,6 @@ struct ext2_struct_u32_list {
int num;
int size;
__u32 *list;
- int badblocks_flags;
};
struct ext2_struct_u32_iterate {
--
2.25.4
From: Li Xi <[email protected]>
This patch only copy the fs to a new one when -m is enabled.
It doesn't actually start any thread. When pass1 test finishes,
the new fs is copied back to the original context.
This patch handles the fs fields in dblist, inode_map and block_map
properly.
Signed-off-by: Li Xi <[email protected]>
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/pass1.c | 41 +++++++++++++++++++++++++++++++++++++----
1 file changed, 37 insertions(+), 4 deletions(-)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index a9244201..c0df4330 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -49,6 +49,8 @@
#include "e2fsck.h"
#include <ext2fs/ext2_ext_attr.h>
+/* todo remove this finally */
+#include <ext2fs/ext2fsP.h>
#include <e2p/e2p.h>
#include "problem.h"
@@ -2084,10 +2086,23 @@ endit:
ctx->invalid_bitmaps++;
}
+static void e2fsck_pass1_copy_fs(ext2_filsys dest, ext2_filsys src)
+{
+ memcpy(dest, src, sizeof(struct struct_ext2_filsys));
+ if (dest->dblist)
+ dest->dblist->fs = dest;
+ if (dest->inode_map)
+ dest->inode_map->fs = dest;
+ if (dest->block_map)
+ dest->block_map->fs = dest;
+}
+
static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thread_ctx)
{
errcode_t retval;
e2fsck_t thread_context;
+ ext2_filsys thread_fs;
+ ext2_filsys global_fs = global_ctx->fs;
retval = ext2fs_get_mem(sizeof(struct e2fsck_struct), &thread_context);
if (retval) {
@@ -2095,18 +2110,32 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre
return retval;
}
memcpy(thread_context, global_ctx, sizeof(struct e2fsck_struct));
- thread_context->fs->priv_data = thread_context;
thread_context->global_ctx = global_ctx;
+ retval = ext2fs_get_mem(sizeof(struct struct_ext2_filsys), &thread_fs);
+ if (retval) {
+ com_err(global_ctx->program_name, retval, "while allocating memory");
+ goto out_context;
+ }
+
+ e2fsck_pass1_copy_fs(thread_fs, global_fs);
+ thread_fs->priv_data = thread_context;
+
+ thread_context->fs = thread_fs;
*thread_ctx = thread_context;
return 0;
+out_context:
+ ext2fs_free_mem(&thread_context);
+ return retval;
}
static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
{
- int flags = global_ctx->flags;
+ int flags = global_ctx->flags;
+ ext2_filsys thread_fs = thread_ctx->fs;
+ ext2_filsys global_fs = global_ctx->fs;
#ifdef HAVE_SETJMP_H
- jmp_buf old_jmp;
+ jmp_buf old_jmp;
memcpy(old_jmp, global_ctx->abort_loc, sizeof(jmp_buf));
#endif
@@ -2118,7 +2147,11 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
global_ctx->flags |= (flags & E2F_FLAG_SIGNAL_MASK) |
(global_ctx->flags & E2F_FLAG_SIGNAL_MASK);
- global_ctx->fs->priv_data = global_ctx;
+ e2fsck_pass1_copy_fs(global_fs, thread_fs);
+ global_fs->priv_data = global_ctx;
+ global_ctx->fs = global_fs;
+
+ ext2fs_free_mem(&thread_ctx->fs);
ext2fs_free_mem(&thread_ctx);
return 0;
}
--
2.25.4
From: Li Xi <[email protected]>
Signed-off-by: Li Xi <[email protected]>
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/pass1.c | 39 +++++++++++++++++++++++++++++++++++++++
1 file changed, 39 insertions(+)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index d2f4ba79..efab125d 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -2153,6 +2153,11 @@ do { \
} \
} while (0)
+#define PASS1_MERGE_CTX_COUNT(_dest, _src, _field) \
+do { \
+ _dest->_field = _field + _src->_field; \
+} while (0)
+
static errcode_t pass1_open_io_channel(ext2_filsys fs,
const char *io_options,
io_manager manager, int flags)
@@ -2487,6 +2492,23 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
ext2fs_block_bitmap inodes_to_rebuild = global_ctx->inodes_to_rebuild;
ext2_icount_t inode_count = global_ctx->inode_count;
ext2_icount_t inode_link_info = global_ctx->inode_link_info;
+ __u32 fs_directory_count = global_ctx->fs_directory_count;
+ __u32 fs_regular_count = global_ctx->fs_regular_count;
+ __u32 fs_blockdev_count = global_ctx->fs_blockdev_count;
+ __u32 fs_chardev_count = global_ctx->fs_chardev_count;
+ __u32 fs_links_count = global_ctx->fs_links_count;
+ __u32 fs_symlinks_count = global_ctx->fs_symlinks_count;
+ __u32 fs_fast_symlinks_count = global_ctx->fs_fast_symlinks_count;
+ __u32 fs_fifo_count = global_ctx->fs_fifo_count;
+ __u32 fs_total_count = global_ctx->fs_total_count;
+ __u32 fs_badblocks_count = global_ctx->fs_badblocks_count;
+ __u32 fs_sockets_count = global_ctx->fs_sockets_count;
+ __u32 fs_ind_count = global_ctx->fs_ind_count;
+ __u32 fs_dind_count = global_ctx->fs_dind_count;
+ __u32 fs_tind_count = global_ctx->fs_tind_count;
+ __u32 fs_fragmented = global_ctx->fs_fragmented;
+ __u32 fs_fragmented_dir = global_ctx->fs_fragmented_dir;
+ __u32 large_files = global_ctx->large_files;
#ifdef HAVE_SETJMP_H
jmp_buf old_jmp;
@@ -2512,6 +2534,23 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
e2fsck_pass1_merge_dir_info(global_ctx, thread_ctx);
global_ctx->inode_count = inode_count;
global_ctx->inode_link_info = inode_link_info;
+ PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_directory_count);
+ PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_regular_count);
+ PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_blockdev_count);
+ PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_chardev_count);
+ PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_links_count);
+ PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_symlinks_count);
+ PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_fast_symlinks_count);
+ PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_fifo_count);
+ PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_total_count);
+ PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_badblocks_count);
+ PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_sockets_count);
+ PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_ind_count);
+ PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_dind_count);
+ PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_tind_count);
+ PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_fragmented);
+ PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_fragmented_dir);
+ PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, large_files);
/* Keep the global singal flags*/
global_ctx->flags |= (flags & E2F_FLAG_SIGNAL_MASK) |
--
2.25.4
From: Li Xi <[email protected]>
Signed-off-by: Li Xi <[email protected]>
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/pass1.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index efab125d..68b7ae26 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -2273,6 +2273,7 @@ static int _e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
ext2fs_block_bitmap block_map;
ext2_badblocks_list badblocks;
ext2_dblist dblist;
+ int flags;
dest_io = dest->io;
dest_image_io = dest->image_io;
@@ -2280,6 +2281,7 @@ static int _e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
block_map = dest->block_map;
badblocks = dest->badblocks;
dblist = dest->dblist;
+ flags = dest->flags;
memcpy(dest, src, sizeof(struct struct_ext2_filsys));
dest->io = dest_io;
dest->image_io = dest_image_io;
@@ -2287,6 +2289,9 @@ static int _e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
dest->block_map = block_map;
dest->badblocks = badblocks;
dest->dblist = dblist;
+ dest->flags = src->flags | flags;
+ if (!(src->flags & EXT2_FLAG_VALID) || !(flags & EXT2_FLAG_VALID))
+ ext2fs_unmark_valid(dest);
/*
* PASS1_MERGE_FS_BITMAP might return directly from this function,
* so please do NOT leave any garbage behind after returning.
--
2.25.4
From: Li Xi <[email protected]>
icache of fs will be rebuilt when needed, so after copying
fs, icache can be inited to NULL.
Signed-off-by: Li Xi <[email protected]>
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/pass1.c | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index b212cdde..22597b12 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -2102,6 +2102,13 @@ static errcode_t e2fsck_pass1_copy_fs(ext2_filsys dest, ext2_filsys src)
dest->inode_map->fs = dest;
if (dest->block_map)
dest->block_map->fs = dest;
+
+ /* icache will be rebuilt if needed, so do not copy from @src */
+ if (src->icache) {
+ ext2fs_free_inode_cache(src->icache);
+ src->icache = NULL;
+ }
+ dest->icache = NULL;
return 0;
}
@@ -2116,6 +2123,13 @@ static void e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
dest->inode_map->fs = dest;
if (dest->block_map)
dest->block_map->fs = dest;
+
+ /* icache will be rebuilt if needed, so do not copy from @src */
+ if (src->icache) {
+ ext2fs_free_inode_cache(src->icache);
+ src->icache = NULL;
+ }
+ dest->icache = NULL;
}
static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thread_ctx)
--
2.25.4
From: Wang Shilong <[email protected]>
This will fix t_dangerous test failure with 2 threads.
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/pass1.c | 28 ++++++++++++++++++++++++++++
1 file changed, 28 insertions(+)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 868c8777..3e608f3b 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -2494,6 +2494,27 @@ static errcode_t e2fsck_pass1_merge_icounts(e2fsck_t global_ctx, e2fsck_t thread
return 0;
}
+static int e2fsck_pass1_merge_dirs_to_hash(e2fsck_t global_ctx, e2fsck_t thread_ctx)
+{
+ int retval = 0;
+
+ if (thread_ctx->dirs_to_hash) {
+ if (!global_ctx->dirs_to_hash)
+ retval = ext2fs_badblocks_copy(thread_ctx->dirs_to_hash,
+ &global_ctx->dirs_to_hash);
+ else
+ retval = ext2fs_badblocks_merge(thread_ctx->dirs_to_hash,
+ global_ctx->dirs_to_hash);
+
+ if (retval)
+ return retval;
+
+ ext2fs_badblocks_list_free(thread_ctx->dirs_to_hash);
+ thread_ctx->dirs_to_hash = 0;
+ }
+ return retval;
+}
+
static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx)
{
errcode_t retval;
@@ -2535,6 +2556,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
__u32 large_files = global_ctx->large_files;
int dx_dir_info_size = global_ctx->dx_dir_info_size;
int dx_dir_info_count = global_ctx->dx_dir_info_count;
+ ext2_u32_list dirs_to_hash = global_ctx->dirs_to_hash;
#ifdef HAVE_SETJMP_H
jmp_buf old_jmp;
@@ -2602,6 +2624,12 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
_("while merging icounts\n"));
return retval;
}
+ global_ctx->dirs_to_hash = dirs_to_hash;
+ retval = e2fsck_pass1_merge_dirs_to_hash(global_ctx, thread_ctx);
+ if (retval) {
+ com_err(global_ctx->program_name, 0, _("while merging dirs to hash\n"));
+ return retval;
+ }
/*
* PASS1_COPY_CTX_BITMAP might return directly from this function,
--
2.25.4
From: Wang Shilong <[email protected]>
e2fsck might restart after pass1, so we should keep
flags if possible, this patch try to fix f_illitable_flexbg failure
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/pass1.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 3e608f3b..182e1cd8 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -2604,9 +2604,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_fragmented_dir);
PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, large_files);
- /* Keep the global singal flags*/
- global_ctx->flags |= (flags & E2F_FLAG_SIGNAL_MASK) |
- (global_ctx->flags & E2F_FLAG_SIGNAL_MASK);
+ global_ctx->flags |= flags;
retval = e2fsck_pass1_merge_fs(global_fs, thread_fs);
if (retval) {
--
2.25.4
From: Wang Shilong <[email protected]>
Every threads calculate its own quota accounting,
merge them after threads finish.
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/pass1.c | 23 +++++++++++++++++++++++
lib/support/mkquota.c | 19 +++++++++++++++++++
lib/support/quotaio.h | 2 ++
3 files changed, 44 insertions(+)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 182e1cd8..645666cc 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -56,6 +56,7 @@
#include <e2p/e2p.h>
#include "problem.h"
+#include "support/dict.h"
#ifdef NO_INLINE_FUNCS
#define _INLINE_
@@ -2429,6 +2430,11 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx,
log_out(thread_context, _("Scan group range [%d, %d)\n"),
tinfo->et_group_start, tinfo->et_group_end);
thread_context->fs = thread_fs;
+ retval = quota_init_context(&thread_context->qctx, thread_fs, 0);
+ if (retval) {
+ com_err(global_ctx->program_name, retval, "while init quota context");
+ goto out_fs;
+ }
*thread_ctx = thread_context;
return 0;
out_fs:
@@ -2515,6 +2521,20 @@ static int e2fsck_pass1_merge_dirs_to_hash(e2fsck_t global_ctx, e2fsck_t thread_
return retval;
}
+static void e2fsck_pass1_merge_quota_ctx(e2fsck_t global_ctx, e2fsck_t thread_ctx)
+{
+ dict_t *dict;
+ enum quota_type qtype;
+
+ for (qtype = 0; qtype < MAXQUOTAS; qtype++) {
+ dict = thread_ctx->qctx->quota_dict[qtype];
+ if (dict)
+ quota_merge_and_update_usage(
+ global_ctx->qctx->quota_dict[qtype], dict);
+ }
+ quota_release_context(&thread_ctx->qctx);
+}
+
static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx)
{
errcode_t retval;
@@ -2557,6 +2577,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
int dx_dir_info_size = global_ctx->dx_dir_info_size;
int dx_dir_info_count = global_ctx->dx_dir_info_count;
ext2_u32_list dirs_to_hash = global_ctx->dirs_to_hash;
+ quota_ctx_t qctx = global_ctx->qctx;
#ifdef HAVE_SETJMP_H
jmp_buf old_jmp;
@@ -2628,6 +2649,8 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
com_err(global_ctx->program_name, 0, _("while merging dirs to hash\n"));
return retval;
}
+ global_ctx->qctx = qctx;
+ e2fsck_pass1_merge_quota_ctx(global_ctx, thread_ctx);
/*
* PASS1_COPY_CTX_BITMAP might return directly from this function,
diff --git a/lib/support/mkquota.c b/lib/support/mkquota.c
index 6f7ae6d6..745106b0 100644
--- a/lib/support/mkquota.c
+++ b/lib/support/mkquota.c
@@ -639,6 +639,25 @@ out:
return err;
}
+errcode_t quota_merge_and_update_usage(dict_t *dest, dict_t *src)
+{
+ dnode_t *n;
+ struct dquot *src_dq, *dest_dq;
+
+ for (n = dict_first(src); n; n = dict_next(src, n)) {
+ src_dq = dnode_get(n);
+ if (!src_dq)
+ continue;
+ dest_dq = get_dq(dest, src_dq->dq_id);
+ if (dest_dq == NULL)
+ return -ENOMEM;
+ dest_dq->dq_dqb.dqb_curspace += src_dq->dq_dqb.dqb_curspace;
+ dest_dq->dq_dqb.dqb_curinodes += src_dq->dq_dqb.dqb_curinodes;
+ }
+
+ return 0;
+}
+
/*
* Compares the measured quota in qctx->quota_dict with that in the quota inode
* on disk and updates the limits in qctx->quota_dict. 'usage_inconsistent' is
diff --git a/lib/support/quotaio.h b/lib/support/quotaio.h
index 60689700..6077268a 100644
--- a/lib/support/quotaio.h
+++ b/lib/support/quotaio.h
@@ -40,6 +40,7 @@
#include "ext2fs/ext2_fs.h"
#include "ext2fs/ext2fs.h"
#include "dqblk_v2.h"
+#include "support/dict.h"
typedef int64_t qsize_t; /* Type in which we store size limitations */
@@ -233,6 +234,7 @@ int quota_file_exists(ext2_filsys fs, enum quota_type qtype);
void quota_set_sb_inum(ext2_filsys fs, ext2_ino_t ino, enum quota_type qtype);
errcode_t quota_compare_and_update(quota_ctx_t qctx, enum quota_type qtype,
int *usage_inconsistent);
+errcode_t quota_merge_and_update_usage(dict_t *dest, dict_t *src);
int parse_quota_opts(const char *opts, int (*func)(char *));
/* parse_qtype.c */
--
2.25.4
From: Li Xi <[email protected]>
These debug codes are added to run the multiple pass1 check
thread one by one in order. If all the codes are correct,
fsck of multiple threads should have exactly the same outcome
with single thread.
Signed-off-by: Li Xi <[email protected]>
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/e2fsck.h | 14 ++++++++++++++
e2fsck/pass1.c | 31 ++++++++++++++++++++++++++++++-
2 files changed, 44 insertions(+), 1 deletion(-)
diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
index 24f164a7..ec5b0fbc 100644
--- a/e2fsck/e2fsck.h
+++ b/e2fsck/e2fsck.h
@@ -446,6 +446,17 @@ struct e2fsck_struct {
__u32 extent_depth_count[MAX_EXTENT_DEPTH_COUNT];
};
+#ifdef DEBUG_THREADS
+/*
+ * Enabling DEBUG_THREADS would cause the parall fsck threads run sequentially
+ */
+struct e2fsck_thread_debug {
+ pthread_mutex_t etd_mutex;
+ pthread_cond_t etd_cond;
+ int etd_finished_threads;
+};
+#endif
+
struct e2fsck_thread_info {
/* ID returned by pthread_create() */
pthread_t eti_thread_id;
@@ -455,6 +466,9 @@ struct e2fsck_thread_info {
int eti_started;
/* Context used for this thread */
e2fsck_t eti_thread_ctx;
+#ifdef DEBUG_THREADS
+ struct e2fsck_thread_debug *eti_debug;
+#endif
};
/* Data structures to evaluate whether an extent tree needs rebuilding. */
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 7accc76c..d2f4ba79 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -1200,7 +1200,8 @@ void _e2fsck_pass1(e2fsck_t ctx)
ctx->readahead_kb = e2fsck_guess_readahead(ctx->fs);
pass1_readahead(ctx, &ra_group, &ino_threshold);
- if (!(ctx->options & E2F_OPT_PREEN))
+ if (!(ctx->options & E2F_OPT_PREEN) &&
+ ((!ctx->global_ctx) || (ctx->thread_info.et_thread_index == 0)))
fix_problem(ctx, PR_1_PASS_HEADER, &pctx);
if (ext2fs_has_feature_dir_index(fs->super) &&
@@ -2619,6 +2620,17 @@ static void *e2fsck_pass1_thread(void *arg)
{
struct e2fsck_thread_info *info = arg;
e2fsck_t thread_ctx = info->eti_thread_ctx;
+#ifdef DEBUG_THREADS
+ struct e2fsck_thread_debug *thread_debug = info->eti_debug;
+#endif
+
+#ifdef DEBUG_THREADS
+ pthread_mutex_lock(&thread_debug->etd_mutex);
+ while (info->eti_thread_index > thread_debug->etd_finished_threads) {
+ pthread_cond_wait(&thread_debug->etd_cond, &thread_debug->etd_mutex);
+ }
+ pthread_mutex_unlock(&thread_debug->etd_mutex);
+#endif
#ifdef HAVE_SETJMP_H
/*
@@ -2643,6 +2655,14 @@ out:
thread_ctx->thread_info.et_group_start,
thread_ctx->thread_info.et_group_end,
thread_ctx->thread_info.et_inode_number);
+
+#ifdef DEBUG_THREADS
+ pthread_mutex_lock(&thread_debug->etd_mutex);
+ thread_debug->etd_finished_threads++;
+ pthread_cond_broadcast(&thread_debug->etd_cond);
+ pthread_mutex_unlock(&thread_debug->etd_mutex);
+#endif
+
return NULL;
}
@@ -2656,6 +2676,12 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo,
struct e2fsck_thread_info *tmp_pinfo;
int i;
e2fsck_t thread_ctx;
+#ifdef DEBUG_THREADS
+ struct e2fsck_thread_debug thread_debug =
+ {PTHREAD_MUTEX_INITIALIZER, PTHREAD_COND_INITIALIZER, 0};
+
+ thread_debug.etd_finished_threads = 0;
+#endif
retval = pthread_attr_init(&attr);
if (retval) {
@@ -2676,6 +2702,9 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo,
for (i = 0; i < num_threads; i++) {
tmp_pinfo = &infos[i];
tmp_pinfo->eti_thread_index = i;
+#ifdef DEBUG_THREADS
+ tmp_pinfo->eti_debug = &thread_debug;
+#endif
retval = e2fsck_pass1_thread_prepare(global_ctx, &thread_ctx,
i, num_threads);
if (retval) {
--
2.25.4
From: Li Xi <[email protected]>
Signed-off-by: Li Xi <[email protected]>
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/pass1.c | 18 +++++++++++++-----
lib/ext2fs/dblist.c | 36 ++++++++++++++++++++++++++++++++++++
lib/ext2fs/ext2fs.h | 1 +
3 files changed, 50 insertions(+), 5 deletions(-)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 3c04edfd..7accc76c 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -2266,18 +2266,21 @@ static int _e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
ext2fs_inode_bitmap inode_map;
ext2fs_block_bitmap block_map;
ext2_badblocks_list badblocks;
+ ext2_dblist dblist;
dest_io = dest->io;
dest_image_io = dest->image_io;
inode_map = dest->inode_map;
block_map = dest->block_map;
badblocks = dest->badblocks;
+ dblist = dest->dblist;
memcpy(dest, src, sizeof(struct struct_ext2_filsys));
dest->io = dest_io;
dest->image_io = dest_image_io;
dest->inode_map = inode_map;
dest->block_map = block_map;
dest->badblocks = badblocks;
+ dest->dblist = dblist;
/*
* PASS1_MERGE_FS_BITMAP might return directly from this function,
* so please do NOT leave any garbage behind after returning.
@@ -2286,11 +2289,16 @@ static int _e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
PASS1_MERGE_FS_BITMAP(dest, src, block_map);
if (src->dblist) {
- retval = ext2fs_copy_dblist(src->dblist, &dest->dblist);
- if (retval)
- return retval;
- /* The ext2fs_copy_dblist() uses the src->fs as the fs */
- dest->dblist->fs = dest;
+ if (dest->dblist) {
+ retval = ext2fs_merge_dblist(src->dblist, dest->dblist);
+ if (retval)
+ return retval;
+ } else {
+ /* The ext2fs_copy_dblist() uses the src->fs as the fs */
+ dest->dblist = src->dblist;
+ dest->dblist->fs = dest;
+ src->dblist = NULL;
+ }
}
if (src->badblocks) {
diff --git a/lib/ext2fs/dblist.c b/lib/ext2fs/dblist.c
index bbdb221d..046b1e68 100644
--- a/lib/ext2fs/dblist.c
+++ b/lib/ext2fs/dblist.c
@@ -119,6 +119,42 @@ errcode_t ext2fs_copy_dblist(ext2_dblist src, ext2_dblist *dest)
return 0;
}
+/*
+ * Merge a directory block list @src to @dest
+ */
+errcode_t ext2fs_merge_dblist(ext2_dblist src, ext2_dblist dest)
+{
+ int src_count = src->count;
+ int dest_count = dest->count;
+ int size = src_count + dest_count;
+ int size_entry = sizeof(struct ext2_db_entry2);
+ struct ext2_db_entry2 *array, *array2;
+ errcode_t retval;
+
+ if (src_count == 0)
+ return 0;
+
+ if (src->sorted || dest->sorted)
+ return EINVAL;
+
+ retval = ext2fs_get_array(size, size_entry, &array);
+ if (retval)
+ return retval;
+
+ array2 = array;
+
+ memcpy(array, src->list, src_count * size_entry);
+ array += src_count;
+ memcpy(array, dest->list, dest_count * size_entry);
+ ext2fs_free_mem(&dest->list);
+
+ dest->list = array2;
+ dest->count = src_count + dest_count;
+ dest->size = size;
+
+ return 0;
+}
+
/*
* Close a directory block list
*
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index 5a094da3..37460a31 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -1120,6 +1120,7 @@ extern errcode_t ext2fs_add_dir_block(ext2_dblist dblist, ext2_ino_t ino,
blk_t blk, int blockcnt);
extern errcode_t ext2fs_add_dir_block2(ext2_dblist dblist, ext2_ino_t ino,
blk64_t blk, e2_blkcnt_t blockcnt);
+extern errcode_t ext2fs_merge_dblist(ext2_dblist src, ext2_dblist dest);
extern void ext2fs_dblist_sort(ext2_dblist dblist,
EXT2_QSORT_TYPE (*sortfunc)(const void *,
const void *));
--
2.25.4
From: Li Xi <[email protected]>
-m option is added but no actual functionality is added. This
patch only adds the logic that when -m is specified, one of
-p/-y/-n options should be specified. And when -m is specified,
-C shouldn't be specified and the completion progress report won't
be triggered by sending SIGUSR1/SIGUSR2 signals. This simplifies
the implementation of multi-thread fsck in the future.
Completion progress support with multi-thread fsck will be added
back after multi-thread fsck implementation is finished. Right
now, disable it to simplify the implementation of multi-thread fsck.
Signed-off-by: Li Xi <[email protected]>
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/e2fsck.h | 1 +
e2fsck/unix.c | 30 ++++++++++++++++++++-----
tests/f_multithread/expect.1 | 23 +++++++++++++++++++
tests/f_multithread/expect.2 | 7 ++++++
tests/f_multithread/image.gz | 1 +
tests/f_multithread/name | 1 +
tests/f_multithread/script | 4 ++++
tests/f_multithread_completion/expect.1 | 2 ++
tests/f_multithread_completion/expect.2 | 23 +++++++++++++++++++
tests/f_multithread_completion/image.gz | 1 +
tests/f_multithread_completion/name | 1 +
tests/f_multithread_completion/script | 4 ++++
tests/f_multithread_no/expect.1 | 24 ++++++++++++++++++++
tests/f_multithread_no/expect.2 | 23 +++++++++++++++++++
tests/f_multithread_no/image.gz | 1 +
tests/f_multithread_no/name | 1 +
tests/f_multithread_no/script | 4 ++++
tests/f_multithread_preen/expect.1 | 11 +++++++++
tests/f_multithread_preen/expect.2 | 23 +++++++++++++++++++
tests/f_multithread_preen/image.gz | 1 +
tests/f_multithread_preen/name | 1 +
tests/f_multithread_preen/script | 4 ++++
tests/f_multithread_yes/expect.1 | 2 ++
tests/f_multithread_yes/expect.2 | 23 +++++++++++++++++++
tests/f_multithread_yes/image.gz | 1 +
tests/f_multithread_yes/name | 1 +
tests/f_multithread_yes/script | 4 ++++
27 files changed, 216 insertions(+), 6 deletions(-)
create mode 100644 tests/f_multithread/expect.1
create mode 100644 tests/f_multithread/expect.2
create mode 120000 tests/f_multithread/image.gz
create mode 100644 tests/f_multithread/name
create mode 100644 tests/f_multithread/script
create mode 100644 tests/f_multithread_completion/expect.1
create mode 100644 tests/f_multithread_completion/expect.2
create mode 120000 tests/f_multithread_completion/image.gz
create mode 100644 tests/f_multithread_completion/name
create mode 100644 tests/f_multithread_completion/script
create mode 100644 tests/f_multithread_no/expect.1
create mode 100644 tests/f_multithread_no/expect.2
create mode 120000 tests/f_multithread_no/image.gz
create mode 100644 tests/f_multithread_no/name
create mode 100644 tests/f_multithread_no/script
create mode 100644 tests/f_multithread_preen/expect.1
create mode 100644 tests/f_multithread_preen/expect.2
create mode 120000 tests/f_multithread_preen/image.gz
create mode 100644 tests/f_multithread_preen/name
create mode 100644 tests/f_multithread_preen/script
create mode 100644 tests/f_multithread_yes/expect.1
create mode 100644 tests/f_multithread_yes/expect.2
create mode 120000 tests/f_multithread_yes/image.gz
create mode 100644 tests/f_multithread_yes/name
create mode 100644 tests/f_multithread_yes/script
diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
index b9e2f06e..8b7e1276 100644
--- a/e2fsck/e2fsck.h
+++ b/e2fsck/e2fsck.h
@@ -177,6 +177,7 @@ struct resource_track {
#define E2F_OPT_ICOUNT_FULLMAP 0x20000 /* use an array for inode counts */
#define E2F_OPT_UNSHARE_BLOCKS 0x40000
#define E2F_OPT_CLEAR_UNINIT 0x80000 /* Hack to clear the uninit bit */
+#define E2F_OPT_MULTITHREAD 0x100000 /* Use multiple threads to speedup */
/*
* E2fsck flags
diff --git a/e2fsck/unix.c b/e2fsck/unix.c
index 1cb51672..0a027be6 100644
--- a/e2fsck/unix.c
+++ b/e2fsck/unix.c
@@ -75,13 +75,14 @@ int journal_enable_debug = -1;
static void usage(e2fsck_t ctx)
{
fprintf(stderr,
- _("Usage: %s [-panyrcdfktvDFV] [-b superblock] [-B blocksize]\n"
+ _("Usage: %s [-pamnyrcdfktvDFV] [-b superblock] [-B blocksize]\n"
"\t\t[-l|-L bad_blocks_file] [-C fd] [-j external_journal]\n"
"\t\t[-E extended-options] [-z undo_file] device\n"),
ctx->program_name);
fprintf(stderr, "%s", _("\nEmergency help:\n"
" -p Automatic repair (no questions)\n"
+ " -m multiple threads to speedup fsck\n"
" -n Make no changes to the filesystem\n"
" -y Assume \"yes\" to all questions\n"
" -c Check for bad blocks and add them to the badblock list\n"
@@ -847,7 +848,7 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)
phys_mem_kb = get_memory_size() / 1024;
ctx->readahead_kb = ~0ULL;
- while ((c = getopt(argc, argv, "panyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDkz:")) != EOF)
+ while ((c = getopt(argc, argv, "pamnyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDkz:")) != EOF)
switch (c) {
case 'C':
ctx->progress = e2fsck_update_progress;
@@ -888,6 +889,9 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)
}
ctx->options |= E2F_OPT_PREEN;
break;
+ case 'm':
+ ctx->options |= E2F_OPT_MULTITHREAD;
+ break;
case 'n':
if (ctx->options & (E2F_OPT_YES|E2F_OPT_PREEN))
goto conflict_opt;
@@ -1006,6 +1010,18 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)
_("The -n and -l/-L options are incompatible."));
fatal_error(ctx, 0);
}
+ if (ctx->options & E2F_OPT_MULTITHREAD) {
+ if ((ctx->options & (E2F_OPT_YES|E2F_OPT_NO|E2F_OPT_PREEN)) == 0) {
+ com_err(ctx->program_name, 0, "%s",
+ _("The -m option should be used together with one of -p/-y/-n options."));
+ fatal_error(ctx, 0);
+ }
+ if (ctx->progress) {
+ com_err(ctx->program_name, 0, "%s",
+ _("Only one of the options -C or -m may be specified."));
+ fatal_error(ctx, 0);
+ }
+ }
if (ctx->options & E2F_OPT_NO)
ctx->options |= E2F_OPT_READONLY;
@@ -1112,10 +1128,12 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)
#ifdef SA_RESTART
sa.sa_flags = SA_RESTART;
#endif
- sa.sa_handler = signal_progress_on;
- sigaction(SIGUSR1, &sa, 0);
- sa.sa_handler = signal_progress_off;
- sigaction(SIGUSR2, &sa, 0);
+ if ((ctx->options & E2F_OPT_MULTITHREAD) == 0) {
+ sa.sa_handler = signal_progress_on;
+ sigaction(SIGUSR1, &sa, 0);
+ sa.sa_handler = signal_progress_off;
+ sigaction(SIGUSR2, &sa, 0);
+ }
#endif
/* Update our PATH to include /sbin if we need to run badblocks */
diff --git a/tests/f_multithread/expect.1 b/tests/f_multithread/expect.1
new file mode 100644
index 00000000..e2b954d0
--- /dev/null
+++ b/tests/f_multithread/expect.1
@@ -0,0 +1,23 @@
+ext2fs_open2: Bad magic number in super-block
+../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+Free blocks count wrong for group #0 (7987, counted=7982).
+Fix? yes
+
+Free blocks count wrong (11602, counted=11597).
+Fix? yes
+
+Free inodes count wrong for group #0 (1493, counted=1488).
+Fix? yes
+
+Free inodes count wrong (2997, counted=2992).
+Fix? yes
+
+
+test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
+test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks
+Exit status is 1
diff --git a/tests/f_multithread/expect.2 b/tests/f_multithread/expect.2
new file mode 100644
index 00000000..a833aefc
--- /dev/null
+++ b/tests/f_multithread/expect.2
@@ -0,0 +1,7 @@
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks
+Exit status is 0
diff --git a/tests/f_multithread/image.gz b/tests/f_multithread/image.gz
new file mode 120000
index 00000000..0fd40018
--- /dev/null
+++ b/tests/f_multithread/image.gz
@@ -0,0 +1 @@
+../f_zero_super/image.gz
\ No newline at end of file
diff --git a/tests/f_multithread/name b/tests/f_multithread/name
new file mode 100644
index 00000000..df838ea6
--- /dev/null
+++ b/tests/f_multithread/name
@@ -0,0 +1 @@
+test "e2fsck -m" option
\ No newline at end of file
diff --git a/tests/f_multithread/script b/tests/f_multithread/script
new file mode 100644
index 00000000..0fe96cd0
--- /dev/null
+++ b/tests/f_multithread/script
@@ -0,0 +1,4 @@
+FSCK_OPT="-fy -m"
+SECOND_FSCK_OPT=-yf
+
+. $cmd_dir/run_e2fsck
diff --git a/tests/f_multithread_completion/expect.1 b/tests/f_multithread_completion/expect.1
new file mode 100644
index 00000000..61cac9bb
--- /dev/null
+++ b/tests/f_multithread_completion/expect.1
@@ -0,0 +1,2 @@
+../e2fsck/e2fsck: Only one of the options -C or -m may be specified.
+Exit status is 8
diff --git a/tests/f_multithread_completion/expect.2 b/tests/f_multithread_completion/expect.2
new file mode 100644
index 00000000..e2b954d0
--- /dev/null
+++ b/tests/f_multithread_completion/expect.2
@@ -0,0 +1,23 @@
+ext2fs_open2: Bad magic number in super-block
+../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+Free blocks count wrong for group #0 (7987, counted=7982).
+Fix? yes
+
+Free blocks count wrong (11602, counted=11597).
+Fix? yes
+
+Free inodes count wrong for group #0 (1493, counted=1488).
+Fix? yes
+
+Free inodes count wrong (2997, counted=2992).
+Fix? yes
+
+
+test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
+test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks
+Exit status is 1
diff --git a/tests/f_multithread_completion/image.gz b/tests/f_multithread_completion/image.gz
new file mode 120000
index 00000000..0fd40018
--- /dev/null
+++ b/tests/f_multithread_completion/image.gz
@@ -0,0 +1 @@
+../f_zero_super/image.gz
\ No newline at end of file
diff --git a/tests/f_multithread_completion/name b/tests/f_multithread_completion/name
new file mode 100644
index 00000000..a959045d
--- /dev/null
+++ b/tests/f_multithread_completion/name
@@ -0,0 +1 @@
+test "e2fsck -m" option conflicts with "-C"
\ No newline at end of file
diff --git a/tests/f_multithread_completion/script b/tests/f_multithread_completion/script
new file mode 100644
index 00000000..bf23cd61
--- /dev/null
+++ b/tests/f_multithread_completion/script
@@ -0,0 +1,4 @@
+FSCK_OPT="-fy -m -C 1"
+SECOND_FSCK_OPT=-yf
+
+. $cmd_dir/run_e2fsck
diff --git a/tests/f_multithread_no/expect.1 b/tests/f_multithread_no/expect.1
new file mode 100644
index 00000000..d14c4083
--- /dev/null
+++ b/tests/f_multithread_no/expect.1
@@ -0,0 +1,24 @@
+ext2fs_open2: Bad magic number in super-block
+../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+Free blocks count wrong for group #0 (7987, counted=7982).
+Fix? no
+
+Free blocks count wrong (11602, counted=11597).
+Fix? no
+
+Free inodes count wrong for group #0 (1493, counted=1488).
+Fix? no
+
+Free inodes count wrong (2997, counted=2992).
+Fix? no
+
+
+test_filesys: ********** WARNING: Filesystem still has errors **********
+
+test_filesys: 11/3008 files (0.0% non-contiguous), 398/12000 blocks
+Exit status is 4
diff --git a/tests/f_multithread_no/expect.2 b/tests/f_multithread_no/expect.2
new file mode 100644
index 00000000..e2b954d0
--- /dev/null
+++ b/tests/f_multithread_no/expect.2
@@ -0,0 +1,23 @@
+ext2fs_open2: Bad magic number in super-block
+../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+Free blocks count wrong for group #0 (7987, counted=7982).
+Fix? yes
+
+Free blocks count wrong (11602, counted=11597).
+Fix? yes
+
+Free inodes count wrong for group #0 (1493, counted=1488).
+Fix? yes
+
+Free inodes count wrong (2997, counted=2992).
+Fix? yes
+
+
+test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
+test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks
+Exit status is 1
diff --git a/tests/f_multithread_no/image.gz b/tests/f_multithread_no/image.gz
new file mode 120000
index 00000000..0fd40018
--- /dev/null
+++ b/tests/f_multithread_no/image.gz
@@ -0,0 +1 @@
+../f_zero_super/image.gz
\ No newline at end of file
diff --git a/tests/f_multithread_no/name b/tests/f_multithread_no/name
new file mode 100644
index 00000000..fa49692e
--- /dev/null
+++ b/tests/f_multithread_no/name
@@ -0,0 +1 @@
+test "e2fsck -m" option works with "-n"
\ No newline at end of file
diff --git a/tests/f_multithread_no/script b/tests/f_multithread_no/script
new file mode 100644
index 00000000..b93deb3a
--- /dev/null
+++ b/tests/f_multithread_no/script
@@ -0,0 +1,4 @@
+FSCK_OPT="-fn -m"
+SECOND_FSCK_OPT=-yf
+
+. $cmd_dir/run_e2fsck
diff --git a/tests/f_multithread_preen/expect.1 b/tests/f_multithread_preen/expect.1
new file mode 100644
index 00000000..b4b0cd9a
--- /dev/null
+++ b/tests/f_multithread_preen/expect.1
@@ -0,0 +1,11 @@
+../e2fsck/e2fsck: Bad magic number in super-block while trying to open test.img
+test_filesys:
+The superblock could not be read or does not describe a valid ext2/ext3/ext4
+filesystem. If the device is valid and it really contains an ext2/ext3/ext4
+filesystem (and not swap or ufs or something else), then the superblock
+is corrupt, and you might try running e2fsck with an alternate superblock:
+ e2fsck -b 8193 <device>
+ or
+ e2fsck -b 32768 <device>
+
+Exit status is 8
diff --git a/tests/f_multithread_preen/expect.2 b/tests/f_multithread_preen/expect.2
new file mode 100644
index 00000000..e2b954d0
--- /dev/null
+++ b/tests/f_multithread_preen/expect.2
@@ -0,0 +1,23 @@
+ext2fs_open2: Bad magic number in super-block
+../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+Free blocks count wrong for group #0 (7987, counted=7982).
+Fix? yes
+
+Free blocks count wrong (11602, counted=11597).
+Fix? yes
+
+Free inodes count wrong for group #0 (1493, counted=1488).
+Fix? yes
+
+Free inodes count wrong (2997, counted=2992).
+Fix? yes
+
+
+test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
+test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks
+Exit status is 1
diff --git a/tests/f_multithread_preen/image.gz b/tests/f_multithread_preen/image.gz
new file mode 120000
index 00000000..0fd40018
--- /dev/null
+++ b/tests/f_multithread_preen/image.gz
@@ -0,0 +1 @@
+../f_zero_super/image.gz
\ No newline at end of file
diff --git a/tests/f_multithread_preen/name b/tests/f_multithread_preen/name
new file mode 100644
index 00000000..90d199df
--- /dev/null
+++ b/tests/f_multithread_preen/name
@@ -0,0 +1 @@
+test "e2fsck -m" option works with "-p"
\ No newline at end of file
diff --git a/tests/f_multithread_preen/script b/tests/f_multithread_preen/script
new file mode 100644
index 00000000..ecb79cd6
--- /dev/null
+++ b/tests/f_multithread_preen/script
@@ -0,0 +1,4 @@
+FSCK_OPT="-fp -m"
+SECOND_FSCK_OPT=-yf
+
+. $cmd_dir/run_e2fsck
diff --git a/tests/f_multithread_yes/expect.1 b/tests/f_multithread_yes/expect.1
new file mode 100644
index 00000000..8b780ecf
--- /dev/null
+++ b/tests/f_multithread_yes/expect.1
@@ -0,0 +1,2 @@
+../e2fsck/e2fsck: The -m option should be used together with one of -p/-y/-n options.
+Exit status is 8
diff --git a/tests/f_multithread_yes/expect.2 b/tests/f_multithread_yes/expect.2
new file mode 100644
index 00000000..e2b954d0
--- /dev/null
+++ b/tests/f_multithread_yes/expect.2
@@ -0,0 +1,23 @@
+ext2fs_open2: Bad magic number in super-block
+../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+Free blocks count wrong for group #0 (7987, counted=7982).
+Fix? yes
+
+Free blocks count wrong (11602, counted=11597).
+Fix? yes
+
+Free inodes count wrong for group #0 (1493, counted=1488).
+Fix? yes
+
+Free inodes count wrong (2997, counted=2992).
+Fix? yes
+
+
+test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
+test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks
+Exit status is 1
diff --git a/tests/f_multithread_yes/image.gz b/tests/f_multithread_yes/image.gz
new file mode 120000
index 00000000..0fd40018
--- /dev/null
+++ b/tests/f_multithread_yes/image.gz
@@ -0,0 +1 @@
+../f_zero_super/image.gz
\ No newline at end of file
diff --git a/tests/f_multithread_yes/name b/tests/f_multithread_yes/name
new file mode 100644
index 00000000..3a703195
--- /dev/null
+++ b/tests/f_multithread_yes/name
@@ -0,0 +1 @@
+test "e2fsck -m" option works with "-y"
\ No newline at end of file
diff --git a/tests/f_multithread_yes/script b/tests/f_multithread_yes/script
new file mode 100644
index 00000000..38891f6a
--- /dev/null
+++ b/tests/f_multithread_yes/script
@@ -0,0 +1,4 @@
+FSCK_OPT="-f -m"
+SECOND_FSCK_OPT=-yf
+
+. $cmd_dir/run_e2fsck
--
2.25.4
From: Wang Shilong <[email protected]>
EA related variables are now shared by different
threads, but without any protections.
So this patch try to fix these by seralizing operations.
Optimizations could be done later, since EA blocks
could be shared, need be careful to split and merge.
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/e2fsck.h | 2 +
e2fsck/pass1.c | 184 ++++++++++++++++++++++++++++--------------------
2 files changed, 111 insertions(+), 75 deletions(-)
diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
index 2defab92..dcc5c2d6 100644
--- a/e2fsck/e2fsck.h
+++ b/e2fsck/e2fsck.h
@@ -448,6 +448,8 @@ struct e2fsck_struct {
pthread_mutex_t fs_fix_mutex;
/* protect block_found_map, block_dup_map */
pthread_mutex_t fs_block_map_mutex;
+ /* protect ea related structure */
+ pthread_mutex_t fs_ea_mutex;
};
#ifdef DEBUG_THREADS
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index e8c0618b..a73d35fd 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -161,6 +161,18 @@ static inline void e2fsck_pass1_block_map_unlock(e2fsck_t ctx)
pthread_mutex_unlock(&global_ctx->fs_block_map_mutex);
}
+static inline void e2fsck_pass1_ea_lock(e2fsck_t ctx)
+{
+ e2fsck_get_lock_context(ctx);
+ pthread_mutex_lock(&global_ctx->fs_ea_mutex);
+}
+
+static inline void e2fsck_pass1_ea_unlock(e2fsck_t ctx)
+{
+ e2fsck_get_lock_context(ctx);
+ pthread_mutex_unlock(&global_ctx->fs_ea_mutex);
+}
+
/*
* Check to make sure a device inode is real. Returns 1 if the device
* checks out, 0 if not.
@@ -429,15 +441,16 @@ static void inc_ea_inode_refs(e2fsck_t ctx, struct problem_context *pctx,
struct ext2_ext_attr_entry *first, void *end)
{
struct ext2_ext_attr_entry *entry;
+ e2fsck_t global_ctx = ctx->global_ctx ? ctx->global_ctx : ctx;
for (entry = first;
(void *)entry < end && !EXT2_EXT_IS_LAST_ENTRY(entry);
entry = EXT2_EXT_ATTR_NEXT(entry)) {
if (!entry->e_value_inum)
continue;
- if (!ctx->ea_inode_refs) {
+ if (!global_ctx->ea_inode_refs) {
pctx->errcode = ea_refcount_create(0,
- &ctx->ea_inode_refs);
+ &global_ctx->ea_inode_refs);
if (pctx->errcode) {
pctx->num = 4;
fix_problem(ctx, PR_1_ALLOCATE_REFCOUNT, pctx);
@@ -445,8 +458,8 @@ static void inc_ea_inode_refs(e2fsck_t ctx, struct problem_context *pctx,
return;
}
}
- ea_refcount_increment(ctx->ea_inode_refs, entry->e_value_inum,
- 0);
+ ea_refcount_increment(global_ctx->ea_inode_refs,
+ entry->e_value_inum, 0);
}
}
@@ -572,8 +585,10 @@ fix:
* EA(s) in automatic fashion -bzzz
*/
if (problem == 0 || !fix_problem(ctx, problem, pctx)) {
+ e2fsck_pass1_ea_lock(ctx);
inc_ea_inode_refs(ctx, pctx,
(struct ext2_ext_attr_entry *)start, end);
+ e2fsck_pass1_ea_unlock(ctx);
return;
}
@@ -1282,14 +1297,49 @@ static void _e2fsck_pass1_post(e2fsck_t ctx)
{
struct problem_context pctx;
ext2_filsys fs = ctx->fs;
- char *block_buf;
+ char *block_buf =
+ (char *)e2fsck_allocate_memory(ctx, ctx->fs->blocksize * 3,
+ "block interate buffer");
reserve_block_for_root_repair(ctx);
reserve_block_for_lnf_repair(ctx);
+ /*
+ * If any extended attribute blocks' reference counts need to
+ * be adjusted, either up (ctx->refcount_extra), or down
+ * (ctx->refcount), then fix them.
+ */
+ if (ctx->refcount) {
+ adjust_extattr_refcount(ctx, ctx->refcount, block_buf, -1);
+ ea_refcount_free(ctx->refcount);
+ ctx->refcount = 0;
+ }
+ if (ctx->refcount_extra) {
+ adjust_extattr_refcount(ctx, ctx->refcount_extra,
+ block_buf, +1);
+ ea_refcount_free(ctx->refcount_extra);
+ ctx->refcount_extra = 0;
+ }
+
+ if (ctx->ea_block_quota_blocks) {
+ ea_refcount_free(ctx->ea_block_quota_blocks);
+ ctx->ea_block_quota_blocks = 0;
+ }
+
+ if (ctx->ea_block_quota_inodes) {
+ ea_refcount_free(ctx->ea_block_quota_inodes);
+ ctx->ea_block_quota_inodes = 0;
+ }
+
if (ctx->invalid_bitmaps)
handle_fs_bad_blocks(ctx);
+ /* We don't need the block_ea_map any more */
+ if (ctx->block_ea_map) {
+ ext2fs_free_block_bitmap(ctx->block_ea_map);
+ ctx->block_ea_map = 0;
+ }
+
if (ctx->flags & E2F_FLAG_RESIZE_INODE) {
struct ext2_inode *inode;
int inode_size = EXT2_INODE_SIZE(fs->super);
@@ -1329,10 +1379,6 @@ static void _e2fsck_pass1_post(e2fsck_t ctx)
clear_problem_context(&pctx);
fix_problem(ctx, PR_1_DUP_BLOCKS_PREENSTOP, &pctx);
}
- block_buf =
- (char *)e2fsck_allocate_memory(ctx,
- ctx->fs->blocksize * 3,
- "block interate buffer");
e2fsck_pass1_dupblocks(ctx, block_buf);
ext2fs_free_mem(&block_buf);
}
@@ -2175,40 +2221,6 @@ void _e2fsck_pass1(e2fsck_t ctx)
ext2fs_close_inode_scan(scan);
scan = NULL;
- /*
- * If any extended attribute blocks' reference counts need to
- * be adjusted, either up (ctx->refcount_extra), or down
- * (ctx->refcount), then fix them.
- */
- if (ctx->refcount) {
- adjust_extattr_refcount(ctx, ctx->refcount, block_buf, -1);
- ea_refcount_free(ctx->refcount);
- ctx->refcount = 0;
- }
- if (ctx->refcount_extra) {
- adjust_extattr_refcount(ctx, ctx->refcount_extra,
- block_buf, +1);
- ea_refcount_free(ctx->refcount_extra);
- ctx->refcount_extra = 0;
- }
-
- if (ctx->ea_block_quota_blocks) {
- ea_refcount_free(ctx->ea_block_quota_blocks);
- ctx->ea_block_quota_blocks = 0;
- }
-
- if (ctx->ea_block_quota_inodes) {
- ea_refcount_free(ctx->ea_block_quota_inodes);
- ctx->ea_block_quota_inodes = 0;
- }
-
-
- /* We don't need the block_ea_map any more */
- if (ctx->block_ea_map) {
- ext2fs_free_block_bitmap(ctx->block_ea_map);
- ctx->block_ea_map = 0;
- }
-
/* We don't need the encryption policy => ID map any more */
destroy_encryption_policy_map(ctx);
@@ -2759,7 +2771,6 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
ext2fs_inode_bitmap inode_imagic_map = global_ctx->inode_imagic_map;
ext2fs_inode_bitmap inode_reg_map = global_ctx->inode_reg_map;
ext2fs_block_bitmap block_dup_map = global_ctx->block_dup_map;
- ext2fs_block_bitmap block_ea_map = global_ctx->block_ea_map;
ext2fs_block_bitmap inodes_to_rebuild = global_ctx->inodes_to_rebuild;
ext2_icount_t inode_count = global_ctx->inode_count;
ext2_icount_t inode_link_info = global_ctx->inode_link_info;
@@ -2788,6 +2799,12 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
int *invalid_inode_bitmap_flag = global_ctx->invalid_inode_bitmap_flag;
int *invalid_inode_table_flag = global_ctx->invalid_inode_table_flag;
int invalid_bitmaps = global_ctx->invalid_bitmaps;
+ ext2_refcount_t refcount = global_ctx->refcount;
+ ext2_refcount_t refcount_extra = global_ctx->refcount_extra;
+ ext2_refcount_t ea_block_quota_blocks = global_ctx->ea_block_quota_blocks;
+ ext2_refcount_t ea_block_quota_inodes = global_ctx->ea_block_quota_inodes;
+ ext2fs_block_bitmap block_ea_map = global_ctx->block_ea_map;
+ ext2_refcount_t ea_inode_refs = global_ctx->ea_inode_refs;
#ifdef HAVE_SETJMP_H
jmp_buf old_jmp;
@@ -2805,7 +2822,6 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
global_ctx->inode_imagic_map = inode_imagic_map;
global_ctx->inodes_to_rebuild = inodes_to_rebuild;
global_ctx->inode_reg_map = inode_reg_map;
- global_ctx->block_ea_map = block_ea_map;
global_ctx->block_dup_map = block_dup_map;
global_ctx->dir_info = dir_info;
e2fsck_pass1_merge_dir_info(global_ctx, thread_ctx);
@@ -2815,6 +2831,12 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
e2fsck_pass1_merge_dx_dir(global_ctx, thread_ctx);
global_ctx->inode_count = inode_count;
global_ctx->inode_link_info = inode_link_info;
+ global_ctx->refcount = refcount;
+ global_ctx->refcount_extra = refcount_extra;
+ global_ctx->ea_block_quota_blocks = ea_block_quota_blocks;
+ global_ctx->ea_block_quota_inodes = ea_block_quota_inodes;
+ global_ctx->block_ea_map = block_ea_map;
+ global_ctx->ea_inode_refs = ea_inode_refs;
PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_directory_count);
PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_regular_count);
PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_blockdev_count);
@@ -2875,7 +2897,6 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
PASS1_MERGE_CTX_BITMAP(global_ctx, thread_ctx, inode_imagic_map);
PASS1_MERGE_CTX_BITMAP(global_ctx, thread_ctx, inode_reg_map);
PASS1_MERGE_CTX_BITMAP(global_ctx, thread_ctx, inodes_to_rebuild);
- PASS1_MERGE_CTX_BITMAP(global_ctx, thread_ctx, block_ea_map);
return 0;
}
@@ -2898,7 +2919,6 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
PASS1_FREE_CTX_BITMAP(thread_ctx, inode_imagic_map);
PASS1_FREE_CTX_BITMAP(thread_ctx, inode_reg_map);
PASS1_FREE_CTX_BITMAP(thread_ctx, inodes_to_rebuild);
- PASS1_FREE_CTX_BITMAP(thread_ctx, block_ea_map);
ext2fs_free_icount(thread_ctx->inode_count);
ext2fs_free_icount(thread_ctx->inode_link_info);
e2fsck_free_dir_info(thread_ctx);
@@ -3105,6 +3125,7 @@ static void e2fsck_pass1_multithread(e2fsck_t global_ctx)
pthread_mutex_init(&global_ctx->fs_fix_mutex, NULL);
pthread_mutex_init(&global_ctx->fs_block_map_mutex, NULL);
+ pthread_mutex_init(&global_ctx->fs_ea_mutex, NULL);
if (ext2fs_has_feature_flex_bg(global_ctx->fs->super))
flexbg_size = 1 << global_ctx->fs->super->s_log_groups_per_flex;
@@ -3421,12 +3442,10 @@ static void adjust_extattr_refcount(e2fsck_t ctx, ext2_refcount_t refcount,
should_be = header->h_refcount + adjust_sign * (int)count;
pctx.num = should_be;
if (fix_problem(ctx, PR_1_EXTATTR_REFCOUNT, &pctx)) {
- e2fsck_pass1_fix_lock(ctx);
header->h_refcount = should_be;
pctx.errcode = ext2fs_write_ext_attr3(fs, blk,
block_buf,
pctx.ino);
- e2fsck_pass1_fix_unlock(ctx);
if (pctx.errcode) {
fix_problem(ctx, PR_1_EXTATTR_WRITE_ABORT,
&pctx);
@@ -3453,6 +3472,7 @@ static int check_ext_attr(e2fsck_t ctx, struct problem_context *pctx,
__u64 quota_inodes = 0;
region_t region = 0;
int failed_csum = 0;
+ e2fsck_t global_ctx = ctx->global_ctx ? ctx->global_ctx : ctx;
ea_block_quota->blocks = 0;
ea_block_quota->inodes = 0;
@@ -3476,26 +3496,30 @@ static int check_ext_attr(e2fsck_t ctx, struct problem_context *pctx,
}
/* If ea bitmap hasn't been allocated, create it */
- if (!ctx->block_ea_map) {
+ e2fsck_pass1_ea_lock(ctx);
+ if (!global_ctx->block_ea_map) {
pctx->errcode = e2fsck_allocate_block_bitmap(fs,
_("ext attr block map"),
EXT2FS_BMAP64_RBTREE, "block_ea_map",
- &ctx->block_ea_map);
+ &global_ctx->block_ea_map);
if (pctx->errcode) {
pctx->num = 2;
fix_problem(ctx, PR_1_ALLOCATE_BBITMAP_ERROR, pctx);
ctx->flags |= E2F_FLAG_ABORT;
+ e2fsck_pass1_ea_unlock(ctx);
return 0;
}
}
/* Create the EA refcount structure if necessary */
- if (!ctx->refcount) {
- pctx->errcode = ea_refcount_create(0, &ctx->refcount);
+ if (!global_ctx->refcount) {
+ pctx->errcode = ea_refcount_create(0,
+ &global_ctx->refcount);
if (pctx->errcode) {
pctx->num = 1;
fix_problem(ctx, PR_1_ALLOCATE_REFCOUNT, pctx);
ctx->flags |= E2F_FLAG_ABORT;
+ e2fsck_pass1_ea_unlock(ctx);
return 0;
}
}
@@ -3506,37 +3530,44 @@ static int check_ext_attr(e2fsck_t ctx, struct problem_context *pctx,
#endif
/* Have we seen this EA block before? */
- if (ext2fs_fast_test_block_bitmap2(ctx->block_ea_map, blk)) {
+ if (ext2fs_fast_test_block_bitmap2(global_ctx->block_ea_map,
+ blk)) {
ea_block_quota->blocks = EXT2FS_C2B(fs, 1);
ea_block_quota->inodes = 0;
- if (ctx->ea_block_quota_blocks) {
- ea_refcount_fetch(ctx->ea_block_quota_blocks, blk,
- "a_blocks);
+ if (global_ctx->ea_block_quota_blocks) {
+ ea_refcount_fetch(global_ctx->ea_block_quota_blocks,
+ blk, "a_blocks);
if (quota_blocks)
ea_block_quota->blocks = quota_blocks;
}
- if (ctx->ea_block_quota_inodes)
- ea_refcount_fetch(ctx->ea_block_quota_inodes, blk,
- &ea_block_quota->inodes);
+ if (global_ctx->ea_block_quota_inodes)
+ ea_refcount_fetch(global_ctx->ea_block_quota_inodes,
+ blk, &ea_block_quota->inodes);
- if (ea_refcount_decrement(ctx->refcount, blk, 0) == 0)
+ if (ea_refcount_decrement(global_ctx->refcount,
+ blk, 0) == 0) {
+ e2fsck_pass1_ea_unlock(ctx);
return 1;
+ }
/* Ooops, this EA was referenced more than it stated */
- if (!ctx->refcount_extra) {
+ if (!global_ctx->refcount_extra) {
pctx->errcode = ea_refcount_create(0,
- &ctx->refcount_extra);
+ &global_ctx->refcount_extra);
if (pctx->errcode) {
pctx->num = 2;
fix_problem(ctx, PR_1_ALLOCATE_REFCOUNT, pctx);
ctx->flags |= E2F_FLAG_ABORT;
+ e2fsck_pass1_ea_unlock(ctx);
return 0;
}
}
- ea_refcount_increment(ctx->refcount_extra, blk, 0);
+ ea_refcount_increment(global_ctx->refcount_extra, blk, 0);
+ e2fsck_pass1_ea_unlock(ctx);
return 1;
}
+ e2fsck_pass1_ea_unlock(ctx);
/*
* OK, we haven't seen this EA block yet. So we need to
@@ -3663,44 +3694,47 @@ static int check_ext_attr(e2fsck_t ctx, struct problem_context *pctx,
return 0;
}
+ e2fsck_pass1_ea_lock(ctx);
if (quota_blocks != EXT2FS_C2B(fs, 1U)) {
- if (!ctx->ea_block_quota_blocks) {
+ if (!global_ctx->ea_block_quota_blocks) {
pctx->errcode = ea_refcount_create(0,
- &ctx->ea_block_quota_blocks);
+ &global_ctx->ea_block_quota_blocks);
if (pctx->errcode) {
pctx->num = 3;
goto refcount_fail;
}
}
- ea_refcount_store(ctx->ea_block_quota_blocks, blk,
- quota_blocks);
+ ea_refcount_store(global_ctx->ea_block_quota_blocks,
+ blk, quota_blocks);
}
if (quota_inodes) {
- if (!ctx->ea_block_quota_inodes) {
+ if (!global_ctx->ea_block_quota_inodes) {
pctx->errcode = ea_refcount_create(0,
- &ctx->ea_block_quota_inodes);
+ &global_ctx->ea_block_quota_inodes);
if (pctx->errcode) {
pctx->num = 4;
refcount_fail:
fix_problem(ctx, PR_1_ALLOCATE_REFCOUNT, pctx);
ctx->flags |= E2F_FLAG_ABORT;
+ e2fsck_pass1_ea_unlock(ctx);
return 0;
}
}
- ea_refcount_store(ctx->ea_block_quota_inodes, blk,
- quota_inodes);
+ ea_refcount_store(global_ctx->ea_block_quota_inodes,
+ blk, quota_inodes);
}
ea_block_quota->blocks = quota_blocks;
ea_block_quota->inodes = quota_inodes;
- inc_ea_inode_refs(ctx, pctx, first, end);
- ea_refcount_store(ctx->refcount, blk, header->h_refcount - 1);
+ inc_ea_inode_refs(global_ctx, pctx, first, end);
+ ea_refcount_store(global_ctx->refcount, blk, header->h_refcount - 1);
e2fsck_pass1_block_map_lock(ctx);
mark_block_used(ctx, blk);
e2fsck_pass1_block_map_unlock(ctx);
- ext2fs_fast_mark_block_bitmap2(ctx->block_ea_map, blk);
+ ext2fs_fast_mark_block_bitmap2(global_ctx->block_ea_map, blk);
+ e2fsck_pass1_ea_unlock(ctx);
return 1;
clear_extattr:
--
2.25.4
From: Li Xi <[email protected]>
This patch copies badblocks when the copying fs.
Signed-off-by: Li Xi <[email protected]>
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/pass1.c | 55 ++++++++++++++++++++++++++++++++++++++++++--------
1 file changed, 47 insertions(+), 8 deletions(-)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index b836e666..a1feb166 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -2145,10 +2145,23 @@ static errcode_t e2fsck_pass1_copy_fs(ext2_filsys dest, ext2_filsys src)
src->dblist = NULL;
}
+ if (src->badblocks) {
+ retval = ext2fs_badblocks_copy(src->badblocks, &dest->badblocks);
+ if (retval)
+ goto out_dblist;
+
+ ext2fs_badblocks_list_free(src->badblocks);
+ src->badblocks = NULL;
+ }
return 0;
+
+out_dblist:
+ ext2fs_free_dblist(dest->dblist);
+ dest->dblist = NULL;
+ return retval;
}
-static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
+static int _e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
{
errcode_t retval = 0;
@@ -2160,6 +2173,32 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
PASS1_COPY_FS_BITMAP(dest, src, inode_map);
PASS1_COPY_FS_BITMAP(dest, src, block_map);
+ if (src->dblist) {
+ retval = ext2fs_copy_dblist(src->dblist, &dest->dblist);
+ if (retval)
+ return retval;
+ /* The ext2fs_copy_dblist() uses the src->fs as the fs */
+ dest->dblist->fs = dest;
+ }
+
+ if (src->badblocks) {
+ retval = ext2fs_badblocks_copy(src->badblocks, &dest->badblocks);
+ if (retval)
+ goto out_dblist;
+ }
+ return 0;
+out_dblist:
+ ext2fs_free_dblist(dest->dblist);
+ dest->dblist = NULL;
+ return retval;
+}
+
+static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
+{
+ errcode_t retval;
+
+ retval = _e2fsck_pass1_merge_fs(dest, src);
+
/* icache will be rebuilt if needed, so do not copy from @src */
if (src->icache) {
ext2fs_free_inode_cache(src->icache);
@@ -2167,16 +2206,16 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
}
dest->icache = NULL;
- if (dest->dblist) {
- retval = ext2fs_copy_dblist(src->dblist, &dest->dblist);
- if (retval == 0) {
- /* The ext2fs_copy_dblist() uses the src->fs as the fs */
- dest->dblist->fs = dest;
- }
-
+ if (src->dblist) {
ext2fs_free_dblist(src->dblist);
src->dblist = NULL;
}
+
+ if (src->badblocks) {
+ ext2fs_badblocks_list_free(src->badblocks);
+ src->badblocks = NULL;
+ }
+
return retval;
}
--
2.25.4
From: Li Xi <[email protected]>
When multi-threads are used for check, each thread needs to jump
to different group in pass1 check. This patch adds the group
jumping support. But still, only one thread is used to check.
Signed-off-by: Li Xi <[email protected]>
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/e2fsck.h | 18 +++++++++++++--
e2fsck/logfile.c | 5 +++--
e2fsck/pass1.c | 46 +++++++++++++++++++++++++++++++++------
e2fsck/problem.c | 5 +++++
e2fsck/problem.h | 3 +++
lib/ext2fs/ext2_err.et.in | 3 +++
6 files changed, 69 insertions(+), 11 deletions(-)
diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
index 93387bd6..896f5f39 100644
--- a/e2fsck/e2fsck.h
+++ b/e2fsck/e2fsck.h
@@ -227,6 +227,20 @@ typedef struct e2fsck_struct *e2fsck_t;
#define MAX_EXTENT_DEPTH_COUNT 5
+/*
+ * Fields that used for multi-thread
+ */
+struct e2fsck_thread {
+ /* The start group number for this thread */
+ dgrp_t et_group_start;
+ /* The end (not included) group number for this thread*/
+ dgrp_t et_group_end;
+ /* The next group number to check */
+ dgrp_t et_group_next;
+ /* Thread index */
+ int et_thread_index;
+};
+
struct e2fsck_struct {
/* ---- Following fields are never updated during the pass1 ---- */
/* Global context to get the cancel flag */
@@ -378,8 +392,8 @@ struct e2fsck_struct {
*/
ext2_ino_t stashed_ino;
struct ext2_inode *stashed_inode;
- /* Thread index, if global_ctx is null, this field is useless */
- int thread_index;
+ /* if @global_ctx is null, this field is useless */
+ struct e2fsck_thread thread_info;
/*
* Directory information
diff --git a/e2fsck/logfile.c b/e2fsck/logfile.c
index 17bfc86e..8b5de135 100644
--- a/e2fsck/logfile.c
+++ b/e2fsck/logfile.c
@@ -311,8 +311,9 @@ static FILE *set_up_log_file(e2fsck_t ctx, const char *key, const char *fn)
expand_logfn(ctx, log_fn, &s);
if (ctx->global_ctx) {
- assert(ctx->thread_index < 1000);
- sprintf(string_index, "%03d", ctx->thread_index);
+ assert(ctx->thread_info.et_thread_index < 1000);
+ sprintf(string_index, "%03d",
+ ctx->thread_info.et_thread_index);
append_string(&s, ".", 1);
append_string(&s, string_index, 0);
}
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 35806f29..45915513 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -1368,6 +1368,19 @@ void _e2fsck_pass1(e2fsck_t ctx)
/* Set up ctx->lost_and_found if possible */
(void) e2fsck_get_lost_and_found(ctx, 0);
+ if (ctx->global_ctx) {
+#if 0
+ printf("jumping to %d\n", ctx->thread_info.et_group_start);
+#endif
+ pctx.errcode = ext2fs_inode_scan_goto_blockgroup(scan,
+ ctx->thread_info.et_group_start);
+ if (pctx.errcode) {
+ fix_problem(ctx, PR_1_PASS_HEADER, &pctx);
+ ctx->flags |= E2F_FLAG_ABORT;
+ goto endit;
+ }
+ }
+
while (1) {
if (ino % (fs->super->s_inodes_per_group * 4) == 1) {
if (e2fsck_mmp_update(fs))
@@ -1411,6 +1424,8 @@ void _e2fsck_pass1(e2fsck_t ctx)
ext2fs_mark_inode_bitmap2(ctx->inode_used_map, ino);
continue;
}
+ if (pctx.errcode == EXT2_ET_SCAN_FINISHED)
+ break;
if (pctx.errcode &&
pctx.errcode != EXT2_ET_INODE_CSUM_INVALID &&
pctx.errcode != EXT2_ET_INODE_IS_GARBAGE) {
@@ -2284,12 +2299,15 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
return retval;
}
-static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thread_ctx)
+static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx,
+ e2fsck_t *thread_ctx,
+ int thread_index)
{
- errcode_t retval;
- e2fsck_t thread_context;
- ext2_filsys thread_fs;
- ext2_filsys global_fs = global_ctx->fs;
+ errcode_t retval;
+ e2fsck_t thread_context;
+ ext2_filsys thread_fs;
+ ext2_filsys global_fs = global_ctx->fs;
+ struct e2fsck_thread *tinfo;
assert(global_ctx->inode_used_map == NULL);
assert(global_ctx->inode_dir_map == NULL);
@@ -2325,9 +2343,15 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre
}
thread_fs->priv_data = thread_context;
- thread_context->thread_index = 0;
+ thread_context->thread_info.et_thread_index = thread_index;
set_up_logging(thread_context);
+ assert(thread_index == 0);
+ tinfo = &thread_context->thread_info;
+ tinfo->et_group_start = 0;
+ tinfo->et_group_next = 0;
+ tinfo->et_group_end = thread_fs->group_desc_count;
+
thread_context->fs = thread_fs;
*thread_ctx = thread_context;
return 0;
@@ -2487,7 +2511,7 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo,
for (i = 0; i < num_threads; i++) {
tmp_pinfo = &infos[i];
tmp_pinfo->eti_thread_index = i;
- retval = e2fsck_pass1_thread_prepare(global_ctx, &thread_ctx);
+ retval = e2fsck_pass1_thread_prepare(global_ctx, &thread_ctx, i);
if (retval) {
com_err(global_ctx->program_name, retval,
_("while preparing pass1 thread\n"));
@@ -2566,6 +2590,7 @@ static errcode_t scan_callback(ext2_filsys fs,
{
struct scan_callback_struct *scan_struct;
e2fsck_t ctx;
+ struct e2fsck_thread *tinfo;
scan_struct = (struct scan_callback_struct *) priv_data;
ctx = scan_struct->ctx;
@@ -2577,6 +2602,13 @@ static errcode_t scan_callback(ext2_filsys fs,
ctx->fs->group_desc_count))
return EXT2_ET_CANCEL_REQUESTED;
+ if (ctx->global_ctx) {
+ tinfo = &ctx->thread_info;
+ tinfo->et_group_next++;
+ if (tinfo->et_group_next >= tinfo->et_group_end)
+ return EXT2_ET_SCAN_FINISHED;
+ }
+
return 0;
}
diff --git a/e2fsck/problem.c b/e2fsck/problem.c
index e79c853b..22c2652c 100644
--- a/e2fsck/problem.c
+++ b/e2fsck/problem.c
@@ -1269,6 +1269,11 @@ static struct e2fsck_problem problem_table[] = {
N_("Encrypted @i %i has corrupt encryption @a.\n"),
PROMPT_CLEAR_INODE, 0, 0, 0, 0 },
+ /* Failed to goto block group */
+ { PR_1_SCAN_GOTO,
+ N_("failed to goto block group"),
+ PROMPT_NONE, PR_FATAL, 0, 0, 0 },
+
/* Pass 1b errors */
/* Pass 1B: Rescan for duplicate/bad blocks */
diff --git a/e2fsck/problem.h b/e2fsck/problem.h
index 4185e517..c2a1cbdf 100644
--- a/e2fsck/problem.h
+++ b/e2fsck/problem.h
@@ -707,6 +707,9 @@ struct problem_context {
/* Encrypted inode has corrupt encryption extended attribute */
#define PR_1_CORRUPT_ENCRYPTION_XATTR 0x01008B
+/* Failed to goto block group */
+#define PR_1_SCAN_GOTO 0x01008C
+
/*
* Pass 1b errors
*/
diff --git a/lib/ext2fs/ext2_err.et.in b/lib/ext2fs/ext2_err.et.in
index 0c76fee6..cdb37423 100644
--- a/lib/ext2fs/ext2_err.et.in
+++ b/lib/ext2fs/ext2_err.et.in
@@ -548,4 +548,7 @@ ec EXT2_ET_EA_INODE_CORRUPTED,
ec EXT2_ET_NO_GDESC,
"Group descriptors not loaded"
+ec EXT2_ET_SCAN_FINISHED,
+ "Scanning finished"
+
end
--
2.25.4
From: Wang Shilong <[email protected]>
Flex_bg might be enabled, if this is enabled it makes
more sense to split based on this.
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/pass1.c | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index c5107956..868c8777 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -2838,6 +2838,23 @@ static void e2fsck_pass1_multithread(e2fsck_t global_ctx)
struct e2fsck_thread_info *infos = NULL;
int num_threads = 1;
errcode_t retval;
+ unsigned flexbg_size = 1;
+ int max_threads;
+
+ if (ext2fs_has_feature_flex_bg(global_ctx->fs->super))
+ flexbg_size = 1 << global_ctx->fs->super->s_log_groups_per_flex;
+
+ max_threads = global_ctx->fs->group_desc_count / flexbg_size;
+ if (max_threads == 0)
+ num_threads = 1;
+ else if (max_threads % num_threads) {
+ int times = max_threads / num_threads;
+
+ if (times == 0)
+ num_threads = 1;
+ else
+ num_threads = max_threads / times;
+ }
init_ext2_max_sizes();
retval = e2fsck_pass1_threads_start(&infos, num_threads, global_ctx);
--
2.25.4
From: Li Xi <[email protected]>
Adding the assert would simplify the copying of context.
Signed-off-by: Li Xi <[email protected]>
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/pass1.c | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 22597b12..1ee6b5bc 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -46,6 +46,7 @@
#ifdef HAVE_ERRNO_H
#include <errno.h>
#endif
+#include <assert.h>
#include "e2fsck.h"
#include <ext2fs/ext2_ext_attr.h>
@@ -2139,6 +2140,18 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre
ext2_filsys thread_fs;
ext2_filsys global_fs = global_ctx->fs;
+ assert(global_ctx->inode_used_map == NULL);
+ assert(global_ctx->inode_dir_map == NULL);
+ assert(global_ctx->inode_bb_map == NULL);
+ assert(global_ctx->inode_imagic_map == NULL);
+ assert(global_ctx->inode_reg_map == NULL);
+ assert(global_ctx->inodes_to_rebuild == NULL);
+
+ assert(global_ctx->block_found_map == NULL);
+ assert(global_ctx->block_dup_map == NULL);
+ assert(global_ctx->block_ea_map == NULL);
+ assert(global_ctx->block_metadata_map == NULL);
+
retval = ext2fs_get_mem(sizeof(struct e2fsck_struct), &thread_context);
if (retval) {
com_err(global_ctx->program_name, retval, "while allocating memory");
--
2.25.4
From: Wang Shilong <[email protected]>
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/pass1.c | 71 ++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 71 insertions(+)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index f1bb1fc5..e8c0618b 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -2508,6 +2508,65 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
return retval;
}
+static void e2fsck_pass1_copy_invalid_bitmaps(e2fsck_t global_ctx,
+ e2fsck_t thread_ctx)
+{
+ int i, j;
+ int grp_start = thread_ctx->thread_info.et_group_start;
+ int grp_end = thread_ctx->thread_info.et_group_end;
+ int total = grp_end - grp_start;
+
+ thread_ctx->invalid_inode_bitmap_flag = e2fsck_allocate_memory(global_ctx,
+ sizeof(int) * total, "invalid_inode_bitmap");
+ thread_ctx->invalid_block_bitmap_flag = e2fsck_allocate_memory(global_ctx,
+ sizeof(int) * total, "invalid_block_bitmap");
+ thread_ctx->invalid_inode_table_flag = e2fsck_allocate_memory(global_ctx,
+ sizeof(int) * total, "invalid_inode_table");
+ thread_ctx->invalid_bitmaps = 0;
+
+ for (i = grp_start, j = 0; i < grp_end; i++, j++) {
+ thread_ctx->invalid_block_bitmap_flag[j] =
+ global_ctx->invalid_block_bitmap_flag[i];
+ thread_ctx->invalid_inode_bitmap_flag[j] =
+ global_ctx->invalid_inode_bitmap_flag[i];
+ thread_ctx->invalid_inode_table_flag[j] =
+ global_ctx->invalid_inode_table_flag[i];
+ if (thread_ctx->invalid_block_bitmap_flag[j])
+ thread_ctx->invalid_bitmaps++;
+ if (thread_ctx->invalid_inode_bitmap_flag[j])
+ thread_ctx->invalid_bitmaps++;
+ if (thread_ctx->invalid_inode_table_flag[j])
+ thread_ctx->invalid_bitmaps++;
+
+ }
+}
+
+static void e2fsck_pass1_merge_invalid_bitmaps(e2fsck_t global_ctx,
+ e2fsck_t thread_ctx)
+{
+ int i, j;
+ int grp_start = thread_ctx->thread_info.et_group_start;
+ int grp_end = thread_ctx->thread_info.et_group_end;
+
+ for (i = grp_start, j = 0; i < grp_end; i++, j++) {
+ global_ctx->invalid_block_bitmap_flag[i] =
+ thread_ctx->invalid_block_bitmap_flag[j];
+ global_ctx->invalid_inode_bitmap_flag[i] =
+ thread_ctx->invalid_inode_bitmap_flag[j];
+ global_ctx->invalid_inode_table_flag[i] =
+ thread_ctx->invalid_inode_table_flag[j];
+ if (thread_ctx->invalid_block_bitmap_flag[j])
+ global_ctx->invalid_bitmaps++;
+ if (thread_ctx->invalid_inode_bitmap_flag[j])
+ global_ctx->invalid_bitmaps++;
+ if (thread_ctx->invalid_inode_table_flag[j])
+ global_ctx->invalid_bitmaps++;
+ }
+ ext2fs_free_mem(&thread_ctx->invalid_block_bitmap_flag);
+ ext2fs_free_mem(&thread_ctx->invalid_inode_bitmap_flag);
+ ext2fs_free_mem(&thread_ctx->invalid_inode_table_flag);
+}
+
static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx,
e2fsck_t *thread_ctx,
int thread_index,
@@ -2584,6 +2643,7 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx,
goto out_fs;
}
*thread_ctx = thread_context;
+ e2fsck_pass1_copy_invalid_bitmaps(global_ctx, thread_context);
return 0;
out_fs:
ext2fs_free_mem(&thread_fs);
@@ -2724,6 +2784,10 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
int dx_dir_info_count = global_ctx->dx_dir_info_count;
ext2_u32_list dirs_to_hash = global_ctx->dirs_to_hash;
quota_ctx_t qctx = global_ctx->qctx;
+ int *invalid_block_bitmap_flag = global_ctx->invalid_block_bitmap_flag;
+ int *invalid_inode_bitmap_flag = global_ctx->invalid_inode_bitmap_flag;
+ int *invalid_inode_table_flag = global_ctx->invalid_inode_table_flag;
+ int invalid_bitmaps = global_ctx->invalid_bitmaps;
#ifdef HAVE_SETJMP_H
jmp_buf old_jmp;
@@ -2795,6 +2859,11 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
}
global_ctx->qctx = qctx;
e2fsck_pass1_merge_quota_ctx(global_ctx, thread_ctx);
+ global_ctx->invalid_block_bitmap_flag = invalid_block_bitmap_flag;
+ global_ctx->invalid_inode_bitmap_flag = invalid_inode_bitmap_flag;
+ global_ctx->invalid_inode_table_flag = invalid_inode_table_flag;
+ global_ctx->invalid_bitmaps = invalid_bitmaps;
+ e2fsck_pass1_merge_invalid_bitmaps(global_ctx, thread_ctx);
/*
* PASS1_COPY_CTX_BITMAP might return directly from this function,
@@ -2846,6 +2915,8 @@ static int e2fsck_pass1_threads_join(struct e2fsck_thread_info *infos,
int i;
struct e2fsck_thread_info *pinfo;
+ /* merge invalid bitmaps will recalculate it */
+ global_ctx->invalid_bitmaps = 0;
for (i = 0; i < num_threads; i++) {
pinfo = &infos[i];
--
2.25.4
From: Wang Shilong <[email protected]>
We could only use @found_map_block to find free blocks
after we have collectd all used blocks, so something like
handle_fs_bad_blocks(), ext2fs_create_resize_inode(),
e2fsck_pass1_dupblocks() really should be handled after
all threads has been finished.
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/e2fsck.h | 2 +
e2fsck/pass1.c | 317 +++++++++++++++++----------
tests/f_multithread/expect.1 | 2 +-
tests/f_multithread_logfile/expect.1 | 2 +-
tests/f_multithread_no/expect.1 | 2 +-
5 files changed, 202 insertions(+), 123 deletions(-)
diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
index 7dee2299..2defab92 100644
--- a/e2fsck/e2fsck.h
+++ b/e2fsck/e2fsck.h
@@ -446,6 +446,8 @@ struct e2fsck_struct {
__u32 extent_depth_count[MAX_EXTENT_DEPTH_COUNT];
/* serialize fix operation for multiple threads */
pthread_mutex_t fs_fix_mutex;
+ /* protect block_found_map, block_dup_map */
+ pthread_mutex_t fs_block_map_mutex;
};
#ifdef DEBUG_THREADS
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 87e96787..f1bb1fc5 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -132,24 +132,35 @@ static void process_inodes(e2fsck_t ctx, char *block_buf,
static __u64 ext2_max_sizes[EXT2_MAX_BLOCK_LOG_SIZE -
EXT2_MIN_BLOCK_LOG_SIZE + 1];
+#define e2fsck_get_lock_context(ctx) \
+ e2fsck_t global_ctx = ctx->global_ctx; \
+ if (!global_ctx) \
+ global_ctx = ctx; \
+
static void e2fsck_pass1_fix_lock(e2fsck_t ctx)
{
- e2fsck_t global_ctx = ctx->global_ctx;
- if (!global_ctx)
- global_ctx = ctx;
-
+ e2fsck_get_lock_context(ctx);
pthread_mutex_lock(&global_ctx->fs_fix_mutex);
}
static void e2fsck_pass1_fix_unlock(e2fsck_t ctx)
{
- e2fsck_t global_ctx = ctx->global_ctx;
- if (!global_ctx)
- global_ctx = ctx;
-
+ e2fsck_get_lock_context(ctx);
pthread_mutex_unlock(&global_ctx->fs_fix_mutex);
}
+static inline void e2fsck_pass1_block_map_lock(e2fsck_t ctx)
+{
+ e2fsck_get_lock_context(ctx);
+ pthread_mutex_lock(&global_ctx->fs_block_map_mutex);
+}
+
+static inline void e2fsck_pass1_block_map_unlock(e2fsck_t ctx)
+{
+ e2fsck_get_lock_context(ctx);
+ pthread_mutex_unlock(&global_ctx->fs_block_map_mutex);
+}
+
/*
* Check to make sure a device inode is real. Returns 1 if the device
* checks out, 0 if not.
@@ -779,11 +790,15 @@ static void check_is_really_dir(e2fsck_t ctx, struct problem_context *pctx,
if (i >= 4)
not_device++;
+ e2fsck_pass1_block_map_lock(ctx);
if (blk < ctx->fs->super->s_first_data_block ||
blk >= ext2fs_blocks_count(ctx->fs->super) ||
ext2fs_fast_test_block_bitmap2(ctx->block_found_map,
- blk))
+ blk)) {
+ e2fsck_pass1_block_map_unlock(ctx);
return; /* Invalid block, can't be dir */
+ }
+ e2fsck_pass1_block_map_unlock(ctx);
}
blk = inode->i_block[0];
}
@@ -919,19 +934,15 @@ static void reserve_block_for_root_repair(e2fsck_t ctx)
errcode_t err;
ext2_filsys fs = ctx->fs;
- e2fsck_pass1_fix_lock(ctx);
ctx->root_repair_block = 0;
if (ext2fs_test_inode_bitmap2(ctx->inode_used_map, EXT2_ROOT_INO))
- goto out;
+ return;
err = ext2fs_new_block2(fs, 0, ctx->block_found_map, &blk);
if (err)
- goto out;
+ return;
ext2fs_mark_block_bitmap2(ctx->block_found_map, blk);
ctx->root_repair_block = blk;
-out:
- e2fsck_pass1_fix_unlock(ctx);
- return;
}
static void reserve_block_for_lnf_repair(e2fsck_t ctx)
@@ -942,18 +953,15 @@ static void reserve_block_for_lnf_repair(e2fsck_t ctx)
static const char name[] = "lost+found";
ext2_ino_t ino;
- e2fsck_pass1_fix_lock(ctx);
ctx->lnf_repair_block = 0;
if (!ext2fs_lookup(fs, EXT2_ROOT_INO, name, sizeof(name)-1, 0, &ino))
- goto out;
+ return;
err = ext2fs_new_block2(fs, 0, ctx->block_found_map, &blk);
if (err)
- goto out;
+ return;
ext2fs_mark_block_bitmap2(ctx->block_found_map, blk);
ctx->lnf_repair_block = blk;
-out:
- e2fsck_pass1_fix_unlock(ctx);
return;
}
@@ -1219,6 +1227,118 @@ static int e2fsck_should_abort(e2fsck_t ctx)
return 0;
}
+/*
+ * We need call mark_table_blocks() before multiple
+ * thread start, since all known system blocks should be
+ * marked and checked later.
+ */
+static int _e2fsck_pass1_prepare(e2fsck_t ctx)
+{
+ struct problem_context pctx;
+ ext2_filsys fs = ctx->fs;
+
+ clear_problem_context(&pctx);
+ if (!(ctx->options & E2F_OPT_PREEN))
+ fix_problem(ctx, PR_1_PASS_HEADER, &pctx);
+
+ pctx.errcode = e2fsck_allocate_subcluster_bitmap(ctx->fs,
+ _("in-use block map"), EXT2FS_BMAP64_RBTREE,
+ "block_found_map", &ctx->block_found_map);
+ if (pctx.errcode) {
+ pctx.num = 1;
+ fix_problem(ctx, PR_1_ALLOCATE_BBITMAP_ERROR, &pctx);
+ ctx->flags |= E2F_FLAG_ABORT;
+ return pctx.errcode;
+ }
+ pctx.errcode = e2fsck_allocate_block_bitmap(ctx->fs,
+ _("metadata block map"), EXT2FS_BMAP64_RBTREE,
+ "block_metadata_map", &ctx->block_metadata_map);
+ if (pctx.errcode) {
+ pctx.num = 1;
+ fix_problem(ctx, PR_1_ALLOCATE_BBITMAP_ERROR, &pctx);
+ ctx->flags |= E2F_FLAG_ABORT;
+ return pctx.errcode;
+ }
+
+ mark_table_blocks(ctx);
+ pctx.errcode = ext2fs_convert_subcluster_bitmap(ctx->fs,
+ &ctx->block_found_map);
+ if (pctx.errcode) {
+ fix_problem(ctx, PR_1_CONVERT_SUBCLUSTER, &pctx);
+ ctx->flags |= E2F_FLAG_ABORT;
+ return pctx.errcode;
+ }
+
+ if (ext2fs_has_feature_mmp(fs->super) &&
+ fs->super->s_mmp_block > fs->super->s_first_data_block &&
+ fs->super->s_mmp_block < ext2fs_blocks_count(fs->super))
+ ext2fs_mark_block_bitmap2(ctx->block_found_map,
+ fs->super->s_mmp_block);
+
+ return 0;
+}
+
+static void _e2fsck_pass1_post(e2fsck_t ctx)
+{
+ struct problem_context pctx;
+ ext2_filsys fs = ctx->fs;
+ char *block_buf;
+
+ reserve_block_for_root_repair(ctx);
+ reserve_block_for_lnf_repair(ctx);
+
+ if (ctx->invalid_bitmaps)
+ handle_fs_bad_blocks(ctx);
+
+ if (ctx->flags & E2F_FLAG_RESIZE_INODE) {
+ struct ext2_inode *inode;
+ int inode_size = EXT2_INODE_SIZE(fs->super);
+ inode = e2fsck_allocate_memory(ctx, inode_size,
+ "scratch inode");
+
+ clear_problem_context(&pctx);
+ pctx.errcode = ext2fs_create_resize_inode(fs);
+ if (pctx.errcode) {
+ if (!fix_problem(ctx, PR_1_RESIZE_INODE_CREATE,
+ &pctx)) {
+ ctx->flags |= E2F_FLAG_ABORT;
+ ext2fs_free_mem(&inode);
+ ext2fs_free_mem(&block_buf);
+ return;
+ }
+ pctx.errcode = 0;
+ }
+ if (!pctx.errcode) {
+ e2fsck_read_inode(ctx, EXT2_RESIZE_INO, inode,
+ "recreate inode");
+ inode->i_mtime = ctx->now;
+ e2fsck_write_inode(ctx, EXT2_RESIZE_INO, inode,
+ "recreate inode");
+ }
+ ctx->flags &= ~E2F_FLAG_RESIZE_INODE;
+ ext2fs_free_mem(&inode);
+ }
+
+ if (ctx->flags & E2F_FLAG_RESTART) {
+ ext2fs_free_mem(&block_buf);
+ return;
+ }
+
+ if (ctx->block_dup_map) {
+ if (ctx->options & E2F_OPT_PREEN) {
+ clear_problem_context(&pctx);
+ fix_problem(ctx, PR_1_DUP_BLOCKS_PREENSTOP, &pctx);
+ }
+ block_buf =
+ (char *)e2fsck_allocate_memory(ctx,
+ ctx->fs->blocksize * 3,
+ "block interate buffer");
+ e2fsck_pass1_dupblocks(ctx, block_buf);
+ ext2fs_free_mem(&block_buf);
+ }
+}
+
+
void _e2fsck_pass1(e2fsck_t ctx)
{
int i;
@@ -1257,10 +1377,6 @@ void _e2fsck_pass1(e2fsck_t ctx)
ctx->readahead_kb = e2fsck_guess_readahead(ctx->fs);
pass1_readahead(ctx, &ra_group, &ino_threshold);
- if (!(ctx->options & E2F_OPT_PREEN) &&
- ((!ctx->global_ctx) || (ctx->thread_info.et_thread_index == 0)))
- fix_problem(ctx, PR_1_PASS_HEADER, &pctx);
-
if (ext2fs_has_feature_dir_index(fs->super) &&
!(ctx->options & E2F_OPT_NO)) {
if (ext2fs_u32_list_create(&ctx->dirs_to_hash, 50))
@@ -1308,24 +1424,6 @@ void _e2fsck_pass1(e2fsck_t ctx)
ctx->flags |= E2F_FLAG_ABORT;
return;
}
- pctx.errcode = e2fsck_allocate_subcluster_bitmap(fs,
- _("in-use block map"), EXT2FS_BMAP64_RBTREE,
- "block_found_map", &ctx->block_found_map);
- if (pctx.errcode) {
- pctx.num = 1;
- fix_problem(ctx, PR_1_ALLOCATE_BBITMAP_ERROR, &pctx);
- ctx->flags |= E2F_FLAG_ABORT;
- return;
- }
- pctx.errcode = e2fsck_allocate_block_bitmap(fs,
- _("metadata block map"), EXT2FS_BMAP64_RBTREE,
- "block_metadata_map", &ctx->block_metadata_map);
- if (pctx.errcode) {
- pctx.num = 1;
- fix_problem(ctx, PR_1_ALLOCATE_BBITMAP_ERROR, &pctx);
- ctx->flags |= E2F_FLAG_ABORT;
- return;
- }
pctx.errcode = e2fsck_setup_icount(ctx, "inode_link_info", 0, NULL,
&ctx->inode_link_info);
if (pctx.errcode) {
@@ -1367,14 +1465,6 @@ void _e2fsck_pass1(e2fsck_t ctx)
}
}
- mark_table_blocks(ctx);
- pctx.errcode = ext2fs_convert_subcluster_bitmap(fs,
- &ctx->block_found_map);
- if (pctx.errcode) {
- fix_problem(ctx, PR_1_CONVERT_SUBCLUSTER, &pctx);
- ctx->flags |= E2F_FLAG_ABORT;
- goto endit;
- }
block_buf = (char *) e2fsck_allocate_memory(ctx, fs->blocksize * 3,
"block interate buffer");
if (EXT2_INODE_SIZE(fs->super) == EXT2_GOOD_OLD_INODE_SIZE)
@@ -1408,12 +1498,6 @@ void _e2fsck_pass1(e2fsck_t ctx)
fs->super->s_mkfs_time < fs->super->s_inodes_count))
low_dtime_check = 0;
- if (ext2fs_has_feature_mmp(fs->super) &&
- fs->super->s_mmp_block > fs->super->s_first_data_block &&
- fs->super->s_mmp_block < ext2fs_blocks_count(fs->super))
- ext2fs_mark_block_bitmap2(ctx->block_found_map,
- fs->super->s_mmp_block);
-
/* Set up ctx->lost_and_found if possible */
(void) e2fsck_get_lost_and_found(ctx, 0);
@@ -1756,8 +1840,10 @@ void _e2fsck_pass1(e2fsck_t ctx)
failed_csum = 0;
}
+ e2fsck_pass1_block_map_lock(ctx);
pctx.errcode = ext2fs_copy_bitmap(ctx->block_found_map,
&pb.fs_meta_blocks);
+ e2fsck_pass1_block_map_unlock(ctx);
if (pctx.errcode) {
pctx.num = 4;
fix_problem(ctx, PR_1_ALLOCATE_BBITMAP_ERROR, &pctx);
@@ -2089,9 +2175,6 @@ void _e2fsck_pass1(e2fsck_t ctx)
ext2fs_close_inode_scan(scan);
scan = NULL;
- reserve_block_for_root_repair(ctx);
- reserve_block_for_lnf_repair(ctx);
-
/*
* If any extended attribute blocks' reference counts need to
* be adjusted, either up (ctx->refcount_extra), or down
@@ -2119,11 +2202,6 @@ void _e2fsck_pass1(e2fsck_t ctx)
ctx->ea_block_quota_inodes = 0;
}
- if (ctx->invalid_bitmaps) {
- e2fsck_pass1_fix_lock(ctx);
- handle_fs_bad_blocks(ctx);
- e2fsck_pass1_fix_unlock(ctx);
- }
/* We don't need the block_ea_map any more */
if (ctx->block_ea_map) {
@@ -2134,31 +2212,6 @@ void _e2fsck_pass1(e2fsck_t ctx)
/* We don't need the encryption policy => ID map any more */
destroy_encryption_policy_map(ctx);
- if (ctx->flags & E2F_FLAG_RESIZE_INODE) {
- clear_problem_context(&pctx);
- e2fsck_pass1_fix_lock(ctx);
- pctx.errcode = ext2fs_create_resize_inode(fs);
- e2fsck_pass1_fix_unlock(ctx);
- if (pctx.errcode) {
- if (!fix_problem(ctx, PR_1_RESIZE_INODE_CREATE,
- &pctx)) {
- ctx->flags |= E2F_FLAG_ABORT;
- goto endit;
- }
- pctx.errcode = 0;
- }
- if (!pctx.errcode) {
- e2fsck_read_inode(ctx, EXT2_RESIZE_INO, inode,
- "recreate inode");
- e2fsck_pass1_fix_lock(ctx);
- inode->i_mtime = ctx->now;
- e2fsck_write_inode(ctx, EXT2_RESIZE_INO, inode,
- "recreate inode");
- e2fsck_pass1_fix_unlock(ctx);
- }
- ctx->flags &= ~E2F_FLAG_RESIZE_INODE;
- }
-
if (ctx->flags & E2F_FLAG_RESTART) {
/*
* Only the master copy of the superblock and block
@@ -2170,15 +2223,6 @@ void _e2fsck_pass1(e2fsck_t ctx)
goto endit;
}
- if (ctx->block_dup_map) {
- if (ctx->options & E2F_OPT_PREEN) {
- clear_problem_context(&pctx);
- fix_problem(ctx, PR_1_DUP_BLOCKS_PREENSTOP, &pctx);
- }
- e2fsck_pass1_fix_lock(ctx);
- e2fsck_pass1_dupblocks(ctx, block_buf);
- e2fsck_pass1_fix_unlock(ctx);
- }
ctx->flags |= E2F_FLAG_ALLOC_OK;
endit:
e2fsck_use_inode_shortcuts(ctx, 0);
@@ -2483,10 +2527,10 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx,
assert(global_ctx->inode_reg_map == NULL);
assert(global_ctx->inodes_to_rebuild == NULL);
- assert(global_ctx->block_found_map == NULL);
assert(global_ctx->block_dup_map == NULL);
+ assert(global_ctx->block_found_map != NULL);
+ assert(global_ctx->block_metadata_map != NULL);
assert(global_ctx->block_ea_map == NULL);
- assert(global_ctx->block_metadata_map == NULL);
retval = ext2fs_get_mem(sizeof(struct e2fsck_struct), &thread_context);
if (retval) {
@@ -2654,10 +2698,8 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
ext2fs_inode_bitmap inode_bb_map = global_ctx->inode_bb_map;
ext2fs_inode_bitmap inode_imagic_map = global_ctx->inode_imagic_map;
ext2fs_inode_bitmap inode_reg_map = global_ctx->inode_reg_map;
- ext2fs_block_bitmap block_found_map = global_ctx->block_found_map;
ext2fs_block_bitmap block_dup_map = global_ctx->block_dup_map;
ext2fs_block_bitmap block_ea_map = global_ctx->block_ea_map;
- ext2fs_block_bitmap block_metadata_map = global_ctx->block_metadata_map;
ext2fs_block_bitmap inodes_to_rebuild = global_ctx->inodes_to_rebuild;
ext2_icount_t inode_count = global_ctx->inode_count;
ext2_icount_t inode_link_info = global_ctx->inode_link_info;
@@ -2699,10 +2741,8 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
global_ctx->inode_imagic_map = inode_imagic_map;
global_ctx->inodes_to_rebuild = inodes_to_rebuild;
global_ctx->inode_reg_map = inode_reg_map;
- global_ctx->block_found_map = block_found_map;
- global_ctx->block_dup_map = block_dup_map;
global_ctx->block_ea_map = block_ea_map;
- global_ctx->block_metadata_map = block_metadata_map;
+ global_ctx->block_dup_map = block_dup_map;
global_ctx->dir_info = dir_info;
e2fsck_pass1_merge_dir_info(global_ctx, thread_ctx);
global_ctx->dx_dir_info = dx_dir_info;
@@ -2766,10 +2806,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
PASS1_MERGE_CTX_BITMAP(global_ctx, thread_ctx, inode_imagic_map);
PASS1_MERGE_CTX_BITMAP(global_ctx, thread_ctx, inode_reg_map);
PASS1_MERGE_CTX_BITMAP(global_ctx, thread_ctx, inodes_to_rebuild);
- PASS1_MERGE_CTX_BITMAP(global_ctx, thread_ctx, block_found_map);
- PASS1_MERGE_CTX_BITMAP(global_ctx, thread_ctx, block_dup_map);
PASS1_MERGE_CTX_BITMAP(global_ctx, thread_ctx, block_ea_map);
- PASS1_MERGE_CTX_BITMAP(global_ctx, thread_ctx, block_metadata_map);
return 0;
}
@@ -2792,10 +2829,7 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
PASS1_FREE_CTX_BITMAP(thread_ctx, inode_imagic_map);
PASS1_FREE_CTX_BITMAP(thread_ctx, inode_reg_map);
PASS1_FREE_CTX_BITMAP(thread_ctx, inodes_to_rebuild);
- PASS1_FREE_CTX_BITMAP(thread_ctx, block_found_map);
- PASS1_FREE_CTX_BITMAP(thread_ctx, block_dup_map);
PASS1_FREE_CTX_BITMAP(thread_ctx, block_ea_map);
- PASS1_FREE_CTX_BITMAP(thread_ctx, block_metadata_map);
ext2fs_free_icount(thread_ctx->inode_count);
ext2fs_free_icount(thread_ctx->inode_link_info);
e2fsck_free_dir_info(thread_ctx);
@@ -2994,7 +3028,12 @@ static void e2fsck_pass1_multithread(e2fsck_t global_ctx)
unsigned flexbg_size = 1;
int max_threads;
+ retval = _e2fsck_pass1_prepare(global_ctx);
+ if (retval)
+ goto out_abort;
+
pthread_mutex_init(&global_ctx->fs_fix_mutex, NULL);
+ pthread_mutex_init(&global_ctx->fs_block_map_mutex, NULL);
if (ext2fs_has_feature_flex_bg(global_ctx->fs->super))
flexbg_size = 1 << global_ctx->fs->super->s_log_groups_per_flex;
@@ -3033,6 +3072,7 @@ out_abort:
void e2fsck_pass1(e2fsck_t ctx)
{
e2fsck_pass1_multithread(ctx);
+ _e2fsck_pass1_post(ctx);
}
#undef FINISH_INODE_LOOP
@@ -3218,7 +3258,12 @@ static void alloc_imagic_map(e2fsck_t ctx)
*/
static _INLINE_ void mark_block_used(e2fsck_t ctx, blk64_t block)
{
- struct problem_context pctx;
+ struct problem_context pctx;
+ e2fsck_t global_ctx;
+
+ global_ctx = ctx->global_ctx;
+ if (!global_ctx)
+ global_ctx = ctx;
clear_problem_context(&pctx);
@@ -3227,11 +3272,15 @@ static _INLINE_ void mark_block_used(e2fsck_t ctx, blk64_t block)
!(ctx->options & E2F_OPT_UNSHARE_BLOCKS)) {
return;
}
- if (!ctx->block_dup_map) {
+ /**
+ * this should be safe because this operation has
+ * been serialized by mutex.
+ */
+ if (!global_ctx->block_dup_map) {
pctx.errcode = e2fsck_allocate_block_bitmap(ctx->fs,
_("multiply claimed block map"),
EXT2FS_BMAP64_RBTREE, "block_dup_map",
- &ctx->block_dup_map);
+ &global_ctx->block_dup_map);
if (pctx.errcode) {
pctx.num = 3;
fix_problem(ctx, PR_1_ALLOCATE_BBITMAP_ERROR,
@@ -3241,7 +3290,7 @@ static _INLINE_ void mark_block_used(e2fsck_t ctx, blk64_t block)
return;
}
}
- ext2fs_fast_mark_block_bitmap2(ctx->block_dup_map, block);
+ ext2fs_fast_mark_block_bitmap2(global_ctx->block_dup_map, block);
} else {
ext2fs_fast_mark_block_bitmap2(ctx->block_found_map, block);
}
@@ -3254,14 +3303,16 @@ static _INLINE_ void mark_block_used(e2fsck_t ctx, blk64_t block)
static _INLINE_ void mark_blocks_used(e2fsck_t ctx, blk64_t block,
unsigned int num)
{
- if (ext2fs_test_block_bitmap_range2(ctx->block_found_map, block, num))
+ e2fsck_pass1_block_map_lock(ctx);
+ if (ext2fs_test_block_bitmap_range2(ctx->block_found_map, block, num)) {
ext2fs_mark_block_bitmap_range2(ctx->block_found_map, block, num);
- else {
+ } else {
unsigned int i;
for (i = 0; i < num; i += EXT2FS_CLUSTER_RATIO(ctx->fs))
mark_block_used(ctx, block + i);
}
+ e2fsck_pass1_block_map_unlock(ctx);
}
/*
@@ -3575,7 +3626,9 @@ refcount_fail:
inc_ea_inode_refs(ctx, pctx, first, end);
ea_refcount_store(ctx->refcount, blk, header->h_refcount - 1);
+ e2fsck_pass1_block_map_lock(ctx);
mark_block_used(ctx, blk);
+ e2fsck_pass1_block_map_unlock(ctx);
ext2fs_fast_mark_block_bitmap2(ctx->block_ea_map, blk);
return 1;
@@ -3960,7 +4013,9 @@ report_problem:
pctx->str = "EXT2_EXTENT_UP";
return;
}
+ e2fsck_pass1_block_map_lock(ctx);
mark_block_used(ctx, blk);
+ e2fsck_pass1_block_map_unlock(ctx);
pb->num_blocks++;
goto next;
}
@@ -4067,6 +4122,7 @@ alloc_later:
pb->last_block,
extent.e_pblk,
extent.e_lblk)) {
+ e2fsck_pass1_block_map_lock(ctx);
for (i = 0; i < extent.e_len; i++) {
pctx->blk = extent.e_lblk + i;
pctx->blk2 = extent.e_pblk + i;
@@ -4074,6 +4130,7 @@ alloc_later:
mark_block_used(ctx, extent.e_pblk + i);
mark_block_used(ctx, extent.e_pblk + i);
}
+ e2fsck_pass1_block_map_unlock(ctx);
}
/*
@@ -4716,6 +4773,7 @@ static int process_block(ext2_filsys fs,
*block_nr = 0;
return 0;
}
+
if (!p->suppress && (p->num_illegal_blocks % 12) == 0) {
if (fix_problem(ctx, PR_1_TOO_MANY_BAD_BLOCKS, pctx)) {
p->clear = 1;
@@ -4757,8 +4815,11 @@ static int process_block(ext2_filsys fs,
* being in use; all of the other blocks are handled
* by mark_table_blocks()).
*/
- if (blockcnt == BLOCK_COUNT_DIND)
+ if (blockcnt == BLOCK_COUNT_DIND) {
+ e2fsck_pass1_block_map_lock(ctx);
mark_block_used(ctx, blk);
+ e2fsck_pass1_block_map_unlock(ctx);
+ }
p->num_blocks++;
} else if (!(ctx->fs->cluster_ratio_bits &&
p->previous_block &&
@@ -4766,15 +4827,19 @@ static int process_block(ext2_filsys fs,
EXT2FS_B2C(ctx->fs, p->previous_block)) &&
(blk & EXT2FS_CLUSTER_MASK(ctx->fs)) ==
((unsigned) blockcnt & EXT2FS_CLUSTER_MASK(ctx->fs)))) {
+ e2fsck_pass1_block_map_lock(ctx);
mark_block_used(ctx, blk);
+ e2fsck_pass1_block_map_unlock(ctx);
p->num_blocks++;
} else if (has_unaligned_cluster_map(ctx, p->previous_block,
p->last_block, blk, blockcnt)) {
pctx->blk = blockcnt;
pctx->blk2 = blk;
fix_problem(ctx, PR_1_MISALIGNED_CLUSTER, pctx);
+ e2fsck_pass1_block_map_lock(ctx);
mark_block_used(ctx, blk);
mark_block_used(ctx, blk);
+ e2fsck_pass1_block_map_unlock(ctx);
}
if (blockcnt >= 0)
p->last_block = blockcnt;
@@ -4841,10 +4906,12 @@ static int process_bad_block(ext2_filsys fs,
}
if (blockcnt < 0) {
+ e2fsck_pass1_block_map_lock(ctx);
if (ext2fs_test_block_bitmap2(p->fs_meta_blocks, blk)) {
p->bbcheck = 1;
if (fix_problem(ctx, PR_1_BB_FS_BLOCK, pctx)) {
*block_nr = 0;
+ e2fsck_pass1_block_map_unlock(ctx);
return BLOCK_CHANGED;
}
} else if (ext2fs_test_block_bitmap2(ctx->block_found_map,
@@ -4853,12 +4920,17 @@ static int process_bad_block(ext2_filsys fs,
if (fix_problem(ctx, PR_1_BBINODE_BAD_METABLOCK,
pctx)) {
*block_nr = 0;
+ e2fsck_pass1_block_map_unlock(ctx);
return BLOCK_CHANGED;
}
- if (e2fsck_should_abort(ctx))
+ if (e2fsck_should_abort(ctx)) {
+ e2fsck_pass1_block_map_unlock(ctx);
return BLOCK_ABORT;
- } else
+ }
+ } else {
mark_block_used(ctx, blk);
+ }
+ e2fsck_pass1_block_map_unlock(ctx);
return 0;
}
#if 0
@@ -4871,10 +4943,13 @@ static int process_bad_block(ext2_filsys fs,
* there's an overlap between the filesystem table blocks
* (bitmaps and inode table) and the bad block list.
*/
+ e2fsck_pass1_block_map_lock(ctx);
if (!ext2fs_test_block_bitmap2(ctx->block_found_map, blk)) {
ext2fs_mark_block_bitmap2(ctx->block_found_map, blk);
+ e2fsck_pass1_block_map_unlock(ctx);
return 0;
}
+ e2fsck_pass1_block_map_unlock(ctx);
/*
* Try to find the where the filesystem block was used...
*/
@@ -5029,6 +5104,7 @@ static void new_table_block(e2fsck_t ctx, blk64_t first_block, dgrp_t group,
fix_problem(ctx, (old_block ? PR_1_RELOC_FROM_TO :
PR_1_RELOC_TO), &pctx);
pctx.blk2 = 0;
+ e2fsck_pass1_block_map_lock(ctx);
for (i = 0; i < num; i++) {
pctx.blk = i;
ext2fs_mark_block_bitmap2(ctx->block_found_map, (*new_block)+i);
@@ -5049,6 +5125,7 @@ static void new_table_block(e2fsck_t ctx, blk64_t first_block, dgrp_t group,
if (pctx.errcode)
fix_problem(ctx, PR_1_RELOC_WRITE_ERR, &pctx);
}
+ e2fsck_pass1_block_map_unlock(ctx);
ext2fs_free_mem(&buf);
}
diff --git a/tests/f_multithread/expect.1 b/tests/f_multithread/expect.1
index 8d2acd2b..4db68d9e 100644
--- a/tests/f_multithread/expect.1
+++ b/tests/f_multithread/expect.1
@@ -1,7 +1,7 @@
ext2fs_open2: Bad magic number in super-block
../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
+Pass 1: Checking inodes, blocks, and sizes
[Thread 0] Scan group range [0, 2)
-[Thread 0] Pass 1: Checking inodes, blocks, and sizes
[Thread 0] Scanned group range [0, 2), inodes 3008
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
diff --git a/tests/f_multithread_logfile/expect.1 b/tests/f_multithread_logfile/expect.1
index 8d2acd2b..4db68d9e 100644
--- a/tests/f_multithread_logfile/expect.1
+++ b/tests/f_multithread_logfile/expect.1
@@ -1,7 +1,7 @@
ext2fs_open2: Bad magic number in super-block
../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
+Pass 1: Checking inodes, blocks, and sizes
[Thread 0] Scan group range [0, 2)
-[Thread 0] Pass 1: Checking inodes, blocks, and sizes
[Thread 0] Scanned group range [0, 2), inodes 3008
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
diff --git a/tests/f_multithread_no/expect.1 b/tests/f_multithread_no/expect.1
index f85a3382..eda2fcac 100644
--- a/tests/f_multithread_no/expect.1
+++ b/tests/f_multithread_no/expect.1
@@ -1,7 +1,7 @@
ext2fs_open2: Bad magic number in super-block
../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
+Pass 1: Checking inodes, blocks, and sizes
[Thread 0] Scan group range [0, 2)
-[Thread 0] Pass 1: Checking inodes, blocks, and sizes
[Thread 0] Scanned group range [0, 2), inodes 3008
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
--
2.25.4
From: Wang Shilong <[email protected]>
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/e2fsck.h | 1 +
e2fsck/pass1.c | 8 ++++++--
e2fsck/unix.c | 3 ++-
tests/f_multithread/script | 2 +-
tests/f_multithread_completion/script | 2 +-
tests/f_multithread_logfile/script | 2 +-
tests/f_multithread_no/script | 2 +-
tests/f_multithread_preen/script | 2 +-
tests/f_multithread_yes/script | 2 +-
9 files changed, 15 insertions(+), 9 deletions(-)
diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
index dcc5c2d6..cd1bab07 100644
--- a/e2fsck/e2fsck.h
+++ b/e2fsck/e2fsck.h
@@ -444,6 +444,7 @@ struct e2fsck_struct {
__u32 fs_fragmented_dir;
__u32 large_files;
__u32 extent_depth_count[MAX_EXTENT_DEPTH_COUNT];
+ __u32 fs_num_threads;
/* serialize fix operation for multiple threads */
pthread_mutex_t fs_fix_mutex;
/* protect block_found_map, block_dup_map */
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index a73d35fd..40fe6b36 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -3114,11 +3114,14 @@ static void init_ext2_max_sizes()
static void e2fsck_pass1_multithread(e2fsck_t global_ctx)
{
struct e2fsck_thread_info *infos = NULL;
- int num_threads = 1;
+ int num_threads = global_ctx->fs_num_threads;
errcode_t retval;
unsigned flexbg_size = 1;
int max_threads;
+ if (num_threads < 1)
+ num_threads = 1;
+
retval = _e2fsck_pass1_prepare(global_ctx);
if (retval)
goto out_abort;
@@ -3136,11 +3139,12 @@ static void e2fsck_pass1_multithread(e2fsck_t global_ctx)
int times = max_threads / num_threads;
if (times == 0)
- num_threads = 1;
+ num_threads = max_threads;
else
num_threads = max_threads / times;
}
+ global_ctx->fs_num_threads = num_threads;
init_ext2_max_sizes();
retval = e2fsck_pass1_threads_start(&infos, num_threads, global_ctx);
if (retval) {
diff --git a/e2fsck/unix.c b/e2fsck/unix.c
index 3124019a..485899b3 100644
--- a/e2fsck/unix.c
+++ b/e2fsck/unix.c
@@ -848,7 +848,7 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)
phys_mem_kb = get_memory_size() / 1024;
ctx->readahead_kb = ~0ULL;
- while ((c = getopt(argc, argv, "pamnyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDkz:")) != EOF)
+ while ((c = getopt(argc, argv, "pam:nyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDkz:")) != EOF)
switch (c) {
case 'C':
ctx->progress = e2fsck_update_progress;
@@ -891,6 +891,7 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)
break;
case 'm':
ctx->options |= E2F_OPT_MULTITHREAD;
+ ctx->fs_num_threads = atoi(optarg);
break;
case 'n':
if (ctx->options & (E2F_OPT_YES|E2F_OPT_PREEN))
diff --git a/tests/f_multithread/script b/tests/f_multithread/script
index 0fe96cd0..83cd0f03 100644
--- a/tests/f_multithread/script
+++ b/tests/f_multithread/script
@@ -1,4 +1,4 @@
-FSCK_OPT="-fy -m"
+FSCK_OPT="-fy -m1"
SECOND_FSCK_OPT=-yf
. $cmd_dir/run_e2fsck
diff --git a/tests/f_multithread_completion/script b/tests/f_multithread_completion/script
index bf23cd61..0ec13816 100644
--- a/tests/f_multithread_completion/script
+++ b/tests/f_multithread_completion/script
@@ -1,4 +1,4 @@
-FSCK_OPT="-fy -m -C 1"
+FSCK_OPT="-fy -m1 -C 1"
SECOND_FSCK_OPT=-yf
. $cmd_dir/run_e2fsck
diff --git a/tests/f_multithread_logfile/script b/tests/f_multithread_logfile/script
index d7042a03..ae497298 100644
--- a/tests/f_multithread_logfile/script
+++ b/tests/f_multithread_logfile/script
@@ -1,5 +1,5 @@
LOG_FNAME="f_multithread_logfile_xxx"
-FSCK_OPT="-fy -m -y -E log_filename=$LOG_FNAME"
+FSCK_OPT="-fy -m1 -y -E log_filename=$LOG_FNAME"
SKIP_VERIFY="true"
ONE_PASS_ONLY="true"
SKIP_CLEANUP="true"
diff --git a/tests/f_multithread_no/script b/tests/f_multithread_no/script
index b93deb3a..db791e11 100644
--- a/tests/f_multithread_no/script
+++ b/tests/f_multithread_no/script
@@ -1,4 +1,4 @@
-FSCK_OPT="-fn -m"
+FSCK_OPT="-fn -m1"
SECOND_FSCK_OPT=-yf
. $cmd_dir/run_e2fsck
diff --git a/tests/f_multithread_preen/script b/tests/f_multithread_preen/script
index ecb79cd6..8965f4a7 100644
--- a/tests/f_multithread_preen/script
+++ b/tests/f_multithread_preen/script
@@ -1,4 +1,4 @@
-FSCK_OPT="-fp -m"
+FSCK_OPT="-fp -m1"
SECOND_FSCK_OPT=-yf
. $cmd_dir/run_e2fsck
diff --git a/tests/f_multithread_yes/script b/tests/f_multithread_yes/script
index 38891f6a..8b4aa9b8 100644
--- a/tests/f_multithread_yes/script
+++ b/tests/f_multithread_yes/script
@@ -1,4 +1,4 @@
-FSCK_OPT="-f -m"
+FSCK_OPT="-f -m1"
SECOND_FSCK_OPT=-yf
. $cmd_dir/run_e2fsck
--
2.25.4
From: Wang Shilong <[email protected]>
Several improvments for this patch:
1) move readahead_kb detection to preparing phase.
2) inode readahead should be aware of thread block group
boundary.
3) make readahead_kb aware of multiple threads.
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/pass1.c | 90 ++++++++++++++++++++++++++++++--------------------
1 file changed, 55 insertions(+), 35 deletions(-)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 0cc4e60d..457c713f 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -1177,13 +1177,14 @@ static void pass1_readahead(e2fsck_t ctx, dgrp_t *group, ext2_ino_t *next_ino)
dgrp_t start = *group, grp;
blk64_t blocks_to_read = 0;
errcode_t err = EXT2_ET_INVALID_ARGUMENT;
+ dgrp_t grp_end = ctx->thread_info.et_group_end;
if (ctx->readahead_kb == 0)
goto out;
/* Keep iterating groups until we have enough to readahead */
inodes_per_block = EXT2_INODES_PER_BLOCK(ctx->fs->super);
- for (grp = start; grp < ctx->fs->group_desc_count; grp++) {
+ for (grp = start; grp < grp_end; grp++) {
if (ext2fs_bg_flags_test(ctx->fs, grp, EXT2_BG_INODE_UNINIT))
continue;
inodes_in_group = ctx->fs->super->s_inodes_per_group -
@@ -1275,6 +1276,38 @@ static int e2fsck_should_abort(e2fsck_t ctx)
return 0;
}
+/**
+ * Even though we could specify number of threads,
+ * but it might be more than the whole filesystem
+ * block groups, correct it here.
+ */
+static void e2fsck_pass1_set_thread_num(e2fsck_t ctx)
+{
+ unsigned flexbg_size = 1;
+ ext2_filsys fs = ctx->fs;
+ int num_threads = ctx->fs_num_threads;
+ int max_threads;
+
+ if (num_threads < 1)
+ num_threads = 1;
+
+ if (ext2fs_has_feature_flex_bg(fs->super))
+ flexbg_size = 1 << fs->super->s_log_groups_per_flex;
+
+ max_threads = fs->group_desc_count / flexbg_size;
+ if (max_threads == 0)
+ num_threads = 1;
+ else if (max_threads % num_threads) {
+ int times = max_threads / num_threads;
+
+ if (times == 0)
+ num_threads = max_threads;
+ else
+ num_threads = max_threads / times;
+ }
+ ctx->fs_num_threads = num_threads;
+}
+
/*
* We need call mark_table_blocks() before multiple
* thread start, since all known system blocks should be
@@ -1284,6 +1317,20 @@ static int _e2fsck_pass1_prepare(e2fsck_t ctx)
{
struct problem_context pctx;
ext2_filsys fs = ctx->fs;
+ unsigned long long readahead_kb;
+
+ e2fsck_pass1_set_thread_num(ctx);
+ /* If we can do readahead, figure out how many groups to pull in. */
+ if (!e2fsck_can_readahead(ctx->fs))
+ ctx->readahead_kb = 0;
+ else if (ctx->readahead_kb == ~0ULL)
+ ctx->readahead_kb = e2fsck_guess_readahead(ctx->fs);
+
+ /* don't use more than 1/10 of memory for threads checking */
+ readahead_kb = get_memory_size() / (10 * ctx->fs_num_threads);
+ /* maybe better disable RA if this is too small? */
+ if (ctx->readahead_kb > readahead_kb)
+ ctx->readahead_kb = readahead_kb;
clear_problem_context(&pctx);
if (!(ctx->options & E2F_OPT_PREEN))
@@ -1467,13 +1514,7 @@ void _e2fsck_pass1(e2fsck_t ctx)
init_resource_track(&rtrack, ctx->fs->io);
clear_problem_context(&pctx);
- /* If we can do readahead, figure out how many groups to pull in. */
- if (!e2fsck_can_readahead(ctx->fs))
- ctx->readahead_kb = 0;
- else if (ctx->readahead_kb == ~0ULL)
- ctx->readahead_kb = e2fsck_guess_readahead(ctx->fs);
pass1_readahead(ctx, &ra_group, &ino_threshold);
-
if (ext2fs_has_feature_dir_index(fs->super) &&
!(ctx->options & E2F_OPT_NO)) {
if (ext2fs_u32_list_create(&ctx->dirs_to_hash, 50))
@@ -2997,7 +3038,7 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
}
static int e2fsck_pass1_threads_join(struct e2fsck_thread_info *infos,
- int num_threads, e2fsck_t global_ctx)
+ e2fsck_t global_ctx)
{
errcode_t rc;
errcode_t ret = 0;
@@ -3006,7 +3047,7 @@ static int e2fsck_pass1_threads_join(struct e2fsck_thread_info *infos,
/* merge invalid bitmaps will recalculate it */
global_ctx->invalid_bitmaps = 0;
- for (i = 0; i < num_threads; i++) {
+ for (i = 0; i < global_ctx->fs_num_threads; i++) {
pinfo = &infos[i];
if (!pinfo->eti_started)
@@ -3083,7 +3124,7 @@ out:
}
static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo,
- int num_threads, e2fsck_t global_ctx)
+ e2fsck_t global_ctx)
{
struct e2fsck_thread_info *infos;
pthread_attr_t attr;
@@ -3098,6 +3139,7 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo,
thread_debug.etd_finished_threads = 0;
#endif
+ int num_threads = global_ctx->fs_num_threads;
retval = pthread_attr_init(&attr);
if (retval) {
@@ -3152,7 +3194,7 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo,
}
if (retval) {
- e2fsck_pass1_threads_join(infos, num_threads, global_ctx);
+ e2fsck_pass1_threads_join(infos, global_ctx);
return retval;
}
*pinfo = infos;
@@ -3183,13 +3225,7 @@ static void init_ext2_max_sizes()
static void e2fsck_pass1_multithread(e2fsck_t global_ctx)
{
struct e2fsck_thread_info *infos = NULL;
- int num_threads = global_ctx->fs_num_threads;
errcode_t retval;
- unsigned flexbg_size = 1;
- int max_threads;
-
- if (num_threads < 1)
- num_threads = 1;
retval = _e2fsck_pass1_prepare(global_ctx);
if (retval)
@@ -3198,31 +3234,15 @@ static void e2fsck_pass1_multithread(e2fsck_t global_ctx)
pthread_mutex_init(&global_ctx->fs_fix_mutex, NULL);
pthread_rwlock_init(&global_ctx->fs_block_map_rwlock, NULL);
pthread_mutex_init(&global_ctx->fs_ea_mutex, NULL);
- if (ext2fs_has_feature_flex_bg(global_ctx->fs->super))
- flexbg_size = 1 << global_ctx->fs->super->s_log_groups_per_flex;
-
- max_threads = global_ctx->fs->group_desc_count / flexbg_size;
- if (max_threads == 0)
- num_threads = 1;
- else if (max_threads % num_threads) {
- int times = max_threads / num_threads;
-
- if (times == 0)
- num_threads = max_threads;
- else
- num_threads = max_threads / times;
- }
-
- global_ctx->fs_num_threads = num_threads;
init_ext2_max_sizes();
- retval = e2fsck_pass1_threads_start(&infos, num_threads, global_ctx);
+ retval = e2fsck_pass1_threads_start(&infos, global_ctx);
if (retval) {
com_err(global_ctx->program_name, retval,
_("while starting pass1 threads\n"));
goto out_abort;
}
- retval = e2fsck_pass1_threads_join(infos, num_threads, global_ctx);
+ retval = e2fsck_pass1_threads_join(infos, global_ctx);
if (retval) {
com_err(global_ctx->program_name, retval,
_("while joining pass1 threads\n"));
--
2.25.4
From: Wang Shilong <[email protected]>
Now @block_found_map is no longer shared by multiple threads,
and @block_dup_map need be checked again after threads finish.
This patch also fix a bug in bitmap merging codes, this could
make following pass skip many checkings and fixes..
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/e2fsck.h | 3 +-
e2fsck/pass1.c | 177 +++++++++++++++++-------------
lib/ext2fs/bitmaps.c | 5 +-
lib/ext2fs/bitops.h | 2 +
lib/ext2fs/blkmap64_rb.c | 38 ++++++-
lib/ext2fs/bmap64.h | 3 +-
lib/ext2fs/ext2fs.h | 9 +-
lib/ext2fs/gen_bitmap64.c | 49 ++++++++-
lib/ext2fs/icount.c | 6 +-
tests/f_itable_collision/expect.1 | 3 -
10 files changed, 199 insertions(+), 96 deletions(-)
diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
index cd1bab07..83be5353 100644
--- a/e2fsck/e2fsck.h
+++ b/e2fsck/e2fsck.h
@@ -202,6 +202,7 @@ struct resource_track {
#define E2F_FLAG_TIME_INSANE 0x2000 /* Time is insane */
#define E2F_FLAG_PROBLEMS_FIXED 0x4000 /* At least one problem was fixed */
#define E2F_FLAG_ALLOC_OK 0x8000 /* Can we allocate blocks? */
+#define E2F_FLAG_DUP_BLOCK 0x20000 /* dup block found during pass1 */
#define E2F_RESET_FLAGS (E2F_FLAG_TIME_INSANE | E2F_FLAG_PROBLEMS_FIXED)
@@ -448,7 +449,7 @@ struct e2fsck_struct {
/* serialize fix operation for multiple threads */
pthread_mutex_t fs_fix_mutex;
/* protect block_found_map, block_dup_map */
- pthread_mutex_t fs_block_map_mutex;
+ pthread_rwlock_t fs_block_map_rwlock;
/* protect ea related structure */
pthread_mutex_t fs_ea_mutex;
};
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 40fe6b36..0cc4e60d 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -149,16 +149,28 @@ static void e2fsck_pass1_fix_unlock(e2fsck_t ctx)
pthread_mutex_unlock(&global_ctx->fs_fix_mutex);
}
-static inline void e2fsck_pass1_block_map_lock(e2fsck_t ctx)
+static inline void e2fsck_pass1_block_map_w_lock(e2fsck_t ctx)
{
e2fsck_get_lock_context(ctx);
- pthread_mutex_lock(&global_ctx->fs_block_map_mutex);
+ pthread_rwlock_wrlock(&global_ctx->fs_block_map_rwlock);
}
-static inline void e2fsck_pass1_block_map_unlock(e2fsck_t ctx)
+static inline void e2fsck_pass1_block_map_w_unlock(e2fsck_t ctx)
{
e2fsck_get_lock_context(ctx);
- pthread_mutex_unlock(&global_ctx->fs_block_map_mutex);
+ pthread_rwlock_unlock(&global_ctx->fs_block_map_rwlock);
+}
+
+static inline void e2fsck_pass1_block_map_r_lock(e2fsck_t ctx)
+{
+ e2fsck_get_lock_context(ctx);
+ pthread_rwlock_rdlock(&global_ctx->fs_block_map_rwlock);
+}
+
+static inline void e2fsck_pass1_block_map_r_unlock(e2fsck_t ctx)
+{
+ e2fsck_get_lock_context(ctx);
+ pthread_rwlock_unlock(&global_ctx->fs_block_map_rwlock);
}
static inline void e2fsck_pass1_ea_lock(e2fsck_t ctx)
@@ -702,6 +714,32 @@ static void check_inode_extra_space(e2fsck_t ctx, struct problem_context *pctx,
}
+static _INLINE_ int is_blocks_used(e2fsck_t ctx, blk64_t block,
+ unsigned int num)
+{
+ int retval;
+
+ /* used to avoid duplicate output from below */
+ retval = ext2fs_test_block_bitmap_range2_valid(ctx->block_found_map,
+ block, num);
+ if (!retval)
+ return 0;
+
+ retval = ext2fs_test_block_bitmap_range2(ctx->block_found_map, block, num);
+ if (retval) {
+ e2fsck_pass1_block_map_r_lock(ctx);
+ if (ctx->global_ctx)
+ retval = ext2fs_test_block_bitmap_range2(
+ ctx->global_ctx->block_found_map, block, num);
+ e2fsck_pass1_block_map_r_unlock(ctx);
+ if (retval)
+ return 0;
+ }
+
+ return 1;
+}
+
+
/*
* Check to see if the inode might really be a directory, despite i_mode
*
@@ -805,15 +843,10 @@ static void check_is_really_dir(e2fsck_t ctx, struct problem_context *pctx,
if (i >= 4)
not_device++;
- e2fsck_pass1_block_map_lock(ctx);
if (blk < ctx->fs->super->s_first_data_block ||
blk >= ext2fs_blocks_count(ctx->fs->super) ||
- ext2fs_fast_test_block_bitmap2(ctx->block_found_map,
- blk)) {
- e2fsck_pass1_block_map_unlock(ctx);
+ is_blocks_used(ctx, blk, 1))
return; /* Invalid block, can't be dir */
- }
- e2fsck_pass1_block_map_unlock(ctx);
}
blk = inode->i_block[0];
}
@@ -1284,6 +1317,19 @@ static int _e2fsck_pass1_prepare(e2fsck_t ctx)
return pctx.errcode;
}
+ pctx.errcode = e2fsck_allocate_block_bitmap(ctx->fs,
+ _("multiply claimed block map"),
+ EXT2FS_BMAP64_RBTREE, "block_dup_map",
+ &ctx->block_dup_map);
+ if (pctx.errcode) {
+ pctx.num = 3;
+ fix_problem(ctx, PR_1_ALLOCATE_BBITMAP_ERROR,
+ &pctx);
+ /* Should never get here */
+ ctx->flags |= E2F_FLAG_ABORT;
+ return pctx.errcode;
+ }
+
if (ext2fs_has_feature_mmp(fs->super) &&
fs->super->s_mmp_block > fs->super->s_first_data_block &&
fs->super->s_mmp_block < ext2fs_blocks_count(fs->super))
@@ -1375,12 +1421,17 @@ static void _e2fsck_pass1_post(e2fsck_t ctx)
}
if (ctx->block_dup_map) {
+ if (!(ctx->flags & E2F_FLAG_DUP_BLOCK)) {
+ ext2fs_free_mem(&block_buf);
+ return;
+ }
if (ctx->options & E2F_OPT_PREEN) {
clear_problem_context(&pctx);
fix_problem(ctx, PR_1_DUP_BLOCKS_PREENSTOP, &pctx);
}
e2fsck_pass1_dupblocks(ctx, block_buf);
ext2fs_free_mem(&block_buf);
+ ctx->flags &= ~E2F_FLAG_DUP_BLOCK;
}
}
@@ -1886,10 +1937,10 @@ void _e2fsck_pass1(e2fsck_t ctx)
failed_csum = 0;
}
- e2fsck_pass1_block_map_lock(ctx);
- pctx.errcode = ext2fs_copy_bitmap(ctx->block_found_map,
+ e2fsck_pass1_block_map_r_lock(ctx);
+ pctx.errcode = ext2fs_copy_bitmap(ctx->global_ctx->block_found_map,
&pb.fs_meta_blocks);
- e2fsck_pass1_block_map_unlock(ctx);
+ e2fsck_pass1_block_map_r_unlock(ctx);
if (pctx.errcode) {
pctx.num = 4;
fix_problem(ctx, PR_1_ALLOCATE_BBITMAP_ERROR, &pctx);
@@ -2281,7 +2332,7 @@ do { \
_src->_map_field = NULL; \
} else { \
_ret = ext2fs_merge_bitmap(_src->_map_field, \
- _dest->_map_field); \
+ _dest->_map_field, NULL);\
if (_ret) \
return _ret; \
} \
@@ -2298,7 +2349,7 @@ do { \
_src->_map_field = NULL; \
} else { \
_ret = ext2fs_merge_bitmap(_src->_map_field, \
- _dest->_map_field); \
+ _dest->_map_field, NULL);\
if (_ret) \
return _ret; \
} \
@@ -2598,7 +2649,7 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx,
assert(global_ctx->inode_reg_map == NULL);
assert(global_ctx->inodes_to_rebuild == NULL);
- assert(global_ctx->block_dup_map == NULL);
+ assert(global_ctx->block_dup_map != NULL);
assert(global_ctx->block_found_map != NULL);
assert(global_ctx->block_metadata_map != NULL);
assert(global_ctx->block_ea_map == NULL);
@@ -2609,8 +2660,15 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx,
return retval;
}
memcpy(thread_context, global_ctx, sizeof(struct e2fsck_struct));
- thread_context->global_ctx = global_ctx;
+ thread_context->block_dup_map = NULL;
+
+ retval = e2fsck_allocate_block_bitmap(global_ctx->fs,
+ _("in-use block map"), EXT2FS_BMAP64_RBTREE,
+ "block_found_map", &thread_context->block_found_map);
+ if (retval)
+ goto out_context;
+ thread_context->global_ctx = global_ctx;
retval = ext2fs_get_mem(sizeof(struct struct_ext2_filsys), &thread_fs);
if (retval) {
com_err(global_ctx->program_name, retval, "while allocating memory");
@@ -2660,6 +2718,8 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx,
out_fs:
ext2fs_free_mem(&thread_fs);
out_context:
+ if (thread_context->block_found_map)
+ ext2fs_free_mem(&thread_context->block_found_map);
ext2fs_free_mem(&thread_context);
return retval;
}
@@ -2697,7 +2757,6 @@ static void e2fsck_pass1_merge_dx_dir(e2fsck_t global_ctx, e2fsck_t thread_ctx)
e2fsck_merge_dx_dir(global_ctx, thread_ctx);
}
-
#define PASS1_MERGE_CTX_ICOUNT(_dest, _src, _field) \
do { \
if (_src->_field) { \
@@ -2770,7 +2829,6 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
ext2fs_inode_bitmap inode_bb_map = global_ctx->inode_bb_map;
ext2fs_inode_bitmap inode_imagic_map = global_ctx->inode_imagic_map;
ext2fs_inode_bitmap inode_reg_map = global_ctx->inode_reg_map;
- ext2fs_block_bitmap block_dup_map = global_ctx->block_dup_map;
ext2fs_block_bitmap inodes_to_rebuild = global_ctx->inodes_to_rebuild;
ext2_icount_t inode_count = global_ctx->inode_count;
ext2_icount_t inode_link_info = global_ctx->inode_link_info;
@@ -2805,6 +2863,8 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
ext2_refcount_t ea_block_quota_inodes = global_ctx->ea_block_quota_inodes;
ext2fs_block_bitmap block_ea_map = global_ctx->block_ea_map;
ext2_refcount_t ea_inode_refs = global_ctx->ea_inode_refs;
+ ext2fs_block_bitmap block_found_map = global_ctx->block_found_map;
+ ext2fs_block_bitmap block_dup_map = global_ctx->block_dup_map;
#ifdef HAVE_SETJMP_H
jmp_buf old_jmp;
@@ -2823,6 +2883,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
global_ctx->inodes_to_rebuild = inodes_to_rebuild;
global_ctx->inode_reg_map = inode_reg_map;
global_ctx->block_dup_map = block_dup_map;
+ global_ctx->block_found_map = block_found_map;
global_ctx->dir_info = dir_info;
e2fsck_pass1_merge_dir_info(global_ctx, thread_ctx);
global_ctx->dx_dir_info = dx_dir_info;
@@ -2881,6 +2942,13 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
}
global_ctx->qctx = qctx;
e2fsck_pass1_merge_quota_ctx(global_ctx, thread_ctx);
+ e2fsck_pass1_block_map_w_lock(thread_ctx);
+ retval = ext2fs_merge_bitmap(thread_ctx->block_found_map,
+ global_ctx->block_found_map,
+ global_ctx->block_dup_map);
+ e2fsck_pass1_block_map_w_unlock(thread_ctx);
+ if (retval == EEXIST)
+ global_ctx->flags |= E2F_FLAG_DUP_BLOCK;
global_ctx->invalid_block_bitmap_flag = invalid_block_bitmap_flag;
global_ctx->invalid_inode_bitmap_flag = invalid_inode_bitmap_flag;
global_ctx->invalid_inode_table_flag = invalid_inode_table_flag;
@@ -2919,6 +2987,7 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
PASS1_FREE_CTX_BITMAP(thread_ctx, inode_imagic_map);
PASS1_FREE_CTX_BITMAP(thread_ctx, inode_reg_map);
PASS1_FREE_CTX_BITMAP(thread_ctx, inodes_to_rebuild);
+ PASS1_FREE_CTX_BITMAP(thread_ctx, block_found_map);
ext2fs_free_icount(thread_ctx->inode_count);
ext2fs_free_icount(thread_ctx->inode_link_info);
e2fsck_free_dir_info(thread_ctx);
@@ -3127,7 +3196,7 @@ static void e2fsck_pass1_multithread(e2fsck_t global_ctx)
goto out_abort;
pthread_mutex_init(&global_ctx->fs_fix_mutex, NULL);
- pthread_mutex_init(&global_ctx->fs_block_map_mutex, NULL);
+ pthread_rwlock_init(&global_ctx->fs_block_map_rwlock, NULL);
pthread_mutex_init(&global_ctx->fs_ea_mutex, NULL);
if (ext2fs_has_feature_flex_bg(global_ctx->fs->super))
flexbg_size = 1 << global_ctx->fs->super->s_log_groups_per_flex;
@@ -3355,38 +3424,18 @@ static void alloc_imagic_map(e2fsck_t ctx)
static _INLINE_ void mark_block_used(e2fsck_t ctx, blk64_t block)
{
struct problem_context pctx;
- e2fsck_t global_ctx;
-
- global_ctx = ctx->global_ctx;
- if (!global_ctx)
- global_ctx = ctx;
+ e2fsck_t global_ctx = ctx->global_ctx ? ctx->global_ctx : ctx;
clear_problem_context(&pctx);
-
- if (ext2fs_fast_test_block_bitmap2(ctx->block_found_map, block)) {
+ if (is_blocks_used(ctx, block, 1)) {
if (ext2fs_has_feature_shared_blocks(ctx->fs->super) &&
!(ctx->options & E2F_OPT_UNSHARE_BLOCKS)) {
return;
}
- /**
- * this should be safe because this operation has
- * been serialized by mutex.
- */
- if (!global_ctx->block_dup_map) {
- pctx.errcode = e2fsck_allocate_block_bitmap(ctx->fs,
- _("multiply claimed block map"),
- EXT2FS_BMAP64_RBTREE, "block_dup_map",
- &global_ctx->block_dup_map);
- if (pctx.errcode) {
- pctx.num = 3;
- fix_problem(ctx, PR_1_ALLOCATE_BBITMAP_ERROR,
- &pctx);
- /* Should never get here */
- ctx->flags |= E2F_FLAG_ABORT;
- return;
- }
- }
+ ctx->flags |= E2F_FLAG_DUP_BLOCK;
+ e2fsck_pass1_block_map_w_lock(ctx);
ext2fs_fast_mark_block_bitmap2(global_ctx->block_dup_map, block);
+ e2fsck_pass1_block_map_w_unlock(ctx);
} else {
ext2fs_fast_mark_block_bitmap2(ctx->block_found_map, block);
}
@@ -3399,8 +3448,7 @@ static _INLINE_ void mark_block_used(e2fsck_t ctx, blk64_t block)
static _INLINE_ void mark_blocks_used(e2fsck_t ctx, blk64_t block,
unsigned int num)
{
- e2fsck_pass1_block_map_lock(ctx);
- if (ext2fs_test_block_bitmap_range2(ctx->block_found_map, block, num)) {
+ if (!is_blocks_used(ctx, block, num)) {
ext2fs_mark_block_bitmap_range2(ctx->block_found_map, block, num);
} else {
unsigned int i;
@@ -3408,7 +3456,6 @@ static _INLINE_ void mark_blocks_used(e2fsck_t ctx, blk64_t block,
for (i = 0; i < num; i += EXT2FS_CLUSTER_RATIO(ctx->fs))
mark_block_used(ctx, block + i);
}
- e2fsck_pass1_block_map_unlock(ctx);
}
/*
@@ -3734,9 +3781,12 @@ refcount_fail:
inc_ea_inode_refs(global_ctx, pctx, first, end);
ea_refcount_store(global_ctx->refcount, blk, header->h_refcount - 1);
- e2fsck_pass1_block_map_lock(ctx);
- mark_block_used(ctx, blk);
- e2fsck_pass1_block_map_unlock(ctx);
+ /**
+ * It might be racy that this block has been merged in the
+ * global found map.
+ */
+ if (!is_blocks_used(ctx, blk, 1))
+ ext2fs_fast_mark_block_bitmap2(ctx->block_found_map, blk);
ext2fs_fast_mark_block_bitmap2(global_ctx->block_ea_map, blk);
e2fsck_pass1_ea_unlock(ctx);
return 1;
@@ -4122,9 +4172,7 @@ report_problem:
pctx->str = "EXT2_EXTENT_UP";
return;
}
- e2fsck_pass1_block_map_lock(ctx);
mark_block_used(ctx, blk);
- e2fsck_pass1_block_map_unlock(ctx);
pb->num_blocks++;
goto next;
}
@@ -4231,7 +4279,6 @@ alloc_later:
pb->last_block,
extent.e_pblk,
extent.e_lblk)) {
- e2fsck_pass1_block_map_lock(ctx);
for (i = 0; i < extent.e_len; i++) {
pctx->blk = extent.e_lblk + i;
pctx->blk2 = extent.e_pblk + i;
@@ -4239,7 +4286,6 @@ alloc_later:
mark_block_used(ctx, extent.e_pblk + i);
mark_block_used(ctx, extent.e_pblk + i);
}
- e2fsck_pass1_block_map_unlock(ctx);
}
/*
@@ -4925,9 +4971,7 @@ static int process_block(ext2_filsys fs,
* by mark_table_blocks()).
*/
if (blockcnt == BLOCK_COUNT_DIND) {
- e2fsck_pass1_block_map_lock(ctx);
mark_block_used(ctx, blk);
- e2fsck_pass1_block_map_unlock(ctx);
}
p->num_blocks++;
} else if (!(ctx->fs->cluster_ratio_bits &&
@@ -4936,19 +4980,15 @@ static int process_block(ext2_filsys fs,
EXT2FS_B2C(ctx->fs, p->previous_block)) &&
(blk & EXT2FS_CLUSTER_MASK(ctx->fs)) ==
((unsigned) blockcnt & EXT2FS_CLUSTER_MASK(ctx->fs)))) {
- e2fsck_pass1_block_map_lock(ctx);
mark_block_used(ctx, blk);
- e2fsck_pass1_block_map_unlock(ctx);
p->num_blocks++;
} else if (has_unaligned_cluster_map(ctx, p->previous_block,
p->last_block, blk, blockcnt)) {
pctx->blk = blockcnt;
pctx->blk2 = blk;
fix_problem(ctx, PR_1_MISALIGNED_CLUSTER, pctx);
- e2fsck_pass1_block_map_lock(ctx);
mark_block_used(ctx, blk);
mark_block_used(ctx, blk);
- e2fsck_pass1_block_map_unlock(ctx);
}
if (blockcnt >= 0)
p->last_block = blockcnt;
@@ -5015,31 +5055,25 @@ static int process_bad_block(ext2_filsys fs,
}
if (blockcnt < 0) {
- e2fsck_pass1_block_map_lock(ctx);
if (ext2fs_test_block_bitmap2(p->fs_meta_blocks, blk)) {
p->bbcheck = 1;
if (fix_problem(ctx, PR_1_BB_FS_BLOCK, pctx)) {
*block_nr = 0;
- e2fsck_pass1_block_map_unlock(ctx);
return BLOCK_CHANGED;
}
- } else if (ext2fs_test_block_bitmap2(ctx->block_found_map,
- blk)) {
+ } else if (is_blocks_used(ctx, blk, 1)) {
p->bbcheck = 1;
if (fix_problem(ctx, PR_1_BBINODE_BAD_METABLOCK,
pctx)) {
*block_nr = 0;
- e2fsck_pass1_block_map_unlock(ctx);
return BLOCK_CHANGED;
}
if (e2fsck_should_abort(ctx)) {
- e2fsck_pass1_block_map_unlock(ctx);
return BLOCK_ABORT;
}
} else {
mark_block_used(ctx, blk);
}
- e2fsck_pass1_block_map_unlock(ctx);
return 0;
}
#if 0
@@ -5052,13 +5086,10 @@ static int process_bad_block(ext2_filsys fs,
* there's an overlap between the filesystem table blocks
* (bitmaps and inode table) and the bad block list.
*/
- e2fsck_pass1_block_map_lock(ctx);
- if (!ext2fs_test_block_bitmap2(ctx->block_found_map, blk)) {
+ if (!is_blocks_used(ctx, blk, 1)) {
ext2fs_mark_block_bitmap2(ctx->block_found_map, blk);
- e2fsck_pass1_block_map_unlock(ctx);
return 0;
}
- e2fsck_pass1_block_map_unlock(ctx);
/*
* Try to find the where the filesystem block was used...
*/
@@ -5213,7 +5244,6 @@ static void new_table_block(e2fsck_t ctx, blk64_t first_block, dgrp_t group,
fix_problem(ctx, (old_block ? PR_1_RELOC_FROM_TO :
PR_1_RELOC_TO), &pctx);
pctx.blk2 = 0;
- e2fsck_pass1_block_map_lock(ctx);
for (i = 0; i < num; i++) {
pctx.blk = i;
ext2fs_mark_block_bitmap2(ctx->block_found_map, (*new_block)+i);
@@ -5234,7 +5264,6 @@ static void new_table_block(e2fsck_t ctx, blk64_t first_block, dgrp_t group,
if (pctx.errcode)
fix_problem(ctx, PR_1_RELOC_WRITE_ERR, &pctx);
}
- e2fsck_pass1_block_map_unlock(ctx);
ext2fs_free_mem(&buf);
}
diff --git a/lib/ext2fs/bitmaps.c b/lib/ext2fs/bitmaps.c
index 74a8c347..4cd664d3 100644
--- a/lib/ext2fs/bitmaps.c
+++ b/lib/ext2fs/bitmaps.c
@@ -47,9 +47,10 @@ errcode_t ext2fs_copy_bitmap(ext2fs_generic_bitmap src,
}
errcode_t ext2fs_merge_bitmap(ext2fs_generic_bitmap src,
- ext2fs_generic_bitmap dest)
+ ext2fs_generic_bitmap dest,
+ ext2fs_generic_bitmap dup)
{
- return ext2fs_merge_generic_bmap(src, dest);
+ return ext2fs_merge_generic_bmap(src, dest, dup);
}
void ext2fs_set_bitmap_padding(ext2fs_generic_bitmap map)
diff --git a/lib/ext2fs/bitops.h b/lib/ext2fs/bitops.h
index 505b3c9c..1facc8dd 100644
--- a/lib/ext2fs/bitops.h
+++ b/lib/ext2fs/bitops.h
@@ -120,6 +120,8 @@ extern int ext2fs_test_block_bitmap_range(ext2fs_block_bitmap bitmap,
extern void ext2fs_set_bitmap_padding(ext2fs_generic_bitmap map);
extern __u32 ext2fs_get_generic_bitmap_start(ext2fs_generic_bitmap bitmap);
extern __u32 ext2fs_get_generic_bitmap_end(ext2fs_generic_bitmap bitmap);
+extern int ext2fs_test_block_bitmap_range2_valid(ext2fs_block_bitmap bitmap,
+ blk64_t block, unsigned int num);
/* 64-bit versions */
diff --git a/lib/ext2fs/blkmap64_rb.c b/lib/ext2fs/blkmap64_rb.c
index 42a10536..2337302f 100644
--- a/lib/ext2fs/blkmap64_rb.c
+++ b/lib/ext2fs/blkmap64_rb.c
@@ -969,27 +969,53 @@ static void rb_print_stats(ext2fs_generic_bitmap_64 bitmap EXT2FS_ATTR((unused))
#endif
static errcode_t rb_merge_bmap(ext2fs_generic_bitmap_64 src,
- ext2fs_generic_bitmap_64 dest)
+ ext2fs_generic_bitmap_64 dest,
+ ext2fs_generic_bitmap_64 dup)
{
- struct ext2fs_rb_private *src_bp, *dest_bp;
+ struct ext2fs_rb_private *src_bp, *dest_bp, *dup_bp = NULL;
struct bmap_rb_extent *src_ext;
struct rb_node *src_node;
- errcode_t retval = 0;
+ int retval = 0;
+ int dup_found = 0;
src_bp = (struct ext2fs_rb_private *) src->private;
dest_bp = (struct ext2fs_rb_private *) dest->private;
+ if (dup)
+ dup_bp = (struct ext2fs_rb_private *) dup->private;
src_bp->rcursor = NULL;
dest_bp->rcursor = NULL;
src_node = ext2fs_rb_first(&src_bp->root);
while (src_node) {
src_ext = node_to_extent(src_node);
- rb_insert_extent(src_ext->start, src_ext->count, dest_bp);
-
+ retval = rb_test_clear_bmap_extent(dest,
+ src_ext->start + src->start,
+ src_ext->count);
+ if (retval) {
+ rb_insert_extent(src_ext->start, src_ext->count,
+ dest_bp);
+ } else {
+ /* unlikely case, do it one by one block */
+ __u64 i;
+
+ for (i = src_ext->start;
+ i < src_ext->start + src_ext->count; i++) {
+ retval = rb_test_clear_bmap_extent(dest, i + src->start, 1);
+ if (retval) {
+ rb_insert_extent(i, 1, dest_bp);
+ } else {
+ if (dup_bp)
+ rb_insert_extent(i, 1, dup_bp);
+ dup_found = 1;
+ }
+ }
+ }
src_node = ext2fs_rb_next(src_node);
}
- return retval;
+ if (dup_found && dup)
+ return EEXIST;
+ return 0;
}
struct ext2_bitmap_ops ext2fs_blkmap64_rbtree = {
diff --git a/lib/ext2fs/bmap64.h b/lib/ext2fs/bmap64.h
index 09a5886b..68a4bb0a 100644
--- a/lib/ext2fs/bmap64.h
+++ b/lib/ext2fs/bmap64.h
@@ -73,7 +73,8 @@ struct ext2_bitmap_ops {
errcode_t (*copy_bmap)(ext2fs_generic_bitmap_64 src,
ext2fs_generic_bitmap_64 dest);
errcode_t (*merge_bmap)(ext2fs_generic_bitmap_64 src,
- ext2fs_generic_bitmap_64 dest);
+ ext2fs_generic_bitmap_64 dest,
+ ext2fs_generic_bitmap_64 dup);
errcode_t (*resize_bmap)(ext2fs_generic_bitmap_64 bitmap,
__u64 new_end,
__u64 new_real_end);
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index 37460a31..38ae2dee 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -839,8 +839,10 @@ extern void ext2fs_free_block_bitmap(ext2fs_block_bitmap bitmap);
extern void ext2fs_free_inode_bitmap(ext2fs_inode_bitmap bitmap);
extern errcode_t ext2fs_copy_bitmap(ext2fs_generic_bitmap src,
ext2fs_generic_bitmap *dest);
-errcode_t ext2fs_merge_bitmap(ext2fs_generic_bitmap src,
- ext2fs_generic_bitmap dest);
+
+extern errcode_t ext2fs_merge_bitmap(ext2fs_generic_bitmap src,
+ ext2fs_generic_bitmap dest,
+ ext2fs_generic_bitmap dup);
extern errcode_t ext2fs_write_inode_bitmap(ext2_filsys fs);
extern errcode_t ext2fs_write_block_bitmap (ext2_filsys fs);
extern errcode_t ext2fs_read_inode_bitmap (ext2_filsys fs);
@@ -1439,7 +1441,8 @@ errcode_t ext2fs_resize_generic_bmap(ext2fs_generic_bitmap bmap,
__u64 new_end,
__u64 new_real_end);
errcode_t ext2fs_merge_generic_bmap(ext2fs_generic_bitmap gen_src,
- ext2fs_generic_bitmap gen_dest);
+ ext2fs_generic_bitmap gen_dest,
+ ext2fs_generic_bitmap gen_dup);
errcode_t ext2fs_compare_generic_bmap(errcode_t neq,
ext2fs_generic_bitmap bm1,
ext2fs_generic_bitmap bm2);
diff --git a/lib/ext2fs/gen_bitmap64.c b/lib/ext2fs/gen_bitmap64.c
index 7bae443f..c27a52e4 100644
--- a/lib/ext2fs/gen_bitmap64.c
+++ b/lib/ext2fs/gen_bitmap64.c
@@ -340,24 +340,32 @@ errcode_t ext2fs_copy_generic_bmap(ext2fs_generic_bitmap gen_src,
}
errcode_t ext2fs_merge_generic_bmap(ext2fs_generic_bitmap gen_src,
- ext2fs_generic_bitmap gen_dest)
+ ext2fs_generic_bitmap gen_dest,
+ ext2fs_generic_bitmap gen_dup)
{
ext2fs_generic_bitmap_64 src = (ext2fs_generic_bitmap_64) gen_src;
ext2fs_generic_bitmap_64 dest = (ext2fs_generic_bitmap_64) gen_dest;
+ ext2fs_generic_bitmap_64 dup = (ext2fs_generic_bitmap_64) gen_dup;
if (!src || !dest)
return EINVAL;
- if (!EXT2FS_IS_64_BITMAP(src) || !EXT2FS_IS_64_BITMAP(dest))
+ if (!EXT2FS_IS_64_BITMAP(src) || !EXT2FS_IS_64_BITMAP(dest) ||
+ (dup && !EXT2FS_IS_64_BITMAP(dup)))
return EINVAL;
- if (src->bitmap_ops != dest->bitmap_ops)
+ if (src->bitmap_ops != dest->bitmap_ops ||
+ (dup && src->bitmap_ops != dup->bitmap_ops))
+ return EINVAL;
+
+ if (src->cluster_bits != dest->cluster_bits ||
+ (dup && dup->cluster_bits != src->cluster_bits))
return EINVAL;
if (src->bitmap_ops->merge_bmap == NULL)
return EOPNOTSUPP;
- return src->bitmap_ops->merge_bmap(src, dest);
+ return src->bitmap_ops->merge_bmap(src, dest, dup);
}
errcode_t ext2fs_resize_generic_bmap(ext2fs_generic_bitmap gen_bmap,
@@ -712,6 +720,39 @@ int ext2fs_test_block_bitmap_range2(ext2fs_block_bitmap gen_bmap,
return bmap->bitmap_ops->test_clear_bmap_extent(bmap, block, num);
}
+int ext2fs_test_block_bitmap_range2_valid(ext2fs_block_bitmap bitmap,
+ blk64_t block, unsigned int num)
+{
+ ext2fs_generic_bitmap_64 bmap = (ext2fs_generic_bitmap_64)bitmap;
+ __u64 end = block + num;
+
+ if (!bmap)
+ return 0;
+
+ if (EXT2FS_IS_32_BITMAP(bmap)) {
+ if ((block & ~0xffffffffULL) ||
+ ((block+num-1) & ~0xffffffffULL)) {
+ return 0;
+ }
+ }
+
+ if (!EXT2FS_IS_64_BITMAP(bmap))
+ return 0;
+
+ /* convert to clusters if necessary */
+ block >>= bmap->cluster_bits;
+ end += (1 << bmap->cluster_bits) - 1;
+ end >>= bmap->cluster_bits;
+ num = end - block;
+
+ if ((block < bmap->start) || (block > bmap->end) ||
+ (block+num-1 > bmap->end))
+ return 0;
+
+ return 1;
+}
+
+
void ext2fs_mark_block_bitmap_range2(ext2fs_block_bitmap gen_bmap,
blk64_t block, unsigned int num)
{
diff --git a/lib/ext2fs/icount.c b/lib/ext2fs/icount.c
index 729f993f..7ec490dc 100644
--- a/lib/ext2fs/icount.c
+++ b/lib/ext2fs/icount.c
@@ -785,12 +785,14 @@ errcode_t ext2fs_icount_merge(ext2_icount_t src, ext2_icount_t dest)
if (src->fullmap)
return ext2fs_icount_merge_full_map(src, dest);
- retval = ext2fs_merge_bitmap(src->single, dest->single);
+ retval = ext2fs_merge_bitmap(src->single,
+ dest->single, NULL);
if (retval)
return retval;
if (src->multiple) {
- retval = ext2fs_merge_bitmap(src->multiple, dest->multiple);
+ retval = ext2fs_merge_bitmap(src->multiple,
+ dest->multiple, NULL);
if (retval)
return retval;
}
diff --git a/tests/f_itable_collision/expect.1 b/tests/f_itable_collision/expect.1
index 01c85d4d..7e98baa8 100644
--- a/tests/f_itable_collision/expect.1
+++ b/tests/f_itable_collision/expect.1
@@ -1,6 +1,5 @@
Pass 1: Checking inodes, blocks, and sizes
Inode 12 block 37 conflicts with critical metadata, skipping block checks.
-Illegal block number passed to ext2fs_test_block_bitmap #268435455 for in-use block map
Illegal block number passed to ext2fs_mark_block_bitmap #268435455 for in-use block map
Inode 12, i_blocks is 48, should be 56. Fix? yes
@@ -27,9 +26,7 @@ Clear inode? yes
Restarting e2fsck from the beginning...
Pass 1: Checking inodes, blocks, and sizes
Inode 12 block 37 conflicts with critical metadata, skipping block checks.
-Illegal block number passed to ext2fs_test_block_bitmap #4294967294 for in-use block map
Illegal block number passed to ext2fs_mark_block_bitmap #4294967294 for in-use block map
-Illegal block number passed to ext2fs_test_block_bitmap #268435455 for in-use block map
Illegal block number passed to ext2fs_mark_block_bitmap #268435455 for in-use block map
Running additional passes to resolve blocks claimed by more than one inode...
--
2.25.4
From: Wang Shilong <[email protected]>
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/e2fsck.h | 1 +
e2fsck/encrypted_files.c | 175 +++++++++++++++++++++++++++++++++------
e2fsck/pass1.c | 12 ++-
3 files changed, 160 insertions(+), 28 deletions(-)
diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
index 153f2e21..4a4f1098 100644
--- a/e2fsck/e2fsck.h
+++ b/e2fsck/e2fsck.h
@@ -585,6 +585,7 @@ __u32 find_encryption_policy(e2fsck_t ctx, ext2_ino_t ino);
void destroy_encryption_policy_map(e2fsck_t ctx);
void destroy_encrypted_file_info(e2fsck_t ctx);
+int merge_two_encrypted_files(e2fsck_t src_ctx, e2fsck_t dest_ctx);
/* extents.c */
errcode_t e2fsck_rebuild_extents_later(e2fsck_t ctx, ext2_ino_t ino);
diff --git a/e2fsck/encrypted_files.c b/e2fsck/encrypted_files.c
index 16be2d6d..40540963 100644
--- a/e2fsck/encrypted_files.c
+++ b/e2fsck/encrypted_files.c
@@ -280,6 +280,9 @@ out:
static int handle_nomem(e2fsck_t ctx, struct problem_context *pctx,
size_t size_needed)
{
+ if (!pctx)
+ return -ENOMEM;
+
pctx->num = size_needed;
fix_problem(ctx, PR_1_ALLOCATE_ENCRYPTED_INODE_LIST, pctx);
/* Should never get here */
@@ -287,11 +290,155 @@ static int handle_nomem(e2fsck_t ctx, struct problem_context *pctx,
return 0;
}
+static int increase_file_ranges_capacity(e2fsck_t ctx,
+ struct encrypted_file_info *info,
+ struct problem_context *pctx)
+{
+ int size = sizeof(struct encrypted_file_range);
+
+ if (info->file_ranges_count == info->file_ranges_capacity) {
+ /* Double the capacity by default. */
+ size_t new_capacity = info->file_ranges_capacity * 2;
+
+ /* ... but go from 0 to 128 right away. */
+ if (new_capacity < 128)
+ new_capacity = 128;
+
+ /* We won't need more than the filesystem's inode count. */
+ if (new_capacity > ctx->fs->super->s_inodes_count)
+ new_capacity = ctx->fs->super->s_inodes_count;
+
+ /* To be safe, ensure the capacity really increases. */
+ if (new_capacity < info->file_ranges_capacity + 1)
+ new_capacity = info->file_ranges_capacity + 1;
+
+ if (ext2fs_resize_mem(info->file_ranges_capacity * size,
+ new_capacity * size, &info->file_ranges) != 0)
+ return handle_nomem(ctx, pctx,
+ new_capacity * size);
+
+ info->file_ranges_capacity = new_capacity;
+ }
+
+ return 0;
+}
+
+int find_entry_insert(e2fsck_t dest_ctx,
+ struct encrypted_file_range *insert_range)
+{
+ size_t l, r, m;
+ struct encrypted_file_range *range;
+ int merge_left = 0, merge_right = 0;
+ struct encrypted_file_info *dest_info = dest_ctx->encrypted_files;
+ int ret;
+
+ l = 0;
+ r = dest_info->file_ranges_count;
+ if (r < 1)
+ return -EINVAL;
+
+ while (l < r) {
+ m = l + (r - l) / 2;
+ range = &dest_info->file_ranges[m];
+
+ if (insert_range->first_ino < range->first_ino)
+ r = m;
+ else if (insert_range->first_ino > range->last_ino)
+ l = m + 1;
+ else /* should not happen */ {
+ return -EINVAL;
+ }
+ }
+
+ /* check wheather it could be merged left */
+ if (l >= 1) {
+ range = &dest_info->file_ranges[l - 1];
+ if (range->last_ino + 1 ==
+ insert_range->first_ino &&
+ range->policy_id == insert_range->policy_id) {
+ range->last_ino = insert_range->last_ino;
+ merge_left = 1;
+ }
+ }
+
+ /* check wheather it could be merged right */
+ if (l < dest_info->file_ranges_count - 1) {
+ range = &dest_info->file_ranges[l + 1];
+ if (range->first_ino ==
+ insert_range->last_ino + 1 &&
+ range->policy_id == insert_range->policy_id) {
+ range->first_ino = insert_range->first_ino;
+ merge_right = 1;
+ }
+ }
+ /* check if we could shrink array */
+ if (merge_left && merge_right) {
+ for (m = l; m < dest_info->file_ranges_count - 1;
+ m++)
+ dest_info->file_ranges[m] =
+ dest_info->file_ranges[m + 1];
+
+ dest_info->file_ranges_count--;
+ return 0;
+ } else if (merge_left || merge_right) { /* return directly */
+ return 0;
+ }
+
+ ret = increase_file_ranges_capacity(dest_ctx, dest_info, NULL);
+ if (ret)
+ return ret;
+
+ /* move forward */
+ for (m = dest_info->file_ranges_count; m >= l; m--)
+ dest_info->file_ranges[m + 1] =
+ dest_info->file_ranges[m];
+
+ dest_info->file_ranges[l] = *insert_range;
+ dest_info->file_ranges_count++;
+ return 0;
+}
+
+int merge_two_encrypted_files(e2fsck_t src_ctx, e2fsck_t dest_ctx)
+{
+ struct encrypted_file_info *src_info = src_ctx->encrypted_files;
+ struct encrypted_file_info *dest_info = dest_ctx->encrypted_files;
+ struct encrypted_file_range *range;
+ __u32 policy_id;
+ errcode_t retval;
+ size_t i;
+
+ /* nothing to merge */
+ if (!src_info)
+ return 0;
+
+ if (!dest_info) {
+ dest_ctx->encrypted_files = src_info;
+ src_ctx->encrypted_files = NULL;
+ return 0;
+ }
+
+ for (i = 0; i < src_info->file_ranges_count; i++) {
+ range = &src_info->file_ranges[i];
+ retval = get_encryption_policy_id(dest_ctx, range->first_ino,
+ &policy_id);
+ if (retval != 0)
+ return retval;
+ /* reset policy id */
+ range->policy_id = policy_id;
+ retval = find_entry_insert(dest_ctx, range);
+ if (retval)
+ return retval;
+ }
+
+ return 0;
+}
+
static int append_ino_and_policy_id(e2fsck_t ctx, struct problem_context *pctx,
ext2_ino_t ino, __u32 policy_id)
{
struct encrypted_file_info *info = ctx->encrypted_files;
struct encrypted_file_range *range;
+ int ret;
/* See if we can just extend the last range. */
if (info->file_ranges_count > 0) {
@@ -310,32 +457,10 @@ static int append_ino_and_policy_id(e2fsck_t ctx, struct problem_context *pctx,
}
}
/* Nope, a new range is needed. */
+ ret = increase_file_ranges_capacity(ctx, info, pctx);
+ if (ret)
+ return ret;
- if (info->file_ranges_count == info->file_ranges_capacity) {
- /* Double the capacity by default. */
- size_t new_capacity = info->file_ranges_capacity * 2;
-
- /* ... but go from 0 to 128 right away. */
- if (new_capacity < 128)
- new_capacity = 128;
-
- /* We won't need more than the filesystem's inode count. */
- if (new_capacity > ctx->fs->super->s_inodes_count)
- new_capacity = ctx->fs->super->s_inodes_count;
-
- /* To be safe, ensure the capacity really increases. */
- if (new_capacity < info->file_ranges_capacity + 1)
- new_capacity = info->file_ranges_capacity + 1;
-
- if (ext2fs_resize_mem(info->file_ranges_capacity *
- sizeof(*range),
- new_capacity * sizeof(*range),
- &info->file_ranges) != 0)
- return handle_nomem(ctx, pctx,
- new_capacity * sizeof(*range));
-
- info->file_ranges_capacity = new_capacity;
- }
range = &info->file_ranges[info->file_ranges_count++];
range->first_ino = ino;
range->last_ino = ino;
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index ac3ffa7b..06e7d753 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -2288,9 +2288,6 @@ void _e2fsck_pass1(e2fsck_t ctx)
ext2fs_close_inode_scan(scan);
scan = NULL;
- /* We don't need the encryption policy => ID map any more */
- destroy_encryption_policy_map(ctx);
-
if (ctx->ea_block_quota_blocks) {
ea_refcount_free(ctx->ea_block_quota_blocks);
ctx->ea_block_quota_blocks = 0;
@@ -3045,6 +3042,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
ext2_refcount_t ea_inode_refs = global_ctx->ea_inode_refs;
ext2fs_block_bitmap block_found_map = global_ctx->block_found_map;
ext2fs_block_bitmap block_dup_map = global_ctx->block_dup_map;
+ struct encrypted_file_info *dest_info = global_ctx->encrypted_files;
#ifdef HAVE_SETJMP_H
jmp_buf old_jmp;
@@ -3079,6 +3077,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
global_ctx->ea_block_quota_inodes = ea_block_quota_inodes;
global_ctx->block_ea_map = block_ea_map;
global_ctx->ea_inode_refs = ea_inode_refs;
+ global_ctx->encrypted_files = dest_info;
PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_directory_count);
PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_regular_count);
PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_blockdev_count);
@@ -3130,6 +3129,12 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
global_ctx->invalid_inode_table_flag = invalid_inode_table_flag;
global_ctx->invalid_bitmaps = invalid_bitmaps;
e2fsck_pass1_merge_invalid_bitmaps(global_ctx, thread_ctx);
+ retval = merge_two_encrypted_files(thread_ctx, global_ctx);
+ if (retval) {
+ com_err(global_ctx->program_name, 0,
+ _("while merging encrypted files"));
+ return retval;
+ }
/*
* PASS1_COPY_CTX_BITMAP might return directly from this function,
@@ -3198,6 +3203,7 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
ea_refcount_free(thread_ctx->refcount_orig);
thread_ctx->refcount_orig = NULL;
}
+ destroy_encrypted_file_info(thread_ctx);
e2fsck_free_dir_info(thread_ctx);
ext2fs_free_mem(&thread_ctx);
--
2.25.4
From: Wang Shilong <[email protected]>
We tried to copy thread context to global context directly
and then copy back some saved variables before merging.
Since we have finished almost all necessary variables
in the e2fsck context, we could simplify codes, and
this could help us understand what is missing rather
than hide problems.
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/pass1.c | 153 +++++++++----------------------------------------
1 file changed, 27 insertions(+), 126 deletions(-)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index dc710e4d..7d6e531a 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -2390,11 +2390,6 @@ do { \
} \
} while (0)
-#define PASS1_MERGE_CTX_COUNT(_dest, _src, _field) \
-do { \
- _dest->_field = _field + _src->_field; \
-} while (0)
-
static errcode_t pass1_open_io_channel(ext2_filsys fs,
const char *io_options,
io_manager manager, int flags)
@@ -2511,6 +2506,7 @@ static int _e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
ext2_badblocks_list badblocks;
ext2_dblist dblist;
int flags;
+ e2fsck_t dest_ctx = dest->priv_data;
dest_io = dest->io;
dest_image_io = dest->image_io;
@@ -2526,6 +2522,7 @@ static int _e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
dest->block_map = block_map;
dest->badblocks = badblocks;
dest->dblist = dblist;
+ dest->priv_data = dest_ctx;
dest->flags = src->flags | flags;
if (!(src->flags & EXT2_FLAG_VALID) || !(flags & EXT2_FLAG_VALID))
ext2fs_unmark_valid(dest);
@@ -2990,133 +2987,42 @@ static errcode_t e2fsck_pass1_merge_ea_refcount(e2fsck_t global_ctx,
return retval;
}
-static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx)
+static errcode_t e2fsck_pass1_merge_context(e2fsck_t global_ctx,
+ e2fsck_t thread_ctx)
{
- errcode_t retval;
- int flags = global_ctx->flags;
- ext2_filsys thread_fs = thread_ctx->fs;
- ext2_filsys global_fs = global_ctx->fs;
- FILE *global_logf = global_ctx->logf;
- FILE *global_problem_logf = global_ctx->problem_logf;
- struct dir_info_db *dir_info = global_ctx->dir_info;
- struct dx_dir_info *dx_dir_info = global_ctx->dx_dir_info;
- ext2fs_inode_bitmap inode_used_map = global_ctx->inode_used_map;
- ext2fs_inode_bitmap inode_dir_map = global_ctx->inode_dir_map;
- ext2fs_inode_bitmap inode_bb_map = global_ctx->inode_bb_map;
- ext2fs_inode_bitmap inode_imagic_map = global_ctx->inode_imagic_map;
- ext2fs_inode_bitmap inode_reg_map = global_ctx->inode_reg_map;
- ext2fs_block_bitmap inodes_to_rebuild = global_ctx->inodes_to_rebuild;
- ext2fs_inode_bitmap inode_bad_map = global_ctx->inode_bad_map;
- ext2_icount_t inode_count = global_ctx->inode_count;
- ext2_icount_t inode_link_info = global_ctx->inode_link_info;
- __u32 fs_directory_count = global_ctx->fs_directory_count;
- __u32 fs_regular_count = global_ctx->fs_regular_count;
- __u32 fs_blockdev_count = global_ctx->fs_blockdev_count;
- __u32 fs_chardev_count = global_ctx->fs_chardev_count;
- __u32 fs_links_count = global_ctx->fs_links_count;
- __u32 fs_symlinks_count = global_ctx->fs_symlinks_count;
- __u32 fs_fast_symlinks_count = global_ctx->fs_fast_symlinks_count;
- __u32 fs_fifo_count = global_ctx->fs_fifo_count;
- __u32 fs_total_count = global_ctx->fs_total_count;
- __u32 fs_badblocks_count = global_ctx->fs_badblocks_count;
- __u32 fs_sockets_count = global_ctx->fs_sockets_count;
- __u32 fs_ind_count = global_ctx->fs_ind_count;
- __u32 fs_dind_count = global_ctx->fs_dind_count;
- __u32 fs_tind_count = global_ctx->fs_tind_count;
- __u32 fs_fragmented = global_ctx->fs_fragmented;
- __u32 fs_fragmented_dir = global_ctx->fs_fragmented_dir;
- __u32 large_files = global_ctx->large_files;
- int dx_dir_info_size = global_ctx->dx_dir_info_size;
- int dx_dir_info_count = global_ctx->dx_dir_info_count;
- ext2_u32_list dirs_to_hash = global_ctx->dirs_to_hash;
- quota_ctx_t qctx = global_ctx->qctx;
- int *invalid_block_bitmap_flag = global_ctx->invalid_block_bitmap_flag;
- int *invalid_inode_bitmap_flag = global_ctx->invalid_inode_bitmap_flag;
- int *invalid_inode_table_flag = global_ctx->invalid_inode_table_flag;
- int invalid_bitmaps = global_ctx->invalid_bitmaps;
- ext2_refcount_t refcount = global_ctx->refcount;
- ext2_refcount_t refcount_extra = global_ctx->refcount_extra;
- ext2_refcount_t refcount_orig = global_ctx->refcount_orig;
- ext2_refcount_t ea_block_quota_blocks = global_ctx->ea_block_quota_blocks;
- ext2_refcount_t ea_block_quota_inodes = global_ctx->ea_block_quota_inodes;
- ext2fs_block_bitmap block_ea_map = global_ctx->block_ea_map;
- ext2_refcount_t ea_inode_refs = global_ctx->ea_inode_refs;
- ext2fs_block_bitmap block_found_map = global_ctx->block_found_map;
- ext2fs_block_bitmap block_dup_map = global_ctx->block_dup_map;
- struct encrypted_file_info *dest_info = global_ctx->encrypted_files;
-
-#ifdef HAVE_SETJMP_H
- jmp_buf old_jmp;
-
- memcpy(old_jmp, global_ctx->abort_loc, sizeof(jmp_buf));
-#endif
- memcpy(global_ctx, thread_ctx, sizeof(struct e2fsck_struct));
-#ifdef HAVE_SETJMP_H
- memcpy(global_ctx->abort_loc, old_jmp, sizeof(jmp_buf));
-#endif
+ errcode_t retval;
- global_ctx->inode_used_map = inode_used_map;
- global_ctx->inode_dir_map = inode_dir_map;
- global_ctx->inode_bb_map = inode_bb_map;
- global_ctx->inode_imagic_map = inode_imagic_map;
- global_ctx->inodes_to_rebuild = inodes_to_rebuild;
- global_ctx->inode_reg_map = inode_reg_map;
- global_ctx->block_dup_map = block_dup_map;
- global_ctx->block_found_map = block_found_map;
- global_ctx->inode_bad_map = inode_bad_map;
- global_ctx->dir_info = dir_info;
- e2fsck_pass1_merge_dir_info(global_ctx, thread_ctx);
- global_ctx->dx_dir_info = dx_dir_info;
- global_ctx->dx_dir_info_count = dx_dir_info_count;
- global_ctx->dx_dir_info_size = dx_dir_info_size;
- e2fsck_pass1_merge_dx_dir(global_ctx, thread_ctx);
- global_ctx->inode_count = inode_count;
- global_ctx->inode_link_info = inode_link_info;
- global_ctx->refcount = refcount;
- global_ctx->refcount_extra = refcount_extra;
- global_ctx->refcount_orig = refcount_orig;
- global_ctx->ea_block_quota_blocks = ea_block_quota_blocks;
- global_ctx->ea_block_quota_inodes = ea_block_quota_inodes;
- global_ctx->block_ea_map = block_ea_map;
- global_ctx->ea_inode_refs = ea_inode_refs;
- global_ctx->encrypted_files = dest_info;
- PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_directory_count);
- PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_regular_count);
- PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_blockdev_count);
- PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_chardev_count);
- PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_links_count);
- PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_symlinks_count);
- PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_fast_symlinks_count);
- PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_fifo_count);
- PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_total_count);
- PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_badblocks_count);
- PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_sockets_count);
- PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_ind_count);
- PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_dind_count);
- PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_tind_count);
- PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_fragmented);
- PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_fragmented_dir);
- PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, large_files);
-
- global_ctx->flags |= flags;
-
- retval = e2fsck_pass1_merge_fs(global_fs, thread_fs);
+ global_ctx->fs_directory_count += thread_ctx->fs_directory_count;
+ global_ctx->fs_regular_count += thread_ctx->fs_regular_count;
+ global_ctx->fs_blockdev_count += thread_ctx->fs_blockdev_count;
+ global_ctx->fs_chardev_count += thread_ctx->fs_chardev_count;
+ global_ctx->fs_links_count += thread_ctx->fs_links_count;
+ global_ctx->fs_symlinks_count += thread_ctx->fs_symlinks_count;
+ global_ctx->fs_fast_symlinks_count += thread_ctx->fs_fast_symlinks_count;
+ global_ctx->fs_fifo_count += thread_ctx->fs_fifo_count;
+ global_ctx->fs_total_count += thread_ctx->fs_total_count;
+ global_ctx->fs_badblocks_count += thread_ctx->fs_badblocks_count;
+ global_ctx->fs_sockets_count += thread_ctx->fs_sockets_count;
+ global_ctx->fs_ind_count += thread_ctx->fs_ind_count;
+ global_ctx->fs_dind_count += thread_ctx->fs_dind_count;
+ global_ctx->fs_tind_count += thread_ctx->fs_tind_count;
+ global_ctx->fs_fragmented += thread_ctx->fs_fragmented;
+ global_ctx->fs_fragmented_dir += thread_ctx->fs_fragmented_dir;
+ global_ctx->large_files += thread_ctx->large_files;
+ global_ctx->flags |= thread_ctx->flags;
+ e2fsck_pass1_merge_dir_info(global_ctx, thread_ctx);
+ e2fsck_pass1_merge_dx_dir(global_ctx, thread_ctx);
+ retval = e2fsck_pass1_merge_fs(global_ctx->fs, thread_ctx->fs);
if (retval) {
com_err(global_ctx->program_name, 0, _("while merging fs\n"));
return retval;
}
- global_fs->priv_data = global_ctx;
- global_ctx->fs = global_fs;
- global_ctx->logf = global_logf;
- global_ctx->problem_logf = global_problem_logf;
- global_ctx->global_ctx = NULL;
retval = e2fsck_pass1_merge_icounts(global_ctx, thread_ctx);
if (retval) {
com_err(global_ctx->program_name, 0,
_("while merging icounts\n"));
return retval;
}
- global_ctx->dirs_to_hash = dirs_to_hash;
retval = e2fsck_pass1_merge_dirs_to_hash(global_ctx, thread_ctx);
if (retval) {
com_err(global_ctx->program_name, 0, _("while merging dirs to hash\n"));
@@ -3124,12 +3030,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
}
e2fsck_pass1_merge_ea_inode_refs(global_ctx, thread_ctx);
e2fsck_pass1_merge_ea_refcount(global_ctx, thread_ctx);
- global_ctx->qctx = qctx;
e2fsck_pass1_merge_quota_ctx(global_ctx, thread_ctx);
- global_ctx->invalid_block_bitmap_flag = invalid_block_bitmap_flag;
- global_ctx->invalid_inode_bitmap_flag = invalid_inode_bitmap_flag;
- global_ctx->invalid_inode_table_flag = invalid_inode_table_flag;
- global_ctx->invalid_bitmaps = invalid_bitmaps;
e2fsck_pass1_merge_invalid_bitmaps(global_ctx, thread_ctx);
retval = merge_two_encrypted_files(thread_ctx, global_ctx);
if (retval) {
@@ -3172,7 +3073,7 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
{
errcode_t retval;
- retval = e2fsck_pass1_thread_join_one(global_ctx, thread_ctx);
+ retval = e2fsck_pass1_merge_context(global_ctx, thread_ctx);
ext2fs_free_mem(&thread_ctx->fs);
if (thread_ctx->logf)
fclose(thread_ctx->logf);
--
2.25.4
From: Wang Shilong <[email protected]>
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/pass1.c | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index d56b7128..45e8090b 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -1375,9 +1375,12 @@ static void _e2fsck_pass1_post(e2fsck_t ctx)
{
struct problem_context pctx;
ext2_filsys fs = ctx->fs;
+ char *block_buf;
- char *block_buf =
- (char *)e2fsck_allocate_memory(ctx, ctx->fs->blocksize * 3,
+ if (e2fsck_should_abort(ctx))
+ return;
+
+ block_buf = (char *)e2fsck_allocate_memory(ctx, ctx->fs->blocksize * 3,
"block interate buffer");
reserve_block_for_root_repair(ctx);
reserve_block_for_lnf_repair(ctx);
@@ -1455,6 +1458,8 @@ static void _e2fsck_pass1_post(e2fsck_t ctx)
ext2fs_free_mem(&block_buf);
ctx->flags &= ~E2F_FLAG_DUP_BLOCK;
}
+
+ ctx->flags |= E2F_FLAG_ALLOC_OK;
}
@@ -2309,7 +2314,6 @@ void _e2fsck_pass1(e2fsck_t ctx)
goto endit;
}
- ctx->flags |= E2F_FLAG_ALLOC_OK;
endit:
e2fsck_use_inode_shortcuts(ctx, 0);
ext2fs_free_mem(&inodes_to_process);
--
2.25.4
From: Wang Shilong <[email protected]>
It will be possible that threads might append E2F_OPT_YES,
so we need merge options to global, test f_yesall cover this.
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/pass1.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 7d6e531a..b4adb8fa 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -3010,6 +3010,8 @@ static errcode_t e2fsck_pass1_merge_context(e2fsck_t global_ctx,
global_ctx->fs_fragmented_dir += thread_ctx->fs_fragmented_dir;
global_ctx->large_files += thread_ctx->large_files;
global_ctx->flags |= thread_ctx->flags;
+ /* threads might enable E2F_OPT_YES */
+ global_ctx->options |= thread_ctx->options;
e2fsck_pass1_merge_dir_info(global_ctx, thread_ctx);
e2fsck_pass1_merge_dx_dir(global_ctx, thread_ctx);
retval = e2fsck_pass1_merge_fs(global_ctx->fs, thread_ctx->fs);
--
2.25.4
From: Wang Shilong <[email protected]>
For smaller fs(less than TiB filesystem), it doesn't
make sense to setup threads, and default behavior will
be kept as single threads.
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/pass1.c | 60 +++++++++++++++++++++++++++-----------------------
1 file changed, 32 insertions(+), 28 deletions(-)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 45e8090b..969475b4 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -1293,6 +1293,27 @@ static void e2fsck_pass1_set_thread_num(e2fsck_t ctx)
ctx->fs_num_threads = num_threads;
}
+static void init_ext2_max_sizes()
+{
+ int i;
+ __u64 max_sizes;
+
+ /*
+ * Init ext2_max_sizes which will be immutable and shared between
+ * threads
+ */
+#define EXT2_BPP(bits) (1ULL << ((bits) - 2))
+
+ for (i = EXT2_MIN_BLOCK_LOG_SIZE; i <= EXT2_MAX_BLOCK_LOG_SIZE; i++) {
+ max_sizes = EXT2_NDIR_BLOCKS + EXT2_BPP(i);
+ max_sizes = max_sizes + EXT2_BPP(i) * EXT2_BPP(i);
+ max_sizes = max_sizes + EXT2_BPP(i) * EXT2_BPP(i) * EXT2_BPP(i);
+ max_sizes = (max_sizes * (1UL << i));
+ ext2_max_sizes[i - EXT2_MIN_BLOCK_LOG_SIZE] = max_sizes;
+ }
+#undef EXT2_BPP
+}
+
/*
* We need call mark_table_blocks() before multiple
* thread start, since all known system blocks should be
@@ -1304,6 +1325,7 @@ static int _e2fsck_pass1_prepare(e2fsck_t ctx)
ext2_filsys fs = ctx->fs;
unsigned long long readahead_kb;
+ init_ext2_max_sizes();
e2fsck_pass1_set_thread_num(ctx);
/* If we can do readahead, figure out how many groups to pull in. */
if (!e2fsck_can_readahead(ctx->fs))
@@ -1944,6 +1966,10 @@ void _e2fsck_pass1(e2fsck_t ctx)
if (ino == EXT2_BAD_INO) {
struct process_block_struct pb;
+ e2fsck_t global_ctx = ctx;
+
+ if (ctx->global_ctx)
+ global_ctx = ctx->global_ctx;
if ((failed_csum || inode->i_mode || inode->i_uid ||
inode->i_gid || inode->i_links_count ||
@@ -1959,7 +1985,7 @@ void _e2fsck_pass1(e2fsck_t ctx)
}
e2fsck_pass1_block_map_r_lock(ctx);
- pctx.errcode = ext2fs_copy_bitmap(ctx->global_ctx->block_found_map,
+ pctx.errcode = ext2fs_copy_bitmap(global_ctx->block_found_map,
&pb.fs_meta_blocks);
e2fsck_pass1_block_map_r_unlock(ctx);
if (pctx.errcode) {
@@ -3297,39 +3323,13 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo,
return 0;
}
-static void init_ext2_max_sizes()
-{
- int i;
- __u64 max_sizes;
-
- /*
- * Init ext2_max_sizes which will be immutable and shared between
- * threads
- */
-#define EXT2_BPP(bits) (1ULL << ((bits) - 2))
-
- for (i = EXT2_MIN_BLOCK_LOG_SIZE; i <= EXT2_MAX_BLOCK_LOG_SIZE; i++) {
- max_sizes = EXT2_NDIR_BLOCKS + EXT2_BPP(i);
- max_sizes = max_sizes + EXT2_BPP(i) * EXT2_BPP(i);
- max_sizes = max_sizes + EXT2_BPP(i) * EXT2_BPP(i) * EXT2_BPP(i);
- max_sizes = (max_sizes * (1UL << i));
- ext2_max_sizes[i - EXT2_MIN_BLOCK_LOG_SIZE] = max_sizes;
- }
-#undef EXT2_BPP
-}
-
static void e2fsck_pass1_multithread(e2fsck_t global_ctx)
{
struct e2fsck_thread_info *infos = NULL;
errcode_t retval;
- retval = _e2fsck_pass1_prepare(global_ctx);
- if (retval)
- goto out_abort;
-
pthread_mutex_init(&global_ctx->fs_fix_mutex, NULL);
pthread_rwlock_init(&global_ctx->fs_block_map_rwlock, NULL);
- init_ext2_max_sizes();
retval = e2fsck_pass1_threads_start(&infos, global_ctx);
if (retval) {
com_err(global_ctx->program_name, retval,
@@ -3351,7 +3351,11 @@ out_abort:
void e2fsck_pass1(e2fsck_t ctx)
{
- e2fsck_pass1_multithread(ctx);
+ _e2fsck_pass1_prepare(ctx);
+ if (ctx->options & E2F_OPT_MULTITHREAD)
+ e2fsck_pass1_multithread(ctx);
+ else
+ _e2fsck_pass1(ctx);
_e2fsck_pass1_post(ctx);
}
--
2.25.4
From: Wang Shilong <[email protected]>
Before proceeding next inodes, waitting existed
fixing finished.
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/e2fsck.h | 2 +-
e2fsck/pass1.c | 72 +++++++++++++++++++++++++++++++++++++++++++------
2 files changed, 65 insertions(+), 9 deletions(-)
diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
index 4a4f1098..86ec04ec 100644
--- a/e2fsck/e2fsck.h
+++ b/e2fsck/e2fsck.h
@@ -448,7 +448,7 @@ struct e2fsck_struct {
__u32 extent_depth_count[MAX_EXTENT_DEPTH_COUNT];
__u32 fs_num_threads;
/* serialize fix operation for multiple threads */
- pthread_mutex_t fs_fix_mutex;
+ pthread_rwlock_t fs_fix_rwlock;
/* protect block_found_map, block_dup_map */
pthread_rwlock_t fs_block_map_rwlock;
};
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 969475b4..52af4f13 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -137,16 +137,40 @@ static __u64 ext2_max_sizes[EXT2_MAX_BLOCK_LOG_SIZE -
if (!global_ctx) \
global_ctx = ctx; \
+/**
+ * before we hold write lock, read lock should
+ * has been held.
+ */
static void e2fsck_pass1_fix_lock(e2fsck_t ctx)
{
+ int err;
+
e2fsck_get_lock_context(ctx);
- pthread_mutex_lock(&global_ctx->fs_fix_mutex);
+ err = pthread_rwlock_trywrlock(&global_ctx->fs_fix_rwlock);
+ assert(err != 0);
+ pthread_rwlock_unlock(&global_ctx->fs_fix_rwlock);
+ pthread_rwlock_wrlock(&global_ctx->fs_fix_rwlock);
}
static void e2fsck_pass1_fix_unlock(e2fsck_t ctx)
{
e2fsck_get_lock_context(ctx);
- pthread_mutex_unlock(&global_ctx->fs_fix_mutex);
+ /* unlock write lock */
+ pthread_rwlock_unlock(&global_ctx->fs_fix_rwlock);
+ /* get read lock again */
+ pthread_rwlock_rdlock(&global_ctx->fs_fix_rwlock);
+}
+
+static void e2fsck_pass1_check_lock(e2fsck_t ctx)
+{
+ e2fsck_get_lock_context(ctx);
+ pthread_rwlock_rdlock(&global_ctx->fs_fix_rwlock);
+}
+
+static void e2fsck_pass1_check_unlock(e2fsck_t ctx)
+{
+ e2fsck_get_lock_context(ctx);
+ pthread_rwlock_unlock(&global_ctx->fs_fix_rwlock);
}
static inline void e2fsck_pass1_block_map_w_lock(e2fsck_t ctx)
@@ -1655,6 +1679,7 @@ void _e2fsck_pass1(e2fsck_t ctx)
}
while (1) {
+ e2fsck_pass1_check_lock(ctx);
if (ino % (fs->super->s_inodes_per_group * 4) == 1) {
if (e2fsck_mmp_update(fs))
fatal_error(ctx, 0);
@@ -1665,8 +1690,10 @@ void _e2fsck_pass1(e2fsck_t ctx)
if (ino > ino_threshold)
pass1_readahead(ctx, &ra_group, &ino_threshold);
ehandler_operation(old_op);
- if (e2fsck_should_abort(ctx))
+ if (e2fsck_should_abort(ctx)) {
+ e2fsck_pass1_check_unlock(ctx);
goto endit;
+ }
if (pctx.errcode == EXT2_ET_BAD_BLOCK_IN_INODE_TABLE) {
/*
* If badblocks says badblocks is bad, offer to clear
@@ -1691,25 +1718,32 @@ void _e2fsck_pass1(e2fsck_t ctx)
ctx->flags |= E2F_FLAG_ABORT;
} else
ctx->flags |= E2F_FLAG_RESTART;
+ e2fsck_pass1_check_unlock(ctx);
goto endit;
}
if (!ctx->inode_bb_map)
alloc_bb_map(ctx);
ext2fs_mark_inode_bitmap2(ctx->inode_bb_map, ino);
ext2fs_mark_inode_bitmap2(ctx->inode_used_map, ino);
+ e2fsck_pass1_check_unlock(ctx);
continue;
}
- if (pctx.errcode == EXT2_ET_SCAN_FINISHED)
+ if (pctx.errcode == EXT2_ET_SCAN_FINISHED) {
+ e2fsck_pass1_check_unlock(ctx);
break;
+ }
if (pctx.errcode &&
pctx.errcode != EXT2_ET_INODE_CSUM_INVALID &&
pctx.errcode != EXT2_ET_INODE_IS_GARBAGE) {
fix_problem(ctx, PR_1_ISCAN_ERROR, &pctx);
ctx->flags |= E2F_FLAG_ABORT;
+ e2fsck_pass1_check_unlock(ctx);
goto endit;
}
- if (!ino)
+ if (!ino) {
+ e2fsck_pass1_check_unlock(ctx);
break;
+ }
if (ctx->global_ctx)
ctx->thread_info.et_inode_number++;
pctx.ino = ino;
@@ -1764,6 +1798,7 @@ void _e2fsck_pass1(e2fsck_t ctx)
pctx.num = inode->i_links_count;
fix_problem(ctx, PR_1_ICOUNT_STORE, &pctx);
ctx->flags |= E2F_FLAG_ABORT;
+ e2fsck_pass1_check_unlock(ctx);
goto endit;
}
} else if ((ino >= EXT2_FIRST_INODE(fs->super)) &&
@@ -1780,6 +1815,7 @@ void _e2fsck_pass1(e2fsck_t ctx)
}
}
FINISH_INODE_LOOP(ctx, ino, &pctx, failed_csum);
+ e2fsck_pass1_check_unlock(ctx);
continue;
}
@@ -1800,6 +1836,7 @@ void _e2fsck_pass1(e2fsck_t ctx)
&pctx);
if (res < 0) {
/* skip FINISH_INODE_LOOP */
+ e2fsck_pass1_check_unlock(ctx);
continue;
}
}
@@ -1822,6 +1859,7 @@ void _e2fsck_pass1(e2fsck_t ctx)
e2fsck_clear_inode(ctx, ino, inode, 0, "pass1");
e2fsck_pass1_fix_unlock(ctx);
/* skip FINISH_INODE_LOOP */
+ e2fsck_pass1_check_unlock(ctx);
continue;
}
}
@@ -1868,6 +1906,7 @@ void _e2fsck_pass1(e2fsck_t ctx)
pctx.errcode = err;
ctx->flags |= E2F_FLAG_ABORT;
e2fsck_pass1_fix_unlock(ctx);
+ e2fsck_pass1_check_unlock(ctx);
goto endit;
}
inode->i_flags &= ~EXT4_INLINE_DATA_FL;
@@ -1883,6 +1922,7 @@ void _e2fsck_pass1(e2fsck_t ctx)
/* Some other kind of non-xattr error? */
pctx.errcode = err;
ctx->flags |= E2F_FLAG_ABORT;
+ e2fsck_pass1_check_unlock(ctx);
goto endit;
}
}
@@ -1922,6 +1962,7 @@ void _e2fsck_pass1(e2fsck_t ctx)
ext2fs_mark_inode_bitmap2(ctx->inode_used_map,
ino);
/* skip FINISH_INODE_LOOP */
+ e2fsck_pass1_check_unlock(ctx);
continue;
}
}
@@ -1992,6 +2033,7 @@ void _e2fsck_pass1(e2fsck_t ctx)
pctx.num = 4;
fix_problem(ctx, PR_1_ALLOCATE_BBITMAP_ERROR, &pctx);
ctx->flags |= E2F_FLAG_ABORT;
+ e2fsck_pass1_check_unlock(ctx);
goto endit;
}
pb.ino = EXT2_BAD_INO;
@@ -2009,16 +2051,19 @@ void _e2fsck_pass1(e2fsck_t ctx)
if (pctx.errcode) {
fix_problem(ctx, PR_1_BLOCK_ITERATE, &pctx);
ctx->flags |= E2F_FLAG_ABORT;
+ e2fsck_pass1_check_unlock(ctx);
goto endit;
}
if (pb.bbcheck)
if (!fix_problem(ctx, PR_1_BBINODE_BAD_METABLOCK_PROMPT, &pctx)) {
ctx->flags |= E2F_FLAG_ABORT;
+ e2fsck_pass1_check_unlock(ctx);
goto endit;
}
ext2fs_mark_inode_bitmap2(ctx->inode_used_map, ino);
clear_problem_context(&pctx);
FINISH_INODE_LOOP(ctx, ino, &pctx, failed_csum);
+ e2fsck_pass1_check_unlock(ctx);
continue;
} else if (ino == EXT2_ROOT_INO) {
/*
@@ -2064,6 +2109,7 @@ void _e2fsck_pass1(e2fsck_t ctx)
}
check_blocks(ctx, &pctx, block_buf, NULL);
FINISH_INODE_LOOP(ctx, ino, &pctx, failed_csum);
+ e2fsck_pass1_check_unlock(ctx);
continue;
}
if ((inode->i_links_count ||
@@ -2095,6 +2141,7 @@ void _e2fsck_pass1(e2fsck_t ctx)
}
check_blocks(ctx, &pctx, block_buf, NULL);
FINISH_INODE_LOOP(ctx, ino, &pctx, failed_csum);
+ e2fsck_pass1_check_unlock(ctx);
continue;
}
if ((inode->i_links_count ||
@@ -2137,11 +2184,13 @@ void _e2fsck_pass1(e2fsck_t ctx)
}
check_blocks(ctx, &pctx, block_buf, NULL);
FINISH_INODE_LOOP(ctx, ino, &pctx, failed_csum);
+ e2fsck_pass1_check_unlock(ctx);
continue;
}
if (!inode->i_links_count) {
FINISH_INODE_LOOP(ctx, ino, &pctx, failed_csum);
+ e2fsck_pass1_check_unlock(ctx);
continue;
}
/*
@@ -2250,12 +2299,14 @@ void _e2fsck_pass1(e2fsck_t ctx)
ctx->fs_symlinks_count++;
if (inode->i_flags & EXT4_INLINE_DATA_FL) {
FINISH_INODE_LOOP(ctx, ino, &pctx, failed_csum);
+ e2fsck_pass1_check_unlock(ctx);
continue;
} else if (ext2fs_is_fast_symlink(inode)) {
ctx->fs_fast_symlinks_count++;
check_blocks(ctx, &pctx, block_buf,
&ea_ibody_quota);
FINISH_INODE_LOOP(ctx, ino, &pctx, failed_csum);
+ e2fsck_pass1_check_unlock(ctx);
continue;
}
}
@@ -2303,16 +2354,21 @@ void _e2fsck_pass1(e2fsck_t ctx)
FINISH_INODE_LOOP(ctx, ino, &pctx, failed_csum);
- if (e2fsck_should_abort(ctx))
+ if (e2fsck_should_abort(ctx)) {
+ e2fsck_pass1_check_unlock(ctx);
goto endit;
+ }
if (process_inode_count >= ctx->process_inode_size) {
process_inodes(ctx, block_buf, inodes_to_process,
&process_inode_count);
- if (e2fsck_should_abort(ctx))
+ if (e2fsck_should_abort(ctx)) {
+ e2fsck_pass1_check_unlock(ctx);
goto endit;
+ }
}
+ e2fsck_pass1_check_unlock(ctx);
}
process_inodes(ctx, block_buf, inodes_to_process,
&process_inode_count);
@@ -3328,7 +3384,7 @@ static void e2fsck_pass1_multithread(e2fsck_t global_ctx)
struct e2fsck_thread_info *infos = NULL;
errcode_t retval;
- pthread_mutex_init(&global_ctx->fs_fix_mutex, NULL);
+ pthread_rwlock_init(&global_ctx->fs_fix_rwlock, NULL);
pthread_rwlock_init(&global_ctx->fs_block_map_rwlock, NULL);
retval = e2fsck_pass1_threads_start(&infos, global_ctx);
if (retval) {
--
2.25.4
From: Wang Shilong <[email protected]>
Change-Id: I250de2d510e3c71974f6c853d5f6ff01229d5573
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/dx_dirinfo.c | 65 +++++++++++++++++++++++++++++++++++++++++++++
e2fsck/e2fsck.h | 1 +
e2fsck/pass1.c | 25 +++++++++++++++++
3 files changed, 91 insertions(+)
diff --git a/e2fsck/dx_dirinfo.c b/e2fsck/dx_dirinfo.c
index caca3e30..ed77271b 100644
--- a/e2fsck/dx_dirinfo.c
+++ b/e2fsck/dx_dirinfo.c
@@ -79,6 +79,71 @@ void e2fsck_add_dx_dir(e2fsck_t ctx, ext2_ino_t ino, struct ext2_inode *inode,
"dx_block info array");
}
+/*
+ * Merge two sorted dir info to @dest
+ */
+void e2fsck_merge_dx_dir(e2fsck_t global_ctx, e2fsck_t thread_ctx)
+{
+ struct dx_dir_info *src_array = thread_ctx->dx_dir_info;
+ struct dx_dir_info *dest_array = global_ctx->dx_dir_info;
+ int size_dx_info = sizeof(struct dx_dir_info);
+ ext2_ino_t size = global_ctx->dx_dir_info_size;
+ ext2_ino_t src_count = thread_ctx->dx_dir_info_count;
+ ext2_ino_t dest_count = global_ctx->dx_dir_info_count;
+ ext2_ino_t total_count = src_count + dest_count;
+ struct dx_dir_info *array;
+ struct dx_dir_info *array_ptr;
+ ext2_ino_t src_index = 0, dest_index = 0;
+
+ if (thread_ctx->dx_dir_info_count == 0)
+ return;
+
+ if (size < total_count)
+ size = total_count;
+
+ array = e2fsck_allocate_memory(global_ctx, size * size_dx_info,
+ "directory map");
+ array_ptr = array;
+ /*
+ * This can be improved by binary search and memcpy, but codes
+ * would be complexer. And if the groups distributed to each
+ * thread are stided, this implementation won't be too bad comparing
+ * to the optimiztion.
+ */
+ while (src_index < src_count || dest_index < dest_count) {
+ if (src_index >= src_count) {
+ memcpy(array_ptr, &dest_array[dest_index],
+ (dest_count - dest_index) * size_dx_info);
+ break;
+ }
+ if (dest_index >= dest_count) {
+ memcpy(array_ptr, &src_array[src_index],
+ (src_count - src_index) * size_dx_info);
+ break;
+ }
+ if (src_array[src_index].ino < dest_array[dest_index].ino) {
+ *array_ptr = src_array[src_index];
+ src_index++;
+ } else {
+ /*
+ assert(src_array[src_index].ino >
+ dest_array[dest_index].ino);
+ */
+ *array_ptr = dest_array[dest_index];
+ dest_index++;
+ }
+ array_ptr++;
+ }
+
+ if (global_ctx->dx_dir_info)
+ ext2fs_free_mem(&global_ctx->dx_dir_info);
+ if (thread_ctx->dx_dir_info)
+ ext2fs_free_mem(&thread_ctx->dx_dir_info);
+ global_ctx->dx_dir_info = array;
+ global_ctx->dx_dir_info_size = size;
+ global_ctx->dx_dir_info_count = total_count;
+}
+
/*
* get_dx_dir_info() --- given an inode number, try to find the directory
* information entry for it.
diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
index ec5b0fbc..8930e278 100644
--- a/e2fsck/e2fsck.h
+++ b/e2fsck/e2fsck.h
@@ -532,6 +532,7 @@ extern int e2fsck_dir_info_get_parent(e2fsck_t ctx, ext2_ino_t ino,
ext2_ino_t *parent);
extern int e2fsck_dir_info_get_dotdot(e2fsck_t ctx, ext2_ino_t ino,
ext2_ino_t *dotdot);
+extern void e2fsck_merge_dx_dir(e2fsck_t global_ctx, e2fsck_t thread_ctx);
/* dx_dirinfo.c */
extern void e2fsck_add_dx_dir(e2fsck_t ctx, ext2_ino_t ino,
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 68b7ae26..c5107956 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -2454,6 +2454,24 @@ static void e2fsck_pass1_merge_dir_info(e2fsck_t global_ctx, e2fsck_t thread_ctx
global_ctx->dir_info);
}
+static void e2fsck_pass1_merge_dx_dir(e2fsck_t global_ctx, e2fsck_t thread_ctx)
+{
+ if (thread_ctx->dx_dir_info == NULL)
+ return;
+
+ if (global_ctx->dx_dir_info == NULL) {
+ /* TODO: tdb needs to be handled properly */
+ global_ctx->dx_dir_info = thread_ctx->dx_dir_info;
+ global_ctx->dx_dir_info_size = thread_ctx->dx_dir_info_size;
+ global_ctx->dx_dir_info_count = thread_ctx->dx_dir_info_count;
+ thread_ctx->dx_dir_info = NULL;
+ return;
+ }
+
+ e2fsck_merge_dx_dir(global_ctx, thread_ctx);
+}
+
+
#define PASS1_MERGE_CTX_ICOUNT(_dest, _src, _field) \
do { \
if (_src->_field) { \
@@ -2485,6 +2503,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
FILE *global_logf = global_ctx->logf;
FILE *global_problem_logf = global_ctx->problem_logf;
struct dir_info_db *dir_info = global_ctx->dir_info;
+ struct dx_dir_info *dx_dir_info = global_ctx->dx_dir_info;
ext2fs_inode_bitmap inode_used_map = global_ctx->inode_used_map;
ext2fs_inode_bitmap inode_dir_map = global_ctx->inode_dir_map;
ext2fs_inode_bitmap inode_bb_map = global_ctx->inode_bb_map;
@@ -2514,6 +2533,8 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
__u32 fs_fragmented = global_ctx->fs_fragmented;
__u32 fs_fragmented_dir = global_ctx->fs_fragmented_dir;
__u32 large_files = global_ctx->large_files;
+ int dx_dir_info_size = global_ctx->dx_dir_info_size;
+ int dx_dir_info_count = global_ctx->dx_dir_info_count;
#ifdef HAVE_SETJMP_H
jmp_buf old_jmp;
@@ -2537,6 +2558,10 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
global_ctx->block_metadata_map = block_metadata_map;
global_ctx->dir_info = dir_info;
e2fsck_pass1_merge_dir_info(global_ctx, thread_ctx);
+ global_ctx->dx_dir_info = dx_dir_info;
+ global_ctx->dx_dir_info_count = dx_dir_info_count;
+ global_ctx->dx_dir_info_size = dx_dir_info_size;
+ e2fsck_pass1_merge_dx_dir(global_ctx, thread_ctx);
global_ctx->inode_count = inode_count;
global_ctx->inode_link_info = inode_link_info;
PASS1_MERGE_CTX_COUNT(global_ctx, thread_ctx, fs_directory_count);
--
2.25.4
From: Wang Shilong <[email protected]>
Allow different threads to fix at the same time could
be dangerous and eror-prone now, and most of time
parallel scanning and checking is important.
So this patch try to add a mutex to serialize
fix operations during pass1.
And the good benefit of this, we don't need block
allocations and free, superblock updates protection
any more, since only fix operations during pass1
could touch them.
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/e2fsck.h | 2 +
e2fsck/pass1.c | 164 +++++++++++++++++++++++++++++++++++++++++++++---
2 files changed, 156 insertions(+), 10 deletions(-)
diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
index 8930e278..7dee2299 100644
--- a/e2fsck/e2fsck.h
+++ b/e2fsck/e2fsck.h
@@ -444,6 +444,8 @@ struct e2fsck_struct {
__u32 fs_fragmented_dir;
__u32 large_files;
__u32 extent_depth_count[MAX_EXTENT_DEPTH_COUNT];
+ /* serialize fix operation for multiple threads */
+ pthread_mutex_t fs_fix_mutex;
};
#ifdef DEBUG_THREADS
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 645666cc..87e96787 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -132,6 +132,24 @@ static void process_inodes(e2fsck_t ctx, char *block_buf,
static __u64 ext2_max_sizes[EXT2_MAX_BLOCK_LOG_SIZE -
EXT2_MIN_BLOCK_LOG_SIZE + 1];
+static void e2fsck_pass1_fix_lock(e2fsck_t ctx)
+{
+ e2fsck_t global_ctx = ctx->global_ctx;
+ if (!global_ctx)
+ global_ctx = ctx;
+
+ pthread_mutex_lock(&global_ctx->fs_fix_mutex);
+}
+
+static void e2fsck_pass1_fix_unlock(e2fsck_t ctx)
+{
+ e2fsck_t global_ctx = ctx->global_ctx;
+ if (!global_ctx)
+ global_ctx = ctx;
+
+ pthread_mutex_unlock(&global_ctx->fs_fix_mutex);
+}
+
/*
* Check to make sure a device inode is real. Returns 1 if the device
* checks out, 0 if not.
@@ -272,8 +290,10 @@ static void check_extents_inlinedata(e2fsck_t ctx,
if (!fix_problem(ctx, PR_1_SPECIAL_EXTENTS_IDATA, pctx))
return;
+ e2fsck_pass1_fix_lock(ctx);
pctx->inode->i_flags &= ~BAD_SPECIAL_FLAGS;
e2fsck_write_inode(ctx, pctx->ino, pctx->inode, "pass1");
+ e2fsck_pass1_fix_unlock(ctx);
}
#undef BAD_SPECIAL_FLAGS
@@ -290,8 +310,10 @@ static void check_immutable(e2fsck_t ctx, struct problem_context *pctx)
if (!fix_problem(ctx, PR_1_SET_IMMUTABLE, pctx))
return;
+ e2fsck_pass1_fix_lock(ctx);
pctx->inode->i_flags &= ~BAD_SPECIAL_FLAGS;
e2fsck_write_inode(ctx, pctx->ino, pctx->inode, "pass1");
+ e2fsck_pass1_fix_unlock(ctx);
}
/*
@@ -308,8 +330,10 @@ static void check_size(e2fsck_t ctx, struct problem_context *pctx)
if (!fix_problem(ctx, PR_1_SET_NONZSIZE, pctx))
return;
+ e2fsck_pass1_fix_lock(ctx);
ext2fs_inode_size_set(ctx->fs, inode, 0);
e2fsck_write_inode(ctx, pctx->ino, pctx->inode, "pass1");
+ e2fsck_pass1_fix_unlock(ctx);
}
/*
@@ -378,9 +402,11 @@ static problem_t check_large_ea_inode(e2fsck_t ctx,
if (!(inode.i_flags & EXT4_EA_INODE_FL)) {
pctx->num = entry->e_value_inum;
if (fix_problem(ctx, PR_1_ATTR_SET_EA_INODE_FL, pctx)) {
+ e2fsck_pass1_fix_lock(ctx);
inode.i_flags |= EXT4_EA_INODE_FL;
ext2fs_write_inode(ctx->fs, entry->e_value_inum,
&inode);
+ e2fsck_pass1_fix_unlock(ctx);
} else {
return PR_1_ATTR_NO_EA_INODE_FL;
}
@@ -541,11 +567,13 @@ fix:
}
/* simply remove all possible EA(s) */
+ e2fsck_pass1_fix_lock(ctx);
*((__u32 *)header) = 0UL;
e2fsck_write_inode_full(ctx, pctx->ino, pctx->inode,
EXT2_INODE_SIZE(sb), "pass1");
ea_ibody_quota->blocks = 0;
ea_ibody_quota->inodes = 0;
+ e2fsck_pass1_fix_unlock(ctx);
}
static int check_inode_extra_negative_epoch(__u32 xtime, __u32 extra) {
@@ -596,12 +624,14 @@ static void check_inode_extra_space(e2fsck_t ctx, struct problem_context *pctx,
inode->i_extra_isize & 3)) {
if (!fix_problem(ctx, PR_1_EXTRA_ISIZE, pctx))
return;
+ e2fsck_pass1_fix_lock(ctx);
if (inode->i_extra_isize < min || inode->i_extra_isize > max)
inode->i_extra_isize = sb->s_want_extra_isize;
else
inode->i_extra_isize = (inode->i_extra_isize + 3) & ~3;
e2fsck_write_inode_full(ctx, pctx->ino, pctx->inode,
EXT2_INODE_SIZE(sb), "pass1");
+ e2fsck_pass1_fix_unlock(ctx);
}
/* check if there is no place for an EA header */
@@ -630,6 +660,7 @@ static void check_inode_extra_space(e2fsck_t ctx, struct problem_context *pctx,
if (!fix_problem(ctx, PR_1_EA_TIME_OUT_OF_RANGE, pctx))
return;
+ e2fsck_pass1_fix_lock(ctx);
if (CHECK_INODE_EXTRA_NEGATIVE_EPOCH(inode, atime))
inode->i_atime_extra &= ~EXT4_EPOCH_MASK;
if (CHECK_INODE_EXTRA_NEGATIVE_EPOCH(inode, ctime))
@@ -640,6 +671,7 @@ static void check_inode_extra_space(e2fsck_t ctx, struct problem_context *pctx,
inode->i_mtime_extra &= ~EXT4_EPOCH_MASK;
e2fsck_write_inode_full(ctx, pctx->ino, pctx->inode,
EXT2_INODE_SIZE(sb), "pass1");
+ e2fsck_pass1_fix_unlock(ctx);
}
}
@@ -797,10 +829,12 @@ static void check_is_really_dir(e2fsck_t ctx, struct problem_context *pctx,
isdir:
if (fix_problem(ctx, PR_1_TREAT_AS_DIRECTORY, pctx)) {
+ e2fsck_pass1_fix_lock(ctx);
inode->i_mode = (inode->i_mode & 07777) | LINUX_S_IFDIR;
e2fsck_write_inode_full(ctx, pctx->ino, inode,
EXT2_INODE_SIZE(ctx->fs->super),
"check_is_really_dir");
+ e2fsck_pass1_fix_unlock(ctx);
}
}
@@ -871,8 +905,11 @@ static errcode_t recheck_bad_inode_checksum(ext2_filsys fs, ext2_ino_t ino,
if (!fix_problem(ctx, PR_1_INODE_ONLY_CSUM_INVALID, pctx))
return 0;
+
+ e2fsck_pass1_fix_lock(ctx);
retval = ext2fs_write_inode_full(fs, ino, (struct ext2_inode *)&inode,
sizeof(inode));
+ e2fsck_pass1_fix_unlock(ctx);
return retval;
}
@@ -882,15 +919,19 @@ static void reserve_block_for_root_repair(e2fsck_t ctx)
errcode_t err;
ext2_filsys fs = ctx->fs;
+ e2fsck_pass1_fix_lock(ctx);
ctx->root_repair_block = 0;
if (ext2fs_test_inode_bitmap2(ctx->inode_used_map, EXT2_ROOT_INO))
- return;
+ goto out;
err = ext2fs_new_block2(fs, 0, ctx->block_found_map, &blk);
if (err)
- return;
+ goto out;
ext2fs_mark_block_bitmap2(ctx->block_found_map, blk);
ctx->root_repair_block = blk;
+out:
+ e2fsck_pass1_fix_unlock(ctx);
+ return;
}
static void reserve_block_for_lnf_repair(e2fsck_t ctx)
@@ -901,15 +942,19 @@ static void reserve_block_for_lnf_repair(e2fsck_t ctx)
static const char name[] = "lost+found";
ext2_ino_t ino;
+ e2fsck_pass1_fix_lock(ctx);
ctx->lnf_repair_block = 0;
if (!ext2fs_lookup(fs, EXT2_ROOT_INO, name, sizeof(name)-1, 0, &ino))
- return;
+ goto out;
err = ext2fs_new_block2(fs, 0, ctx->block_found_map, &blk);
if (err)
- return;
+ goto out;
ext2fs_mark_block_bitmap2(ctx->block_found_map, blk);
ctx->lnf_repair_block = blk;
+out:
+ e2fsck_pass1_fix_unlock(ctx);
+ return;
}
static errcode_t get_inline_data_ea_size(ext2_filsys fs, ext2_ino_t ino,
@@ -1008,8 +1053,10 @@ static int fix_inline_data_extents_file(e2fsck_t ctx,
if (ext2fs_extent_header_verify(inode->i_block,
sizeof(inode->i_block)) == 0 &&
fix_problem(ctx, PR_1_CLEAR_INLINE_DATA_FOR_EXTENT, pctx)) {
+ e2fsck_pass1_fix_lock(ctx);
inode->i_flags &= ~EXT4_INLINE_DATA_FL;
dirty = 1;
+ e2fsck_pass1_fix_unlock(ctx);
goto out;
}
@@ -1023,8 +1070,10 @@ static int fix_inline_data_extents_file(e2fsck_t ctx,
if (EXT2_I_SIZE(inode) <
EXT4_MIN_INLINE_DATA_SIZE + max_inline_ea_size &&
fix_problem(ctx, PR_1_CLEAR_EXTENT_FOR_INLINE_DATA, pctx)) {
+ e2fsck_pass1_fix_lock(ctx);
inode->i_flags &= ~EXT4_EXTENTS_FL;
dirty = 1;
+ e2fsck_pass1_fix_unlock(ctx);
goto out;
}
@@ -1034,6 +1083,7 @@ static int fix_inline_data_extents_file(e2fsck_t ctx,
*/
if (could_be_block_map(fs, inode) &&
fix_problem(ctx, PR_1_CLEAR_EXTENT_INLINE_DATA_FLAGS, pctx)) {
+ e2fsck_pass1_fix_lock(ctx);
#ifdef WORDS_BIGENDIAN
int i;
@@ -1043,18 +1093,24 @@ static int fix_inline_data_extents_file(e2fsck_t ctx,
inode->i_flags &= ~(EXT4_EXTENTS_FL | EXT4_INLINE_DATA_FL);
dirty = 1;
+ e2fsck_pass1_fix_unlock(ctx);
goto out;
}
/* Oh well, just clear the busted inode. */
if (fix_problem(ctx, PR_1_CLEAR_EXTENT_INLINE_DATA_INODE, pctx)) {
+ e2fsck_pass1_fix_lock(ctx);
e2fsck_clear_inode(ctx, ino, inode, 0, "pass1");
+ e2fsck_pass1_fix_unlock(ctx);
return -1;
}
out:
- if (dirty)
+ if (dirty) {
+ e2fsck_pass1_fix_lock(ctx);
e2fsck_write_inode(ctx, ino, inode, "pass1");
+ e2fsck_pass1_fix_unlock(ctx);
+ }
return 0;
}
@@ -1398,7 +1454,9 @@ void _e2fsck_pass1(e2fsck_t ctx)
&pctx)) {
errcode_t err;
+ e2fsck_pass1_fix_lock(ctx);
e2fsck_clear_inode(ctx, ino, inode, 0, "pass1");
+ e2fsck_pass1_fix_unlock(ctx);
ext2fs_badblocks_list_free(ctx->fs->badblocks);
ctx->fs->badblocks = NULL;
err = ext2fs_read_bb_inode(ctx->fs,
@@ -1439,7 +1497,9 @@ void _e2fsck_pass1(e2fsck_t ctx)
inode->i_links_count > 0 &&
fix_problem(ctx, PR_1_INODE_IS_GARBAGE, &pctx)) {
pctx.errcode = 0;
+ e2fsck_pass1_fix_lock(ctx);
e2fsck_clear_inode(ctx, ino, inode, 0, "pass1");
+ e2fsck_pass1_fix_unlock(ctx);
}
failed_csum = pctx.errcode != 0;
@@ -1463,10 +1523,12 @@ void _e2fsck_pass1(e2fsck_t ctx)
if (inode->i_dtime && low_dtime_check &&
inode->i_dtime < ctx->fs->super->s_inodes_count) {
if (fix_problem(ctx, PR_1_LOW_DTIME, &pctx)) {
+ e2fsck_pass1_fix_lock(ctx);
inode->i_dtime = inode->i_links_count ?
0 : ctx->now;
e2fsck_write_inode(ctx, ino, inode,
"pass1");
+ e2fsck_pass1_fix_unlock(ctx);
failed_csum = 0;
}
}
@@ -1485,9 +1547,11 @@ void _e2fsck_pass1(e2fsck_t ctx)
if (!inode->i_dtime && inode->i_mode) {
if (fix_problem(ctx,
PR_1_ZERO_DTIME, &pctx)) {
+ e2fsck_pass1_fix_lock(ctx);
inode->i_dtime = ctx->now;
e2fsck_write_inode(ctx, ino, inode,
"pass1");
+ e2fsck_pass1_fix_unlock(ctx);
failed_csum = 0;
}
}
@@ -1524,11 +1588,15 @@ void _e2fsck_pass1(e2fsck_t ctx)
pctx.errcode = get_inline_data_ea_size(fs, ino, &size);
if (!pctx.errcode &&
fix_problem(ctx, PR_1_INLINE_DATA_FEATURE, &pctx)) {
+ e2fsck_pass1_fix_lock(ctx);
ext2fs_set_feature_inline_data(sb);
ext2fs_mark_super_dirty(fs);
+ e2fsck_pass1_fix_unlock(ctx);
inlinedata_fs = 1;
} else if (fix_problem(ctx, PR_1_INLINE_DATA_SET, &pctx)) {
+ e2fsck_pass1_fix_lock(ctx);
e2fsck_clear_inode(ctx, ino, inode, 0, "pass1");
+ e2fsck_pass1_fix_unlock(ctx);
/* skip FINISH_INODE_LOOP */
continue;
}
@@ -1570,10 +1638,12 @@ void _e2fsck_pass1(e2fsck_t ctx)
/* broken EA or no system.data EA; truncate */
if (fix_problem(ctx, PR_1_INLINE_DATA_NO_ATTR,
&pctx)) {
+ e2fsck_pass1_fix_lock(ctx);
err = ext2fs_inode_size_set(fs, inode, 0);
if (err) {
pctx.errcode = err;
ctx->flags |= E2F_FLAG_ABORT;
+ e2fsck_pass1_fix_unlock(ctx);
goto endit;
}
inode->i_flags &= ~EXT4_INLINE_DATA_FL;
@@ -1581,6 +1651,7 @@ void _e2fsck_pass1(e2fsck_t ctx)
sizeof(inode->i_block));
e2fsck_write_inode(ctx, ino, inode,
"pass1");
+ e2fsck_pass1_fix_unlock(ctx);
failed_csum = 0;
}
break;
@@ -1613,12 +1684,16 @@ void _e2fsck_pass1(e2fsck_t ctx)
if ((ext2fs_extent_header_verify(inode->i_block,
sizeof(inode->i_block)) == 0) &&
fix_problem(ctx, PR_1_EXTENT_FEATURE, &pctx)) {
+ e2fsck_pass1_fix_lock(ctx);
ext2fs_set_feature_extents(sb);
ext2fs_mark_super_dirty(fs);
extent_fs = 1;
+ e2fsck_pass1_fix_unlock(ctx);
} else if (fix_problem(ctx, PR_1_EXTENTS_SET, &pctx)) {
clear_inode:
+ e2fsck_pass1_fix_lock(ctx);
e2fsck_clear_inode(ctx, ino, inode, 0, "pass1");
+ e2fsck_pass1_fix_unlock(ctx);
if (ino == EXT2_BAD_INO)
ext2fs_mark_inode_bitmap2(ctx->inode_used_map,
ino);
@@ -1653,12 +1728,14 @@ void _e2fsck_pass1(e2fsck_t ctx)
if ((ext2fs_extent_header_verify(ehp,
sizeof(inode->i_block)) == 0) &&
(fix_problem(ctx, PR_1_UNSET_EXTENT_FL, &pctx))) {
+ e2fsck_pass1_fix_lock(ctx);
inode->i_flags |= EXT4_EXTENTS_FL;
#ifdef WORDS_BIGENDIAN
memcpy(inode->i_block, tmp_block,
sizeof(inode->i_block));
#endif
e2fsck_write_inode(ctx, ino, inode, "pass1");
+ e2fsck_pass1_fix_unlock(ctx);
failed_csum = 0;
}
}
@@ -1671,9 +1748,11 @@ void _e2fsck_pass1(e2fsck_t ctx)
(inode->i_flags & EXT4_INLINE_DATA_FL) ||
inode->i_file_acl) &&
fix_problem(ctx, PR_1_INVALID_BAD_INODE, &pctx)) {
+ e2fsck_pass1_fix_lock(ctx);
memset(inode, 0, sizeof(struct ext2_inode));
e2fsck_write_inode(ctx, ino, inode,
"clear bad inode");
+ e2fsck_pass1_fix_unlock(ctx);
failed_csum = 0;
}
@@ -1732,9 +1811,11 @@ void _e2fsck_pass1(e2fsck_t ctx)
*/
if (inode->i_dtime && inode->i_links_count) {
if (fix_problem(ctx, PR_1_ROOT_DTIME, &pctx)) {
+ e2fsck_pass1_fix_lock(ctx);
inode->i_dtime = 0;
e2fsck_write_inode(ctx, ino, inode,
"pass1");
+ e2fsck_pass1_fix_unlock(ctx);
failed_csum = 0;
}
}
@@ -1744,9 +1825,11 @@ void _e2fsck_pass1(e2fsck_t ctx)
if (!LINUX_S_ISREG(inode->i_mode) &&
fix_problem(ctx, PR_1_JOURNAL_BAD_MODE,
&pctx)) {
+ e2fsck_pass1_fix_lock(ctx);
inode->i_mode = LINUX_S_IFREG;
e2fsck_write_inode(ctx, ino, inode,
"pass1");
+ e2fsck_pass1_fix_unlock(ctx);
failed_csum = 0;
}
check_blocks(ctx, &pctx, block_buf, NULL);
@@ -1760,8 +1843,10 @@ void _e2fsck_pass1(e2fsck_t ctx)
memset(inode, 0, inode_size);
ext2fs_icount_store(ctx->inode_link_info,
ino, 0);
+ e2fsck_pass1_fix_lock(ctx);
e2fsck_write_inode_full(ctx, ino, inode,
inode_size, "pass1");
+ e2fsck_pass1_fix_unlock(ctx);
failed_csum = 0;
}
} else if (quota_inum_is_reserved(fs, ino)) {
@@ -1771,9 +1856,11 @@ void _e2fsck_pass1(e2fsck_t ctx)
if (!LINUX_S_ISREG(inode->i_mode) &&
fix_problem(ctx, PR_1_QUOTA_BAD_MODE,
&pctx)) {
+ e2fsck_pass1_fix_lock(ctx);
inode->i_mode = LINUX_S_IFREG;
e2fsck_write_inode(ctx, ino, inode,
"pass1");
+ e2fsck_pass1_fix_unlock(ctx);
failed_csum = 0;
}
check_blocks(ctx, &pctx, block_buf, NULL);
@@ -1784,11 +1871,13 @@ void _e2fsck_pass1(e2fsck_t ctx)
inode->i_blocks || inode->i_block[0]) &&
fix_problem(ctx, PR_1_QUOTA_INODE_NOT_CLEAR,
&pctx)) {
- memset(inode, 0, inode_size);
ext2fs_icount_store(ctx->inode_link_info,
ino, 0);
+ e2fsck_pass1_fix_lock(ctx);
+ memset(inode, 0, inode_size);
e2fsck_write_inode_full(ctx, ino, inode,
inode_size, "pass1");
+ e2fsck_pass1_fix_unlock(ctx);
failed_csum = 0;
}
} else if (ino < EXT2_FIRST_INODE(fs->super)) {
@@ -1808,9 +1897,11 @@ void _e2fsck_pass1(e2fsck_t ctx)
}
if (problem) {
if (fix_problem(ctx, problem, &pctx)) {
+ e2fsck_pass1_fix_lock(ctx);
inode->i_mode = 0;
e2fsck_write_inode(ctx, ino, inode,
"pass1");
+ e2fsck_pass1_fix_unlock(ctx);
failed_csum = 0;
}
}
@@ -1871,9 +1962,11 @@ void _e2fsck_pass1(e2fsck_t ctx)
ino);
} else {
if (fix_problem(ctx, PR_1_SET_IMAGIC, &pctx)) {
+ e2fsck_pass1_fix_lock(ctx);
inode->i_flags &= ~EXT2_IMAGIC_FL;
e2fsck_write_inode(ctx, ino,
inode, "pass1");
+ e2fsck_pass1_fix_unlock(ctx);
failed_csum = 0;
}
}
@@ -1890,8 +1983,10 @@ void _e2fsck_pass1(e2fsck_t ctx)
LINUX_S_ISLNK(inode->i_mode) &&
!ext2fs_inode_has_valid_blocks2(fs, inode) &&
fix_problem(ctx, PR_1_FAST_SYMLINK_EXTENT_FL, &pctx)) {
+ e2fsck_pass1_fix_lock(ctx);
inode->i_flags &= ~EXT4_EXTENTS_FL;
e2fsck_write_inode(ctx, ino, inode, "pass1");
+ e2fsck_pass1_fix_unlock(ctx);
failed_csum = 0;
}
@@ -2024,8 +2119,11 @@ void _e2fsck_pass1(e2fsck_t ctx)
ctx->ea_block_quota_inodes = 0;
}
- if (ctx->invalid_bitmaps)
+ if (ctx->invalid_bitmaps) {
+ e2fsck_pass1_fix_lock(ctx);
handle_fs_bad_blocks(ctx);
+ e2fsck_pass1_fix_unlock(ctx);
+ }
/* We don't need the block_ea_map any more */
if (ctx->block_ea_map) {
@@ -2038,7 +2136,9 @@ void _e2fsck_pass1(e2fsck_t ctx)
if (ctx->flags & E2F_FLAG_RESIZE_INODE) {
clear_problem_context(&pctx);
+ e2fsck_pass1_fix_lock(ctx);
pctx.errcode = ext2fs_create_resize_inode(fs);
+ e2fsck_pass1_fix_unlock(ctx);
if (pctx.errcode) {
if (!fix_problem(ctx, PR_1_RESIZE_INODE_CREATE,
&pctx)) {
@@ -2050,9 +2150,11 @@ void _e2fsck_pass1(e2fsck_t ctx)
if (!pctx.errcode) {
e2fsck_read_inode(ctx, EXT2_RESIZE_INO, inode,
"recreate inode");
+ e2fsck_pass1_fix_lock(ctx);
inode->i_mtime = ctx->now;
e2fsck_write_inode(ctx, EXT2_RESIZE_INO, inode,
"recreate inode");
+ e2fsck_pass1_fix_unlock(ctx);
}
ctx->flags &= ~E2F_FLAG_RESIZE_INODE;
}
@@ -2073,7 +2175,9 @@ void _e2fsck_pass1(e2fsck_t ctx)
clear_problem_context(&pctx);
fix_problem(ctx, PR_1_DUP_BLOCKS_PREENSTOP, &pctx);
}
+ e2fsck_pass1_fix_lock(ctx);
e2fsck_pass1_dupblocks(ctx, block_buf);
+ e2fsck_pass1_fix_unlock(ctx);
}
ctx->flags |= E2F_FLAG_ALLOC_OK;
endit:
@@ -2890,6 +2994,7 @@ static void e2fsck_pass1_multithread(e2fsck_t global_ctx)
unsigned flexbg_size = 1;
int max_threads;
+ pthread_mutex_init(&global_ctx->fs_fix_mutex, NULL);
if (ext2fs_has_feature_flex_bg(global_ctx->fs->super))
flexbg_size = 1 << global_ctx->fs->super->s_log_groups_per_flex;
@@ -3194,10 +3299,12 @@ static void adjust_extattr_refcount(e2fsck_t ctx, ext2_refcount_t refcount,
should_be = header->h_refcount + adjust_sign * (int)count;
pctx.num = should_be;
if (fix_problem(ctx, PR_1_EXTATTR_REFCOUNT, &pctx)) {
+ e2fsck_pass1_fix_lock(ctx);
header->h_refcount = should_be;
pctx.errcode = ext2fs_write_ext_attr3(fs, blk,
block_buf,
pctx.ino);
+ e2fsck_pass1_fix_unlock(ctx);
if (pctx.errcode) {
fix_problem(ctx, PR_1_EXTATTR_WRITE_ABORT,
&pctx);
@@ -3426,8 +3533,10 @@ static int check_ext_attr(e2fsck_t ctx, struct problem_context *pctx,
*/
if (failed_csum &&
fix_problem(ctx, PR_1_EA_BLOCK_ONLY_CSUM_INVALID, pctx)) {
+ e2fsck_pass1_fix_lock(ctx);
pctx->errcode = ext2fs_write_ext_attr3(fs, blk, block_buf,
pctx->ino);
+ e2fsck_pass1_fix_unlock(ctx);
if (pctx->errcode)
return 0;
}
@@ -3473,8 +3582,10 @@ refcount_fail:
clear_extattr:
if (region)
region_free(region);
+ e2fsck_pass1_fix_lock(ctx);
ext2fs_file_acl_block_set(fs, inode, 0);
e2fsck_write_inode(ctx, ino, inode, "check_ext_attr");
+ e2fsck_pass1_fix_unlock(ctx);
return 0;
}
@@ -3704,10 +3815,12 @@ static void scan_extent_node(e2fsck_t ctx, struct problem_context *pctx,
if (try_repairs && is_dir && problem == 0 &&
(extent.e_flags & EXT2_EXTENT_FLAGS_UNINIT) &&
fix_problem(ctx, PR_1_UNINIT_DBLOCK, pctx)) {
+ e2fsck_pass1_fix_lock(ctx);
extent.e_flags &= ~EXT2_EXTENT_FLAGS_UNINIT;
pb->inode_modified = 1;
pctx->errcode = ext2fs_extent_replace(ehandle, 0,
&extent);
+ e2fsck_pass1_fix_unlock(ctx);
if (pctx->errcode)
return;
failed_csum = 0;
@@ -3751,13 +3864,17 @@ report_problem:
}
e2fsck_read_bitmaps(ctx);
pb->inode_modified = 1;
+ e2fsck_pass1_fix_lock(ctx);
pctx->errcode =
ext2fs_extent_delete(ehandle, 0);
+ e2fsck_pass1_fix_unlock(ctx);
if (pctx->errcode) {
pctx->str = "ext2fs_extent_delete";
return;
}
+ e2fsck_pass1_fix_lock(ctx);
pctx->errcode = ext2fs_extent_fix_parents(ehandle);
+ e2fsck_pass1_fix_unlock(ctx);
if (pctx->errcode &&
pctx->errcode != EXT2_ET_NO_CURRENT_NODE) {
pctx->str = "ext2fs_extent_fix_parents";
@@ -3821,9 +3938,11 @@ report_problem:
pctx->num = e_info.curr_level - 1;
problem = PR_1_EXTENT_INDEX_START_INVALID;
if (fix_problem(ctx, problem, pctx)) {
+ e2fsck_pass1_fix_lock(ctx);
pb->inode_modified = 1;
pctx->errcode =
ext2fs_extent_fix_parents(ehandle);
+ e2fsck_pass1_fix_unlock(ctx);
if (pctx->errcode) {
pctx->str = "ext2fs_extent_fix_parents";
return;
@@ -3887,15 +4006,19 @@ report_problem:
pctx->blk = extent.e_lblk;
pctx->blk2 = new_lblk;
if (fix_problem(ctx, PR_1_COLLAPSE_DBLOCK, pctx)) {
+ e2fsck_pass1_fix_lock(ctx);
extent.e_lblk = new_lblk;
pb->inode_modified = 1;
pctx->errcode = ext2fs_extent_replace(ehandle,
0, &extent);
+ e2fsck_pass1_fix_unlock(ctx);
if (pctx->errcode) {
pctx->errcode = 0;
goto alloc_later;
}
+ e2fsck_pass1_fix_lock(ctx);
pctx->errcode = ext2fs_extent_fix_parents(ehandle);
+ e2fsck_pass1_fix_unlock(ctx);
if (pctx->errcode)
goto failed_add_dir_block;
pctx->errcode = ext2fs_extent_goto(ehandle,
@@ -3991,8 +4114,10 @@ alloc_later:
/* Failed csum but passes checks? Ask to fix checksum. */
if (failed_csum &&
fix_problem(ctx, PR_1_EXTENT_ONLY_CSUM_INVALID, pctx)) {
+ e2fsck_pass1_fix_lock(ctx);
pb->inode_modified = 1;
pctx->errcode = ext2fs_extent_replace(ehandle, 0, &extent);
+ e2fsck_pass1_fix_unlock(ctx);
if (pctx->errcode)
return;
}
@@ -4017,9 +4142,12 @@ static void check_blocks_extents(e2fsck_t ctx, struct problem_context *pctx,
eh = (struct ext3_extent_header *) &inode->i_block[0];
retval = ext2fs_extent_header_verify(eh, sizeof(inode->i_block));
if (retval) {
- if (fix_problem(ctx, PR_1_MISSING_EXTENT_HEADER, pctx))
+ if (fix_problem(ctx, PR_1_MISSING_EXTENT_HEADER, pctx)) {
+ e2fsck_pass1_fix_lock(ctx);
e2fsck_clear_inode(ctx, ino, inode, 0,
"check_blocks_extents");
+ e2fsck_pass1_fix_unlock(ctx);
+ }
pctx->errcode = 0;
return;
}
@@ -4027,9 +4155,12 @@ static void check_blocks_extents(e2fsck_t ctx, struct problem_context *pctx,
/* ...since this function doesn't fail if i_block is zeroed. */
pctx->errcode = ext2fs_extent_open2(fs, ino, inode, &ehandle);
if (pctx->errcode) {
- if (fix_problem(ctx, PR_1_READ_EXTENT, pctx))
+ if (fix_problem(ctx, PR_1_READ_EXTENT, pctx)) {
+ e2fsck_pass1_fix_lock(ctx);
e2fsck_clear_inode(ctx, ino, inode, 0,
"check_blocks_extents");
+ e2fsck_pass1_fix_unlock(ctx);
+ }
pctx->errcode = 0;
return;
}
@@ -4066,8 +4197,10 @@ static void check_blocks_extents(e2fsck_t ctx, struct problem_context *pctx,
fix_problem(ctx, PR_1_EXTENT_ITERATE_FAILURE, pctx)) {
pb->num_blocks = 0;
inode->i_blocks = 0;
+ e2fsck_pass1_fix_lock(ctx);
e2fsck_clear_inode(ctx, ino, inode, E2F_FLAG_RESTART,
"check_blocks_extents");
+ e2fsck_pass1_fix_unlock(ctx);
pctx->errcode = 0;
}
ext2fs_extent_free(ehandle);
@@ -4243,8 +4376,10 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx,
}
if (pb.clear) {
+ e2fsck_pass1_fix_lock(ctx);
e2fsck_clear_inode(ctx, ino, inode, E2F_FLAG_RESTART,
"check_blocks");
+ e2fsck_pass1_fix_unlock(ctx);
return;
}
@@ -4260,7 +4395,9 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx,
if (!pb.num_blocks && pb.is_dir &&
!(inode->i_flags & EXT4_INLINE_DATA_FL)) {
if (fix_problem(ctx, PR_1_ZERO_LENGTH_DIR, pctx)) {
+ e2fsck_pass1_fix_lock(ctx);
e2fsck_clear_inode(ctx, ino, inode, 0, "check_blocks");
+ e2fsck_pass1_fix_unlock(ctx);
ctx->fs_directory_count--;
return;
}
@@ -4339,6 +4476,7 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx,
pctx->num = (pb.last_block + 1) * fs->blocksize;
pctx->group = bad_size;
if (fix_problem(ctx, PR_1_BAD_I_SIZE, pctx)) {
+ e2fsck_pass1_fix_lock(ctx);
if (LINUX_S_ISDIR(inode->i_mode))
pctx->num &= 0xFFFFFFFFULL;
ext2fs_inode_size_set(fs, inode, pctx->num);
@@ -4349,6 +4487,7 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx,
inode->i_flags &= ~EXT4_INLINE_DATA_FL;
}
dirty_inode++;
+ e2fsck_pass1_fix_unlock(ctx);
}
pctx->num = 0;
}
@@ -4362,8 +4501,10 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx,
(inode->osd2.linux2.l_i_blocks_hi != 0)))) {
pctx->num = pb.num_blocks;
if (fix_problem(ctx, PR_1_BAD_I_BLOCKS, pctx)) {
+ e2fsck_pass1_fix_lock(ctx);
inode->i_blocks = pb.num_blocks;
inode->osd2.linux2.l_i_blocks_hi = pb.num_blocks >> 32;
+ e2fsck_pass1_fix_unlock(ctx);
dirty_inode++;
}
pctx->num = 0;
@@ -4392,8 +4533,11 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx,
e2fsck_rehash_dir_later(ctx, ino);
out:
- if (dirty_inode)
+ if (dirty_inode) {
+ e2fsck_pass1_fix_lock(ctx);
e2fsck_write_inode(ctx, ino, inode, "check_blocks");
+ e2fsck_pass1_fix_unlock(ctx);
+ }
}
#if 0
--
2.25.4
From: Wang Shilong <[email protected]>
With this patch, we no longer share ea related
refcounts globally, so that mutex lock for ea could
be dropped, this is important for Lustre, since Lustre
backend filesystem use xattrs heavily.
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/e2fsck.h | 3 +-
e2fsck/pass1.c | 310 +++++++++++++++++++++++++++++---------
lib/ext2fs/bitmaps.c | 6 +-
lib/ext2fs/blkmap64_rb.c | 20 ++-
lib/ext2fs/bmap64.h | 3 +-
lib/ext2fs/ext2fs.h | 6 +-
lib/ext2fs/gen_bitmap64.c | 15 +-
lib/ext2fs/icount.c | 5 +-
8 files changed, 277 insertions(+), 91 deletions(-)
diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
index 83be5353..153f2e21 100644
--- a/e2fsck/e2fsck.h
+++ b/e2fsck/e2fsck.h
@@ -365,6 +365,7 @@ struct e2fsck_struct {
ext2_refcount_t refcount;
ext2_refcount_t refcount_extra;
+ ext2_refcount_t refcount_orig;
/*
* Quota blocks and inodes to be charged for each ea block.
@@ -450,8 +451,6 @@ struct e2fsck_struct {
pthread_mutex_t fs_fix_mutex;
/* protect block_found_map, block_dup_map */
pthread_rwlock_t fs_block_map_rwlock;
- /* protect ea related structure */
- pthread_mutex_t fs_ea_mutex;
};
#ifdef DEBUG_THREADS
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 457c713f..ac3ffa7b 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -173,18 +173,6 @@ static inline void e2fsck_pass1_block_map_r_unlock(e2fsck_t ctx)
pthread_rwlock_unlock(&global_ctx->fs_block_map_rwlock);
}
-static inline void e2fsck_pass1_ea_lock(e2fsck_t ctx)
-{
- e2fsck_get_lock_context(ctx);
- pthread_mutex_lock(&global_ctx->fs_ea_mutex);
-}
-
-static inline void e2fsck_pass1_ea_unlock(e2fsck_t ctx)
-{
- e2fsck_get_lock_context(ctx);
- pthread_mutex_unlock(&global_ctx->fs_ea_mutex);
-}
-
/*
* Check to make sure a device inode is real. Returns 1 if the device
* checks out, 0 if not.
@@ -453,16 +441,15 @@ static void inc_ea_inode_refs(e2fsck_t ctx, struct problem_context *pctx,
struct ext2_ext_attr_entry *first, void *end)
{
struct ext2_ext_attr_entry *entry;
- e2fsck_t global_ctx = ctx->global_ctx ? ctx->global_ctx : ctx;
for (entry = first;
(void *)entry < end && !EXT2_EXT_IS_LAST_ENTRY(entry);
entry = EXT2_EXT_ATTR_NEXT(entry)) {
if (!entry->e_value_inum)
continue;
- if (!global_ctx->ea_inode_refs) {
+ if (!ctx->ea_inode_refs) {
pctx->errcode = ea_refcount_create(0,
- &global_ctx->ea_inode_refs);
+ &ctx->ea_inode_refs);
if (pctx->errcode) {
pctx->num = 4;
fix_problem(ctx, PR_1_ALLOCATE_REFCOUNT, pctx);
@@ -470,7 +457,7 @@ static void inc_ea_inode_refs(e2fsck_t ctx, struct problem_context *pctx,
return;
}
}
- ea_refcount_increment(global_ctx->ea_inode_refs,
+ ea_refcount_increment(ctx->ea_inode_refs,
entry->e_value_inum, 0);
}
}
@@ -597,10 +584,8 @@ fix:
* EA(s) in automatic fashion -bzzz
*/
if (problem == 0 || !fix_problem(ctx, problem, pctx)) {
- e2fsck_pass1_ea_lock(ctx);
inc_ea_inode_refs(ctx, pctx,
(struct ext2_ext_attr_entry *)start, end);
- e2fsck_pass1_ea_unlock(ctx);
return;
}
@@ -1414,16 +1399,6 @@ static void _e2fsck_pass1_post(e2fsck_t ctx)
ctx->refcount_extra = 0;
}
- if (ctx->ea_block_quota_blocks) {
- ea_refcount_free(ctx->ea_block_quota_blocks);
- ctx->ea_block_quota_blocks = 0;
- }
-
- if (ctx->ea_block_quota_inodes) {
- ea_refcount_free(ctx->ea_block_quota_inodes);
- ctx->ea_block_quota_inodes = 0;
- }
-
if (ctx->invalid_bitmaps)
handle_fs_bad_blocks(ctx);
@@ -2316,6 +2291,16 @@ void _e2fsck_pass1(e2fsck_t ctx)
/* We don't need the encryption policy => ID map any more */
destroy_encryption_policy_map(ctx);
+ if (ctx->ea_block_quota_blocks) {
+ ea_refcount_free(ctx->ea_block_quota_blocks);
+ ctx->ea_block_quota_blocks = 0;
+ }
+
+ if (ctx->ea_block_quota_inodes) {
+ ea_refcount_free(ctx->ea_block_quota_inodes);
+ ctx->ea_block_quota_inodes = 0;
+ }
+
if (ctx->flags & E2F_FLAG_RESTART) {
/*
* Only the master copy of the superblock and block
@@ -2373,7 +2358,8 @@ do { \
_src->_map_field = NULL; \
} else { \
_ret = ext2fs_merge_bitmap(_src->_map_field, \
- _dest->_map_field, NULL);\
+ _dest->_map_field, NULL,\
+ NULL); \
if (_ret) \
return _ret; \
} \
@@ -2390,7 +2376,8 @@ do { \
_src->_map_field = NULL; \
} else { \
_ret = ext2fs_merge_bitmap(_src->_map_field, \
- _dest->_map_field, NULL);\
+ _dest->_map_field, NULL,\
+ NULL); \
if (_ret) \
return _ret; \
} \
@@ -2855,6 +2842,157 @@ static void e2fsck_pass1_merge_quota_ctx(e2fsck_t global_ctx, e2fsck_t thread_ct
quota_release_context(&thread_ctx->qctx);
}
+static errcode_t e2fsck_pass1_merge_ea_inode_refs(e2fsck_t global_ctx,
+ e2fsck_t thread_ctx)
+{
+ ea_value_t count;
+ blk64_t blk;
+ errcode_t retval;
+
+ if (!thread_ctx->ea_inode_refs)
+ return 0;
+
+ if (!global_ctx->ea_inode_refs) {
+ global_ctx->ea_inode_refs = thread_ctx->ea_inode_refs;
+ thread_ctx->ea_inode_refs = NULL;
+ return 0;
+ }
+
+ ea_refcount_intr_begin(thread_ctx->ea_inode_refs);
+ while (1) {
+ if ((blk = ea_refcount_intr_next(thread_ctx->ea_inode_refs,
+ &count)) == 0)
+ break;
+ if (!global_ctx->block_ea_map ||
+ !ext2fs_fast_test_block_bitmap2(global_ctx->block_ea_map,
+ blk)) {
+ retval = ea_refcount_store(global_ctx->ea_inode_refs,
+ blk, count);
+ if (retval)
+ return retval;
+ }
+ }
+
+ return retval;
+}
+
+static ea_value_t ea_refcount_usage(e2fsck_t ctx, blk64_t blk,
+ ea_value_t *orig)
+{
+ ea_value_t count_cur;
+ ea_value_t count_extra = 0;
+ ea_value_t count_orig;
+
+ ea_refcount_fetch(ctx->refcount_orig, blk, &count_orig);
+ ea_refcount_fetch(ctx->refcount, blk, &count_cur);
+ /* most of time this is not needed */
+ if (ctx->refcount_extra && count_cur == 0)
+ ea_refcount_fetch(ctx->refcount_extra, blk, &count_extra);
+
+ if (!count_orig)
+ count_orig = *orig;
+ else if (orig)
+ *orig = count_orig;
+
+ return count_orig + count_extra - count_cur;
+}
+
+static errcode_t e2fsck_pass1_merge_ea_refcount(e2fsck_t global_ctx,
+ e2fsck_t thread_ctx)
+{
+ ea_value_t count;
+ blk64_t blk;
+ errcode_t retval = 0;
+
+ if (!thread_ctx->refcount)
+ return 0;
+
+ if (!global_ctx->refcount) {
+ global_ctx->refcount = thread_ctx->refcount;
+ thread_ctx->refcount = NULL;
+ global_ctx->refcount_extra = thread_ctx->refcount;
+ thread_ctx->refcount_extra = NULL;
+ return 0;
+ }
+
+ ea_refcount_intr_begin(thread_ctx->refcount);
+ while (1) {
+ if ((blk = ea_refcount_intr_next(thread_ctx->refcount,
+ &count)) == 0)
+ break;
+ /**
+ * this EA has never seen before, so just store its
+ * refcount and refcount_extra into global_ctx if needed.
+ */
+ if (!global_ctx->block_ea_map ||
+ !ext2fs_fast_test_block_bitmap2(global_ctx->block_ea_map,
+ blk)) {
+ ea_value_t extra;
+
+ retval = ea_refcount_store(global_ctx->refcount,
+ blk, count);
+ if (retval)
+ return retval;
+
+ if (count > 0 || !thread_ctx->refcount_extra)
+ continue;
+ ea_refcount_fetch(thread_ctx->refcount_extra, blk,
+ &extra);
+ if (extra == 0)
+ continue;
+
+ if (!global_ctx->refcount_extra) {
+ retval = ea_refcount_create(0,
+ &global_ctx->refcount_extra);
+ if (retval)
+ return retval;
+ }
+ retval = ea_refcount_store(global_ctx->refcount_extra,
+ blk, extra);
+ if (retval)
+ return retval;
+
+ } else {
+ ea_value_t orig;
+ ea_value_t thread_usage;
+ ea_value_t global_usage;
+ ea_value_t new;
+
+ thread_usage = ea_refcount_usage(thread_ctx,
+ blk, &orig);
+ global_usage = ea_refcount_usage(global_ctx,
+ blk, &orig);
+ if (thread_usage + global_usage <= orig) {
+ new = orig - thread_usage - global_usage;
+ retval = ea_refcount_store(global_ctx->refcount,
+ blk, new);
+ if (retval)
+ return retval;
+ continue;
+ }
+ /* update it is as zero */
+ retval = ea_refcount_store(global_ctx->refcount,
+ blk, 0);
+ if (retval)
+ return retval;
+ /* Ooops, this EA was referenced more than it stated */
+ if (!global_ctx->refcount_extra) {
+ retval = ea_refcount_create(0,
+ &global_ctx->refcount_extra);
+ if (retval)
+ return retval;
+ }
+ new = global_usage + thread_usage - orig;
+ retval = ea_refcount_store(global_ctx->refcount_extra,
+ blk, new);
+ if (retval)
+ return retval;
+ }
+ }
+
+ return retval;
+}
+
static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx)
{
errcode_t retval;
@@ -2900,6 +3038,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
int invalid_bitmaps = global_ctx->invalid_bitmaps;
ext2_refcount_t refcount = global_ctx->refcount;
ext2_refcount_t refcount_extra = global_ctx->refcount_extra;
+ ext2_refcount_t refcount_orig = global_ctx->refcount_orig;
ext2_refcount_t ea_block_quota_blocks = global_ctx->ea_block_quota_blocks;
ext2_refcount_t ea_block_quota_inodes = global_ctx->ea_block_quota_inodes;
ext2fs_block_bitmap block_ea_map = global_ctx->block_ea_map;
@@ -2935,6 +3074,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
global_ctx->inode_link_info = inode_link_info;
global_ctx->refcount = refcount;
global_ctx->refcount_extra = refcount_extra;
+ global_ctx->refcount_orig = refcount_orig;
global_ctx->ea_block_quota_blocks = ea_block_quota_blocks;
global_ctx->ea_block_quota_inodes = ea_block_quota_inodes;
global_ctx->block_ea_map = block_ea_map;
@@ -2981,15 +3121,10 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
com_err(global_ctx->program_name, 0, _("while merging dirs to hash\n"));
return retval;
}
+ e2fsck_pass1_merge_ea_inode_refs(global_ctx, thread_ctx);
+ e2fsck_pass1_merge_ea_refcount(global_ctx, thread_ctx);
global_ctx->qctx = qctx;
e2fsck_pass1_merge_quota_ctx(global_ctx, thread_ctx);
- e2fsck_pass1_block_map_w_lock(thread_ctx);
- retval = ext2fs_merge_bitmap(thread_ctx->block_found_map,
- global_ctx->block_found_map,
- global_ctx->block_dup_map);
- e2fsck_pass1_block_map_w_unlock(thread_ctx);
- if (retval == EEXIST)
- global_ctx->flags |= E2F_FLAG_DUP_BLOCK;
global_ctx->invalid_block_bitmap_flag = invalid_block_bitmap_flag;
global_ctx->invalid_inode_bitmap_flag = invalid_inode_bitmap_flag;
global_ctx->invalid_inode_table_flag = invalid_inode_table_flag;
@@ -3006,8 +3141,23 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
PASS1_MERGE_CTX_BITMAP(global_ctx, thread_ctx, inode_imagic_map);
PASS1_MERGE_CTX_BITMAP(global_ctx, thread_ctx, inode_reg_map);
PASS1_MERGE_CTX_BITMAP(global_ctx, thread_ctx, inodes_to_rebuild);
+ PASS1_MERGE_CTX_BITMAP(global_ctx, thread_ctx, block_ea_map);
- return 0;
+ /*
+ * This need be done after merging block_ea_map
+ * because ea block might be shared, we need exclude
+ * them from dup blocks.
+ */
+ retval = ext2fs_merge_bitmap(thread_ctx->block_found_map,
+ global_ctx->block_found_map,
+ global_ctx->block_dup_map,
+ global_ctx->block_ea_map);
+ if (retval == EEXIST) {
+ global_ctx->flags |= E2F_FLAG_DUP_BLOCK;
+ retval = 0;
+ }
+
+ return retval;
}
static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
@@ -3029,8 +3179,25 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
PASS1_FREE_CTX_BITMAP(thread_ctx, inode_reg_map);
PASS1_FREE_CTX_BITMAP(thread_ctx, inodes_to_rebuild);
PASS1_FREE_CTX_BITMAP(thread_ctx, block_found_map);
+ PASS1_FREE_CTX_BITMAP(thread_ctx, block_ea_map);
ext2fs_free_icount(thread_ctx->inode_count);
ext2fs_free_icount(thread_ctx->inode_link_info);
+ if (thread_ctx->refcount) {
+ ea_refcount_free(thread_ctx->refcount);
+ thread_ctx->refcount = NULL;
+ }
+ if (thread_ctx->refcount_extra) {
+ ea_refcount_free(thread_ctx->refcount_extra);
+ thread_ctx->refcount_extra = NULL;
+ }
+ if (thread_ctx->ea_inode_refs) {
+ ea_refcount_free(thread_ctx->ea_inode_refs);
+ thread_ctx->ea_inode_refs = NULL;
+ }
+ if (thread_ctx->refcount_orig) {
+ ea_refcount_free(thread_ctx->refcount_orig);
+ thread_ctx->refcount_orig = NULL;
+ }
e2fsck_free_dir_info(thread_ctx);
ext2fs_free_mem(&thread_ctx);
@@ -3233,7 +3400,6 @@ static void e2fsck_pass1_multithread(e2fsck_t global_ctx)
pthread_mutex_init(&global_ctx->fs_fix_mutex, NULL);
pthread_rwlock_init(&global_ctx->fs_block_map_rwlock, NULL);
- pthread_mutex_init(&global_ctx->fs_ea_mutex, NULL);
init_ext2_max_sizes();
retval = e2fsck_pass1_threads_start(&infos, global_ctx);
if (retval) {
@@ -3567,30 +3733,35 @@ static int check_ext_attr(e2fsck_t ctx, struct problem_context *pctx,
}
/* If ea bitmap hasn't been allocated, create it */
- e2fsck_pass1_ea_lock(ctx);
- if (!global_ctx->block_ea_map) {
+ if (!ctx->block_ea_map) {
pctx->errcode = e2fsck_allocate_block_bitmap(fs,
_("ext attr block map"),
EXT2FS_BMAP64_RBTREE, "block_ea_map",
- &global_ctx->block_ea_map);
+ &ctx->block_ea_map);
if (pctx->errcode) {
pctx->num = 2;
fix_problem(ctx, PR_1_ALLOCATE_BBITMAP_ERROR, pctx);
ctx->flags |= E2F_FLAG_ABORT;
- e2fsck_pass1_ea_unlock(ctx);
return 0;
}
}
/* Create the EA refcount structure if necessary */
- if (!global_ctx->refcount) {
+ if (!ctx->refcount) {
pctx->errcode = ea_refcount_create(0,
- &global_ctx->refcount);
+ &ctx->refcount);
+ if (pctx->errcode) {
+ pctx->num = 1;
+ fix_problem(ctx, PR_1_ALLOCATE_REFCOUNT, pctx);
+ ctx->flags |= E2F_FLAG_ABORT;
+ return 0;
+ }
+ pctx->errcode = ea_refcount_create(0,
+ &ctx->refcount_orig);
if (pctx->errcode) {
pctx->num = 1;
fix_problem(ctx, PR_1_ALLOCATE_REFCOUNT, pctx);
ctx->flags |= E2F_FLAG_ABORT;
- e2fsck_pass1_ea_unlock(ctx);
return 0;
}
}
@@ -3601,44 +3772,39 @@ static int check_ext_attr(e2fsck_t ctx, struct problem_context *pctx,
#endif
/* Have we seen this EA block before? */
- if (ext2fs_fast_test_block_bitmap2(global_ctx->block_ea_map,
+ if (ext2fs_fast_test_block_bitmap2(ctx->block_ea_map,
blk)) {
ea_block_quota->blocks = EXT2FS_C2B(fs, 1);
ea_block_quota->inodes = 0;
- if (global_ctx->ea_block_quota_blocks) {
- ea_refcount_fetch(global_ctx->ea_block_quota_blocks,
+ if (ctx->ea_block_quota_blocks) {
+ ea_refcount_fetch(ctx->ea_block_quota_blocks,
blk, "a_blocks);
if (quota_blocks)
ea_block_quota->blocks = quota_blocks;
}
- if (global_ctx->ea_block_quota_inodes)
- ea_refcount_fetch(global_ctx->ea_block_quota_inodes,
+ if (ctx->ea_block_quota_inodes)
+ ea_refcount_fetch(ctx->ea_block_quota_inodes,
blk, &ea_block_quota->inodes);
- if (ea_refcount_decrement(global_ctx->refcount,
- blk, 0) == 0) {
- e2fsck_pass1_ea_unlock(ctx);
+ if (ea_refcount_decrement(ctx->refcount,
+ blk, 0) == 0)
return 1;
- }
/* Ooops, this EA was referenced more than it stated */
- if (!global_ctx->refcount_extra) {
+ if (!ctx->refcount_extra) {
pctx->errcode = ea_refcount_create(0,
- &global_ctx->refcount_extra);
+ &ctx->refcount_extra);
if (pctx->errcode) {
pctx->num = 2;
fix_problem(ctx, PR_1_ALLOCATE_REFCOUNT, pctx);
ctx->flags |= E2F_FLAG_ABORT;
- e2fsck_pass1_ea_unlock(ctx);
return 0;
}
}
- ea_refcount_increment(global_ctx->refcount_extra, blk, 0);
- e2fsck_pass1_ea_unlock(ctx);
+ ea_refcount_increment(ctx->refcount_extra, blk, 0);
return 1;
}
- e2fsck_pass1_ea_unlock(ctx);
/*
* OK, we haven't seen this EA block yet. So we need to
@@ -3765,50 +3931,48 @@ static int check_ext_attr(e2fsck_t ctx, struct problem_context *pctx,
return 0;
}
- e2fsck_pass1_ea_lock(ctx);
if (quota_blocks != EXT2FS_C2B(fs, 1U)) {
- if (!global_ctx->ea_block_quota_blocks) {
+ if (!ctx->ea_block_quota_blocks) {
pctx->errcode = ea_refcount_create(0,
- &global_ctx->ea_block_quota_blocks);
+ &ctx->ea_block_quota_blocks);
if (pctx->errcode) {
pctx->num = 3;
goto refcount_fail;
}
}
- ea_refcount_store(global_ctx->ea_block_quota_blocks,
+ ea_refcount_store(ctx->ea_block_quota_blocks,
blk, quota_blocks);
}
if (quota_inodes) {
- if (!global_ctx->ea_block_quota_inodes) {
+ if (!ctx->ea_block_quota_inodes) {
pctx->errcode = ea_refcount_create(0,
- &global_ctx->ea_block_quota_inodes);
+ &ctx->ea_block_quota_inodes);
if (pctx->errcode) {
pctx->num = 4;
refcount_fail:
fix_problem(ctx, PR_1_ALLOCATE_REFCOUNT, pctx);
ctx->flags |= E2F_FLAG_ABORT;
- e2fsck_pass1_ea_unlock(ctx);
return 0;
}
}
- ea_refcount_store(global_ctx->ea_block_quota_inodes,
+ ea_refcount_store(ctx->ea_block_quota_inodes,
blk, quota_inodes);
}
ea_block_quota->blocks = quota_blocks;
ea_block_quota->inodes = quota_inodes;
- inc_ea_inode_refs(global_ctx, pctx, first, end);
- ea_refcount_store(global_ctx->refcount, blk, header->h_refcount - 1);
+ inc_ea_inode_refs(ctx, pctx, first, end);
+ ea_refcount_store(ctx->refcount, blk, header->h_refcount - 1);
+ ea_refcount_store(ctx->refcount_orig, blk, header->h_refcount);
/**
* It might be racy that this block has been merged in the
* global found map.
*/
if (!is_blocks_used(ctx, blk, 1))
ext2fs_fast_mark_block_bitmap2(ctx->block_found_map, blk);
- ext2fs_fast_mark_block_bitmap2(global_ctx->block_ea_map, blk);
- e2fsck_pass1_ea_unlock(ctx);
+ ext2fs_fast_mark_block_bitmap2(ctx->block_ea_map, blk);
return 1;
clear_extattr:
diff --git a/lib/ext2fs/bitmaps.c b/lib/ext2fs/bitmaps.c
index 4cd664d3..000df234 100644
--- a/lib/ext2fs/bitmaps.c
+++ b/lib/ext2fs/bitmaps.c
@@ -48,9 +48,11 @@ errcode_t ext2fs_copy_bitmap(ext2fs_generic_bitmap src,
errcode_t ext2fs_merge_bitmap(ext2fs_generic_bitmap src,
ext2fs_generic_bitmap dest,
- ext2fs_generic_bitmap dup)
+ ext2fs_generic_bitmap dup,
+ ext2fs_generic_bitmap dup_allowed)
{
- return ext2fs_merge_generic_bmap(src, dest, dup);
+ return ext2fs_merge_generic_bmap(src, dest, dup,
+ dup_allowed);
}
void ext2fs_set_bitmap_padding(ext2fs_generic_bitmap map)
diff --git a/lib/ext2fs/blkmap64_rb.c b/lib/ext2fs/blkmap64_rb.c
index 2337302f..3ec4f4de 100644
--- a/lib/ext2fs/blkmap64_rb.c
+++ b/lib/ext2fs/blkmap64_rb.c
@@ -970,7 +970,8 @@ static void rb_print_stats(ext2fs_generic_bitmap_64 bitmap EXT2FS_ATTR((unused))
static errcode_t rb_merge_bmap(ext2fs_generic_bitmap_64 src,
ext2fs_generic_bitmap_64 dest,
- ext2fs_generic_bitmap_64 dup)
+ ext2fs_generic_bitmap_64 dup,
+ ext2fs_generic_bitmap_64 dup_allowed)
{
struct ext2fs_rb_private *src_bp, *dest_bp, *dup_bp = NULL;
struct bmap_rb_extent *src_ext;
@@ -1004,9 +1005,20 @@ static errcode_t rb_merge_bmap(ext2fs_generic_bitmap_64 src,
if (retval) {
rb_insert_extent(i, 1, dest_bp);
} else {
- if (dup_bp)
- rb_insert_extent(i, 1, dup_bp);
- dup_found = 1;
+ if (dup_allowed) {
+ retval = rb_test_clear_bmap_extent(dup_allowed,
+ i + src->start, 1);
+ /* not existed in dup_allowed */
+ if (retval) {
+ dup_found = 1;
+ if (dup_bp)
+ rb_insert_extent(i, 1, dup_bp);
+ } /* else we conside it not duplicated */
+ } else {
+ if (dup_bp)
+ rb_insert_extent(i, 1, dup_bp);
+ dup_found = 1;
+ }
}
}
}
diff --git a/lib/ext2fs/bmap64.h b/lib/ext2fs/bmap64.h
index 68a4bb0a..555193ee 100644
--- a/lib/ext2fs/bmap64.h
+++ b/lib/ext2fs/bmap64.h
@@ -74,7 +74,8 @@ struct ext2_bitmap_ops {
ext2fs_generic_bitmap_64 dest);
errcode_t (*merge_bmap)(ext2fs_generic_bitmap_64 src,
ext2fs_generic_bitmap_64 dest,
- ext2fs_generic_bitmap_64 dup);
+ ext2fs_generic_bitmap_64 dup,
+ ext2fs_generic_bitmap_64 dup_allowed);
errcode_t (*resize_bmap)(ext2fs_generic_bitmap_64 bitmap,
__u64 new_end,
__u64 new_real_end);
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index 38ae2dee..44e569e6 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -842,7 +842,8 @@ extern errcode_t ext2fs_copy_bitmap(ext2fs_generic_bitmap src,
extern errcode_t ext2fs_merge_bitmap(ext2fs_generic_bitmap src,
ext2fs_generic_bitmap dest,
- ext2fs_generic_bitmap dup);
+ ext2fs_generic_bitmap dup,
+ ext2fs_generic_bitmap dup_allowed);
extern errcode_t ext2fs_write_inode_bitmap(ext2_filsys fs);
extern errcode_t ext2fs_write_block_bitmap (ext2_filsys fs);
extern errcode_t ext2fs_read_inode_bitmap (ext2_filsys fs);
@@ -1442,7 +1443,8 @@ errcode_t ext2fs_resize_generic_bmap(ext2fs_generic_bitmap bmap,
__u64 new_real_end);
errcode_t ext2fs_merge_generic_bmap(ext2fs_generic_bitmap gen_src,
ext2fs_generic_bitmap gen_dest,
- ext2fs_generic_bitmap gen_dup);
+ ext2fs_generic_bitmap gen_dup,
+ ext2fs_generic_bitmap dup_allowed);
errcode_t ext2fs_compare_generic_bmap(errcode_t neq,
ext2fs_generic_bitmap bm1,
ext2fs_generic_bitmap bm2);
diff --git a/lib/ext2fs/gen_bitmap64.c b/lib/ext2fs/gen_bitmap64.c
index c27a52e4..a8f8fde2 100644
--- a/lib/ext2fs/gen_bitmap64.c
+++ b/lib/ext2fs/gen_bitmap64.c
@@ -341,31 +341,36 @@ errcode_t ext2fs_copy_generic_bmap(ext2fs_generic_bitmap gen_src,
errcode_t ext2fs_merge_generic_bmap(ext2fs_generic_bitmap gen_src,
ext2fs_generic_bitmap gen_dest,
- ext2fs_generic_bitmap gen_dup)
+ ext2fs_generic_bitmap gen_dup,
+ ext2fs_generic_bitmap gen_dup_allowed)
{
ext2fs_generic_bitmap_64 src = (ext2fs_generic_bitmap_64) gen_src;
ext2fs_generic_bitmap_64 dest = (ext2fs_generic_bitmap_64) gen_dest;
ext2fs_generic_bitmap_64 dup = (ext2fs_generic_bitmap_64) gen_dup;
+ ext2fs_generic_bitmap_64 dup_allowed = (ext2fs_generic_bitmap_64) gen_dup_allowed;
if (!src || !dest)
return EINVAL;
if (!EXT2FS_IS_64_BITMAP(src) || !EXT2FS_IS_64_BITMAP(dest) ||
- (dup && !EXT2FS_IS_64_BITMAP(dup)))
+ (dup && !EXT2FS_IS_64_BITMAP(dup)) || (dup_allowed &&
+ !EXT2FS_IS_64_BITMAP(dup_allowed)))
return EINVAL;
if (src->bitmap_ops != dest->bitmap_ops ||
- (dup && src->bitmap_ops != dup->bitmap_ops))
+ (dup && src->bitmap_ops != dup->bitmap_ops) ||
+ (dup_allowed && src->bitmap_ops != dup_allowed->bitmap_ops))
return EINVAL;
if (src->cluster_bits != dest->cluster_bits ||
- (dup && dup->cluster_bits != src->cluster_bits))
+ (dup && dup->cluster_bits != src->cluster_bits) ||
+ (dup_allowed && dup->cluster_bits != dup_allowed->cluster_bits))
return EINVAL;
if (src->bitmap_ops->merge_bmap == NULL)
return EOPNOTSUPP;
- return src->bitmap_ops->merge_bmap(src, dest, dup);
+ return src->bitmap_ops->merge_bmap(src, dest, dup, dup_allowed);
}
errcode_t ext2fs_resize_generic_bmap(ext2fs_generic_bitmap gen_bmap,
diff --git a/lib/ext2fs/icount.c b/lib/ext2fs/icount.c
index 7ec490dc..8fe6ff4e 100644
--- a/lib/ext2fs/icount.c
+++ b/lib/ext2fs/icount.c
@@ -786,13 +786,14 @@ errcode_t ext2fs_icount_merge(ext2_icount_t src, ext2_icount_t dest)
return ext2fs_icount_merge_full_map(src, dest);
retval = ext2fs_merge_bitmap(src->single,
- dest->single, NULL);
+ dest->single, NULL, NULL);
if (retval)
return retval;
if (src->multiple) {
retval = ext2fs_merge_bitmap(src->multiple,
- dest->multiple, NULL);
+ dest->multiple, NULL,
+ NULL);
if (retval)
return retval;
}
--
2.25.4
From: Wang Shilong <[email protected]>
Change-Id: I19138835c8532eab8f91b711ba25300b33329902
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/pass1.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 06e7d753..dc710e4d 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -3006,6 +3006,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
ext2fs_inode_bitmap inode_imagic_map = global_ctx->inode_imagic_map;
ext2fs_inode_bitmap inode_reg_map = global_ctx->inode_reg_map;
ext2fs_block_bitmap inodes_to_rebuild = global_ctx->inodes_to_rebuild;
+ ext2fs_inode_bitmap inode_bad_map = global_ctx->inode_bad_map;
ext2_icount_t inode_count = global_ctx->inode_count;
ext2_icount_t inode_link_info = global_ctx->inode_link_info;
__u32 fs_directory_count = global_ctx->fs_directory_count;
@@ -3062,6 +3063,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
global_ctx->inode_reg_map = inode_reg_map;
global_ctx->block_dup_map = block_dup_map;
global_ctx->block_found_map = block_found_map;
+ global_ctx->inode_bad_map = inode_bad_map;
global_ctx->dir_info = dir_info;
e2fsck_pass1_merge_dir_info(global_ctx, thread_ctx);
global_ctx->dx_dir_info = dx_dir_info;
@@ -3141,6 +3143,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
* so please do NOT leave any garbage behind after returning.
*/
PASS1_MERGE_CTX_BITMAP(global_ctx, thread_ctx, inode_used_map);
+ PASS1_MERGE_CTX_BITMAP(global_ctx, thread_ctx, inode_bad_map);
PASS1_MERGE_CTX_BITMAP(global_ctx, thread_ctx, inode_dir_map);
PASS1_MERGE_CTX_BITMAP(global_ctx, thread_ctx, inode_bb_map);
PASS1_MERGE_CTX_BITMAP(global_ctx, thread_ctx, inode_imagic_map);
@@ -3185,6 +3188,7 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
PASS1_FREE_CTX_BITMAP(thread_ctx, inodes_to_rebuild);
PASS1_FREE_CTX_BITMAP(thread_ctx, block_found_map);
PASS1_FREE_CTX_BITMAP(thread_ctx, block_ea_map);
+ PASS1_FREE_CTX_BITMAP(thread_ctx, inode_bad_map);
ext2fs_free_icount(thread_ctx->inode_count);
ext2fs_free_icount(thread_ctx->inode_link_info);
if (thread_ctx->refcount) {
--
2.25.4
From: Wang Shilong <[email protected]>
tests covered by f_extent_htree.
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/pass1.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 3d6af9fb..d56b7128 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -2991,6 +2991,7 @@ static errcode_t e2fsck_pass1_merge_context(e2fsck_t global_ctx,
e2fsck_t thread_ctx)
{
errcode_t retval;
+ int i;
global_ctx->fs_directory_count += thread_ctx->fs_directory_count;
global_ctx->fs_regular_count += thread_ctx->fs_regular_count;
@@ -3018,6 +3019,11 @@ static errcode_t e2fsck_pass1_merge_context(e2fsck_t global_ctx,
*/
global_ctx->lost_and_found = 0;
+ /* merge extent depth count */
+ for (i = 0; i < MAX_EXTENT_DEPTH_COUNT; i++)
+ global_ctx->extent_depth_count[i] +=
+ thread_ctx->extent_depth_count[i];
+
e2fsck_pass1_merge_dir_info(global_ctx, thread_ctx);
e2fsck_pass1_merge_dx_dir(global_ctx, thread_ctx);
retval = e2fsck_pass1_merge_fs(global_ctx->fs, thread_ctx->fs);
--
2.25.4
From: Wang Shilong <[email protected]>
e2fsck init memory according to filesystem inodes/dir numbers
recorded in the superblock, this should be aware of filesystem
number of threads, otherwise, oom happen.
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/pass1.c | 1 +
lib/ext2fs/dblist.c | 2 ++
lib/ext2fs/ext2fs.h | 3 ++-
lib/ext2fs/icount.c | 4 ++++
4 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 990e4b04..ca8d30c4 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -1315,6 +1315,7 @@ static void e2fsck_pass1_set_thread_num(e2fsck_t ctx)
num_threads = max_threads / times;
}
ctx->fs_num_threads = num_threads;
+ ctx->fs->fs_num_threads = num_threads;
}
static void init_ext2_max_sizes()
diff --git a/lib/ext2fs/dblist.c b/lib/ext2fs/dblist.c
index 046b1e68..55e58306 100644
--- a/lib/ext2fs/dblist.c
+++ b/lib/ext2fs/dblist.c
@@ -58,6 +58,8 @@ static errcode_t make_dblist(ext2_filsys fs, ext2_ino_t size,
if (retval)
goto cleanup;
dblist->size = (num_dirs * 2) + 12;
+ if (fs->fs_num_threads)
+ dblist->size /= fs->fs_num_threads;
}
len = (size_t) sizeof(struct ext2_db_entry2) * dblist->size;
dblist->count = count;
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index 44e569e6..9e77f6d1 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -255,10 +255,11 @@ struct struct_ext2_filsys {
int cluster_ratio_bits;
__u16 default_bitmap_type;
__u16 pad;
+ __u32 fs_num_threads;
/*
* Reserved for future expansion
*/
- __u32 reserved[5];
+ __u32 reserved[4];
/*
* Reserved for the use of the calling application.
diff --git a/lib/ext2fs/icount.c b/lib/ext2fs/icount.c
index 8fe6ff4e..25ec75c2 100644
--- a/lib/ext2fs/icount.c
+++ b/lib/ext2fs/icount.c
@@ -237,6 +237,8 @@ errcode_t ext2fs_create_icount_tdb(ext2_filsys fs EXT2FS_NO_TDB_UNUSED,
* value.
*/
num_inodes = fs->super->s_inodes_count - fs->super->s_free_inodes_count;
+ if (fs->fs_num_threads)
+ num_inodes /= fs->fs_num_threads;
icount->tdb = tdb_open(fn, num_inodes, TDB_NOLOCK | TDB_NOSYNC,
O_RDWR | O_CREAT | O_TRUNC, 0600);
@@ -288,6 +290,8 @@ errcode_t ext2fs_create_icount2(ext2_filsys fs, int flags, unsigned int size,
if (retval)
goto errout;
icount->size += fs->super->s_inodes_count / 50;
+ if (fs->fs_num_threads)
+ icount->size /= fs->fs_num_threads;
}
bytes = (size_t) (icount->size * sizeof(struct ext2_icount_el));
--
2.25.4
From: Wang Shilong <[email protected]>
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/pass1.c | 22 ++++++++--------------
1 file changed, 8 insertions(+), 14 deletions(-)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 52af4f13..990e4b04 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -2513,6 +2513,7 @@ static errcode_t e2fsck_pass1_copy_fs(ext2_filsys dest, e2fsck_t src_context,
memcpy(dest, src, sizeof(struct struct_ext2_filsys));
dest->inode_map = NULL;
dest->block_map = NULL;
+ dest->icache = NULL;
/*
* PASS1_COPY_FS_BITMAP might return directly from this function,
@@ -2521,13 +2522,6 @@ static errcode_t e2fsck_pass1_copy_fs(ext2_filsys dest, e2fsck_t src_context,
PASS1_COPY_FS_BITMAP(dest, src, inode_map);
PASS1_COPY_FS_BITMAP(dest, src, block_map);
- /* icache will be rebuilt if needed, so do not copy from @src */
- if (src->icache) {
- ext2fs_free_inode_cache(src->icache);
- src->icache = NULL;
- }
- dest->icache = NULL;
-
if (src->dblist) {
retval = ext2fs_copy_dblist(src->dblist, &dest->dblist);
if (retval)
@@ -2593,6 +2587,7 @@ static int _e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
ext2_dblist dblist;
int flags;
e2fsck_t dest_ctx = dest->priv_data;
+ struct ext2_inode_cache *icache = dest->icache;
dest_io = dest->io;
dest_image_io = dest->image_io;
@@ -2610,6 +2605,7 @@ static int _e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
dest->dblist = dblist;
dest->priv_data = dest_ctx;
dest->flags = src->flags | flags;
+ dest->icache = icache;
if (!(src->flags & EXT2_FLAG_VALID) || !(flags & EXT2_FLAG_VALID))
ext2fs_unmark_valid(dest);
/*
@@ -2654,17 +2650,15 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
retval = _e2fsck_pass1_merge_fs(dest, src);
- if (src->inode_map)
- ext2fs_free_generic_bmap(src->inode_map);
- if (src->block_map)
- ext2fs_free_generic_bmap(src->block_map);
-
- /* icache will be rebuilt if needed, so do not copy from @src */
if (src->icache) {
ext2fs_free_inode_cache(src->icache);
src->icache = NULL;
}
- dest->icache = NULL;
+
+ if (src->inode_map)
+ ext2fs_free_generic_bmap(src->inode_map);
+ if (src->block_map)
+ ext2fs_free_generic_bmap(src->block_map);
if (src->dblist) {
ext2fs_free_dblist(src->dblist);
--
2.25.4
From: Wang Shilong <[email protected]>
This should not be kept, the reaons is similar to what
e2fsck_pass1 has done before.
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/pass1.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index b4adb8fa..3d6af9fb 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -3012,6 +3012,12 @@ static errcode_t e2fsck_pass1_merge_context(e2fsck_t global_ctx,
global_ctx->flags |= thread_ctx->flags;
/* threads might enable E2F_OPT_YES */
global_ctx->options |= thread_ctx->options;
+ /*
+ * The l+f inode may have been cleared, so zap it now and
+ * later passes will recalculate it if necessary
+ */
+ global_ctx->lost_and_found = 0;
+
e2fsck_pass1_merge_dir_info(global_ctx, thread_ctx);
e2fsck_pass1_merge_dx_dir(global_ctx, thread_ctx);
retval = e2fsck_pass1_merge_fs(global_ctx->fs, thread_ctx->fs);
--
2.25.4
From: Li Xi <[email protected]>
Signed-off-by: Li Xi <[email protected]>
Signed-off-by: Wang Shilong <[email protected]>
---
e2fsck/pass1.c | 36 +++++++++++++++-
lib/ext2fs/ext2fs.h | 1 +
lib/ext2fs/icount.c | 101 ++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 137 insertions(+), 1 deletion(-)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index e343ec00..3c04edfd 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -2435,6 +2435,28 @@ static void e2fsck_pass1_merge_dir_info(e2fsck_t global_ctx, e2fsck_t thread_ctx
global_ctx->dir_info);
}
+#define PASS1_MERGE_CTX_ICOUNT(_dest, _src, _field) \
+do { \
+ if (_src->_field) { \
+ if (_dest->_field == NULL) { \
+ _dest->_field = _src->_field; \
+ _src->_field = NULL; \
+ } else { \
+ errcode_t _ret; \
+ _ret = ext2fs_icount_merge(_src->_field, _dest->_field); \
+ if (_ret) \
+ return _ret; \
+ } \
+ } \
+} while (0)
+
+static errcode_t e2fsck_pass1_merge_icounts(e2fsck_t global_ctx, e2fsck_t thread_ctx)
+{
+ PASS1_MERGE_CTX_ICOUNT(global_ctx, thread_ctx, inode_count);
+ PASS1_MERGE_CTX_ICOUNT(global_ctx, thread_ctx, inode_link_info);
+ return 0;
+}
+
static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx)
{
errcode_t retval;
@@ -2454,7 +2476,9 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
ext2fs_block_bitmap block_ea_map = global_ctx->block_ea_map;
ext2fs_block_bitmap block_metadata_map = global_ctx->block_metadata_map;
ext2fs_block_bitmap inodes_to_rebuild = global_ctx->inodes_to_rebuild;
-
+ ext2_icount_t inode_count = global_ctx->inode_count;
+ ext2_icount_t inode_link_info = global_ctx->inode_link_info;
+
#ifdef HAVE_SETJMP_H
jmp_buf old_jmp;
@@ -2477,6 +2501,8 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
global_ctx->block_metadata_map = block_metadata_map;
global_ctx->dir_info = dir_info;
e2fsck_pass1_merge_dir_info(global_ctx, thread_ctx);
+ global_ctx->inode_count = inode_count;
+ global_ctx->inode_link_info = inode_link_info;
/* Keep the global singal flags*/
global_ctx->flags |= (flags & E2F_FLAG_SIGNAL_MASK) |
@@ -2492,6 +2518,12 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
global_ctx->logf = global_logf;
global_ctx->problem_logf = global_problem_logf;
global_ctx->global_ctx = NULL;
+ retval = e2fsck_pass1_merge_icounts(global_ctx, thread_ctx);
+ if (retval) {
+ com_err(global_ctx->program_name, 0,
+ _("while merging icounts\n"));
+ return retval;
+ }
/*
* PASS1_COPY_CTX_BITMAP might return directly from this function,
@@ -2533,6 +2565,8 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
PASS1_FREE_CTX_BITMAP(thread_ctx, block_dup_map);
PASS1_FREE_CTX_BITMAP(thread_ctx, block_ea_map);
PASS1_FREE_CTX_BITMAP(thread_ctx, block_metadata_map);
+ ext2fs_free_icount(thread_ctx->inode_count);
+ ext2fs_free_icount(thread_ctx->inode_link_info);
e2fsck_free_dir_info(thread_ctx);
ext2fs_free_mem(&thread_ctx);
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index bdb72251..5a094da3 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -1509,6 +1509,7 @@ extern errcode_t ext2fs_icount_decrement(ext2_icount_t icount, ext2_ino_t ino,
__u16 *ret);
extern errcode_t ext2fs_icount_store(ext2_icount_t icount, ext2_ino_t ino,
__u16 count);
+extern errcode_t ext2fs_icount_merge(ext2_icount_t src, ext2_icount_t dest);
extern ext2_ino_t ext2fs_get_icount_size(ext2_icount_t icount);
errcode_t ext2fs_icount_validate(ext2_icount_t icount, FILE *);
diff --git a/lib/ext2fs/icount.c b/lib/ext2fs/icount.c
index 888a90b2..729f993f 100644
--- a/lib/ext2fs/icount.c
+++ b/lib/ext2fs/icount.c
@@ -13,6 +13,7 @@
#if HAVE_UNISTD_H
#include <unistd.h>
#endif
+#include <assert.h>
#include <string.h>
#include <stdio.h>
#include <sys/stat.h>
@@ -701,6 +702,106 @@ errcode_t ext2fs_icount_store(ext2_icount_t icount, ext2_ino_t ino,
return 0;
}
+errcode_t ext2fs_icount_merge_full_map(ext2_icount_t src, ext2_icount_t dest)
+{
+ /* TODO: add the support for full map */
+ return EOPNOTSUPP;
+}
+
+errcode_t ext2fs_icount_merge_el(ext2_icount_t src, ext2_icount_t dest)
+{
+ int src_count = src->count;
+ int dest_count = dest->count;
+ int size = src_count + dest_count;
+ int size_entry = sizeof(struct ext2_icount_el);
+ struct ext2_icount_el *array;
+ struct ext2_icount_el *array_ptr;
+ struct ext2_icount_el *src_array = src->list;
+ struct ext2_icount_el *dest_array = dest->list;
+ int src_index = 0;
+ int dest_index = 0;
+ errcode_t retval;
+
+ if (src_count == 0)
+ return 0;
+
+ retval = ext2fs_get_array(size, size_entry, &array);
+ if (retval)
+ return retval;
+
+ array_ptr = array;
+ /*
+ * This can be improved by binary search and memcpy, but codes would
+ * be complexer. And if number of bad blocks is small, the optimization
+ * won't improve performance a lot.
+ */
+ while (src_index < src_count || dest_index < dest_count) {
+ if (src_index >= src_count) {
+ memcpy(array_ptr, &dest_array[dest_index],
+ (dest_count - dest_index) * size_entry);
+ break;
+ }
+ if (dest_index >= dest_count) {
+ memcpy(array_ptr, &src_array[src_index],
+ (src_count - src_index) * size_entry);
+ break;
+ }
+ if (src_array[src_index].ino < dest_array[dest_index].ino) {
+ *array_ptr = src_array[src_index];
+ src_index++;
+ } else {
+ assert(src_array[src_index].ino >
+ dest_array[dest_index].ino);
+ *array_ptr = dest_array[dest_index];
+ dest_index++;
+ }
+ array_ptr++;
+ }
+
+ ext2fs_free_mem(&dest->list);
+ dest->list = array;
+ dest->count = src_count + dest_count;
+ dest->size = size;
+ dest->last_lookup = NULL;
+ return 0;
+}
+
+errcode_t ext2fs_icount_merge(ext2_icount_t src, ext2_icount_t dest)
+{
+ errcode_t retval;
+
+ if (src->fullmap && !dest->fullmap)
+ return EINVAL;
+
+ if (!src->fullmap && dest->fullmap)
+ return EINVAL;
+
+ if (src->multiple && !dest->multiple)
+ return EINVAL;
+
+ if (!src->multiple && dest->multiple)
+ return EINVAL;
+
+ if (src->fullmap)
+ return ext2fs_icount_merge_full_map(src, dest);
+
+ retval = ext2fs_merge_bitmap(src->single, dest->single);
+ if (retval)
+ return retval;
+
+ if (src->multiple) {
+ retval = ext2fs_merge_bitmap(src->multiple, dest->multiple);
+ if (retval)
+ return retval;
+ }
+
+ retval = ext2fs_icount_merge_el(src, dest);
+ if (retval)
+ return retval;
+
+ return 0;
+}
+
ext2_ino_t ext2fs_get_icount_size(ext2_icount_t icount)
{
if (!icount || icount->magic != EXT2_ET_MAGIC_ICOUNT)
--
2.25.4