Hello,
I've updated the ext4 online defragmentation patches.
I've addressed the review comment from Mingming, and there are some cleanups
but no functional changes since the previous version.
Changelog:
- 0.9 (May 30, 2008)
- Create some new functions (ext4_defrag_fill_ar(),
ext4_defrag_check_phase() ...) to separate block allocation function,
since the phase mode plug into the allocation function isn't good.
- Add the description of ext4_defrag() which is main function.
- Add the capability check.
- Some cleanups.
- 0.8 (April 8, 2008)
Fix sparse warnings and change the construction of patches.
- 0.7 (January 10, 2008)
Interchange the data blocks of the target and temporary files
in an atomic manner.
Outline for ext4 online defragmentation:
Ext4 online defrag has the following three functions.
no option Solving a single file fragmentation.
Single file fragmentation is solved by moving file
data to contiguous free blocks.
-r Solving a relevant file fragmentation.
Relevant file fragmentation could be solved by moving
the files under the specified directory close together with
the block containing the directory data.
-f Solving a free space fragmentation.
If there is no contiguous free blocks in the filesystem,
the other files are moved to make sufficient space to allocate
contiguous blocks for the target file.
Notes:
- Ext4 online defarg needs "mballoc" and "extents" mount options,
and it only supports 4KB block size (It will go away soon).
- Current mballoc in the ext4 patch queue (linux-2.6.26-rc4) doesn't
work fine, ext4_mb_init_caches() which is changed by
ext4-add-error-handling-in-ext4-mb_laod_buddy.patch returns -EIO.
You can reproduce this kind of issue easily with following procedure.
1. mke2fs -j -E test_fs /devname
2. mount -t ext4dev -o mballoc,extents /devname /mountpoint
3. mkdir DIR /mountpint
mkdir: cannot create directory `/mnt/mp2/DIR1': No space left on device
Next steps:
1. Implement FS_IOC_FIEMAP instead of EXT4_EXTENTS_INFO to reduce ioctl count.
2. Remove the restriction of block size (Enable defrag with 1KB and 2KB).
3. Remove the limit of target file size (now 128MB) in -f mode.
* The post about -f mode can be found at:
http://marc.info/?l=linux-ext4&m=118239067704899&w=4
Summary of patches:
* The following patches are new ext4 online defrag patches.
Mingming, please replace these patches with the ext4 patch queue's.
[PATCH 1/8] Main function of ext4 online defrag and ioctl implementation
- Create the temporary inode and do defrag per
defrag_size (defalut 64MB).
[PATCH 2/8] Allocate new contiguous blocks with mballoc
- Search contiguous free blocks with mutil-block allocation
and allocate them for the temporary inode.
[PATCH 3/8] Read and write file data with memory page
- Read the file data from the old blocks to the page cache and
write the file data on the page into the new blocks.
[PATCH 4/8] Exchange the blocks between two inodes
- Exchange the data blocks between the temporary inode and
the original inode.
[PATCH 5/8] Defragmentation for the relevant files (-r mode)
- Relevant file fragmentation could be solved by moving
the files under the specified directory close together with
the block containing the directory data.
[PATCH 6/8] Check the free space fragmentation (-f mode)
- Check the free space fragmentation in the block group
where target file is located.
[PATCH 7/8] Move victim files for the target file (-f mode)
- Move victim files to make sufficient space and reallocate
the contiguous blocks for the target file.
[PATCH 8/8] Online defrag command
- The defrag command. Usage is as follows:
- Defrag for a single file.
# e4defrag file-name
- Defrag for all files on ext4.
# e4defrag device-name
- Put the multiple files closer together.
# e4defrag -r directory-name
- Defrag for free space fragmentation.
# e4defrag -f file-name
Performance test:
I created 1GB files with parallel writing then ran the e4defrag to it.
As a result, I got the following improvement.
<Before> <After>
--------------------------------------------------------------------
Fragments 127 -> 20
I/O performance(sec) 40.3 -> 36 (10% improved)
Kernel: 2.6.26-rc4 + ext4 patch queue Arch: x86_64
Mount options: extents, mballoc, delalloc, data=writeback
* "Fragments" is the number of fragments in 1GB file shown by filefrag.
"I/O performance" is the execution time for reading 1GB file
with "cat" command (cat file* > /dev/null).
Any comments are welcome.
Regards,
Akira Fujita
Thanks for the update.
On Fri, 2008-05-30 at 20:17 +0900, Akira Fujita wrote:
> - Current mballoc in the ext4 patch queue (linux-2.6.26-rc4) doesn't
> work fine, ext4_mb_init_caches() which is changed by
> ext4-add-error-handling-in-ext4-mb_laod_buddy.patch returns -EIO.
> You can reproduce this kind of issue easily with following
> procedure.
> 1. mke2fs -j -E test_fs /devname
> 2. mount -t ext4dev -o mballoc,extents /devname /mountpoint
> 3. mkdir DIR /mountpint
> mkdir: cannot create directory `/mnt/mp2/DIR1': No space left on
> device
>
>
Ccing Shen Feng for this bug...
> Next steps:
> 1. Implement FS_IOC_FIEMAP instead of EXT4_EXTENTS_INFO to reduce ioctl count.
> 2. Remove the restriction of block size (Enable defrag with 1KB and 2KB).
> 3. Remove the limit of target file size (now 128MB) in -f mode.
> * The post about -f mode can be found at:
> http://marc.info/?l=linux-ext4&m=118239067704899&w=4
>
> Summary of patches:
> * The following patches are new ext4 online defrag patches.
> Mingming, please replace these patches with the ext4 patch queue's.
>
Done.
Regards,
Mingming
ext4: Fix ext4_mb_init_cache return error
From: Mingming Cao <[email protected]>
ext4_mb_init_cache() incorrectly always return EIO on success. This
causes the caller of ext4_mb_init_cache() fail when it checks
the return value.
Signed-off-by: Mingming Cao <[email protected]>
---
fs/ext4/mballoc.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
Index: linux-2.6.26-rc4/fs/ext4/mballoc.c
===================================================================
--- linux-2.6.26-rc4.orig/fs/ext4/mballoc.c 2008-06-01 12:55:19.000000000 -0700
+++ linux-2.6.26-rc4/fs/ext4/mballoc.c 2008-06-01 12:55:23.000000000 -0700
@@ -798,7 +798,7 @@ static int ext4_mb_init_cache(struct pag
for (i = 0; i < groups_per_page && bh[i]; i++)
wait_on_buffer(bh[i]);
- err = -EIO;
+ err = 0;
for (i = 0; i < groups_per_page && bh[i]; i++)
if (!buffer_uptodate(bh[i]))
goto out;
Mingming Cao Wrote:
> ext4: Fix ext4_mb_init_cache return error
>
> From: Mingming Cao <[email protected]>
>
> ext4_mb_init_cache() incorrectly always return EIO on success. This
> causes the caller of ext4_mb_init_cache() fail when it checks
> the return value.
>
> Signed-off-by: Mingming Cao <[email protected]>
> ---
> fs/ext4/mballoc.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> Index: linux-2.6.26-rc4/fs/ext4/mballoc.c
> ===================================================================
> --- linux-2.6.26-rc4.orig/fs/ext4/mballoc.c 2008-06-01 12:55:19.000000000 -0700
> +++ linux-2.6.26-rc4/fs/ext4/mballoc.c 2008-06-01 12:55:23.000000000 -0700
> @@ -798,7 +798,7 @@ static int ext4_mb_init_cache(struct pag
> for (i = 0; i < groups_per_page && bh[i]; i++)
> wait_on_buffer(bh[i]);
>
> - err = -EIO;
> + err = 0;
> for (i = 0; i < groups_per_page && bh[i]; i++)
> if (!buffer_uptodate(bh[i]))
> goto out;
>
>
I think the fix should be
err = -EIO;
for (i = 0; i < groups_per_page && bh[i]; i++)
if (!buffer_uptodate(bh[i]))
goto out;
+ err = 0;
Is that right?
Shen Feng
On Mon, 2008-06-02 at 09:16 +0800, Shen Feng wrote:
>
> Mingming Cao Wrote:
> > ext4: Fix ext4_mb_init_cache return error
> >
> > From: Mingming Cao <[email protected]>
> >
> > ext4_mb_init_cache() incorrectly always return EIO on success. This
> > causes the caller of ext4_mb_init_cache() fail when it checks
> > the return value.
> >
> > Signed-off-by: Mingming Cao <[email protected]>
> > ---
> > fs/ext4/mballoc.c | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > Index: linux-2.6.26-rc4/fs/ext4/mballoc.c
> > ===================================================================
> > --- linux-2.6.26-rc4.orig/fs/ext4/mballoc.c 2008-06-01 12:55:19.000000000 -0700
> > +++ linux-2.6.26-rc4/fs/ext4/mballoc.c 2008-06-01 12:55:23.000000000 -0700
> > @@ -798,7 +798,7 @@ static int ext4_mb_init_cache(struct pag
> > for (i = 0; i < groups_per_page && bh[i]; i++)
> > wait_on_buffer(bh[i]);
> >
> > - err = -EIO;
> > + err = 0;
> > for (i = 0; i < groups_per_page && bh[i]; i++)
> > if (!buffer_uptodate(bh[i]))
> > goto out;
> >
> >
>
> I think the fix should be
>
> err = -EIO;
> for (i = 0; i < groups_per_page && bh[i]; i++)
> if (!buffer_uptodate(bh[i]))
> goto out;
> + err = 0;
>
> Is that right?
>
you are right, thanks for point this out.
Mingming
> Shen Feng
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html