hi,
I try to find ways to reduce the start time of programs or the whole
operating system. The idea to increase disk performance is to gather all
related files on disk physically close together. Unlike the last patches
from Kazuya Mio for "relevant file defragmentation" files used by system
boot are mostly not in the same directory.
First i look at the EXT4_IOC_MOVE_EXT call.
To reduce the amount of disk seeks i want to move the extents from
several files into a single donor file. To do so i increment the
move_extent.donor_start by the size of each file. But i got an error.
The error is caused by the lines in the kernel source
fs/ext4/move_extent.c:998
998 /* Start offset should be same */
999 if (orig_start != donor_start) {
1000 ext4_debug("ext4 move extent: orig and donor's start "
1001 "offset are not same [ino:orig %lu, donor %lu]\n",
1002 orig_inode->i_ino, donor_inode->i_ino);
1003 return -EINVAL;
1004 }
Is there a reason why the offset of the original file and the donor file
must be the same?
As i can see the patch for relevant file defragmentation in e4defrag
supports only directories. May it be possible to select any desired file?
What would you suggest as the best way to put files physically together?
If I could help, tell me. :)
Regards,
andy
Hi Andreas,
Thanks for the comments.
2010/08/25 21:00, [email protected] wrote:
> Is there a reason why the offset of the original file and the donor file
> must be the same?
e4defrag creates a donor file whose size is the same of the original file by
fallocate. There is a possibility that the original file will be corrupted
after moving an extent if the offset of the original file and the donor file
are different. So they are checked in the kernel space, but it may be
unnecessary from the point of view of the ioctl.
> As i can see the patch for relevant file defragmentation in e4defrag
> supports only directories. May it be possible to select any desired file?
That's interesting. I came up with the new interface of e4defrag -r.
What do you think the following implementation idea?
Usage: e4defrag -r directory...| device...
e4defrag -r base_file move_file... <--- new
1. Defrag base_file to reduce fragmentation of extents (call EXT4_IOC_MOVE_EXT)
2. Preallocate physical blocks near the data blocks of base_file
3. Move move_file's extents to the blocks that are allocated by (2).
4. Repeat (2) and (3) for all files specified as move_file
Regards,
Kazuya Mio
2010/8/26 Kazuya Mio <[email protected]>:
> Hi Andreas,
> Thanks for the comments.
>
> 2010/08/25 21:00, [email protected] wrote:
>> Is there a reason why the offset of the original file and the donor file
>> must be the same?
>
> e4defrag creates a donor file whose size is the same of the original file by
> fallocate. There is a possibility that the original file will be corrupted
> after moving an extent if the offset of the original file and the donor file
> are different. So they are checked in the kernel space, but it may be
> unnecessary from the point of view of the ioctl.
>
>> As i can see the patch for relevant file defragmentation in e4defrag
>> supports only directories. May it be possible to select any desired file?
>
> That's interesting. I came up with the new interface of e4defrag -r.
> What do you think the following implementation idea?
>
> Usage: e4defrag -r directory...| device...
> ? ? ? e4defrag -r base_file move_file... ? ? <--- new
>
> 1. Defrag base_file to reduce fragmentation of extents (call EXT4_IOC_MOVE_EXT)
> 2. Preallocate physical blocks near the data blocks of base_file
> 3. Move move_file's extents to the blocks that are allocated by (2).
> 4. Repeat (2) and (3) for all files specified as move_file
>
> Regards,
> Kazuya Mio
I suspect the original idea would work better because it is more
likely to pack the libs / files into perfectly contiguous block
ranges.
I too have never understood why the donor offsets have to match the
original offsets, although I had previously assumed the issue could be
worked around via making the donor file sparse and only have the block
range of interest allocated.
This use case is a specific example of where it would be beneficial to
eliminate that artificial limitation.
Greg
hi,
>> Usage: e4defrag -r directory...| device...
>> e4defrag -r base_file move_file...<--- new
>>
The new interface looks nice. As I see it's easy to implement cause only
the main-function must be modified. If you have no objections, I will do
it myself and send you the patch to review it.
> I suspect the original idea would work better because it is more
> likely to pack the libs / files into perfectly contiguous block
> ranges.
I don't know if it makes a difference for the block allocator. Even
though you know the size of all files to be moved.
For a better result, may it be possible to create a base_file whose
inode is part of an empty block group?
Andreas
On 2010-08-26, at 14:16, [email protected] wrote:
>>> Usage: e4defrag -r directory...| device...
>>> e4defrag -r base_file move_file...<--- new
>>>
>
> The new interface looks nice. As I see it's easy to implement cause only the main-function must be modified. If you have no objections, I will do it myself and send you the patch to review it.
>
>> I suspect the original idea would work better because it is more
>> likely to pack the libs / files into perfectly contiguous block
>> ranges.
>
> I don't know if it makes a difference for the block allocator. Even though you know the size of all files to be moved.
>
> For a better result, may it be possible to create a base_file whose inode is part of an empty block group?
There is a /proc/fs/ext4/*/inode_goal that will cause new inodes allocated in the filesystem to be after the specified goal inode.
I suppose it would also be possible to have e4defrag simply take an inode number in the form "<inode_number>" in place of "base_file", as debugfs does.
Cheers, Andreas
2010/08/27 5:16, [email protected] wrote:
>>> Usage: e4defrag -r directory...| device...
>>> e4defrag -r base_file move_file...<--- new
>>>
>
> The new interface looks nice. As I see it's easy to implement cause only the main-function must be modified.
I think so.
> If you have no objections, I will do it myself and send you the patch to review it.
That would be great.
Regards,
Kazuya Mio