I'm writing this to Theodore Ts'o because he has worked on the package
in the past, and currently maintains e2fsprogs. I am CCing lkml since
there are likely to be other interested parties, and there does not seem
to be a separate list for e2fsprogs.
The old defrag package for doing offline defragmentation of ext2 has not
been maintained in many years. About two years ago I fixed a few bugs
that were causing it to not work on ext3 and placed the changes in
Ubuntu. I would like to see this program continue to live since it
still serves a purpose, but because it has had no upstream maintainer in
so long, Debian and thus Ubuntu have dropped the package. I am trying
to resurrect it.
I was wondering why it was never integrated into e2fsprogs, and if this
could now be done. It seems to me that e2fsprogs would be the proper
home for it, and a neutral upstream for everyone. If not, then I will
try to maintain it as an Ubuntu native package in a bazaar branch on
launchpad.
On Wed, Oct 22, 2008 at 12:10:23PM -0400, Phillip Susi wrote:
> I'm writing this to Theodore Ts'o because he has worked on the package
> in the past, and currently maintains e2fsprogs. I am CCing lkml since
> there are likely to be other interested parties, and there does not seem
> to be a separate list for e2fsprogs.
>
> The old defrag package for doing offline defragmentation of ext2 has not
> been maintained in many years. About two years ago I fixed a few bugs
> that were causing it to not work on ext3 and placed the changes in
> Ubuntu. I would like to see this program continue to live since it
> still serves a purpose, but because it has had no upstream maintainer in
> so long, Debian and thus Ubuntu have dropped the package. I am trying
> to resurrect it.
>
> I was wondering why it was never integrated into e2fsprogs, and if this
> could now be done. It seems to me that e2fsprogs would be the proper
> home for it, and a neutral upstream for everyone. If not, then I will
> try to maintain it as an Ubuntu native package in a bazaar branch on
> launchpad.
The problem with the defrag code has always been that I haven't had
time to give it proper love and attention, and it needs a *lot* of
work before I would be willing to stand behind it as a useful and
"safe" tool to use. Basically, I don't want to get the hate mail from
people who lose their data due to bugs or other potential failures
from e2defrag.
I have a vague memory it doesn't even work if the filesystem blocksize
is greater than 1k, but I'm not 100% certain that is still true.
There is also an on-line defragmetter that requires kernel support
that some folks from NEC are working on for ext4 (which can also
support ext3 filesystems). My thinking is that it's more worthwhile
to focus my attentions on helping Akira Fujita and Takashi Sato finish
theiron-line ext4 defragmentation patches than to worry about
e2defrag. Of course, if you'd like to give some love and attention to
e2defrag, that's great. But e2defrag doesn't use the ext2fs
libraries, and it would be a huge amount of work to make sure it
supports things like extended attributes and other newer format
enhancements that have been made to the ext3 filesystem.
Regards,
- Ted
Theodore Tso wrote:
> The problem with the defrag code has always been that I haven't had
> time to give it proper love and attention, and it needs a *lot* of
> work before I would be willing to stand behind it as a useful and
> "safe" tool to use. Basically, I don't want to get the hate mail from
> people who lose their data due to bugs or other potential failures
> from e2defrag.
Indeed, it rightly should be covered with warnings about it possibly
munching your data and should only be used on fully backed up
filesystems. As far as I know currently though, it works properly and
I'm willing to use what free time I have to fix any bugs that are found,
but I need to decide where to house the source in a proper VCS instead
of just continuing to add dpatches to the ubuntu source package.
> I have a vague memory it doesn't even work if the filesystem blocksize
> is greater than 1k, but I'm not 100% certain that is still true.
I have an active 110 gig ext3 root fs ( that is only about 8% used )
that I tested on with no errors ( after removing one disk from the
mirror and using that to test with ). It uses 4k blocks, has_journal,
resize_inode, dir_index, filetype, sparse_super, large_file. I think it
won't support blocks larger than 4k, but I don't think the kernel will
either. I also verified the md5sums of all files on the disk after the
defrag.
> There is also an on-line defragmetter that requires kernel support
> that some folks from NEC are working on for ext4 (which can also
> support ext3 filesystems). My thinking is that it's more worthwhile
> to focus my attentions on helping Akira Fujita and Takashi Sato finish
> theiron-line ext4 defragmentation patches than to worry about
> e2defrag. Of course, if you'd like to give some love and attention to
> e2defrag, that's great. But e2defrag doesn't use the ext2fs
> libraries, and it would be a huge amount of work to make sure it
> supports things like extended attributes and other newer format
> enhancements that have been made to the ext3 filesystem.
I've been reading a lot in the last few hours about the proposed online
defragmenter, and while it seems quite interesting, it appears to not be
ready yet, whereas e2defrag is readily available. It will be
interesting to compare the two approaches.
I have been trying to identify any new features that cause problems and
fix any that arise. Aren't extended attributes stored in the extra
space of large inodes? If so, I would need to format a new fs using
large inodes to test. I don't think e2defrag currently supports large
inodes, but it shouldn't be too hard to add support. Other than being
aware of the correct size of the inodes, I don't see anything special it
would need to do to support extended attributes if they are just extra
data in the inode, or can they be stored in data blocks and so it would
need to understand the block pointers in the inode?
Are there any other ext3 features that might cause trouble? I know
there are plenty in ext4, but I want to make sure everything works well
in ext3 first.
On Wed, Oct 22, 2008 at 04:01:48PM -0400, Phillip Susi wrote:
>
> I have been trying to identify any new features that cause problems and
> fix any that arise. Aren't extended attributes stored in the extra
> space of large inodes? If so, I would need to format a new fs using
> large inodes to test.
Extended attributes *can* be stored in the extra space of large
inodes, but they can also be stored in separate, stand-alone data
blocks --- and just to make life entertaining, if multiple files have
exactly the same set of extended attributes (for example, if all of
the files inone directory have the same ACL and/or SELinux property),
an extended attribute block can be shared across multiple inodes for
space efficiency.
> Are there any other ext3 features that might cause trouble? I know
> there are plenty in ext4, but I want to make sure everything works well
> in ext3 first.
That's probably the main one; and as you say, there are plenty of new
features that will require special handling for e2defrag, especially
since it doesn't use the standard libext2fs library.
- Ted