Ext4 Developer Interlock Call: 05/07/2007 Meeting Minutes
Attendees: Mingming Cao, Dave Kleikamp, Ted Ts'o, Aneesh Veetil, Takashi
Sato, Jose Santos, Avantika Mathur,
Minutes can be accessed at:
http://ext4.wiki.kernel.org/index.php/Ext4_Developer%27s_Conference_Call
Discussed pushing ext4 patches upstream.
- Preallocation Patches: These patches are almost ready to push, once we
have consensus from all architecture maintainers on the format of the
system call. Mingming will help with this.
- Nanosecond Timestamps: Ted will take a closer look at this patch
before pushing
- It was proposed to push everything in the ext4 git tree up until the
unstable patches. Before pushing these patches, we must run tests on
the patches and use the check-patches scripts to perform sanity checking.
e2fsprogs:
- Ted recently posted e2fsprogs-1.39-tyt3; which includes patches
supporting extents. This is a work in progress.
- Jose used the patches to e2fsprogs sent by Valerie Clement to create
large filesystems (with >32-bit block numbers)
-- Ted suggested that these patches be added to e2fsprogs-tyt3, with
a flag set when in use, indicating that no shared libraries should be used.
- Ted is planning to discontinue use of Mercurial for e2fsprogs, and
switch entirely to git. By using git, we can have a stable and
development branch for e2fsprogs. This will also reduce the work Ted
does to create a patch series.
Migration:
- Aneesh looked at the online defragmentation code, and feels it may not
be usable for migration because it is solely extents based. Mingming
pointed out that the new defrag patches also have support for indirect
mapped blocks, Aneesh will look into these new patches.
- Ted suggested doing the migration through two ioctls, the first
migrating the inode online via ioctl, and the second using the
online-defrag ioctl.
Metablock Groups:
- Mingming mentioned that in the metablock group feature, the inode
metadata is not moved with the rest of the metadata. Ted will check
this, as he thought this had been implemented.
- Online resize does not use metablock groups, and is currently limited
to 2TB.
- Also need to remove sanity checks in the mount code
- Once these issues are fixed, the metablock group feature can be turned
on by default for ext4 filesysytems
- The extents option will be turned on by default for all ext4
filesystems with the extents feature enabled.
On May 09, 2007 10:02 -0700, Avantika Mathur wrote:
> Metablock Groups:
>
> - Mingming mentioned that in the metablock group feature, the inode
> metadata is not moved with the rest of the metadata. Ted will check
> this, as he thought this had been implemented.
>
> - Online resize does not use metablock groups, and is currently limited
> to 2TB.
>
> - Also need to remove sanity checks in the mount code
>
> - Once these issues are fixed, the metablock group feature can be turned
> on by default for ext4 filesysytems
In my recent investigation of META_BG, it appears that BOTH the kernel
and e2fsprogs do NOT handle group metadata (bitmaps, itable) outside the
group.
In the kernel ext[234]_check_descriptors() requires the bitmaps and itable
to be inside the group even though META_BG is supported for a long time.
In e2fsck/super.c check_super_block() also requires that the bitmaps and
itable be inside the group even though e2fsck has claimed META_BG for a
long time.
That means it is virtually impossible to allow the group metadata to move
under the META_BG feature. It might be possible to predicate this change
under both 64BIT and META_BG, though this is somewhat non-standard.
Alternately, we could allow the relocation of bitmaps and itables under
BIG_BG, which is itself covers the relocation aspects and also allows
the ability to change the group size.
Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.
On Wed, 2007-05-09 at 10:37 -0700, Andreas Dilger wrote:
> On May 09, 2007 10:02 -0700, Avantika Mathur wrote:
> > Metablock Groups:
> >
> > - Mingming mentioned that in the metablock group feature, the inode
> > metadata is not moved with the rest of the metadata. Ted will check
> > this, as he thought this had been implemented.
> >
> > - Online resize does not use metablock groups, and is currently limited
> > to 2TB.
> >
> > - Also need to remove sanity checks in the mount code
> >
> > - Once these issues are fixed, the metablock group feature can be turned
> > on by default for ext4 filesysytems
>
> In my recent investigation of META_BG, it appears that BOTH the kernel
> and e2fsprogs do NOT handle group metadata (bitmaps, itable) outside the
> group.
>
> In the kernel ext[234]_check_descriptors() requires the bitmaps and itable
> to be inside the group even though META_BG is supported for a long time.
>
> In e2fsck/super.c check_super_block() also requires that the bitmaps and
> itable be inside the group even though e2fsck has claimed META_BG for a
> long time.
>
> That means it is virtually impossible to allow the group metadata to move
> under the META_BG feature.
Understand it's not straightforward to turn on this feature by default.
Just try to understand, why the kernel and e2fsprogs issues you
mentioned above are not possible to fix to support full META_BG
feature?
> It might be possible to predicate this change
> under both 64BIT and META_BG, though this is somewhat non-standard.
>
> Alternately, we could allow the relocation of bitmaps and itables under
> BIG_BG, which is itself covers the relocation aspects and also allows
> the ability to change the group size.
>
Well, I think the previous discussion about BIG_BG feature leads us to
believe that turn on META_BG feature is good enough to support larger fs
and reduce fragmentation(which is the new BIG_BG feature is trying to
address). After all the relocation of metadata (bitmaps, itables and
group descriptors) is what META_BG feature means.
Regards,
Mingming
On May 09, 2007 11:10 -0700, Mingming Cao wrote:
> On Wed, 2007-05-09 at 10:37 -0700, Andreas Dilger wrote:
> > That means it is virtually impossible to allow the group metadata to move
> > under the META_BG feature.
>
> Understand it's not straightforward to turn on this feature by default.
> Just try to understand, why the kernel and e2fsprogs issues you
> mentioned above are not possible to fix to support full META_BG
> feature?
Because older kernels and e2fsprogs will think they understand META_BG,
and will consider the filesystem corrupt due to "badly placed" bitmaps
and itable. That is exactly what FEATURE flags are supposed to protect
against, so we can't change the semantics of the META_BG feature at this
time. I understand that we WANTED to have META_BG have different
semantics, but the implementation didn't follow the design for this flag.
> > It might be possible to predicate this change
> > under both 64BIT and META_BG, though this is somewhat non-standard.
> >
> > Alternately, we could allow the relocation of bitmaps and itables under
> > BIG_BG, which is itself covers the relocation aspects and also allows
> > the ability to change the group size.
> >
>
> Well, I think the previous discussion about BIG_BG feature leads us to
> believe that turn on META_BG feature is good enough to support larger fs
> and reduce fragmentation(which is the new BIG_BG feature is trying to
> address). After all the relocation of metadata (bitmaps, itables and
> group descriptors) is what META_BG feature means.
That is what we WANTED the META_BG feature to mean. What was implemented
in both ext2/3/4 and e2fsck is that META_BG means "group descriptor backups
are in 1st, 2nd, last group in metagroup" and nothing else. At least the
code is consistent between the two.
So that leaves us in the position that we either need to create a new
INCOMPAT feature flag to allow relocation of bitmaps/itable, OR we
need to make this conditional upon both META_BG and 64BIT being set.
The latter is a bit confusing and is subject to error if only one or
the other flag is checked by accident. If we make a new INCOMPAT flag
then could almost as easily implement BIG_BG and give us more flexibility
for the future, though I haven't looked at that code in a while.
Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.