Now that we're done pushing patches to ext4 for the merge windows
(unless some really nasty bugs/regressions turn up), it's time to start
thinking which patches that have been pushed to mainline since 2.6.27
should be nominated for the 2.6.27.x -stable tree.
To do that, I've put together a couple of branches, all based off of
2.6.27, at the ext4 git tree:
http://git.kernel.org/?p=linux/kernel/git/tytso/ext4.git
The first branch, ext4-stable, contains *all* ext4-related patches
pushed to Linus since 2.6.27. This is intended for use by people who
want to apply patches from the ext4 patch queue during the merge window,
and also as convenience for ext4 developers trying to decide which
patches should be cherry picked for -stable.
>From that list, I picked a conservative set of commits that clearly met
the stable kernel criteria in the for-stable branch:
6d1c365... ext4: Do mballoc init before doing filesystem recovery
1708a97... ext4: Free ext4_prealloc_space using kmem_cache_free
bfe5327... ext4: fix xattr deadlock
cefdd90... jbd2: Fix buffer head leak when writing the commit block
a75c1c2... jbd2: abort instead of waiting for nonexistent transaction
73622b0... ext4: fix initialization of UNINIT bitmap blocks
66b63de... ext4/jbd2: Avoid WARN() messages when failing to write to the superbl
89376a6... ext4: Renumber EXT4_IOC_MIGRATE
8ecff40... ext4: elevate write count for migrate ioctl
75ff03e... ext4: add missing unlock in ext4_check_descriptors() on error path
cd2032b... jbd2: fix /proc setup for devices that contain '/' in their names
a27e215... ext4: fix #11321: create /proc/ext4/*/stats more carefully
903872b... Update flex_bg free blocks and free inodes counters when resizing.
70f61d7... ext4: Avoid printk floods in the face of directory corruption
This can be found here:
git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4.git for-stable
http://git.kernel.org/?p=linux/kernel/git/tytso/ext4.git;a=shortlog;h=for-stable
The more troublesome set of patches are the ones to fix the ENOSPC when
using delayed allocations problems. These patches violate the stable
criteria in being much harder to review, as well as exceeding the 100
line patch limit. The for-stable-enospc branch includes the for-stable
patches listed above, as well as the following:
1eede11... ext4: Properly update i_disksize.
623865a... ext4: truncate block allocated on a failed ext4_write_begin
e7683a4... ext4: Retry block allocation if we have free blocks left
9337b86... ext4: Don't add the inode to journal handle until after the block is
83519a2... ext4: Fix ext4 nomballoc allocator for ENOSPC
0aa4055... ext4: Signed arithmetic fix
52ea747... ext4: Switch to non delalloc mode when we are low on free blocks coun
1e69693... ext4: Add percpu dirty block accounting.
22e5981... ext4: Retry block reservation
d1e9424... ext4: Make sure all the block allocation paths reserve blocks
6be82f3... ext4: invalidate pages if delalloc block allocation fails.
git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4.git for-stable-enospc
http://git.kernel.org/?p=linux/kernel/git/tytso/ext4.git;a=shortlog;h=for-stable-enospc
I'm a bit concerned about asking Greg or Chris to take the delayed
allocation ENOSPC patches, although they really are critical for someone
considering using ext4 in production (and granted, they would probably
be better waiting until 2.6.28).
So I think the for-stable branch is probably the right set to nominate
for 2.6.27.x. Comments? Can folks give it some quick testing before
I formally send it off to the stable folks?
- Ted
On Sat, Oct 18, 2008 at 10:18:35AM -0400, Theodore Ts'o wrote:
>
>
> I'm a bit concerned about asking Greg or Chris to take the delayed
> allocation ENOSPC patches, although they really are critical for someone
> considering using ext4 in production (and granted, they would probably
> be better waiting until 2.6.28).
Heh, you do realize that hundreds of thousands of users are going to be
using 2.6.27.x "for production" as at least 3 distros are being based
off of it?
So, either you send these patches to us if you want to have a sane idea
of what the distros are using, or you rely on the distros themselves to
get it right in cherry-picking from upstream here. I know one distro
already has done this, and I'm sure others have as well.
It comes down to who you trust, a random distro engineer, or you and
your group of engineers :)
If you feel the second tree is something that you can support, and you
feel is necessary for users if they want to use ext4 on this kernel
release, I'd be glad to review them for inclusion.
thanks,
greg k-h
On Sat, Oct 18, 2008 at 10:03:05AM -0700, Greg KH wrote:
>
> Heh, you do realize that hundreds of thousands of users are going to be
> using 2.6.27.x "for production" as at least 3 distros are being based
> off of it?
>
> So, either you send these patches to us if you want to have a sane idea
> of what the distros are using, or you rely on the distros themselves to
> get it right in cherry-picking from upstream here. I know one distro
> already has done this, and I'm sure others have as well.
>
> It comes down to who you trust, a random distro engineer, or you and
> your group of engineers :)
Heh**2. What Fedora 10 at least is going to be doing is grabbing the
equivalent of ext4-stable. If it were completely up to me, and Eric
Sandeen from Red Hat has made the same wish (although I suspectly
somewhat in jest) on #ext4, it would be great if we could pour all of
ext4-stable (i.e., what we got added into the malinline during the
2.6.28 merge window) into 2.6.27.x. That would be easier for me to
support as well. Only problem is that it rather blatently violates
the criteria as set forth in Documentation/stable_kernel_rules.txt.
(i.e., there are factor-of-five performance enhacements, code
cleanups, ext4dev gets renamed to ext4, etc.)
Regardless what goes into 2.6.27.x, I'll probably have to maintain a
patchset of everything else "left over" for those distributions that
want to be aggressive about letting users starting to play with ext4.
> If you feel the second tree is something that you can support, and you
> feel is necessary for users if they want to use ext4 on this kernel
> release, I'd be glad to review them for inclusion.
If distribution users are going to use ext4 for real, then ENOSPC
issues are a real problem, and they should be solved. But there are a
large number of patches involved here, and many of them go over 100
lines (although granted the wc -l listed below includes the commit
comments):
161 /tmp/p/0015-ext4-invalidate-pages-if-delalloc-block-allocation.patch
211 /tmp/p/0016-ext4-Make-sure-all-the-block-allocation-paths-reser.patch
123 /tmp/p/0017-ext4-Retry-block-reservation.patch
307 /tmp/p/0018-ext4-Add-percpu-dirty-block-accounting.patch
113 /tmp/p/0019-ext4-Switch-to-non-delalloc-mode-when-we-are-low-on.patch
93 /tmp/p/0020-ext4-Signed-arithmetic-fix.patch
64 /tmp/p/0021-ext4-Fix-ext4-nomballoc-allocator-for-ENOSPC.patch
86 /tmp/p/0022-ext4-Don-t-add-the-inode-to-journal-handle-until-af.patch
190 /tmp/p/0023-ext4-Retry-block-allocation-if-we-have-free-blocks.patch
50 /tmp/p/0024-ext4-truncate-block-allocated-on-a-failed-ext4_writ.patch
169 /tmp/p/0025-ext4-Properly-update-i_disksize.patch
I am very confident about these patches, since they hit the ext4 tree
sometime in 2.6.27-rc2 or -rc3, so they've gotten quite a bit of
testing. I just put them at the end of the for-stable-enospc branch
because they do technically violate the -stable rules.
So it's ultimately to you and Chris. I'm happy to support any one of
for-stable, for-stable-enospc, or ext4-stable in 2.6.27.x. The last
is less work for me since I won't have to create "this-is-in-mainline-
please-consider-this-for-your-distro" patch sets. The real question
is what you're willing to review, and whether other -stable reviewers
will end up complaining and sending NACK's because they violate the
-stable rules. (To be fair, if some other subsystem did this and I
saw such patches, with my stable reviewer hat on I'd at least send up
a red flag.)
- Ted
Theodore Tso wrote:
> On Sat, Oct 18, 2008 at 10:03:05AM -0700, Greg KH wrote:
>> Heh, you do realize that hundreds of thousands of users are going to be
>> using 2.6.27.x "for production" as at least 3 distros are being based
>> off of it?
>>
>> So, either you send these patches to us if you want to have a sane idea
>> of what the distros are using, or you rely on the distros themselves to
>> get it right in cherry-picking from upstream here. I know one distro
>> already has done this, and I'm sure others have as well.
>>
>> It comes down to who you trust, a random distro engineer, or you and
>> your group of engineers :)
>
> Heh**2. What Fedora 10 at least is going to be doing is grabbing the
> equivalent of ext4-stable. If it were completely up to me, and Eric
> Sandeen from Red Hat has made the same wish (although I suspectly
> somewhat in jest) on #ext4, it would be great if we could pour all of
> ext4-stable (i.e., what we got added into the malinline during the
> 2.6.28 merge window) into 2.6.27.x.
Well, in reality Fedora 10 will be getting these patches, whether they
come in via 2.6.27.x or not. (Many/most are already in, most of the
rest will follow). We've been advocating for ext4 in Fedora, so it
makes sense for us to keep it up to date with the latest patches in any
case. (I've been slightly gunshy over the writeback changes just
because there was a bit of flux up until the end, but they may still
make it; also I'm holding off on fiemap until it officially sees the
light of day.) But I guess I'm one of those rare "random distro
engineers" who is also one of "Ted's group of engineers." ;)
So don't fret over Fedora w.r.t. 2.6.27.x; I think we've got it covered.
Fedora 10 will also get 2.6.28 at some point. So I'm not personally
worried. I wonder how anxious other distros are to ship bona-fide ext4
in 2.6.27?
Anyway; I hope to find some time to look over the -stable nominated
patches, at first glance it looks like a pretty sane patchset. I
wouldn't really advocate dumping all of 2.6.28's ext4 code into
2.6.27-stable, that'd be some pretty serious rule-bending I think.
-Eric
On Sat, Oct 18, 2008 at 03:28:56PM -0400, Theodore Tso wrote:
> On Sat, Oct 18, 2008 at 10:03:05AM -0700, Greg KH wrote:
> >
> > Heh, you do realize that hundreds of thousands of users are going to be
> > using 2.6.27.x "for production" as at least 3 distros are being based
> > off of it?
> >
> > So, either you send these patches to us if you want to have a sane idea
> > of what the distros are using, or you rely on the distros themselves to
> > get it right in cherry-picking from upstream here. I know one distro
> > already has done this, and I'm sure others have as well.
> >
> > It comes down to who you trust, a random distro engineer, or you and
> > your group of engineers :)
>
> Heh**2. What Fedora 10 at least is going to be doing is grabbing the
> equivalent of ext4-stable. If it were completely up to me, and Eric
> Sandeen from Red Hat has made the same wish (although I suspectly
> somewhat in jest) on #ext4, it would be great if we could pour all of
> ext4-stable (i.e., what we got added into the malinline during the
> 2.6.28 merge window) into 2.6.27.x. That would be easier for me to
> support as well. Only problem is that it rather blatently violates
> the criteria as set forth in Documentation/stable_kernel_rules.txt.
> (i.e., there are factor-of-five performance enhacements, code
> cleanups, ext4dev gets renamed to ext4, etc.)
>
> Regardless what goes into 2.6.27.x, I'll probably have to maintain a
> patchset of everything else "left over" for those distributions that
> want to be aggressive about letting users starting to play with ext4.
Fair enough, want to send us the patches you know you want us to take
for 2.6.27 right now?
> > If you feel the second tree is something that you can support, and you
> > feel is necessary for users if they want to use ext4 on this kernel
> > release, I'd be glad to review them for inclusion.
>
> If distribution users are going to use ext4 for real, then ENOSPC
> issues are a real problem, and they should be solved. But there are a
> large number of patches involved here, and many of them go over 100
> lines (although granted the wc -l listed below includes the commit
> comments):
>
> 161 /tmp/p/0015-ext4-invalidate-pages-if-delalloc-block-allocation.patch
> 211 /tmp/p/0016-ext4-Make-sure-all-the-block-allocation-paths-reser.patch
> 123 /tmp/p/0017-ext4-Retry-block-reservation.patch
> 307 /tmp/p/0018-ext4-Add-percpu-dirty-block-accounting.patch
> 113 /tmp/p/0019-ext4-Switch-to-non-delalloc-mode-when-we-are-low-on.patch
> 93 /tmp/p/0020-ext4-Signed-arithmetic-fix.patch
> 64 /tmp/p/0021-ext4-Fix-ext4-nomballoc-allocator-for-ENOSPC.patch
> 86 /tmp/p/0022-ext4-Don-t-add-the-inode-to-journal-handle-until-af.patch
> 190 /tmp/p/0023-ext4-Retry-block-allocation-if-we-have-free-blocks.patch
> 50 /tmp/p/0024-ext4-truncate-block-allocated-on-a-failed-ext4_writ.patch
> 169 /tmp/p/0025-ext4-Properly-update-i_disksize.patch
>
> I am very confident about these patches, since they hit the ext4 tree
> sometime in 2.6.27-rc2 or -rc3, so they've gotten quite a bit of
> testing. I just put them at the end of the for-stable-enospc branch
> because they do technically violate the -stable rules.
>
> So it's ultimately to you and Chris. I'm happy to support any one of
> for-stable, for-stable-enospc, or ext4-stable in 2.6.27.x. The last
> is less work for me since I won't have to create "this-is-in-mainline-
> please-consider-this-for-your-distro" patch sets. The real question
> is what you're willing to review, and whether other -stable reviewers
> will end up complaining and sending NACK's because they violate the
> -stable rules. (To be fair, if some other subsystem did this and I
> saw such patches, with my stable reviewer hat on I'd at least send up
> a red flag.)
I'm still undecided about this. For now, I'm thinking we should hold
off, but as 2.6.27 gets a bit more "stable", I might be willing to
reconsider just because of what it is being used as (for a base for a
wide range of distros/users.)
Sound fair?
thanks,
greg k-h