Hi Linus,
Please pull from:
git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4.git for_linus
It has a number random cleanups and bug fixes, and two new features.
The first is uninitialized block groups, which allows fast mke2fs
operations plus as well as speeding up e2fsck by allowing it to skip
parts of the inode tables that haven't been used yet. A checksum has
been added to block group descriptors so we can tell detect corruption
in that data structure, since we are depending on it being accurate so
we know what portion of the inode table is really in use.
The second feature is flexible block groups, which allows inode tables
and block/inode bitmaps to be grouped together outside of the block
group that they administer, to allow contiguous regions of data blocks
to span multiple block groups, which helps for large files significantly
bigger than the size of a block group (i.e., 32 gigabytes on a 4k block
filesystem).
All of these patches have been baking in -mm for a while.
Regards,
- Ted
Andreas Dilger (1):
Ext4: Uninitialized Block Groups
Aneesh Kumar K.V (8):
ext4: Fix sparse warnings
ext4: Convert bg_block_bitmap to bg_block_bitmap_lo
ext4: Convert bg_inode_bitmap and bg_inode_table
ext4: Convert s_blocks_count to s_blocks_count_lo
ext4: Convert s_r_blocks_count and s_free_blocks_count
ext4: Convert ext4_extent.ee_start to ext4_extent.ee_start_lo
ext4: Convert ext4_extent_idx.ei_leaf to ext4_extent_idx.ei_leaf_lo
ext4: sparse fixes
Coly Li (1):
ext4: Remove (partial, never completed) fragment support
Eric Sandeen (3):
ext4: remove #ifdef CONFIG_EXT4_INDEX
ext4: fix setup_new_group_blocks locking
ext4: lighten up resize transaction requirements
Jan Kara (1):
jbd2: fix commit code to properly abort journal
Jose R. Santos (2):
JBD2: debug code cleanup.
ext4: FLEX_BG Kernel support v2.
Mingming Cao (6):
JBD: JBD slab allocation cleanups
JBD2: jbd2 slab allocation cleanups
JBD: replace jbd_kmalloc with kmalloc directly
JBD2: replace jbd_kmalloc with kmalloc directly.
JBD2/Ext4: Convert kmalloc to kzalloc in jbd2/ext4
jbd2: JBD_XXX to JBD2_XXX naming cleanup
fs/Kconfig | 1 +
fs/ext4/balloc.c | 112 ++++++++++++++++++++++++++++-
fs/ext4/dir.c | 7 --
fs/ext4/extents.c | 14 ++--
fs/ext4/fsync.c | 2 +-
fs/ext4/group.h | 27 +++++++
fs/ext4/ialloc.c | 151 +++++++++++++++++++++++++++++++++++----
fs/ext4/inode.c | 18 ++----
fs/ext4/namei.c | 20 -----
fs/ext4/resize.c | 59 +++++++++++-----
fs/ext4/super.c | 97 +++++++++++++++++--------
fs/ext4/xattr.c | 7 +-
fs/jbd/commit.c | 6 +-
fs/jbd/journal.c | 99 ++------------------------
fs/jbd/transaction.c | 12 ++--
fs/jbd2/commit.c | 16 ++--
fs/jbd2/journal.c | 128 ++++-----------------------------
fs/jbd2/recovery.c | 2 +-
fs/jbd2/revoke.c | 4 +-
fs/jbd2/transaction.c | 19 +++---
include/linux/ext4_fs.h | 103 ++++++++++----------------
include/linux/ext4_fs_extents.h | 4 +-
include/linux/ext4_fs_i.h | 5 --
include/linux/ext4_fs_sb.h | 3 -
include/linux/ext4_jbd2.h | 6 +-
include/linux/jbd.h | 17 +++--
include/linux/jbd2.h | 49 +++++++------
include/linux/poison.h | 3 +-
28 files changed, 527 insertions(+), 464 deletions(-)
create mode 100644 fs/ext4/group.h
On Wed, 17 Oct 2007, Theodore Ts'o wrote:
> Hi Linus,
>
> Please pull from:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4.git for_linus
>
> It has a number random cleanups and bug fixes, and two new features.
> The first is uninitialized block groups, which allows fast mke2fs
> operations plus as well as speeding up e2fsck by allowing it to skip
> parts of the inode tables that haven't been used yet.
nice feature, is there any work on a tool to go through a well-used
filesystem and mark unused block groups as uninitialized? (I would guess
that such a tool may want to move files to make this so)
David Lang
> A checksum has
> been added to block group descriptors so we can tell detect corruption
> in that data structure, since we are depending on it being accurate so
> we know what portion of the inode table is really in use.
>
> The second feature is flexible block groups, which allows inode tables
> and block/inode bitmaps to be grouped together outside of the block
> group that they administer, to allow contiguous regions of data blocks
> to span multiple block groups, which helps for large files significantly
> bigger than the size of a block group (i.e., 32 gigabytes on a 4k block
> filesystem).
>
> All of these patches have been baking in -mm for a while.
>
> Regards,
>
> - Ted
>
> Andreas Dilger (1):
> Ext4: Uninitialized Block Groups
>
> Aneesh Kumar K.V (8):
> ext4: Fix sparse warnings
> ext4: Convert bg_block_bitmap to bg_block_bitmap_lo
> ext4: Convert bg_inode_bitmap and bg_inode_table
> ext4: Convert s_blocks_count to s_blocks_count_lo
> ext4: Convert s_r_blocks_count and s_free_blocks_count
> ext4: Convert ext4_extent.ee_start to ext4_extent.ee_start_lo
> ext4: Convert ext4_extent_idx.ei_leaf to ext4_extent_idx.ei_leaf_lo
> ext4: sparse fixes
>
> Coly Li (1):
> ext4: Remove (partial, never completed) fragment support
>
> Eric Sandeen (3):
> ext4: remove #ifdef CONFIG_EXT4_INDEX
> ext4: fix setup_new_group_blocks locking
> ext4: lighten up resize transaction requirements
>
> Jan Kara (1):
> jbd2: fix commit code to properly abort journal
>
> Jose R. Santos (2):
> JBD2: debug code cleanup.
> ext4: FLEX_BG Kernel support v2.
>
> Mingming Cao (6):
> JBD: JBD slab allocation cleanups
> JBD2: jbd2 slab allocation cleanups
> JBD: replace jbd_kmalloc with kmalloc directly
> JBD2: replace jbd_kmalloc with kmalloc directly.
> JBD2/Ext4: Convert kmalloc to kzalloc in jbd2/ext4
> jbd2: JBD_XXX to JBD2_XXX naming cleanup
>
> fs/Kconfig | 1 +
> fs/ext4/balloc.c | 112 ++++++++++++++++++++++++++++-
> fs/ext4/dir.c | 7 --
> fs/ext4/extents.c | 14 ++--
> fs/ext4/fsync.c | 2 +-
> fs/ext4/group.h | 27 +++++++
> fs/ext4/ialloc.c | 151 +++++++++++++++++++++++++++++++++++----
> fs/ext4/inode.c | 18 ++----
> fs/ext4/namei.c | 20 -----
> fs/ext4/resize.c | 59 +++++++++++-----
> fs/ext4/super.c | 97 +++++++++++++++++--------
> fs/ext4/xattr.c | 7 +-
> fs/jbd/commit.c | 6 +-
> fs/jbd/journal.c | 99 ++------------------------
> fs/jbd/transaction.c | 12 ++--
> fs/jbd2/commit.c | 16 ++--
> fs/jbd2/journal.c | 128 ++++-----------------------------
> fs/jbd2/recovery.c | 2 +-
> fs/jbd2/revoke.c | 4 +-
> fs/jbd2/transaction.c | 19 +++---
> include/linux/ext4_fs.h | 103 ++++++++++----------------
> include/linux/ext4_fs_extents.h | 4 +-
> include/linux/ext4_fs_i.h | 5 --
> include/linux/ext4_fs_sb.h | 3 -
> include/linux/ext4_jbd2.h | 6 +-
> include/linux/jbd.h | 17 +++--
> include/linux/jbd2.h | 49 +++++++------
> include/linux/poison.h | 3 +-
> 28 files changed, 527 insertions(+), 464 deletions(-)
> create mode 100644 fs/ext4/group.h
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
On Wed, 17 Oct 2007, Theodore Ts'o wrote:
>
> Please pull from:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4.git for_linus
This conflicts in nontrivial ways with
commit 7c9e69faa28027913ee059c285a5ea8382e24b5d
Author: Aneesh Kumar K.V <[email protected]>
Date: Tue Oct 16 23:27:02 2007 -0700
ext2/ext3/ext4: add block bitmap validation
which I just merged from -mm.
I suspect it's trivial to fix up for somebody who knows/understands the
code, but I'm not that person, and I cannot verify the end result.
So can you please do the merge, and I'll pull the merged tree?
Linus
On Wed, Oct 17, 2007 at 08:59:53AM -0700, Linus Torvalds wrote:
> > Please pull from:
> >
> > git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4.git for_linus
>
> This conflicts in nontrivial ways with
>
> which I just merged from -mm.
>
> I suspect it's trivial to fix up for somebody who knows/understands the
> code, but I'm not that person, and I cannot verify the end result.
>
> So can you please do the merge, and I'll pull the merged tree?
>
OK, I've rebased and fixed up the patches with your latest git tree.
Could you please pull from:
git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4.git for_linus
now? Thanks!!
- Ted
On Oct 16, 2007 21:51 -0700, [email protected] wrote:
> On Wed, 17 Oct 2007, Theodore Ts'o wrote:
> >It has a number random cleanups and bug fixes, and two new features.
> >The first is uninitialized block groups, which allows fast mke2fs
> >operations plus as well as speeding up e2fsck by allowing it to skip
> >parts of the inode tables that haven't been used yet.
>
> nice feature, is there any work on a tool to go through a well-used
> filesystem and mark unused block groups as uninitialized? (I would guess
> that such a tool may want to move files to make this so)
Yes, just set the feature flag via tune2fs and then run e2fsck on it.
The second e2fsck shown below is just a demonstration of the speedup.
# tune2fs -O uninit_groups /dev/foo
tune2fs 1.39.cfs9 (7-Apr-2007)
Please run e2fsck on the filesystem.
# time e2fsck -fy /dev/foo
e2fsck 1.39.cfs9 (7-Apr-2007)
Group descriptor 0 checksum is invalid. Fix? yes
{repeats for all groups}
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
lustre-OST0000: 9099/1640160 files (0.4% non-contiguous), 221996/6554520
blocks
real 0m17.273s
user 0m4.930s
sys 0m1.749s
# time e2fsck -fy /dev/hda3
e2fsck 1.39.cfs9 (7-Apr-2007)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
lustre-OST0000: 9099/1640160 files (0.4% non-contiguous), 221996/6554520
blocks
real 0m2.412s
user 0m0.604s
sys 0m0.077s
The caveats are:
(a) this is a read-only feature, so you can't mount such a filesystem r/w
on an older kernel. You can disable it with "tune2fs -O ^uninit_groups"
and run a full e2fsck on it again.
(b) I don't think there is an official e2fsprogs release with support for this
feature yet (it's in the pipe, however).
(c) The actual speedup depends on how full the filesystem is, but since ext*
usually has way too many inodes, it is generally pretty good.
Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.
On Wed, 17 Oct 2007 08:59:53 -0700 (PDT)
Linus Torvalds <[email protected]> wrote:
> On Wed, 17 Oct 2007, Theodore Ts'o wrote:
> >
> > Please pull from:
> >
> > git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4.git for_linus
>
> This conflicts in nontrivial ways with
>
> commit 7c9e69faa28027913ee059c285a5ea8382e24b5d
> Author: Aneesh Kumar K.V <[email protected]>
> Date: Tue Oct 16 23:27:02 2007 -0700
>
> ext2/ext3/ext4: add block bitmap validation
>
> which I just merged from -mm.
<catching up>
There shouldn't have been conflicts here - if there were I wouldn't have
sent those patches. Unless there were things in the ext4 pull which
weren't present in the ext4 quilt tree which I included in 2.6.23-mm1?
On Thu, 25 Oct 2007, Andrew Morton wrote:
>
> There shouldn't have been conflicts here - if there were I wouldn't have
> sent those patches. Unless there were things in the ext4 pull which
> weren't present in the ext4 quilt tree which I included in 2.6.23-mm1?
Well, you merge your patch-series by patching.
You should have noticed by now that GNU patch in particular will happily
apply a patch whether it conflicts or not. So it's entirely possible that
it didn't conflict for you, but applied cleanly and sanely.
Linus
On Thu, 25 Oct 2007 16:44:21 -0700 (PDT)
Linus Torvalds <[email protected]> wrote:
>
>
> On Thu, 25 Oct 2007, Andrew Morton wrote:
> >
> > There shouldn't have been conflicts here - if there were I wouldn't have
> > sent those patches. Unless there were things in the ext4 pull which
> > weren't present in the ext4 quilt tree which I included in 2.6.23-mm1?
>
> Well, you merge your patch-series by patching.
>
> You should have noticed by now that GNU patch in particular will happily
> apply a patch whether it conflicts or not. So it's entirely possible that
> it didn't conflict for you, but applied cleanly and sanely.
>
hrm, could be. It would be strange for that to happen quietly with fuzz=1
and to still produce a compileable result.
But there weren't any patches in this git-merge which weren't in 2.6.23-mm1
so maybe something like that happened. Or maybe that fact that this pull
only contained _some_ of the ext4 patches which were in -mm somehow affected
things.
Oh well, I should have sent the ext4 changes via Ted anyway.