All,
I recently had a major data loss when expanding an LVM logical volume
past 2TB in size. I have preserved the 2.25TB just in case I might be
able to recover some data. But this backup is not my reason for
writing.
I have done several tests and these are the results. First the error
condition, where I create a 1GB file system, expand the volume to 2TB,
resize the file system and have it become corrupted:
linux-ek23:/mnt # lvcreate -L 1G -n main storage
Logical volume "main" created
linux-ek23:/mnt # mkfs.ext4 /dev/storage/main
mke2fs 1.42.6 (21-Sep-2012)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=128 blocks, Stripe width=640 blocks
65536 inodes, 262144 blocks
13107 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=268435456
8 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376
Allocating group tables: done
Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done
linux-ek23:/mnt # lvextend -L 2T /dev/storage/main
Extending logical volume main to 2.00 TiB
Logical volume main successfully resized
linux-ek23:/mnt # fsck.ext4 /dev/storage/main
e2fsck 1.42.6 (21-Sep-2012)
/dev/storage/main: clean, 11/65536 files, 12635/262144 blocks
linux-ek23:/mnt # resize2fs -p /dev/storage/main
resize2fs 1.42.6 (21-Sep-2012)
Resizing the filesystem on /dev/storage/main to 536870912 (4k) blocks.
Begin pass 2 (max = 5)
Relocating blocks XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Begin pass 3 (max = 8)
Scanning inode table XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Begin pass 5 (max = 1)
Moving inode table XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
The filesystem on /dev/storage/main is now 536870912 blocks long.
linux-ek23:/mnt # fsck.ext4 /dev/storage/main
e2fsck 1.42.6 (21-Sep-2012)
ext2fs_check_desc: Corrupt group descriptor: bad block for block bitmap
fsck.ext4: Group descriptors look bad... trying backup blocks...
/dev/storage/main was not cleanly unmounted, check forced.
Pass 1: Checking inodes, blocks, and sizes
Group 1's inode table at 609 conflicts with some other fs block.
Relocate<y>?
/dev/storage/main: e2fsck canceled.
/dev/storage/main: ***** FILE SYSTEM WAS MODIFIED *****
Following is a successful attempt, but instead of starting with a 1GB
ext4 file system, I start with a 2GB ext4 file system. There is no
error after resizing to 2TB:
linux-ek23:/mnt # lvcreate -L 2G -n main storage
Logical volume "main" created
linux-ek23:/mnt # mkfs.ext4 /dev/storage/main
mke2fs 1.42.6 (21-Sep-2012)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=128 blocks, Stripe width=640 blocks
131072 inodes, 524288 blocks
26214 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=536870912
16 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912
Allocating group tables: done
Writing inode tables: done
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done
linux-ek23:/mnt # lvextend -L 2T /dev/storage/main
Extending logical volume main to 2.00 TiB
Logical volume main successfully resized
linux-ek23:/mnt # fsck.ext4 /dev/storage/main
e2fsck 1.42.6 (21-Sep-2012)
/dev/storage/main: clean, 11/131072 files, 25388/524288 blocks
linux-ek23:/mnt # resize2fs -p /dev/storage/main
resize2fs 1.42.6 (21-Sep-2012)
Resizing the filesystem on /dev/storage/main to 536870912 (4k) blocks.
The filesystem on /dev/storage/main is now 536870912 blocks long.
linux-ek23:/mnt # fsck.ext4 /dev/storage/main
e2fsck 1.42.6 (21-Sep-2012)
/dev/storage/main: clean, 11/134217728 files, 8440346/536870912 blocks
I know it is unusual to start so small and expand like this, but I
doubt I am the only one to have experienced this. How could I have
prepared my small 1GB file system so that it would be able to handle
an expansion past 2TB? Any help would be appreciated.
Regards,
--
John Jolly - [email protected] - http://john.jolly.name
On 3/27/13 10:44 PM, John Jolly wrote:
> All,
>
> I recently had a major data loss when expanding an LVM logical volume
> past 2TB in size. I have preserved the 2.25TB just in case I might be
> able to recover some data. But this backup is not my reason for
> writing.
>
> I have done several tests and these are the results. First the error
> condition, where I create a 1GB file system, expand the volume to 2TB,
> resize the file system and have it become corrupted:
I checked the recent code in the master & pu branches of git, and the
same problem persists.
...
> I know it is unusual to start so small and expand like this, but I
> doubt I am the only one to have experienced this. How could I have
> prepared my small 1GB file system so that it would be able to handle
> an expansion past 2TB? Any help would be appreciated.
not sure; you haven't done anything wrong, this simply looks like
a resize bug, of which there have been a few lately I'm afraid.
-Eric
> Regards,
> --
> John Jolly - [email protected] - http://john.jolly.name
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
On Wed, Mar 27, 2013 at 10:57:25PM -0500, Eric Sandeen wrote:
> > I know it is unusual to start so small and expand like this, but I
> > doubt I am the only one to have experienced this. How could I have
> > prepared my small 1GB file system so that it would be able to handle
> > an expansion past 2TB? Any help would be appreciated.
>
> not sure; you haven't done anything wrong, this simply looks like
> a resize bug, of which there have been a few lately I'm afraid.
Note that what might have corrupted your file system may be different
from the reproduction case which you've sent (which I've also
confirmed). There was a number of off-line resize bugs fixed between
1.42.6 and 1.42.7. In general, off-line resize has been a bit more
problematic recently than on-line resize. (Although if you're using
an older kernel, online resize didn't work when starting with a
super-small file system and then grow it to something large.)
Thanks for sending this bug report; we'll get right on it.
Can you say a bit more about the resize2fs failure you had with your
production file system? Was it a similar "start really small and grow
very big" pattern? Or something else?
- Ted
Signed-off-by: "Theodore Ts'o" <[email protected]>
---
tests/r_1024_small_bg/script | 27 +++++++
tests/r_64bit_big_expand/script | 27 +++++++
tests/r_bigalloc_big_expand/script | 27 +++++++
tests/r_ext4_big_expand/script | 27 +++++++
tests/r_ext4_small_bg/script | 27 +++++++
tests/scripts/resize_test | 139 +++++++++++++++++++++++++++++++++++++
6 files changed, 274 insertions(+)
create mode 100644 tests/r_1024_small_bg/script
create mode 100644 tests/r_64bit_big_expand/script
create mode 100644 tests/r_bigalloc_big_expand/script
create mode 100644 tests/r_ext4_big_expand/script
create mode 100644 tests/r_ext4_small_bg/script
create mode 100755 tests/scripts/resize_test
diff --git a/tests/r_1024_small_bg/script b/tests/r_1024_small_bg/script
new file mode 100644
index 0000000..fafcf91
--- /dev/null
+++ b/tests/r_1024_small_bg/script
@@ -0,0 +1,27 @@
+if test -x $RESIZE2FS_EXE -a -x $DEBUGFS_EXE; then
+
+test_description="ext2 1024 blocksize with small block groups"
+FEATURES="-t ext2 -O ^resize_inode -b 1024 -g 1024"
+SIZE_1=64M
+SIZE_2=2G
+LOG=$test_name.log
+E2FSCK=../e2fsck/e2fsck
+
+
+. $cmd_dir/scripts/resize_test
+
+if resize_test
+then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ touch $test_name.failed
+fi
+
+unset FEATURES SIZE_1 SIZE_2 LOG E2FSCK
+
+else #if test -x $RESIZE2FS_EXE -a -x $DEBUGFS_EXE; then
+ echo "$test_name: $test_description: skipped"
+fi
+
diff --git a/tests/r_64bit_big_expand/script b/tests/r_64bit_big_expand/script
new file mode 100644
index 0000000..3b34a62
--- /dev/null
+++ b/tests/r_64bit_big_expand/script
@@ -0,0 +1,27 @@
+if test -x $RESIZE2FS_EXE -a -x $DEBUGFS_EXE; then
+
+test_description="very large fs growth using ext4 w/64bit"
+FEATURES="-t ext4 -O 64bit"
+SIZE_1=512M
+SIZE_2=2T
+LOG=$test_name.log
+E2FSCK=../e2fsck/e2fsck
+
+
+. $cmd_dir/scripts/resize_test
+
+if resize_test
+then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ touch $test_name.failed
+fi
+
+unset FEATURES SIZE_1 SIZE_2 LOG E2FSCK
+
+else #if test -x $RESIZE2FS_EXE -a -x $DEBUGFS_EXE; then
+ echo "$test_name: $test_description: skipped"
+fi
+
diff --git a/tests/r_bigalloc_big_expand/script b/tests/r_bigalloc_big_expand/script
new file mode 100644
index 0000000..2b9cc63
--- /dev/null
+++ b/tests/r_bigalloc_big_expand/script
@@ -0,0 +1,27 @@
+if test -x $RESIZE2FS_EXE -a -x $DEBUGFS_EXE; then
+
+test_description="ext4 with bigalloc"
+FEATURES="-t ext4 -O bigalloc"
+SIZE_1=512M
+SIZE_2=2T
+LOG=$test_name.log
+E2FSCK=../e2fsck/e2fsck
+RESIZE2FS_OPTS=-f
+
+. $cmd_dir/scripts/resize_test
+
+if resize_test
+then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ touch $test_name.failed
+fi
+
+unset FEATURES SIZE_1 SIZE_2 LOG E2FSCK
+
+else #if test -x $RESIZE2FS_EXE -a -x $DEBUGFS_EXE; then
+ echo "$test_name: $test_description: skipped"
+fi
+
diff --git a/tests/r_ext4_big_expand/script b/tests/r_ext4_big_expand/script
new file mode 100644
index 0000000..fb31d7a
--- /dev/null
+++ b/tests/r_ext4_big_expand/script
@@ -0,0 +1,27 @@
+if test -x $RESIZE2FS_EXE -a -x $DEBUGFS_EXE; then
+
+test_description="very large fs growth using ext4"
+FEATURES="-t ext4"
+SIZE_1=512M
+SIZE_2=2T
+LOG=$test_name.log
+E2FSCK=../e2fsck/e2fsck
+
+
+. $cmd_dir/scripts/resize_test
+
+if resize_test
+then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ touch $test_name.failed
+fi
+
+unset FEATURES SIZE_1 SIZE_2 LOG E2FSCK
+
+else #if test -x $RESIZE2FS_EXE -a -x $DEBUGFS_EXE; then
+ echo "$test_name: $test_description: skipped"
+fi
+
diff --git a/tests/r_ext4_small_bg/script b/tests/r_ext4_small_bg/script
new file mode 100644
index 0000000..553cbd8
--- /dev/null
+++ b/tests/r_ext4_small_bg/script
@@ -0,0 +1,27 @@
+if test -x $RESIZE2FS_EXE -a -x $DEBUGFS_EXE; then
+
+test_description="ext4 1024 blocksize with small block groups"
+FEATURES="-t ext4 -O ^resize_inode -b 1024 -g 512"
+SIZE_1=64M
+SIZE_2=2G
+LOG=$test_name.log
+E2FSCK=../e2fsck/e2fsck
+
+
+. $cmd_dir/scripts/resize_test
+
+if resize_test
+then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ touch $test_name.failed
+fi
+
+unset FEATURES SIZE_1 SIZE_2 LOG E2FSCK
+
+else #if test -x $RESIZE2FS_EXE -a -x $DEBUGFS_EXE; then
+ echo "$test_name: $test_description: skipped"
+fi
+
diff --git a/tests/scripts/resize_test b/tests/scripts/resize_test
new file mode 100755
index 0000000..964150e
--- /dev/null
+++ b/tests/scripts/resize_test
@@ -0,0 +1,139 @@
+#!/bin/sh
+
+resize_test () {
+
+rm -f $TMPFILE
+touch $TMPFILE
+echo $MKE2FS $FEATURES -qF $TMPFILE $SIZE_1 > $LOG
+$MKE2FS $FEATURES -qF $TMPFILE $SIZE_1 >> $LOG
+
+OUT_TMP=$(mktemp -t csum-tmp.XXXXXX)
+
+date > $OUT_TMP
+cat $E2FSCK >> $OUT_TMP
+echo $CRCSUM $OUT_TMP >> $LOG 2>&1
+CSUM_1=$($CRCSUM $OUT_TMP)
+echo Checksum is $CSUM_1 >> $LOG
+
+echo Setting up file system >> $LOG
+$DEBUGFS -w $TMPFILE >> $LOG 2>&1 << EOF
+mkdir test
+cd test
+write $OUT_TMP e2fsck
+ls /test
+stat /test/e2fsck
+quit
+EOF
+echo " " >> $LOG
+
+rm -f $OUT_TMP
+
+echo $FSCK -fy $TMPFILE >> $LOG 2>&1
+$FSCK -fy $TMPFILE >> $LOG 2>&1
+
+echo $RESIZE2FS $RESIZE2FS_OPTS -d 31 $TMPFILE $SIZE_2 >> $LOG 2>&1
+if ! $RESIZE2FS $RESIZE2FS_OPTS -d 31 $TMPFILE $SIZE_2 >> $LOG 2>&1
+then
+ return 1
+fi
+
+echo $FSCK -fp $TMPFILE >> $LOG 2>&1
+if ! $FSCK -fp $TMPFILE >> $LOG 2>&1
+then
+ dumpe2fs $TMPFILE >> $LOG
+ return 1
+fi
+
+echo $DEBUGFS -R "dump /test/e2fsck $OUT_TMP" $TMPFILE >> $LOG 2>&1
+$DEBUGFS -R "dump /test/e2fsck $OUT_TMP" $TMPFILE >> $LOG 2>&1
+
+echo $CRCSUM $OUT_TMP >> $LOG 2>&1
+CSUM_2=$($CRCSUM $OUT_TMP)
+echo Checksum is $CSUM_2 >> $LOG
+
+if test "$CSUM_1" != "$CSUM_2"
+then
+ return 1
+fi
+
+echo $RESIZE2FS $RESIZE2FS_OPTS -d 31 -M $TMPFILE $SIZE_2 >> $LOG 2>&1
+if ! $RESIZE2FS $RESIZE2FS_OPTS -d 31 -M $TMPFILE $SIZE_2 >> $LOG 2>&1
+then
+ return 1
+fi
+
+echo $FSCK -fp $TMPFILE >> $LOG 2>&1
+if ! $FSCK -fp $TMPFILE >> $LOG 2>&1
+then
+ dumpe2fs $TMPFILE >> $LOG
+ return 1
+fi
+
+echo $DEBUGFS -R "dump /test/e2fsck $OUT_TMP" $TMPFILE >> $LOG 2>&1
+$DEBUGFS -R "dump /test/e2fsck $OUT_TMP" $TMPFILE >> $LOG 2>&1
+
+echo $CRCSUM $OUT_TMP >> $LOG 2>&1
+CSUM_2=$($CRCSUM $OUT_TMP)
+echo Checksum is $CSUM_2 >> $LOG
+
+if test "$CSUM_1" != "$CSUM_2"
+then
+ return 1
+fi
+
+echo $RESIZE2FS $RESIZE2FS_OPTS -d 31 -M $TMPFILE $SIZE_2 >> $LOG 2>&1
+if ! $RESIZE2FS $RESIZE2FS_OPTS -d 31 -M $TMPFILE $SIZE_2 >> $LOG 2>&1
+then
+ return 1
+fi
+
+echo $FSCK -fp $TMPFILE >> $LOG 2>&1
+if ! $FSCK -fp $TMPFILE >> $LOG 2>&1
+then
+ dumpe2fs $TMPFILE >> $LOG
+ return 1
+fi
+
+echo $DEBUGFS -R "dump /test/e2fsck $OUT_TMP" $TMPFILE >> $LOG 2>&1
+$DEBUGFS -R "dump /test/e2fsck $OUT_TMP" $TMPFILE >> $LOG 2>&1
+
+echo $CRCSUM $OUT_TMP >> $LOG 2>&1
+CSUM_2=$($CRCSUM $OUT_TMP)
+echo Checksum is $CSUM_2 >> $LOG
+
+if test "$CSUM_1" != "$CSUM_2"
+then
+ return 1
+fi
+
+echo $RESIZE2FS $RESIZE2FS_OPTS -d 31 -M $TMPFILE $SIZE_2 >> $LOG 2>&1
+if ! $RESIZE2FS $RESIZE2FS_OPTS -d 31 -M $TMPFILE $SIZE_2 >> $LOG 2>&1
+then
+ return 1
+fi
+
+echo $FSCK -fp $TMPFILE >> $LOG 2>&1
+if ! $FSCK -fp $TMPFILE >> $LOG 2>&1
+then
+ dumpe2fs $TMPFILE >> $LOG
+ return 1
+fi
+
+echo $DEBUGFS -R "dump /test/e2fsck $OUT_TMP" $TMPFILE >> $LOG 2>&1
+$DEBUGFS -R "dump /test/e2fsck $OUT_TMP" $TMPFILE >> $LOG 2>&1
+
+echo $CRCSUM $OUT_TMP >> $LOG 2>&1
+CSUM_2=$($CRCSUM $OUT_TMP)
+echo Checksum is $CSUM_2 >> $LOG
+
+rm $OUT_TMP
+unset OUT_TMP
+
+if test "$CSUM_1" != "$CSUM_2"
+then
+ return 1
+fi
+
+return 0
+
+}
--
1.7.12.rc0.22.gcdd159b
This fixes the reported failure when resizing an ext4 file system which
started out a as a very small file system, which is then resizied to a
very big file system. I've also added 5 more resize2fs tests to the
e2fsprogs regression test suite.
Theodore Ts'o (4):
resize2fs: fix off-line resize of file systems with flex_bg &&
!resize_inode
mke2fs: don't display bigalloc/quota fs feature warnings in quiet
mode
tests: create crcsum progam to support resizing tests
tests: add more tests for off-line resizing
misc/mke2fs.c | 6 +-
resize/resize2fs.c | 58 ++++++++++++----
tests/progs/Makefile.in | 6 +-
tests/progs/crcsum.c | 70 +++++++++++++++++++
tests/r_1024_small_bg/script | 27 +++++++
tests/r_64bit_big_expand/script | 27 +++++++
tests/r_bigalloc_big_expand/script | 27 +++++++
tests/r_ext4_big_expand/script | 27 +++++++
tests/r_ext4_small_bg/script | 27 +++++++
tests/scripts/resize_test | 139 +++++++++++++++++++++++++++++++++++++
tests/test_config | 1 +
11 files changed, 399 insertions(+), 16 deletions(-)
create mode 100644 tests/progs/crcsum.c
create mode 100644 tests/r_1024_small_bg/script
create mode 100644 tests/r_64bit_big_expand/script
create mode 100644 tests/r_bigalloc_big_expand/script
create mode 100644 tests/r_ext4_big_expand/script
create mode 100644 tests/r_ext4_small_bg/script
create mode 100755 tests/scripts/resize_test
--
1.7.12.rc0.22.gcdd159b
The only checksum program which we can reliably count upon being
installed on all systems is "sum", which is not a particular robust
checksum. The problem with using md5sum or sha1sum is it hat it may
not be installed on all systems. So create a crcsum program which is
used so we can validate that a data file on a resized file system has
not been corrupted.
Signed-off-by: "Theodore Ts'o" <[email protected]>
---
tests/progs/Makefile.in | 6 ++++-
tests/progs/crcsum.c | 70 +++++++++++++++++++++++++++++++++++++++++++++++++
tests/test_config | 1 +
3 files changed, 76 insertions(+), 1 deletion(-)
create mode 100644 tests/progs/crcsum.c
diff --git a/tests/progs/Makefile.in b/tests/progs/Makefile.in
index 0e28192..e3c1ef4 100644
--- a/tests/progs/Makefile.in
+++ b/tests/progs/Makefile.in
@@ -13,7 +13,7 @@ INSTALL = @INSTALL@
MK_CMDS= _SS_DIR_OVERRIDE=../../lib/ss ../../lib/ss/mk_cmds
-PROGS= test_icount
+PROGS= test_icount crcsum
TEST_REL_OBJS= test_rel.o test_rel_cmds.o
@@ -34,6 +34,10 @@ test_rel: $(TEST_REL_OBJS) $(DEPLIBS)
$(E) " LD $@"
$(Q) $(LD) $(ALL_LDFLAGS) -o test_rel $(TEST_REL_OBJS) $(LIBS)
+crcsum: crcsum.o $(DEPLIBS)
+ $(E) " LD $@"
+ $(Q) $(LD) $(ALL_LDFLAGS) -o crcsum crcsum.o $(LIBS)
+
test_rel_cmds.c: test_rel_cmds.ct
$(E) " MK_CMDS $@"
$(Q) $(MK_CMDS) $(srcdir)/test_rel_cmds.ct
diff --git a/tests/progs/crcsum.c b/tests/progs/crcsum.c
new file mode 100644
index 0000000..bee979b
--- /dev/null
+++ b/tests/progs/crcsum.c
@@ -0,0 +1,70 @@
+/*
+ * crcsum.c
+ *
+ * Copyright (C) 2013 Theodore Ts'o.
+ *
+ * %Begin-Header%
+ * This file may be redistributed under the terms of the GNU Public
+ * License.
+ * %End-Header%
+ */
+
+#include "config.h"
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <string.h>
+#include <unistd.h>
+#ifdef HAVE_GETOPT_H
+#include <getopt.h>
+#endif
+#include <fcntl.h>
+
+#include "et/com_err.h"
+#include "ss/ss.h"
+#include "ext2fs/ext2fs.h"
+
+
+int main(int argc, char **argv)
+{
+ int c;
+ uint32_t crc = ~0;
+ uint32_t (*csum_func)(uint32_t crc, unsigned char const *p,
+ size_t len);
+ FILE *f;
+
+ csum_func = ext2fs_crc32c_le;
+
+ while ((c = getopt (argc, argv, "B")) != EOF) {
+ switch (c) {
+ case 'B':
+ csum_func = ext2fs_crc32c_be;
+ break;
+ default:
+ com_err(argv[0], 0, "Usage: crcsum [-b] [file]\n");
+ return 1;
+ }
+ }
+
+ if (optind == argc)
+ f = stdin;
+ else {
+ f = fopen(argv[optind], "r");
+ if (!f) {
+ com_err(argv[0], errno, "while trying to open %s\n",
+ argv[optind]);
+ exit(1);
+ }
+ }
+
+ while (!feof(f)) {
+ unsigned char buf[4096];
+
+ int c = fread(buf, 1, sizeof(buf), f);
+
+ if (c)
+ crc = csum_func(crc, buf, c);
+ }
+ printf("%u\n", crc);
+ return 0;
+}
diff --git a/tests/test_config b/tests/test_config
index 0ba8b5e..36b53b7 100644
--- a/tests/test_config
+++ b/tests/test_config
@@ -19,6 +19,7 @@ RESIZE2FS="$USE_VALGRIND $RESIZE2FS_EXE"
E2UNDO_EXE="../misc/e2undo"
TEST_REL=../tests/progs/test_rel
TEST_ICOUNT=../tests/progs/test_icount
+CRCSUM=../tests/progs/crcsum
LD_LIBRARY_PATH=../lib:../lib/ext2fs:../lib/e2p:../lib/et:../lib/ss
DYLD_LIBRARY_PATH=../lib:../lib/ext2fs:../lib/e2p:../lib/et:../lib/ss
export LD_LIBRARY_PATH
--
1.7.12.rc0.22.gcdd159b
When doing an off-line resize2fs of an initially very small file
system, it's possible to run out of reserved gdt blocks (which are
reserved via the resize inode). Once we run out, we need to move the
allocation bitmaps and inode table out of the way to grow the gdt
blocks. Unfortunately, when moving these metadata blocks, it was
possible that a block that had been just been newly allocated for a
new block group could also get allocated for a metadata block for an
existing block group that was being moved.
To prevent this, after we grow the gdt blocks and allocate the
metadata blocks for the new block groups, make sure all of these
blocks are marked as reserved.
Signed-off-by: "Theodore Ts'o" <[email protected]>
Reported-by: John Jolly <[email protected]>
---
resize/resize2fs.c | 58 ++++++++++++++++++++++++++++++++++++++++++------------
1 file changed, 45 insertions(+), 13 deletions(-)
diff --git a/resize/resize2fs.c b/resize/resize2fs.c
index c9458ea..7c4f86a 100644
--- a/resize/resize2fs.c
+++ b/resize/resize2fs.c
@@ -51,6 +51,8 @@ static errcode_t move_itables(ext2_resize_t rfs);
static errcode_t fix_resize_inode(ext2_filsys fs);
static errcode_t ext2fs_calculate_summary_stats(ext2_filsys fs);
static errcode_t fix_sb_journal_backup(ext2_filsys fs);
+static errcode_t mark_table_blocks(ext2_filsys fs,
+ ext2fs_block_bitmap bmap);
/*
* Some helper CPP macros
@@ -306,9 +308,6 @@ static void free_gdp_blocks(ext2_filsys fs,
/*
* This routine is shared by the online and offline resize routines.
* All of the information which is adjusted in memory is done here.
- *
- * The reserve_blocks parameter is only needed when shrinking the
- * filesystem.
*/
errcode_t adjust_fs_info(ext2_filsys fs, ext2_filsys old_fs,
ext2fs_block_bitmap reserve_blocks, blk64_t new_size)
@@ -407,12 +406,21 @@ retry:
real_end = (((blk64_t) EXT2_BLOCKS_PER_GROUP(fs->super) *
fs->group_desc_count)) - 1 +
fs->super->s_first_data_block;
- retval = ext2fs_resize_block_bitmap2(ext2fs_blocks_count(fs->super)-1,
- real_end, fs->block_map);
-
+ retval = ext2fs_resize_block_bitmap2(new_size - 1,
+ real_end, fs->block_map);
if (retval) goto errout;
/*
+ * If we are growing the file system, also grow the size of
+ * the reserve_blocks bitmap
+ */
+ if (reserve_blocks && new_size > ext2fs_blocks_count(old_fs->super)) {
+ retval = ext2fs_resize_block_bitmap2(new_size - 1,
+ real_end, reserve_blocks);
+ if (retval) goto errout;
+ }
+
+ /*
* Reallocate the group descriptors as necessary.
*/
if (old_fs->desc_blocks != fs->desc_blocks) {
@@ -512,6 +520,15 @@ retry:
else
old_desc_blocks = fs->desc_blocks +
fs->super->s_reserved_gdt_blocks;
+
+ /*
+ * If we changed the number of block_group descriptor blocks,
+ * we need to make sure they are all marked as reserved in the
+ * file systems's block allocation map.
+ */
+ for (i = 0; i < old_fs->group_desc_count; i++)
+ ext2fs_reserve_super_and_bgd(fs, i, fs->block_map);
+
for (i = old_fs->group_desc_count;
i < fs->group_desc_count; i++) {
memset(ext2fs_group_desc(fs, fs->group_desc, i), 0,
@@ -578,6 +595,17 @@ retry:
}
retval = 0;
+ /*
+ * Mark all of the metadata blocks as reserved so they won't
+ * get allocated by the call to ext2fs_allocate_group_table()
+ * in blocks_to_move(), where we allocate new blocks to
+ * replace those allocation bitmap and inode table blocks
+ * which have to get relocated to make space for an increased
+ * number of the block group descriptors.
+ */
+ if (reserve_blocks)
+ mark_table_blocks(fs, reserve_blocks);
+
errout:
return (retval);
}
@@ -716,6 +744,7 @@ static errcode_t mark_table_blocks(ext2_filsys fs,
ext2fs_block_bitmap bmap)
{
dgrp_t i;
+ blk64_t blk;
for (i = 0; i < fs->group_desc_count; i++) {
ext2fs_reserve_super_and_bgd(fs, i, bmap);
@@ -723,21 +752,24 @@ static errcode_t mark_table_blocks(ext2_filsys fs,
/*
* Mark the blocks used for the inode table
*/
- ext2fs_mark_block_bitmap_range2(bmap,
- ext2fs_inode_table_loc(fs, i),
- fs->inode_blocks_per_group);
+ blk = ext2fs_inode_table_loc(fs, i);
+ if (blk)
+ ext2fs_mark_block_bitmap_range2(bmap, blk,
+ fs->inode_blocks_per_group);
/*
* Mark block used for the block bitmap
*/
- ext2fs_mark_block_bitmap2(bmap,
- ext2fs_block_bitmap_loc(fs, i));
+ blk = ext2fs_block_bitmap_loc(fs, i);
+ if (blk)
+ ext2fs_mark_block_bitmap2(bmap, blk);
/*
* Mark block used for the inode bitmap
*/
- ext2fs_mark_block_bitmap2(bmap,
- ext2fs_inode_bitmap_loc(fs, i));
+ blk = ext2fs_inode_bitmap_loc(fs, i);
+ if (blk)
+ ext2fs_mark_block_bitmap2(bmap, blk);
}
return 0;
}
--
1.7.12.rc0.22.gcdd159b
Signed-off-by: "Theodore Ts'o" <[email protected]>
---
misc/mke2fs.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/misc/mke2fs.c b/misc/mke2fs.c
index fe5ce7d..7448255 100644
--- a/misc/mke2fs.c
+++ b/misc/mke2fs.c
@@ -1942,13 +1942,15 @@ profile_error:
exit(1);
}
- if (fs_param.s_feature_ro_compat & EXT4_FEATURE_RO_COMPAT_BIGALLOC)
+ if (!quiet &&
+ (fs_param.s_feature_ro_compat & EXT4_FEATURE_RO_COMPAT_BIGALLOC))
fprintf(stderr, _("\nWarning: the bigalloc feature is still "
"under development\n"
"See https://ext4.wiki.kernel.org/"
"index.php/Bigalloc for more information\n\n"));
- if (fs_param.s_feature_ro_compat & EXT4_FEATURE_RO_COMPAT_QUOTA)
+ if (!quiet &&
+ (fs_param.s_feature_ro_compat & EXT4_FEATURE_RO_COMPAT_QUOTA))
fprintf(stderr, _("\nWarning: the quota feature is still "
"under development\n"
"See https://ext4.wiki.kernel.org/"
--
1.7.12.rc0.22.gcdd159b