From: Eryu Guan Subject: Re: [PATCH v5] ext4: Ext4 online resize with bigalloc tests. Date: Thu, 25 Jan 2018 13:00:19 +0800 Message-ID: <20180125050019.GF18267@eguan.usersys.redhat.com> References: <20180124063748.GA16904@eguan.usersys.redhat.com> <20180124225812.69852-1-harshads@google.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: fstests@vger.kernel.org, linux-ext4@vger.kernel.org To: harshads Return-path: Content-Disposition: inline In-Reply-To: <20180124225812.69852-1-harshads@google.com> Sender: fstests-owner@vger.kernel.org List-Id: linux-ext4.vger.kernel.org On Wed, Jan 24, 2018 at 02:58:12PM -0800, harshads wrote: > Add tests to verify Ext4 online resizing feature with bigalloc feature > enabled. We test various resizing scenarios with different cluster > sizes. > > Signed-off-by: Harshad Shirwadkar Thanks for all the revisions! I made some minor edits and queued it for next release. Please see below. > --- > tests/ext4/030 | 171 ++++++++++++++++++++++++++++++++++++++++++++++++++++ > tests/ext4/030.out | 172 +++++++++++++++++++++++++++++++++++++++++++++++++++++ > tests/ext4/group | 1 + > 3 files changed, 344 insertions(+) > create mode 100755 tests/ext4/030 > create mode 100644 tests/ext4/030.out > > diff --git a/tests/ext4/030 b/tests/ext4/030 > new file mode 100755 > index 00000000..1d42b514 > --- /dev/null > +++ b/tests/ext4/030 > @@ -0,0 +1,171 @@ > +#! /bin/bash > +# FS QA Test ext4/030 > +# > +# Ext4 online resize tests of small and crucial resizes with bigalloc > +# feature. > +# > +#----------------------------------------------------------------------- > +# Copyright (c) 2017 Google, Inc. All Rights Reserved. > +# > +# Author: Harshad Shirwadkar > +# > +# This program is free software; you can redistribute it and/or > +# modify it under the terms of the GNU General Public License as > +# published by the Free Software Foundation. > +# > +# This program is distributed in the hope that it would be useful, > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +# GNU General Public License for more details. > +# > +# You should have received a copy of the GNU General Public License > +# along with this program; if not, write the Free Software Foundation, > +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA > +#----------------------------------------------------------------------- > + > +seq=`basename $0` > +seqres=$RESULT_DIR/$seq > +echo "QA output created by $seq" > + > +here=`pwd` > +tmp=/tmp/$$ > +status=1 # failure is the default! > +trap "_cleanup; exit \$status" 0 1 2 3 15 > + > +BLK_SIZ=4096 > +CLUSTER_SIZ=4096 > + > +IMG_FILE=$SCRATCH_DIR/$seq.fs > +IMG_MNT=$SCRATCH_DIR/$seq.mnt > + > +## Num clusters to blocks. > +c2b() > +{ > + local clusters=$1 > + echo $(($clusters * $CLUSTER_SIZ / $BLK_SIZ)) > +} > + > +_ext4_online_resize() I removed the leading underscore for local function. > +{ > + ## sizes are in number of blocks. > + local original_size=$1 > + local final_size=$2 > + local check_if_supported=$([ "$3" = "1" ] && echo "1" || echo "0") local check_if_supported=${3:-0} > + > + ## Start with a clean image file. > + echo "" > ${IMG_FILE} > + echo "+++ truncate image file to $final_size" | \ > + tee -a $seqres.full > + $XFS_IO_PROG -f -c "truncate $(($final_size * $BLK_SIZ))" ${IMG_FILE} > + LOOP_DEVICE=`_create_loop_device $IMG_FILE || _fail "losetup failed"` No need to call _fail here, _create_loop_device already did that. > + > + echo "+++ create fs on image file $original_size" | \ > + tee -a $seqres.full > + > + ${MKFS_PROG}.${FSTYP} -F -O bigalloc,resize_inode -C $CLUSTER_SIZ \ > + -b $BLK_SIZ ${LOOP_DEVICE} $original_size >> \ > + $seqres.full 2>&1 || _fail "mkfs failed" > + > + echo "+++ mount image file" | tee -a $seqres.full > + $MOUNT_PROG -t ${FSTYP} ${LOOP_DEVICE} ${IMG_MNT} > \ > + /dev/null 2>&1 || _fail "mount failed" > + > + echo "+++ resize fs to $final_size" | tee -a $seqres.full > + > + resize_output=$(resize2fs -f ${LOOP_DEVICE} $final_size 2>&1) I changed this to let resize2fs dump output to a tmp file, saving multi-line outputs into a variable would cause subtle problems that are hard to find/debug. e.g. echoing a such variable to a pipe without proper quotes would change the output in subtle ways, and we did hit such problems before. > + if [ $? -ne 0 ]; then > + if [ $check_if_supported -eq 1 ]; then > + echo $resize_output | \ > + grep "resize2fs: Operation not supported While checking for on-line resizing support" \ > + && _notrun "online resizing not supported with bigalloc" > + fi > + _fail "resize failed" > + fi > + echo $resize_output >> $seqres.full Then grep the tmp file and append it to $seqres.full. Thanks, Eryu > + echo "+++ umount fs" | tee -a $seqres.full > + $UMOUNT_PROG ${IMG_MNT} > + > + echo "+++ check fs" | tee -a $seqres.full > + _check_generic_filesystem $LOOP_DEVICE >> $seqres.full 2>&1 || \ > + _fail "fsck should not fail" > + _destroy_loop_device $LOOP_DEVICE && LOOP_DEVICE= > +} > + > +_cleanup() > +{ > + cd / > + [ -n "$LOOP_DEVICE" ] && _destroy_loop_device $LOOP_DEVICE > /dev/null 2>&1 > + rm -f $tmp.* > + $UMOUNT_PROG ${IMG_MNT} > /dev/null 2>&1 > + rm -f ${IMG_FILE} > /dev/null 2>&1 > +} > + > +# get standard environment and checks > +. ./common/rc > + > +# remove previous $seqres.full before test > +rm -f $seqres.full > + > +# real QA test starts here > +_supported_fs ext4 > +_supported_os Linux > + > +_require_loop > +_require_scratch > +# We use resize_inode to make sure that block group descriptor table > +# can be extended. > +_require_scratch_ext4_feature "bigalloc,resize_inode" > + > +_scratch_mkfs >>$seqres.full 2>&1 > +_scratch_mount > + > +rm -f $seqres.full > + > +mkdir -p $IMG_MNT || _fail "cannot create loopback mount point" > + > +# Check if online resizing with bigalloc is supported by the kernel > +_ext4_online_resize 4096 8192 1 > + > +## We perform resizing to various multiples of block group sizes to > +## ensure that we cover maximum edge cases in the kernel code. > +for CLUSTER_SIZ in 4096 16384 65536; do > + echo "++ set cluster size to $CLUSTER_SIZ" | tee -a $seqres.full > + > + ## > + ## Extend last group tests > + ## > + > + ## Extending a 1/2 block group to a 2/3 block group. > + _ext4_online_resize $(c2b 16384) $(c2b 24576) > + ## Extending a 2/3rd block group to one cluster less than a > + ## full block group. > + _ext4_online_resize $(c2b 24576) $(c2b 32767) > + ## Extending a 2/3rd block group to a full block group. > + _ext4_online_resize $(c2b 24576) $(c2b 32768) > + > + ## > + ## Extend last group and add more block groups. > + ## > + > + ## Extending a 2/3rd block group to 2 block groups. > + _ext4_online_resize $(c2b 24576) $(c2b 65536) > + ## Extending a 2/3rd block group to 15 block groups and one > + ## cluster. > + _ext4_online_resize $(c2b 24576) $(c2b 491521) > + ## Extending a 2/3rd block group to 15 and a half block groups. > + _ext4_online_resize $(c2b 24576) $(c2b 507904) > + ## Extending a 2/3rd block group to 16 block groups. > + _ext4_online_resize $(c2b 24576) $(c2b 524288) > + ## Extending a 2/3rd block group to 160 block groups. > + _ext4_online_resize $(c2b 24576) $(c2b 5242880) > + > + ## > + ## Convert to meta_bg. > + ## > + > + ## Extending a 2/3rd block group to 1280 block groups. > + _ext4_online_resize $(c2b 24576) $(c2b 41943040) > +done > + > +status=0 > +exit > diff --git a/tests/ext4/030.out b/tests/ext4/030.out > new file mode 100644 > index 00000000..49304654 > --- /dev/null > +++ b/tests/ext4/030.out > @@ -0,0 +1,172 @@ > +QA output created by 030 > ++++ truncate image file to 8192 > ++++ create fs on image file 4096 > ++++ mount image file > ++++ resize fs to 8192 > ++++ umount fs > ++++ check fs > +++ set cluster size to 4096 > ++++ truncate image file to 24576 > ++++ create fs on image file 16384 > ++++ mount image file > ++++ resize fs to 24576 > ++++ umount fs > ++++ check fs > ++++ truncate image file to 32767 > ++++ create fs on image file 24576 > ++++ mount image file > ++++ resize fs to 32767 > ++++ umount fs > ++++ check fs > ++++ truncate image file to 32768 > ++++ create fs on image file 24576 > ++++ mount image file > ++++ resize fs to 32768 > ++++ umount fs > ++++ check fs > ++++ truncate image file to 65536 > ++++ create fs on image file 24576 > ++++ mount image file > ++++ resize fs to 65536 > ++++ umount fs > ++++ check fs > ++++ truncate image file to 491521 > ++++ create fs on image file 24576 > ++++ mount image file > ++++ resize fs to 491521 > ++++ umount fs > ++++ check fs > ++++ truncate image file to 507904 > ++++ create fs on image file 24576 > ++++ mount image file > ++++ resize fs to 507904 > ++++ umount fs > ++++ check fs > ++++ truncate image file to 524288 > ++++ create fs on image file 24576 > ++++ mount image file > ++++ resize fs to 524288 > ++++ umount fs > ++++ check fs > ++++ truncate image file to 5242880 > ++++ create fs on image file 24576 > ++++ mount image file > ++++ resize fs to 5242880 > ++++ umount fs > ++++ check fs > ++++ truncate image file to 41943040 > ++++ create fs on image file 24576 > ++++ mount image file > ++++ resize fs to 41943040 > ++++ umount fs > ++++ check fs > +++ set cluster size to 16384 > ++++ truncate image file to 98304 > ++++ create fs on image file 65536 > ++++ mount image file > ++++ resize fs to 98304 > ++++ umount fs > ++++ check fs > ++++ truncate image file to 131068 > ++++ create fs on image file 98304 > ++++ mount image file > ++++ resize fs to 131068 > ++++ umount fs > ++++ check fs > ++++ truncate image file to 131072 > ++++ create fs on image file 98304 > ++++ mount image file > ++++ resize fs to 131072 > ++++ umount fs > ++++ check fs > ++++ truncate image file to 262144 > ++++ create fs on image file 98304 > ++++ mount image file > ++++ resize fs to 262144 > ++++ umount fs > ++++ check fs > ++++ truncate image file to 1966084 > ++++ create fs on image file 98304 > ++++ mount image file > ++++ resize fs to 1966084 > ++++ umount fs > ++++ check fs > ++++ truncate image file to 2031616 > ++++ create fs on image file 98304 > ++++ mount image file > ++++ resize fs to 2031616 > ++++ umount fs > ++++ check fs > ++++ truncate image file to 2097152 > ++++ create fs on image file 98304 > ++++ mount image file > ++++ resize fs to 2097152 > ++++ umount fs > ++++ check fs > ++++ truncate image file to 20971520 > ++++ create fs on image file 98304 > ++++ mount image file > ++++ resize fs to 20971520 > ++++ umount fs > ++++ check fs > ++++ truncate image file to 167772160 > ++++ create fs on image file 98304 > ++++ mount image file > ++++ resize fs to 167772160 > ++++ umount fs > ++++ check fs > +++ set cluster size to 65536 > ++++ truncate image file to 393216 > ++++ create fs on image file 262144 > ++++ mount image file > ++++ resize fs to 393216 > ++++ umount fs > ++++ check fs > ++++ truncate image file to 524272 > ++++ create fs on image file 393216 > ++++ mount image file > ++++ resize fs to 524272 > ++++ umount fs > ++++ check fs > ++++ truncate image file to 524288 > ++++ create fs on image file 393216 > ++++ mount image file > ++++ resize fs to 524288 > ++++ umount fs > ++++ check fs > ++++ truncate image file to 1048576 > ++++ create fs on image file 393216 > ++++ mount image file > ++++ resize fs to 1048576 > ++++ umount fs > ++++ check fs > ++++ truncate image file to 7864336 > ++++ create fs on image file 393216 > ++++ mount image file > ++++ resize fs to 7864336 > ++++ umount fs > ++++ check fs > ++++ truncate image file to 8126464 > ++++ create fs on image file 393216 > ++++ mount image file > ++++ resize fs to 8126464 > ++++ umount fs > ++++ check fs > ++++ truncate image file to 8388608 > ++++ create fs on image file 393216 > ++++ mount image file > ++++ resize fs to 8388608 > ++++ umount fs > ++++ check fs > ++++ truncate image file to 83886080 > ++++ create fs on image file 393216 > ++++ mount image file > ++++ resize fs to 83886080 > ++++ umount fs > ++++ check fs > ++++ truncate image file to 671088640 > ++++ create fs on image file 393216 > ++++ mount image file > ++++ resize fs to 671088640 > ++++ umount fs > ++++ check fs > diff --git a/tests/ext4/group b/tests/ext4/group > index 257bb646..f29d3de6 100644 > --- a/tests/ext4/group > +++ b/tests/ext4/group > @@ -32,6 +32,7 @@ > 027 auto quick fsmap > 028 auto quick fsmap > 029 auto quick fsmap > +030 auto quick ioctl resize > 271 auto rw quick > 301 aio auto ioctl rw stress defrag > 302 aio auto ioctl rw stress defrag > -- > 2.16.0.rc1.238.g530d649a79-goog >