Received-SPF: pass (google.com: domain of linux-ext4+bounces-1763-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
From: Luis Henriques <luis.henriques@linux.dev>
To: Andreas Dilger <adilger@dilger.ca>
Cc: Theodore Ts'o <tytso@mit.edu>,  Ext4 Developers List
 <linux-ext4@vger.kernel.org>
Subject: Re: [PATCH e2fsprogs] e2fsck: update quota accounting after
 directory optimization
In-Reply-To: <F7E15565-D316-4F02-93D4-CC081AB421C0@dilger.ca> (Andreas
	Dilger's message of "Wed, 27 Mar 2024 13:25:49 -0600")
References: <20240327154352.22648-1-luis.henriques@linux.dev>
	<F7E15565-D316-4F02-93D4-CC081AB421C0@dilger.ca>
Date: Wed, 27 Mar 2024 21:16:55 +0000
Message-ID: <87wmpn74u0.fsf@camandro.org>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain

Andreas Dilger <adilger@dilger.ca> writes:

> On Mar 27, 2024, at 9:43 AM, Luis Henriques (SUSE) <luis.henriques@linux.dev> wrote:
>> 
>> In "Pass 3A: Optimizing directories", a directory may have it's size reduced.
>> If that happens and quota is enabled in the filesystem, the quota information
>> will be incorrect because it doesn't take the rehash into account.
>> 
>> This patch simply updates the quota data accordingly, after the directory is
>> written and it's size has been updated.
>
> Hi Luis,
> thanks for the patch.  It looks reasonable, and might (partially) explain
> why quota accounting occasionally reports inconsistencies by a few blocks
> after running e2fsck.  You can add my Reviewed-by: to the patch.

Awesome, thank you for your review.  In the meantime, I have already a
similar fix for another case where quota inconsistencies occur in e2fsck.
I'm still running a few more tests, but I should be sending another patch
soon.

> Could you please include an e2fsck regression test for this, to confirm
> that it is working and continues to work in the future?  It should be
> possible to use something like the following to create a test case:
>
>     # cd tests
>     # make testnew
>     # tune2fs -O quota,project f_testnew/image
>     # mkdir /mnt/tmp
>     # mount -t ext4 -o loop f_testnew/image /mnt/tmp
>     # mkdir /mnt/tmp/subdir
>     # chattr -p 1000 -P /mnt/tmp/subdir
>     # touch /mnt/tmp/subdir/long-filenames-for-test-{1..1024}
>     # rm /mnt/tmp/subdir/long-filenames-for-test-{1..1024..2}
>     # umount /mnt/tmp
>     # echo "directory optimization updates quota" > f_testnew/name
>     # make testend
>     # mv f_testnew f_quota_shrinkdir
>
> and confirm in expect.[12] that the quota did not need to be repaired
> afterward by e2fsck (i.e. there shouldn't be any error messages about
> inconsistent quota).  Running this test with an unpatched e2fsck should
> report that the quotas had to be fixed and the test should fail.

Right, it makes sense to add a regression test, of course.  I'll need to
get familiar with the framework used but I'll have a look and see what I
can come up with.

> On a related note, it would be convenient if "make testnew" passed an
> environment variable (e.g. $TESTNEW_OPTS) to mke2fs so more options
> could be set at format time instead of using tune2fs afterward:
>
>     # TESTNEW_OPTS="-O quota,project" make testnew
>
> It isn't a big deal in this case, but might be useful in the future.

Hmm... Okay, as I said I'll need to get familiar with this but that seems
to make sense.

Cheers,
-- 
Luis

> Cheers, Andreas
>
>> Link: https://bugzilla.kernel.org/show_bug.cgi?id=218626
>> Signed-off-by: Luis Henriques (SUSE) <luis.henriques@linux.dev>
>> ---
>> e2fsck/rehash.c | 27 +++++++++++++++++++++------
>> 1 file changed, 21 insertions(+), 6 deletions(-)
>> 
>> diff --git a/e2fsck/rehash.c b/e2fsck/rehash.c
>> index c1da7d52724e..4847d172e5fe 100644
>> --- a/e2fsck/rehash.c
>> +++ b/e2fsck/rehash.c
>> @@ -987,14 +987,18 @@ errcode_t e2fsck_rehash_dir(e2fsck_t ctx, ext2_ino_t ino,
>> {
>> 	ext2_filsys 		fs = ctx->fs;
>> 	errcode_t		retval;
>> -	struct ext2_inode 	inode;
>> +	struct ext2_inode_large	inode;
>> 	char			*dir_buf = 0;
>> 	struct fill_dir_struct	fd = { NULL, NULL, 0, 0, 0, NULL,
>> 				       0, 0, 0, 0, 0, 0 };
>> 	struct out_dir		outdir = { 0, 0, 0, 0 };
>> -	struct name_cmp_ctx name_cmp_ctx = {0, NULL};
>> +	struct name_cmp_ctx	name_cmp_ctx = {0, NULL};
>> +	__u64			osize;
>> 
>> -	e2fsck_read_inode(ctx, ino, &inode, "rehash_dir");
>> +	e2fsck_read_inode_full(ctx, ino, EXT2_INODE(&inode),
>> +			       sizeof(inode), "rehash_dir");
>> +
>> +	osize = EXT2_I_SIZE(&inode);
>> 
>> 	if (ext2fs_has_feature_inline_data(fs->super) &&
>> 	   (inode.i_flags & EXT4_INLINE_DATA_FL))
>> @@ -1013,7 +1017,7 @@ errcode_t e2fsck_rehash_dir(e2fsck_t ctx, ext2_ino_t ino,
>> 	fd.ino = ino;
>> 	fd.ctx = ctx;
>> 	fd.buf = dir_buf;
>> -	fd.inode = &inode;
>> +	fd.inode = EXT2_INODE(&inode);
>> 	fd.dir = ino;
>> 	if (!ext2fs_has_feature_dir_index(fs->super) ||
>> 	    (inode.i_size / fs->blocksize) < 2)
>> @@ -1092,14 +1096,25 @@ resort:
>> 			goto errout;
>> 	}
>> 
>> -	retval = write_directory(ctx, fs, &outdir, ino, &inode, fd.compress);
>> +	retval = write_directory(ctx, fs, &outdir, ino, EXT2_INODE(&inode),
>> +				 fd.compress);
>> 	if (retval)
>> 		goto errout;
>> 
>> +	if ((osize > EXT2_I_SIZE(&inode)) &&
>> +	    (ino != quota_type2inum(PRJQUOTA, fs->super)) &&
>> +	    (ino != fs->super->s_orphan_file_inum) &&
>> +	    (ino == EXT2_ROOT_INO || ino >= EXT2_FIRST_INODE(ctx->fs->super)) &&
>> +	    !(inode.i_flags & EXT4_EA_INODE_FL)) {
>> +		quota_data_sub(ctx->qctx, &inode,
>> +			       ino, osize - EXT2_I_SIZE(&inode));
>> +	}
>> +
>> 	if (ctx->options & E2F_OPT_CONVERT_BMAP)
>> 		retval = e2fsck_rebuild_extents_later(ctx, ino);
>> 	else
>> -		retval = e2fsck_check_rebuild_extents(ctx, ino, &inode, pctx);
>> +		retval = e2fsck_check_rebuild_extents(ctx, ino,
>> +						      EXT2_INODE(&inode), pctx);
>> errout:
>> 	ext2fs_free_mem(&dir_buf);
>> 	ext2fs_free_mem(&fd.harray);
>> 
>
>
> Cheers, Andreas