Received-SPF: pass (google.com: domain of linux-ext4-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Andreas Dilger <adilger@dilger.ca>
Message-Id: <83D3872A-E269-477D-8D2E-CA2AF80D658A@dilger.ca>
Content-Type: multipart/signed;
 boundary="Apple-Mail=_353F2A8B-789B-4AAB-95B5-AA663F50BB99";
 protocol="application/pgp-signature"; micalg=pgp-sha256
Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\))
Subject: Re: [RFCv1 18/72] libext2fs: Add support to get average group count
Date:   Wed, 14 Dec 2022 14:24:50 -0700
In-Reply-To: <7e4f563719aee1970dd1058ca45b0609ae4c7c5f.1667822611.git.ritesh.list@gmail.com>
Cc:     Theodore Ts'o <tytso@mit.edu>, linux-ext4@vger.kernel.org,
        Harshad Shirwadkar <harshadshirwadkar@gmail.com>,
        Wang Shilong <wangshilong1991@gmail.com>, Li Xi <lixi@ddn.com>
To:     "Ritesh Harjani (IBM)" <ritesh.list@gmail.com>
References: <cover.1667822611.git.ritesh.list@gmail.com>
 <7e4f563719aee1970dd1058ca45b0609ae4c7c5f.1667822611.git.ritesh.list@gmail.com>
Precedence: bulk


--Apple-Mail=_353F2A8B-789B-4AAB-95B5-AA663F50BB99
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

On Nov 7, 2022, at 5:21 AM, Ritesh Harjani (IBM) <ritesh.list@gmail.com> =
wrote:
>=20
> From: Wang Shilong <wshilong@ddn.com>
>=20
> number of threads in pfsck should not exceed flex bg numbers.
> This patch adds the support in libext2fs to calculate
> ext2fs_get_avg_group() which returns an average group
> count which each thread has to scan.
>=20
> fs->fs_num_threads will be set by the client, in this case e2fsck.
> No. of threads will be passed along with -m option while running =
e2fsck.
> That will also set fs->fs_num_threads, which will help in controlling
> the amount of memory consumed to maintain in memory data structures =
(per
> thread) in case of multiple parallel threads (pfsck) to avoid oom.
>=20
> Signed-off-by: Wang Shilong <wshilong@ddn.com>
> Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
> ---
> lib/ext2fs/ext2fs.h | 32 +++++++++++++++++++++++++++++++-
> 1 file changed, 31 insertions(+), 1 deletion(-)
>=20
> diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
> index b1505f95..6b4926ce 100644
> --- a/lib/ext2fs/ext2fs.h
> +++ b/lib/ext2fs/ext2fs.h
> @@ -279,10 +279,11 @@ struct struct_ext2_filsys {
> 	int				cluster_ratio_bits;
> 	__u16				default_bitmap_type;
> 	__u16				pad;
> +	__u32				fs_num_threads;

=46rom later cleanup patch, fs_num_threads should just use "__u16 pad" =
field.

Otherwise, looks OK.


> 	/*
> 	 * Reserved for future expansion
> 	 */
> -	__u32				reserved[5];
> +	__u32				reserved[4];
>=20
> 	/*
> 	 * Reserved for the use of the calling application.
> @@ -2231,6 +2232,35 @@ ext2fs_orphan_block_tail(ext2_filsys fs, char =
*buf)
> 		sizeof(struct ext4_orphan_block_tail));
> }
>=20
> +static dgrp_t ext2fs_get_avg_group(ext2_filsys fs)
> +{
> +#ifdef HAVE_PTHREAD
> +	dgrp_t average_group;
> +	unsigned flexbg_size;
> +
> +	if (fs->fs_num_threads <=3D 1)
> +		return fs->group_desc_count;
> +
> +	average_group =3D fs->group_desc_count / fs->fs_num_threads;
> +	if (average_group <=3D 1)
> +		return 1;
> +
> +	if (ext2fs_has_feature_flex_bg(fs->super)) {
> +		int times =3D 1;
> +
> +		flexbg_size =3D 1 << fs->super->s_log_groups_per_flex;
> +		if (average_group % flexbg_size) {
> +			times =3D average_group / flexbg_size;
> +			average_group =3D times * flexbg_size;
> +		}
> +	}
> +
> +	return average_group;
> +#else
> +	return fs->group_desc_count;
> +#endif
> +}
> +
> #undef _INLINE_
> #endif
>=20
> --
> 2.37.3
>=20


Cheers, Andreas


--Apple-Mail=_353F2A8B-789B-4AAB-95B5-AA663F50BB99
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
	filename=signature.asc
Content-Type: application/pgp-signature;
	name=signature.asc
Content-Description: Message signed with OpenPGP

-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org

iQIzBAEBCAAdFiEEDb73u6ZejP5ZMprvcqXauRfMH+AFAmOaPyIACgkQcqXauRfM
H+BXnQ//d3kV9OyLF1tT9Mxbzv0Cwo/RJ6tu+trpQESCqi6LouRr2taGuv1hkTX7
y741tq0kITV6SLlQw411VV0/Gpz3kvGF7MXWb4UE11DAtVEeErGvKUn2v1WhbM2r
yGYQ//PeblWoBbMFDoSzgOWUyeQXy6ZimZak8XjEh/a4Yr/zFPmeb6ei1dAbe0xG
KPrkqxASQy6RW2wMxjabkPhcV6ntaSdJpYjdZzFKBC3UaiDz9IIYimrjf2QT/CND
BxcwcGTAVEX4WlrsONY/pfEqMYsWMcJ8tsOz2TyqVdWmFH4aXHK7lO+JJFJhayC+
L6+jiKySLaZFZwZw9DNvJ4PqGTrV2XThbCul9AsolUKIkqcQr2577ccud6qM3tF0
WZ6TZe/w6Im5d6poehpCP8O8gJ+2d9+xYcvH7y5ZSUFeSaY43cf8+34uB+h50wRU
BD51ZIAh88KAoi1eBe+r46PtmR8OeJRvjEQRo30OW5M4hAbIcpCAu2ugUo4fK/mH
0t50WfCuctM4Rj/jQjD2tFhrSgvY+qBiLseAGFGLR7NrZK+nIAZe8nyq1NBwdGMm
1RGgUEHMqGDLsTOM++0uhBfmoCQfv3JxwTdyRtC7vZ1ZDyDurXGw0EwhiWR/Guzj
VHN4LbF9pKEK8SKrrvWZk2pb1ONGg4che1ypL6n7fNYAQ4ATNj8=
=xxnC
-----END PGP SIGNATURE-----

--Apple-Mail=_353F2A8B-789B-4AAB-95B5-AA663F50BB99--