One of the side-effects of mb_optimize_scan was that the optimized
functions to select next group to try were called even before we tried
the goal group. As a result we no longer allocate files close to
corresponding inodes as well as we don't try to expand currently
allocated extent in the same group. This results in reaim regression
with workfile.disk workload of upto 8% with many clients on my test
machine:
baseline mb_optimize_scan
Hmean disk-1 2114.16 ( 0.00%) 2099.37 ( -0.70%)
Hmean disk-41 87794.43 ( 0.00%) 83787.47 * -4.56%*
Hmean disk-81 148170.73 ( 0.00%) 135527.05 * -8.53%*
Hmean disk-121 177506.11 ( 0.00%) 166284.93 * -6.32%*
Hmean disk-161 220951.51 ( 0.00%) 207563.39 * -6.06%*
Hmean disk-201 208722.74 ( 0.00%) 203235.59 ( -2.63%)
Hmean disk-241 222051.60 ( 0.00%) 217705.51 ( -1.96%)
Hmean disk-281 252244.17 ( 0.00%) 241132.72 * -4.41%*
Hmean disk-321 255844.84 ( 0.00%) 245412.84 * -4.08%*
Also this is causing huge regression (time increased by a factor of 5 or
so) when untarring archive with lots of small files on some eMMC storage
cards.
Fix the problem by making sure we try goal group first.
Fixes: 196e402adf2e ("ext4: improve cr 0 / cr 1 group scanning")
CC: [email protected]
Reported-and-tested-by: Stefan Wahren <[email protected]>
Tested-by: Ojaswin Mujoo <[email protected]>
Reviewed-by: Ritesh Harjani (IBM) <[email protected]>
Link: https://lore.kernel.org/all/20220727105123.ckwrhbilzrxqpt24@quack3/
Link: https://lore.kernel.org/all/[email protected]/
Signed-off-by: Jan Kara <[email protected]>
---
fs/ext4/mballoc.c | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c
index bd8f8b5c3d30..41e1cfecac3b 100644
--- a/fs/ext4/mballoc.c
+++ b/fs/ext4/mballoc.c
@@ -1049,8 +1049,10 @@ static void ext4_mb_choose_next_group(struct ext4_allocation_context *ac,
{
*new_cr = ac->ac_criteria;
- if (!should_optimize_scan(ac) || ac->ac_groups_linear_remaining)
+ if (!should_optimize_scan(ac) || ac->ac_groups_linear_remaining) {
+ *group = next_linear_group(ac, *group, ngroups);
return;
+ }
if (*new_cr == 0) {
ext4_mb_choose_next_group_cr0(ac, new_cr, group, ngroups);
@@ -2636,7 +2638,7 @@ static noinline_for_stack int
ext4_mb_regular_allocator(struct ext4_allocation_context *ac)
{
ext4_group_t prefetch_grp = 0, ngroups, group, i;
- int cr = -1;
+ int cr = -1, new_cr;
int err = 0, first_err = 0;
unsigned int nr = 0, prefetch_ios = 0;
struct ext4_sb_info *sbi;
@@ -2711,13 +2713,11 @@ ext4_mb_regular_allocator(struct ext4_allocation_context *ac)
ac->ac_groups_linear_remaining = sbi->s_mb_max_linear_groups;
prefetch_grp = group;
- for (i = 0; i < ngroups; group = next_linear_group(ac, group, ngroups),
- i++) {
- int ret = 0, new_cr;
+ for (i = 0, new_cr = cr; i < ngroups; i++,
+ ext4_mb_choose_next_group(ac, &new_cr, &group, ngroups)) {
+ int ret = 0;
cond_resched();
-
- ext4_mb_choose_next_group(ac, &new_cr, &group, ngroups);
if (new_cr != cr) {
cr = new_cr;
goto repeat;
--
2.35.3
On Thu, 8 Sep 2022 11:21:24 +0200, Jan Kara wrote:
> One of the side-effects of mb_optimize_scan was that the optimized
> functions to select next group to try were called even before we tried
> the goal group. As a result we no longer allocate files close to
> corresponding inodes as well as we don't try to expand currently
> allocated extent in the same group. This results in reaim regression
> with workfile.disk workload of upto 8% with many clients on my test
> machine:
>
> [...]
Applied, thanks!
[1/5] ext4: Make mballoc try target group first even with mb_optimize_scan
commit: 4fca50d440cc5d4dc570ad5484cc0b70b381bc2a
[2/5] ext4: Avoid unnecessary spreading of allocations among groups
commit: 1940265ede6683f6317cba0d428ce6505eaca944
[3/5] ext4: Make directory inode spreading reflect flexbg size
commit: 613c5a85898d1cd44e68f28d65eccf64a8ace9cf
[4/5] ext4: Use locality group preallocation for small closed files
commit: a9f2a2931d0e197ab28c6007966053fdababd53f
[5/5] ext4: Use buckets for cr 1 block scan instead of rbtree
commit: 83e80a6e3543f37f74c8e48a5f305b054b65ce2a
Best regards,
--
Theodore Ts'o <[email protected]>
On Wed 21-09-22 22:52:34, Theodore Ts'o wrote:
> On Thu, 8 Sep 2022 11:21:24 +0200, Jan Kara wrote:
> > One of the side-effects of mb_optimize_scan was that the optimized
> > functions to select next group to try were called even before we tried
> > the goal group. As a result we no longer allocate files close to
> > corresponding inodes as well as we don't try to expand currently
> > allocated extent in the same group. This results in reaim regression
> > with workfile.disk workload of upto 8% with many clients on my test
> > machine:
> >
> > [...]
>
> Applied, thanks!
>
> [1/5] ext4: Make mballoc try target group first even with mb_optimize_scan
> commit: 4fca50d440cc5d4dc570ad5484cc0b70b381bc2a
> [2/5] ext4: Avoid unnecessary spreading of allocations among groups
> commit: 1940265ede6683f6317cba0d428ce6505eaca944
> [3/5] ext4: Make directory inode spreading reflect flexbg size
> commit: 613c5a85898d1cd44e68f28d65eccf64a8ace9cf
> [4/5] ext4: Use locality group preallocation for small closed files
> commit: a9f2a2931d0e197ab28c6007966053fdababd53f
> [5/5] ext4: Use buckets for cr 1 block scan instead of rbtree
> commit: 83e80a6e3543f37f74c8e48a5f305b054b65ce2a
Thanks Ted! I just have locally a small fixup to the series that was reported
by Smatch. It is attached, either fold it into the last patch or just merge
it as a separate patch. Thanks!
Honza
--
Jan Kara <[email protected]>
SUSE Labs, CR
On Thu 22-09-22 11:15:42, Jan Kara wrote:
> On Wed 21-09-22 22:52:34, Theodore Ts'o wrote:
> > On Thu, 8 Sep 2022 11:21:24 +0200, Jan Kara wrote:
> > > One of the side-effects of mb_optimize_scan was that the optimized
> > > functions to select next group to try were called even before we tried
> > > the goal group. As a result we no longer allocate files close to
> > > corresponding inodes as well as we don't try to expand currently
> > > allocated extent in the same group. This results in reaim regression
> > > with workfile.disk workload of upto 8% with many clients on my test
> > > machine:
> > >
> > > [...]
> >
> > Applied, thanks!
> >
> > [1/5] ext4: Make mballoc try target group first even with mb_optimize_scan
> > commit: 4fca50d440cc5d4dc570ad5484cc0b70b381bc2a
> > [2/5] ext4: Avoid unnecessary spreading of allocations among groups
> > commit: 1940265ede6683f6317cba0d428ce6505eaca944
> > [3/5] ext4: Make directory inode spreading reflect flexbg size
> > commit: 613c5a85898d1cd44e68f28d65eccf64a8ace9cf
> > [4/5] ext4: Use locality group preallocation for small closed files
> > commit: a9f2a2931d0e197ab28c6007966053fdababd53f
> > [5/5] ext4: Use buckets for cr 1 block scan instead of rbtree
> > commit: 83e80a6e3543f37f74c8e48a5f305b054b65ce2a
>
> Thanks Ted! I just have locally a small fixup to the series that was reported
> by Smatch. It is attached, either fold it into the last patch or just merge
> it as a separate patch. Thanks!
Ted, I've noticed you've merged my mballoc fixes (and pushed them to Linus)
without this fixup. Can you please merge it? The use of uninitialized
variable seems rare but possible...
Honza
> From 8885b11fb253e08ecfa90a28beffb01719af84f5 Mon Sep 17 00:00:00 2001
> From: Jan Kara <[email protected]>
> Date: Thu, 22 Sep 2022 11:09:29 +0200
> Subject: [PATCH] ext4: Fixup possible uninitialized variable access in
> ext4_mb_choose_next_group_cr1()
>
> Variable 'grp' may be left uninitialized if there's no group with
> suitable average fragment size (or larger). Fix the problem by
> initializing it earlier.
>
> Fixes: 83e80a6e3543 ("ext4: use buckets for cr 1 block scan instead of rbtree")
> Reported-by: Dan Carpenter <[email protected]>
> Signed-off-by: Jan Kara <[email protected]>
> ---
> fs/ext4/mballoc.c | 3 +--
> 1 file changed, 1 insertion(+), 2 deletions(-)
>
> diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c
> index 71f5b67d7f28..9dad93059945 100644
> --- a/fs/ext4/mballoc.c
> +++ b/fs/ext4/mballoc.c
> @@ -910,7 +910,7 @@ static void ext4_mb_choose_next_group_cr1(struct ext4_allocation_context *ac,
> int *new_cr, ext4_group_t *group, ext4_group_t ngroups)
> {
> struct ext4_sb_info *sbi = EXT4_SB(ac->ac_sb);
> - struct ext4_group_info *grp, *iter;
> + struct ext4_group_info *grp = NULL, *iter;
> int i;
>
> if (unlikely(ac->ac_flags & EXT4_MB_CR1_OPTIMIZED)) {
> @@ -927,7 +927,6 @@ static void ext4_mb_choose_next_group_cr1(struct ext4_allocation_context *ac,
> read_unlock(&sbi->s_mb_avg_fragment_size_locks[i]);
> continue;
> }
> - grp = NULL;
> list_for_each_entry(iter, &sbi->s_mb_avg_fragment_size[i],
> bb_avg_fragment_size_node) {
> if (sbi->s_mb_stats)
> --
> 2.35.3
>
--
Jan Kara <[email protected]>
SUSE Labs, CR
On Mon, Sep 26, 2022 at 11:11:26AM +0200, Jan Kara wrote:
>
> Ted, I've noticed you've merged my mballoc fixes (and pushed them to Linus)
> without this fixup. Can you please merge it? The use of uninitialized
> variable seems rare but possible...
Oops, sorry my bad. I had forgotten to merge the fix before I pushed
them to Linus. I'm preparing a follow-up push to Linus right now.
Once a smoke test passes, I'll sned out the pull request.
- Ted