2015-08-31 09:46:45

by Zhang Zhen

Subject: [PATCH] fs/ext4: fix potential endless loop in ext4_mb_discard_group_preallocations()

In ext4_mb_discard_group_preallocations(), if free is always less than needed,
and some PAs are always used in every loop, it will be endless loop.

Here we pick a random value to limit the max number of loop.

Signed-off-by: Zhang Zhen <[email protected]>
---
fs/ext4/mballoc.c | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c
index 34b610e..553fbde 100644
--- a/fs/ext4/mballoc.c
+++ b/fs/ext4/mballoc.c
@@ -3836,6 +3836,7 @@ ext4_mb_discard_group_preallocations(struct super_block *sb,
int err;
int busy = 0;
int free = 0;
+ int tried = 0;

mb_debug(1, "discard preallocation for group %u\n", group);

@@ -3886,9 +3887,11 @@ repeat:
list_add(&pa->u.pa_tmp_list, &list);
}

- /* if we still need more blocks and some PAs were used, try again */
- if (free < needed && busy) {
+ /* if we still need more blocks and some PAs were used, try again,
+ here 20 is a ramdon value. */
+ if (free < needed && busy && tried < 20) {
busy = 0;
+ tried++;
ext4_unlock_group(sb, group);
cond_resched();
goto repeat;
--
1.9.1


.

2015-08-31 22:34:40

by Andreas Dilger

Subject: Re: [PATCH] fs/ext4: fix potential endless loop in ext4_mb_discard_group_preallocations()

On Aug 31, 2015, at 3:46 AM, Zhang Zhen <[email protected]> wrote:
>
> In ext4_mb_discard_group_preallocations(), if free is always less
> than needed, and some PAs are always used in every loop, it will
> be endless loop.
>
> Here we pick a random value to limit the max number of loop.
>
> Signed-off-by: Zhang Zhen <[email protected]>
> ---
> fs/ext4/mballoc.c | 7 +++++--
> 1 file changed, 5 insertions(+), 2 deletions(-)
>
> diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c
> index 34b610e..553fbde 100644
> --- a/fs/ext4/mballoc.c
> +++ b/fs/ext4/mballoc.c
> @@ -3836,6 +3836,7 @@ ext4_mb_discard_group_preallocations(struct super_block *sb,
> int err;
> int busy = 0;
> int free = 0;
> + int tried = 0;
>
> mb_debug(1, "discard preallocation for group %u\n", group);
>
> @@ -3886,9 +3887,11 @@ repeat:
> list_add(&pa->u.pa_tmp_list, &list);
> }
>
> - /* if we still need more blocks and some PAs were used, try again */
> - if (free < needed && busy) {
> + /* if we still need more blocks and some PAs were used, try again,
> + here 20 is a ramdon value. */

(typo) s/ramdon/random/

> + if (free < needed && busy && tried < 20) {
> busy = 0;
> + tried++;
> ext4_unlock_group(sb, group);
> cond_resched();
> goto repeat;

Looks OK otherwise.

Cheers, Andreas

2015-09-01 01:16:52

by Zhang Zhen

Subject: Re: [PATCH] fs/ext4: fix potential endless loop in ext4_mb_discard_group_preallocations()

On 2015/9/1 6:34, Andreas Dilger wrote:
> On Aug 31, 2015, at 3:46 AM, Zhang Zhen <[email protected]> wrote:
>>
>> In ext4_mb_discard_group_preallocations(), if free is always less
>> than needed, and some PAs are always used in every loop, it will
>> be endless loop.
>>
>> Here we pick a random value to limit the max number of loop.
>>
>> Signed-off-by: Zhang Zhen <[email protected]>
>> ---
>> fs/ext4/mballoc.c | 7 +++++--
>> 1 file changed, 5 insertions(+), 2 deletions(-)
>>
>> diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c
>> index 34b610e..553fbde 100644
>> --- a/fs/ext4/mballoc.c
>> +++ b/fs/ext4/mballoc.c
>> @@ -3836,6 +3836,7 @@ ext4_mb_discard_group_preallocations(struct super_block *sb,
>> int err;
>> int busy = 0;
>> int free = 0;
>> + int tried = 0;
>>
>> mb_debug(1, "discard preallocation for group %u\n", group);
>>
>> @@ -3886,9 +3887,11 @@ repeat:
>> list_add(&pa->u.pa_tmp_list, &list);
>> }
>>
>> - /* if we still need more blocks and some PAs were used, try again */
>> - if (free < needed && busy) {
>> + /* if we still need more blocks and some PAs were used, try again,
>> + here 20 is a ramdon value. */
>
> (typo) s/ramdon/random/
>
My mistake, thanks!

>> + if (free < needed && busy && tried < 20) {
>> busy = 0;
>> + tried++;
>> ext4_unlock_group(sb, group);
>> cond_resched();
>> goto repeat;
>
> Looks OK otherwise.
>
> Cheers, Andreas
>
>
>
>
>
>
>

2015-09-01 06:41:38

by Zhang Zhen

Subject: [PATCH v2] fs/ext4: fix potential endless loop in ext4_mb_discard_group_preallocations()

In ext4_mb_discard_group_preallocations(), if free is always less than needed,
and some PAs are always used in every loop, it will be endless loop.

Here we pick a random value to limit the max number of loop.

Change v1 -> v2:
- fix typo error, s/ramdon/random/

Signed-off-by: Zhang Zhen <[email protected]>
Acked-by: Andreas Dilger <[email protected]>
---
fs/ext4/mballoc.c | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c
index 34b610e..553fbde 100644
--- a/fs/ext4/mballoc.c
+++ b/fs/ext4/mballoc.c
@@ -3836,6 +3836,7 @@ ext4_mb_discard_group_preallocations(struct super_block *sb,
int err;
int busy = 0;
int free = 0;
+ int tried = 0;

mb_debug(1, "discard preallocation for group %u\n", group);

@@ -3886,9 +3887,11 @@ repeat:
list_add(&pa->u.pa_tmp_list, &list);
}

- /* if we still need more blocks and some PAs were used, try again */
- if (free < needed && busy) {
+ /* if we still need more blocks and some PAs were used, try again,
+ here 20 is a random value. */
+ if (free < needed && busy && tried < 20) {
busy = 0;
+ tried++;
ext4_unlock_group(sb, group);
cond_resched();
goto repeat;
--
1.9.1


.

2015-09-16 19:45:14

by Jan Kara

Subject: Re: [PATCH] fs/ext4: fix potential endless loop in ext4_mb_discard_group_preallocations()

On Mon 31-08-15 17:46:24, Zhang Zhen wrote:
> In ext4_mb_discard_group_preallocations(), if free is always less than needed,
> and some PAs are always used in every loop, it will be endless loop.
>
> Here we pick a random value to limit the max number of loop.

Were you able to trigger this in practice or is it just a theoretical
concern?

My slight concern is that in theory we could prematurely declare ENOSPC
with this patch since ext4_mb_discard_preallocations() doesn't reliably
discard all the preallocations anymore. But probably that's acceptable.
But we should add a comment before ext4_mb_discard_group_preallocations()
saying that the functions needn't free all the preallocations.

Honza
>
> Signed-off-by: Zhang Zhen <[email protected]>
> ---
> fs/ext4/mballoc.c | 7 +++++--
> 1 file changed, 5 insertions(+), 2 deletions(-)
>
> diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c
> index 34b610e..553fbde 100644
> --- a/fs/ext4/mballoc.c
> +++ b/fs/ext4/mballoc.c
> @@ -3836,6 +3836,7 @@ ext4_mb_discard_group_preallocations(struct super_block *sb,
> int err;
> int busy = 0;
> int free = 0;
> + int tried = 0;
>
> mb_debug(1, "discard preallocation for group %u\n", group);
>
> @@ -3886,9 +3887,11 @@ repeat:
> list_add(&pa->u.pa_tmp_list, &list);
> }
>
> - /* if we still need more blocks and some PAs were used, try again */
> - if (free < needed && busy) {
> + /* if we still need more blocks and some PAs were used, try again,
> + here 20 is a ramdon value. */
> + if (free < needed && busy && tried < 20) {
> busy = 0;
> + tried++;
> ext4_unlock_group(sb, group);
> cond_resched();
> goto repeat;
> --
> 1.9.1
>
>
> .
>
>
>
>
--
Jan Kara <[email protected]>
SUSE Labs, CR

2015-09-17 01:29:07

by Zhang Zhen

Subject: Re: [PATCH] fs/ext4: fix potential endless loop in ext4_mb_discard_group_preallocations()

On 2015/9/17 3:45, Jan Kara wrote:
> On Mon 31-08-15 17:46:24, Zhang Zhen wrote:
>> In ext4_mb_discard_group_preallocations(), if free is always less than needed,
>> and some PAs are always used in every loop, it will be endless loop.
>>
>> Here we pick a random value to limit the max number of loop.
>
> Were you able to trigger this in practice or is it just a theoretical
> concern?
>
We found the problem in a stress test.
It lead to system hungtask.

Thanks!
> My slight concern is that in theory we could prematurely declare ENOSPC
> with this patch since ext4_mb_discard_preallocations() doesn't reliably
> discard all the preallocations anymore. But probably that's acceptable.
> But we should add a comment before ext4_mb_discard_group_preallocations()
> saying that the functions needn't free all the preallocations.
>
> Honza
>>
>> Signed-off-by: Zhang Zhen <[email protected]>
>> ---
>> fs/ext4/mballoc.c | 7 +++++--
>> 1 file changed, 5 insertions(+), 2 deletions(-)
>>
>> diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c
>> index 34b610e..553fbde 100644
>> --- a/fs/ext4/mballoc.c
>> +++ b/fs/ext4/mballoc.c
>> @@ -3836,6 +3836,7 @@ ext4_mb_discard_group_preallocations(struct super_block *sb,
>> int err;
>> int busy = 0;
>> int free = 0;
>> + int tried = 0;
>>
>> mb_debug(1, "discard preallocation for group %u\n", group);
>>
>> @@ -3886,9 +3887,11 @@ repeat:
>> list_add(&pa->u.pa_tmp_list, &list);
>> }
>>
>> - /* if we still need more blocks and some PAs were used, try again */
>> - if (free < needed && busy) {
>> + /* if we still need more blocks and some PAs were used, try again,
>> + here 20 is a ramdon value. */
>> + if (free < needed && busy && tried < 20) {
>> busy = 0;
>> + tried++;
>> ext4_unlock_group(sb, group);
>> cond_resched();
>> goto repeat;
>> --
>> 1.9.1
>>
>>
>> .
>>
>>
>>
>>