2023-02-09 02:55:41

by Yajun Deng

Subject: [PATCH] mm/page_alloc: optimize the loop in find_suitable_fallback()

There is no need to execute the next loop if it not return in the first
loop. So add a break at the end of the loop.

There are only three rows in fallbacks, so reduce the first index size
from MIGRATE_TYPES to MIGRATE_PCPTYPES.

Signed-off-by: Yajun Deng <[email protected]>
---
mm/page_alloc.c | 11 +++++------
1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 1113483fa6c5..536e8d838fb5 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -2603,7 +2603,7 @@ struct page *__rmqueue_smallest(struct zone *zone, unsigned int order,
*
* The other migratetypes do not have fallbacks.
*/
-static int fallbacks[MIGRATE_TYPES][MIGRATE_PCPTYPES - 1] = {
+static int fallbacks[MIGRATE_PCPTYPES][MIGRATE_PCPTYPES - 1] = {
[MIGRATE_UNMOVABLE] = { MIGRATE_RECLAIMABLE, MIGRATE_MOVABLE },
[MIGRATE_MOVABLE] = { MIGRATE_RECLAIMABLE, MIGRATE_UNMOVABLE },
[MIGRATE_RECLAIMABLE] = { MIGRATE_UNMOVABLE, MIGRATE_MOVABLE },
@@ -2861,7 +2861,7 @@ int find_suitable_fallback(struct free_area *area, unsigned int order,
int i;
int fallback_mt;

- if (area->nr_free == 0)
+ if (area->nr_free == 0 || !migratetype_is_mergeable(migratetype))
return -1;

*can_steal = false;
@@ -2873,11 +2873,10 @@ int find_suitable_fallback(struct free_area *area, unsigned int order,
if (can_steal_fallback(order, migratetype))
*can_steal = true;

- if (!only_stealable)
- return fallback_mt;
-
- if (*can_steal)
+ if (!only_stealable || *can_steal)
return fallback_mt;
+ else
+ break;
}

return -1;
--
2.25.1

2023-02-09 08:12:46

by Vlastimil Babka

Subject: Re: [PATCH] mm/page_alloc: optimize the loop in find_suitable_fallback()

On 2/9/23 03:44, Yajun Deng wrote:
> There is no need to execute the next loop if it not return in the first
> loop. So add a break at the end of the loop.
>
> There are only three rows in fallbacks, so reduce the first index size
> from MIGRATE_TYPES to MIGRATE_PCPTYPES.
>
> Signed-off-by: Yajun Deng <[email protected]>

Acked-by: Vlastimil Babka <[email protected]>

> ---
> mm/page_alloc.c | 11 +++++------
> 1 file changed, 5 insertions(+), 6 deletions(-)
>
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 1113483fa6c5..536e8d838fb5 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -2603,7 +2603,7 @@ struct page *__rmqueue_smallest(struct zone *zone, unsigned int order,
> *
> * The other migratetypes do not have fallbacks.
> */
> -static int fallbacks[MIGRATE_TYPES][MIGRATE_PCPTYPES - 1] = {
> +static int fallbacks[MIGRATE_PCPTYPES][MIGRATE_PCPTYPES - 1] = {
> [MIGRATE_UNMOVABLE] = { MIGRATE_RECLAIMABLE, MIGRATE_MOVABLE },
> [MIGRATE_MOVABLE] = { MIGRATE_RECLAIMABLE, MIGRATE_UNMOVABLE },
> [MIGRATE_RECLAIMABLE] = { MIGRATE_UNMOVABLE, MIGRATE_MOVABLE },
> @@ -2861,7 +2861,7 @@ int find_suitable_fallback(struct free_area *area, unsigned int order,
> int i;
> int fallback_mt;
>
> - if (area->nr_free == 0)
> + if (area->nr_free == 0 || !migratetype_is_mergeable(migratetype))

Just curious, did you the check for extra safety or did you find (by running
or code inspection) that this can be indeed called with a non-mergeable
migratetype, and cause out of bounds access of the shrinked fallbacks array?

BTW, I noticed the commment on migratetype_is_mergeable() contains:

"See fallbacks[MIGRATE_TYPES][3] in page_alloc.c. "

Should probably change it to e.g. "See fallbacks[][] array ..." so we don't
have to keep it in exact sync...

> return -1;
>
> *can_steal = false;
> @@ -2873,11 +2873,10 @@ int find_suitable_fallback(struct free_area *area, unsigned int order,
> if (can_steal_fallback(order, migratetype))
> *can_steal = true;
>
> - if (!only_stealable)
> - return fallback_mt;
> -
> - if (*can_steal)
> + if (!only_stealable || *can_steal)
> return fallback_mt;
> + else
> + break;
> }
>
> return -1;

2023-02-09 08:44:16

by Yajun Deng

Subject: Re: [PATCH] mm/page_alloc: optimize the loop in find_suitable_fallback()

February 9, 2023 4:12 PM, "Vlastimil Babka" <[email protected]> wrote:

> On 2/9/23 03:44, Yajun Deng wrote:
>
>> There is no need to execute the next loop if it not return in the first
>> loop. So add a break at the end of the loop.
>>
>> There are only three rows in fallbacks, so reduce the first index size
>> from MIGRATE_TYPES to MIGRATE_PCPTYPES.
>>
>> Signed-off-by: Yajun Deng <[email protected]>
>
> Acked-by: Vlastimil Babka <[email protected]>
>
>> ---
>> mm/page_alloc.c | 11 +++++------
>> 1 file changed, 5 insertions(+), 6 deletions(-)
>>
>> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
>> index 1113483fa6c5..536e8d838fb5 100644
>> --- a/mm/page_alloc.c
>> +++ b/mm/page_alloc.c
>> @@ -2603,7 +2603,7 @@ struct page *__rmqueue_smallest(struct zone *zone, unsigned int order,
>> *
>> * The other migratetypes do not have fallbacks.
>> */
>> -static int fallbacks[MIGRATE_TYPES][MIGRATE_PCPTYPES - 1] = {
>> +static int fallbacks[MIGRATE_PCPTYPES][MIGRATE_PCPTYPES - 1] = {
>> [MIGRATE_UNMOVABLE] = { MIGRATE_RECLAIMABLE, MIGRATE_MOVABLE },
>> [MIGRATE_MOVABLE] = { MIGRATE_RECLAIMABLE, MIGRATE_UNMOVABLE },
>> [MIGRATE_RECLAIMABLE] = { MIGRATE_UNMOVABLE, MIGRATE_MOVABLE },
>> @@ -2861,7 +2861,7 @@ int find_suitable_fallback(struct free_area *area, unsigned int order,
>> int i;
>> int fallback_mt;
>>
>> - if (area->nr_free == 0)
>> + if (area->nr_free == 0 || !migratetype_is_mergeable(migratetype))
>
> Just curious, did you the check for extra safety or did you find (by running
> or code inspection) that this can be indeed called with a non-mergeable
> migratetype, and cause out of bounds access of the shrinked fallbacks array?
>

No, I'm not sure if it is called with a non-mergeable migratetype.
It is just for extra safety.

> BTW, I noticed the commment on migratetype_is_mergeable() contains:
>
> "See fallbacks[MIGRATE_TYPES][3] in page_alloc.c. "
>
> Should probably change it to e.g. "See fallbacks[][] array ..." so we don't
> have to keep it in exact sync...
>

Yes, this comment should be changed.
So do I need to submit a v2 patch?

>> return -1;
>>
>> *can_steal = false;
>> @@ -2873,11 +2873,10 @@ int find_suitable_fallback(struct free_area *area, unsigned int order,
>> if (can_steal_fallback(order, migratetype))
>> *can_steal = true;
>>
>> - if (!only_stealable)
>> - return fallback_mt;
>> -
>> - if (*can_steal)
>> + if (!only_stealable || *can_steal)
>> return fallback_mt;
>> + else
>> + break;
>> }
>>
>> return -1;

2023-02-09 09:23:39

by Vlastimil Babka

Subject: Re: [PATCH] mm/page_alloc: optimize the loop in find_suitable_fallback()

On 2/9/23 09:44, Yajun Deng wrote:
> February 9, 2023 4:12 PM, "Vlastimil Babka" <[email protected]> wrote:
>
>> On 2/9/23 03:44, Yajun Deng wrote:
>>
>>> There is no need to execute the next loop if it not return in the first
>>> loop. So add a break at the end of the loop.
>>>
>>> There are only three rows in fallbacks, so reduce the first index size
>>> from MIGRATE_TYPES to MIGRATE_PCPTYPES.
>>>
>>> Signed-off-by: Yajun Deng <[email protected]>
>>
>> Acked-by: Vlastimil Babka <[email protected]>
>>
>>> ---
>>> mm/page_alloc.c | 11 +++++------
>>> 1 file changed, 5 insertions(+), 6 deletions(-)
>>>
>>> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
>>> index 1113483fa6c5..536e8d838fb5 100644
>>> --- a/mm/page_alloc.c
>>> +++ b/mm/page_alloc.c
>>> @@ -2603,7 +2603,7 @@ struct page *__rmqueue_smallest(struct zone *zone, unsigned int order,
>>> *
>>> * The other migratetypes do not have fallbacks.
>>> */
>>> -static int fallbacks[MIGRATE_TYPES][MIGRATE_PCPTYPES - 1] = {
>>> +static int fallbacks[MIGRATE_PCPTYPES][MIGRATE_PCPTYPES - 1] = {
>>> [MIGRATE_UNMOVABLE] = { MIGRATE_RECLAIMABLE, MIGRATE_MOVABLE },
>>> [MIGRATE_MOVABLE] = { MIGRATE_RECLAIMABLE, MIGRATE_UNMOVABLE },
>>> [MIGRATE_RECLAIMABLE] = { MIGRATE_UNMOVABLE, MIGRATE_MOVABLE },
>>> @@ -2861,7 +2861,7 @@ int find_suitable_fallback(struct free_area *area, unsigned int order,
>>> int i;
>>> int fallback_mt;
>>>
>>> - if (area->nr_free == 0)
>>> + if (area->nr_free == 0 || !migratetype_is_mergeable(migratetype))
>>
>> Just curious, did you the check for extra safety or did you find (by running
>> or code inspection) that this can be indeed called with a non-mergeable
>> migratetype, and cause out of bounds access of the shrinked fallbacks array?
>>
>
> No, I'm not sure if it is called with a non-mergeable migratetype.
> It is just for extra safety.

OK, I agree with that.

>> BTW, I noticed the commment on migratetype_is_mergeable() contains:
>>
>> "See fallbacks[MIGRATE_TYPES][3] in page_alloc.c. "
>>
>> Should probably change it to e.g. "See fallbacks[][] array ..." so we don't
>> have to keep it in exact sync...
>>
>
> Yes, this comment should be changed.
> So do I need to submit a v2 patch?

Please do, with my acked-by.

>>> return -1;
>>>
>>> *can_steal = false;
>>> @@ -2873,11 +2873,10 @@ int find_suitable_fallback(struct free_area *area, unsigned int order,
>>> if (can_steal_fallback(order, migratetype))
>>> *can_steal = true;
>>>
>>> - if (!only_stealable)
>>> - return fallback_mt;
>>> -
>>> - if (*can_steal)
>>> + if (!only_stealable || *can_steal)
>>> return fallback_mt;
>>> + else
>>> + break;
>>> }
>>>
>>> return -1;