> ?? 2019??12??18?գ?22:09??Chris Down <[email protected]> д????
>
> Hi Hui,
>
> In general cgroup v1 is in maintenance mode -- that is, excepting specific bugfixes, we don't plan to add new features.
Hi Chris,
Thanks for your review.
I will move to v2.
>
> Hui Zhu writes:
>> Currently, memcg has some config to limit memory usage and config
>> the shrink behavior.
>> In the memory-constrained environment, put different priority tasks
>> into different cgroups with different memory limits to protect the
>> performance of the high priority tasks. Because the global memory
>> shrink will affect the performance of all tasks. The memory limit
>> cgroup can make shrink happen inside the cgroup. Then it can decrease
>> the memory shrink of the high priority task to protect its performance.
>>
>> But the memory footprint of the task is not static. It will change as
>> the working pressure changes. And the version changes will affect it too.
>> Then set the appropriate memory limit to decrease the global memory shrink
>> is a difficult job and lead to wasted memory or performance loss sometimes.
>>
>> This commit adds global shrink priority to memcg to try to handle this
>> problem.
>
> I have significant concerns with exposing scan priority to userspace. This is an incredibly difficult metric for users to reason about since it's a reclaim implementation feature and would add to an already confusing and fragmented API in v1.
>
> Have you considered using memory protection (memory.low, memory.min) for this instead? It sounds like it can achieve the results you want, in that it allows you to direct and prioritise reclaim in a way that allows for ballparking (ie. it is compatible with applications with variable memory footprints).
>
Memory.min, low, high can affect the global shrink behavior. They can help task keep some pages to help protect performance.
But what I want is the low priority tasks (the tasks that performance is not very important) do more shrink first. And when low priority tasks doesn??t have enough pages to be dropped and system need more free page, shrink the high priority task??s pages. Because at this time, system??s stable is more important than the performance of priority task.
With memory.min and memory.low, I have no idea to config them to support this. That is why I add global shrink priority.
Thanks,
Hui
> Thanks,
>
> Chris
Hi Hui,
teawater writes:
>Memory.min, low, high can affect the global shrink behavior. They can help
>task keep some pages to help protect performance.
>
>But what I want is the low priority tasks (the tasks that performance is not
>very important) do more shrink first. And when low priority tasks doesn’t
>have enough pages to be dropped and system need more free page, shrink the
>high priority task’s pages. Because at this time, system’s stable is more
>important than the performance of priority task.
>With memory.min and memory.low, I have no idea to config them to support this.
>That is why I add global shrink priority.
For sure, that's what I'm suggesting you use memory.{min,low} for -- you define
some subset of the cgroup hierarchy as "protected", and then you bias reclaim
away from protected cgroups (and thus *towards* unprotected cgroups) by biasing
the size of LRU scanning. See my patch that went into 5.4 and the examples in
the commit message:
commit 9783aa9917f8ae24759e67bf882f1aba32fe4ea1
Author: Chris Down <[email protected]>
Date: Sun Oct 6 17:58:32 2019 -0700
mm, memcg: proportional memory.{low,min} reclaim
You can see how we're using memory.{low,min} to achieve this in this case
study[0]. It's not exactly equivalent technically to your solution, but the end
goals are similar.
Thanks,
Chris
0: https://facebookmicrosites.github.io/cgroup2/docs/overview.html#case-study-the-fbtax2-project
> ?? 2019??12??19?գ?19:26??Chris Down <[email protected]> д????
>
> Hi Hui,
>
> teawater writes:
>> Memory.min, low, high can affect the global shrink behavior. They can help task keep some pages to help protect performance.
>>
>> But what I want is the low priority tasks (the tasks that performance is not very important) do more shrink first. And when low priority tasks doesn??t have enough pages to be dropped and system need more free page, shrink the high priority task??s pages. Because at this time, system??s stable is more important than the performance of priority task.
>> With memory.min and memory.low, I have no idea to config them to support this. That is why I add global shrink priority.
>
> For sure, that's what I'm suggesting you use memory.{min,low} for -- you define some subset of the cgroup hierarchy as "protected", and then you bias reclaim away from protected cgroups (and thus *towards* unprotected cgroups) by biasing the size of LRU scanning. See my patch that went into 5.4 and the examples in the commit message:
>
> commit 9783aa9917f8ae24759e67bf882f1aba32fe4ea1
> Author: Chris Down <[email protected]>
> Date: Sun Oct 6 17:58:32 2019 -0700
>
> mm, memcg: proportional memory.{low,min} reclaim
>
> You can see how we're using memory.{low,min} to achieve this in this case study[0]. It's not exactly equivalent technically to your solution, but the end goals are similar.
>
> Thanks,
>
> Chris
>
> 0: https://facebookmicrosites.github.io/cgroup2/docs/overview.html#case-study-the-fbtax2-project
Hi Chris,
Really appreciate for your help. I will try to use it handle my problem.
Best,
Hui
> ?? 2019??12??19?գ?19:26??Chris Down <[email protected]> д????
>
> Hi Hui,
>
> teawater writes:
>> Memory.min, low, high can affect the global shrink behavior. They can help task keep some pages to help protect performance.
>>
>> But what I want is the low priority tasks (the tasks that performance is not very important) do more shrink first. And when low priority tasks doesn??t have enough pages to be dropped and system need more free page, shrink the high priority task??s pages. Because at this time, system??s stable is more important than the performance of priority task.
>> With memory.min and memory.low, I have no idea to config them to support this. That is why I add global shrink priority.
>
> For sure, that's what I'm suggesting you use memory.{min,low} for -- you define some subset of the cgroup hierarchy as "protected", and then you bias reclaim away from protected cgroups (and thus *towards* unprotected cgroups) by biasing the size of LRU scanning. See my patch that went into 5.4 and the examples in the commit message:
>
> commit 9783aa9917f8ae24759e67bf882f1aba32fe4ea1
> Author: Chris Down <[email protected]>
> Date: Sun Oct 6 17:58:32 2019 -0700
>
> mm, memcg: proportional memory.{low,min} reclaim
>
> You can see how we're using memory.{low,min} to achieve this in this case study[0]. It's not exactly equivalent technically to your solution, but the end goals are similar.
>
> Thanks,
>
> Chris
>
> 0: https://facebookmicrosites.github.io/cgroup2/docs/overview.html#case-study-the-fbtax2-project
Hi Chris,
I have another idea about global shrink priority.
In the memory-constrained and complex multitasking environment such as an Android system may require more complex performance priority.
For example, the tasks of app in the font, they need high priority because low priority will affect the user experience at once.
The tasks of app in background should have lower priority than the first one. And sometime, each app should have different priority. Because some apps are frequently used. They should have high priority than other background apps.
The daemons should have lower priority than background apps. Because most of them will not affect the user experience.
Do you think this environment is suitable for global shrink priority.
Best,
Hui
Hi Hui,
teawater writes:
>In the memory-constrained and complex multitasking environment such as an Android system may require more complex performance priority.
>For example, the tasks of app in the font, they need high priority because low priority will affect the user experience at once.
>The tasks of app in background should have lower priority than the first one. And sometime, each app should have different priority. Because some apps are frequently used. They should have high priority than other background apps.
>The daemons should have lower priority than background apps. Because most of them will not affect the user experience.
In general I don't think it's meaningful to speculate about whether it would
help or not without data and evidence gathering. It would really depend on how
the system is composed overall. Is this a real problem you're seeing, or just
something hypothetical?
If there is a real case to discuss, we can certainly discuss it. That said, at
the very least I think the API needs to be easier to reason about rather than
just exposing mm internals, and there needs to be a demonstration that it
solves a real problem and existing controls are insufficient :-)
Thanks,
Chris
> ?? 2019??12??29?գ?22:02??Chris Down <[email protected]> д????
>
> Hi Hui,
>
> teawater writes:
>> In the memory-constrained and complex multitasking environment such as an Android system may require more complex performance priority.
>> For example, the tasks of app in the font, they need high priority because low priority will affect the user experience at once.
>> The tasks of app in background should have lower priority than the first one. And sometime, each app should have different priority. Because some apps are frequently used. They should have high priority than other background apps.
>> The daemons should have lower priority than background apps. Because most of them will not affect the user experience.
>
> In general I don't think it's meaningful to speculate about whether it would help or not without data and evidence gathering. It would really depend on how the system is composed overall. Is this a real problem you're seeing, or just something hypothetical?
>
> If there is a real case to discuss, we can certainly discuss it. That said, at the very least I think the API needs to be easier to reason about rather than just exposing mm internals, and there needs to be a demonstration that it solves a real problem and existing controls are insufficient :-)
>
> Thanks,
>
> Chris
Thanks! Agree with you.
Best,
Hui