The full series is also available here:
https://github.com/hansendc/linux/tree/automigrate-20210304
which also inclues some vm.zone_reclaim_mode sysctl ABI fixup
prerequisites.
The meat of this patch is in:
[PATCH 05/10] mm/migrate: demote pages during reclaim
Which also has the most changes since the last post. This version is
mostly to address review comments from Yang Shi and Oscar Salvador.
Review comments are documented in the individual patch changelogs.
This also contains a few prerequisite patches that fix up an issue
with the vm.zone_reclaim_mode sysctl ABI.
Changes since (automigrate-20210122):
* move from GFP_HIGHUSER -> GFP_HIGHUSER_MOVABLE since pages *are*
movable.
* Separate out helpers that check for being able to relaim anonymous
pages versus being able to meaningfully scan the anon LRU.
--
We're starting to see systems with more and more kinds of memory such
as Intel's implementation of persistent memory.
Let's say you have a system with some DRAM and some persistent memory.
Today, once DRAM fills up, reclaim will start and some of the DRAM
contents will be thrown out. Allocations will, at some point, start
falling over to the slower persistent memory.
That has two nasty properties. First, the newer allocations can end
up in the slower persistent memory. Second, reclaimed data in DRAM
are just discarded even if there are gobs of space in persistent
memory that could be used.
This set implements a solution to these problems. At the end of the
reclaim process in shrink_page_list() just before the last page
refcount is dropped, the page is migrated to persistent memory instead
of being dropped.
While I've talked about a DRAM/PMEM pairing, this approach would
function in any environment where memory tiers exist.
This is not perfect. It "strands" pages in slower memory and never
brings them back to fast DRAM. Other things need to be built to
promote hot pages back to DRAM.
This is also all based on an upstream mechanism that allows
persistent memory to be onlined and used as if it were volatile:
http://lkml.kernel.org/r/[email protected]
== Open Issues ==
* For cpusets and memory policies that restrict allocations
to PMEM, is it OK to demote to PMEM? Do we need a cgroup-
level API to opt-in or opt-out of these migrations?
* Could be more aggressive about where anon LRU scanning occurs
since it no longer necessarily involves I/O. get_scan_count()
for instance says: "If we have no swap space, do not bother
scanning anon pages"
--
Documentation/admin-guide/sysctl/vm.rst | 9
include/linux/migrate.h | 20 +
include/linux/swap.h | 3
include/linux/vm_event_item.h | 2
include/trace/events/migrate.h | 3
include/uapi/linux/mempolicy.h | 1
mm/compaction.c | 3
mm/gup.c | 4
mm/internal.h | 5
mm/memory-failure.c | 4
mm/memory_hotplug.c | 4
mm/mempolicy.c | 8
mm/migrate.c | 369 +++++++++++++++++++++++++++++---
mm/page_alloc.c | 13 -
mm/vmscan.c | 173 +++++++++++++--
mm/vmstat.c | 2
16 files changed, 560 insertions(+), 63 deletions(-)
--
Changes since (automigrate-20200818):
* Fall back to normal reclaim when demotion fails
* Fix some compile issues, when page migration and NUMA are off
Changes since (automigrate-20201007):
* separate out checks for "can scan anon LRU" from "can actually
swap anon pages right now". Previous series conflated them
and may have been overly aggressive scanning LRU
* add MR_DEMOTION to tracepoint header
* remove unnecessary hugetlb page check
Changes since (https://lwn.net/Articles/824830/):
* Use higher-level migrate_pages() API approach from Yang Shi's
earlier patches.
* made sure to actually check node_reclaim_mode's new bit
* disabled migration entirely before introducing RECLAIM_MIGRATE
* Replace GFP_NOWAIT with explicit __GFP_KSWAPD_RECLAIM and
comment why we want that.
* Comment on effects of that keep multiple source nodes from
sharing target nodes
Cc: Yang Shi <[email protected]>
Cc: David Rientjes <[email protected]>
Cc: Huang Ying <[email protected]>
Cc: Dan Williams <[email protected]>
Cc: David Hildenbrand <[email protected]>
Cc: osalvador <[email protected]>
Cc: Huang Ying <[email protected]>
From: Dave Hansen <[email protected]>
Anonymous pages are kept on their own LRU(s). These lists could
theoretically always be scanned and maintained. But, without swap,
there is currently nothing the kernel can *do* with the results of a
scanned, sorted LRU for anonymous pages.
A check for '!total_swap_pages' currently serves as a valid check as
to whether anonymous LRUs should be maintained. However, another
method will be added shortly: page demotion.
Abstract out the 'total_swap_pages' checks into a helper, give it a
logically significant name, and check for the possibility of page
demotion.
Signed-off-by: Dave Hansen <[email protected]>
Cc: David Rientjes <[email protected]>
Cc: Huang Ying <[email protected]>
Cc: Dan Williams <[email protected]>
Cc: David Hildenbrand <[email protected]>
Cc: osalvador <[email protected]>
---
b/mm/vmscan.c | 28 +++++++++++++++++++++++++---
1 file changed, 25 insertions(+), 3 deletions(-)
diff -puN mm/vmscan.c~mm-vmscan-anon-can-be-aged mm/vmscan.c
--- a/mm/vmscan.c~mm-vmscan-anon-can-be-aged 2021-03-04 15:35:58.935806422 -0800
+++ b/mm/vmscan.c 2021-03-04 15:35:58.942806422 -0800
@@ -2517,6 +2517,26 @@ out:
}
}
+/*
+ * Anonymous LRU management is a waste if there is
+ * ultimately no way to reclaim the memory.
+ */
+bool anon_should_be_aged(struct lruvec *lruvec)
+{
+ struct pglist_data *pgdat = lruvec_pgdat(lruvec);
+
+ /* Aging the anon LRU is valuable if swap is present: */
+ if (total_swap_pages > 0)
+ return true;
+
+ /* Also valuable if anon pages can be demoted: */
+ if (next_demotion_node(pgdat->node_id) >= 0)
+ return true;
+
+ /* No way to reclaim anon pages. Should not age anon LRUs: */
+ return false;
+}
+
static void shrink_lruvec(struct lruvec *lruvec, struct scan_control *sc)
{
unsigned long nr[NR_LRU_LISTS];
@@ -2626,7 +2646,8 @@ static void shrink_lruvec(struct lruvec
* Even if we did not try to evict anon pages at all, we want to
* rebalance the anon lru active/inactive ratio.
*/
- if (total_swap_pages && inactive_is_low(lruvec, LRU_INACTIVE_ANON))
+ if (anon_should_be_aged(lruvec) &&
+ inactive_is_low(lruvec, LRU_INACTIVE_ANON))
shrink_active_list(SWAP_CLUSTER_MAX, lruvec,
sc, LRU_ACTIVE_ANON);
}
@@ -3455,10 +3476,11 @@ static void age_active_anon(struct pglis
struct mem_cgroup *memcg;
struct lruvec *lruvec;
- if (!total_swap_pages)
+ lruvec = mem_cgroup_lruvec(NULL, pgdat);
+
+ if (!anon_should_be_aged(lruvec))
return;
- lruvec = mem_cgroup_lruvec(NULL, pgdat);
if (!inactive_is_low(lruvec, LRU_INACTIVE_ANON))
return;
_
From: Yang Shi <[email protected]>
Account the number of demoted pages into reclaim_state->nr_demoted.
Add pgdemote_kswapd and pgdemote_direct VM counters showed in
/proc/vmstat.
[ daveh:
- __count_vm_events() a bit, and made them look at the THP
size directly rather than getting data from migrate_pages()
]
Signed-off-by: Yang Shi <[email protected]>
Signed-off-by: Dave Hansen <[email protected]>
Cc: David Rientjes <[email protected]>
Cc: Huang Ying <[email protected]>
Cc: Dan Williams <[email protected]>
Cc: David Hildenbrand <[email protected]>
Cc: osalvador <[email protected]>
--
Changes since 202010:
* remove unused scan-control 'demoted' field
---
b/include/linux/vm_event_item.h | 2 ++
b/mm/vmscan.c | 5 +++++
b/mm/vmstat.c | 2 ++
3 files changed, 9 insertions(+)
diff -puN include/linux/vm_event_item.h~mm-vmscan-add-page-demotion-counter include/linux/vm_event_item.h
--- a/include/linux/vm_event_item.h~mm-vmscan-add-page-demotion-counter 2021-03-04 15:35:57.698806425 -0800
+++ b/include/linux/vm_event_item.h 2021-03-04 15:35:57.719806425 -0800
@@ -33,6 +33,8 @@ enum vm_event_item { PGPGIN, PGPGOUT, PS
PGREUSE,
PGSTEAL_KSWAPD,
PGSTEAL_DIRECT,
+ PGDEMOTE_KSWAPD,
+ PGDEMOTE_DIRECT,
PGSCAN_KSWAPD,
PGSCAN_DIRECT,
PGSCAN_DIRECT_THROTTLE,
diff -puN mm/vmscan.c~mm-vmscan-add-page-demotion-counter mm/vmscan.c
--- a/mm/vmscan.c~mm-vmscan-add-page-demotion-counter 2021-03-04 15:35:57.700806425 -0800
+++ b/mm/vmscan.c 2021-03-04 15:35:57.724806425 -0800
@@ -1118,6 +1118,11 @@ static unsigned int demote_page_list(str
target_nid, MIGRATE_ASYNC, MR_DEMOTION,
&nr_succeeded);
+ if (current_is_kswapd())
+ __count_vm_events(PGDEMOTE_KSWAPD, nr_succeeded);
+ else
+ __count_vm_events(PGDEMOTE_DIRECT, nr_succeeded);
+
return nr_succeeded;
}
diff -puN mm/vmstat.c~mm-vmscan-add-page-demotion-counter mm/vmstat.c
--- a/mm/vmstat.c~mm-vmscan-add-page-demotion-counter 2021-03-04 15:35:57.708806425 -0800
+++ b/mm/vmstat.c 2021-03-04 15:35:57.726806425 -0800
@@ -1244,6 +1244,8 @@ const char * const vmstat_text[] = {
"pgreuse",
"pgsteal_kswapd",
"pgsteal_direct",
+ "pgdemote_kswapd",
+ "pgdemote_direct",
"pgscan_kswapd",
"pgscan_direct",
"pgscan_direct_throttle",
_
On Thu, Mar 4, 2021 at 4:01 PM Dave Hansen <[email protected]> wrote:
>
>
> From: Yang Shi <[email protected]>
>
> Account the number of demoted pages into reclaim_state->nr_demoted.
>
> Add pgdemote_kswapd and pgdemote_direct VM counters showed in
> /proc/vmstat.
>
> [ daveh:
> - __count_vm_events() a bit, and made them look at the THP
> size directly rather than getting data from migrate_pages()
> ]
>
> Signed-off-by: Yang Shi <[email protected]>
> Signed-off-by: Dave Hansen <[email protected]>
> Cc: David Rientjes <[email protected]>
> Cc: Huang Ying <[email protected]>
> Cc: Dan Williams <[email protected]>
> Cc: David Hildenbrand <[email protected]>
> Cc: osalvador <[email protected]>
>
> --
>
> Changes since 202010:
> * remove unused scan-control 'demoted' field
Reviewed-by: Yang Shi <[email protected]>
> ---
>
> b/include/linux/vm_event_item.h | 2 ++
> b/mm/vmscan.c | 5 +++++
> b/mm/vmstat.c | 2 ++
> 3 files changed, 9 insertions(+)
>
> diff -puN include/linux/vm_event_item.h~mm-vmscan-add-page-demotion-counter include/linux/vm_event_item.h
> --- a/include/linux/vm_event_item.h~mm-vmscan-add-page-demotion-counter 2021-03-04 15:35:57.698806425 -0800
> +++ b/include/linux/vm_event_item.h 2021-03-04 15:35:57.719806425 -0800
> @@ -33,6 +33,8 @@ enum vm_event_item { PGPGIN, PGPGOUT, PS
> PGREUSE,
> PGSTEAL_KSWAPD,
> PGSTEAL_DIRECT,
> + PGDEMOTE_KSWAPD,
> + PGDEMOTE_DIRECT,
> PGSCAN_KSWAPD,
> PGSCAN_DIRECT,
> PGSCAN_DIRECT_THROTTLE,
> diff -puN mm/vmscan.c~mm-vmscan-add-page-demotion-counter mm/vmscan.c
> --- a/mm/vmscan.c~mm-vmscan-add-page-demotion-counter 2021-03-04 15:35:57.700806425 -0800
> +++ b/mm/vmscan.c 2021-03-04 15:35:57.724806425 -0800
> @@ -1118,6 +1118,11 @@ static unsigned int demote_page_list(str
> target_nid, MIGRATE_ASYNC, MR_DEMOTION,
> &nr_succeeded);
>
> + if (current_is_kswapd())
> + __count_vm_events(PGDEMOTE_KSWAPD, nr_succeeded);
> + else
> + __count_vm_events(PGDEMOTE_DIRECT, nr_succeeded);
> +
> return nr_succeeded;
> }
>
> diff -puN mm/vmstat.c~mm-vmscan-add-page-demotion-counter mm/vmstat.c
> --- a/mm/vmstat.c~mm-vmscan-add-page-demotion-counter 2021-03-04 15:35:57.708806425 -0800
> +++ b/mm/vmstat.c 2021-03-04 15:35:57.726806425 -0800
> @@ -1244,6 +1244,8 @@ const char * const vmstat_text[] = {
> "pgreuse",
> "pgsteal_kswapd",
> "pgsteal_direct",
> + "pgdemote_kswapd",
> + "pgdemote_direct",
> "pgscan_kswapd",
> "pgscan_direct",
> "pgscan_direct_throttle",
> _
>
On Thu, Mar 4, 2021 at 4:01 PM Dave Hansen <[email protected]> wrote:
>
>
> From: Dave Hansen <[email protected]>
>
> Anonymous pages are kept on their own LRU(s). These lists could
> theoretically always be scanned and maintained. But, without swap,
> there is currently nothing the kernel can *do* with the results of a
> scanned, sorted LRU for anonymous pages.
>
> A check for '!total_swap_pages' currently serves as a valid check as
> to whether anonymous LRUs should be maintained. However, another
> method will be added shortly: page demotion.
>
> Abstract out the 'total_swap_pages' checks into a helper, give it a
> logically significant name, and check for the possibility of page
> demotion.
Reviewed-by: Yang Shi <[email protected]>
>
> Signed-off-by: Dave Hansen <[email protected]>
> Cc: David Rientjes <[email protected]>
> Cc: Huang Ying <[email protected]>
> Cc: Dan Williams <[email protected]>
> Cc: David Hildenbrand <[email protected]>
> Cc: osalvador <[email protected]>
> ---
>
> b/mm/vmscan.c | 28 +++++++++++++++++++++++++---
> 1 file changed, 25 insertions(+), 3 deletions(-)
>
> diff -puN mm/vmscan.c~mm-vmscan-anon-can-be-aged mm/vmscan.c
> --- a/mm/vmscan.c~mm-vmscan-anon-can-be-aged 2021-03-04 15:35:58.935806422 -0800
> +++ b/mm/vmscan.c 2021-03-04 15:35:58.942806422 -0800
> @@ -2517,6 +2517,26 @@ out:
> }
> }
>
> +/*
> + * Anonymous LRU management is a waste if there is
> + * ultimately no way to reclaim the memory.
> + */
> +bool anon_should_be_aged(struct lruvec *lruvec)
> +{
> + struct pglist_data *pgdat = lruvec_pgdat(lruvec);
> +
> + /* Aging the anon LRU is valuable if swap is present: */
> + if (total_swap_pages > 0)
> + return true;
> +
> + /* Also valuable if anon pages can be demoted: */
> + if (next_demotion_node(pgdat->node_id) >= 0)
> + return true;
> +
> + /* No way to reclaim anon pages. Should not age anon LRUs: */
> + return false;
> +}
> +
> static void shrink_lruvec(struct lruvec *lruvec, struct scan_control *sc)
> {
> unsigned long nr[NR_LRU_LISTS];
> @@ -2626,7 +2646,8 @@ static void shrink_lruvec(struct lruvec
> * Even if we did not try to evict anon pages at all, we want to
> * rebalance the anon lru active/inactive ratio.
> */
> - if (total_swap_pages && inactive_is_low(lruvec, LRU_INACTIVE_ANON))
> + if (anon_should_be_aged(lruvec) &&
> + inactive_is_low(lruvec, LRU_INACTIVE_ANON))
> shrink_active_list(SWAP_CLUSTER_MAX, lruvec,
> sc, LRU_ACTIVE_ANON);
> }
> @@ -3455,10 +3476,11 @@ static void age_active_anon(struct pglis
> struct mem_cgroup *memcg;
> struct lruvec *lruvec;
>
> - if (!total_swap_pages)
> + lruvec = mem_cgroup_lruvec(NULL, pgdat);
> +
> + if (!anon_should_be_aged(lruvec))
> return;
>
> - lruvec = mem_cgroup_lruvec(NULL, pgdat);
> if (!inactive_is_low(lruvec, LRU_INACTIVE_ANON))
> return;
>
> _
>
On Thu, Mar 4, 2021 at 4:00 PM Dave Hansen <[email protected]> wrote:
>
>
> The full series is also available here:
>
> https://github.com/hansendc/linux/tree/automigrate-20210304
>
> which also inclues some vm.zone_reclaim_mode sysctl ABI fixup
> prerequisites.
>
> The meat of this patch is in:
>
> [PATCH 05/10] mm/migrate: demote pages during reclaim
>
> Which also has the most changes since the last post. This version is
> mostly to address review comments from Yang Shi and Oscar Salvador.
> Review comments are documented in the individual patch changelogs.
>
> This also contains a few prerequisite patches that fix up an issue
> with the vm.zone_reclaim_mode sysctl ABI.
>
> Changes since (automigrate-20210122):
> * move from GFP_HIGHUSER -> GFP_HIGHUSER_MOVABLE since pages *are*
> movable.
> * Separate out helpers that check for being able to relaim anonymous
> pages versus being able to meaningfully scan the anon LRU.
>
> --
>
> We're starting to see systems with more and more kinds of memory such
> as Intel's implementation of persistent memory.
>
> Let's say you have a system with some DRAM and some persistent memory.
> Today, once DRAM fills up, reclaim will start and some of the DRAM
> contents will be thrown out. Allocations will, at some point, start
> falling over to the slower persistent memory.
>
> That has two nasty properties. First, the newer allocations can end
> up in the slower persistent memory. Second, reclaimed data in DRAM
> are just discarded even if there are gobs of space in persistent
> memory that could be used.
>
> This set implements a solution to these problems. At the end of the
> reclaim process in shrink_page_list() just before the last page
> refcount is dropped, the page is migrated to persistent memory instead
> of being dropped.
>
> While I've talked about a DRAM/PMEM pairing, this approach would
> function in any environment where memory tiers exist.
>
> This is not perfect. It "strands" pages in slower memory and never
> brings them back to fast DRAM. Other things need to be built to
> promote hot pages back to DRAM.
>
> This is also all based on an upstream mechanism that allows
> persistent memory to be onlined and used as if it were volatile:
>
> http://lkml.kernel.org/r/[email protected]
>
> == Open Issues ==
>
> * For cpusets and memory policies that restrict allocations
> to PMEM, is it OK to demote to PMEM? Do we need a cgroup-
> level API to opt-in or opt-out of these migrations?
I'm wondering if such usecases, which don't want to have memory
allocate on pmem, will allow memory swapped out or reclaimed? If swap
is allowed then I failed to see why migrating to pmem should be
disallowed. If swap is not allowed, they should call mlock, then the
memory won't be migrated to pmem as well.
> * Could be more aggressive about where anon LRU scanning occurs
> since it no longer necessarily involves I/O. get_scan_count()
> for instance says: "If we have no swap space, do not bother
> scanning anon pages"
Yes, I agree. Johannes's patchset
(https://lore.kernel.org/linux-mm/[email protected]/#r)
has lifted the swappiness to 200 so anonymous lru could be scanned
more aggressively. We definitely could tweak this if needed.
>
> --
>
> Documentation/admin-guide/sysctl/vm.rst | 9
> include/linux/migrate.h | 20 +
> include/linux/swap.h | 3
> include/linux/vm_event_item.h | 2
> include/trace/events/migrate.h | 3
> include/uapi/linux/mempolicy.h | 1
> mm/compaction.c | 3
> mm/gup.c | 4
> mm/internal.h | 5
> mm/memory-failure.c | 4
> mm/memory_hotplug.c | 4
> mm/mempolicy.c | 8
> mm/migrate.c | 369 +++++++++++++++++++++++++++++---
> mm/page_alloc.c | 13 -
> mm/vmscan.c | 173 +++++++++++++--
> mm/vmstat.c | 2
> 16 files changed, 560 insertions(+), 63 deletions(-)
>
> --
>
> Changes since (automigrate-20200818):
> * Fall back to normal reclaim when demotion fails
> * Fix some compile issues, when page migration and NUMA are off
>
> Changes since (automigrate-20201007):
> * separate out checks for "can scan anon LRU" from "can actually
> swap anon pages right now". Previous series conflated them
> and may have been overly aggressive scanning LRU
> * add MR_DEMOTION to tracepoint header
> * remove unnecessary hugetlb page check
>
> Changes since (https://lwn.net/Articles/824830/):
> * Use higher-level migrate_pages() API approach from Yang Shi's
> earlier patches.
> * made sure to actually check node_reclaim_mode's new bit
> * disabled migration entirely before introducing RECLAIM_MIGRATE
> * Replace GFP_NOWAIT with explicit __GFP_KSWAPD_RECLAIM and
> comment why we want that.
> * Comment on effects of that keep multiple source nodes from
> sharing target nodes
>
> Cc: Yang Shi <[email protected]>
> Cc: David Rientjes <[email protected]>
> Cc: Huang Ying <[email protected]>
> Cc: Dan Williams <[email protected]>
> Cc: David Hildenbrand <[email protected]>
> Cc: osalvador <[email protected]>
> Cc: Huang Ying <[email protected]>
>
>
...
>> == Open Issues ==
>>
>> * For cpusets and memory policies that restrict allocations
>> to PMEM, is it OK to demote to PMEM? Do we need a cgroup-
>> level API to opt-in or opt-out of these migrations?
>
> I'm wondering if such usecases, which don't want to have memory
> allocate on pmem, will allow memory swapped out or reclaimed? If swap
> is allowed then I failed to see why migrating to pmem should be
> disallowed. If swap is not allowed, they should call mlock, then the
> memory won't be migrated to pmem as well.
Agreed. I have a hard time imagining there are a lot of folks that can
tolerate the massive overhead from swapping, but can't tolerate the much
smaller overhead of going to pmem instead of DRAM.
Dave Hansen <[email protected]> wrote:
> From: Dave Hansen <[email protected]>
>
> Anonymous pages are kept on their own LRU(s). These lists could
> theoretically always be scanned and maintained. But, without swap,
> there is currently nothing the kernel can *do* with the results of a
> scanned, sorted LRU for anonymous pages.
>
> A check for '!total_swap_pages' currently serves as a valid check as
> to whether anonymous LRUs should be maintained. However, another
> method will be added shortly: page demotion.
>
> Abstract out the 'total_swap_pages' checks into a helper, give it a
> logically significant name, and check for the possibility of page
> demotion.
Reviewed-by: Greg Thelen <[email protected]>
> Signed-off-by: Dave Hansen <[email protected]>
> Cc: David Rientjes <[email protected]>
> Cc: Huang Ying <[email protected]>
> Cc: Dan Williams <[email protected]>
> Cc: David Hildenbrand <[email protected]>
> Cc: osalvador <[email protected]>
> ---
>
> b/mm/vmscan.c | 28 +++++++++++++++++++++++++---
> 1 file changed, 25 insertions(+), 3 deletions(-)
>
> diff -puN mm/vmscan.c~mm-vmscan-anon-can-be-aged mm/vmscan.c
> --- a/mm/vmscan.c~mm-vmscan-anon-can-be-aged 2021-03-04 15:35:58.935806422 -0800
> +++ b/mm/vmscan.c 2021-03-04 15:35:58.942806422 -0800
> @@ -2517,6 +2517,26 @@ out:
> }
> }
>
> +/*
> + * Anonymous LRU management is a waste if there is
> + * ultimately no way to reclaim the memory.
> + */
> +bool anon_should_be_aged(struct lruvec *lruvec)
Should this be static?
> +{
> + struct pglist_data *pgdat = lruvec_pgdat(lruvec);
> +
> + /* Aging the anon LRU is valuable if swap is present: */
> + if (total_swap_pages > 0)
> + return true;
> +
> + /* Also valuable if anon pages can be demoted: */
> + if (next_demotion_node(pgdat->node_id) >= 0)
> + return true;
> +
> + /* No way to reclaim anon pages. Should not age anon LRUs: */
> + return false;
> +}
> +
> static void shrink_lruvec(struct lruvec *lruvec, struct scan_control *sc)
> {
> unsigned long nr[NR_LRU_LISTS];
> @@ -2626,7 +2646,8 @@ static void shrink_lruvec(struct lruvec
> * Even if we did not try to evict anon pages at all, we want to
> * rebalance the anon lru active/inactive ratio.
> */
> - if (total_swap_pages && inactive_is_low(lruvec, LRU_INACTIVE_ANON))
> + if (anon_should_be_aged(lruvec) &&
> + inactive_is_low(lruvec, LRU_INACTIVE_ANON))
> shrink_active_list(SWAP_CLUSTER_MAX, lruvec,
> sc, LRU_ACTIVE_ANON);
> }
> @@ -3455,10 +3476,11 @@ static void age_active_anon(struct pglis
> struct mem_cgroup *memcg;
> struct lruvec *lruvec;
>
> - if (!total_swap_pages)
> + lruvec = mem_cgroup_lruvec(NULL, pgdat);
> +
> + if (!anon_should_be_aged(lruvec))
> return;
>
> - lruvec = mem_cgroup_lruvec(NULL, pgdat);
> if (!inactive_is_low(lruvec, LRU_INACTIVE_ANON))
> return;
>
> _