From: Joonsoo Kim <[email protected]>
Hello,
This patchset implements workingset protection and detection on
the anonymous LRU list.
* SUBJECT
workingset protection
* PROBLEM
In current implementation, newly created or swap-in anonymous page is
started on the active list. Growing the active list results in rebalancing
active/inactive list so old pages on the active list are demoted to the
inactive list. Hence, hot page on the active list isn't protected at all.
Following is an example of this situation.
Assume that 50 hot pages on active list and system can contain total
100 pages. Numbers denote the number of pages on active/inactive
list (active | inactive). (h) stands for hot pages and (uo) stands for
used-once pages.
1. 50 hot pages on active list
50(h) | 0
2. workload: 50 newly created (used-once) pages
50(uo) | 50(h)
3. workload: another 50 newly created (used-once) pages
50(uo) | 50(uo), swap-out 50(h)
As we can see, hot pages are swapped-out and it would cause swap-in later.
* SOLUTION
Since this is what we want to avoid, this patchset implements workingset
protection. Like as the file LRU list, newly created or swap-in anonymous
page is started on the inactive list. Also, like as the file LRU list,
if enough reference happens, the page will be promoted. This simple
modification changes the above example as following.
1. 50 hot pages on active list
50(h) | 0
2. workload: 50 newly created (used-once) pages
50(h) | 50(uo)
3. workload: another 50 newly created (used-once) pages
50(h) | 50(uo), swap-out 50(uo)
hot pages remains in the active list. :)
* EXPERIMENT
I tested this scenario on my test bed and confirmed that this problem
happens on current implementation. I also checked that it is fixed by
this patchset.
I did another test to show the performance effect of this patchset.
- ebizzy (with modified random function)
ebizzy is the test program that main thread allocates lots of memory and
child threads access them randomly during the given times. Swap-in/out
will happen if allocated memory is larger than the system memory.
The random function that represents the zipf distribution is used to
make hot/cold memory. Hot/cold ratio is controlled by the parameter. If
the parameter is high, hot memory is accessed much larger than cold one.
If the parameter is low, the number of access on each memory would be
similar. I uses various parameters in order to show the effect of
patchset on various hot/cold ratio workload.
My test setup is a virtual machine with 8 cpus and 1024MB memory.
Result format is as following.
Parameter 0.1 ... 1.3
Allocated memory size
Throughput for base (larger is better)
Throughput for patchset (larger is better)
Improvement (larger is better)
* single thread
0.1 0.3 0.5 0.7 0.9 1.1 1.3
<512M>
7009.0 7372.0 7774.0 8523.0 9569.0 10724.0 11936.0
6973.0 7342.0 7745.0 8576.0 9441.0 10730.0 12033.0
-0.01 -0.0 -0.0 0.01 -0.01 0.0 0.01
<768M>
915.0 1039.0 1275.0 1687.0 2328.0 3486.0 5445.0
920.0 1037.0 1238.0 1689.0 2384.0 3638.0 5381.0
0.01 -0.0 -0.03 0.0 0.02 0.04 -0.01
<1024M>
425.0 471.0 539.0 753.0 1183.0 2130.0 3839.0
414.0 468.0 553.0 770.0 1242.0 2187.0 3932.0
-0.03 -0.01 0.03 0.02 0.05 0.03 0.02
<1280M>
320.0 346.0 410.0 556.0 871.0 1654.0 3298.0
316.0 346.0 411.0 550.0 892.0 1652.0 3293.0
-0.01 0.0 0.0 -0.01 0.02 -0.0 -0.0
<1536M>
273.0 290.0 341.0 458.0 733.0 1381.0 2925.0
271.0 293.0 344.0 462.0 740.0 1398.0 2969.0
-0.01 0.01 0.01 0.01 0.01 0.01 0.02
<2048M>
77.0 79.0 95.0 147.0 276.0 690.0 1816.0
91.0 94.0 115.0 170.0 321.0 770.0 2018.0
0.18 0.19 0.21 0.16 0.16 0.12 0.11
* multi thread (8)
0.1 0.3 0.5 0.7 0.9 1.1 1.3
<512M>
29083.0 29648.0 30145.0 31668.0 33964.0 38414.0 43707.0
29238.0 29701.0 30301.0 31328.0 33809.0 37991.0 43667.0
0.01 0.0 0.01 -0.01 -0.0 -0.01 -0.0
<768M>
3332.0 3699.0 4673.0 5830.0 8307.0 12969.0 17665.0
3579.0 3992.0 4432.0 6111.0 8699.0 12604.0 18061.0
0.07 0.08 -0.05 0.05 0.05 -0.03 0.02
<1024M>
1921.0 2141.0 2484.0 3296.0 5391.0 8227.0 14574.0
1989.0 2155.0 2609.0 3565.0 5463.0 8170.0 15642.0
0.04 0.01 0.05 0.08 0.01 -0.01 0.07
<1280M>
1524.0 1625.0 1931.0 2581.0 4155.0 6959.0 12443.0
1560.0 1707.0 2016.0 2714.0 4262.0 7518.0 13910.0
0.02 0.05 0.04 0.05 0.03 0.08 0.12
<1536M>
1303.0 1399.0 1550.0 2137.0 3469.0 6712.0 12944.0
1356.0 1465.0 1701.0 2237.0 3583.0 6830.0 13580.0
0.04 0.05 0.1 0.05 0.03 0.02 0.05
<2048M>
172.0 184.0 215.0 289.0 514.0 1318.0 4153.0
175.0 190.0 225.0 329.0 606.0 1585.0 5170.0
0.02 0.03 0.05 0.14 0.18 0.2 0.24
As we can see, as allocated memory grows, patched kernel get the better
result. Maximum improvement is 21% for the single thread test and
24% for the multi thread test.
* SUBJECT
workingset detection
* PROBLEM
Later part of the patchset implements the workingset detection for
the anonymous LRU list. There is a corner case that workingset protection
could cause thrashing. If we can avoid thrashing by workingset detection,
we can get the better performance.
Following is an example of thrashing due to the workingset protection.
1. 50 hot pages on active list
50(h) | 0
2. workload: 50 newly created (will be hot) pages
50(h) | 50(wh)
3. workload: another 50 newly created (used-once) pages
50(h) | 50(uo), swap-out 50(wh)
4. workload: 50 (will be hot) pages
50(h) | 50(wh), swap-in 50(wh)
5. workload: another 50 newly created (used-once) pages
50(h) | 50(uo), swap-out 50(wh)
6. repeat 4, 5
Without workingset detection, this kind of workload cannot be promoted
and thrashing happens forever.
* SOLUTION
Therefore, this patchset implements workingset detection.
All the infrastructure for workingset detecion is already implemented,
so there is not much work to do. First, extend workingset detection
code to deal with the anonymous LRU list. Then, make swap cache handles
the exceptional value for the shadow entry. Lastly, install/retrieve
the shadow value into/from the swap cache and check the refault distance.
* EXPERIMENT
I made a test program to imitates above scenario and confirmed that
problem exists. Then, I checked that this patchset fixes it.
My test setup is a virtual machine with 8 cpus and 6100MB memory. But,
the amount of the memory that the test program can use is about 280 MB.
This is because the system uses large ram-backed swap and large ramdisk
to capture the trace.
Test scenario is like as below.
1. allocate cold memory (512MB)
2. allocate hot-1 memory (96MB)
3. activate hot-1 memory (96MB)
4. allocate another hot-2 memory (96MB)
5. access cold memory (128MB)
6. access hot-2 memory (96MB)
7. repeat 5, 6
Since hot-1 memory (96MB) is on the active list, the inactive list can
contains roughly 190MB pages. hot-2 memory's re-access interval
(96+128 MB) is more 190MB, so it cannot be promoted without workingset
detection and swap-in/out happens repeatedly. With this patchset,
workingset detection works and promotion happens. Therefore, swap-in/out
occurs less.
Here is the result. (average of 5 runs)
type swap-in swap-out
base 863240 989945
patch 681565 809273
As we can see, patched kernel do less swap-in/out.
Note that, this result is gotten from v5.1. Although workingset detection
works on v5.1, it doesn't work well on v5.5. It looks like that recent
code change on workingset.c is the reason of this problem. I will
track it soon.
Patchset is based on v5.5.
Patchset can also be available at
https://github.com/JoonsooKim/linux/tree/improve-anonymous-lru-management-v1.00-v5.5
Enjoy it.
Thanks.
Joonsoo Kim (9):
mm/vmscan: make active/inactive ratio as 1:1 for anon lru
mm/vmscan: protect the workingset on anonymous LRU
mm/workingset: extend the workingset detection for anon LRU
mm/swapcache: support to handle the value in swapcache
mm/workingset: use the node counter if memcg is the root memcg
mm/workingset: handle the page without memcg
mm/swap: implement workingset detection for anonymous LRU
mm/vmscan: restore active/inactive ratio for anonymous LRU
mm/swap: count a new anonymous page as a reclaim_state's rotate
include/linux/mmzone.h | 14 ++++++++-----
include/linux/swap.h | 18 ++++++++++++-----
kernel/events/uprobes.c | 2 +-
mm/huge_memory.c | 6 +++---
mm/khugepaged.c | 2 +-
mm/memcontrol.c | 12 ++++++++----
mm/memory.c | 16 +++++++++------
mm/migrate.c | 2 +-
mm/shmem.c | 3 ++-
mm/swap.c | 40 ++++++++++++++++++++++++++++++-------
mm/swap_state.c | 52 ++++++++++++++++++++++++++++++++++++++++++-------
mm/swapfile.c | 2 +-
mm/userfaultfd.c | 2 +-
mm/vmscan.c | 43 +++++++++++++++++++++++++++++++---------
mm/vmstat.c | 6 ++++--
mm/workingset.c | 47 +++++++++++++++++++++++++++++++-------------
16 files changed, 199 insertions(+), 68 deletions(-)
--
2.7.4
From: Joonsoo Kim <[email protected]>
In the following patch, workingset detection will be applied to
anonymous LRU. To prepare it, this patch adds some code to
distinguish/handle the both LRUs.
Signed-off-by: Joonsoo Kim <[email protected]>
---
include/linux/mmzone.h | 14 +++++++++-----
mm/memcontrol.c | 12 ++++++++----
mm/vmscan.c | 15 ++++++++++-----
mm/vmstat.c | 6 ++++--
mm/workingset.c | 35 ++++++++++++++++++++++-------------
5 files changed, 53 insertions(+), 29 deletions(-)
diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index 5334ad8..b78fd8c 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -220,8 +220,12 @@ enum node_stat_item {
NR_ISOLATED_ANON, /* Temporary isolated pages from anon lru */
NR_ISOLATED_FILE, /* Temporary isolated pages from file lru */
WORKINGSET_NODES,
- WORKINGSET_REFAULT,
- WORKINGSET_ACTIVATE,
+ WORKINGSET_REFAULT_BASE,
+ WORKINGSET_REFAULT_ANON = WORKINGSET_REFAULT_BASE,
+ WORKINGSET_REFAULT_FILE,
+ WORKINGSET_ACTIVATE_BASE,
+ WORKINGSET_ACTIVATE_ANON = WORKINGSET_ACTIVATE_BASE,
+ WORKINGSET_ACTIVATE_FILE,
WORKINGSET_RESTORE,
WORKINGSET_NODERECLAIM,
NR_ANON_MAPPED, /* Mapped anonymous pages */
@@ -304,10 +308,10 @@ enum lruvec_flags {
struct lruvec {
struct list_head lists[NR_LRU_LISTS];
struct zone_reclaim_stat reclaim_stat;
- /* Evictions & activations on the inactive file list */
- atomic_long_t inactive_age;
+ /* Evictions & activations on the inactive list */
+ atomic_long_t inactive_age[2];
/* Refaults at the time of last reclaim cycle */
- unsigned long refaults;
+ unsigned long refaults[2];
/* Various lruvec state flags (enum lruvec_flags) */
unsigned long flags;
#ifdef CONFIG_MEMCG
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 6c83cf4..8f4473d 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -1431,10 +1431,14 @@ static char *memory_stat_format(struct mem_cgroup *memcg)
seq_buf_printf(&s, "%s %lu\n", vm_event_name(PGMAJFAULT),
memcg_events(memcg, PGMAJFAULT));
- seq_buf_printf(&s, "workingset_refault %lu\n",
- memcg_page_state(memcg, WORKINGSET_REFAULT));
- seq_buf_printf(&s, "workingset_activate %lu\n",
- memcg_page_state(memcg, WORKINGSET_ACTIVATE));
+ seq_buf_printf(&s, "workingset_refault_anon %lu\n",
+ memcg_page_state(memcg, WORKINGSET_REFAULT_ANON));
+ seq_buf_printf(&s, "workingset_refault_file %lu\n",
+ memcg_page_state(memcg, WORKINGSET_REFAULT_FILE));
+ seq_buf_printf(&s, "workingset_activate_anon %lu\n",
+ memcg_page_state(memcg, WORKINGSET_ACTIVATE_ANON));
+ seq_buf_printf(&s, "workingset_activate_file %lu\n",
+ memcg_page_state(memcg, WORKINGSET_ACTIVATE_FILE));
seq_buf_printf(&s, "workingset_nodereclaim %lu\n",
memcg_page_state(memcg, WORKINGSET_NODERECLAIM));
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 4122a84..74c3ade 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -2735,7 +2735,10 @@ static bool shrink_node(pg_data_t *pgdat, struct scan_control *sc)
if (!sc->force_deactivate) {
unsigned long refaults;
- if (inactive_is_low(target_lruvec, LRU_INACTIVE_ANON))
+ refaults = lruvec_page_state(target_lruvec,
+ WORKINGSET_ACTIVATE_ANON);
+ if (refaults != target_lruvec->refaults[0] ||
+ inactive_is_low(target_lruvec, LRU_INACTIVE_ANON))
sc->may_deactivate |= DEACTIVATE_ANON;
else
sc->may_deactivate &= ~DEACTIVATE_ANON;
@@ -2746,8 +2749,8 @@ static bool shrink_node(pg_data_t *pgdat, struct scan_control *sc)
* rid of any stale active pages quickly.
*/
refaults = lruvec_page_state(target_lruvec,
- WORKINGSET_ACTIVATE);
- if (refaults != target_lruvec->refaults ||
+ WORKINGSET_ACTIVATE_FILE);
+ if (refaults != target_lruvec->refaults[1] ||
inactive_is_low(target_lruvec, LRU_INACTIVE_FILE))
sc->may_deactivate |= DEACTIVATE_FILE;
else
@@ -3026,8 +3029,10 @@ static void snapshot_refaults(struct mem_cgroup *target_memcg, pg_data_t *pgdat)
unsigned long refaults;
target_lruvec = mem_cgroup_lruvec(target_memcg, pgdat);
- refaults = lruvec_page_state(target_lruvec, WORKINGSET_ACTIVATE);
- target_lruvec->refaults = refaults;
+ refaults = lruvec_page_state(target_lruvec, WORKINGSET_ACTIVATE_ANON);
+ target_lruvec->refaults[0] = refaults;
+ refaults = lruvec_page_state(target_lruvec, WORKINGSET_ACTIVATE_FILE);
+ target_lruvec->refaults[1] = refaults;
}
/*
diff --git a/mm/vmstat.c b/mm/vmstat.c
index 78d5337..3cdf8e9 100644
--- a/mm/vmstat.c
+++ b/mm/vmstat.c
@@ -1146,8 +1146,10 @@ const char * const vmstat_text[] = {
"nr_isolated_anon",
"nr_isolated_file",
"workingset_nodes",
- "workingset_refault",
- "workingset_activate",
+ "workingset_refault_anon",
+ "workingset_refault_file",
+ "workingset_activate_anon",
+ "workingset_activate_file",
"workingset_restore",
"workingset_nodereclaim",
"nr_anon_pages",
diff --git a/mm/workingset.c b/mm/workingset.c
index 474186b..d04f70a 100644
--- a/mm/workingset.c
+++ b/mm/workingset.c
@@ -15,6 +15,7 @@
#include <linux/dax.h>
#include <linux/fs.h>
#include <linux/mm.h>
+#include <linux/mm_inline.h>
/*
* Double CLOCK lists
@@ -156,7 +157,7 @@
*
* Implementation
*
- * For each node's file LRU lists, a counter for inactive evictions
+ * For each node's anon/file LRU lists, a counter for inactive evictions
* and activations is maintained (node->inactive_age).
*
* On eviction, a snapshot of this counter (along with some bits to
@@ -213,7 +214,8 @@ static void unpack_shadow(void *shadow, int *memcgidp, pg_data_t **pgdat,
*workingsetp = workingset;
}
-static void advance_inactive_age(struct mem_cgroup *memcg, pg_data_t *pgdat)
+static void advance_inactive_age(struct mem_cgroup *memcg, pg_data_t *pgdat,
+ int is_file)
{
/*
* Reclaiming a cgroup means reclaiming all its children in a
@@ -230,7 +232,7 @@ static void advance_inactive_age(struct mem_cgroup *memcg, pg_data_t *pgdat)
struct lruvec *lruvec;
lruvec = mem_cgroup_lruvec(memcg, pgdat);
- atomic_long_inc(&lruvec->inactive_age);
+ atomic_long_inc(&lruvec->inactive_age[is_file]);
} while (memcg && (memcg = parent_mem_cgroup(memcg)));
}
@@ -248,18 +250,19 @@ void *workingset_eviction(struct page *page, struct mem_cgroup *target_memcg)
unsigned long eviction;
struct lruvec *lruvec;
int memcgid;
+ int is_file = page_is_file_cache(page);
/* Page is fully exclusive and pins page->mem_cgroup */
VM_BUG_ON_PAGE(PageLRU(page), page);
VM_BUG_ON_PAGE(page_count(page), page);
VM_BUG_ON_PAGE(!PageLocked(page), page);
- advance_inactive_age(page_memcg(page), pgdat);
+ advance_inactive_age(page_memcg(page), pgdat, is_file);
lruvec = mem_cgroup_lruvec(target_memcg, pgdat);
/* XXX: target_memcg can be NULL, go through lruvec */
memcgid = mem_cgroup_id(lruvec_memcg(lruvec));
- eviction = atomic_long_read(&lruvec->inactive_age);
+ eviction = atomic_long_read(&lruvec->inactive_age[is_file]);
return pack_shadow(memcgid, pgdat, eviction, PageWorkingset(page));
}
@@ -278,13 +281,16 @@ void workingset_refault(struct page *page, void *shadow)
struct lruvec *eviction_lruvec;
unsigned long refault_distance;
struct pglist_data *pgdat;
- unsigned long active_file;
+ unsigned long active;
struct mem_cgroup *memcg;
unsigned long eviction;
struct lruvec *lruvec;
unsigned long refault;
bool workingset;
int memcgid;
+ int is_file = page_is_file_cache(page);
+ enum lru_list active_lru = page_lru_base_type(page) + LRU_ACTIVE_FILE;
+ enum node_stat_item workingset_stat;
unpack_shadow(shadow, &memcgid, &pgdat, &eviction, &workingset);
@@ -309,8 +315,8 @@ void workingset_refault(struct page *page, void *shadow)
if (!mem_cgroup_disabled() && !eviction_memcg)
goto out;
eviction_lruvec = mem_cgroup_lruvec(eviction_memcg, pgdat);
- refault = atomic_long_read(&eviction_lruvec->inactive_age);
- active_file = lruvec_page_state(eviction_lruvec, NR_ACTIVE_FILE);
+ refault = atomic_long_read(&eviction_lruvec->inactive_age[is_file]);
+ active = lruvec_page_state(eviction_lruvec, active_lru);
/*
* Calculate the refault distance
@@ -341,19 +347,21 @@ void workingset_refault(struct page *page, void *shadow)
memcg = page_memcg(page);
lruvec = mem_cgroup_lruvec(memcg, pgdat);
- inc_lruvec_state(lruvec, WORKINGSET_REFAULT);
+ workingset_stat = WORKINGSET_REFAULT_BASE + is_file;
+ inc_lruvec_state(lruvec, workingset_stat);
/*
* Compare the distance to the existing workingset size. We
* don't act on pages that couldn't stay resident even if all
* the memory was available to the page cache.
*/
- if (refault_distance > active_file)
+ if (refault_distance > active)
goto out;
SetPageActive(page);
- advance_inactive_age(memcg, pgdat);
- inc_lruvec_state(lruvec, WORKINGSET_ACTIVATE);
+ advance_inactive_age(memcg, pgdat, is_file);
+ workingset_stat = WORKINGSET_ACTIVATE_BASE + is_file;
+ inc_lruvec_state(lruvec, workingset_stat);
/* Page was active prior to eviction */
if (workingset) {
@@ -371,6 +379,7 @@ void workingset_refault(struct page *page, void *shadow)
void workingset_activation(struct page *page)
{
struct mem_cgroup *memcg;
+ int is_file = page_is_file_cache(page);
rcu_read_lock();
/*
@@ -383,7 +392,7 @@ void workingset_activation(struct page *page)
memcg = page_memcg_rcu(page);
if (!mem_cgroup_disabled() && !memcg)
goto out;
- advance_inactive_age(memcg, page_pgdat(page));
+ advance_inactive_age(memcg, page_pgdat(page), is_file);
out:
rcu_read_unlock();
}
--
2.7.4
From: Joonsoo Kim <[email protected]>
In current implementation, newly created or swap-in anonymous page
is started on active list. Growing active list results in rebalancing
active/inactive list so old pages on active list are demoted to inactive
list. Hence, the page on active list isn't protected at all.
Following is an example of this situation.
Assume that 50 hot pages on active list. Numbers denote the number of
pages on active/inactive list (active | inactive).
1. 50 hot pages on active list
50(h) | 0
2. workload: 50 newly created (used-once) pages
50(uo) | 50(h)
3. workload: another 50 newly created (used-once) pages
50(uo) | 50(uo), swap-out 50(h)
This patch tries to fix this issue.
Like as file LRU, newly created or swap-in anonymous pages will be
inserted to the inactive list. They are promoted to active list if
enough reference happens. This simple modification changes the above
example as following.
1. 50 hot pages on active list
50(h) | 0
2. workload: 50 newly created (used-once) pages
50(h) | 50(uo)
3. workload: another 50 newly created (used-once) pages
50(h) | 50(uo), swap-out 50(uo)
As you can see, hot pages on active list would be protected.
Note that, this implementation has a drawback that the page cannot
be promoted and will be swapped-out if re-access interval is greater than
the size of inactive list but less than the size of total(active+inactive).
To solve this potential issue, following patch will apply workingset
detection that is applied to file LRU some day before.
Signed-off-by: Joonsoo Kim <[email protected]>
---
include/linux/swap.h | 2 +-
kernel/events/uprobes.c | 2 +-
mm/huge_memory.c | 6 +++---
mm/khugepaged.c | 2 +-
mm/memory.c | 9 ++++-----
mm/migrate.c | 2 +-
mm/swap.c | 13 +++++++------
mm/swapfile.c | 2 +-
mm/userfaultfd.c | 2 +-
mm/vmscan.c | 21 +++++++++++++++++++--
10 files changed, 39 insertions(+), 22 deletions(-)
diff --git a/include/linux/swap.h b/include/linux/swap.h
index 1e99f7a..954e13e 100644
--- a/include/linux/swap.h
+++ b/include/linux/swap.h
@@ -344,7 +344,7 @@ extern void deactivate_page(struct page *page);
extern void mark_page_lazyfree(struct page *page);
extern void swap_setup(void);
-extern void lru_cache_add_active_or_unevictable(struct page *page,
+extern void lru_cache_add_inactive_or_unevictable(struct page *page,
struct vm_area_struct *vma);
/* linux/mm/vmscan.c */
diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c
index ece7e13..14156fc 100644
--- a/kernel/events/uprobes.c
+++ b/kernel/events/uprobes.c
@@ -190,7 +190,7 @@ static int __replace_page(struct vm_area_struct *vma, unsigned long addr,
get_page(new_page);
page_add_new_anon_rmap(new_page, vma, addr, false);
mem_cgroup_commit_charge(new_page, memcg, false, false);
- lru_cache_add_active_or_unevictable(new_page, vma);
+ lru_cache_add_inactive_or_unevictable(new_page, vma);
} else
/* no new page, just dec_mm_counter for old_page */
dec_mm_counter(mm, MM_ANONPAGES);
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index a880932..6356dfd 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -638,7 +638,7 @@ static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf,
entry = maybe_pmd_mkwrite(pmd_mkdirty(entry), vma);
page_add_new_anon_rmap(page, vma, haddr, true);
mem_cgroup_commit_charge(page, memcg, false, true);
- lru_cache_add_active_or_unevictable(page, vma);
+ lru_cache_add_inactive_or_unevictable(page, vma);
pgtable_trans_huge_deposit(vma->vm_mm, vmf->pmd, pgtable);
set_pmd_at(vma->vm_mm, haddr, vmf->pmd, entry);
add_mm_counter(vma->vm_mm, MM_ANONPAGES, HPAGE_PMD_NR);
@@ -1282,7 +1282,7 @@ static vm_fault_t do_huge_pmd_wp_page_fallback(struct vm_fault *vmf,
set_page_private(pages[i], 0);
page_add_new_anon_rmap(pages[i], vmf->vma, haddr, false);
mem_cgroup_commit_charge(pages[i], memcg, false, false);
- lru_cache_add_active_or_unevictable(pages[i], vma);
+ lru_cache_add_inactive_or_unevictable(pages[i], vma);
vmf->pte = pte_offset_map(&_pmd, haddr);
VM_BUG_ON(!pte_none(*vmf->pte));
set_pte_at(vma->vm_mm, haddr, vmf->pte, entry);
@@ -1435,7 +1435,7 @@ vm_fault_t do_huge_pmd_wp_page(struct vm_fault *vmf, pmd_t orig_pmd)
pmdp_huge_clear_flush_notify(vma, haddr, vmf->pmd);
page_add_new_anon_rmap(new_page, vma, haddr, true);
mem_cgroup_commit_charge(new_page, memcg, false, true);
- lru_cache_add_active_or_unevictable(new_page, vma);
+ lru_cache_add_inactive_or_unevictable(new_page, vma);
set_pmd_at(vma->vm_mm, haddr, vmf->pmd, entry);
update_mmu_cache_pmd(vma, vmf->address, vmf->pmd);
if (!page) {
diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index b679908..246c155 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -1092,7 +1092,7 @@ static void collapse_huge_page(struct mm_struct *mm,
page_add_new_anon_rmap(new_page, vma, address, true);
mem_cgroup_commit_charge(new_page, memcg, false, true);
count_memcg_events(memcg, THP_COLLAPSE_ALLOC, 1);
- lru_cache_add_active_or_unevictable(new_page, vma);
+ lru_cache_add_inactive_or_unevictable(new_page, vma);
pgtable_trans_huge_deposit(mm, pmd, pgtable);
set_pmd_at(mm, address, pmd, _pmd);
update_mmu_cache_pmd(vma, address, pmd);
diff --git a/mm/memory.c b/mm/memory.c
index 45442d9..5f7813a 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -2513,7 +2513,7 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf)
ptep_clear_flush_notify(vma, vmf->address, vmf->pte);
page_add_new_anon_rmap(new_page, vma, vmf->address, false);
mem_cgroup_commit_charge(new_page, memcg, false, false);
- lru_cache_add_active_or_unevictable(new_page, vma);
+ lru_cache_add_inactive_or_unevictable(new_page, vma);
/*
* We call the notify macro here because, when using secondary
* mmu page tables (such as kvm shadow page tables), we want the
@@ -3038,11 +3038,10 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
if (unlikely(page != swapcache && swapcache)) {
page_add_new_anon_rmap(page, vma, vmf->address, false);
mem_cgroup_commit_charge(page, memcg, false, false);
- lru_cache_add_active_or_unevictable(page, vma);
+ lru_cache_add_inactive_or_unevictable(page, vma);
} else {
do_page_add_anon_rmap(page, vma, vmf->address, exclusive);
mem_cgroup_commit_charge(page, memcg, true, false);
- activate_page(page);
}
swap_free(entry);
@@ -3186,7 +3185,7 @@ static vm_fault_t do_anonymous_page(struct vm_fault *vmf)
inc_mm_counter_fast(vma->vm_mm, MM_ANONPAGES);
page_add_new_anon_rmap(page, vma, vmf->address, false);
mem_cgroup_commit_charge(page, memcg, false, false);
- lru_cache_add_active_or_unevictable(page, vma);
+ lru_cache_add_inactive_or_unevictable(page, vma);
setpte:
set_pte_at(vma->vm_mm, vmf->address, vmf->pte, entry);
@@ -3449,7 +3448,7 @@ vm_fault_t alloc_set_pte(struct vm_fault *vmf, struct mem_cgroup *memcg,
inc_mm_counter_fast(vma->vm_mm, MM_ANONPAGES);
page_add_new_anon_rmap(page, vma, vmf->address, false);
mem_cgroup_commit_charge(page, memcg, false, false);
- lru_cache_add_active_or_unevictable(page, vma);
+ lru_cache_add_inactive_or_unevictable(page, vma);
} else {
inc_mm_counter_fast(vma->vm_mm, mm_counter_file(page));
page_add_file_rmap(page, false);
diff --git a/mm/migrate.c b/mm/migrate.c
index 86873b6..ef034c0 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -2784,7 +2784,7 @@ static void migrate_vma_insert_page(struct migrate_vma *migrate,
page_add_new_anon_rmap(page, vma, addr, false);
mem_cgroup_commit_charge(page, memcg, false, false);
if (!is_zone_device_page(page))
- lru_cache_add_active_or_unevictable(page, vma);
+ lru_cache_add_inactive_or_unevictable(page, vma);
get_page(page);
if (flush) {
diff --git a/mm/swap.c b/mm/swap.c
index 5341ae9..18b2735 100644
--- a/mm/swap.c
+++ b/mm/swap.c
@@ -448,23 +448,24 @@ void lru_cache_add(struct page *page)
}
/**
- * lru_cache_add_active_or_unevictable
+ * lru_cache_add_inactive_or_unevictable
* @page: the page to be added to LRU
* @vma: vma in which page is mapped for determining reclaimability
*
- * Place @page on the active or unevictable LRU list, depending on its
+ * Place @page on the inactive or unevictable LRU list, depending on its
* evictability. Note that if the page is not evictable, it goes
* directly back onto it's zone's unevictable list, it does NOT use a
* per cpu pagevec.
*/
-void lru_cache_add_active_or_unevictable(struct page *page,
+void lru_cache_add_inactive_or_unevictable(struct page *page,
struct vm_area_struct *vma)
{
+ bool evictable;
+
VM_BUG_ON_PAGE(PageLRU(page), page);
- if (likely((vma->vm_flags & (VM_LOCKED | VM_SPECIAL)) != VM_LOCKED))
- SetPageActive(page);
- else if (!TestSetPageMlocked(page)) {
+ evictable = (vma->vm_flags & (VM_LOCKED | VM_SPECIAL)) != VM_LOCKED;
+ if (!evictable && !TestSetPageMlocked(page)) {
/*
* We use the irq-unsafe __mod_zone_page_stat because this
* counter is not modified from interrupt context, and the pte
diff --git a/mm/swapfile.c b/mm/swapfile.c
index bb3261d..6bdcbf9 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -1888,7 +1888,7 @@ static int unuse_pte(struct vm_area_struct *vma, pmd_t *pmd,
} else { /* ksm created a completely new copy */
page_add_new_anon_rmap(page, vma, addr, false);
mem_cgroup_commit_charge(page, memcg, false, false);
- lru_cache_add_active_or_unevictable(page, vma);
+ lru_cache_add_inactive_or_unevictable(page, vma);
}
swap_free(entry);
/*
diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
index 1b0d7ab..875e329 100644
--- a/mm/userfaultfd.c
+++ b/mm/userfaultfd.c
@@ -120,7 +120,7 @@ static int mcopy_atomic_pte(struct mm_struct *dst_mm,
inc_mm_counter(dst_mm, MM_ANONPAGES);
page_add_new_anon_rmap(page, dst_vma, dst_addr, false);
mem_cgroup_commit_charge(page, memcg, false, false);
- lru_cache_add_active_or_unevictable(page, dst_vma);
+ lru_cache_add_inactive_or_unevictable(page, dst_vma);
set_pte_at(dst_mm, dst_addr, dst_pte, _dst_pte);
diff --git a/mm/vmscan.c b/mm/vmscan.c
index e772f3f..4122a84 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -1010,8 +1010,15 @@ static enum page_references page_check_references(struct page *page,
return PAGEREF_RECLAIM;
if (referenced_ptes) {
- if (PageSwapBacked(page))
- return PAGEREF_ACTIVATE;
+ if (PageSwapBacked(page)) {
+ if (referenced_page) {
+ ClearPageReferenced(page);
+ return PAGEREF_ACTIVATE;
+ }
+
+ SetPageReferenced(page);
+ return PAGEREF_KEEP;
+ }
/*
* All mapped pages start out with page table
* references from the instantiating fault, so we need
@@ -2056,6 +2063,15 @@ static void shrink_active_list(unsigned long nr_to_scan,
}
}
+ /*
+ * Now, newly created anonymous page isn't appened to the
+ * active list. We don't need to clear the reference bit here.
+ */
+ if (PageSwapBacked(page)) {
+ ClearPageReferenced(page);
+ goto deactivate;
+ }
+
if (page_referenced(page, 0, sc->target_mem_cgroup,
&vm_flags)) {
nr_rotated += hpage_nr_pages(page);
@@ -2074,6 +2090,7 @@ static void shrink_active_list(unsigned long nr_to_scan,
}
}
+deactivate:
ClearPageActive(page); /* we are de-activating */
SetPageWorkingset(page);
list_add(&page->lru, &l_inactive);
--
2.7.4
From: Joonsoo Kim <[email protected]>
Swapcache doesn't handle the value since there is no case using the value.
In the following patch, workingset detection for anonymous page will be
implemented and it stores the value into the swapcache. So, we need to
handle it and this patch implement handling.
Signed-off-by: Joonsoo Kim <[email protected]>
---
include/linux/swap.h | 5 +++--
mm/swap_state.c | 23 ++++++++++++++++++++---
mm/vmscan.c | 2 +-
3 files changed, 24 insertions(+), 6 deletions(-)
diff --git a/include/linux/swap.h b/include/linux/swap.h
index 954e13e..0df8b3f 100644
--- a/include/linux/swap.h
+++ b/include/linux/swap.h
@@ -410,7 +410,8 @@ extern void show_swap_cache_info(void);
extern int add_to_swap(struct page *page);
extern int add_to_swap_cache(struct page *, swp_entry_t, gfp_t);
extern int __add_to_swap_cache(struct page *page, swp_entry_t entry);
-extern void __delete_from_swap_cache(struct page *, swp_entry_t entry);
+extern void __delete_from_swap_cache(struct page *page,
+ swp_entry_t entry, void *shadow);
extern void delete_from_swap_cache(struct page *);
extern void free_page_and_swap_cache(struct page *);
extern void free_pages_and_swap_cache(struct page **, int);
@@ -571,7 +572,7 @@ static inline int add_to_swap_cache(struct page *page, swp_entry_t entry,
}
static inline void __delete_from_swap_cache(struct page *page,
- swp_entry_t entry)
+ swp_entry_t entry, void *shadow)
{
}
diff --git a/mm/swap_state.c b/mm/swap_state.c
index 8e7ce9a..3fbbe45 100644
--- a/mm/swap_state.c
+++ b/mm/swap_state.c
@@ -117,6 +117,10 @@ int add_to_swap_cache(struct page *page, swp_entry_t entry, gfp_t gfp)
pgoff_t idx = swp_offset(entry);
XA_STATE_ORDER(xas, &address_space->i_pages, idx, compound_order(page));
unsigned long i, nr = compound_nr(page);
+ unsigned long nrexceptional = 0;
+ void *old;
+
+ xas_set_update(&xas, workingset_update_node);
VM_BUG_ON_PAGE(!PageLocked(page), page);
VM_BUG_ON_PAGE(PageSwapCache(page), page);
@@ -132,10 +136,14 @@ int add_to_swap_cache(struct page *page, swp_entry_t entry, gfp_t gfp)
goto unlock;
for (i = 0; i < nr; i++) {
VM_BUG_ON_PAGE(xas.xa_index != idx + i, page);
+ old = xas_load(&xas);
+ if (xa_is_value(old))
+ nrexceptional++;
set_page_private(page + i, entry.val + i);
xas_store(&xas, page);
xas_next(&xas);
}
+ address_space->nrexceptional -= nrexceptional;
address_space->nrpages += nr;
__mod_node_page_state(page_pgdat(page), NR_FILE_PAGES, nr);
ADD_CACHE_INFO(add_total, nr);
@@ -155,24 +163,33 @@ int add_to_swap_cache(struct page *page, swp_entry_t entry, gfp_t gfp)
* This must be called only on pages that have
* been verified to be in the swap cache.
*/
-void __delete_from_swap_cache(struct page *page, swp_entry_t entry)
+void __delete_from_swap_cache(struct page *page,
+ swp_entry_t entry, void *shadow)
{
struct address_space *address_space = swap_address_space(entry);
int i, nr = hpage_nr_pages(page);
pgoff_t idx = swp_offset(entry);
XA_STATE(xas, &address_space->i_pages, idx);
+ /* Do not apply workingset detection for the hugh page */
+ if (nr > 1)
+ shadow = NULL;
+
+ xas_set_update(&xas, workingset_update_node);
+
VM_BUG_ON_PAGE(!PageLocked(page), page);
VM_BUG_ON_PAGE(!PageSwapCache(page), page);
VM_BUG_ON_PAGE(PageWriteback(page), page);
for (i = 0; i < nr; i++) {
- void *entry = xas_store(&xas, NULL);
+ void *entry = xas_store(&xas, shadow);
VM_BUG_ON_PAGE(entry != page, entry);
set_page_private(page + i, 0);
xas_next(&xas);
}
ClearPageSwapCache(page);
+ if (shadow)
+ address_space->nrexceptional += nr;
address_space->nrpages -= nr;
__mod_node_page_state(page_pgdat(page), NR_FILE_PAGES, -nr);
ADD_CACHE_INFO(del_total, nr);
@@ -247,7 +264,7 @@ void delete_from_swap_cache(struct page *page)
struct address_space *address_space = swap_address_space(entry);
xa_lock_irq(&address_space->i_pages);
- __delete_from_swap_cache(page, entry);
+ __delete_from_swap_cache(page, entry, NULL);
xa_unlock_irq(&address_space->i_pages);
put_swap_page(page, entry);
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 74c3ade..99588ba 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -909,7 +909,7 @@ static int __remove_mapping(struct address_space *mapping, struct page *page,
if (PageSwapCache(page)) {
swp_entry_t swap = { .val = page_private(page) };
mem_cgroup_swapout(page, swap);
- __delete_from_swap_cache(page, swap);
+ __delete_from_swap_cache(page, swap, NULL);
xa_unlock_irqrestore(&mapping->i_pages, flags);
put_swap_page(page, swap);
} else {
--
2.7.4
From: Joonsoo Kim <[email protected]>
In the following patch, workingset detection is implemented for the
swap cache. Swap cache's node is usually allocated by kswapd and it
isn't charged by kmemcg since it is from the kernel thread. So the swap
cache's shadow node is managed by the node list of the list_lru rather
than the memcg specific one.
If counting the shadow node on the root memcg happens to reclaim the slab
object, the shadow node count returns the number of the shadow node on
the node list of the list_lru since root memcg has the kmem_cache_id, -1.
However, the size of pages on the LRU is calculated by using the specific
memcg, so mismatch happens. This causes the number of shadow node not to
be increased to the enough size and, therefore, workingset detection
cannot work correctly. This patch fixes this bug by checking if the memcg
is the root memcg or not. If it is the root memcg, instead of using
the memcg-specific LRU, the system-wide LRU is used to calculate proper
size of the shadow node so that the number of the shadow node can grow
as expected.
Signed-off-by: Joonsoo Kim <[email protected]>
---
mm/workingset.c | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/mm/workingset.c b/mm/workingset.c
index d04f70a..636aafc 100644
--- a/mm/workingset.c
+++ b/mm/workingset.c
@@ -468,7 +468,13 @@ static unsigned long count_shadow_nodes(struct shrinker *shrinker,
* PAGE_SIZE / xa_nodes / node_entries * 8 / PAGE_SIZE
*/
#ifdef CONFIG_MEMCG
- if (sc->memcg) {
+ /*
+ * Kernel allocation on root memcg isn't regarded as allocation of
+ * specific memcg. So, if sc->memcg is the root memcg, we need to
+ * use the count for the node rather than one for the specific
+ * memcg.
+ */
+ if (sc->memcg && !mem_cgroup_is_root(sc->memcg)) {
struct lruvec *lruvec;
int i;
--
2.7.4
From: Joonsoo Kim <[email protected]>
When implementing workingset detection for anonymous page, I found
some swapcache pages with NULL memcg. From the code reading, I found
two reasons.
One is the case that swap-in readahead happens. The other is the
corner case related to the shmem cache. These two problems should be
fixed, but, it's not straight-forward to fix. For example, when swap-off,
all swapped-out pages are read into swapcache. In this case, who's the
owner of the swapcache page?
Since this problem doesn't look trivial, I decide to leave the issue and
handles this corner case on the place where the error occurs.
Signed-off-by: Joonsoo Kim <[email protected]>
---
mm/workingset.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/mm/workingset.c b/mm/workingset.c
index 636aafc..20286d6 100644
--- a/mm/workingset.c
+++ b/mm/workingset.c
@@ -257,6 +257,10 @@ void *workingset_eviction(struct page *page, struct mem_cgroup *target_memcg)
VM_BUG_ON_PAGE(page_count(page), page);
VM_BUG_ON_PAGE(!PageLocked(page), page);
+ /* page_memcg() can be NULL if swap-in readahead happens */
+ if (!page_memcg(page))
+ return NULL;
+
advance_inactive_age(page_memcg(page), pgdat, is_file);
lruvec = mem_cgroup_lruvec(target_memcg, pgdat);
--
2.7.4
From: Joonsoo Kim <[email protected]>
This patch implements workingset detection for anonymous LRU.
All the infrastructure is implemented by the previous patches so this patch
just activates the workingset detection by installing/retrieving
the shadow entry.
Signed-off-by: Joonsoo Kim <[email protected]>
---
include/linux/swap.h | 11 +++++++++--
mm/memory.c | 7 ++++++-
mm/shmem.c | 3 ++-
mm/swap_state.c | 31 ++++++++++++++++++++++++++-----
mm/vmscan.c | 7 +++++--
5 files changed, 48 insertions(+), 11 deletions(-)
diff --git a/include/linux/swap.h b/include/linux/swap.h
index 0df8b3f..fb4772e 100644
--- a/include/linux/swap.h
+++ b/include/linux/swap.h
@@ -408,7 +408,9 @@ extern struct address_space *swapper_spaces[];
extern unsigned long total_swapcache_pages(void);
extern void show_swap_cache_info(void);
extern int add_to_swap(struct page *page);
-extern int add_to_swap_cache(struct page *, swp_entry_t, gfp_t);
+extern void *get_shadow_from_swap_cache(swp_entry_t entry);
+extern int add_to_swap_cache(struct page *page, swp_entry_t entry,
+ gfp_t gfp, void **shadowp);
extern int __add_to_swap_cache(struct page *page, swp_entry_t entry);
extern void __delete_from_swap_cache(struct page *page,
swp_entry_t entry, void *shadow);
@@ -565,8 +567,13 @@ static inline int add_to_swap(struct page *page)
return 0;
}
+static inline void *get_shadow_from_swap_cache(swp_entry_t entry)
+{
+ return NULL;
+}
+
static inline int add_to_swap_cache(struct page *page, swp_entry_t entry,
- gfp_t gfp_mask)
+ gfp_t gfp_mask, void **shadowp)
{
return -1;
}
diff --git a/mm/memory.c b/mm/memory.c
index 5f7813a..91a2097 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -2925,10 +2925,15 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
page = alloc_page_vma(GFP_HIGHUSER_MOVABLE, vma,
vmf->address);
if (page) {
+ void *shadow;
+
__SetPageLocked(page);
__SetPageSwapBacked(page);
set_page_private(page, entry.val);
- lru_cache_add_anon(page);
+ shadow = get_shadow_from_swap_cache(entry);
+ if (shadow)
+ workingset_refault(page, shadow);
+ lru_cache_add(page);
swap_readpage(page, true);
}
} else {
diff --git a/mm/shmem.c b/mm/shmem.c
index 8793e8c..c6663ad 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -1370,7 +1370,8 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc)
list_add(&info->swaplist, &shmem_swaplist);
if (add_to_swap_cache(page, swap,
- __GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN) == 0) {
+ __GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN,
+ NULL) == 0) {
spin_lock_irq(&info->lock);
shmem_recalc_inode(inode);
info->swapped++;
diff --git a/mm/swap_state.c b/mm/swap_state.c
index 3fbbe45..7f7cb19 100644
--- a/mm/swap_state.c
+++ b/mm/swap_state.c
@@ -107,11 +107,24 @@ void show_swap_cache_info(void)
printk("Total swap = %lukB\n", total_swap_pages << (PAGE_SHIFT - 10));
}
+void *get_shadow_from_swap_cache(swp_entry_t entry)
+{
+ struct address_space *address_space = swap_address_space(entry);
+ pgoff_t idx = swp_offset(entry);
+ struct page *page;
+
+ page = find_get_entry(address_space, idx);
+ if (xa_is_value(page))
+ return page;
+ return NULL;
+}
+
/*
* add_to_swap_cache resembles add_to_page_cache_locked on swapper_space,
* but sets SwapCache flag and private instead of mapping and index.
*/
-int add_to_swap_cache(struct page *page, swp_entry_t entry, gfp_t gfp)
+int add_to_swap_cache(struct page *page, swp_entry_t entry,
+ gfp_t gfp, void **shadowp)
{
struct address_space *address_space = swap_address_space(entry);
pgoff_t idx = swp_offset(entry);
@@ -137,8 +150,11 @@ int add_to_swap_cache(struct page *page, swp_entry_t entry, gfp_t gfp)
for (i = 0; i < nr; i++) {
VM_BUG_ON_PAGE(xas.xa_index != idx + i, page);
old = xas_load(&xas);
- if (xa_is_value(old))
+ if (xa_is_value(old)) {
nrexceptional++;
+ if (shadowp)
+ *shadowp = old;
+ }
set_page_private(page + i, entry.val + i);
xas_store(&xas, page);
xas_next(&xas);
@@ -226,7 +242,7 @@ int add_to_swap(struct page *page)
* Add it to the swap cache.
*/
err = add_to_swap_cache(page, entry,
- __GFP_HIGH|__GFP_NOMEMALLOC|__GFP_NOWARN);
+ __GFP_HIGH|__GFP_NOMEMALLOC|__GFP_NOWARN, NULL);
if (err)
/*
* add_to_swap_cache() doesn't return -EEXIST, so we can safely
@@ -380,6 +396,7 @@ struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask,
struct page *found_page = NULL, *new_page = NULL;
struct swap_info_struct *si;
int err;
+ void *shadow;
*new_page_allocated = false;
do {
@@ -435,11 +452,15 @@ struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask,
/* May fail (-ENOMEM) if XArray node allocation failed. */
__SetPageLocked(new_page);
__SetPageSwapBacked(new_page);
- err = add_to_swap_cache(new_page, entry, gfp_mask & GFP_KERNEL);
+ shadow = NULL;
+ err = add_to_swap_cache(new_page, entry,
+ gfp_mask & GFP_KERNEL, &shadow);
if (likely(!err)) {
/* Initiate read into locked page */
SetPageWorkingset(new_page);
- lru_cache_add_anon(new_page);
+ if (shadow)
+ workingset_refault(new_page, shadow);
+ lru_cache_add(new_page);
*new_page_allocated = true;
return new_page;
}
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 99588ba..a1892e7 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -867,6 +867,7 @@ static int __remove_mapping(struct address_space *mapping, struct page *page,
{
unsigned long flags;
int refcount;
+ void *shadow = NULL;
BUG_ON(!PageLocked(page));
BUG_ON(mapping != page_mapping(page));
@@ -909,12 +910,13 @@ static int __remove_mapping(struct address_space *mapping, struct page *page,
if (PageSwapCache(page)) {
swp_entry_t swap = { .val = page_private(page) };
mem_cgroup_swapout(page, swap);
- __delete_from_swap_cache(page, swap, NULL);
+ if (reclaimed && !mapping_exiting(mapping))
+ shadow = workingset_eviction(page, target_memcg);
+ __delete_from_swap_cache(page, swap, shadow);
xa_unlock_irqrestore(&mapping->i_pages, flags);
put_swap_page(page, swap);
} else {
void (*freepage)(struct page *);
- void *shadow = NULL;
freepage = mapping->a_ops->freepage;
/*
@@ -1485,6 +1487,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
SetPageActive(page);
stat->nr_activate[type] += nr_pages;
count_memcg_page_event(page, PGACTIVATE);
+ workingset_activation(page);
}
keep_locked:
unlock_page(page);
--
2.7.4
From: Joonsoo Kim <[email protected]>
Now, workingset detection is implemented for anonymous LRU.
We don't have to worry about the misfound for workingset due to
the ratio of active/inactive. Let's restore the ratio.
Signed-off-by: Joonsoo Kim <[email protected]>
---
mm/vmscan.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/mm/vmscan.c b/mm/vmscan.c
index a1892e7..81ff725 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -2237,7 +2237,7 @@ static bool inactive_is_low(struct lruvec *lruvec, enum lru_list inactive_lru)
active = lruvec_page_state(lruvec, NR_LRU_BASE + active_lru);
gb = (inactive + active) >> (30 - PAGE_SHIFT);
- if (gb && is_file_lru(inactive_lru))
+ if (gb)
inactive_ratio = int_sqrt(10 * gb);
else
inactive_ratio = 1;
--
2.7.4
From: Joonsoo Kim <[email protected]>
reclaim_stat's rotate is used for controlling the ratio of scanning page
between file and anonymous LRU. All new anonymous pages are counted
for rotate before the patch, protecting anonymous pages on active LRU, and,
it makes that reclaim on anonymous LRU is less happened than file LRU.
Now, situation is changed. all new anonymous pages are not added
to the active LRU so rotate would be far less than before. It will cause
that reclaim on anonymous LRU happens more and it would result in bad
effect on some system that is optimized for previous setting.
Therefore, this patch counts a new anonymous page as a reclaim_state's
rotate. Although it is non-logical to add this count to
the reclaim_state's rotate in current algorithm, reducing the regression
would be more important.
I found this regression on kernel-build test and it is roughly 2~5%
performance degradation. With this workaround, performance is completely
restored.
Signed-off-by: Joonsoo Kim <[email protected]>
---
mm/swap.c | 27 ++++++++++++++++++++++++++-
1 file changed, 26 insertions(+), 1 deletion(-)
diff --git a/mm/swap.c b/mm/swap.c
index 18b2735..c3584af 100644
--- a/mm/swap.c
+++ b/mm/swap.c
@@ -187,6 +187,9 @@ int get_kernel_page(unsigned long start, int write, struct page **pages)
}
EXPORT_SYMBOL_GPL(get_kernel_page);
+static void __pagevec_lru_add_fn(struct page *page, struct lruvec *lruvec,
+ void *arg);
+
static void pagevec_lru_move_fn(struct pagevec *pvec,
void (*move_fn)(struct page *page, struct lruvec *lruvec, void *arg),
void *arg)
@@ -207,6 +210,19 @@ static void pagevec_lru_move_fn(struct pagevec *pvec,
spin_lock_irqsave(&pgdat->lru_lock, flags);
}
+ if (move_fn == __pagevec_lru_add_fn) {
+ struct list_head *entry = &page->lru;
+ unsigned long next = (unsigned long)entry->next;
+ unsigned long rotate = next & 2;
+
+ if (rotate) {
+ VM_BUG_ON(arg);
+
+ next = next & ~2;
+ entry->next = (struct list_head *)next;
+ arg = (void *)rotate;
+ }
+ }
lruvec = mem_cgroup_page_lruvec(page, pgdat);
(*move_fn)(page, lruvec, arg);
}
@@ -475,6 +491,14 @@ void lru_cache_add_inactive_or_unevictable(struct page *page,
hpage_nr_pages(page));
count_vm_event(UNEVICTABLE_PGMLOCKED);
}
+
+ if (PageSwapBacked(page) && evictable) {
+ struct list_head *entry = &page->lru;
+ unsigned long next = (unsigned long)entry->next;
+
+ next = next | 2;
+ entry->next = (struct list_head *)next;
+ }
lru_cache_add(page);
}
@@ -927,6 +951,7 @@ static void __pagevec_lru_add_fn(struct page *page, struct lruvec *lruvec,
{
enum lru_list lru;
int was_unevictable = TestClearPageUnevictable(page);
+ unsigned long rotate = (unsigned long)arg;
VM_BUG_ON_PAGE(PageLRU(page), page);
@@ -962,7 +987,7 @@ static void __pagevec_lru_add_fn(struct page *page, struct lruvec *lruvec,
if (page_evictable(page)) {
lru = page_lru(page);
update_page_reclaim_stat(lruvec, page_is_file_cache(page),
- PageActive(page));
+ PageActive(page) | rotate);
if (was_unevictable)
count_vm_event(UNEVICTABLE_PGRESCUED);
} else {
--
2.7.4
Hello,
2020년 2월 12일 (수) 오후 12:35, Hillf Danton <[email protected]>님이 작성:
>
>
> On Mon, 10 Feb 2020 22:20:37 -0800 (PST)
> > From: Joonsoo Kim <[email protected]>
> >
> > reclaim_stat's rotate is used for controlling the ratio of scanning page
> > between file and anonymous LRU. All new anonymous pages are counted
> > for rotate before the patch, protecting anonymous pages on active LRU, and,
> > it makes that reclaim on anonymous LRU is less happened than file LRU.
> >
> > Now, situation is changed. all new anonymous pages are not added
> > to the active LRU so rotate would be far less than before. It will cause
> > that reclaim on anonymous LRU happens more and it would result in bad
> > effect on some system that is optimized for previous setting.
> >
> > Therefore, this patch counts a new anonymous page as a reclaim_state's
> > rotate. Although it is non-logical to add this count to
> > the reclaim_state's rotate in current algorithm, reducing the regression
> > would be more important.
> >
> > I found this regression on kernel-build test and it is roughly 2~5%
> > performance degradation. With this workaround, performance is completely
> > restored.
> >
> > Signed-off-by: Joonsoo Kim <[email protected]>
> > ---
> > mm/swap.c | 27 ++++++++++++++++++++++++++-
> > 1 file changed, 26 insertions(+), 1 deletion(-)
> >
> > diff --git a/mm/swap.c b/mm/swap.c
> > index 18b2735..c3584af 100644
> > --- a/mm/swap.c
> > +++ b/mm/swap.c
> > @@ -187,6 +187,9 @@ int get_kernel_page(unsigned long start, int write, struct page **pages)
> > }
> > EXPORT_SYMBOL_GPL(get_kernel_page);
> >
> > +static void __pagevec_lru_add_fn(struct page *page, struct lruvec *lruvec,
> > + void *arg);
> > +
> > static void pagevec_lru_move_fn(struct pagevec *pvec,
> > void (*move_fn)(struct page *page, struct lruvec *lruvec, void *arg),
> > void *arg)
> > @@ -207,6 +210,19 @@ static void pagevec_lru_move_fn(struct pagevec *pvec,
> > spin_lock_irqsave(&pgdat->lru_lock, flags);
> > }
> >
> > + if (move_fn == __pagevec_lru_add_fn) {
> > + struct list_head *entry = &page->lru;
> > + unsigned long next = (unsigned long)entry->next;
> > + unsigned long rotate = next & 2;
> > +
> > + if (rotate) {
> > + VM_BUG_ON(arg);
> > +
> > + next = next & ~2;
> > + entry->next = (struct list_head *)next;
> > + arg = (void *)rotate;
> > + }
> > + }
> > lruvec = mem_cgroup_page_lruvec(page, pgdat);
> > (*move_fn)(page, lruvec, arg);
> > }
> > @@ -475,6 +491,14 @@ void lru_cache_add_inactive_or_unevictable(struct page *page,
> > hpage_nr_pages(page));
> > count_vm_event(UNEVICTABLE_PGMLOCKED);
> > }
> > +
> > + if (PageSwapBacked(page) && evictable) {
> > + struct list_head *entry = &page->lru;
> > + unsigned long next = (unsigned long)entry->next;
> > +
> > + next = next | 2;
> > + entry->next = (struct list_head *)next;
> > + }
> > lru_cache_add(page);
> > }
> >
> > @@ -927,6 +951,7 @@ static void __pagevec_lru_add_fn(struct page *page, struct lruvec *lruvec,
> > {
> > enum lru_list lru;
> > int was_unevictable = TestClearPageUnevictable(page);
> > + unsigned long rotate = (unsigned long)arg;
> >
> > VM_BUG_ON_PAGE(PageLRU(page), page);
> >
> > @@ -962,7 +987,7 @@ static void __pagevec_lru_add_fn(struct page *page, struct lruvec *lruvec,
> > if (page_evictable(page)) {
> > lru = page_lru(page);
> > update_page_reclaim_stat(lruvec, page_is_file_cache(page),
> > - PageActive(page));
> > + PageActive(page) | rotate);
>
>
> Is it likely to rotate a page if we know it's not active?
>
> update_page_reclaim_stat(lruvec, page_is_file_cache(page),
> - PageActive(page));
> + PageActive(page) ||
> + !page_is_file_cache(page));
>
My intention is that only newly created anonymous pages contributes
the rotate count.
With your code suggestion, other case for anonymous pages could also contributes
the rotate count since __pagevec_lru_add_fn() is used else where.
Thanks.
2020년 2월 11일 (화) 오후 3:20, <[email protected]>님이 작성:
>
> From: Joonsoo Kim <[email protected]>
>
> In the following patch, workingset detection will be applied to
> anonymous LRU. To prepare it, this patch adds some code to
> distinguish/handle the both LRUs.
>
> Signed-off-by: Joonsoo Kim <[email protected]>
> ---
> include/linux/mmzone.h | 14 +++++++++-----
> mm/memcontrol.c | 12 ++++++++----
> mm/vmscan.c | 15 ++++++++++-----
> mm/vmstat.c | 6 ++++--
> mm/workingset.c | 35 ++++++++++++++++++++++-------------
> 5 files changed, 53 insertions(+), 29 deletions(-)
This patch should be changed as following.
- enum lru_list active_lru = page_lru_base_type(page) + LRU_ACTIVE_FILE;
+ enum lru_list active_lru = page_lru_base_type(page) + LRU_ACTIVE;
Whole fixed patch is as following.
--------------------->8----------------------
From 2b0691140d11c4e9a0f1500dda831b70697b2a00 Mon Sep 17 00:00:00 2001
From: Joonsoo Kim <[email protected]>
Date: Fri, 15 Nov 2019 09:40:22 +0900
Subject: [PATCH] mm/workingset: extend the workingset detection for anon LRU
In the following patch, workingset detection will be applied to
anonymous LRU. To prepare it, this patch adds some code to
distinguish/handle the both LRUs.
Signed-off-by: Joonsoo Kim <[email protected]>
---
include/linux/mmzone.h | 14 +++++++++-----
mm/memcontrol.c | 12 ++++++++----
mm/vmscan.c | 15 ++++++++++-----
mm/vmstat.c | 6 ++++--
mm/workingset.c | 35 ++++++++++++++++++++++-------------
5 files changed, 53 insertions(+), 29 deletions(-)
diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index 5334ad8fc7bd..b78fd8c7284b 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -220,8 +220,12 @@ enum node_stat_item {
NR_ISOLATED_ANON, /* Temporary isolated pages from anon lru */
NR_ISOLATED_FILE, /* Temporary isolated pages from file lru */
WORKINGSET_NODES,
- WORKINGSET_REFAULT,
- WORKINGSET_ACTIVATE,
+ WORKINGSET_REFAULT_BASE,
+ WORKINGSET_REFAULT_ANON = WORKINGSET_REFAULT_BASE,
+ WORKINGSET_REFAULT_FILE,
+ WORKINGSET_ACTIVATE_BASE,
+ WORKINGSET_ACTIVATE_ANON = WORKINGSET_ACTIVATE_BASE,
+ WORKINGSET_ACTIVATE_FILE,
WORKINGSET_RESTORE,
WORKINGSET_NODERECLAIM,
NR_ANON_MAPPED, /* Mapped anonymous pages */
@@ -304,10 +308,10 @@ enum lruvec_flags {
struct lruvec {
struct list_head lists[NR_LRU_LISTS];
struct zone_reclaim_stat reclaim_stat;
- /* Evictions & activations on the inactive file list */
- atomic_long_t inactive_age;
+ /* Evictions & activations on the inactive list */
+ atomic_long_t inactive_age[2];
/* Refaults at the time of last reclaim cycle */
- unsigned long refaults;
+ unsigned long refaults[2];
/* Various lruvec state flags (enum lruvec_flags) */
unsigned long flags;
#ifdef CONFIG_MEMCG
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 6c83cf4ed970..8f4473d6ff9c 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -1431,10 +1431,14 @@ static char *memory_stat_format(struct
mem_cgroup *memcg)
seq_buf_printf(&s, "%s %lu\n", vm_event_name(PGMAJFAULT),
memcg_events(memcg, PGMAJFAULT));
- seq_buf_printf(&s, "workingset_refault %lu\n",
- memcg_page_state(memcg, WORKINGSET_REFAULT));
- seq_buf_printf(&s, "workingset_activate %lu\n",
- memcg_page_state(memcg, WORKINGSET_ACTIVATE));
+ seq_buf_printf(&s, "workingset_refault_anon %lu\n",
+ memcg_page_state(memcg, WORKINGSET_REFAULT_ANON));
+ seq_buf_printf(&s, "workingset_refault_file %lu\n",
+ memcg_page_state(memcg, WORKINGSET_REFAULT_FILE));
+ seq_buf_printf(&s, "workingset_activate_anon %lu\n",
+ memcg_page_state(memcg, WORKINGSET_ACTIVATE_ANON));
+ seq_buf_printf(&s, "workingset_activate_file %lu\n",
+ memcg_page_state(memcg, WORKINGSET_ACTIVATE_FILE));
seq_buf_printf(&s, "workingset_nodereclaim %lu\n",
memcg_page_state(memcg, WORKINGSET_NODERECLAIM));
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 4122a841dfce..74c3adefc933 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -2735,7 +2735,10 @@ static bool shrink_node(pg_data_t *pgdat,
struct scan_control *sc)
if (!sc->force_deactivate) {
unsigned long refaults;
- if (inactive_is_low(target_lruvec, LRU_INACTIVE_ANON))
+ refaults = lruvec_page_state(target_lruvec,
+ WORKINGSET_ACTIVATE_ANON);
+ if (refaults != target_lruvec->refaults[0] ||
+ inactive_is_low(target_lruvec, LRU_INACTIVE_ANON))
sc->may_deactivate |= DEACTIVATE_ANON;
else
sc->may_deactivate &= ~DEACTIVATE_ANON;
@@ -2746,8 +2749,8 @@ static bool shrink_node(pg_data_t *pgdat, struct
scan_control *sc)
* rid of any stale active pages quickly.
*/
refaults = lruvec_page_state(target_lruvec,
- WORKINGSET_ACTIVATE);
- if (refaults != target_lruvec->refaults ||
+ WORKINGSET_ACTIVATE_FILE);
+ if (refaults != target_lruvec->refaults[1] ||
inactive_is_low(target_lruvec, LRU_INACTIVE_FILE))
sc->may_deactivate |= DEACTIVATE_FILE;
else
@@ -3026,8 +3029,10 @@ static void snapshot_refaults(struct mem_cgroup
*target_memcg, pg_data_t *pgdat)
unsigned long refaults;
target_lruvec = mem_cgroup_lruvec(target_memcg, pgdat);
- refaults = lruvec_page_state(target_lruvec, WORKINGSET_ACTIVATE);
- target_lruvec->refaults = refaults;
+ refaults = lruvec_page_state(target_lruvec, WORKINGSET_ACTIVATE_ANON);
+ target_lruvec->refaults[0] = refaults;
+ refaults = lruvec_page_state(target_lruvec, WORKINGSET_ACTIVATE_FILE);
+ target_lruvec->refaults[1] = refaults;
}
/*
diff --git a/mm/vmstat.c b/mm/vmstat.c
index 78d53378db99..3cdf8e9b0ba2 100644
--- a/mm/vmstat.c
+++ b/mm/vmstat.c
@@ -1146,8 +1146,10 @@ const char * const vmstat_text[] = {
"nr_isolated_anon",
"nr_isolated_file",
"workingset_nodes",
- "workingset_refault",
- "workingset_activate",
+ "workingset_refault_anon",
+ "workingset_refault_file",
+ "workingset_activate_anon",
+ "workingset_activate_file",
"workingset_restore",
"workingset_nodereclaim",
"nr_anon_pages",
diff --git a/mm/workingset.c b/mm/workingset.c
index 474186b76ced..5fb8f85d1fec 100644
--- a/mm/workingset.c
+++ b/mm/workingset.c
@@ -15,6 +15,7 @@
#include <linux/dax.h>
#include <linux/fs.h>
#include <linux/mm.h>
+#include <linux/mm_inline.h>
/*
* Double CLOCK lists
@@ -156,7 +157,7 @@
*
* Implementation
*
- * For each node's file LRU lists, a counter for inactive evictions
+ * For each node's anon/file LRU lists, a counter for inactive evictions
* and activations is maintained (node->inactive_age).
*
* On eviction, a snapshot of this counter (along with some bits to
@@ -213,7 +214,8 @@ static void unpack_shadow(void *shadow, int
*memcgidp, pg_data_t **pgdat,
*workingsetp = workingset;
}
-static void advance_inactive_age(struct mem_cgroup *memcg, pg_data_t *pgdat)
+static void advance_inactive_age(struct mem_cgroup *memcg, pg_data_t *pgdat,
+ int is_file)
{
/*
* Reclaiming a cgroup means reclaiming all its children in a
@@ -230,7 +232,7 @@ static void advance_inactive_age(struct mem_cgroup
*memcg, pg_data_t *pgdat)
struct lruvec *lruvec;
lruvec = mem_cgroup_lruvec(memcg, pgdat);
- atomic_long_inc(&lruvec->inactive_age);
+ atomic_long_inc(&lruvec->inactive_age[is_file]);
} while (memcg && (memcg = parent_mem_cgroup(memcg)));
}
@@ -248,18 +250,19 @@ void *workingset_eviction(struct page *page,
struct mem_cgroup *target_memcg)
unsigned long eviction;
struct lruvec *lruvec;
int memcgid;
+ int is_file = page_is_file_cache(page);
/* Page is fully exclusive and pins page->mem_cgroup */
VM_BUG_ON_PAGE(PageLRU(page), page);
VM_BUG_ON_PAGE(page_count(page), page);
VM_BUG_ON_PAGE(!PageLocked(page), page);
- advance_inactive_age(page_memcg(page), pgdat);
+ advance_inactive_age(page_memcg(page), pgdat, is_file);
lruvec = mem_cgroup_lruvec(target_memcg, pgdat);
/* XXX: target_memcg can be NULL, go through lruvec */
memcgid = mem_cgroup_id(lruvec_memcg(lruvec));
- eviction = atomic_long_read(&lruvec->inactive_age);
+ eviction = atomic_long_read(&lruvec->inactive_age[is_file]);
return pack_shadow(memcgid, pgdat, eviction, PageWorkingset(page));
}
@@ -278,13 +281,16 @@ void workingset_refault(struct page *page, void *shadow)
struct lruvec *eviction_lruvec;
unsigned long refault_distance;
struct pglist_data *pgdat;
- unsigned long active_file;
+ unsigned long active;
struct mem_cgroup *memcg;
unsigned long eviction;
struct lruvec *lruvec;
unsigned long refault;
bool workingset;
int memcgid;
+ int is_file = page_is_file_cache(page);
+ enum lru_list active_lru = page_lru_base_type(page) + LRU_ACTIVE;
+ enum node_stat_item workingset_stat;
unpack_shadow(shadow, &memcgid, &pgdat, &eviction, &workingset);
@@ -309,8 +315,8 @@ void workingset_refault(struct page *page, void *shadow)
if (!mem_cgroup_disabled() && !eviction_memcg)
goto out;
eviction_lruvec = mem_cgroup_lruvec(eviction_memcg, pgdat);
- refault = atomic_long_read(&eviction_lruvec->inactive_age);
- active_file = lruvec_page_state(eviction_lruvec, NR_ACTIVE_FILE);
+ refault = atomic_long_read(&eviction_lruvec->inactive_age[is_file]);
+ active = lruvec_page_state(eviction_lruvec, active_lru);
/*
* Calculate the refault distance
@@ -341,19 +347,21 @@ void workingset_refault(struct page *page, void *shadow)
memcg = page_memcg(page);
lruvec = mem_cgroup_lruvec(memcg, pgdat);
- inc_lruvec_state(lruvec, WORKINGSET_REFAULT);
+ workingset_stat = WORKINGSET_REFAULT_BASE + is_file;
+ inc_lruvec_state(lruvec, workingset_stat);
/*
* Compare the distance to the existing workingset size. We
* don't act on pages that couldn't stay resident even if all
* the memory was available to the page cache.
*/
- if (refault_distance > active_file)
+ if (refault_distance > active)
goto out;
SetPageActive(page);
- advance_inactive_age(memcg, pgdat);
- inc_lruvec_state(lruvec, WORKINGSET_ACTIVATE);
+ advance_inactive_age(memcg, pgdat, is_file);
+ workingset_stat = WORKINGSET_ACTIVATE_BASE + is_file;
+ inc_lruvec_state(lruvec, workingset_stat);
/* Page was active prior to eviction */
if (workingset) {
@@ -371,6 +379,7 @@ void workingset_refault(struct page *page, void *shadow)
void workingset_activation(struct page *page)
{
struct mem_cgroup *memcg;
+ int is_file = page_is_file_cache(page);
rcu_read_lock();
/*
@@ -383,7 +392,7 @@ void workingset_activation(struct page *page)
memcg = page_memcg_rcu(page);
if (!mem_cgroup_disabled() && !memcg)
goto out;
- advance_inactive_age(memcg, page_pgdat(page));
+ advance_inactive_age(memcg, page_pgdat(page), is_file);
out:
rcu_read_unlock();
}
--
2.17.1
2020년 2월 11일 (화) 오후 3:20, <[email protected]>님이 작성:
>
> From: Joonsoo Kim <[email protected]>
>
> Hello,
>
> This patchset implements workingset protection and detection on
> the anonymous LRU list.
>
snip...
>
> Note that, this result is gotten from v5.1. Although workingset detection
> works on v5.1, it doesn't work well on v5.5. It looks like that recent
> code change on workingset.c is the reason of this problem. I will
> track it soon.
Problem is fixed now.
It's due to my mistake on the rebase.
Replace "mm/workingset: extend the workingset detection for anon LRU"
with the patch on the following link.
https://lkml.kernel.org/r/CAAmzW4NGwvTiE_unACAcSZUH9V3tO0qR=ZPxi=q9s=zDi53DeQ@mail.gmail.com
Thanks.
Greeting,
FYI, we noticed a 19.5% improvement of fio.read_bw_MBps due to commit:
commit: 323c95f0957c8a7d59cd9d05f43f19dfc9bbc584 ("[PATCH 3/9] mm/workingset: extend the workingset detection for anon LRU")
url: https://github.com/0day-ci/linux/commits/js1304-gmail-com/workingset-protection-detection-on-the-anonymous-LRU-list/20200213-111547
in testcase: fio-basic
on test machine: 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 256G memory
with following parameters:
disk: 2pmem
fs: ext4
runtime: 200s
nr_task: 50%
time_based: tb
rw: randrw
bs: 2M
ioengine: mmap
test_size: 200G
cpufreq_governor: performance
ucode: 0x500002c
test-description: Fio is a tool that will spawn a number of threads or processes doing a particular type of I/O action as specified by the user.
test-url: https://github.com/axboe/fio
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
bs/compiler/cpufreq_governor/disk/fs/ioengine/kconfig/nr_task/rootfs/runtime/rw/tbox_group/test_size/testcase/time_based/ucode:
2M/gcc-7/performance/2pmem/ext4/mmap/x86_64-rhel-7.6/50%/debian-x86_64-20191114.cgz/200s/randrw/lkp-csl-2sp6/200G/fio-basic/tb/0x500002c
commit:
fffae40d0e ("mm/vmscan: protect the workingset on anonymous LRU")
323c95f095 ("mm/workingset: extend the workingset detection for anon LRU")
fffae40d0ee3fd1e 323c95f0957c8a7d59cd9d05f43
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
1:4 -25% :4 dmesg.WARNING:at_ip___perf_sw_event/0x
:4 25% 1:4 dmesg.WARNING:at_ip_generic_make_request/0x
2:4 45% 4:4 perf-profile.calltrace.cycles-pp.sync_regs.error_entry
2:4 50% 4:4 perf-profile.calltrace.cycles-pp.error_entry
2:4 53% 4:4 perf-profile.children.cycles-pp.error_entry
0:4 6% 0:4 perf-profile.self.cycles-pp.error_entry
%stddev %change %stddev
\ | \
0.28 ± 11% -0.1 0.16 ± 28% fio.latency_100ms%
8.36 ± 8% -5.9 2.50 ± 5% fio.latency_10ms%
6.10 ± 17% -5.3 0.82 ± 6% fio.latency_20ms%
0.00 ±173% +0.0 0.01 ± 10% fio.latency_250ms%
0.71 ± 16% +2.8 3.56 ± 4% fio.latency_250us%
13.45 ± 5% +9.4 22.85 ± 2% fio.latency_2ms%
29.90 ± 2% -7.3 22.58 ± 2% fio.latency_4ms%
8.43 ± 7% +9.4 17.84 ± 4% fio.latency_500us%
7.43 ± 3% -3.3 4.13 ± 12% fio.latency_750us%
4336 ± 2% +19.5% 5182 ± 2% fio.read_bw_MBps
7430144 ± 16% -64.3% 2654208 fio.read_clat_90%_us
11501568 ± 9% -70.8% 3358720 fio.read_clat_95%_us
17170432 ± 8% -57.7% 7258112 ± 4% fio.read_clat_99%_us
2960419 ± 7% -53.3% 1381107 fio.read_clat_mean_us
4105907 ± 14% -45.6% 2232352 ± 11% fio.read_clat_stddev
2168 ± 2% +19.5% 2591 ± 2% fio.read_iops
1.942e+09 ± 2% -22.0% 1.514e+09 ± 2% fio.time.file_system_inputs
1.783e+09 ± 2% +19.5% 2.131e+09 ± 2% fio.time.file_system_outputs
216950 ± 12% -48.8% 111051 ± 16% fio.time.involuntary_context_switches
2.427e+08 ± 2% -22.0% 1.892e+08 ± 2% fio.time.major_page_faults
1.271e+08 ± 5% +66.8% 2.12e+08 ± 2% fio.time.minor_page_faults
1619 ± 9% -33.6% 1074 ± 2% fio.time.percent_of_cpu_this_job_got
2851 ± 9% -40.1% 1708 ± 2% fio.time.system_time
404.54 ± 6% +11.4% 450.81 ± 2% fio.time.user_time
869238 ± 2% +19.5% 1038826 ± 2% fio.workload
4350 ± 2% +19.5% 5199 ± 2% fio.write_bw_MBps
38404096 -12.9% 33456128 fio.write_clat_90%_us
39452672 -14.0% 33947648 fio.write_clat_95%_us
45744128 -19.5% 36831232 ± 3% fio.write_clat_99%_us
17993869 ± 4% -9.2% 16330675 ± 2% fio.write_clat_mean_us
16656042 ± 5% -6.2% 15615331 fio.write_clat_stddev
2175 ± 2% +19.5% 2599 ± 2% fio.write_iops
31.58 ± 4% +5.8 37.39 mpstat.cpu.all.iowait%
18.54 ± 7% -6.5 12.04 ± 2% mpstat.cpu.all.sys%
1.008e+08 ± 10% -37.9% 62633861 ± 18% numa-numastat.node0.local_node
1.008e+08 ± 9% -37.9% 62658517 ± 18% numa-numastat.node0.numa_hit
48.29 +1.1% 48.80 iostat.cpu.idle
31.27 ± 4% +18.4% 37.03 iostat.cpu.iowait
18.38 ± 7% -35.0% 11.94 ± 2% iostat.cpu.system
618.00 ± 6% -27.0% 451.25 turbostat.Avg_MHz
22.14 ± 6% -6.0 16.17 turbostat.Busy%
1.361e+08 ± 7% -26.6% 99961074 ± 3% turbostat.IRQ
224.76 -4.7% 214.16 turbostat.PkgWatt
4580495 +85.7% 8505839 slabinfo.buffer_head.active_objs
119518 +82.7% 218335 slabinfo.buffer_head.active_slabs
4661221 +82.7% 8515085 slabinfo.buffer_head.num_objs
119518 +82.7% 218335 slabinfo.buffer_head.num_slabs
19148 ± 3% +9.2% 20917 ± 3% slabinfo.kmalloc-512.num_objs
30.75 ± 4% +19.5% 36.75 vmstat.cpu.wa
4753706 ± 2% -21.9% 3710468 ± 2% vmstat.io.bi
4361808 ± 2% +19.7% 5220006 ± 2% vmstat.io.bo
619.75 ± 8% +86.0% 1153 ± 5% vmstat.memory.buff
31.25 ± 4% +15.2% 36.00 vmstat.procs.b
19.00 ± 9% -28.9% 13.50 ± 3% vmstat.procs.r
409038 ± 5% -18.7% 332463 ± 2% vmstat.system.in
9785202 +221.1% 31421653 meminfo.Active
60190 ± 8% -34.3% 39569 ± 3% meminfo.Active(anon)
9725011 +222.7% 31382083 meminfo.Active(file)
619.25 ± 7% +86.7% 1156 ± 5% meminfo.Buffers
32198569 -68.1% 10269890 meminfo.Inactive
28407610 -77.2% 6464247 meminfo.Inactive(file)
650015 +60.3% 1041779 meminfo.KReclaimable
650015 +60.3% 1041779 meminfo.SReclaimable
829409 +47.3% 1221631 meminfo.Slab
2564 ± 4% +42.5% 3654 ± 4% meminfo.Writeback
4673366 ± 9% +204.3% 14223125 numa-meminfo.node0.Active
4670584 ± 9% +204.4% 14218202 numa-meminfo.node0.Active(file)
3328794 ± 5% -10.7% 2972288 numa-meminfo.node0.Dirty
16103425 ± 3% -60.3% 6392546 ± 2% numa-meminfo.node0.Inactive
13636450 ± 5% -77.2% 3114750 ± 3% numa-meminfo.node0.Inactive(file)
361437 ± 2% +64.5% 594674 ± 2% numa-meminfo.node0.KReclaimable
361437 ± 2% +64.5% 594674 ± 2% numa-meminfo.node0.SReclaimable
454217 +51.0% 686089 ± 2% numa-meminfo.node0.Slab
5117666 ± 8% +236.1% 17201387 numa-meminfo.node1.Active
57417 ± 8% -39.6% 34652 ± 7% numa-meminfo.node1.Active(anon)
5060248 ± 8% +239.2% 17166734 numa-meminfo.node1.Active(file)
16086677 ± 3% -76.0% 3867738 ± 5% numa-meminfo.node1.Inactive
14767296 ± 5% -77.4% 3343591 ± 2% numa-meminfo.node1.Inactive(file)
288589 ± 3% +54.9% 446992 ± 3% numa-meminfo.node1.KReclaimable
288589 ± 3% +54.9% 446992 ± 3% numa-meminfo.node1.SReclaimable
375202 ± 2% +42.7% 535430 ± 2% numa-meminfo.node1.Slab
1310 ± 9% +57.4% 2062 ± 5% numa-meminfo.node1.Writeback
0.02 ± 38% +151.1% 0.04 ± 4% sched_debug.cfs_rq:/.nr_spread_over.avg
0.14 ± 30% +62.3% 0.22 ± 11% sched_debug.cfs_rq:/.nr_spread_over.stddev
702.00 ± 3% +20.3% 844.48 ± 11% sched_debug.cfs_rq:/.runnable_load_avg.max
8658 ± 41% -88.5% 994.62 ±366% sched_debug.cfs_rq:/.spread0.avg
-5693 +147.0% -14059 sched_debug.cfs_rq:/.spread0.min
280.56 ± 11% -20.9% 221.88 ± 5% sched_debug.cfs_rq:/.util_avg.avg
16889 ± 8% +31.2% 22166 ± 13% sched_debug.cpu.nr_switches.avg
0.21 ± 23% +44.0% 0.30 ± 7% sched_debug.cpu.nr_uninterruptible.avg
15025 ± 9% +35.5% 20353 ± 14% sched_debug.cpu.sched_count.avg
16991 ± 21% +31.0% 22263 ± 16% sched_debug.cpu.sched_count.stddev
6755 ± 11% +39.0% 9388 ± 14% sched_debug.cpu.sched_goidle.avg
478.75 ± 16% +37.8% 659.79 ± 19% sched_debug.cpu.sched_goidle.min
8464 ± 21% +31.2% 11108 ± 16% sched_debug.cpu.sched_goidle.stddev
7465 ± 9% +35.5% 10114 ± 14% sched_debug.cpu.ttwu_count.avg
3184 ± 15% +79.5% 5715 ± 14% sched_debug.cpu.ttwu_local.max
349.50 ± 15% +32.8% 464.02 ± 16% sched_debug.cpu.ttwu_local.min
516.10 ± 9% +58.8% 819.50 ± 16% sched_debug.cpu.ttwu_local.stddev
135179 +16.5% 157474 ± 8% sched_debug.ktime
20801 ± 9% +15.2% 23973 ± 3% softirqs.CPU13.SCHED
21834 ± 5% +11.4% 24317 ± 4% softirqs.CPU15.SCHED
20068 ± 13% +26.6% 25416 ± 2% softirqs.CPU18.SCHED
21227 ± 8% +18.3% 25107 ± 2% softirqs.CPU19.SCHED
21493 ± 6% +14.2% 24542 ± 4% softirqs.CPU21.SCHED
21659 ± 5% +17.5% 25448 ± 3% softirqs.CPU22.SCHED
21989 ± 4% +17.2% 25773 ± 2% softirqs.CPU23.SCHED
20132 ± 14% +23.2% 24812 ± 2% softirqs.CPU25.SCHED
20294 ± 16% +23.7% 25112 ± 5% softirqs.CPU26.SCHED
21996 ± 2% +14.2% 25115 ± 2% softirqs.CPU3.SCHED
21985 ± 8% +17.2% 25765 ± 3% softirqs.CPU31.SCHED
21974 ± 6% +14.6% 25184 softirqs.CPU37.SCHED
21943 +12.4% 24665 ± 3% softirqs.CPU40.SCHED
21177 ± 9% +17.8% 24954 ± 2% softirqs.CPU41.SCHED
21555 ± 4% +16.1% 25018 softirqs.CPU43.SCHED
21565 ± 2% +15.0% 24804 softirqs.CPU45.SCHED
22158 ± 6% +14.9% 25466 ± 3% softirqs.CPU50.SCHED
22571 ± 7% +13.4% 25597 softirqs.CPU51.SCHED
21477 ± 7% +17.8% 25310 softirqs.CPU53.SCHED
21648 ± 4% +15.7% 25037 ± 2% softirqs.CPU54.SCHED
22303 ± 4% +10.0% 24524 ± 5% softirqs.CPU57.SCHED
22250 ± 8% +13.7% 25302 ± 2% softirqs.CPU6.SCHED
21945 ± 6% +15.1% 25253 softirqs.CPU62.SCHED
20860 ± 9% +23.0% 25655 ± 3% softirqs.CPU64.SCHED
22491 ± 4% +7.5% 24180 ± 5% softirqs.CPU81.SCHED
19570 ± 21% +28.5% 25157 ± 2% softirqs.CPU83.SCHED
1167199 ± 9% +204.5% 3554443 numa-vmstat.node0.nr_active_file
833733 ± 5% -11.3% 739589 numa-vmstat.node0.nr_dirty
3409188 ± 5% -77.1% 779009 ± 3% numa-vmstat.node0.nr_inactive_file
83.50 ± 22% -60.8% 32.75 ± 16% numa-vmstat.node0.nr_isolated_file
90376 ± 2% +64.5% 148639 ± 2% numa-vmstat.node0.nr_slab_reclaimable
1394052 ± 7% +282.6% 5333857 ± 17% numa-vmstat.node0.nr_vmscan_immediate_reclaim
315.25 ± 3% +22.5% 386.25 ± 13% numa-vmstat.node0.nr_writeback
1167965 ± 9% +204.3% 3554646 numa-vmstat.node0.nr_zone_active_file
3409621 ± 5% -77.1% 780149 ± 3% numa-vmstat.node0.nr_zone_inactive_file
835951 ± 5% -11.3% 741848 numa-vmstat.node0.nr_zone_write_pending
62116218 ± 6% -31.7% 42426214 ± 23% numa-vmstat.node0.numa_hit
62089335 ± 6% -31.8% 42365222 ± 23% numa-vmstat.node0.numa_local
18221559 ± 10% -100.0% 0.00 numa-vmstat.node0.workingset_activate
13281 ± 7% -22.3% 10326 ± 8% numa-vmstat.node0.workingset_nodes
33316100 ± 6% -100.0% 0.00 numa-vmstat.node0.workingset_refault
10372861 ± 12% -100.0% 1.25 ± 66% numa-vmstat.node0.workingset_restore
14346 ± 8% -39.6% 8669 ± 7% numa-vmstat.node1.nr_active_anon
1263972 ± 8% +239.6% 4292221 numa-vmstat.node1.nr_active_file
61372170 ± 9% +31.3% 80554020 ± 8% numa-vmstat.node1.nr_dirtied
3692694 ± 5% -77.4% 835792 numa-vmstat.node1.nr_inactive_file
72150 ± 3% +54.8% 111693 ± 3% numa-vmstat.node1.nr_slab_reclaimable
1341628 ± 12% +750.8% 11415031 ± 6% numa-vmstat.node1.nr_vmscan_immediate_reclaim
331.25 ± 10% +66.0% 549.75 ± 9% numa-vmstat.node1.nr_writeback
60469655 ± 9% +31.7% 79613669 ± 8% numa-vmstat.node1.nr_written
14347 ± 8% -39.6% 8670 ± 7% numa-vmstat.node1.nr_zone_active_anon
1264780 ± 8% +239.4% 4292348 numa-vmstat.node1.nr_zone_active_file
3693150 ± 5% -77.3% 837009 numa-vmstat.node1.nr_zone_inactive_file
19519796 ± 8% -100.0% 0.00 numa-vmstat.node1.workingset_activate
12632 ± 5% -20.7% 10012 ± 4% numa-vmstat.node1.workingset_nodes
36195648 ± 4% -100.0% 0.00 numa-vmstat.node1.workingset_refault
11279550 ± 9% -100.0% 0.25 ±173% numa-vmstat.node1.workingset_restore
15.72 +11.1% 17.46 perf-stat.i.MPKI
7.221e+09 ± 3% -14.9% 6.144e+09 ± 2% perf-stat.i.branch-instructions
29447507 ± 3% -13.8% 25382627 ± 2% perf-stat.i.branch-misses
67.78 +2.0 69.77 perf-stat.i.cache-miss-rate%
1.55 ± 4% -14.7% 1.33 perf-stat.i.cpi
5.857e+10 ± 7% -27.4% 4.252e+10 perf-stat.i.cpu-cycles
148.26 ± 2% -12.2% 130.23 perf-stat.i.cpu-migrations
154.03 ± 3% -24.7% 115.95 perf-stat.i.cycles-between-cache-misses
0.20 ± 2% -0.0 0.15 ± 3% perf-stat.i.dTLB-load-miss-rate%
20033658 ± 4% -34.6% 13108913 perf-stat.i.dTLB-load-misses
9.948e+09 ± 3% -13.4% 8.617e+09 ± 3% perf-stat.i.dTLB-loads
0.37 ± 3% -0.1 0.28 ± 3% perf-stat.i.dTLB-store-miss-rate%
20362529 ± 4% -30.4% 14162666 ± 5% perf-stat.i.dTLB-store-misses
5.532e+09 ± 2% -10.6% 4.945e+09 ± 2% perf-stat.i.dTLB-stores
73.19 -1.4 71.76 perf-stat.i.iTLB-load-miss-rate%
12785996 ± 2% -10.5% 11440556 ± 3% perf-stat.i.iTLB-load-misses
4812663 ± 2% -6.0% 4526122 perf-stat.i.iTLB-loads
3.712e+10 ± 3% -14.2% 3.185e+10 ± 2% perf-stat.i.instructions
2895 -4.1% 2777 perf-stat.i.instructions-per-iTLB-miss
0.66 ± 4% +16.5% 0.77 perf-stat.i.ipc
1203000 ± 2% -22.0% 938451 ± 2% perf-stat.i.major-faults
632441 ± 5% +66.7% 1054315 ± 2% perf-stat.i.minor-faults
1835441 ± 3% +8.6% 1992767 ± 2% perf-stat.i.page-faults
15.80 +10.3% 17.43 perf-stat.overall.MPKI
67.75 +2.1 69.87 perf-stat.overall.cache-miss-rate%
1.58 ± 4% -15.3% 1.34 perf-stat.overall.cpi
147.21 ± 3% -25.5% 109.71 perf-stat.overall.cycles-between-cache-misses
0.20 ± 2% -0.0 0.15 ± 3% perf-stat.overall.dTLB-load-miss-rate%
0.37 ± 3% -0.1 0.29 ± 3% perf-stat.overall.dTLB-store-miss-rate%
72.65 -1.0 71.64 perf-stat.overall.iTLB-load-miss-rate%
2902 -4.1% 2784 perf-stat.overall.instructions-per-iTLB-miss
0.64 ± 4% +17.8% 0.75 perf-stat.overall.ipc
8598809 -28.3% 6163102 perf-stat.overall.path-length
7.185e+09 ± 3% -14.9% 6.114e+09 ± 2% perf-stat.ps.branch-instructions
29306041 ± 3% -13.8% 25261462 ± 2% perf-stat.ps.branch-misses
5.829e+10 ± 7% -27.4% 4.232e+10 perf-stat.ps.cpu-cycles
147.55 ± 2% -12.2% 129.61 perf-stat.ps.cpu-migrations
19935636 ± 4% -34.6% 13047562 perf-stat.ps.dTLB-load-misses
9.899e+09 ± 3% -13.4% 8.575e+09 ± 2% perf-stat.ps.dTLB-loads
20263384 ± 4% -30.4% 14096250 ± 5% perf-stat.ps.dTLB-store-misses
5.505e+09 ± 2% -10.6% 4.92e+09 ± 2% perf-stat.ps.dTLB-stores
12723152 ± 2% -10.5% 11384126 ± 3% perf-stat.ps.iTLB-load-misses
4789226 ± 2% -6.0% 4503341 perf-stat.ps.iTLB-loads
3.694e+10 ± 3% -14.2% 3.169e+10 ± 2% perf-stat.ps.instructions
1197099 ± 2% -22.0% 933922 ± 2% perf-stat.ps.major-faults
629469 ± 5% +66.7% 1049045 ± 2% perf-stat.ps.minor-faults
1826569 ± 3% +8.6% 1982969 ± 2% perf-stat.ps.page-faults
7.476e+12 ± 3% -14.4% 6.403e+12 ± 2% perf-stat.total.instructions
104876 ± 15% -55.2% 46978 ± 8% proc-vmstat.allocstall_movable
6670 ± 13% +90.1% 12681 ± 23% proc-vmstat.allocstall_normal
270.00 ± 34% -62.4% 101.50 ± 33% proc-vmstat.compact_fail
277.50 ± 33% -61.0% 108.25 ± 36% proc-vmstat.compact_stall
629.25 ± 35% +596.1% 4380 ± 8% proc-vmstat.kswapd_low_wmark_hit_quickly
15034 ± 8% -34.2% 9894 ± 3% proc-vmstat.nr_active_anon
2432763 +222.5% 7846521 proc-vmstat.nr_active_file
2.229e+08 ± 2% +19.5% 2.664e+08 ± 2% proc-vmstat.nr_dirtied
1732719 -3.1% 1679210 proc-vmstat.nr_dirty
280680 -4.7% 267623 proc-vmstat.nr_free_pages
7100824 -77.3% 1613637 proc-vmstat.nr_inactive_file
164.75 ± 9% -54.9% 74.25 ± 8% proc-vmstat.nr_isolated_file
9582341 -2.0% 9389351 proc-vmstat.nr_mapped
121263 -4.0% 116373 proc-vmstat.nr_shmem
162508 +60.3% 260428 proc-vmstat.nr_slab_reclaimable
5464481 ± 4% +578.9% 37099437 ± 2% proc-vmstat.nr_vmscan_immediate_reclaim
639.25 ± 3% +43.5% 917.50 ± 4% proc-vmstat.nr_writeback
2.217e+08 ± 2% +19.4% 2.646e+08 ± 2% proc-vmstat.nr_written
15036 ± 8% -34.2% 9895 ± 3% proc-vmstat.nr_zone_active_anon
2434338 +222.3% 7846813 proc-vmstat.nr_zone_active_file
7101719 -77.2% 1615989 proc-vmstat.nr_zone_inactive_file
1737241 -3.1% 1683939 proc-vmstat.nr_zone_write_pending
40016131 ± 6% -16.6% 33361886 ± 6% proc-vmstat.numa_foreign
25745506 ± 14% +56.4% 40263559 ± 5% proc-vmstat.numa_hint_faults
19974893 ± 14% +49.8% 29926746 ± 8% proc-vmstat.numa_hint_faults_local
2.039e+08 ± 2% -22.5% 1.581e+08 ± 2% proc-vmstat.numa_hit
1808 ± 2% -3.3% 1749 proc-vmstat.numa_huge_pte_updates
2.039e+08 ± 2% -22.5% 1.58e+08 ± 2% proc-vmstat.numa_local
40016131 ± 6% -16.6% 33361886 ± 6% proc-vmstat.numa_miss
40047410 ± 6% -16.6% 33393063 ± 6% proc-vmstat.numa_other
251563 ± 29% +420.2% 1308599 ± 10% proc-vmstat.numa_pages_migrated
635.00 ± 35% +596.9% 4425 ± 8% proc-vmstat.pageoutrun
1.127e+08 ± 2% -55.0% 50732767 proc-vmstat.pgactivate
9619324 ± 4% -37.5% 6013199 ± 7% proc-vmstat.pgalloc_dma32
2.354e+08 ± 2% -20.7% 1.868e+08 ± 2% proc-vmstat.pgalloc_normal
1.829e+08 ± 2% -97.1% 5298999 ± 4% proc-vmstat.pgdeactivate
7.145e+08 ± 3% +6.8% 7.627e+08 ± 2% proc-vmstat.pgfault
2.353e+08 ± 2% -22.4% 1.827e+08 ± 3% proc-vmstat.pgfree
2.427e+08 ± 2% -22.0% 1.892e+08 ± 2% proc-vmstat.pgmajfault
427089 ± 38% +278.2% 1615323 ± 11% proc-vmstat.pgmigrate_success
9.71e+08 ± 2% -22.0% 7.569e+08 ± 2% proc-vmstat.pgpgin
8.867e+08 ± 2% +19.4% 1.059e+09 ± 2% proc-vmstat.pgpgout
1.829e+08 ± 2% -97.1% 5298820 ± 4% proc-vmstat.pgrefill
5485949 ± 3% +574.1% 36983260 ± 2% proc-vmstat.pgrotated
1.984e+08 ± 12% -78.9% 41928600 ± 14% proc-vmstat.pgscan_direct
3.395e+08 ± 3% +11.0% 3.768e+08 proc-vmstat.pgscan_kswapd
89080449 ± 12% -87.6% 11029910 ± 8% proc-vmstat.pgsteal_direct
1.439e+08 ± 3% +17.0% 1.684e+08 ± 2% proc-vmstat.pgsteal_kswapd
37738291 ± 5% -100.0% 0.00 proc-vmstat.workingset_activate
25988 ± 2% -22.6% 20107 ± 2% proc-vmstat.workingset_nodes
69506838 ± 2% -100.0% 0.00 proc-vmstat.workingset_refault
21650634 ± 5% -100.0% 1.50 ± 33% proc-vmstat.workingset_restore
87.50 ±168% -100.0% 0.00 interrupts.114:PCI-MSI.31981647-edge.i40e-eth0-TxRx-78
2.75 ±110% +5090.9% 142.75 ±103% interrupts.61:PCI-MSI.31981594-edge.i40e-eth0-TxRx-25
50.25 ±153% -98.5% 0.75 ±110% interrupts.83:PCI-MSI.31981616-edge.i40e-eth0-TxRx-47
0.25 ±173% +28300.0% 71.00 ±159% interrupts.86:PCI-MSI.31981619-edge.i40e-eth0-TxRx-50
43979343 ± 10% -35.3% 28436814 ± 6% interrupts.CAL:Function_call_interrupts
309869 ± 12% -48.7% 158885 ± 43% interrupts.CPU0.CAL:Function_call_interrupts
1496 ± 20% -59.7% 603.25 ± 30% interrupts.CPU0.NMI:Non-maskable_interrupts
1496 ± 20% -59.7% 603.25 ± 30% interrupts.CPU0.PMI:Performance_monitoring_interrupts
711.25 ± 39% +120.0% 1564 ± 46% interrupts.CPU0.RES:Rescheduling_interrupts
378389 ± 11% -40.7% 224400 ± 36% interrupts.CPU0.TLB:TLB_shootdowns
564422 ± 6% -51.1% 276198 ± 18% interrupts.CPU11.CAL:Function_call_interrupts
675725 ± 13% -47.0% 358389 ± 15% interrupts.CPU11.TLB:TLB_shootdowns
1834 ± 31% -58.9% 754.25 ± 16% interrupts.CPU12.NMI:Non-maskable_interrupts
1834 ± 31% -58.9% 754.25 ± 16% interrupts.CPU12.PMI:Performance_monitoring_interrupts
483343 ± 18% -43.3% 273844 ± 35% interrupts.CPU13.CAL:Function_call_interrupts
4095 ± 34% -72.2% 1137 ± 55% interrupts.CPU13.NMI:Non-maskable_interrupts
4095 ± 34% -72.2% 1137 ± 55% interrupts.CPU13.PMI:Performance_monitoring_interrupts
662959 ± 25% -52.6% 313982 ± 30% interrupts.CPU13.TLB:TLB_shootdowns
560563 ± 24% -49.1% 285449 ± 9% interrupts.CPU14.CAL:Function_call_interrupts
693290 ± 34% -47.1% 366618 ± 5% interrupts.CPU14.TLB:TLB_shootdowns
558990 ± 21% -61.4% 215573 ± 24% interrupts.CPU15.CAL:Function_call_interrupts
2204 ± 31% -67.0% 727.50 ± 35% interrupts.CPU15.NMI:Non-maskable_interrupts
2204 ± 31% -67.0% 727.50 ± 35% interrupts.CPU15.PMI:Performance_monitoring_interrupts
655709 ± 30% -64.3% 233774 ± 18% interrupts.CPU15.TLB:TLB_shootdowns
531963 ± 30% -51.7% 257180 ± 61% interrupts.CPU16.CAL:Function_call_interrupts
3104 ± 49% -66.2% 1050 ± 71% interrupts.CPU16.NMI:Non-maskable_interrupts
3104 ± 49% -66.2% 1050 ± 71% interrupts.CPU16.PMI:Performance_monitoring_interrupts
758.50 ± 52% +58.1% 1199 ± 18% interrupts.CPU16.RES:Rescheduling_interrupts
624333 ± 30% -53.8% 288559 ± 54% interrupts.CPU16.TLB:TLB_shootdowns
414942 ± 20% -51.1% 203078 ± 32% interrupts.CPU18.CAL:Function_call_interrupts
480458 ± 21% -52.4% 228645 ± 25% interrupts.CPU18.TLB:TLB_shootdowns
1109 ± 44% +205.5% 3390 ± 41% interrupts.CPU2.RES:Rescheduling_interrupts
461027 ± 25% -57.0% 198359 ± 30% interrupts.CPU20.CAL:Function_call_interrupts
3798 ± 63% -81.0% 720.75 ± 86% interrupts.CPU20.NMI:Non-maskable_interrupts
3798 ± 63% -81.0% 720.75 ± 86% interrupts.CPU20.PMI:Performance_monitoring_interrupts
564227 ± 37% -62.5% 211819 ± 31% interrupts.CPU20.TLB:TLB_shootdowns
513743 ± 29% -48.3% 265392 ± 32% interrupts.CPU21.CAL:Function_call_interrupts
644566 ± 43% -57.1% 276789 ± 32% interrupts.CPU21.TLB:TLB_shootdowns
503293 ± 25% -63.1% 185627 ± 52% interrupts.CPU22.CAL:Function_call_interrupts
645000 ± 13% -67.5% 209945 ± 56% interrupts.CPU22.TLB:TLB_shootdowns
553939 ± 27% -61.8% 211825 ± 28% interrupts.CPU23.CAL:Function_call_interrupts
687580 ± 25% -67.1% 225909 ± 29% interrupts.CPU23.TLB:TLB_shootdowns
474.00 ± 52% +258.5% 1699 ± 57% interrupts.CPU24.RES:Rescheduling_interrupts
2.25 ±123% +6233.3% 142.50 ±103% interrupts.CPU25.61:PCI-MSI.31981594-edge.i40e-eth0-TxRx-25
4043 ± 63% -71.3% 1160 ± 31% interrupts.CPU25.NMI:Non-maskable_interrupts
4043 ± 63% -71.3% 1160 ± 31% interrupts.CPU25.PMI:Performance_monitoring_interrupts
451692 ± 11% -39.6% 272701 ± 25% interrupts.CPU26.CAL:Function_call_interrupts
815.25 ± 39% +110.4% 1715 ± 29% interrupts.CPU26.RES:Rescheduling_interrupts
584283 ± 11% -41.2% 343585 ± 33% interrupts.CPU26.TLB:TLB_shootdowns
517137 ± 11% -41.0% 305137 ± 22% interrupts.CPU27.CAL:Function_call_interrupts
2414 ± 26% -45.2% 1323 ± 26% interrupts.CPU27.NMI:Non-maskable_interrupts
2414 ± 26% -45.2% 1323 ± 26% interrupts.CPU27.PMI:Performance_monitoring_interrupts
614104 ± 7% -39.7% 370314 ± 19% interrupts.CPU27.TLB:TLB_shootdowns
441205 ± 12% -32.1% 299453 ± 20% interrupts.CPU28.CAL:Function_call_interrupts
2273 ± 24% -39.7% 1369 ± 19% interrupts.CPU28.NMI:Non-maskable_interrupts
2273 ± 24% -39.7% 1369 ± 19% interrupts.CPU28.PMI:Performance_monitoring_interrupts
544289 ± 4% -31.6% 372065 ± 21% interrupts.CPU28.TLB:TLB_shootdowns
520217 ± 28% -31.2% 357653 ± 14% interrupts.CPU29.CAL:Function_call_interrupts
3744 ± 63% -55.3% 1674 ± 27% interrupts.CPU29.NMI:Non-maskable_interrupts
3744 ± 63% -55.3% 1674 ± 27% interrupts.CPU29.PMI:Performance_monitoring_interrupts
814.25 ± 53% +128.8% 1863 ± 39% interrupts.CPU29.RES:Rescheduling_interrupts
613340 ± 29% -31.8% 418352 ± 11% interrupts.CPU29.TLB:TLB_shootdowns
516303 ± 28% -48.8% 264227 ± 29% interrupts.CPU3.CAL:Function_call_interrupts
1951 ± 32% -71.4% 558.25 ± 28% interrupts.CPU3.NMI:Non-maskable_interrupts
1951 ± 32% -71.4% 558.25 ± 28% interrupts.CPU3.PMI:Performance_monitoring_interrupts
649962 ± 17% -54.0% 298822 ± 40% interrupts.CPU3.TLB:TLB_shootdowns
518481 ± 11% -22.2% 403431 ± 14% interrupts.CPU30.CAL:Function_call_interrupts
678.50 ± 47% +154.1% 1724 ± 16% interrupts.CPU30.RES:Rescheduling_interrupts
740472 ± 16% -40.8% 438014 ± 16% interrupts.CPU30.TLB:TLB_shootdowns
497599 ± 13% -26.9% 363979 ± 2% interrupts.CPU31.CAL:Function_call_interrupts
560143 ± 11% -21.8% 438031 ± 10% interrupts.CPU31.TLB:TLB_shootdowns
919.75 ± 37% +68.1% 1546 ± 16% interrupts.CPU32.RES:Rescheduling_interrupts
526266 ± 22% -40.5% 312935 ± 27% interrupts.CPU33.CAL:Function_call_interrupts
2250 ± 15% -38.3% 1387 ± 3% interrupts.CPU33.NMI:Non-maskable_interrupts
2250 ± 15% -38.3% 1387 ± 3% interrupts.CPU33.PMI:Performance_monitoring_interrupts
959.25 ± 31% +197.4% 2852 ± 59% interrupts.CPU33.RES:Rescheduling_interrupts
589843 ± 17% -40.4% 351483 ± 9% interrupts.CPU34.CAL:Function_call_interrupts
724580 ± 19% -43.0% 412731 ± 9% interrupts.CPU34.TLB:TLB_shootdowns
3061 ± 51% -66.5% 1026 ± 54% interrupts.CPU35.NMI:Non-maskable_interrupts
3061 ± 51% -66.5% 1026 ± 54% interrupts.CPU35.PMI:Performance_monitoring_interrupts
527518 ± 26% -41.3% 309832 ± 20% interrupts.CPU36.CAL:Function_call_interrupts
1935 ± 29% -41.1% 1139 ± 25% interrupts.CPU36.NMI:Non-maskable_interrupts
1935 ± 29% -41.1% 1139 ± 25% interrupts.CPU36.PMI:Performance_monitoring_interrupts
605919 ± 27% -33.5% 402973 ± 36% interrupts.CPU36.TLB:TLB_shootdowns
1968 ± 19% -43.1% 1120 ± 33% interrupts.CPU37.NMI:Non-maskable_interrupts
1968 ± 19% -43.1% 1120 ± 33% interrupts.CPU37.PMI:Performance_monitoring_interrupts
552091 ± 17% -37.9% 343088 ± 20% interrupts.CPU38.CAL:Function_call_interrupts
741448 ± 25% -42.9% 423500 ± 21% interrupts.CPU38.TLB:TLB_shootdowns
893.75 ± 22% +66.8% 1490 ± 34% interrupts.CPU39.RES:Rescheduling_interrupts
651520 ± 10% -32.9% 437091 ± 33% interrupts.CPU39.TLB:TLB_shootdowns
443156 ± 27% -74.1% 114892 ± 21% interrupts.CPU4.CAL:Function_call_interrupts
536804 ± 32% -73.9% 139975 ± 23% interrupts.CPU4.TLB:TLB_shootdowns
2376 ± 9% -31.2% 1635 ± 25% interrupts.CPU40.NMI:Non-maskable_interrupts
2376 ± 9% -31.2% 1635 ± 25% interrupts.CPU40.PMI:Performance_monitoring_interrupts
634311 ± 13% -30.5% 441067 ± 19% interrupts.CPU42.TLB:TLB_shootdowns
605811 ± 24% -44.1% 338572 ± 5% interrupts.CPU45.CAL:Function_call_interrupts
824.00 ± 43% +169.8% 2223 ± 39% interrupts.CPU45.RES:Rescheduling_interrupts
789721 ± 28% -45.9% 427130 ± 12% interrupts.CPU45.TLB:TLB_shootdowns
527252 ± 26% -41.4% 309202 ± 18% interrupts.CPU46.CAL:Function_call_interrupts
747063 ± 34% -55.4% 333517 ± 18% interrupts.CPU46.TLB:TLB_shootdowns
50.00 ±154% -99.5% 0.25 ±173% interrupts.CPU47.83:PCI-MSI.31981616-edge.i40e-eth0-TxRx-47
552975 ± 11% -35.1% 358953 ± 23% interrupts.CPU47.CAL:Function_call_interrupts
698382 ± 5% -34.2% 459438 ± 31% interrupts.CPU47.TLB:TLB_shootdowns
668725 ± 25% -71.0% 193871 ± 59% interrupts.CPU48.CAL:Function_call_interrupts
2351 ± 23% -63.0% 869.00 ± 74% interrupts.CPU48.NMI:Non-maskable_interrupts
2351 ± 23% -63.0% 869.00 ± 74% interrupts.CPU48.PMI:Performance_monitoring_interrupts
1036 ± 39% +143.7% 2524 ± 59% interrupts.CPU48.RES:Rescheduling_interrupts
723105 ± 28% -72.2% 200997 ± 57% interrupts.CPU48.TLB:TLB_shootdowns
681246 ± 37% -69.7% 206357 ± 66% interrupts.CPU49.CAL:Function_call_interrupts
734750 ± 36% -70.8% 214575 ± 68% interrupts.CPU49.TLB:TLB_shootdowns
532908 ± 33% -54.9% 240375 ± 53% interrupts.CPU50.CAL:Function_call_interrupts
622804 ± 27% -59.6% 251480 ± 56% interrupts.CPU50.TLB:TLB_shootdowns
2612 ± 56% -64.4% 929.00 ± 37% interrupts.CPU54.RES:Rescheduling_interrupts
1525 ± 19% -45.9% 825.25 ± 38% interrupts.CPU6.NMI:Non-maskable_interrupts
1525 ± 19% -45.9% 825.25 ± 38% interrupts.CPU6.PMI:Performance_monitoring_interrupts
658.25 ± 61% +183.8% 1868 ± 51% interrupts.CPU60.RES:Rescheduling_interrupts
1826 ± 29% -66.0% 620.75 ± 53% interrupts.CPU61.NMI:Non-maskable_interrupts
1826 ± 29% -66.0% 620.75 ± 53% interrupts.CPU61.PMI:Performance_monitoring_interrupts
2182 ± 35% -53.4% 1017 ± 20% interrupts.CPU62.NMI:Non-maskable_interrupts
2182 ± 35% -53.4% 1017 ± 20% interrupts.CPU62.PMI:Performance_monitoring_interrupts
479189 ± 47% -50.5% 237071 ± 37% interrupts.CPU64.TLB:TLB_shootdowns
417282 ± 22% -36.7% 263997 ± 22% interrupts.CPU7.CAL:Function_call_interrupts
826.50 ± 48% +73.7% 1435 ± 27% interrupts.CPU7.RES:Rescheduling_interrupts
533501 ± 13% -45.4% 291545 ± 28% interrupts.CPU7.TLB:TLB_shootdowns
618059 ± 18% -53.0% 290711 ± 27% interrupts.CPU72.CAL:Function_call_interrupts
680178 ± 18% -53.6% 315576 ± 36% interrupts.CPU72.TLB:TLB_shootdowns
2597 ± 16% -43.7% 1463 ± 24% interrupts.CPU73.NMI:Non-maskable_interrupts
2597 ± 16% -43.7% 1463 ± 24% interrupts.CPU73.PMI:Performance_monitoring_interrupts
459.50 ± 41% +431.0% 2440 ± 26% interrupts.CPU73.RES:Rescheduling_interrupts
521.75 ± 55% +281.9% 1992 ± 21% interrupts.CPU74.RES:Rescheduling_interrupts
2656 ± 21% -45.7% 1441 ± 33% interrupts.CPU75.NMI:Non-maskable_interrupts
2656 ± 21% -45.7% 1441 ± 33% interrupts.CPU75.PMI:Performance_monitoring_interrupts
524.75 ± 54% +141.9% 1269 ± 32% interrupts.CPU75.RES:Rescheduling_interrupts
87.00 ±169% -100.0% 0.00 interrupts.CPU78.114:PCI-MSI.31981647-edge.i40e-eth0-TxRx-78
650.00 ± 35% +102.7% 1317 ± 31% interrupts.CPU78.RES:Rescheduling_interrupts
903.25 ± 25% +76.1% 1590 ± 14% interrupts.CPU84.NMI:Non-maskable_interrupts
903.25 ± 25% +76.1% 1590 ± 14% interrupts.CPU84.PMI:Performance_monitoring_interrupts
421.00 ± 38% +147.6% 1042 ± 39% interrupts.CPU85.RES:Rescheduling_interrupts
807.25 ± 29% +56.5% 1263 ± 24% interrupts.CPU89.RES:Rescheduling_interrupts
521370 ± 3% -46.6% 278562 ± 38% interrupts.CPU9.CAL:Function_call_interrupts
613194 ± 20% -46.4% 328688 ± 32% interrupts.CPU9.TLB:TLB_shootdowns
440.00 ± 33% +163.4% 1158 ± 26% interrupts.CPU93.RES:Rescheduling_interrupts
208.75 ± 9% -40.4% 124.50 ± 3% interrupts.IWI:IRQ_work_interrupts
204364 ± 7% -31.0% 140939 ± 2% interrupts.NMI:Non-maskable_interrupts
204364 ± 7% -31.0% 140939 ± 2% interrupts.PMI:Performance_monitoring_interrupts
111435 ± 10% +35.2% 150649 ± 6% interrupts.RES:Rescheduling_interrupts
52728394 ± 10% -38.7% 32297958 ± 4% interrupts.TLB:TLB_shootdowns
22.74 ± 16% -18.0 4.75 ± 6% perf-profile.calltrace.cycles-pp.pagecache_get_page.filemap_fault.ext4_filemap_fault.__do_fault.handle_pte_fault
29.75 ± 13% -16.9 12.83 ± 2% perf-profile.calltrace.cycles-pp.filemap_fault.ext4_filemap_fault.__do_fault.handle_pte_fault.__handle_mm_fault
30.06 ± 13% -16.9 13.17 ± 2% perf-profile.calltrace.cycles-pp.__do_fault.handle_pte_fault.__handle_mm_fault.handle_mm_fault.do_page_fault
30.04 ± 13% -16.9 13.15 ± 2% perf-profile.calltrace.cycles-pp.ext4_filemap_fault.__do_fault.handle_pte_fault.__handle_mm_fault.handle_mm_fault
17.51 ± 18% -16.4 1.14 ± 28% perf-profile.calltrace.cycles-pp.__alloc_pages_slowpath.__alloc_pages_nodemask.pagecache_get_page.filemap_fault.ext4_filemap_fault
18.25 ± 17% -16.1 2.16 ± 12% perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.pagecache_get_page.filemap_fault.ext4_filemap_fault.__do_fault
15.88 ± 19% -15.2 0.66 ± 61% perf-profile.calltrace.cycles-pp.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.pagecache_get_page.filemap_fault
15.86 ± 19% -15.2 0.66 ± 61% perf-profile.calltrace.cycles-pp.shrink_node.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask
15.86 ± 19% -15.2 0.66 ± 61% perf-profile.calltrace.cycles-pp.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.pagecache_get_page
15.78 ± 19% -15.1 0.64 ± 61% perf-profile.calltrace.cycles-pp.shrink_lruvec.shrink_node.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath
37.45 ± 11% -14.3 23.12 ± 2% perf-profile.calltrace.cycles-pp.handle_pte_fault.__handle_mm_fault.handle_mm_fault.do_page_fault.page_fault
38.00 ± 11% -14.0 24.01 ± 2% perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.do_page_fault.page_fault
38.59 ± 11% -13.7 24.93 ± 2% perf-profile.calltrace.cycles-pp.handle_mm_fault.do_page_fault.page_fault
39.53 ± 11% -13.0 26.49 ± 2% perf-profile.calltrace.cycles-pp.do_page_fault.page_fault
39.80 ± 11% -12.8 26.95 ± 2% perf-profile.calltrace.cycles-pp.page_fault
12.67 ± 19% -12.0 0.63 ± 61% perf-profile.calltrace.cycles-pp.shrink_inactive_list.shrink_lruvec.shrink_node.do_try_to_free_pages.try_to_free_pages
6.30 ± 14% -6.2 0.13 ±173% perf-profile.calltrace.cycles-pp.shrink_page_list.shrink_inactive_list.shrink_lruvec.shrink_node.do_try_to_free_pages
6.03 ± 27% -6.0 0.00 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irq.shrink_inactive_list.shrink_lruvec.shrink_node
5.53 ± 25% -5.5 0.00 perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.shrink_inactive_list.shrink_lruvec.shrink_node.do_try_to_free_pages
4.11 ± 12% -2.1 2.05 ± 3% perf-profile.calltrace.cycles-pp.add_to_page_cache_lru.pagecache_get_page.filemap_fault.ext4_filemap_fault.__do_fault
2.75 ± 28% -2.0 0.72 ± 12% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.pagevec_lru_move_fn.deactivate_file_page.invalidate_mapping_pages
2.85 ± 27% -2.0 0.83 ± 10% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.pagevec_lru_move_fn.deactivate_file_page.invalidate_mapping_pages.generic_fadvise
3.32 ± 9% -2.0 1.35 ± 14% perf-profile.calltrace.cycles-pp.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_lruvec.shrink_node
3.19 ± 23% -1.8 1.35 ± 7% perf-profile.calltrace.cycles-pp.pagevec_lru_move_fn.deactivate_file_page.invalidate_mapping_pages.generic_fadvise.ksys_fadvise64_64
3.36 ± 22% -1.7 1.70 ± 5% perf-profile.calltrace.cycles-pp.deactivate_file_page.invalidate_mapping_pages.generic_fadvise.ksys_fadvise64_64.__x64_sys_fadvise64
2.33 ± 17% -1.3 0.98 ± 2% perf-profile.calltrace.cycles-pp.pagevec_lru_move_fn.__lru_cache_add.add_to_page_cache_lru.pagecache_get_page.filemap_fault
2.39 ± 16% -1.3 1.07 ± 3% perf-profile.calltrace.cycles-pp.__lru_cache_add.add_to_page_cache_lru.pagecache_get_page.filemap_fault.ext4_filemap_fault
2.16 ± 8% -0.7 1.50 ± 11% perf-profile.calltrace.cycles-pp.page_referenced.shrink_page_list.shrink_inactive_list.shrink_lruvec.shrink_node
1.90 ± 9% -0.6 1.29 ± 12% perf-profile.calltrace.cycles-pp.rmap_walk_file.page_referenced.shrink_page_list.shrink_inactive_list.shrink_lruvec
1.69 ± 8% -0.4 1.33 ± 4% perf-profile.calltrace.cycles-pp.on_each_cpu_cond_mask.flush_tlb_mm_range.ptep_clear_flush.page_mkclean_one.rmap_walk_file
1.75 ± 7% -0.3 1.44 ± 3% perf-profile.calltrace.cycles-pp.flush_tlb_mm_range.ptep_clear_flush.page_mkclean_one.rmap_walk_file.page_mkclean
1.49 ± 8% -0.3 1.19 ± 6% perf-profile.calltrace.cycles-pp.smp_call_function_single.on_each_cpu_cond_mask.flush_tlb_mm_range.ptep_clear_flush.page_mkclean_one
1.82 ± 7% -0.3 1.54 ± 4% perf-profile.calltrace.cycles-pp.ptep_clear_flush.page_mkclean_one.rmap_walk_file.page_mkclean.clear_page_dirty_for_io
0.61 ± 8% +0.3 0.89 ± 8% perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.pagecache_get_page.filemap_fault.ext4_filemap_fault
2.60 ± 8% +0.4 3.01 ± 3% perf-profile.calltrace.cycles-pp.filemap_map_pages.handle_pte_fault.__handle_mm_fault.handle_mm_fault.do_page_fault
2.08 ± 6% +0.4 2.51 ± 4% perf-profile.calltrace.cycles-pp.page_mkclean_one.rmap_walk_file.page_mkclean.clear_page_dirty_for_io.mpage_submit_page
0.73 ± 11% +0.4 1.16 ± 16% perf-profile.calltrace.cycles-pp.isolate_lru_pages.shrink_inactive_list.shrink_lruvec.shrink_node.balance_pgdat
0.93 ± 9% +0.5 1.38 ± 4% perf-profile.calltrace.cycles-pp.pagevec_lookup_entries.invalidate_mapping_pages.generic_fadvise.ksys_fadvise64_64.__x64_sys_fadvise64
0.92 ± 9% +0.5 1.38 ± 3% perf-profile.calltrace.cycles-pp.find_get_entries.pagevec_lookup_entries.invalidate_mapping_pages.generic_fadvise.ksys_fadvise64_64
0.77 ± 7% +0.5 1.24 ± 5% perf-profile.calltrace.cycles-pp.get_io_u
0.00 +0.6 0.61 ± 5% perf-profile.calltrace.cycles-pp.fault_dirty_shared_page.do_wp_page.handle_pte_fault.__handle_mm_fault.handle_mm_fault
2.46 ± 10% +0.6 3.08 ± 4% perf-profile.calltrace.cycles-pp.page_mkclean.clear_page_dirty_for_io.mpage_submit_page.mpage_process_page_bufs.mpage_prepare_extent_to_map
2.23 ± 6% +0.6 2.85 ± 4% perf-profile.calltrace.cycles-pp.rmap_walk_file.page_mkclean.clear_page_dirty_for_io.mpage_submit_page.mpage_process_page_bufs
0.00 +0.6 0.63 ± 9% perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
1.22 ± 10% +0.7 1.87 ± 4% perf-profile.calltrace.cycles-pp.block_page_mkwrite.ext4_page_mkwrite.do_page_mkwrite.do_wp_page.handle_pte_fault
4.32 ± 6% +0.7 4.98 ± 4% perf-profile.calltrace.cycles-pp.__memcpy_mcsafe.pmem_do_bvec.pmem_make_request.generic_make_request.submit_bio
4.41 ± 6% +0.7 5.08 ± 4% perf-profile.calltrace.cycles-pp.pmem_do_bvec.pmem_make_request.generic_make_request.submit_bio.ext4_mpage_readpages
3.50 ± 5% +0.7 4.20 ± 3% perf-profile.calltrace.cycles-pp.clear_page_dirty_for_io.mpage_submit_page.mpage_process_page_bufs.mpage_prepare_extent_to_map.ext4_writepages
0.00 +0.8 0.77 ± 8% perf-profile.calltrace.cycles-pp.__set_page_dirty.mark_buffer_dirty.__block_commit_write.block_page_mkwrite.ext4_page_mkwrite
5.00 ± 6% +0.8 5.78 ± 4% perf-profile.calltrace.cycles-pp.pmem_make_request.generic_make_request.submit_bio.ext4_mpage_readpages.filemap_fault
0.39 ± 57% +0.8 1.20 ± 5% perf-profile.calltrace.cycles-pp.__block_commit_write.block_page_mkwrite.ext4_page_mkwrite.do_page_mkwrite.do_wp_page
0.00 +0.9 0.90 ± 4% perf-profile.calltrace.cycles-pp.__test_set_page_writeback.ext4_bio_write_page.mpage_submit_page.mpage_process_page_bufs.mpage_prepare_extent_to_map
5.42 ± 6% +0.9 6.33 ± 4% perf-profile.calltrace.cycles-pp.generic_make_request.submit_bio.ext4_mpage_readpages.filemap_fault.ext4_filemap_fault
5.46 ± 6% +0.9 6.38 ± 4% perf-profile.calltrace.cycles-pp.submit_bio.ext4_mpage_readpages.filemap_fault.ext4_filemap_fault.__do_fault
0.00 +0.9 0.94 ± 23% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
0.75 ± 9% +1.0 1.78 ± 19% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle
1.42 ± 10% +1.0 2.46 ± 3% perf-profile.calltrace.cycles-pp.ext4_page_mkwrite.do_page_mkwrite.do_wp_page.handle_pte_fault.__handle_mm_fault
1.44 ± 10% +1.1 2.50 ± 3% perf-profile.calltrace.cycles-pp.do_page_mkwrite.do_wp_page.handle_pte_fault.__handle_mm_fault.handle_mm_fault
6.32 ± 5% +1.1 7.40 ± 4% perf-profile.calltrace.cycles-pp.ext4_mpage_readpages.filemap_fault.ext4_filemap_fault.__do_fault.handle_pte_fault
0.42 ± 57% +1.1 1.53 ± 4% perf-profile.calltrace.cycles-pp.mark_buffer_dirty.__block_commit_write.block_page_mkwrite.ext4_page_mkwrite.do_page_mkwrite
0.83 ± 8% +1.1 1.95 ± 19% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
0.65 ± 4% +1.2 1.87 ± 11% perf-profile.calltrace.cycles-pp.test_clear_page_writeback.end_page_writeback.ext4_finish_bio.ext4_end_bio.pmem_make_request
0.71 ± 5% +1.7 2.40 ± 3% perf-profile.calltrace.cycles-pp.end_page_writeback.ext4_finish_bio.ext4_end_bio.pmem_make_request.generic_make_request
1.92 ± 5% +1.7 3.61 perf-profile.calltrace.cycles-pp.ext4_finish_bio.ext4_end_bio.pmem_make_request.generic_make_request.submit_bio
1.93 ± 5% +1.7 3.63 perf-profile.calltrace.cycles-pp.ext4_end_bio.pmem_make_request.generic_make_request.submit_bio.ext4_io_submit
1.93 ± 8% +1.8 3.68 ± 3% perf-profile.calltrace.cycles-pp.do_wp_page.handle_pte_fault.__handle_mm_fault.handle_mm_fault.do_page_fault
8.26 ± 9% +1.8 10.04 ± 5% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.posix_fadvise
8.26 ± 9% +1.8 10.04 ± 5% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.posix_fadvise
8.24 ± 9% +1.8 10.02 ± 5% perf-profile.calltrace.cycles-pp.__x64_sys_fadvise64.do_syscall_64.entry_SYSCALL_64_after_hwframe.posix_fadvise
8.24 ± 9% +1.8 10.02 ± 5% perf-profile.calltrace.cycles-pp.ksys_fadvise64_64.__x64_sys_fadvise64.do_syscall_64.entry_SYSCALL_64_after_hwframe.posix_fadvise
8.24 ± 9% +1.8 10.02 ± 5% perf-profile.calltrace.cycles-pp.generic_fadvise.ksys_fadvise64_64.__x64_sys_fadvise64.do_syscall_64.entry_SYSCALL_64_after_hwframe
8.28 ± 9% +1.8 10.07 ± 5% perf-profile.calltrace.cycles-pp.posix_fadvise
3.07 ± 9% +2.1 5.21 ± 8% perf-profile.calltrace.cycles-pp.mpage_process_page_bufs.mpage_prepare_extent_to_map.ext4_writepages.do_writepages.__filemap_fdatawrite_range
3.29 ± 9% +2.3 5.56 ± 8% perf-profile.calltrace.cycles-pp.mpage_prepare_extent_to_map.ext4_writepages.do_writepages.__filemap_fdatawrite_range.generic_fadvise
3.30 ± 9% +2.3 5.59 ± 8% perf-profile.calltrace.cycles-pp.do_writepages.__filemap_fdatawrite_range.generic_fadvise.ksys_fadvise64_64.__x64_sys_fadvise64
3.30 ± 9% +2.3 5.59 ± 8% perf-profile.calltrace.cycles-pp.ext4_writepages.do_writepages.__filemap_fdatawrite_range.generic_fadvise.ksys_fadvise64_64
3.30 ± 9% +2.3 5.59 ± 8% perf-profile.calltrace.cycles-pp.__filemap_fdatawrite_range.generic_fadvise.ksys_fadvise64_64.__x64_sys_fadvise64.do_syscall_64
4.45 ± 11% +2.4 6.80 ± 13% perf-profile.calltrace.cycles-pp.shrink_page_list.shrink_inactive_list.shrink_lruvec.shrink_node.balance_pgdat
3.45 ± 16% +3.3 6.74 ± 3% perf-profile.calltrace.cycles-pp.__memcpy_flushcache.write_pmem.pmem_do_bvec.pmem_make_request.generic_make_request
3.49 ± 16% +3.3 6.81 ± 3% perf-profile.calltrace.cycles-pp.write_pmem.pmem_do_bvec.pmem_make_request.generic_make_request.submit_bio
3.51 ± 16% +3.3 6.86 ± 3% perf-profile.calltrace.cycles-pp.pmem_do_bvec.pmem_make_request.generic_make_request.submit_bio.ext4_io_submit
7.42 ± 7% +4.7 12.16 ± 2% perf-profile.calltrace.cycles-pp.mpage_process_page_bufs.mpage_prepare_extent_to_map.ext4_writepages.do_writepages.__writeback_single_inode
7.77 ± 6% +5.0 12.77 perf-profile.calltrace.cycles-pp.mpage_prepare_extent_to_map.ext4_writepages.do_writepages.__writeback_single_inode.writeback_sb_inodes
7.94 ± 7% +5.0 12.96 perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork
7.96 ± 7% +5.0 13.00 perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork
7.79 ± 6% +5.1 12.85 perf-profile.calltrace.cycles-pp.do_writepages.__writeback_single_inode.writeback_sb_inodes.__writeback_inodes_wb.wb_writeback
7.79 ± 6% +5.1 12.85 perf-profile.calltrace.cycles-pp.ext4_writepages.do_writepages.__writeback_single_inode.writeback_sb_inodes.__writeback_inodes_wb
7.79 ± 6% +5.1 12.85 perf-profile.calltrace.cycles-pp.wb_workfn.process_one_work.worker_thread.kthread.ret_from_fork
7.79 ± 6% +5.1 12.85 perf-profile.calltrace.cycles-pp.wb_writeback.wb_workfn.process_one_work.worker_thread.kthread
7.79 ± 6% +5.1 12.85 perf-profile.calltrace.cycles-pp.__writeback_inodes_wb.wb_writeback.wb_workfn.process_one_work.worker_thread
7.79 ± 6% +5.1 12.85 perf-profile.calltrace.cycles-pp.writeback_sb_inodes.__writeback_inodes_wb.wb_writeback.wb_workfn.process_one_work
7.79 ± 6% +5.1 12.85 perf-profile.calltrace.cycles-pp.__writeback_single_inode.writeback_sb_inodes.__writeback_inodes_wb.wb_writeback.wb_workfn
5.48 ± 12% +5.1 10.57 ± 2% perf-profile.calltrace.cycles-pp.pmem_make_request.generic_make_request.submit_bio.ext4_io_submit.ext4_bio_write_page
5.49 ± 12% +5.1 10.58 ± 2% perf-profile.calltrace.cycles-pp.generic_make_request.submit_bio.ext4_io_submit.ext4_bio_write_page.mpage_submit_page
5.50 ± 12% +5.1 10.58 ± 2% perf-profile.calltrace.cycles-pp.submit_bio.ext4_io_submit.ext4_bio_write_page.mpage_submit_page.mpage_process_page_bufs
5.50 ± 12% +5.1 10.58 ± 2% perf-profile.calltrace.cycles-pp.ext4_io_submit.ext4_bio_write_page.mpage_submit_page.mpage_process_page_bufs.mpage_prepare_extent_to_map
6.60 ± 11% +6.0 12.63 ± 2% perf-profile.calltrace.cycles-pp.ext4_bio_write_page.mpage_submit_page.mpage_process_page_bufs.mpage_prepare_extent_to_map.ext4_writepages
15.73 ± 8% +6.1 21.88 ± 5% perf-profile.calltrace.cycles-pp.ret_from_fork
15.73 ± 8% +6.1 21.88 ± 5% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork
10.14 ± 7% +6.8 16.91 ± 2% perf-profile.calltrace.cycles-pp.mpage_submit_page.mpage_process_page_bufs.mpage_prepare_extent_to_map.ext4_writepages.do_writepages
22.84 ± 16% -18.1 4.78 ± 6% perf-profile.children.cycles-pp.pagecache_get_page
29.77 ± 13% -16.9 12.85 ± 2% perf-profile.children.cycles-pp.filemap_fault
30.04 ± 13% -16.9 13.15 ± 2% perf-profile.children.cycles-pp.ext4_filemap_fault
30.07 ± 13% -16.9 13.18 ± 2% perf-profile.children.cycles-pp.__do_fault
18.08 ± 18% -16.8 1.31 ± 26% perf-profile.children.cycles-pp.__alloc_pages_slowpath
18.83 ± 17% -16.5 2.35 ± 14% perf-profile.children.cycles-pp.__alloc_pages_nodemask
16.42 ± 19% -15.5 0.89 ± 37% perf-profile.children.cycles-pp.try_to_free_pages
16.40 ± 19% -15.5 0.89 ± 37% perf-profile.children.cycles-pp.do_try_to_free_pages
24.16 ± 14% -14.4 9.73 ± 11% perf-profile.children.cycles-pp.shrink_node
24.07 ± 14% -14.4 9.67 ± 11% perf-profile.children.cycles-pp.shrink_lruvec
37.48 ± 11% -14.3 23.17 ± 2% perf-profile.children.cycles-pp.handle_pte_fault
38.05 ± 11% -14.0 24.07 ± 2% perf-profile.children.cycles-pp.__handle_mm_fault
38.64 ± 11% -13.6 25.00 ± 2% perf-profile.children.cycles-pp.handle_mm_fault
39.58 ± 11% -13.0 26.58 ± 2% perf-profile.children.cycles-pp.do_page_fault
39.84 ± 11% -12.8 27.01 ± 2% perf-profile.children.cycles-pp.page_fault
15.30 ± 22% -12.8 2.48 ± 12% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
19.34 ± 14% -9.8 9.52 ± 10% perf-profile.children.cycles-pp.shrink_inactive_list
8.85 ± 23% -8.1 0.72 ± 23% perf-profile.children.cycles-pp._raw_spin_lock_irq
4.72 ± 13% -4.6 0.11 ± 68% perf-profile.children.cycles-pp.shrink_active_list
6.92 ± 18% -4.0 2.88 ± 6% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
11.04 ± 10% -3.8 7.26 ± 11% perf-profile.children.cycles-pp.shrink_page_list
5.65 ± 20% -3.0 2.67 ± 3% perf-profile.children.cycles-pp.pagevec_lru_move_fn
4.12 ± 12% -2.1 2.06 ± 3% perf-profile.children.cycles-pp.add_to_page_cache_lru
3.42 ± 10% -1.9 1.51 ± 13% perf-profile.children.cycles-pp.__remove_mapping
3.37 ± 22% -1.7 1.70 ± 5% perf-profile.children.cycles-pp.deactivate_file_page
3.11 ± 15% -1.5 1.57 ± 5% perf-profile.children.cycles-pp.on_each_cpu_cond_mask
2.41 ± 16% -1.3 1.08 ± 2% perf-profile.children.cycles-pp.__lru_cache_add
2.96 ± 10% -1.3 1.66 ± 9% perf-profile.children.cycles-pp.page_referenced
1.43 ± 25% -1.2 0.23 ± 11% perf-profile.children.cycles-pp.try_to_unmap_flush
1.43 ± 25% -1.2 0.23 ± 11% perf-profile.children.cycles-pp.arch_tlbbatch_flush
1.29 ± 24% -1.0 0.31 ± 15% perf-profile.children.cycles-pp.smp_call_function_many_cond
1.64 ± 10% -0.7 0.92 ± 8% perf-profile.children.cycles-pp.page_referenced_one
0.80 ± 9% -0.7 0.14 ± 23% perf-profile.children.cycles-pp.workingset_refault
0.80 ± 13% -0.6 0.18 ± 15% perf-profile.children.cycles-pp.wake_all_kswapds
0.78 ± 14% -0.6 0.17 ± 16% perf-profile.children.cycles-pp.wakeup_kswapd
1.80 ± 10% -0.6 1.23 ± 5% perf-profile.children.cycles-pp.smp_call_function_single
1.73 ± 11% -0.5 1.22 ± 15% perf-profile.children.cycles-pp.isolate_lru_pages
1.19 ± 20% -0.5 0.68 ± 29% perf-profile.children.cycles-pp.create_empty_buffers
1.21 ± 19% -0.5 0.71 ± 28% perf-profile.children.cycles-pp.create_page_buffers
1.74 ± 10% -0.5 1.25 ± 5% perf-profile.children.cycles-pp.page_vma_mapped_walk
0.85 ± 26% -0.4 0.44 ± 46% perf-profile.children.cycles-pp.alloc_page_buffers
1.61 ± 10% -0.4 1.22 ± 9% perf-profile.children.cycles-pp.__list_del_entry_valid
0.80 ± 28% -0.4 0.40 ± 50% perf-profile.children.cycles-pp.alloc_buffer_head
0.64 ± 32% -0.4 0.27 ± 71% perf-profile.children.cycles-pp.__slab_alloc
1.54 ± 17% -0.4 1.17 ± 16% perf-profile.children.cycles-pp.__block_write_begin_int
0.86 ± 26% -0.4 0.49 ± 40% perf-profile.children.cycles-pp.kmem_cache_alloc
0.63 ± 32% -0.4 0.27 ± 73% perf-profile.children.cycles-pp.___slab_alloc
0.81 ± 10% -0.4 0.46 ± 16% perf-profile.children.cycles-pp.move_pages_to_lru
0.48 ± 43% -0.3 0.15 ±138% perf-profile.children.cycles-pp.new_slab
0.35 ± 10% -0.3 0.09 ± 15% perf-profile.children.cycles-pp.advance_inactive_age
0.40 ± 8% -0.2 0.19 ± 9% perf-profile.children.cycles-pp.workingset_eviction
0.49 ± 8% -0.2 0.30 ± 13% perf-profile.children.cycles-pp.try_to_free_buffers
0.35 ± 10% -0.2 0.19 ± 21% perf-profile.children.cycles-pp.jbd2_journal_try_to_free_buffers
0.33 ± 8% -0.2 0.18 ± 22% perf-profile.children.cycles-pp.jbd2_journal_grab_journal_head
0.37 ± 9% -0.1 0.26 ± 4% perf-profile.children.cycles-pp.page_add_file_rmap
0.32 ± 13% -0.1 0.20 ± 18% perf-profile.children.cycles-pp.page_evictable
0.28 ± 12% -0.1 0.19 ± 19% perf-profile.children.cycles-pp.page_remove_rmap
0.14 ± 6% -0.1 0.07 ± 12% perf-profile.children.cycles-pp.vma_interval_tree_iter_first
0.09 ± 11% -0.1 0.03 ±100% perf-profile.children.cycles-pp.get_partial_node
0.17 ± 10% -0.1 0.11 ± 15% perf-profile.children.cycles-pp.drop_buffers
0.19 ± 13% -0.1 0.12 ± 12% perf-profile.children.cycles-pp.free_buffer_head
0.24 ± 10% -0.0 0.19 ± 3% perf-profile.children.cycles-pp.ext4_da_get_block_prep
0.15 ± 10% -0.0 0.10 ± 12% perf-profile.children.cycles-pp.mem_cgroup_page_lruvec
0.12 ± 14% -0.0 0.08 ± 15% perf-profile.children.cycles-pp.__slab_free
0.10 ± 11% -0.0 0.06 ± 13% perf-profile.children.cycles-pp.ptep_clear_flush_young
0.06 ± 9% +0.0 0.07 ± 5% perf-profile.children.cycles-pp.__indirect_thunk_start
0.06 ± 11% +0.0 0.09 ± 12% perf-profile.children.cycles-pp.generic_update_time
0.08 ± 11% +0.0 0.11 ± 8% perf-profile.children.cycles-pp.fput_many
0.14 ± 7% +0.0 0.17 ± 7% perf-profile.children.cycles-pp.mempool_alloc
0.13 ± 6% +0.0 0.17 ± 6% perf-profile.children.cycles-pp._find_next_bit
0.09 ± 24% +0.0 0.12 ± 4% perf-profile.children.cycles-pp.try_to_wake_up
0.08 ± 5% +0.0 0.11 ± 14% perf-profile.children.cycles-pp.task_tick_fair
0.07 ± 19% +0.0 0.10 ± 10% perf-profile.children.cycles-pp.io_serial_in
0.04 ± 58% +0.0 0.08 ± 10% perf-profile.children.cycles-pp.ext4_dirty_inode
0.06 ± 9% +0.0 0.09 ± 8% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
0.09 ± 19% +0.0 0.13 ± 18% perf-profile.children.cycles-pp.wait_for_xmitr
0.09 ± 19% +0.0 0.13 ± 18% perf-profile.children.cycles-pp.serial8250_console_putchar
0.06 ± 7% +0.0 0.10 ± 4% perf-profile.children.cycles-pp.__unlock_page_memcg
0.07 ± 6% +0.0 0.11 ± 6% perf-profile.children.cycles-pp.pmd_devmap_trans_unstable
0.09 ± 5% +0.0 0.13 ± 6% perf-profile.children.cycles-pp.bio_add_page
0.11 ± 7% +0.0 0.15 ± 10% perf-profile.children.cycles-pp.PageHuge
0.10 ± 21% +0.0 0.14 ± 16% perf-profile.children.cycles-pp.serial8250_console_write
0.07 ± 7% +0.0 0.11 ± 13% perf-profile.children.cycles-pp.schedule
0.07 ± 5% +0.0 0.12 ± 11% perf-profile.children.cycles-pp.ext4_convert_inline_data
0.13 ± 14% +0.0 0.18 ± 10% perf-profile.children.cycles-pp.blk_throtl_bio
0.06 ± 20% +0.0 0.11 ± 7% perf-profile.children.cycles-pp.native_write_msr
0.17 ± 6% +0.0 0.22 ± 8% perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode
0.10 ± 12% +0.0 0.15 ± 8% perf-profile.children.cycles-pp.read_tsc
0.01 ±173% +0.1 0.06 ± 13% perf-profile.children.cycles-pp.update_load_avg
0.04 ± 57% +0.1 0.09 ± 13% perf-profile.children.cycles-pp.__next_timer_interrupt
0.11 ± 17% +0.1 0.16 ± 6% perf-profile.children.cycles-pp.irq_work_run_list
0.01 ±173% +0.1 0.07 ± 23% perf-profile.children.cycles-pp.pick_next_task_fair
0.00 +0.1 0.05 ± 8% perf-profile.children.cycles-pp.__bio_try_merge_page
0.10 ± 17% +0.1 0.15 ± 7% perf-profile.children.cycles-pp.console_unlock
0.06 ± 9% +0.1 0.11 ± 11% perf-profile.children.cycles-pp.page_mapped
0.00 +0.1 0.06 ± 9% perf-profile.children.cycles-pp.sched_clock_cpu
0.22 ± 9% +0.1 0.28 perf-profile.children.cycles-pp.bio_alloc_bioset
0.43 ± 9% +0.1 0.49 ± 3% perf-profile.children.cycles-pp.__mod_memcg_state
0.10 ± 7% +0.1 0.16 ± 5% perf-profile.children.cycles-pp.vmacache_find
0.08 ± 16% +0.1 0.14 ± 6% perf-profile.children.cycles-pp.lru_deactivate_file_fn
0.00 +0.1 0.06 ± 7% perf-profile.children.cycles-pp._raw_spin_trylock
0.01 ±173% +0.1 0.07 ± 10% perf-profile.children.cycles-pp.mem_cgroup_wb_domain
0.12 ± 13% +0.1 0.18 ± 6% perf-profile.children.cycles-pp.xas_clear_mark
0.08 ± 10% +0.1 0.14 ± 5% perf-profile.children.cycles-pp.down_read_trylock
0.03 ±100% +0.1 0.09 ± 9% perf-profile.children.cycles-pp.lapic_next_deadline
0.00 +0.1 0.06 ± 11% perf-profile.children.cycles-pp.note_gp_changes
0.06 ± 13% +0.1 0.12 ± 11% perf-profile.children.cycles-pp.__sb_start_write
0.00 +0.1 0.06 ± 6% perf-profile.children.cycles-pp.ext4_inode_journal_mode
0.32 ± 11% +0.1 0.39 ± 3% perf-profile.children.cycles-pp.find_get_entry
0.00 +0.1 0.07 ± 17% perf-profile.children.cycles-pp.migrate_pages
0.04 ± 57% +0.1 0.11 ± 15% perf-profile.children.cycles-pp.load_balance
0.00 +0.1 0.07 ± 6% perf-profile.children.cycles-pp.current_time
0.00 +0.1 0.07 ± 6% perf-profile.children.cycles-pp.run_timer_softirq
0.08 +0.1 0.15 ± 10% perf-profile.children.cycles-pp.balance_dirty_pages
0.07 ± 13% +0.1 0.14 ± 6% perf-profile.children.cycles-pp.get_next_timer_interrupt
0.04 ± 58% +0.1 0.11 ± 11% perf-profile.children.cycles-pp.inc_zone_page_state
0.01 ±173% +0.1 0.09 ± 10% perf-profile.children.cycles-pp.ktime_get_update_offsets_now
0.11 ± 3% +0.1 0.19 ± 4% perf-profile.children.cycles-pp.find_vma
0.01 ±173% +0.1 0.09 ± 9% perf-profile.children.cycles-pp.xas_set_mark
0.23 ± 7% +0.1 0.31 ± 5% perf-profile.children.cycles-pp.xas_find
0.07 ± 12% +0.1 0.15 ± 7% perf-profile.children.cycles-pp.__fprop_inc_percpu_max
0.21 ± 7% +0.1 0.29 ± 5% perf-profile.children.cycles-pp.__might_sleep
0.29 ± 11% +0.1 0.36 ± 9% perf-profile.children.cycles-pp._cond_resched
0.00 +0.1 0.08 ± 19% perf-profile.children.cycles-pp.find_busiest_group
0.18 ± 6% +0.1 0.26 ± 4% perf-profile.children.cycles-pp.xas_start
0.13 ± 6% +0.1 0.22 ± 14% perf-profile.children.cycles-pp.scheduler_tick
0.12 ± 15% +0.1 0.20 ± 7% perf-profile.children.cycles-pp.__mark_inode_dirty
0.13 ± 12% +0.1 0.21 ± 6% perf-profile.children.cycles-pp.file_update_time
0.00 +0.1 0.09 ± 18% perf-profile.children.cycles-pp.migrate_misplaced_page
0.04 ± 57% +0.1 0.13 ± 19% perf-profile.children.cycles-pp.rebalance_domains
0.08 ± 5% +0.1 0.18 ± 8% perf-profile.children.cycles-pp.invalidate_inode_page
0.04 ± 58% +0.1 0.14 ± 3% perf-profile.children.cycles-pp.finish_mkwrite_fault
0.35 ± 8% +0.1 0.45 ± 4% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.12 ± 16% +0.1 0.23 ± 22% perf-profile.children.cycles-pp.tick_nohz_next_event
0.26 ± 7% +0.1 0.38 ± 8% perf-profile.children.cycles-pp.flush_tlb_func_common
0.12 ± 6% +0.1 0.24 ± 3% perf-profile.children.cycles-pp.set_page_dirty
0.35 ± 6% +0.1 0.47 ± 5% perf-profile.children.cycles-pp.generic_make_request_checks
0.20 ± 7% +0.1 0.31 ± 6% perf-profile.children.cycles-pp.native_flush_tlb_one_user
0.14 ± 13% +0.1 0.26 ± 8% perf-profile.children.cycles-pp.dec_zone_page_state
0.45 ± 6% +0.1 0.57 ± 7% perf-profile.children.cycles-pp.___might_sleep
0.14 ± 17% +0.1 0.27 ± 22% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
0.21 ± 2% +0.1 0.33 ± 9% perf-profile.children.cycles-pp.___perf_sw_event
0.14 ± 10% +0.1 0.27 ± 5% perf-profile.children.cycles-pp.balance_dirty_pages_ratelimited
0.11 ± 7% +0.1 0.24 ± 6% perf-profile.children.cycles-pp.__xa_set_mark
0.14 ± 23% +0.1 0.27 ± 11% perf-profile.children.cycles-pp.rcu_core
0.11 ± 6% +0.1 0.25 ± 6% perf-profile.children.cycles-pp.__xa_clear_mark
0.03 ±100% +0.1 0.17 ± 42% perf-profile.children.cycles-pp.tick_irq_enter
0.04 ± 58% +0.2 0.20 ± 35% perf-profile.children.cycles-pp.irq_enter
0.00 +0.2 0.17 ± 6% perf-profile.children.cycles-pp.pagevec_move_tail_fn
0.18 ± 15% +0.2 0.36 ± 5% perf-profile.children.cycles-pp.__pagevec_release
0.31 ± 2% +0.2 0.51 ± 7% perf-profile.children.cycles-pp.__perf_sw_event
0.24 ± 11% +0.2 0.43 ± 15% perf-profile.children.cycles-pp.update_process_times
0.52 ± 7% +0.2 0.72 ± 6% perf-profile.children.cycles-pp.account_page_dirtied
0.24 ± 12% +0.2 0.45 ± 16% perf-profile.children.cycles-pp.tick_sched_handle
0.33 ± 2% +0.2 0.55 ± 7% perf-profile.children.cycles-pp.find_get_pages_range_tag
0.33 ± 2% +0.2 0.55 ± 6% perf-profile.children.cycles-pp.pagevec_lookup_range_tag
0.94 ± 7% +0.2 1.17 ± 2% perf-profile.children.cycles-pp.unlock_page
0.05 ± 59% +0.2 0.29 ± 8% perf-profile.children.cycles-pp.pagevec_move_tail
0.53 ± 9% +0.2 0.76 ± 5% perf-profile.children.cycles-pp.page_mapping
0.29 ± 7% +0.3 0.56 ± 18% perf-profile.children.cycles-pp.tick_sched_timer
0.22 ± 13% +0.3 0.49 ± 22% perf-profile.children.cycles-pp.clockevents_program_event
0.05 ± 59% +0.3 0.33 ± 5% perf-profile.children.cycles-pp.rotate_reclaimable_page
0.29 ± 12% +0.3 0.57 ± 14% perf-profile.children.cycles-pp.__softirqentry_text_start
0.42 ± 13% +0.3 0.72 ± 4% perf-profile.children.cycles-pp.release_pages
2.77 ± 5% +0.3 3.08 ± 3% perf-profile.children.cycles-pp.page_mkclean
0.43 ± 6% +0.3 0.78 ± 4% perf-profile.children.cycles-pp.percpu_counter_add_batch
0.39 ± 6% +0.3 0.74 ± 15% perf-profile.children.cycles-pp.__hrtimer_run_queues
0.39 ± 8% +0.4 0.75 ± 18% perf-profile.children.cycles-pp.irq_exit
0.28 ± 15% +0.4 0.64 ± 8% perf-profile.children.cycles-pp.menu_select
0.75 ± 7% +0.4 1.16 ± 5% perf-profile.children.cycles-pp.__set_page_dirty
2.63 ± 8% +0.4 3.04 ± 3% perf-profile.children.cycles-pp.filemap_map_pages
0.72 ± 8% +0.4 1.14 ± 3% perf-profile.children.cycles-pp.xas_load
0.42 ± 5% +0.4 0.86 ± 5% perf-profile.children.cycles-pp.fault_dirty_shared_page
0.39 ± 7% +0.4 0.84 ± 15% perf-profile.children.cycles-pp.ktime_get
0.93 ± 9% +0.5 1.38 ± 4% perf-profile.children.cycles-pp.find_get_entries
0.61 ± 7% +0.5 1.07 ± 4% perf-profile.children.cycles-pp.sync_regs
0.93 ± 9% +0.5 1.38 ± 3% perf-profile.children.cycles-pp.pagevec_lookup_entries
0.78 ± 7% +0.5 1.26 ± 5% perf-profile.children.cycles-pp.get_io_u
0.46 ± 7% +0.5 0.97 ± 4% perf-profile.children.cycles-pp.__set_page_dirty_buffers
0.96 ± 8% +0.6 1.54 ± 3% perf-profile.children.cycles-pp.mark_buffer_dirty
2.93 ± 12% +0.6 3.55 ± 4% perf-profile.children.cycles-pp.block_page_mkwrite
0.70 ± 4% +0.6 1.33 ± 2% perf-profile.children.cycles-pp.__test_set_page_writeback
4.37 ± 6% +0.7 5.02 ± 4% perf-profile.children.cycles-pp.__memcpy_mcsafe
1.09 ± 9% +0.7 1.76 ± 5% perf-profile.children.cycles-pp.native_irq_return_iret
0.69 ± 8% +0.7 1.39 ± 15% perf-profile.children.cycles-pp.hrtimer_interrupt
1.11 ± 8% +0.7 1.81 ± 3% perf-profile.children.cycles-pp.__block_commit_write
3.53 ± 5% +0.7 4.27 ± 3% perf-profile.children.cycles-pp.clear_page_dirty_for_io
3.40 ± 12% +1.0 4.42 ± 3% perf-profile.children.cycles-pp.ext4_page_mkwrite
3.42 ± 12% +1.0 4.46 ± 3% perf-profile.children.cycles-pp.do_page_mkwrite
1.00 ± 6% +1.1 2.06 ± 3% perf-profile.children.cycles-pp.test_clear_page_writeback
6.32 ± 5% +1.1 7.41 ± 4% perf-profile.children.cycles-pp.ext4_mpage_readpages
1.15 ± 8% +1.2 2.36 ± 14% perf-profile.children.cycles-pp.smp_apic_timer_interrupt
1.26 ± 7% +1.3 2.59 ± 13% perf-profile.children.cycles-pp.apic_timer_interrupt
1.07 ± 7% +1.3 2.42 ± 3% perf-profile.children.cycles-pp.end_page_writeback
1.94 ± 5% +1.7 3.66 perf-profile.children.cycles-pp.ext4_end_bio
1.94 ± 5% +1.7 3.66 perf-profile.children.cycles-pp.ext4_finish_bio
8.51 ± 9% +1.7 10.24 ± 5% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
8.51 ± 9% +1.7 10.24 ± 5% perf-profile.children.cycles-pp.do_syscall_64
1.94 ± 8% +1.8 3.69 ± 4% perf-profile.children.cycles-pp.do_wp_page
8.24 ± 9% +1.8 10.02 ± 5% perf-profile.children.cycles-pp.__x64_sys_fadvise64
8.24 ± 9% +1.8 10.02 ± 5% perf-profile.children.cycles-pp.ksys_fadvise64_64
8.24 ± 9% +1.8 10.02 ± 5% perf-profile.children.cycles-pp.generic_fadvise
8.28 ± 9% +1.8 10.07 ± 5% perf-profile.children.cycles-pp.posix_fadvise
3.30 ± 9% +2.3 5.59 ± 8% perf-profile.children.cycles-pp.__filemap_fdatawrite_range
3.47 ± 16% +3.3 6.81 ± 3% perf-profile.children.cycles-pp.__memcpy_flushcache
3.50 ± 16% +3.4 6.88 ± 3% perf-profile.children.cycles-pp.write_pmem
7.95 ± 7% +4.1 12.00 perf-profile.children.cycles-pp.pmem_do_bvec
7.94 ± 7% +5.0 12.96 perf-profile.children.cycles-pp.process_one_work
7.96 ± 7% +5.0 13.00 perf-profile.children.cycles-pp.worker_thread
7.79 ± 6% +5.1 12.85 perf-profile.children.cycles-pp.wb_workfn
7.79 ± 6% +5.1 12.85 perf-profile.children.cycles-pp.wb_writeback
7.79 ± 6% +5.1 12.85 perf-profile.children.cycles-pp.__writeback_inodes_wb
7.79 ± 6% +5.1 12.85 perf-profile.children.cycles-pp.writeback_sb_inodes
7.79 ± 6% +5.1 12.85 perf-profile.children.cycles-pp.__writeback_single_inode
5.52 ± 12% +5.2 10.67 ± 2% perf-profile.children.cycles-pp.ext4_io_submit
10.52 ± 7% +5.9 16.44 perf-profile.children.cycles-pp.pmem_make_request
6.60 ± 11% +6.0 12.63 ± 2% perf-profile.children.cycles-pp.ext4_bio_write_page
10.95 ± 7% +6.1 17.01 perf-profile.children.cycles-pp.generic_make_request
10.98 ± 6% +6.1 17.05 perf-profile.children.cycles-pp.submit_bio
15.73 ± 8% +6.1 21.88 ± 5% perf-profile.children.cycles-pp.ret_from_fork
15.73 ± 8% +6.1 21.88 ± 5% perf-profile.children.cycles-pp.kthread
10.14 ± 7% +6.8 16.92 ± 2% perf-profile.children.cycles-pp.mpage_submit_page
10.50 ± 7% +6.9 17.36 ± 2% perf-profile.children.cycles-pp.mpage_process_page_bufs
11.06 ± 7% +7.3 18.36 ± 2% perf-profile.children.cycles-pp.mpage_prepare_extent_to_map
11.09 ± 7% +7.4 18.45 ± 2% perf-profile.children.cycles-pp.do_writepages
11.09 ± 7% +7.4 18.45 ± 2% perf-profile.children.cycles-pp.ext4_writepages
15.29 ± 22% -12.8 2.47 ± 12% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
1.16 ± 26% -1.0 0.20 ± 21% perf-profile.self.cycles-pp.smp_call_function_many_cond
0.76 ± 13% -0.6 0.17 ± 17% perf-profile.self.cycles-pp.wakeup_kswapd
1.66 ± 10% -0.6 1.09 ± 5% perf-profile.self.cycles-pp.smp_call_function_single
1.60 ± 10% -0.4 1.21 ± 9% perf-profile.self.cycles-pp.__list_del_entry_valid
0.42 ± 8% -0.3 0.10 ± 23% perf-profile.self.cycles-pp.workingset_refault
0.34 ± 10% -0.2 0.09 ± 15% perf-profile.self.cycles-pp.advance_inactive_age
0.99 ± 9% -0.2 0.79 ± 7% perf-profile.self.cycles-pp.page_vma_mapped_walk
0.74 ± 10% -0.2 0.55 ± 5% perf-profile.self.cycles-pp.get_page_from_freelist
0.43 ± 9% -0.2 0.25 ± 16% perf-profile.self.cycles-pp.move_pages_to_lru
0.62 ± 9% -0.2 0.47 ± 7% perf-profile.self.cycles-pp.__mod_lruvec_state
0.32 ± 9% -0.1 0.17 ± 25% perf-profile.self.cycles-pp.jbd2_journal_grab_journal_head
0.37 ± 9% -0.1 0.24 ± 18% perf-profile.self.cycles-pp.__remove_mapping
0.37 ± 10% -0.1 0.26 ± 10% perf-profile.self.cycles-pp.page_referenced_one
0.23 ± 12% -0.1 0.13 ± 17% perf-profile.self.cycles-pp.page_evictable
0.17 ± 6% -0.1 0.10 ± 11% perf-profile.self.cycles-pp.workingset_eviction
0.16 ± 9% -0.1 0.10 ± 11% perf-profile.self.cycles-pp.page_add_file_rmap
0.14 ± 15% -0.1 0.08 ± 6% perf-profile.self.cycles-pp.pagecache_get_page
0.17 ± 12% -0.1 0.11 ± 15% perf-profile.self.cycles-pp.drop_buffers
0.12 ± 5% -0.1 0.06 ± 14% perf-profile.self.cycles-pp.vma_interval_tree_iter_first
0.26 ± 11% -0.1 0.20 ± 9% perf-profile.self.cycles-pp.isolate_lru_pages
0.12 ± 11% -0.1 0.06 ± 13% perf-profile.self.cycles-pp.page_referenced
0.14 ± 8% -0.1 0.09 ± 13% perf-profile.self.cycles-pp.mem_cgroup_page_lruvec
0.12 ± 11% -0.1 0.07 ± 12% perf-profile.self.cycles-pp.page_remove_rmap
0.12 ± 14% -0.0 0.08 ± 15% perf-profile.self.cycles-pp.__slab_free
0.10 ± 11% -0.0 0.06 ± 13% perf-profile.self.cycles-pp.ptep_clear_flush_young
0.08 ± 10% -0.0 0.06 perf-profile.self.cycles-pp.ext4_da_get_block_prep
0.10 ± 7% +0.0 0.12 ± 3% perf-profile.self.cycles-pp.rcu_all_qs
0.06 ± 6% +0.0 0.08 ± 5% perf-profile.self.cycles-pp.__lru_cache_add
0.07 ± 17% +0.0 0.10 ± 8% perf-profile.self.cycles-pp.xas_find
0.07 ± 10% +0.0 0.10 ± 8% perf-profile.self.cycles-pp.fput_many
0.12 ± 8% +0.0 0.16 ± 7% perf-profile.self.cycles-pp._find_next_bit
0.15 ± 8% +0.0 0.18 ± 7% perf-profile.self.cycles-pp.alloc_set_pte
0.08 ± 10% +0.0 0.12 ± 7% perf-profile.self.cycles-pp.generic_make_request_checks
0.08 ± 14% +0.0 0.11 ± 6% perf-profile.self.cycles-pp.pmem_do_bvec
0.06 ± 14% +0.0 0.09 ± 4% perf-profile.self.cycles-pp.lru_deactivate_file_fn
0.07 ± 19% +0.0 0.10 ± 10% perf-profile.self.cycles-pp.io_serial_in
0.08 ± 19% +0.0 0.11 ± 4% perf-profile.self.cycles-pp.free_unref_page_list
0.09 ± 11% +0.0 0.13 ± 11% perf-profile.self.cycles-pp.PageHuge
0.06 ± 6% +0.0 0.10 ± 7% perf-profile.self.cycles-pp.page_mkclean_one
0.07 ± 6% +0.0 0.11 ± 8% perf-profile.self.cycles-pp.swapgs_restore_regs_and_return_to_usermode
0.41 ± 6% +0.0 0.45 ± 8% perf-profile.self.cycles-pp.filemap_fault
0.12 ± 13% +0.0 0.15 ± 7% perf-profile.self.cycles-pp.xas_store
0.10 ± 4% +0.0 0.14 ± 10% perf-profile.self.cycles-pp.ptep_clear_flush
0.06 ± 7% +0.0 0.10 ± 11% perf-profile.self.cycles-pp.pmd_devmap_trans_unstable
0.04 ± 57% +0.0 0.08 ± 5% perf-profile.self.cycles-pp.__unlock_page_memcg
0.10 ± 12% +0.0 0.14 ± 9% perf-profile.self.cycles-pp.read_tsc
0.08 ± 14% +0.0 0.12 ± 4% perf-profile.self.cycles-pp.down_read_trylock
0.07 +0.0 0.12 ± 9% perf-profile.self.cycles-pp.ext4_convert_inline_data
0.07 ± 11% +0.0 0.12 ± 5% perf-profile.self.cycles-pp.flush_tlb_mm_range
0.06 ± 6% +0.0 0.11 ± 6% perf-profile.self.cycles-pp.set_page_dirty
0.07 ± 17% +0.0 0.11 ± 9% perf-profile.self.cycles-pp.pagevec_lru_move_fn
0.06 ± 13% +0.0 0.11 ± 11% perf-profile.self.cycles-pp.__mark_inode_dirty
0.06 ± 20% +0.0 0.11 ± 7% perf-profile.self.cycles-pp.native_write_msr
0.42 ± 8% +0.1 0.47 ± 3% perf-profile.self.cycles-pp.__mod_memcg_state
0.00 +0.1 0.05 ± 8% perf-profile.self.cycles-pp.__indirect_thunk_start
0.12 ± 13% +0.1 0.18 ± 14% perf-profile.self.cycles-pp._cond_resched
0.10 ± 9% +0.1 0.15 ± 4% perf-profile.self.cycles-pp.vmacache_find
0.00 +0.1 0.06 ± 9% perf-profile.self.cycles-pp.note_gp_changes
0.11 ± 11% +0.1 0.17 ± 10% perf-profile.self.cycles-pp.xas_clear_mark
0.00 +0.1 0.06 ± 7% perf-profile.self.cycles-pp.write_pmem
0.00 +0.1 0.06 ± 7% perf-profile.self.cycles-pp._raw_spin_trylock
0.01 ±173% +0.1 0.07 ± 10% perf-profile.self.cycles-pp.mem_cgroup_wb_domain
0.00 +0.1 0.06 ± 11% perf-profile.self.cycles-pp.pagevec_move_tail_fn
0.00 +0.1 0.06 ± 13% perf-profile.self.cycles-pp.__sb_start_write
0.00 +0.1 0.06 ± 17% perf-profile.self.cycles-pp.balance_dirty_pages
0.00 +0.1 0.06 ± 6% perf-profile.self.cycles-pp.ext4_inode_journal_mode
0.18 ± 8% +0.1 0.25 ± 5% perf-profile.self.cycles-pp.__might_sleep
0.04 ± 59% +0.1 0.11 ± 9% perf-profile.self.cycles-pp.balance_dirty_pages_ratelimited
0.13 ± 8% +0.1 0.20 ± 10% perf-profile.self.cycles-pp.pmem_make_request
0.01 ±173% +0.1 0.08 ± 13% perf-profile.self.cycles-pp.xas_set_mark
0.03 ±100% +0.1 0.10 ± 11% perf-profile.self.cycles-pp.inc_zone_page_state
0.03 ±100% +0.1 0.10 ± 11% perf-profile.self.cycles-pp.block_page_mkwrite
0.01 ±173% +0.1 0.09 ± 10% perf-profile.self.cycles-pp.page_mapped
0.00 +0.1 0.07 ± 5% perf-profile.self.cycles-pp.invalidate_inode_page
0.10 ± 10% +0.1 0.17 ± 6% perf-profile.self.cycles-pp.__perf_sw_event
0.10 ± 10% +0.1 0.17 ± 10% perf-profile.self.cycles-pp.page_fault
0.00 +0.1 0.07 ± 22% perf-profile.self.cycles-pp.fault_dirty_shared_page
0.08 ± 8% +0.1 0.16 ± 7% perf-profile.self.cycles-pp.__block_commit_write
0.00 +0.1 0.08 ± 11% perf-profile.self.cycles-pp.ktime_get_update_offsets_now
0.09 ± 13% +0.1 0.16 ± 11% perf-profile.self.cycles-pp.mark_buffer_dirty
0.16 ± 5% +0.1 0.24 ± 4% perf-profile.self.cycles-pp.xas_start
0.01 ±173% +0.1 0.09 ± 8% perf-profile.self.cycles-pp.ext4_page_mkwrite
0.35 ± 6% +0.1 0.44 ± 7% perf-profile.self.cycles-pp.mpage_process_page_bufs
0.17 ± 2% +0.1 0.27 ± 11% perf-profile.self.cycles-pp.___perf_sw_event
0.11 ± 9% +0.1 0.22 ± 5% perf-profile.self.cycles-pp.mpage_prepare_extent_to_map
0.12 ± 13% +0.1 0.23 ± 8% perf-profile.self.cycles-pp.dec_zone_page_state
0.44 ± 7% +0.1 0.55 ± 6% perf-profile.self.cycles-pp.___might_sleep
0.20 ± 7% +0.1 0.31 ± 6% perf-profile.self.cycles-pp.native_flush_tlb_one_user
0.28 ± 7% +0.1 0.41 ± 4% perf-profile.self.cycles-pp.clear_page_dirty_for_io
0.23 ± 6% +0.1 0.36 ± 4% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
0.22 ± 4% +0.1 0.36 ± 8% perf-profile.self.cycles-pp.do_wp_page
0.20 ± 6% +0.1 0.35 ± 4% perf-profile.self.cycles-pp.do_page_fault
0.10 ± 12% +0.1 0.25 ± 5% perf-profile.self.cycles-pp.cpuidle_enter_state
0.17 ± 13% +0.2 0.32 ± 8% perf-profile.self.cycles-pp.deactivate_file_page
0.26 ± 4% +0.2 0.43 ± 7% perf-profile.self.cycles-pp.handle_pte_fault
0.22 ± 14% +0.2 0.39 ± 7% perf-profile.self.cycles-pp.invalidate_mapping_pages
0.09 ± 9% +0.2 0.27 ± 3% perf-profile.self.cycles-pp.__block_write_begin_int
0.29 ± 3% +0.2 0.48 ± 6% perf-profile.self.cycles-pp.find_get_pages_range_tag
0.21 ± 6% +0.2 0.41 ± 3% perf-profile.self.cycles-pp.__test_set_page_writeback
0.49 ± 8% +0.2 0.71 ± 5% perf-profile.self.cycles-pp.page_mapping
0.12 ± 27% +0.2 0.35 ± 3% perf-profile.self.cycles-pp.menu_select
0.90 ± 7% +0.2 1.14 ± 2% perf-profile.self.cycles-pp.unlock_page
0.25 ± 5% +0.2 0.49 ± 4% perf-profile.self.cycles-pp.ext4_bio_write_page
0.33 ± 8% +0.2 0.58 ± 6% perf-profile.self.cycles-pp.percpu_counter_add_batch
0.17 ± 11% +0.3 0.43 ± 8% perf-profile.self.cycles-pp.__set_page_dirty_buffers
0.36 ± 12% +0.3 0.63 ± 4% perf-profile.self.cycles-pp.release_pages
0.35 ± 9% +0.3 0.64 ± 4% perf-profile.self.cycles-pp.handle_mm_fault
0.51 ± 6% +0.3 0.81 ± 8% perf-profile.self.cycles-pp.__handle_mm_fault
0.22 ± 6% +0.3 0.53 ± 7% perf-profile.self.cycles-pp.test_clear_page_writeback
1.61 ± 7% +0.3 1.94 ± 3% perf-profile.self.cycles-pp.filemap_map_pages
0.54 ± 9% +0.3 0.87 ± 5% perf-profile.self.cycles-pp.xas_load
0.86 ± 5% +0.4 1.23 ± 4% perf-profile.self.cycles-pp.ext4_finish_bio
0.69 ± 9% +0.4 1.06 ± 4% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.86 ± 9% +0.4 1.25 ± 4% perf-profile.self.cycles-pp.find_get_entries
0.30 ± 8% +0.4 0.70 ± 17% perf-profile.self.cycles-pp.ktime_get
0.61 ± 6% +0.4 1.06 ± 4% perf-profile.self.cycles-pp.sync_regs
0.76 ± 7% +0.5 1.23 ± 5% perf-profile.self.cycles-pp.get_io_u
4.28 ± 6% +0.6 4.92 ± 4% perf-profile.self.cycles-pp.__memcpy_mcsafe
1.09 ± 9% +0.7 1.76 ± 5% perf-profile.self.cycles-pp.native_irq_return_iret
3.43 ± 16% +3.3 6.72 ± 3% perf-profile.self.cycles-pp.__memcpy_flushcache
fio.write_bw_MBps
6000 +--------------------------------------------------------------------+
| O O |
5000 |-+ O O O O O O O O O O O O O O O O O O O O O O O |
| O .+. .+. |
|.+.+.+.+.+.+.+ + +.+.+.+.+.+..+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.|
4000 |-+ |
| |
3000 |-+ |
| |
2000 |-+ |
| |
| |
1000 |-+ |
| |
0 +--------------------------------------------------------------------+
fio.read_clat_mean_us
3.5e+06 +-----------------------------------------------------------------+
|.+ .+ .+ .+. +. |
3e+06 |-++ .+.+ .+.+.+.+.+ + .+ + .++.+ +.+. + +.|
| +.+.+.+.+ +.+ +.+.+ +.+.+ + |
2.5e+06 |-+ |
| |
2e+06 |-+ |
| |
1.5e+06 |-+ O O O O O O O O OO O O |
| O O O O OO O O O O O O O O |
1e+06 |-+ |
| |
500000 |-+ |
| |
0 +-----------------------------------------------------------------+
fio.read_clat_90__us
9e+06 +-------------------------------------------------------------------+
| + +. : + |
8e+06 |.+ .+. + : + +.+.+ .+.+ +. .+.+ : :|
7e+06 |-+: +.+ +.+ : .+.+.+ : +.+ : +.+.+. + + + : :|
| : + + : + : + + + |
6e+06 |-+ + + + |
5e+06 |-+ |
| |
4e+06 |-+ |
3e+06 |-+ |
| O O O O O O O O O O O O O O O O O O O O O O O O O O |
2e+06 |-+ |
1e+06 |-+ |
| |
0 +-------------------------------------------------------------------+
fio.read_clat_95__us
1.4e+07 +-----------------------------------------------------------------+
|.+ .+ .+ +.+. +. |
1.2e+07 |-++ +. .+.+.+.+.+ + .+ + .++. + +. + + |
| +.+.+.+. + ++.+ +.+.+ +.+.+ + + +|
1e+07 |-+ + |
| |
8e+06 |-+ |
| |
6e+06 |-+ |
| |
4e+06 |-+ O |
| O O O O O OO O O O O O O O O O O O O O O OO O O |
2e+06 |-+ |
| |
0 +-----------------------------------------------------------------+
fio.read_clat_99__us
2.5e+07 +-----------------------------------------------------------------+
| |
| +. |
2e+07 |.+. + +. .+. .+. ++. + +.+ +. |
| +. : + + +.+.+.+ +. .+ +. + + + + +.|
| +.+.+. : ++.+ +.+ +.+ + |
1.5e+07 |-+ + |
| |
1e+07 |-+ |
| O |
| O O O O O O O O O O O OO O O |
5e+06 |-+ O O O O O O O O O O |
| |
| |
0 +-----------------------------------------------------------------+
fio.latency_500us_
25 +----------------------------------------------------------------------+
| |
| |
20 |-+ O O O O O O O O O O O O |
| O O O O O O O O O O |
| O O O O |
15 |-+ |
| |
10 |-+ |
| .+.+..+. .+.+.+. .+. .+.+. .+. .+. .|
|.+.+.+ + +.+.+.+.+.+..+.+ +.+.+.+ + +..+.+ +.+ |
5 |-+ |
| |
| |
0 +----------------------------------------------------------------------+
fio.latency_2ms_
25 +----------------------------------------------------------------------+
| O O O O O O O O O O |
| O O O O O O O O O O O O O |
20 |-+ O |
| |
| |
15 |.+. .+.+.+..+.+.+.+.+.+.+.+.+.+.+.. .+.+.+.+. .+.+.+. .+.+..+.+. .+.+.|
| + + + + + |
10 |-+ |
| |
| |
5 |-+ |
| |
| |
0 +----------------------------------------------------------------------+
fio.latency_4ms_
35 +----------------------------------------------------------------------+
| |
30 |.+.+.+.+.+..+.+.+.+.+.+.+.+.+.+.+..+.+.+.+.+.+.+.+. .+.+. .+.+.+.+.|
| + +..+ |
25 |-+ O O O O O O O O O |
| O O O O O O O O O O O O O O O O O |
20 |-+ |
| |
15 |-+ |
| |
10 |-+ |
| |
5 |-+ |
| |
0 +----------------------------------------------------------------------+
fio.latency_10ms_
10 +----------------------------------------------------------------------+
9 |-+ .+ + +.+ |
| .+ + .+. + + .+.+.. + :|
8 |.+. .+.+.+..+ +.+ + +.+.+.. .+.+.+.+. .+.+.+.+ +.+.+ :|
7 |-+ + + + |
| |
6 |-+ |
5 |-+ |
4 |-+ |
| |
3 |-+ O O O O O O O O |
2 |-+ O O O O O O O O O O O O O O O O O O |
| |
1 |-+ |
0 +----------------------------------------------------------------------+
fio.latency_20ms_
8 +-----------------------------------------------------------------------+
| +. |
7 |.+ .+ +. .+ .+ : + |
6 |-+: + : +. .. +.+ : .+ + .+. .+.+.+..+. : :|
| : .+..+ : : + + : .+ + .+ + + :|
5 |-+ +.+ + : +.+ +.+ +.+ |
| + |
4 |-+ |
| |
3 |-+ |
2 |-+ |
| |
1 |-+ O |
| O O O O O O O O O O O O O O O O O O O O O O O O O |
0 +-----------------------------------------------------------------------+
fio.time.system_time
3500 +--------------------------------------------------------------------+
| + +. |
3000 |-+ + + .+. .+. .+ : + |
|.+. .+.+.+. .+ +.+.+.+ +.+.+.. .+.+.+.+. .+.+.+.+.+ + + : +|
2500 |-+ + + + + + |
| |
2000 |-+ |
| O O O O O O O O O O O O O O O O O O O O O O O O O |
1500 |-+ O |
| |
1000 |-+ |
| |
500 |-+ |
| |
0 +--------------------------------------------------------------------+
fio.time.percent_of_cpu_this_job_got
2000 +--------------------------------------------------------------------+
1800 |-+ +. |
| .+. .+. .+. .+ : + |
1600 |.+. .+.+.+.+.+ +.+.+.+ +.+.+.. .+.+.+.+. .+.+.+.+.+ + + : +|
1400 |-+ + + + + |
| |
1200 |-+ O O O O O O O O O O O O O |
1000 |-+ O O O O O O O O O O O O O |
800 |-+ |
| |
600 |-+ |
400 |-+ |
| |
200 |-+ |
0 +--------------------------------------------------------------------+
fio.time.major_page_faults
3e+08 +-----------------------------------------------------------------+
| |
2.5e+08 |-+ .+ .+. .+.+.|
|.+.+.+.+.+.+.+ +.+.+.+ +.+.+.+.+.+.+.+.+.+.+.++.+. .+.+.+ |
| + |
2e+08 |-+ O O O O OO O O O O O O O O O O O O O O O OO O O |
| O |
1.5e+08 |-+ |
| |
1e+08 |-+ |
| |
| |
5e+07 |-+ |
| |
0 +-----------------------------------------------------------------+
fio.time.minor_page_faults
2.5e+08 +-----------------------------------------------------------------+
| O O O O O |
| O O OO O O O O O O O O O O O O OO O O |
2e+08 |-+ O |
| |
| |
1.5e+08 |-+ .+ .+. .+.+. .+. |
|.+.+.+.+.+.+.+ + +.+.+.+.+.+.+.+.+.+.+.+.+.+.++ +.+.+ +.|
1e+08 |-+ |
| |
| |
5e+07 |-+ |
| |
| |
0 +-----------------------------------------------------------------+
fio.time.file_system_inputs
2.5e+09 +-----------------------------------------------------------------+
| |
| + |
2e+09 |.+ .+.+.+.+. + +.+. .+.+.+. .+.+.+. .+.+. .+. .+. .+.+.|
| +.+ + +.+ +.+ +.+ ++ +.+ + |
| O O O O OO O O O O O O O O O O O |
1.5e+09 |-+ O O O O O O O O O |
| |
1e+09 |-+ |
| |
| |
5e+08 |-+ |
| |
| |
0 +-----------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
2020년 2월 28일 (금) 오후 4:42, kernel test robot <[email protected]>님이 작성:
>
> Greeting,
>
> FYI, we noticed a 19.5% improvement of fio.read_bw_MBps due to commit:
Hello, all.
Please forget this improvement report.
My revision 1 patchset has a bug on this patch and it looks like it
causes some improvement. :)
I have fixed this bug at revision 2 patchset.
Thanks.