This patch series adds "smart scanning" for KSM.
What is smart scanning?
=======================
KSM evaluates all the candidate pages for each scan. It does not use historic
information from previous scans. This has the effect that candidate pages that
couldn't be used for KSM de-duplication continue to be evaluated for each scan.
The idea of "smart scanning" is to keep historic information. With the historic
information we can temporarily skip the candidate page for one or several scans.
Details:
========
"Smart scanning" is to keep two small counters to store if the page has been
used for KSM. One counter stores how often we already tried to use the page for
KSM and the other counter stores when a page will be used as a candidate page
again.
How often we skip the candidate page depends how often a page failed KSM
de-duplication. The code skips a maximum of 8 times. During testing this has
shown to be a good compromise for different workloads.
New sysfs knob:
===============
Smart scanning is not enabled by default. With /sys/kernel/mm/ksm/smart_scan
smart scanning can be enabled.
Monitoring:
===========
To monitor how effective smart scanning is a new sysfs knob has been introduced.
/sys/kernel/mm/pages_skipped report how many pages have been skipped by smart
scanning.
Results:
========
- Various workloads have shown a 20% - 25% reduction in page scans
For the instagram workload for instance, the number of pages scanned has been
reduced from over 20M pages per scan to less than 15M pages.
- Less pages scans also resulted in an overall higher de-duplication rate as
some shorter lived pages could be de-duplicated additionally
- Less pages scanned allows to reduce the pages_to_scan parameter
and this resulted in a 25% reduction in terms of CPU.
- The improvements have been observed for workloads that enable KSM with
madvise as well as prctl
Changes:
- V2:
- Renamed function inc_skip_age() to skip_age()
- Added comment to skip_age() function
- Renamed function skip_rmap_item() to should_skip_rmap_item()
- Added more comments to should_skip_rmap_item function
- Added explicit modification of age with overflow check
Stefan Roesch (4):
mm/ksm: add "smart" page scanning mode
mm/ksm: add pages_skipped metric
mm/ksm: document smart scan mode
mm/ksm: document pages_skipped sysfs knob
Documentation/admin-guide/mm/ksm.rst | 11 +++
mm/ksm.c | 120 +++++++++++++++++++++++++++
2 files changed, 131 insertions(+)
base-commit: 15bcc9730fcd7526a3b92eff105d6701767a53bb
--
2.39.3
This adds documentation for the smart scan mode of KSM.
Signed-off-by: Stefan Roesch <[email protected]>
Reviewed-by: David Hildenbrand <[email protected]>
---
Documentation/admin-guide/mm/ksm.rst | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/Documentation/admin-guide/mm/ksm.rst b/Documentation/admin-guide/mm/ksm.rst
index 776f244bdae4..1762219baf51 100644
--- a/Documentation/admin-guide/mm/ksm.rst
+++ b/Documentation/admin-guide/mm/ksm.rst
@@ -155,6 +155,15 @@ stable_node_chains_prune_millisecs
scan. It's a noop if not a single KSM page hit the
``max_page_sharing`` yet.
+smart_scan
+ By default KSM checks every candidate page for each scan. It does
+ not take into account historic information. When smart scan is
+ enabled, pages that have previously not been de-duplicated get
+ skipped. How often these pages are skipped depends on how often
+ de-duplication has already been tried and failed. By default this
+ optimization is disabled. The ``pages_skipped`` metric shows how
+ effetive the setting is.
+
The effectiveness of KSM and MADV_MERGEABLE is shown in ``/sys/kernel/mm/ksm/``:
general_profit
--
2.39.3