Received-SPF: pass (google.com: domain of linux-kernel+bounces-188492-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
From: Chengming Zhou <chengming.zhou@linux.dev>
Subject: [PATCH 0/4] mm/ksm: cmp_and_merge_page() optimizations and cleanup
Date: Fri, 24 May 2024 16:56:49 +0800
Message-Id: <20240524-b4-ksm-scan-optimize-v1-0-053b31bd7ab4@linux.dev>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
To: Andrew Morton <akpm@linux-foundation.org>, david@redhat.com, 
 aarcange@redhat.com, hughd@google.com, shr@devkernel.io
Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, 
 zhouchengming@bytedance.com, Chengming Zhou <chengming.zhou@linux.dev>

Hello,

This series mainly optimizes cmp_and_merge_page() to have more efficient
separate code flow for ksm page and non-ksm anon page.

- ksm page: don't need to calculate the checksum obviously.
- anon page: don't need to search stable tree if changing fast and try
  to merge with zero page before searching ksm page on stable tree.

Please see the patch-2 for details.

Patch-3 is cleanup also a little optimization for the chain()/chain_prune
interfaces, which made the stable_tree_search()/stable_tree_insert() over
complex.

In patch-4, fix behaviors in stable_tree_search() when handle migrating
stable_node: return the migrated ksm page if no shareable ksm page found
on the stable tree, so our rmap_item can be added directly.

I have done simple testing using "hackbench -g 1 -l 300000" (maybe I need
to use a better workload) on my machine, have seen a little CPU usage
decrease of ksmd and some improvements of cmp_and_merge_page() latency:

Before:

- ksm page
[128, 256)            21 |                                                    |
[256, 512)         12509 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[512, 1K)            769 |@@@                                                 |
[1K, 2K)              99 |                                                    |
[2K, 4K)               4 |                                                    |
[4K, 8K)               2 |                                                    |
[8K, 16K)              8 |                                                    |

- anon page
[512, 1K)             19 |                                                    |
[1K, 2K)            7160 |@@@@@@@@@@@                                         |
[2K, 4K)           33516 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[4K, 8K)           33172 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ |
[8K, 16K)          11305 |@@@@@@@@@@@@@@@@@                                   |
[16K, 32K)          1303 |@@                                                  |
[32K, 64K)            16 |                                                    |
[64K, 128K)            6 |                                                    |
[128K, 256K)           6 |                                                    |
[256K, 512K)           9 |                                                    |
[512K, 1M)             3 |                                                    |
[1M, 2M)               2 |                                                    |
[2M, 4M)               1 |                                                    |

After:

- ksm page
[128, 256)             9 |                                                    |
[256, 512)           915 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[512, 1K)             41 |@@                                                  |
[1K, 2K)               1 |                                                    |
[2K, 4K)               1 |                                                    |

- anon page
[512, 1K)            374 |                                                    |
[1K, 2K)            5367 |@@@@                                                |
[2K, 4K)           64362 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[4K, 8K)           27721 |@@@@@@@@@@@@@@@@@@@@@@                              |
[8K, 16K)           1047 |                                                    |
[16K, 32K)            63 |                                                    |
[32K, 64K)             7 |                                                    |
[64K, 128K)            6 |                                                    |
[128K, 256K)           5 |                                                    |
[256K, 512K)           3 |                                                    |
[512K, 1M)             1 |                                                    |

We can see the latency of cmp_and_merge_page() when handling non-ksm
anon pages has been improved.

Thanks for review and comments!

Signed-off-by: Chengming Zhou <chengming.zhou@linux.dev>
---
Chengming Zhou (4):
      mm/ksm: refactor out try_to_merge_with_zero_page()
      mm/ksm: don't waste time searching stable tree for fast changing page
      mm/ksm: optimize the chain()/chain_prune() interfaces
      mm/ksm: use ksm page itself if no another ksm page is found on stable tree

 mm/ksm.c | 266 ++++++++++++++++++++-------------------------------------------
 1 file changed, 84 insertions(+), 182 deletions(-)
---
base-commit: 2218eca02bc4203f68b8fb7e1116e5a2601506d1
change-id: 20240524-b4-ksm-scan-optimize-d2fd9401c357

Best regards,
-- 
Chengming Zhou <chengming.zhou@linux.dev>