Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) client-ip=23.128.96.31;
Message-ID: <4509a3b4-16a6-f63e-1dd5-e20c7eadf87d@redhat.com>
Date:   Fri, 6 Oct 2023 14:01:09 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.15.1
Content-Language: en-US
To:     Stefan Roesch <shr@devkernel.io>, kernel-team@fb.com
Cc:     akpm@linux-foundation.org, hannes@cmpxchg.org, riel@surriel.com,
        linux-kernel@vger.kernel.org, linux-mm@kvack.org
References: <20231004190249.829015-1-shr@devkernel.io>
From:   David Hildenbrand <david@redhat.com>
Organization: Red Hat
Subject: Re: [PATCH v1 0/4] mm/ksm: Add ksm advisor
In-Reply-To: <20231004190249.829015-1-shr@devkernel.io>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 04.10.23 21:02, Stefan Roesch wrote:
> What is the KSM advisor?
> =========================
> The ksm advisor automatically manages the pages_to_scan setting to
> achieve a target scan time. The target scan time defines how many seconds
> it should take to scan all the candidate KSM pages. In other words the
> pages_to_scan rate is changed by the advisor to achieve the target scan
> time.
> 
> Why do we need a KSM advisor?
> ==============================
> The number of candidate pages for KSM is dynamic. It can often be observed
> that during the startup of an application more candidate pages need to be
> processed. Without an advisor the pages_to_scan parameter needs to be
> sized for the maximum number of candidate pages. With the scan time
> advisor the pages_to_scan parameter based can be changed based on demand.
> 
> Algorithm
> ==========
> The algorithm calculates the change value based on the target scan time
> and the previous scan time. To avoid pertubations an exponentially
> weighted moving average is applied.
> 
> The algorithm has a max and min
> value to:
> - guarantee responsiveness to changes
> - to avoid to spend too much CPU
> 
> Parameters to influence the KSM scan advisor
> =============================================
> The respective parameters are:
> - ksm_advisor_mode
>    0: None (default), 1: scan time advisor
> - ksm_advisor_target_scan_time
>    how many seconds a scan should of all candidate pages take
> - ksm_advisor_min_pages
>    minimum value for pages_to_scan per batch
> - ksm_advisor_max_pages
>    maximum value for pages_to_scan per batch
> 
> The parameters are exposed as knobs in /sys/kernel/mm/ksm.
> By default the scan time advisor is disabled.

What would be the main reason to not have this enabled as default?

IIUC, it is kind-of an auto-tuning of pages_to_scan. Would "auto-tuning" 
describe it better than "advisor" ?

[...]

> How is defining a target scan time better?
> ===========================================
> For an administrator it is more logical to set a target scan time.. The
> administrator can determine how many pages are scanned on each scan.
> Therefore setting a target scan time makes more sense.
> 
> In addition the administrator might have a good idea about the
> memory sizing of its respective workloads.

Is there any way you could imagine where we could have this just do 
something reasonable without any user input? IOW, true auto-tuning?

I read above:
 > - guarantee responsiveness to changes
 > - to avoid to spend too much CPU

whereby both things are accountable/measurable to use that as the input 
for auto-tuning?


I just had a family NMI, so my todo list is quite lengthy. Hoping I cna 
take a closer look next week.

-- 
Cheers,

David / dhildenb