Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) client-ip=2620:137:e000::3:8;
Message-ID: <54fa32a9-36a8-f296-9be8-d2b28770f82e@amd.com>
Date:   Wed, 13 Sep 2023 10:58:23 +0530
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Thunderbird/102.6.1
Subject: Re: [RFC PATCH V1 0/6] sched/numa: Enhance disjoint VMA scanning
To:     Raghavendra K T <raghavendra.kt@amd.com>,
        linux-kernel@vger.kernel.org, linux-mm@kvack.org
Cc:     Ingo Molnar <mingo@redhat.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Mel Gorman <mgorman@suse.de>,
        Andrew Morton <akpm@linux-foundation.org>,
        David Hildenbrand <david@redhat.com>, rppt@kernel.org,
        Juri Lelli <juri.lelli@redhat.com>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Bharata B Rao <bharata@amd.com>,
        Aithal Srikanth <sraithal@amd.com>,
        kernel test robot <oliver.sang@intel.com>,
        K Prateek Nayak <kprateek.nayak@amd.com>
References: <cover.1693287931.git.raghavendra.kt@amd.com>
Content-Language: en-US
From:   Swapnil Sapkal <Swapnil.Sapkal@amd.com>
In-Reply-To: <cover.1693287931.git.raghavendra.kt@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?YnlmeEQ5QWd1ZmhIUEtiUzhXQlNPTUJjRXpMQ3FqN2NkL2t3SnNHTk54bEtV?=
 =?utf-8?B?VE9TZDllb2NvZWxuVFd1QW1TcjFXc1R4WlpFc2xIZUxKb3pLcHJISnBMYTNs?=
 =?utf-8?B?NFNCdDR0eDA2RGI0TTFjM3VwVHRCYVFsRnNWa0tKVHFqMThPbGRmVXdPekNa?=
 =?utf-8?B?S0tlT1gyMk91SktDSEFpUmhRQkd2MitwbWk0MWQ2SnVGSHlPbFF1dHZ4UWdj?=
 =?utf-8?B?dDQ0aXViSE42ZVFhZHJ2WU5qZENIVHBOd2svZ1FObjZDYVQvMHg4ZS9UcTJz?=
 =?utf-8?B?RW1mY2I3cDFvVmhZY2l0K2gzc0c4SEsxRkJJTDNJMEMyZjlVcU9GNXN6czRS?=
 =?utf-8?B?eHN3bFlmN0kzek5zaHJDekt5V0xxdU5LRS9QUSt3a3IwVVh5UVFaeXIrampp?=
 =?utf-8?B?L1AvUTVKVWxlWDBQQTBOSmNFUjNSMGNQMTRXNlk3UGJBQitIS29DNm0vSU0y?=
 =?utf-8?B?aC9rVGFjeVBuYy8rWFd0cWE3UEpnc0ZxbVVZK3FsUWNjWTVRVTd3all0ZHpt?=
 =?utf-8?B?RE95elZqWSs5ckhMOXg4YWdvWGpyQmZCU1BibWtDTEFjd1M2ekx6OU1BTHA0?=
 =?utf-8?B?eC9rOENubWNnWTRHL0QxNm1tUjRGcThiSFArTHpXelNnUFZ3SnRqaG01TEl3?=
 =?utf-8?B?NTh5d0hXaFpYVHdlYVpmRndxTjR0WENaVEVFNGVvVFZHVmtQNjFmNVFaSlBx?=
 =?utf-8?B?aVpCUlAwUzFFcUZXSFBsVGFPYmcyTldGVWdrdFIwRzBjTVBnN3RTamlnN0g3?=
 =?utf-8?B?bk51V2pPaGxSbXphSEZvSUsyOERmZ3U1dSt3QTFDM3ZBTVZodnYweHZnTC9P?=
 =?utf-8?B?MmpHc3NUeEE4b3ZGbnlyOGQyQWQvalRKWFVHeEJqUlNaTUJwa0tGdjBzeEw2?=
 =?utf-8?B?VG8xaXA4Z01kTGkyRlFnWCtPbnlOL1V0WHJEa1dWMmNOL0ZXSkhqTERoN1lN?=
 =?utf-8?B?V21YUFBEK3N3WE5KVExhQXd0ei9QZlBPYi9VY1NJZCtZRnVsMmthUGt0UU1B?=
 =?utf-8?B?TWFJaFBGVEZBTFQ0QkprV0tTOEFFaGlPbEZTVUl4djdEQjB1dmx6OGlrWmND?=
 =?utf-8?B?a0RQMDB2ZS9VTGdDOFV2YWh5S1FWRTBUUWdtQTJ3dkRMRStxWnE1Skk0TEsw?=
 =?utf-8?B?S0xiNnB2K2RSTUJGWnF2cHBpaFRUU2VIbUtreERlR2RmYXhkUnpvUHNOWjNK?=
 =?utf-8?B?eUJUd25YSzhrM3NUdFUrSUFSNTRYVkhwK1JWbGtoaEN1Q3BtTFZxMUF5SnVM?=
 =?utf-8?B?S0RTRFMvWTlHRGsvRHd3Q3NTeDVieGIwckUzT29jVjhKQVJqUDV4RlM2V21j?=
 =?utf-8?B?TWdNYmdlV1p3TDFORWdBM0prY0FsbFord0VuQUczbXM2bWFlcllYQW5SaTky?=
 =?utf-8?B?RkFCNkFzOUFkZk1Ec1hkcjZXMFozWmVkZVkvb2RGOExhaDBBOWVEVVBoaXdv?=
 =?utf-8?B?NWtVeU5pTm1wOHVDdDRtWHFKRnBLVUxhVzNYQ0pkN1NybEtZcFBRZmJ3OFNX?=
 =?utf-8?B?N29nSFY2ZE02eStKZTMwVE41T21jSTZ0V2FoZm1uc2dNNTNwRlYvSzl0ZGNp?=
 =?utf-8?B?RHErbWhHYzhVcGVSYmRDT25xN2dFS0w0ZVJTOFNhQUUyUnNJY2tHUXdxUHo4?=
 =?utf-8?B?Mk5NOCtXbUozZ1AxaC9ycWJRMDNzZ0d2NVVDSjV1UXpMT1lNMW5JcFJtU2FF?=
 =?utf-8?B?V21pTFlJZnBOZEtJbmxVRHpGUUI4akt3Z3QvR0pXQ2ZJWGtTbVJZWXd2Zzl6?=
 =?utf-8?B?NkxLQXExejhRY3YzeFpremVDYktPV0pNNXQ0bWwxZzJHYWpLcjh1dTdMWWpl?=
 =?utf-8?B?a3hPZ1ZDbEhCMHlpeVpDV25IV2ZtbnRLZ1VPSHhaK1hVWjI2eWpFMEUwSXVL?=
 =?utf-8?B?bTZlWkh0SDRQcHlCQjFNdnYrMUd6MFNlSUZWSmY5MzVmSWdqYlBsUjV5WjFN?=
 =?utf-8?B?clRNTHpzRzdjRlgvV0tCZlRpQTJ1OU1IOVRoaVdFYndId1VuMFE5RGRBaC9W?=
 =?utf-8?B?cVFBWkNBUVRWN0FKTnpXNTRNOVIrQ2FYNmY1Wjd1U1lHaWpaUVcrNTVBS0w5?=
 =?utf-8?B?ZWFMUmE5N0RTUndzMFFNeHppWGNZWWtPOHlPU25EdkN4TmRLMmxKbEppbHdh?=
 =?utf-8?Q?4mmOYUn/GjDQ0a6ZLFV0id5I8?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ee7328f3-1ac4-45d8-b7a6-08dbb41a4852
X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB6165.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2023 05:28:38.2669
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: WvXBBF39CC7FM0S5am0SGz4qGVQ0WhJEq/T9yyr4vwjHbEAZO3NabXxubgKzRhVHtC7oeghe/U+qIckbwhnUJg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR12MB7784
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Tue, 12 Sep 2023 22:29:07 -0700 (PDT)
X-Spam-Status: No, score=-2.3 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS
	autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email

Hello Raghu,

On 8/29/2023 11:36 AM, Raghavendra K T wrote:
> Since commit fc137c0ddab2 ("sched/numa: enhance vma scanning logic") [1]
> VMA scanning is allowed if:
> 1) The task had accessed the VMA.
>   Rationale: Reduce overhead for the tasks that had not
> touched VMA. Also filter out unnecessary scanning.
> 
> 2) Early phase of the VMA scan where mm->numa_scan_seq is less than 2.
>   Rationale: Understanding initial characteristics of VMAs and also
>   prevent VMA scanning unfairness.
> 
> While that works for most of the times to reduce scanning overhead,
>   there are some corner cases associated with it.
> 
> This was found in an internal LKP run and also reported by [2]. There was
> an attempt to fix.
> 
> Link: https://lore.kernel.org/linux-mm/cover.1685506205.git.raghavendra.kt@amd.com/T/
> 
> This is a fully different series after Mel's feedback to address the issue
>   and also a continuation of enhancing VMA scanning for NUMA balancing.
> 
> Problem statement (Disjoint VMA set):
> ======================================
> Let's look at some of the corner cases with a below example of tasks and their
> access pattern.
> 
> Consider N tasks (threads) of a process.
> Set1 tasks accessing vma_x (group of VMAs)
> Set2 tasks accessing vma_y (group of VMAs)
> 
>               Set1                      Set2
>          -------------------         --------------------
>          | task_1..task_n/2 |       | task_n/2+1..task_n |
>          -------------------         --------------------	
>                   |                             |
>                   V                             V
>          -------------------         --------------------
>          |     vma_x       |         |     vma_y         |
>          -------------------         --------------------	
> 
> Corner cases:
> (a) Out of N tasks, not all of them gets fair opportunity to scan. (PeterZ).
> suppose Set1 tasks gets more opportunity to scan (May be because of the
> activity pattern of tasks or other reasons in current design) in the above
> example, then vma_x gets scanned more number of times than vma_y.
> 
> some experiment is also done here which illustrates this unfairness:
> Link: https://lore.kernel.org/lkml/c730dee0-a711-8a8e-3eb1-1bfdd21e6add@amd.com/
> 
> (b) Sizes of vmas can differ.
> Suppose size of vma_y is far greater than the size of vma_x, then a bigger
> portion of vma_y can potentially be left unscanned since scanning is bounded
> by scan_size of 256MB (default) for each iteration.
> 
> (c) Highly active threads trap a few VMAs frequently, and some of the VMAs not
> accessed for long time can potentially get starved of scanning indefinitely
> (Mel). There is a possibility of lack of enough hints/details about VMAs if it
> is needed later for migration.
> 
> (d) Allocation of memory in some specific manner (Mel).
> One example could be, Suppose a main thread allocates memory and it is not
> active. When other threads tries to act upon it, they may not have much
> hints about it, if the corresponding VMA was not scanned.
> 
> (e) VMAs that are created after two full scans of mm (mm->numa_scan_seq > 2)
> will never get scanned. (Observed rarely but very much possible depending on
> workload behaviour).
> 
> Above this, a combination of some of the above (e.g., (a) and (b)) can
> potentially amplifyi/worsen the side effect.
> 
> This patchset, tries to address the above issues by enhancing unconditional
> VMA scanning logic.
> 
> High level ideas:
> =================
> Idea-1) Depending on vma_size, populate a per vma_scan_select value, decrement it
> and when it hits zero do force scan (Mel).
> vma_scan_select value is again repopulated when it hits zero.
> 
> This is how VMA scanning phases looks like after implementation:
> 
> |<---p1--->|<-----p2----->|<-----p2----->|...
> 
> Algorithm:
> p1: New VMA, initial phase do not scan till scan_delay.
> 
> p2: Allow scanning if the task has accessed VMA or vma_scan_select hit zero.
> 
> Reinitialize vma_scan_select and repeat p2.
> 
> pros/cons:
> +  : Ratelimiting is inbuilt to the approach
> +  : vma_size is taken into account for scanning
> +/-: Scanning continues forever
> -  : Changes in vma size is taken care after force scan. i.e.,
>     vma_scan_select is repopulated only after vma_scan_select hits zero.
> 
> Idea-1 can potentially cover all the issues mentioned above.
> 
> Idea-2) Take bitmask_weight of latest access_pids value (suggested by Bharata).
> If number of tasks accessing vma is >= 1, unconditionally allow scanning.
> 
> Idea-3 ) Take bitmask_weight of access_pid history of VMA. If number of tasks
> accessing VMA is > THRESHOLD (=3), unconditionally allow scanning.
> 
> Rationale (Idea-2,3): Do not miss out scanning of critical VMAs.
> 
> Idea-4) Have a per vma_scan_seq. allow the unconditional scan till vma_scan_seq
> reaches a value proportional (or equal) to vma_size/scan_size.
> This a complimentary to Idea-1.
> 
> this is how VMA scanning phases looks like after implementation:
> 
> |<--p1--->|<-----p2----->|<-----p3----->|<-----p4----->...||<-----p2----->|<-----p3----->|<-----p4-----> ...||
>                                                          RESET                                               RESET
> Algorithm:
> p1: New VMA, initial phase do not scan till scan_delay.
> 
> p2: Allow scanning if task has accessed VMA or vma_scan_seq has reached till
>   f(vma_size)/scan_size) for e.g., f = 1/2 * vma_size/scan_size.
> 
> p3: Allow scanning if task has accessed VMA or vma_scan_seq has reached till
>   f(vma_size)/scan_size in a rate limited manner. This is an optional phase.
> 
> p4: Allow scanning iff task has accessed VMA.
> 
> Reset after p4 (optional).
> 
> Repeat p2, p3 p4
> 
> Motivation: Allow agressive scanning in the beginning followed by a rate
> limited scanning. And then completely disallow scanning to avoid unnecessary
> scanning. Reset time could be a function of scan_delay and chosen long enough
> to aid long running task to forget history and start afresh.
> 
> +  : Ratelimiting need to be taken care separately if needed.
> +/-: Scanning continues only if RESET of vma_scan_seq is implemented.
> +  : changes in vma size is taken care in every scan.
> 
>   Current patch series implements Ideas 1, 2, 3 + extension of access PID history
> idea from PeterZ.
> 
> Results:
> ======
> Base: 6.5.0-rc6+ (4853c74bd7ab)
> SUT: Milan w/ 2 numa nodes 256 cpus
> 
> mmtest		numa01_THREAD_ALLOC manual run:
> 
> 		base		patched
> real		1m22.758s	1m9.200s
> user		249m49.540s	229m30.039s
> sys		0m25.040s	3m10.451s
> 	
> numa_pte_updates 	6985	1573363
> numa_hint_faults 	2705	1022623
> numa_hint_faults_local 	2279	389633
> numa_pages_migrated 	426	632990
> 
> kernbench
> 			base			patched
> Amean     user-256    21989.09 (   0.00%)    21677.36 *   1.42%*
> Amean     syst-256    10171.34 (   0.00%)    10818.28 *  -6.36%*
> Amean     elsp-256      166.81 (   0.00%)      168.40 *  -0.95%*
> 
> Duration User       65973.18    65038.00
> Duration System     30538.92    32478.59
> Duration Elapsed      529.52      533.09
> 
> Ops NUMA PTE updates                  976844.00      962680.00
> Ops NUMA hint faults                  226763.00      245620.00
> Ops NUMA pages migrated               220146.00      207025.00
> Ops AutoNUMA cost                       1144.84        1238.77
> 
> Improvements in other benchmarks I have tested.
> Time based:
> Hashjoin	4.21%
> Btree	 	2.04%
> XSbench		0.36%
> 
> Throughput based:
> Graph500 	-3.62%
> Nas.bt		3.69%
> Nas.ft		21.91%
> 
> Note: VMA scanning improvements [1] has refined scanning so much that
> system overhead we re-introduce with additional scan look glaringly
> high. But If we consider the difference between before [1] and current
> series, overall scanning overhead is considerably reduced.
> 
> 1. Link: https://lore.kernel.org/lkml/cover.1677672277.git.raghavendra.kt@amd.com/T/#t
> 2. Link: https://lore.kernel.org/lkml/cover.1683033105.git.raghavendra.kt@amd.com/
> 
> Note: Patch description is again repeated in some patches to avoid any
> need to copy from cover letter again.
> 
> Peter Zijlstra (1):
>    sched/numa: Increase tasks' access history
> 
> Raghavendra K T (5):
>    sched/numa: Move up the access pid reset logic
>    sched/numa: Add disjoint vma unconditional scan logic
>    sched/numa: Remove unconditional scan logic using mm numa_scan_seq
>    sched/numa: Allow recently accessed VMAs to be scanned
>    sched/numa: Allow scanning of shared VMAs
> 
>   include/linux/mm.h       |  12 +++--
>   include/linux/mm_types.h |   5 +-
>   kernel/sched/fair.c      | 109 ++++++++++++++++++++++++++++++++-------
>   3 files changed, 102 insertions(+), 24 deletions(-)
> 

I have tested this series on 4th generation EPYC processor. I am seeing improvement in autonuma-benchmark with the series.

o System Details

- 4th Generation EPYC System
- 2 x 128C/256T
- NPS1 mode

o Kernels

base:   4853c74bd7ab Merge tag 'parisc-for-6.5-rc7' of git://git.kernel.org/pub/scm/linux/kernel/git/deller/parisc-linux


==================================================================
Test          : autonuma-benchmark
Units         : Time in seconds
Interpretation: Lower is better
Statistic     : AMean
==================================================================
commit:
   base (4853c74bd7ab)
   base + this_series

   base (4853c74bd7ab)         base + this_series
---------------- ---------------------------
          %stddev     %change         %stddev
              \          |                \
     522.58           -11.2%     464.23        autonuma-benchmark.numa01.seconds
     273.93            -1.2%     270.75        autonuma-benchmark.numa01_THREAD_ALLOC.seconds
       0.60            +0.0%       0.60        autonuma-benchmark.numa02.seconds
     102.68            +3.7%     106.50        autonuma-benchmark.numa02_SMT.seconds

Tested-by: Swapnil Sapkal <Swapnil.Sapkal@amd.com>

--
Thanks and Regards,
Swapnil