Received-SPF: pass (google.com: domain of linux-kernel+bounces-82723-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Date: Tue, 27 Feb 2024 00:36:56 -0500
From: Gregory Price <gregory.price@memverge.com>
To: "Huang, Ying" <ying.huang@intel.com>
Cc: Gregory Price <gourry.memverge@gmail.com>, linux-mm@kvack.org,
	linux-kernel@vger.kernel.org, hannes@cmpxchg.org,
	dan.j.williams@intel.com, dave.jiang@intel.com
Subject: Re: [RFC 1/1] mm/mempolicy: introduce system default interleave
 weights
Message-ID: <Zd10+G4XIrPoojJE@memverge.com>
References: <20240220202529.2365-1-gregory.price@memverge.com>
 <20240220202529.2365-2-gregory.price@memverge.com>
 <87wmqxht4c.fsf@yhuang6-desk2.ccr.corp.intel.com>
 <ZdgxaLSBznupVmJK@memverge.com>
 <87sf1jh7es.fsf@yhuang6-desk2.ccr.corp.intel.com>
 <ZdygZ8ZidfaORg8F@memverge.com>
 <87edcyeo78.fsf@yhuang6-desk2.ccr.corp.intel.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <87edcyeo78.fsf@yhuang6-desk2.ccr.corp.intel.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?wcflbr0mrgJ8UcSFqL4RYiIis3sxQhAX6miXRbbjsiW6tdxZuOq6VPvW+R7V?=
 =?us-ascii?Q?Se1xk6u0+ugGY9P/nrmcNEFuAOudlVZqJh5J5lTH0vsCs88816fFddxpCT/N?=
 =?us-ascii?Q?w9Dlj1yA+SCCSgEu1f9yNase8wY+FIY5dksJf6ikuIpnmuzQqB3yTd9Xn5JL?=
 =?us-ascii?Q?qeoourg+MOo70tVCppwana33qzlUDrfaZqzSrsBF2BU9Yn4JMgXuRfNpnGm3?=
 =?us-ascii?Q?n4Jy4tPqTsKFyMuv+ax2JKVNP3k5LfGWgJzSb0I3R5kJy4DshDCDmjz62gEC?=
 =?us-ascii?Q?rG1wlED7CZQeUX3G74igcyAp4wGY4cPrxm9pUuMgIcpLEfFrTVkWunCIPgTQ?=
 =?us-ascii?Q?nkko6aTIvJP2bmhWnCL/rhacZARf1nilODeR7+4I9OACvWf7We0d35Rd/TKh?=
 =?us-ascii?Q?RYzJ8iNmZvRXimGPMi01mjpXg8+A1eA6f7narSXOW5AN3fAFLGxXSrIHDPm4?=
 =?us-ascii?Q?9vNJn3xYG23UePEVm4GVoP1nElqqIdXKVOwg0XQVmQwPuL7Abp51p4jw9WDa?=
 =?us-ascii?Q?LL0YIg7oqIM+yjkWC2oFB6gaxocty7PZ+QwCYolmhL+lMYTb3CRDwVO+wX4I?=
 =?us-ascii?Q?6dB09OAoj5NHcSQnadxfnIi6Sy0EeisDAgaZ5Y4gizw739MJkrW1lGZwMTf/?=
 =?us-ascii?Q?aEebTZ88Om6yTUvlmlwZh1BbTmldAAkP/W5rqcPP1HOVVPGiu+9jDaPmuWlZ?=
 =?us-ascii?Q?7aVBYDZ67+y6TBpDROhD0ICSULnpAtCLT/9qtT76p2ECgTXBwvreBui9ggMc?=
 =?us-ascii?Q?jrPEoD46B55acB2vdSTA55JA54t3SAB6AuGqqmhWSLcTErq2Qy+e6XRleivJ?=
 =?us-ascii?Q?pOt/yvahkdl9dMhfHB7Wecv9EMWDf0KG46sFahJhVR8o8e1FNrRrP6ygASKh?=
 =?us-ascii?Q?aGAAFLbBLFkpF5wlZzWXCoXfOIxusj6aRKl50PGqQw0FhCF3jASg0PsUBZNj?=
 =?us-ascii?Q?lOPH5WdShNzC8hyi0+SRxxqTv6+6SR/z0So4L2BxFnodF0Pv39316yRbUFIp?=
 =?us-ascii?Q?38JCN8P80HnhPM/o+PUjE3bMAgrMwQ7orZHTQWYvKbLh2ilf9cGCsSCRB+8Z?=
 =?us-ascii?Q?LnH7GD8ONX62AanEYHeQvnmMrgOkQhQ2Un05LaL0EY5KNzx5CM9wtL8RFeNr?=
 =?us-ascii?Q?cNB0s7vuckTed3NkDMM0d2f/+2jWtLzyvALcfywW1D5Jt+b5nkTU9aw9rWdC?=
 =?us-ascii?Q?WQZrkMjdLGS6dNMamHNtAsEDF1icmhMbtWuTnW6ZZhU/fLVR23sPbt6wU+vd?=
 =?us-ascii?Q?PiSMcf7uYEFDLeBSYjVWxHg1b/NSHP3UuYi1xZu3EKV6hC6wEOAvP2kDA4of?=
 =?us-ascii?Q?k7fw5Dz3A8K1QkLNNvwZUb46DTnPS85WunOVTOwE6VIcWoMyIIAeZfyPqe2G?=
 =?us-ascii?Q?OiE69afJ2GPgsQHmFJMES3acBCcFTxB76hguBjeBgekODkvZ+hneXRFMihbm?=
 =?us-ascii?Q?eku4IIep/F2ZhUWLYVHhH/S6rVBGFNv6upFCch8+VDSfjXAPMPONOgyO7WPl?=
 =?us-ascii?Q?L4f1U0AOaXJm//2qwjEaQEdy9Cjp9vOxSY/ZZdAN62DH8FC8Rzq+fFw+pCwp?=
 =?us-ascii?Q?C2rGPN93tWbyxGsG/20zS+bz+fhZ6d2++rlyE2SLwH+AVq5EpMnZnVDcY4T3?=
 =?us-ascii?Q?XQ=3D=3D?=
X-OriginatorOrg: memverge.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e58cdd02-1c2f-4889-7f56-08dc37562108
X-MS-Exchange-CrossTenant-AuthSource: SJ0PR17MB5512.namprd17.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Feb 2024 05:37:04.3699
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 5c90cb59-37e7-4c81-9c07-00473d5fb682
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 6pcQVdkb7kpxrFwGo/EPLWqz5gfdVsRog8t4UK9qOvzaEimXrbHjiJsKVlthcgbnTecpqKKMYhsB3Sk5+Zrof6i6kryH5vLC6LmVUWeGflg=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR17MB5183

On Tue, Feb 27, 2024 at 08:38:19AM +0800, Huang, Ying wrote:
> Gregory Price <gregory.price@memverge.com> writes:
> > Where are the 100 nodes coming from?
> 
> If you have a real large machine with more than 100 nodes, and some of
> them are CXL memory nodes, then it's possible that most nodes will have
> interleave weight "1" because the sum of all interleave weights is
> "100".  Then, even if you use only one socket, the interleave weight of
> DRAM and CXL MEM could be all "1", lead to useless default value.  So, I
> suggest don't cap the sum of interleave weights.

I have to press this issue: Is this an actual, practical, concern?

It seems to me in this type of scenario, there are larger, more complex
numa topology issues that make the use of the general, global weighted
mempolicy system entirely impractical.  This is a bit outside the scope

> > So, long winded winded way of saying:
> > - Could we use a larger default number? Yes.
> > - Does that actually help us? Not really, we want smaller numbers.
> 
> The larger number will be reduced after GCD.
>

I suppose another strategy is to calculate the interleave weights
un-bounded from the raw bandwidth - but continuously force reductions
(through some yet-undefined algorithm) until at least one node reaches a
weight of `1`.  This suffers from the opposite problem: what if the top
node has a value greater than 255? Do we just cap it at 255? That seems
the opposite form of problematic.

(Large numbers are quite pointless, as it is essentially the antithesis
of interleave)

> > I think I'll draft up an LSF/MM chat to see if we can garner more input.
> > If large-numa systems are a real issue, then yes we need to address it.
> 
> Sounds good to me!

Working on it.

~Gregory