Received-SPF: pass (google.com: domain of linux-kernel+bounces-82738-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161;
Date: Tue, 27 Feb 2024 01:11:50 -0500
From: Gregory Price <gregory.price@memverge.com>
To: "Huang, Ying" <ying.huang@intel.com>
Cc: Gregory Price <gourry.memverge@gmail.com>, linux-mm@kvack.org,
	linux-kernel@vger.kernel.org, hannes@cmpxchg.org,
	dan.j.williams@intel.com, dave.jiang@intel.com
Subject: Re: [RFC 1/1] mm/mempolicy: introduce system default interleave
 weights
Message-ID: <Zd19JvKrhMho20Fg@memverge.com>
References: <20240220202529.2365-1-gregory.price@memverge.com>
 <20240220202529.2365-2-gregory.price@memverge.com>
 <87wmqxht4c.fsf@yhuang6-desk2.ccr.corp.intel.com>
 <ZdgxaLSBznupVmJK@memverge.com>
 <87sf1jh7es.fsf@yhuang6-desk2.ccr.corp.intel.com>
 <ZdygZ8ZidfaORg8F@memverge.com>
 <87edcyeo78.fsf@yhuang6-desk2.ccr.corp.intel.com>
 <Zd10+G4XIrPoojJE@memverge.com>
 <87a5nme9c1.fsf@yhuang6-desk2.ccr.corp.intel.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <87a5nme9c1.fsf@yhuang6-desk2.ccr.corp.intel.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?kfmvDleHfAyT6XDjnPM6YJupn/3AoCxfPbTqdbDJM1DvpKRfuGsP4elrp4RZ?=
 =?us-ascii?Q?iZ3VqGF0MW9wqZvcWY71kSGnJ5Ysp9Hk2jA+slvDklwf9Z7NDHH5PG5VZWWU?=
 =?us-ascii?Q?qxeGYrURbAV/cNUJQhxNN1NKOrzIcEgNZ4/cDW/c1EnwnxOnJqBn16hRD36R?=
 =?us-ascii?Q?W5JZb5ZallItNiPtjwXRhPt+a9Ahl47iGZNhcKpPS+4VYjIaKyqBdr0baQRI?=
 =?us-ascii?Q?02qmcesca1y+RqI67JdrTyWpTfFntBmaO6My0hEyFw1V5GrtFu7cTkHOAsrX?=
 =?us-ascii?Q?XZ1qmfhGVB8kk7yHeZbmdXY2oVJTiE0LNH492uQ4JRRy6CSqsjW85SRH+YnE?=
 =?us-ascii?Q?krxBAqzoUKI2lTqEK6xDGa/9BqeBocnPem2PUdHuunHvbDq/gUxePnCf/Dc1?=
 =?us-ascii?Q?nmmIvjX8g7+Zf7JoWshio8N0lFn3KAU8RME1jdvJfPMoj+AjF2b6yVDoqox3?=
 =?us-ascii?Q?DLKiqWajMOqh49nSc1M5t9xsyLYJGxOqrYDWrbpcxe1kdeMuTEvdKzxZ9ism?=
 =?us-ascii?Q?GRnWoNauDuAbyCy8ys/rkSw3h6V9z7J4XfU8yUY/LA/XYQzqX2WvGztfXGLk?=
 =?us-ascii?Q?Gz6eqC/uLMLWnqlx4Iu8Sr3t6S1Ebyj0Bl4+Pr78LO6NZtIXkUfSTgtGM0u7?=
 =?us-ascii?Q?xT3DaKNpotfYZCiUITJLRrnTZ4QQBoFzjHsBZNEc/PEQ+tqRy21KiPzeW2nQ?=
 =?us-ascii?Q?q3uPtJb2zlzHz1Hxk2PdJm9aENBFPgwDITp3Ykhjb/rMwRlJXuzjfpI1kbC9?=
 =?us-ascii?Q?rf1fw+OQMg52/QUHCBBalA/jrnTtuIJf8HOR7ldNu5Lu12NnhP8weYxj7Owd?=
 =?us-ascii?Q?NaBq6jwi1NwSkKMzFnbDHeZh2D1cZHse7S/mWa1z/FpadhTGqpPNE6Bbw4J7?=
 =?us-ascii?Q?0hccRCuVdbrs0zPHGLX3N+EHz4dldJEP+u87yRpCl8lRPtnH9H659M002SWd?=
 =?us-ascii?Q?yKKpd9ESxCGUJF/vzgVolQOmlFEgPSikFCIam3kq78s5YGKE67uBQnoePO/U?=
 =?us-ascii?Q?UiwrItn5DGaan1yI3XRhUf+k5BS/OErsOsPSqLYd3lnLD2lED5FCrwcilk63?=
 =?us-ascii?Q?L68AeLW7E3sFo6s6KQe7I+l+0tdH5chWMj7LyhXC9N2/bx2N9Qj6AE/7NANb?=
 =?us-ascii?Q?D8MzwpIrfqSvqa31ZhWi5bnQb5OkNmoNZmBr0w2GC6bCKk+3ZyUXLnCZO6Bu?=
 =?us-ascii?Q?VBZn0egLvZNLNaJwasjLb5ktc/ACgt4/h173M4/JcsJo69qPXyUu2ijNI4tV?=
 =?us-ascii?Q?wFaKdsTvXLwdSgBYAQJgfFAxqtllZDkJlOxowMSmsQM+4/l9G1VmOYhD7WHE?=
 =?us-ascii?Q?uGtxbpXMqD2cW35a9YpDOi6PLf5t58T6Ujj0Rk7+CVfTGtHyCJXP4LvFcN4B?=
 =?us-ascii?Q?NYVY3zg2RQicuBYSOuxN0aU2eg4f8DsLi4J6natAATBanAnjCZQ0+rR7KzQy?=
 =?us-ascii?Q?O+5HvXQLe7TvYgOTJUKayopKhm8ggfC8OUsqOUy0v//uJPwnRMNbAxIEGG6+?=
 =?us-ascii?Q?DsAKR6srEHBtM+l7AO9fEjxagocWUXoQQUiJjwHyMeWH0P/DSJGwXTq6RyVa?=
 =?us-ascii?Q?3tH14o0h1qv2uuB6GX6tss1llyRFPLe1Ark60RoPLOEIasBL3ERbtXgmB5se?=
 =?us-ascii?Q?IA=3D=3D?=
X-OriginatorOrg: memverge.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e7508887-b014-4ee6-ad7c-08dc375b028f
X-MS-Exchange-CrossTenant-AuthSource: SJ0PR17MB5512.namprd17.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Feb 2024 06:12:00.6075
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 5c90cb59-37e7-4c81-9c07-00473d5fb682
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: g7tNIbTDpHXxDfo/kqy+JG45OCzaLJ2hFw7RWbVFnvqcA2V6ochz9IQzqi9ET6lW1KrJx7WgxJqBbq5f6Dn8ib6suS8DfBng/x8cZHNB0jA=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR17MB6246

On Tue, Feb 27, 2024 at 01:59:26PM +0800, Huang, Ying wrote:
> Gregory Price <gregory.price@memverge.com> writes:
> 
> > I have to press this issue: Is this an actual, practical, concern?
> 
> I don't know who have large machine like that.  But I guess that it's
> possible in the long run.
>

Certainly possible, although that seems like a hyper-specialized case of
a supercomputer.  I suppose still worth considering for a bit.

> > I suppose another strategy is to calculate the interleave weights
> > un-bounded from the raw bandwidth - but continuously force reductions
> > (through some yet-undefined algorithm) until at least one node reaches a
> > weight of `1`.  This suffers from the opposite problem: what if the top
> > node has a value greater than 255? Do we just cap it at 255? That seems
> > the opposite form of problematic.
> >
> > (Large numbers are quite pointless, as it is essentially the antithesis
> > of interleave)
> 
> Yes.  So I suggest to use a relative small number as the default weight
> to start with for normal DRAM.  We will have to floor/ceiling the weight
> value.

Yeah more concretely, I was thinking something like

unsigned int *temp_weights; /* sizeof nr_node_ids */

memcpy(temp_weights, node_bandwidth);
while min(temp_weights) > 1:
    - attempt GCD reduction
    - if failed (GCD=1), adjust all odd numbers to be even (+1), try again

for weight in temp_weights:
    iw_table[N] = (weight > 255) ? 255 : (unsigned char)weight;

Something like this.  Of course this breaks if you have two nodes with a
massively different bandwidth ratio (> 255:1), but that seems
unrealistic given the intent of the devices.

~Gregory