Received-SPF: pass (google.com: domain of linux-kernel+bounces-16053-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Date: Wed, 3 Jan 2024 17:09:59 -0500
From: Gregory Price <gregory.price@memverge.com>
To: "Huang, Ying" <ying.huang@intel.com>
Cc: Gregory Price <gourry.memverge@gmail.com>, linux-mm@kvack.org,
	linux-doc@vger.kernel.org, linux-fsdevel@vger.kernel.org,
	linux-kernel@vger.kernel.org, linux-api@vger.kernel.org,
	x86@kernel.org, akpm@linux-foundation.org, arnd@arndb.de,
	tglx@linutronix.de, luto@kernel.org, mingo@redhat.com, bp@alien8.de,
	dave.hansen@linux.intel.com, hpa@zytor.com, mhocko@kernel.org,
	tj@kernel.org, corbet@lwn.net, rakie.kim@sk.com,
	hyeongtak.ji@sk.com, honggyu.kim@sk.com, vtavarespetr@micron.com,
	peterz@infradead.org, jgroves@micron.com, ravis.opensrc@micron.com,
	sthanneeru@micron.com, emirakhur@micron.com, Hasan.Maruf@amd.com,
	seungjun.ha@samsung.com,
	Srinivasulu Thanneeru <sthanneeru.opensrc@micron.com>
Subject: Re: [PATCH v5 02/11] mm/mempolicy: introduce
 MPOL_WEIGHTED_INTERLEAVE for weighted interleaving
Message-ID: <ZZXbN4+2nVbE/lRe@memverge.com>
References: <20231223181101.1954-1-gregory.price@memverge.com>
 <20231223181101.1954-3-gregory.price@memverge.com>
 <8734vof3kq.fsf@yhuang6-desk2.ccr.corp.intel.com>
 <ZYp6ZRLZQVtTHest@memverge.com>
 <878r58dt31.fsf@yhuang6-desk2.ccr.corp.intel.com>
 <ZZRybDPSoLme8Ldh@memverge.com>
 <87mstnc6jz.fsf@yhuang6-desk2.ccr.corp.intel.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <87mstnc6jz.fsf@yhuang6-desk2.ccr.corp.intel.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?zL8jtbJxVZBzaUDntqQj2TQ1mCTsdZoeTkdzEoLdP2cxLsPFcZBb2OfWDkqC?=
 =?us-ascii?Q?/5dsoztCCSn1q+YMlHbf70cGFMa1ZCjVi8AIhwxnULDKcQEMAQfJE7sj4ky4?=
 =?us-ascii?Q?NDM7pP99PDMetyfXbDr540lXh1xiD5HJtDc6CVS/0xl0ApmDOXh6mmHdSJN8?=
 =?us-ascii?Q?UWz+PjTyJgY+ooQAw30sXbRC5JzRhR3soShXavVTbAoPBiupbp0lRzvso0pe?=
 =?us-ascii?Q?7QOOH/eaptYoM2mV+oeebcNAw9WHXOQ7QgVBotb0eOHPKDr939beomYGAafN?=
 =?us-ascii?Q?93+PEGBQhFvEsi/IRECgV9+tPe4h9kyNMOWAq7viKXp7XF0aIO9n+fuCzU/f?=
 =?us-ascii?Q?WYOlfOlldfNixiqfvoixhbmI54MAfN9x8QGl38jPmvNultcSfDL54YVnbbPt?=
 =?us-ascii?Q?36henJSa0ERytLcdT0+apZD5wdMTfN0XNx5zz8aILmP7fIsWbwW2a4PkEVjL?=
 =?us-ascii?Q?X2TneNHwSB9+L6QKTXktb9B1Cx7vrwiorMZyXLb/CSDmiWPbD3V1y98Gfof5?=
 =?us-ascii?Q?mNM9XYRyi3ERviBAz9DH40r2mXIgxkEhNMYHESBS3Q0WDjxgUptAu/0SQ32x?=
 =?us-ascii?Q?030lyJ05VhreccplCIKZWX3GJ1LlRBsfbtsb77PnAbN7MnnEIAdiR/CGhWk7?=
 =?us-ascii?Q?minXJamTDGIefru5c6dSy3CdxRp+7fKUB5diGB/tAy491MvRNIn5Rr97Lyes?=
 =?us-ascii?Q?tOYBp4r4ePhKw6gZ1l8qZ3dbzi43yTDnoINdLF5bO5uwsRrPRCzyZJwt2F1e?=
 =?us-ascii?Q?vRCuYc6bvd+vLrgZIz6o+b5h7KIExKUQNsg79QF/rcIEjc7e/egU00tkZjXG?=
 =?us-ascii?Q?tL+NLT0naJlswUjYlLIRALLr91vqAEUn0FYDn8UjCSiroTXyNkdqCyh1sVtv?=
 =?us-ascii?Q?ZfoHIAaQ67MzImNpyS4nmkTHMudpaZVZIvU+jIGSD1LHi8lGBS2HsK9NS7TV?=
 =?us-ascii?Q?yJZnT/vSCtcoJL/ST4+l29RUZViveIDY0h0CPmh+thycQCAdaS76ay98p9FN?=
 =?us-ascii?Q?2j3K2Kb7ny9m1e7WY5VhZ8tf//gj0d3eWJE1PebkhYJoi8d2mSNdW7BRNwr/?=
 =?us-ascii?Q?1Z6eHOSyfo5/Iw8F3yzG79RfCWUD9JcUNIJFyOVpYPtJDIARUQkKkAkOfAhS?=
 =?us-ascii?Q?y/pbjsNqKb4ZB4/yYfFwx4GnQUoqMS6niZ4ZKPUzUBbZ7pNT6Pwh2KPC33YQ?=
 =?us-ascii?Q?HuCYVNijoLoAZDqaUHDKFu/FhxhE2BfR/TwpIWw7GPetglzZDe82CqiMuOCg?=
 =?us-ascii?Q?NM4PcycfF4WLwfdQZeUH5vRG1BlhZq1WEpDb07HrFySeJ7W14n2rOiEhHh8P?=
 =?us-ascii?Q?9eCgN48DBXLF5vjqNfxlufBZPeKemPEXj6GEU+n4GlnDaI/FOMO3qYpFTvv1?=
 =?us-ascii?Q?thHcsPFa7dYcwT13H+8MfLH0+gZ+OyAZvh8egVNHJOfJRISaXRT/hSt+eLd+?=
 =?us-ascii?Q?CXSufqxswfoGNBM5Mr8sM1Xn4l7LEVJMkqe+TlrPXeatvaShnazvpa6txLXM?=
 =?us-ascii?Q?dN8J8qNgpAxvGimUJSw9xDe5bcwHeA2Ci6NbPIzZX+HOXzxEMbhcAtp21b3I?=
 =?us-ascii?Q?xCxWyJ8a144zx+l7XKYxDZBftObAnH6E8b77l5reNu+XJY91huQnX69gmEU8?=
 =?us-ascii?Q?JQ=3D=3D?=
X-OriginatorOrg: memverge.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 04d55387-bc81-4944-03d3-08dc0ca8bfa9
X-MS-Exchange-CrossTenant-AuthSource: SJ0PR17MB5512.namprd17.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jan 2024 22:10:09.0564
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 5c90cb59-37e7-4c81-9c07-00473d5fb682
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: uZIVSZ7MvjqjEAL9nQR+KKWsRkzBVgMD4TZU01L1QQJpFP2aNz8PcaV1PU/sY/n506Zzb/sac4nncr/vGn/3uzIBWGUA6TYrhUh49Vc+yGs=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR17MB6786

On Wed, Jan 03, 2024 at 01:46:56PM +0800, Huang, Ying wrote:
> Gregory Price <gregory.price@memverge.com> writes:
> > I'm specifically concerned about:
> > 	weighted_interleave_nid
> > 	alloc_pages_bulk_array_weighted_interleave
> >
> > I'm unsure whether kmalloc/kfree is safe (and non-offensive) in those
> > contexts. If kmalloc/kfree is safe fine, this problem is trivial.
> >
> > If not, there is no good solution to this without pre-allocating a
> > scratch area per-task.
> 
> You need to audit whether it's safe for all callers.  I guess that you
> need to allocate pages after calling, so you can use the same GFP flags
> here.
> 

After picking away i realized that this code is usually going to get
called during page fault handling - duh.  So kmalloc is almost never
safe (or can fail), and we it's nasty to try to handle those errors.

Instead of doing that, I simply chose to implement the scratch space
in the mempolicy structure

mempolicy->wil.scratch_weights[MAX_NUMNODES].

We eat an extra 1kb of memory in the mempolicy, but it gives us a safe
scratch space we can use any time the task is allocating memory, and
prevents the need for any fancy error handling.  That seems like a
perfectly reasonable tradeoff.

> >
> > Weights are collected individually onto the stack because we have to sum
> > them up before we actually apply the weights.
> >
> > A stale weight is not offensive.  RCU is not needed and doesn't help.
> 
> When you copy weights from iw_table[] to stack, it's possible for
> compiler to cache its contents in register, or merge, split the memory
> operations.  At the same time, iw_table[] may be changed simultaneously
> via sysfs interface.  So, we need a mechanism to guarantee that we read
> the latest contents consistently.
> 

Fair enough, I went ahead and added a similar interaction.

~Gregoryg