Received-SPF: pass (google.com: domain of linux-kernel+bounces-108022-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Date: Tue, 19 Mar 2024 14:32:17 -0400
From: Gregory Price <gregory.price@memverge.com>
To: Matthew Wilcox <willy@infradead.org>
Cc: Gregory Price <gourry.memverge@gmail.com>, linux-mm@kvack.org,
	linux-api@vger.kernel.org, linux-arch@vger.kernel.org,
	linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org,
	ying.huang@intel.com, dan.j.williams@intel.com, honggyu.kim@sk.com,
	corbet@lwn.net, arnd@arndb.de, luto@kernel.org,
	akpm@linux-foundation.org, shuah@kernel.org
Subject: Re: [RFC v3 3/3] ktest: sys_move_phys_pages ktest
Message-ID: <ZfnaMa6x/O68ENsP@memverge.com>
References: <20240319172609.332900-1-gregory.price@memverge.com>
 <20240319172609.332900-4-gregory.price@memverge.com>
 <ZfnQ7n_7cZvk9BkC@casper.infradead.org>
 <ZfnWCRwcZJ4KBmSH@memverge.com>
 <ZfnXcbd3h3Rj4IIS@casper.infradead.org>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <ZfnXcbd3h3Rj4IIS@casper.infradead.org>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?Op609ko6MWsmhA78P7+5bOSgdqfbk2oC6xZYeuolw8HPRcMrxrA5jgne91zZ?=
 =?us-ascii?Q?bCgEYts/AvSfhjejvkPSi55zKOMgbCwLP1NKzqr5AfPkWehjtVldiIdzr0GX?=
 =?us-ascii?Q?kHcfAhjnnO/NMS6GTwjC4NqyEYbfIy0vGJ+balICYbqm6gYcJfeMqAQiGbnk?=
 =?us-ascii?Q?ZnWDMaC86VQcMQq/TsMnvfPsOApzx/GiYPuUKPy48vWMqCQdXkE/+zsgUd6T?=
 =?us-ascii?Q?7huA8MImlsn8SnPCThsBZASbv/GXl60o2uWHGhHBfYOsC4+Vl9zCWMy3LNho?=
 =?us-ascii?Q?31LTPmQ+tOaWeuH9Lc4Zj0bsIr8uG9RsB4OVFpyAQzJMrOLgaFKnJ/xAFuWq?=
 =?us-ascii?Q?gj0/EXcMgqBA8DVa74JNRrtWpqGtJOUgb+PYu6SW6G1XXIg3/HYkyjqIR3Wn?=
 =?us-ascii?Q?hSb83jVClAK0FqFxx3azoZVA+yCVqXf9e6kzz7/HTi2qlwVRukKFVQepSjtI?=
 =?us-ascii?Q?WE1xC2qOZWAAEAdqcdJ1G7FGtkgdcyLC13wfwA6C69/4d4sdupvsyTlRMR9w?=
 =?us-ascii?Q?m5U12EtIGhVIlffRM6j50yhSmOJSvfGj3nbumvsm7GFwb8qRpk5ScZ0ogo9T?=
 =?us-ascii?Q?Gw8XVwM+RBt85vZNYd19F4InSzg4axTlFlPyphkzJYtPrE8e8X1CBDSrcc8/?=
 =?us-ascii?Q?kbwgd4+WboEl0LXp1HJ8kQfNcssNdxxafDw2eohSj4j53YD4Jn89t4eW5V2h?=
 =?us-ascii?Q?JZnmMVWcA5cvk80XfGrtPTF3MudjDU4lqOo0sNyXv73Zg/73nsI6oO2sKBwI?=
 =?us-ascii?Q?K1xsvGZFMITb9WkWcCYdi6v06ZQHqsM+UCSrH7THRBEvnfPaVDoLQj/tXyAR?=
 =?us-ascii?Q?RlDVsu8mbV/JsMDTvV9DyTxQ1+mk1W71geQfBTrPtn+GMujHvqOCltM37uzY?=
 =?us-ascii?Q?8BHkY8fUcRy0xZ3eWuocQj/sAzdBAaKUW/x7mXuiqW9GxuRucfu0L9cAHDSO?=
 =?us-ascii?Q?KYMNFRfD6xA+V8E/lla/C9FPhtgQVhYglOoGA147JCoWF3y/1xrj19Jx3C9n?=
 =?us-ascii?Q?75SkFXUltkx+i1wnvZJsExWeVpMoPDfG3ucFw1S+B4Uk/Gwd2nfD47r6ejoR?=
 =?us-ascii?Q?1BJVY/TsgEq8M30O3h3DGdKzZXh5bO9l0y95ZqwNE+zN+uT5o3w/QA9ACh8R?=
 =?us-ascii?Q?e6pX4Y2Z9U387lEgOXQ+AtIJOH1TIwMttZLMYzFaIGDStbd/JgrkPTssZaTo?=
 =?us-ascii?Q?AD9tci6aDqO8WlMd37ChkgmLDBBJ4uZRC3DlnmHjC9qbH3F0piEh7idpF7pj?=
 =?us-ascii?Q?aV02+FXunf9kaBrnL0RfGi11XpEhnFPS4Ai5iZ/oH5G2GMtjGykvBDEcPD9P?=
 =?us-ascii?Q?RiJS8QFzw6NoTeasxhk5BYXOXvEDElxVfn+OgJH5y3AP4MIA46tWedDtnTS8?=
 =?us-ascii?Q?l/fZ1F5zXf3ERPJp7kCXjRt6q/nXjQha5lMjAt1KInYtLhKYkCdnlNbVbsQy?=
 =?us-ascii?Q?SEGPdHlqHK4jNQXLuHL1h0QHmVFleCI/fpU9YpkUwZGgYq2gjZ0PtbqmzJBm?=
 =?us-ascii?Q?U34CImzhTgm9gP9t8u95RvsG7ezuBkK6l5aD7v9jRccGpormwQPz2NmydAjM?=
 =?us-ascii?Q?FZq1Ux4JYgRYD2uZTR69R6N5psfntGpEZc+d4fFGLEP3JYLZKfvEcx6zh2U5?=
 =?us-ascii?Q?OA=3D=3D?=
X-OriginatorOrg: memverge.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a2dc5df7-9846-447e-3a28-08dc4842ecc6
X-MS-Exchange-CrossTenant-AuthSource: SJ0PR17MB5512.namprd17.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Mar 2024 18:32:25.8840
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 5c90cb59-37e7-4c81-9c07-00473d5fb682
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: yYLxySmzOpQiptwxMEWtFPcFzm735g4X8+nw+TSXisvY6J2djBl9EZ9IubbrPTS4JwfzUi4HqSDMKg6tEO6pO8x6G34/EIvlmn3ZXobhK6E=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR17MB6241

On Tue, Mar 19, 2024 at 06:20:33PM +0000, Matthew Wilcox wrote:
> On Tue, Mar 19, 2024 at 02:14:33PM -0400, Gregory Price wrote:
> > On Tue, Mar 19, 2024 at 05:52:46PM +0000, Matthew Wilcox wrote:
> > > On Tue, Mar 19, 2024 at 01:26:09PM -0400, Gregory Price wrote:
> > > > Implement simple ktest that looks up the physical address via
> > > > /proc/self/pagemap and migrates the page based on that information.
> > > 
> > > What?  LOL.  No.
> > > 
> > 
> > Certainly the test is stupid and requires admin, but I could not
> > come up an easier test to demonstrate the concept - and the docs
> > say to include a test with all syscall proposals.
> > 
> > Am I missing something else important?
> > (stupid question: of course I am, but alas I must ask it)
> 
> It's not that the test is stupid.  It's the concept that's stupid.

Ok i'll bite.

The 2 major ways page-hotness is detected right now is page-faults
(induced or otherwise) and things like IBS/PEBS.

page-faults cause overhead, and IBS/PEBS actually miss upwards of ~66%
of all traffic (if you want the details i can dig up the presentation,
but TL;DR: prefetcher traffic is missed entirely).

so OCP folks have been proposing hotness-tracking offloaded to the
memory devices themselves:

https://www.opencompute.org/documents/ocp-cms-hotness-tracking-requirements-white-paper-pdf-1

(it's come along further than this white paper, but i need to dig up
the new information).

These devices are incapable of providing virtual addressing information,
and doing reverse lookups of addresses is inordinately expensive from
user space.  This leaves: Do it all in a kernel task, or give user space
an an interface to operate on data provided by the device.

The syscall design is mostly being posted right now to collaborate via
public channels, but if the idea is so fundamentally offensive then i'll
drop it and relay the opinion accordingly.

~Gregory