Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) client-ip=2620:137:e000::3:5;
Date:   Wed, 22 Nov 2023 18:53:42 -0500
From:   Gregory Price <gregory.price@memverge.com>
To:     Vinicius Petrucci <vpetrucci@gmail.com>
Cc:     akpm@linux-foundation.org, linux-mm@vger.kernel.org,
        linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org,
        linux-arch@vger.kernel.org, linux-api@vger.kernel.org,
        minchan@kernel.org, dave.hansen@linux.intel.com, x86@kernel.org,
        Jonathan.Cameron@huawei.com, aneesh.kumar@linux.ibm.com,
        ying.huang@intel.com, dan.j.williams@intel.com,
        hezhongkun.hzk@bytedance.com, fvdl@google.com, surenb@google.com,
        rientjes@google.com, hannes@cmpxchg.org, mhocko@suse.com,
        Hasan.Maruf@amd.com, jgroves@micron.com, ravis.opensrc@micron.com,
        sthanneeru@micron.com, emirakhur@micron.com,
        vtavarespetr@micron.com
Subject: Re: [RFC PATCH] mm/mbind: Introduce process_mbind() syscall for
 external memory binding
Message-ID: <ZV6Uhsg6WLBtNqU3@memverge.com>
References: <ZV5zGROLefrsEcHJ@r13-u19.micron.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <ZV5zGROLefrsEcHJ@r13-u19.micron.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?MKcQ4+Y+wOD2TMmkSwDmes5A14AGxMyrXaX7WxYogehH690jL2PAuvuZgfq/?=
 =?us-ascii?Q?YFg27lI4+NRsbKHfQcYAuCFBrKnoFPCGcndVMNwoCjmB5aFuOlzDJHlZs0vV?=
 =?us-ascii?Q?e3X5N5RJXfLSrInqvn6M0tbAanckIcM4ZzMCYviUK68WvbhnpQCQuqecRRgj?=
 =?us-ascii?Q?36ZaGF8tZu1HzcSGnB1HT/l+Fbb6KobiIkNzFBWxyD8uUafdpIDluJ+7+5yM?=
 =?us-ascii?Q?q33fxNKSMkLp04SDr27DeCKnJVZImdZctEAdxANHo/aDUzLcRupx8zUW80gx?=
 =?us-ascii?Q?LMSIAsuNqS7azE7BBEHXXVQ8xHWoaX3Q4G/4S+gZg0+XfLgCRtEnQ2Dvlqb/?=
 =?us-ascii?Q?Xe4M3t7zDLIjVcaQsbdGBVm3im8QT63nD8r3LvqdAd8Kf9caKofjaMT6Lt0/?=
 =?us-ascii?Q?Q+t3EknS06UKbGU4tLgWFcHdSuEKzE7uibAqSvQQZfs9jqbsw84FG3QCF3WR?=
 =?us-ascii?Q?XvcWFYclyalU6FKAtR1vVf8ty4XhXQ6CY1Mnc6N+mkqIFZ7InVRex595TseV?=
 =?us-ascii?Q?Md5HLnT5WM1TSwKwZfuSbjQXzsYVEP7ib5UUmdN28TyDeS9quuE3QFCJGQoP?=
 =?us-ascii?Q?UJhP+zPQPrbGPGUIHeQnpS/TIsI7V/V5O5OYBT1wsz/ISYwk2mp2NQ+jDgbG?=
 =?us-ascii?Q?cJRRFc+9XPPJq7qFz6gMyuWrn90RJbIdkQra6Gs0CQJ4LgkZt6qwgfQRJzEA?=
 =?us-ascii?Q?b2u2fCkRqBMc4DSAFmcX7v4cc2fTOTDHtxbBb85z19OUsckAL0lcHoXHqGk5?=
 =?us-ascii?Q?4/vuICIIuW3vrqNvUBxLxJJYbFHqfrKeab70XBukhCGwCZ2MNo6gWAfNBLbn?=
 =?us-ascii?Q?PDgW0IAXm8+uTWZF19uSrk2S4jhNRCjr/RLlwVXKtxlRYtWMqZ5WjLpxB+F8?=
 =?us-ascii?Q?PUJhosws/NdPWUvIgstXVgcxblk37c8HKswKp0ZZw9lUtjI4fgKBfaD9mkiU?=
 =?us-ascii?Q?7fFhh9/WpTeVkfN035F+g8CVCKhxrgaqm9aDDmhjJwudmgp0Efzb2NAYDNbr?=
 =?us-ascii?Q?LNsxInLXS6wdq8l9pFcWxqP5hCD0SOhtocrYZsqoXBEhJzZ1vMVvURiDBaom?=
 =?us-ascii?Q?uTkjZKNLR4JtR5fQc3kLos6zxf4e34scUo1cWIlh+2N4wFKPmrNd13mJ4Xoa?=
 =?us-ascii?Q?YKZFBh2994evk4C1FTzi4jTaQCQp8o+7SnHEIdT15qw+R2ZpXgu5nUuwVvCh?=
 =?us-ascii?Q?WrBu1Fo8f0R/QDq6DHdxSTTDGyaHFtv5XhvAybMkr2hiOkQ5MT+lOxy/15j4?=
 =?us-ascii?Q?Qif/0sVIsXZPtN+xjXit7kCcsMX1FE0g34owyU2h/m0akakdvcYdTrfj9L0A?=
 =?us-ascii?Q?/098VNp6z2A65/ZHVOTWls9NGf28sP2azlsQEY5axHU4xhRge1kQfc/mHqBj?=
 =?us-ascii?Q?BLCiKM4Fph/8ToLFLAPSxckS3Et0wHzfQUJrFi7FyvfpAXlqFu8VxqsShyVD?=
 =?us-ascii?Q?RH4JzQQ6C5/hsTOGhdQTGDg5+n2hN3Lxk2b7YbDXR88eJJavICz1GMaUdVQT?=
 =?us-ascii?Q?F7ugrhq1UVj5e/PK/foRrVT0YVtG5RGjWJBXxtU6kFc+QR9BD361lgvHs+uF?=
 =?us-ascii?Q?fbpocdilpZGTV6OXXh65d6W9eKfd+Z/b7JGNsPTvHQeI2Arx9gE0ki85pCcb?=
 =?us-ascii?Q?zQ=3D=3D?=
X-OriginatorOrg: memverge.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 181ca7c3-51f4-4485-57df-08dbebb6468f
X-MS-Exchange-CrossTenant-AuthSource: SJ0PR17MB5512.namprd17.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Nov 2023 23:53:50.5010
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 5c90cb59-37e7-4c81-9c07-00473d5fb682
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 3d5vrYzz7icOIHqI+rC6PRnpx7Rb8ZWikj4jI7kh2MCtcgVNkuiyl4j9XKff1kqiAUrxE5aBh5sU2ZjGvNYIRxd6mKUkjYZUzGEswDI2HQw=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR17MB6336
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,
	DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,
	SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Wed, 22 Nov 2023 15:54:10 -0800 (PST)

On Wed, Nov 22, 2023 at 03:31:05PM -0600, Vinicius Petrucci wrote:
> From: Vinicius Tavares Petrucci <vtavarespetr@micron.com>
> 
> The proposed API is as follows:
> 
> long process_mbind(int pidfd, 
>                 const struct iovec *iovec, 
>                 unsigned long vlen, 
>                 unsigned long mode, 
>                 const unsigned long *nmask,
>                 unsigned int flags);
> 
> The `pidfd` argument is used to select the process that is identified by the PID file 
> descriptor provided in pidfd. (See pidofd_open(2) for more information)
>

This is probably a more maintainable interface than the pid_t interface
in my RFC, but I have not used pidfd extensively myself.

I can pivot my RFC to utilize pidfd as a whole and pop this on top, if
the other interfaces are generally useful.

> Please note the initial `maxnode` parameter from `mbind` was omitted 
> to ensure the API doesn't exceed 6 arguments. Instead, the constant 
> MAX_NUMNODES was utilized.
> 

I don't think this will work, users have traditionally been allowed to
shorten their nodemasks, and also for some level of portability.

We may want to consider an arg structure, rather than just chopping an
argument off.

> diff --git a/mm/mempolicy.c b/mm/mempolicy.c
> index 10a590ee1c89..91ee300fa728 100644
> --- a/mm/mempolicy.c
> +++ b/mm/mempolicy.c
> @@ -1215,11 +1215,10 @@ static struct folio *alloc_migration_target_by_mpol(struct folio *src,
>  }
>  #endif
>  
> -static long do_mbind(unsigned long start, unsigned long len,
> +static long do_mbind(struct mm_struct *mm, unsigned long start, unsigned long len,
>  		     unsigned short mode, unsigned short mode_flags,
>  		     nodemask_t *nmask, unsigned long flags)
>  {
> -	struct mm_struct *mm = current->mm;
>  	struct vm_area_struct *vma, *prev;
>  	struct vma_iterator vmi;
>  	struct migration_mpol mmpol;
> @@ -1465,10 +1464,84 @@ static inline int sanitize_mpol_flags(int *mode, unsigned short *flags)
>  	return 0;
>  }

This is a completely insufficient change to do_mbind.  do_mbind utilizes
`current` in a variety of places for nodemask (cpuset) validation and to
acquire the task's lock.  This will not work the way you intend it to,
you end up mixing up node masks between current and target task.

see here:
https://lore.kernel.org/all/20231122211200.31620-7-gregory.price@memverge.com/

I had to make do_mbind operate on the target task explicitly.

We may want to combine this change and with my change so that your iovec
changes can be re-used, because that is a very nice feature.

> +	unsigned long maxnode = MAX_NUMNODES;
> +	int err;
> +	nodemask_t nodes;
> +
> +	err = sanitize_mpol_flags(&lmode, &mode_flags);
> +	if (err)
> +		goto out;
> +
> +	err = get_nodes(&nodes, nmask, maxnode);

per above, userland MAX_NUMNODES may or may not be equal to kernel
MAX_NUMNODES, so i don't think you can just chop this argument off.


I think we can combine our RFCs here and get this sorted out.

~Gregory