Received-SPF: pass (google.com: domain of linux-kernel+bounces-5772-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Date: Tue, 19 Dec 2023 13:09:02 -0500
From: Gregory Price <gregory.price@memverge.com>
To: "Huang, Ying" <ying.huang@intel.com>
Cc: Gregory Price <gourry.memverge@gmail.com>, linux-mm@kvack.org,
	linux-doc@vger.kernel.org, linux-fsdevel@vger.kernel.org,
	linux-kernel@vger.kernel.org, linux-api@vger.kernel.org,
	x86@kernel.org, akpm@linux-foundation.org, arnd@arndb.de,
	tglx@linutronix.de, luto@kernel.org, mingo@redhat.com, bp@alien8.de,
	dave.hansen@linux.intel.com, hpa@zytor.com, mhocko@kernel.org,
	tj@kernel.org, corbet@lwn.net, rakie.kim@sk.com,
	hyeongtak.ji@sk.com, honggyu.kim@sk.com, vtavarespetr@micron.com,
	peterz@infradead.org, jgroves@micron.com, ravis.opensrc@micron.com,
	sthanneeru@micron.com, emirakhur@micron.com, Hasan.Maruf@amd.com,
	seungjun.ha@samsung.com, Johannes Weiner <hannes@cmpxchg.org>,
	Hasan Al Maruf <hasanalmaruf@fb.com>, Hao Wang <haowang3@fb.com>,
	Dan Williams <dan.j.williams@intel.com>,
	Michal Hocko <mhocko@suse.com>,
	Zhongkun He <hezhongkun.hzk@bytedance.com>,
	Frank van der Linden <fvdl@google.com>,
	John Groves <john@jagalactic.com>,
	Jonathan Cameron <Jonathan.Cameron@huawei.com>
Subject: Re: [PATCH v4 00/11] mempolicy2, mbind2, and weighted interleave
Message-ID: <ZYHcPiU2IzHr/tbQ@memverge.com>
References: <20231218194631.21667-1-gregory.price@memverge.com>
 <87wmtanba2.fsf@yhuang6-desk2.ccr.corp.intel.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <87wmtanba2.fsf@yhuang6-desk2.ccr.corp.intel.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?8vVI8WMMe2BfP7jCoOY2RC05UlpJpM0uHTfWxnccPNlRpLBKTlZT9Zi5HKPR?=
 =?us-ascii?Q?/j+K3W7n3yH2m2ThCEV3I9SouqDDO+LWyy81g64kU2MAIalBCmpE9zTUjIJW?=
 =?us-ascii?Q?/P0ePH+YSwLNgK6ONTZKsmhMNaQtbkjC1ysRuE7fa7VWY8aZMkyirVzitrB6?=
 =?us-ascii?Q?laXS+MlR8rzCN5/hFmXTogM+plUczMA/F1tcg5Q6hPCfiBLVq15VqqHjxccU?=
 =?us-ascii?Q?ybw1K3lqTkVRrYQr7jyNF74VRKtONWook/hh1cAn/lpQgR8bLZgGph63350N?=
 =?us-ascii?Q?89WtmsG9cqkO2Xq0Kk5fHJGVnq409Kj4TX5RxKI3Uv/9ZOVkOjOuyyfsgUPQ?=
 =?us-ascii?Q?VMJKrQUa3ip/gd7/7RBqDyecpxZ2y9l7MDS9W2stDZ2483MXE7I8Z5gdkIgw?=
 =?us-ascii?Q?7bnBTv69rLMf0y10oY2H3XKB7bNPIWG54aRo0UzIlEnqgBKsj3yv699Sz1vI?=
 =?us-ascii?Q?kQ4Y+0uUlhXO/0/zle+1IliJNBaxYBUN5uX6udhbU+pkuUfckX+rZKDENMAx?=
 =?us-ascii?Q?x+G5Db/ipIp+32sAeBDOAOwx3Lx0QbJkkhkDRMYFzLm1NdnICpfkT0g78/Nz?=
 =?us-ascii?Q?HnjqSxxX8FQuIuiLs7eCBYjUN4Bji0yqKUGErpruEe/MJCSug/xxI7M6hbF/?=
 =?us-ascii?Q?umiq4Qe8HOHNQapHwCfUcTl/nes9Iyo7tbqoLPEGnQyn71WcW3IiQjBZcTyh?=
 =?us-ascii?Q?X/nxyhfz/o9ns+RfyqYZysW14EAsUaYFnY3q0LTlw1L6lbcnKGXzJOUHTb4a?=
 =?us-ascii?Q?o/jdBV1qoOO5VXPNVlkesmB0Mf2j68edShvUlLy5YbkpbiZt1dVvHEIGkk5M?=
 =?us-ascii?Q?UyUSHaHulgzn7myE4ugk8f2QfUpsgiJN/8hQoK4+gWWa4/JkBI13uyfeKkkn?=
 =?us-ascii?Q?8UHMUhbMwsv3ePzxoN5FsxkqIoaMMMpgYahTX/7ocRN978x5Xi9C8xa/FG8l?=
 =?us-ascii?Q?XKfePxlGrBZDcjCVHrOZ5c4kDNsWqyqcIzyQuJSmwqJAKug+l5SBgoQfTKoo?=
 =?us-ascii?Q?1dqnTOz7WvZz7REX0E3DLtUo7ZiRkd07UhLao2LbCHNPomcx+DvEauP1IFBt?=
 =?us-ascii?Q?Vxqfw7jCSUiyNMhVhAqBMtDzs23r5lljhPqnjuORzJG+0ax7eUvM9BZ/dAiW?=
 =?us-ascii?Q?xyPsAhMgY5y5gNZL2VcTmvcvPzry6wnc/vMWWapHJlKE/b+fx3tK7oe5c1VW?=
 =?us-ascii?Q?7xtqpX/kY+Wd0XzWURVjQxIv3Ky2x/Hjy4L6KdvCTOF9q69K4pTJkcmhcNB5?=
 =?us-ascii?Q?wJeyuHYsbescMcmaKi7pgtpQVhl1AwBm/U0EKTq8h530xT9lb71muhVkvkSn?=
 =?us-ascii?Q?K8T9RJXN0W83b89OUy2iGrwiXMHpRVSxUnT9iUUZmOlSLOEzjYCrbB5iJ9XL?=
 =?us-ascii?Q?4lVX6ccY2g8J43hrhmWZ4n0DfU8AXsmJNF1s3KjLwd+QaCscgj+G3iCpiO+G?=
 =?us-ascii?Q?WqdeYnBEAY8dMIZVxA1CEprvMgL1Uujgp12wUWVT+LlW2ZPkp8DvpWnjlQj1?=
 =?us-ascii?Q?S8lrWShK6oqh/GBFIAVWvWIeVmkUjYEytwjmx6cKr6R6VRePhUn2w77uMDOZ?=
 =?us-ascii?Q?MzVBXV/STVqbG/r/qtOfytcd8dPFicfp/z1MJPnxoLsYtSE7x5nKl8q40A2n?=
 =?us-ascii?Q?kg=3D=3D?=
X-OriginatorOrg: memverge.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 047b7cb4-bb97-4138-04cf-08dc00bd982d
X-MS-Exchange-CrossTenant-AuthSource: SJ0PR17MB5512.namprd17.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Dec 2023 18:09:08.3407
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 5c90cb59-37e7-4c81-9c07-00473d5fb682
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: k1OtG37z6u48cOprkEcDcGVk4g/HcCPGzNw+prGsE/8ZjiQcmAixYIe4EPGQ7HwwA9dusC/DwCF7V+PI3FJtVwJde0EhCNTT/7VfoYPls5s=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLAPR17MB4097

On Tue, Dec 19, 2023 at 11:04:05AM +0800, Huang, Ying wrote:
> Gregory Price <gourry.memverge@gmail.com> writes:
> 
> > This patch set extends the mempolicy interface to enable new
> > mempolicies which may require extended data to operate.
> >
> > MPOL_WEIGHTED_INTERLEAVE is included as an example extension.
> 
> Per my understanding, it's better to describe why we need this patchset
> at the beginning.  Per my understanding, weighted interleave is used to
> expand DRAM bandwidth for workloads with real high memory bandwidth
> requirements.  Without it, DRAM bandwidth will be saturated, which leads
> to poor performance.
> 

Will add more details, thanks.

> > struct mempolicy_args {
> >     unsigned short mode;            /* policy mode */
> >     unsigned short mode_flags;      /* policy mode flags */
> >     int home_node;                  /* mbind: use MPOL_MF_HOME_NODE */
> >     nodemask_t *policy_nodes;       /* get/set/mbind */
> >     unsigned char *il_weights;      /* for mode MPOL_WEIGHTED_INTERLEAVE */
> >     int policy_node;                /* get: policy node information */
> > };
> 
> Because we use more and more parameters to describe the mempolicy, I
> think it's a good idea to replace some parameters with struct.  But I
> don't think it's a good idea to put unrelated stuff into the struct.
> For example,
> 
> struct mempolicy_param {
>     unsigned short mode;            /* policy mode */
>     unsigned short mode_flags;      /* policy mode flags */
>     int home_node;                  /* mbind: use MPOL_MF_HOME_NODE */
>     nodemask_t *policy_nodes;
>     unsigned char *il_weights;      /* for mode MPOL_WEIGHTED_INTERLEAVE */
> };
> 
> describe the parameters to create the mempolicy.  It can be used by
> set/get_mempolicy() and mbind().  So, I think that it's a good
> abstraction.  But "policy_node" has nothing to do with set_mempolicy()
> and mbind().  So I think that we shouldn't add it into the struct.  It's
> totally OK to use different parameters for different functions.  For
> example,
> 
> long do_set_mempolicy(struct mempolicy_param *mparam);
> long do_mbind(unsigned long start, unsigned long len,
>                 struct mempolicy_param *mparam, unsigned long flags);
> long do_get_task_mempolicy(struct mempolicy_param *mparam, int
>                 *policy_node);
> 
> This isn't the full list.  My point is to use separate parameter for
> something specific for some function.
>

this is the internal structure, but i get the point, we can drop it from
the structure and extend the arg list internally.

I'd originally thought to just remove the policy_node stuff all
together from get_mempolicy2().  Do you prefer to have a separate struct
for set/get interfaces so that the get interface struct can be extended?

All the MPOL_F_NODE "alternate data fetch" mechanisms from
get_mempolicy() feel like more of a wart than a feature.  And presently
the only data returned in policy_node is the next allocation node for
interleave.  That's not even particularly useful, so I'm of a mind to
remove it.

Assuming we remove policy_node altogether... do we still break up the
set/get interface into separate structures to avoid this in the future?

> > struct mpol_args {
> >         /* Basic mempolicy settings */
> >         __u16 mode;
> >         __u16 mode_flags;
> >         __s32 home_node;
> >         __aligned_u64 pol_nodes;
> >         __aligned_u64 *il_weights;      /* of size pol_maxnodes */
> >         __u64 pol_maxnodes;
> >         __s32 policy_node;
> > };
> 
> Same as my idea above.  I think we shouldn't add policy_node for
> set_mempolicy2()/mbind2().  That will make users confusing.  We can use
> a different struct for get_mempolicy2().
> 

See above.

~Gregory