Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 3 Aug 2022 14:41:20 +0800
From:   Feng Tang <feng.tang@intel.com>
To:     Michal Hocko <mhocko@suse.com>
CC:     Muchun Song <songmuchun@bytedance.com>,
        "akpm@linux-foundation.org" <akpm@linux-foundation.org>,
        "bwidawsk@kernel.org" <bwidawsk@kernel.org>,
        "dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Mike Kravetz <mike.kravetz@oracle.com>
Subject: Re: [PATCH] mm: mempolicy: fix policy_nodemask() for
 MPOL_PREFERRED_MANY case
Message-ID: <YuoYkPk+YzdPNvmN@feng-snb>
References: <20220801084207.39086-1-songmuchun@bytedance.com>
 <YueXhmiFcI8iw3OI@dhcp22.suse.cz>
 <YuecP/RKXWz7QAs0@feng-skl>
 <YuidPA9knCOoaT0c@FVFYT0MHHV2J>
 <Yui7hWZYMX31ktOr@feng-skl>
 <YujGy8EIeZc1Avc7@FVFYT0MHHV2J>
 <YujUyCIBjFj+FzX5@feng-skl>
 <YujoLQt09Js/sSQL@dhcp22.suse.cz>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <YujoLQt09Js/sSQL@dhcp22.suse.cz>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?zqDpoNIXJyXqnaUQhIfM9BwiiVL0M3wIAg3Bb28qbE6+p2aDPcYGhdZXoHFB?=
 =?us-ascii?Q?Vdqtfc43EZGSu9vpDQrRSasZ1a+y3eDp5dLLEhXLyknxOot/R13Mv5Wl0peP?=
 =?us-ascii?Q?7dcwTYyQ421yZUByaBxYKd/3/BsA0WOYu34nfs4ygi3B0MjUWEg0ckPnt7oo?=
 =?us-ascii?Q?0YqRmkWpC3ljRcQ0kVJXtfHRoq+Un/FJ840AgcoZB2ioIwvoePcF1xIgJjSn?=
 =?us-ascii?Q?ZGLjyJkrpcfBN3mVwTHTaJTxwOcJr1DXbLcfexs0dAEhDRrwFLNvUxz4YpLl?=
 =?us-ascii?Q?jCF9zYoNpMtZnlR6VsdTpL+04rdok9CEd0Whps0HsCck2QJJ+wygCiwtfyzY?=
 =?us-ascii?Q?p4P+95yRHPmnHw1iiJmnIDgmEsJMFNXyXw58PxqWOSq5bpQXbVpIW33saLAp?=
 =?us-ascii?Q?1FlFUIkHEoutlcCd4sF54M8BeIZ1CP2C5JzQA8AlONH2YA5yAk90OhNpIRMr?=
 =?us-ascii?Q?Lmg2+H5Jqlq89ChrflwauDl2NzaEOcUE14Sx7iZEN2UkfYwxuwJ1dvuyAy2k?=
 =?us-ascii?Q?pFBqG/k8uuq5hKbwOoH7ul7yRARCa8F1bbWxKonqtLTNtUKpneV1R21KQjY0?=
 =?us-ascii?Q?Yt/Sxrhy+7+9hAc1gx64KlT/o6nVl4KAMkrhr3HljFcehSAC36U+xjxeki5t?=
 =?us-ascii?Q?RZ9Nmzuo0W+K37EifxPp4pEWwJCgOgye3m6k12dmmSdEkFZN3jjfhefz4vrk?=
 =?us-ascii?Q?4UIh04D3AedLfFCK9LwgB605oDPw1hSEJLKmJxqcbVw+TM2G43MtkI6QhPn3?=
 =?us-ascii?Q?0eZN+KCasG9+DQRAs0xuMNaPrvmJs5gwV4MMreoOoCOH3pWB0tOju2z/R6N0?=
 =?us-ascii?Q?qPDTru7aajgiX1lAP5pUfXrzjaWFs7CwJoJMcwiZdV7piTU0OtBSfTPzG4zp?=
 =?us-ascii?Q?jXm7s0DDzn7dlqdSvt6bPm7OqaVKlQLAOU3YTlNMLebYS7nTfYg9Z3r0uvo7?=
 =?us-ascii?Q?BIjgfivFuQMHpaUinkjJ6g8pMtQKXbV/ej4FwBUNRAtT8Ptatu/nMfjI04sd?=
 =?us-ascii?Q?n0nsOtsCnprrjxzckHSrW78NYKdCc9aG4VCd7nccQ/zDALkhEX6b46NQdeVM?=
 =?us-ascii?Q?ZhFrO0+gfOaZBc/BQF7QzSF1pKF+ZtGVRzBoNJxMhBVeRCix3CGYRbByIv6r?=
 =?us-ascii?Q?+eJpcRQldtVzvMFjgIozLS3WP3JMzr8nulEPFWV3YKa4AskCmmF2lNRl+IkO?=
 =?us-ascii?Q?m2Dmp5h4smUmGM3Jfgh79XTlk3TZBn3t89kVFvIexXIfniRdkoH9jq9OJC8p?=
 =?us-ascii?Q?x3ojKVqTbndVc5TiEZYMjEUG6Nr/jl2nZ1dLgc9rUA7ONpZzWslbt6s7Wp3r?=
 =?us-ascii?Q?rdLxMu5mPntoMcn+qj6tOj7STcWT61P6jrWzu23AcpylCjfOU8pYYru8qKvL?=
 =?us-ascii?Q?ch5/3tykM6kTpwZ+lHstG+HBLgyEN77F+Viv1DYx+TSnZrURE3Mm7QJOsBTo?=
 =?us-ascii?Q?b1i4aHIKwmppzB9x5GhannTbajhEy0eHFgASOHUAUIwZmRoAyOnLaQROgasl?=
 =?us-ascii?Q?q334sbUpv977CychFGWHPrTIAV9u2sEoz226keQYH/Vc/a7PfXVKd8btGwgb?=
 =?us-ascii?Q?vITXiaiz1lQfw3pljbzBhWCFspqKNyif9hJgT4mm?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 8f596738-0744-44d8-05f3-08da751b34b1
X-MS-Exchange-CrossTenant-AuthSource: MN0PR11MB6304.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Aug 2022 06:41:33.6804
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Jo7XIadSLcvthOcpeKuB6g4UqLYzY2OmHddSxXztefjz5ZTumN1uAXJW1ba0ry9EXIfA5pDdJzDRAJYO1Noscw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR11MB6750
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-5.0 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Tue, Aug 02, 2022 at 05:02:37PM +0800, Michal Hocko wrote:
> Please make sure to CC Mike on hugetlb related changes.

OK.

> I didn't really get to grasp your proposed solution but it feels goind
> sideways. The real issue is that hugetlb uses a dedicated allocation
> scheme which is not fully MPOL_PREFERRED_MANY aware AFAICS. I do not
> think we should be tricking that by providing some fake nodemasks and
> what not.
> 
> The good news is that allocation from the pool is MPOL_PREFERRED_MANY
> aware because it first tries to allocation from the preffered node mask
> and then fall back to the full nodemask (dequeue_huge_page_vma).
> If the existing pools cannot really satisfy that allocation then it
> tries to allocate a new hugetlb page (alloc_fresh_huge_page) which also
> performs 2 stage allocation with the node mask and no node masks. But
> both of them might fail.
> 
> The bad news is that other allocation functions - including those that
> allocate to the pool are not fully MPOL_PREFERRED_MANY aware. E.g.
> __nr_hugepages_store_common paths which use the allocating process
> policy to fill up the pool so the pool could be under provisioned if
> that context is using MPOL_PREFERRED_MANY.

Thanks for the check!

So you mean if the prferred nodes don't have enough pages, we should
also fallback to all like dequeue_huge_page_vma() does?

Or we can user a policy API which return nodemask for MPOL_BIND and 
NULL for all other policies, like allowed_mems_nr() needs.

--- a/include/linux/mempolicy.h
+++ b/include/linux/mempolicy.h
@@ -158,6 +158,18 @@ static inline nodemask_t *policy_nodemask_current(gfp_t gfp)
 	return policy_nodemask(gfp, mpol);
 }
 
+#ifdef CONFIG_HUGETLB_FS
+static inline nodemask_t *strict_policy_nodemask_current(void)
+{
+	struct mempolicy *mpol = get_task_policy(current);
+
+	if (mpol->mode == MPOL_BIND)
+		return &mpol->nodes;
+
+	return NULL;
+}
+#endif
+

> Wrt. allowed_mems_nr (i.e. hugetlb_acct_memory) this is a reservation
> code and I have to admit I do not really remember details there. This is
> a subtle code and my best guess would be that policy_nodemask_current
> should be hugetlb specific and only care about MPOL_BIND.

The API needed by allowed_mem_nr() is a little different as it has gfp
flag and cpuset config to consider.

Thanks,
Feng

[snip]