Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 3 Aug 2022 09:36:41 +0200
From:   Michal Hocko <mhocko@suse.com>
To:     Feng Tang <feng.tang@intel.com>
Cc:     Muchun Song <songmuchun@bytedance.com>,
        "akpm@linux-foundation.org" <akpm@linux-foundation.org>,
        "bwidawsk@kernel.org" <bwidawsk@kernel.org>,
        "dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Mike Kravetz <mike.kravetz@oracle.com>
Subject: Re: [PATCH] mm: mempolicy: fix policy_nodemask() for
 MPOL_PREFERRED_MANY case
Message-ID: <YuolieBmdaIzoS3t@dhcp22.suse.cz>
References: <20220801084207.39086-1-songmuchun@bytedance.com>
 <YueXhmiFcI8iw3OI@dhcp22.suse.cz>
 <YuecP/RKXWz7QAs0@feng-skl>
 <YuidPA9knCOoaT0c@FVFYT0MHHV2J>
 <Yui7hWZYMX31ktOr@feng-skl>
 <YujGy8EIeZc1Avc7@FVFYT0MHHV2J>
 <YujUyCIBjFj+FzX5@feng-skl>
 <YujoLQt09Js/sSQL@dhcp22.suse.cz>
 <YuoYkPk+YzdPNvmN@feng-snb>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <YuoYkPk+YzdPNvmN@feng-snb>
Precedence: bulk

On Wed 03-08-22 14:41:20, Feng Tang wrote:
> On Tue, Aug 02, 2022 at 05:02:37PM +0800, Michal Hocko wrote:
> > Please make sure to CC Mike on hugetlb related changes.
> 
> OK.
> 
> > I didn't really get to grasp your proposed solution but it feels goind
> > sideways. The real issue is that hugetlb uses a dedicated allocation
> > scheme which is not fully MPOL_PREFERRED_MANY aware AFAICS. I do not
> > think we should be tricking that by providing some fake nodemasks and
> > what not.
> > 
> > The good news is that allocation from the pool is MPOL_PREFERRED_MANY
> > aware because it first tries to allocation from the preffered node mask
> > and then fall back to the full nodemask (dequeue_huge_page_vma).
> > If the existing pools cannot really satisfy that allocation then it
> > tries to allocate a new hugetlb page (alloc_fresh_huge_page) which also
> > performs 2 stage allocation with the node mask and no node masks. But
> > both of them might fail.
> > 
> > The bad news is that other allocation functions - including those that
> > allocate to the pool are not fully MPOL_PREFERRED_MANY aware. E.g.
> > __nr_hugepages_store_common paths which use the allocating process
> > policy to fill up the pool so the pool could be under provisioned if
> > that context is using MPOL_PREFERRED_MANY.
> 
> Thanks for the check!
> 
> So you mean if the prferred nodes don't have enough pages, we should
> also fallback to all like dequeue_huge_page_vma() does?
> 
> Or we can user a policy API which return nodemask for MPOL_BIND and 
> NULL for all other policies, like allowed_mems_nr() needs.
> 
> --- a/include/linux/mempolicy.h
> +++ b/include/linux/mempolicy.h
> @@ -158,6 +158,18 @@ static inline nodemask_t *policy_nodemask_current(gfp_t gfp)
>  	return policy_nodemask(gfp, mpol);
>  }
>  
> +#ifdef CONFIG_HUGETLB_FS
> +static inline nodemask_t *strict_policy_nodemask_current(void)
> +{
> +	struct mempolicy *mpol = get_task_policy(current);
> +
> +	if (mpol->mode == MPOL_BIND)
> +		return &mpol->nodes;
> +
> +	return NULL;
> +}
> +#endif

Yes something like this, except that I would also move this into hugetlb
proper because this doesn't seem generally useful.
 
> > Wrt. allowed_mems_nr (i.e. hugetlb_acct_memory) this is a reservation
> > code and I have to admit I do not really remember details there. This is
> > a subtle code and my best guess would be that policy_nodemask_current
> > should be hugetlb specific and only care about MPOL_BIND.
> 
> The API needed by allowed_mem_nr() is a little different as it has gfp
> flag and cpuset config to consider.

Why would gfp mask matter? 
-- 
Michal Hocko
SUSE Labs