Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Fri, 7 Dec 2018 14:27:22 -0800 (PST)
From:   David Rientjes <rientjes@google.com>
To:     Vlastimil Babka <vbabka@suse.cz>
cc:     Linus Torvalds <torvalds@linux-foundation.org>,
        Andrea Arcangeli <aarcange@redhat.com>,
        mgorman@techsingularity.net, mhocko@kernel.org,
        ying.huang@intel.com, s.priebe@profihost.ag,
        Linux List Kernel Mailing <linux-kernel@vger.kernel.org>,
        alex.williamson@redhat.com, lkp@01.org, kirill@shutemov.name,
        Andrew Morton <akpm@linux-foundation.org>,
        zi.yan@cs.rutgers.edu
Subject: Re: [patch for-4.20] Revert "mm, thp: consolidate THP gfp handling
 into alloc_hugepage_direct_gfpmask"
In-Reply-To: <296d9813-1dfe-f27c-8d2c-6f140d8a5b0d@suse.cz>
Message-ID: <alpine.DEB.2.21.1812071421030.148470@chino.kir.corp.google.com>
References: <alpine.DEB.2.21.1812051445390.35948@chino.kir.corp.google.com> <alpine.DEB.2.21.1812061341130.144733@chino.kir.corp.google.com> <alpine.DEB.2.21.1812061343240.144733@chino.kir.corp.google.com> <296d9813-1dfe-f27c-8d2c-6f140d8a5b0d@suse.cz>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Fri, 7 Dec 2018, Vlastimil Babka wrote:

> > This reverts commit 89c83fb539f95491be80cdd5158e6f0ce329e317.
> > 
> > There are a couple of issues with 89c83fb539f9 independent of its partial 
> > revert in 2f0799a0ffc0 ("mm, thp: restore node-local hugepage 
> > allocations"):
> > 
> > Firstly, the interaction between alloc_hugepage_direct_gfpmask() and 
> > alloc_pages_vma() is racy wrt __GFP_THISNODE and MPOL_BIND.  
> > alloc_hugepage_direct_gfpmask() makes sure not to set __GFP_THISNODE for 
> > an MPOL_BIND policy but the policy used in alloc_pages_vma() may not be 
> > the same for shared vma policies, triggering the WARN_ON_ONCE() in 
> > policy_node().
> 
> AFAICS 2f0799a0ffc0 removed the policy check in
> alloc_hugepage_direct_gfpmask() comlpetely, so it's not racy and the
> warning will always trigger for a MPOL_BIND policy right now?
> 

Yup, looks like you hit it on the head.  This revert should have been done 
alongside 2f0799a0ffc0 ("mm, thp: restore node-local hugepage 
allocations"), I didn't appreciate how invasive the consolidation patch 
was.

I noticed the race in 89c83fb539f9 ("mm, thp: consolidate THP gfp handling 
into alloc_hugepage_direct_gfpmask") that is fixed by the revert, but as 
you noted it didn't cleanup the second part which is the balancing act for 
gfp flags between alloc_hugepage_direct_gfpmask() and alloc_pages_vma().  
Syzbot found this to trigger the WARN_ON_ONCE() you mention.

So we certainly need this patch for 4.20 as a follow-up to 2f0799a0ffc0.  
It's likely better to regroup and discuss NUMA aspects of all thp 
allocations separately with a stable 4.20.