Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Tue, 25 Feb 2020 18:51:30 -0800
From:   Andrew Morton <akpm@linux-foundation.org>
To:     Mel Gorman <mgorman@techsingularity.net>
Cc:     Michal Hocko <mhocko@suse.com>, Vlastimil Babka <vbabka@suse.cz>,
        Ivan Babrou <ivan@cloudflare.com>,
        Rik van Riel <riel@surriel.com>,
        Linux-MM <linux-mm@kvack.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 0/3] Limit runaway reclaim due to watermark boosting
Message-Id: <20200225185130.6a32a8a6920d11b4c098e90e@linux-foundation.org>
In-Reply-To: <20200225141534.5044-1-mgorman@techsingularity.net>
References: <20200225141534.5044-1-mgorman@techsingularity.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Tue, 25 Feb 2020 14:15:31 +0000 Mel Gorman <mgorman@techsingularity.net> wrote:

> Ivan Babrou reported the following

http://lkml.kernel.org/r/CABWYdi1eOUD1DHORJxTsWPMT3BcZhz++xP1pXhT=x4SgxtgQZA@mail.gmail.com
is helpful.

> 	Commit 1c30844d2dfe ("mm: reclaim small amounts of memory when
> 	an external fragmentation event occurs") introduced undesired
> 	effects in our environment.
> 
> 	  * NUMA with 2 x CPU
> 	  * 128GB of RAM
> 	  * THP disabled
> 	  * Upgraded from 4.19 to 5.4
> 
> 	Before we saw free memory hover at around 1.4GB with no
> 	spikes. After the upgrade we saw some machines decide that they
> 	need a lot more than that, with frequent spikes above 10GB,
> 	often only on a single numa node.
> 
> There have been a few reports recently that might be watermark boost
> related. Unfortunately, finding someone that can reproduce the problem
> and test a patch has been problematic.  This series intends to limit
> potential damage only.

It's problematic that we don't understand what's happening.  And these
palliatives can only reduce our ability to do that.

Rik seems to have the means to reproduce this (or something similar)
and it seems Ivan can test patches three weeks hence.  So how about a
debug patch which will help figure out what's going on in there?