Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Subject: Re: [RFC] mm: Proactive compaction
To:     Vlastimil Babka <vbabka@suse.cz>, Nitin Gupta <nigupta@nvidia.com>,
        "dan.j.williams@intel.com" <dan.j.williams@intel.com>,
        "mhocko@suse.com" <mhocko@suse.com>,
        "mgorman@techsingularity.net" <mgorman@techsingularity.net>,
        "akpm@linux-foundation.org" <akpm@linux-foundation.org>
Cc:     "cai@lca.pw" <cai@lca.pw>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
        "aryabinin@virtuozzo.com" <aryabinin@virtuozzo.com>,
        "jannh@google.com" <jannh@google.com>, "guro@fb.com" <guro@fb.com>,
        "hannes@cmpxchg.org" <hannes@cmpxchg.org>,
        "keescook@chromium.org" <keescook@chromium.org>,
        "yuzhao@google.com" <yuzhao@google.com>,
        "arunks@codeaurora.org" <arunks@codeaurora.org>,
        "willy@infradead.org" <willy@infradead.org>,
        "janne.huttunen@nokia.com" <janne.huttunen@nokia.com>,
        "khlebnikov@yandex-team.ru" <khlebnikov@yandex-team.ru>
References: <20190816214413.15006-1-nigupta@nvidia.com>
 <87634ddc-8bfd-8311-46c4-35f7dc32d42f@suse.cz>
 <7bbd5322ed7a7fcb349c83952f8fc17448cd07d8.camel@nvidia.com>
 <71d7fba0-bd6f-3ac5-1fd8-9a8ff6fc6b8b@suse.cz>
From:   Khalid Aziz <khalid.aziz@oracle.com>
Organization: Oracle Corp
Message-ID: <d33fce4d-6018-0235-5391-debc8974eda5@oracle.com>
Date:   Tue, 24 Sep 2019 08:11:09 -0600
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.8.0
MIME-Version: 1.0
In-Reply-To: <71d7fba0-bd6f-3ac5-1fd8-9a8ff6fc6b8b@suse.cz>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On 9/24/19 7:39 AM, Vlastimil Babka wrote:
> On 9/20/19 1:37 AM, Nitin Gupta wrote:
>> On Tue, 2019-08-20 at 10:46 +0200, Vlastimil Babka wrote:
>>>
>>> That's a lot of control knobs - how is an admin supposed to tune them=
 to
>>> their
>>> needs?
>>
>>
>> Yes, it's difficult for an admin to get so many tunable right unless
>> targeting a very specific workload.
>>
>> How about a simpler solution where we exposed just one tunable per-nod=
e:
>>    /sys/.../node-x/compaction_effort
>> which accepts [0, 100]
>>
>> This parallels /proc/sys/vm/swappiness but for compaction. With this
>> single number, we can estimate per-order [low, high] watermarks for ex=
ternal
>> fragmentation like this:
>>  - For now, map this range to [low, medium, high] which correponds to =
specific
>> low, high thresholds for extfrag.
>>  - Apply more relaxed thresholds for higher-order than for lower order=
s.
>>
>> With this single tunable we remove the burden of setting per-order exp=
licit
>> [low, high] thresholds and it should be easier to experiment with.
>=20
> What about instead autotuning by the numbers of allocations hitting
> direct compaction recently? IIRC there were attempts in the past (mysel=
f
> included) and recently Khalid's that was quite elaborated.
>=20

I do think the right way forward with this longstanding problem is to
take the burden of managing free memory away from end user and let the
kernel autotune itself to the demands of workload. We can start with a
simpler algorithm in the kernel that adapts to workload and refine it as
we move forward. As long as initial implementation performs at least as
well as current free page management, we have a workable path for
improvements. I am moving the implementation I put together in kernel to
a userspace daemon just to test it out on larger variety of workloads.
It is more limited in userspace with limited access to statistics the
algorithm needs to perform trend analysis so I would rather be doing
this in the kernel.

--
Khalid