Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <bfa13f35-bca9-c4e8-25f3-e8021f85f223@redhat.com>
Date:   Fri, 7 Jul 2023 15:24:33 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.12.0
Content-Language: en-US
To:     Matthew Wilcox <willy@infradead.org>
Cc:     Ryan Roberts <ryan.roberts@arm.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>,
        Yin Fengwei <fengwei.yin@intel.com>,
        Yu Zhao <yuzhao@google.com>,
        Catalin Marinas <catalin.marinas@arm.com>,
        Will Deacon <will@kernel.org>,
        Anshuman Khandual <anshuman.khandual@arm.com>,
        Yang Shi <shy828301@gmail.com>,
        linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org,
        linux-mm@kvack.org
References: <20230703135330.1865927-1-ryan.roberts@arm.com>
 <78159ed0-a233-9afb-712f-2df1a4858b22@redhat.com>
 <4d4c45a2-0037-71de-b182-f516fee07e67@arm.com>
 <d9cb4563-c622-9660-287b-a2f35121aec7@redhat.com>
 <ZKgPIXSrxqymWrsv@casper.infradead.org>
From:   David Hildenbrand <david@redhat.com>
Organization: Red Hat
Subject: Re: [PATCH v2 0/5] variable-order, large folios for anonymous memory
In-Reply-To: <ZKgPIXSrxqymWrsv@casper.infradead.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 07.07.23 15:12, Matthew Wilcox wrote:
> On Fri, Jul 07, 2023 at 01:40:53PM +0200, David Hildenbrand wrote:
>> On 06.07.23 10:02, Ryan Roberts wrote:
>> But can you comment on the page migration part (IOW did you try it already)?
>>
>> For example, memory hotunplug, CMA, MCE handling, compaction all rely on
>> page migration of something that was allocated using GFP_MOVABLE to actually
>> work.
>>
>> Compaction seems to skip any higher-order folios, but the question is if the
>> udnerlying migration itself works.
>>
>> If it already works: great! If not, this really has to be tackled early,
>> because otherwise we'll be breaking the GFP_MOVABLE semantics.
> 
> I have looked at this a bit.  _Migration_ should be fine.  _Compaction_
> is not.

Thanks! Very nice if at least ordinary migration works.

> 
> If you look at a function like folio_migrate_mapping(), it all seems
> appropriately folio-ised.  There might be something in there that is
> slightly wrong, but that would just be a bug to fix, not a huge
> architectural problem.
> 
> The problem comes in the callers of migrate_pages().  They pass a
> new_folio_t callback.  alloc_migration_target() is the usual one passed
> and as far as I can tell is fine.  I've seen no problems reported with it.
> 
> compaction_alloc() is a disaster, and I don't know how to fix it.
> The compaction code has its own allocator which is populated with order-0
> folios.  How it populates that freelist is awful ... see split_map_pages()

Yeah, all that code was written under the assumption that we're moving 
order-0 pages (which is what the anon+pagecache pages part).

 From what I recall, we're allocating order-0 pages from the high memory 
addresses, so we can migrate from low memory addresses, effectively 
freeing up low memory addresses and filling high memory addresses.

Adjusting that will be ... interesting. Instead of allocating order-0 
pages from high addresses, we might want to allocate "as large as 
possible" ("grab what we can") from high addresses and then have our own 
kind of buddy for allocating from that pool a compaction destination 
page, depending on our source page. Nasty.

What should always work is the split->migrate. But that's definitely not 
what we want in many cases.

> 
>> Is swapping working as expected? zswap?
> 
> Suboptimally.  Swap will split folios in order to swap them.  Somebody
> needs to fix that, but it should work.

Good!

It would be great to have some kind of a feature matrix that tells us 
what works perfectly, sub-optimally, barely, not at all (and what has 
not been tested). Maybe (likely!) we'll also find things that are 
sub-optimal for ordinary THP (like swapping, not even sure about).

I suspect that KSM should work mostly fine with flexible-thp. When 
deduplciating, we'll simply split the compound page and proceed as 
expected. But might be worth testing as well.

-- 
Cheers,

David / dhildenb