Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) client-ip=2620:137:e000::3:8;
Message-ID: <4978d86c-c55f-cf26-796d-237c81efeab3@linux.alibaba.com>
Date:   Thu, 2 Nov 2023 15:10:49 +0800
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Thunderbird/102.15.1
Subject: Re: [PATCH] mm: huge_memory: batch tlb flush when splitting a
 pte-mapped THP
To:     Alistair Popple <apopple@nvidia.com>
Cc:     akpm@linux-foundation.org, shy828301@gmail.com,
        ying.huang@intel.com, linux-mm@kvack.org,
        linux-kernel@vger.kernel.org
References: <431d9fb6823036369dcb1d3b2f63732f01df21a7.1698488264.git.baolin.wang@linux.alibaba.com>
 <877cn22e0d.fsf@nvdebian.thelocal>
From:   Baolin Wang <baolin.wang@linux.alibaba.com>
In-Reply-To: <877cn22e0d.fsf@nvdebian.thelocal>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk


On 11/1/2023 2:13 PM, Alistair Popple wrote:
> 
> Baolin Wang <baolin.wang@linux.alibaba.com> writes:
> 
>> I can observe an obvious tlb flush hotpot when splitting a pte-mapped THP on
> 
>   A tlb flush hotpot does sound delicious, but I think you meant hotspot :-)

Ah, yes. Hope Andrew can help to fix it :)

>> my ARM64 server, and the distribution of this hotspot is as follows:
>>
>>     - 16.85% split_huge_page_to_list
>>        + 7.80% down_write
>>        - 7.49% try_to_migrate
>>           - 7.48% rmap_walk_anon
>>                7.23% ptep_clear_flush
>>        + 1.52% __split_huge_page
>>
>> The reason is that the split_huge_page_to_list() will build migration entries
>> for each subpage of a pte-mapped Anon THP by try_to_migrate(), or unmap for
>> file THP, and it will clear and tlb flush for each subpage's pte. Moreover,
>> the split_huge_page_to_list() will set TTU_SPLIT_HUGE_PMD flag to ensure
>> the THP is already a pte-mapped THP before splitting it to some normal pages.
> 
> The only other user of TTU_SPLIT_HUGE_PMD is vmscan which also sets
> TTU_BATCH_FLUSH so we could make the former imply the latter but that
> seem dangerous given the requirement to call try_to_unmap_flush() so
> best not to.
> 
> Reviewed-by: Alistair Popple <apopple@nvidia.com>

Thanks for reviewing, and also thanks to Ying and Yang.