Hi all,
This series contains cleanups to remove dedicated macro and remove
unnecessary tlb_remove_page_size() for huge zero pmd. Also this adds
missing read-only THP checking for transparent_hugepage_enabled() and
avoids discarding hugepage if other processes are mapping it. More
details can be found in the respective changelogs. Thanks!
v1->v2:
collect Reviewed-by tag
add missing check for read-only THP per Yang Shi
Miaohe Lin (5):
mm/huge_memory.c: remove dedicated macro HPAGE_CACHE_INDEX_MASK
mm/huge_memory.c: use page->deferred_list
mm/huge_memory.c: add missing read-only THP checking in
transparent_hugepage_enabled()
mm/huge_memory.c: remove unnecessary tlb_remove_page_size() for huge
zero pmd
mm/huge_memory.c: don't discard hugepage if other processes are
mapping it
include/linux/huge_mm.h | 27 +++++++++++++++++++--------
mm/huge_memory.c | 15 +++++++++------
mm/khugepaged.c | 4 +---
mm/shmem.c | 3 +--
4 files changed, 30 insertions(+), 19 deletions(-)
--
2.23.0
Rewrite the pgoff checking logic to remove macro HPAGE_CACHE_INDEX_MASK
which is only used here to simplify the code.
Reviewed-by: Yang Shi <[email protected]>
Reviewed-by: Anshuman Khandual <[email protected]>
Signed-off-by: Miaohe Lin <[email protected]>
---
include/linux/huge_mm.h | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
index 9626fda5efce..0a526f211fec 100644
--- a/include/linux/huge_mm.h
+++ b/include/linux/huge_mm.h
@@ -152,15 +152,13 @@ static inline bool __transparent_hugepage_enabled(struct vm_area_struct *vma)
bool transparent_hugepage_enabled(struct vm_area_struct *vma);
-#define HPAGE_CACHE_INDEX_MASK (HPAGE_PMD_NR - 1)
-
static inline bool transhuge_vma_suitable(struct vm_area_struct *vma,
unsigned long haddr)
{
/* Don't have to check pgoff for anonymous vma */
if (!vma_is_anonymous(vma)) {
- if (((vma->vm_start >> PAGE_SHIFT) & HPAGE_CACHE_INDEX_MASK) !=
- (vma->vm_pgoff & HPAGE_CACHE_INDEX_MASK))
+ if (!IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - vma->vm_pgoff,
+ HPAGE_PMD_NR))
return false;
}
--
2.23.0
Commit aa88b68c3b1d ("thp: keep huge zero page pinned until tlb flush")
introduced tlb_remove_page() for huge zero page to keep it pinned until
flush is complete and prevents the page from being split under us. But
huge zero page is kept pinned until all relevant mm_users reach zero since
the commit 6fcb52a56ff6 ("thp: reduce usage of huge zero page's atomic
counter"). So tlb_remove_page_size() for huge zero pmd is unnecessary now.
Signed-off-by: Miaohe Lin <[email protected]>
---
mm/huge_memory.c | 3 ---
1 file changed, 3 deletions(-)
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index e24a96de2e37..af30338ac49c 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -1680,12 +1680,9 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,
if (arch_needs_pgtable_deposit())
zap_deposited_table(tlb->mm, pmd);
spin_unlock(ptl);
- if (is_huge_zero_pmd(orig_pmd))
- tlb_remove_page_size(tlb, pmd_page(orig_pmd), HPAGE_PMD_SIZE);
} else if (is_huge_zero_pmd(orig_pmd)) {
zap_deposited_table(tlb->mm, pmd);
spin_unlock(ptl);
- tlb_remove_page_size(tlb, pmd_page(orig_pmd), HPAGE_PMD_SIZE);
} else {
struct page *page = NULL;
int flush_needed = 1;
--
2.23.0
If other processes are mapping any other subpages of the hugepage, i.e. in
pte-mapped thp case, page_mapcount() will return 1 incorrectly. Then we
would discard the page while other processes are still mapping it. Fix it
by using total_mapcount() which can tell whether other processes are still
mapping it.
Fixes: b8d3c4c3009d ("mm/huge_memory.c: don't split THP page when MADV_FREE syscall is called")
Reviewed-by: Yang Shi <[email protected]>
Signed-off-by: Miaohe Lin <[email protected]>
---
mm/huge_memory.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index af30338ac49c..87b0241394f4 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -1607,7 +1607,7 @@ bool madvise_free_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,
* If other processes are mapping this page, we couldn't discard
* the page unless they all do MADV_FREE so let's skip the page.
*/
- if (page_mapcount(page) != 1)
+ if (total_mapcount(page) != 1)
goto out;
if (!trylock_page(page))
--
2.23.0
On 29.04.21 15:26, Miaohe Lin wrote:
> Rewrite the pgoff checking logic to remove macro HPAGE_CACHE_INDEX_MASK
> which is only used here to simplify the code.
>
> Reviewed-by: Yang Shi <[email protected]>
> Reviewed-by: Anshuman Khandual <[email protected]>
> Signed-off-by: Miaohe Lin <[email protected]>
> ---
> include/linux/huge_mm.h | 6 ++----
> 1 file changed, 2 insertions(+), 4 deletions(-)
>
> diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
> index 9626fda5efce..0a526f211fec 100644
> --- a/include/linux/huge_mm.h
> +++ b/include/linux/huge_mm.h
> @@ -152,15 +152,13 @@ static inline bool __transparent_hugepage_enabled(struct vm_area_struct *vma)
>
> bool transparent_hugepage_enabled(struct vm_area_struct *vma);
>
> -#define HPAGE_CACHE_INDEX_MASK (HPAGE_PMD_NR - 1)
> -
> static inline bool transhuge_vma_suitable(struct vm_area_struct *vma,
> unsigned long haddr)
> {
> /* Don't have to check pgoff for anonymous vma */
> if (!vma_is_anonymous(vma)) {
> - if (((vma->vm_start >> PAGE_SHIFT) & HPAGE_CACHE_INDEX_MASK) !=
> - (vma->vm_pgoff & HPAGE_CACHE_INDEX_MASK))
> + if (!IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - vma->vm_pgoff,
> + HPAGE_PMD_NR))
> return false;
I'd have used
if (!IS_ALIGNED(PHYS_PFN(vma->vm_start) - vma->vm_pgoff,
Reviewed-by: David Hildenbrand <[email protected]>
--
Thanks,
David / dhildenb
On 29.04.21 15:26, Miaohe Lin wrote:
> Commit aa88b68c3b1d ("thp: keep huge zero page pinned until tlb flush")
> introduced tlb_remove_page() for huge zero page to keep it pinned until
> flush is complete and prevents the page from being split under us. But
> huge zero page is kept pinned until all relevant mm_users reach zero since
> the commit 6fcb52a56ff6 ("thp: reduce usage of huge zero page's atomic
> counter"). So tlb_remove_page_size() for huge zero pmd is unnecessary now.
>
> Signed-off-by: Miaohe Lin <[email protected]>
> ---
> mm/huge_memory.c | 3 ---
> 1 file changed, 3 deletions(-)
>
> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> index e24a96de2e37..af30338ac49c 100644
> --- a/mm/huge_memory.c
> +++ b/mm/huge_memory.c
> @@ -1680,12 +1680,9 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,
> if (arch_needs_pgtable_deposit())
> zap_deposited_table(tlb->mm, pmd);
> spin_unlock(ptl);
> - if (is_huge_zero_pmd(orig_pmd))
> - tlb_remove_page_size(tlb, pmd_page(orig_pmd), HPAGE_PMD_SIZE);
> } else if (is_huge_zero_pmd(orig_pmd)) {
> zap_deposited_table(tlb->mm, pmd);
> spin_unlock(ptl);
> - tlb_remove_page_size(tlb, pmd_page(orig_pmd), HPAGE_PMD_SIZE);
> } else {
> struct page *page = NULL;
> int flush_needed = 1;
>
This sounds sane to me
Acked-by: David Hildenbrand <[email protected]>
--
Thanks,
David / dhildenb
On Thu, Apr 29, 2021 at 6:27 AM Miaohe Lin <[email protected]> wrote:
>
> Commit aa88b68c3b1d ("thp: keep huge zero page pinned until tlb flush")
> introduced tlb_remove_page() for huge zero page to keep it pinned until
> flush is complete and prevents the page from being split under us. But
> huge zero page is kept pinned until all relevant mm_users reach zero since
> the commit 6fcb52a56ff6 ("thp: reduce usage of huge zero page's atomic
> counter"). So tlb_remove_page_size() for huge zero pmd is unnecessary now.
By reading the git history, it seems the lifecycle of huge zero page
is bound to process instead of page table due to the latter commit.
The patch looks correct to me. Reviewed-by: Yang Shi
<[email protected]>
>
> Signed-off-by: Miaohe Lin <[email protected]>
> ---
> mm/huge_memory.c | 3 ---
> 1 file changed, 3 deletions(-)
>
> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> index e24a96de2e37..af30338ac49c 100644
> --- a/mm/huge_memory.c
> +++ b/mm/huge_memory.c
> @@ -1680,12 +1680,9 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,
> if (arch_needs_pgtable_deposit())
> zap_deposited_table(tlb->mm, pmd);
> spin_unlock(ptl);
> - if (is_huge_zero_pmd(orig_pmd))
> - tlb_remove_page_size(tlb, pmd_page(orig_pmd), HPAGE_PMD_SIZE);
> } else if (is_huge_zero_pmd(orig_pmd)) {
> zap_deposited_table(tlb->mm, pmd);
> spin_unlock(ptl);
> - tlb_remove_page_size(tlb, pmd_page(orig_pmd), HPAGE_PMD_SIZE);
> } else {
> struct page *page = NULL;
> int flush_needed = 1;
> --
> 2.23.0
>
>
On 2021/4/29 22:48, David Hildenbrand wrote:
> On 29.04.21 15:26, Miaohe Lin wrote:
>> Rewrite the pgoff checking logic to remove macro HPAGE_CACHE_INDEX_MASK
>> which is only used here to simplify the code.
>>
>> Reviewed-by: Yang Shi <[email protected]>
>> Reviewed-by: Anshuman Khandual <[email protected]>
>> Signed-off-by: Miaohe Lin <[email protected]>
>> ---
>> include/linux/huge_mm.h | 6 ++----
>> 1 file changed, 2 insertions(+), 4 deletions(-)
>>
>> diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
>> index 9626fda5efce..0a526f211fec 100644
>> --- a/include/linux/huge_mm.h
>> +++ b/include/linux/huge_mm.h
>> @@ -152,15 +152,13 @@ static inline bool __transparent_hugepage_enabled(struct vm_area_struct *vma)
>> bool transparent_hugepage_enabled(struct vm_area_struct *vma);
>> -#define HPAGE_CACHE_INDEX_MASK (HPAGE_PMD_NR - 1)
>> -
>> static inline bool transhuge_vma_suitable(struct vm_area_struct *vma,
>> unsigned long haddr)
>> {
>> /* Don't have to check pgoff for anonymous vma */
>> if (!vma_is_anonymous(vma)) {
>> - if (((vma->vm_start >> PAGE_SHIFT) & HPAGE_CACHE_INDEX_MASK) !=
>> - (vma->vm_pgoff & HPAGE_CACHE_INDEX_MASK))
>> + if (!IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - vma->vm_pgoff,
>> + HPAGE_PMD_NR))
>> return false;
>
> I'd have used
>
> if (!IS_ALIGNED(PHYS_PFN(vma->vm_start) - vma->vm_pgoff,
>
It's because I want keep the code style consistent with hugepage_vma_check().
There is similiar code in hugepage_vma_check():
return IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - vma->vm_pgoff,
HPAGE_PMD_NR);
> Reviewed-by: David Hildenbrand <[email protected]>
Many thanks for review!
>