Firstly, the no_page_table() is meaningless for hugetlb which is a no-op
there, because a hugetlb page always satisfies:
- vma_is_anonymous() == false
- vma->vm_ops->fault != NULL
So we can already safely remove it in hugetlb_follow_page_mask(), alongside
with the page* variable.
Meanwhile, what we do in follow_hugetlb_page() actually makes sense for a
dump: we try to fault in the page only if the page cache is already
allocated. Let's do the same here for follow_page_mask() on hugetlb.
It should so far has zero effect on real dumps, because that still goes
into follow_hugetlb_page(). But this may start to influence a bit on
follow_page() users who mimics a "dump page" scenario, but hopefully in a
good way. This also paves way for unifying the hugetlb gup-slow.
Signed-off-by: Peter Xu <[email protected]>
---
mm/gup.c | 9 ++-------
mm/hugetlb.c | 9 +++++++++
2 files changed, 11 insertions(+), 7 deletions(-)
diff --git a/mm/gup.c b/mm/gup.c
index dbe96d266670..aa0668505d61 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -781,7 +781,6 @@ static struct page *follow_page_mask(struct vm_area_struct *vma,
struct follow_page_context *ctx)
{
pgd_t *pgd;
- struct page *page;
struct mm_struct *mm = vma->vm_mm;
ctx->page_mask = 0;
@@ -794,12 +793,8 @@ static struct page *follow_page_mask(struct vm_area_struct *vma,
* hugetlb_follow_page_mask is only for follow_page() handling here.
* Ordinary GUP uses follow_hugetlb_page for hugetlb processing.
*/
- if (is_vm_hugetlb_page(vma)) {
- page = hugetlb_follow_page_mask(vma, address, flags);
- if (!page)
- page = no_page_table(vma, flags);
- return page;
- }
+ if (is_vm_hugetlb_page(vma))
+ return hugetlb_follow_page_mask(vma, address, flags);
pgd = pgd_offset(mm, address);
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 270ec0ecd5a1..82dfdd96db4c 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -6501,6 +6501,15 @@ struct page *hugetlb_follow_page_mask(struct vm_area_struct *vma,
spin_unlock(ptl);
out_unlock:
hugetlb_vma_unlock_read(vma);
+
+ /*
+ * Fixup retval for dump requests: if pagecache doesn't exist,
+ * don't try to allocate a new page but just skip it.
+ */
+ if (!page && (flags & FOLL_DUMP) &&
+ !hugetlbfs_pagecache_present(h, vma, address))
+ page = ERR_PTR(-EFAULT);
+
return page;
}
--
2.40.1
On 06/13/23 17:53, Peter Xu wrote:
> Firstly, the no_page_table() is meaningless for hugetlb which is a no-op
> there, because a hugetlb page always satisfies:
>
> - vma_is_anonymous() == false
> - vma->vm_ops->fault != NULL
>
> So we can already safely remove it in hugetlb_follow_page_mask(), alongside
> with the page* variable.
>
> Meanwhile, what we do in follow_hugetlb_page() actually makes sense for a
> dump: we try to fault in the page only if the page cache is already
> allocated. Let's do the same here for follow_page_mask() on hugetlb.
>
> It should so far has zero effect on real dumps, because that still goes
> into follow_hugetlb_page(). But this may start to influence a bit on
> follow_page() users who mimics a "dump page" scenario, but hopefully in a
> good way. This also paves way for unifying the hugetlb gup-slow.
>
> Signed-off-by: Peter Xu <[email protected]>
> ---
> mm/gup.c | 9 ++-------
> mm/hugetlb.c | 9 +++++++++
> 2 files changed, 11 insertions(+), 7 deletions(-)
Thanks Peter!
Reviewed-by: Mike Kravetz <[email protected]>
--
Mike Kravetz
On 13.06.23 23:53, Peter Xu wrote:
> Firstly, the no_page_table() is meaningless for hugetlb which is a no-op
> there, because a hugetlb page always satisfies:
>
> - vma_is_anonymous() == false
> - vma->vm_ops->fault != NULL
>
> So we can already safely remove it in hugetlb_follow_page_mask(), alongside
> with the page* variable.
>
> Meanwhile, what we do in follow_hugetlb_page() actually makes sense for a
> dump: we try to fault in the page only if the page cache is already
> allocated. Let's do the same here for follow_page_mask() on hugetlb.
>
> It should so far has zero effect on real dumps, because that still goes
> into follow_hugetlb_page(). But this may start to influence a bit on
> follow_page() users who mimics a "dump page" scenario, but hopefully in a
> good way. This also paves way for unifying the hugetlb gup-slow.
>
> Signed-off-by: Peter Xu <[email protected]>
> ---
> mm/gup.c | 9 ++-------
> mm/hugetlb.c | 9 +++++++++
> 2 files changed, 11 insertions(+), 7 deletions(-)
>
> diff --git a/mm/gup.c b/mm/gup.c
> index dbe96d266670..aa0668505d61 100644
> --- a/mm/gup.c
> +++ b/mm/gup.c
> @@ -781,7 +781,6 @@ static struct page *follow_page_mask(struct vm_area_struct *vma,
> struct follow_page_context *ctx)
> {
> pgd_t *pgd;
> - struct page *page;
> struct mm_struct *mm = vma->vm_mm;
>
> ctx->page_mask = 0;
> @@ -794,12 +793,8 @@ static struct page *follow_page_mask(struct vm_area_struct *vma,
> * hugetlb_follow_page_mask is only for follow_page() handling here.
> * Ordinary GUP uses follow_hugetlb_page for hugetlb processing.
> */
> - if (is_vm_hugetlb_page(vma)) {
> - page = hugetlb_follow_page_mask(vma, address, flags);
> - if (!page)
> - page = no_page_table(vma, flags);
> - return page;
> - }
> + if (is_vm_hugetlb_page(vma))
> + return hugetlb_follow_page_mask(vma, address, flags);
>
> pgd = pgd_offset(mm, address);
>
> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
> index 270ec0ecd5a1..82dfdd96db4c 100644
> --- a/mm/hugetlb.c
> +++ b/mm/hugetlb.c
> @@ -6501,6 +6501,15 @@ struct page *hugetlb_follow_page_mask(struct vm_area_struct *vma,
> spin_unlock(ptl);
> out_unlock:
> hugetlb_vma_unlock_read(vma);
> +
> + /*
> + * Fixup retval for dump requests: if pagecache doesn't exist,
> + * don't try to allocate a new page but just skip it.
> + */
> + if (!page && (flags & FOLL_DUMP) &&
> + !hugetlbfs_pagecache_present(h, vma, address))
> + page = ERR_PTR(-EFAULT);
> +
> return page;
> }
>
Makes sense to me:
Reviewed-by: David Hildenbrand <[email protected]>
--
Cheers,
David / dhildenb