2020-05-18 06:05:55

by John Hubbard

Subject: [PATCH] orangefs: convert get_user_pages() --> pin_user_pages()

This code was using get_user_pages*(), in a "Case 2" scenario
(DMA/RDMA), using the categorization from [1]. That means that it's
time to convert the get_user_pages*() + put_page() calls to
pin_user_pages*() + unpin_user_pages() calls.

There is some helpful background in [2]: basically, this is a small
part of fixing a long-standing disconnect between pinning pages, and
file systems' use of those pages.

[1] Documentation/core-api/pin_user_pages.rst

[2] "Explicit pinning of user-space pages":
https://lwn.net/Articles/807108/

Signed-off-by: John Hubbard <[email protected]>
---

fs/orangefs/orangefs-bufmap.c | 9 +++------
1 file changed, 3 insertions(+), 6 deletions(-)

diff --git a/fs/orangefs/orangefs-bufmap.c b/fs/orangefs/orangefs-bufmap.c
index 2bb916d68576..538e839590ef 100644
--- a/fs/orangefs/orangefs-bufmap.c
+++ b/fs/orangefs/orangefs-bufmap.c
@@ -168,10 +168,7 @@ static DEFINE_SPINLOCK(orangefs_bufmap_lock);
static void
orangefs_bufmap_unmap(struct orangefs_bufmap *bufmap)
{
- int i;
-
- for (i = 0; i < bufmap->page_count; i++)
- put_page(bufmap->page_array[i]);
+ unpin_user_pages(bufmap->page_array, bufmap->page_count);
}

static void
@@ -268,7 +265,7 @@ orangefs_bufmap_map(struct orangefs_bufmap *bufmap,
int offset = 0, ret, i;

/* map the pages */
- ret = get_user_pages_fast((unsigned long)user_desc->ptr,
+ ret = pin_user_pages_fast((unsigned long)user_desc->ptr,
bufmap->page_count, FOLL_WRITE, bufmap->page_array);

if (ret < 0)
@@ -280,7 +277,7 @@ orangefs_bufmap_map(struct orangefs_bufmap *bufmap,

for (i = 0; i < ret; i++) {
SetPageError(bufmap->page_array[i]);
- put_page(bufmap->page_array[i]);
+ unpin_user_page(bufmap->page_array[i]);
}
return -ENOMEM;
}
--
2.26.2

2020-05-18 06:27:19

by John Hubbard

Subject: Re: [PATCH] orangefs: convert get_user_pages() --> pin_user_pages()

On 2020-05-17 23:01, John Hubbard wrote:
> This code was using get_user_pages*(), in a "Case 2" scenario
> (DMA/RDMA), using the categorization from [1]. That means that it's
> time to convert the get_user_pages*() + put_page() calls to
> pin_user_pages*() + unpin_user_pages() calls.

Oops, actually, maybe that description is wrong. This is probably
closer to "Case 1: Direct IO"...right? I don't see an O_DIRECT
anywhere, but there is an ioctl to do ORANGEFS_DEV_MAP, which seems
like effectively it's doing Direct IO.

(I got lulled into commit log complacency, due to sending quite a few
non-filesystem patches that were Case 2.)


thanks,
--
John Hubbard
NVIDIA

>
> There is some helpful background in [2]: basically, this is a small
> part of fixing a long-standing disconnect between pinning pages, and
> file systems' use of those pages.
>
> [1] Documentation/core-api/pin_user_pages.rst
>
> [2] "Explicit pinning of user-space pages":
> https://lwn.net/Articles/807108/
>
> Signed-off-by: John Hubbard <[email protected]>
> ---
>
> fs/orangefs/orangefs-bufmap.c | 9 +++------
> 1 file changed, 3 insertions(+), 6 deletions(-)
>
> diff --git a/fs/orangefs/orangefs-bufmap.c b/fs/orangefs/orangefs-bufmap.c
> index 2bb916d68576..538e839590ef 100644
> --- a/fs/orangefs/orangefs-bufmap.c
> +++ b/fs/orangefs/orangefs-bufmap.c
> @@ -168,10 +168,7 @@ static DEFINE_SPINLOCK(orangefs_bufmap_lock);
> static void
> orangefs_bufmap_unmap(struct orangefs_bufmap *bufmap)
> {
> - int i;
> -
> - for (i = 0; i < bufmap->page_count; i++)
> - put_page(bufmap->page_array[i]);
> + unpin_user_pages(bufmap->page_array, bufmap->page_count);
> }
>
> static void
> @@ -268,7 +265,7 @@ orangefs_bufmap_map(struct orangefs_bufmap *bufmap,
> int offset = 0, ret, i;
>
> /* map the pages */
> - ret = get_user_pages_fast((unsigned long)user_desc->ptr,
> + ret = pin_user_pages_fast((unsigned long)user_desc->ptr,
> bufmap->page_count, FOLL_WRITE, bufmap->page_array);
>
> if (ret < 0)
> @@ -280,7 +277,7 @@ orangefs_bufmap_map(struct orangefs_bufmap *bufmap,
>
> for (i = 0; i < ret; i++) {
> SetPageError(bufmap->page_array[i]);
> - put_page(bufmap->page_array[i]);
> + unpin_user_page(bufmap->page_array[i]);
> }
> return -ENOMEM;
> }
>