Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: [PATCH] orangefs: convert get_user_pages() --> pin_user_pages()
To:     LKML <linux-kernel@vger.kernel.org>
CC:     Mike Marshall <hubcap@omnibond.com>,
        Martin Brandenburg <martin@omnibond.com>,
        <devel@lists.orangefs.org>
References: <20200518060139.2828423-1-jhubbard@nvidia.com>
From:   John Hubbard <jhubbard@nvidia.com>
Message-ID: <582e6272-ca21-6300-19dd-7776e21ce002@nvidia.com>
Date:   Sun, 17 May 2020 23:24:58 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.8.0
MIME-Version: 1.0
In-Reply-To: <20200518060139.2828423-1-jhubbard@nvidia.com>
Content-Type: text/plain; charset="utf-8"; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On 2020-05-17 23:01, John Hubbard wrote:
> This code was using get_user_pages*(), in a "Case 2" scenario
> (DMA/RDMA), using the categorization from [1]. That means that it's
> time to convert the get_user_pages*() + put_page() calls to
> pin_user_pages*() + unpin_user_pages() calls.

Oops, actually, maybe that description is wrong. This is probably
closer to "Case 1: Direct IO"...right? I don't see an O_DIRECT
anywhere, but there is an ioctl to do ORANGEFS_DEV_MAP, which seems
like effectively it's doing Direct IO.

(I got lulled into commit log complacency, due to sending quite a few
non-filesystem patches that were Case 2.)


thanks,
-- 
John Hubbard
NVIDIA

> 
> There is some helpful background in [2]: basically, this is a small
> part of fixing a long-standing disconnect between pinning pages, and
> file systems' use of those pages.
> 
> [1] Documentation/core-api/pin_user_pages.rst
> 
> [2] "Explicit pinning of user-space pages":
>      https://lwn.net/Articles/807108/
> 
> Signed-off-by: John Hubbard <jhubbard@nvidia.com>
> ---
> 
>   fs/orangefs/orangefs-bufmap.c | 9 +++------
>   1 file changed, 3 insertions(+), 6 deletions(-)
> 
> diff --git a/fs/orangefs/orangefs-bufmap.c b/fs/orangefs/orangefs-bufmap.c
> index 2bb916d68576..538e839590ef 100644
> --- a/fs/orangefs/orangefs-bufmap.c
> +++ b/fs/orangefs/orangefs-bufmap.c
> @@ -168,10 +168,7 @@ static DEFINE_SPINLOCK(orangefs_bufmap_lock);
>   static void
>   orangefs_bufmap_unmap(struct orangefs_bufmap *bufmap)
>   {
> -	int i;
> -
> -	for (i = 0; i < bufmap->page_count; i++)
> -		put_page(bufmap->page_array[i]);
> +	unpin_user_pages(bufmap->page_array, bufmap->page_count);
>   }
>   
>   static void
> @@ -268,7 +265,7 @@ orangefs_bufmap_map(struct orangefs_bufmap *bufmap,
>   	int offset = 0, ret, i;
>   
>   	/* map the pages */
> -	ret = get_user_pages_fast((unsigned long)user_desc->ptr,
> +	ret = pin_user_pages_fast((unsigned long)user_desc->ptr,
>   			     bufmap->page_count, FOLL_WRITE, bufmap->page_array);
>   
>   	if (ret < 0)
> @@ -280,7 +277,7 @@ orangefs_bufmap_map(struct orangefs_bufmap *bufmap,
>   
>   		for (i = 0; i < ret; i++) {
>   			SetPageError(bufmap->page_array[i]);
> -			put_page(bufmap->page_array[i]);
> +			unpin_user_page(bufmap->page_array[i]);
>   		}
>   		return -ENOMEM;
>   	}
>