Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Tue, 27 Oct 2020 10:15:09 -0300
From:   Jason Gunthorpe <jgg@ziepe.ca>
To:     Jan Kara <jack@suse.cz>
Cc:     John Hubbard <jhubbard@nvidia.com>, linux-kernel@vger.kernel.org,
        Andrea Arcangeli <aarcange@redhat.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        Christoph Hellwig <hch@lst.de>,
        Hugh Dickins <hughd@google.com>, Jann Horn <jannh@google.com>,
        Kirill Shutemov <kirill@shutemov.name>,
        Kirill Tkhai <ktkhai@virtuozzo.com>,
        Linux-MM <linux-mm@kvack.org>, Michal Hocko <mhocko@suse.com>,
        Oleg Nesterov <oleg@redhat.com>, Peter Xu <peterx@redhat.com>
Subject: Re: [PATCH 1/2] mm: reorganize internal_get_user_pages_fast()
Message-ID: <20201027131509.GU36674@ziepe.ca>
References: <1-v1-281e425c752f+2df-gup_fork_jgg@nvidia.com>
 <16c50bb0-431d-5bfb-7b80-a8af0b4da90f@nvidia.com>
 <20201027093301.GA16090@quack2.suse.cz>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20201027093301.GA16090@quack2.suse.cz>
Precedence: bulk

On Tue, Oct 27, 2020 at 10:33:01AM +0100, Jan Kara wrote:
> On Fri 23-10-20 21:44:17, John Hubbard wrote:
> > On 10/23/20 5:19 PM, Jason Gunthorpe wrote:
> > > +	start += (unsigned long)nr_pinned << PAGE_SHIFT;
> > > +	pages += nr_pinned;
> > > +	ret = __gup_longterm_unlocked(start, nr_pages - nr_pinned, gup_flags,
> > > +				      pages);
> > > +	if (ret < 0) {
> > >   		/* Have to be a bit careful with return values */
> > 
> > ...and can we move that comment up one level, so that it reads:
> > 
> > 	/* Have to be a bit careful with return values */
> > 	if (ret < 0) {
> > 		if (nr_pinned)
> > 			return nr_pinned;
> > 		return ret;
> > 	}
> > 	return ret + nr_pinned;
> > 
> > Thinking about this longer term, it would be nice if the whole gup/pup API
> > set just stopped pretending that anyone cares about partial success, because
> > they *don't*. If we had return values of "0 or -ERRNO" throughout, and an
> > additional set of API wrappers that did some sort of limited retry just like
> > some of the callers do, that would be a happier story.
> 
> Actually there are callers that care about partial success. See e.g.
> iov_iter_get_pages() usage in fs/direct_io.c:dio_refill_pages() or
> bio_iov_iter_get_pages(). These places handle partial success just fine and
> not allowing partial success from GUP could regress things...

I looked through a bunch of call sites, and there are a wack that
actually do only want a complete return and are carrying a bunch of
code to fix it:

	pvec = kvmalloc_array(npages, sizeof(struct page *), GFP_KERNEL);
	if (!pvec)
		return -ENOMEM;

	do {
		unsigned num_pages = npages - pinned;
		uint64_t ptr = userptr->ptr + pinned * PAGE_SIZE;
		struct page **pages = pvec + pinned;

		ret = pin_user_pages_fast(ptr, num_pages,
					  !userptr->ro ? FOLL_WRITE : 0, pages);
		if (ret < 0) {
			unpin_user_pages(pvec, pinned);
			kvfree(pvec);
			return ret;
		}

		pinned += ret;

	} while (pinned < npages);

Is really a lot better if written as:

   	pvec = kvmalloc_array(npages, sizeof(struct page *), GFP_KERNEL);
	if (!pvec)
		return -ENOMEM;
	ret = pin_user_pages_fast(userptr->ptr, npages, FOLL_COMPLETE |
	                          (!userptr->ro ? FOLL_WRITE : 0),
				  pvec);
        if (ret) {
              kvfree(pvec);
	      return ret;
        }

(eg FOLL_COMPLETE says to return exactly npages or fail)

Some code assumes things work that way already anyhow:

	/* Pin user pages for DMA Xfer */
	err = pin_user_pages_unlocked(user_dma.uaddr, user_dma.page_count,
			dma->map, FOLL_FORCE);

	if (user_dma.page_count != err) {
		IVTV_DEBUG_WARN("failed to map user pages, returned %d instead of %d\n",
			   err, user_dma.page_count);
		if (err >= 0) {
			unpin_user_pages(dma->map, err);
			return -EINVAL;
		}
		return err;
	}

Actually I'm quite surprised I didn't find too many missing the tricky
unpin_user_pages() on the error path - eg
videobuf_dma_init_user_locked() is wrong.

Jason