Received: by 2002:a25:ef43:0:0:0:0:0 with SMTP id w3csp1048929ybm; Wed, 27 May 2020 14:54:03 -0700 (PDT) X-Google-Smtp-Source: ABdhPJytvxsx7BAMnqL+Pnfh2yhNqprCXg5W1WeszdJS8jes9T97GvvZQs2yE+bphfrTWxKGU/aV X-Received: by 2002:a05:6402:2211:: with SMTP id cq17mr168510edb.171.1590616443543; Wed, 27 May 2020 14:54:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1590616443; cv=none; d=google.com; s=arc-20160816; b=nG3J3DnlBVL+EZJeYCt86UGNjnG1lx4iCj2OhfoVhgE/XeAQSI8+EEfDvyakU6StQg OcOvaa5+LQ+QHdT/oJtYQsZx+IZbIHyhYnMLsns7FCzDhQYmp9kS32+xsc+0PMR+3NM7 JBBhKUKRJMCWfrU4i4I2MWFm+gnAa2GJ7mJOmUcmiacR8cb3cTxVtP6e0q2NUuDh8cj3 vBOJ4yiAx30nw6l+VfHovuczOAAFXAitdCV/vhtMwfuW/6RnjtzpLwDQqZyqAhcTiDUN FbXL/isChp/kotoL3MNLjxuKU8DwaamCihqC2fksVDIr+OTHNWq4KPwvu7y0tREdFWGu XBeA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:dkim-signature:content-transfer-encoding :content-language:in-reply-to:mime-version:user-agent:date :message-id:from:references:to:subject; bh=cJGxgyxUMS7+SGXB5PmVaX93dA/VtdD9tLRCxZnqybQ=; b=nUU1re842O1BDVgR2jJOuupOBem2tcXsnTcAQ0wPK7ySoi4RWI6WENfcQ0n/a/VSsQ Lj2K5zb844bi/hgpx9d2tue2m2QmPu4U6MG9GBBt4DR73bBWOIz2rc2BaLJMoGWYjcOS HXEtIdgDtIcIfQ5mzx58kvFBPOxtexiiD+EfH8wKioxsiBq0NOxeOtwj1EwImMD36Z1Y eF3mabMaQ9ZO1t2SeFLMAHoks/p5O+kn+WiHvAwWcbsWXm2DePDpwnAm+/s9KQIaLnqK iuRSouG/5+xlRJpmwoOoYeXLo7S8VxZyeVzTIZ4dbWk+JhWMXrKPLjZ6Tshyq71qtGoT RZJw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@nvidia.com header.s=n1 header.b=LFjk88MN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=nvidia.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id n19si2300603edr.438.2020.05.27.14.53.41; Wed, 27 May 2020 14:54:03 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@nvidia.com header.s=n1 header.b=LFjk88MN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=nvidia.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728491AbgE0THK (ORCPT + 99 others); Wed, 27 May 2020 15:07:10 -0400 Received: from hqnvemgate25.nvidia.com ([216.228.121.64]:4357 "EHLO hqnvemgate25.nvidia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725294AbgE0THJ (ORCPT ); Wed, 27 May 2020 15:07:09 -0400 Received: from hqpgpgate101.nvidia.com (Not Verified[216.228.121.13]) by hqnvemgate25.nvidia.com (using TLS: TLSv1.2, DES-CBC3-SHA) id ; Wed, 27 May 2020 12:05:45 -0700 Received: from hqmail.nvidia.com ([172.20.161.6]) by hqpgpgate101.nvidia.com (PGP Universal service); Wed, 27 May 2020 12:07:09 -0700 X-PGP-Universal: processed; by hqpgpgate101.nvidia.com on Wed, 27 May 2020 12:07:09 -0700 Received: from [10.2.87.74] (10.124.1.5) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Wed, 27 May 2020 19:07:08 +0000 Subject: Re: [PATCH] drm/radeon: Convert get_user_pages() --> pin_user_pages() To: Souptick Joarder , , , , , , References: <1590526802-3008-1-git-send-email-jrdr.linux@gmail.com> <69a033cf-63b2-7da6-6a5e-a5bbc94b8afb@nvidia.com> <20200527084852.GN206103@phenom.ffwll.local> <20200527085117.GO206103@phenom.ffwll.local> From: John Hubbard X-Nvconfidentiality: public Message-ID: Date: Wed, 27 May 2020 12:07:08 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.8.1 MIME-Version: 1.0 In-Reply-To: <20200527085117.GO206103@phenom.ffwll.local> X-Originating-IP: [10.124.1.5] X-ClientProxiedBy: HQMAIL101.nvidia.com (172.20.187.10) To HQMAIL107.nvidia.com (172.20.187.13) Content-Type: text/plain; charset="utf-8"; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1590606345; bh=cJGxgyxUMS7+SGXB5PmVaX93dA/VtdD9tLRCxZnqybQ=; h=X-PGP-Universal:Subject:To:References:From:X-Nvconfidentiality: Message-ID:Date:User-Agent:MIME-Version:In-Reply-To: X-Originating-IP:X-ClientProxiedBy:Content-Type:Content-Language: Content-Transfer-Encoding; b=LFjk88MNH0RKsXsLEslCB9oYqWO/+qMh7mtsWwHzujVQNvjhVM3PMi5vWNhtfXxso crnY9sWfLYHUbXyb1qBHZ2T3NZDECLkFwKYXj5JCxdqvsHElOFJDUqLmcZZUmwz8vw kNSQ1hzCmul6R2atxy7vC4gHnhTLSubpC9+XFFM2R5zhstiVlSIQRp6Lp3IE/8L2SM CucGGzQ8NCIIWK2j5pvWa5Jr7e6iP/ggLLIduMbh7MVZHCREBVJ5dVoD8IhyWo0M2N 1yF68YtM+toIRVk1NkUTaZRPr5GOd1ry/MnsmndslsBkJVaLj7Xb9e0kKPDZlgLWIs sDMfQem0oi7Vw== Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2020-05-27 01:51, Daniel Vetter wrote: > On Wed, May 27, 2020 at 10:48:52AM +0200, Daniel Vetter wrote: >> On Tue, May 26, 2020 at 03:57:45PM -0700, John Hubbard wrote: >>> On 2020-05-26 14:00, Souptick Joarder wrote: >>>> This code was using get_user_pages(), in a "Case 2" scenario >>>> (DMA/RDMA), using the categorization from [1]. That means that it's >>>> time to convert the get_user_pages() + release_pages() calls to >>>> pin_user_pages() + unpin_user_pages() calls. >>>> >>>> There is some helpful background in [2]: basically, this is a small >>>> part of fixing a long-standing disconnect between pinning pages, and >>>> file systems' use of those pages. >>>> >>>> [1] Documentation/core-api/pin_user_pages.rst >>>> >>>> [2] "Explicit pinning of user-space pages": >>>> https://lwn.net/Articles/807108/ >> >> I don't think this is a case 2 here, nor is it any of the others. Feels >> like not covered at all by the doc. >> >> radeon has a mmu notifier (might be a bit broken, but hey whatever there's >> other drivers which have the same concept, but less broken). So when you >> do an munmap, radeon will release the page refcount. > Aha, thanks Daniel. I withdraw my misinformed ACK, then. > I forgot to add: It's also not case 3, since there's no hw page fault > support. It's all faked in software, and explicitly synchronizes against > pending io (or preempts it, that depends a bit upon the jobs running). > This is what case 3 was *intended* to cover, but it looks like case 3 needs to be written a little better. I'll attempt that, and Cc you on the actual patch to -mm. (I think we also need a case 5 for an unrelated scenario, too, so it's time.) thanks, -- John Hubbard NVIDIA >> Which case it that? >> >> Note that currently only amdgpu doesn't work like that for gpu dma >> directly to userspace ranges, it uses hmm and afaiui doens't hold a full >> page pin refcount. >> >> Cheers, Daniel >> >> >>>> >>>> Signed-off-by: Souptick Joarder >>>> Cc: John Hubbard >>>> >>>> Hi, >>>> >>>> I'm compile tested this, but unable to run-time test, so any testing >>>> help is much appriciated. >>>> --- >>>> drivers/gpu/drm/radeon/radeon_ttm.c | 6 +++--- >>>> 1 file changed, 3 insertions(+), 3 deletions(-) >>>> >>>> diff --git a/drivers/gpu/drm/radeon/radeon_ttm.c b/drivers/gpu/drm/radeon/radeon_ttm.c >>>> index 5d50c9e..e927de2 100644 >>>> --- a/drivers/gpu/drm/radeon/radeon_ttm.c >>>> +++ b/drivers/gpu/drm/radeon/radeon_ttm.c >>>> @@ -506,7 +506,7 @@ static int radeon_ttm_tt_pin_userptr(struct ttm_tt *ttm) >>>> uint64_t userptr = gtt->userptr + pinned * PAGE_SIZE; >>>> struct page **pages = ttm->pages + pinned; >>>> - r = get_user_pages(userptr, num_pages, write ? FOLL_WRITE : 0, >>>> + r = pin_user_pages(userptr, num_pages, write ? FOLL_WRITE : 0, >>>> pages, NULL); >>>> if (r < 0) >>>> goto release_pages; >>>> @@ -535,7 +535,7 @@ static int radeon_ttm_tt_pin_userptr(struct ttm_tt *ttm) >>>> kfree(ttm->sg); >>>> release_pages: >>>> - release_pages(ttm->pages, pinned); >>>> + unpin_user_pages(ttm->pages, pinned); >>>> return r; >>>> } >>>> @@ -562,7 +562,7 @@ static void radeon_ttm_tt_unpin_userptr(struct ttm_tt *ttm) >>>> set_page_dirty(page); >>> >>> >>> Maybe we also need a preceding patch, to fix the above? It should be >>> set_page_dirty_lock(), rather than set_page_dirty(), unless I'm overlooking >>> something (which is very possible!). >>> >>> Either way, from a tunnel vision perspective of changing gup to pup, this >>> looks good to me, so >>> >>> Acked-by: John Hubbard >>> >>> >>> thanks, >>> -- >>> John Hubbard >>> NVIDIA >>> >>>> mark_page_accessed(page); >>>> - put_page(page); >>>> + unpin_user_page(page); >>>> } >>>> sg_free_table(ttm->sg); >>>> >>> >> >> -- >> Daniel Vetter >> Software Engineer, Intel Corporation >> http://blog.ffwll.ch >