Received-SPF: pass (google.com: domain of linux-ext4-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates
 12.22.5.235 as permitted sender) receiver=protection.outlook.com;
 client-ip=12.22.5.235; helo=mail.nvidia.com;
From:   Alistair Popple <apopple@nvidia.com>
To:     Alex Sierra <alex.sierra@amd.com>,
        Joao Martins <joao.m.martins@oracle.com>
CC:     <amd-gfx@lists.freedesktop.org>, <dri-devel@lists.freedesktop.org>,
        <hch@lst.de>, <jglisse@redhat.com>, <willy@infradead.org>,
        <akpm@linux-foundation.org>, <Felix.Kuehling@amd.com>,
        <linux-mm@kvack.org>, <rcampbell@nvidia.com>,
        <linux-ext4@vger.kernel.org>, <linux-xfs@vger.kernel.org>,
        <jgg@nvidia.com>
Subject: Re: [PATCH v3 03/10] mm/gup: fail get_user_pages for LONGTERM dev coherent type
Date:   Fri, 21 Jan 2022 00:18:15 +1100
Message-ID: <2432841.uOC12ROH9Y@nvdebian>
In-Reply-To: <008b5ff4-dd53-d652-46dd-5ce771c69a0a@oracle.com>
References: <20220110223201.31024-1-alex.sierra@amd.com> <20220110223201.31024-4-alex.sierra@amd.com> <008b5ff4-dd53-d652-46dd-5ce771c69a0a@oracle.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 7Bit
Content-Type: text/plain; charset="us-ascii"
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Jan 2022 13:18:21.6071
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 7c433adb-d488-4440-8849-08d9dc1754f9
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.235];Helo=[mail.nvidia.com]
X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT013.eop-nam11.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR12MB5043
Precedence: bulk
List-ID: <linux-ext4.vger.kernel.org>
X-Mailing-List: linux-ext4@vger.kernel.org

On Thursday, 20 January 2022 11:36:21 PM AEDT Joao Martins wrote:
> On 1/10/22 22:31, Alex Sierra wrote:
> > Avoid long term pinning for Coherent device type pages. This could
> > interfere with their own device memory manager. For now, we are just
> > returning error for PIN_LONGTERM Coherent device type pages. Eventually,
> > these type of pages will get migrated to system memory, once the device
> > migration pages support is added.
> > 
> > Signed-off-by: Alex Sierra <alex.sierra@amd.com>
> > ---
> >  mm/gup.c | 7 +++++++
> >  1 file changed, 7 insertions(+)
> > 
> > diff --git a/mm/gup.c b/mm/gup.c
> > index 886d6148d3d0..9c8a075d862d 100644
> > --- a/mm/gup.c
> > +++ b/mm/gup.c
> > @@ -1720,6 +1720,12 @@ static long check_and_migrate_movable_pages(unsigned long nr_pages,
> >  		 * If we get a movable page, since we are going to be pinning
> >  		 * these entries, try to move them out if possible.
> >  		 */
> > +		if (is_device_page(head)) {
> > +			WARN_ON_ONCE(is_device_private_page(head));
> > +			ret = -EFAULT;
> > +			goto unpin_pages;
> > +		}
> > +
> 
> Wouldn't be more efficient for you failing earlier instead of after all the pages are pinned?

Rather than failing I think the plan is to migrate the device coherent pages
like we do for ZONE_MOVABLE, so leaving this here is a good place holder until
that is done. Currently we are missing some functionality required to do that
but I am hoping to post a series fixing that soon.

> Filesystem DAX suffers from a somewhat similar issue[0] -- albeit it's more related to
> blocking FOLL_LONGTERM in gup-fast while gup-slow can still do it. Coherent devmap appears
> to want to block it in all gup.
> 
> On another thread Jason was suggesting about having different pgmap::flags to capture
> these special cases[1] instead of selecting what different pgmap types can do in various
> different places.
> 
> [0] https://lore.kernel.org/linux-mm/6a18179e-65f7-367d-89a9-d5162f10fef0@oracle.com/
> [1] https://lore.kernel.org/linux-mm/20211019160136.GH3686969@ziepe.ca/
>