Received: by 2002:ac0:a5b6:0:0:0:0:0 with SMTP id m51-v6csp4229338imm;
        Mon, 18 Jun 2018 11:15:59 -0700 (PDT)
X-Google-Smtp-Source: ADUXVKL8DLu/PQjmFFw3rSdpQ15894mQ3bo3modo8qjAdU46wLUgc1OuUuDtRw5cHfYdk983BWA8
X-Received: by 2002:aa7:8345:: with SMTP id z5-v6mr14813206pfm.251.1529345759465;
        Mon, 18 Jun 2018 11:15:59 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; t=1529345759; cv=none;
        d=google.com; s=arc-20160816;
        b=omyLmIM9E1qBIxGHcyoc8LGfayhYCHN1Jys0DCbm2qtOnGlOxTGkMmL22B4Aq5tIh/
         2zEMSz8gq9P8qHBm21XoMXiqhTWhgbABBOavtj08glnkf+YdQMF8vnD3SG3RbLxzm0Tu
         onhY4c7nDu7eVGoBlG7YwFmpeFY/+U2c5mncaSWr0TZrnuuQB/jrqUVinPJ2+fGAlYp/
         IWdwW4lnRTDNf3rqa5LIFT2C3LgQ7zEcIfo9oLHTyUAaLgEzu1h+P5aTey0onTTLIqTl
         HdX31VyT9Oo+pM4OcxPfwH/1FklxqkTNRKZbvHyHHgJrm9u+qahXRZRYkgEZxQF+e3U1
         CbFQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816;
        h=list-id:precedence:sender:content-transfer-encoding
         :content-language:in-reply-to:mime-version:user-agent:date
         :message-id:from:references:cc:to:subject:arc-authentication-results;
        bh=P+x/Kzly5FHJ5E1qa5NeaGqFVdrW5ZAcLGtk9q3algQ=;
        b=XyOKKE/wKDDADnMEFTeN/PkmHNSzBbmquM2lZtDv4kj9bvZiecwRLc2OuRGI+Nj3YL
         Kz0LkOYjUGlPz6ZTpxlq1O8ELdXCrO45Tj7sveFnDKIbSFcJcOeADFvOo+AQkcBIEii2
         b0QOl3xWhuMW8YlmEOGpk4r8LTT/n2/pUZJ+qwEJhU7D26iALcFXmUzg7N57mubNYCSd
         sbjgGBOv+m8qDyIz0jIyoMY0uV97Ye7b0JYf1nSoOXy91cvzi51yDytjNpGQ1h0gqLd1
         vFvb9HXIwlsbrOgj3oeT6wsKq2qwMokO65uudnuO/0mourg5SwZ8W8pn3Q0Jd9VdA2Iu
         t3oQ==
ARC-Authentication-Results: i=1; mx.google.com;
       spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org;
       dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=nvidia.com
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67])
        by mx.google.com with ESMTP id l12-v6si12511149pgp.683.2018.06.18.11.15.45;
        Mon, 18 Jun 2018 11:15:59 -0700 (PDT)
Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Authentication-Results: mx.google.com;
       spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org;
       dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=nvidia.com
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S935794AbeFRSPD (ORCPT <rfc822;linux.htchiang@gmail.com>
        + 99 others); Mon, 18 Jun 2018 14:15:03 -0400
Received: from hqemgate14.nvidia.com ([216.228.121.143]:2587 "EHLO
        hqemgate14.nvidia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S935533AbeFRSPC (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Mon, 18 Jun 2018 14:15:02 -0400
Received: from hqpgpgate101.nvidia.com (Not Verified[216.228.121.13]) by hqemgate14.nvidia.com (using TLS: TLSv1, AES128-SHA)
        id <B5b27f6ac0001>; Mon, 18 Jun 2018 11:15:08 -0700
Received: from HQMAIL107.nvidia.com ([172.20.161.6])
  by hqpgpgate101.nvidia.com (PGP Universal service);
  Mon, 18 Jun 2018 11:15:02 -0700
X-PGP-Universal: processed;
        by hqpgpgate101.nvidia.com on Mon, 18 Jun 2018 11:15:02 -0700
Received: from [10.2.175.123] (10.2.175.123) by HQMAIL107.nvidia.com
 (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1347.2; Mon, 18 Jun
 2018 18:15:01 +0000
Subject: Re: [PATCH 2/2] mm: set PG_dma_pinned on get_user_pages*()
To:     Dan Williams <dan.j.williams@intel.com>
CC:     Christoph Hellwig <hch@lst.de>, Jason Gunthorpe <jgg@ziepe.ca>,
        <john.hubbard@gmail.com>, Matthew Wilcox <willy@infradead.org>,
        Michal Hocko <mhocko@kernel.org>,
        Christopher Lameter <cl@linux.com>, Jan Kara <jack@suse.cz>,
        Linux MM <linux-mm@kvack.org>,
        LKML <linux-kernel@vger.kernel.org>,
        linux-rdma <linux-rdma@vger.kernel.org>
References: <20180617012510.20139-1-jhubbard@nvidia.com>
 <20180617012510.20139-3-jhubbard@nvidia.com>
 <CAPcyv4i=eky-QrPcLUEqjsASuRUrFEWqf79hWe0mU8xtz6Jk-w@mail.gmail.com>
 <20180617200432.krw36wrcwidb25cj@ziepe.ca>
 <CAPcyv4gayKk_zHDYAvntware12qMXWjnnL_FDJNUQsJS_zNfDw@mail.gmail.com>
 <311eba48-60f1-b6cc-d001-5cc3ed4d76a9@nvidia.com>
 <20180618081258.GB16991@lst.de>
 <d4817192-6db0-2f3f-7c67-6078b69686d3@nvidia.com>
 <CAPcyv4iacHYxGmyWokFrVsmxvLj7=phqp2i0tv8z6AT-mYuEEA@mail.gmail.com>
X-Nvconfidentiality: public
From:   John Hubbard <jhubbard@nvidia.com>
Message-ID: <3898ef6b-2fa0-e852-a9ac-d904b47320d5@nvidia.com>
Date:   Mon, 18 Jun 2018 11:14:40 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.8.0
MIME-Version: 1.0
In-Reply-To: <CAPcyv4iacHYxGmyWokFrVsmxvLj7=phqp2i0tv8z6AT-mYuEEA@mail.gmail.com>
X-Originating-IP: [10.2.175.123]
X-ClientProxiedBy: HQMAIL102.nvidia.com (172.18.146.10) To
 HQMAIL107.nvidia.com (172.20.187.13)
Content-Type: text/plain; charset="utf-8"
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 06/18/2018 10:56 AM, Dan Williams wrote:
> On Mon, Jun 18, 2018 at 10:50 AM, John Hubbard <jhubbard@nvidia.com> wrote:
>> On 06/18/2018 01:12 AM, Christoph Hellwig wrote:
>>> On Sun, Jun 17, 2018 at 01:28:18PM -0700, John Hubbard wrote:
>>>> Yes. However, my thinking was: get_user_pages() can become a way to indicate that
>>>> these pages are going to be treated specially. In particular, the caller
>>>> does not really want or need to support certain file operations, while the
>>>> page is flagged this way.
>>>>
>>>> If necessary, we could add a new API call.
>>>
>>> That API call is called get_user_pages_longterm.
>>
>> OK...I had the impression that this was just semi-temporary API for dax, but
>> given that it's an exported symbol, I guess it really is here to stay.
> 
> The plan is to go back and provide api changes that bypass
> get_user_page_longterm() for RDMA. However, for VFIO and others, it's
> not clear what we could do. In the VFIO case the guest would need to
> be prepared handle the revocation.
 
OK, let's see if I understand that plan correctly:

1. Change RDMA users (this could be done entirely in the various device drivers'
code, unless I'm overlooking something) to use mmu notifiers, and to do their
DMA to/from non-pinned pages.

2. Return early from get_user_pages_longterm, if the memory is...marked for
RDMA? (How? Same sort of page flag that I'm floating here, or something else?)
That would avoid the problem with pinned pages getting their buffer heads
removed--by disallowing the pinning. Makes sense.

Also, is there anything I can help with here, so that things can happen sooner?