Received: by 2002:ac0:a581:0:0:0:0:0 with SMTP id m1-v6csp3508178imm;
        Sun, 1 Jul 2018 23:47:22 -0700 (PDT)
X-Google-Smtp-Source: ADUXVKL3MtgO4Q9puZOO1XuCUBa1/ci0hEHtmPxCSzie7v/1BaNDm3LpVDE/7az9jv0oii9bZTWB
X-Received: by 2002:a65:56cd:: with SMTP id w13-v6mr21013715pgs.227.1530514042017;
        Sun, 01 Jul 2018 23:47:22 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; t=1530514041; cv=none;
        d=google.com; s=arc-20160816;
        b=mcOCrGq5xnsrJOaZv1A0qQbArx7y5NXGanrK1p+CyjukEdt3rzL4sTgM/nOyKeiZjU
         WFAvvX0Y0kMkyfF1liMUpPZUgBKdu41n2tP5VMLeiPyOnsFxBlTzFzrqe7snS7W88dd+
         s8r6NHyl0CH9cxDLkV2tUeho7xIGoHqExu3Sy6ZvszrIp7dT9kJ0BOvwbuHslrCe4c7c
         CRfBANM7Z39ptszEq1T+a3yKK2YIhXtmmy1tgtpSLa4p3117/f+I6UxRW5f2HVkVtpuX
         R8UCxVqx+AT0jKioKYGqE0YKfXWMcVydeg4GWFtVnQolXyQ+BkNFGFrcBCQQrC0cl+B4
         3ggQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816;
        h=list-id:precedence:sender:content-transfer-encoding
         :content-language:in-reply-to:mime-version:user-agent:date
         :message-id:from:references:cc:to:subject:arc-authentication-results;
        bh=QIxMQk6oQqP/DvDmlYUVDqjWhuC+46BR/2Tgk6rzO/c=;
        b=bzO15FY7CeQb+MPqeV5SgRWt9/+GMcdnOkPpOvfaFal07ufbfMEwruhhydNv/oB5ID
         ehKSew5NaBV0gm/dXSFZFol2UF0ytgbNiwdcZYWIvS4mwZBobNQi3hT9L/jlWv3IGwBs
         ey0GGatByEn7PH9jlXQUFoXp86KpPznR5g9AL16nbOtZMav9nu3v7jVz0L4GvQZldj8H
         KdAu11gbiQ867TZBhO2rPzP0ihtVUK+i9LYJfskFhMtYmpLTPgzsu/N0gVh3CKxZXZ7A
         6XH7UFEa+sgfgA3bFyo2BJgVIx1yWXdqTArActtPBN/jnMDDCnLpVKt8GrshbhMa9Mf7
         ggfg==
ARC-Authentication-Results: i=1; mx.google.com;
       spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org;
       dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=nvidia.com
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67])
        by mx.google.com with ESMTP id a26-v6si14734667pfn.281.2018.07.01.23.47.07;
        Sun, 01 Jul 2018 23:47:21 -0700 (PDT)
Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Authentication-Results: mx.google.com;
       spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org;
       dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=nvidia.com
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1753261AbeGBGLJ (ORCPT <rfc822;zoe.song.ucas@gmail.com>
        + 99 others); Mon, 2 Jul 2018 02:11:09 -0400
Received: from hqemgate15.nvidia.com ([216.228.121.64]:14716 "EHLO
        hqemgate15.nvidia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S1753148AbeGBGLG (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Mon, 2 Jul 2018 02:11:06 -0400
Received: from hqpgpgate101.nvidia.com (Not Verified[216.228.121.13]) by hqemgate15.nvidia.com (using TLS: TLSv1, AES128-SHA)
        id <B5b39c1d70000>; Sun, 01 Jul 2018 23:10:32 -0700
Received: from HQMAIL107.nvidia.com ([172.20.161.6])
  by hqpgpgate101.nvidia.com (PGP Universal service);
  Sun, 01 Jul 2018 23:11:06 -0700
X-PGP-Universal: processed;
        by hqpgpgate101.nvidia.com on Sun, 01 Jul 2018 23:11:06 -0700
Received: from [10.110.48.28] (10.110.48.28) by HQMAIL107.nvidia.com
 (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1347.2; Mon, 2 Jul
 2018 06:11:05 +0000
Subject: Re: [PATCH 2/2] mm: set PG_dma_pinned on get_user_pages*()
To:     Leon Romanovsky <leon@kernel.org>, Jan Kara <jack@suse.cz>
CC:     Jason Gunthorpe <jgg@ziepe.ca>, Michal Hocko <mhocko@kernel.org>,
        Dan Williams <dan.j.williams@intel.com>,
        Christoph Hellwig <hch@lst.de>,
        John Hubbard <john.hubbard@gmail.com>,
        Matthew Wilcox <willy@infradead.org>,
        Christopher Lameter <cl@linux.com>,
        Linux MM <linux-mm@kvack.org>,
        LKML <linux-kernel@vger.kernel.org>,
        linux-rdma <linux-rdma@vger.kernel.org>
References: <20180626134757.GY28965@dhcp22.suse.cz>
 <20180626164825.fz4m2lv6hydbdrds@quack2.suse.cz>
 <20180627113221.GO32348@dhcp22.suse.cz>
 <20180627115349.cu2k3ainqqdrrepz@quack2.suse.cz>
 <20180627115927.GQ32348@dhcp22.suse.cz>
 <20180627124255.np2a6rxy6rb6v7mm@quack2.suse.cz>
 <20180627145718.GB20171@ziepe.ca>
 <20180627170246.qfvucs72seqabaef@quack2.suse.cz>
 <1f6e79c5-5801-16d2-18a6-66bd0712b5b8@nvidia.com>
 <20180628091743.khhta7nafuwstd3m@quack2.suse.cz>
 <20180702055251.GV3014@mtr-leonro.mtl.com>
From:   John Hubbard <jhubbard@nvidia.com>
X-Nvconfidentiality: public
Message-ID: <235a23e3-6e02-234c-3e20-b2dddc93e568@nvidia.com>
Date:   Sun, 1 Jul 2018 23:10:04 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.8.0
MIME-Version: 1.0
In-Reply-To: <20180702055251.GV3014@mtr-leonro.mtl.com>
X-Originating-IP: [10.110.48.28]
X-ClientProxiedBy: HQMAIL103.nvidia.com (172.20.187.11) To
 HQMAIL107.nvidia.com (172.20.187.13)
Content-Type: text/plain; charset="windows-1252"
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 07/01/2018 10:52 PM, Leon Romanovsky wrote:
> On Thu, Jun 28, 2018 at 11:17:43AM +0200, Jan Kara wrote:
>> On Wed 27-06-18 19:42:01, John Hubbard wrote:
>>> On 06/27/2018 10:02 AM, Jan Kara wrote:
>>>> On Wed 27-06-18 08:57:18, Jason Gunthorpe wrote:
>>>>> On Wed, Jun 27, 2018 at 02:42:55PM +0200, Jan Kara wrote:
>>>>>> On Wed 27-06-18 13:59:27, Michal Hocko wrote:
>>>>>>> On Wed 27-06-18 13:53:49, Jan Kara wrote:
>>>>>>>> On Wed 27-06-18 13:32:21, Michal Hocko wrote:
>>>>>>> [...]
>>> One question though: I'm still vague on the best actions to take in the
>>> following functions:
>>>
>>>     page_mkclean_one
>>>     try_to_unmap_one
>>>
>>> At the moment, they are both just doing an evil little early-out:
>>>
>>> 	if (PageDmaPinned(page))
>>> 		return false;
>>>
>>> ...but we talked about maybe waiting for the condition to clear, instead?
>>> Thoughts?
>>
>> What needs to happen in page_mkclean() depends on the caller. Most of the
>> callers really need to be sure the page is write-protected once
>> page_mkclean() returns. Those are:
>>
>>   pagecache_isize_extended()
>>   fb_deferred_io_work()
>>   clear_page_dirty_for_io() if called for data-integrity writeback - which
>>     is currently known only in its caller (e.g. write_cache_pages()) where
>>     it can be determined as wbc->sync_mode == WB_SYNC_ALL. Getting this
>>     information into page_mkclean() will require some plumbing and
>>     clear_page_dirty_for_io() has some 50 callers but it's doable.
>>
>> clear_page_dirty_for_io() for cleaning writeback (wbc->sync_mode !=
>> WB_SYNC_ALL) can just skip pinned pages and we probably need to do that as
>> otherwise memory cleaning would get stuck on pinned pages until RDMA
>> drivers release its pins.
> 
> Sorry for naive question, but won't it create too much dirty pages
> so writeback will be called "non-stop" to rebalance watermarks without
> ability to progress?
> 

That is an interesting point. 

Holding off page writeback of this region does seem like it could cause
problems under memory pressure. Maybe adjusting the watermarks so that we
tell the writeback  system, "all is well, just ignore this region until
we're done with it" might help? Any ideas here are welcome...

Longer term, maybe some additional work could allow the kernel to be able
to writeback the gup-pinned pages (while DMA is happening--snapshots), but
that seems like a pretty big overhaul.

thanks,
-- 
John Hubbard
NVIDIA