Received: by 2002:ac0:946b:0:0:0:0:0 with SMTP id j40csp1196461imj; Sat, 16 Feb 2019 23:52:45 -0800 (PST) X-Google-Smtp-Source: AHgI3IYfWxw+W7ODGf5erLGBZA8IAnPY6phpOqFCaTMPTxtq8GtLW+3miL4pvVGpC45OTdvZJuBq X-Received: by 2002:a63:e80e:: with SMTP id s14mr13201823pgh.30.1550389965340; Sat, 16 Feb 2019 23:52:45 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550389965; cv=none; d=google.com; s=arc-20160816; b=nvosw6c4ilp5g/weovSqdamrZyW2kuuCGSEA0lEL4QRkX7ow//NH3sn3o0cBygx/Y5 B0yUuPXRcMMk2FEX9pj+2JKuHtcsNAtEZchn7bFeGIikZ2YSn5zwoFDnsjGO+KPqmwwS gB24a6+QizwlYmWwov/f7RFgvUML6tvyvXVKu7YCoEqIpc8fTUXhsJIcPB0P/CUz/fs2 IXcYIiIjfHHq/77ysmsTo6LR4/OfeIWFwpCjRNBxlZYH/cZbT8IDybdUpjgMmrPBt5x4 d6CTb+oPo/56dj1upVQHdqD/GMXhjPKiMguYyczbcM3tOWr5Nn/m8gspb0y0aPhUU3ih tGkQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:user-agent:in-reply-to :content-disposition:mime-version:references:message-id:subject:cc :to:from:date; bh=X3ip6CkqDDWn1gLVD0Z2SslNhPWtMl54rEztwbBR7aw=; b=eI+gFXVaydeqinv8WUf0uwi8SlEhpM5hV0qOIOqJKAJyMig8Jz6FMY9IPn42tPwJy/ Y8PrZorTZQw1VeZiQY4zcqs6rIFpDOmk4lpyNLnE2W5PWXQKAyA5rCk19SMHquLPP30H pUlEDcCRoAoL0e0+ssTD33Qz8SEbj1SFVFnq4DBqkKacBEQLxof9HDgCtqU8YCAUuoky Yd87FqS7XpC8MEDgq/ErKEkoTHGSLNJC506zPCuWGMgW0bd+OHk72BjdFhvI16rqzIAS 5nLo4kAPqQxcKJ7FAmFfIphBIai9qKmz2ybnO+pkxebTWwsHE2D2H7PWgYkqWe5N6Uc7 S34A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id z10si9770332pgf.549.2019.02.16.23.52.19; Sat, 16 Feb 2019 23:52:45 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2387425AbfBPWm4 (ORCPT + 99 others); Sat, 16 Feb 2019 17:42:56 -0500 Received: from ipmail06.adl6.internode.on.net ([150.101.137.145]:45428 "EHLO ipmail06.adl6.internode.on.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726523AbfBPWmz (ORCPT ); Sat, 16 Feb 2019 17:42:55 -0500 Received: from ppp59-167-129-252.static.internode.on.net (HELO dastard) ([59.167.129.252]) by ipmail06.adl6.internode.on.net with ESMTP; 17 Feb 2019 09:12:52 +1030 Received: from dave by dastard with local (Exim 4.80) (envelope-from ) id 1gv8fa-0001EO-S9; Sun, 17 Feb 2019 09:42:50 +1100 Date: Sun, 17 Feb 2019 09:42:50 +1100 From: Dave Chinner To: Ira Weiny Cc: Jason Gunthorpe , Christopher Lameter , Matthew Wilcox , Jerome Glisse , Dan Williams , Jan Kara , Doug Ledford , lsf-pc@lists.linux-foundation.org, linux-rdma , Linux MM , Linux Kernel Mailing List , John Hubbard , Michal Hocko Subject: Re: [LSF/MM TOPIC] Discuss least bad options for resolving longterm-GUP usage by RDMA Message-ID: <20190216224250.GV20493@dastard> References: <20190211180654.GB24692@ziepe.ca> <20190214202622.GB3420@redhat.com> <20190214205049.GC12668@bombadil.infradead.org> <20190214213922.GD3420@redhat.com> <20190215011921.GS20493@dastard> <01000168f1d25e3a-2857236c-a7cc-44b8-a5f3-f51c2cfe6ce4-000000@email.amazonses.com> <20190215180852.GJ12668@bombadil.infradead.org> <01000168f26d9e0c-aef3255a-5059-4657-b241-dae66663bbea-000000@email.amazonses.com> <20190215220031.GB8001@ziepe.ca> <20190215233828.GB30818@iweiny-DESK2.sc.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190215233828.GB30818@iweiny-DESK2.sc.intel.com> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Feb 15, 2019 at 03:38:29PM -0800, Ira Weiny wrote: > On Fri, Feb 15, 2019 at 03:00:31PM -0700, Jason Gunthorpe wrote: > > On Fri, Feb 15, 2019 at 06:31:36PM +0000, Christopher Lameter wrote: > > > On Fri, 15 Feb 2019, Matthew Wilcox wrote: > > > > > > > > Since RDMA is something similar: Can we say that a file that is used for > > > > > RDMA should not use the page cache? > > > > > > > > That makes no sense. The page cache is the standard synchronisation point > > > > for filesystems and processes. The only problems come in for the things > > > > which bypass the page cache like O_DIRECT and DAX. > > > > > > It makes a lot of sense since the filesystems play COW etc games with the > > > pages and RDMA is very much like O_DIRECT in that the pages are modified > > > directly under I/O. It also bypasses the page cache in case you have > > > not noticed yet. > > > > It is quite different, O_DIRECT modifies the physical blocks on the > > storage, bypassing the memory copy. > > > > Really? I thought O_DIRECT allowed the block drivers to write to/from user > space buffers. But the _storage_ was still under the control of the block > drivers? Yup, in a nutshell. Even O_DIRECT on DAX doesn't modify the physical storage directly - it ends up in the pmem driver and it does a memcpy() to move the data to/from the physical storage and the user space buffer. It's exactly the same IO path as moving data to/from the physical storage into the page cache pages.... Cheers, Dave. -- Dave Chinner david@fromorbit.com