Received: by 2002:ac0:8c8e:0:0:0:0:0 with SMTP id r14csp1090057ima; Wed, 6 Feb 2019 13:34:03 -0800 (PST) X-Google-Smtp-Source: AHgI3IbJB3Sx5PFrYAdDMZGgp95ss4EwYcU0UHCiCQuUJrlfSLpBm0TI+MjBjtVhTVTvpuugYtza X-Received: by 2002:a63:e655:: with SMTP id p21mr11492965pgj.70.1549488843834; Wed, 06 Feb 2019 13:34:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1549488843; cv=none; d=google.com; s=arc-20160816; b=ZXCvINQhVrQBAwBCMAqyp5Csf5HmMmimyB/gRdExtnr9YvYaR9CP+fEP4Kl86AAocG P/j9AyIWF1JEUyplCQ71GHYgMRMA/TQH9IjqmI6Afgm1k/aAX8akzlU3JEJodkXdVLsR BXVaq66Llg00nV1sgoNfZuUgK78nmT5zi7rp0R6272C+r7u0JbMyrm1/aMZDO5sVbN4A fUX/np5uP9Dg1rFTdDJf60zwLBkMTvc9LMtV9+iviZSwoZyI89e8KI858zwzFwnUycdJ myvABIB3olXSMWWxjaHsaDqQ779fuxUUdVAWonh9hwxl8Fz7k2w+AEmxejsuB+4yVUV6 pqxQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:user-agent:in-reply-to :content-disposition:mime-version:references:message-id:subject:cc :to:from:date; bh=NcyN0yHV1Ys9MIRPSBDNu4WzDmgXwJtP8q9wZrDgVXM=; b=vtHdvcSA4OJwHKZiq427SxuOPmOlI2O0iV7irEPdp5RGgaEUAnCPdyxlU1czOQsaRN hMbn4DvD+cacLpweGsnAkdIxwdR2Y7IMQtZ8gIukpfCJlS3JBpgd/B15q2f//HoydUIQ XAOzklGprHLT/WvqUXLl4ERZaJqGKrZA9bm4U8UrxaSM5vY6OqhV/LNlmh5ZZqP7DHq5 riUug6G2eosDrlEEtUwrPOSlydeq0/3CAuVrUgt4ukle0IL8cIgNKn7FYfCkLKGeCvlm 3cGu8vpAWDqLcENbvpmmO1kQdpKOOOfwQ/Sqb+IhQiJY5sTVJazR5jc6BkohuZ5J2Fbz zw2Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g12si7003373pgd.567.2019.02.06.13.33.46; Wed, 06 Feb 2019 13:34:03 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726655AbfBFVcS (ORCPT + 99 others); Wed, 6 Feb 2019 16:32:18 -0500 Received: from ipmail06.adl2.internode.on.net ([150.101.137.129]:24220 "EHLO ipmail06.adl2.internode.on.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726037AbfBFVcR (ORCPT ); Wed, 6 Feb 2019 16:32:17 -0500 Received: from ppp59-167-129-252.static.internode.on.net (HELO dastard) ([59.167.129.252]) by ipmail06.adl2.internode.on.net with ESMTP; 07 Feb 2019 08:01:49 +1030 Received: from dave by dastard with local (Exim 4.80) (envelope-from ) id 1grUnL-0003Lg-7s; Thu, 07 Feb 2019 08:31:47 +1100 Date: Thu, 7 Feb 2019 08:31:47 +1100 From: Dave Chinner To: Ira Weiny Cc: lsf-pc@lists.linux-foundation.org, linux-rdma@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, John Hubbard , Jan Kara , Jerome Glisse , Dan Williams , Matthew Wilcox , Doug Ledford , Michal Hocko , Jason Gunthorpe Subject: Re: [LSF/MM TOPIC] Discuss least bad options for resolving longterm-GUP usage by RDMA Message-ID: <20190206213147.GA6173@dastard> References: <20190205175059.GB21617@iweiny-DESK2.sc.intel.com> <20190205180120.GC21617@iweiny-DESK2.sc.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190205180120.GC21617@iweiny-DESK2.sc.intel.com> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Feb 05, 2019 at 10:01:20AM -0800, Ira Weiny wrote: > I had an old invalid address for Jason Gunthorpe in my address book... > > Correcting his email in the thread. Probably should have cc'd linux-fsdevel, too, but it's too late for that now.... > On Tue, Feb 05, 2019 at 09:50:59AM -0800, 'Ira Weiny' wrote: > > > > The problem: Once we have pages marked as GUP-pinned how should various > > subsystems work with those markings. > > > > The current work for John Hubbards proposed solutions (part 1 and 2) is > > progressing.[1] But the final part (3) of his solution is also going to take > > some work. > > > > In Johns presentation he lists 3 alternatives for gup-pinned pages: > > > > 1) Hold off try_to_unmap > > 2) Allow writeback while pinned (via bounce buffers) > > [Note this will not work for DAX] > > 3) Use a "revocable reservation" (or lease) on those pages > > 4) Pin the blocks as busy in the FS allocator > > > > The problem with lease's on pages used by RDMA is that the references to > > these pages is not local to the machine. Once the user has been given access > > to the page they, through the use of a remote tokens, give a reference to that > > page to remote nodes. This is the core essence of RDMA, and like it or not, > > something which is increasingly used by major Linux users. > > > > Therefore we need to discuss the extent by which leases are appropriate and > > what happens should a lease be revoked which a user does not respond to. > > > > As John Hubbard put it: > > > > "Other filesystem features that need to replace the page with a new one can > > be inhibited for pages that are GUP-pinned. This will, however, alter and > > limit some of those filesystem features. The only fix for that would be to > > require GUP users monitor and respond to CPU page table updates. Subsystems > > such as ODP and HMM do this, for example. This aspect of the problem is > > still under discussion." > > > > -- John Hubbard[2] > > > > The following people have been involved in previous conversations and would be key to > > the face to face discussion. > > > > John Hubbard > > Jan Kara > > Dave Chinner Just FYI, I won't be at LSFMM. Puerto Rico is about as physically far away from me as you can get on this planet. There's 40 hours in transit from airport to airport, and that doesn't include the 5 hours of travel the day before (and hence overnight stay) to be able to get to the first airport in time for the first flight. I'm looking at a transit time - if all goes well - of over 60 hours just to get to the conference. And it's looks like it will be just as bad on the way back. 6 days travel for a 2 day conference makes no sense at all. Cheers, Dave. -- Dave Chinner david@fromorbit.com