Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1762394AbYBOUCW (ORCPT ); Fri, 15 Feb 2008 15:02:22 -0500 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1754815AbYBOUCO (ORCPT ); Fri, 15 Feb 2008 15:02:14 -0500 Received: from netops-testserver-3-out.sgi.com ([192.48.171.28]:51118 "EHLO relay.sgi.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1754677AbYBOUCN (ORCPT ); Fri, 15 Feb 2008 15:02:13 -0500 Date: Fri, 15 Feb 2008 12:02:12 -0800 (PST) From: Christoph Lameter X-X-Sender: clameter@schroedinger.engr.sgi.com To: Caitlin Bestler cc: linux-kernel@vger.kernel.org, avi@qumranet.com, linux-mm@kvack.org, general@lists.openfabrics.org, kvm-devel@lists.sourceforge.net Subject: RE: [ofa-general] Re: Demand paging for memory regions In-Reply-To: <78C9135A3D2ECE4B8162EBDCE82CAD77030E25F1@nekter> Message-ID: References: <47B2174E.5000708@opengridcomputing.com> <8A71B368A89016469F72CD08050AD334026D5C23@maui.asicdesigners.com> <47B45994.7010805@opengridcomputing.com> <469958e00802141217i3a3d16a1k1232d69b8ba54471@mail.gmail.com> <469958e00802141443g33448abcs3efa6d6c4aec2b56@mail.gmail.com> <78C9135A3D2ECE4B8162EBDCE82CAD77030E2456@nekter> <78C9135A3D2ECE4B8162EBDCE82CAD77030E25BA@nekter> <78C9135A3D2ECE4B8162EBDCE82CAD77030E25F1@nekter> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 1625 Lines: 35 On Fri, 15 Feb 2008, Caitlin Bestler wrote: > So that would mean that mlock is used by the application before it > registers memory for direct access, and then it is up to the RDMA > layer and the OS to negotiate actual pinning of the addresses for > whatever duration is required. Right. > There is no *protocol* barrier to replacing pages within a Memory > Region as long as it is done in a way that keeps the content of > those page coherent. But existing devices have their own ideas > on how this is done and existing devices are notoriously poor at > learning new tricks. Hmmmm.. Okay. But that is mainly a device driver maintenance issue. > Merely mlocking pages deals with the end-to-end RDMA semantics. > What still needs to be addressed is how a fastpath interface > would dynamically pin and unpin. Yielding pins for short-term > suspensions (and flushing cached translations) deals with the > rest. Understanding the range of support that existing devices > could provide with software updates would be the next step if > you wanted to pursue this. That is addressed on the VM level by the mmu_notifier which started this whole thread. The RDMA layers need to subscribe to this notifier and then do whatever the hardware requires to unpin and pin memory. I can only go as far as dealing with the VM layer. If you have any issues there I'd be glad to help. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/