Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1761160AbYBMUgz (ORCPT ); Wed, 13 Feb 2008 15:36:55 -0500 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1755417AbYBMUgo (ORCPT ); Wed, 13 Feb 2008 15:36:44 -0500 Received: from relay1.sgi.com ([192.48.171.29]:40894 "EHLO relay.sgi.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1756473AbYBMUgm (ORCPT ); Wed, 13 Feb 2008 15:36:42 -0500 Date: Wed, 13 Feb 2008 12:36:42 -0800 (PST) From: Christoph Lameter X-X-Sender: clameter@schroedinger.engr.sgi.com To: Jason Gunthorpe cc: Roland Dreier , Rik van Riel , steiner@sgi.com, Andrea Arcangeli , a.p.zijlstra@chello.nl, izike@qumranet.com, linux-kernel@vger.kernel.org, avi@qumranet.com, linux-mm@kvack.org, daniel.blueman@quadrics.com, Robin Holt , general@lists.openfabrics.org, Andrew Morton , kvm-devel@lists.sourceforge.net Subject: Re: [ofa-general] Re: Demand paging for memory regions In-Reply-To: <20080213195144.GE31435@obsidianresearch.com> Message-ID: References: <47B2174E.5000708@opengridcomputing.com> <20080212232329.GC31435@obsidianresearch.com> <20080213012638.GD31435@obsidianresearch.com> <20080213032533.GC32047@obsidianresearch.com> <20080213195144.GE31435@obsidianresearch.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 1397 Lines: 29 On Wed, 13 Feb 2008, Jason Gunthorpe wrote: > Unfortunately it really has little to do with the drivers - changes, > for instance, need to be made to support this in the user space MPI > libraries. The RDMA ops do not pass through the kernel, userspace > talks directly to the hardware which complicates building any sort of > abstraction. Ok so the notifiers have to be handed over to the user space library that has the function of the device driver here... > That is where I think you run into trouble, if you ask the MPI people > to add code to their critical path to support swapping they probably > will not be too interested. At a minimum to support your idea you need > to check on every RDMA if the remote page is mapped... Plus the > overheads Christian was talking about in the OOB channel(s). You only need to check if a handle has been receiving invalidates. If not then you can just go ahead as now. You can use the notifier to take down the whole region if any reclaim occur against it (probably best and simples to implement approach). Then you mark the handle so that the mapping is reestablished before the next operation. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/