Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934510AbbDXUAW (ORCPT ); Fri, 24 Apr 2015 16:00:22 -0400 Received: from resqmta-ch2-10v.sys.comcast.net ([69.252.207.42]:56737 "EHLO resqmta-ch2-10v.sys.comcast.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933502AbbDXUAU (ORCPT ); Fri, 24 Apr 2015 16:00:20 -0400 Date: Fri, 24 Apr 2015 15:00:18 -0500 (CDT) From: Christoph Lameter X-X-Sender: cl@gentwo.org To: Jerome Glisse cc: Benjamin Herrenschmidt , paulmck@linux.vnet.ibm.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, jglisse@redhat.com, mgorman@suse.de, aarcange@redhat.com, riel@redhat.com, airlied@redhat.com, aneesh.kumar@linux.vnet.ibm.com, Cameron Buschardt , Mark Hairgrove , Geoffrey Gerfin , John McKenna , akpm@linux-foundation.org Subject: Re: Interacting with coherent memory on external devices In-Reply-To: <20150424192859.GF3840@gmail.com> Message-ID: References: <1429756456.4915.22.camel@kernel.crashing.org> <20150423161105.GB2399@gmail.com> <20150424150829.GA3840@gmail.com> <20150424164325.GD3840@gmail.com> <20150424171957.GE3840@gmail.com> <20150424192859.GF3840@gmail.com> Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2743 Lines: 56 On Fri, 24 Apr 2015, Jerome Glisse wrote: > > Still no answer as to why is that not possible with the current scheme? > > You keep on talking about pointers and I keep on responding that this is a > > matter of making the address space compatible on both sides. > > So if do that in a naive way, how can we migrate a chunk of memory to video > memory while still handling properly the case where CPU try to access that > same memory while it is migrated to the GPU memory. Well that the same issue that the migration code is handling which I submitted a long time ago to the kernel. > Without modifying a single line of mm code, the only way to do this is to > either unmap from the cpu page table the range being migrated or to mprotect > it in some way. In both case the cpu access will trigger some kind of fault. Yes that is how Linux migration works. If you can fix that then how about improving page migration in Linux between NUMA nodes first? > This is not the behavior we want. What we want is same address space while > being able to migrate system memory to device memory (who make that decision > should not be part of that discussion) while still gracefully handling any > CPU access. Well then there could be a situation where you have concurrent write access. How do you reconcile that then? Somehow you need to stall one or the other until the transaction is complete. > This means if CPU access it we want to migrate memory back to system memory. > To achieve this there is no way around adding couple of if inside the mm > page fault code path. Now do you want each driver to add its own if branch > or do you want a common infrastructure to do just that ? If you can improve the page migration in general then we certainly would love that. Having faultless migration is certain a good thing for a lot of functionality that depends on page migration. > As i keep saying the solution you propose is what we have today, today we > have fake share address space through the trick of remapping system memory > at same address inside the GPU address space and also enforcing the use of > a special memory allocator that goes behind the back of mm code. Hmmm... I'd like to know more details about that. > As you pointed out, not using GPU memory is a waste and we want to be able > to use it. Now Paul have more sofisticated hardware that offer oportunities > to do thing in a more transparent and efficient way. Does this also work between NUMA nodes in a Power8 system? -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/