Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932516AbbENHjW (ORCPT ); Thu, 14 May 2015 03:39:22 -0400 Received: from cantor2.suse.de ([195.135.220.15]:50461 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753064AbbENHjU (ORCPT ); Thu, 14 May 2015 03:39:20 -0400 Message-ID: <55545124.7090804@suse.cz> Date: Thu, 14 May 2015 09:39:16 +0200 From: Vlastimil Babka User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: Benjamin Herrenschmidt CC: Christoph Lameter , Rik van Riel , Jerome Glisse , "Paul E. McKenney" , linux-kernel@vger.kernel.org, linux-mm@kvack.org, jglisse@redhat.com, mgorman@suse.de, aarcange@redhat.com, airlied@redhat.com, aneesh.kumar@linux.vnet.ibm.com, Cameron Buschardt , Mark Hairgrove , Geoffrey Gerfin , John McKenna , akpm@linux-foundation.org Subject: Re: Interacting with coherent memory on external devices References: <20150424150829.GA3840@gmail.com> <20150424164325.GD3840@gmail.com> <20150424171957.GE3840@gmail.com> <20150424192859.GF3840@gmail.com> <20150425114633.GI5561@linux.vnet.ibm.com> <20150427154728.GA26980@gmail.com> <553E6405.1060007@redhat.com> <1430178843.16571.134.camel@kernel.crashing.org> <55535B6E.5090700@suse.cz> <1431560326.20218.94.camel@kernel.crashing.org> In-Reply-To: <1431560326.20218.94.camel@kernel.crashing.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 1614 Lines: 33 On 05/14/2015 01:38 AM, Benjamin Herrenschmidt wrote: > On Wed, 2015-05-13 at 16:10 +0200, Vlastimil Babka wrote: >> Sorry for reviving oldish thread... > > Well, that's actually appreciated since this is constructive discussion > of the kind I was hoping to trigger initially :-) I'll look at I hoped so :) > ZONE_MOVABLE, I wasn't aware of its existence. > > Don't we still have the problem that ZONEs must be somewhat contiguous > chunks ? Ie, my "CAPI memory" will be interleaved in the physical > address space somewhat.. This is due to the address space on some of > those systems where you'll basically have something along the lines of: > > [ node 0 mem ] [ node 0 CAPI dev ] .... [ node 1 mem] [ node 1 CAPI dev] ... Oh, I see. The VM code should cope with that, but some operations would be inefficiently looping over the holes in the CAPI zone by 2MB pageblock per iteration. This would include compaction scanning, which would suck if you need those large contiguous allocations as you said. Interleaving works better if it's done with a smaller granularity. But I guess you could just represent the CAPI as multiple NUMA nodes, each with single ZONE_MOVABLE zone. Especially if "node 0 CAPI dev" and "node 1 CAPI dev" differs in other characteristics than just using a different range of PFNs... otherwise what's the point of this split anyway? -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/