Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754322Ab2E2QpC (ORCPT ); Tue, 29 May 2012 12:45:02 -0400 Received: from casper.infradead.org ([85.118.1.10]:58830 "EHLO casper.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751397Ab2E2QpA convert rfc822-to-8bit (ORCPT ); Tue, 29 May 2012 12:45:00 -0400 Message-ID: <1338309855.26856.130.camel@twins> Subject: Re: [PATCH 35/35] autonuma: page_autonuma From: Peter Zijlstra To: Andrea Arcangeli Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Hillf Danton , Dan Smith , Linus Torvalds , Andrew Morton , Thomas Gleixner , Ingo Molnar , Paul Turner , Suresh Siddha , Mike Galbraith , "Paul E. McKenney" , Lai Jiangshan , Bharata B Rao , Lee Schermerhorn , Rik van Riel , Johannes Weiner , Srivatsa Vaddagiri , Christoph Lameter Date: Tue, 29 May 2012 18:44:15 +0200 In-Reply-To: <1337965359-29725-36-git-send-email-aarcange@redhat.com> References: <1337965359-29725-1-git-send-email-aarcange@redhat.com> <1337965359-29725-36-git-send-email-aarcange@redhat.com> Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7BIT X-Mailer: Evolution 3.2.2- Mime-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2557 Lines: 66 On Fri, 2012-05-25 at 19:02 +0200, Andrea Arcangeli wrote: > Move the AutoNUMA per page information from the "struct page" to a > separate page_autonuma data structure allocated in the memsection > (with sparsemem) or in the pgdat (with flatmem). > > This is done to avoid growing the size of the "struct page" and the > page_autonuma data is only allocated if the kernel has been booted on > real NUMA hardware (or if noautonuma is passed as parameter to the > kernel). > Argh, please fold this change back into the series proper. If you want to keep it.. as it is its not really an improvement IMO, see below. > +struct page_autonuma { > + /* > + * FIXME: move to pgdat section along with the memcg and allocate > + * at runtime only in presence of a numa system. > + */ > + /* > + * To modify autonuma_last_nid lockless the architecture, > + * needs SMP atomic granularity < sizeof(long), not all archs > + * have that, notably some alpha. Archs without that requires > + * autonuma_last_nid to be a long. > + */ Looking at arch/alpha/include/asm/xchg.h it looks to have that just fine, so maybe we simply don't support SMP on those early Alphas that had that weirdness. > +#if BITS_PER_LONG > 32 > + int autonuma_migrate_nid; > + int autonuma_last_nid; > +#else > +#if MAX_NUMNODES >= 32768 > +#error "too many nodes" > +#endif > + /* FIXME: remember to check the updates are atomic */ > + short autonuma_migrate_nid; > + short autonuma_last_nid; > +#endif > + struct list_head autonuma_migrate_node; > + > + /* > + * To find the page starting from the autonuma_migrate_node we > + * need a backlink. > + */ > + struct page *page; > +}; This makes a shadow page frame of 32 bytes per page, or ~0.8% of memory. This isn't in fact an improvement. The suggestion done by Rik was to have something like a sqrt(nr_pages) (?) scaled array of such things containing the list_head and page pointer -- and leave the two nids in the regular page frame. Although I think you've got to fight the memcg people over that last word in struct page. That places a limit on the amount of pages that can be in migration concurrently, but also greatly reduces the memory overhead. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/