Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1760132AbaGOXRi (ORCPT ); Tue, 15 Jul 2014 19:17:38 -0400 Received: from mail-ie0-f170.google.com ([209.85.223.170]:57137 "EHLO mail-ie0-f170.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1759322AbaGOXRd (ORCPT ); Tue, 15 Jul 2014 19:17:33 -0400 Date: Tue, 15 Jul 2014 16:17:31 -0700 (PDT) From: David Rientjes X-X-Sender: rientjes@chino.kir.corp.google.com To: Dave Hansen cc: Andrew Morton , Andrea Arcangeli , Mel Gorman , Rik van Riel , "Kirill A. Shutemov" , Bob Liu , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [patch] mm, thp: only collapse hugepages to nodes with affinity In-Reply-To: <53C4B251.5000505@intel.com> Message-ID: References: <53C4B251.5000505@intel.com> User-Agent: Alpine 2.02 (DEB 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, 14 Jul 2014, Dave Hansen wrote: > > + if (node == NUMA_NO_NODE) { > > + node = page_to_nid(page); > > + } else { > > + int distance = node_distance(page_to_nid(page), node); > > + > > + /* > > + * Do not migrate to memory that would not be reclaimed > > + * from. > > + */ > > + if (distance > RECLAIM_DISTANCE) > > + goto out_unmap; > > + } > > Isn't the reclaim behavior based on zone_reclaim_mode and not > RECLAIM_DISTANCE directly? And isn't that reclaim behavior disabled by > default? > Seems that RECLAIM_DISTANCE has taken on a life of its own independent of zone_reclaim_mode as a heuristic, such as its use in creating sched domains which would be unrelated. > I think you should at least be consulting zone_reclaim_mode. > Good point, and it matches what the comment is saying about whether we'd actually reclaim from the remote node to allocate thp on fault or not. I'll add it. After this change, we'll also need to consider the behavior of thp at fault and whether remote HPAGE_PMD_SIZE memory when local memory is low/fragmented is better than local PAGE_SIZE memory. In my page fault latency testing on true NUMA machines it's convincing that it's not. This makes me believe that, somewhat similar to this patch, when we allocate thp memory at fault and zone_reclaim_mode is non-zero that we should set only nodes with numa_node_id() <= RECLAIM_DISTANCE and then otherwise fallback to the PAGE_SIZE fault path. I've been hesitant to make that exact change, though, because it's a systemwide setting and I really hope to avoid a prctl() that controls zone reclaim for a particular process. Perhaps the NUMA balancing work makes this more dependable. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/