Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752038AbbDUBqs (ORCPT ); Mon, 20 Apr 2015 21:46:48 -0400 Received: from szxga03-in.huawei.com ([119.145.14.66]:58966 "EHLO szxga03-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751350AbbDUBqp (ORCPT ); Mon, 20 Apr 2015 21:46:45 -0400 Message-ID: <5535A879.8050302@huawei.com> Date: Tue, 21 Apr 2015 09:31:37 +0800 From: Xishi Qiu User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: Yasuaki Ishimatsu CC: Gu Zheng , Andrew Morton , Yasuaki Ishimatsu , Kamezawa Hiroyuki , , Tang Chen , Xiexiuqi , Mel Gorman , David Rientjes , Linux MM , LKML Subject: Re: [PATCH 1/2 V2] memory-hotplug: fix BUG_ON in move_freepages() References: <5530E578.9070505@huawei.com> <5531679d.4642ec0a.1beb.3569@mx.google.com> <55345979.2020502@cn.fujitsu.com> <55346859.30605@huawei.com> <553472b0.4ad2ec0a.3abe.ffffd0f6@mx.google.com> <55347592.4050400@huawei.com> <55354434.2902ec0a.14ab.fffffff6@mx.google.com> In-Reply-To: <55354434.2902ec0a.14ab.fffffff6@mx.google.com> Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 7bit X-Originating-IP: [10.177.25.179] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A020202.5535A88A.0009,ss=1,re=0.000,recu=0.000,reip=0.000,cl=1,cld=1,fgs=0, ip=0.0.0.0, so=2013-05-26 15:14:31, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: 77543297141c0bf8bd5cacd7bbb27712 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3986 Lines: 138 On 2015/4/21 2:23, Yasuaki Ishimatsu wrote: > > On Mon, 20 Apr 2015 11:42:10 +0800 > Xishi Qiu wrote: > >> On 2015/4/20 11:29, Yasuaki Ishimatsu wrote: >> >>> >>> On Mon, 20 Apr 2015 10:45:45 +0800 >>> Xishi Qiu wrote: >>> >>>> On 2015/4/20 9:42, Gu Zheng wrote: >>>> >>>>> Hi Xishi, >>>>> On 04/18/2015 04:05 AM, Yasuaki Ishimatsu wrote: >>>>> >>>>>> >>>>>> Your patches will fix your issue. >>>>>> But, if BIOS reports memory first at node hot add, pgdat can >>>>>> not be initialized. >>>>>> >>>>>> Memory hot add flows are as follows: >>>>>> >>>>>> add_memory >>>>>> ... >>>>>> -> hotadd_new_pgdat() >>>>>> ... >>>>>> -> node_set_online(nid) >>>>>> >>>>>> When calling hotadd_new_pgdat() for a hot added node, the node is >>>>>> offline because node_set_online() is not called yet. So if applying >>>>>> your patches, the pgdat is not initialized in this case. >>>>> >>>>> Ishimtasu's worry is reasonable. And I am afraid the fix here is a bit >>>>> over-kill. >>>>> >>>>>> >>>>>> Thanks, >>>>>> Yasuaki Ishimatsu >>>>>> >>>>>> On Fri, 17 Apr 2015 18:50:32 +0800 >>>>>> Xishi Qiu wrote: >>>>>> >>>>>>> Hot remove nodeXX, then hot add nodeXX. If BIOS report cpu first, it will call >>>>>>> hotadd_new_pgdat(nid, 0), this will set pgdat->node_start_pfn to 0. As nodeXX >>>>>>> exists at boot time, so pgdat->node_spanned_pages is the same as original. Then >>>>>>> free_area_init_core()->memmap_init() will pass a wrong start and a nonzero size. >>>>> >>>>> As your analysis said the root cause here is passing a *0* as the node_start_pfn, >>>>> then the chaos occurred when init the zones. And this only happens to the re-hotadd >>>>> node, so how about using the saved *node_start_pfn* (via get_pfn_range_for_nid(nid, &start_pfn, &end_pfn)) >>>>> instead if we find "pgdat->node_start_pfn == 0 && !node_online(XXX)"? >>>>> >>>>> Thanks, >>>>> Gu >>>>> >>>> >>>> Hi Gu, >>>> >>>> I first considered this method, but if the hot added node's start and size are different >>>> from before, it makes the chaos. >>>> >>> >>>> e.g. >>>> nodeXX (8-16G) >>>> remove nodeXX >>>> BIOS report cpu first and online it >>>> hotadd nodeXX >>>> use the original value, so pgdat->node_start_pfn is set to 8G, and size is 8G >>>> BIOS report mem(10-12G) >>>> call add_memory()->__add_zone()->grow_zone_span()/grow_pgdat_span() >>>> the start is still 8G, not 10G, this is chaos! >>> >>> If you set CONFIG_HAVE_MEMBLOCK_NODE_MAP, kernel shows the following >>> pr_info()'s message. >>> >>> void __paginginit free_area_init_node(int nid, unsigned long *zones_size, >>> unsigned long node_start_pfn, unsigned long *zholes_size) >>> { >>> ... >>> #ifdef CONFIG_HAVE_MEMBLOCK_NODE_MAP >>> get_pfn_range_for_nid(nid, &start_pfn, &end_pfn); >>> pr_info("Initmem setup node %d [mem %#018Lx-%#018Lx]\n", nid, >>> (u64)start_pfn << PAGE_SHIFT, ((u64)end_pfn << PAGE_SHIFT) - 1); >>> #endif >>> } >>> >>> Is the memory range of the message "8G - 16G"? >>> If so, the reason is that memblk is not deleted at memory hot remove. >>> >>> Thanks, >>> Yasuaki Ishimatsu >>> >> >> Hi Yasuaki, >> > >> By reading the code, I find memblk is not deleted at memory hot remove. >> I am not sure whether we should remove it. If remove it, we should also reset >> "arch_zone_lowest_possible_pfn", right? It seems a little complicated. > > I think memblk should be added/removed by hot adding/removing memory. > But, arch_zone_lowest_possible_pfn should not be changed. > Ok, thanks for your suggestion. > Thanks, > Yasuaki Ishimatsu > >> >> Thanks, >> Xishi Qiu >> >>> >>> >>>> >>>> Thanks, >>>> Xishi Qiu >>>> >>> >>> . >>> >> >> >> > > . > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/