Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752183AbcLGLZu (ORCPT ); Wed, 7 Dec 2016 06:25:50 -0500 Received: from szxga01-in.huawei.com ([58.251.152.64]:9493 "EHLO szxga01-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751675AbcLGLZt (ORCPT ); Wed, 7 Dec 2016 06:25:49 -0500 Message-ID: <5847F0AA.5050109@huawei.com> Date: Wed, 7 Dec 2016 19:21:14 +0800 From: Xishi Qiu User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: Scott Branden CC: Arnd Bergmann , Russell King , Catalin Marinas , Will Deacon , Ard Biesheuvel , Mark Rutland , , BCM Kernel Feedback , Tang Chen , , , "Luck, Tony" Subject: Re: [RFC PATCH 0/2] arm64: memory-hotplug: Add Memory Hotplug support References: <1480637999-4320-1-git-send-email-scott.branden@broadcom.com> <5840D333.9050801@huawei.com> <9cfa95ca-a9e9-299d-d020-7e565562dcf5@broadcom.com> <5840E659.6000104@huawei.com> <0f24c35b-10d5-d857-356d-01a21be48c2c@broadcom.com> In-Reply-To: <0f24c35b-10d5-d857-356d-01a21be48c2c@broadcom.com> Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: 7bit X-Originating-IP: [10.177.25.179] X-CFilter-Loop: Reflected Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 4816 Lines: 153 On 2016/12/7 16:43, Scott Branden wrote: > Hi Xishi, > > I followed you suggestions and found pfn_valid is always true. Answers to your questions inline. > > I could keep debugging this but hope Marcin sends out some code - I'm quite willing to test and help clean up the patchset. > > On 16-12-01 07:11 PM, Xishi Qiu wrote: >> On 2016/12/2 10:38, Scott Branden wrote: >> >>> Hi Xishi, >>> >>> Thanks for the reply - please see comments below. >>> >>> On 16-12-01 05:49 PM, Xishi Qiu wrote: >>>> On 2016/12/2 8:19, Scott Branden wrote: >>>> >>>>> This patchset is sent for comment to add memory hotplug support for ARM64 >>>>> based platforms. It follows hotplug code added for other architectures >>>>> in the linux kernel. >>>>> >>>>> I tried testing the memory hotplug feature following documentation from >>>>> Documentation/memory-hotplug.txt. I don't think it is working as expected >>>>> - see below: >>>>> >>>>> To add memory to the system I did the following: >>>>> echo 0x400000000 > /sys/devices/system/memory/probe >>>>> >>>>> The memory is displayed as system ram: >>>>> cat /proc/iomem: >>>>> 74000000-77ffffff : System RAM >>>>> 74080000-748dffff : Kernel code >>>>> 74950000-749d2fff : Kernel data >>>>> 400000000-43fffffff : System RAM >>>>> >>>>> But does not seem to be added to the kernel memory. >>>>> /proc/meminfo did not change. >>>>> >>>>> What else needs to be done so the memory is added to the kernel memory >>>>> pool for normal allocation? >>>>> >>>> >>>> Hi Scott, >>>> >>>> Do you mean it still don't support hod-add after apply this patchset? >>> >>> After applying the patch it appears to partially support hot-add. Please let me know if you think it is working as expected? >>> >>> The memory probe functions in that the memory is registered with the system and shows up in /proc/iomem. But, the memory is not available in /proc/meminfo. Do you think something else needs to be adjusted for ARM64 to hotadd the memory >>> >>> I just found another clue: >>> under /sys/devices/system/memory I only see one memory entry (before or after I try to hotadd additional memory). >>> >>> /sys/devices/system/memory # ls >>> auto_online_blocks memory0 uevent >>> block_size_bytes probe >>> >>> In arch/arm64/include/asm/sparsemem.h if I change SECTION_SIZE_BITS from 30 to 28 and recompile I get the following: >>> /sys/devices/system/memory # ls >>> auto_online_blocks memory7 uevent >>> block_size_bytes probe >>> >>> >>> In arch/arm64/include/asm/sparsemem.h if I change SECTION_SIZE_BITS from 30 to 27 and recompile I get the following: >>> /sys/devices/system/memory # ls >>> auto_online_blocks memory14 uevent >>> block_size_bytes probe >>> >>> If looks to me like something is not working properly in the ARM64 implementation. I should expect to see multiple memoryX entries under /sys/devices/system/memory? >>> >> >> Hi Scott, >> >> 1. Do you enable the following configs? >> CONFIG_SPARSEMEM >> MEMORY_HOTPLUG >> CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE > Yes, these configs are enabled >> >> 2. I find you missed create mapping in arch_add_memory(), and x86 has it. > Could you please explain this further? The patch I submitted hass arch_add_memory identical to the ia64 implementation. Hi Scott, I think we should create page table first for the new hotadd memory. e.g. create_mapping_late(start, __phys_to_virt(start), size, PAGE_KERNEL); I don't know why ia64 don't have this step. CC Tony >> >> 3. We will add memblock first, so pfn_valid() maybe always return true(in the >> following function), and this will lead __add_section() failed. Please check >> it. > You are correct - pfn_valid always returns true. The function is in arch/arm64/mm/init.c and different than the one you indicated below: > > #ifdef CONFIG_HAVE_ARCH_PFN_VALID > int pfn_valid(unsigned long pfn) > { > return memblock_is_map_memory(pfn << PAGE_SHIFT); > } > EXPORT_SYMBOL(pfn_valid); > #endif > >> >> int pfn_valid(unsigned long pfn) >> { >> return (pfn & PFN_MASK) == pfn && memblock_is_memory(pfn << PAGE_SHIFT); >> } >> >> add_memory >> add_memory_resource >> memblock_add_node >> arch_add_memory >> __add_pages >> __add_section >> pfn_valid >> >> Thanks, >> Xishi Qiu >> >>> >>> >>>> >>>> Thanks, >>>> Xishi Qiu >>>> >>>>> Scott Branden (2): >>>>> arm64: memory-hotplug: Add MEMORY_HOTPLUG, MEMORY_HOTREMOVE, >>>>> MEMORY_PROBE >>>>> arm64: defconfig: enable MEMORY_HOTPLUG config options >>>>> >>>>> arch/arm64/Kconfig | 10 ++++++++++ >>>>> arch/arm64/configs/defconfig | 3 +++ >>>>> arch/arm64/mm/init.c | 42 ++++++++++++++++++++++++++++++++++++++++++ >>>>> 3 files changed, 55 insertions(+) >>>>> >>>> >>>> >>>> >>> >>> . >>> >> >> >> > > . >