Received: by 2002:ad5:474a:0:0:0:0:0 with SMTP id i10csp2738391imu; Mon, 17 Dec 2018 07:04:39 -0800 (PST) X-Google-Smtp-Source: AFSGD/UYKU72qqOt1gX9wkkWRSZ5lEg4X0q8KPsUCgqP8CUzW6jtc3zsMo2rakj9K4VBQNqiwbmA X-Received: by 2002:a63:85c6:: with SMTP id u189mr11960071pgd.156.1545059079207; Mon, 17 Dec 2018 07:04:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1545059079; cv=none; d=google.com; s=arc-20160816; b=0FEO/xvElXZNbaYGtmm9jZyvNcTHeFqyd1T1kovX/VjJno1QVr/Hs0PEVa1Mf+dvpe UPmOJsmzJdIDB5fLMGsHRDT6ctetuiAsYryCMqEhfR7lBWucFeXc3c9+hwyEbb9+d1EO M2VTV6jUvjDdNngquvBpoJKpX+PJ/cOY3cU68MloXfg2XG1tCU+IX/RtGVNlJoV77dhn Cup3P8WuG1nIGsP92ovP9hrnnlziygQZrafyKX6RodNJWy79OUz69jbe4nAYRAIGNKFA QBsmSl3bjnyeYxklCdVBv5TnJbQJqCqdgU4m+ogpVN1aBAs2B4JqEqDXLEQOwVOZbG1E l98g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:user-agent:message-id:references :in-reply-to:organization:subject:cc:to:from:date :content-transfer-encoding:mime-version; bh=C8j0za7FFbqUekO21SCvjHgI+fzBGnz/xnUsn5E0eGc=; b=OgHtTHtLISaLaqN2okdiggflrSS39nTYCpRZVx5jlW5MEzUwjGZyVxJrY+NbRYrBF8 DLwel6ifQax5X+klwGKuQO6P/SX+5wPmDwTkAmlo6xoIpJChsawDYZHNQ6EpATFKihp8 MB1Qb+wjm4mT60SIwQmRDeRHwN05sD+Doeshz2FM30Skz48QWSApDUYa1XJ3FF18Jexz m1+sv69RnnMAFZdj9RTfRf3BeLzs4NuiZdND0oZVFsdaSUyHyh1MLjq8GabwNpwtuw+D uawbcyOajWhBlSwOFKP+cgW1cGqaenk3ja58kQyRrz06G5KuJ/epJvsqpdv7Aop0rSye T3og== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id k22si10578459pls.14.2018.12.17.07.04.21; Mon, 17 Dec 2018 07:04:39 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2387628AbeLQPAg (ORCPT + 99 others); Mon, 17 Dec 2018 10:00:36 -0500 Received: from mailgate-2.ics.forth.gr ([139.91.1.5]:42914 "EHLO mailgate-2.ics.forth.gr" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1733240AbeLQPA1 (ORCPT ); Mon, 17 Dec 2018 10:00:27 -0500 Received: from av1.ics.forth.gr (av3in [139.91.1.77]) by mailgate-2.ics.forth.gr (8.14.4/ICS-FORTH/V10-1.8-GATE) with ESMTP id wBHExMmr001882; Mon, 17 Dec 2018 14:59:24 GMT X-AuditID: 8b5b9d4d-90dff7000000235c-08-5c17b9c9032a Received: from enigma.ics.forth.gr (webmail.ics.forth.gr [139.91.1.35]) by av1.ics.forth.gr (SMTP Outbound / FORTH / ICS) with SMTP id 8B.2C.09052.9C9B71C5; Mon, 17 Dec 2018 16:59:22 +0200 (EET) Received: from webmail.ics.forth.gr (localhost [127.0.0.1]) by enigma.ics.forth.gr (8.15.1//ICS-FORTH/V10.5.0C-EXTNULL-SSL-SASL) with ESMTP id wBHExJbC026827; Mon, 17 Dec 2018 16:59:20 +0200 X-ICS-AUTH-INFO: Authenticated user: at ics.forth.gr MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Mon, 17 Dec 2018 16:59:19 +0200 From: Nick Kossifidis To: Logan Gunthorpe Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-sh@vger.kernel.org, Rob Herring , Albert Ou , Andrew Waterman , Arnd Bergmann , Palmer Dabbelt , Stephen Bates , Zong Li , Olof Johansson , Andrew Morton , Michael Clark , Christoph Hellwig Subject: Re: [PATCH v2 6/6] RISC-V: Implement sparsemem Organization: FORTH In-Reply-To: <20181015175702.9036-7-logang@deltatee.com> References: <20181015175702.9036-1-logang@deltatee.com> <20181015175702.9036-7-logang@deltatee.com> Message-ID: <4b591ba933363e29392dba218ef63267@mailhost.ics.forth.gr> X-Sender: mick@mailhost.ics.forth.gr User-Agent: Roundcube Webmail/1.1.2 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrDIsWRmVeSWpSXmKPExsXSHc2orHtqp3iMwbMJrBZz1q9hs9hyO8Zi 6+9Z7BZ/Jx1jt1i5+iiTxabH11gtLu+aw2Zxb81/Vottn1vYLOb8mcJssaT1OJvFxw2fWSxO Xf/MZrF5wgJWi/97drBbfD4zj9Xi+cpeNgdBjz2nZzF7/P41idHj2dVnjB6HO76we2xa1cnm senTJHaPEzN+s3hsXlLvceVEE6vH7psNbB6rOvexe2xb/JLV41LzdXaPz5vkAviiuGxSUnMy y1KL9O0SuDKe/j3HWNCiUbHoeyNbA2OTQhcjJ4eEgInE49aZzF2MXBxCAkcYJXqON7NDOIcY JX68XcUKUWUqMXtvJyOIzSsgKHFy5hMWEJtZwEJi6pX9jBC2vETz1tnMIDaLgKpE+/eN7CA2 m4CmxPxLB4HqOThEgOwtr3xB5jMLfGCRmPvhItgcYQFziYePu5hAbH4BYYlPdy+C7eUEmt+2 +AdYjZBAqsS8S2ehbnCRWHRxBQvEbSoSH34/ANslKqAs8eLEdNYJjEKzkJw6C8mps5CcuoCR eRWjQGKZsV5mcrFeWn5RSYZeetEmRnAcz/XdwXhugf0hRgEORiUeXs39YjFCrIllxZW5wIDh YFYS4e21BArxpiRWVqUW5ccXleakFh9ilOZgURLnPfwiPEhIID2xJDU7NbUgtQgmy8TBKdXA WGsvmfPvnOm0uCdu15l5Feb+br7ieOLMMimxeAM+7lef9h3b7bBjUfjpdn0Gv86qjnOHVFaL NzMVV59fHJ7V//VFlIW74B2+Vfsnu8o8Fk4VqvTaYZuwovJYkrhT9jT+NL2amteHJ+7I5rR8 dnbiis2+Z1zZVU/saDghPGORhMt9zm/5H4SvKbEUZyQaajEXFScCAEfgw+ffAgAA X-Greylist: inspected by milter-greylist-4.6.2 (mailgate-2.ics.forth.gr [139.91.1.5]); Mon, 17 Dec 2018 14:59:25 +0000 (GMT) for IP:'139.91.1.77' DOMAIN:'av3in' HELO:'av1.ics.forth.gr' FROM:'mick@ics.forth.gr' RCPT:'' X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mailgate-2.ics.forth.gr [139.91.1.5]); Mon, 17 Dec 2018 14:59:25 +0000 (GMT) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hello Logan, Στις 2018-10-15 20:57, Logan Gunthorpe έγραψε: > This patch implements sparsemem support for risc-v which helps pave the > way for memory hotplug and eventually P2P support. > > We introduce Kconfig options for virtual and physical address bits > which > are used to calculate the size of the vmemmap and set the > MAX_PHYSMEM_BITS. > > The vmemmap is located directly before the VMALLOC region and sized > such that we can allocate enough pages to populate all the virtual > address space in the system (similar to the way it's done in arm64). > > During initialization, call memblocks_present() and sparse_init(), > and provide a stub for vmemmap_populate() (all of which is similar to > arm64). > > Signed-off-by: Logan Gunthorpe > Reviewed-by: Palmer Dabbelt > Cc: Albert Ou > Cc: Andrew Waterman > Cc: Olof Johansson > Cc: Michael Clark > Cc: Rob Herring > Cc: Zong Li > --- > arch/riscv/Kconfig | 23 +++++++++++++++++++++++ > arch/riscv/include/asm/pgtable.h | 21 +++++++++++++++++---- > arch/riscv/include/asm/sparsemem.h | 11 +++++++++++ > arch/riscv/kernel/setup.c | 4 +++- > arch/riscv/mm/init.c | 8 ++++++++ > 5 files changed, 62 insertions(+), 5 deletions(-) > create mode 100644 arch/riscv/include/asm/sparsemem.h > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig > index a344980287a5..a1b5d758a542 100644 > --- a/arch/riscv/Kconfig > +++ b/arch/riscv/Kconfig > @@ -52,12 +52,32 @@ config ZONE_DMA32 > bool > default y if 64BIT > > +config VA_BITS > + int > + default 32 if 32BIT > + default 39 if 64BIT > + > +config PA_BITS > + int > + default 34 if 32BIT > + default 56 if 64BIT > + > config PAGE_OFFSET > hex > default 0xC0000000 if 32BIT && MAXPHYSMEM_2GB > default 0xffffffff80000000 if 64BIT && MAXPHYSMEM_2GB > default 0xffffffe000000000 if 64BIT && MAXPHYSMEM_128GB > > +config ARCH_FLATMEM_ENABLE > + def_bool y > + > +config ARCH_SPARSEMEM_ENABLE > + def_bool y > + select SPARSEMEM_VMEMMAP_ENABLE > + > +config ARCH_SELECT_MEMORY_MODEL > + def_bool ARCH_SPARSEMEM_ENABLE > + > config STACKTRACE_SUPPORT > def_bool y > > @@ -92,6 +112,9 @@ config PGTABLE_LEVELS > config HAVE_KPROBES > def_bool n > > +config HAVE_ARCH_PFN_VALID > + def_bool y > + > menu "Platform type" > > choice > diff --git a/arch/riscv/include/asm/pgtable.h > b/arch/riscv/include/asm/pgtable.h > index 16301966d65b..e1162336f5ea 100644 > --- a/arch/riscv/include/asm/pgtable.h > +++ b/arch/riscv/include/asm/pgtable.h > @@ -89,6 +89,23 @@ extern pgd_t swapper_pg_dir[]; > #define __S110 PAGE_SHARED_EXEC > #define __S111 PAGE_SHARED_EXEC > > +#define VMALLOC_SIZE (KERN_VIRT_SIZE >> 1) > +#define VMALLOC_END (PAGE_OFFSET - 1) > +#define VMALLOC_START (PAGE_OFFSET - VMALLOC_SIZE) > + > +/* > + * Roughly size the vmemmap space to be large enough to fit enough > + * struct pages to map half the virtual address space. Then > + * position vmemmap directly below the VMALLOC region. > + */ > +#define VMEMMAP_SHIFT \ > + (CONFIG_VA_BITS - PAGE_SHIFT - 1 + STRUCT_PAGE_MAX_SHIFT) > +#define VMEMMAP_SIZE (1UL << VMEMMAP_SHIFT) > +#define VMEMMAP_END (VMALLOC_START - 1) > +#define VMEMMAP_START (VMALLOC_START - VMEMMAP_SIZE) > + > +#define vmemmap ((struct page *)VMEMMAP_START) > + > /* > * ZERO_PAGE is a global shared page that is always zero, > * used for zero-mapped memory areas, etc. > @@ -411,10 +428,6 @@ static inline void pgtable_cache_init(void) > /* No page table caches to initialize */ > } > > -#define VMALLOC_SIZE (KERN_VIRT_SIZE >> 1) > -#define VMALLOC_END (PAGE_OFFSET - 1) > -#define VMALLOC_START (PAGE_OFFSET - VMALLOC_SIZE) > - > /* > * Task size is 0x40000000000 for RV64 or 0xb800000 for RV32. > * Note that PGDIR_SIZE must evenly divide TASK_SIZE. > diff --git a/arch/riscv/include/asm/sparsemem.h > b/arch/riscv/include/asm/sparsemem.h > new file mode 100644 > index 000000000000..215530b24336 > --- /dev/null > +++ b/arch/riscv/include/asm/sparsemem.h > @@ -0,0 +1,11 @@ > +/* SPDX-License-Identifier: GPL-2.0 */ > + > +#ifndef __ASM_SPARSEMEM_H > +#define __ASM_SPARSEMEM_H > + > +#ifdef CONFIG_SPARSEMEM > +#define MAX_PHYSMEM_BITS CONFIG_PA_BITS > +#define SECTION_SIZE_BITS 30 Having memory blocks of a minimum size of 1GB doesn't make much sense. It makes it harder to implement hotplug on top of this since we'll only able to add/remove 1GB at a time. ARM used to do the same and they switched to 27bits (https://patchwork.kernel.org/patch/9172845/), ARM64 still uses 1GB, x86 also uses 27bits and most archs also use something below 30. I believe we should go for 27bits as well or even better have this as a compile time option. BTW memblocks_present is on master now (got merged 3 days ago). Regards, N.