Received: by 2002:a25:8b12:0:0:0:0:0 with SMTP id i18csp3944988ybl; Mon, 12 Aug 2019 08:53:34 -0700 (PDT) X-Google-Smtp-Source: APXvYqyIRS27cSSmMN2f7N8h+SVjqmcFil8jFi13r8oOJwXJTT8NwGiG0cf08beGp9H7kfyv/M4j X-Received: by 2002:a17:902:e613:: with SMTP id cm19mr31393398plb.299.1565625214020; Mon, 12 Aug 2019 08:53:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1565625214; cv=none; d=google.com; s=arc-20160816; b=0NT0oZCjbBV4AmrzDnYUBcPzGiOh3Y0j4gyvoti9XL2gj36w+vQKsHSac4oVQvYFIh 1K/zmkHfjhl5apkGeNOlFIsDqbiu8lB1SvETM+zUdeV5vbLsMJJ8OE4jV+1+Mr/5+UGc W1BaH7IN9oFBI4ZWKlfpytXb3D2/bTHjgfa9Ve7ofQK4L3Hrv96bgrlrZtn0fra6cEvW CVHI2zS0tHrtlr7WAHP/uEz/aI/xxvWO/YjaXjJDY106ErznrZqh9GgRyd2XFPh4bB2g eOUtNmki8Ayreii5wpGPJNeJu2yT+Brv+PA0ZxAodbtu+G7xfMYYJalfKgqZ7thLPvop wvmQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:subject:content-transfer-encoding :content-language:in-reply-to:mime-version:user-agent:date :message-id:from:references:cc:to; bh=Ol525TPzEUcgxJSYx6TOxzYD87erecsZVWaDI6H/HRw=; b=NqCHv+eY5ywStKFWv2PpHr2HA6UyzHThfInFpNc3BPpnNtfp2e6vvODTQc9b9SepTN ibRUDF78EpZzcTsphha/7zp1qH0//W2cnDcwyw/fRH2KP7rg3+TH9rGSXxy+zjSN1Sda sjUrnFj1WPFro6P6549jzoVnYVwM7jNgON3X9cSU3k7c0TiNwCKqAvNLBv78xi3dMrkP gbQFaLgKUAOE5Leu/zuiUp8MJ52PpqaZ/QmoGi3UXSmAL9HU1hfcmYjlqbDnNPEHiSAl xGf7qES55rOGbj/LDCIXLzD4CODEA5xu1fzLDjBxtt1q2FZv7YoEzFi5tu8Ku/RxDJ7P riGA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id h96si12449209plb.51.2019.08.12.08.53.18; Mon, 12 Aug 2019 08:53:34 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726568AbfHLPwI (ORCPT + 99 others); Mon, 12 Aug 2019 11:52:08 -0400 Received: from ale.deltatee.com ([207.54.116.67]:36846 "EHLO ale.deltatee.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726219AbfHLPwI (ORCPT ); Mon, 12 Aug 2019 11:52:08 -0400 Received: from [172.16.1.162] by ale.deltatee.com with esmtp (Exim 4.89) (envelope-from ) id 1hxCbx-0006Ga-2Z; Mon, 12 Aug 2019 09:51:54 -0600 To: Greentime Hu Cc: greentime.hu@sifive.com, paul.walmsley@sifive.com, Rob Herring , Albert Ou , Andrew Waterman , Palmer Dabbelt , Linux Kernel Mailing List , Stephen Bates , Zong Li , Olof Johansson , linux-riscv@lists.infradead.org, Michael Clark , Christoph Hellwig References: <20190109203911.7887-1-logang@deltatee.com> <20190109203911.7887-3-logang@deltatee.com> <0926a261-520e-4c40-f926-ddd40bb8ce44@deltatee.com> From: Logan Gunthorpe Message-ID: <96156909-1453-d487-ff66-a041d67c74d6@deltatee.com> Date: Mon, 12 Aug 2019 09:51:50 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-CA Content-Transfer-Encoding: 8bit X-SA-Exim-Connect-IP: 172.16.1.162 X-SA-Exim-Rcpt-To: hch@lst.de, michaeljclark@mac.com, linux-riscv@lists.infradead.org, olof@lixom.net, zong@andestech.com, sbates@raithlin.com, linux-kernel@vger.kernel.org, palmer@sifive.com, andrew@sifive.com, aou@eecs.berkeley.edu, robh@kernel.org, paul.walmsley@sifive.com, greentime.hu@sifive.com, green.hu@gmail.com X-SA-Exim-Mail-From: logang@deltatee.com X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on ale.deltatee.com X-Spam-Level: X-Spam-Status: No, score=-8.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, GREYLIST_ISWHITE autolearn=ham autolearn_force=no version=3.4.2 Subject: Re: [PATCH v4 2/2] RISC-V: Implement sparsemem X-SA-Exim-Version: 4.2.1 (built Tue, 02 Aug 2016 21:08:31 +0000) X-SA-Exim-Scanned: Yes (on ale.deltatee.com) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2019-08-11 10:01 p.m., Greentime Hu wrote: > Hi Logan, > > Logan Gunthorpe 於 2019年8月10日 週六 上午3:03寫道: >> >> >> >> On 2019-08-09 11:01 a.m., Greentime Hu wrote: >>> Hi Logan, >>> >>> Logan Gunthorpe 於 2019年8月9日 週五 下午11:47寫道: >>>> >>>> >>>> >>>> On 2019-08-08 10:23 p.m., Greentime Hu wrote: >>>>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig >>>>> index 3f12b069af1d..208b3e14ccd8 100644 >>>>> --- a/arch/riscv/Kconfig >>>>> +++ b/arch/riscv/Kconfig >>>>> @@ -116,7 +116,8 @@ config PGTABLE_LEVELS >>>>> default 2 >>>>> >>>>> config HAVE_ARCH_PFN_VALID >>>>> - def_bool y >>>>> + bool >>>>> + default !SPARSEMEM_VMEMMAP >>>>> >>>>> menu "Platform type" >>>>> >>>>> diff --git a/arch/riscv/include/asm/page.h b/arch/riscv/include/asm/page.h >>>>> index 8ddb6c7fedac..6991f7a5a4a7 100644 >>>>> --- a/arch/riscv/include/asm/page.h >>>>> +++ b/arch/riscv/include/asm/page.h >>>>> @@ -93,16 +93,20 @@ extern unsigned long min_low_pfn; >>>>> #define virt_to_pfn(vaddr) (phys_to_pfn(__pa(vaddr))) >>>>> #define pfn_to_virt(pfn) (__va(pfn_to_phys(pfn))) >>>>> >>>>> +#if !defined(CONFIG_SPARSEMEM_VMEMMAP) >>>>> +#define pfn_valid(pfn) \ >>>>> + (((pfn) >= pfn_base) && (((pfn)-pfn_base) < max_mapnr)) >>>>> #define virt_to_page(vaddr) (pfn_to_page(virt_to_pfn(vaddr))) >>>>> #define page_to_virt(page) (pfn_to_virt(page_to_pfn(page))) >>>>> +#else >>>>> +#define virt_to_page(vaddr) ((struct page *)((((u64)vaddr - >>>>> va_pa_offset) / PAGE_SIZE) * sizeof(struct page) + VMEMMAP_START)) >>>>> +#define page_to_virt(pg) ((void *)(((((u64)pg - VMEMMAP_START) / >>>>> sizeof(struct page)) * PAGE_SIZE) + va_pa_offset)) >>>>> +#endif >>>> >>>> This doesn't make sense to me at all. It should always use pfn_to_page() >>>> for virt_to_page() and the generic pfn_to_page()/page_to_pfn() >>>> implementations essentially already do what you are doing in a cleaner >>>> way. So I'd be really surprised if this does anything at all. >>>> >>> >>> Thank you for point me out that. I just checked the generic >>> implementation and I should use that one. >>> Sorry I didn't check the generic one and just implement it again. >>> I think the only patch we need is the first part to use generic >>> pfn_valid(). I just tested it and yes it can boot successfully in dts >>> with hole. >>> >>> It will fail in this check ((pfn)-pfn_base) < max_mapnr. >> >> Sounds to me like max_mapnr is not set correctly. See the code in >> setup_bootmem(). Seems like 'mem_size' should be set to the largest >> memory block, not just the one that contains the kernel... >> >> >>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig >>> index 3f12b069af1d..208b3e14ccd8 100644 >>> --- a/arch/riscv/Kconfig >>> +++ b/arch/riscv/Kconfig >>> @@ -116,7 +116,8 @@ config PGTABLE_LEVELS >>> default 2 >>> >>> config HAVE_ARCH_PFN_VALID >>> - def_bool y >>> + bool >>> + default !SPARSEMEM_VMEMMAP >>> >>> menu "Platform type" >>> >>> diff --git a/arch/riscv/include/asm/page.h b/arch/riscv/include/asm/page.h >>> index 8ddb6c7fedac..80d28fa1e2eb 100644 >>> --- a/arch/riscv/include/asm/page.h >>> +++ b/arch/riscv/include/asm/page.h >>> @@ -100,8 +100,10 @@ extern unsigned long min_low_pfn; >>> #define page_to_bus(page) (page_to_phys(page)) >>> #define phys_to_page(paddr) (pfn_to_page(phys_to_pfn(paddr))) >>> >>> +#if !defined(CONFIG_SPARSEMEM_VMEMMAP) >>> #define pfn_valid(pfn) \ >>> (((pfn) >= pfn_base) && (((pfn)-pfn_base) < max_mapnr)) >>> +#endif >>> >>> #define ARCH_PFN_OFFSET (pfn_base) >> >> >> This patch still makes no sense. I'm not sure why we have an arch >> specific pfn_valid() because it's very similar to the generic one. But >> my guess is there's a reason for it and it's not doing what it is >> supposed when you remove it for the sparsemem case. > > It will use another pfn_valid() implementation in > include/linux/mmzone.h if CONFIG_SPARSEMEM and > !CONFIG_HAVE_ARCH_PFN_VALID > It will be this one. > > static inline int pfn_valid(unsigned long pfn) > { > if (pfn_to_section_nr(pfn) >= NR_MEM_SECTIONS) > return 0; > return valid_section(__nr_to_section(pfn_to_section_nr(pfn))); > } Ah, ok I see. "page.h" is only included in no-mmu arches. Which explains why riscv re-implements that macro. Couple follow up questions then: * Did you test the memory-with-hole scenario without the sparsemem patches? It seems pfn_valid() will be wrong regardless of sparse/flat mem. * Any chance we can just use the generic pfn_valid() function in all cases not just sparsemem? Can you test that? Thanks, Logan