Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
To:     Palmer Dabbelt <palmer@sifive.com>
Cc:     linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org,
        sbates@raithlin.com, Christoph Hellwig <hch@lst.de>,
        aou@eecs.berkeley.edu
References: <mhng-8e27a4f1-db30-46cf-b987-51bba8cd96c7@palmer-si-x1e>
From:   Logan Gunthorpe <logang@deltatee.com>
Message-ID: <479792a3-d049-9d93-8bd7-465156ce813d@deltatee.com>
Date:   Fri, 26 Apr 2019 10:37:15 -0600
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.6.1
MIME-Version: 1.0
In-Reply-To: <mhng-8e27a4f1-db30-46cf-b987-51bba8cd96c7@palmer-si-x1e>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Subject: Re: [PATCH 0/7] RISC-V: Sparsmem, Memory Hotplug and pte_devmap for
 P2P
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk


On 2019-04-24 5:23 p.m., Palmer Dabbelt wrote:
> On Wed, 27 Mar 2019 14:36:36 PDT (-0700), logang@deltatee.com wrote:
>> Hi,
>>
>> This patchset enables P2P on the RISC-V architecture. To do this on the
>> current kernel, we only need to be able to back IO memory with struct
>> pages using devm_memremap_pages(). This requires ARCH_HAS_ZONE_DEVICE,
>> ARCH_ENABLE_MEMORY_HOTPLUG, and ARCH_ENABLE_MEMORY_HOTREMOVE; which in
>> turn requires ARCH_SPARSEMEM_ENABLE. We also need to ensure that the
>> IO memory regions in hardware can be covered by the linear region
>> so that there is a linear relation ship between the virtual address and
>> the struct page address in the vmemmap region.
>>
>> While our reason to do this work is for P2P, these features are all
>> useful, more generally, and also enable other kernel features.
>>
>> The first patch in the series implements sparse mem. It was already
>> submitted and reviewed last cycle, only forgotten. It has been rebased
>> onto v5.1-rc2.
>>
>> Patches 2 through 4 rework the architecture's virtual address space
>> mapping trying to get as much of the IO regions covered by the linear
>> mapping. With Sv39, we do not have enough address space to cover all the
>> typical hardware regions but we can get the majority of it.
>>
>> Patch 5 and 6 implement memory hotplug and remove. These are relatively
>> straight forward additions similar to other arches.
>>
>> Patch 7 implements pte_devmap which allows us to set
>> ARCH_HAS_ZONE_DEVICE.
>>
>> The patchset was tested in QEMU and on a HiFive Unleashed board.
>> However, we were unable to actually test P2P transactions with this
>> exact set because we have been unable to get PCI working with v5.1-rc2.
>> We were able to get it running on a 4.19 era kernel (with a bunch of
>> out-of-tree patches for PCI on a Microsemi PolarFire board).
>>
>> This series is based on v5.1-rc2 and a git tree is available here:
>>
>> https://github.com/sbates130272/linux-p2pmem riscv-p2p-v1
> 
> Looks like these don't build on rv32 when applied on top of 5.1-rc6.  We 
> now
> have rv32_defconfig, which should make it easier to tests these sorts of
> things.

Thanks for the note. I've queued up the fixes for this. However, I'm 
still a bit stuck on the memory hot remove stuff. I'm waiting for the 
similar work in arm64 to be done so I can reuse some of it[1].

The first patch of this series that implements sparsemem builds on rv32 
and was generally accepted in previous cycles; so I'd appreciate if you 
can pick that one up for v5.2.

Thanks.

Logan


[1] 
https://lore.kernel.org/lkml/1555221553-18845-1-git-send-email-anshuman.khandual@arm.com/T/#u