Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Thu, 24 Oct 2019 09:51:25 -0700 (PDT)
From:   Paul Walmsley <paul.walmsley@sifive.com>
To:     Logan Gunthorpe <logang@deltatee.com>
cc:     Yash Shah <yash.shah@sifive.com>,
        "Paul Walmsley ( Sifive)" <paul.walmsley@g.sifive.com>,
        "Palmer Dabbelt ( Sifive)" <palmer@g.sifive.com>,
        "linux-riscv@lists.infradead.org" <linux-riscv@lists.infradead.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "sorear2@gmail.com" <sorear2@gmail.com>,
        "aou@eecs.berkeley.edu" <aou@eecs.berkeley.edu>,
        "alex@ghiti.fr" <alex@ghiti.fr>,
        "catalin.marinas@arm.com" <catalin.marinas@arm.com>,
        "Anup.Patel@wdc.com" <Anup.Patel@wdc.com>,
        "rppt@linux.ibm.com" <rppt@linux.ibm.com>,
        Sachin Ghadi <sachin.ghadi@sifive.com>,
        Greentime Hu <greentime.hu@g.sifive.com>,
        "gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
        "tglx@linutronix.de" <tglx@linutronix.de>,
        "will@kernel.org" <will@kernel.org>,
        "allison@lohutok.net" <allison@lohutok.net>
Subject: Re: [PATCH] RISC-V: Add PCIe I/O BAR memory mapping
In-Reply-To: <c4817ec1-4e50-5646-68f0-caeb0ab6f0bf@deltatee.com>
Message-ID: <alpine.DEB.2.21.9999.1910240937350.20010@viisi.sifive.com>
References: <1571908438-4802-1-git-send-email-yash.shah@sifive.com> <c4817ec1-4e50-5646-68f0-caeb0ab6f0bf@deltatee.com>
User-Agent: Alpine 2.21.9999 (DEB 301 2018-08-15)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Thu, 24 Oct 2019, Logan Gunthorpe wrote:

> On 2019-10-24 3:14 a.m., Yash Shah wrote:
> > For I/O BARs to work correctly on RISC-V Linux, we need to establish a
> > reserved memory region for them, so that drivers that wish to use I/O BARs
> > can issue reads and writes against a memory region that is mapped to the
> > host PCIe controller's I/O BAR MMIO mapping.
> 
> I don't think other arches do this. 

$ git grep 'define PCI_IOBASE' arch/ 
arch/arm/include/asm/io.h:#define PCI_IOBASE            ((void __iomem *)PCI_IO_VIRT_BASE)
arch/arm64/include/asm/io.h:#define PCI_IOBASE          ((void __iomem *)PCI_IO_START)
arch/m68k/include/asm/io_no.h:#define PCI_IOBASE        ((void __iomem *) PCI_IO_PA)
arch/microblaze/include/asm/io.h:#define PCI_IOBASE     ((void __iomem *)_IO_BASE)
arch/unicore32/include/asm/io.h:#define PCI_IOBASE      PKUNITY_PCILIO_BASE
arch/xtensa/include/asm/io.h:#define PCI_IOBASE         ((void __iomem *)XCHAL_KIO_BYPASS_VADDR)
$

This is for the old x86-style, non-memory mapped I/O address space the 
legacy PCI stuff that one would use in{b,w,l}()/out{b,w,l}() for.

Yash, you might consider updating your patch description to note that this 
is for "legacy I/O BARs (i.e., non-MMIO BARs)" or something similar.  That 
might make things clearer.

> ioremap() typically just uses virtual address space in the VMALLOC 
> region, PCI doesn't need it's own range. As far as I know the ioremap() 
> implementation in riscv already does this.
> 
> In any case, 16MB for PCI bar space seems woefully inadequate.

The modern MMIO PCI resources wind up in jost controller apertures, which 
as you note, are usually much larger.  They don't go in this legacy space.

Regarding sizing - I haven't seen any PCIe cards with more than 64KiB of 
legacy I/O resources.  (16MiB / 64KiB) = 256, so 16MiB sounds reasonable 
from that point of view?  ARM64 is using that.


- Paul