Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) client-ip=23.128.96.38;
Date:   Fri, 3 Nov 2023 13:43:48 -0500
From:   Bjorn Helgaas <helgaas@kernel.org>
To:     David Epping <david.epping@missinglinkelectronics.com>
Cc:     linux-arm-kernel@lists.infradead.org,
        Dinh Nguyen <dinguyen@kernel.org>,
        Ley Foon Tan <ley.foon.tan@intel.com>,
        Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>,
        linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org,
        Bjorn Helgaas <bhelgaas@google.com>,
        Krzysztof =?utf-8?Q?Wilczy=C5=84ski?= <kw@linux.com>,
        Zhichang Yuan <yuanzhichang@hisilicon.com>,
        Gabriele Paoloni <gabriele.paoloni@huawei.com>,
        John Garry <john.garry@huawei.com>,
        Joyce Ooi <joyce.ooi@intel.com>
Subject: Re: arm: mach-socfpga: PCIe Root IO TLP support for Cyclone V
Message-ID: <20231103184348.GA162150@bhelgaas>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <ZUDd04c7FXUeusxK@nucnuc.mle>
Precedence: bulk

[+cc Zhichang, Gabriele, John (INDIRECT_PIO); Joyce (altera maintainer)]

On Tue, Oct 31, 2023 at 11:58:27AM +0100, David Epping wrote:
> Hello ARM PCIe and especially Intel Altera SOCFPGA maintainers,
> 
> the Intel Altera Cyclone V PCIe Root Complex drivers afaik currently
> don’t support sending IO TLPs.  The Root Complex IP Core, seemingly
> unlike many other ARM Root Complexes, does not offer a memory
> mapping for the IO address space, but instead relies on indirect
> addressing via address and data registers. To my knowledge this IO
> space access has not been implemented in Linux, yet, and is
> currently only used for PCIe Configuration TLPs on this hardware,
> which use the same mechanism.
> 
> To support an AX99100 endpoint (which requires IO BARs for some of
> its features) connected to a Cyclone V PCIe root, we added support
> for IO TLPs to Kernels 4.14, 5.4, and 5.15.  For your reference the
> 4.14 patches are attached. They do _not_ apply to current mainline
> Linux and are meant to give you an idea of our current approach.  As
> you can see, in order to implement our own inb() and outb() family
> of functions, we had to remove multi-platform support for
> ARCH_SOCFPGA.
> 
> I would like to get some feedback on:
>
> a) Are you interested in adding IO TLP support for this FPGA/CPU to
> mainline? We would implement patches for the current mainline in
> that case and post them for discussion.
>
> b) Do you see an option to implement the IO space access functions
> without dropping multi-platform support? Is that a prerequisite for
> going mainline?

I don't understand INDIRECT_PIO very well, but does it help with this
at all?

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/lib/Kconfig?id=v6.6#n92
https://git.kernel.org/linus/031e3601869c

Beginning of thread, including full patch (for v4.14, not current
kernel) at: https://lore.kernel.org/all/ZUDd04c7FXUeusxK@nucnuc.mle/

Bjorn