Received-SPF: pass (google.com: domain of linux-kernel+bounces-154997-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Date: Tue, 23 Apr 2024 13:33:01 +0200
From: Herve Codina <herve.codina@bootlin.com>
To: Chris Packham <Chris.Packham@alliedtelesis.co.nz>
Cc: Bjorn Helgaas <helgaas@kernel.org>, "linux-kernel@vger.kernel.org"
 <linux-kernel@vger.kernel.org>, "linux-pci@vger.kernel.org"
 <linux-pci@vger.kernel.org>, "linux-acpi@vger.kernel.org"
 <linux-acpi@vger.kernel.org>
Subject: Re: local bus enumeration beyond a PCI device
Message-ID: <20240423133301.217f7ab2@bootlin.com>
In-Reply-To: <b7589f13-e851-4ec3-ade7-43981b345cb1@alliedtelesis.co.nz>
References: <20240418184531.GA245970@bhelgaas>
	<b7589f13-e851-4ec3-ade7-43981b345cb1@alliedtelesis.co.nz>
Organization: Bootlin
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Hi Chris,

On Tue, 23 Apr 2024 01:05:21 +0000
Chris Packham <Chris.Packham@alliedtelesis.co.nz> wrote:

> Hi,
> 
> On 19/04/24 06:45, Bjorn Helgaas wrote:
> > [+cc Herve]
> >
> > On Thu, Apr 18, 2024 at 12:24:06AM +0000, Chris Packham wrote:  
> >> Hi,
> >>
> >> We've got a custom x86_64 based design that is using an ASIX9100 to
> >> provide a PCI to local bus bridge. Attached to that local bus is an FPGA
> >> which mostly provides some GPIOs accessed via registers on the local
> >> bus. Right now we've got a custom driver that bundles everything
> >> together so effectively we've got a PCI device that provides GPIOs.  
> > What's the local bus?  The ASIX9100 (for which Google doesn't find any
> > details) would have a PCI interface on the primary (upstream) side.
> > What's the local bus on the secondary (downstream) side?  Below you
> > mention "PCI bridge", which normally means both the primary and
> > secondary sides are PCI buses.  
> 
> The ASIX9100 is a multi purpose chip with a few options for providing 
> GPIO/I2C/SPI/etc. In our hardware configuration we're using it to 
> provide a 16-bit parallel bus which we're using for register access to 
> the FPGA. There's an outside chance someone might ask us to hang other 
> parallel devices like an SRAM or some NOR Flash off it as well. These 
> are all things we've done on other products with more traditional 
> embedded SoCs that have a parallel bus interface.
> 
> > If the local bus is not PCI, I guess the ASIX9100 would look to the OS
> > like an endpoint, i.e., PCI_HEADER_TYPE_NORMAL, and the ASIX9100
> > driver would handle any "bridge" functionality completely internally?  
> 
> I might be using the terminology wrong. Essentially the ASIX9100 is a 
> PCI-e endpoint (so a device from the PCI subsystems point of view) but 
> that device can be a I2C, SPI or parallel bus controller onto which 
> other devices are connected.Hence it's providing a bridge between PCI 
> and 16-bit parallel busses.
> 
> > Maybe Herve's work at
> > https://lore.kernel.org/r/20240325153919.199337-1-herve.codina@bootlin.com
> > would be relevant?  
> 
> I've had a quick look at that series. I'm not 100% sure it'll help. The 
> platform I'm dealing with is x86_64 and doesn't have device tree support 
> (if someone can say "hey do xyz and you'll get DT on x86" that'd be 
> great).But I'll take a closer look at it and at the original "Generate 
> device tree node for pci devices" series.

You can have DT support on x86.
CONFIG_OF=y
CONFIG_PCI_DYNAMIC_OF_NODES = y

I use a PCIe board from Microchip (Lan966x) and I describe the internal board
IPs using a DT overlay.
My PCI driver loads the overlay at probe() and call of_platform_default_populate()
to instantiate the devices.

My DT overlay contains nodes for:
- clock controller
- reset
- i2c bus
- pinctrl and gpio
- Network switch
- MDIO controller

I have it working on x86 system.
What is mainly missing upstream for x86 system is the DT node creation for the
host PCI bridge (root complex). Indeed, on x86 the description comes from ACPI
and a DT description of this root bridge is needed.
Got a draft code locally (not sent upstream yet) that creates this first bridge
on ACPI based systems.

PCI-to-PCI bridges and PCI devices are already handled:
  https://elixir.bootlin.com/linux/v6.9-rc5/source/drivers/pci/of.c#L625
At runtime, based on the PCI enumeration, corresponding DT nodes are created.

Best regards,
Hervé

> 
> >> But as things can change based on the FPGA program I'd like some
> >> flexibility to treat it separately from the PCI bridge. So really I'd
> >> like to have a PCI device driver for the ASIX9100 that provides a local
> >> bus controller and a (platform?) driver for the FPGA that provides the
> >> GPIOs where I can have different compatibles for the different
> >> implementations.
> >>
> >> Then in the ACPI overlay I'd have something like
> >>
> >>       Scope (\_SB.PCI0.D0B0)
> >>       {
> >>           Device (ASIX)
> >>           {
> >>               Name (_ADR, 0x0000)
> >>
> >>               Device (FPGA)
> >>               {
> >>                           Name (_HID, "PRP0001")
> >>                           Name (_DSD, Package ()
> >>                           {
> >> ToUUID("daffd814-6eba-4d8c-8a91-bc9bbf4aa301"),
> >>                                       Package ()
> >>                                       {
> >>                                                   Package () {
> >> "compatible", "my-platform-driver-for-fpga" },
> >>                                       }
> >>                           })
> >>               }
> >>           }
> >>       }
> >>
> >>      Scope(\_SB)
> >>      {
> >>           Device(OTHR)
> >>           {
> >>               GpioIo (Exclusive, PullUp, 0, 0, IoRestrictionInputOnly,
> >> "\\_SB.PCI0.D0B0.ASIX.FPGA",) { 0 }
> >>           }
> >>      }
> >>
> >> Is it even possible to register a host controller for another platform bus?

With DT overlay, you can register any devices available on your PCI board.
You just need to describe them in the overlay and be sure that a driver exist to
handle them based on their compatible property.

Also, this solution was designed to work on both DT and ACPI based systems.
My Microchip board works on x86 and ARM.

Best regards,
Hervé