Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 18 Jul 2022 13:14:25 -0500
From:   Bjorn Helgaas <helgaas@kernel.org>
To:     Jim Quinlan <jim2101024@gmail.com>
Cc:     linux-pci@vger.kernel.org,
        Nicolas Saenz Julienne <nsaenz@kernel.org>,
        Bjorn Helgaas <bhelgaas@google.com>,
        Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>,
        Cyril Brulebois <kibi@debian.org>,
        bcm-kernel-feedback-list@broadcom.com, james.quinlan@broadcom.com,
        Florian Fainelli <f.fainelli@gmail.com>,
        Lorenzo Pieralisi <lpieralisi@kernel.org>,
        Rob Herring <robh@kernel.org>,
        Krzysztof =?utf-8?Q?Wilczy=C5=84ski?= <kw@linux.com>,
        "moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE" 
        <linux-rpi-kernel@lists.infradead.org>,
        "moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE" 
        <linux-arm-kernel@lists.infradead.org>,
        open list <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v2 2/6] PCI: brcmstb: Split brcm_pcie_setup() into two
 funcs
Message-ID: <20220718181425.GA1431580@bhelgaas>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20220716222454.29914-3-jim2101024@gmail.com>
Precedence: bulk

On Sat, Jul 16, 2022 at 06:24:49PM -0400, Jim Quinlan wrote:
> Currently, the function does the setup for establishing PCIe link-up
> with the downstream device, and it does the actual link-up as well.
> The calling sequence is (roughly) the following in the probe:
> 
> -> brcm_pcie_probe()
>     -> brcm_pcie_setup();                       /* Set-up and link-up */
>     -> pci_host_probe(bridge);
> 
> This commit splits the setup function in two: brcm_pcie_setup(), which only
> does the set-up, and brcm_pcie_start_link(), which only does the link-up.
> The reason why we are doing this is to lay a foundation for subsequent
> commits so that we can turn on any power regulators, as described in the
> root port's DT node, prior to doing link-up.

All drivers that care about power regulators turn them on before
link-up, but typically those regulators are described directly under
the host bridge itself.

IIUC the difference here is that you have regulators described under
Root Ports (not the host bridge/Root Complex itself), so you don't
know about them until you've enumerated the Root Ports.
brcm_pcie_probe() can't turn them on directly because it doesn't know
what Root Ports are present and doesn't know about regulators below
them.

So I think brcm_pcie_setup() does initialization that doesn't depend
on the link or any downstream devices, and brcm_pcie_start_link() does
things that depend on the link being up.  Right?

If so, "start_link" might be a slight misnomer since AFAICT
brcm_pcie_start_link() doesn't do anything to initiate link-up except
maybe deasserting fundamental reset.  Some drivers start the LTSSM or
explicitly enable link training, but brcm_pcie_start_link() doesn't
seem to do anything like that.

brcm_pcie_start_link() still does brcm_pcie_set_outbound_win().  Does
that really depend on the link being up?  If that only affects the
Root Port, maybe it could be done before link-up?

> We do this by defining an
> add_bus() callback which is invoked during enumeraion.  At the end of this
> patchset the probe function trace will look something like this:
> 
> -> brcm_pcie_probe()
>     -> brcm_pcie_setup();                       /* Set-up only */
>     -> pci_host_probe(bridge);
>         -> [enumeration]
>             -> pci_alloc_child_bus()
>                 -> bus->ops->add_bus(bus);      /* We've set this op */
>                 -> brcm_pcie_add_bus()          /* Our callback      */
>                      -> [turn on regulators]    /* Main objective!   */
>                      -> brcm_pcie_start_link()  /* Link-up           */