Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) client-ip=2620:137:e000::1:18;
Date:   Mon, 21 Mar 2022 11:08:20 -0500
From:   Rob Herring <robh@kernel.org>
To:     dann frazier <dann.frazier@canonical.com>
Cc:     Marc Zyngier <maz@kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        linux-arm-kernel <linux-arm-kernel@lists.infradead.org>,
        PCI <linux-pci@vger.kernel.org>,
        Toan Le <toan@os.amperecomputing.com>,
        Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>,
        Krzysztof =?utf-8?Q?Wilczy=C5=84ski?= <kw@linux.com>,
        Bjorn Helgaas <bhelgaas@google.com>,
        =?iso-8859-1?Q?St=E9phane?= Graber <stgraber@ubuntu.com>,
        Android Kernel Team <kernel-team@android.com>
Subject: Re: [PATCH v2 0/2] PCI: xgene: Restore working PCIe functionnality
Message-ID: <Yjii9LUC+u/gmijj@robh.at.kernel.org>
References: <20220321104843.949645-1-maz@kernel.org>
 <CAL_JsqJacC6GbNebTfYyUEScROCFN4+Fg2v1_iYFfqAvW4E9Vw@mail.gmail.com>
 <YjiewB5Nz5CyFuI0@xps13.dannf>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <YjiewB5Nz5CyFuI0@xps13.dannf>
Precedence: bulk

On Mon, Mar 21, 2022 at 09:50:24AM -0600, dann frazier wrote:
> On Mon, Mar 21, 2022 at 10:17:34AM -0500, Rob Herring wrote:
> > On Mon, Mar 21, 2022 at 5:49 AM Marc Zyngier <maz@kernel.org> wrote:
> > >
> > > Since 6dce5aa59e0b ("PCI: xgene: Use inbound resources for setup") was
> > > merged in the 5.5 time frame, PCIe on the venerable XGene platform has
> > > been unusable: 6dce5aa59e0b broke both XGene-1 (Mustang and m400) and
> > > XGene-2 (Merlin), while the addition of c7a75d07827a ("PCI: xgene: Fix
> > > IB window setup") fixed XGene-2, but left the rest of the zoo
> > > unusable.
> > >
> > > It is understood that this systems come with "creative" DTs that don't
> > > match the expectations of modern kernels. However, there is little to
> > > be gained by forcing these changes on users -- the firmware is not
> > > upgradable, and the current owner of the IP will deny that these
> > > machines have ever existed.
> > 
> > The gain for fixing this properly is not having drivers do their own
> > dma-ranges parsing. We've seen what happens when drivers do their own
> > parsing of standard properties (e.g. interrupt-map). Currently, we
> > don't have any drivers doing their own parsing:
> > 
> > $ git grep of_pci_dma_range_parser_init
> > drivers/of/address.c:int of_pci_dma_range_parser_init(struct
> > of_pci_range_parser *parser,
> > drivers/of/address.c:EXPORT_SYMBOL_GPL(of_pci_dma_range_parser_init);
> > drivers/of/address.c:#define of_dma_range_parser_init
> > of_pci_dma_range_parser_init
> > drivers/of/unittest.c:  if (of_pci_dma_range_parser_init(&parser, np)) {
> > drivers/pci/of.c:       err = of_pci_dma_range_parser_init(&parser, dev_node);
> > include/linux/of_address.h:extern int
> > of_pci_dma_range_parser_init(struct of_pci_range_parser *parser,
> > include/linux/of_address.h:static inline int
> > of_pci_dma_range_parser_init(struct of_pci_range_parser *parser,
> > 
> > And we can probably further refactor this to be private to drivers/pci/of.c.
> > 
> > For XGene-2 the issue is simply that the driver depends on the order
> > of dma-ranges entries.
> > 
> > For XGene-1, I'd still like to understand what the issue is. Reverting
> > the first fix and fixing 'dma-ranges' should have fixed it. I need a
> > dump of how the IB registers are initialized in both cases.
> 
> Happy to provide that for the m400 if told how :)

Something like the below patch. This should be with the 'dma-ranges' 
DT change and only c7a75d07827a reverted.

8<-------------------------------------------------------------------
diff --git a/drivers/pci/controller/pci-xgene.c b/drivers/pci/controller/pci-xgene.c
index 0d5acbfc7143..6a435c31f45e 100644
--- a/drivers/pci/controller/pci-xgene.c
+++ b/drivers/pci/controller/pci-xgene.c
@@ -78,6 +78,7 @@ static u32 xgene_pcie_readl(struct xgene_pcie *port, u32 reg)
 
 static void xgene_pcie_writel(struct xgene_pcie *port, u32 reg, u32 val)
 {
+	dev_info(port->dev, "0x%04x <- 0x%08x\n", reg, val);
 	writel(val, port->csr_base + reg);
 }
 
@@ -508,7 +509,9 @@ static void xgene_pcie_setup_ib_reg(struct xgene_pcie *port,
 	case 0:
 		xgene_pcie_set_ib_mask(port, BRIDGE_CFG_4, flags, size);
 		bar_addr = cfg_base + PCI_BASE_ADDRESS_0;
+		dev_info(port->dev, "BAR0L <- 0x%08x\n", bar_low);
 		writel(bar_low, bar_addr);
+		dev_info(port->dev, "BAR0H <- 0x%08x\n", upper_32_bits(cpu_addr));
 		writel(upper_32_bits(cpu_addr), bar_addr + 0x4);
 		pim_reg = PIM1_1L;
 		break;