Received-SPF: softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
Date:   Mon, 21 Mar 2022 15:06:29 -0600
From:   dann frazier <dann.frazier@canonical.com>
To:     Rob Herring <robh@kernel.org>
Cc:     Marc Zyngier <maz@kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        linux-arm-kernel <linux-arm-kernel@lists.infradead.org>,
        PCI <linux-pci@vger.kernel.org>,
        Toan Le <toan@os.amperecomputing.com>,
        Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>,
        Krzysztof =?utf-8?Q?Wilczy=C5=84ski?= <kw@linux.com>,
        Bjorn Helgaas <bhelgaas@google.com>,
        =?iso-8859-1?Q?St=E9phane?= Graber <stgraber@ubuntu.com>,
        Android Kernel Team <kernel-team@android.com>
Subject: Re: [PATCH v2 0/2] PCI: xgene: Restore working PCIe functionnality
Message-ID: <Yjjo1emkGKPpaYY2@xps13.dannf>
References: <20220321104843.949645-1-maz@kernel.org>
 <CAL_JsqJacC6GbNebTfYyUEScROCFN4+Fg2v1_iYFfqAvW4E9Vw@mail.gmail.com>
 <87h77rxnyl.wl-maz@kernel.org>
 <CAL_JsqK57KpZmzCE=86dLcHK4Ws_0w0ga4_qoYUe2GwFNpDzRw@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAL_JsqK57KpZmzCE=86dLcHK4Ws_0w0ga4_qoYUe2GwFNpDzRw@mail.gmail.com>
Precedence: bulk

On Mon, Mar 21, 2022 at 01:03:27PM -0500, Rob Herring wrote:
> On Mon, Mar 21, 2022 at 11:36 AM Marc Zyngier <maz@kernel.org> wrote:
> >
> > On Mon, 21 Mar 2022 15:17:34 +0000,
> > Rob Herring <robh@kernel.org> wrote:
> > >
> > > On Mon, Mar 21, 2022 at 5:49 AM Marc Zyngier <maz@kernel.org> wrote:
> > > >
> > > > Since 6dce5aa59e0b ("PCI: xgene: Use inbound resources for setup") was
> > > > merged in the 5.5 time frame, PCIe on the venerable XGene platform has
> > > > been unusable: 6dce5aa59e0b broke both XGene-1 (Mustang and m400) and
> > > > XGene-2 (Merlin), while the addition of c7a75d07827a ("PCI: xgene: Fix
> > > > IB window setup") fixed XGene-2, but left the rest of the zoo
> > > > unusable.
> > > >
> > > > It is understood that this systems come with "creative" DTs that don't
> > > > match the expectations of modern kernels. However, there is little to
> > > > be gained by forcing these changes on users -- the firmware is not
> > > > upgradable, and the current owner of the IP will deny that these
> > > > machines have ever existed.
> > >
> > > The gain for fixing this properly is not having drivers do their own
> > > dma-ranges parsing. We've seen what happens when drivers do their own
> > > parsing of standard properties (e.g. interrupt-map).
> >
> > We have, and we added the required exceptions for the legacy platforms
> > that the code base supported until then. We didn't leave things broken
> > just because we didn't like the way things were done a long time ago.
> >
> > > Currently, we don't have any drivers doing their own parsing:
> > >
> > > $ git grep of_pci_dma_range_parser_init
> > > drivers/of/address.c:int of_pci_dma_range_parser_init(struct
> > > of_pci_range_parser *parser,
> > > drivers/of/address.c:EXPORT_SYMBOL_GPL(of_pci_dma_range_parser_init);
> > > drivers/of/address.c:#define of_dma_range_parser_init
> > > of_pci_dma_range_parser_init
> > > drivers/of/unittest.c:  if (of_pci_dma_range_parser_init(&parser, np)) {
> > > drivers/pci/of.c:       err = of_pci_dma_range_parser_init(&parser, dev_node);
> > > include/linux/of_address.h:extern int
> > > of_pci_dma_range_parser_init(struct of_pci_range_parser *parser,
> > > include/linux/of_address.h:static inline int
> > > of_pci_dma_range_parser_init(struct of_pci_range_parser *parser,
> > >
> > > And we can probably further refactor this to be private to drivers/pci/of.c.
> > >
> > > For XGene-2 the issue is simply that the driver depends on the order
> > > of dma-ranges entries.
> > >
> > > For XGene-1, I'd still like to understand what the issue is. Reverting
> > > the first fix and fixing 'dma-ranges' should have fixed it. I need a
> > > dump of how the IB registers are initialized in both cases. I'm not
> > > saying changing 'dma-ranges' in the firmware is going to be required
> > > here. There's a couple of other ways we could fix that without a
> > > firmware change, but first I need to understand why it broke.
> >
> > Reverting 6dce5aa59e0b was enough for me, without changing anything
> > else.
> 
> Meaning c7a75d07827a didn't matter for you. I'm not sure that it would.
> 
> Can you tell me what 'dma-ranges' contains on your system?
> 
> > m400 probably uses an even older firmware (AFAIR, it was stuck
> > with an ancient version of u-boot that HP never updated, while Mustang
> > had a few updates). In any case, that DT cannot be changed.
> 
> How is Dann changing it then? I assume he's not changing the firmware,
> but overriding it. That could be a possible solution.

Correct, I'm just overriding it for testing. I'm using the pxelinux
emulation provided by the m400's u-boot, which supports an FDT field:

---------
$ cat /srv/tftp/pxelinux.cfg/default
DEFAULT default

LABEL default
  KERNEL uImage
  APPEND initrd=uInitrd console=ttyS0,9600n8r ro root=LABEL=cloudimg-rootfs
  FDT m400.dtb
---------

This loads the specified file into ${fdt_addr_r}, overriding the blob
that the firmware had already loaded there.

> Do the DT's in the kernel tree correspond to anything anyone is
> using?

Upstream apm-mustang.dtb is what Ubuntu uses for Mustang boards w/
u-boot firmware. That used to work fine, but I haven't tried lately.

  -dann