Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20211129173637.303201-1-robh@kernel.org> <Yf2wTLjmcRj+AbDv@xps13.dannf>
 <CAL_Jsq+4b4Yy8rJGJv9j9j_TCm6mTAkW5fzcDuuW-jOoiZ2GLg@mail.gmail.com>
In-Reply-To: <CAL_Jsq+4b4Yy8rJGJv9j9j_TCm6mTAkW5fzcDuuW-jOoiZ2GLg@mail.gmail.com>
From:   dann frazier <dann.frazier@canonical.com>
Date:   Sat, 5 Feb 2022 14:12:57 -0700
Message-ID: <CALdTtnuK+D7gNbEDgHbrc29pFFCR3XYAHqrK3=X_hQxUx-Seow@mail.gmail.com>
Subject: Re: [PATCH] PCI: xgene: Fix IB window setup
To:     Rob Herring <robh@kernel.org>
Cc:     Toan Le <toan@os.amperecomputing.com>,
        Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>,
        =?UTF-8?Q?Krzysztof_Wilczy=C5=84ski?= <kw@linux.com>,
        Bjorn Helgaas <bhelgaas@google.com>,
        Andrew Murray <amurray@thegoodpenguin.co.uk>,
        =?UTF-8?Q?St=C3=A9phane_Graber?= <stgraber@ubuntu.com>,
        stable <stable@vger.kernel.org>, PCI <linux-pci@vger.kernel.org>,
        linux-arm-kernel <linux-arm-kernel@lists.infradead.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Sat, Feb 5, 2022 at 9:05 AM Rob Herring <robh@kernel.org> wrote:
>
> On Fri, Feb 4, 2022 at 5:01 PM dann frazier <dann.frazier@canonical.com> =
wrote:
> >
> > On Mon, Nov 29, 2021 at 11:36:37AM -0600, Rob Herring wrote:
> > > Commit 6dce5aa59e0b ("PCI: xgene: Use inbound resources for setup")
> > > broke PCI support on XGene. The cause is the IB resources are now sor=
ted
> > > in address order instead of being in DT dma-ranges order. The result =
is
> > > which inbound registers are used for each region are swapped. I don't
> > > know the details about this h/w, but it appears that IB region 0
> > > registers can't handle a size greater than 4GB. In any case, limiting
> > > the size for region 0 is enough to get back to the original assignmen=
t
> > > of dma-ranges to regions.
> >
> > hey Rob!
> >
> > I've been seeing a panic on HP Moonshoot m400 cartridges (X-Gene1) -
> > only during network installs - that I also bisected down to commit
> > 6dce5aa59e0b ("PCI: xgene: Use inbound resources for setup"). I was
> > hoping that this patch that fixed the issue on St=C3=A9phane's X-Gene2
> > system would also fix my issue, but no luck. In fact, it seems to just
> > makes it fail differently. Reverting both patches is required to get a
> > v5.17-rc kernel to boot.
> >
> > I've collected the following logs - let me know if anything else would
> > be useful.
> >
> > 1) v5.17-rc2+ (unmodified):
> >    http://dannf.org/bugs/m400-no-reverts.log
> >    Note that the mlx4 driver fails initialization.
> >
> > 2) v5.17-rc2+, w/o the commit that fixed St=C3=A9phane's system:
> >    http://dannf.org/bugs/m400-xgene2-fix-reverted.log
> >    Note the mlx4 MSI-X timeout, and later panic.
> >
> > 3) v5.17-rc2+, w/ both commits reverted (works)
> >    http://dannf.org/bugs/m400-both-reverted.log
>
> The ranges and dma-ranges addresses don't appear to match up with any
> upstream dts files. Can you send me the DT?

Sure: http://dannf.org/bugs/fdt

 -dann

> Otherwise, we're going to need some debugging added to
> xgene_pcie_setup_ib_reg() to see if the register setup changed. I can
> come up with something next week.
>
> Rob