Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Thu, 20 Jan 2022 13:37:13 -0600
From:   Bjorn Helgaas <helgaas@kernel.org>
To:     Pali =?iso-8859-1?Q?Roh=E1r?= <pali@kernel.org>
Cc:     Thomas Petazzoni <thomas.petazzoni@bootlin.com>,
        Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>,
        Rob Herring <robh@kernel.org>,
        Krzysztof =?utf-8?Q?Wilczy=C5=84ski?= <kw@linux.com>,
        Bjorn Helgaas <bhelgaas@google.com>,
        Marek =?iso-8859-1?Q?Beh=FAn?= <kabel@kernel.org>,
        linux-pci@vger.kernel.org, linux-arm-kernel@lists.infradead.org,
        linux-kernel@vger.kernel.org
Subject: Re: [PATCH 08/15] PCI: mvebu: Propagate errors when updating
 PCI_IO_BASE and PCI_MEM_BASE registers
Message-ID: <20220120193713.GA1060589@bhelgaas>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20220120190826.wkhkcx53lmafq2yp@pali>
Precedence: bulk

On Thu, Jan 20, 2022 at 08:08:26PM +0100, Pali Roh?r wrote:
> On Thursday 20 January 2022 11:50:47 Bjorn Helgaas wrote:
> > On Thu, Jan 13, 2022 at 11:35:23AM +0100, Pali Roh?r wrote:
> > > On Wednesday 12 January 2022 18:19:21 Bjorn Helgaas wrote:
> > > > On Sat, Jan 08, 2022 at 12:46:58AM +0100, Pali Roh?r wrote:
> > > > > On Friday 07 January 2022 17:16:17 Bjorn Helgaas wrote:
> > > > > > On Fri, Jan 07, 2022 at 11:28:26PM +0100, Pali Roh?r wrote:
> > > > > > > On Friday 07 January 2022 15:55:04 Bjorn Helgaas wrote:
> > > > > > > > On Thu, Nov 25, 2021 at 01:45:58PM +0100, Pali Roh?r wrote:
> > > > > > > > > Properly propagate failure from
> > > > > > > > > mvebu_pcie_add_windows() function back to the caller
> > > > > > > > > mvebu_pci_bridge_emul_base_conf_write() and
> > > > > > > > > correctly updates PCI_IO_BASE, PCI_MEM_BASE and
> > > > > > > > > PCI_IO_BASE_UPPER16 registers on error.  On error
> > > > > > > > > set base value higher than limit value which
> > > > > > > > > indicates that address range is disabled. 

> > Regardless, this means PCI thinks [mem 0xe0000000-0xe7ffffff] is
> > available on bus 00 and can be assigned to devices on bus 00
> > according to the normal PCI rules (BARs aligned on size, PCI
> > bridge windows aligned on 1MB and multiple of 1MB in size).  IIUC,
> > mvebu imposes additional alignment constraints on the bridge
> > windows.
> > 
> > These are the bridge window assignments from your dmesg:
> > 
> > > pci 0000:00:01.0: BAR 8: assigned [mem 0xe0000000-0xe00fffff]
> > > pci 0000:00:02.0: BAR 8: assigned [mem 0xe0200000-0xe04fffff]
> > > pci 0000:00:03.0: BAR 8: assigned [mem 0xe0100000-0xe01fffff]
> > 
> > > pci 0000:00:01.0: PCI bridge to [bus 01]
> > > pci 0000:00:01.0:   bridge window [mem 0xe0000000-0xe00fffff]
> > > pci 0000:00:02.0: PCI bridge to [bus 02]
> > > pci 0000:00:02.0:   bridge window [mem 0xe0200000-0xe04fffff]
> > > pci 0000:00:03.0: PCI bridge to [bus 03]
> > > pci 0000:00:03.0:   bridge window [mem 0xe0100000-0xe01fffff]
> > 
> > The PCI core knows nothing about the mvebu constraints.  Are we
> > just lucky here that when PCI assigned these bridge windows, they
> > happen to be supported on mvebu?  What happens if PCI decides it
> > needs 29MB on bus 01?
> 
> In this case pci-mvebu.c split 29MB window into continuous ranges of
> power of two (16MB + 8MB + 4MB + 1MB) and then register each range
> to mbus slot. Code is in function mvebu_pcie_add_windows():
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pci/controller/pci-mvebu.c?h=v5.15#n300
> 
> So at the end there is continuous space of 29MB PCIe window, just it
> "eats" 4 mbus slots.
> 
> This function may fail (if there is not enough free mbus slots) and
> this patch is propagating that failure back to the caller.

This failure cannot occur in conforming PCI hardware.  I guess if you
want to propagate the error from mvebu_pcie_add_windows() back to
mvebu_pci_bridge_emul_base_conf_write() and do something there, I'm OK
with that.

But change the commit log so it doesn't say "... and correctly update
PCI_IO_BASE, PCI_MEM_BASE and PCI_IO_BASE_UPPER16" because this is
completely device-specific behavior and is not "correct" per any PCI
spec.

Instead, say something about how mvebu doesn't support arbitrary
windows and we're disabling the window completely if we can't provide
what's requested.  

Maybe this error warrants a clue in dmesg?  How would a user figure
out what's going on in this situation?  From the patch, it looks like
we would assign resources to a device, but the device just would not
work because the root port window was silently disabled?

Bjorn