Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <20210412123936.25555-1-pali@kernel.org> <CAL_JsqLSse=W3TFu=Wc=eEAV4fKDGfsQ6JUvO3KyG_pnGTVg6A@mail.gmail.com>
 <20210415083640.ntg6kv6ayppxldgd@pali> <20210415104537.403de52e@thinkpad>
In-Reply-To: <20210415104537.403de52e@thinkpad>
From:   Rob Herring <robh+dt@kernel.org>
Date:   Thu, 15 Apr 2021 10:13:17 -0500
Message-ID: <CAL_JsqL2gjprb3MDv8KPSpe0CUBFjGajnMbF71DM+F9Yewp2uw@mail.gmail.com>
Subject: Re: [PATCH] arm64: dts: marvell: armada-37xx: Set linux,pci-domain to zero
To:     Marek Behun <marek.behun@nic.cz>,
        =?UTF-8?Q?Pali_Roh=C3=A1r?= <pali@kernel.org>
Cc:     Andrew Lunn <andrew@lunn.ch>,
        Gregory Clement <gregory.clement@bootlin.com>,
        Sebastian Hesselbarth <sebastian.hesselbarth@gmail.com>,
        linux-arm-kernel <linux-arm-kernel@lists.infradead.org>,
        devicetree@vger.kernel.org,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        PCI <linux-pci@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Thu, Apr 15, 2021 at 3:45 AM Marek Behun <marek.behun@nic.cz> wrote:
>
> On Thu, 15 Apr 2021 10:36:40 +0200
> Pali Roh=C3=A1r <pali@kernel.org> wrote:
>
> > On Tuesday 13 April 2021 13:17:29 Rob Herring wrote:
> > > On Mon, Apr 12, 2021 at 7:41 AM Pali Roh=C3=A1r <pali@kernel.org> wro=
te:
> > > >
> > > > Since commit 526a76991b7b ("PCI: aardvark: Implement driver 'remove=
'
> > > > function and allow to build it as module") PCIe controller driver f=
or
> > > > Armada 37xx can be dynamically loaded and unloaded at runtime. Also=
 driver
> > > > allows dynamic binding and unbinding of PCIe controller device.
> > > >
> > > > Kernel PCI subsystem assigns by default dynamically allocated PCI d=
omain
> > > > number (starting from zero) for this PCIe controller every time whe=
n device
> > > > is bound. So PCI domain changes after every unbind / bind operation=
.
> > >
> > > PCI host bridges as a module are relatively new, so seems likely a bu=
g to me.
> >
> > Why a bug? It is there since 5.10 and it is working.

I mean historically, the PCI subsystem didn't even support host
bridges as a module. They weren't even proper drivers and it was all
arch specific code. Most of the host bridge drivers are still built-in
only. This seems like a small detail that was easily overlooked.
unbind is not a well tested path.

> > > > Alternative way for assigning PCI domain number is to use static al=
located
> > > > numbers defined in Device Tree. This option has requirement that ev=
ery PCI
> > > > controller in system must have defined PCI bus number in Device Tre=
e.
> > >
> > > That seems entirely pointless from a DT point of view with a single P=
CI bridge.
> >
> > If domain id is not specified in DT then kernel uses counter and assign=
s
> > counter++. So it is not pointless if we want to have stable domain id.
>
> What Rob is trying to say is that
> - the bug is that kernel assigns counter++
> - device-tree should not be used to fix problems with how kernel does
>   things
> - if a device has only one PCIe controller, it is pointless to define
>   it's pci-domain. If there were multiple controllers, then it would
>   make sense, but there is only one

Yes. I think what we want here is a domain bitmap rather than a
counter and we assign the lowest free bit. That could also allow for
handling a mixture of fixed domain numbers and dynamically assigned
ones.

You could create scenarios where the numbers change on you, but it
wouldn't be any different than say plugging in USB serial adapters.
You get the same ttyUSBx device when you re-attach unless there's been
other ttyUSBx devices attached/detached.

Rob