Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <1656618906-29881-1-git-send-email-radhey.shyam.pandey@amd.com>
 <Yr66xEMB/ORr0Xcp@lunn.ch> <MN0PR12MB59531DFD084FA947084D91B6B7819@MN0PR12MB5953.namprd12.prod.outlook.com>
 <CAGETcx_BUR3EPDLgp9v0Uk9N=8BtYRjFyhpJTQa9kEMHtkgdwQ@mail.gmail.com> <MN0PR12MB59539E587A8B46FDC190FB7AB78B9@MN0PR12MB5953.namprd12.prod.outlook.com>
In-Reply-To: <MN0PR12MB59539E587A8B46FDC190FB7AB78B9@MN0PR12MB5953.namprd12.prod.outlook.com>
From:   Saravana Kannan <saravanak@google.com>
Date:   Fri, 15 Jul 2022 12:08:42 -0700
Message-ID: <CAGETcx9dh1hfqoFSRwNf3fGbH5Wsdhah8RT5R-JrGOS-rDFX3g@mail.gmail.com>
Subject: Re: [PATCH net-next v2] net: macb: In shared MDIO usecase make MDIO
 producer ethernet node to probe first
To:     "Pandey, Radhey Shyam" <radhey.shyam.pandey@amd.com>
Cc:     Andrew Lunn <andrew@lunn.ch>,
        "nicolas.ferre@microchip.com" <nicolas.ferre@microchip.com>,
        "claudiu.beznea@microchip.com" <claudiu.beznea@microchip.com>,
        "davem@davemloft.net" <davem@davemloft.net>,
        "edumazet@google.com" <edumazet@google.com>,
        "kuba@kernel.org" <kuba@kernel.org>,
        "pabeni@redhat.com" <pabeni@redhat.com>,
        "hkallweit1@gmail.com" <hkallweit1@gmail.com>,
        "linux@armlinux.org.uk" <linux@armlinux.org.uk>,
        "gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
        "rafael@kernel.org" <rafael@kernel.org>,
        "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "git (AMD-Xilinx)" <git@amd.com>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Fri, Jul 15, 2022 at 12:00 PM Pandey, Radhey Shyam
<radhey.shyam.pandey@amd.com> wrote:
>
> > -----Original Message-----
> > From: Saravana Kannan <saravanak@google.com>
> > Sent: Wednesday, July 6, 2022 12:28 AM
> > To: Pandey, Radhey Shyam <radhey.shyam.pandey@amd.com>
> > Cc: Andrew Lunn <andrew@lunn.ch>; nicolas.ferre@microchip.com;
> > claudiu.beznea@microchip.com; davem@davemloft.net;
> > edumazet@google.com; kuba@kernel.org; pabeni@redhat.com;
> > hkallweit1@gmail.com; linux@armlinux.org.uk;
> > gregkh@linuxfoundation.org; rafael@kernel.org; netdev@vger.kernel.org;
> > linux-kernel@vger.kernel.org; git (AMD-Xilinx) <git@amd.com>
> > Subject: Re: [PATCH net-next v2] net: macb: In shared MDIO usecase make
> > MDIO producer ethernet node to probe first
> >
> > On Tue, Jul 5, 2022 at 11:49 AM Pandey, Radhey Shyam
> > <radhey.shyam.pandey@amd.com> wrote:
> > >
> > > > -----Original Message-----
> > > > From: Andrew Lunn <andrew@lunn.ch>
> > > > Sent: Friday, July 1, 2022 2:44 PM
> > > > To: Pandey, Radhey Shyam <radhey.shyam.pandey@amd.com>
> > > > Cc: nicolas.ferre@microchip.com; claudiu.beznea@microchip.com;
> > > > davem@davemloft.net; edumazet@google.com; kuba@kernel.org;
> > > > pabeni@redhat.com; hkallweit1@gmail.com; linux@armlinux.org.uk;
> > > > gregkh@linuxfoundation.org; rafael@kernel.org;
> > saravanak@google.com;
> > > > netdev@vger.kernel.org; linux-kernel@vger.kernel.org; git
> > > > (AMD-Xilinx) <git@amd.com>
> > > > Subject: Re: [PATCH net-next v2] net: macb: In shared MDIO usecase
> > > > make MDIO producer ethernet node to probe first
> > > >
> > > > On Fri, Jul 01, 2022 at 01:25:06AM +0530, Radhey Shyam Pandey wrote:
> > > > > In shared MDIO suspend/resume usecase for ex. with MDIO producer
> > > > > (0xff0c0000) eth1 and MDIO consumer(0xff0b0000) eth0 there is a
> > > > > constraint that ethernet interface(ff0c0000) MDIO bus producer has
> > > > > to be resumed before the consumer ethernet interface(ff0b0000).
> > > > >
> > > > > However above constraint is not met when GEM0(ff0b0000) is resumed
> > first.
> > > > > There is phy_error on GEM0 and interface becomes non-functional on
> > > > resume.
> > > > >
> > > > > suspend:
> > > > > [ 46.477795] macb ff0c0000.ethernet eth1: Link is Down [
> > > > > 46.483058] macb ff0c0000.ethernet: gem-ptp-timer ptp clock
> > unregistered.
> > > > > [ 46.490097] macb ff0b0000.ethernet eth0: Link is Down [
> > > > > 46.495298] macb ff0b0000.ethernet: gem-ptp-timer ptp clock
> > unregistered.
> > > > >
> > > > > resume:
> > > > > [ 46.633840] macb ff0b0000.ethernet eth0: configuring for
> > > > > phy/sgmii link mode macb_mdio_read -> pm_runtime_get_sync(GEM1)
> > it
> > > > > return -
> > > > EACCES error.
> > > > >
> > > > > The suspend/resume is dependent on probe order so to fix this
> > > > > dependency ensure that MDIO producer ethernet node is always
> > > > > probed first followed by MDIO consumer ethernet node.
> > > > >
> > > > > During MDIO registration find out if MDIO bus is shared and check
> > > > > if MDIO producer platform node(traverse by 'phy-handle' property)
> > > > > is bound. If not bound then defer the MDIO consumer ethernet node
> > probe.
> > > > > Doing it ensures that in suspend/resume MDIO producer is resumed
> > > > > followed by MDIO consumer ethernet node.
> > > >
> > > > I don't think there is anything specific to MACB here. There are
> > > > Freescale boards which have an MDIO bus shared by two interfaces etc.
> > > >
> > > > Please try to solve this in a generic way, not specific to one MAC
> > > > and MDIO combination.
> > >
> > > Thanks for the review.  I want to get your thoughts on the outline of
> > > the generic solution. Is the current approach fine and we can extend
> > > it for all shared MDIO use cases/ or do we see any limitations?
> > >
> > > a) Figure out if the MDIO bus is shared.  (new binding or reuse
> > > existing)
> > > b) If the MDIO bus is shared based on DT property then figure out if
> > > the MDIO producer platform device is probed. If not, defer MDIO
> > > consumer MDIO bus registration.
> >
> > Radhey,
> >
> > I think Andrew added me because he's pointing you towards fw_devlink.
> >
> > Andrew,
> >
> > I have intentionally not added phy-handle support to fw_devlink because it
> > would also prevent the generic driver from binding/cause issues with DSA. I
> > have some high level ideas on fixing that but haven't gotten around to it yet.
>
> Thanks, just want to understand on implementation when phy-handle support is
> added to fw_devlink. Does it ensure that supplier node is probed first? Or it uses
> device_link framework to specify suspend/resume dependency and don't care
> on consumer/producer probe order.

fw_devlink will enforce probe ordering and suspend/resume ordering.
Btw, fw_devlink uses device links underneath. It just used the
firmware (Eg: DT) to figure out the dependencies. That's why it's
called fw_devlink.

-Saravana