Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <7e365fa4-7a50-382c-5a99-288a417a82a7@gmail.com>
Date:   Thu, 3 Aug 2023 11:24:04 -0700
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.13.0
Subject: Re: [PATCH net-next v2 02/19] net: phy: add a shutdown procedure
Content-Language: en-US
To:     =?UTF-8?Q?Uwe_Kleine-K=c3=b6nig?= <u.kleine-koenig@pengutronix.de>,
        Ioana Ciornei <ciorneiioana@gmail.com>
Cc:     Andrew Lunn <andrew@lunn.ch>,
        Heiner Kallweit <hkallweit1@gmail.com>,
        Russell King <linux@armlinux.org.uk>,
        Jakub Kicinski <kuba@kernel.org>, netdev@vger.kernel.org,
        linux-kernel@vger.kernel.org,
        Ioana Ciornei <ioana.ciornei@nxp.com>,
        Alexandru Ardelean <alexandru.ardelean@analog.com>,
        Andre Edich <andre.edich@microchip.com>,
        Antoine Tenart <atenart@kernel.org>,
        Baruch Siach <baruch@tkos.co.il>,
        Christophe Leroy <christophe.leroy@c-s.fr>,
        Divya Koppera <Divya.Koppera@microchip.com>,
        Hauke Mehrtens <hauke@hauke-m.de>,
        Jerome Brunet <jbrunet@baylibre.com>,
        Kavya Sree Kotagiri <kavyasree.kotagiri@microchip.com>,
        Linus Walleij <linus.walleij@linaro.org>,
        Marco Felsch <m.felsch@pengutronix.de>,
        Marek Vasut <marex@denx.de>,
        Martin Blumenstingl <martin.blumenstingl@googlemail.com>,
        Mathias Kresin <dev@kresin.me>,
        Maxim Kochetkov <fido_max@inbox.ru>,
        Michael Walle <michael@walle.cc>,
        Neil Armstrong <narmstrong@baylibre.com>,
        Nisar Sayed <Nisar.Sayed@microchip.com>,
        Oleksij Rempel <o.rempel@pengutronix.de>,
        Philippe Schenker <philippe.schenker@toradex.com>,
        Willy Liu <willy.liu@realtek.com>,
        Yuiko Oshino <yuiko.oshino@microchip.com>
References: <20201101125114.1316879-1-ciorneiioana@gmail.com>
 <20201101125114.1316879-3-ciorneiioana@gmail.com>
 <20230803181640.yzxsk2xphwryxww4@pengutronix.de>
From:   Florian Fainelli <f.fainelli@gmail.com>
In-Reply-To: <20230803181640.yzxsk2xphwryxww4@pengutronix.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk

On 8/3/23 11:16, Uwe Kleine-König wrote:
> Hello,
> 
> this patch became commit e2f016cf775129c050d6c79483073423db15c79a and is
> contained in v5.11-rc1.
> 
> It broke wake-on-lan on my NAS (an ARM machine with an Armada 370 SoC,
> armada-370-netgear-rn104.dts). The used phy driver is marvell.c. I only
> report it now as I just upgraded that machine from Debian 11 (with
> kernel 5.10.x) to Debian 12 (with kernel 6.1.x).
> 
> Commenting out phy_disable_interrupts(...) in v6.1.41's phy_shutdown()
> fixes the problem for me.
> 
> On Sun, Nov 01, 2020 at 02:50:57PM +0200, Ioana Ciornei wrote:
>> In case of a board which uses a shared IRQ we can easily end up with an
>> IRQ storm after a forced reboot.
>>
>> For example, a 'reboot -f' will trigger a call to the .shutdown()
>> callbacks of all devices. Because phylib does not implement that hook,
>> the PHY is not quiesced, thus it can very well leave its IRQ enabled.
>>
>> At the next boot, if that IRQ line is found asserted by the first PHY
>> driver that uses it, but _before_ the driver that is _actually_ keeping
>> the shared IRQ asserted is probed, the IRQ is not going to be
>> acknowledged, thus it will keep being fired preventing the boot process
>> of the kernel to continue. This is even worse when the second PHY driver
>> is a module.
>>
>> To fix this, implement the .shutdown() callback and disable the
>> interrupts if these are used.
> 
> I don't know how this should interact with wake-on-lan, but I would
> expect that there is a way to fix this without reintroducing the problem
> fixed by this change. However I cannot say if this needs fixing in the
> generic phy code or the phy driver. Any hints?

It depends upon what the PHY drivers and underlying hardware are capable 
and willing to do. Some PHY drivers will shutdown the TX path completely 
since you do not need that part to receive Wake-on-LAN packets and pass 
them up to the PHY and/or MAC Wake-on-LAN matching logic. This would 
invite us to let individual PHY drivers make a decision as to what they 
want to do in a .shutdown() routine that would then need to be added to 
each and every driver that wants to do something special. In the absence 
of said routine, you could default to calling phy_disable_interrupts() 
unless the PHY has WoL enabled?

phydev::wol_enabled reflects whether the PHY and/or the MAC has 
Wake-on-LAN enabled which could you could key off to "nullify" what the 
shutdown does.

>   
>> Note that we are still susceptible to IRQ storms if the previous kernel
>> exited with a panic or if the bootloader left the shared IRQ active, but
>> there is absolutely nothing we can do about these cases.
> 
> I'd say the bootloader could handle that, knowing that for some machines
> changing the bootloader isn't an option.

There is also the case of the boot loader not touching any 
PHY/MAC/networking and just booting as fast as possible to the kernel. 
It really becomes a problem that can be distributed against multiple SW 
agents to solve it, though clearly the kernel could do it too, so why 
not keep it there I guess.
-- 
Florian